The AI glossary

Annotation

The act of humans labeling model outputs - pass/fail, quality score, or category - for evaluation data.

API Token

The credential your application uses to call Infere, carrying its own routing, budgets, and security settings.

Auto-Recharge

An automatic top-up that buys credits when the balance falls below a set threshold.

Auto-Router

Infere's AI-powered router that analyzes each prompt and picks the optimal model for cost, speed, and quality.

B

Benchmark

A fixed, standardized test used to compare model capability across versions and vendors.

C

Prompting

Chain-of-Thought (CoT)

A prompting technique that asks the model to reason step by step before giving a final answer.

Read → RAG & Data

Chunking

Splitting source documents into smaller passages so retrieval can fetch just the relevant parts.

Circuit Breaker

A guard that stops sending traffic to a failing provider for a cooldown period instead of retrying endlessly.

Content Moderation

Filtering or flagging harmful, abusive, or disallowed content in inputs and outputs.

Context Compression

Reducing the number of input tokens sent to a model by trimming or summarizing context.

Context Window

The maximum number of tokens (input plus output) a model can consider in a single request.

Cost Attribution

Mapping LLM spend to the team, feature, user, or model that caused it.

Cost Optimization

The tactics that lower LLM spend: routing, caching, compression, output limits, batching, and budgets.

Credit Balance

The remaining prepaid funds available for requests, tracked to high precision.

Credits

The prepaid balance Infere draws down as you use models; credits never expire.

D

Safety & Security

Data Retention

How long request and prompt data is stored, and how it is deleted after that.

Dataset

A curated collection of inputs (and often expected outputs) used to run evaluations.

Drift

A gradual change in model behavior, quality, or cost over time, even when the code has not changed.

E

Embedding

A numeric vector that represents the meaning of text, used for search, retrieval, and clustering.

Error Rate

The share of requests that fail, by provider, model, or error type.

Evaluator

An automated check that scores a model or prompt output against defined criteria.

F

Gateway & Routing

Fallback Chain

An ordered list of backup models the gateway retries with if the primary model fails.

Few-Shot & Zero-Shot Prompting

Guiding a model with a few worked examples (few-shot) or none (zero-shot).

Fine-Tuning

Further training a base model on your own examples to specialize its behavior and output style.

Frontier Model

The newest, most capable - and usually most expensive - model from a provider, reserved for the hardest reasoning tasks.

G

Golden Dataset

A trusted, human-verified dataset used as the reference for evaluating changes.

Ground Truth

The correct or expected answer for a task, used as the reference when scoring model output.

Read → RAG & Data

Groundedness

How well a model's answer is supported by the context it was given, rather than invented.

Guardrails

Rules and checks that constrain what goes into and comes out of a model.

H

Hallucination

When a model states something false or unsupported as if it were true, confidently and fluently.

Human-in-the-Loop (HITL)

Routing the hardest evaluation cases to human reviewers to label and calibrate automated judges.

I

Platform

Infercom

Infere's own inference provider, serving a range of open-weight models within the same catalog.

L

Fundamentals

Large Language Model (LLM)

A neural network trained on large text corpora to predict the next token and, in turn, generate, summarize, translate, and reason over language.

Latency

The total time a request takes from send to complete response.

LLM Gateway

A proxy that sits in front of multiple model providers and exposes one API, one key, and one bill.

LLM-as-a-Judge

Using one model to grade the output of another against a rubric, at scale and at low cost.

M

Fundamentals

Model

A specific trained artifact identified by an ID, such as claude-sonnet-5 or gpt-5.6-terra, that you select per request.

Model Registry

Infere's live, read-only catalog of every available model with capabilities, context, and per-token pricing.

Model Router

Logic that selects which model handles each request based on cost, capability, and quality.

Monitoring

Continuously watching production metrics - latency, errors, cost, quality - and alerting when they move.

Multi-Provider Fallback

Serving one request across several providers so another picks up when one fails.

Multimodal

A model that can accept or generate more than text - typically images, and sometimes audio or video.

O

Observability

Understanding what a system did and why, from its logs, traces, and metrics - in production.

Offline vs Online Evaluation

Offline evaluation runs before deploy on a fixed dataset; online evaluation scores real production traffic.

Open-Weight Model

A model whose weights are publicly downloadable, so it can be self-hosted or served by third parties.

OpenAI-Compatible API

An API that matches the OpenAI request/response shape, so existing SDKs work with only a base URL change.

Organization

A shared account above workspaces, with members, roles, and a common credit balance.

P

Cost & Billing

Pass-Through Pricing

Charging exactly the provider's per-token rate with no gateway markup.

Pass@k

The share of tasks solved at least once across k attempts - a measure of how often a model can succeed.

Percentiles (P50 / P95 / P99)

Latency or cost at given percentiles of traffic; tail percentiles reveal the experience users actually hit.

PII (Personally Identifiable Information)

Data that identifies a person - names, emails, IDs - which may need redaction before reaching a model.

PML (Prompt Markup Language)

Infere's file format for prompts and evaluators, with model config and variables in frontmatter.

Prepaid Billing

Paying for usage in advance by funding a balance, instead of being invoiced after the fact.

Prompt

The input you send to a model - system instructions, user messages, and any context or examples.

Prompt Caching

A provider feature that reuses a repeated prompt prefix and bills cache hits at a fraction of the input rate.

Prompt Enhancement

Infere feature that uses AI to rewrite a prompt for a specific target model to improve output quality.

Prompt Injection

An attack where untrusted input overrides a model's instructions or leaks its system prompt.

Prompt Repository

A Git-backed repository that stores prompts and PML files as versioned source code.

Prompt Template

A reusable prompt with placeholders that are filled in at runtime for each request.

Property

A custom tag attached to a request (for example user, feature, or environment) used to slice analytics.

Provider

The vendor that serves a model. Infere routes across OpenAI, Anthropic, Fireworks AI, Together AI, and Infercom.

Q

Quality Gate

An automated threshold that blocks a deploy - or flags a request - when evaluation scores regress.

R

RAG (Retrieval-Augmented Generation)

Feeding a model relevant retrieved context so it answers from your data instead of its memory alone.

Rate Limit

A cap on how many requests or tokens can be sent in a time window.

Request Ledger

A searchable record of every request that passes through the gateway, with model, tokens, cost, and latency.

Read → RAG & Data

Retrieval

Fetching the passages most relevant to a query, usually by vector similarity, before generation.

Rubric

The written criteria and scale an evaluator - human or LLM - uses to score output.

S

Semantic Search

Search ranked by meaning (via embeddings) rather than exact keyword matches.

Span

A single unit of work within a trace, such as one model call or one tool invocation.

Spend Cap

A hard upper bound on spending over a period, which blocks further requests once reached.

Streaming

Delivering a model's response token by token as it is generated, instead of waiting for the full output.

System Prompt

The high-level instructions that set a model's role, rules, and tone before the user's messages.

T

Prompting

U

Platform

Uptime SLA

A service-level commitment on availability, backed by multi-provider fallback and circuit breakers.

W

Platform

Workspace

An isolated container that holds its own API tokens, prompts, datasets, and usage.

Workspace Isolation

Keeping each workspace's data, tokens, prompts, and usage separated from every other workspace.