Observability is the ability to understand what a system did and why, using its logs, traces, and metrics. For LLM applications that means capturing prompts, model, tokens, cost, latency, and quality for every request - and connecting them to the prompt version and evaluator that produced the result.
Why it matters
LLM behavior is probabilistic and multi-step, so traditional uptime monitoring is not enough. You need to see the prompt, model, tokens, cost, latency, and quality for every request.
How it works
Requests are logged to a ledger, grouped into traces, and tagged with properties so you can slice them. Because the gateway sees every call, this data is captured without instrumenting each service.