Time-to-first-token measures how long until the first streamed token arrives. It is dominated by queueing and prompt processing, so long prompts and busy providers push it up. TTFT and total latency together describe the streaming experience better than either alone.
Why it matters
In a streaming UI, the wait before the first token is the wait users notice most. TTFT is often what makes an app feel fast or sluggish.
How it works
TTFT is dominated by queueing and prompt processing, so long prompts and busy providers push it up. Reducing prompt size and routing to fast models are the primary ways to lower it.