Latency Budget Calculator

Plan how a streamed LLM response will feel to your users. Enter output length, generation speed, and time to first token to see when text appears, when it finishes, and how it compares to human reading speed.

Output tokens

Generation speed preset

Tokens per second (editable)

Time to first token (seconds)

First token appears

Full response done

Approx. words (~0.75/token)

Stream speed vs reading speed

What the user sees over time

Elapsed	Tokens visible	Approx. words	% of response

Perceived speed tips:

Always stream. A response that starts in 0.8s and finishes in 8s feels faster than one that appears complete at 4s. Waiting on a blank screen is the worst state.
Show progress immediately: a typing indicator or skeleton the instant the request is sent, first tokens as soon as they arrive.
Cut time to first token: prompt caching, shorter system prompts, and smaller models all reduce TTFT, and TTFT dominates how "snappy" the app feels.
The reading-speed trick: people read around 4 words per second (~240 wpm). At ~0.75 words per token that is roughly 5-6 tokens per second. Any stream faster than ~6 tok/s outpaces the reader, so it feels instant while they read - speed beyond that only matters for content users skim or that machines consume (code, JSON, agents).

Speed presets are typical serving ranges, approximate as of July 2026 - actual tok/s varies by model size, provider, load, and context length. Measure your own with a stopwatch or the API's usage timings.