First-token latency vs sustained throughput, measured through the local proxy on the Anthropic /v1/messages path — the same route the models actually serve on.
context
sort
provider
latency ↔ throughput map
← faster first token · higher throughput ↑ · top-left = best
Long-context mode (~180k-token prompt, real 213k–285k by each tokenizer).
Here TTFB is the whole story — it's now the prefill tax for reading the giant prompt, and it fans out from
0.6s (codex) to 27s (gemini-3.1-pro). The codex sub-second numbers are suspiciously
fast for a 200k prefill and likely reflect ChatGPT-side prompt caching, not a true full read. TPS here is overall
out ÷ total. gpt-oss-120b exceeded its context limit; codex-spark returned
empty; fable-5 was rate-limited.
Cached long-generation (~50k-token cacheable prefix → max_tokens=32000, 1 rep).
A ~50k-token Chinese setting bible is sent, primed by a warm-up call, then one measured call asks the model to
write a long-form novel continuously. This mode measures sustained decode over minutes, not prefill —
runs last from 9s to 12min. Two caveats, both real:
(1) the cache mostly did not hit. Only sonnet-5 and opus-4-8 reported a true
cache_read (51,469 tok, marked cache hit); the proxy runs
routing: round-robin, so the warm-up and the measured call can land on different auth accounts, and a
prompt cache is per-account. Non-Anthropic routes don't report cache fields at all. (2) nobody wrote 100k字.
Asked for maximum length, only opus-4-6-thinking reached the 32k-token cap; every other model stopped
on end_turn at 6k–28k characters. The binding limit is the model's willingness to keep writing, not
max_tokens. gpt-5.6-terra and luna declined outright (<300 tok);
fable-5 was rate-limited again. Note gpt-oss-120b and codex-spark, which
failed the 200k test, both succeed here.