Time to first token, or TTFT, measures how long passes between sending your request and the first piece of the response appearing. It's what gives the feeling that the model "started responding quickly".

How it works

When you send a prompt, several things happen before the first token comes out:

  • The request travels over the network to the server.
  • The request may wait in a queue if the server is busy.
  • The model reads and processes the entire prompt, a phase called prefill.
  • Only then is the first token generated and sent back.

For this reason, very long prompts increase TTFT: there is more text to process before starting. Reasoning models can also take longer, because they "think" before showing the answer.

TTFT only makes sense when measured properly with streaming responses. Without streaming, you only receive everything at the end and you don't see the first token separately.

Practical example

You build a chat for customers. In a test, the complete response takes 8 seconds, but the first text appears in 0.6 second. The customer feels the system is agile, because they see the text appearing. If the first token took 5 seconds, even with the same total duration, it would feel stuck.

A simple way to measure:

Python · ttft.py
import time
inicio = time.time()
for pedaco in resposta_em_streaming:
    print("TTFT:", time.time() - inicio)
    break

Here, resposta_em_streaming represents the stream returned by your provider's client.

What it's for

TTFT is the central metric for user experience in chats and voice assistants, where every tenth of a second counts. It also serves to compare providers and models and to diagnose bottlenecks, such as queue, network or prompts that are too large.

To reduce TTFT, you can shorten the prompt, use prompt cache for repeated parts, choose a smaller model or a server closer by.

Difference between TTFT and tokens per second

TTFT measures the initial wait. Tokens per second measures the pace after the response starts. A model can have low TTFT and write slowly, or start slow and then write fast. To evaluate well, look at both measures and the total latency.

Caveats

The value varies with time of day, load and prompt size. Make several measurements and use the median or percentiles, not a single test. Always compare with the same prompt.