What is Response Streaming? Meaning and examples

What is Response Streaming?

Response Streaming: Sending the model's response token by token instead of waiting for the complete response, reducing perceived latency for the user.

Updated on 11/9/2026 · By Marcelo Santiago

How it works

Response streaming is sending the model's output as it is generated, token by token, instead of delivering everything at once at the end. The server uses a persistent connection, usually SSE or WebSocket, and sends each fragment as soon as the model produces it.

From the user's point of view, the first word appears in milliseconds and the text fills in on screen. This greatly improves the experience, even if the total generation time is the same.

Practical example

In a chat API call, you enable streaming mode by passing a parameter. The response arrives as a sequence of events:

Terminal

$ curl https://api.exemplo.com/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -d '{"model":"gpt-4o","stream":true,"messages":[{"role":"user","content":"Explique LoRA"}]}'

Each line of the stream carries a text fragment. Your code accumulates and displays it in the interface. Metrics like time to first token (TTFT) measure how long it takes for the first fragment to appear.

What it's for

Streaming is used to improve the perceived speed in chat interfaces, assistants and copilots. It is essential in products where users expect a fast response. It also allows you to cancel generation midway, saving tokens and cost when the response is already sufficient.

VS Code Extension

Enjoyed learning about Response Streaming? Put it into practice with AI

Orchestrate squads of agents (Claude, OpenCode, Codex, Antigravity and more) in VS Code with complete control, isolated git worktrees, and your own APIs or subscriptions with zero markup.

Parallel multi-agentsAPIs or subscriptionsAgents for multitasking

Explore bridgecode

Frequently asked questions about Response Streaming

Does streaming reduce API cost?

Not directly. Cost depends on the tokens generated. Streaming lets you cancel early, which can save tokens.

Does streaming work with any model?

It depends on the provider. Most modern chat APIs offer streaming, but you need to enable it in the call parameter.

What is the difference between streaming and batch?

Streaming sends the response piece by piece, while batch delivers everything at once at the end. Streaming improves the experience, batch simplifies the code.