Listen up. If you're building modern applications, understanding AI Streaming is non-negotiable. This is where simple logic turns into intelligent behavior.
1Why Waiting for the Full Response Feels Slow
A long completion can take many seconds to generate in full, and a standard blocking request makes the user stare at a spinner for the entire duration. Streaming changes the delivery model: instead of one big JSON blob at the end, the response arrives as a sequence of small chunks the moment each one is generated.
This is exactly the token-by-token 'typing' effect visible in apps like ChatGPT, and it's a delivery mechanism, not a change to how the model itself generates text.
// Example
console.log("Running AI concept...");AI logic processed successfully.
2Enabling stream: true
A single boolean flag ā stream: true ā changes the API's response shape entirely: instead of one JSON object, the connection stays open and delivers a series of small delta chunks, each containing just the next piece of generated text.
Everything else about the request (model, messages, temperature) stays identical; streaming only changes how the response is delivered, not what gets generated.
const response = await openai.chat.completions.create({
model: "gpt-4",
messages: [{ role: "user", content: "Write a story." }],
stream: true,
});AI logic processed successfully.
3Reading the Stream with getReader() and TextDecoder
On the client, response.body.getReader() gives you a ReadableStream reader that yields raw binary chunks (Uint8Array) as they arrive over the network, which a TextDecoder then converts back into readable text.
The read loop keeps calling reader.read() until it returns done: true, at which point the stream is fully consumed and the full message has been reconstructed piece by piece.
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
}AI logic processed successfully.
4Building the Typing Effect in React
Each decoded chunk gets appended to component state with a functional update, setContent(prev => prev + newChunk), so React re-renders with slightly more text every time a chunk lands ā the visual 'typing' effect is nothing more than many rapid small state updates.
Using the functional form (prev => ...) rather than referencing the outer content variable directly matters here, since chunks can arrive faster than React's render cycle and a stale closure would silently drop text.
const [content, setContent] = useState('');
// Inside the stream loop:
setContent((prev) => prev + newChunk);AI logic processed successfully.
5Streaming's Real Metric: Time to First Token
Streaming doesn't make the model generate the full response any faster ā total generation time is roughly unchanged. What it improves is Time to First Token: the user sees output starting almost immediately instead of waiting for the entire response to finish before anything appears.
This perceived-speed improvement is the entire point of streaming; measuring TTFT, not total response time, is how you should evaluate whether it's actually working.
Status: Streaming...
AI logic processed successfully.
6Streaming Beyond Plain Text
The same delta-chunk mechanism carries structured data too ā tool_calls can arrive incrementally across multiple chunks, meaning an agent's function-call arguments get assembled piece by piece just like text does, requiring your parsing code to accumulate partial JSON across chunks rather than expect it complete in one.
Some providers extend this further to stream progress on non-text generation like images, though plain text chat remains the most common streaming use case.
// Multi-modal streaming
for await (const chunk of stream) {
if (chunk.choices[0].delta.tool_calls) {
processToolChunk(chunk);
}
}AI logic processed successfully.
7Streams Fail Differently Than Regular Requests
A streaming request's initial HTTP response can succeed (status 200, connection opened) while the stream itself fails midway ā a network blip or provider timeout partway through generation is a distinct failure mode from a normal request's all-or-nothing success/failure.
Error handling has to live inside the read loop itself (wrapping each reader.read() call), not just around the initial fetch, or a mid-stream failure goes completely unhandled.
try {
const { value, done } = await reader.read();
} catch (err) {
console.error('Stream interrupted', err);
}AI logic processed successfully.
8Streaming as the Default for Chat UIs
Between the perceived-latency win and how expected the typing effect now is in AI products, streaming has effectively become the default choice for any conversational interface, with blocking requests reserved for cases where you genuinely need the complete response before doing anything (structured extraction, background jobs).
The implementation cost (a read loop, a decoder, incremental state updates) is a one-time investment that pays off on every single request afterward.
UX: Instant
AI logic processed successfully.
9Next: Enforcing Structured JSON Output
Streaming solves perceived latency for conversational text, but many features need the response in a specific, parseable shape rather than free-form prose ā the next lesson covers JSON Mode, which guarantees the model's output is valid, structured JSON your code can consume directly.
JSON Mode Next
AI logic processed successfully.
10Step-by-Step Breakdown
AI responses can be slow. 'Streaming' allows you to deliver parts of the response as they are generated, making your app feel instant.
To enable streaming, set the 'stream' property to true in your API request. This changes the response from a single JSON to a stream of chunks.
On the frontend, you'll need to read the response body as a stream. Each chunk must be decoded from binary into text.
Checkpoint: Which JavaScript object is used to convert binary stream data back into human-readable text?
- āJSON.parse()
- āTextDecoder
As each chunk arrives, you append it to your state. This creates the 'typing' effect seen in apps like ChatGPT.
Streaming reduces 'Time to First Token' (TTFT), significantly improving the perceived performance of your AI application.
Checkpoint: What is the primary UX benefit of using streaming instead of a standard blocking request?
- āIt makes the API calls cheaper
- āIt reduces the perceived latency for the user
Streaming isn't just for chat. You can stream tool calls and even image generation progress in some advanced workflows.
Error handling is different in streams. You must catch errors within the read loop, as the initial HTTP request might succeed while the stream fails later.
Checkpoint: What does TTFT stand for in performance monitoring?
- āTime to First Token
- āTotal Time for Transcript
Streaming mastered! Your users will love the snappy, real-time feel of your AI interfaces.
Next, we'll learn how to handle structured data with 'AI JSON Mode'.
Detect a Real Stream Completion. Finish detecting the special marker that signals a streamed response has finished.
Level Up š
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported.
Fully supported.
Fully supported.
Fully supported.
Accessibility (A11y)
1Don't Let Streamed Text Spam Screen Readers Word-by-Word
Wrapping a streaming message container in aria-live="polite" without any throttling causes a screen reader to attempt announcing every single incremental update, producing an unusable flood of partial-word interruptions ā batch updates (e.g. announce once generation completes, or every few hundred milliseconds) instead of on every chunk.
<div aria-live="polite" aria-atomic="true">{finalizedContent}</div>SEO Implications
- 1
Streamed Content Still Needs a Complete Final State for SSR
If a streamed AI response is meant to be part of indexable page content (not just an ephemeral chat UI), the server-rendered version must include the complete, finished text ā streaming is a client-perceived delivery optimization and has no equivalent benefit or meaning for a crawler receiving a single HTML response.
Best Practices
Debounce UI Updates for Very Fast Streams
On a fast connection, chunks can arrive fast enough to trigger dozens of re-renders per second; batching multiple chunks into a single state update every ~50ms keeps the UI smooth without perceptibly changing the typing effect.
Always Close the Reader in a finally Block
Whether the stream completes normally or throws mid-way, call reader.releaseLock() or otherwise clean up in a finally block, so a failed stream doesn't leave a dangling open connection.
Frequent Bugs
Appending raw decoded chunks directly without handling multi-byte UTF-8 characters split across chunk boundaries.
TextDecoder.decode() without the { stream: true } option can corrupt a multi-byte character (like an emoji or accented letter) that happens to be split across two separate network chunks. Pass { stream: true } to decode() during the loop, and a final decode() call with no arguments to flush any trailing bytes.
Real-World Examples
A Streaming Chat Component with Debounced Rendering
A chat UI buffers incoming chunks in a ref, and a setInterval every 50ms flushes the buffer into React state in one batch, giving users a smooth typing effect without triggering a re-render for every single small chunk that arrives from a fast connection.
const bufferRef = useRef('');
setInterval(() => {
if (bufferRef.current) {
setContent(prev => prev + bufferRef.current);
bufferRef.current = '';
}
}, 50);