Streaming Architecture and LLM Integration with Next.js and the Vercel AI SDK

Since Large Language Models (LLMs — OpenAI, Anthropic, Gemini, etc.) entered our lives, the shape of web applications has changed. The traditional HTTP request-response cycle falls short for AI-powered apps, because a full LLM response can sometimes take 10-15 seconds to generate. Leaving a user staring at an empty loading screen for 15 seconds is an unacceptable user experience (UX).

The solution is streaming architecture. In this article, we'll look at how to architect a modern, scalable AI integration using the Next.js App Router and the Vercel AI SDK.

Why Streaming Instead of a Traditional REST API?

In a traditional architecture, the server sends a request to the AI model, waits for the full response to complete, and sends the entire response to the client as a single JSON blob.

In a streaming architecture, Next.js (running on Node.js or the Edge runtime) forwards each token coming back from the LLM straight to the browser as it arrives. Users then see the result appear instantly, as if someone were typing it out in front of them.

Using the Vercel AI SDK with Server Actions

Next.js Server Actions remove the need to build a dedicated API route (/api/...) — we can call a server-side function directly from our frontend code.

The example below shows how to set up an optimized stream using the Vercel AI SDK's streamText method:

// app/actions.ts (Server Action)
"use server";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
import { createStreamableValue } from "ai/rsc";

export async function generateProductDescription(prompt: string) {
  // 1. Create a streamable state that can be passed between client and server
  const stream = createStreamableValue("");

  // 2. Connect to the AI model asynchronously and start the stream
  (async () => {
    const { textStream } = await streamText({
      model: openai("gpt-4-turbo"),
      prompt: `You are an SEO expert. Write a compelling description for this product: ${prompt}`,
    });

    // 3. Append each incoming token to the streamable state
    for await (const delta of textStream) {
      stream.update(delta);
    }

    // 4. Close the stream once it's done
    stream.done();
  })();

  // Return the stream object to the client
  return { output: stream.value };
}