Streaming format

The server-sent events a streamed reply sends, and how a stream ends.

With "stream": true, chat completions answers with server-sent events. This page is the format on the wire. Streaming shows how to read a stream with an SDK.

A stream

Each event is a data: line and a blank line. A line that starts with a colon is a comment. Here, … stands for the fields every chunk repeats: object, created, model, and obfuscation.

Response
data: {"id":"chatcmpl-a1b2",…,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-a1b2",…,"choices":[{"index":0,"delta":{"content":"Rain"},"finish_reason":null}]}

data: {"id":"chatcmpl-a1b2",…,"choices":[{"index":0,"delta":{"content":" on the roof"},"finish_reason":null}]}

: keepalive

data: {"id":"chatcmpl-a1b2",…,"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-a1b2",…,"choices":[],"usage":{"prompt_tokens":14,"completion_tokens":19,"total_tokens":33}}

data: [DONE]

Events

LineWhat it is
data: {…}A chat.completion.chunk, the next piece of the reply.
data: {"choices": [], "usage": …}With stream_options.include_usage, one chunk after the reply carries usage and an empty choices list. Only Privacy Veil's closing chunks, when it's on, follow it. No other chunk carries usage.
: keepaliveA comment Morse sends every 25 seconds while some reasoning models think. Clients that follow the server-sent events spec skip it.
data: {"error": …}The provider failed after the stream started. It's the last event, in the error shape with the code upstream_error, and no [DONE] follows.
data: [DONE]The reply is complete.

The chunk

ChatCompletionChunk
type ChatCompletionChunk = {
  id: string; // the same on every chunk of a reply
  object: "chat.completion.chunk";
  created: number; // Unix seconds
  model: string;
  choices: {
    index: number;
    delta: {
      role?: string;
      content?: string | null; // the next piece of the reply's text
      reasoning_content?: string; // the next piece of a reasoning model's visible thinking
      tool_calls?: object[]; // pieces of tool calls, in OpenAI's format
    };
    finish_reason: string | null; // set on the chunk that ends the choice
  }[];
  usage?: Usage; // on one chunk at the end, with include_usage
  obfuscation: string; // padding, ignore it
};

Each chunk carries obfuscation, a random string of 16 to 64 letters and digits, so a chunk's size doesn't give away the words in it. It holds no data.

How a stream ends

  • data: [DONE]: the reply is whole.
  • An error event and no [DONE]: the status was already 200, so read the error's code instead.
  • The connection closes with neither: treat the reply as incomplete.

An error before the first event isn't streamed. It comes back as an ordinary error reply, with its HTTP status.

Stop a stream

Close the connection. If a stream stops early, for any reason, you pay for what it delivered.