串流

設定 stream: true 接收 server-sent events。位元組從供應商直通，usage 在最後一個 chunk。

運作方式

chunk 為標準 OpenAI chat.completion.chunk。NXIO 會強制 stream_options.include_usage，因此 [DONE] 前的最後一個 data 事件帶有含費用的 usage。計費在串流結束後依該 usage 進行。

TypeScript

const stream = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [{ role: 'user', content: 'Write a haiku about servers.' }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
  if (chunk.usage) console.log('\n', chunk.usage); // final chunk carries usage + cost
}

raw SSE

data: {"id":"gen-…","choices":[{"index":0,"delta":{"content":"Hel"},"finish_reason":null}]}

data: {"id":"gen-…","choices":[{"index":0,"delta":{"content":"lo"},"finish_reason":"stop"}]}

data: {"id":"gen-…","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":2,"total_tokens":14,"prompt_tokens_details":{"cached_tokens":10,"cache_write_tokens":0},"completion_tokens_details":{"reasoning_tokens":0},"cost":0.000066,"is_byok":false}}

data: [DONE]

串流中途錯誤

header 送出後 HTTP 狀態已是 200。若供應商中途失敗，NXIO 會送出 finish_reason 為 "error" 且帶頂層 error 物件的最後一個 chunk，然後 [DONE]。在任何位元組送出前失敗的請求則回一般 HTTP 錯誤。

串流中錯誤

data: {"id":null,"object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"error"}],"error":{"code":502,"message":"Upstream provider rejected the request","metadata":{"error_type":"server","provider_name":"upstream"}}}

data: [DONE]

供應商沒回 usage 時

極少數情況串流結束卻沒有 usage chunk（連線中斷、供應商問題）。此時 NXIO 以 token 估算計費，並在 GET /generation 標記 usage_source = estimated。估算刻意偏保守。