Quick Start
Streaming Responses
Enablestream: true for real-time responses via Server-Sent Events:
AI SDK Format
For Vercel AI SDK compatibility, add the header:start, text-delta, source-document, reasoning-delta, tool-input-delta, finish
Reasoning (Extended Thinking)
EnableenableReasoning: true to see the model’s thinking process. Supported by models with extended thinking capabilities — check the model catalog for availability.
Structured Outputs (JSON Mode)
Force JSON schema compliance withresponseFormat:
{"name": "iPhone 15 Pro", "price": 999}
Tool Calling (Function Calling)
Define tools the model can invoke:Continuing Conversations
UsechatId to continue an existing chat:
FAQ
How does streaming work?
The API sends Server-Sent Events (SSE) with incremental content. Eachdata: line contains a JSON object with delta (new text) and finished (boolean). Parse events as they arrive for real-time display.
What’s the max conversation length?
Limited by the model’s context window. The API automatically truncates old messages if needed. Check the model catalog for context window sizes.Are responses cached?
No. Each request generates a fresh completion. For idempotent behavior, use the sameIdempotency-Key header.
How do I count tokens before sending?
The response includes actual token counts inusage. For pre-request estimates, use a tokenizer library compatible with your model’s provider.
Related
Models API
Create and configure models
Knowledge Bases
Add documents for RAG