Generate chat completion

Generate a chat completion from a prompt using one model in your project. Your prompt messages can include several roles. This endpoints supports streaming

POST /api/v1/chat/completions

Generate chat completion

cURL

curl --request POST \
  --url https://api.example.com/api/v1/chat/completions \
  --header 'Content-Type: application/json' \
  --data '
{
  "messages": [
    {
      "content": "<string>",
      "role": "<string>",
      "name": "<string>",
      "completion_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "metadata": "<unknown>"
    }
  ],
  "model": "<string>",
  "stop": [
    "<string>"
  ],
  "max_tokens": 1,
  "temperature": 123,
  "top_p": 123,
  "max_ttft_ms": 1,
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "session_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "user": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "ab_campaign": "76d1fab3-214c-47ef-bb04-16270639bf89",
  "n": 1,
  "labels": {
    "key": "value"
  },
  "metadata": {
    "key": "value"
  },
  "system_prompt_args": {},
  "tags": [
    "<string>"
  ],
  "use_tools": true,
  "tools": [
    {
      "id": "<string>",
      "enabled": true
    }
  ],
  "store": true
}
'

200

{
  "id": "<string>",
  "created": 123,
  "choices": [
    {
      "index": 1,
      "message": {
        "id": "<string>",
        "role": "<string>",
        "content": "<string>"
      },
      "completion_id": "<string>",
      "model": "<string>",
      "finish_reason": "<string>"
    }
  ],
  "session_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "usage": {
    "completion_tokens": 1,
    "prompt_tokens": 1,
    "total_tokens": 1
  }
}

Body

application/json

messages · object[] · required

Show child attributes

model · string · required

can be of the form {project}/{model} or {project}. In the latter it will use the default model

stop · string[] | null

max_tokens · integer<int32> | null

Required range: x >= 0

temperature · number<float> | null

top_p · number<float> | null

max_ttft_ms · integer<int64> | null

Required range: x >= 0

stream · boolean

stream_options · object

Show child attributes

session_id · string<uuid> | null

user · string<uuid> | null

ab_campaign · null | string

id or key of the entity

Example:

"76d1fab3-214c-47ef-bb04-16270639bf89"

n · integer<int32>

Required range: x >= 0

labels · object

dictionnary with key and values as string

Show child attributes

Example:

{ "key": "value" }

metadata · object

alias for labels

Show child attributes

Example:

{ "key": "value" }

system_prompt_args · object

Will be used to render system prompt template

Show child attributes

tags · string[] | null

use_tools · boolean

tools · object[] | null

Override tool configuration for this request - enables/disables specific tools

Show child attributes

store · boolean | null

Response

200

application/json

a stream will be returned if streaming = true in the request

id · string · required

created · number · required

Unix Timestamp in seconds

Example:

"1720712536"

choices · object[] · required

Show child attributes

session_id · string<uuid> · required

usage · object · required

Show child attributes