XAI | Voximplant Voice AI

XAI provides VoxEngine clients for the xAI Grok Voice Agent API and Realtime TTS over WebSocket.

Use XAI.createVoiceAgentAPIClient(...) to create a VoiceAgentAPIClient for voice-agent scenarios. After the client is created, call methods such as sendMediaTo, responseCreate, and addEventListener on that client instance.

Use XAI.createRealtimeTTSPlayer(...) when you only need realtime text-to-speech playback.

Grok connector overview

Learn how Grok Voice Agent fits into a VoxEngine call flow.

Answer an inbound call

Start from a complete inbound Grok Voice Agent scenario.

Grok features

See tool calls, context, and event-driven interaction patterns.

Place an outbound call

Start from a complete outbound Grok Voice Agent scenario.

Usage: required module import and basic flow.
Factory functions: create the XAI Voice Agent client.
VoiceAgentAPIClientParameters: API key, tracing, privacy, and WebSocket options.
VoiceAgentAPIClient: runtime client object returned by the factory.
Methods: media, response, and connection control methods.
Events: WebSocket media start and end payloads.
VoiceAgentAPIEvents: xAI Voice Agent API event names and payload fields.

Usage

Add the module before using the namespace:

require(Modules.XAI);

Create the client, bridge media, and listen for both WebSocket media events and Voice Agent API events.

Factory functions

createRealtimeTTSPlayer

Creates a new XAI.RealtimeTTSPlayer instance. You can attach media streams later via the XAI.RealtimeTTSPlayer.sendMediaTo or VoxEngine.sendMediaBetween methods.

createRealtimeTTSPlayer(parameters?: {
  statistics?: boolean;
  trace?: boolean;
  privacy?: boolean;
  connectionParameters?: Object;
  apiKey?: string;
}): RealtimeTTSPlayer

The optional parameters object is typed as XAI.RealtimeTTSPlayerParameters.

Parameters

Parameter	Type	Req.	Description
`parameters`	`RealtimeTTSPlayerParameters`	✗	XAI.RealtimeTTSPlayer parameters. Can be passed as arguments to the `XAI.createRealtimeTTSPlayer` method.
↳ `statistics`	`boolean`	✗	Enables statistics functionality.
↳ `trace`	`boolean`	✗	Whether to enable the tracing functionality. If tracing is enabled, a URL to the trace file appears in the ‘websocket.created’ message. The file contains all sent and received WebSocket messages in the plain text format. The file is uploaded to the S3 storage. NOTE: enable this only for diagnostic purposes. You can provide the trace file to our support team to help investigate issues.
↳ `privacy`	`boolean`	✗	Whether to enable the privacy functionality. If privacy is enabled, the logging for the WebSocket connection is disabled. NOTE: the default value is false.
↳ `connectionParameters`	`Object`	✗	Object to provide parameters for the xAI WebSocket connection. Must contain `voice` and `language`. Find more information in the documentation.
↳ `apiKey`	`string`	✗	xAI API key. Use your xAI API key if you have your own xAI account.

Returns

Type	Description
`RealtimeTTSPlayer`	The requested `RealtimeTTSPlayer` value.

createVoiceAgentAPIClient

Creates a new XAI.VoiceAgentAPIClient instance.

createVoiceAgentAPIClient(parameters: {
  statistics?: boolean;
  trace?: boolean;
  privacy?: boolean;
  onWebSocketClose?: (event: object) => void;
  xAIApiKey: string;
  model?: string;
  conversationId?: string;
  reasoningEffort?: string;
}): Promise<XAI.VoiceAgentAPIClient>

The required parameters object is typed as XAI.VoiceAgentAPIClientParameters.

Parameters

Parameter	Type	Req.	Description
`parameters`	`VoiceAgentAPIClientParameters`	✓	XAI.VoiceAgentAPIClient parameters. Can be passed as arguments to the `XAI.createVoiceAgentAPIClient` method.
↳ `statistics`	`boolean`	✗	Enables statistics functionality.
↳ `trace`	`boolean`	✗	Whether to enable the tracing functionality. If tracing is enabled, a URL to the trace file appears in the ‘websocket.created’ message. The file contains all sent and received WebSocket messages in the plain text format. The file is uploaded to the S3 storage. NOTE: enable this only for diagnostic purposes. You can provide the trace file to our support team to help investigate issues.
↳ `privacy`	`boolean`	✗	Whether to enable the privacy functionality. If privacy is enabled, the logging for the WebSocket connection is disabled. NOTE: the default value is false.
↳ `onWebSocketClose`	`(event: object) => void`	✗	A callback function that is called when the WebSocket connection is closed.
↳ `xAIApiKey`	`string`	✓	The xAI API key for the XAI VoiceAgent API.
↳ `model`	`string`	✗	The model to use for the XAI VoiceAgent API. https://docs.x.ai/developers/model-capabilities/audio/voice-agent#model-selection. Note: The default value is grok-voice-fast-1.0.
↳ `conversationId`	`string`	✗	Conversation id from a previous `XAI.VoiceAgentAPIEvents.ConversationCreated` event. Pass it when creating a new client to resume the session after a WebSocket disconnect (xAI session resumption). Enable caching with `resumption: { enabled: true }` in `XAI.VoiceAgentAPIClient.sessionUpdate`.
↳ `reasoningEffort`	`string`	✗	Controls whether the model uses reasoning (`high` \| `none`). Passed as the WebSocket `reasoning.effort` query parameter. Defaults to `high` on the xAI side when omitted.

Returns

Type	Description
`Promise<XAI.VoiceAgentAPIClient>`	Resolves to the `XAI.VoiceAgentAPIClient` instance.

VoiceAgentAPIClient

Methods

addEventListener

Adds a handler for the specified XAI.VoiceAgentAPIEvents or XAI.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

addEventListener(event: XAI.Events | XAI.VoiceAgentAPIEvents | string, callback: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`XAI.Events \| XAI.VoiceAgentAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✓	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

clearMediaBuffer

Clears the XAI WebSocket media buffer.

clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`ClearMediaBufferParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

close

Closes the XAI connection (over WebSocket) or connection attempt.

close(): void

Parameters

This method does not accept parameters.

Returns

Type	Description
`void`	Does not return a value.

conversationItemCreate

Create a new conversation item. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.create

conversationItemCreate(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓	xAI conversation.item.create client message. Common fields include type, event_id, and item; item can be a user message, assistant message, function call, or function call output. See the partner API reference.

Returns

Type	Description
`void`	Does not return a value.

Example parameters:

{
  "type": "conversation.item.create",
  "event_id": "event_345",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Hello"
      }
    ]
  }
}

conversationItemDelete

Delete a conversation item by ID. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.delete

conversationItemDelete(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

conversationItemTruncate

Truncate a previous assistant audio message item. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.truncate

conversationItemTruncate(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

id

Returns the VoiceAgentAPIClient id.

id(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

inputAudioBufferClear

Clear input audio buffer. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.clear

inputAudioBufferClear(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓	xAI input_audio_buffer.clear client message. Common fields include type and optional event_id. See the partner API reference.

Returns

Type	Description
`void`	Does not return a value.

Example parameters:

{
  "type": "input_audio_buffer.clear"
}

inputAudioBufferCommit

Commit the input audio buffer as a user message. Only available when turn_detection type is null. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.commit

inputAudioBufferCommit(parameters?: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✗

Returns

Type	Description
`void`	Does not return a value.

removeEventListener

Removes a handler for the specified XAI.VoiceAgentAPIEvents or XAI.Events event.

removeEventListener(event: XAI.Events | XAI.VoiceAgentAPIEvents | string, callback?: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`XAI.Events \| XAI.VoiceAgentAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✗	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

responseCancel

Send this event to cancel an in-progress response. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.cancel

responseCancel(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

responseCreate

Request the server to create a new assistant response when using client side vad. (This is handled automatically when using server side vad.) Prefer nested { response: { ... } } body per xAI docs; flat fields are also accepted by the connector. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.create

responseCreate(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓	xAI response.create client message. Common fields include type, optional event_id, and optional response configuration. See the partner API reference.

Returns

Type	Description
`void`	Does not return a value.

Example parameters:

{
  "type": "response.create"
}

sendMediaTo

Starts sending media from the XAI (via WebSocket) to the media unit. XAI works in real time.

sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓
`parameters`	`SendMediaParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

sessionUpdate

Send this event to update the session’s configuration. https://docs.x.ai/developers/rest-api-reference/inference/voice#session.update

sessionUpdate(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓	xAI session.update client message. Common fields include type, optional event_id, and session, which can configure prompt, voice, audio formats, turn detection, and tools. See the partner API reference.

Returns

Type	Description
`void`	Does not return a value.

Example parameters:

{
  "type": "session.update",
  "session": {
    "voice": "aria",
    "instructions": "You are a helpful assistant",
    "turn_detection": {
      "type": "server_vad"
    }
  }
}

stopMediaTo

Stops sending media from the XAI (via WebSocket) to the media unit.

stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓

Returns

Type	Description
`void`	Does not return a value.

webSocketId

Returns the XAI WebSocket id.

webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

Events

These events describe audio received through the XAI WebSocket media bridge.

Events.WebSocketMediaStarted

Triggered when the audio stream sent by a third party through an xAI WebSocket starts playing.

Event constant: Events.WebSocketMediaStarted

Payload

Field	Type	Req.	Description
`client`	`VoiceAgentAPIClient`	✓	The XAI.VoiceAgentAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`encoding`	`string`	✗	Audio encoding formats.
`customParameters`	`{ [key: string]: string }`	✗	Custom parameters.

Events.WebSocketMediaEnded

Triggered after the end of the audio stream sent by a third party through an xAI WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

Field	Type	Req.	Description
`client`	`VoiceAgentAPIClient`	✓	The XAI.VoiceAgentAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`mediaInfo`	`WebSocketMediaInfo`	✗	Information about the audio stream that can be obtained after the stream stops or pauses (1 second of silence).

VoiceAgentAPIEvents

These events mirror server messages from the xAI Voice Agent API. The data field contains the provider event payload.

All VoiceAgentAPIEvents callbacks receive these common fields:

Field	Type	Description
`client`	`XAI.VoiceAgentAPIClient`	The XAI.VoiceAgentAPIClient instance.
`data`	`Object`	Pass-through xAI server event payload.

Per-event payload tables below show only event-specific fields (and any provider payload enrichment for data).

Unknown

The unknown event.

Event constant: VoiceAgentAPIEvents.Unknown

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

Error

Sent when an error occurs. Contains error code and message. https://docs.x.ai/developers/rest-api-reference/inference/voice#error

Event constant: VoiceAgentAPIEvents.Error

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

SessionCreated

Returned when a session is created. https://docs.x.ai/developers/rest-api-reference/inference/voice#session.created

Event constant: VoiceAgentAPIEvents.SessionCreated

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ConversationCreated

The first message at connection. Notifies the client that a conversation session has been created. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.created

Event constant: VoiceAgentAPIEvents.ConversationCreated

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The first message on connection. Notifies the client that a conversation session has been created. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `conversation.created`.
`data.conversation`	`object`	✓	The conversation object.

Example data:

{
  "event_id": "event_9101",
  "type": "conversation.created",
  "conversation": {
    "id": "conv_001",
    "object": "realtime.conversation"
  }
}

SessionUpdated

Acknowledge the client’s “session.update” message that the session has been updated. https://docs.x.ai/developers/rest-api-reference/inference/voice#session.updated

Event constant: VoiceAgentAPIEvents.SessionUpdated

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Acknowledges the client’s session.update message that the session has been configured. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `session.updated`.
`data.session`	`object`	✓	The updated session configuration.

Example data:

{
  "event_id": "event_123",
  "type": "session.updated",
  "session": {
    "model": "grok-voice-fast-1.0",
    "instructions": "You are a helpful assistant.",
    "voice": "Eve",
    "turn_detection": {
      "type": "server_vad"
    }
  }
}

ConversationItemAdded

Responding to the client that a new user message has been added to conversation history, or if an assistance response has been added to conversation history. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.added

Event constant: VoiceAgentAPIEvents.ConversationItemAdded

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	A new user or assistant message has been added to the conversation history. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `conversation.item.added`.
`data.previous_item_id`	`string`	✓	ID of the preceding item in conversation history.
`data.item`	`object`	✓	The conversation item that was added.

Example data:

{
  "event_id": "event_1920",
  "type": "conversation.item.added",
  "previous_item_id": "msg_002",
  "item": {
    "id": "msg_003",
    "object": "realtime.item",
    "type": "message",
    "status": "completed",
    "role": "user",
    "content": [
      {
        "type": "input_audio",
        "transcript": "hello how are you"
      }
    ]
  }
}

ConversationItemDeleted

Returned when an item in the conversation is deleted. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.deleted

Event constant: VoiceAgentAPIEvents.ConversationItemDeleted

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ConversationItemTruncated

Returned when a conversation item is truncated. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.truncated

Event constant: VoiceAgentAPIEvents.ConversationItemTruncated

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ConversationItemInputAudioTranscriptionCompleted

Notify the client the audio transcription for input has been completed. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.input_audio_transcription.completed

Event constant: VoiceAgentAPIEvents.ConversationItemInputAudioTranscriptionCompleted

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Audio transcription for the user’s input has been completed. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`”conversation.item.input_audio_transcription.completed”`	✓	Event type.
`data.item_id`	`string`	✓	ID of the conversation item whose audio was transcribed.
`data.transcript`	`string`	✓	The transcribed text.

Example data:

{
  "event_id": "event_2122",
  "type": "conversation.item.input_audio_transcription.completed",
  "item_id": "msg_003",
  "transcript": "Hello, how are you?"
}

ConversationItemInputAudioTranscriptionUpdated

Returned when input audio transcription is updated. https://docs.x.ai/developers/rest-api-reference/inference/voice#conversation.item.input_audio_transcription.updated

Event constant: VoiceAgentAPIEvents.ConversationItemInputAudioTranscriptionUpdated

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

InputAudioBufferCommitted

Input audio buffer has been committed. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.committed

Event constant: VoiceAgentAPIEvents.InputAudioBufferCommitted

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Input audio buffer has been committed as a user message. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `input_audio_buffer.committed`.
`data.previous_item_id`	`string`	✓	ID of the preceding conversation item.
`data.item_id`	`string`	✓	ID of the newly created user message item.

Example data:

{
  "event_id": "event_1121",
  "type": "input_audio_buffer.committed",
  "previous_item_id": "msg_001",
  "item_id": "msg_002"
}

InputAudioBufferCleared

Input audio buffer has been cleared. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.cleared

Event constant: VoiceAgentAPIEvents.InputAudioBufferCleared

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Confirms the input audio buffer has been cleared. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `input_audio_buffer.cleared`.

Example data:

{
  "event_id": "event_1122",
  "type": "input_audio_buffer.cleared"
}

InputAudioBufferSpeechStarted

Notify the client the server’s VAD has detected the start of a speech. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.speech_started

Event constant: VoiceAgentAPIEvents.InputAudioBufferSpeechStarted

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Notifies that the server’s VAD detected the start of speech. Only available with server_vad turn detection. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `input_audio_buffer.speech_started`.
`data.item_id`	`string`	✓	ID of the associated message item.
`data.audio_start_ms`	`integer`	✓	Millisecond offset in the audio buffer where speech was detected.

Example data:

{
  "event_id": "event_1516",
  "type": "input_audio_buffer.speech_started",
  "item_id": "msg_003"
}

InputAudioBufferSpeechStopped

Notify the client the server’s VAD has detected the end of a speech. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.speech_stopped

Event constant: VoiceAgentAPIEvents.InputAudioBufferSpeechStopped

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Notifies that the server’s VAD detected the end of speech. Only available with server_vad turn detection. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `input_audio_buffer.speech_stopped`.
`data.item_id`	`string`	✓	ID of the associated message item.
`data.audio_end_ms`	`integer`	✓	Millisecond offset in the audio buffer where speech ended.

Example data:

{
  "event_id": "event_1516",
  "type": "input_audio_buffer.speech_stopped",
  "item_id": "msg_003"
}

InputAudioBufferTimeoutTriggered

Returned when the input audio buffer idle timeout is triggered. https://docs.x.ai/developers/rest-api-reference/inference/voice#input_audio_buffer.timeout_triggered

Event constant: VoiceAgentAPIEvents.InputAudioBufferTimeoutTriggered

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseCreated

A new assistant response turn is in progress. Audio delta created from this assistant turn will have the same response id. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.created

Event constant: VoiceAgentAPIEvents.ResponseCreated

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	A new assistant response turn is in progress. Audio deltas from this turn share the same response_id. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.created`.
`data.response`	`object`	✓	The response object.

Example data:

{
  "event_id": "event_2930",
  "type": "response.created",
  "response": {
    "id": "resp_001",
    "object": "realtime.response",
    "status": "in_progress",
    "output": []
  }
}

ResponseDone

The assistant’s response is completed. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.done

Event constant: VoiceAgentAPIEvents.ResponseDone

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The assistant’s response is completed. Sent after all audio and transcript deltas. Ready for the client to add a new conversation item. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.done`.
`data.response`	`object`	✓	The completed response object.

Example data:

{
  "event_id": "event_3132",
  "type": "response.done",
  "response": {
    "id": "resp_001",
    "object": "realtime.response",
    "status": "completed"
  }
}

ResponseOutputItemAdded

A new assistant response is added to message history. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_item.added

Event constant: VoiceAgentAPIEvents.ResponseOutputItemAdded

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	A new assistant response item is added to the message history. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.output_item.added`.
`data.response_id`	`string`	✓	ID of the response this item belongs to.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.item`	`object`	✓	The output item that was added.

Example data:

{
  "event_id": "event_3334",
  "type": "response.output_item.added",
  "response_id": "resp_001",
  "output_index": 0,
  "item": {
    "id": "msg_007",
    "object": "realtime.item",
    "type": "message",
    "status": "in_progress",
    "role": "assistant",
    "content": []
  }
}

ResponseOutputItemDone

A new assistant response is done. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_item.done

Event constant: VoiceAgentAPIEvents.ResponseOutputItemDone

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	An output item is complete. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.output_item.done`.
`data.response_id`	`string`	✓	ID of the response this item belongs to.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.item`	`object`	✓	The completed output item.

Example data:

{
  "event_id": "event_3335",
  "type": "response.output_item.done",
  "response_id": "resp_001",
  "output_index": 0,
  "item": {
    "id": "msg_007",
    "object": "realtime.item",
    "type": "message",
    "status": "completed",
    "role": "assistant",
    "content": []
  }
}

ResponseOutputTextDelta

Text delta of the assistant response. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_text.delta

Event constant: VoiceAgentAPIEvents.ResponseOutputTextDelta

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseOutputTextDone

The text of the assistant response has finished generating. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_text.done

Event constant: VoiceAgentAPIEvents.ResponseOutputTextDone

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseOutputAudioTranscriptDelta

Audio transcript delta of the assistant response. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_audio_transcript.delta

Event constant: VoiceAgentAPIEvents.ResponseOutputAudioTranscriptDelta

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Streaming text transcript delta of the assistant’s audio response. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.output_audio_transcript.delta`.
`data.response_id`	`string`	✓	ID of the response.
`data.item_id`	`string`	✓	ID of the output item.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.content_index`	`integer`	✓	Index of the content part within the item.
`data.delta`	`string`	✓	Text transcript fragment.

Example data:

{
  "event_id": "event_4950",
  "type": "response.output_audio_transcript.delta",
  "response_id": "resp_001",
  "item_id": "msg_008",
  "delta": "Hello! I'm doing"
}

ResponseOutputAudioTranscriptDone

The audio transcript delta of the assistant response has finished generating. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_audio_transcript.done

Event constant: VoiceAgentAPIEvents.ResponseOutputAudioTranscriptDone

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The audio transcript for this assistant turn has finished generating. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.output_audio_transcript.done`.
`data.response_id`	`string`	✓	ID of the response.
`data.item_id`	`string`	✓	ID of the output item.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.content_index`	`integer`	✓	Index of the content part within the item.
`data.transcript`	`string`	✓	The complete transcript text.

Example data:

{
  "event_id": "event_5152",
  "type": "response.output_audio_transcript.done",
  "response_id": "resp_001",
  "item_id": "msg_008"
}

ResponseOutputAudioDone

Notifies client that the audio for this turn has finished generating. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.output_audio.done

Event constant: VoiceAgentAPIEvents.ResponseOutputAudioDone

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Audio generation for this assistant turn has finished. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.output_audio.done`.
`data.response_id`	`string`	✓	ID of the response.
`data.item_id`	`string`	✓	ID of the output item.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.content_index`	`integer`	✓	Index of the content part within the item.

Example data:

{
  "event_id": "event_5152",
  "type": "response.output_audio.done",
  "response_id": "resp_001",
  "item_id": "msg_008"
}

ResponseContentPartAdded

Notifies client that the content part added. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.content_part.added

Event constant: VoiceAgentAPIEvents.ResponseContentPartAdded

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	A content part starts within an output item. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.content_part.added`.
`data.response_id`	`string`	✓	ID of the response.
`data.item_id`	`string`	✓	ID of the output item.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.content_index`	`integer`	✓	Index of the content part within the item.
`data.part`	`object`	✓	The content part.

Example data:

{
  "event_id": "event_3336",
  "type": "response.content_part.added",
  "response_id": "resp_001",
  "item_id": "msg_007",
  "output_index": 0,
  "content_index": 0,
  "part": {
    "type": "audio"
  }
}

ResponseContentPartDone

Notifies client that the content part done. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.content_part.done

Event constant: VoiceAgentAPIEvents.ResponseContentPartDone

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	A content part finishes. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.content_part.done`.
`data.response_id`	`string`	✓	ID of the response.
`data.item_id`	`string`	✓	ID of the output item.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.content_index`	`integer`	✓	Index of the content part within the item.
`data.part`	`object`	✓	The completed content part.

Example data:

{
  "event_id": "event_3337",
  "type": "response.content_part.done",
  "response_id": "resp_001",
  "item_id": "msg_007",
  "output_index": 0,
  "content_index": 0,
  "part": {
    "type": "audio"
  }
}

ResponseFunctionCallArgumentsDelta

Function call arguments delta. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.function_call_arguments.delta

Event constant: VoiceAgentAPIEvents.ResponseFunctionCallArgumentsDelta

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseFunctionCallArgumentsDone

Function call triggered with complete arguments. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.function_call_arguments.done

Event constant: VoiceAgentAPIEvents.ResponseFunctionCallArgumentsDone

Payload

Field	Type	Req.	Description
`data`	`{ customEvent?: string; payload?: Object; }`	✗	A function call has been triggered with complete arguments. Your code should execute the function and return results via `conversation.item.create` with type `function_call_output`. See the partner event documentation.
`data.event_id`	`string`	✓	Unique event identifier.
`data.type`	`string`	✓	Always `response.function_call_arguments.done`.
`data.response_id`	`string`	✓	ID of the response.
`data.item_id`	`string`	✓	ID of the function call item.
`data.output_index`	`integer`	✓	Index of the output item in the response.
`data.call_id`	`string`	✓	Unique ID for this function call. Pass this as `call_id` in the `conversation.item.create` event with type `function_call_output`.
`data.name`	`string`	✓	Name of the function to call.
`data.arguments`	`string`	✓	JSON string of the function arguments.

Example data:

{
  "event_id": "event_fc01",
  "type": "response.function_call_arguments.done",
  "response_id": "resp_001",
  "item_id": "msg_009",
  "output_index": 0,
  "call_id": "call_001",
  "name": "get_weather",
  "arguments": "{\"location\": \"San Francisco\"}"
}

ResponseMCPCallArgumentsDelta

MCP call arguments delta. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_call_arguments.delta

Event constant: VoiceAgentAPIEvents.ResponseMCPCallArgumentsDelta

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseMCPCallArgumentsDone

MCP call arguments completed. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_call_arguments.done

Event constant: VoiceAgentAPIEvents.ResponseMCPCallArgumentsDone

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseMCPCallInProgress

MCP call is in progress. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_call.in_progress

Event constant: VoiceAgentAPIEvents.ResponseMCPCallInProgress

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseMCPCallCompleted

MCP call completed. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_call.completed

Event constant: VoiceAgentAPIEvents.ResponseMCPCallCompleted

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ResponseMCPCallFailed

MCP call failed. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_call.failed

Event constant: VoiceAgentAPIEvents.ResponseMCPCallFailed

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

MCPListToolsInProgress

MCP list tools is in progress. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_list_tools.in_progress

Event constant: VoiceAgentAPIEvents.MCPListToolsInProgress

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

MCPListToolsCompleted

MCP list tools completed. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_list_tools.completed

Event constant: VoiceAgentAPIEvents.MCPListToolsCompleted

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

MCPListToolsFailed

MCP list tools failed. https://docs.x.ai/developers/rest-api-reference/inference/voice#response.mcp_list_tools.failed

Event constant: VoiceAgentAPIEvents.MCPListToolsFailed

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

WebSocketError

The WebSocket error response event.

Event constant: VoiceAgentAPIEvents.WebSocketError

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

ConnectorInformation

Contains information about connector.

Event constant: VoiceAgentAPIEvents.ConnectorInformation

Payload

No event-specific payload columns are listed here; this callback still receives the common client and data fields. For data, see the partner documentation for the exact JSON shape.

Related guides

Contents

Usage

Factory functions

createRealtimeTTSPlayer

createVoiceAgentAPIClient

VoiceAgentAPIClient

Methods

addEventListener

clearMediaBuffer

close

conversationItemCreate

conversationItemDelete

conversationItemTruncate

id

inputAudioBufferClear

inputAudioBufferCommit

removeEventListener

responseCancel

responseCreate

sendMediaTo

sessionUpdate

stopMediaTo

webSocketId

Events

Events.WebSocketMediaStarted

Events.WebSocketMediaEnded

VoiceAgentAPIEvents

Unknown

Error

SessionCreated

ConversationCreated

SessionUpdated

ConversationItemAdded

ConversationItemDeleted

ConversationItemTruncated

ConversationItemInputAudioTranscriptionCompleted

ConversationItemInputAudioTranscriptionUpdated

InputAudioBufferCommitted

InputAudioBufferCleared

InputAudioBufferSpeechStarted

InputAudioBufferSpeechStopped

InputAudioBufferTimeoutTriggered

ResponseCreated

ResponseDone

ResponseOutputItemAdded

ResponseOutputItemDone

ResponseOutputTextDelta

ResponseOutputTextDone

ResponseOutputAudioTranscriptDelta

ResponseOutputAudioTranscriptDone

ResponseOutputAudioDone

ResponseContentPartAdded

ResponseContentPartDone

ResponseFunctionCallArgumentsDelta

ResponseFunctionCallArgumentsDone

ResponseMCPCallArgumentsDelta

ResponseMCPCallArgumentsDone

ResponseMCPCallInProgress

ResponseMCPCallCompleted

ResponseMCPCallFailed

MCPListToolsInProgress

MCPListToolsCompleted

MCPListToolsFailed

WebSocketError

ConnectorInformation