XAI

XAI provides VoxEngine clients for the xAI Grok Voice Agent API and Realtime TTS over WebSocket.

Use XAI.createVoiceAgentAPIClient(...) to create a VoiceAgentAPIClient for voice-agent scenarios. After the client is created, call methods such as sendMediaTo, responseCreate, and addEventListener on that client instance.

Use XAI.createRealtimeTTSPlayer(...) when you only need realtime text-to-speech playback.

Related guides

Contents

• Usage: required module import and basic flow.
• Factory functions: create the XAI Voice Agent client.
• VoiceAgentAPIClientParameters: API key, tracing, privacy, and WebSocket options.
• VoiceAgentAPIClient: runtime client object returned by the factory.
• Methods: media, response, and connection control methods.
• Events: WebSocket media start and end payloads.
• VoiceAgentAPIEvents: xAI Voice Agent API event names and payload fields.

Usage

Add the module before using the namespace:

1
require(Modules.XAI);

Create the client, bridge media, and listen for both WebSocket media events and Voice Agent API events.

Factory functions

createRealtimeTTSPlayer

Creates a new XAI.RealtimeTTSPlayer instance. You can attach media streams later via the XAI.RealtimeTTSPlayer.sendMediaTo or VoxEngine.sendMediaBetween methods.

1
createRealtimeTTSPlayer(parameters?: {
2
  statistics?: boolean;
3
  trace?: boolean;
4
  privacy?: boolean;
5
  connectionParameters?: Object;
6
  apiKey?: string;
7
}): RealtimeTTSPlayer

The optional parameters object is typed as XAI.RealtimeTTSPlayerParameters.

Parameters

Returns

createVoiceAgentAPIClient

Creates a new XAI.VoiceAgentAPIClient instance.

1
createVoiceAgentAPIClient(parameters: {
2
  statistics?: boolean;
3
  trace?: boolean;
4
  privacy?: boolean;
5
  onWebSocketClose?: (event: object) => void;
6
  xAIApiKey: string;
7
  model?: string;
8
}): Promise<XAI.VoiceAgentAPIClient>

The required parameters object is typed as XAI.VoiceAgentAPIClientParameters.

Parameters

Returns

VoiceAgentAPIClient

Methods

addEventListener

Adds a handler for the specified XAI.VoiceAgentAPIEvents or XAI.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

1
addEventListener(event: XAI.Events | XAI.VoiceAgentAPIEvents | string, callback: (event: object) => any): void

Parameters

Returns

clearMediaBuffer

Clears the XAI WebSocket media buffer.

1
clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Returns

close

Closes the XAI connection (over WebSocket) or connection attempt.

1
close(): void

Parameters

This method does not accept parameters.

Returns

conversationItemCreate

Create a new user message. https://docs.x.ai/docs/guides/voice/agent#client

1
conversationItemCreate(parameters: Object): void

Parameters

Returns

Example parameters:

1
{
2
  "type": "conversation.item.create",
3
  "event_id": "event_345",
4
  "item": {
5
    "type": "message",
6
    "role": "user",
7
    "content": [
8
      {
9
        "type": "input_text",
10
        "text": "Hello"
11
      }
12
    ]
13
  }
14
}

id

Returns the VoiceAgentAPIClient id.

1
id(): string

Parameters

This method does not accept parameters.

Returns

inputAudioBufferClear

Clear input audio buffer. https://docs.x.ai/docs/guides/voice/agent#client-1

1
inputAudioBufferClear(parameters: Object): void

Parameters

Returns

Example parameters:

1
{
2
  "type": "input_audio_buffer.clear"
3
}

removeEventListener

Removes a handler for the specified XAI.VoiceAgentAPIEvents or XAI.Events event.

1
removeEventListener(event: XAI.Events | XAI.VoiceAgentAPIEvents | string, callback?: (event: object) => any): void

Parameters

Returns

responseCreate

Request the server to create a new assistant response when using client side vad. (This is handled automatically when using server side vad.) https://docs.x.ai/docs/guides/voice/agent#client-2

1
responseCreate(parameters: Object): void

Parameters

Returns

Example parameters:

1
{
2
  "type": "response.create"
3
}

sendMediaTo

Starts sending media from the XAI (via WebSocket) to the media unit. XAI works in real time.

1
sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Returns

sessionUpdate

Send this event to update the session’s configuration. https://docs.x.ai/docs/guides/voice/agent#client-events-1

1
sessionUpdate(parameters: Object): void

Parameters

Returns

Example parameters:

1
{
2
  "type": "session.update",
3
  "session": {
4
    "voice": "aria",
5
    "instructions": "You are a helpful assistant",
6
    "turn_detection": {
7
      "type": "server_vad"
8
    }
9
  }
10
}

stopMediaTo

Stops sending media from the XAI (via WebSocket) to the media unit.

1
stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Returns

webSocketId

Returns the XAI WebSocket id.

1
webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Events

These events describe audio received through the XAI WebSocket media bridge.

Events.WebSocketMediaStarted

Triggered when the audio stream sent by a third party through an xAI WebSocket is started playing.

Event constant: Events.WebSocketMediaStarted

Payload

Events.WebSocketMediaEnded

Triggers after the end of the audio stream sent by a third party through an xAI WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

VoiceAgentAPIEvents

These events mirror server messages from the xAI Voice Agent API. The data field contains the provider event payload.

All VoiceAgentAPIEvents callbacks receive these common fields:

Per-event payload tables below show only event-specific fields (and any provider payload enrichment for data).

1
{
2
  "event_id": "event_9101",
3
  "type": "conversation.created",
4
  "conversation": {
5
    "id": "conv_001",
6
    "object": "realtime.conversation"
7
  }
8
}

1
{
2
  "event_id": "event_123",
3
  "type": "session.updated",
4
  "session": {
5
    "model": "grok-voice-fast-1.0",
6
    "instructions": "You are a helpful assistant.",
7
    "voice": "Eve",
8
    "turn_detection": {
9
      "type": "server_vad"
10
    }
11
  }
12
}

1
{
2
  "event_id": "event_1920",
3
  "type": "conversation.item.added",
4
  "previous_item_id": "msg_002",
5
  "item": {
6
    "id": "msg_003",
7
    "object": "realtime.item",
8
    "type": "message",
9
    "status": "completed",
10
    "role": "user",
11
    "content": [
12
      {
13
        "type": "input_audio",
14
        "transcript": "hello how are you"
15
      }
16
    ]
17
  }
18
}

1
{
2
  "event_id": "event_2122",
3
  "type": "conversation.item.input_audio_transcription.completed",
4
  "item_id": "msg_003",
5
  "transcript": "Hello, how are you?"
6
}

1
{
2
  "event_id": "event_1121",
3
  "type": "input_audio_buffer.committed",
4
  "previous_item_id": "msg_001",
5
  "item_id": "msg_002"
6
}

1
{
2
  "event_id": "event_1122",
3
  "type": "input_audio_buffer.cleared"
4
}

1
{
2
  "event_id": "event_1516",
3
  "type": "input_audio_buffer.speech_started",
4
  "item_id": "msg_003"
5
}

1
{
2
  "event_id": "event_1516",
3
  "type": "input_audio_buffer.speech_stopped",
4
  "item_id": "msg_003"
5
}

1
{
2
  "event_id": "event_2930",
3
  "type": "response.created",
4
  "response": {
5
    "id": "resp_001",
6
    "object": "realtime.response",
7
    "status": "in_progress",
8
    "output": []
9
  }
10
}

1
{
2
  "event_id": "event_3132",
3
  "type": "response.done",
4
  "response": {
5
    "id": "resp_001",
6
    "object": "realtime.response",
7
    "status": "completed"
8
  }
9
}

1
{
2
  "event_id": "event_3334",
3
  "type": "response.output_item.added",
4
  "response_id": "resp_001",
5
  "output_index": 0,
6
  "item": {
7
    "id": "msg_007",
8
    "object": "realtime.item",
9
    "type": "message",
10
    "status": "in_progress",
11
    "role": "assistant",
12
    "content": []
13
  }
14
}

1
{
2
  "event_id": "event_3335",
3
  "type": "response.output_item.done",
4
  "response_id": "resp_001",
5
  "output_index": 0,
6
  "item": {
7
    "id": "msg_007",
8
    "object": "realtime.item",
9
    "type": "message",
10
    "status": "completed",
11
    "role": "assistant",
12
    "content": []
13
  }
14
}

1
{
2
  "event_id": "event_4950",
3
  "type": "response.output_audio_transcript.delta",
4
  "response_id": "resp_001",
5
  "item_id": "msg_008",
6
  "delta": "Hello! I'm doing"
7
}

1
{
2
  "event_id": "event_5152",
3
  "type": "response.output_audio_transcript.done",
4
  "response_id": "resp_001",
5
  "item_id": "msg_008"
6
}

1
{
2
  "event_id": "event_5152",
3
  "type": "response.output_audio.done",
4
  "response_id": "resp_001",
5
  "item_id": "msg_008"
6
}

1
{
2
  "event_id": "event_3336",
3
  "type": "response.content_part.added",
4
  "response_id": "resp_001",
5
  "item_id": "msg_007",
6
  "output_index": 0,
7
  "content_index": 0,
8
  "part": {
9
    "type": "audio"
10
  }
11
}

1
{
2
  "event_id": "event_3337",
3
  "type": "response.content_part.done",
4
  "response_id": "resp_001",
5
  "item_id": "msg_007",
6
  "output_index": 0,
7
  "content_index": 0,
8
  "part": {
9
    "type": "audio"
10
  }
11
}

1
{
2
  "event_id": "event_fc01",
3
  "type": "response.function_call_arguments.done",
4
  "response_id": "resp_001",
5
  "item_id": "msg_009",
6
  "output_index": 0,
7
  "call_id": "call_001",
8
  "name": "get_weather",
9
  "arguments": "{\"location\": \"San Francisco\"}"
10
}