ElevenLabs

ElevenLabs provides a VoxEngine client for connecting a call or media unit to the ElevenLabs Agents WebSocket API.

Use ElevenLabs.createAgentsClient(...) to create an AgentsClient for the current scenario.

Related guides

Contents

• Usage: required module import and basic flow.
• Factory functions: create the ElevenLabs Agents client.
• Methods: conversation, tools, and contextual client messages.
• Events: WebSocket media bridge events.
• AgentsEvents: ElevenLabs Agents event names and payload fields.

Usage

Add the module before using the namespace:

1
require(Modules.ElevenLabs);

Create the client, bridge media, and listen for ElevenLabs Agents events.

Factory functions

createAgentsClient

Creates a new ElevenLabs.AgentsClient instance.

1
createAgentsClient(parameters: AgentsClientParameters): Promise<ElevenLabs.AgentsClient>

Parameters

Returns

createRealtimeTTSPlayer

Creates a new ElevenLabs.RealtimeTTSPlayer instance with the specified text (TTS is used to play the text). You can attach media streams later via the ElevenLabs.RealtimeTTSPlayer.sendMediaTo or VoxEngine.sendMediaBetween methods. NOTE: this method uses 11labs initializeConnection(https://elevenlabs.io/docs/api-reference/text-to-speech/v-1-text-to-speech-voice-id-stream-input#send.initializeConnection) method internally.

1
createRealtimeTTSPlayer(text: string, parameters?: RealtimeTTSPlayerParameters): RealtimeTTSPlayer

Parameters

Returns

AgentsClient

Methods

addEventListener

Adds a handler for the specified ElevenLabs.AgentsEvents or ElevenLabs.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

1
addEventListener(event: ElevenLabs.Events | ElevenLabs.AgentsEvents | string, callback: (event: object) => any): void

Parameters

Returns

clearMediaBuffer

Clears the ElevenLabs WebSocket media buffer.

1
clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Returns

clientToolResult

Result of the client tool call. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Client-Tool-Result

1
clientToolResult(parameters: Object): void

Parameters

Returns

close

Closes the ElevenLabs connection (over WebSocket) or connection attempt.

1
close(): void

Parameters

This method does not accept parameters.

Returns

contextualUpdate

Allows to send non-interrupting background information to the conversation. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Contextual-Update

1
contextualUpdate(parameters: Object): void

Parameters

Returns

conversationInitiationClientData

Defines what can be customized when starting a conversation. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Conversation-Initiation-Client-Data

1
conversationInitiationClientData(parameters: Object): void

Parameters

Returns

id

Returns the AgentsClient id.

1
id(): string

Parameters

This method does not accept parameters.

Returns

removeEventListener

Removes a handler for the specified ElevenLabs.AgentsEvents or ElevenLabs.Events event.

1
removeEventListener(event: ElevenLabs.Events | ElevenLabs.AgentsEvents | string, callback?: (event: object) => any): void

Parameters

Returns

sendMediaTo

Starts sending media from the ElevenLabs (via WebSocket) to the media unit. ElevenLabs works in real time.

1
sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Returns

stopMediaTo

Stops sending media from the ElevenLabs (via WebSocket) to the media unit.

1
stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Returns

userMessage

Allows to send user text messages to the conversation. https://elevenlabs.io/docs/agents-platform/customization/events/client-to-server-events#user-messages

1
userMessage(parameters: Object): void

Parameters

Returns

webSocketId

Returns the ElevenLabs WebSocket id.

1
webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Events

These events describe audio received through the ElevenLabs WebSocket media bridge.

WebSocketMediaStarted

Triggered when the audio stream sent by a third party through an ElevenLabs WebSocket is started playing.

Event constant: Events.WebSocketMediaStarted

Payload

WebSocketMediaEnded

Triggers after the end of the audio stream sent by a third party through an ElevenLabs WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

AgentsEvents

These events mirror server messages from the ElevenLabs Agents WebSocket API. The data field contains the provider event payload.

1
{
2
  "type": "conversation_initiation_metadata"
3
}

1
{
2
  "type": "ping"
3
}

1
{
2
  "type": "user_transcript"
3
}

1
{
2
  "type": "agent_response"
3
}

1
{
2
  "type": "agent_response_correction"
3
}

1
{
2
  "type": "interruption"
3
}

1
{
2
  "type": "client_tool_call"
3
}

1
{
2
  "type": "internal_tentative_agent_response"
3
}

1
{
2
  "type": "agent_tool_response"
3
}