ElevenLabs | Voximplant Voice AI

ElevenLabs provides a VoxEngine client for connecting a call or media unit to the ElevenLabs Agents WebSocket API.

Use ElevenLabs.createAgentsClient(...) to create an AgentsClient for the current scenario.

Learn how ElevenLabs Agents fits into a VoxEngine call flow.

Usage: required module import and basic flow.
Factory functions: create the ElevenLabs Agents client.
Methods: conversation, tools, and contextual client messages.
Events: WebSocket media bridge events.
AgentsEvents: ElevenLabs Agents event names and payload fields.

Usage

Add the module before using the namespace:

require(Modules.ElevenLabs);

Create the client, bridge media, and listen for ElevenLabs Agents events.

Factory functions

createAgentsClient

Creates a new ElevenLabs.AgentsClient instance.

createAgentsClient(parameters: AgentsClientParameters): Promise<ElevenLabs.AgentsClient>

Parameters

Parameter	Type	Req.	Description
`parameters`	`AgentsClientParameters`	✓

Returns

Type	Description
`Promise<ElevenLabs.AgentsClient>`	Resolves to the `ElevenLabs.AgentsClient` instance.

createRealtimeTTSPlayer

Creates a new ElevenLabs.RealtimeTTSPlayer instance with the specified text (TTS is used to play the text). You can attach media streams later via the ElevenLabs.RealtimeTTSPlayer.sendMediaTo or VoxEngine.sendMediaBetween methods. NOTE: this method uses 11labs initializeConnection method internally.

createRealtimeTTSPlayer(text: string, parameters?: RealtimeTTSPlayerParameters): RealtimeTTSPlayer

Parameters

Parameter	Type	Req.	Description
`text`	`string`	✓
`parameters`	`RealtimeTTSPlayerParameters`	✗

Returns

Type	Description
`RealtimeTTSPlayer`	The requested `RealtimeTTSPlayer` value.

AgentsClient

Methods

addEventListener

Adds a handler for the specified ElevenLabs.AgentsEvents or ElevenLabs.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

addEventListener(event: ElevenLabs.Events | ElevenLabs.AgentsEvents | string, callback: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`ElevenLabs.Events \| ElevenLabs.AgentsEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✓	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

clearMediaBuffer

Clears the ElevenLabs WebSocket media buffer.

clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`ClearMediaBufferParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

clientToolResult

Result of the client tool call. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Client-Tool-Result

clientToolResult(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

close

Closes the ElevenLabs connection (over WebSocket) or connection attempt.

close(): void

Parameters

This method does not accept parameters.

Returns

Type	Description
`void`	Does not return a value.

contextualUpdate

Allows to send non-interrupting background information to the conversation. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Contextual-Update

contextualUpdate(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

conversationInitiationClientData

Defines what can be customized when starting a conversation. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Conversation-Initiation-Client-Data

conversationInitiationClientData(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

id

Returns the AgentsClient id.

id(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

removeEventListener

Removes a handler for the specified ElevenLabs.AgentsEvents or ElevenLabs.Events event.

removeEventListener(event: ElevenLabs.Events | ElevenLabs.AgentsEvents | string, callback?: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`ElevenLabs.Events \| ElevenLabs.AgentsEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✗	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

sendMediaTo

Starts sending media from the ElevenLabs (via WebSocket) to the media unit. ElevenLabs works in real time.

sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓
`parameters`	`SendMediaParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

stopMediaTo

Stops sending media from the ElevenLabs (via WebSocket) to the media unit.

stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓

Returns

Type	Description
`void`	Does not return a value.

userMessage

Allows to send user text messages to the conversation. https://elevenlabs.io/docs/agents-platform/customization/events/client-to-server-events#user-messages

userMessage(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

webSocketId

Returns the ElevenLabs WebSocket id.

webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

Events

These events describe audio received through the ElevenLabs WebSocket media bridge.

Events.WebSocketMediaStarted

Triggered when the audio stream sent by a third party through an ElevenLabs WebSocket is started playing.

Event constant: Events.WebSocketMediaStarted

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`encoding`	`string`	✗	Audio encoding formats.
`customParameters`	`{ [key: string]: string }`	✗	Custom parameters.

Events.WebSocketMediaEnded

Triggers after the end of the audio stream sent by a third party through an ElevenLabs WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`mediaInfo`	`WebSocketMediaInfo`	✗	Information about the audio stream that can be obtained after the stream stops or pauses (1 second of silence).

AgentsEvents

These events mirror server messages from the ElevenLabs Agents WebSocket API. The data field contains the provider event payload.

Unknown

The unknown event.

Event constant: AgentsEvents.Unknown

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

HTTPResponse

The HTTP response event.

Event constant: AgentsEvents.HTTPResponse

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

ConversationInitiationMetadata

Automatically sent when starting a conversation. Initializes conversation settings and parameters. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#conversation_initiation_metadata

Event constant: AgentsEvents.ConversationInitiationMetadata

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Initial connection metadata See the partner event documentation.
`data.type`	`”conversation_initiation_metadata”`	✓	Provider-defined field.
`data.conversation_initiation_metadata_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "conversation_initiation_metadata"
}

Ping

Health check event requiring immediate response. Used to maintain WebSocket connection. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#ping

Event constant: AgentsEvents.Ping

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Keepalive ping from server See the partner event documentation.
`data.type`	`"ping"`	✓	Provider-defined field.
`data.ping_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "ping"
}

UserTranscript

Contains finalized speech-to-text results. Represents complete user utterances. Used for conversation history. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#user_transcript

Event constant: AgentsEvents.UserTranscript

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Final transcription of user speech See the partner event documentation.
`data.type`	`”user_transcript”`	✓	Provider-defined field.
`data.user_transcription_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "user_transcript"
}

AgentResponse

Contains complete agent message. Sent with first audio chunk. Used for display and history. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent_response

Event constant: AgentsEvents.AgentResponse

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Agent’s text response See the partner event documentation.
`data.type`	`”agent_response”`	✓	Provider-defined field.
`data.agent_response_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "agent_response"
}

AgentResponseCorrection

Contains truncated response after interruption. Updates displayed message. Maintains conversation accuracy. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent_response_correction

Event constant: AgentsEvents.AgentResponseCorrection

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Corrected response after interruption See the partner event documentation.
`data.type`	`”agent_response_correction”`	✓	Provider-defined field.
`data.agent_response_correction_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "agent_response_correction"
}

Interruption

Contains event id of interrupted event. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Interruption

Event constant: AgentsEvents.Interruption

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	User interrupted agent speech See the partner event documentation.
`data.type`	`”interruption”`	✓	Provider-defined field.
`data.interruption_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "interruption"
}

ContextualUpdate

Contains contextual information to be added to the conversation state. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Contextual-Update

Event constant: AgentsEvents.ContextualUpdate

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The ‘payload’ parameter contains the event’s data.

ClientToolCall

Represents a function call the agent wants the client to execute. Contains tool name, tool call ID, and parameters. Requires client-side execution of the function and sending the result back to the server. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#client_tool_call

Event constant: AgentsEvents.ClientToolCall

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Tool for client to execute See the partner event documentation.
`data.type`	`”client_tool_call”`	✓	Provider-defined field.
`data.client_tool_call`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "client_tool_call"
}

VadScore

Voice Activity Detection score event. Indicates the probability that the user is speaking. Values range from 0 to 1, where higher values indicate higher confidence of speech. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#vad_score

Event constant: AgentsEvents.VadScore

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The ‘payload’ parameter contains the event’s data.

InternalTentativeAgentResponse

Contains preliminary text from the agent. https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Internal-Tentative-Agent-Response

Event constant: AgentsEvents.InternalTentativeAgentResponse

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Internal tentative agent response (not for public use) See the partner event documentation.
`data.type`	`”internal_tentative_agent_response”`	✓	Provider-defined field.
`data.tentative_agent_response_internal_event`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "internal_tentative_agent_response"
}

WebSocketError

The WebSocket error response event.

Event constant: AgentsEvents.WebSocketError

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The ‘payload’ parameter contains the event’s data.

ConnectorInformation

Contains information about connector.

Event constant: AgentsEvents.ConnectorInformation

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	The ‘payload’ parameter contains the event’s data.

AgentToolResponse

Indicates when the agent has executed a tool function. Contains tool metadata and execution status. Provides visibility into agent tool usage during conversations. https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent_tool_response

Event constant: AgentsEvents.AgentToolResponse

Payload

Field	Type	Req.	Description
`client`	`AgentsClient`	✓	The ElevenLabs.AgentsClient instance.
`data`	`{ customEvent?: string; payload?: Object; }`	✗	Result of agent tool execution See the partner event documentation.
`data.type`	`”agent_tool_response”`	✓	Provider-defined field.
`data.agent_tool_response`	`unknown`	✓	Provider-defined field.

Example data:

{
  "type": "agent_tool_response"
}

Related guides

Contents

Usage

Factory functions

createAgentsClient

createRealtimeTTSPlayer

AgentsClient

Methods

addEventListener

clearMediaBuffer

clientToolResult

close

contextualUpdate

conversationInitiationClientData

id

removeEventListener

sendMediaTo

stopMediaTo

userMessage

webSocketId

Events

Events.WebSocketMediaStarted

Events.WebSocketMediaEnded

AgentsEvents

Unknown

HTTPResponse

ConversationInitiationMetadata

Ping

UserTranscript

AgentResponse

AgentResponseCorrection

Interruption

ContextualUpdate

ClientToolCall

VadScore

InternalTentativeAgentResponse

WebSocketError

ConnectorInformation

AgentToolResponse