> For a complete documentation index, fetch https://docs.voximplant.ai/llms.txt # ElevenLabs ElevenLabs provides a VoxEngine client for connecting a call or media unit to the ElevenLabs Agents WebSocket API. Use `ElevenLabs.createAgentsClient(...)` to create an `AgentsClient` for the current scenario. ## Related guides Learn how ElevenLabs Agents fits into a VoxEngine call flow. ## Contents * [Usage](#usage): required module import and basic flow. * [Factory functions](#factory-functions): create the ElevenLabs Agents client. * [Methods](#methods): conversation, tools, and contextual client messages. * [Events](#events): WebSocket media bridge events. * [AgentsEvents](#agentsevents): ElevenLabs Agents event names and payload fields. ## Usage Add the module before using the namespace: ```js require(Modules.ElevenLabs); ``` Create the client, bridge media, and listen for ElevenLabs Agents events. ## Factory functions ### createAgentsClient Creates a new [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. ```ts createAgentsClient(parameters: AgentsClientParameters): Promise ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | ------------------------------------------ | ---- | ----------- | | `parameters` | AgentsClientParameters | ✓ | | **Returns** | Type | Description | | ---------------------------------------------- | -------------------------------------------------------------------- | | Promise\ | Resolves to the [`ElevenLabs.AgentsClient`](#agentsclient) instance. | ### createRealtimeTTSPlayer Creates a new [ElevenLabs.RealtimeTTSPlayer](/api-reference/voxengine/eleven-labs/realtime-tts-player) instance with the specified text (TTS is used to play the text). You can attach media streams later via the `ElevenLabs.RealtimeTTSPlayer.sendMediaTo` or `VoxEngine.sendMediaBetween` methods. NOTE: this method uses 11labs `initializeConnection`([https://elevenlabs.io/docs/api-reference/text-to-speech/v-1-text-to-speech-voice-id-stream-input#send.initializeConnection](https://elevenlabs.io/docs/api-reference/text-to-speech/v-1-text-to-speech-voice-id-stream-input#send.initializeConnection)) method internally. ```ts createRealtimeTTSPlayer(text: string, parameters?: RealtimeTTSPlayerParameters): RealtimeTTSPlayer ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | ------------------------------------------------------ | ---- | ----------- | | `text` | `string` | ✓ | | | `parameters` | RealtimeTTSPlayerParameters | ✗ | | **Returns** | Type | Description | | ------------------------------------- | ---------------------------------------- | | RealtimeTTSPlayer | The requested `RealtimeTTSPlayer` value. | ## AgentsClient ## Methods ### addEventListener Adds a handler for the specified [ElevenLabs.AgentsEvents](/api-reference/voxengine/eleven-labs#agentsevents) or [ElevenLabs.Events](/api-reference/voxengine/eleven-labs#events) event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called. ```ts addEventListener(event: ElevenLabs.Events | ElevenLabs.AgentsEvents | string, callback: (event: object) => any): void ``` **Parameters** | Parameter | Type | Req. | Description | | ---------- | --------------------------------------------------------------------------------- | ---- | --------------------------------------------- | | `event` | ElevenLabs.Events \| ElevenLabs.AgentsEvents \| string | ✓ | Event constant or event name to subscribe to. | | `callback` | (event: object) => any | ✓ | Function called when the event is emitted. | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### clearMediaBuffer Clears the ElevenLabs WebSocket media buffer. ```ts clearMediaBuffer(parameters?: ClearMediaBufferParameters): void ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | ----------------------------------------------------- | ---- | ----------- | | `parameters` | ClearMediaBufferParameters | ✗ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### clientToolResult Result of the client tool call. [https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Client-Tool-Result](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Client-Tool-Result) ```ts clientToolResult(parameters: Object): void ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | -------- | ---- | ----------- | | `parameters` | `Object` | ✓ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### close Closes the ElevenLabs connection (over WebSocket) or connection attempt. ```ts close(): void ``` **Parameters** This method does not accept parameters. **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### contextualUpdate Allows to send non-interrupting background information to the conversation. [https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Contextual-Update](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Contextual-Update) ```ts contextualUpdate(parameters: Object): void ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | -------- | ---- | ----------- | | `parameters` | `Object` | ✓ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### conversationInitiationClientData Defines what can be customized when starting a conversation. [https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Conversation-Initiation-Client-Data](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#send.Conversation-Initiation-Client-Data) ```ts conversationInitiationClientData(parameters: Object): void ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | -------- | ---- | ----------- | | `parameters` | `Object` | ✓ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### id Returns the AgentsClient id. ```ts id(): string ``` **Parameters** This method does not accept parameters. **Returns** | Type | Description | | -------- | --------------------------- | | `string` | The requested string value. | ### removeEventListener Removes a handler for the specified [ElevenLabs.AgentsEvents](/api-reference/voxengine/eleven-labs#agentsevents) or [ElevenLabs.Events](/api-reference/voxengine/eleven-labs#events) event. ```ts removeEventListener(event: ElevenLabs.Events | ElevenLabs.AgentsEvents | string, callback?: (event: object) => any): void ``` **Parameters** | Parameter | Type | Req. | Description | | ---------- | --------------------------------------------------------------------------------- | ---- | --------------------------------------------- | | `event` | ElevenLabs.Events \| ElevenLabs.AgentsEvents \| string | ✓ | Event constant or event name to subscribe to. | | `callback` | (event: object) => any | ✗ | Function called when the event is emitted. | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### sendMediaTo Starts sending media from the ElevenLabs (via WebSocket) to the media unit. ElevenLabs works in real time. ```ts sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | --------------------------------------- | ---- | ----------- | | `mediaUnit` | `VoxMediaUnit` | ✓ | | | `parameters` | SendMediaParameters | ✗ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### stopMediaTo Stops sending media from the ElevenLabs (via WebSocket) to the media unit. ```ts stopMediaTo(mediaUnit: VoxMediaUnit): void ``` **Parameters** | Parameter | Type | Req. | Description | | ----------- | -------------- | ---- | ----------- | | `mediaUnit` | `VoxMediaUnit` | ✓ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### userMessage Allows to send user text messages to the conversation. [https://elevenlabs.io/docs/agents-platform/customization/events/client-to-server-events#user-messages](https://elevenlabs.io/docs/agents-platform/customization/events/client-to-server-events#user-messages) ```ts userMessage(parameters: Object): void ``` **Parameters** | Parameter | Type | Req. | Description | | ------------ | -------- | ---- | ----------- | | `parameters` | `Object` | ✓ | | **Returns** | Type | Description | | ------ | ------------------------ | | `void` | Does not return a value. | ### webSocketId Returns the ElevenLabs WebSocket id. ```ts webSocketId(): string ``` **Parameters** This method does not accept parameters. **Returns** | Type | Description | | -------- | --------------------------- | | `string` | The requested string value. | ## Events These events describe audio received through the ElevenLabs WebSocket media bridge. ### WebSocketMediaStarted Triggered when the audio stream sent by a third party through an ElevenLabs WebSocket is started playing. Event constant: `Events.WebSocketMediaStarted` **Payload** | Field | Type | Req. | Description | | ------------------------------------ | ---------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `tag` | `string` | ✗ | Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time. | | `encoding` | `string` | ✗ | Audio encoding formats. | | customParameters | \{ \[key: string]: string } | ✗ | Custom parameters. | ### WebSocketMediaEnded Triggers after the end of the audio stream sent by a third party through an ElevenLabs WebSocket (**1 second of silence**). Event constant: `Events.WebSocketMediaEnded` **Payload** | Field | Type | Req. | Description | | ----------- | -------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `tag` | `string` | ✗ | Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time. | | `mediaInfo` | WebSocketMediaInfo | ✗ | Information about the audio stream that can be obtained after the stream stops or pauses (**1 second of silence**). | ## AgentsEvents These events mirror server messages from the ElevenLabs Agents WebSocket API. The `data` field contains the provider event payload. The unknown event. Event constant: `AgentsEvents.Unknown` **Payload** | Field | Type | Req. | Description | | -------- | -------------- | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | This event payload is provider-specific. See the [partner event documentation](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket) for the full payload shape. | The HTTP response event. Event constant: `AgentsEvents.HTTPResponse` **Payload** | Field | Type | Req. | Description | | -------- | -------------- | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | This event payload is provider-specific. See the [partner event documentation](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket) for the full payload shape. | Automatically sent when starting a conversation. Initializes conversation settings and parameters. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#conversation\_initiation\_metadata](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#conversation_initiation_metadata) Event constant: `AgentsEvents.ConversationInitiationMetadata` **Payload** | Field | Type | Req. | Description | | --------------------------------------------- | ------------------------------------------------- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Initial connection metadata See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "conversation\_initiation\_metadata" | ✓ | Provider-defined field. | | `data.conversation_initiation_metadata_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "conversation_initiation_metadata" } ``` Health check event requiring immediate response. Used to maintain WebSocket connection. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#ping](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#ping) Event constant: `AgentsEvents.Ping` **Payload** | Field | Type | Req. | Description | | ----------------- | -------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Keepalive ping from server See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | `"ping"` | ✓ | Provider-defined field. | | `data.ping_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "ping" } ``` Contains finalized speech-to-text results. Represents complete user utterances. Used for conversation history. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#user\_transcript](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#user_transcript) Event constant: `AgentsEvents.UserTranscript` **Payload** | Field | Type | Req. | Description | | ------------------------------- | ------------------------------- | ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Final transcription of user speech See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "user\_transcript" | ✓ | Provider-defined field. | | `data.user_transcription_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "user_transcript" } ``` Contains complete agent message. Sent with first audio chunk. Used for display and history. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent\_response](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent_response) Event constant: `AgentsEvents.AgentResponse` **Payload** | Field | Type | Req. | Description | | --------------------------- | ------------------------------ | ---- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Agent's text response See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "agent\_response" | ✓ | Provider-defined field. | | `data.agent_response_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "agent_response" } ``` Contains truncated response after interruption. Updates displayed message. Maintains conversation accuracy. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent\_response\_correction](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent_response_correction) Event constant: `AgentsEvents.AgentResponseCorrection` **Payload** | Field | Type | Req. | Description | | -------------------------------------- | ------------------------------------------ | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Corrected response after interruption See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "agent\_response\_correction" | ✓ | Provider-defined field. | | `data.agent_response_correction_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "agent_response_correction" } ``` Contains event id of interrupted event. [https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Interruption](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Interruption) Event constant: `AgentsEvents.Interruption` **Payload** | Field | Type | Req. | Description | | ------------------------- | --------------------------- | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | User interrupted agent speech See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "interruption" | ✓ | Provider-defined field. | | `data.interruption_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "interruption" } ``` Contains contextual information to be added to the conversation state. [https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Contextual-Update](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Contextual-Update) Event constant: `AgentsEvents.ContextualUpdate` **Payload** | Field | Type | Req. | Description | | -------- | -------------- | ---- | ------------------------------------------------------------------------------------------ | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | The event's data. | Represents a function call the agent wants the client to execute. Contains tool name, tool call ID, and parameters. Requires client-side execution of the function and sending the result back to the server. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#client\_tool\_call](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#client_tool_call) Event constant: `AgentsEvents.ClientToolCall` **Payload** | Field | Type | Req. | Description | | ----------------------- | --------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Tool for client to execute See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "client\_tool\_call" | ✓ | Provider-defined field. | | `data.client_tool_call` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "client_tool_call" } ``` Voice Activity Detection score event. Indicates the probability that the user is speaking. Values range from 0 to 1, where higher values indicate higher confidence of speech. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#vad\_score](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#vad_score) Event constant: `AgentsEvents.VadScore` **Payload** | Field | Type | Req. | Description | | -------- | -------------- | ---- | ------------------------------------------------------------------------------------------ | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | The event's data. | Contains preliminary text from the agent. [https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Internal-Tentative-Agent-Response](https://elevenlabs.io/docs/agents-platform/api-reference/agents-platform/websocket#receive.Internal-Tentative-Agent-Response) Event constant: `AgentsEvents.InternalTentativeAgentResponse` **Payload** | Field | Type | Req. | Description | | ---------------------------------------------- | --------------------------------------------------- | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Internal tentative agent response (not for public use) See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "internal\_tentative\_agent\_response" | ✓ | Provider-defined field. | | `data.tentative_agent_response_internal_event` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "internal_tentative_agent_response" } ``` The WebSocket error response event. Event constant: `AgentsEvents.WebSocketError` **Payload** | Field | Type | Req. | Description | | -------- | -------------- | ---- | ------------------------------------------------------------------------------------------ | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | The event's data. | Contains information about connector. Event constant: `AgentsEvents.ConnectorInformation` **Payload** | Field | Type | Req. | Description | | -------- | -------------- | ---- | ------------------------------------------------------------------------------------------ | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | The event's data. | Indicates when the agent has executed a tool function. Contains tool metadata and execution status. Provides visibility into agent tool usage during conversations. [https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent\_tool\_response](https://elevenlabs.io/docs/agents-platform/customization/events/client-events#agent_tool_response) Event constant: `AgentsEvents.AgentToolResponse` **Payload** | Field | Type | Req. | Description | | -------------------------- | ------------------------------------ | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | `AgentsClient` | ✓ | The [ElevenLabs.AgentsClient](/api-reference/voxengine/eleven-labs#agentsclient) instance. | | `data` | `Object` | ✗ | Result of agent tool execution See the [partner event documentation](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket). | | `data.type` | "agent\_tool\_response" | ✓ | Provider-defined field. | | `data.agent_tool_response` | `unknown` | ✓ | Provider-defined field. | Example `data`: ```json { "type": "agent_tool_response" } ```