Ultravox | Voximplant Voice AI

Ultravox provides a VoxEngine client for connecting a call or media unit to the Ultravox WebSocket API.

Use Ultravox.createWebSocketAPIClient(...) to create a WebSocketAPIClient for the current scenario.

Learn how Ultravox fits into a VoxEngine call flow.

Usage: required module import and basic flow.
Factory functions: create the Ultravox WebSocket API client.
Methods: client-to-server data messages and media helpers.
Events: WebSocket media bridge events.
WebSocketAPIEvents: Ultravox WebSocket API event names and payload fields.

Usage

Add the module before using the namespace:

1 require(Modules.Ultravox);

Create the client, bridge media, and listen for Ultravox WebSocket API events.

Factory functions

createWebSocketAPIClient

Creates a new Ultravox.WebSocketAPIClient instance.

1 createWebSocketAPIClient(parameters: WebSocketAPIClientParameters): Promise<Ultravox.WebSocketAPIClient>

Parameters

Parameter	Type	Req.	Description
`parameters`	`WebSocketAPIClientParameters`	✓

Returns

Type	Description
`Promise<Ultravox.WebSocketAPIClient>`	Resolves to the `Ultravox.WebSocketAPIClient` instance.

WebSocketAPIClient

Methods

addEventListener

Adds a handler for the specified Ultravox.WebSocketAPIEvents or Ultravox.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

1 addEventListener(event: Ultravox.Events | Ultravox.WebSocketAPIEvents | string, callback: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`Ultravox.Events \| Ultravox.WebSocketAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✓	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

clearMediaBuffer

Clears the Ultravox WebSocket media buffer.

1 clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`ClearMediaBufferParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

clientToolResult

Contains the result of a client tool invocation. https://docs.ultravox.ai/apps/datamessages#clienttoolresult-and-dataconnectiontoolresult

1 clientToolResult(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

close

Closes the Ultravox connection (over WebSocket) or connection attempt.

1 close(): void

Parameters

This method does not accept parameters.

Returns

Type	Description
`void`	Does not return a value.

dataConnectionToolResult

Contains the result of a client tool invocation. https://docs.ultravox.ai/apps/datamessages#clienttoolresult-and-dataconnectiontoolresult

1 dataConnectionToolResult(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

forcedAgentMessage

Forces the agent to say a specific message or invoke tools. https://docs.ultravox.ai/apps/datamessages#forcedagentmessage

1 forcedAgentMessage(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

hangUp

Instructs the agent to end the call with an optional farewell message. https://docs.ultravox.ai/apps/datamessages#hangup

1 hangUp(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

id

Returns the WebSocketAPIClient id.

1 id(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

inputTextMessage

Deprecated

Used to send a user message to the agent via text. NOTE: this method is deprecated, use userTextMessage instead.

1 inputTextMessage(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

removeEventListener

Removes a handler for the specified Ultravox.WebSocketAPIEvents or Ultravox.Events event.

1 removeEventListener(event: Ultravox.Events | Ultravox.WebSocketAPIEvents | string, callback?: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`Ultravox.Events \| Ultravox.WebSocketAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✗	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

sendMediaTo

Starts sending media from the Ultravox (via WebSocket) to the media unit. Ultravox works in real time.

1 sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓
`parameters`	`SendMediaParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

setOutputMedium

Sets server’s output medium to text or voice. https://docs.ultravox.ai/datamessages#setoutputmedium

1 setOutputMedium(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

stopMediaTo

Stops sending media from the Ultravox (via WebSocket) to the media unit.

1 stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓

Returns

Type	Description
`void`	Does not return a value.

userTextMessage

A user message sent via text. The message appears to the agent as if it came from the user. https://docs.ultravox.ai/apps/datamessages#usertextmessage

1 userTextMessage(parameters: Object): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

webSocketId

Returns the Ultravox WebSocket id.

1 webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

Events

These events describe audio received through the Ultravox WebSocket media bridge.

WebSocketMediaStarted

Triggered when the audio stream sent by a third party through an Ultravox WebSocket is started playing.

Event constant: Events.WebSocketMediaStarted

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`encoding`	`string`	✗	Audio encoding formats.
`customParameters`	`{ [key: string]: string }`	✗	Custom parameters.

WebSocketMediaEnded

Triggers after the end of the audio stream sent by a third party through an Ultravox WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`mediaInfo`	`WebSocketMediaInfo`	✗	Information about the audio stream that can be obtained after the stream stops or pauses (1 second of silence).

WebSocketAPIEvents

These events mirror server messages from the Ultravox WebSocket API. The data field contains the provider event payload.

Unknown

The unknown event.

Event constant: WebSocketAPIEvents.Unknown

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

HTTPResponse

The HTTP response event.

Event constant: WebSocketAPIEvents.HTTPResponse

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

State

Indicates the server’s current state. https://docs.ultravox.ai/apps/datamessages#state

Event constant: WebSocketAPIEvents.State

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

Transcript

Contains text for an utterance made during the call. https://docs.ultravox.ai/apps/datamessages#transcript

Event constant: WebSocketAPIEvents.Transcript

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

ClientToolInvocation

Sent by the server to ask the client or data connection to invoke a tool with the given parameters. https://docs.ultravox.ai/apps/datamessages#clienttoolinvocation-and-dataconnectiontoolinvocation

Event constant: WebSocketAPIEvents.ClientToolInvocation

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	This event payload is provider-specific. See the partner event documentation for the full payload shape.

DataConnectionToolInvocation

Sent by the server to ask the client or data connection to invoke a tool with the given parameters. https://docs.ultravox.ai/apps/datamessages#clienttoolinvocation-and-dataconnectiontoolinvocation

Event constant: WebSocketAPIEvents.DataConnectionToolInvocation

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	The event’s data.

Debug

Useful for application debugging. https://docs.ultravox.ai/apps/datamessages#debug

Event constant: WebSocketAPIEvents.Debug

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	The event’s data.

PlaybackClearBuffer

Used to clear buffered output audio. WebSocket only. https://docs.ultravox.ai/apps/datamessages#playbackclearbuffer

Event constant: WebSocketAPIEvents.PlaybackClearBuffer

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	The event’s data.

WebSocketError

The WebSocket error response event.

Event constant: WebSocketAPIEvents.WebSocketError

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	The event’s data.

ConnectorInformation

Contains information about connector.

Event constant: WebSocketAPIEvents.ConnectorInformation

Payload

Field	Type	Req.	Description
`client`	`WebSocketAPIClient`	✓	The Ultravox.WebSocketAPIClient instance.
`data`	`Object`	✗	The event’s data.

Related guides

Contents

Usage

Factory functions

.fa-secondary{opacity:.4} createWebSocketAPIClient

WebSocketAPIClient

Methods

.fa-secondary{opacity:.4} addEventListener

.fa-secondary{opacity:.4} clearMediaBuffer

.fa-secondary{opacity:.4} clientToolResult

.fa-secondary{opacity:.4} close

.fa-secondary{opacity:.4} dataConnectionToolResult

.fa-secondary{opacity:.4} forcedAgentMessage

.fa-secondary{opacity:.4} hangUp

.fa-secondary{opacity:.4} id

.fa-secondary{opacity:.4} inputTextMessage

Deprecated

.fa-secondary{opacity:.4} removeEventListener

.fa-secondary{opacity:.4} sendMediaTo

.fa-secondary{opacity:.4} setOutputMedium

.fa-secondary{opacity:.4} stopMediaTo

.fa-secondary{opacity:.4} userTextMessage

.fa-secondary{opacity:.4} webSocketId

Events

.fa-secondary{opacity:.4} WebSocketMediaStarted

.fa-secondary{opacity:.4} WebSocketMediaEnded

WebSocketAPIEvents

Unknown

HTTPResponse

State

Transcript

ClientToolInvocation

DataConnectionToolInvocation

Debug

PlaybackClearBuffer

WebSocketError

ConnectorInformation

createWebSocketAPIClient

addEventListener

clearMediaBuffer

clientToolResult

close

dataConnectionToolResult

forcedAgentMessage

hangUp

id

inputTextMessage

removeEventListener

sendMediaTo

setOutputMedium

stopMediaTo

userTextMessage

webSocketId

WebSocketMediaStarted

WebSocketMediaEnded