Yandex
Yandex provides a VoxEngine client for connecting a call or media unit to the Yandex Realtime API over WebSocket.
Use Yandex.createRealtimeAPIClient(...) to create a RealtimeAPIClient for the current scenario.
Related guides
Contents
- Usage: required module import and basic flow.
- Factory functions: create the Yandex Realtime API client.
- Methods: session, items, and response control methods.
- Events: WebSocket media bridge events.
- RealtimeAPIEvents: Yandex Realtime API event names and payload fields.
Usage
Add the module before using the namespace:
Create the client, bridge media, and listen for Yandex Realtime API events.
Factory functions
createRealtimeAPIClient
Creates a Yandex.RealtimeAPIClient instance.
Parameters
Returns
RealtimeAPIClient
Methods
addEventListener
Adds a handler for the specified Yandex.RealtimeAPIEvents or Yandex.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.
Parameters
Returns
clearMediaBuffer
Clears the Yandex WebSocket media buffer.
Parameters
Returns
close
Closes the Yandex connection (over WebSocket) or connection attempt.
Parameters
This method does not accept parameters.
Returns
conversationItemCreate
Add a new Item to the Conversation’s context.
Parameters
Returns
conversationItemDelete
Send this event when you want to remove any item from the conversation history.
Parameters
Returns
conversationItemRetrieve
Send this event when you want to retrieve the server’s representation of a specific item in the conversation history.
Parameters
Returns
conversationItemTruncate
Send this event to truncate a previous assistant message’s audio.
Parameters
Returns
id
Returns the RealtimeAPIClient id.
Parameters
This method does not accept parameters.
Returns
removeEventListener
Removes a handler for the specified Yandex.RealtimeAPIEvents or Yandex.Events event.
Parameters
Returns
responseCancel
Send this event to cancel an in-progress response.
Parameters
Returns
responseCreate
This event instructs the server to create a Response, which means triggering model inference.
Parameters
Returns
sendMediaTo
Starts sending media from the Yandex (via WebSocket) to the media unit. Yandex works in real time.
Parameters
Returns
sessionUpdate
Send this event to update the session’s configuration.
Parameters
Returns
stopMediaTo
Stops sending media from the Yandex (via WebSocket) to the media unit.
Parameters
Returns
webSocketId
Returns the Yandex WebSocket id.
Parameters
This method does not accept parameters.
Returns
Events
These events describe audio received through the Yandex WebSocket media bridge.
WebSocketMediaStarted
Triggered when the audio stream sent by a third party through an Yandex WebSocket is started playing.
Event constant: Events.WebSocketMediaStarted
Payload
WebSocketMediaEnded
Triggers after the end of the audio stream sent by a third party through an Yandex WebSocket (1 second of silence).
Event constant: Events.WebSocketMediaEnded
Payload
RealtimeAPIEvents
These events mirror server messages from the Yandex Realtime API. The data field contains the provider event payload.
Unknown
The unknown event.
Event constant: RealtimeAPIEvents.Unknown
Payload
HTTPResponse
The HTTP response event.
Event constant: RealtimeAPIEvents.HTTPResponse
Payload
WebSocketError
The WebSocket error response event.
Event constant: RealtimeAPIEvents.WebSocketError
Payload
ConnectorInformation
Contains information about connector.
Event constant: RealtimeAPIEvents.ConnectorInformation
Payload
Error
Returned when an error occurs, which could be a client problem or a server problem.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerError
Event constant: RealtimeAPIEvents.Error
Payload
SessionCreated
Returned when a Session is created.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerSessionCreated
Event constant: RealtimeAPIEvents.SessionCreated
Payload
SessionUpdated
Returned when a session is updated with a session.update event, unless there is an error.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerSessionUpdated
Event constant: RealtimeAPIEvents.SessionUpdated
Payload
ConversationItemCreated
Returned when a new Item is created in the Conversation.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemCreated
Event constant: RealtimeAPIEvents.ConversationItemCreated
Payload
ConversationItemRetrieved
Returned when a conversation item is retrieved with conversation.item.retrieve.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemRetrieved
Event constant: RealtimeAPIEvents.ConversationItemRetrieved
Payload
ConversationItemInputAudioTranscriptionCompleted
This event is the output of audio transcription for user audio written to the user audio buffer.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemInputAudioTranscriptionCompleted
Event constant: RealtimeAPIEvents.ConversationItemInputAudioTranscriptionCompleted
Payload
ConversationItemInputAudioTranscriptionDelta
Returned when the text value of an input audio transcription content part is updated with incremental transcription results.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemInputAudioTranscriptionDelta
Event constant: RealtimeAPIEvents.ConversationItemInputAudioTranscriptionDelta
Payload
ConversationItemInputAudioTranscriptionSegment
Returned when an input audio transcription segment is identified for an item.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemInputAudioTranscriptionSegment
Event constant: RealtimeAPIEvents.ConversationItemInputAudioTranscriptionSegment
Payload
ConversationItemInputAudioTranscriptionFailed
Returned when input audio transcription is configured, and a transcription request for a user message failed.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemInputAudioTranscriptionFailed
Event constant: RealtimeAPIEvents.ConversationItemInputAudioTranscriptionFailed
Payload
ConversationItemTruncated
Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemTruncated
Event constant: RealtimeAPIEvents.ConversationItemTruncated
Payload
ConversationItemDeleted
Returned when an item in the conversation is deleted by the client with a conversation.item.delete event.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerConversationItemDeleted
Event constant: RealtimeAPIEvents.ConversationItemDeleted
Payload
InputAudioBufferCommitted
Returned when an input audio buffer is committed.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerInputAudioBufferCommitted
Event constant: RealtimeAPIEvents.InputAudioBufferCommitted
Payload
InputAudioBufferCleared
Returned when the input audio buffer is cleared by the client with an input_audio_buffer.clear event.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerInputAudioBufferCleared
Event constant: RealtimeAPIEvents.InputAudioBufferCleared
Payload
InputAudioBufferDTMFEventReceived
Returned when a DTMF event is received (telephony integrations). https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerInputAudioBufferDtmfEventReceived
Event constant: RealtimeAPIEvents.InputAudioBufferDTMFEventReceived
Payload
InputAudioBufferSpeechStarted
Sent by the server when in server_vad mode to indicate that speech has been detected in the audio buffer.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerInputAudioBufferSpeechStarted
Event constant: RealtimeAPIEvents.InputAudioBufferSpeechStarted
Payload
InputAudioBufferSpeechStopped
Returned in server_vad mode when the server detects the end of speech in the audio buffer.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerInputAudioBufferSpeechStopped
Event constant: RealtimeAPIEvents.InputAudioBufferSpeechStopped
Payload
InputAudioBufferTimeoutTriggered
Returned when the Server VAD timeout is triggered for the input audio buffer.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerInputAudioBufferTimeoutTriggered
Event constant: RealtimeAPIEvents.InputAudioBufferTimeoutTriggered
Payload
OutputAudioBufferCleared
Returned when the output audio buffer is cleared.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerOutputAudioBufferCleared
Event constant: RealtimeAPIEvents.OutputAudioBufferCleared
Payload
OutputAudioBufferStarted
Returned when the output audio buffer starts playing/streaming.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerOutputAudioBufferStarted
Event constant: RealtimeAPIEvents.OutputAudioBufferStarted
Payload
OutputAudioBufferStopped
Returned when the output audio buffer stops.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerOutputAudioBufferStopped
Event constant: RealtimeAPIEvents.OutputAudioBufferStopped
Payload
ResponseCreated
Returned when a new Response is created.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseCreated
Event constant: RealtimeAPIEvents.ResponseCreated
Payload
ResponseDone
Returned when a Response is done streaming.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseDone
Event constant: RealtimeAPIEvents.ResponseDone
Payload
ResponseOutputItemAdded
Returned when a new Item is created during Response generation.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseOutputItemAdded
Event constant: RealtimeAPIEvents.ResponseOutputItemAdded
Payload
ResponseOutputItemDone
Returned when an Item is done streaming.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseOutputItemDone
Event constant: RealtimeAPIEvents.ResponseOutputItemDone
Payload
ResponseContentPartAdded
Returned when a new content part is added to an assistant message item during response generation.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseContentPartAdded
Event constant: RealtimeAPIEvents.ResponseContentPartAdded
Payload
ResponseContentPartDone
Returned when a content part is done streaming in an assistant message item.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseContentPartDone
Event constant: RealtimeAPIEvents.ResponseContentPartDone
Payload
ResponseOutputTextDelta
Returned when the text value of an “output_text” content part is updated.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseOutputTextDelta
Event constant: RealtimeAPIEvents.ResponseOutputTextDelta
Payload
ResponseOutputTextDone
Returned when the text value of an “output_text” content part is done streaming.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseOutputTextDone
Event constant: RealtimeAPIEvents.ResponseOutputTextDone
Payload
ResponseOutputAudioTranscriptDelta
Returned when the model-generated transcription of audio output is updated.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseOutputAudioTranscriptDelta
Event constant: RealtimeAPIEvents.ResponseOutputAudioTranscriptDelta
Payload
ResponseOutputAudioTranscriptDone
Returned when the model-generated transcription of audio output is done streaming.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseOutputAudioTranscriptDone
Event constant: RealtimeAPIEvents.ResponseOutputAudioTranscriptDone
Payload
ResponseOutputAudioDone
Returned when the model-generated audio is done.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseMcpCallArgumentsDone
Event constant: RealtimeAPIEvents.ResponseOutputAudioDone
Payload
ResponseFunctionCallArgumentsDelta
Returned when the model-generated function call arguments are updated.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseFunctionCallArgumentsDelta
Event constant: RealtimeAPIEvents.ResponseFunctionCallArgumentsDelta
Payload
ResponseFunctionCallArgumentsDone
Returned when the model-generated function call arguments are done streaming.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseFunctionCallArgumentsDone
Event constant: RealtimeAPIEvents.ResponseFunctionCallArgumentsDone
Payload
ResponseMCPCallArgumentsDelta
Returned when MCP tool call arguments are updated during response generation.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseMcpCallArgumentsDelta
Event constant: RealtimeAPIEvents.ResponseMCPCallArgumentsDelta
Payload
ResponseMCPCallArgumentsDone
Returned when MCP tool call arguments are finalized during response generation.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseMcpCallArgumentsDone
Event constant: RealtimeAPIEvents.ResponseMCPCallArgumentsDone
Payload
ResponseMCPCallInProgress
Returned when an MCP tool call has started and is in progress.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseMcpCallInProgress
Event constant: RealtimeAPIEvents.ResponseMCPCallInProgress
Payload
ResponseMCPCallCompleted
Returned when an MCP tool call has completed successfully.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseMcpCallCompleted
Event constant: RealtimeAPIEvents.ResponseMCPCallCompleted
Payload
ResponseMCPCallFailed
Returned when an MCP tool call has failed.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerResponseMcpCallFailed
Event constant: RealtimeAPIEvents.ResponseMCPCallFailed
Payload
MCPListToolsInProgress
Returned when listing MCP tools is in progress for an item.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerMcpListToolsInProgress
Event constant: RealtimeAPIEvents.MCPListToolsInProgress
Payload
MCPListToolsCompleted
Returned when listing MCP tools has completed for an item.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerMcpListToolsCompleted
Event constant: RealtimeAPIEvents.MCPListToolsCompleted
Payload
MCPListToolsFailed
Returned when listing MCP tools has failed for an item.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerMcpListToolsFailed
Event constant: RealtimeAPIEvents.MCPListToolsFailed
Payload
RateLimitsUpdated
Emitted at the beginning of a Response to indicate the updated rate limits.https://yandex.cloud/ru/docs/ai-studio/serverEvents/realtimeServerRateLimitsUpdated
Event constant: RealtimeAPIEvents.RateLimitsUpdated
Payload
