Skip to navigation

RealtimeTTSPlayer

Realtime TTS player for xAI Grok speech synthesis in VoxEngine.
View as Markdown

XAI provides a real-time Grok TTS player that streams generated audio to calls, WebSockets, and other VoxEngine media units.

Use XAI.createRealtimeTTSPlayer(...) to create a RealtimeTTSPlayer. Send text.delta, text.done, and text.clear messages to stream utterances and handle barge-in without reconnecting.

VoxEngine uses PCM audio at 16000 Hz. Keep the adapter audio format when configuring the xAI connection.

Contents

RealtimeTTSPlayer

Methods

clearBuffer

Clears a XAI.RealtimeTTSPlayer buffer.

clearBuffer(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

send

Send message object to the xAI provider context.

send(parameters: Object): void

Parameters

ParameterTypeReq.Description
parametersObject✓Send text.delta to append text, text.done to finish the current utterance, or text.clear to cancel it for barge-in. The WebSocket remains open for subsequent utterances. Preserve spaces and punctuation across text.delta chunks. For VoxEngine barge-in, clearBuffer() cancels the provider utterance and clears queued playback audio. See the partner API reference.

Returns

TypeDescription
voidDoes not return a value.

Example parameters messages:

text.delta

{
"type": "text.delta",
"delta": "Hello from Grok TTS."
}

text.done

{
"type": "text.done"
}

text.clear

{
"type": "text.clear"
}

Inherited Methods

XAI.RealtimeTTSPlayer extends the shared player base (BasePlayer in the typings). Those base methods use the same playback contract documented on the public Player page, and they emit PlayerEvents.

addEventListener

Inherited from the shared player base. See Player.addEventListener.

Adds a handler for the specified PlayerEvents event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

addEventListener(event: PlayerEvents | string, callback: (event: object) => any): void

Parameters

ParameterTypeReq.Description
eventPlayerEvents | string✓Event constant or event name to subscribe to.
callback(event: object) => any✓Function called when the event is emitted.

Returns

TypeDescription
voidDoes not return a value.

id

Inherited from the shared player base. See Player.id.

Returns the player’s id.

id(): string

Parameters

This method does not accept parameters.

Returns

TypeDescription
stringThe requested string value.

pause

Inherited from the shared player base. See Player.pause.

Pauses playback. To continue the playback use the Player.resume method.

pause(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

removeEventListener

Inherited from the shared player base. See Player.removeEventListener.

Removes a handler for the specified PlayerEvents event.

removeEventListener(event: PlayerEvents | string, callback?: (event: object) => any): void

Parameters

ParameterTypeReq.Description
eventPlayerEvents | string✓Event constant or event name to subscribe to.
callback(event: object) => any✗Function called when the event is emitted.

Returns

TypeDescription
voidDoes not return a value.

resume

Inherited from the shared player base. See Player.resume.

Resumes playback after the Player.pause method is called.

resume(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

sendMediaTo

Inherited from the shared player base. See Player.sendMediaTo.

Starts sending media from the player to the media unit.

sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

ParameterTypeReq.Description
mediaUnitVoxMediaUnit✓
parametersSendMediaParameters✗

Returns

TypeDescription
voidDoes not return a value.

stop

Inherited from the shared player base. See Player.stop.

Stops playback. The current player’s instance is destroyed.

stop(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

stopMediaTo

Inherited from the shared player base. See Player.stopMediaTo.

Stops sending media from the player to the media unit.

stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

ParameterTypeReq.Description
mediaUnitVoxMediaUnit✓

Returns

TypeDescription
voidDoes not return a value.

PlayerEvents

These are inherited PlayerEvents. Event callbacks receive the shared player event object; player is the XAI.RealtimeTTSPlayer instance that emitted the event. Some events include additional playback fields noted below.

Triggered when Player created.

Event constant: PlayerEvents.Created

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.

Triggers by the createURLPlayer and createTTSPlayer methods when

  1. the audio file download to the Voximplant cache is finished;
  2. the audio file is found in the cache (i.e., it is in the cache before).

Event constant: PlayerEvents.PlaybackReady

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.

Triggered when playback is started. Note that if the createURLPlayer method is called with the onPause parameter set to true, the event is not triggered; it is triggered after the Player.resume method call.

Event constant: PlayerEvents.Started

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.
durationnumber✓Playback duration

Triggers as a result of the Player.stop method call.

Event constant: PlayerEvents.Stopped

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.

Triggered when playback has finished successfully or with an error.

Event constant: PlayerEvents.PlaybackFinished

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.
errorstring✗Error message

Triggered when playback has finished with an error.

Event constant: PlayerEvents.Error

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.
errorstring✓Error message

Triggered when Player.addMarker is reached.

Event constant: PlayerEvents.PlaybackMarkerReached

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.
offsetnumber✓The marker offset

Triggered when an audio file is playing faster than it is being loaded.

Event constant: PlayerEvents.PlaybackBuffering

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.

Triggered when an audio chunk playback is finished. Note that this event is triggered only for RealtimeTTSPlayer instances.

For VoxTTS.RealtimeTTSPlayer, the optional final field indicates whether the completed flush ends the current phrase or is an intermediate auto-flush.

Event constant: PlayerEvents.AudioChunksPlaybackFinished

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The XAI.RealtimeTTSPlayer instance that emitted the event.
timeToFirstBytenumber✓Time to first byte (TTFB) in milliseconds. Represents the delay between sending the request and receiving the first byte of audio data.
finalboolean✗Whether the completed flush ends the current phrase. Present only for VoxTTS.RealtimeTTSPlayer (true or false); omitted for other RealtimeTTS providers. - true — final (normal) flush; phrase playback is complete. - false — intermediate auto-flush; more audio is expected.