VoxEngineVoice AI

Inworld

Realtime TTS player for Inworld speech synthesis in VoxEngine.
View as Markdown

Inworld provides a real-time TTS player that can send generated audio to calls, WebSockets, and other VoxEngine media units.

Use Inworld.createRealtimeTTSPlayer(...) to create a RealtimeTTSPlayer for the current scenario. After the player is created, call methods such as send, sendMediaTo, and addEventListener on that player instance.

OpenAI with Inworld TTS

Use Inworld as the realtime TTS layer in an OpenAI voice pipeline.

Realtime TTS pattern

See where realtime speech synthesis fits into VoxEngine call flows.

Contents

  • Usage: required module import and basic flow.
  • Factory functions: create the realtime TTS player.
  • RealtimeTTSPlayerParameters: API key, provider context, and WebSocket options.
  • RealtimeTTSPlayer: runtime player object returned by the factory.
  • Methods: direct Inworld player methods.
  • Inherited Methods: playback and media methods inherited from the shared player base.
  • PlayerEvents: player event names and playback lifecycle events.

Usage

Add the module before using the namespace:

1require(Modules.Inworld);

Create a player, send provider messages, and bridge playback with sendMediaTo or VoxEngine.sendMediaBetween.

Factory functions

createRealtimeTTSPlayer

Creates a new Inworld.RealtimeTTSPlayer instance. You can attach media streams later via the Inworld.RealtimeTTSPlayer.sendMediaTo or VoxEngine.sendMediaBetween methods.

1createRealtimeTTSPlayer(parameters?: {
2  statistics?: boolean;
3  trace?: boolean;
4  privacy?: boolean;
5  createContextParameters?: Object;
6  apiKey?: string;
7}): RealtimeTTSPlayer

The optional parameters object is typed as Inworld.RealtimeTTSPlayerParameters.

Parameters

ParameterTypeReq.Description
parametersRealtimeTTSPlayerParameters✗Inworld.RealtimeTTSPlayer parameters. Can be passed as arguments to the Inworld.createRealtimeTTSPlayer method.
↳ statisticsboolean✗Enables statistics functionality.
↳ traceboolean✗Whether to enable the tracing functionality. If tracing is enabled, a URL to the trace file appears in the ‘websocket.created’ message. The file contains all sent and received WebSocket messages in the plain text format. The file is uploaded to the S3 storage. NOTE: enable this only for diagnostic purposes. You can provide the trace file to our support team to help investigating issues.
↳ privacyboolean✗Whether to enable the privacy functionality. If privacy is enabled, the logging for the WebSocket connection is disabled. NOTE: the default value is false.
↳ createContextParametersObject✗Object to provide parameters directly to the Inworld provider Create Context message. Find more information in the documentation(https://docs.inworld.ai/api-reference/ttsAPI/texttospeech/synthesize-speech-websocket).
↳ apiKeystring✗Inworld API key. Use your Inworld API key if you have your own Inworld account.

Returns

TypeDescription
RealtimeTTSPlayerThe requested RealtimeTTSPlayer value.

RealtimeTTSPlayer

Methods

clearBuffer

Clears an Inworld.RealtimeTTSPlayer buffer.

1clearBuffer(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

send

Send message object to the Inworld provider context.

1send(parameters: Object): void

Parameters

ParameterTypeReq.Description
parametersObject✓Inworld TTS WebSocket message. Common message shapes: send_text, flush_context, close_context. See the partner API reference.

Returns

TypeDescription
voidDoes not return a value.

Example parameters:

1{
2  "send_text": {
3    "text": "Hello, what a wonderful day to be a text-to-speech model!",
4    "flush_context": {}
5  },
6  "contextId": "ctx-1"
7}

Inherited Methods

Inworld.RealtimeTTSPlayer extends the shared player base (BasePlayer in the typings). Those base methods use the same playback contract documented on the public Player page, and they emit PlayerEvents.

addEventListener

Inherited from the shared player base. See Player.addEventListener.

Adds a handler for the specified PlayerEvents event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

1addEventListener(event: PlayerEvents | string, callback: (event: object) => any): void

Parameters

ParameterTypeReq.Description
eventPlayerEvents | string✓Event constant or event name to subscribe to.
callback(event: object) => any✓Function called when the event is emitted.

Returns

TypeDescription
voidDoes not return a value.

id

Inherited from the shared player base. See Player.id.

Returns the player’s id.

1id(): string

Parameters

This method does not accept parameters.

Returns

TypeDescription
stringThe requested string value.

pause

Inherited from the shared player base. See Player.pause.

Pauses playback. To continue the playback use the Player.resume method.

1pause(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

removeEventListener

Inherited from the shared player base. See Player.removeEventListener.

Removes a handler for the specified PlayerEvents event.

1removeEventListener(event: PlayerEvents | string, callback?: (event: object) => any): void

Parameters

ParameterTypeReq.Description
eventPlayerEvents | string✓Event constant or event name to subscribe to.
callback(event: object) => any✗Function called when the event is emitted.

Returns

TypeDescription
voidDoes not return a value.

resume

Inherited from the shared player base. See Player.resume.

Resumes playback after the Player.pause method is called.

1resume(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

sendMediaTo

Inherited from the shared player base. See Player.sendMediaTo.

Starts sending media from the player to the media unit.

1sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

ParameterTypeReq.Description
mediaUnitVoxMediaUnit✓
parametersSendMediaParameters✗

Returns

TypeDescription
voidDoes not return a value.

stop

Inherited from the shared player base. See Player.stop.

Stops playback. The current player’s instance is destroyed.

1stop(): void

Parameters

This method does not accept parameters.

Returns

TypeDescription
voidDoes not return a value.

stopMediaTo

Inherited from the shared player base. See Player.stopMediaTo.

Stops sending media from the player to the media unit.

1stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

ParameterTypeReq.Description
mediaUnitVoxMediaUnit✓

Returns

TypeDescription
voidDoes not return a value.

PlayerEvents

These are inherited PlayerEvents. Event callbacks receive the shared player event object; player is the Inworld.RealtimeTTSPlayer instance that emitted the event. Some events include additional playback fields noted below.

Triggered when Player created.

Event constant: PlayerEvents.Created

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.

Triggers by the createURLPlayer and createTTSPlayer methods when

  1. the audio file download to the Voximplant cache is finished;
  2. the audio file is found in the cache (i.e., it is in the cache before).

Event constant: PlayerEvents.PlaybackReady

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.

Triggered when playback is started. Note that if the createURLPlayer method is called with the onPause parameter set to true, the event is not triggered; it is triggered after the Player.resume method call.

Event constant: PlayerEvents.Started

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.
durationnumber✓Playback duration

Triggers as a result of the Player.stop method call.

Event constant: PlayerEvents.Stopped

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.

Triggered when playback has finished successfully or with an error.

Event constant: PlayerEvents.PlaybackFinished

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.
errorstring✗Error message

Triggered when playback has finished with an error.

Event constant: PlayerEvents.Error

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.
errorstring✓Error message

Triggered when Player.addMarker is reached.

Event constant: PlayerEvents.PlaybackMarkerReached

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.
offsetnumber✓The marker offset

Triggered when an audio file is playing faster than it is being loaded.

Event constant: PlayerEvents.PlaybackBuffering

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.

Triggered when an audio chunk playback is finished. Note that this event is triggered only for RealtimeTTSPlayer instances.

Event constant: PlayerEvents.AudioChunksPlaybackFinished

Payload

FieldTypeReq.Description
playerRealtimeTTSPlayer✓The Inworld.RealtimeTTSPlayer instance that emitted the event.
timeToFirstBytenumber✓Time to first byte (TTFB) in milliseconds. Represents the delay between sending the request and receiving the first byte of audio data.