Gemini

Voice agent client for Google Gemini Live API real-time voice scenarios.

Gemini

Voice agent client for Google Gemini Live API real-time voice scenarios.

Gemini provides a VoxEngine client for connecting a call or media unit to the Google Gemini Live API over WebSocket.

Use Gemini.createLiveAPIClient(...) to create a LiveAPIClient for the current scenario.

Gemini connector overview

Learn how Gemini Live fits into a VoxEngine call flow.

Usage: required module import and basic flow.
Factory functions: create the Gemini Live client.
Methods: media, response, and connection control methods.
Events: WebSocket media bridge events.
LiveAPIEvents: Gemini Live API event names and payload fields.

Usage

Add the module before using the namespace:

1 require(Modules.Gemini);

Create the client, bridge media, and listen for Live API events.

Factory functions

createLiveAPIClient

Creates a Gemini.LiveAPIClient instance.

1 createLiveAPIClient(parameters: LiveAPIClientParameters): Promise<LiveAPIClient>

Parameters

Parameter	Type	Req.	Description
`parameters`	`LiveAPIClientParameters`	✓

Returns

Type	Description
`Promise<LiveAPIClient>`	Resolves to the `Gemini.LiveAPIClient` instance.

LiveAPIClient

Methods

addEventListener

Adds a handler for the specified Gemini.LiveAPIEvents or Gemini.Events event. Use only functions as handlers; anything except a function leads to the error and scenario termination when a handler is called.

1 addEventListener(event: Events | LiveAPIEvents | string, callback: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`Events \| LiveAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✓	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

clearMediaBuffer

Clears the LiveAPI WebSocket media buffer.

1 clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`ClearMediaBufferParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

close

Closes the LiveAPI connection (over WebSocket) or connection attempt.

1 close(): void

Parameters

This method does not accept parameters.

Returns

Type	Description
`void`	Does not return a value.

id

Returns the LiveAPIClient id.

1 id(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

removeEventListener

Removes a handler for the specified Gemini.LiveAPIEvents or Gemini.Events event.

1 removeEventListener(event: Events | LiveAPIEvents | string, callback?: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`Events \| LiveAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✗	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

sendClientContent

Transmits a LiveClientContent over the established connection. https://pkg.go.dev/google.golang.org/genai@v1.57.0#Session.SendClientContent

1 sendClientContent(input: Object): void

Parameters

Parameter	Type	Req.	Description
`input`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

sendMediaTo

Starts sending media from the LiveAPI (via WebSocket) to the media unit. LiveAPI works in real time.

1 sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓
`parameters`	`SendMediaParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

sendRealtimeInput

Transmits a LiveClientRealtimeInput over the established connection. https://pkg.go.dev/google.golang.org/genai@v1.57.0#Session.SendRealtimeInput

1 sendRealtimeInput(input: Object): void

Parameters

Parameter	Type	Req.	Description
`input`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

sendToolResponse

Transmits a LiveClientToolResponse over the established connection. https://pkg.go.dev/google.golang.org/genai@v1.57.0#Session.SendToolResponse

1 sendToolResponse(input: Object): void

Parameters

Parameter	Type	Req.	Description
`input`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

stopMediaTo

Stops sending media from the LiveAPI (via WebSocket) to the media unit.

1 stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓

Returns

Type	Description
`void`	Does not return a value.

webSocketId

Returns the LiveAPI WebSocket id.

1 webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

Events

These events describe audio received through the Gemini WebSocket media bridge.

WebSocketMediaStarted

Triggered when the audio stream sent by a third party through a Gemini WebSocket is started playing.

Event constant: Events.WebSocketMediaStarted

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`encoding`	`string`	✗	Audio encoding formats.
`customParameters`	`{ [key: string]: string }`	✗	Custom parameters.

WebSocketMediaEnded

Triggers after the end of the audio stream sent by a third party through a Gemini WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`mediaInfo`	`WebSocketMediaInfo`	✗	Information about the audio stream that can be obtained after the stream stops or pauses (1 second of silence).

LiveAPIEvents

These events mirror server messages from the Gemini Live API. The data field contains the provider event payload.

Unknown

The unknown event.

Event constant: LiveAPIEvents.Unknown

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ServerContent

Content generated by the model in response to client messages. https://pkg.go.dev/google.golang.org/genai@v1.57.0#LiveServerContent

Event constant: LiveAPIEvents.ServerContent

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ToolCall

Request for the client to execute the function_calls and return the responses with the matching ids. https://pkg.go.dev/google.golang.org/genai@v1.57.0#LiveServerToolCall

Event constant: LiveAPIEvents.ToolCall

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ToolCallCancellation

Notification for the client that a previously issued ToolCallMessage with the specified ids should have been not executed and should be cancelled. https://pkg.go.dev/google.golang.org/genai@v1.57.0#LiveServerToolCallCancellation

Event constant: LiveAPIEvents.ToolCallCancellation

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ConnectorInformation

Contains information about connector.

Event constant: LiveAPIEvents.ConnectorInformation

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

Gemini provides a VoxEngine client for connecting a call or media unit to the Google Gemini Live API over WebSocket.

Use Gemini.createLiveAPIClient(...) to create a LiveAPIClient for the current scenario.

Gemini connector overview

Learn how Gemini Live fits into a VoxEngine call flow.

Usage: required module import and basic flow.
Factory functions: create the Gemini Live client.
Methods: media, response, and connection control methods.
Events: WebSocket media bridge events.
LiveAPIEvents: Gemini Live API event names and payload fields.

Usage

Add the module before using the namespace:

1 require(Modules.Gemini);

Create the client, bridge media, and listen for Live API events.

Factory functions

createLiveAPIClient

Creates a Gemini.LiveAPIClient instance.

1 createLiveAPIClient(parameters: LiveAPIClientParameters): Promise<LiveAPIClient>

Parameters

Parameter	Type	Req.	Description
`parameters`	`LiveAPIClientParameters`	✓

Returns

Type	Description
`Promise<LiveAPIClient>`	Resolves to the `Gemini.LiveAPIClient` instance.

LiveAPIClient

Methods

addEventListener

1 addEventListener(event: Events | LiveAPIEvents | string, callback: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`Events \| LiveAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✓	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

clearMediaBuffer

Clears the LiveAPI WebSocket media buffer.

1 clearMediaBuffer(parameters?: ClearMediaBufferParameters): void

Parameters

Parameter	Type	Req.	Description
`parameters`	`ClearMediaBufferParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

close

Closes the LiveAPI connection (over WebSocket) or connection attempt.

1 close(): void

Parameters

This method does not accept parameters.

Returns

Type	Description
`void`	Does not return a value.

id

Returns the LiveAPIClient id.

1 id(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

removeEventListener

Removes a handler for the specified Gemini.LiveAPIEvents or Gemini.Events event.

1 removeEventListener(event: Events | LiveAPIEvents | string, callback?: (event: object) => any): void

Parameters

Parameter	Type	Req.	Description
`event`	`Events \| LiveAPIEvents \| string`	✓	Event constant or event name to subscribe to.
`callback`	`(event: object) => any`	✗	Function called when the event is emitted.

Returns

Type	Description
`void`	Does not return a value.

sendClientContent

Transmits a LiveClientContent over the established connection. https://pkg.go.dev/google.golang.org/genai@v1.57.0#Session.SendClientContent

1 sendClientContent(input: Object): void

Parameters

Parameter	Type	Req.	Description
`input`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

sendMediaTo

Starts sending media from the LiveAPI (via WebSocket) to the media unit. LiveAPI works in real time.

1 sendMediaTo(mediaUnit: VoxMediaUnit, parameters?: SendMediaParameters): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓
`parameters`	`SendMediaParameters`	✗

Returns

Type	Description
`void`	Does not return a value.

sendRealtimeInput

Transmits a LiveClientRealtimeInput over the established connection. https://pkg.go.dev/google.golang.org/genai@v1.57.0#Session.SendRealtimeInput

1 sendRealtimeInput(input: Object): void

Parameters

Parameter	Type	Req.	Description
`input`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

sendToolResponse

Transmits a LiveClientToolResponse over the established connection. https://pkg.go.dev/google.golang.org/genai@v1.57.0#Session.SendToolResponse

1 sendToolResponse(input: Object): void

Parameters

Parameter	Type	Req.	Description
`input`	`Object`	✓

Returns

Type	Description
`void`	Does not return a value.

stopMediaTo

Stops sending media from the LiveAPI (via WebSocket) to the media unit.

1 stopMediaTo(mediaUnit: VoxMediaUnit): void

Parameters

Parameter	Type	Req.	Description
`mediaUnit`	`VoxMediaUnit`	✓

Returns

Type	Description
`void`	Does not return a value.

webSocketId

Returns the LiveAPI WebSocket id.

1 webSocketId(): string

Parameters

This method does not accept parameters.

Returns

Type	Description
`string`	The requested string value.

Events

These events describe audio received through the Gemini WebSocket media bridge.

WebSocketMediaStarted

Triggered when the audio stream sent by a third party through a Gemini WebSocket is started playing.

Event constant: Events.WebSocketMediaStarted

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`encoding`	`string`	✗	Audio encoding formats.
`customParameters`	`{ [key: string]: string }`	✗	Custom parameters.

WebSocketMediaEnded

Triggers after the end of the audio stream sent by a third party through a Gemini WebSocket (1 second of silence).

Event constant: Events.WebSocketMediaEnded

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`tag`	`string`	✗	Special tag to name audio streams sent over one WebSocket connection. With it, one can send 2 audios to 2 different media units at the same time.
`mediaInfo`	`WebSocketMediaInfo`	✗	Information about the audio stream that can be obtained after the stream stops or pauses (1 second of silence).

LiveAPIEvents

These events mirror server messages from the Gemini Live API. The data field contains the provider event payload.

Unknown

The unknown event.

Event constant: LiveAPIEvents.Unknown

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ServerContent

Content generated by the model in response to client messages. https://pkg.go.dev/google.golang.org/genai@v1.57.0#LiveServerContent

Event constant: LiveAPIEvents.ServerContent

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ToolCall

Request for the client to execute the function_calls and return the responses with the matching ids. https://pkg.go.dev/google.golang.org/genai@v1.57.0#LiveServerToolCall

Event constant: LiveAPIEvents.ToolCall

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ToolCallCancellation

Event constant: LiveAPIEvents.ToolCallCancellation

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

ConnectorInformation

Contains information about connector.

Event constant: LiveAPIEvents.ConnectorInformation

Payload

Field	Type	Req.	Description
`client`	`LiveAPIClient`	✓	The Gemini.LiveAPIClient instance.
`data`	`Object`	✗	The event’s data.

Related guides

Contents

Usage

Factory functions

.fa-secondary{opacity:.4} createLiveAPIClient

LiveAPIClient

Methods

.fa-secondary{opacity:.4} addEventListener

.fa-secondary{opacity:.4} clearMediaBuffer

.fa-secondary{opacity:.4} close

.fa-secondary{opacity:.4} id

.fa-secondary{opacity:.4} removeEventListener

.fa-secondary{opacity:.4} sendClientContent

.fa-secondary{opacity:.4} sendMediaTo

.fa-secondary{opacity:.4} sendRealtimeInput

.fa-secondary{opacity:.4} sendToolResponse

.fa-secondary{opacity:.4} stopMediaTo

.fa-secondary{opacity:.4} webSocketId

Events

.fa-secondary{opacity:.4} WebSocketMediaStarted

.fa-secondary{opacity:.4} WebSocketMediaEnded

LiveAPIEvents

Unknown

ServerContent

ToolCall

ToolCallCancellation

ConnectorInformation

Related guides

Contents

Usage

Factory functions

.fa-secondary{opacity:.4} createLiveAPIClient

LiveAPIClient

Methods

.fa-secondary{opacity:.4} addEventListener

.fa-secondary{opacity:.4} clearMediaBuffer

.fa-secondary{opacity:.4} close

.fa-secondary{opacity:.4} id

.fa-secondary{opacity:.4} removeEventListener

.fa-secondary{opacity:.4} sendClientContent

.fa-secondary{opacity:.4} sendMediaTo

.fa-secondary{opacity:.4} sendRealtimeInput

.fa-secondary{opacity:.4} sendToolResponse

.fa-secondary{opacity:.4} stopMediaTo

.fa-secondary{opacity:.4} webSocketId

Events

.fa-secondary{opacity:.4} WebSocketMediaStarted

.fa-secondary{opacity:.4} WebSocketMediaEnded

LiveAPIEvents

Unknown

ServerContent

ToolCall

ToolCallCancellation

ConnectorInformation

createLiveAPIClient

addEventListener

clearMediaBuffer

close

id

removeEventListener

sendClientContent

sendMediaTo

sendRealtimeInput

sendToolResponse

stopMediaTo

webSocketId

WebSocketMediaStarted

WebSocketMediaEnded

createLiveAPIClient

addEventListener

clearMediaBuffer

close

id

removeEventListener

sendClientContent

sendMediaTo

sendRealtimeInput

sendToolResponse

stopMediaTo

webSocketId

WebSocketMediaStarted

WebSocketMediaEnded