***

title: Overview
subtitle: OpenAI Realtime API in VoxEngine
------------------------------------------

## Benefits

The native OpenAI module connects Voximplant calls to the OpenAI Realtime API for low‑latency, speech‑to‑speech interactions.
VoxEngine handles telephony, media conversion, and WebSocket streaming so you can focus on agent behavior.

Capability and feature highlights:

* Bridge PSTN, SIP, WebRTC, or WhatsApp calls into OpenAI Realtime with a single VoxEngine scenario.
* Real-time conversations with speech input, speech output, and partial transcript events.
* Barge‑in with server VAD and media buffer control for natural turn‑taking.
* Function calling for external actions (weather, transfers, CRM, etc.).
* Flexible output modes: audio for full speech‑to‑speech, text for half‑cascade pipelines.

## Demo video

OpenAI Realtime demo (general):

<iframe width="100%" height="360" src="https://www.youtube.com/embed/ryJAcb1RAFs" title="OpenAI Realtime API demo" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowFullScreen />

## Architecture

```mermaid
graph LR
  Caller["PSTN/SIP/WhatsApp/WebRTC"] <-->|"Media + call control"| VoxEngine["VoxEngine Scenario"]
  VoxEngine <-->|"WebSocket: audio + events"| OpenAI["OpenAI Realtime API"]
```

## Prerequisites

* OpenAI API key stored in Voximplant `ApplicationStorage` under `OPENAI_API_KEY`.

## Development notes

* **Native VoxEngine module**: load with `require(Modules.OpenAI)` and create an `OpenAI.RealtimeAPIClient` via `OpenAI.createRealtimeAPIClient({ apiKey, model })`.
* **Session setup**: configure behavior via `sessionUpdate({ session: {...} })` (instructions, voice, turn detection, output modalities).
* **Barge‑in**: listen for `OpenAI.RealtimeAPIEvents.InputAudioBufferSpeechStarted` and call `client.clearMediaBuffer()` to cancel current audio.
* **Function calling**: define tools in the session and handle `ResponseFunctionCallArgumentsDone`; send results via `conversationItemCreate({ item: { type: "function_call_output", ... }})`.
* **Output modes**: use `output_modalities: ["audio"]` for speech‑to‑speech, or `output_modalities: ["text"]` for half‑cascade pipelines.

See the [OpenAI Realtime API reference](https://platform.openai.com/docs/api-reference/realtime) for full details on events, session updates, and response creation.

## Examples

* [Example: Answering an incoming call](openai-realtime-voice-ai-example-answering-incoming-call)
* [Example: Placing an outbound call](openai-realtime-voice-ai-example-placing-outbound-call)
* [Example: Function calling](openai-realtime-voice-ai-example-function-calling)
* [Example: Half-cascade with ElevenLabs](openai-realtime-voice-ai-example-half-cascade-elevenlabs)
* [Example: Half-cascade with Inworld](openai-realtime-voice-ai-example-half-cascade-inworld)
* [Example: Half-cascade with Cartesia](openai-realtime-voice-ai-example-half-cascade-cartesia)

## Links

### Voximplant

* OpenAI Voice AI connector: [https://voximplant.com/docs/voice-ai/openai](https://voximplant.com/docs/voice-ai/openai)
* OpenAI module API reference: [https://voximplant.com/docs/references/voxengine/openai](https://voximplant.com/docs/references/voxengine/openai)
* Voice AI product overview: [https://voximplant.ai/](https://voximplant.ai/)

### OpenAI

* Realtime API reference: [https://platform.openai.com/docs/api-reference/realtime](https://platform.openai.com/docs/api-reference/realtime)
* Realtime events (client/server): [https://platform.openai.com/docs/api-reference/realtime-client-events](https://platform.openai.com/docs/api-reference/realtime-client-events)
* Realtime guide: [https://platform.openai.com/docs/guides/realtime](https://platform.openai.com/docs/guides/realtime)