> For a complete documentation index, fetch https://docs.voximplant.ai/llms.txt

# Pipeline Options

> For the complete documentation index, see [llms.txt](/llms.txt).

VoxEngine supports a wide variety of Voice AI architectures and components.
The right pipeline depends on your priorities -- low latency, interactivity needs, voice choice, and low-level control are some factors to consider.

[*User↔LLM*](#speech-to-speech)

### Speech-to-speech (S2S)

Fully integrated Voice AI that often offers the lowest overall latency.

Fastest path | Lowest latency

[*LLM↔TTS*](#hybrid-cascade)

### Hybrid-cascade

Realtime LLM for speech input and reasoning, with a separate TTS system for output voice.

More voice choice | Still realtime

[*STT↔LLM↔TTS*](#full-cascade)

### Cascaded

Separate STT, LLM, and TTS stages for maximum provider flexibility and pipeline control.

Most configurable | Most complex

## High-level comparison

| Pipeline         | Best when you want                       | Latency | Voice flexibility                            | Provider flexibility | VoxEngine control |
| ---------------- | ---------------------------------------- | ------- | -------------------------------------------- | -------------------- | ----------------- |
| Speech-to-speech | Streamlined architecture                 | Lowest  | Limited to the realtime provider voice stack | Medium               | Medium            |
| Hybrid-cascade   | Realtime LLM with external TTS           | Low     | High                                         | High for TTS-only    | High              |
| Cascaded         | Independent control of STT, LLM, and TTS | Higher  | High                                         | Highest              | Highest           |

Some vendors present a speech-to-speech (S2S) API externally, but internally manage a cascaded architecture with separate components with varying degrees of control and configuration.
See more details below.

* Start with speech-to-speech when you want to get to a working voice agent quickly.
* Move to hybrid-cascade when you need more TTS voice choices and control, and can tolerate some additional latency.
* Consider full cascade when you need independent control over transcription, reasoning, playback, turn-taking, or
  provider mix and are willing to manage the additional interactions.

## Voximplant Connectors & Pipeline Support

Voximplant always manages the telephony side of the call: routing, answering, bridging media, call control, and any orchestration logic you add in VoxEngine.
Our Voice AI API clients - also referred to as connectors - provide a direct connection from VoxEngine to the provider's environment with a low-latency WebSocket connection.
Voximplant's various connector options differ in how they handle transcription, reasoning, and speech synthesis internally.

LLMs like OpenAI, Gemini, and Grok provide direct access to the speech-enabled LLM.
Speech input and output parameters are managed as part of the connector.

Ultravox provides a speech-native LLM, but integrates TTS within their environment, only exposing a Speech-to-Speech interface to Voximplant.
Similarly, ElevenLabs, Cartesia, and Deepgram provide a speech-to-speech interface to Voximplant, but handle LLM, STT, and TTS elements internally.
Configuration of these internal elements vary by vendor. We refer to the integrations as "indirect speech-to-speech".

All vendors provide some level of end-of-turn detection, with some vendors providing more explicit control over turn-taking and barge-in.

The table below summarizes support by vendor from the Voximplant connector perspective.

| Agent Vendor                                                      | Direct S2S | Indirect S2S | Hybrid | Cascaded |
| ----------------------------------------------------------------- | ---------- | ------------ | ------ | -------- |
| [OpenAI](/voice-ai-orchestration/openai/overview)                 | ✓          |              | ✓      | ✓        |
| [Gemini Live](/voice-ai-orchestration/gemini/overview)            | ✓          |              |        |          |
| [xAI Grok Voice Agent](/voice-ai-orchestration/grok/overview)     | ✓          |              |        |          |
| [Ultravox](/voice-ai-orchestration/ultravox/overview)             |            | ✓            |        |          |
| [ElevenLabs Agents](/voice-ai-orchestration/elevenlabs/overview)  |            | ✓            |        |          |
| [Deepgram Agents](/voice-ai-orchestration/deepgram/overview)      |            | ✓            |        |          |
| [Cartesia Line Agents](/voice-ai-orchestration/cartesia/overview) |            | ✓            |        |          |

## Speech-to-speech

In this pattern, caller audio is bridged to the provider in realtime, and the provider returns speech audio directly back into the call.

![Speech-to-speech architecture in light mode](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/voximplant.docs.buildwithfern.com/06c6fa34c3e2907c70fc10eaee58b887dc70fe302f6b2ad455bc8dd337fe4cac/docs/assets/connectors/s2s-light.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260910%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260910T045127Z&X-Amz-Expires=604800&X-Amz-Signature=9d9dd7c31a6ad6ba747f86215d74e3d0a92605d90fb9b795e98f209c6631f60d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)![Speech-to-speech architecture in dark mode](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/voximplant.docs.buildwithfern.com/4bfae36e253eb1a3938f294667501aca9326192cbc1b6f42ed9920f0c0c688b8/docs/assets/connectors/s2s-dark.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260910%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260910T045127Z&X-Amz-Expires=604800&X-Amz-Signature=bebdda2e18b50f2b26bab8ab08c118aa0fc0dff22cf20efeeb78b768ede5f91f&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

### How it works

* Caller audio is streamed from VoxEngine to a realtime voice agent over WebSockets.
* The provider handles speech recognition, reasoning, and speech generation in one session.
* VoxEngine manages telephony, routing, media bridge, and call control around that session.

### Tradeoffs

#### Advantages

* Lowest end-to-end latency.
* Simple architecture with fewer systems to coordinate.
* Fastest path from prototype to production trial.
* Natural fit for direct realtime connectors with built-in streaming speech.

#### Disadvantages

* Voice quality and speech style are limited to the provider's native output voices.
* You have less control over how STT, reasoning, and speech output are separated.
* Switching one stage often means switching the whole stack.

### Best for

* You want the fastest possible conversational loop.
* You are happy with the provider's built-in voices.
* You want the simplest operational model.

### Related guides

#### Direct speech-to-speech connectors

These connectors expose the speech-enabled LLM directly to VoxEngine.

### [OpenAI](/voice-ai-orchestration/openai/overview)

Direct realtime speech-to-speech with OpenAI.

### [Gemini Live](/voice-ai-orchestration/gemini/overview)

Direct live audio pipeline with Gemini Live.

### [xAI Grok Voice Agent](/voice-ai-orchestration/grok/overview)

Realtime voice interaction with Grok voice agent sessions.

#### Indirect speech-to-speech connectors

These connectors present a speech-to-speech interface to VoxEngine while managing more of the internal speech and agent stack inside the provider environment.

### [Ultravox](/voice-ai-orchestration/ultravox/overview)

Speech-native realtime agent with native speech input and output.

### [ElevenLabs Agents](/voice-ai-orchestration/elevenlabs/overview)

Conversational agent sessions with realtime speech handling.

### [Deepgram Agents](/voice-ai-orchestration/deepgram/overview)

Realtime voice agent flow with Deepgram speech input and output.

### [Cartesia Line Agents](/voice-ai-orchestration/cartesia/overview)

Speech-to-speech agent flow with Cartesia-managed runtime behavior.

## Hybrid-cascade

Hybrid-cascade - sometimes called half-cascade - keeps the realtime LLM in the loop for user speech input and reasoning, but moves speech output to a separate synthesis provider.
This is often the best compromise between latency and voice flexibility.

![Half-cascade architecture in light mode](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/voximplant.docs.buildwithfern.com/023442b4007585055cb0163e006064d9ed84cae7ea7d02b8c856cad4401a2b83/docs/assets/connectors/half-cascade-light.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260910%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260910T045127Z&X-Amz-Expires=604800&X-Amz-Signature=4bfbe0884b5d51ff717263954811d7812569601e385da0e2acfa831c5009a72d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)![Half-cascade architecture in dark mode](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/voximplant.docs.buildwithfern.com/705b77ca97365bdf8a084a09a651d73efcefd6315ea6c3d451ebbc1577b46447/docs/assets/connectors/half-cascade-dark.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260910%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260910T045127Z&X-Amz-Expires=604800&X-Amz-Signature=aa110d8c287c35a807b8dfd34788bddeab1e03ead6da949216312ec4b30be84c&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

### How it works

* Caller audio is streamed to a realtime LLM.
* The realtime LLM returns text rather than audio (or you transcribe the audio).
* VoxEngine sends the text into a TTS engine and streams the generated speech back to the caller.

### Tradeoffs

#### Advantages

* Much broader choice of output voices and TTS providers.
* Lets you tune speaking style, voice identity, and output pricing separately from the LLM.
* Still supports low-latency streaming when paired with realtime TTS.
* Good balance between orchestration control and implementation simplicity.

#### Disadvantages

* More moving parts than direct speech-to-speech.
* Output latency depends on the selected TTS provider and playback strategy.
* Increased cost - you need to pay for realtime LLM and speech synthesis.

### Best for

* You want a specific voice provider or voice quality that your LLM provider does not offer.
* You want more control over output speech without moving to a full cascade.
* You need a practical middle ground between speed and flexibility.

### Related guides

### [OpenAI + Cartesia](/voice-ai-orchestration/openai/half-cascade-cartesia)

Realtime LLM with Cartesia realtime TTS.

### [OpenAI + ElevenLabs](/voice-ai-orchestration/openai/half-cascade-elevenlabs)

Realtime LLM with ElevenLabs streaming TTS.

### [OpenAI + Inworld](/voice-ai-orchestration/openai/half-cascade-inworld)

Realtime LLM with Inworld speech output.

## Full cascade

Full cascade separates speech-to-text, reasoning, and text-to-speech into independent stages. This gives you the most control over the pipeline and the most freedom to mix providers.

![Full-cascade architecture in light mode](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/voximplant.docs.buildwithfern.com/b7909e251c9267ff49c41971ef232b01916748dd54dab9e517f01fbb725579ef/docs/assets/connectors/cascaded-light.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260910%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260910T045127Z&X-Amz-Expires=604800&X-Amz-Signature=4eb1db08080eb2130db808e690f7d8d8dd308269f4108781e7887f82f7b6fbf8&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)![Full-cascade architecture in dark mode](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/voximplant.docs.buildwithfern.com/a99eb42a89d407d8f66e5b2b81550304ab9948a495fafd77c3662a5144d9439e/docs/assets/connectors/cascaded-dark.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260910%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260910T045127Z&X-Amz-Expires=604800&X-Amz-Signature=316810f69de333fcf44823a540f71437a5a48ef7a4c79304389c42226c97010b&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

### How it works

* VoxEngine transcribes caller audio with an STT integration.
* Your scenario sends text to an LLM or OpenAI-compatible endpoint.
* The text response is synthesized through a TTS provider and streamed back into the call.
* Voice activity detection, turn detection, and barge-in control are handled explicitly in the orchestration layer.

### Tradeoffs

#### Advantages

* Maximum freedom to choose the best STT, LLM, and TTS provider for each stage.
* Strong control over turn-taking, interruption logic, and prompt/response handling.
* Works well when you need custom transcription behavior, domain vocabulary, or provider-specific tuning.
* Easier to swap one layer without redesigning the whole pipeline.

#### Disadvantages

* Highest implementation and operational complexity.
* More places to manage latency and streaming coordination.
* Requires deliberate turn-taking and barge-in handling for a natural caller experience.

### Best for

* You need full control over transcription, reasoning, and speech generation.
* You want to mix providers based on quality, language support, or cost.
* You need explicit turn-taking logic or custom orchestration around the LLM.

### Related guides

For turn-taking, interruption, and end-of-turn behavior, see the [Speech Flow Control guides](/voice-ai-orchestration/speech-flow-control/turn-detection).

### [Responses: Deepgram-Groq-Inworld](/voice-ai-orchestration/bring-your-own-llm/full-cascade-groq)

Independent STT, LLM, and TTS with explicit turn-taking.

### [Turn Taking Helper Library](/voice-ai-orchestration/speech-flow-control/turn-taking-helper-library)

Reference guide for VAD, turn detection, and barge-in orchestration.

### [Run TTS in VoxEngine](/getting-started/enhance-and-deploy/telephony-and-voice-ai#run-tts-in-voxengine)

Related guide for playback and TTS orchestration patterns.

## Choosing a pipeline

If you are new to Voice AI starting from scratch, we recommend the following progression:

1. Start with speech-to-speech to validate the call flow and prompt design quickly.
2. Move to hybrid-cascade if you need a different voice or output speech behavior than your speech-LLM provider offers.
3. Move to full cascade when you need full provider selection, explicit turn-taking control, and/or custom orchestration between stages.

That progression keeps the first version simple while leaving room to add flexibility only when the product requirements justify the extra moving parts.