> For a complete documentation index, fetch https://docs.voximplant.ai/llms.txt
# Responses API Example: Microsoft-Grok-Grok TTS
> Combine Microsoft streaming STT, Grok reasoning through the Responses API client, and Grok realtime TTS in a VoxEngine call.
> For the complete documentation index, see [llms.txt](/llms.txt).
## Overview
This cascaded pipeline uses separate components for each stage:
1. Microsoft streaming speech recognition transcribes caller audio.
2. Grok processes completed user turns through VoxEngine's OpenAI-compatible Responses client.
3. Grok realtime TTS turns the response text into speech.
4. [Agentic VoiceDSP](/voice-ai-orchestration/speech-flow-control/agentic-voicedsp) coordinates speech detection, end-of-turn detection, and barge-in.
The sample uses the `grok-4.7` text model at `https://api.x.ai/v1`. Change `GROK_MODEL` to another [Responses-compatible model](https://docs.x.ai/developers/models) available to your xAI account when needed. STT, reasoning, and TTS can each be replaced with another supported provider.
> **Note**
>
> For a direct speech-to-speech integration, use the [Grok Voice Agent](/voice-ai-orchestration/grok/overview). For standalone speech output without STT or an LLM, see [xAI realtime TTS](/feature-guides/speech/grok-tts).
Jump to the [Full VoxEngine scenario](#full-voxengine-scenario).
## Prerequisites
* Create a [VoxEngine scenario](/platform/voxengine/scenarios) and route an incoming call to it through an application's [routing rule](/platform/voxengine/routing-rules).
* Store an xAI API key with Responses API access in Voximplant [Secrets](/platform/voxengine/secrets) as `XAI_API_KEY`. This key is required for the LLM; the scenario also uses it for TTS.
* The example imports `Modules.Agentic` and creates VoiceDSP directly; no preceding helper scenario is required.
## How it works
* After the call connects, the scenario creates Microsoft ASR with interim results and phrase hints.
* It creates `OpenAI.createResponsesAPIClient({ baseUrl: "https://api.x.ai/v1" })`. Voximplant handles provider requests and exposes Responses events through the client.
* `Agentic.createVoiceDSP({ asr: stt, ... })` receives the same Microsoft ASR instance used for call media. VAD starts a user turn; Smart Turn ends it. `TurnStopped.content` supplies the latest transcript snapshot after the configured 0.3-second ASR grace window. The sample waits for this event instead of sending duplicate requests from `TurnInferenceTriggered`, which carries no transcript.
* `ResponseTextDelta` events supply text to `XAI.createRealtimeTTSPlayer()`. `ResponseTextDone` sends `text.done` to finish the TTS utterance.
* Caller media flows independently to ASR and VoiceDSP. VoiceDSP does not replace Microsoft transcription. TTS media flows back to the caller. Noise suppression is omitted.
* A greeting request starts the conversation. `storeContext: true` lets the Responses connector maintain context across requests.
## Barge-in and response lifecycle
When VoiceDSP emits `TurnStarted`, the scenario marks the caller as speaking and invalidates any pending response, including one that has not produced speech yet. The scenario calls `ttsPlayer.clearBuffer()`, which sends `text.clear` and clears queued playback. It also marks the current LLM response as discarded. Both late text deltas and late text completion events are ignored.
The current VoxEngine Responses client exposes `createResponses()` and `close()`, with no per-response cancellation method. The sample lets the interrupted request finish silently and queues the next completed user turn until `ResponseCompleted`. This prevents an old response from resuming when a later user turn ends. It does not cancel provider inference or its billing.
Requests are serialized, and a newer completed user turn replaces any queued turn. On call failure, disconnect, ASR, Responses, VoiceDSP, or TTS error, or the three-minute demo deadline, the scenario closes the Responses client and VoiceDSP, stops ASR and TTS, and terminates. If the caller hangs up during asynchronous setup, the newly created resource is closed when setup resolves.
## Playback and idle tracking
The first accepted text delta signals `signalAssistantTurnStarted()` as text is queued for speech. `ResponseCompleted` is the end of inference, not the end of playback. After `ResponseTextDone`, the next `AudioChunksPlaybackFinished` signals `signalAssistantTurnEnded()`. Intermediate chunk completions do not mark the assistant idle. Interruption clears TTS and ends that assistant turn immediately. The 30-second `TurnIdle` event ends the demo only when no user turn, response request, or assistant playback is active.
## Developer notes
* The prompt requests short, speakable text and occasional [xAI speech tags](https://docs.x.ai/developers/model-capabilities/audio/text-to-speech#speech-tags), such as `[pause]` or `...`.
* The TTS adapter uses PCM at 16 kHz. Configure `voice`, `language`, and the documented [connection options](/feature-guides/speech/grok-tts#connection-options).
* The sample logs Responses lifecycle events and TTS time to first byte. Use these logs when checking your own account and model; they do not replace listening to a test call.
## Full VoxEngine scenario
```javascript title={"voxeengine-grok-full-cascade.js"} maxLines={0}
/**
* Cascaded voice AI: Microsoft STT → Grok Responses → Grok realtime TTS.
* Adapted from Chad Hart's example in https://github.com/voximplant/fern-docs/pull/56.
* VoiceDSP provides turn boundaries; no preceding helper scenario is required.
*/
require(Modules.Agentic);
require(Modules.ASR);
require(Modules.OpenAI);
require(Modules.XAI);
const GROK_MODEL = "grok-4.7";
const SYSTEM_PROMPT = `
You are Voxi, a concise phone assistant for Voximplant. Reply in English.
Keep answers to one or two short sentences and ask one concise question when needed.
If asked about this demo, explain: Microsoft transcribes, Grok reasons, Grok TTS speaks.
Write only speakable text; avoid markdown, URLs, emojis, and stage directions.
Use at most one xAI speech tag per short answer: [pause], [laugh],
..., ..., or ....
`;
VoxEngine.addEventListener(AppEvents.CallAlerting, ({call}) => {
/** @type {ASR | undefined} */
let stt;
/** @type {OpenAI.ResponsesAPIClient | undefined} */
let responsesClient;
/** @type {XAI.RealtimeTTSPlayer | undefined} */
let ttsPlayer;
/** @type {Agentic.VoiceDSP | undefined} */
let voiceDSP;
let finished = false;
let responsePending = false;
let discardResponse = false;
let queuedInput;
let userSpeaking = false;
let assistantSpeaking = false;
let textFinished = false;
let idleRearmTimer;
const terminate = () => {
if (finished) return;
finished = true;
clearTimeout(timeout);
clearTimeout(idleRearmTimer);
stt?.stop();
voiceDSP?.close();
ttsPlayer?.stop();
responsesClient?.close();
VoxEngine.terminate();
};
const timeout = setTimeout(terminate, 180000);
call.addEventListener(CallEvents.Disconnected, terminate);
call.addEventListener(CallEvents.Failed, terminate);
call.addEventListener(CallEvents.Connected, async () => {
try {
const xaiApiKey = VoxEngine.getSecretValue("XAI_API_KEY");
if (!xaiApiKey) throw new Error("Create the XAI_API_KEY Voximplant Secret before running this LLM example.");
stt = VoxEngine.createASR({
profile: ASRProfileList.Microsoft.en_US,
interimResults: true,
phraseHints: ["Voximplant", "VoxEngine", "Grok", "xAI"],
});
stt.addEventListener(ASREvents.ASRError, (event) => {
Logger.write("===ASR.Error=== " + JSON.stringify(event));
terminate();
});
const responsesParameters = {
apiKey: xaiApiKey,
baseUrl: "https://api.x.ai/v1",
storeContext: true,
onWebSocketClose: (event) => {
Logger.write("===Grok.Responses.WebSocket.Close=== " + JSON.stringify(event));
terminate();
},
};
responsesClient = await OpenAI.createResponsesAPIClient(responsesParameters);
if (finished) { responsesClient.close(); return; }
ttsPlayer = XAI.createRealtimeTTSPlayer({
apiKey: xaiApiKey,
connectionParameters: {voice: "ara", language: "en", optimize_streaming_latency: 1},
});
ttsPlayer.addEventListener(PlayerEvents.Error, (event) => {
Logger.write("===Grok.TTS.Error=== " + JSON.stringify(event));
terminate();
});
ttsPlayer.addEventListener(PlayerEvents.Started, () => {
if (finished || userSpeaking) return;
assistantSpeaking = true;
voiceDSP?.signalAssistantTurnStarted();
});
ttsPlayer.addEventListener(PlayerEvents.AudioChunksPlaybackFinished, (event) => {
if (finished) return;
Logger.write("===Grok.TTS.TTFB=== " + event.timeToFirstByte);
// Do not declare the assistant idle between chunks or at LLM completion.
if (textFinished && assistantSpeaking) {
assistantSpeaking = false;
voiceDSP.signalAssistantTurnEnded();
}
});
const requestResponse = (input) => {
if (finished) return;
// One response at a time: interrupted responses finish silently.
if (responsePending) { queuedInput = input; return; }
responsePending = true;
discardResponse = false;
responsesClient.createResponses({model: GROK_MODEL, instructions: SYSTEM_PROMPT, input});
};
const responseFinished = () => {
if (finished) return;
responsePending = false;
if (queuedInput !== undefined) {
const input = queuedInput;
queuedInput = undefined;
requestResponse(input);
} else if (!assistantSpeaking) {
voiceDSP.signalAssistantTurnEnded();
}
};
voiceDSP = await Agentic.createVoiceDSP({
asr: stt,
vadParameters: {threshold: 0.5, minSilenceDurationMs: 350, speechPadMs: 10},
turnDetectorParameters: {threshold: 0.5, stopSecs: 3},
asrGraceTimeout: 0.3,
userTurnStopTimeout: 5,
userTurnIdleTimeout: 30,
startStrategies: [Agentic.VoiceDSPStrategies.createVADStart({enableInterruptions: true})],
stopStrategies: [Agentic.VoiceDSPStrategies.createTurnDetectorStop()],
});
if (finished) { voiceDSP.close(); return; }
voiceDSP.addEventListener(Agentic.VoiceDSPEvents.TurnStarted, (event) => {
if (finished) return;
userSpeaking = true;
clearTimeout(idleRearmTimer);
Logger.write("===USER TURN STARTED=== interrupted=" + event.interrupted);
// A user turn invalidates pending inference even before TTS has started.
discardResponse = true;
queuedInput = undefined;
textFinished = false;
ttsPlayer.clearBuffer();
assistantSpeaking = false;
voiceDSP.signalAssistantTurnEnded();
});
voiceDSP.addEventListener(Agentic.VoiceDSPEvents.TurnStopped, (event) => {
if (finished) return;
userSpeaking = false;
const input = event.content.trim();
Logger.write("===USER===> " + input + " transcript=" + event.transcriptState);
// TurnStopped carries the transcript; TurnInferenceTriggered does not.
if (input) requestResponse(input);
else if (!assistantSpeaking) {
// VoiceDSP resets its active turn after this callback returns.
clearTimeout(idleRearmTimer);
idleRearmTimer = setTimeout(() => {
if (!finished && !responsePending && !userSpeaking && !assistantSpeaking) {
voiceDSP.signalAssistantTurnEnded();
}
}, 0);
}
});
voiceDSP.addEventListener(Agentic.VoiceDSPEvents.TurnIdle, () => {
if (!finished && !responsePending && !userSpeaking && !assistantSpeaking) terminate();
});
voiceDSP.addEventListener(Agentic.VoiceDSPEvents.Error, (event) => {
Logger.write("===VoiceDSP.Error=== " + JSON.stringify(event));
terminate();
});
[Agentic.VoiceDSPEvents.TurnInferenceTriggered, Agentic.VoiceDSPEvents.TurnStopTimeout,
Agentic.VoiceDSPEvents.ConnectorInformation].forEach((eventName) => {
voiceDSP.addEventListener(eventName, (event) => {
Logger.write("===VoiceDSP " + eventName + "=== " + JSON.stringify(event));
});
});
responsesClient.addEventListener(OpenAI.ResponsesAPIEvents.ResponseTextDelta, (event) => {
const payload = /** @type {{delta?: string}} */ (event?.data?.payload || {});
const text = payload.delta;
if (finished || discardResponse || !responsePending || !text || userSpeaking) return;
textFinished = false;
if (!assistantSpeaking) {
assistantSpeaking = true;
voiceDSP.signalAssistantTurnStarted();
}
ttsPlayer.send({type: "text.delta", delta: text});
});
responsesClient.addEventListener(OpenAI.ResponsesAPIEvents.ResponseTextDone, (event) => {
if (finished || discardResponse || !responsePending || userSpeaking) return;
const payload = /** @type {{text?: string}} */ (event?.data?.payload || {});
Logger.write("===AGENT===> " + payload.text);
textFinished = true;
ttsPlayer.send({type: "text.done"});
if (!assistantSpeaking) voiceDSP.signalAssistantTurnEnded();
});
responsesClient.addEventListener(OpenAI.ResponsesAPIEvents.ResponseCompleted, responseFinished);
[OpenAI.ResponsesAPIEvents.ResponseFailed, OpenAI.ResponsesAPIEvents.ResponseIncomplete,
OpenAI.ResponsesAPIEvents.ResponseError, OpenAI.ResponsesAPIEvents.ResponsesAPIError].forEach((eventName) => {
responsesClient.addEventListener(eventName, (event) => {
Logger.write("===Grok.Responses.Error=== " + JSON.stringify(event));
terminate();
});
});
[OpenAI.ResponsesAPIEvents.ResponseCreated, OpenAI.ResponsesAPIEvents.ResponseInProgress,
OpenAI.ResponsesAPIEvents.ResponseCompleted, OpenAI.ResponsesAPIEvents.ConnectorInformation,
OpenAI.ResponsesAPIEvents.Unknown, OpenAI.Events.WebSocketMediaStarted,
OpenAI.Events.WebSocketMediaEnded].forEach((eventName) => {
responsesClient.addEventListener(eventName, (event) => {
Logger.write("===Grok.Responses " + eventName + "=== " + JSON.stringify(event));
});
});
call.sendMediaTo(stt);
call.sendMediaTo(voiceDSP);
ttsPlayer.sendMediaTo(call);
requestResponse("Greet the caller as Voxi, include one [pause], and ask how you can help.");
} catch (error) {
Logger.write("===Grok pipeline setup failed=== " + error);
terminate();
}
});
call.answer();
});
```
## More info
* [xAI realtime TTS guide](/feature-guides/speech/grok-tts)
* [OpenAI Responses API client](/api-reference/voxengine/openai#responsesapiclient)
* [XAI.RealtimeTTSPlayer](/api-reference/voxengine/xai/realtime-tts-player)
* [VoxEngine.createASR](/api-reference/voxengine/voxengine#createasr)
* [Agentic VoiceDSP](/voice-ai-orchestration/speech-flow-control/agentic-voicedsp)
* [PlayerEvents](/api-reference/voxengine/player-events)
* [xAI Responses API](https://docs.x.ai/developers/model-capabilities/text/generate-text)