> For a complete documentation index, fetch https://docs.voximplant.ai/llms.txt # Example: Chat Completions with Together AI
For the complete documentation index, see llms.txt.
## Overview This full-cascade example answers an inbound call, transcribes caller speech with Voximplant ASR, sends the final transcript to Together AI through the OpenAI-compatible Chat Completions interface, and speaks the completed response back into the call. Use this pattern when you want to keep Voximplant call control, ASR, and TTS in VoxEngine while using a third-party LLM provider that exposes an OpenAI-compatible Chat Completions API. **⬇️ Jump to the [Full VoxEngine scenario](#full-voxengine-scenario).** This example: 1. answers an inbound call 2. transcribes caller speech using Voximplant's built-in [ASR module](/api-reference/voxengine/asr) 3. sends each final transcript to Together AI through `OpenAI.createChatCompletionsAPIClient()` 4. logs streamed text deltas 5. speaks the completed response back to the caller using Voximplant's default TTS provider ## Prerequisites * A [Together AI account](https://api.together.ai/) with active billing or credits. * A serverless Together AI model that supports Chat Completions, for example `meta-llama/Llama-3.3-70B-Instruct-Turbo`. * Store the Together AI API key in Voximplant [Secrets](/platform/voxengine/secrets) under `TOGETHER_API_KEY`. * Create a Voximplant routing rule that points your test destination to this scenario. ## Architecture ```mermaid graph LR Caller[PSTN/SIP/WhatsApp/App users/WebRTC] --> VoxEngine[VoxEngine Scenario] VoxEngine -->|audio| ASR[Voximplant ASR] ASR -->|final transcript| Chat[OpenAI Chat Completions client] Chat -->|baseUrl: api.together.ai/v1| Together[Together AI] Chat -->|streamed text| VoxEngine VoxEngine -->|call.say text to speech| Caller ``` ## How it works * The scenario reads `TOGETHER_API_KEY` from Voximplant Secrets. * `baseUrl` points the OpenAI Chat Completions client at Together AI's OpenAI-compatible API root. * `model` uses a Together AI serverless model name. * VoxEngine ASR turns caller speech into text for the Chat Completions request. * Streamed content deltas are logged as `===LLM_DELTA===`. * The final response is logged as `===LLM_DONE===` and played back into the call with `call.say()`. Modern Voice AI flows utilize turn detection and more sophisticated speech engines. See the [Responses API example](/voice-ai-orchestration/bring-your-own-llm/full-cascade-groq) for a more advanced flow that utilizes Voximplant's turn detection and external speech integrations for improved speech quality and interactivity. ## Usage highlights * Create an OpenAI Chat Completions client with `OpenAI.createChatCompletionsAPIClient(...)`. * Set `baseUrl: "https://api.together.ai/v1"` so the VoxEngine OpenAI module sends requests to Together AI. * Use a Together model name in the `model` field instead of an OpenAI model name. * Keep conversation state in the scenario by appending user and assistant messages to the local `messages` array. * Use `call.say()` for a simple example response path. For lower latency and provider-specific voice selection, use a streaming TTS module instead. ## Full VoxEngine scenario ```javascript title={"voxeengine-openai-compatible-chat.js"} maxLines={0} /** * Bring your own LLM demo: Chat Completions through an OpenAI-compatible API. * Scenario: answer an incoming call, transcribe caller speech, and respond with an external LLM. */ require(Modules.OpenAI); require(Modules.ASR); // -------------------- OpenAI-compatible LLM settings -------------------- const LLM_BASE_URL = "https://api.together.ai/v1"; const LLM_MODEL = "meta-llama/Llama-3.3-70B-Instruct-Turbo"; const SYSTEM_PROMPT = "You are a helpful phone assistant. Keep replies short and natural."; VoxEngine.addEventListener(AppEvents.CallAlerting, async ({call}) => { let asr; let chatClient; let responseText = ""; let responseCompleted = false; let isResponding = false; const messages = [{role: "system", content: SYSTEM_PROMPT}]; // Termination function - clean up ASR and the LLM client const terminate = () => { asr?.stop(); chatClient?.close(); VoxEngine.terminate(); }; // Speak the completed assistant response back to the caller const playFinalResponse = (text) => { if (responseCompleted) return; responseCompleted = true; isResponding = false; const finalText = text || responseText || "The LLM responded successfully."; Logger.write(`===LLM_DONE=== ${finalText}`); messages.push({role: "assistant", content: finalText}); call.say(finalText); }; // Send the caller's final transcript to the LLM const askLLM = (input) => { if (isResponding) { Logger.write("===SKIP_INPUT_WHILE_RESPONDING==="); Logger.write(input); return; } isResponding = true; responseCompleted = false; responseText = ""; messages.push({role: "user", content: input}); chatClient.createChatCompletions({ model: LLM_MODEL, messages, stream: true, max_tokens: 80, }); }; call.addEventListener(CallEvents.Disconnected, terminate); call.addEventListener(CallEvents.Failed, terminate); try { call.answer(); // call.record({hd_audio: true, stereo: true}); // Optional: record the call // Create an OpenAI-compatible Chat Completions client chatClient = await OpenAI.createChatCompletionsAPIClient({ apiKey: VoxEngine.getSecretValue("TOGETHER_API_KEY"), baseUrl: LLM_BASE_URL, storeContext: false, onWebSocketClose: (event) => { Logger.write("===LLM.WebSocket.Close==="); if (event) Logger.write(JSON.stringify(event)); terminate(); }, }); // Stream text deltas as they arrive chatClient.addEventListener(OpenAI.ChatCompletionsAPIEvents.ContentDelta, (event) => { const delta = event.data.payload.delta; if (!delta) return; responseText += delta; Logger.write(`===LLM_DELTA=== ${delta}`); }); // Final content event from the Chat Completions client chatClient.addEventListener(OpenAI.ChatCompletionsAPIEvents.ContentDone, (event) => { playFinalResponse(event.data.payload.content); }); // Raw chunk event used as a fallback completion signal chatClient.addEventListener(OpenAI.ChatCompletionsAPIEvents.Chunk, (event) => { const payload = event.data.payload; const finishReason = payload.choices?.[0]?.finish_reason; if (finishReason === "stop") playFinalResponse(); }); // ---------------------- Error and diagnostic events ----------------------- chatClient.addEventListener(OpenAI.ChatCompletionsAPIEvents.ChatCompletionsAPIError, (event) => { Logger.write("===ChatCompletions.Error==="); Logger.write(JSON.stringify(event)); }); chatClient.addEventListener(OpenAI.ChatCompletionsAPIEvents.ConnectorInformation, (event) => { Logger.write("===ChatCompletions.ConnectorInformation==="); if (event?.data) Logger.write(JSON.stringify(event.data)); }); // Create ASR and send caller audio to it asr = VoxEngine.createASR({ profile: ASRProfileList.Google.en_US, }); // Send final ASR transcripts to the LLM asr.addEventListener(ASREvents.Result, (event) => { if (event.isFinal === false) return; const text = event.text?.trim(); if (!text) return; Logger.write(`===USER=== ${text}`); askLLM(text); }); asr.addEventListener(ASREvents.ASRError, (event) => { Logger.write("===ASR_ERROR==="); Logger.write(JSON.stringify(event)); }); // Bridge caller audio to ASR and greet the caller call.sendMediaTo(asr); call.say("Connected to the LLM. Ask me something."); } catch (error) { Logger.write("===UNHANDLED_ERROR==="); Logger.write(error); terminate(); } }); ``` ## More info * OpenAI module API: [https://voximplant.com/docs/references/voxengine/openai](https://voximplant.com/docs/references/voxengine/openai) * ASR module API: [https://voximplant.com/docs/references/voxengine/asr](https://voximplant.com/docs/references/voxengine/asr) * Voximplant Secrets: [/platform/voxengine/secrets](/platform/voxengine/secrets) * Together AI documentation: [https://docs.together.ai/](https://docs.together.ai/)