Overview
Qwen Cloud realtime voice in VoxEngine
For the complete documentation index, see llms.txt.
Qwen Cloud’s realtime audio API is available in VoxEngine as a native connector.
The client streams caller audio to qwen-audio-3.1-realtime-plus and plays the model’s speech back into the call.
Benefits
- Built-in WebSocket connector. VoxEngine opens the Qwen Cloud session and converts call audio, so the scenario keeps call control, routing, and telephony.
- Direct speech-to-speech. One model handles recognition, reasoning, and speech output. The default model is
qwen-audio-3.1-realtime-plus. - Session control from VoxEngine. Send instructions, voice, languages, and turn detection with
sessionUpdateafterSessionCreated. - Barge-in. Clear the playback buffer when Qwen reports
InputAudioBufferSpeechStarted. - Function calling. Declare tools in the session, run them in the scenario, and return
function_call_outputbefore the next response.
Architecture
VoxEngine sits between the caller and Qwen Cloud.
The scenario creates a VoiceAgentAPIClient, applies session.update before the first audio frame, then bridges media.
Tool calls come back as events. The scenario runs the function and sends the result over the same client.
Prerequisites
- A Qwen Cloud API key. The client sends it as
Authorization: Bearer. - Store that key in Voximplant Secrets under
QWEN_API_KEY. - Optional: a Qwen Cloud workspace id, passed as
workspaceand sent asX-DashScope-WorkSpace.
Development notes
- Load the module with
require(Modules.QwenCloud)and create the client withQwenCloud.createVoiceAgentAPIClient. modeldefaults toqwen-audio-3.1-realtime-plus. The examples set it explicitly.- On
SessionCreated, callsessionUpdatebefore bridging audio.turn_detectioncan only change before the first audio frame.voiceis applied on the firstsession.update. - On
SessionUpdated, callVoxEngine.sendMediaBetween(call, client). - Greet the caller by inserting a short user text item with
conversationItemCreate, then callresponseCreate. The Qwenresponse.createevent acceptsresponse.modalitiesandresponse.voice, so the greeting text goes in the conversation item. - For barge-in, call
clearMediaBufferonInputAudioBufferSpeechStarted. - The examples use the English system voice
beth_v3.1. The model default islonganqian_v3.1. Other 3.1 system voices includebetty_v3.1,cally_v3.1,longanhuan_v3.1,longanlingxin_v3.1,longanfengyue_v3.1, andxunanchuan_v3.1. A cloned voice id can be passed invoice. - Set
input_audio_transcription.languageandoutput_audio.languageforqwen-audio-3.1-realtime-plus. Output language steers speech. Also say the reply language ininstructionsif the spoken text must stay in that language. turn_detection.typeisserver_vadby default. Set it tosmart_turnfor semantic turn detection, or setturn_detectiontonullfor push-to-talk. Push-to-talk then needsinputAudioBufferCommitandresponseCreate.enable_searchandtoolscannot be enabled together.- Function tools use a nested
functionobject. See Function calling.
server_vad starts a response when the caller stops speaking. silence_duration_ms defaults to 800 and can be set from 200 to 6000.