{"id":2838,"date":"2026-10-22T04:38:29","date_gmt":"2026-10-22T11:38:29","guid":{"rendered":"https:\/\/www.five.reviews\/?p=2838"},"modified":"2026-10-22T04:38:30","modified_gmt":"2026-10-22T11:38:30","slug":"use-mistral-voxtral-3","status":"publish","type":"post","link":"https:\/\/www.five.reviews\/how-to\/use-mistral-voxtral-3\/","title":{"rendered":"How to Use Mistral Voxtral 3: Setup, API and Voice AI Guide"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Adding speech recognition, spoken replies, or a voice assistant to an application takes more than picking a model. You need the right model for each job, a working API key, audio in a format the API accepts, and a plan for latency, cost, and errors. This guide shows how to use Mistral Voxtral 3 from setup to a working voice agent. You will make a file transcription request, stream live audio over a WebSocket, generate speech, and see how the Voice Agents SDK coordinates the full conversation loop. The guide also covers pricing and the errors developers meet most often.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Quick Summary: What You Need to Know About Voxtral 3<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Voxtral 3 is Mistral\u2019s family of audio models for <a href=\"https:\/\/www.five.reviews\/ai-tools\/gemini-3-5-transcribe\/\" target=\"_blank\" rel=\"noreferrer noopener\">speech-to-text<\/a>, text-to-speech, and voice agents. You access it through the Mistral API, using an account in Mistral AI Studio and an API key stored in the MISTRAL_API_KEY environment variable. Use voxtral-mini-transcribe-3 for recorded files, voxtral-mini-transcribe-realtime-3 for live audio, and voxtral-mini-tts-3-0 for generated speech. A first transcription request needs the Python SDK, an API key, and one audio file. Pricing depends on audio minutes for transcription and characters for speech. Choose the Voice Agents SDK when your application must listen, reason, call tools, and respond in a live conversation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What Is Mistral Voxtral 3?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/mistral.ai\/news\/voxtral-3\/\" target=\"_blank\" rel=\"noreferrer noopener\">Voxtral 3<\/a> covers three capabilities. The transcription models convert recorded audio to text, with optional speaker labels, timestamps, and custom vocabulary. The real-time transcription model turns live audio into text while the speaker is still talking. The speech models generate audio from text, with preset voices, optional voice cloning, emotion and pace tags, and custom pronunciation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These capabilities are separate, and the distinction affects your design. A transcription API receives audio and returns text. A text-to-speech API receives text and returns audio. A voice agent combines both with a reasoning model, which is a chat model that decides what to say and which tools to call. Voxtral handles the audio stages. The Voice Agents SDK runs transcription, the reasoning model, speech generation, and playback together, and you choose the model for each stage.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Which Voxtral 3 Model Should You Use?<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Task<\/strong><\/td><td><strong>Model ID<\/strong><\/td><td><strong>Mode<\/strong><\/td><td><strong>Choose it when<\/strong><\/td><\/tr><tr><td>Transcribe recordings<\/td><td>voxtral-mini-transcribe-3<\/td><td>Offline<\/td><td>You need transcripts of calls, meetings, or podcasts. Start here.<\/td><\/tr><tr><td>Higher-quality transcripts<\/td><td>voxtral-small-transcribe-3<\/td><td>Offline<\/td><td>The Mini model still makes errors that custom vocabulary does not fix.<\/td><\/tr><tr><td>Live transcription<\/td><td>voxtral-mini-transcribe-realtime-3<\/td><td>Realtime<\/td><td>Text must appear while people speak.<\/td><\/tr><tr><td>Generate speech<\/td><td>voxtral-mini-tts-3-0<\/td><td>Offline<\/td><td>You have complete text, such as a notification or an article.<\/td><\/tr><tr><td>Live speech<\/td><td>voxtral-mini-tts-realtime-3-0<\/td><td>Realtime<\/td><td>Speech is part of a live conversation.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Batch processing is enough when recordings already exist, or the user can wait for a result. Realtime is necessary when audio is still arriving, and your application must respond during the conversation. Transcription models do not generate speech, and speech models do not transcribe.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Some documentation pages write these IDs with a -3-0 suffix. Confirm the exact string your account\u2019s model list returns before you ship.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For summaries, translations, and questions about a recording, Mistral\u2019s audio-in-chat feature uses voxtral-small-latest (Voxtral 2) at launch.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How to Set Up Mistral Voxtral 3<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 1: Create a Mistral AI Studio account<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Sign in or create an account at console.mistral.ai. Studio is where you generate API keys, monitor usage, and test audio in the playground. The Mistral documentation covers activating Studio and generating a key.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 2: Generate and secure your API key<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Create a key in the Studio console, then store it in an environment variable. On macOS or Linux:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">bash<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">export MISTRAL_API_KEY=&#8221;your-api-key&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">On Windows PowerShell:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">powershell<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">$env:MISTRAL_API_KEY=&#8221;your-api-key&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Never hard-code keys in source files. Repositories and logs leak them, and any key placed in browser JavaScript is visible to every visitor. Server code should read the variable. Browser applications should request a session from your server instead.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 3: Install the required Python SDK<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">bash<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">pip install mistralai<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The voice agent SDK needs an extra install covered later in this guide. Raw WebSocket sessions need pip install websockets.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 4: Confirm your environment is ready<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">bash<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">python -c &#8220;import os; from mistralai.client import Mistral; print(&#8216;SDK ok, key set:&#8217;, &#8216;MISTRAL_API_KEY&#8217; in os.environ)&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the import fails, reinstall the package. If the key reads False, set the variable in the same terminal session.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How to Use the Voxtral 3 API for Speech-to-Text<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 1: Prepare an audio recording<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Each request accepts exactly one audio input: a local file (file), a URL the API can reach (file_url), or a file uploaded to the Files API with purpose=audio (file_id). Upload a file when you plan to transcribe it several times. Check the Audio formats and Limits pages for accepted formats and maximum sizes before you process large batches. Delete uploaded files with DELETE \/v1\/files\/{file_id} when you no longer need them.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 2: Make your first transcription request<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">python<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import os<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">from mistralai.client import Mistral<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">client = Mistral(api_key=os.environ[&#8220;MISTRAL_API_KEY&#8221;])<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">try:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;with open(&#8220;customer_call.wav&#8221;, &#8220;rb&#8221;) as audio:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;transcription = client.audio.transcriptions.complete(<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;model=&#8221;voxtral-mini-transcribe-3&#8243;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;file={&#8220;file_name&#8221;: &#8220;customer_call.wav&#8221;, &#8220;content&#8221;: audio},<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;print(transcription.text)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">except FileNotFoundError:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;print(&#8220;Audio file not found. Check the path.&#8221;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">except Exception as error:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;print(f&#8221;Transcription failed: {error}&#8221;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The client reads the key from the environment, so it never appears in code. The file_name value should match the file\u2019s real name and extension. The complete call sends the whole recording and returns the transcript in transcription.text. The response also includes segments, which stay empty unless you request timestamps, and usage.prompt_audio_seconds, which tells you how many billable seconds were processed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 3: Improve transcription quality<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Pin the language with a two-letter code, such as language=fr, when you know it in advance. Leave it out when the language varies between recordings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Add timestamps to see when each part of the call was spoken:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">bash<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">curl https:\/\/api.mistral.ai\/v1\/audio\/transcriptions \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;-H &#8220;Authorization: Bearer $MISTRAL_API_KEY&#8221; \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;-F model=voxtral-mini-transcribe-3 \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;-F file=@customer_call.wav \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;-F timestamp_granularities=segment<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Each segment includes start, end, and text. Word-level timestamps are not available, and requesting them returns an error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For a support call that mentions product names, set context_bias once per term, such as Northwind or Quillara, so the model spells them correctly. Set diarize to true to label each segment by speaker, which requires segment timestamps. Accuracy depends on recording quality, accents, and vocabulary, so test with recordings from your own environment before you rely on the output.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How to Use Voxtral 3 for Real-Time Speech Transcription<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">File transcription sends a finished recording. Real-time transcription opens a WebSocket session, streams audio as it is captured, and returns text while the speaker talks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A session follows four steps: connect, configure, stream, and finish. The server sends session. created when the connection opens. You then send session.update with your settings, and the server echoes them in session. updated. You stream audio with input_audio. append messages, receive transcription.text.delta events, and end with input_audio.end, which triggers transcription. done with the full transcript. Settings are locked once audio starts, so send them first.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The endpoint is wss:\/\/api.mistral.ai\/v1\/audio\/transcriptions\/realtime?model=voxtral-mini-transcribe-realtime-3, with your key in the Authorization header. The following script follows the official quickstart and streams a 16-bit mono WAV file in 100 ms chunks:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">python<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import asyncio, base64, json, os, wave<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import websockets<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">MODEL = &#8220;voxtral-mini-transcribe-realtime-3&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">URL = f&#8221;wss:\/\/api.mistral.ai\/v1\/audio\/transcriptions\/realtime?model={MODEL}&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">HEADERS = {&#8220;Authorization&#8221;: f&#8221;Bearer {os.environ[&#8216;MISTRAL_API_KEY&#8217;]}&#8221;}<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">AUDIO_FILE = &#8220;customer_call.wav&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">async def send_audio(ws, path):<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;with wave.open(path, &#8220;rb&#8221;) as wav:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;while chunk := wav.readframes(wav.getframerate() \/\/ 10):<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await ws.send(json.dumps({&#8220;type&#8221;: &#8220;input_audio.append&#8221;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&#8220;audio&#8221;: base64.b64encode(chunk).decode()}))<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;await ws.send(json.dumps({&#8220;type&#8221;: &#8220;input_audio.end&#8221;}))<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">async def main():<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;with wave.open(AUDIO_FILE, &#8220;rb&#8221;) as wav:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sample_rate = wav.getframerate()<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u00a0\u00a0\u00a0\u00a0async with websocketsconnect(URL, additional_headers=HEADERS) as ws:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await ws.recv()&nbsp; # session.created<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await ws.send(json.dumps({&#8220;type&#8221;: &#8220;session.update&#8221;, &#8220;session&#8221;: {<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&#8220;audio_format&#8221;: {&#8220;encoding&#8221;: &#8220;pcm_s16le&#8221;, &#8220;sample_rate&#8221;: sample_rate},<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&#8220;target_streaming_delay_ms&#8221;: 480,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&#8220;language&#8221;: &#8220;en&#8221;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}}))<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sender = asyncio.create_task(send_audio(ws, AUDIO_FILE))<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;async for message in ws:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;event = json.loads(message)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if event[&#8220;type&#8221;] == &#8220;transcription.text.delta&#8221;:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print(event[&#8220;text&#8221;], end=&#8221;&#8221;, flush=True)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;elif event[&#8220;type&#8221;] == &#8220;transcription.done&#8221;:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print(&#8220;\\nDone.&#8221;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;break<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;elif event[&#8220;type&#8221;] == &#8220;error&#8221;:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print(&#8220;error:&#8221;, event[&#8220;error&#8221;][&#8220;message&#8221;])<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;break<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;sender.cancel()<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">asyncio.run(main())<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Each text delta is final, so your application should append deltas and never replace them. Save the text from transcription. done as the transcript of record.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The target_streaming_delay_ms setting controls the latency trade-off. Values from 80 to 1200 milliseconds, or 2400, are accepted, and 480 is the documented starting point. Shorter delays show text sooner but with less accuracy. Longer delays give the model more context and more accurate text, but the text arrives later.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Test latency at the pace of real audio. Sending a file faster than real time produces misleading measurements. Also note that diarize and turn_detection cannot be enabled in the same session.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How to Use Voxtral 3 for Text-to-Speech<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Generate speech from text<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">python<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import base64<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import os<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">from pathlib import Path<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">from mistralai.client import Mistral<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">client = Mistral(api_key=os.environ[&#8220;MISTRAL_API_KEY&#8221;])<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">response = client.audio.speech.complete(<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;model=&#8221;voxtral-mini-tts-3-0&#8243;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;input=&#8221;Your order shipped this morning and should arrive on Thursday.&#8221;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;voice_id=&#8221;gb_jane_neutral&#8221;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;response_format=&#8221;mp3&#8243;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">audio = base64.b64decode(response.audio_data)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">output = Path(&#8220;order_update.mp3&#8221;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">output.write_bytes(audio)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">print(f&#8221;Saved {output.resolve()} ({len(audio)} bytes)&#8221;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The response returns audio as base64 in audio_data, so decode it before saving. The voice_id must match a preset slug from the Voices page or a voice saved in your workspace. The response_format field accepts mp3 (the default), wav, pcm, flac, or opus.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Choose voices and control delivery<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Preset voices are selected by ID, and each has its own language and accent. Write input in the language you want spoken. Inline emotion and pace tags adjust delivery word by word. The docs list 34 emotions and slow and fast paces. The phoneme_hints field sets the pronunciation of names and terms that the model might misread. The expressiveness setting accepts values from 1.0 to 5.0. Raise it when emotion does not come through, and lower it when the speech sounds forced. Style instructions were not available at launch. For playback before the file is complete, set stream to true.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Understand voice cloning<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Zero-shot cloning reproduces a voice from a short recording. You can pass a base64-encoded recording as ref_audio for a single request, or save the voice in your workspace and reuse its ID. Clone only voices you have permission to use, and disclose synthetic speech where listeners could be misled about who is speaking.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How to Build a Real-Time Voice Agent With Voxtral 3<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Understand the voice-agent workflow<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A voice agent runs one conversational turn through five stages:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>The microphone captures the user\u2019s speech.<\/li>\n\n\n\n<li>Real-time transcription converts speech to text and detects when the user has finished.<\/li>\n\n\n\n<li>The reasoning model reads the conversation and writes a reply. If it needs data, it calls a tool.<\/li>\n\n\n\n<li>Speech generation turns the reply into audio in the agent\u2019s voice.<\/li>\n\n\n\n<li>The SDK plays the audio, and the user can interrupt mid-sentence.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Tools come in two types. Server-side tools run at Mistral, including web search, document libraries, and workspace connectors. Client-side tools run your own functions. On a phone line, no code of yours runs during the call, so phone agents must use server-side tools.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Install and run the SDK<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">On macOS, install PortAudio with brew install portaudio. On Linux, run sudo apt install libportaudio2 libportaudiocpp0 portaudio19-dev libasound-dev libsndfile1-dev -y. Then install the SDK with its audio extra:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">bash<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">pip install &#8220;mistralai[voice-agents]&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Save the following file and run it. The agent listens through your microphone and answers through your speakers:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">python<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import asyncio<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">import os<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">from mistralai.client import Mistral<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">from mistralai.extra.voice_agents import DefaultAudioInterface<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">async def main():<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;async with Mistral(api_key=os.environ[&#8220;MISTRAL_API_KEY&#8221;]) as client:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;session = await client.audio.sessions.init(<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;audio_interface=DefaultAudioInterface(),<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;instructions=&#8221;Please respond to me in the most dramatic way possible.&#8221;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;def on_transcript(event):<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;print(f&#8221;{event.role}: {event.content}&#8221;)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;session.on(&#8220;transcript&#8221;, on_transcript)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;async with session:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await session.wait()<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">asyncio.run(main())<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is a minimal example. Change instructions to define the agent\u2019s behavior. To change the voice, pass tts={&#8220;voice_id&#8221;: &#8220;fr_marie_neutral&#8221;} to the session.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Configure models, voice, and tools<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You choose the model for each stage per agent or per session. Save an agent when several applications or phone lines should share one configuration, because each change creates a new version and you can pin a version in production. For prototypes, pass the full configuration when you connect. Settings you pass on connection override the saved agent for that session only.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Add practical tool calling<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider a support agent that answers order-status questions. A caller asks, \u201cWhere is order 4417?\u201d The reasoning model calls a tool named for order lookup and passes the order number. Your function queries your order system and returns the status. The agent checks the result, then speaks an answer such as \u201cOrder 4417 left the warehouse this morning.\u201d While the tool runs, the SDK can play a short spoken filler so the caller does not hear silence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Validate tool inputs and results before the agent speaks them. Require explicit confirmation before consequential actions, such as cancelling an order. The Tools page documents how to declare a function with its name, description, and parameters.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Handle interruptions and deployment<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Users can talk over the agent, and the SDK manages turn detection and interruptions. The SDK builds on LiveKit for real-time audio transport, but you do not need to know LiveKit to use it. For a web page, your server creates the session so the API key never reaches the browser. Telephony through SIP is available to enterprise customers through an account team. Production systems also need signed webhooks to record call starts, ends, and failures, plus retries, timeouts, and logging. The SDK example above does not include these.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How Much Does Voxtral 3 Cost?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Mistral prices speech models by audio minute for transcription and by characters for speech. Check the pricing page for the current rate of each model before you budget, because rates can change. The listing I reviewed showed these rates:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Capability<\/strong><\/td><td><strong>Listed rate<\/strong><\/td><td><strong>Billing unit<\/strong><\/td><\/tr><tr><td>Transcription (listed as Voxtral Mini Transcribe 2)<\/td><td>$0.004<\/td><td>Per audio minute<\/td><\/tr><tr><td>Voxtral TTS<\/td><td>$0.016<\/td><td>Per 1,000 characters<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Here is an illustrative calculation with those rates. A helpline transcribes 1,000 call minutes per month, costing 1,000 \u00d7 $0.004 = $4.00. It also generates 200,000 characters of spoken replies, costing 200 \u00d7 $0.016 = $3.20. The two components total $7.20 per month.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That total is incomplete. A voice agent also pays for the reasoning model, telephony, hosting, and storage. Build cost estimates from your real call volumes and average call length, not from speech rates alone.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Best Practices, Limitations, and Common Errors<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Best practices for production use<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Test with recordings from your real environment, including the accents, background noise, and vocabulary your users produce.<\/li>\n\n\n\n<li>Add context_bias terms for names and product codes, and measure accuracy before and after.<\/li>\n\n\n\n<li>Keep API keys on the server, and never log raw audio that contains personal data without a retention policy.<\/li>\n\n\n\n<li>Obtain consent before recording callers and before cloning any voice.<\/li>\n\n\n\n<li>Measure end-to-end latency at real-time pace, not from file uploads.<\/li>\n\n\n\n<li>Set timeouts and retries around every API and WebSocket call, and define a fallback message for outages.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Common API errors and how to fix them<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td><strong>Error or symptom<\/strong><\/td><td><strong>Likely cause<\/strong><\/td><td><strong>Fix<\/strong><\/td><\/tr><tr><td>422 Invalid language alpha2 code<\/td><td>The language value is not a two-letter code<\/td><td>Use a code such as en or fr<\/td><\/tr><tr><td>400 Word-level segments are not enabled<\/td><td>Word timestamps were requested<\/td><td>Use timestamp_granularities=segment<\/td><\/tr><tr><td>422 when diarize is set<\/td><td>Diarization requested without segment timestamps<\/td><td>Add timestamp_granularities=segment<\/td><\/tr><tr><td>400 maximum context length<\/td><td>Audio plus context_bias is too long<\/td><td>Shorten the bias list<\/td><\/tr><tr><td>Session fields cannot be changed after audio has started<\/td><td>session.update sent after streaming began<\/td><td>Send settings first, or open a new session<\/td><\/tr><tr><td>Diarization is not compatible with turn detection<\/td><td>Both features enabled in one realtime session<\/td><td>Keep only one, or use separate sessions<\/td><\/tr><tr><td>404 invalid_voice<\/td><td>Voice ID does not exist in the catalog or workspace<\/td><td>Copy the exact ID from the Voices page<\/td><\/tr><tr><td>400 Either ref_audio or voice must be provided<\/td><td>TTS request has no voice<\/td><td>Add voice_id<\/td><\/tr><tr><td>Voice agent captures no audio<\/td><td>Native audio libraries missing<\/td><td>Install PortAudio, then reinstall the extra<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Limitations to understand before deployment<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Word-level timestamps are not available. Real-time segments carry speaker labels but no start or end times. Language coverage differs by model: the languages page lists 50 languages for file transcription, 45 languages and dialects for real-time transcription, and 35 languages with 50-plus dialects for speech generation. Vendor benchmarks may not predict results for your accents, industry terms, or recording conditions. The sub-200 ms latency figure in the documentation should be measured in your own setup.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Final Verdict<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Use batch transcription for recordings, call archives, and meetings you can process after they end. Use real-time transcription for live captions, live note-taking, or any feature that must react while someone speaks. Use text-to-speech when your application needs generated audio, and use the Voice Agents SDK when it must listen, reason, call tools, and respond in a conversation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Choose between the hosted API and any self-hosted option based on your privacy requirements, latency targets, and budget, and verify each deployment option against the model card before you commit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Your next step is to create an API key, transcribe one of your own recordings with the file example above, and then pick the model that matches your application\u2019s mode.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Frequently Asked Questions&nbsp;<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What is Mistral Voxtral 3 used for?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Voxtral 3 converts recorded and live speech to text, generates speech from text, and powers voice agents that listen, reason, and respond. It is accessed through the Mistral API and the Voice Agents SDK.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>How do I get access to the Mistral Voxtral 3 API?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Create an account at console.mistral.ai, activate Studio, and generate an API key. Store the key in the MISTRAL_API_KEY environment variable and call the API with the Python SDK or HTTP requests.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Which Voxtral 3 model is best for real-time transcription?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Use voxtral-mini-transcribe-realtime-3. It is the realtime model, and it supports sessions, turn detection, and a streaming delay you configure.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Can Voxtral 3 convert text into speech?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes. Use voxtral-mini-tts-3-0 for complete text and voxtral-mini-tts-realtime-3-0 for speech in live conversations. Both support preset voices, and the offline model also supports voice cloning.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Does Voxtral 3 support Hindi and other languages?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Language support depends on the model and mode. The languages page lists the number of supported languages for each model but does not name each one in the text I reviewed. Check the list for your specific model before you deploy in Hindi.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>How much does the Voxtral 3 API cost?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Transcription is billed per audio minute, and speech generation is billed per character. The listed rates on Mistral\u2019s pricing page are about $0.004 per minute for transcription and $0.016 per 1,000 characters for TTS, but confirm the current rate for each Voxtral 3 model before budgeting.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Can I use Voxtral 3 to build a voice agent?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes. The Voice Agents SDK runs transcription, a reasoning model, speech generation, and playback together. You still need to supply the agent\u2019s instructions, the tools it can call, and the logic for what happens when a call ends.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Does Voxtral 3 support voice cloning?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes. Zero-shot cloning reproduces a voice from a short recording, either for one request through ref_audio or as a saved voice you reuse by ID. Use cloning only with consent.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Can I deploy Voxtral 3 on my own infrastructure?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The documentation I reviewed does not confirm open weights for every Voxtral 3 model. An earlier realtime model was published under Apache 2.0, so check the model card for the model you plan to use before you assume self-hosting is possible.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>What is the difference between batch and real-time transcription?<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Batch transcription sends a complete recording and returns the transcript once processing finishes. Real-time transcription streams audio over a WebSocket and returns text while the speaker is still talking.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Adding speech recognition, spoken replies, or a voice assistant to an application takes more than picking a model. You need the right model for [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":2839,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[6],"tags":[651,1249,336,1245,1246,652,1250,1247,1248],"content_cluster":[7],"content_type":[21],"search_intent":[24],"tool_category":[28],"class_list":["post-2838","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-how-to","tag-ai-transcription","tag-audio-processing","tag-mistral-ai","tag-mistral-voxtral-3","tag-real-time-transcription","tag-speech-to-text","tag-text-to-speech-2","tag-voice-ai","tag-voxtral-api","content_cluster-how-to","content_type-how-to-guide","search_intent-informational","tool_category-ai-writing"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/posts\/2838","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Fcomments&post=2838"}],"version-history":[{"count":1,"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/posts\/2838\/revisions"}],"predecessor-version":[{"id":2840,"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/posts\/2838\/revisions\/2840"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=\/wp\/v2\/media\/2839"}],"wp:attachment":[{"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2838"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Fcategories&post=2838"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Ftags&post=2838"},{"taxonomy":"content_cluster","embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Fcontent_cluster&post=2838"},{"taxonomy":"content_type","embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Fcontent_type&post=2838"},{"taxonomy":"search_intent","embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Fsearch_intent&post=2838"},{"taxonomy":"tool_category","embeddable":true,"href":"https:\/\/www.five.reviews\/?rest_route=%2Fwp%2Fv2%2Ftool_category&post=2838"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}