Listen up. If you're building modern applications, understanding Voice Transcription in AI & Artificial Intelligence is non-negotiable. This is where simple logic turns into intelligent behavior.
1Speech-to-Text with Whisper
OpenAI's Whisper model converts recorded audio into accurate text transcripts, opening up voice as an input modality — a voice note, a dictated form field, or a spoken command can all be turned into text your application already knows how to handle.
Whisper handles a wide range of accents, background noise, and even multiple languages, making it robust enough for real-world audio rather than just clean studio recordings.
// Example
console.log("Running AI concept...");AI logic processed successfully.
2Capturing Audio with MediaRecorder
Before any transcription can happen, the browser needs to actually record something — navigator.mediaDevices.getUserMedia({ audio: true }) requests microphone access, and the returned stream is handed to a MediaRecorder instance that captures audio into a Blob as the user speaks.
This is purely a browser-side capture step; nothing is sent to any API yet at this point, and the user must explicitly grant microphone permission for it to work.
navigator.mediaDevices.getUserMedia({ audio: true })
.then(stream => {
const mediaRecorder = new MediaRecorder(stream);
mediaRecorder.start();
});AI logic processed successfully.
3Why Whisper Needs multipart/form-data, Not JSON
Unlike every other AI API covered in this course, Whisper's endpoint expects a binary file upload, not a JSON body — JSON can't efficiently represent raw audio bytes. The request is built as a FormData object, appending the audio blob under the 'file' key and the model name ('whisper-1') as a separate field.
This is the same multipart/form-data encoding used by any HTML file upload form, just constructed programmatically instead of through a browser file picker.
const formData = new FormData();
formData.append('file', audioBlob, 'recording.webm');
formData.append('model', 'whisper-1');AI logic processed successfully.
4Proxying Through Your Server, Never the Browser
Exactly like the DALL-E image API, Whisper must never be called directly from client-side code — the FormData gets sent to your own backend route, which attaches the real Authorization header and forwards the request to OpenAI from a server environment where the API key stays hidden.
The audio blob travels client → your server → OpenAI, with your API key only ever appearing in the middle, server-side leg of that chain.
// Server-side (Next.js API)
const response = await fetch(WHISPER_URL, {
method: 'POST',
headers: { 'Authorization': `Bearer ${KEY}` },
body: formData
});AI logic processed successfully.
5Consuming the Transcribed Text
The response is a simple { "text": "..." } object — plain, ready-to-use text with no further parsing required. From here it can be displayed directly to the user as a transcript, or, more commonly, passed straight into a chat completion as the user's message, effectively giving a chat interface a voice input mode.
Because the output is just text, everything covered elsewhere in this course about prompts, context, and function calling applies unchanged once the audio has been transcribed.
{ "text": "Hello, world! I am speaking to an AI." }AI logic processed successfully.
6Tuning Accuracy with language and prompt
Passing a language code ('en') skips Whisper's automatic language detection step and improves both speed and accuracy when you already know what language is being spoken. The prompt field serves a different purpose: it primes the model with contextual vocabulary — brand names, technical jargon, expected topic — so it correctly spells and recognizes domain-specific words it might otherwise mishear.
Neither parameter changes what Whisper fundamentally does; they're both accuracy tuning knobs for known context about the audio.
formData.append('language', 'en');
formData.append('prompt', 'A lesson about AI development.');AI logic processed successfully.
7The Dedicated Translation Endpoint
A separate translations endpoint accepts foreign-language audio and returns English text directly, skipping the two-step process of transcribing to the original language and then running a separate translation call.
The request format is identical FormData with the same audio blob — only the URL changes — making it a drop-in alternative when the desired output is always English regardless of the spoken language.
// Translation Endpoint
const res = await fetch(TRANSLATIONS_URL, {
method: 'POST',
body: formData
});AI logic processed successfully.
8From Capture to Text: The Full Pipeline
End to end: capture audio with MediaRecorder into a Blob, package it into FormData with the model name (and optional language/prompt), POST it from your server (never the client) to Whisper, and consume the returned text — either displaying it or feeding it into a downstream LLM call.
This same pipeline scales from a simple voice-memo feature to a full voice-driven chat assistant, since everything past the transcription step is just ordinary text-based AI work.
Voice: Transcribed
AI logic processed successfully.
9Next: Monitoring the Cost of These Calls
Voice, image, and text generation calls all have real per-request costs that add up in production — the next lesson covers extracting token/usage data and setting up budget safeguards so an AI feature's spending stays predictable.
Cost Monitoring Next
AI logic processed successfully.
10Step-by-Step Breakdown
Voice interfaces are revolutionizing AI apps. OpenAI's Whisper API allows you to transcribe audio to text with incredible accuracy.
First, we need to capture audio. In browsers, we use the MediaRecorder API to record the user's voice into a 'blob'.
Whisper doesn't accept JSON. It expects 'multipart/form-data'. We must append our audio blob and the model name to a FormData object.
Checkpoint: Why do we use FormData instead of a standard JSON object for the Whisper API request?
- →JSON is faster for audio
- →JSON cannot efficiently encode binary audio files
Just like DALL-E, never call Whisper from the frontend. Send the FormData to your server, then have the server talk to OpenAI.
The API returns the transcribed text. You can then feed this text into an LLM or display it to the user.
Checkpoint: What is the maximum file size limit for a single audio file sent to the Whisper API?
- →10 MB
- →25 MB
You can also specify the 'language' to improve accuracy or use the 'prompt' parameter to give the model context about spelling.
If you need translations, Whisper has a dedicated endpoint that transcribes foreign audio directly into English text.
Checkpoint: Which parameter helps Whisper correctly identify domain-specific words or proper nouns in the audio?
- →model
- →prompt
Voice transcription mastered! You can now build apps that listen and understand.
Next, we'll learn how to monitor and manage the costs of these API calls.
Filter Real Low-Confidence Transcripts. Finish filtering out transcript segments the model wasn't confident about.
Level Up 🚀
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported.
Fully supported.
Fully supported.
Fully supported.
Accessibility (A11y)
1Voice Input Must Have a Visible, Keyboard-Operable Alternative
A microphone-based input feature should never be the only way to submit that data — provide a standard text field alongside it, since voice recording is unusable for deaf users, users in noisy or quiet-required environments, and anyone without functioning microphone permissions.
<button aria-label="Start voice recording">🎙️</button>
<textarea aria-label="Or type your message" />SEO Implications
- 1
Recorded Audio and Its Transcript Are Runtime User Data, Not Page Content
Neither the recorded audio blob nor its transcription is ever part of a server-rendered page a crawler would see — this entire feature operates on ephemeral, per-user request data, so its only connection to SEO is (as with the other lessons) making sure this documentation page's own explanation is genuinely unique.
Best Practices
Validate File Size Before Uploading
Whisper enforces a 25 MB file size limit per request — check the recorded blob's size client-side before attempting an upload, so users get immediate feedback instead of waiting for a request to fail after a long recording.
Request Microphone Permission with Clear Context
Trigger the getUserMedia() permission prompt only after the user has taken an action indicating they want to record (clicking a mic button), not on page load — an unexpected permission prompt on load is a common source of users reflexively denying access.
Frequent Bugs
Sending a recorded audio file larger than Whisper's 25 MB limit.
A long recording (especially uncompressed formats) can easily exceed the API's 25 MB cap, causing the request to fail. Check blob.size against the limit before uploading, and either compress the audio, chunk long recordings, or warn the user before they finish an overly long recording.
Real-World Examples
A Voice-to-Text Meeting Notes Feature
A meeting app records short voice memos client-side, uploads each blob through a server proxy to Whisper with a prompt hint listing the team's project names for better proper-noun recognition, and appends the returned transcript to the user's notes alongside a fallback text input for silent environments.
formData.append('prompt', 'Project names: Aurora, Nightingale, Project Falcon.');