Skip to content

Add voice and transcriber fallback docs #847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vtkovapi merged 3 commits into from
Dec 15, 2025
Merged

Add voice and transcriber fallback docs #847

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a866775
docs: add transcriber fallback configuration guide
vtkovapi Dec 15, 2025
123b8d1
docs: update voice fallback configuration guide
vtkovapi Dec 15, 2025
9efbc31
docs: update navigation for voice and transcriber fallback configurat…
vtkovapi Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 174 additions & 0 deletions fern/customization/transcriber-fallback-plan.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
title: Transcriber fallback configuration
subtitle: Configure fallback transcribers that activate automatically if your primary transcriber fails.
slug: customization/transcriber-fallback-plan
---

## Overview

Transcriber fallback configuration ensures your calls continue even if your primary speech-to-text provider experiences issues. Your assistant will sequentially fallback to the transcribers you configure, in the exact order you specify.

**Key benefits:**
- **Call continuity** during provider outages
- **Automatic failover** with no user intervention required
- **Provider diversity** to protect against single points of failure

<Note>
Without a fallback plan configured, your call will end with an error if your chosen transcription provider fails.
</Note>

## How it works

When a transcriber failure occurs, Vapi will:
1. Detect the failure of the primary transcriber
2. Switch to the first fallback transcriber in your plan
3. Continue through your specified list if subsequent failures occur
4. Terminate only if all transcribers in your plan have failed

## Configure via Dashboard

<Steps>
<Step title="Open Transcriber tab">
Navigate to your assistant and select the **Transcriber** tab.
</Step>
<Step title="Expand Fallback Transcribers section">
Scroll down to find the **Fallback Transcribers** collapsible section. A warning indicator appears if no fallback transcribers are configured.
</Step>
<Step title="Add a fallback transcriber">
Click **Add Fallback Transcriber** to configure your first fallback:
- Select a **provider** from the dropdown
- Choose a **model** (if the provider offers multiple models)
- Select a **language** for transcription
</Step>
<Step title="Configure provider-specific settings (optional)">
Expand **Additional Configuration** to access provider-specific settings like numerals formatting, VAD settings, and confidence thresholds.
</Step>
<Step title="Add more fallbacks">
Repeat to add additional fallback transcribers. Order matters—the first fallback in your list is tried first.
</Step>
</Steps>

<Note>
If HIPAA or PCI compliance is enabled on your account or assistant, only **Deepgram** and **Azure** transcribers will be available as fallback options.
</Note>

## Configure via API

Add the `fallbackPlan` property to your assistant's transcriber configuration, and specify the fallback transcribers within the `transcribers` property.

```json
{
"transcriber": {
"provider": "deepgram",
"model": "nova-3",
"language": "en",
"fallbackPlan": {
"transcribers": [
{
"provider": "assembly-ai",
"speechModel": "universal-streaming-multilingual",
"language": "en"
},
{
"provider": "azure",
"language": "en-US"
}
]
}
}
}
```

## Provider-specific settings

Each transcriber provider supports different configuration options. Expand the accordion below to see available settings for each provider.

<AccordionGroup>
<Accordion title="Deepgram">
- **model**: Model selection (`nova-3`, `nova-3-general`, `nova-3-medical`, `nova-2`, `flux-general-en`, etc.).
- **language**: Language code for transcription.
- **keywords**: Keywords with optional boost values for improved recognition (e.g., `["companyname", "productname:2"]`).
- **keyterm**: Keyterm prompting for up to 90% keyword recall rate improvement.
- **smartFormat** (boolean): Enable smart formatting for numbers and dates.
- **eotThreshold** (0.5-0.9): End-of-turn confidence threshold. Only available with Flux models.
- **eotTimeoutMs** (500-10000): Maximum time to wait after speech before finalizing turn. Only available with Flux models. Default is 5000ms.
</Accordion>
<Accordion title="AssemblyAI">
- **language**: Language code (`multi` for multilingual, `en` for English).
- **speechModel**: Streaming speech model (`universal-streaming-english` or `universal-streaming-multilingual`).
- **wordBoost**: Custom vocabulary array (up to 2500 characters total).
- **keytermsPrompt**: Array of keyterms for improved recognition (up to 100 terms, 50 characters each). Costs additional $0.04/hour.
- **endUtteranceSilenceThreshold**: Duration of silence in milliseconds to detect end of utterance.
- **disablePartialTranscripts** (boolean): Set to `true` to disable partial transcripts.
- **confidenceThreshold** (0-1): Minimum confidence threshold for accepting transcriptions. Default is 0.4.
- **vadAssistedEndpointingEnabled** (boolean): Enable VAD-based endpoint detection.
</Accordion>
<Accordion title="Azure">
- **language**: Language code in BCP-47 format (e.g., `en-US`, `es-MX`, `fr-FR`).
- **segmentationSilenceTimeoutMs** (100-5000): Duration of silence after which a phrase is finalized. Configure to adjust sensitivity to pauses.
- **segmentationMaximumTimeMs** (20000-70000): Maximum duration a segment can reach before being cut off.
- **segmentationStrategy**: Controls phrase boundary detection. Options: `Default`, `Time`, or `Semantic`.
</Accordion>
<Accordion title="Gladia">
- **model**: Model selection (`fast`, `accurate`, or `solaria-1`).
- **language**: Language code.
- **confidenceThreshold** (0-1): Minimum confidence for transcription acceptance. Default is 0.4.
- **endpointing** (0.01-10): Time in seconds to wait before considering speech ended.
- **speechThreshold** (0-1): Speech detection sensitivity (0.0 to 1.0).
- **prosody** (boolean): Enable prosody detection (laugh, giggle, music, etc.).
- **audioEnhancer** (boolean): Pre-process audio for improved accuracy (increases latency).
- **transcriptionHint**: Hint text to guide transcription.
- **customVocabularyEnabled** (boolean): Enable custom vocabulary.
- **customVocabularyConfig**: Custom vocabulary configuration with vocabulary array and default intensity.
- **region**: Processing region (`us-west` or `eu-west`).
- **receivePartialTranscripts** (boolean): Enable partial transcript delivery.
</Accordion>
<Accordion title="Speechmatics">
- **model**: Model selection (currently only `default`).
- **language**: Language code.
- **operatingPoint**: Accuracy level. `standard` for faster turnaround, `enhanced` for highest accuracy. Default is `enhanced`.
- **region**: Processing region (`eu` for Europe, `us` for United States). Default is `eu`.
- **enableDiarization** (boolean): Enable speaker identification for multi-speaker conversations.
- **maxDelayMs**: Maximum delay in milliseconds for partial transcripts. Balances latency and accuracy.
</Accordion>
<Accordion title="Google">
- **model**: Gemini model selection.
- **language**: Language selection (e.g., `Multilingual`, `English`, `Spanish`, `French`).
</Accordion>
<Accordion title="OpenAI">
- **model**: OpenAI Realtime STT model selection (required).
- **language**: Language code for transcription.
</Accordion>
<Accordion title="ElevenLabs">
- **model**: Model selection (currently only `scribe_v1`).
- **language**: ISO 639-1 language code.
</Accordion>
<Accordion title="Cartesia">
- **model**: Model selection (currently only `ink-whisper`).
- **language**: ISO 639-1 language code.
</Accordion>
</AccordionGroup>

## Best practices

- Use **different providers** for fallbacks to protect against provider-wide outages.
- Consider **language compatibility** when selecting fallbacks—ensure all fallback transcribers support your required languages.
- Test your fallback configuration to ensure smooth transitions between transcribers.
- For **HIPAA/PCI compliance**, ensure all fallbacks are compliant providers (Deepgram or Azure).

## FAQ

<AccordionGroup>
<Accordion title="Which providers support fallback?">
All major transcriber providers are supported: Deepgram, AssemblyAI, Azure, Gladia, Google, Speechmatics, Cartesia, ElevenLabs, and OpenAI.
</Accordion>
<Accordion title="Does fallback affect pricing?">
No additional fees for using fallback transcribers. You are only billed for the transcriber that processes the audio.
</Accordion>
<Accordion title="How fast is the failover?">
Failover typically occurs within milliseconds of detecting a failure, ensuring minimal disruption to the call.
</Accordion>
<Accordion title="Can I use different languages for fallbacks?">
Yes, each fallback transcriber can have its own language configuration. However, for the best user experience, we recommend using the same or similar languages across all fallbacks.
</Accordion>
</AccordionGroup>
4 changes: 3 additions & 1 deletion fern/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,10 @@ navigation:
path: customization/speech-configuration.mdx
- page: Voice pipeline configuration
path: customization/voice-pipeline-configuration.mdx
- page: Voice fallback plan
- page: Voice fallback configuration
path: voice-fallback-plan.mdx
- page: Transcriber fallback configuration
path: customization/transcriber-fallback-plan.mdx
- page: OpenAI realtime speech-to-speech
path: openai-realtime.mdx
- page: Provider keys
Expand Down
174 changes: 147 additions & 27 deletions fern/voice-fallback-plan.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,62 +1,182 @@
---
title: Voice Fallback Plan
title: Voice fallback configuration
subtitle: Configure fallback voices that activate automatically if your primary voice fails.
slug: voice-fallback-plan
---

<Note>
Voice fallback plans can currently only be configured through the API. We are working on making this available through our dashboard.
</Note>

## Introduction
## Overview

Voice fallback plans give you the ability to continue your call in the event that your primary voice fails. Your assistant will sequentially fallback to only the voices you configure within your plan, in the exact order you specify.
Voice fallback configuration gives you the ability to continue your call in the event that your primary voice fails. Your assistant will sequentially fallback to only the voices you configure within your plan, in the exact order you specify.

<Note>
Without a fallback plan configured, your call will end with an error in the event that your chosen voice provider fails.
</Note>

## How It Works
## How it works

When a voice failure occurs, Vapi will:
1. Detect the failure of the primary voice
2. If a custom fallback plan exists:
- Switch to the first fallback voice in your plan
- Continue through your specified list if subsequent failures occur
- Terminate only if all voices in your plan have failed
- Switch to the first fallback voice in your plan
- Continue through your specified list if subsequent failures occur
- Terminate only if all voices in your plan have failed

## Configuration
## Configure via Dashboard

<Steps>
<Step title="Open Voice tab">
Navigate to your assistant and select the **Voice** tab.
</Step>
<Step title="Expand Fallback Voices section">
Scroll down to find the **Fallback Voices** collapsible section. A warning indicator appears if no fallback voices are configured.
</Step>
<Step title="Add a fallback voice">
Click **Add Fallback Voice** to configure your first fallback:
- Select a **provider** from the dropdown (supports 20+ voice providers)
- Choose a **voice** from the searchable popover (shows gender, language, and deprecated status)
- The **model** is automatically selected based on your voice choice
</Step>
<Step title="Configure provider-specific settings (optional)">
Expand **Additional Configuration** to access provider-specific settings like stability, speed, and emotion controls.
</Step>
<Step title="Add more fallbacks">
Repeat to add additional fallback voices. Order matters—the first fallback in your list is tried first.
</Step>
</Steps>

## Configure via API

Add the `fallbackPlan` property to your assistant's voice configuration, and specify the fallback voices within the `voices` property.
- Please note that fallback voices must be valid JSON configurations, and not strings.
- The order matters. Vapi will choose fallback voices starting from the beginning of the list.

<Note>
Fallback voices must be valid JSON configurations, not strings. The order matters—Vapi will choose fallback voices starting from the beginning of the list.
</Note>

```json
{
"voice": {
"provider": "openai",
"voiceId": "shimmer",
"fallbackPlan": {
"voices": [
{
"provider": "cartesia",
"voiceId": "248be419-c632-4f23-adf1-5324ed7dbf1d"
},
{
"provider": "11labs",
"voiceId": "cgSgspJ2msm6clMCkdW9"
}
]
"voices": [
{
"provider": "cartesia",
"voiceId": "248be419-c632-4f23-adf1-5324ed7dbf1d"
},
{
"provider": "11labs",
"voiceId": "cgSgspJ2msm6clMCkdW9",
"stability": 0.5,
"similarityBoost": 0.75
}
]
}
}
}
```

## Provider-specific settings

Each voice provider supports different configuration options. Expand the accordion below to see available settings for each provider.

<AccordionGroup>
<Accordion title="ElevenLabs">
- **stability** (0-1): Controls voice consistency. Lower values allow more emotional range; higher values produce more stable output.
- **similarityBoost** (0-1): Enhances similarity to the original voice. Higher values make the voice more similar to the reference.
- **style** (0-1): Voice style intensity. Higher values amplify the speaker's style.
- **useSpeakerBoost** (boolean): Enable to boost similarity to the original speaker.
- **speed** (0.7-1.2): Speech speed multiplier. Default is 1.0.
- **optimizeStreamingLatency** (0-4): Controls streaming latency optimization. Default is 3.
- **enableSsmlParsing** (boolean): Enable SSML pronunciation support.
- **model**: Select from `eleven_multilingual_v2`, `eleven_turbo_v2`, `eleven_turbo_v2_5`, `eleven_flash_v2`, `eleven_flash_v2_5`, or `eleven_monolingual_v1`.
</Accordion>
<Accordion title="Cartesia">
- **model**: Model selection (`sonic-english`, `sonic-3`, etc.).
- **language**: Language code for the voice.
- **experimentalControls.speed**: Speech speed adjustment (-1 to 1). Negative values slow down; positive values speed up.
- **experimentalControls.emotion**: Array of emotion configurations (e.g., `["happiness:high", "curiosity:medium"]`).
- **generationConfig** (sonic-3 only):
- **speed** (0.6-1.5): Fine-grained speed control.
- **volume** (0.5-2.0): Volume adjustment.
- **experimental.accentLocalization** (0 or 1): Toggle accent localization.
</Accordion>
<Accordion title="Azure">
- **speed** (0.5-2): Speech rate multiplier. Default is 1.0.
</Accordion>
<Accordion title="OpenAI">
- **speed** (0.25-4): Speech speed multiplier. Default is 1.0.
- **model**: Select from `tts-1`, `tts-1-hd`, or realtime models.
- **instructions**: Voice prompt to control the generated audio style. Does not work with `tts-1` or `tts-1-hd` models.
</Accordion>
<Accordion title="LMNT">
- **speed** (0.25-2): Speech rate multiplier. Default is 1.0.
- **language**: Two-letter ISO 639-1 language code, or `auto` for auto-detection.
</Accordion>
<Accordion title="Rime AI">
- **model**: Select from `arcana`, `mistv2`, or `mist`. Defaults to `arcana`.
- **speed** (0.1+): Speech speed multiplier.
- **pauseBetweenBrackets** (boolean): Enable pause control using angle brackets (e.g., `<200>` for 200ms pause).
- **phonemizeBetweenBrackets** (boolean): Enable phonemization using curly brackets (e.g., `{h'El.o}`).
- **reduceLatency** (boolean): Optimize for reduced streaming latency.
- **inlineSpeedAlpha**: Inline speed control using alpha notation.
</Accordion>
<Accordion title="PlayHT">
- **speed** (0.1-5): Speech rate multiplier.
- **temperature** (0.1-2): Controls voice variance. Lower values are more predictable; higher values allow more variation.
- **emotion**: Emotion preset (e.g., `female_happy`, `male_sad`, `female_angry`, `male_surprised`).
- **voiceGuidance** (1-6): Controls voice uniqueness. Lower values reduce uniqueness.
- **styleGuidance** (1-30): Controls emotion intensity. Higher values create more emotional performance.
- **textGuidance** (1-2): Controls text adherence. Higher values are more accurate to input text.
- **model**: Select from `PlayHT2.0`, `PlayHT2.0-turbo`, `Play3.0-mini`, or `PlayDialog`.
</Accordion>
<Accordion title="Deepgram">
- **model**: Select from `aura` or `aura-2`. Defaults to `aura-2`.
- **mipOptOut** (boolean): Opt out of the Deepgram Model Improvement Partnership program.
</Accordion>
<Accordion title="Hume">
- **model**: Model selection (e.g., `octave2`).
- **description**: Natural language instructions describing how the speech should sound (tone, intonation, pacing, accent).
- **isCustomHumeVoice** (boolean): Indicates whether using a custom Hume voice.
</Accordion>
<Accordion title="Minimax">
- **model**: Select from `speech-02-hd` (high-fidelity) or `speech-02-turbo` (low latency). Defaults to `speech-02-turbo`.
- **emotion**: Emotion preset (`happy`, `sad`, `angry`, `fearful`, `surprised`, `disgusted`, `neutral`).
- **pitch** (-12 to 12): Voice pitch adjustment in semitones.
- **speed** (0.5-2): Speech speed adjustment.
- **volume** (0.5-2): Volume adjustment.
</Accordion>
<Accordion title="WellSaid">
- **model**: Model selection.
- **enableSsml** (boolean): Enable limited SSML translation for input text.
- **libraryIds**: Array of library IDs to use for voice synthesis.
</Accordion>
<Accordion title="Neuphonic">
- **model**: Model selection (e.g., `neu_fast`).
- **language**: Language code (required).
- **speed** (0.25-2): Speech speed multiplier.
</Accordion>
<Accordion title="SmallestAI">
- **model**: Model selection (e.g., `lightning`).
- **speed**: Speech speed multiplier.
</Accordion>
</AccordionGroup>

## Best practices

- Use <b>different providers</b> for your fallback voices to protect against provider-wide outages.
- Use **different providers** for your fallback voices to protect against provider-wide outages.
- Select voices with **similar characteristics** (tone, accent, gender) to maintain consistency in the user experience.
- Test your fallback configuration to ensure smooth transitions between voices.

## How will pricing work?
## FAQ

There is no change to the pricing of the voices. Your call will not incur any extra fees while using fallback voices, and you will be able to see the cost for each voice in your end-of-call report.
<AccordionGroup>
<Accordion title="How will pricing work?">
There is no change to the pricing of the voices. Your call will not incur any extra fees while using fallback voices, and you will be able to see the cost for each voice in your end-of-call report.
</Accordion>
<Accordion title="How many fallback voices can I configure?">
You can configure as many fallback voices as you need. However, we recommend 2-3 fallbacks from different providers for optimal reliability.
</Accordion>
<Accordion title="Will users notice when a fallback is activated?">
Users may notice a brief pause and a change in voice characteristics when switching to a fallback voice. Selecting voices with similar properties helps minimize this disruption.
</Accordion>
</AccordionGroup>
Loading