This guide covers best practices for using audio tags with ElevenLabs Eleven v3 and the platform's AudioTagProcessor.
Overview#
Audio tags control voice synthesis characteristics including:
- Emotional expression
- Physical reactions
- Cognitive states
- Tone and delivery
- Accent variations
- Sound effects
Tag Categories#
Emotional Tags#
Control the emotional quality of speech:
| Tag | Use Case | Best For |
|---|---|---|
[calm] |
Peaceful content | Meditation, sleep |
[excited] |
High-energy content | Motivational, kids |
[happy] |
Positive content | Affirmations, celebration |
[sad] |
Somber content | Grief support, reflection |
[sorrowful] |
Deep sadness | Loss, mourning |
[nervous] |
Anxious expression | Character dialogue |
[frustrated] |
Tension | Problem scenarios |
[angry] |
Intense emotion | Dramatic content |
[fearful] |
Fear expression | Dramatic tension |
[surprised] |
Unexpected moments | Revelation, discovery |
// Good: Match emotion to content
"[calm] Allow yourself to settle into this moment of stillness."
// Good: Transition between emotions
"[nervous] The path ahead seemed uncertain... [calm] But then, a peace washed over me."
Reaction Tags#
Add physical/vocal reactions:
| Tag | Effect | When to Use |
|---|---|---|
[sigh] |
Audible exhale | Relief, relaxation |
[laughs] |
Laughter | Humor, joy |
[gasps] |
Sharp intake | Surprise, revelation |
[whispers] |
Quiet voice | Intimacy, secrets |
[gulps] |
Swallowing sound | Nervousness |
[coughs] |
Cough sound | Clearing, interruption |
[clears throat] |
Throat clear | Preparation to speak |
[sniffs] |
Sniffing | Emotion, detection |
// Good: Natural placement
"[sigh] After a long day, you finally have this moment to yourself."
// Good: Before emotional content
"[whispers] This is just between us..."
Cognitive Tags#
Express thought processes:
| Tag | Effect | Best For |
|---|---|---|
[pauses] |
Brief pause | Thoughtful moments |
[hesitates] |
Uncertainty | Decision points |
[stammers] |
Speech disruption | Nervousness |
[resigned tone] |
Acceptance | Letting go |
[thoughtful] |
Contemplative | Reflection |
[uncertain] |
Doubt | Questions |
// Good: Before important statements
"[pauses] And now, we come to the heart of the practice."
// Good: Showing internal process
"[thoughtful] Consider what this means for you..."
Tone Tags#
Set the overall voice quality:
| Tag | Quality | Content Type |
|---|---|---|
[cheerfully] |
Bright, upbeat | Kids, celebration |
[flatly] |
Emotionless | Contrast, deadpan |
[deadpan] |
No expression | Comedy timing |
[playfully] |
Light, fun | Kids, games |
[sarcastically] |
Ironic | Commentary |
[seriously] |
Grave | Important topics |
[warmly] |
Friendly | Welcome, comfort |
[coldly] |
Distant | Character work |
// Good: Set tone at start
"[warmly] Welcome to your evening meditation practice."
// Good: Tone shift for emphasis
"[warmly] We've had a beautiful journey today. [seriously] But remember..."
Delivery Tags#
Control speech delivery:
| Tag | Effect | Use Case |
|---|---|---|
[pause] |
Silent pause | Between thoughts |
[rushed] |
Fast speech | Urgency |
[drawn out] |
Elongated | Emphasis |
[deliberate] |
Measured | Instructions |
[whispered] |
Very quiet | Intimate moments |
[shouted] |
Loud speech | Urgency (use sparingly) |
// Good: Meditation pacing
"[deliberate] Breathe... in... [pause] and... out..."
// Good: Creating atmosphere
"[whispered] In the quiet of this moment..."
Accent Tags#
Add regional accent qualities:
| Tag | Accent |
|---|---|
[American accent] |
Standard American |
[British accent] |
British English |
[Australian accent] |
Australian |
[Indian accent] |
Indian English |
[Irish accent] |
Irish |
[Scottish accent] |
Scottish |
// Use consistently for character
"[British accent] Good evening. [warmly] Welcome to tonight's meditation."
Content Type Configurations#
Sleep Content#
const sleepConfig = {
preferredTags: ['calm', 'whispers', 'pauses', 'sigh', 'warmly', 'deliberate'],
avoidTags: ['excited', 'shouted', 'rushed', 'angry', 'thunder'],
stabilityMode: 'natural',
tagDensity: 'low'
};
// Example
"[calm] [whispers] Allow your body to sink deeper into rest. [pauses]
Each breath carries you further into peaceful sleep. [sigh]"
Meditation Content#
const meditationConfig = {
preferredTags: ['calm', 'pauses', 'deliberate', 'warmly', 'thoughtful'],
avoidTags: ['rushed', 'shouted', 'angry', 'frustrated'],
stabilityMode: 'natural',
tagDensity: 'medium'
};
// Example
"[calm] [warmly] Welcome to this moment of stillness. [pauses]
[thoughtful] Notice your breath, naturally flowing. [deliberate] In... and out..."
Energizing Content#
const energizingConfig = {
preferredTags: ['excited', 'cheerfully', 'playfully', 'happy'],
avoidTags: ['calm', 'whispers', 'sorrowful', 'sad'],
stabilityMode: 'creative',
tagDensity: 'high'
};
// Example
"[excited] [cheerfully] Good morning! [playfully] Ready to start your day
with energy and purpose? [happy] Let's go!"
Kids Content#
const kidsConfig = {
preferredTags: ['cheerfully', 'playfully', 'excited', 'happy'],
avoidTags: ['sorrowful', 'angry', 'fearful', 'whispers'],
stabilityMode: 'robust',
tagDensity: 'medium'
};
// Example
"[cheerfully] Hi there, little explorer! [playfully] Are you ready
for a magical adventure? [excited] Let's find out what's waiting for us!"
Healing Content#
const healingConfig = {
preferredTags: ['calm', 'warmly', 'pauses', 'sigh', 'thoughtful'],
avoidTags: ['rushed', 'shouted', 'angry', 'coldly'],
stabilityMode: 'natural',
tagDensity: 'medium'
};
// Example
"[warmly] You are safe here. [calm] [pauses] Allow healing energy to flow
through you. [sigh] [thoughtful] With each breath, you are becoming whole again."
Tag Placement Rules#
Rule 1: Start with Emotional Context#
Always establish emotional tone at the beginning:
// Good: Set context immediately
"[calm] Welcome to your meditation practice."
// Avoid: Starting without context
"Welcome to your meditation practice. [calm]"
Rule 2: Minimum Gap Between Tags#
Maintain at least 10 characters between tags:
// Good: Adequate spacing
"[calm] Allow yourself to settle. [pauses] Notice your breath."
// Avoid: Tags too close
"[calm] [pauses] Notice your breath."
Rule 3: Avoid Incompatible Adjacent Tags#
Don't place conflicting tags together:
// Incompatible pairs to avoid:
// - [excited] + [calm]
// - [whispers] + [shouted]
// - [rushed] + [deliberate]
// - [happy] + [sad]
// - [cheerfully] + [flatly]
// - [warmly] + [coldly]
// Bad: Conflicting emotions
"[excited] [calm] Take a moment to rest."
// Good: Choose one appropriate tag
"[calm] Take a moment to rest."
Rule 4: Place Reactions Before Content#
Reactions should precede the affected text:
// Good: Reaction first
"[sigh] After all that work, you can finally relax."
// Less effective: Reaction after
"After all that work, you can finally relax. [sigh]"
Rule 5: Use Cognitive Tags at Transition Points#
Place thinking tags at natural pauses:
// Good: At section transitions
"...and that completes the body scan. [pauses] [thoughtful] Now let us
explore the breath."
// Good: Before reflective questions
"[thoughtful] What has arisen for you in this practice?"
Using the AudioTagProcessor#
Basic Usage#
import { AudioTagProcessor } from './audio-tags/audio-tag-processor';
const processor = new AudioTagProcessor();
// Parse existing tags
const tags = processor.parseTags("[calm] Welcome to meditation.");
// Returns: [{ tag: 'calm', category: 'emotional', position: 0, length: 6 }]
// Validate tags
const validation = processor.validateTags(text);
if (!validation.valid) {
console.log('Errors:', validation.errors);
console.log('Warnings:', validation.warnings);
}
Processing for Content Type#
// Automatically add appropriate tags for content type
const result = processor.processForContentType(
"Welcome to your evening practice.",
'sleep'
);
console.log(result.processedText);
// "[calm] Welcome to your evening practice."
console.log(result.stabilityMode);
// "natural"
Emotional Context Detection#
// Auto-detect emotions from text content
const result = processor.autoInjectEmotionalTags(
"Find peace and tranquility in this serene moment."
);
console.log(result.processedText);
// "[calm] Find peace and tranquility in this serene moment."
console.log(result.tagsAdded);
// [{ tag: 'calm', category: 'emotional', ... }]
Custom Templates#
// Create a custom audio tag template
const template = processor.createTemplate(
'meditation-opening',
'Meditation Opening',
'meditation',
['calm', 'warmly', 'pauses'],
[
{ tag: 'calm', placement: 'start', probability: 1.0 },
{ tag: 'pauses', placement: 'after_pause', probability: 0.5 }
]
);
// Apply template to text
const result = processor.applyTemplate(
"Welcome. This is your time.",
template
);
Dialogue Formatting#
// Format multi-speaker dialogue
const lines = [
{ speaker: 'Teacher', text: 'Welcome to class.', emotion: 'warmly' },
{ speaker: 'Student', text: 'Thank you, teacher.', emotion: 'calm' }
];
const speakers = [
{ name: 'Teacher', voiceId: 'voice-1', defaultTone: 'warmly' },
{ name: 'Student', voiceId: 'voice-2', defaultTone: 'calm' }
];
const formatted = processor.formatDialogue(lines, speakers);
// "Teacher: [warmly] Welcome to class.\n\nStudent: [calm] Thank you, teacher."
Stripping Tags#
// Remove all audio tags
const stripped = processor.stripTags("[calm] Welcome to [pauses] meditation.");
// "Welcome to meditation."
Stability Modes#
Control expressiveness vs. consistency:
| Mode | Description | Best For |
|---|---|---|
creative |
Maximum expressiveness | Emotional content, storytelling |
natural |
Balanced | General use, meditation |
robust |
Maximum consistency | Kids content, predictable output |
// Get recommended mode for content type
const mode = processor.getStabilityMode('meditation');
// Returns: 'natural'
// Get mode description
const desc = processor.getStabilityModeDescription('natural');
// "Balanced expressiveness and consistency. Good for general use."
Tag Density Guidelines#
Low Density (Sleep, Relaxation)#
1-2 tags per paragraph:
"[calm] Allow your body to sink into the bed. Feel the support beneath you.
Your breath naturally slows. [pauses] Each exhale releases a little more tension."
Medium Density (Meditation, Healing)#
2-4 tags per paragraph:
"[calm] [warmly] Welcome to this practice. [pauses] Notice your breath,
naturally flowing. [thoughtful] What sensations arise as you settle?
[deliberate] Bring your attention... here."
High Density (Energizing, Kids)#
4-6 tags per paragraph:
"[excited] [cheerfully] Good morning! [playfully] Ready to wake up your body?
[happy] Let's stretch! [laughs] Reach up high! [excited] And back down!"
Common Mistakes#
1. Overusing Tags#
// Bad: Too many tags
"[calm] [warmly] [pauses] [thoughtful] [deliberate] Welcome."
// Good: Selective, meaningful tags
"[calm] [warmly] Welcome to your practice. [pauses]"
2. Inappropriate Tag Combinations#
// Bad: Conflicting emotions
"[excited] [whispers] Relax now."
// Good: Coherent emotional state
"[calm] [whispers] Relax now."
3. Tags Breaking Natural Speech#
// Bad: Interrupting flow
"Take a deep [pauses] breath."
// Good: Tags between phrases
"Take a deep breath. [pauses] Feel it filling your lungs."
4. Ignoring Content Context#
// Bad: Energizing tags in sleep content
"[excited] Time to fall asleep!"
// Good: Appropriate tags for context
"[calm] [whispers] Time to drift into sleep."
5. Using Unknown Tags#
// Bad: Non-standard tag (will generate warning)
"[sleepy] Time to rest."
// Good: Use supported tags
"[calm] [sigh] Time to rest."
Validation Best Practices#
Always Validate Before Synthesis#
const validation = processor.validateTags(text);
if (!validation.valid) {
throw new Error(`Invalid tags: ${validation.errors.join(', ')}`);
}
if (validation.warnings.length > 0) {
console.warn('Tag warnings:', validation.warnings);
}
Handle Unknown Tags#
const tags = processor.parseTags(text);
const unknownTags = tags.filter(t => t.category === 'unknown');
if (unknownTags.length > 0) {
console.warn('Unknown tags detected:', unknownTags.map(t => t.tag));
// Consider stripping or replacing
}
Check Tag Spacing#
const tags = processor.parseTags(text);
for (let i = 0; i < tags.length - 1; i++) {
const gap = tags[i + 1].position - (tags[i].position + tags[i].length);
if (gap < 10) {
console.warn(`Tags too close: [${tags[i].tag}] and [${tags[i + 1].tag}]`);
}
}
Integration with TTS#
Complete Synthesis Flow#
import { AudioTagProcessor } from './audio-tags/audio-tag-processor';
import { ElevenLabsProvider } from './providers/elevenlabs-provider';
async function synthesizeWithTags(text: string, contentType: string) {
const processor = new AudioTagProcessor();
const tts = new ElevenLabsProvider({ apiKey: process.env.ELEVENLABS_KEY });
// Process text for content type
const processed = processor.processForContentType(text, contentType);
// Validate tags
const validation = processor.validateTags(processed.processedText);
if (!validation.valid) {
throw new Error(`Tag validation failed: ${validation.errors}`);
}
// Synthesize with appropriate settings
const audio = await tts.synthesize({
text: processed.processedText,
voice_id: 'selected-voice-id',
model_id: 'eleven_v3', // Required for audio tags
voice_settings: {
stability: processed.stabilityMode === 'robust' ? 0.7 : 0.5,
similarity_boost: 0.75,
style: processed.stabilityMode === 'creative' ? 0.8 : 0.5
}
});
return audio;
}
Model Compatibility#
Audio tags are fully supported in:
eleven_v3- Full supporteleven_multilingual_v2- Full supporteleven_turbo_v2_5- Limited supporteleven_flash_v2_5- Limited support
Always use eleven_v3 for maximum tag expressiveness.