Eve library

Audio Tag Best Practices Guide

Guide

Revision 1

On this page

This guide covers best practices for using audio tags with ElevenLabs Eleven v3 and the platform's AudioTagProcessor.

Overview#

Audio tags control voice synthesis characteristics including:

  • Emotional expression
  • Physical reactions
  • Cognitive states
  • Tone and delivery
  • Accent variations
  • Sound effects

Tag Categories#

Emotional Tags#

Control the emotional quality of speech:

Tag Use Case Best For
[calm] Peaceful content Meditation, sleep
[excited] High-energy content Motivational, kids
[happy] Positive content Affirmations, celebration
[sad] Somber content Grief support, reflection
[sorrowful] Deep sadness Loss, mourning
[nervous] Anxious expression Character dialogue
[frustrated] Tension Problem scenarios
[angry] Intense emotion Dramatic content
[fearful] Fear expression Dramatic tension
[surprised] Unexpected moments Revelation, discovery
typescript
// Good: Match emotion to content
"[calm] Allow yourself to settle into this moment of stillness."

// Good: Transition between emotions
"[nervous] The path ahead seemed uncertain... [calm] But then, a peace washed over me."

Reaction Tags#

Add physical/vocal reactions:

Tag Effect When to Use
[sigh] Audible exhale Relief, relaxation
[laughs] Laughter Humor, joy
[gasps] Sharp intake Surprise, revelation
[whispers] Quiet voice Intimacy, secrets
[gulps] Swallowing sound Nervousness
[coughs] Cough sound Clearing, interruption
[clears throat] Throat clear Preparation to speak
[sniffs] Sniffing Emotion, detection
typescript
// Good: Natural placement
"[sigh] After a long day, you finally have this moment to yourself."

// Good: Before emotional content
"[whispers] This is just between us..."

Cognitive Tags#

Express thought processes:

Tag Effect Best For
[pauses] Brief pause Thoughtful moments
[hesitates] Uncertainty Decision points
[stammers] Speech disruption Nervousness
[resigned tone] Acceptance Letting go
[thoughtful] Contemplative Reflection
[uncertain] Doubt Questions
typescript
// Good: Before important statements
"[pauses] And now, we come to the heart of the practice."

// Good: Showing internal process
"[thoughtful] Consider what this means for you..."

Tone Tags#

Set the overall voice quality:

Tag Quality Content Type
[cheerfully] Bright, upbeat Kids, celebration
[flatly] Emotionless Contrast, deadpan
[deadpan] No expression Comedy timing
[playfully] Light, fun Kids, games
[sarcastically] Ironic Commentary
[seriously] Grave Important topics
[warmly] Friendly Welcome, comfort
[coldly] Distant Character work
typescript
// Good: Set tone at start
"[warmly] Welcome to your evening meditation practice."

// Good: Tone shift for emphasis
"[warmly] We've had a beautiful journey today. [seriously] But remember..."

Delivery Tags#

Control speech delivery:

Tag Effect Use Case
[pause] Silent pause Between thoughts
[rushed] Fast speech Urgency
[drawn out] Elongated Emphasis
[deliberate] Measured Instructions
[whispered] Very quiet Intimate moments
[shouted] Loud speech Urgency (use sparingly)
typescript
// Good: Meditation pacing
"[deliberate] Breathe... in... [pause] and... out..."

// Good: Creating atmosphere
"[whispered] In the quiet of this moment..."

Accent Tags#

Add regional accent qualities:

Tag Accent
[American accent] Standard American
[British accent] British English
[Australian accent] Australian
[Indian accent] Indian English
[Irish accent] Irish
[Scottish accent] Scottish
typescript
// Use consistently for character
"[British accent] Good evening. [warmly] Welcome to tonight's meditation."

Content Type Configurations#

Sleep Content#

typescript
const sleepConfig = {
  preferredTags: ['calm', 'whispers', 'pauses', 'sigh', 'warmly', 'deliberate'],
  avoidTags: ['excited', 'shouted', 'rushed', 'angry', 'thunder'],
  stabilityMode: 'natural',
  tagDensity: 'low'
};

// Example
"[calm] [whispers] Allow your body to sink deeper into rest. [pauses]
Each breath carries you further into peaceful sleep. [sigh]"

Meditation Content#

typescript
const meditationConfig = {
  preferredTags: ['calm', 'pauses', 'deliberate', 'warmly', 'thoughtful'],
  avoidTags: ['rushed', 'shouted', 'angry', 'frustrated'],
  stabilityMode: 'natural',
  tagDensity: 'medium'
};

// Example
"[calm] [warmly] Welcome to this moment of stillness. [pauses]
[thoughtful] Notice your breath, naturally flowing. [deliberate] In... and out..."

Energizing Content#

typescript
const energizingConfig = {
  preferredTags: ['excited', 'cheerfully', 'playfully', 'happy'],
  avoidTags: ['calm', 'whispers', 'sorrowful', 'sad'],
  stabilityMode: 'creative',
  tagDensity: 'high'
};

// Example
"[excited] [cheerfully] Good morning! [playfully] Ready to start your day
with energy and purpose? [happy] Let's go!"

Kids Content#

typescript
const kidsConfig = {
  preferredTags: ['cheerfully', 'playfully', 'excited', 'happy'],
  avoidTags: ['sorrowful', 'angry', 'fearful', 'whispers'],
  stabilityMode: 'robust',
  tagDensity: 'medium'
};

// Example
"[cheerfully] Hi there, little explorer! [playfully] Are you ready
for a magical adventure? [excited] Let's find out what's waiting for us!"

Healing Content#

typescript
const healingConfig = {
  preferredTags: ['calm', 'warmly', 'pauses', 'sigh', 'thoughtful'],
  avoidTags: ['rushed', 'shouted', 'angry', 'coldly'],
  stabilityMode: 'natural',
  tagDensity: 'medium'
};

// Example
"[warmly] You are safe here. [calm] [pauses] Allow healing energy to flow
through you. [sigh] [thoughtful] With each breath, you are becoming whole again."

Tag Placement Rules#

Rule 1: Start with Emotional Context#

Always establish emotional tone at the beginning:

typescript
// Good: Set context immediately
"[calm] Welcome to your meditation practice."

// Avoid: Starting without context
"Welcome to your meditation practice. [calm]"

Rule 2: Minimum Gap Between Tags#

Maintain at least 10 characters between tags:

typescript
// Good: Adequate spacing
"[calm] Allow yourself to settle. [pauses] Notice your breath."

// Avoid: Tags too close
"[calm] [pauses] Notice your breath."

Rule 3: Avoid Incompatible Adjacent Tags#

Don't place conflicting tags together:

typescript
// Incompatible pairs to avoid:
// - [excited] + [calm]
// - [whispers] + [shouted]
// - [rushed] + [deliberate]
// - [happy] + [sad]
// - [cheerfully] + [flatly]
// - [warmly] + [coldly]

// Bad: Conflicting emotions
"[excited] [calm] Take a moment to rest."

// Good: Choose one appropriate tag
"[calm] Take a moment to rest."

Rule 4: Place Reactions Before Content#

Reactions should precede the affected text:

typescript
// Good: Reaction first
"[sigh] After all that work, you can finally relax."

// Less effective: Reaction after
"After all that work, you can finally relax. [sigh]"

Rule 5: Use Cognitive Tags at Transition Points#

Place thinking tags at natural pauses:

typescript
// Good: At section transitions
"...and that completes the body scan. [pauses] [thoughtful] Now let us
explore the breath."

// Good: Before reflective questions
"[thoughtful] What has arisen for you in this practice?"

Using the AudioTagProcessor#

Basic Usage#

typescript
import { AudioTagProcessor } from './audio-tags/audio-tag-processor';

const processor = new AudioTagProcessor();

// Parse existing tags
const tags = processor.parseTags("[calm] Welcome to meditation.");
// Returns: [{ tag: 'calm', category: 'emotional', position: 0, length: 6 }]

// Validate tags
const validation = processor.validateTags(text);
if (!validation.valid) {
  console.log('Errors:', validation.errors);
  console.log('Warnings:', validation.warnings);
}

Processing for Content Type#

typescript
// Automatically add appropriate tags for content type
const result = processor.processForContentType(
  "Welcome to your evening practice.",
  'sleep'
);

console.log(result.processedText);
// "[calm] Welcome to your evening practice."

console.log(result.stabilityMode);
// "natural"

Emotional Context Detection#

typescript
// Auto-detect emotions from text content
const result = processor.autoInjectEmotionalTags(
  "Find peace and tranquility in this serene moment."
);

console.log(result.processedText);
// "[calm] Find peace and tranquility in this serene moment."

console.log(result.tagsAdded);
// [{ tag: 'calm', category: 'emotional', ... }]

Custom Templates#

typescript
// Create a custom audio tag template
const template = processor.createTemplate(
  'meditation-opening',
  'Meditation Opening',
  'meditation',
  ['calm', 'warmly', 'pauses'],
  [
    { tag: 'calm', placement: 'start', probability: 1.0 },
    { tag: 'pauses', placement: 'after_pause', probability: 0.5 }
  ]
);

// Apply template to text
const result = processor.applyTemplate(
  "Welcome. This is your time.",
  template
);

Dialogue Formatting#

typescript
// Format multi-speaker dialogue
const lines = [
  { speaker: 'Teacher', text: 'Welcome to class.', emotion: 'warmly' },
  { speaker: 'Student', text: 'Thank you, teacher.', emotion: 'calm' }
];

const speakers = [
  { name: 'Teacher', voiceId: 'voice-1', defaultTone: 'warmly' },
  { name: 'Student', voiceId: 'voice-2', defaultTone: 'calm' }
];

const formatted = processor.formatDialogue(lines, speakers);
// "Teacher: [warmly] Welcome to class.\n\nStudent: [calm] Thank you, teacher."

Stripping Tags#

typescript
// Remove all audio tags
const stripped = processor.stripTags("[calm] Welcome to [pauses] meditation.");
// "Welcome to meditation."

Stability Modes#

Control expressiveness vs. consistency:

Mode Description Best For
creative Maximum expressiveness Emotional content, storytelling
natural Balanced General use, meditation
robust Maximum consistency Kids content, predictable output
typescript
// Get recommended mode for content type
const mode = processor.getStabilityMode('meditation');
// Returns: 'natural'

// Get mode description
const desc = processor.getStabilityModeDescription('natural');
// "Balanced expressiveness and consistency. Good for general use."

Tag Density Guidelines#

Low Density (Sleep, Relaxation)#

1-2 tags per paragraph:

typescript
"[calm] Allow your body to sink into the bed. Feel the support beneath you.
Your breath naturally slows. [pauses] Each exhale releases a little more tension."

Medium Density (Meditation, Healing)#

2-4 tags per paragraph:

typescript
"[calm] [warmly] Welcome to this practice. [pauses] Notice your breath,
naturally flowing. [thoughtful] What sensations arise as you settle?
[deliberate] Bring your attention... here."

High Density (Energizing, Kids)#

4-6 tags per paragraph:

typescript
"[excited] [cheerfully] Good morning! [playfully] Ready to wake up your body?
[happy] Let's stretch! [laughs] Reach up high! [excited] And back down!"

Common Mistakes#

1. Overusing Tags#

typescript
// Bad: Too many tags
"[calm] [warmly] [pauses] [thoughtful] [deliberate] Welcome."

// Good: Selective, meaningful tags
"[calm] [warmly] Welcome to your practice. [pauses]"

2. Inappropriate Tag Combinations#

typescript
// Bad: Conflicting emotions
"[excited] [whispers] Relax now."

// Good: Coherent emotional state
"[calm] [whispers] Relax now."

3. Tags Breaking Natural Speech#

typescript
// Bad: Interrupting flow
"Take a deep [pauses] breath."

// Good: Tags between phrases
"Take a deep breath. [pauses] Feel it filling your lungs."

4. Ignoring Content Context#

typescript
// Bad: Energizing tags in sleep content
"[excited] Time to fall asleep!"

// Good: Appropriate tags for context
"[calm] [whispers] Time to drift into sleep."

5. Using Unknown Tags#

typescript
// Bad: Non-standard tag (will generate warning)
"[sleepy] Time to rest."

// Good: Use supported tags
"[calm] [sigh] Time to rest."

Validation Best Practices#

Always Validate Before Synthesis#

typescript
const validation = processor.validateTags(text);

if (!validation.valid) {
  throw new Error(`Invalid tags: ${validation.errors.join(', ')}`);
}

if (validation.warnings.length > 0) {
  console.warn('Tag warnings:', validation.warnings);
}

Handle Unknown Tags#

typescript
const tags = processor.parseTags(text);
const unknownTags = tags.filter(t => t.category === 'unknown');

if (unknownTags.length > 0) {
  console.warn('Unknown tags detected:', unknownTags.map(t => t.tag));
  // Consider stripping or replacing
}

Check Tag Spacing#

typescript
const tags = processor.parseTags(text);

for (let i = 0; i < tags.length - 1; i++) {
  const gap = tags[i + 1].position - (tags[i].position + tags[i].length);
  if (gap < 10) {
    console.warn(`Tags too close: [${tags[i].tag}] and [${tags[i + 1].tag}]`);
  }
}

Integration with TTS#

Complete Synthesis Flow#

typescript
import { AudioTagProcessor } from './audio-tags/audio-tag-processor';
import { ElevenLabsProvider } from './providers/elevenlabs-provider';

async function synthesizeWithTags(text: string, contentType: string) {
  const processor = new AudioTagProcessor();
  const tts = new ElevenLabsProvider({ apiKey: process.env.ELEVENLABS_KEY });

  // Process text for content type
  const processed = processor.processForContentType(text, contentType);

  // Validate tags
  const validation = processor.validateTags(processed.processedText);
  if (!validation.valid) {
    throw new Error(`Tag validation failed: ${validation.errors}`);
  }

  // Synthesize with appropriate settings
  const audio = await tts.synthesize({
    text: processed.processedText,
    voice_id: 'selected-voice-id',
    model_id: 'eleven_v3',  // Required for audio tags
    voice_settings: {
      stability: processed.stabilityMode === 'robust' ? 0.7 : 0.5,
      similarity_boost: 0.75,
      style: processed.stabilityMode === 'creative' ? 0.8 : 0.5
    }
  });

  return audio;
}

Model Compatibility#

Audio tags are fully supported in:

  • eleven_v3 - Full support
  • eleven_multilingual_v2 - Full support
  • eleven_turbo_v2_5 - Limited support
  • eleven_flash_v2_5 - Limited support

Always use eleven_v3 for maximum tag expressiveness.