Fish Audio Drama 3: Direct AI Voice Performances

TL;DR

Fish Audio Drama 3 is a new generation AI text to speech model designed for directed voice performance rather than simple speech generation. Creators can write ordinary performance directions beside a script, insert pauses and sounds, emphasize selected words, switch languages mid line, and produce multi speaker scenes in one pass. It is built for audiobooks, audio dramas, games, animation, dubbing, interactive media, and short form video. Its most important advance is creative control: teams can revise a performance in plain language, regenerate only a selected segment through audio inpainting, and maintain character consistency across long scripts.

For content teams, Drama 3 changes the production model. Instead of generating a voiceover and manually editing it, writers, directors, and developers can treat the script as a performance brief. The result is a faster path from written dialogue to emotionally credible audio, especially for serialized, multilingual, and character led content.

ELI5 Introduction

Imagine you ask someone to read a sentence aloud. A basic text to speech tool says the words, but it does not really know how you want them said.

Drama 3 is different. You can tell it:

• “Say this nervously.”
• “Pause here.”
• “Add a shaky breath.”
• “Whisper only these final words.”
• “Start calm, then become panicked.”
• “Have three characters speak in one scene.”
• “Say this phrase in French, then return to English.”

In simple terms, Drama 3 lets you direct an AI voice like a film director directs an actor. You do not need to learn complicated audio tags first. You write instructions in normal language, place them where they belong in the script, and the model performs the line accordingly.

This matters because voice is not only pronunciation. A line such as “I’m fine” can sound calm, sarcastic, heartbroken, frightened, or defiant. Drama 3 is designed to understand that difference and turn written intent into vocal performance.

Detailed Analysis

What Is Fish Audio Drama 3?

Fish Audio Drama 3 is Fish Audio’s latest text to speech model for voice direction and content creation. It allows creators to describe how each line should be performed, add sounds and pauses, and limit a delivery change to specific words.

The model is positioned as a step beyond conventional AI voiceover. Rather than asking users to choose from a fixed list of emotions, Drama 3 accepts natural language directions such as “barely holding it together, forcing a smile.” It also supports paired tags such as <whisper>, <emphasis>, <soft>, <fast>, <slow>, <stress>, and <forceful> to alter selected words only.

Drama 3 is available through Fish Audio’s web based Text to Speech experience, mobile apps, and API. Developers can use it by setting the model header to drama-3 in an existing POST /v1/tts request.

Why Directed Voice Performance Matters

Traditional text to speech answers one question: “How do I turn this text into speech?”

Directed voice performance answers a more valuable question: “How should this line sound in this story, for this character, at this moment?”

That distinction is central to modern audio content. An audiobook narrator must sustain tension across chapters. A game character must react convincingly to changing circumstances. A brand video may need warmth in one sentence and urgency in the next. A dubbed scene must preserve emotion while moving between languages.

Drama 3 addresses these needs by making performance instructions part of the script itself. A creator can write:

The direction is readable, editable, and reusable. This makes the workflow closer to screenwriting, voice directing, and editorial production than to traditional speech synthesis.

Natural Language Voice Direction

The core innovation is natural language control. Users describe tone, pacing, emotion, intent, and character relationships in their own words. Directions can be written in English or another language, and they do not need to match the language of the script.

For example, a creator can write a direction in Chinese while the spoken line remains in English:

This is valuable for global production teams. A director in one market can brief a performance in the language they think in, while the final audio is produced for another audience.

It also lowers the barrier for non technical creators. Instead of learning a specialized markup language, a writer can use the same vocabulary they would use with a human voice actor.

Word Level Control

Drama 3 supports scoped delivery changes. A creator can select only the words that need to change, rather than regenerating an entire line with a different global setting.

Supported paired tags include:

• <whisper>
• <emphasis>
• <soft>
• <fast>
• <slow>
• <stress>
• <forceful>

For example:

This allows a single sentence to carry a shift in emotional intensity. The first part can sound composed, while the final phrase becomes private, fragile, or urgent.

For storytellers, this is a major practical advantage. Emotional turns often happen inside a sentence, not between sentences. Word level control makes those turns easier to express without splitting a script into many separate audio clips.

Sounds, Pauses, and Performance Texture

Drama 3 lets creators place sounds and pauses exactly where they should occur. Examples include a sigh, a laugh, a breath, or a short pause.

A line can therefore become:

This matters because realistic speech includes more than words. Hesitation, breathing, laughter, and silence communicate emotional state. By allowing creators to place these elements in the script, Drama 3 gives them a more complete performance toolkit.

The model also includes an Auto Tag feature that can suggest directions for a script. Creators can keep the suggestions that work and revise the rest, which provides a useful starting point for teams that need to produce audio at scale.

Multi Speaker Dialogue in One Pass

Drama 3 supports multi speaker dialogue. Creators can generate an entire scene in one request, assign a voice to each speaker, and use speaker tags to distinguish turns.

The documentation shows a scene with a narrator, captain, and engineer. Each character receives a separate voice ID, while speaker tags such as <|speaker:0|>, <|speaker:1|>, and <|speaker:2|> organize the dialogue.

This is especially useful for:

• Audio dramas
• Game dialogue
• Animation
• Interactive fiction
• Branded storytelling
• Educational roleplay
• Podcast style fiction
• Multilingual dubbing

Instead of generating each character separately and stitching clips together, teams can produce a scene as one coordinated performance.

Long Form and Multilingual Consistency

Drama 3 is designed for long scripts and multilingual content. Fish Audio states that the model can maintain the same voice and character across long scripts, switch languages mid scene, and control pronunciation at the phoneme level in languages including English, Chinese, Japanese, and Korean.

The model supports phoneme tags for pronunciation control. For English, it uses CMU Arpabet; for Chinese, pinyin; and for Japanese, romaji. This helps creators pin down names, technical terms, foreign words, and brand names that a model might otherwise pronounce inconsistently.

Multilingual capability is particularly important for global media companies. A character can speak English in one phrase and switch to French, Spanish, German, Japanese, Chinese, Korean, or Italian in the next. Fish Audio notes that language switching is experimental and accuracy varies by language and voice, so teams should test their specific language pairs before production.

Audio Inpainting and Targeted Revision

One of Drama 3’s most commercially important capabilities is audio inpainting. Creators can select part of a performance, update the instructions, and regenerate only that segment while keeping the surrounding audio intact.

This changes the economics of revision. In a conventional workflow, changing one word may require regenerating an entire paragraph, adjusting timing, and reconciling the edit with neighboring audio. With targeted inpainting, teams can correct a mispronounced name, soften one phrase, or change the emotion of a short passage without rebuilding the whole scene.

For serialized content, this is especially valuable. A publisher can update a chapter, localize a phrase, fix a factual correction, or revise a character’s delivery while preserving the rest of the production.

Multi Turn Director Revision

Drama 3 also supports multi turn revision. Instead of starting from scratch, a creator can give feedback on the previous take, such as “gentler, and a bit sadder,” and the model regenerates the performance with that context.

This makes the model behave more like a collaborative voice performer than a one shot generator. It supports an iterative creative process:

1. Write the script.
2. Generate a first take.
3. Listen for emotional accuracy.
4. Give a plain language note.
5. Regenerate the revised take.
6. Retain the version that best serves the story.

This workflow should feel familiar to creative directors, podcast producers, game studios, and localization teams.

Best Practices and Case Examples

Audiobook Production

An audiobook publisher can use Drama 3 to produce a fiction title with a consistent narrator, distinct character voices, and controlled emotional pacing.

For example, a suspense novel could include:

The publisher can revise a single frightened line without rebuilding the chapter, then use phoneme tags to ensure place names and character names are pronounced correctly.

Game and Animation Dialogue

A game studio can use multi speaker generation to create NPC dialogue, cutscenes, and interactive reactions. Because Drama 3 supports multi character scenes and streaming for long text, it is suited to dialogue systems where characters must respond in context.

A practical use case is a branching narrative. The studio can maintain one voice per character, revise individual reactions through inpainting, and update localized versions without recreating every audio asset from scratch.

Audio Drama and Interactive Media

Audio drama teams can use Drama 3 as a production layer between scriptwriting and final mix. Writers can annotate emotion, sound, and pacing directly in the script. Directors can iterate on performances using plain language notes. Sound designers can then layer music, ambience, and effects around the generated dialogue.

This creates a more efficient division of labor: AI handles directed speech generation, while humans retain responsibility for story judgment, casting, final quality, and creative approval.

Brand and Educational Content

Brands can use Drama 3 for product explainers, onboarding audio, training modules, and multilingual campaigns. The key is to use natural language direction to preserve brand personality.

For example:

For education, a tutor character can switch languages mid line, making the model useful for language learning and bilingual instruction. Fish Audio specifically highlights this type of mixed language use case, while advising teams to test accuracy by language and voice.

Governance, Quality, and Responsible Use

AI voice technology creates real commercial value, but it also requires clear governance. Teams should establish rules before scaling production.

• Obtain proper consent and licensing for any cloned voice.
• Maintain records of voice permissions and approved use cases.
• Avoid impersonation of real people without explicit authorization.
• Use human review for news, health, financial, legal, and political content.
• Label synthetic voice where disclosure is legally or ethically appropriate.
• Protect source scripts, voice references, and generated audio with access controls.
• Test pronunciation and cultural appropriateness in every target market.

Drama 3’s voice cloning and multi speaker capabilities make these safeguards especially important. The technology enables compelling performance, but responsible deployment depends on clear ownership, consent, and editorial oversight.

Actionable Next Steps

1. Pilot Drama 3 on one contained project. Choose a short audio drama, product explainer, game scene, or audiobook chapter with clear creative requirements.
2. Create a direction library. Document effective prompts for your brand voice, character archetypes, and common emotional states.
3. Assign clear roles. Let writers own dialogue, creative leads own performance direction, and audio editors own final quality control.
4. Test multilingual pairs early. Validate pronunciation, accent quality, and language switching for every market before committing to a localized production schedule.
5. Use inpainting for iteration. Reserve full regeneration for structural script changes and use targeted editing for pronunciation, pacing, and emotional corrections.
6. Measure what matters. Track revision cycles, time to approved audio, listener completion, comprehension, brand fit, and localization quality.
7. Build a governance checklist. Cover voice consent, licensing, disclosure, data security, and human approval before scaling.