TL;DR

Runway Act Two is a groundbreaking AI-powered performance transfer tool that brings realistic animation to video creation by transferring human gestures, facial expressions, and movements from a source “driving” video to a reference character image or video. As part of Runway's Gen-3 suite, Act Two leverages temporal diffusion transformers to map nuanced performance, including dialogue and body language, onto any character, empowering filmmakers, content creators, and marketers to animate sequences without expensive motion capture equipment or painstaking keyframing. While ideal for storytelling and creative workflows, Act Two is not a fully automatic story continuation generator; instead, it enables high-fidelity, prompt-directed animation and narrative extension through user-supplied performances and clips.

ELI5 Introduction: What Is Runway Act Two?

Imagine you have a cartoon character or a digital avatar, and you want it to move, speak, and react just like you or an actor in a recorded video. Instead of spending hours making each face or hand move, you show the computer your performance, and it magically animates your character to do exactly what you did—smile, wave, talk, or jump. That’s Runway Act Two: a smart AI tool turning your real-life motion into digital animation quickly and realistically. It’s like the movie magic behind Hollywood blockbusters, now accessible to everyone.

What Is Runway Act Two?

Runway Act Two is the flagship motion capture and performance transfer tool within Runway’s Gen-3 AI video suite. Its core function is to animate any character from an image, drawing, or live-action video by analyzing and recreating the movements, facial expressions, and dialogue of a person in a “driving” video.

Unlike basic animation or video editing tools, Act Two directly leverages real human performances to generate animation that faithfully preserves the style, timing, and emotional nuance of the original. This democratizes high-end animation, making complex, studio-quality character motion achievable without expensive hardware or technical animation skills.

It is not intended as an automatic, multi-scene story generator, but is highly effective for crafting narrative sequences, provided the user supplies the necessary video inputs.

Key Features and Capabilities

High-Fidelity Performance Transfer

• Head, face, and body gesture tracking: Accurately replicates everything from subtle eyebrow raises to full-body dance moves, including hand gestures.
• Speech and lip-sync: Dialogue and mouth movements in the output match the speech in your driving video, producing synced and expressive animation.
• Style and visual consistency: Prompt-based controls allow users to define color palette, lighting, and cinematic effects, but the primary driver is always the performance input.

Input and Workflow

• Driving video: A short clip (usually 10–30 seconds, depending on your plan) of the performance you want to transfer (e.g., an actor speaking, moving, emoting).
• Reference character: An image, illustration, or another video—the “performer” for the output animation.
• Result: The reference character is animated to precisely mimic the driving performance, retaining the visual style and context directed by prompt or setting.

Temporal Consistency and Animation Quality

Powered by temporal diffusion transformers, Act Two preserves consistent movement, pose, and transitions throughout each animated sequence. While scenes are currently limited in duration, the temporal modeling ensures natural, lifelike motion.

Integration with Runway’s Ecosystem

Act Two integrates directly with other Runway tools, such as text-to-video, video-to-video, and image-to-video modules, supporting a streamlined, creative pipeline for everything from commercial projects to viral social media clips.

Real-World Applications

Filmmaking & Content Creation

• Easily animate characters for new scenes, dialogue, or alternative takes with minimal setup.
• Expand short clips into longer content by sequencing multiple performance transfers.
• Generate alternate marketing shots or ad variations without reshoots.

Social Media & Marketing

• Brands: Animate mascots, demo spokespeople, or product influencers directly from recorded staff or actors.

Education & Training

• Generate stylized learning videos, lecture explainers, or interactive digital guides featuring animated teachers or avatars.