Pith. sign in

REVIEW 8 cited by

Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2505.22647 v1 pith:GHA2MR6O submitted 2025-05-28 cs.CV

Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation

Zhe Kong , Feng Gao , Yong Zhang , Zhuoliang Kang , Xiaoming Wei , Xunliang Cai , Guanying Chen , Wenhan Luo This is my paper
classification cs.CV
keywords generationaudiomulti-persontalkingmethodstraininganimationaudio-driven
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Audio-driven human animation methods, such as talking head and talking body generation, have made remarkable progress in generating synchronized facial movements and appealing visual quality videos. However, existing methods primarily focus on single human animation and struggle with multi-stream audio inputs, facing incorrect binding problems between audio and persons. Additionally, they exhibit limitations in instruction-following capabilities. To solve this problem, in this paper, we propose a novel task: Multi-Person Conversational Video Generation, and introduce a new framework, MultiTalk, to address the challenges during multi-person generation. Specifically, for audio injection, we investigate several schemes and propose the Label Rotary Position Embedding (L-RoPE) method to resolve the audio and person binding problem. Furthermore, during training, we observe that partial parameter training and multi-task training are crucial for preserving the instruction-following ability of the base model. MultiTalk achieves superior performance compared to other methods on several datasets, including talking head, talking body, and multi-person datasets, demonstrating the powerful generation capabilities of our approach.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation

    cs.CV 2026-06 unverdicted novelty 6.0

    InterTalk is a motion-based real-time framework for flexible multi-round multi-person conversational talking face generation using motion feedback, iterative strategies, and facial component disentanglement, supported...

  2. InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars

    cs.CV 2026-06 unverdicted novelty 6.0

    InteractiveAvatar uses autoregressive distillation, Long-Short Visual Memory, and a Reasoning-Reaction Module to enable real-time, consistent, intent-aware avatar video streaming.

  3. InteractiveAvatar: Real-Time Streaming Video Generation for Consistent and Intent-Aware Avatars

    cs.CV 2026-06 unverdicted novelty 6.0

    InteractiveAvatar is a real-time infinite-streaming avatar video generation system using autoregressive distillation, Long-Short Visual Memory for consistency, and a Reasoning-Reaction Module for intent-aware interactions.

  4. Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation

    cs.CV 2026-05 unverdicted novelty 6.0

    TT-SAC is a parameter-free inference framework that uses a generator-encoder feedback loop to adapt conditioning representations and stabilize identity and motion in audio-driven talking-head videos.

  5. PresentAgent-2: Towards Generalist Multimodal Presentation Agents

    cs.CV 2026-05 unverdicted novelty 6.0

    PresentAgent-2 generates query-driven multimodal presentation videos with research grounding, supporting single-speaker, multi-speaker discussion, and interactive question-answering modes.

  6. OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

    cs.CV 2026-04 unverdicted novelty 6.0

    OmniShow unifies text, image, audio, and pose conditions into an end-to-end model for high-quality human-object interaction video generation and introduces the HOIVG-Bench benchmark, claiming state-of-the-art results.

  7. TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation

    cs.CV 2026-04 unverdicted novelty 5.0

    TurboTalk uses progressive distillation from 4 steps to 1 step with distribution matching and adversarial training to achieve 120x faster single-step audio-driven talking avatar video generation.

  8. AUHead: Realistic Emotional Talking Head Generation via Action Units Control

    cs.CV 2026-02 unverdicted novelty 5.0

    AUHead uses audio-language models to generate Action Unit sequences from speech and feeds them into a controllable diffusion model to synthesize realistic emotional talking-head videos.