Pith. sign in

REVIEW 3 major objections 6 minor 76 references

An immutable visual anchor plus fixed-capacity audio-video memory lets few-step causal generators keep identity and lip-sync stable across long prompt-driven segments in real time.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 17:03 UTC pith:LYP5YWFX

load-bearing objection Solid systems package for long-form A/V avatars; quality and 35-FPS claims are measured under different regimes, so the joint selling point is not fully closed. the 3 major comments →

arxiv 2607.24359 v1 pith:LYP5YWFX submitted 2026-07-27 cs.CV

TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation

Qijun Gan , Chenwei Zhang , Meiguang Jin , Junfeng Ma , Qiu Shen This is my paper
classification cs.CV
keywords digital human generationjoint audio-video diffusionpersistent memoryvisual anchorcausal distillationstage-parallel inferencelong-form consistencylip synchronization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long digital-human videos must keep the same face, clothes, scene, and voice while speech and gestures change under a sequence of prompts. Ordinary caches only remember the recent past, so older identity evidence disappears; keeping every past frame makes attention grow without bound and feeds old mistakes forward. TaoMate splits conditioning into a short position-bearing cache for local motion and phonetics, and a fixed-size persistent memory that holds an immutable visual anchor, a fixed audio reference, and compressed adaptive video and audio states. Those states are read by residual side attention and by reference-aware feature modulation, never by lengthening the active cache. Training deliberately rolls out mixed horizons on partly student-made, lightly noised history while leaving the anchor clean, so the model learns to trust the anchor when recent context is imperfect. Because memory sits outside stage-local caches, successive denoising stages can run on different blocks in parallel, reaching real-time frame rates without a special pipeline retrain. On long Mandarin and English single-presenter continuations the method reports stronger lip-sync, lower color drift, and higher long-range appearance consistency than compared systems.

Core claim

Separating an immutable visual anchor and fixed-capacity adaptive audio-video memory from the bounded active cache is enough to preserve subject appearance and audio-video synchronization across successive prompt-conditioned segments in few-step autoregressive generation, while also enabling training-free stage-parallel real-time inference.

What carries the argument

Anchor-guided persistent multimodal memory: immutable visual anchor and audio reference plus fixed-cardinality adaptive content and appearance states, updated only from stop-gradient completed blocks via an anchor-regularized barycenter (with confidence gating on video), then retrieved by modality-specific residual attention and reference-aware FiLM without extending the active KV cache.

Load-bearing premise

A fixed-size compressed memory seeded from the first block, updated with hand-chosen assimilation and gate rates only from finished blocks, stays rich enough to keep identity and appearance stable over long recursive continuations in real use.

What would settle it

Generate multi-minute single-presenter continuations with several prompt switches and measure whether long-range appearance consistency and color drift collapse once the active cache has turned over many times—especially if the initial anchor is removed or the confidence gate is disabled—while holding the rest of the system fixed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Long-form digital humans can keep a stable face, wardrobe, scene, and voice across prompt-driven action and speech changes without storing full history.
  • Few-step causal audio-video models can run stage-parallel across blocks and exceed playback frame rate on a few GPUs without pipeline-specific retraining.
  • Training that mixes student prefixes and mild history noise while protecting the anchor reduces the train–deploy context gap in recursive generation.
  • Appearance statistics can be conditioned separately from structural content, limiting color and contrast drift at segment boundaries.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same split—immutable reference versus bounded adaptive memory outside the KV cache—could transfer to non-avatar long video or audio-only streaming where identity or timbre must survive context refreshes.
  • If gate rates and anchor weight are too conservative, legitimate lighting or wardrobe changes may be under-absorbed; if too aggressive, drift returns—suggesting a need for content-adaptive or learned assimilation schedules.
  • Real deployments with camera motion, multi-person scenes, or domain shift away from teacher-style synthetic trajectories would stress whether fixed-capacity summaries remain sufficient without enlarging memory rank.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents TaoMate, a framework for real-time long-form audio-video digital-human generation built on a causal few-step generator distilled from a bidirectional LTX-2.3 teacher. Its core mechanism is an anchor-guided persistent memory: an immutable visual anchor and fixed audio reference, plus fixed-capacity adaptive content and appearance states updated (stop-gradient, gated by anchor agreement) from completed blocks and injected via residual side attention and reference-aware FiLM, keeping conditioning cost independent of generated duration. Training uses rollout-aligned causal-context distillation with mixed horizons, student-generated prefixes, and re-noised non-anchor history while the anchor stays clean. The separation of persistent memory from stage-local KV caches enables an anti-diagonal stage-parallel inference schedule across GPUs, reported at 35 output FPS on 3 GPUs. On an author-constructed 60-second Mandarin/English benchmark, TaoMate reports best-or-tied LipSync (5.918), Desync, Alignment, Long Consistency (0.9755), and ColorΔ (0.00260), with an ablation attributing gains to gated memory updates plus FiLM.

Significance. If the claims hold, this is a useful contribution to real-time long-form digital-human generation: a fixed-capacity, duration-independent memory with an explicit ablation (Table 3) isolating the gated-update and FiLM components, multi-metric evaluation against both joint A/V and S2V baselines, and reported multi-GPU throughput exceeding playback rate. The anchor-preserving distillation asymmetry (perturb non-anchor history, keep the anchor clean) is a clean, falsifiable design choice, and the ablation showing ungated FiLM actively hurts is the kind of negative control that strengthens the paper. The stage-parallel schedule, if quality-preserving as claimed, would be practically significant. These strengths are currently offset by the unverified quality-under-parallelism gap and the closed, teacher-derived evaluation.

major comments (3)
  1. [§3.5 / §4.2–4.3, Tables 1–2] The headline claim is a conjunction: Table 1 quality AND 35-FPS stage-parallel throughput, 'without pipeline-specific retraining' (§3.5, §4.3). No experiment measures quality under stage-parallel execution. Table 1's DiT FPS for TaoMate (16.32) matches the single-device serial configuration, so Table 1 metrics were almost certainly collected serially, while the 35 FPS figure (Table 2) comes from the 3-GPU anti-diagonal pipeline. Per §3.5, 'a block completed at stage k is committed only to the cache of stage k... Cleaner outputs from downstream stages are never written back into an upstream cache.' With three denoising steps across three ranks, stage 1's cache for block b therefore contains preceding blocks that have completed only one of three denoising steps — i.e., substantially noisy intermediate latents — whereas serial execution conditions stage 1 on fully denoised outputs. The only
  2. [§4.1–4.2, Table 1] The central quantitative evidence rests on an author-constructed long-form benchmark (§4.1) evaluated with two author-defined stability metrics whose 'exact definitions are provided in the supplementary material' (§4.2), on training/evaluation trajectories synthesized by the same LTX-2.3 teacher from which the student is distilled (§4.1). Baselines additionally 'retain their native resolution, frame rate, and long-form construction procedure,' so the DiT FPS column and possibly the quality metrics are not measured under matched conditions (e.g., LTX-2.3's poor scores may partly reflect being pushed into chunked continuation beyond its native horizon rather than an intrinsic deficit). The Face Consistency metric is MediaPipe normalized geometry, which the authors correctly note measures face-shape stability, not identity — so the 'stable appearance/identity' claim has no identity-specific
  3. [§3.2, Eq. (5); §4.1] The gate ρ_t in Eq. (5) is defined as a mini-batch average (B⁻¹ Σ_b cos(...)), and the gate hyperparameters (τ=0.05, T_g=0.10, g_min=0.10) were presumably calibrated against batch-averaged agreement statistics. At deployment, generation is a single stream (B=1), so the per-sample cosine distribution entering the sigmoid is much noisier than in training, shifting the effective assimilation rate β_v·g_t. This could silently change the memory-update behavior relative to what the ablation in Table 3 validated. Please clarify the batch size used during training rollouts, state the inference-time value of B, and either justify that the gate statistics match or report sensitivity of Table 3's final row to single-sample gating.
minor comments (6)
  1. [Fig. 1 / §4.1] Figure 1 advertises stability over 1,000 seconds, but all quantitative results are on 60-second outputs. Either add quantitative metrics at longer horizons (e.g., 5 minutes) or soften the teaser to 'qualitative.'
  2. [Table 3] Ablation deltas in Table 3 are small (Long Cons. 0.9614→0.9676; Face Cons. changes in the fourth decimal). No variance, number of evaluation samples, or significance is reported. Please state the evaluation set size and, ideally, standard errors, so readers can judge whether the gated-FiLM ordering is stable.
  3. [§3.3–3.4, §4.2] Key elements are deferred to the supplementary material: full distillation objectives and sampling schedules (§3.4), architectural details of memory attention and the parameter-free π readout (§3.3), and the Color∆/Long Cons. definitions (§4.2). Since the supplementary is not part of this submission, the main text is not self-contained; at minimum, summarize the π readout and metric definitions in a short appendix here.
  4. [Table 2] Table 2 compares FPS across systems with different GPU counts and model sizes (LiveAvatar: 14B on 5 GPUs; LongLive-2.0: 5B on 4 GPUs). A per-GPU or normalized throughput figure would make the comparison fairer; please also state the GPU model used for Table 2 (§4.1 names the RTX PRO 5000 only for single-device profiling).
  5. [§4.1] Several hyperparameters appear without sensitivity analysis: (β_v, λ_v)=(0.15, 0.20), (β_a, λ_a)=(0.10, 0.10), (β_c, λ_c)=(0.03, 0.60), memory token budgets (48 video / 64 audio), insertion every 4 layers, PCM weight 0.03 starting at step 400. A small sweep over at least the video assimilation/anchor rates would strengthen the claim that the design is robust rather than tuned to the benchmark.
  6. [§3.2, Eq. (3); Table 1] Notation: in Eq. (3), κ*_t = K(z^v_t) uses the un-normalized latent while π*_t uses the normalized one; a sentence on why channel statistics are extracted pre-normalization would help. Also define 'DiT FPS' inline at first use in Table 1 rather than only in §4.2 text.

Circularity Check

0 steps flagged

No significant circularity: empirical systems paper with external benchmarks, not a self-forcing derivation.

full rationale

TaoMate is an empirical methods paper. Its headline claims (appearance stability, A/V sync, real-time stage-parallel FPS) are supported by comparisons to external baselines (Table 1), multi-GPU throughput numbers (Table 2), and component ablations (Table 3), not by an algebraic derivation that reduces outputs to fitted inputs. Memory evolution (Eqs. 4–5), residual MemAttn/FiLM conditioning (Eq. 6), and rollout-aligned distillation are design choices with stated hyperparameters; they are not presented as first-principles predictions forced by construction. Training on frozen LTX-2.3 teacher trajectories and evaluating with VABench-style LipSync/Desync/Alignment plus MediaPipe face geometry is standard distillation practice with external-style metrics, not self-definitional circularity. Self-citations (e.g., OmniAvatar) appear as baselines, not as load-bearing uniqueness theorems. Concerns about serial vs. stage-parallel quality measurement or train–deploy cache mismatch are experimental-validity issues, not circular derivation. No step reduces a claimed prediction to its own inputs by definition.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 3 invented entities

Load-bearing content is architectural and empirical, not theorem-driven. The claim rests on standard diffusion/distillation background, the modeling choice that identity can be held by an immutable anchor plus fixed-cardinality compressed states, and many hand-set rates that control memory assimilation and training exposure. No new physical entities; invented pieces are engineering modules.

free parameters (8)
  • video memory assimilation/anchor rates (βv, λv) = (0.15, 0.20)
    Control how fast visual content memory tracks new blocks versus the anchor; directly affect drift vs responsiveness.
  • audio memory rates (βa, λa) = (0.10, 0.10)
    Analogous audio update/tether rates; chosen constants, not derived.
  • visual agreement gate (τ, Tg, gmin) = (0.05, 0.10, 0.10)
    Maps anchor cosine agreement to observation confidence; floor prevents freeze. Hand-tuned.
  • appearance state rates (βc, λc) = (0.03, 0.60)
    Strong anchor prior on channel/low-frequency appearance statistics to stop color drift.
  • PCM regularizer weight and start step = weight 0.03 from step 400, every 2 steps
    Auxiliary consistency loss schedule affecting few-step quality.
  • rollout mixture and history noise = horizons 5/10 with p=0.8/0.2; student prefix p=0.1; σ~U(0,0.15)
    Training exposure: horizon mix, student-prefix probability, re-noise level on non-anchor cache history.
  • memory token budgets and insertion stride = 48/64 tokens; every 4 of 48 layers (~97.8M extra params)
    48 video memory tokens (anchor+dynamic), 64 audio tokens, residual memory every 4 layers; capacity/compute tradeoff chosen by authors.
  • optimizer/EMA/CFG teacher scales = gen lr 1e-5, critic 2e-6, EMA 0.99; CFG video 3.0 audio 5.0; 3 steps
    Standard but claim-relevant training knobs for the distilled student.
axioms (5)
  • domain assumption Few-step causal students distilled from bidirectional joint A/V diffusion teachers can approximate the teacher’s local audio-video distribution when trained on self-rollouts (DMD + PCM-style objectives).
    Background of §1–2 and §3.4; inherited from Self/Causal/Mutual Forcing and DMD/PCM literature rather than proved here.
  • ad hoc to paper Long-range identity/appearance/acoustics can be adequately represented by fixed-cardinality detached summaries plus an immutable initial visual anchor, without retaining position-bearing full history.
    Core modeling bet of §3.1–3.2 (Eqs. 1–5); if false, drift returns as duration grows beyond evaluated horizons.
  • ad hoc to paper Stop-gradient memory updates applied only after clean block completion preserve the learned causal transition and allow stage-parallel schedules without pipeline-specific retraining.
    Stated in §3.1 (sg), §3.5, and contributions; underpins both stability accounting and the 35 FPS claim.
  • ad hoc to paper Cosine agreement between spatially normalized block content and the visual anchor is a valid reliability signal for gating visual and appearance updates.
    Eq. 5 and ablation contrast gated vs ungated FiLM; structural assumption about what ‘identity-safe’ updates look like.
  • standard math Standard transformer attention, flow-matching/diffusion latent generation, RoPE segment restarts, and FiLM-style feature modulation behave as in the cited prior literature.
    Used throughout §3 as background machinery.
invented entities (3)
  • Anchor-guided persistent multimodal memory state Mt=(av,aa,mv,ma,κ,π) no independent evidence
    purpose: Separate immutable references from adaptive content and appearance statistics beyond the active KV cache.
    Central postulated data structure of §3.2; effectiveness is only evidenced by this paper’s metrics/ablation.
  • Reference-aware FiLM residual conditioned on dynamic+anchor channel statistics no independent evidence
    purpose: Modulate video features for appearance calibration without dumping statistics through token attention.
    §3.3 conditioning path; FiLM exists prior, this residual wiring to dual appearance stats is paper-specific.
  • Memory-aware stage-parallel inference with stage-local KV and anti-diagonal block/stage wavefront no independent evidence
    purpose: Raise FPS by overlapping denoising stages across blocks while updating persistent memory only from final clean outputs.
    §3.5 / Fig. 3; claims quality-preserving acceleration without HiAR-style pipeline retraining.

pith-pipeline@v1.2.0-grok45-kimik3 · 18784 in / 4611 out tokens · 98848 ms · 2026-07-31T17:03:17.296977+00:00 · methodology

0 comments
read the original abstract

Real-time long-form digital-human generation relies on causal models to extend audio-visual content while preserving subject appearance and audio-video synchronization across successive segments. A bounded cache retains local motion and phonetic context but discards older evidence, whereas attending to the complete generated history is computationally expensive and can propagate accumulated errors. We present \method, an anchor-guided persistent-memory framework for few-step joint audio-video generation. The framework preserves an immutable visual anchor, compresses completed video and audio blocks into fixed-capacity dynamic states, and retrieves those states through modality-specific residual attention without extending the active cache. A reference-aware modulation method additionally conditions video features on dynamic and anchor appearance statistics. Anchor-preserving causal-context distillation varies rollout horizon, prefix provenance, and cache-history reliability while keeping the immutable visual anchor unperturbed. By separating persistent memory from stage-local denoising dependencies, \method further admits stage-parallel execution across blocks, accelerating autoregressive inference without pipeline-specific retraining. We evaluate long-form video continuations with appearance, temporal, synchronization, facial, and speech diagnostics. Results show that \method preserves stable appearance across prompt-conditioned segments and strong audio-visual synchronization under autoregressive generation. Our project page is https://taoliveaigc.github.io/TaoMate.

Figures

Figures reproduced from arXiv: 2607.24359 by Chenwei Zhang, Junfeng Ma, Meiguang Jin, Qijun Gan, Qiu Shen.

Figure 1
Figure 1. Figure 1: TaoMate remains stable over 1,000 seconds. Unlike OmniForcing’s visual degradation, identity drift, and color shift, TaoMate preserves consistency through visual anchoring, persistent A/V memory, and reference-aware FiLM. history to approximate the context distribution encountered during recursive inference. At inference, the separation be￾tween persistent memory and stage-local KV dependencies further ena… view at source ↗
Figure 2
Figure 2. Figure 2: TaoMate Pipeline. The pipeline generates synchronized long-form video and audio causally with anchor-guided persistent memory. A bounded context provides local motion and phonetic cues, while a fixed visual anchor and adaptive modality states supply long-range conditioning through side attention and reference-aware FiLM. shifts. We convert this agreement into a smooth observation￾confidence gate: ρt = B −1… view at source ↗
Figure 3
Figure 3. Figure 3: Memory-aware stage-parallel inference. Each stage [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on one-minute video generation. For two representative prompts, we show frames sampled at 0, 15, 30, and 60 seconds (columns) from videos generated by different methods (rows) [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

76 extracted references · 35 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2501.00103 , year=

    LTX-Video: Realtime Video Latent Diffusion , author=. arXiv preprint arXiv:2501.00103 , year=

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:2601.03233 , year=

    LTX-2: Efficient Joint Audio-Visual Foundation Model , author=. arXiv preprint arXiv:2601.03233 , year=

    Pith/arXiv arXiv

  3. [3]

    arXiv preprint arXiv:2602.19163 , year=

    JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation , author=. arXiv preprint arXiv:2602.19163 , year=

    arXiv

  4. [4]

    arXiv preprint arXiv:2510.01284 , year=

    Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation , author=. arXiv preprint arXiv:2510.01284 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2602.08794 , year=

    MOVA: Towards Scalable and Synchronized Video-Audio Generation , author=. arXiv preprint arXiv:2602.08794 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2401.16423 , year=

    Synchformer: Efficient Synchronization from Sparse Cues , author=. arXiv preprint arXiv:2401.16423 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2503.20215 , year=

    Qwen2.5-Omni Technical Report , author=. arXiv preprint arXiv:2503.20215 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2606.17800 , year=

    MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model , author=. arXiv preprint arXiv:2606.17800 , year=

    Pith/arXiv arXiv

  9. [9]

    Advances in Neural Information Processing Systems , volume=

    Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion , author=. Advances in Neural Information Processing Systems , volume=. 2025 , url=

    2025

  10. [10]

    arXiv preprint arXiv:2602.02214 , year=

    Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation , author=. arXiv preprint arXiv:2602.02214 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2512.04677 , year=

    Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length , author=. arXiv preprint arXiv:2512.04677 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2506.18866 , year=

    OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation , author=. arXiv preprint arXiv:2506.18866 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2508.08248 , year=

    StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation , author=. arXiv preprint arXiv:2508.08248 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2508.18621 , year=

    Wan-S2V: Audio-Driven Cinematic Video Generation , author=. arXiv preprint arXiv:2508.18621 , year=

    Pith/arXiv arXiv

  15. [15]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=. 2025 , url=

    2025

  16. [16]

    International Conference on Learning Representations , year=

    LongLive: Real-time Interactive Long Video Generation , author=. International Conference on Learning Representations , year=

  17. [17]

    arXiv preprint arXiv:2605.18739 , year=

    LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation , author=. arXiv preprint arXiv:2605.18739 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2604.25819 , year=

    Mutual Forcing: Dual-Mode Self-Evolution for Fast Autoregressive Audio-Video Character Generation , author=. arXiv preprint arXiv:2604.25819 , year=

    Pith/arXiv arXiv

  19. [19]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    One-step Diffusion with Distribution Matching Distillation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=. 2024 , url=

    2024

  20. [20]

    Advances in Neural Information Processing Systems , volume=

    Improved Distribution Matching Distillation for Fast Image Synthesis , author=. Advances in Neural Information Processing Systems , volume=. 2024 , doi=

    2024

  21. [21]

    arXiv preprint arXiv:2606.13035 , year=

    TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment , author=. arXiv preprint arXiv:2606.13035 , year=

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2605.13111 , year=

    Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation , author=. arXiv preprint arXiv:2605.13111 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2604.21221 , year=

    Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generation , author=. arXiv preprint arXiv:2604.21221 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2411.19108 , year=

    Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model , author=. arXiv preprint arXiv:2411.19108 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2512.04519 , year=

    VideoSSM: Autoregressive Long Video Generation with Hybrid State-Space Memory , author=. arXiv preprint arXiv:2512.04519 , year=

    arXiv

  26. [26]

    arXiv preprint arXiv:2605.30349 , year=

    AdaState: Self-Evolving Anchors for Streaming Video Generation , author=. arXiv preprint arXiv:2605.30349 , year=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2605.31033 , year=

    SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation , author=. arXiv preprint arXiv:2605.31033 , year=

    Pith/arXiv arXiv

  28. [28]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

    VBench: Comprehensive Benchmark Suite for Video Generative Models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

  29. [29]

    Proceedings of the 28th ACM International Conference on Multimedia , pages=

    A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild , author=. Proceedings of the 28th ACM International Conference on Multimedia , pages=. 2020 , doi=

    2020

  30. [30]

    Proceedings of the 40th International Conference on Machine Learning , series=

    Robust Speech Recognition via Large-Scale Weak Supervision , author=. Proceedings of the 40th International Conference on Machine Learning , series=. 2023 , publisher=

    2023

  31. [31]

    arXiv preprint arXiv:1801.07698 , year=

    ArcFace: Additive Angular Margin Loss for Deep Face Recognition , author=. arXiv preprint arXiv:1801.07698 , year=

    Pith/arXiv arXiv

  32. [32]

    Advances in Neural Information Processing Systems , pages=

    Denoising Diffusion Probabilistic Models , author=. Advances in Neural Information Processing Systems , pages=. 2020 , url=

    2020

  33. [33]

    International Conference on Learning Representations , year=

    Denoising Diffusion Implicit Models , author=. International Conference on Learning Representations , year=

  34. [34]

    arXiv preprint arXiv:2207.12598 , year=

    Classifier-Free Diffusion Guidance , author=. arXiv preprint arXiv:2207.12598 , year=

    Pith/arXiv arXiv

  35. [35]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    High-Resolution Image Synthesis with Latent Diffusion Models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=. 2022 , url=

    2022

  36. [36]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Scalable Diffusion Models with Transformers , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=. 2023 , url=

    2023

  37. [37]

    arXiv preprint arXiv:2210.02303 , year=

    Imagen Video: High Definition Video Generation with Diffusion Models , author=. arXiv preprint arXiv:2210.02303 , year=

    Pith/arXiv arXiv

  38. [38]

    arXiv preprint arXiv:2209.14792 , year=

    Make-A-Video: Text-to-Video Generation without Text-Video Data , author=. arXiv preprint arXiv:2209.14792 , year=

    Pith/arXiv arXiv

  39. [39]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=. 2023 , url=

    2023

  40. [40]

    arXiv preprint arXiv:2311.15127 , year=

    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets , author=. arXiv preprint arXiv:2311.15127 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2307.04725 , year=

    AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning , author=. arXiv preprint arXiv:2307.04725 , year=

    Pith/arXiv arXiv

  42. [42]

    arXiv preprint arXiv:2312.14125 , year=

    VideoPoet: A Large Language Model for Zero-Shot Video Generation , author=. arXiv preprint arXiv:2312.14125 , year=

    Pith/arXiv arXiv

  43. [43]

    ACM Transactions on Graphics , volume=

    MakeItTalk: Speaker-Aware Talking-Head Animation , author=. ACM Transactions on Graphics , volume=. 2020 , doi=

    2020

  44. [44]

    Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence , pages=

    Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion , author=. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence , pages=. 2021 , doi=

    2021

  45. [45]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=. 2023 , doi=

    2023

  46. [46]

    International Conference on Learning Representations , year=

    GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis , author=. International Conference on Learning Representations , year=

  47. [47]

    European Conference on Computer Vision , pages=

    EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions , author=. European Conference on Computer Vision , pages=. 2024 , doi=

    2024

  48. [48]

    arXiv preprint arXiv:2406.08801 , year=

    Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation , author=. arXiv preprint arXiv:2406.08801 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2407.03168 , year=

    LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control , author=. arXiv preprint arXiv:2407.03168 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2311.16498 , year=

    MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model , author=. arXiv preprint arXiv:2311.16498 , year=

    Pith/arXiv arXiv

  51. [51]

    International Conference on Learning Representations , year=

    Progressive Distillation for Fast Sampling of Diffusion Models , author=. International Conference on Learning Representations , year=

  52. [52]

    Proceedings of the 40th International Conference on Machine Learning , series=

    Consistency Models , author=. Proceedings of the 40th International Conference on Machine Learning , series=. 2023 , publisher=

    2023

  53. [53]

    arXiv preprint arXiv:2310.04378 , year=

    Latent Consistency Models: Synthesizing High-Resolution Images with Few-step Inference , author=. arXiv preprint arXiv:2310.04378 , year=

    Pith/arXiv arXiv

  54. [54]

    Advances in Neural Information Processing Systems , year=

    Phased Consistency Models , author=. Advances in Neural Information Processing Systems , year=

  55. [55]

    International Conference on Learning Representations , year=

    An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion , author=. International Conference on Learning Representations , year=

  56. [56]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

    DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

  57. [57]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

    Adding Conditional Control to Text-to-Image Diffusion Models , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

  58. [58]

    International Conference on Learning Representations , year=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. International Conference on Learning Representations , year=

  59. [59]

    2018 , url=

    Perez, Ethan and Strub, Florian and de Vries, Harm and Dumoulin, Vincent and Courville, Aaron , booktitle=. 2018 , url=

    2018

  60. [60]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=. 2019 , doi=

    2019

  61. [61]

    International Conference on Learning Representations , year=

    Compressive Transformers for Long-Range Sequence Modelling , author=. International Conference on Learning Representations , year=

  62. [62]

    International Conference on Learning Representations , year=

    Memorizing Transformers , author=. International Conference on Learning Representations , year=

  63. [63]

    International Conference on Learning Representations , year=

    Efficient Streaming Language Models with Attention Sinks , author=. International Conference on Learning Representations , year=

  64. [64]

    Advances in Neural Information Processing Systems , volume=

    H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , author=. Advances in Neural Information Processing Systems , volume=. 2023 , url=

    2023

  65. [65]

    arXiv preprint arXiv:2310.01889 , year=

    Ring Attention with Blockwise Transformers for Near-Infinite Context , author=. arXiv preprint arXiv:2310.01889 , year=

    Pith/arXiv arXiv

  66. [66]

    arXiv preprint arXiv:1812.01717 , year=

    Towards Accurate Generative Models of Video: A New Metric and Challenges , author=. arXiv preprint arXiv:1812.01717 , year=

    Pith/arXiv arXiv

  67. [67]

    International Conference on Machine Learning , year=

    Learning Transferable Visual Models From Natural Language Supervision , author=. International Conference on Machine Learning , year=

  68. [68]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

    Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , year=

  69. [69]

    arXiv preprint arXiv:1907.06724 , year=

    Real-Time Facial Surface Geometry from Monocular Video on Mobile GPUs , author=. arXiv preprint arXiv:1907.06724 , year=

    Pith/arXiv arXiv 1907

  70. [70]

    Hua, Daili and Wang, Xizhi and Zeng, Bohan and Huang, Xinyi and Liang, Hao and Niu, Junbo and Chen, Xinlong and Xu, Quanqing and Zhang, Wentao , booktitle=

  71. [71]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  72. [72]

    Anchor Forcing: Anchor Memory and Tri-Region

    Yang, Yang and Zhang, Tianyi and Huang, Wei and Chen, Jinwei and Wu, Boxi and He, Xiaofei and Cai, Deng and Li, Bo and Jiang, Peng-Tao , journal=. Anchor Forcing: Anchor Memory and Tri-Region

  73. [73]

    arXiv preprint arXiv:2603.14331 , year=

    AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising , author=. arXiv preprint arXiv:2603.14331 , year=

    arXiv

  74. [74]

    arXiv preprint arXiv:2605.30519 , year=

    OmniMem: Scalable and Adaptive Memory Retrieval for Long Video Generation , author=. arXiv preprint arXiv:2605.30519 , year=

    Pith/arXiv arXiv

  75. [75]

    arXiv preprint arXiv:2606.10671 , year=

    FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion , author=. arXiv preprint arXiv:2606.10671 , year=

    Pith/arXiv arXiv

  76. [76]

    2026 , url=

    Zou, Kai and Zheng, Dian and Liu, Hongbo and Hang, Tiankai and Liu, Bin and Yu, Nenghai , journal=. 2026 , url=

    2026