Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Wan-Streamer v0.2 raises live interactive video from 192×336 to 640×368 while keeping about 200 ms model-side latency by splitting a single-GPU thinker from a multi-GPU Ulysses performer.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 19:07 UTC pith:LS5SX7RN

load-bearing objection Clean systems upgrade note: higher-res streaming AV at claimed same latency via thinker/Ulysses-performer split, but the latency claim is asserted without measurements. the 2 major comments →

arxiv 2607.04443 v3 pith:LS5SX7RN submitted 2026-07-05 cs.CV cs.AIcs.GRcs.LG

Wan-Streamer v0.2: Higher Resolution, Same Latency

Lianghua Huang , Zhi-Fan Wu , Yupeng Shi , Wei Wang , Mengyang Feng , Junjie He , Chen-Wei Xie , Yu Liu
show 18 more authors
Jingren Zhou Ang Wang Bang Zhang Baole Ai Chen Liang Cheng Yu Chongyang Zhong Jinwei Qi Kai Zhu Pandeng Li Peng Zhang Wenyuan Zhang Xinhua Cheng Yitong Huang Yun Zheng Yuxiang Bao Yuzheng Wang Zoubin Bi
This is my paper
classification cs.CV cs.AIcs.GRcs.LG
keywords real-time audio-visual interactionstreaming video generationlatency-preserving servingcontext parallelismdigital humansfull-duplex dialogueinteractive agentsUlysses sequence parallelism
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper shows that a real-time audio-visual conversation model can move from low-resolution close-up framing to a higher-resolution stream where an agent’s posture, gaze, hands, nearby objects, and scene layout stay readable, without stretching the interaction cadence. It keeps the same native-streaming causal formulation as the prior version, so user and agent text, audio, and video still share one timeline. The extra generation cost of 640×368 video is absorbed by a multi-GPU context-parallel “performer” that sequence-shards the long video latents, while a single-GPU “thinker” still owns perception, language/state updates, cache construction, and decoding. Model-side signal-to-signal latency stays near 200 ms at 25 FPS; with a fixed network budget the total remote loop remains about 550 ms. A sympathetic reader cares because that is the difference between a face-only video call and a scene-grounded mid-shot agent that can still answer in real time.

Core claim

v0.2 is a latency-preserving resolution upgrade of the native-streaming end-to-end audio-visual interaction model: it raises the interactive output stream from 192×336 to 640×368 at 25 FPS while preserving approximately 200 ms model-side signal-to-signal latency, by keeping the thinker on one GPU for perception, state/KV construction, and decoding, and moving high-resolution latent generation into a multi-GPU Ulysses-style context-parallel performer.

What carries the argument

The thinker–performer serving split with Ulysses-style context parallelism: a single-GPU thinker builds performer-compatible K/V slices; each performer rank writes them into a pre-sharded local cache and sequence-shards only the long high-resolution video latents for denoising (short audio latents stay unsharded), so added hardware concentrates on visual generation without widening the compact control boundary.

Load-bearing premise

That the multi-GPU performer group finishes high-resolution video latent generation, plus K/V and latent transfer and internal all-to-all/gather communication, inside each 160 ms streaming unit so the model-side path still lands near 200 ms.

What would settle it

Instrument wall-clock time from the moment a 160 ms user streaming unit is available to the thinker until the matching 640×368 audio-video unit is decoded for emission under the described multi-GPU Ulysses setup; if that path systematically exceeds about 200 ms, the latency-preserving claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Interactive agents can be framed as mid-shot scene-grounded characters with legible body, hands, and local layout, not only close-up faces, at the same low-latency cadence.
  • Extra compute can be spent on visual latent generation without forcing a separate language sequence through the performer group.
  • Total remote interaction latency can remain about 550 ms at 640×368 when a 350 ms bidirectional network budget is included.
  • Close-up video-call fidelity improves under the same streaming causality constraints used at lower resolution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same thinker/performer split may extend to still higher resolutions if sequence-parallel efficiency continues to scale with latent length.
  • Tasks that need readable hands and nearby objects in live dialogue (demonstrations, tutoring, shared workspaces) become more practical without cascaded avatar pipelines.
  • Keeping the latency-critical control path single-GPU while parallelizing only generation is a pattern other multimodal streaming systems could reuse.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. Wan-Streamer v0.2 is presented as a latency-preserving systems upgrade of the authors' prior native-streaming end-to-end audio-visual interaction model (v0.1). The modeling formulation is held fixed: user and agent text, audio, and video share one causal timeline under block-causal attention. The claimed advance is raising interactive output from 192×336 to 640×368 at 25 FPS while keeping approximately 200 ms model-side signal-to-signal latency (and ~550 ms total remote latency under a fixed 350 ms bidirectional network budget). To absorb the higher visual cost, the paper splits serving into a single-GPU thinker (streaming perception, short language/state Transformer pass that builds the generation K/V cache, and final decoding) and a multi-GPU Ulysses-style context-parallel performer for next-unit latent generation. Performer ranks write thinker-broadcast K/V into pre-sharded local caches; the long high-resolution video latent sequence is sequence-sharded for denoising with intra-group all-to-all/gather, while short audio latents are left unsharded. The paper argues this concentrates hardware on visual generation without lengthening the latency-critical control path, and that the higher-resolution stream enables scene-grounded mid-shot agents with legible posture, gaze, hands, objects, and local layout.

Significance. If the latency claim is demonstrated, the work is a useful systems contribution at the intersection of full-duplex dialogue, streaming video generation, and interactive digital humans: same interaction cadence at roughly 3–4× linear resolution, with a clean thinker–performer boundary that folds language/state into K/V conditioning so the performer group need not communicate a separate language sequence. The architectural idea—single-GPU low-latency control path plus Ulysses-style sequence-parallel visual latent generation with pre-sharded K/V—is concrete and transferable. The expanded mid-shot visual scope is also practically relevant beyond portrait video-call framing. The manuscript does not ship machine-checked proofs, public code, or parameter-free derivations; its value is engineering. That value hinges on measured evidence that the 160 ms unit budget is actually met under the new topology.

major comments (2)
  1. [Sec. 4; also Abstract, Table 1, Sec. 3, Fig. 2] The central claim of the paper is that 640×368 output is achieved at the same ~200 ms model-side signal-to-signal latency as v0.1 (Abstract; Table 1; Sec. 3; Sec. 4). Sec. 4 defines the measurement boundary carefully (start when a 160 ms user unit is available to the thinker; end when the corresponding AV response unit is decoded) and restates ~200 ms / ~550 ms, but supplies no wall-clock timings, variance, GPU count or type, communication volumes for thinker–performer K/V and latent transfer, Ulysses all-to-all/gather cost, FLOPs, or ablations versus a single-GPU performer. Fig. 2 and Sec. 3 describe a schedule in which performer work must fit inside each 160 ms unit; without measurements this remains an architectural assertion. For a systems paper whose primary result is latency preservation under higher resolution, these numbers are load-bearing and should be reported (including hardw
  2. [Sec. 4 (Qualitative visual observations); also Sec. 2.1–2.2, contributions list] Visual claims (higher-fidelity close-ups and scene-grounded mid-shot agents with legible posture, gaze, hands, objects, and layout) are supported only by qualitative description in Sec. 4. There are no figures of generated frames, no side-by-side v0.1 vs v0.2 comparisons, and no quantitative proxies (identity consistency, hand/pose readability, temporal stability, lip-sync, or user study). The resolution upgrade is the other half of the contribution; even a small set of representative stills or short sequence strips in the main text or appendix would make the mid-shot claim inspectable. Without that, the visual half of the result cannot be evaluated.
minor comments (6)
  1. [Table 1] Table 1 is a useful version comparison, but the latency rows are restated targets rather than measured results; labeling them as targets vs. measured would avoid over-reading the table as empirical evidence.
  2. [Fig. 2] Fig. 2 is informative but dense. A short caption note on which arrows are on the critical path for the ~200 ms signal-to-signal latency (vs. throughput-only paths) would help readers map the figure to the latency definition in Sec. 4.
  3. [Sec. 3] The paper cites DeepSpeed Ulysses [13] for context parallelism but does not state the performer world size, sharding axis for video latents, or whether attention is fully sequence-parallel for all denoising steps. A brief implementation paragraph would improve reproducibility of the serving claim.
  4. [Sec. 4] Sec. 4 notes that public systems report heterogeneous latency endpoints; this is fair. A one-row comparison table of what is and is not claimed relative to those endpoints (even if not apples-to-apples) would reduce ambiguity for readers coming from avatar/product literature.
  5. [Abstract] Minor typography: Abstract and body use both “approximately200 ms” (missing space) and “approximately 200 ms”; normalize spacing. Also “supportsscene-grounded” and similar missing spaces appear in the abstract text.
  6. [References] References include the authors’ own v0.1 as [1] with a June 2026 arXiv date and other 2025–2026 preprints; ensure final versions and DOIs are updated if available at camera-ready.

Circularity Check

0 steps flagged

No significant circularity: engineering upgrade asserts a serving schedule; latency is not derived from a fitted or self-defined quantity.

full rationale

Wan-Streamer v0.2 is a systems/version-upgrade paper, not a first-principles derivation. It inherits the native-streaming formulation and the ~200 ms / ~550 ms measurement protocol from v0.1 by design (Abstract; Sec. 2.1; Table 1; Sec. 4), then describes a thinker/performer serving topology (single-GPU thinker, multi-GPU Ulysses-style performer with pre-sharded K/V and sequence-sharded video denoising) intended to keep that budget at 640×368 (Sec. 3; Fig. 2). Self-citation to v0.1 is expected for a v0.2 note and does not force the latency outcome by construction: the paper does not fit a parameter and rename it a prediction, does not import a uniqueness theorem from the same authors, and does not define the claimed latency in terms of itself. External citations (e.g., DeepSpeed Ulysses [13]) supply the parallelism mechanism rather than smuggling an ansatz that equals the result. The main scientific weakness is under-evidenced wall-clock measurement of the 160 ms unit budget, which is a correctness/evidence gap, not definitional or self-citation circularity. Score 0 with empty steps is therefore the honest finding.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

This is a systems upgrade paper, not a derivation paper. Load-bearing content is mostly inherited modeling from v0.1, standard distributed-attention machinery, and chosen latency/resolution operating points. Free parameters are the operating targets and the external network budget; axioms are the stability of the v0.1 formulation and the assumption that Ulysses CP plus the thinker/performer boundary can absorb the extra visual cost inside the unit time. Invented entities are mostly role names for a serving split already present in spirit in v0.1.

free parameters (4)
  • model-side latency target = ~200 ms
    The ~200 ms signal-to-signal budget is a chosen operating point that defines success; the paper does not derive it from first principles.
  • streaming unit duration = 160 ms
    Work is scheduled in 160 ms units at 25 FPS; this cadence is a design choice that the performer must fit inside.
  • bidirectional network budget = 350 ms
    Total remote latency of ~550 ms assumes a fixed 350 ms network term treated as an external deployment constant.
  • output resolution and frame rate = 640×368 @ 25 FPS
    640×368 at 25 FPS is the chosen visual target that drives the need for multi-GPU generation.
axioms (4)
  • domain assumption The v0.1 native-streaming causal formulation (shared timeline, block-causal attention, committed audio-video latents) remains sufficient when resolution increases.
    Stated throughout Sec. 2.1 and Table 1 as the stable baseline; not re-validated with new training or ablations in this paper.
  • domain assumption Ulysses-style context parallelism with pre-sharded K/V caches can shard the long high-resolution video latent sequence while leaving short audio latents unsharded without breaking the 160 ms unit.
    Core of Sec. 3 and Fig. 2; relies on prior Ulysses systems work [13] plus unstated hardware scale.
  • ad hoc to paper Language/state computation can be fully folded into performer-compatible K/V slices so no separate language sequence must be communicated inside the performer group.
    Explicit design choice in Abstract and Sec. 3 that keeps the thinker–performer boundary compact.
  • domain assumption A 350 ms bidirectional network budget is a reasonable external constant for reporting total remote interaction latency.
    Carried from v0.1 and restated in Abstract and Sec. 4; not measured here.
invented entities (2)
  • Thinker / Performer serving roles (v0.2 topology) no independent evidence
    purpose: Isolate latency-critical perception, state update, KV construction, and decoding on one GPU while concentrating multi-GPU capacity on high-res latent generation.
    Roles are named and specialized for the resolution upgrade; the split is the paper's main systems contribution relative to single-GPU v0.1 generation.
  • Pre-sharded performer-side K/V caches receiving thinker-broadcast slices no independent evidence
    purpose: Allow each performer rank to update a local shard of history without shipping a full language sequence inside the group.
    Described in Abstract and Sec. 3 as the mechanism that preserves a compact thinker–performer boundary.

pith-pipeline@v1.1.0-grok45 · 11773 in / 3647 out tokens · 43677 ms · 2026-07-11T19:07:27.236386+00:00 · methodology

0 comments
read the original abstract

We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Video = World + Event Stream

    cs.CV 2026-07 reject novelty 3.0

    Wan-Streamer v0.3 reformulates streaming video as a persistent world plus a changing event stream and extends an agent's output with open-vocabulary behavior, without reporting any quantitative results.

Reference graph

Works this paper leans on

17 extracted references · 10 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models, June 2026

    Lianghua Huang, Zhi-Fan Wu, Wei Wang, Yupeng Shi, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, and Zoubin Bi. Wan-Streamer v0.1: End-to-end Real-time Interactive Foundatio...

    Pith/arXiv arXiv 2026

  2. [2]

    From turn-taking to synchronous dialogue: A survey of full-duplex spoken language models.arXiv preprint arXiv:2509.14515, 2025

    Yuxuan Chen and Haoyuan Yu. From turn-taking to synchronous dialogue: A survey of full-duplex spoken language models.arXiv preprint arXiv:2509.14515, 2025

    arXiv 2025

  3. [3]

    Moshi: a speech-text foundation model for real-time dialogue.arXiv preprint arXiv:2410.00037, 2024

    Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. Moshi: a speech-text foundation model for real-time dialogue.arXiv preprint arXiv:2410.00037, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025

    Qwen Team. Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Minicpm-o 4.5: Towards real-time full-duplex omni-modal interaction.arXiv preprint arXiv:2604.27393, 2026

    OpenBMB Team. Minicpm-o 4.5: Towards real-time full-duplex omni-modal interaction.arXiv preprint arXiv:2604.27393, 2026

    Pith/arXiv arXiv 2026

  6. [6]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion

    Boyuan Chen, Diego Marti Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. InAdvances in Neural Information Processing Systems, 2024

    2024

  8. [8]

    Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Vasa-1: Lifelike audio-driven talking faces generated in real time.arXiv preprint arXiv:2404.10667, 2024

    Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. Vasa-1: Lifelike audio-driven talking faces generated in real time.arXiv preprint arXiv:2404.10667, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Streamavatar: Streaming diffusion models for real-time interactive human avatars.arXiv preprint arXiv:2512.22065, 2025

    Zhiyao Sun, Ziqiao Peng, Yifeng Ma, Yi Chen, Zhengguang Zhou, Zixiang Zhou, Guozhen Zhang, Youliang Zhang, Yuan Zhou, Qinglin Lu, and Yong-Jin Liu. Streamavatar: Streaming diffusion models for real-time interactive human avatars.arXiv preprint arXiv:2512.22065, 2025

    arXiv 2025

  11. [11]

    Mavid: A multimodal framework for audio-visual dialogue understanding and generation.arXiv preprint arXiv:2512.03034, 2025

    Youxin Pang, Jiajun Liu, Lingfeng Tan, Yong Zhang, Feng Gao, Xiang Deng, Zhuoliang Kang, Xiaoming Wei, and Yebin Liu. Mavid: A multimodal framework for audio-visual dialogue understanding and generation.arXiv preprint arXiv:2512.03034, 2025

    arXiv 2025

  12. [12]

    Body of her: A preliminary study on end-to-end humanoid agent.arXiv preprint arXiv:2408.02879, 2024

    Tenglong Ao. Body of her: A preliminary study on end-to-end humanoid agent.arXiv preprint arXiv:2408.02879, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Doubao realtime voice model.https://seed.bytedance.com/en/realtime_voice, 2025

    ByteDance Seed Team. Doubao realtime voice model.https://seed.bytedance.com/en/realtime_voice, 2025. Model page, January 20, 2025

    2025

  15. [15]

    Hello gpt-4o.https://openai.com/index/hello-gpt-4o/, 2024

    OpenAI. Hello gpt-4o.https://openai.com/index/hello-gpt-4o/, 2024. Blog post, May 13, 2024

    2024

  16. [16]

    Introducing evi 3: The world’s most realistic and instructible speech-language model.https://www

    Hume AI. Introducing evi 3: The world’s most realistic and instructible speech-language model.https://www. hume.ai/blog/introducing-evi-3, 2025. Blog post, 2025

    2025

  17. [17]

    Hallo-live: Real-time streaming joint audio-video avatar generation with asynchronous dual-stream and human-centric preference distillation.arXiv preprint arXiv:2604.23632, 2026

    Chunyu Li, Jiaye Li, Ruiqiao Mei, Haoyuan Xia, Hao Zhu, Jingdong Wang, and Siyu Zhu. Hallo-live: Real-time streaming joint audio-video avatar generation with asynchronous dual-stream and human-centric preference distillation.arXiv preprint arXiv:2604.23632, 2026. 6 Appendix A Contributions and Acknowledgements A.1 Core Contributors Lianghua Huang, Zhi-Fan...

    Pith/arXiv arXiv 2026