Video = World + Event Stream

Ang Wang; Bang Zhang; Baole Ai; Cheng Yu; Chen Liang; Chen-Wei Xie; Chongyang Zhong; Jingren Zhou; Jinwei Qi; Junjie He

REVIEW 3 major objections 4 minor 1 cited by

A video is a persistent world plus a time-varying event stream, and training a streaming model to predict the next event from the world and incoming input gives a general-purpose real-time competence; instantiating this on full-duplex audio

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:19 UTC pith:WGWK2OOL

load-bearing objection A clearly written, honest version note whose central claim — world-event pretraining transfers to real-time tasks — is explicitly not measured; the only concrete new element is open-vocabulary behavior via parenthesized directives. the 3 major comments →

arxiv 2607.15038 v2 pith:WGWK2OOL submitted 2026-07-16 cs.CV

Video = World + Event Stream

Lianghua Huang , Zhi-Fan Wu , Yupeng Shi , Wei Wang , Mengyang Feng , Cheng Yu , Chen Liang , Junjie He

show 19 more authors

Chen-Wei Xie Yu Liu Jingren Zhou Ang Wang Bang Zhang Baole Ai Chongyang Zhong Jinwei Qi Kai Zhu Pandeng Li Peng Zhang Wenyuan Zhang Xinhua Cheng Yitong Huang Yun Zheng Yuxiang Bao Yuzheng Wang Zhiwei Lin Zoubin Bi

This is my paper

classification cs.CV

keywords world-event decompositionreal-time interactionfull-duplex audio-visualstreaming transformervideo pretrainingopen-vocabulary behaviorvision-language-actionlow-latency generation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's central claim is that any video is usefully decomposed into a persistent world (scene, characters, ambient sound, voice) and an event stream (behavior, speech, changes), and that this decomposition defines a general-purpose pretraining objective: given a world and incoming input, predict how the world streams forward in real time. Trained on ordinary video, the resulting competence can be specialized to real-time downstream tasks without changing the streaming response logic. The paper instantiates this on real-time full-duplex audio-visual interaction, where the agent's response is an interleaved token stream of spoken words and free-form, language-described behavior that conditions synchronized audio-video generation. This preserves the previous operating point—640×368 video at 25 FPS, about 200 ms model-side latency, and about 550 ms total interaction latency. A sympathetic reader would care because it recasts interaction modeling as general world-event prediction, potentially making one pretrained model reusable across many real-time tasks.

Core claim

The organizing claim is that a video is a world plus an event stream. The world is the persistent context—environment, visual style, subjects and identities, ambient acoustics, voice—kept stable over a clip or interaction; the event stream is everything that changes: subject behavior, camera motion, environmental change, speech, and sounds. Under this reading, every natural video is a training example for one objective: given the world and an incoming input, predict how the world evolves, unit by unit, in real time. The paper formalizes this as a factorization of the next event given the world, the causal input history, and prior events, making the world a first-class conditioning object tha

What carries the argument

The world-event factorization p(e1:K | W, x1:K) = product over k of p(e_k | W, x≤k, e<k), which promotes the persistent world record W to a separate, once-prefilled conditioning object and then autoregressively predicts time-localized event records e_k. The second piece is the role-play token stream, in which parenthesized behavior directives and spoken words share a single token sequence that conditions joint audio-video generation. The factorization makes the pretraining objective general—any video supplies a world and events—while the shared token stream lets behavior and speech be learned together and rendered by the same causal decoders without adding a latency path.

Load-bearing premise

The load-bearing premise is that the existing audio-video renderers can turn any parenthesized, open-vocabulary behavior description into synchronized, identity-consistent 640×368 video at 25 FPS without introducing a new latency-critical path.

What would settle it

A concrete test: prompt the model with a directive that requires sustained object interaction, such as 'picks up the mug, drinks, and places it back,' and measure whether the mug's identity and position remain consistent across frames and whether model-side latency stays at about 200 ms. If open-vocabulary actions break object consistency, or latency grows with directive complexity, the central claim of expanded expressive range at unchanged latency fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

A single world-event pretraining objective trained on ordinary video can be specialized to different real-time tasks by changing only the world context and the stream-driving input, such as world-model control, first-person interaction, or embodied manipulation.
The agent's event stream can carry open-vocabulary behavior described in natural language, not just a fixed action set, and this behavior is grounded in the configured scene and character.
Interleaving behavior directives with speech in one token stream lets their timing and ordering be learned jointly rather than scheduled externally, and adds negligible token cost.
Because the modeling contract and serving topology are unchanged from the prior version, the same resolution and latency operating point—640×368 at 25 FPS, about 200 ms model-side latency—is preserved.
The decomposition treats recorded video and interactive experience as the same kind of object, making the pretraining data source abundant and scalable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the decomposition generalizes, a world-event model pretrained on video could serve as a shared world model across multiple embodied agents; the paper demonstrates only the audio-visual interaction instance, so this is a reasonable extrapolation rather than a demonstrated result.
The no-new-latency claim depends on behavior directives being short and realizable by the existing decoder; a stress test with long or physically complex directives might reveal capacity or latency limits that the qualitative observations do not cover.
Pretraining quality hinges on the quality of inferred world summaries and event alignments from raw video; noisy annotations would cap the transferable world knowledge even if the factorization is correct.
The interleaved format could extend to multi-character scenes with explicit actor-tagged events, but the paper only shows single-character examples; this extension is implied but not tested.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A clearly written, honest version note whose central claim — world-event pretraining transfers to real-time tasks — is explicitly not measured; the only concrete new element is open-vocabulary behavior via parenthesized directives.

read the letter

You should know two things. First, this is the most honest version note I've read in a while: it says outright that the core factorization "is not a new architecture" and that systematic evaluation is "left to future work." Second, that honesty sits on top of a load-bearing gap — the central claim is never measured. The abstract promises a general-purpose pretraining task that transfers; the paper delivers a factorization and qualitative observations.

What it does well: Eq. (3) is a clean decomposition, and the paper is right that naming the persistent context W is a useful organizational device. The role-play format with parenthesized behavior directives — "(picks up the mug and glances toward the window) Sure, one moment." — is the one concrete new element: free-form, open-vocabulary behavior carried in the language token stream, conditioning audio-video generation without adding a latency-critical path. That is a plausible and testable design choice. The version comparison in Table 1 is neat, and the writing is crisp throughout.

The soft spots are exactly where the reader puts them. Section 2.3 states the general-purpose pretraining objective and then defers evaluation; Section 5 reports "qualitative behavior observations" with no metric, no comparison against v0.2, and no test of whether a directive like "reaches into the grass and picks up a green leaf" is actually rendered faithfully, synchronously, and identity-consistently. So the headline contribution — transferable competence from world-event pretraining — rests entirely on assertion. That is an evidentiary gap, not an internal inconsistency; you can't say the derivation is wrong, only that the paper doesn't test what it claims. The open-vocabulary behavior point is also assumed, not demonstrated: the decoder must realize arbitrary parenthesized actions, and nothing here shows that holds at 640x368/25 FPS.

Who is it for? Practitioners who want a compact description of how a production streaming model organizes its context, and anyone studying how industrial teams frame world models. As a research preprint, it does not substantiate its headline claim — it is a version note with a good conceptual paragraph, not a research result.

I would desk reject at a research venue, or accept as a non-archival systems note. If a referee does see it, the single most useful ask is: measure the transfer claim.

Referee Report

3 major / 4 minor

Summary. The paper proposes a conceptual reframing of video generation as a 'world plus event stream' decomposition. It formalizes the world W as a structured record and events e_k as time-localized records, and factorizes the autoregressive objective in Eq. (3). The authors claim this yields a general-purpose pretraining task over large amounts of ordinary video, producing competence transferable to real-time downstream tasks. They instantiate the framework on real-time full-duplex audio-visual interaction, introducing free-form parenthesized behavior directives in the agent's language stream, and preserve the v0.2 resolution and latency. The experimental section reports only qualitative behavior observations and explicitly defers systematic adaptation and evaluation to future work.

Significance. If validated, the world+event framing could provide a unified perspective on video pretraining and downstream transfer, and the open-vocabulary behavior interface is a useful extension of the v0.2 agent. The paper is clearly written and honest about its scope, explicitly noting that the factorization is not a new architecture. However, the central empirical claims are not tested: there are no quantitative results, no comparison with v0.2, no transfer experiment, and no measurement of behavior-directive fulfillment. The contribution is currently a conceptual reframing rather than a demonstrated method. Eq. (3) is mathematically valid but tautologically true for any sequence split into context and changes; it does not establish the claimed generalization. The paper does provide a concrete instantiation and preserves latency, which is a meaningful engineering constraint, but the headline claim remains unsupported.

major comments (3)

[Section 2.3, Section 5] The abstract claims: 'This yields a general-purpose pretraining task over large amounts of real video ... The resulting competence can be specialized to a broad family of real-time downstream tasks.' Yet Section 2.3 explicitly states: 'This paper studies only the real-time full-duplex audio-visual interaction specialization and leaves systematic adaptation and evaluation of the others to future work.' Section 5 reports only 'Qualitative behavior observations' with no quantitative metrics, no comparison to v0.2, no measurement of behavior-directive fulfillment, and no transfer experiment. The central claim of transferable competence is therefore completely unsupported by empirical evidence. This is load-bearing: the contribution rests on the claimed transfer, not merely on the factorization.
[Section 3.2] The claim that 'no new latency-critical path is introduced' depends on the assumption that open-vocabulary behavior directives (e.g., '(reaches into the grass and picks up a green leaf)') can be realized by the same audio-video decoders that previously conditioned on language. No evidence is provided that the pretrained decoder can render arbitrary parenthesized directives into synchronized, identity-consistent video without additional capacity or a new conditioning mechanism. If open-vocabulary actions require extra model capacity or a dedicated action-conditioning path, the expressive-range claim fails even if the world-event decomposition is valid.
[Eq. (3), Section 2.2] The factorization in Eq. (3) is a rewriting of the v0.1 autoregressive objective with W named explicitly; the paper itself notes that 'The v0.1 autoregressive factorization is the special case in which W is folded into the history.' The substantive claim that pretraining on inferred worlds and events produces reusable world-event dynamics is not established by the equation. Moreover, the procedure for inferring W and event labels from raw video is unspecified—there is no description of annotation, automatic labeling, or supervision source—so the pretraining objective is not concretely defined. This underspecification prevents replication and weakens the method section.

minor comments (4)

[Section 5] The first paragraph refers to 'the same response boundary as v0.1/v0.2' without defining it. Since this is a self-contained paper, the boundary should be restated or explicitly referenced.
[Section 3.3] The statement that 'Behavior directives are short language tokens, so they add negligible cost to the token-causal path' is an assertion without analysis. A complexity estimate or a pointer to measurements would make this claim checkable.
[Figure 1] The time interval labels in Figure 1 are visually dense and ambiguous, with overlapping brackets. Consider a cleaner layout or a table to separate event types and time intervals.
[Conclusion] The conclusion states that the model 'supports general-purpose video pretraining and transfer' without the qualifier 'in principle,' whereas Section 2.3 explicitly defers evaluation. The wording should be softened to match the paper's actual evidence.

Circularity Check

1 steps flagged

The 'world-event pretraining' objective is Eq. (3), which the paper admits is the v0.1 autoregressive factorization with W folded into history; the central transfer claim is untested.

specific steps

renaming known result [Section 2.2, Eq. (3); Section 2.3; Table 1]
"The v0.1 autoregressive factorization is the special case in which W is folded into the history, x is the user observation, and e_k is the agent’s text/audio/video response. ... This is not a new architecture: it is a reading of the same interleaved causal sequence in which durable conditioning is named W and time-localized change is named e_k."

The advertised general-purpose pretraining task is Eq. (3), but the paper itself states that Eq. (3) is the v0.1 autoregressive factorization with W folded into history. Thus 'given a world and incoming input, predict how the world moves, changes, and responds' is exactly the existing next-token/flow-matching objective repackaged by naming W and e_k. No independent loss, data pipeline, or transfer measurement is provided (Sec. 2.3 defers evaluation; Sec. 5 is qualitative only), so the claim that the decomposition 'yields' a new general-purpose competence reduces to a relabeling of the prior objective rather than a derived consequence.

full rationale

The paper does not contain a hidden fitted parameter called a prediction, nor an imported uniqueness theorem from self-citations. Its one identifiable circular move is the announcement of a 'general-purpose world-event pretraining' objective: the formal objective is Eq. (3), which the text immediately identifies as the v0.1 autoregressive factorization with W folded into the history. Hence the headline 'given a world and incoming input, predict how the world moves' is a relabeling of the existing causal next-token/flow-matching objective; calling it world-event pretraining adds nomenclature rather than a new loss or derivation. The paper is transparent about this ('This is not a new architecture'), which mitigates the severity, and the instantiation contains independent ingredients (open-vocabulary behavior in a role-play format, application to general video corpora). However, the claimed transfer competence is not measured anywhere: Sec. 2.3 explicitly defers systematic evaluation of other tasks to future work, and Sec. 5 reports only qualitative observations. That is an evidentiary gap rather than circularity; under the instructions, absence of evidence was not itself scored as circularity. Weighted: one central renaming step, honestly disclosed, with no additional load-bearing self-citation chain — overall partial circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 3 invented entities

The paper reports no fitted numbers; the runtime specifications are inherited operating-point choices. The central scientific assumptions are domain assumptions about representability and transfer.

free parameters (2)

Streaming unit length = 160 ms
Fixed operating-point choice inherited from v0.2; not fitted to data in this paper.
Bidirectional network budget = 350 ms
External deployment assumption used to compute total latency; not a learned parameter.

axioms (5)

standard math Standard probability chain rule allows p(e1:K | W,x) = ∏ p(ek | W,x≤k,e<k).
Equation 3 in §2.2 is a routine factorization; accepted.
domain assumption Every natural video can be represented as a persistent world W and time-aligned event records e_k, and event annotations can be inferred at scale.
§2.1/§2.3 and Figure 1 assume this representability and annotation pipeline; no details are given.
domain assumption Pretraining to predict next events on ordinary video yields transferable real-time interaction competence.
§2.3 asserts this and admits evaluation is future work; it is the load-bearing transfer claim.
domain assumption The v0.2 serving topology and causal decoders remain valid unchanged so latency is preserved.
§4 and §5 state latency is inherited from v0.2 and 'network term is an external deployment assumption'; no runtime measured here.
standard math Conditional flow matching for continuous audio-video latents (referenced from v0.1) is an accepted generative method.
Used in §2.2; not derived in this paper.

invented entities (3)

World record W = {w_j} no independent evidence
purpose: Names persistent context (scene, character, voice, ambient sound) as explicit conditioning object.
A notational construct; no falsifiable handle outside the paper.
Event record e_k = (τ_k, c_k, d_k) no independent evidence
purpose: Names time-localized changes/behavior; defines pretraining supervision.
A data-format construct; no independent evidence that these records can be produced at scale or that predicting them transfers.
Parenthesized behavior directives in agent language stream no independent evidence
purpose: Open-vocabulary action channel that conditions audio-video generation.
New output format; no evaluation of whether decoded video realizes directives.

pith-pipeline@v1.3.0-alltime-deepseek · 6672 in / 12835 out tokens · 133474 ms · 2026-08-02T00:19:57.999570+00:00 · methodology

0 comments

read the original abstract

We present Wan-Streamer v0.3, which reframes our native-streaming interaction model under a single organizing view: a video is a world plus an event stream. The world is the persistent context in which a video unfolds, including the environment, scene, subjects, ambient acoustic conditions, voice characteristics, and other relatively stable conditions. The event stream is everything that changes over time within that world, including scene or environmental changes, subject behavior, speech, and other sounds. This yields a general-purpose pretraining task over large amounts of real video: given a world and incoming input, predict how the world moves, changes, and responds in real time. The resulting competence can be specialized to a broad family of real-time downstream tasks. We instantiate it on real-time full-duplex audio-visual interaction, where the event stream is the agent's speech together with free-form behavior. Functionally, the model's multimodal understanding process is vision-language-action-like: it maps multimodal user input to language-form speech and behavior actions. Wan-Streamer v0.3 preserves the v0.2 operating point: 640x368 video at 25 FPS, a 160 ms streaming unit, approximately 200 ms model-side response latency, and approximately 550 ms total interaction latency under a 350 ms bidirectional network budget.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention
cs.CV 2026-07 conditional novelty 6.0

A fixed multi-scale patch/attention schedule matched to each frame's noise level makes streaming video diffusion ~40% faster, while homogeneous-noise-level DMD improves training-inference consistency.

Reference graph

Works this paper leans on

15 extracted references · 8 linked inside Pith · cited by 1 Pith paper

[1]

Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models, June 2026

Lianghua Huang, Zhi-Fan Wu, Wei Wang, Yupeng Shi, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, and Zoubin Bi. Wan-Streamer v0.1: End-to-end Real-time Interactive Foundatio...

Pith/arXiv arXiv 2026
[2]

Wan-Streamer v0.2: Higher Resolution, Same Latency, July 2026

Lianghua Huang, Zhi-Fan Wu, Yupeng Shi, Wei Wang, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuxiang Bao, Yuzheng Wang, and Zoubin Bi. Wan-Streamer v0.2: Higher Resoluti...

Pith/arXiv arXiv 2026
[3]

Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

Pith/arXiv arXiv 2023
[4]

Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025

Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025

Pith/arXiv arXiv 2025
[5]

Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

Pith/arXiv arXiv 2026
[6]

360explorer: Exploring 4d controllable world in panoramic videos

Xinhua Cheng, Haiyang Zhou, Wangbo Yu, Tanghui Jia, Bin Lin, Yunyang Ge, Weiqi Li, and Li Yuan. 360explorer: Exploring 4d controllable world in panoramic videos. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 3300–3308, 2026

2026
[7]

Realwonder: Real-time physical action-conditioned video generation.arXiv preprint arXiv:2603.05449, 2026

Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, and Jiajun Wu. Realwonder: Real-time physical action-conditioned video generation.arXiv preprint arXiv:2603.05449, 2026

arXiv 2026
[8]

Body of her: A preliminary study on end-to-end humanoid agent.arXiv preprint arXiv:2408.02879, 2024

Tenglong Ao. Body of her: A preliminary study on end-to-end humanoid agent.arXiv preprint arXiv:2408.02879, 2024

Pith/arXiv arXiv 2024
[9]

Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

Pith/arXiv arXiv 2026
[10]

Flowact-r1: Towards interactive humanoid video generation.arXiv preprint arXiv:2601.10103, 2026

Lizhen Wang, Yongming Zhu, Zhipeng Ge, Youwei Zheng, Longhao Zhang, Tianshu Hu, Shiyang Qin, et al. Flowact-r1: Towards interactive humanoid video generation.arXiv preprint arXiv:2601.10103, 2026

arXiv 2026
[11]

Doubao realtime voice model.https://seed.bytedance.com/en/realtime_voice, 2025

ByteDance Seed Team. Doubao realtime voice model.https://seed.bytedance.com/en/realtime_voice, 2025. Model page, January 20, 2025

2025
[12]

Hello gpt-4o.https://openai.com/index/hello-gpt-4o/, 2024

OpenAI. Hello gpt-4o.https://openai.com/index/hello-gpt-4o/, 2024. Blog post, May 13, 2024

2024
[13]

Introducing evi 3: The world’s most realistic and instructible speech-language model.https://www

Hume AI. Introducing evi 3: The world’s most realistic and instructible speech-language model.https://www. hume.ai/blog/introducing-evi-3, 2025. Blog post, 2025

2025
[14]

Streamavatar: Streaming diffusion models for real-time interactive human avatars.arXiv preprint arXiv:2512.22065, 2025

Zhiyao Sun, Ziqiao Peng, Yifeng Ma, Yi Chen, Zhengguang Zhou, Zixiang Zhou, Guozhen Zhang, Youliang Zhang, Yuan Zhou, Qinglin Lu, and Yong-Jin Liu. Streamavatar: Streaming diffusion models for real-time interactive human avatars.arXiv preprint arXiv:2512.22065, 2025

arXiv 2025
[15]

Hallo-live: Real-time streaming joint audio-video avatar generation with asynchronous dual-stream and human-centric preference distillation.arXiv preprint arXiv:2604.23632, 2026

Chunyu Li, Jiaye Li, Ruiqiao Mei, Haoyuan Xia, Hao Zhu, Jingdong Wang, and Siyu Zhu. Hallo-live: Real-time streaming joint audio-video avatar generation with asynchronous dual-stream and human-centric preference distillation.arXiv preprint arXiv:2604.23632, 2026. 7 Appendix A Contributions and Acknowledgements A.1 Core Contributors Lianghua Huang∗, Zhi-Fa...

Pith/arXiv arXiv 2026

[1] [1]

Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models, June 2026

Lianghua Huang, Zhi-Fan Wu, Wei Wang, Yupeng Shi, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, and Zoubin Bi. Wan-Streamer v0.1: End-to-end Real-time Interactive Foundatio...

Pith/arXiv arXiv 2026

[2] [2]

Wan-Streamer v0.2: Higher Resolution, Same Latency, July 2026

Lianghua Huang, Zhi-Fan Wu, Yupeng Shi, Wei Wang, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuxiang Bao, Yuzheng Wang, and Zoubin Bi. Wan-Streamer v0.2: Higher Resoluti...

Pith/arXiv arXiv 2026

[3] [3]

Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhandari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models.arXiv preprint arXiv:2309.14509, 2023

Pith/arXiv arXiv 2023

[4] [4]

Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025

Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025

Pith/arXiv arXiv 2025

[5] [5]

Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

Pith/arXiv arXiv 2026

[6] [6]

360explorer: Exploring 4d controllable world in panoramic videos

Xinhua Cheng, Haiyang Zhou, Wangbo Yu, Tanghui Jia, Bin Lin, Yunyang Ge, Weiqi Li, and Li Yuan. 360explorer: Exploring 4d controllable world in panoramic videos. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 3300–3308, 2026

2026

[7] [7]

Realwonder: Real-time physical action-conditioned video generation.arXiv preprint arXiv:2603.05449, 2026

Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, and Jiajun Wu. Realwonder: Real-time physical action-conditioned video generation.arXiv preprint arXiv:2603.05449, 2026

arXiv 2026

[8] [8]

Body of her: A preliminary study on end-to-end humanoid agent.arXiv preprint arXiv:2408.02879, 2024

Tenglong Ao. Body of her: A preliminary study on end-to-end humanoid agent.arXiv preprint arXiv:2408.02879, 2024

Pith/arXiv arXiv 2024

[9] [9]

Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

Pith/arXiv arXiv 2026

[10] [10]

Flowact-r1: Towards interactive humanoid video generation.arXiv preprint arXiv:2601.10103, 2026

Lizhen Wang, Yongming Zhu, Zhipeng Ge, Youwei Zheng, Longhao Zhang, Tianshu Hu, Shiyang Qin, et al. Flowact-r1: Towards interactive humanoid video generation.arXiv preprint arXiv:2601.10103, 2026

arXiv 2026

[11] [11]

Doubao realtime voice model.https://seed.bytedance.com/en/realtime_voice, 2025

ByteDance Seed Team. Doubao realtime voice model.https://seed.bytedance.com/en/realtime_voice, 2025. Model page, January 20, 2025

2025

[12] [12]

Hello gpt-4o.https://openai.com/index/hello-gpt-4o/, 2024

OpenAI. Hello gpt-4o.https://openai.com/index/hello-gpt-4o/, 2024. Blog post, May 13, 2024

2024

[13] [13]

Introducing evi 3: The world’s most realistic and instructible speech-language model.https://www

Hume AI. Introducing evi 3: The world’s most realistic and instructible speech-language model.https://www. hume.ai/blog/introducing-evi-3, 2025. Blog post, 2025

2025

[14] [14]

Streamavatar: Streaming diffusion models for real-time interactive human avatars.arXiv preprint arXiv:2512.22065, 2025

Zhiyao Sun, Ziqiao Peng, Yifeng Ma, Yi Chen, Zhengguang Zhou, Zixiang Zhou, Guozhen Zhang, Youliang Zhang, Yuan Zhou, Qinglin Lu, and Yong-Jin Liu. Streamavatar: Streaming diffusion models for real-time interactive human avatars.arXiv preprint arXiv:2512.22065, 2025

arXiv 2025

[15] [15]

Hallo-live: Real-time streaming joint audio-video avatar generation with asynchronous dual-stream and human-centric preference distillation.arXiv preprint arXiv:2604.23632, 2026

Chunyu Li, Jiaye Li, Ruiqiao Mei, Haoyuan Xia, Hao Zhu, Jingdong Wang, and Siyu Zhu. Hallo-live: Real-time streaming joint audio-video avatar generation with asynchronous dual-stream and human-centric preference distillation.arXiv preprint arXiv:2604.23632, 2026. 7 Appendix A Contributions and Acknowledgements A.1 Core Contributors Lianghua Huang∗, Zhi-Fa...

Pith/arXiv arXiv 2026