Pith.

Computer Vision and Pattern Recognition · arXiv:2610.01762 v1 · submitted 1 Oct 2026 · 47 references

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction

Xiangyu Zeng, Yuandong Yang, Zhiqiu Zhang, Yuhan Zhu, Xinhao Li, Qingyi Si, Dingyu Yao, Changlian Ma

show 16 more authors

Haoran Chen, Xinyu Chen, Yansong Shi, Junhao Zhou, Yifei Li, Jun Zhang, Chuanyu Qin, Chenxu Yang, Xinlei Yu, Kun Ouyang, Yuchen Shao, Qianshan Wei, Changhai Zhou, Jun Gao, Jiaqi Wang, Limin Wang

The paper says

“Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception.”

more of the abstract

We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events. Streaming caption targets supervise the interpretation of observed video prefixes during training. At inference, model-generated records complement a recent visual window, providing reusable factual context without revisiting historical visual features. Proactive State Transition Learning (PSTL) reduces the dominance of repeated waiting states by preserving supervision at all output anchors and selecting representative state-change and state-persistence tokens. We further develop a streaming data synthesis pipeline that aligns output content and timing with available evidence. Combining the resulting streaming captions and QA with cleaned open-source data yields OneStreamer-1M, a broad-coverage streaming video interaction dataset with over one million records spanning diverse tasks. Our 4B model achieves the best results among the compared methods across all eight evaluated streaming video understanding benchmarks. Ablations show that retaining generated captions improves historical QA without degrading real-time perception. PSTL also outperforms dense state supervision while supervising only 27.5% of annotated state tokens. Together, these results support proactive generation as a shared learning interface connecting perception, memory formation, and timely response in streaming video interaction.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 2 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. The mechanism is real and the ablations are the strongest part; the 'best on all eight' headline is contingent on decontamination that is currently unverifiable. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 47 references say.

The paper says, at Appendix B.1, Table 6, Table 1

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The central headline—best results on all eight benchmarks (abstract, [b8],[b40]; Table 1)—rests on the assertion in B.1 that 'any training sample whose source video appears in a benchmark test set is removed, including clips taken from non-overlapping temporal intervals of the same video.'

more of objection 1

This is an assertion, not a demonstrated result, and no code, checkpoints, or dataset artifacts are released to check it. The concern is not generic train-test overlap: Table 6 shows OneStreamer-1M aggregates training data from Streamo (which feeds ProactiveVQA/OVO-Timing/OmniMMI), ViSpeak, MMDuet2, StreamForest (which feeds ODVBench), and VideoChat-OL (which feeds OVBench), i.e., several evaluated benchmarks draw from the same source works. The additional claim that non-overlapping temporal clips of the same source video are removed requires reliable same-video identification, whose procedure is not described. If tracing is incomplete, shared content would inflate margins on several benchmarks simultaneously, independent of whether PHCM and PSTL work. Please provide a quantitative decontamination audit: the matching procedure, the number of training samples removed per source work and per benchmark, and the rate of near-duplicate detection, ideally with a released overlap-check script. The mechanism claims in Tables 2, 3, and 17 are protected by their matched controls, but the headline claim currently cannot be verified.

The paper says, at Appendix D, Section 5.1, Table 12

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The ProactiveVideoQA result relies on an automated judge: 'we evaluate ProactiveVideoQA at 2 fps and use Qwen3-235B-A22B as the automated judge.' For comparison models with previously reported results, numbers are taken directly from the respective papers (Appendix D), which may have used different judges or PAUC protocols, so the ProactiveVQA comparison is not necessarily apples-to-apples.

more of objection 2

Separately, for re-evaluated models the paper reports the higher of two benchmark-level averages from two evaluation settings. This is conservative toward the baselines (it cannot inflate OneStreamer's relative margin), but it is unusual and should be justified: report both settings for each re-evaluated model, and state explicitly whether the judge model and the judge prompt are held constant across OneStreamer and all re-evaluated baselines on ProactiveVideoQA.

The paper says, at Table 3, Section 5.2 Finding 3

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The PSTL claim is stated as 'outperforms dense state-token supervision,' but the dense cross-entropy baseline (26.1 on ProactiveVQA) is far below the dense focal-loss baseline (47.0), which is within 1.7 points of PSTL's 48.7. The text in Finding 3 does acknowledge that focal loss 'substantially outperforms' CE under dense supervision, but the abstract's phrasing ('PSTL also outperforms dense state supervision') is stronger than what Table 3 supports against the best dense alternative.

more of objection 3

This does not invalidate the contribution—PSTL does beat the focal baseline on all three proactive benchmarks—but the claim should be sharpened to name the dense-focal comparison explicitly in the abstract and conclusion for accuracy.

4 minor objections

The paper says, at Table 2

Pith says: minor objection 1 of 4

The claim in Finding 2 that PHCM 'exceeds Full's memory performance' is supported only on ASI (71.6 vs 67.6); on EPM, PHCM is slightly below Full (62.6 vs 63.0). The sentence in §5.2 already notes this, but the conclusion's phrasing 'improves historical QA without degrading real-time perception' should likewise acknowledge the small EPM shortfall for precision.

The paper says, at Tables 8 and 15

Pith says: minor objection 2 of 4

Several cells are missing value separators, e.g., Table 8 '75.861.3' and Table 15 '55.0065.0082.0055.00' and '4.104.63'. Please insert spaces or alignment separators, as these are currently difficult to read and partly ambiguous.

show 2 more

The paper says, at Figure 1 and Section 5.1

Pith says: minor objection 3 of 4

The benchmark is named 'Vispeak' in Figure 1 and as 'ViSpeak' elsewhere in the text; please normalize the spelling. Also, Figure 1's legend ('light purple portions indicating the Qwen3-VL baseline') is not evident from the figure as rendered.

The paper says, at Appendix F.3

Pith says: minor objection 4 of 4

The failure case (Fig. 12) honestly documents that the model does not correct an earlier incorrect response when later evidence reveals an error. This is an acknowledged limitation, and I do not treat it as a finding; I note it here only to encourage stating it in the main text's limitations discussion as well, since it bears on the 'evidence-grounded' claims.

What would settle it

Release the training-to-source-video provenance table (or deduplication logs) for OneStreamer-1M, and run an independent near-duplicate video check — e.g., DINOv2 or perceptual-hash features with a reported similarity threshold — between every training video and the test video sets of the four same-source benchmarks (ViSpeak, ODVBench, OVBench, and OVO-Timing/ProactiveVQA source sets). Report the overlap count and threshold. If overlap is zero, the decontamination claim is substantiated and the headline stands; if nonzero, recompute all eight benchmark margins with overlapping training samples excluded and report whether OneStreamer still leads on each.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 2 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 37 s · 31,632 tokens read, 5,670 written
  3. Editor wrote the verdict above 16 s · 32,822 tokens read, 2,495 written
  4. Referee major revision: 3 major objections, 4 minor 43 s · 33,823 tokens read, 1,888 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The mechanism work is genuinely good, and the headline is the only load-bearing piece that wobbles.

What's new: PHCM replaces distant visual history with model-generated, time-grounded text captions plus coarser event summaries. That is a real departure from prior memory work (Flash-Vstream, Streamo's state tokens) — query-independent memory writing inside the causal generation loop. PSTL is a smaller but concrete refinement: quota-based selective supervision of state tokens, preserving output anchors while capping repeated waiting states. The OneStreamer-1M corpus with calibrated response times is a useful resource even setting the results aside.

The ablations are the strong part, and Table 17 is the one that earns the causal claim. Same fact-recording prompt at inference, but the model trained without proactive caption supervision gets essentially zero memory gain from its own generated descriptions (-0.1 and +0.3), while OneStreamer gets +2.0 and +0.6. That separates supervision from mere mechanism — the right control. Table 2 holds checkpoint and window fixed and shows PHCM beating both Full and FIFO on aggregate; Table 3 holds data and text supervision fixed across state-selection strategies with a supervision-matched random baseline at the same 27.5% ratio. These are well-designed, and PSTL's gains survive them. The authors also disclose a genuine failure case in F.3 rather than cherry-picking. Credit where due: this is careful work.

Now the soft spots, in proportion. The mechanism claims hold up. The "best on all eight" headline does not, and the stress-test note is right: B.1's decontamination is load-bearing for those margins. Four of the eight benchmarks draw from the same source pipelines whose training data sits inside OneStreamer-1M, and the appendix asserts source-video-level removal including non-overlapping clips — that requires reliable same-video identification across clips, which is a substantial engineering task, and there are no code, checkpoints, or dataset artifacts to check it. If tracing is incomplete, several margins inflate simultaneously, independent of whether the architecture works. Believe the mechanism; treat the headline as contingent.

The other issues are milder than the reader's list suggests. Single-run with no error bars is field convention — minor. The benchmark-specific settings are disclosed, including a documented policy of reporting the higher of two settings for re-evaluated models at whole-benchmark level; disclosed and reasonably fair, though not a standard I'd want abused. No released artifacts, with synthesis on closed models, is the real second-order concern.

Anyone working on streaming video LLMs or online memory should read this; the Table 17 control alone justifies it. It deserves a serious referee — accept for review, with decontamination verification and artifact release as the conditions before the headline is taken at face value.

The strongest objection

The central headline — best results among compared methods on all eight benchmarks (b8, b40, b124) — is load-bearing on the completeness of the source-video-level decontamination asserted in Appendix B.1: 'Any training sample whose source video appears in a benchmark test set is removed, including clips taken from non-overlapping temporal intervals of the same video.' This is an assertion, not a demonstrated result, and no code, checkpoints, or dataset artifacts are released to check it. What makes this more than a generic training-test overlap worry is that four of the eight evaluated benchmarks come from the very works whose training data is inside OneStreamer-1M: ViSpeak (benchmark [6], training from ViSpeak-Instruct, Table 6), ODVBench (StreamForest [39], 337,570 spatial and 92,283 temporal-grounding records), OVBench (VideoChat-OL [8], AVA spatial QA), plus Streamo/MMDuet2 data feeding ProactiveVQA/OVO-Timing/OmniMMI. For YouTube-Collected and pipeline-derived examples, tracing every benchmark test video back to its identity is a substantial engineering task, and the additional claim that non-overlapping clips of the same source video are removed requires reliable same-video identification, not just clip-level matching. If tracing is incomplete, shared content would inflate margins on several benchmarks simultaneously, independent of whether PHCM and PSTL work. The mechanism claims are better protected: Table 2's same-checkpoint FIFO/PHCM comparison, Table 17's retention-without-supervision control, and Table 3's supervision-matched random baseline are internally controlled and would not be explained away by contamination. But the headline 'best on all eight' and the specific margin numbers (e.g., 72.1 vs 65.3 on OVOBench, 2.87 vs 2.48 on ViSpeak) are exactly what the unverified decontamination protects. The authors' honesty about the failure case (F.3) does not resolve this gap.

The reader's weakest assumption — completeness of source-video-level decontamination (B.1) — is, in my reading, the single most load-bearing condition for the headline claim. The training corpus visibly incorporates training data from ViSpeak [6], StreamForest [39], VideoChat-OL [8], and Streamo [28], four works whose own benchmarks (ViSpeak, ODVBench, OVBench, and contributors to ProactiveVQA/OVO-Timing) are among the eight evaluated. The only protection against same-source leakage is the asserted source-video-level removal, which is neither demonstrated nor independently checkable because no artifacts are released. I confirm the reader's identification and add the structural point that this is not a generic contamination worry: the overlap is structurally plausible for these specific benchmark-sourced pairs, which is why it deserves an explicit provenance check. I do not find a separate, stronger concern. The mechanism claims rest on genuinely controlled ablations (same-checkpoint FIFO vs PHCM in Table 2, the retention-without-supervision control in Table 17, and the supervision-matched random baseline in Table 3), and I found no internal inconsistency in the PSTL quota construction (Eq. 1–2) or its claimed supervision ratio of 27.5%. The remaining weaknesses the reader lists — single-run numbers with 1–5 point gaps, benchmark-specific evaluation settings for OneStreamer versus native baselines, and the use of closed models in data synthesis — are real but secondary; they weaken confidence in the exact margins, not the mechanism. The CONDITIONAL verdict is the right calibration: the mechanism claims are well supported, while the headline should not be taken at face value until the decontamination is independently verifiable. My assessment does not move the verdict.

The referee's report

If the headline results hold, the paper makes a useful architectural contribution: treating proactive generation as a shared interface for perception, memory, and response timing is a clean idea, and the ablations are unusually well controlled. Table 2 compares FIFO vs PHCM with the same checkpoint and Recent-16 window; Table 3 holds training data, input sequences, and text supervision fixed while varying only state-token selection; Table 17 separates caption-supervision benefit from memory-at-inference by using each model's own descriptions as memory. The 27.5% supervision ratio is an emergent consequence of the PSTL quota (Eq. 1–2), not a tuned parameter. These controlled demonstrations are the paper's strongest asset and largely support the mechanism claims. The residual risk is concentrated in the 'best on all eight' headline, which depends on the completeness of the source-video-level decontamination asserted in Appendix B.1, for which no artifacts are released.

Pith's reading of the paper (a machine's summary, not the authors')

OneStreamer is a four-billion-parameter streaming video language model built on one idea: a model that watches an unbounded stream should write down, as it goes, factual records that may matter to questions that have not been asked yet, and it should learn that recording and the answering itself through the same act of generation. The paper claims this shared proactive generation interface lets a single model form reusable factual memory without giving up real-time perception, and it reports the best scores among the compared methods on all eight streaming-video benchmarks it evaluates. The two load-bearing pieces are a caption memory that turns observed moments into time-grounded local descriptions and event summaries, and a selective supervision scheme for the control tokens that say when to keep watching, when to record, and when to respond. Why this matters: video arrives with no script of what will be asked, and a mechanism that keeps evidence in cheap text while only ever looking at recent frames is the kind a wearable or live assistant could actually run at scale.

What carries the argument. Two mechanisms carry the argument. Proactive Hierarchical Caption Memory (PHCM) is the memory-writing half: during training, causal streaming caption targets teach the model to interpret each observed video prefix, and at inference the model generates and retains time-grounded local-detail captions (</Observe>) and event-level summaries (</Summary>) that supplement a Recent-N FIFO window of visual tokens, so historical evidence is never revisited through old visual features. Proactive State Transition Learning (PSTL) is the timing half: it supervises the control tokens marking continued observation (</Silence>), emerging-but-insufficient evidence (</Standby>), and output generation (</Response>), preserving supervision at all output anchors and capping frequent transition groups to a per-sequence quota $q = \max_{X}\max_{a \in A_\tau} n_{X\to a}$, so repeated silence labels cannot dominate the state loss while state persistence still receives supervision. A third piece supplies the data: a streaming synthesis pipeline that aligns each caption and QA target's content and release time with the evidence available at that moment, yielding the OneStreamer-1M corpus of more than a million records. The claim being tested is that generation itself can serve as the shared substrate for perception, memory formation, and response timing.

Core claim. The paper claims that query-independent evidence recording and task response can be learned jointly through a single causal generation process, and that this unification is what lets a streaming video LLM retain distant facts while staying current. Concretely, the model writes two kinds of time-stamped records — dense local captions introduced by the </Observe> token and sparser semantic summaries of completed events introduced by </Summary> — appends them to its text history as their source frames leave a Recent-N visual window, and answers later questions using those records alongside a small window of recent frames. On a controlled comparison, retaining these generated records raises the OVOBench-Backward ASI from 63.5 to 71.6 while holding real-time perception steady, which the paper reads as evidence that proactive caption generation is genuinely query-independent memory writing. The paper further claims that its selective state supervision, PSTL, beats dense supervision on proactive-response benchmarks while supervising only 27.5% of annotated state tokens, because it keeps supervision at every output anchor and samples representative state-change and state-persistence positions rather than drowning in repeated silence labels. The result reported is a 4B model that scores highest among the compared methods on all eight evaluated benchmarks, with an average relative gain of 25.0% over its Qwen3-VL base and 6.1% over the strongest competing method on each benchmark.

The premise it rests on. The load-bearing premise is that the source-video-level decontamination described in Appendix B.1 is complete: because the training corpus absorbs data from several of the same projects that contributed the eight evaluated benchmarks, "best on all eight" holds only if every training sample whose source video appears in a benchmark test set was truly removed, including clips from non-overlapping temporal intervals of the same video.

If the paper is right (Pith's inferences, not the authors')

A streaming model can beat both a full-visual-history model and a recent-window-only model on the same hardware budget: PHCM exceeds FIFO on real-time perception (OVOBench Real-Time 80.9 to 81.4, StreamingBench 86.3 to 86.9) while surpassing the full-history model on memory (ASI 71.6 vs 67.6), so the paper reads the result as the best of both rather than a compromise.

Selective supervision beats dense supervision for response timing: PSTL at 27.5% supervised state tokens outperforms 100% dense cross-entropy, the focal loss used by a prior streaming method, and a supervision-matched random baseline, implying that which tokens are supervised matters more than how many.

Caption supervision helps perception even when the captions are not used at inference: removing the 60K proactive caption examples lowers offline perception scores across all six evaluation settings in the paper's ablation, so the practice of recording itself sharpens interpretation.

The approach scales on a live stream: on a 360-second clip, PHCM cuts context tokens by 93.1%, GPU memory by 60.4%, and time-to-first-token from 4.560 s to 0.124 s relative to full visual history, while keeping online memory construction under the 1 s per-update budget.

Because a shared generation process writes captions, answers, and control states alike, the same architecture handles memory QA, perception QA, and proactive QA from one set of learned distributions, which is the paper's stated reason the results transfer across all eight benchmarks.

The paper's own failure case (Appendix F.3) shows the model does not revise an inaccurate description once later frames reveal the error; extending the same generation interface with an explicit self-correction state is a natural next step the paper does not take.

Because the training corpus aggregates data from the very works whose benchmarks are being scored, the "best on all eight" margins depend on the completeness of the source-video decontamination; a holdout variant that excludes those datasets' videos entirely would test whether the gains survive without any possible train-test overlap.

PHCM stores evidence as compact text rather than visual tokens, so the same idea could port to agents that must archive days or weeks of footage, where caption text itself would eventually need summarization or eviction policies at coarser granularities.

The response-time calibration (advancing responses by 13.72 s on average, with 86.62% of records made earlier) reads as a generally usable recipe for any streaming task where "when to answer" can be learned from a likelihood or similarity criterion, independent of these particular benchmarks.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception. We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events. Streaming caption targets supervise the interpretation of observed video prefixes during training. At inference, model-generated records complement a recent visual window, providing reusable factual context without revisiting historical visual features. Proactive State Transition Learning (PSTL) reduces the dominance of repeated waiting states by preserving supervision at all output anchors and selecting representative state-change and state-persistence tokens. We further develop a streaming data synthesis pipeline that aligns output content and timing with available evidence. Combining the resulting streaming captions and QA with cleaned open-source data yields OneStreamer-1M, a broad-coverage streaming video interaction dataset with over one million records spanning diverse tasks. Our 4B model achieves the best results among the compared methods across all eight evaluated streaming video understanding benchmarks. Ablations show that retaining generated captions improves historical QA without degrading real-time perception. PSTL also outperforms dense state supervision while supervising only 27.5% of annotated state tokens. Together, these results support proactive generation as a shared learning interface connecting perception, memory formation, and timely response in streaming video interaction.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-02

The authors' main claim, as the classifier picked it out 1.00

“Together, these results support proactive generation as a shared learning interface connecting perception, memory formation, and timely response in streaming video interaction.”

How the main claim is stated: asserted 0.81

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.13 · human participants 0.04 · controlled comparison 0.72 · randomized assignment 0.01 · states quantitative result 0.98 · compares with prior results 0.96 · mentions code or data availability 0.10

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. background 0.98 0.07 0.08 asserted 1.00
The challenge is to form reusable factual memory without compromising real-time perception. background 0.68 0.05 0.03 asserted 0.99
We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. contribution 0.96 0.94 0.11 asserted 1.00
Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of completed events. method 0.87 0.89 0.05 asserted 1.00
Streaming caption targets supervise the interpretation of observed video prefixes during training. method 0.99 0.79 0.02 asserted 0.98
At inference, model-generated records complement a recent visual window, providing reusable factual context without revisiting historical visual features. method 0.97 0.65 0.09 asserted 1.00
Proactive State Transition Learning (PSTL) reduces the dominance of repeated waiting states by preserving supervision at all output anchors and selecting representative state-change and state-persistence tokens. method 0.91 0.94 0.05 asserted 1.00
We further develop a streaming data synthesis pipeline that aligns output content and timing with available evidence. method 0.92 0.95 0.03 asserted 0.99
Combining the resulting streaming captions and QA with cleaned open-source data yields OneStreamer-1M, a broad-coverage streaming video interaction dataset with over one million records spanning diverse tasks. method 0.56 0.96 0.04 asserted 1.00
Our 4B model achieves the best results among the compared methods across all eight evaluated streaming video understanding benchmarks. result 1.00 0.97 0.20 asserted 1.00
Ablations show that retaining generated captions improves historical QA without degrading real-time perception. result 1.00 0.96 0.10 asserted 1.00
PSTL also outperforms dense state supervision while supervising only 27.5% of annotated state tokens. result 1.00 0.96 0.06 asserted 1.00
Together, these results support proactive generation as a shared learning interface connecting perception, memory formation, and timely response in streaming video interaction. interpretation 1.00 0.75 0.88 asserted 0.81

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.95

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-02: 1 request, 9,371 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

References (47)
  1. [1] Shehreen Azad, Vibhav Vineet, and Yogesh S Rawat. Streamready: Learning what to answer and when in long streaming videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 40494–40504, 2026 on Pith
  2. [2] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025 on PithPith's reviewarXiv
  3. [3] Joya Chen, Zhaoyang Lv, Shiwei Wu, Kevin Qinghong Lin, Chenan Song, Difei Gao, Jia-Wei Liu, Ziteng Gao, Dongxing Mao, and Mike Zheng Shou. Videollm-online: Online video large language model for streaming video. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognit... on Pith
  4. [4] Joya Chen, Ziyun Zeng, Yiqi Lin, Wei Li, Zejun Ma, and Mike Zheng Shou. Livecc: Learning video llm with streaming speech transcription at scale. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 29083–29095. IEEE, 2025 on Pith
  5. [5] Xin Ding, Hao Wu, Yifan Yang, Shiqi Jiang, Qianxi Zhang, Donglin Bai, Zhibo Chen, and Ting Cao. Streammind: Unlocking full frame rate streaming video dialogue through event-gated cognition. In2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 13448–13459. ... on Pith
  6. [6] Shenghao Fu, Qize Yang, Yuan-Ming Li, Yi-Xing Peng, Kun-Yu Lin, Xihan Wei, Jian-Fang Hu, Xiaohua Xie, and Wei-Shi Zheng. Vispeak: Visual instruction feedback in streaming videos. In2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 21778–21788. IEEE, 2025 on Pith
  7. [7] Yiran Guan, Liang Yin, Dingkang Liang, Jianzhong Ju, Zhenbo Luo, Jian Luan, Yuliang Liu, and Xiang Bai. Video streaming thinking: Videollms can watch and think simultaneously. InEuropean Conference on Computer Vision, pages 80–99. Springer, 2026 on Pith
  8. [8] Zhenpeng Huang, Xinhao Li, Jiaqi Li, Jing Wang, Xiangyu Zeng, Cheng Liang, Tao Wu, Xi Chen, Liang Li, and Limin Wang. Online video understanding: Ovbench and videochat-online. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3328–3338. IEEE, 2025 on Pith
  9. [9] Wei Li, Bing Hu, Rui Shao, Leyang Shen, and Liqiang Nie. Lion-fs: Fast & slow video-language thinker as online video assistant. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3240–3251. IEEE, 2025 on Pith
  10. [10] Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, et al. Videochat-flash: Hierarchical compression for long-context video modeling. InInternational Conference on Learning Representations, volume 2026, pages... on Pith
  11. [11] Xinhao Li, Yuhan Zhu, Xiangyu Zeng, Yuhao Dong, Haoning Wu, Zhiqiu Zhang, Yuandong Yang, Changlian Ma, Qingyu Zhang, Yansong Shi, et al. Videochat3: Fully open video mllm for efficient and generalist video understanding.arXiv preprint arXiv:2607.14935, 2026 on PithPith's reviewarXiv
  12. [12] Junming Lin, Zheng Fang, Chi Chen, Haoxuan Cheng, Zihao Wan, Fuwen Luo, Ziyue Wang, Peng Li, Yang Liu, and Maosong Sun. Streamingbench: Assessing the gap for mllms to achieve streaming video understanding. InICASSP 2026-2026 IEEE International Conference on Acoustics, Speech a... on Pith
  13. [13] Zikang Liu, Longteng Guo, Handong Li, Ru Zhen, Xingjian He, Ruyi Ji, Xiaoming Ren, Yanhao Zhang, Haonan Lu, and Jing Liu. Thinking in streaming video.arXiv preprint arXiv:2603.12938, 2026. 10 on PitharXiv
  14. [14] Xudong Lu, Yang Bo, Jinpeng Chen, Shuhan Li, Xintong Guo, Huankang Guan, Fang Liu, Dunyuan Xu, Peiwen Sun, Heyang Sun, et al. Aura: Always-on understanding and real-time assistance via video streams.arXiv preprint arXiv:2604.04184, 2026 on PithPith's reviewarXiv
  15. [15] Junbo Niu, Yifei Li, Ziyang Miao, Chunjiang Ge, Yuanhang Zhou, Qihao He, Xiaoyi Dong, Haodong Duan, Shuangrui Ding, Rui Qian, et al. Ovo-bench: How far is your video-llms from real-world online video understanding? In2025 IEEE/CVF Conference on Computer Vision and Pattern Reco... on Pith
  16. [16] Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reaction. In2025 IEEE/CVF Conference on Computer Vision and Pattern ... on Pith
  17. [17] Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrish- nan Varadarajan, Florian Bordes, et al. Longvu: Spatiotemporal adaptive compression for long video-language understanding. arXiv preprint arXiv:2410.17434, 2024 on PithPith's reviewarXiv
  18. [18] Yujiao Shen, Shulin Tian, Jingkang Yang, and Ziwei Liu. A simple baseline for streaming video understanding.arXiv preprint arXiv:2604.02317, 2026 on PitharXiv
  19. [19] Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition... on Pith
  20. [20] Chao Wang, Xudong Tan, Jianjian Cao, Kangcong Li, and Tao Chen. Curvestream: boosting streaming video understanding in mllms via curvature-aware hierarchical visual memory management. InEuropean Conference on Computer Vision, pages 605–623. Springer, 2026 on Pith
show 27 more
  1. [21] Haibo Wang, Bo Feng, Zhengfeng Lai, Mingze Xu, Shiyu Li, Weifeng Ge, Afshin Dehghan, Meng Cao, and Ping Huang. Streambridge: Turning your offline video large language model into a proactive streaming assistant.Advances in Neural Information Processing Systems, 38:132332–132359, 2025 on Pith
  2. [22] Lu Wang, Zhuoran Jin, Yupu Hao, Yubo Chen, Kang Liu, Yulong Ao, and Jun Zhao. Think while watching: Online streaming segment-level memory for multi-turn video reasoning in multimodal large language models.arXiv preprint arXiv:2603.11896, 2026 on PitharXiv
  3. [23] Pengyu Wang, Chenkun Tan, Shaojun Zhou, Qirui Zhou, Yanxin Chen, Xingyang He, Huazheng Zeng, Jijun Cheng, Chenghao Wang, Xiaomeng Qian, et al. Moss-vl technical report.arXiv preprint arXiv:2608.15045, 2026 on PithPith's reviewarXiv
  4. [24] Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvideo2.5: Empowering video mllms with long and rich context modeling.arXiv preprint arXiv:2501.12386, 2025 on PithPith's reviewarXiv
  5. [25] Yueqian Wang, Xiaojun Meng, Yifan Wang, Huishuai Zhang, and Dongyan Zhao. Proactivevideoqa: A comprehensive benchmark evaluating proactive interactions in video large language models.arXiv preprint arXiv:2507.09313, 2025 on PithPith's reviewarXiv
  6. [26] Yueqian Wang, Songxiang Liu, Disong Wang, Nuo Xu, Guanglu Wan, Huishuai Zhang, and Dongyan Zhao. Mmduet2: Enhancing proactive interaction of video mllms with multi-turn reinforcement learning. InInternational Conference on Learning Representations, volume 2026, pages 23257–23270, 2026 on Pith
  7. [27] Yuxuan Wang, Yueqian Wang, Bo Chen, Tong Wu, Dongyan Zhao, and Zilong Zheng. Omnimmi: A comprehensive multi-modal interaction benchmark in streaming video contexts. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18925–18935. IEEE, 2025 on Pith
  8. [28] Jiaer Xia, Peixian Chen, Mengdan Zhang, Xing Sun, and Kaiyang Zhou. Streaming video instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 31219–31229, 2026 on Pith
  9. [29] Ming Xie, Zizheng Huang, Xudong Tan, Chao Wang, Xiangyu Zeng, Wenxiao Wu, Tao Chen, Limin Wang, and Yanwei Fu. Streamov: Streaming omni-video understanding via evidence-guided memory and response triggering.arXiv preprint arXiv:2605.25621, 2026 on PithPith's reviewarXiv
  10. [30] Yiweng Xie, Bo He, Junke Wang, Xiangyu Zheng, Ziyi Ye, and Zuxuan Wu. Fluxmem: Adaptive hierarchical memory for streaming video understanding.arXiv preprint arXiv:2603.02096, 2026 on PitharXiv
  11. [31] Ruyi Xu, Guangxuan Xiao, Yukang Chen, Liuning He, Kelly Peng, Yao Lu, and Song Han. Streamingvlm: Real-time understanding for infinite video streams. InInternational Conference on Learning Representations, volume 2026, pages 61463–61475, 2026. 11 on Pith
  12. [32] Weicai Yan, Yuhong Dai, Qi Ran, Haodong Li, Wang Lin, Tao Jin, Xing Xie, Hao Liao, and Jianxun Lian. Proact-vl: A proactive videollm for real-time ai companions.arXiv preprint arXiv:2603.03447, 2026 on PithPith's reviewarXiv
  13. [33] Haolin Yang, Feilong Tang, Lingxiao Zhao, Xinlin Zhuang, Yifan Lu, Xiang An, Ming Hu, Xiaofeng Zhang, Abdalla Swikir, Junjun He, et al. Streamagent: Towards anticipatory agents for streaming video understanding.arXiv preprint arXiv:2508.01875, 2025 on PithPith's reviewarXiv
  14. [34] Senqiao Yang, Kaichen Zhang, Zhaoyang Jia, Jinghao Guo, Yifei Shen, Xinjie Zhang, Xiaoyi Zhang, Haoqing Wang, Xiao Li, Peng Zhang, et al. Mage-vl: An efficient codec-native streaming multimodal foundation model.arXiv preprint arXiv:2607.24904, 2026 on PithPith's reviewarXiv
  15. [35] Zhenyu Yang, Kairui Zhang, Yuhang Hu, Bing Wang, Shengsheng Qian, Bin Wen, Fan Yang, Tingting Gao, Weiming Dong, and Changsheng Xu. Livestar: Live streaming assistant for real-world online video understanding.Advances in Neural Information Processing Systems, 38:31266–31304, 2025 on Pith
  16. [36] Dingyu Yao, Junhao Zhou, Chenxu Yang, Chuanyu Qin, Haowen Hou, Zheming Liang, Congcong Wang, Yuhang Cao, Shenglong Ye, Shuai Xie, et al. Joyai-vl-interaction: Real-time vision-language interaction intelligence.arXiv preprint arXiv:2606.14777, 2026 on PithPith's reviewarXiv
  17. [37] Linli Yao, Yicheng Li, Yuancheng Wei, Lei Li, Shuhuai Ren, Yuanxin Liu, Kun Ouyang, Lean Wang, Shicheng Li, Sida Li, et al. Timechat-online: 80% visual tokens are naturally redundant in streaming videos. InProceedings of the 33rd ACM International Conference on Multimedia, pag... on Pith
  18. [38] Xiangyu Zeng, Kunchang Li, Chenting Wang, Xinhao Li, Tianxiang Jiang, Ziang Yan, Songze Li, Yansong Shi, Zhengrong Yue, Yi Wang, et al. Timesuite: Improving mllms for long video understanding via grounded tuning. InInternational Conference on Learning Representations, volume 2... on Pith
  19. [39] Xiangyu Zeng, Kefan Qiu, Qingyu Zhang, Xinhao Li, Jing Wang, Jiaxin Li, Ziang Yan, Kun Tian, Meng Tian, Xinhai Zhao, et al. Streamforest: Efficient online video understanding with persistent event memory.Advances in Neural Information Processing Systems, 38:75804–75835, 2025 on Pith
  20. [40] Xiangyu Zeng, Zhiqiu Zhang, Yuhan Zhu, Xinhao Li, Zikang Wang, Changlian Ma, Qingyu Zhang, Zizheng Huang, Kun Ouyang, Tianxiang Jiang, et al. Video-o3: Native interleaved clue seeking for long video multi-hop reasoning.arXiv preprint arXiv:2601.23224, 2026 on PithPith's reviewarXiv
  21. [41] Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-vstream: Memory-based real-time understanding for long video streams.arXiv preprint arXiv:2406.08085, 2024 on PithPith's reviewarXiv
  22. [42] Jialiang Zhang, Junlong Tong, Junyan Lin, Hao Wu, Yirong Sun, Yunpu Ma, and Xiaoyu Shen. Think-as-you-see: Streaming chain-of-thought reasoning for large vision-language models.arXiv preprint arXiv:2603.02872, 2026 on PitharXiv
  23. [43] Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision.arXiv preprint arXiv:2406.16852, 2024 on PithPith's reviewarXiv
  24. [44] Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Llava-video: Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024 on PithPith's reviewarXiv
  25. [45] Yikai Zheng, Xin Ding, Yifan Yang, Shiqi Jiang, Hao Wu, Qianxi Zhang, Weijun Wang, Ting Cao, and Yunxin Liu. Em-garde: A propose-match framework for proactive streaming video understanding.arXiv preprint arXiv:2603.19054, 2026 on PithPith's reviewarXiv
  26. [46] Junjie Zhou, Ke Mei, Lei Li, Tianyi Wang, Fengyun Rao, and Jing Lyu. Wemm-embedding: Wechat multi-modal embedding technical report.arXiv preprint arXiv:2608.24053, 2026 on PithPith's reviewarXiv
  27. [47] Xingyi Zhou, Anurag Arnab, Shyamal Buch, Shen Yan, Austin Myers, Xuehan Xiong, Arsha Nagrani, and Cordelia Schmid. Streaming dense video captioning. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18243–18252. IEEE, 2024. 12 Appendix Content... on Pith
Cited by 8 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment