Pith.

Computer Vision and Pattern Recognition · arXiv:2610.32332 v1 · submitted 23 Oct 2026 · 61 references

4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents

Yanhao Jia, Yi Zhu, Yiran Zhong, Weigao Sun, Feida Zhu, Pengkun Jiao, Sihan Yang, Ming Ma

show 2 more authors

Steven Hoi, Erik Cambria

The paper says

“Humans efficiently process multimodal streams spanning hours to years, building persistent, fine-grained spatiotemporal memories that support reasoning about and predicting long-horizon events.”

more of the abstract

In contrast, existing benchmarks focus on short- and medium-context tasks, and most foundation models struggle to preserve fine-grained information over long horizons. To evaluate and improve long-horizon spatiotemporal reasoning, we introduce the LongVideo dataset and the 4DMemoryBench. LongVideo comprises about 900 hours of video and over 5,000 manually reviewed questions covering visual, textual, and spatiotemporal reasoning across diverse context lengths. We further develop the 4DMemory Harness, which integrates four complementary skills, 22+ specialized tool models, and a model-specific retrieval-augmented generation system. Evaluating 13 leading models, we find that standalone foundation models struggle with long-horizon tasks, while the 4DMemory Harness improves all of them by over 50%. Compared to previous benchmark, the same models exhibit a performance drop of more than 10% on 4DMemoryBench. We hope 4DMemory will facilitate the development of spatiotemporal agents capable of addressing complex, long-horizon tasks.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 26 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Does not hold up as stated. The benchmark is a genuine step past prior long-video evaluation, but the paper's headline harness claim is not supported by its own experiments. This line is wrong

4 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 61 references say.

The paper says, at Abstract, Contribution 3, Table 2, Figure 2, S4.4

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.94):

“The results charac- terize the evaluated reasoner–observer–tool sys- tems under the stated metric; without matched standalone-model runs or component ablations, they do not measure a causal gain from the har- ness or identify a unique failure mechanism.”

Pith says: Objection 1 of 4 · major

The claim that the 4DMemory Harness 'improves all of them by over 50%' (abstract, b5; Contribution 3, b44) is not an experimental result reported in this paper. Table 2 contains only harness-conditioned joint scores; there is no matched standalone-model run on the same questions anywhere in the paper or supplement.

more of objection 1

Figure 2 compares 4DMemory scores against published Video-MME/LVBench scores, which are different questions, different judging protocols, and different access conditions, so it cannot serve as the baseline. Section S4.4 says directly: 'without matched standalone-model runs or component ablations, they do not measure a causal gain from the harness.' The paper also never defines what input a standalone model would receive for a 97.98-hour question, so the comparison is undefined as well as unreported. This is load-bearing because the same missing comparison underlies the 'standalone models struggle' narrative and the paper's central quantitative message. The authors must either report matched no-harness baselines (with the standalone input protocol specified) or remove/reframe the improvement claim.

The paper says, at Abstract, Section 2.2, S2.1, S2.2, S5.2

The review did not quote this passage.

Pith says: Objection 2 of 4 · major

The dataset's validity rests on 5,214 reference answers produced by an automated pipeline: a generator composes questions from clip summaries and ASR transcripts, a verifier checks correctness and clip necessity, and a single automatic quality threshold filters records. The abstract's phrase 'over 5,000 manually reviewed questions' is contradicted by this description: no human annotation counts, no human agreement statistics on …

more of objection 2

… reference answers or evidence necessity, and no validation of the verifier are reported, and S5.2 lists 'source-level human verification' as future work. Similarly, both judging protocols are LLM-based, and Section S4.4 acknowledges that inter-judge agreement is not quantified. Since every reported model score is computed against these references, a material error rate in the reference answers would invalidate the benchmark's difficulty and all model rankings. The paper needs at least a human-verified sample (answer correctness and evidence necessity) with agreement statistics, and the abstract's wording should be corrected to match the actual pipeline.

The paper says, at Section 2.2 vs. Section 2.4, S2.3

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

The harness's memory documents are constructed from the same clip summaries, ASR transcripts, scene metadata, and speaker-labelled dialogue that serve as the evidence cards for QA generation (Section 2.2; S2.2). Retrieval therefore searches text that literally contains the evidence used to compose the questions, which gives the harness a privileged alignment with the annotation pipeline that an independently constructed memory would not have.

more of objection 3

This does not make the benchmark circular (the questions are externally grounded in video content), but it is a correctness risk for the claim that harness scores reflect general memory and retrieval skill. A concrete test would be to measure how often reference answers are recoverable from the memory documents alone without any video observation, or to compare retrieval success against memory built from independent annotations. The paper currently contains no such leakage analysis.

The paper says, at Contribution 1, Table 1, Section 2.1

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

Contribution 1 claims 4DMemoryBench is 'the first multimodal video benchmark in which an individual question can bind more than 100 hours of ordered source video,' but Table 1 and Section 2.1 both state that the maximum question scope is 97.98 hours (the complete-series scope).

more of objection 4

A 97.98-hour scope is not more than 100 hours, so the headline novelty claim contradicts the paper's own numbers. This needs to be corrected in one direction or the other, and the '850+ hours' figure in Contribution 1 should also be reconciled with the 887.480-hour total stated in Section 2.1.

4 minor objections

The paper says, at Sections 2.2 and 2.3

Pith says: minor objection 1 of 4

Sections 2.2 and 2.3 are near-duplicates: both are titled 'Compositional QA construction' and repeat the same content with minor variations. One of them should be removed or folded into the other.

The paper says, at S2.3, Section 2.1, Table 3 discussion

Pith says: minor objection 2 of 4

Several presentation errors need fixing: the S2.3 heading reads 'Harnees Details'; Section 2.1 has 'Appdendix Figure S1'; the Table 3 discussion says entries are 'provisional, as marked in 3' with a dangling reference to the table number; and Section S4.1 asserts a difference between group sizes that are stated as identical ('36 and 24 here, compared with 36 and 24 in the earlier release').

show 2 more

The paper says, at Abstract, Contribution 1, Table 1, S2

Pith says: minor objection 3 of 4

The corpus size is reported inconsistently as 887.480 hours (S2), approximately 887 hours (Table 1), '850+ hours' (Contribution 1), and 'approximately 900 hours' (abstract), and the video count is given as 'over 1,000 standalone long videos' (Introduction) versus 1,025 standalone videos (S2). These figures should be harmonized.

The paper says, at S3.1, S3.4

Pith says: minor objection 4 of 4

Reproducibility is limited because the exact API endpoint identifiers and resolved reasoner/observer routes are said to be preserved only in run manifests that are not included with the paper or supplement; the authors should provide the manifests or their hashes.

What would settle it

Run matched no-harness baselines for all 13 models on the same 5,214-question snapshot: use the same media observer and input pathway, the same Opus-5 and three-judge protocols, and the same S4.1 denominators, but disable memory retrieval, video_memory_query, and all three skill tools so the model answers directly from the bounded source input. Compute per-model relative gain (Overall_harness - Overall_standalone)/Overall_standalone; the abstract claim requires every model to exceed 50%. A minimal decisive version: run this for GPT-5.6 Sol and Gemini 3.7-Flash, which have complete judging under both protocols; if either gain is below 50%, the 'all 13 models' claim fails.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 26 Oct 2026

  1. Reader does not hold up confidence moderate
  2. Skeptic partly agrees 1 min 8 s · 28,292 tokens read, 10,056 written
  3. Editor wrote the verdict above 34 s · 29,196 tokens read, 4,118 written
  4. Referee reject: 4 major objections, 4 minor 1 min 2 s · 28,623 tokens read, 9,432 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Read 4DMemory. The benchmark is a real resource: 887 hours, 5,214 QA pairs, and a per-question temporal scope up to 97.98 hours, which is genuinely past prior long-video benchmarks. The TV-series pathway—episode/season/series composition, deterministic programs, timestamped evidence chains, and the separation of question-independent memory from question-conditioned retrieval—is well designed and clearly described. The paper is also unusually candid: the supplementary says the results do not measure a causal harness gain.

The problem is that candid sentence sits in the appendix. The abstract and contribution 3 claim the harness improves all 13 models by over 50%. That claim has no matched comparison anywhere. Table 2 reports only harness-conditioned scores. Figure 2 is a cross-benchmark comparison, not standalone-versus-harness on the same questions. S4.4 states: without matched standalone-model runs or component ablations, the results do not measure a causal gain from the harness. So the headline result as written is unsupported by the paper's own experiments. That is load-bearing because the "standalone models struggle" narrative rests on the same missing baseline, and "standalone" is never defined for a 97.98-hour question.

Second soft spot: the reference answers and judging are LLM-mediated, with no human agreement statistics. The generator, verifier, and judges are documented; none is validated. The reader's weakest assumption is pointed in the right direction: if a material fraction of reference answers are wrong, ambiguous, or shortcut-prone, every reported model score is contaminated. The authors acknowledge automated annotation and model-based judging introduce uncertainty, but they do not quantify it. The memory documents are also built from the same clip summaries, ASR, and scene metadata used to generate the QA pairs, so the harness has a privileged alignment with the annotation pipeline. That does not invalidate the benchmark as a difficulty probe, but it blocks clean attribution of its gains to general memory and retrieval.

Minor but real: some comparisons rest on tiny groups—four complete-series visual questions, five-question max/min subtypes—which the authors flag and treat correctly. The corpus is two sitcoms plus web videos, narrower than the title's "long-horizon spatiotemporal memory" suggests.

Who this is for: people building long-video benchmarks or agentic retrieval systems. The design ideas are reusable, and the task-level signals—ordering beats counting, scope interacts with task composition—are useful even if absolute numbers are provisional.

Recommendation: send it to peer review. The dataset and harness concept deserve referee time. The referees need matched standalone baselines, human-judge agreement numbers, and either support for or removal of the 50 percent claim before it can stand as stated.

The strongest objection

The most load-bearing problem is that the paper's central quantitative claim is not an experimental result in the paper. The abstract (b5) and contribution 3 (b44) assert that the 4DMemory Harness improves all 13 models by over 50%. Table 2, however, reports only harness-conditioned scores: every column is computed from runs in which retrieval, memory, and skill tools were active, and the standalone scores the comparison requires are absent. Figure 2's Video-MME/LVBench comparison is cross-benchmark and cannot serve as a matched no-harness baseline. The supplementary discussion (S4.4, b523) states: 'without matched standalone-model runs or component ablations, they do not measure a causal gain from the harness.' That sentence directly contradicts the abstract. Moreover, the paper never specifies what a standalone model is allowed to see on a 97.98-hour question, so the comparison is undefined as well as unreported. This is load-bearing because the same missing comparison underlies the 'standalone models struggle' narrative and every downstream interpretation of the reported scores.

The reader's weakest assumption is reference-answer reliability, and that concern is genuine: with no human agreement statistics or verifier validation, the benchmark scores inherit annotation risk. But for the central claim as stated, the missing matched baseline is more load-bearing because the paper itself disavows causal attribution while the abstract asserts it. Even a perfectly annotated benchmark would not support 'harness improves all 13 models by over 50%' without matched no-harness runs on the same questions. This is not a request to re-prove a cited result; it is a request for the controlled experiment that the reported tables omit. The construction details, dual judging protocols, explicit snapshot denominators, and candid limitations are strengths, and the 97.98-hour per-question scope is a real contribution. However, the main empirical claim is unsupported as written, so the reader's REJECT stands; I would not downgrade to CONDITIONAL because the missing baseline is the central result rather than a peripheral gap. My primary concern differs from the reader's stated weakest assumption, hence partial agreement.

The referee's report

If substantiated, the benchmark would fill a genuine gap: no existing video benchmark binds multi-evidence questions to tens of hours of ordered source video, and the four-metric decomposition (Invocation, Ordered, Order-relaxed, Overall) is a useful way to separate tool use from reasoning outcome. The paper is also unusually candid: Section S4.4 explicitly disclaims causal attribution of the reported scores, Section S5.1 lists the main limitations, small sample groups are flagged, and the operational detail in S2.3-S3.3 (chunk sizes, retrieval budgets, cache provenance) supports replication. The counting-versus-ordering gap is a plausible and interesting finding. However, the two headline claims are not supported as written: the 50% improvement claim has no matched comparison anywhere in the paper, and the 'more than 100 hours' claim contradicts the stated 97.98-hour maximum scope. The benchmark's validity additionally rests on an automated QA pipeline with no reported human validation, which is a serious gap for a measurement instrument.

Pith's reading of the paper (a machine's summary, not the authors')

The paper's project is to make long-horizon spatiotemporal memory — remembering and reasoning over events separated by dozens of hours of footage — both measurable and improvable. It claims that 4DMemoryBench, built from roughly 887 hours of video (1,025 standalone clips and 402 episodes of two television series) with 5,214 question–answer pairs, is the first multimodal video benchmark in which a single question can bind more than 100 hours of ordered source video, concretely spanning up to 97.98 hours across 278 episodes. It further claims that the 4DMemory Harness, a model-agnostic layer combining question-independent persistent memory, bounded retrieval, and specialized tools, lifts all 13 evaluated foundation models by over 50% relative to their standalone performance, while the same models exhibit a performance drop of more than 10% on 4DMemoryBench relative to earlier benchmarks. The evaluation also separates tool invocation from ordered execution from correct answers, and finds distributed counting across separated scenes to be the consistent bottleneck, with the best score at 47.58%. If these claims hold, the field gains a measurement instrument for long-horizon video reasoning that did not exist before, and a retraining-free way to raise any foundation model's performance on it.

What carries the argument. The load-bearing mechanism has two halves. First, evidence-chain questions: each item bundles ordered source videos with private annotations (reference answer, supporting evidence, optional executable program), and questions are admitted only if they combine two to four clips through an explicit operation, survive a verifier that removes each supporting clip to check joint necessity, and pass clarity, evidence-sufficiency, compositionality, difficulty, and shortcut screens. Second, the 4DMemory Harness: an inference-time shell organized around the principle of task-independent storage and goal-directed recall, in which a multimodal observer converts media into persistent documents carrying source identity and timestamps (summary, transcript, on-screen text, entities, actions, events, spatial relations), a TF-IDF retriever with one-unit temporal-neighbor expansion selects up to 12 seed documents within a text budget, and the reasoner can issue further video_memory_query calls and invoke the video-processing, visual-grounding, and text-statistics skills. Four metrics — Invocation, Ordered, Order-relaxed, and Overall — carry the argument that initiating tool use, satisfying ordered execution, and producing a correct answer are separable achievements, which is why the gap between near-universal invocation and a 77.79% best joint score is the paper's main diagnostic result.

Core claim. On the authors' own terms, the central claim is that long-horizon 4D reasoning — the "4D" being space plus time — is a systems property, not a context-window property, and that it can be engineered. The benchmark is constructed so that answering a question requires composing two to four pieces of evidence scattered in time: the standalone pathway generates candidates from time-ordered clip cards using explicit operations such as comparison, temporal ordering, state change, cross-modal binding, and distributed counting, and a verifier checks each answer and tests each clip's necessity by removing it in turn; the television pathway derives reference answers from deterministic programs over dialogue, ASR, and clip-level visual statistics at episode, season, and series scope. The harness then implements task-independent storage and goal-directed recall: an observer writes timestamped, question-independent memory documents; at question time a lexical retriever with temporal-neighbor expansion supplies bounded evidence; and three skills (video processing, SAM 3.1-based visual grounding, and deterministic text statistics) together with iterative video_memory_query calls support the reasoning model under execution budgets. Evaluated on all 5,214 questions by 13 API-served models, the paper reports that standalone foundation models struggle, that every model improves by over 50% when harness-equipped, and that tool invocation (at least 98.7% for 12 of 13 systems) far outstrips joint success (best 77.79%), with distributed counting the universal weak point. The authors are careful to frame the scores as characterizing the reasoner–observer–tool system as a whole, noting that without matched standalone-model runs the results do not isolate a causal gain from the harness.

The premise it rests on. The load-bearing premise is that the 5,214 reference answers and their evidence chains — produced by an automated generator–verifier pipeline filtered by a single quality threshold, with no human-annotation statistics or verifier validation reported in the paper — are correct and unambiguous; if a material share are wrong, shortcut-prone, or ambiguous, every reported model score and the benchmark's claimed difficulty are invalid, which is why the paper's own concession that automated annotation and model-based judging introduce uncertainty matters.

If the paper is right (Pith's inferences, not the authors')

If the benchmark measures what it claims, long-video ability should be reported per question scope rather than per context window: a question's evidence can span about 98 hours while the model only ever sees bounded retrieved evidence.

The harness's over-50% improvement across all 13 models, if it replicates, makes long-horizon video reasoning an achievable plug-in capability for any API-served foundation model without fine-tuning or model-internal changes.

The universal counting deficit — every system scores worse on distributed counting than on ordering, with a best of 47.58% — gives developers a concrete target: quantity-aware memory with explicit temporal provenance rather than summary-based recall.

The separation of Invocation from Overall means evaluations of agentic video systems must score tool use, execution order, and final answers separately, or they will overstate practical readiness.

Rankings shift sharply across episode, season, and series scopes, so leaderboards built on this benchmark should report per-scope, per-subtype scores instead of a single aggregate.

Editorial extension: the authors explicitly leave agent fine-tuning unexplored, so the most direct test of the benchmark as a training resource would be to fine-tune a representative model on the 5,214 QA pairs and their evidence chains and probe on unseen videos and series.

Editorial extension: because the paper itself notes the absence of matched standalone-model runs and component ablations, a clean ablation (memory construction alone, retrieval alone, tools alone) would show which part of the harness carries the reported over-50% gain.

Editorial extension: the reported judge sensitivity (one model swings from 79.17% to 70.83% on complete-series spatiotemporal questions depending on the judging protocol) suggests ranks on this benchmark will depend on the judge, and publishing matched item-level judge votes would let comparisons be re-derived fairly.

Editorial extension: if the serialized-question pipeline generalizes beyond sitcoms, the same construction could be applied to continuous streams such as egocentric recordings or fixed cameras, directly testing the paper's stated future direction of memory updating and entity persistence.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Humans efficiently process multimodal streams spanning hours to years, building persistent, fine-grained spatiotemporal memories that support reasoning about and predicting long-horizon events. In contrast, existing benchmarks focus on short- and medium-context tasks, and most foundation models struggle to preserve fine-grained information over long horizons. To evaluate and improve long-horizon spatiotemporal reasoning, we introduce the LongVideo dataset and the 4DMemoryBench. LongVideo comprises about 900 hours of video and over 5,000 manually reviewed questions covering visual, textual, and spatiotemporal reasoning across diverse context lengths. We further develop the 4DMemory Harness, which integrates four complementary skills, 22+ specialized tool models, and a model-specific retrieval-augmented generation system. Evaluating 13 leading models, we find that standalone foundation models struggle with long-horizon tasks, while the 4DMemory Harness improves all of them by over 50%. Compared to previous benchmark, the same models exhibit a performance drop of more than 10% on 4DMemoryBench. We hope 4DMemory will facilitate the development of spatiotemporal agents capable of addressing complex, long-horizon tasks.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-26

The authors' main claim, as the classifier picked it out 0.99

“Evaluating 13 leading models, we find that standalone foundation models struggle with long-horizon tasks, while the 4DMemory Harness improves all of them by over 50\%.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.11 · states limitation 0.20 · human participants 0.30 · controlled comparison 0.74 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.09

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Humans efficiently process multimodal streams spanning hours to years, building persistent, fine-grained spatiotemporal memories that support reasoning about and predicting long-horizon events. background 1.00 0.05 0.03 asserted 1.00
In contrast, existing benchmarks focus on short- and medium-context tasks, and most foundation models struggle to preserve fine-grained information over long horizons. gap 0.94 0.07 0.04 asserted 0.99
To evaluate and improve long-horizon spatiotemporal reasoning, we introduce the LongVideo dataset and the 4DMemoryBench. contribution 1.00 0.93 0.07 asserted 0.98
LongVideo comprises about 900 hours of video and over 5,000 manually reviewed questions covering visual, textual, and spatiotemporal reasoning across diverse context lengths. method 0.68 0.92 0.03 asserted 1.00
We further develop the 4DMemory Harness, which integrates four complementary skills, 22+ specialized tool models, and a model-specific retrieval-augmented generation system. method 0.84 0.95 0.05 asserted 1.00
Evaluating 13 leading models, we find that standalone foundation models struggle with long-horizon tasks, while the 4DMemory Harness improves all of them by over 50\%. result 1.00 0.97 0.61 asserted 1.00
Compared to previous benchmark, the same models exhibit a performance drop of more than 10\% on 4DMemoryBench. result 1.00 0.90 0.13 asserted 1.00
We hope 4DMemory will facilitate the development of spatiotemporal agents capable of addressing complex, long-horizon tasks. future work 0.99 0.07 0.10 hedged possibility 0.61

What kind of paper the classifier reports this is: dataset or benchmark 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-26: 1 request, 6,348 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. Motivation for 4DMemory Agent. (a) Traditional models with limited active context may lose access to
Figure 2 of the paper
Figure 2. Comparison of previous Video-MME (top) and LVBench (bottom) scores with 4DMemory results for the same models. All models shown score lower on 4DMemory, with dashed arrows indicating the decrease in percentage points. context windows exceeding 1M tokens, these set￾tings alone do not establish whether models can match human abilities to extract and retain fine￾grained evidence, or preserve, integrate, and use key information after receiving tens of hours of additional video. Recent work has begun to eval￾uate cross-day recall from egocentric video (Chen et al., 2026), but its source…
Figure 3 of the paper
Figure 3. Overview of the LongVideo construction pipeline. Hours-level videos undergo audio-text screening, clip
Figure 4 of the paper
Figure 4. Illustration of 4DMemory Agent framework. (a) The harness builds persistent, question-independent

Figures are the authors', reproduced from the paper.

References (61)
  1. [1] Encoding specificity and retrieval processes in episodic memory. , author=. Psychological review , volume=. 1973 , publisher= on Pith
  2. [2] Reconsolidation of episodic memories: A subtle reminder triggers integration of new information , author=. Learning & memory , volume=. 2007 , publisher= on Pith
  3. [3] Longvideobench: A benchmark for long-context interleaved video-language understanding , author=. Advances in Neural Information Processing Systems , volume= on Pith
  4. [4] Hourvideo: 1-hour video-language understanding , author=. Advances in Neural Information Processing Systems , volume= on Pith
  5. [5] Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
  6. [6] Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents , author=. arXiv preprint arXiv:2609.29892 , year= on PithPith's reviewarXiv
  7. [7] Seeing sound, hearing sight: Uncovering modality bias and conflict of ai models in sound localization , author=. Advances in neural information processing systems , volume= on Pith
  8. [8] Uni-retrieval: A multi-style retrieval framework for stem’s education , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  9. [9] From query to explanation: Uni-rag for multi-modal retrieval-augmented learning in stem , author=. arXiv preprint arXiv:2507.03868 , year= on PithPith's reviewarXiv
  10. [10] Towards Spatial Reasoning and Understanding via Modeling Modality Conflict, Bias and Alignment , author=. researchgate , year= on Pith
  11. [11] MMOE: Modernizing Diffusion Transformers with Efficient Expert Design , author=. arXiv preprint arXiv:2607.24665 , year= on PithPith's reviewarXiv
  12. [12] Vulnerabilities and Defenses in Audio-Visual Attacks: A Survey From Audio to Multimodal Models , author=. ACM Computing Surveys , year= on Pith
  13. [13] Unifle: Uniform fusion of multiple lora experts for backdoor defense in large language models , author=. IEEE Transactions on Dependable and Secure Computing , year= on Pith
  14. [14] ChronoPhyBench: Do MLLMs Truly Understand the World or Merely Exploit Language Priors? , author=. arXiv preprint arXiv:2606.07962 , year= on PithPith's reviewarXiv
  15. [15] Mlvu: Benchmarking multi-task long video understanding , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
  16. [16] Lvbench: An extreme long video understanding benchmark , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization= on Pith
  17. [17] Tempcompass: Do video llms really understand videos? , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages= on Pith
  18. [18] Thinking in space: How multimodal large language models see, remember, and recall spaces , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
  19. [19] Egoschema: A diagnostic benchmark for very long-form video language understanding , author=. Advances in Neural Information Processing Systems , volume= on Pith
  20. [20] Mmbench-video: A long-form multi-shot benchmark for holistic video understanding , author=. Advances in Neural Information Processing Systems , volume= on Pith
show 41 more
  1. [21] Infinibench: A benchmark for large multi-modal models in long-form movies and tv shows , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages= on Pith
  2. [22] EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory , author=. arXiv preprint arXiv:2608.13113 , year= on PithPith's reviewarXiv
  3. [23] RULER: What's the real context size of your long-context language models? , author=. arXiv preprint arXiv:2404.06654 , year= on PithPith's reviewarXiv
  4. [24] Context length alone hurts LLM performance despite perfect retrieval , author=. arXiv preprint arXiv:2510.05381 , year= on PitharXiv
  5. [25] Evaluating very long-term conversational memory of llm agents , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  6. [26] SkillsBench: Benchmarking how well agent skills work across diverse tasks , author=. arXiv preprint arXiv:2602.12670 , year= on PithPith's reviewarXiv
  7. [27] Skill retrieval augmentation for agentic ai , author=. arXiv preprint arXiv:2604.24594 , year= on PithPith's reviewarXiv
  8. [28] LongFuncEval: Measuring the effectiveness of long context models for function calling , author=. arXiv preprint arXiv:2505.10570 , year= on PithPith's reviewarXiv
  9. [29] arXiv preprint arXiv:2403.05530 , year =. doi:10.48550/arXiv.2403.05530 , url = on PithPith's reviewarXivdoi:10.48550/arxiv.2403.05530
  10. [30] Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Fan, Yang and Dang, Kai and Du, Mengfei and Ren, Xuancheng and Men, Rui and Liu, Dayiheng and Zhou, Chang and Zhou, Jin... on PithPith's reviewarXivdoi:10.48550/arxiv.2409.12191
  11. [31] Bai, Shuai and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Song, Sibo and Dang, Kai and Wang, Peng and Wang, Shijie and Tang, Jun and Zhong, Humen and Zhu, Yuanzhi and Yang, Mingkun and Li, Zhaohai and Wan, Jianqiang and Wang, Pengfei and Ding, Wei and Fu,... on PithPith's reviewarXivdoi:10.48550/arxiv.2502.13923
  12. [32] Video-chatgpt: Towards detailed video understanding via large vision and language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  13. [33] Llava-onevision: Easy visual task transfer , author=. arXiv preprint arXiv:2408.03326 , year= on PithPith's reviewarXiv
  14. [34] Longvila: Scaling long-context visual language models for long videos , author=. International Conference on Learning Representations , volume= on Pith
  15. [35] Llama-vid: An image is worth 2 tokens in large language models , author=. European conference on computer vision , pages=. 2024 , organization= on Pith
  16. [36] Videoagent: Long-form video understanding with large language model as agent , author=. European Conference on Computer Vision , pages=. 2024 , organization= on Pith
  17. [37] Videoagent: A memory-augmented multimodal agent for video understanding , author=. European Conference on Computer Vision , pages=. 2024 , organization= on Pith
  18. [38] Drvideo: Document retrieval based long video understanding , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
  19. [39] Deep video discovery: Agentic search with tool use for long-form video understanding , author=. Advances in Neural Information Processing Systems , volume= on Pith
  20. [40] Re-thinking temporal search for long-form video understanding , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
  21. [41] Longvt: Incentivizing" thinking with long videos" via native tool calling , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
  22. [42] Li, Keliang and Li, Yansong and Shen, Hongze and Liu, Mengdi and Chang, Hong and Shan, Shiguang , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2026 , pages = on Pith
  23. [43] Worldmm: Dynamic multimodal memory agent for long video reasoning , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
  24. [44] Videoarm: Agentic reasoning over hierarchical memory for long-form video understanding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
  25. [45] Li, Chenglin and Chen, Qianglong and Han, Feng and Wang, Yikun and Yin, Xingxi and Gong, Yan and Li, Ruilin and Zhang, Yin and Wang, Jiaqi , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings , month =. 2026 , pages = on Pith
  26. [46] Envharness: Awakening static worlds for agent learning , author=. arXiv preprint arXiv:2608.19880 , year= on PithPith's reviewarXiv
  27. [47] AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents , author=. arXiv preprint arXiv:2607.02255 , year= on PithPith's reviewarXiv
  28. [48] MemoHarness: Agent Harnesses That Learn from Experience , author=. arXiv preprint arXiv:2607.14159 , year= on PithPith's reviewarXiv
  29. [49] Moviechat: From dense token to sparse memory for long video understanding , author=. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2024 , organization= on Pith
  30. [50] Shen, Xiaoqian and Xiong, Yunyang and Zhao, Changsheng and Wu, Lemeng and Chen, Jun and Zhu, Chenchen and Liu, Zechun and Xiao, Fanyi and Varadarajan, Balakrishnan and Bordes, Florian and Liu, Zhuang and Xu, Hu and Kim, Hyunwoo J. and Soran, Bilge and Krishnamoorthi, Raghurama... on Pith
  31. [51] Videotree: Adaptive tree-based video representation for llm reasoning on long videos , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
  32. [52] Wu, Di and Wang, Hongwei and Yu, Wenhao and Zhang, Yuwei and Chang, Kai-Wei and Yu, Dong , booktitle =. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory , volume = on Pith
  33. [53] Mmrc: A large-scale benchmark for understanding multimodal large language model in real-world conversation , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  34. [54] Mem-gallery: Benchmarking multimodal long-term conversational memory for mllm agents , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  35. [55] Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory , author=. International Conference on Learning Representations , volume= on Pith
  36. [56] Memlens: Benchmarking multimodal long-term memory in large vision-language models , author=. arXiv preprint arXiv:2605.14906 , year= on PithPith's reviewarXiv
  37. [57] Mem2actbench: A benchmark for evaluating long-term memory utilization in task-oriented autonomous agents , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages= on Pith
  38. [58] Memoryarena: Benchmarking agent memory in interdependent multi-session agentic tasks , author=. arXiv preprint arXiv:2602.16313 , year= on PithPith's reviewarXiv
  39. [59] Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents , author=. arXiv preprint arXiv:2608.15008 , year= on PithPith's reviewarXiv
  40. [60] Lora: Low-rank adaptation of large language models , author=. arXiv preprint arXiv:2106.09685 , year= on PithPith's reviewarXiv
  41. [61] Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume= on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment