{"id":"0fa8ddaf-65f2-4c9c-8c13-63dad940ec88","arxiv_id":"2505.13851","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper issues a grand challenge to build neuro-symbolic video agents that search video, call tools, and generate video, using the authors' NSVS-TL and NeuS-V as starting points.","lead":"This paper is a challenge statement, not a new experiment. It argues that future video systems should combine neural perception with symbolic temporal logic, and it asks the community to build datasets and benchmarks for video agents that search, act, and generate.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing necessity claim in Section 2 ('video agents would need to be built via neuro-symbolic methods') is under-supported: Figure 3 compares a bespoke hybrid system to foundation models on self-built benchmarks with no learned temporal baseline or scaling analysis, so the assertion that…","rationale":"The paper is a position/challenge paper, so it is not required to prove the impossibility of pure deep learning. But the reader's conditional verdict is appropriate precisely because the strongest sentence in Section 2 is a factual necessity claim, not merely a research agenda. My stress-test identified the same soft spot as the reader: the temporal-reasoning bottleneck is asserted from a narrow, self-constructed comparison, and no experiment rules out the scaling counterfactual. A single well-designed baseline comparison would materially change how much weight the central claim can bear. I also confirmed the internal contradiction regarding neuro-symbolic video generation, which should be corrected regardless. Because the reader already made the verdict CONDITIONAL and my concern is the same class of evidence gap, I do not recommend moving the verdict; I would keep it CONDITIONAL (UNCHANGED).","tokens_in":11892,"tokens_out":4956,"duration_ms":49077,"concrete_test":"Reproduce Figure 3(a) on the TLV dataset with a purely learned temporal baseline rather than a general-purpose foundation model: fine-tune a long-context video-language model with explicit temporal grounding (e.g., a VideoLLM with timestamped decoding or a video transformer with memory) on the TLV training split, holding compute roughly matched to NSVS-TL, and measure F1 on the same complex queries, split by number of temporal constraints. If the learned baseline reaches or exceeds NSVS-TL's F1, or if F1 improves monotonically with model capacity/context length, the necessity claim in Section 2 is undermined. If performance saturates with scale while NSVS-TL continues to improve on the same queries, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an architectural necessity claim: Section 2 concludes that 'video agents would need to be built via neuro-symbolic methods' because deep learning struggles with temporal dependencies. The support for that claim is (i) NSVS-TL outperforming multimodal foundation models on the TLV benchmark (Figure 3a) and (ii) degradation of text-to-video outputs with query complexity on NeuS-V (Figure 3b). Both benchmarks are introduced by the same authors, and the baselines are general-purpose foundation models, not competitive learned temporal systems. A necessity claim requires evidence that the gap is intrinsic to end-to-end learning rather than a function of model scale, training data, or evaluation protocol. No scaling curves, no compute-matched comparisons, no fine-tuned video-language temporal grounding baseline, and no statistical analysis are provided. The paper also asserts in Section 3 that 'neuro-symbolic methods for solving this problem [video generation] have not been explored,' immediately after citing the authors' own Choi et al. 2025a ('We'll fix it in post: improving text-to-video generation with neuro-symbolic feedback'), an internal inconsistency that further weakens the evidence narrative. The neuro-symbolic program is a reasonable research hypothesis, but the paper's strongest claim overreaches its evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This position paper argues that the next generation of 'video agents'—systems that search and understand video, act in the real world, and generate synthetic video—must be built with neuro-symbolic architectures that combine deep perception with formal temporal-logic representations. The authors support this necessity claim primarily with two strands of evidence drawn from their own prior systems: NSVS-TL outperforms multimodal foundation models on complex video search (Figure 3a), and text-to-video models degrade in temporal fidelity as query complexity increases on the NeuS-V benchmark (Figure 3b). The paper then formalizes a grand challenge, specifying inputs, outputs, and metrics for video agents, and offers TLV as a preliminary dataset, F1 for event search, tool-calling accuracy, and VBench/NeuS-V for generation evaluation. It closes with open questions on multimodal triggers, choice of formal language, and multi-camera multi-agent settings.","tokens_in":12128,"tokens_out":3091,"duration_ms":30977,"significance":"If the central claim is accepted, the paper provides a useful community roadmap: it clearly names three pillars of video-agent capability, proposes concrete evaluation axes, and identifies temporal reasoning as a likely bottleneck. Its most valuable concrete contributions are the articulation of an evaluation protocol for tool-calling agents and the suggestion of combining visual-quality and temporal-fidelity metrics for generative video. The paper is also honest about the preliminary nature of the TLV dataset and explicitly invites the community to build better benchmarks. However, the significance is limited by the fact that the load-bearing necessity claim—that deep learning alone cannot achieve the needed temporal reasoning—is not established in this manuscript; the supporting evidence is drawn entirely from the authors' own benchmarks with no re-analysis, no learned temporal baselines, and no scaling analysis. The paper is best read as a research hypothesis and call to action rather than a demonstrated result.","major_comments":[{"comment":"The sentence 'video agents would need to be built via neuro-symbolic methods' is a categorical necessity claim, but the support in Figure 3 is a comparison of the authors' NSVS-TL and NeuS-V systems against general-purpose foundation models on benchmarks introduced by the same authors. There is no fine-tuned video-language temporal-grounding baseline, no compute-matched comparison, no scaling curve showing that the gap persists as model size or training data grows, and no error bars or significance testing. As written, the evidence supports 'currently outperform on these benchmarks,' not 'must be neuro-symbolic.' I recommend either softening the claim throughout Section 2 and the abstract or substantially expanding the evidence with learned baselines and scaling analysis.","section":"Section 2, final paragraph"},{"comment":"The sentence 'to our knowledge, neuro-symbolic methods for solving this problem have not been explored' is immediately followed by a citation to the authors' own Choi et al. 2025a, 'We'll fix it in post: improving text-to-video generation with neuro-symbolic feedback.' This is an internal inconsistency on a point that is partially load-bearing for the claimed gap. If the intended meaning is that no prior work has combined formal verification with the generative pipeline at training time, that distinction needs to be stated explicitly.","section":"Section 3, Video Generation paragraph"},{"comment":"The quantitative evidence that foundation models 'struggle' and 'deteriorate' is presented as two small panels without the evaluation protocol: which specific models were tested, how many prompts, how queries were varied in complexity, and what variance exists across runs. Because NeuS-V is both the proposed evaluation metric and part of the evidence for the neuro-symbolic approach, this creates a circularity concern. The manuscript would be strengthened by a short description of the evaluation setup or a reference to a version of the underlying benchmarks with this information, and by reporting confidence intervals.","section":"Figures 3(a) and 3(b), Section 6"},{"comment":"The architecture discussion assumes that neural perception can reliably produce atomic events for formal verification, but this reliability assumption is the soft underbelly of the entire approach. If perception is noisy at the atomic-event level, formal guarantees over the abstracted event sequence do not directly transfer to the video. The paper acknowledges the orchestration challenge only obliquely; a challenge paper on this topic should state the perception-verification gap as a first-class open problem, since it bears directly on the promised 'guarantees on system behavior.'","section":"Section 7"}],"minor_comments":[{"comment":"The description of the TLV dataset as compiled from 'static images' from Waymo and NuScenes is confusing for a video-search benchmark; please clarify whether the evaluation actually exercises temporal video dynamics or whether temporal structure is imposed by the specification-pairing procedure. This will affect how readers interpret the claimed temporal-reasoning bottleneck.","section":"Section 5"},{"comment":"Typo: 'long-from video understanding' should be 'long-form video understanding.'","section":"Section 3, Video Agents paragraph"},{"comment":"The text uses 'open-sourced API's' with an apostrophe; use 'APIs.'","section":"Section 4, Inputs/Tools"},{"comment":"The reference title contains a typo: 'formalf verification' should be 'formal verification.'","section":"References, Sharan et al. 2024"},{"comment":"The paper states that data, benchmarks, and example agents are provided on GitHub, but no URL is given in the manuscript; please include the repository link.","section":"Section 4, final paragraph"}],"recommendation":"major_revision","confidential_remarks":"This is a vision/challenge paper, so the evidence bar should not be the same as for a technical systems paper. However, the categorical phrasing of the necessity claim in Section 2 is the paper's central thesis, and it is currently supported only by the authors' own benchmark comparisons. The internal inconsistency about neuro-symbolic video generation also needs to be resolved. I believe the paper can be made acceptable by softening the claim, adding a more rigorous discussion of the evidence, and clarifying the perception-to-verification gap; hence major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things before deciding how much time this paper deserves. First, it is a position paper, not a research contribution: it proposes a grand challenge for neuro-symbolic video agents and repackages the authors' earlier NSVS-TL and NeuS-V systems as starting points. Second, the central assertion in Section 2 that video agents \"would need to be built via neuro-symbolic methods\" is stated more strongly than the evidence supports. The paper's own Figure 3 compares a bespoke hybrid system against general-purpose foundation models on self-built benchmarks, with no learned temporal baseline, no scaling curves, and no compute-matched comparison. That is a real soft spot, though for a position paper it is a defensible hypothesis if worded as one instead of as a necessity claim.\n\nWhat the paper does well: it writes clearly, motivates the problem with concrete scenarios (home security, delivery, break-in), and formalizes a challenge with inputs, outputs, metrics, and open questions in Section 7. The three-pillar structure—search, action, generation—is sensible and timely. The authors honestly note that their TLV dataset lacks tool-invocation annotations, which invites the community to extend it. The proposed use of VBench for visual quality and NeuS-V for temporal fidelity is a reasonable starting point.\n\nThe more serious issue is internal consistency. Section 3 says \"to our knowledge, neuro-symbolic methods for solving this problem [video generation] have not been explored,\" immediately after citing the authors' own 2025a paper \"We'll fix it in post,\" which is exactly a neuro-symbolic feedback method for text-to-video improvement. That is a direct contradiction and needs fixing. Also, the paper says resources are provided \"on GitHub\" but never gives a URL, and the benchmarks it leans on are the authors' own, so external readers cannot independently verify the reported gaps.\n\nThe citation pattern is heavily self-referential, but that is not unusual for a position paper built on two prior systems. The formalized challenge definition is modestly new, and the paper does not pretend to offer new experiments. For a venue that welcomes vision papers or community challenges, this deserves a serious referee: the framing is useful, the writing is good, and the open questions are worth airing. I would ask the authors to downgrade the \"need\" claim to a strong hypothesis, fix the contradiction, and provide the missing links before acceptance. It would spark a good reading-group discussion in neuro-symbolic AI, though I would not cite it in my own work—it does not advance the technical state of the art.","headline":"A clearly written position paper that usefully packages the authors' prior neuro-symbolic video benchmarks as a community challenge, but the claim that video agents *need* neuro-symbolic methods is a research hypothesis, not an established result, and one internal contradiction about video generation needs correcting.","tokens_in":12686,"tokens_out":2727,"would_cite":false,"duration_ms":25609,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Proactive video agents will need deep learning plus temporal logic, not deep learning alone.","keywords":["neuro-symbolic AI","video agents","temporal reasoning","temporal logic","video understanding","tool calling","text-to-video generation","formal verification"],"falsifier":"A direct test would be a TLV-style benchmark of temporally structured search and tool-timing queries on which a purely learned video-language model—trained at scale and allowed long-context or memory mechanisms—matches the verification-based agent's precision and recall on queries with four or more ordered events. Alternatively, measuring whether NSVS-TL's performance degrades when its neural perception module outputs noisy atomic events would reveal how much of the claimed advantage comes from verification rather than perception.","tokens_in":11704,"feed_emoji":"🎬","tokens_out":5807,"duration_ms":51145,"temperature":0.7,"pith_summary":"This paper argues that proactive video agents—systems that watch a video stream, reason about what is happening, and act on it—cannot be built from deep learning alone. The obstacle is temporal reasoning: neural models recognize objects and short activities well, but they lose track of sequencing and dependencies across long videos. The authors' proposed route is neuro-symbolic: use neural perception to detect atomic events, then express the user's query as a temporal-logic specification and verify it against a state-machine model of the video. If the argument holds, future video systems will combine learned perception with formal reasoning, supported by new benchmarks that annotate not just events but also tool actions and their timing.","feed_headline":"Deep learning alone can't build proactive video agents","feed_subtitle":"Long-video temporal reasoning needs formal logic and state machines, not just bigger models.","key_machinery":"The load-bearing mechanism is the decomposition of a video query into atomic events plus a temporal-logic specification, with the video represented as a probabilistic automaton whose states come from neural perception. Search then becomes a verification problem: does the automaton satisfy the user's temporal-logic query, and which frames correspond to the satisfying run? The same machinery is applied twice in the paper—NSVS-TL for search and NeuS-V for evaluating generated video—and it is the reason the approach can offer interpretability and formal guarantees while neural modules handle spatial perception.","core_discovery":"On the paper's own terms, the central claim is that the next generation of video systems will be neuro-symbolic video agents built on three pillars: video search and understanding, integration of understanding with real-world action through tool calling, and video generation for testing and editing. For each pillar, the paper argues that purely learned foundation models fall short—NSVS-TL shows they struggle with temporally complex search queries, and NeuS-V shows text-to-video generation loses temporal fidelity as prompts grow more complex. The proposed remedy is to decompose a query into atomic events, translate it into temporal logic, map the video into a probabilistic automaton using off-the-shelf neural perception, and then treat search and evaluation as formal verification problems. The paper therefore issues a community challenge to build datasets, metrics, and agents that combine these neural and symbolic components.","pith_inferences":["If the necessity claim holds, the reliability gap between learned and verified video agents should widen as query length and event count grow; the paper's Figure 3 data already suggest this for generation.","The same pipeline could extend beyond vision to audio triggers, multi-camera viewpoints, and multiple agents, but those settings would require distributed temporal-logic reasoning that the paper raises but does not solve.","A purely scaled learned model that matches neuro-symbolic performance on TLV-style temporal search and tool-timing would refute the necessity claim, so the challenge is falsifiable in principle.","Formal verification at the agent level could eventually give safety guarantees for home-security and autonomous-driving actions, but only if neural perception errors are tightly bounded; the paper's open questions acknowledge this dependency."],"forward_implications":["Temporally structured video search can be treated as verification over perception-derived automata rather than as end-to-end pattern matching, which is how the paper's NSVS-TL demonstration outperforms foundation models on complex queries.","Text-to-video generation must be judged not only by visual quality but by temporal fidelity, with prompts converted to atomic-event specifications and satisfaction probabilities as in NeuS-V.","Video-agent datasets need frame-level event annotations plus the tools to be invoked and the temporal order of invocations; the TLV dataset supplies the first part and leaves the second as an open challenge.","Tool calling by a video agent must be evaluated for event-specific selection, temporal alignment with the prompt, and correctness of the invoked inputs, not just whether a plausible tool was chosen.","Video generation will serve dual roles: synthetic edge-case testing for agents and privacy-preserving editing for post-hoc analysis."],"supporting_citations":[{"why":"Introduces NSVS-TL and the TLV dataset, the neuro-symbolic search pipeline and benchmark that the paper's argument against pure deep learning rests on.","marker":"(Choi et al., 2024)"},{"why":"Introduces NeuS-V, the formal-verification metric used to show text-to-video models lose temporal fidelity as prompt complexity rises.","marker":"(Sharan et al., 2024)"},{"why":"Supplies the model-checking and temporal-logic foundations that the neuro-symbolic verification machinery is built on.","marker":"(Baier and Katoen, 2008)"},{"why":"Provides the tool-orchestration pattern that video agents need for turning detected events into API calls and actions.","marker":"(Chase, 2022)"},{"why":"Defines the ToolBench-style agent benchmark that the paper's proposed video-agent task format is modeled on.","marker":"(Qin et al., 2023)"},{"why":"Supplies VBench, the visual-quality standard the paper pairs with NeuS-V for evaluating generated video.","marker":"(Huang et al., 2024)"},{"why":"Represents the LLM/VLM agent baseline whose lack of temporal grounding motivates the neuro-symbolic design.","marker":"(Yao et al., 2023)"}],"fun_headline_variants":["Video agents need logic, not just larger models","Neuro-symbolic approach: the key to temporal video reasoning","A challenge: build video agents that reason and act","Beyond deep learning: neuro-symbolic video agents","Temporal reasoning in video demands symbolic structure"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that temporal reasoning over long videos is a bottleneck that scaling current deep learning cannot overcome; if learned models solve long-horizon temporal dependencies, the case for a neuro-symbolic architecture loses its necessity.","fun_headline_variants_meta":{"raw":{"variants":["Video agents need logic, not just larger models","Neuro-symbolic approach: the key to temporal video reasoning","A challenge: build video agents that reason and act","Beyond deep learning: neuro-symbolic video agents","Temporal reasoning in video demands symbolic structure"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000279,"raw_usage":{"total_tokens":1654,"prompt_tokens":939,"completion_tokens":715,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":640}},"tokens_in":555,"tokens_out":715,"duration_ms":6877,"temperature":1.0,"reasoning_tokens":640,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:08:28.500564+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be a TLV-style benchmark of temporally structured search and tool-timing queries on which a purely learned video-language model—trained at scale and allowed long-context or memory mechanisms—matches the verification-based agent's precision and recall on queries with four or more ordered events. Alternatively, measuring whether NSVS-TL's performance degrades when its neural perception module outputs noisy atomic events would reveal how much of the claimed advantage comes from verification rather than perception.","supporting_citations":[{"cited_title":"Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification","cited_arxiv_id":"2411.16718","evidence_quote":"Introduces NeuS-V, the formal-verification metric used to show text-to-video models lose temporal fidelity as prompt complexity rises."},{"cited_title":"Principles of Model Checking","cited_arxiv_id":null,"evidence_quote":"Supplies the model-checking and temporal-logic foundations that the neuro-symbolic verification machinery is built on."},{"cited_title":"Langchain, 2022","cited_arxiv_id":null,"evidence_quote":"Provides the tool-orchestration pattern that video agents need for turning detected events into API calls and actions."}],"review_version":1}