{"id":"0c8be78f-a96e-4711-b612-020983cd1a45","arxiv_id":"2501.15953","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A graph-based memory that tracks entity relations over time improves LLM-driven long-video question answering accuracy and frame efficiency.","lead":"GraphVideoAgent adds a dynamic graph memory that tracks objects and their relations across frames, helping an LLM agent pick the right frames and answer questions about long videos. On two benchmarks it beats a strong video-agent baseline by about two percentage points while using about eight frames per video.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SOTA gains over VideoAgent (2.2 pts EgoSchema, 2.0 pts NExT-QA) come from single runs without variance or significance tests; these margins may be stochastic, so the central graph-memory claim is not yet supported.","rationale":"I read the paper as claiming that an explicit entity-relation graph memory outperforms sequential memory for long-form video QA. For that claim to hold, the observed accuracy differences must reflect the graph component rather than chance. The paper reports no repeated runs or uncertainty intervals. The margins over the key baseline are small (2.2 and 2.0 points), exactly the scale at which GPT-4 nondeterminism and prompt-order effects operate. Without a statistical check, the central conclusion is underdetermined. This is not an accusation of dishonesty; it is a standard correctness requirement for a stochastic agent evaluation. The confidence-calibration issue in Section 3.3 is real, but because the baseline also uses a similar self-reflection stopping rule, it cannot explain a differential gain and thus is not the load-bearing assumption for the paper's novelty claim. I therefore keep the reader's CONDITIONAL verdict: the idea is plausible and the qualitative pattern of ablations supports it, but release of code and a variance-aware evaluation are needed before the state-of-the-art claim can be accepted.","tokens_in":17124,"tokens_out":6256,"duration_ms":61334,"concrete_test":"Run GraphVideoAgent and the exact VideoAgent baseline (same VLM captioner, same GPT-4 version and temperature) at least 10 times on the EgoSchema public subset and NExT-QA val with different random seeds. Report mean accuracy, standard deviation, and a paired bootstrap 95% confidence interval for the difference. If the CI for the 2.2/2.0 point differences includes zero, the SOTA and graph-memory-validity claims collapse; if it excludes zero, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that GraphVideoAgent's dynamic entity-relation graph improves video QA accuracy depends on the reported gains being real rather than run-to-run noise. Tables 1 and 3 report one accuracy value per system with no standard deviation, number of seeds, or significance test. The gap over the primary baseline VideoAgent is only 2.2 points on EgoSchema full and 2.0 points on NExT-QA. Because the agent uses stochastic GPT-4 generations and prompt-dependent retrieval decisions, single-run differences of this size can easily flip. The paper's own ablation (Table 5) shows a 4.2-point drop when removing the multi-dimension structure, which is encouraging, but it is also unreplicated. The reader's concern about the level-3 confidence stopping rule in Section 3.3 is plausible, but that rule is shared by the baseline and does not isolate the graph's contribution. The missing error analysis directly targets the differentiator and is therefore more load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GraphVideoAgent, an LLM-driven agent for long-form video question answering that maintains a dynamic entity-relation graph built from sampled frame captions. The graph tracks entities, their spatial/interaction/action relations, and state changes, and the agent uses it to guide iterative frame retrieval and answer generation with a self-reported confidence stop rule. The authors report state-of-the-art accuracy on EgoSchema (56.3% full, 62.7% subset) and NExT-QA (73.3%) while using only about 8.1–8.2 frames per video on average, and they present ablations on LLM choice, graph components, and entity scale, plus case studies.","tokens_in":17352,"tokens_out":4649,"duration_ms":43236,"significance":"If the reported results are reliable, the core idea—replacing sequential memory with an explicit, dynamically updated entity-relation graph for long-video QA—is valuable and could improve both accuracy and frame efficiency. The paper provides a concrete instantiation of this idea, with a working system and comparisons against strong baselines, and the ablations are directionally consistent with the graph contributing to performance. However, the evidence is not yet statistically robust: the gains over the primary baseline are small (+2.2 and +2.0 points), no variance or significance tests are reported, and key algorithmic components (the graph update function and the frame-scoring weights) are under-specified. The entity-scale analysis in particular rests on only 50 questions from an undescribed dataset, so the scalability claim is not supported. The manuscript also contains template placeholders and citation inconsistencies that must be corrected. These issues can be fixed with additional experiments and more detailed exposition, but the current form is not publishable without revision.","major_comments":[{"comment":"The central claim that GraphVideoAgent outperforms VideoAgent by 2.2 points on EgoSchema and 2.0 points on NExT-QA is not supported by any measure of statistical reliability. The system depends on stochastic GPT-4 generations and retrieval decisions, yet only single accuracy values are reported in Tables 1, 2, and 3. Under run-to-run variation, a 2-point gap on a benchmark like EgoSchema is often within the noise of the evaluation. Please report multiple seeds (at least 3–5) with standard deviations or confidence intervals, and ideally a paired significance test over videos for the main comparisons and for the ablations in Tables 5 and 6.","section":"§4.2, Tables 1 and 3"},{"comment":"The method is under-specified in ways that prevent reproduction and independent evaluation. The global graph update function U in Eq. (2) is never defined; the text says new frames and relations are 'seamlessly integrated' but no algorithm is given for how nodes, edges, or state histories are added, merged, or pruned. Similarly, Eq. (3) defines the frame score with weights alpha=0.5, beta=0.3, gamma=0.2, but no sensitivity analysis or justification is provided, and the threshold tau=3 is stated without calibration. Please provide a precise algorithmic description of U (or pseudocode), the exact definitions of s_graph, s_visual, and s_temporal, and a sensitivity analysis for the weights and tau.","section":"§3.2, Eq. (2) and §3.3, Eq. (3)"},{"comment":"The adaptive retrieval loop stops when the LLM self-reports confidence level 3, but the paper gives no calibration or error analysis showing that level-3 self-reports actually correspond to correct answers. This rule is load-bearing for the claimed efficiency of using only ~8.2 frames: if the model is overconfident on partial evidence, early termination will systematically hurt accuracy. The baseline VideoAgent shares a similar rule, so this does not invalidate the graph's contribution, but it does affect the interpretation of the frame-efficiency result. Please report the distribution of confidence levels, the correlation between confidence and accuracy, and a breakdown of errors by stopping round.","section":"§3.3, confidence stopping rule"},{"comment":"The entity-scale ablation uses only 50 multiple-choice questions from a 'multi-entity hour-long video dataset' that is not described at all—its size, annotation process, video sources, and question categories are missing. With 50 questions, the reported differences (e.g., 64.0 vs. 60.0 for 2–3 entities, 52.0 vs. 42.0 for 7+ entities) have very large confidence intervals, and the claim of 'superior scalability' is not statistically supported. Please either describe this dataset fully and provide error bars or significance tests, or run the entity-scale analysis on a larger, existing benchmark with known properties.","section":"§4.3, Table 6 and §4.1"}],"minor_comments":[{"comment":"The abstract contains typos and missing spaces, e.g., 'GraphVideoAgentachieveda2.2%improvementoverexistingmethodswhilerequiringanalysisofonly8.2framesonaverage', and the improvements should be written as percentages (2.2% and 2.0%) for consistency with the rest of the paper.","section":"Abstract"},{"comment":"The citation 'VideoAgent [63]' is incorrect: reference [63] is 'A Simple LLM Framework for Long-Range Video Question-Answering' (LLoVi), not VideoAgent. This should be corrected, and the duplicate references [63] and [64] to the same paper should be merged.","section":"§2.1"},{"comment":"The manuscript contains placeholder ACM template text that should have been removed before submission: 'Conference acronym ’XX, June 03–05, 2018, Woodstock, NY', 'CCS Concepts Do Not Use This Code', and 'Make sure to enter the correct conference title from your rights confirmation email'. These artifacts indicate an incomplete submission and must be fixed.","section":"Template/formatting"},{"comment":"The sentence 'The performance difference between the subset (60.2%) and full test set (54.1%) reflects the distinct characteristics of our evaluation protocol' is confusing because those numbers appear to be VideoAgent's subset and full-set results, not GraphVideoAgent's. Rewrite this passage to clearly describe the two evaluation regimes and the reported numbers.","section":"§4.2"},{"comment":"The description of the multi-entity dataset in §4.1 is a single sentence with no details, and Table 6 does not state how the entity count per question was determined or how the 50 questions were selected. Please provide full dataset documentation and selection criteria.","section":"§4.1 and Table 6"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline paper. The core idea is interesting and the experimental design is on the right track, but the current evidence is too thin to support the strong 'state-of-the-art' and 'validates' claims. The missing error bars, under-specified graph update, and tiny entity-scale ablation are all fixable with additional work, but as submitted the paper is not yet publishable. I also note the template placeholders, which suggest the manuscript was not prepared for submission; the authors should be asked to complete and polish the paper if they seek publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is replacing sequential memory with a dynamic entity-relation graph memory inside an LLM agent for long-form video QA. That is genuinely new as far as I can tell: prior agent pipelines used flat or sequential memory, and static scene graphs were not used this way for iterative frame retrieval. The graph with spatial, interaction, and action edges plus temporal state tracking is a sensible design, and the paper's own ablations show consistent drops when you remove components. No circularity: the prompts and graph scoring are not tuned on test labels. The frame budget is also impressively low, and the numbers on EgoSchema and NExT-QA are consistent across question categories, which is more than nothing.\n\nThe soft spots are real and they line up with the stress-test note. The headline gains over VideoAgent are +2.2 on EgoSchema full and +2.0 on NExT-QA overall. Tables 1 and 3 report one number per system, no standard deviations, no seeds, no significance test. GPT-4 is stochastic and the retrieval loop is prompt-dependent, so a 2-point gap can easily flip. The entity-scale ablation uses only 50 questions, and the graph construction is underspecified: how exactly nodes and edges are extracted from captions, what S(e,f) and R(e,f) in Eq. (1) are, and how the update U in Eq. (2) works are all left vague. The confidence-level stopping rule is a shared baseline property and does not isolate the graph's contribution, so the missing variance analysis is the load-bearing issue, not the stopping rule.\n\nThe paper is also not finished as a manuscript. There are ACM template placeholders, a 2018 copyright line, a fake CCS Concepts block, and citation errors—VideoAgent appears as [45] and [63] with different descriptions. That suggests the authors uploaded an early draft. It does not change the scientific substance, but it does mean the paper should not be published as is.\n\nBottom line: the central idea is worth taking seriously, and the ablations give it some internal support, but the evidence is not yet robust enough to accept. If I were an editor, I would send it to review anyway because the method is plausible and relevant to a busy subfield, but I would expect the referees to demand code, implementation details, and error bars before acceptance. I would not trust the 2-point SOTA claim until I see multi-seed variance or significance numbers. The paper is worth a reading-group discussion, mostly to think through the graph-memory design, not to copy the numbers.","headline":"A plausible graph-memory idea for long-video QA, but the reported gains over VideoAgent rest on single runs without variance, so the paper needs code and statistics before the claim is trustworthy.","tokens_in":17856,"tokens_out":2001,"would_cite":true,"duration_ms":21985,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GraphVideoAgent claims that replacing sequential video memory with a dynamic entity-relation graph lets an LLM agent answer long-video questions using about 8 frames per video.","keywords":["LLM agent","long-form video understanding","entity relation graph","video question answering","adaptive frame selection","temporal reasoning","graph memory"],"falsifier":"Run GraphVideoAgent on EgoSchema while logging the self-reported confidence and the set of frames seen at each iteration, then compare answers produced at level 3 with answers produced when the agent is forced to retrieve additional rounds; if level-3 answers are no more accurate than forced-continue answers, the confidence-based stopping rule is not the mechanism behind the reported accuracy.","tokens_in":16912,"feed_emoji":"🎬","tokens_out":8026,"duration_ms":70046,"temperature":0.7,"pith_summary":"The paper proposes GraphVideoAgent, a system that answers questions about long videos by maintaining a dynamic graph of entities and their relations as it samples frames. The central claim is that this graph, rather than a sequential memory of frames, is what lets an LLM agent find the decisive moments in a long video. If the claim is right, structured semantic memory improves both accuracy and efficiency: the paper reports 56.3% on EgoSchema and 73.3% on NExT-QA while using only 8.2 and 8.1 frames per video on average. A sympathetic reading is that the graph's temporal tracking of entities and relations, not a stronger vision backbone, drives the improvement.","feed_headline":"Graph memory answers long videos in 8 frames","feed_subtitle":"Replacing sequential video memory with a dynamic entity graph lifts EgoSchema to 56.3% and NExT-QA to 73.3%.","key_machinery":"The load-bearing object is a multi-level dynamic entity-relation graph $G=(V,E)$, whose nodes are entity profiles $(F_i, x_i, c_i, s_i)$ tracking appearance frames, visual features, caption descriptions, and state-change histories, and whose edges are typed relations (spatial, interaction, action) extracted via dependency parsing. Temporal coherence is maintained by the score $T(e,f)=\\alpha S(e,f)+(1-\\alpha)R(e,f)$, which combines entity state consistency and relation persistence, and the graph is updated across frames by $G_{t+1}=U(G_t,F_{t+1},R_{t+1})$. The LLM agent reads this graph, scores candidate frames with $S(f)=0.5\\,s_{\\text{graph}}(f)+0.3\\,s_{\\text{visual}}(f)+0.2\\,s_{\\text{temporal}}(f)$, and stops retrieval when its self-reflection reaches confidence level 3.","core_discovery":"GraphVideoAgent is a video question-answering pipeline in which a vision-language model captions uniformly sampled frames, NLP tools extract entities and relations into a dynamic graph, and an LLM agent iteratively selects additional frames until a self-reported confidence level of 3 is reached. The paper's central claim is that this graph memory, not the raw frame sequence, is what lets the agent answer causal and temporal questions correctly with very few frames. On EgoSchema it reports 56.3% on the full test set (62.7% on the public subset) using 8.2 frames per video on average, and on NExT-QA it reports 73.3% overall using 8.1 frames, with gains in every question category over the VideoAgent baseline. The conclusion states that these results validate the effectiveness of structured semantic memory in video understanding tasks.","pith_inferences":["A direct test of the stopping rule would log confidence and correctness per query; if level-3 stops are not more accurate than forced extra retrievals, the reported frame efficiency would not reflect a reliable signal.","Because the graph is built from captions, swapping the captioning model while freezing the graph machinery would separate the contribution of richer text input from the contribution of graph structure.","The graph is built per question in this design; reusing it across multiple questions on the same video is an untested extension that could amortize construction cost."],"forward_implications":["If the central claim holds, graph-guided retrieval reaches sufficient confidence after about 8 frames, so per-video inference cost stays near constant even on hour-long inputs.","The EgoSchema full-set result of 56.3% with 8.2 frames would put the method above the reported VideoAgent baseline (54.1%) and roughly level with proprietary models that process many more frames.","On NExT-QA, the method reports gains on every question type, with the largest absolute gain over VideoAgent on causal questions (74.6% versus 72.7%).","The ablations imply that the multi-level graph structure, not the LLM backbone alone, accounts for the largest part of the measured improvement."],"supporting_citations":[{"why":"Supplies the iterative LLM-agent frame-selection baseline (VideoAgent) that this method extends with graph memory and must outperform.","marker":"[45]"},{"why":"Supplies the EgoSchema benchmark whose full-set and public-subset accuracies are the paper's primary long-form results.","marker":"[31]"},{"why":"Supplies the NExT-QA benchmark with causal, temporal, and descriptive splits where the method reports 73.3% overall.","marker":"[52]"},{"why":"Supplies the LLoVi long-form QA baseline that processes about 180 frames, the comparison showing frame-count efficiency.","marker":"[63]"},{"why":"Supplies the GPT-4 LLM agent backbone; the paper's LLM ablation shows it outperforms smaller LLM choices by a wide margin.","marker":"[34]"},{"why":"Supplies EVA-CLIP visual features used in entity node profiles and in the visual component of frame scoring.","marker":"[39]"},{"why":"Supplies LaViLa egocentric captioning, which feeds the entity and relation extraction for EgoSchema videos.","marker":"[66]"},{"why":"Supplies spaCy for named entity recognition, noun-phrase chunking, and dependency parsing that build graph nodes and edges.","marker":"[16]"}],"fun_headline_variants":["Graph memory answers long videos with just 8 frames","LLM entity graphs turn 8 frames into full video understanding","Dynamic entity graph: 8 frames beat frame-by-frame memory","Video QA in 8 frames via entity relation graphs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system trusts the LLM's self-reported level-3 confidence as proof that the already-gathered frames contain the answer, but the paper provides no calibration data showing that level-3 self-reports track actual correctness.","fun_headline_variants_meta":{"raw":{"variants":["Graph memory answers long videos with just 8 frames","LLM entity graphs turn 8 frames into full video understanding","Dynamic entity graph: 8 frames beat frame-by-frame memory","Video QA in 8 frames via entity relation graphs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000745,"raw_usage":{"total_tokens":3331,"prompt_tokens":962,"completion_tokens":2369,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":2302}},"tokens_in":578,"tokens_out":2369,"duration_ms":16945,"temperature":1.0,"reasoning_tokens":2302,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:49:37.535577+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GraphVideoAgent on EgoSchema while logging the self-reported confidence and the set of frames seen at each iteration, then compare answers produced at level 3 with answers produced when the agent is forced to retrieve additional rounds; if level-3 answers are no more accurate than forced-continue answers, the confidence-based stopping rule is not the mechanism behind the reported accuracy.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the iterative LLM-agent frame-selection baseline (VideoAgent) that this method extends with graph memory and must outperform."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the EgoSchema benchmark whose full-set and public-subset accuracies are the paper's primary long-form results."},{"cited_title":"InProceedings of the IEEE/CVF international conference on computer vision","cited_arxiv_id":null,"evidence_quote":"Supplies the LLoVi long-form QA baseline that processes about 180 frames, the comparison showing frame-count efficiency."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies LaViLa egocentric captioning, which feeds the entity and relation extraction for EgoSchema videos."},{"cited_title":"2020.spaCy: Industrial-strength Natural Language Processing in Python","cited_arxiv_id":null,"evidence_quote":"Supplies spaCy for named entity recognition, noun-phrase chunking, and dependency parsing that build graph nodes and edges."}],"review_version":1}