{"id":"4d81984c-bead-4af4-8171-4b7c61d62c51","arxiv_id":"2607.10190","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Structured physical memories (captions, event graphs, rule cues) retrieved and verified at inference time consistently improve frozen VLMs on physical-plausibility benchmarks without fine-tuning.","lead":"PhysMRV is a training-free system that turns training videos into a hierarchical bank of scene captions, physical-event graphs, and rule cues, then retrieves them to help frozen video-language models judge whether events obey physics. It matters because current VLMs still fail basic physical-plausibility tests that block reliable real-world use.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The load-bearing risk is that graph/rule construction by the same VLM family may inject correlated labels rather than independent physical constraints, so retrieval gains may not prove transferable physical verification.","rationale":"The reader’s weakest assumption is exactly the right load-bearing point: faithfulness of Qwen3-VL/GPT-constructed graphs and rule/cue records as carriers of physical constraints (§3.2–3.3). I agree rather than invent a different concern. The paper’s ablations (Table 2) and bias analysis (Table 3) make a credible case that matched structured memories help, and gains are not pure random ICL; that is enough to keep CONDITIONAL rather than REJECT. What remains unsettled is whether the pipeline measures physical structure or correlated LLM-generated physics language—especially because train and eval graphs share the same extractor and TRAVL is the sole memory source. A human fidelity audit plus a corrected-graph re-run is the minimal concrete check that would settle whether the central claim is about physical verification or about better-aligned synthetic scaffolding. No stronger internal inconsistency is present; artifacts and train–eval documentation remain secondary requirements already noted by the reader.","tokens_in":14237,"tokens_out":677,"duration_ms":8666,"concrete_test":"On a stratified 100-video subset of GRASP L2 and IntPhys2, have two human annotators independently label (i) whether each extracted event graph correctly encodes the critical physical relation (support/contact/identity/continuity) and (ii) whether the top-1 retrieved rule family matches the true violation type. Report graph fidelity and rule-match rates; then re-run Main vs Top-3-memories-only after replacing low-fidelity graphs with human-corrected graphs. If gains collapse when graphs are corrected (or if fidelity <70% on implausible cases), the claim that structured physical memories supply transferable physical evidence is weakened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that physics-aware retrieval of hierarchical memories (captions + event graphs + rule/cue records) plus verification improves frozen VLMs on physical-plausibility benchmarks without training (Table 1). That claim requires the offline Memory Bank and test-time graphs to carry true physical structure that is independent of the verifier’s surface priors.\n\n§3.2–3.3 construct both train graphs and eval graphs with Qwen3-VL-8B, and rule families/cues with GPT-5.5 from TRAVL VQA+explanations. Physical-Event Reranking (Eqs. 5–6) then matches those graphs component-wise. If the extractor systematically under- or over-describes the same violation types that the frozen VLMs already mishandle (e.g., identity continuity, unsupported motion), then S_graph can retrieve surface-correlated but physically mismatched anchors, and the verifier may be scoring against self-consistent LLM physics language rather than ground-truth constraints. Table 2 shows graph reranking beats semantic/CLIP, but that only shows internal consistency of the pipeline, not external fidelity. The bias correction in Table 3 is real, yet still compatible with cue-driven polarity shift rather than instance-level physical verification.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"PhysMRV is a training-free inference framework that improves frozen VLMs on physical-plausibility reasoning by converting TRAVL training videos into a Hierarchical Physical Memory Bank (scene captions, physical-event graphs, and rule-family principles with positive/negative cues), then retrieving rule-diverse precedents via caption–question semantic recall followed by event-graph component matching (Eqs. 5–6) and prompting a frozen VLM to verify the current video against those structured criteria. On ImplausiBench, IntPhys2, and GRASP Level 2, across Qwen3-VL, Cosmos-Reason2, InternVL3.5-8B, and LLaVA-OneVision-2, the full pipeline consistently beats direct prompting and component controls (Table 1), with ablations isolating matched retrieval, verifier context, top-1 vs top-3 diversity, and graph vs semantic/CLIP retrieval (Table 2), plus a class-conditional bias analysis (Table 3).","tokens_in":14625,"tokens_out":1628,"duration_ms":23695,"significance":"If the gains reflect transferable physical structure rather than correlated LLM physics language, the work is a useful systems contribution: a plug-and-play, non-parametric scaffold for physical plausibility that requires no parameter updates and is evaluated across three benchmarks and four backbones. Strengths include thorough component controls and retrieval-strategy ablations (Tables 1–2), explicit diagnosis of the severe positive-class prior in direct VLMs (Table 3), qualitative failure-mode corrections (Figure 4), and honest reporting that LLaVA-OneVision-2 barely benefits. The hierarchical memory design (caption / event graph / rule cues) and physics-aware coarse-to-fine retrieval are concrete and reusable. The result is incremental relative to RAG and physics-context builders, but the combination of structured physical memory with verification and bias analysis is a solid empirical step for frozen-VLM physical reasoning.","major_comments":[{"comment":"§3.2–3.3 and Eqs. (5)–(6): Train and eval event graphs are both produced by Qwen3-VL-8B, and rule families/cues by GPT-5.5 from TRAVL VQA+explanations. Physical reranking therefore matches LLM-extracted structure to LLM-extracted structure. Table 2 shows graph reranking beats semantic/CLIP under the same verifier, which demonstrates internal pipeline consistency but not that graphs/cues are faithful carriers of ground-truth physical constraints independent of the verifier’s surface priors. Without human or physics-engine validation of graph/cue quality, or an ablation that swaps the extractor family (or uses human-authored graphs on a subset), the central claim that retrieval supplies transferable physical evidence remains under-supported relative to the alternative that the system retrieves self-consistent physics language and shifts answer polarity.","section":"§3.2–3.3, Eqs. (5)–(6), Table 2"},{"comment":"§4 / Table 1: Memory is built exclusively from TRAVL training videos, while ImplausiBench is from the same TRAVL/ImplausiBench paper family (Motamed et al., 2025). The manuscript does not quantify scenario, template, or visual overlap between the 3,482 TRAVL train videos and ImplausiBench’s 150 paired scenarios. If memory anchors are near-neighbors of eval items, gains on ImplausiBench (and possibly shared physical templates elsewhere) may partly reflect in-distribution case retrieval rather than general physical verification. A contamination or nearest-neighbor analysis (e.g., max S_graph / caption similarity of eval items to the bank, and performance stratified by similarity) is needed for the cross-benchmark claim.","section":"§4, Table 1, Memory Bank construction"},{"comment":"Table 3 and §4.3–§5: The bias analysis is valuable and shows large negative-class gains, but on GRASP the correction is a large polarity rebalancing (Qwen3-VL 0.9453/0.0220 → 0.6636/0.4961; InternVL flips toward negative bias). Aggregate accuracy can rise from cue-driven prior correction without reliable per-instance physical verification. The conclusion itself flags calibrated instance-level verification as open. To make the central claim load-bearing, report error slices conditioned on retrieved rule-family match vs mismatch, and cases where retrieved cues are wrong yet the model still flips—or where graphs miss the true violation—so that gains cannot be explained solely as checklist-induced polarity shift.","section":"Table 3, §4.3, §5"},{"comment":"§3.3: Component weights α_p are fixed uniformly at 0.2, with top-50 recall and rule-diverse top-3 selection as free design choices. Sensitivity of S_graph and final accuracy to α_p, pool size, and diversity constraint is not reported. Because physical-event reranking is presented as the key differentiator from semantic RAG, at least a small sensitivity study (or learned/ablated weights) is needed to show that the reported gains are not brittle to these hyperparameters.","section":"§3.3, Eq. (5)"}],"minor_comments":[{"comment":"Naming is inconsistent: title/abstract use PhysMRV; body often uses PHYSMRV; conclusion says “training-only” while the rest says “training-free.” Unify terminology.","section":"Throughout / §5"},{"comment":"“LLaV A-OneVision-2” is repeatedly spaced incorrectly; fix to LLaVA-OneVision-2.","section":"Table 1, §4.1"},{"comment":"Figure 2 lists “GPT 5.5”; implementation details say “GPT-5.5 API.” Clarify the exact model identifier and whether rule generation is reproducible (temperature, prompts).","section":"Figure 2, Implementation Details"},{"comment":"ImplausiBench scoring is a custom option-letter parser without an official script (§4 Metrics). Release the scorer and parsing rules with the paper to make Table 1 reproducible.","section":"§4 Metrics"},{"comment":"Related work on PhysMem, PCBs, and TRAVL is appropriate; a short explicit comparison table (training-free vs trained, memory structure, retrieval signal) would help position the contribution.","section":"§2"},{"comment":"Eq. (9) writes ŷ_i = F_θ(P_i) while Eq. (4) includes (v_i, q_i, P_i). State clearly whether raw video frames are still passed to the verifier or only the structured prompt.","section":"§3.1, §3.4"}],"recommendation":"major_revision","confidential_remarks":"Fit is reasonable for a systems/empirical ML venue. Novelty is incremental (structured RAG + verification for physics) rather than conceptual breakthrough; the decision should hinge on whether authors can substantiate graph/cue fidelity and TRAVL–ImplausiBench independence. I would not reject on the skeptic’s circularity concern alone—it is a real risk but addressable with validation experiments—but I would not accept without those fixes. No citation-pattern red flags beyond the natural concentration on TRAVL/ImplausiBench."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean empirical systems paper. The punchline is that a hierarchical memory bank (caption + physical-event graph + 15-family rule/cue cards) built from TRAVL training videos, retrieved by semantic recall then graph-component rerank with rule-family diversity, and fed as verification evidence, improves frozen VLMs on ImplausiBench, IntPhys2, and GRASP L2 without any parameter updates. Biggest lift is ~+9.6 pp on GRASP for Qwen3-VL; three of four backbones move, LLaVA-OneVision-2 barely does.\n\nWhat is actually new is not “RAG for video” (Video-RAG, MemoryBank, PhysMem, PCBs already exist) but the combination: structured event graphs for physical reranking, a fixed 15-family rule catalog with positive/negative cues, and an explicit verify-against-cues step rather than just stuffing similar text into the prompt. The experimental design is the strong part. Table 1 has component controls (caption, graph, memory-only, full). Table 2 isolates random vs matched memories, top-1 vs diverse top-3, and semantic/CLIP vs graph rerank. Table 3 shows the severe yes-prior in direct VLMs and that PhysMRV rebalances it, especially on GRASP. That is honest and useful.\n\nSoft spots, in proportion. Gains are moderate and backbone-dependent; the method does not fix a model that cannot use the evidence. Memory construction and test-time graphs both use Qwen3-VL-8B, rules/cues use GPT-5.5, so the stress-test worry about correlated LLM physics language is real—Table 2 shows internal consistency of the pipeline more than external fidelity of the graphs. No code/data release yet, and train–eval separation relative to TRAVL/ImplausiBench should be documented carefully. Free knobs (αp=0.2, top-50, top-3 diversity) are fixed without much sensitivity. None of that sinks the central claim; it just means the numbers need a referee and a release before we treat them as settled.\n\nWho it is for: people working on physical commonsense in VLMs, embodied priors, and training-free scaffolding. Citation pattern is appropriate. Math is light (cosine + weighted component match) and fine for the job. I would bring it to reading group, cite it if I am writing on physical VLM failures or memory-augmented verification, and send it to peer review.","headline":"Solid training-free systems paper: hierarchical physical memory + graph rerank + rule-cue verification lifts several frozen VLMs on three hard plausibility benches; gains real but moderate and backbone-dependent.","tokens_in":15221,"tokens_out":614,"would_cite":true,"duration_ms":5949,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Structured physical memories let frozen video-language models check physical plausibility without any training.","keywords":["physical plausibility reasoning","video-language models","hierarchical memory bank","physics-aware retrieval","event graphs","training-free verification","frozen VLMs"],"falsifier":"Replace the event-graph reranker with pure visual or pure semantic retrieval while keeping the same verifier prompt and memory bank; if accuracy and negative-class detection no longer improve over direct prompting on GRASP Level 2 and IntPhys2, the claim that structured physical matching is necessary collapses.","tokens_in":15125,"feed_emoji":"🧠","tokens_out":869,"duration_ms":7263,"temperature":0.7,"pith_summary":"Video-language models often recognize objects and actions yet still fail to judge whether an event obeys basic physical laws such as support, continuity, or contact. PhysMRV argues that the missing piece is not more fine-tuning but an external bank of structured physical precedents built from training videos. Each memory holds a scene caption, a physical-event graph of object interactions and state changes, and a rule-level summary with positive and negative diagnostic cues drawn from a fifteen-family physics catalog. At test time a coarse-to-fine retriever first recalls scenes by caption-question similarity, then reranks them by event-graph match and keeps a small set of rule-diverse anchors. A frozen model is then prompted to verify the current video against those retrieved rules and cues rather than answer from surface priors alone. Across three hard benchmarks and several strong backbones the method raises accuracy without updating any model parameters, and it measurably reduces the models’ strong bias toward calling every event plausible.","feed_headline":"Frozen video models check physics with structured memory","feed_subtitle":"Scene captions, event graphs and rule cues raise accuracy without any training","key_machinery":"Hierarchical Physical Memory Bank plus physics-aware coarse-to-fine retrieval: offline memories store scene captions, physical-event graphs, and rule-family cues; inference first recalls by caption-question embedding similarity, then reranks by weighted event-component match and selects rule-diverse top anchors that a frozen VLM uses as verification criteria.","core_discovery":"A hierarchical Physical Memory Bank of captions, event graphs, and rule-cue records, retrieved by physics-aware coarse-to-fine matching and used as explicit verification evidence, consistently improves frozen VLMs’ physical-plausibility judgments over direct prompting on ImplausiBench, IntPhys2, and GRASP Level 2 without any parameter updates.","pith_inferences":["The same hierarchical memory design could be applied to other latent-constraint domains where surface similarity is misleading, such as causal or counterfactual video reasoning.","Because gains depend on the backbone’s ability to use retrieved evidence, future work may need lighter adapters that help weaker instruction-following models actually read the verification cards.","Calibrated per-instance verification that grounds decisions in concrete object states and contacts observed in the current video remains the natural next bottleneck once memory retrieval is in place."],"forward_implications":["Frozen or proprietary VLMs can gain physical-plausibility skill at inference time without fine-tuning or access to model weights.","Retrieval for physical reasoning should be conditioned on interaction structure and rule families rather than surface semantic or visual similarity alone.","Explicit positive and negative diagnostic cues from matched precedents can correct the strong prior that frozen models treat almost every event as physically normal.","A reusable fifteen-family physics catalog plus event-graph memories can serve as a plug-and-play scaffold for other physical-reasoning benchmarks and backbones."],"fun_headline_variants":["Hierarchical memory bank boosts frozen VLMs on physics checks","Structured physical memories verify video plausibility without training","PhysMRV retrieves scenes graphs and rules for physics judgment","Frozen VLMs gain physical evidence from captions events and rules","Coarse-to-fine memory retrieval improves frozen model physics scores"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The event graphs and rule-cue records produced by the offline caption and language models must faithfully capture the true physical constraints, so that matching on them yields transferable physical precedents rather than merely similar-looking but physically mismatched cases.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical memory bank boosts frozen VLMs on physics checks","Structured physical memories verify video plausibility without training","PhysMRV retrieves scenes graphs and rules for physics judgment","Frozen VLMs gain physical evidence from captions events and rules","Coarse-to-fine memory retrieval improves frozen model physics scores"]},"model":"grok-4.5","effort":"low","cost_usd":0.005468,"raw_usage":{"total_tokens":1466,"prompt_tokens":784,"num_sources_used":0,"completion_tokens":64,"cost_in_usd_ticks":54680000,"prompt_tokens_details":{"text_tokens":784,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":618,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":784,"tokens_out":64,"duration_ms":6246,"temperature":1.0,"reasoning_tokens":618,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T13:37:18.599172+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Replace the event-graph reranker with pure visual or pure semantic retrieval while keeping the same verifier prompt and memory bank; if accuracy and negative-class detection no longer improve over direct prompting on GRASP Level 2 and IntPhys2, the claim that structured physical matching is necessary collapses.","supporting_citations":[],"review_version":1}