{"id":"d4f461b2-87ee-4bc8-bd65-9dc5f0760dca","arxiv_id":"2508.01984","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"IMoRe couples a MAC-style memory network with program-function embeddings and multi-level ViT motion features to reach state-of-the-art accuracy on Babel-QA and a new HuMMan-QA benchmark.","lead":"This paper introduces IMoRe, a model that answers natural language questions about 3D human motion by letting structured program functions guide an implicit memory-based reasoning process, without hand-built functional modules. It reports accuracy gains over prior work on the Babel-QA benchmark and introduces a new HuMMan-QA dataset to show the approach transfers across motion datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Backbone confound: the SOTA comparison against NSPose is not controlled; ViT+NSPose is an incompatible baseline, so the claimed superiority of implicit program reasoning is not isolated.","rationale":"The reader identified external program availability as the weakest assumption, which is a real limitation. However, the more load-bearing concern for the central 'outperforms explicit neuro-symbolic execution' claim is the uncontrolled backbone difference in the main comparison. The ablation 'ViT + NSPose' (Table 3, row A) is not a fair control: NSPose was designed for a different input representation, and forcing it onto ViT features without adaptation gives a misleadingly low baseline. Thus the observed superiority could be explained by the stronger pretrained encoder rather than by the implicit program-guided reasoning mechanism. This does not invalidate the paper's contribution, but it means the headline claim is not yet fully established. Since the reader's verdict is already CONDITIONAL, this additional concern does not change the verdict; it reinforces the conditionality. The proposed matched-backbone experiment would settle whether the reasoning module itself carries the advantage, and should be reported before accepting the SOTA claim without qualification.","tokens_in":155,"tokens_out":3990,"duration_ms":57418,"concrete_test":"Re-run the Babel-QA comparison with matched backbones: (1) fine-tune NSPose on the Motion ViT features by adding a learned linear projection before NSPose's first module and tune its hyperparameters; (2) also train IMoRe with the original NSPose encoder. If NSPose+ViT reaches at least 0.609 overall, or IMoRe with the NSPose encoder drops below 0.578, the reported SOTA gap is not attributable to implicit program reasoning and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that implicit program-guided reasoning outperforms explicit neuro-symbolic execution. The headline numbers in Table 1 compare IMoRe (using the pre-trained Motion ViT of [38]) against NSPose (using its original encoder). Section 4.3 attempts to control for this with 'ViT + NSPose', but this baseline drops to 0.440, below NSPose's native 0.578. The authors read this as evidence that the gain is due to reasoning ('the better performance of our IMoRe comes from a better reasoning module'), but the drop more plausibly shows that NSPose's explicit modules are not compatible with patch-based ViT features; this is a strawman baseline. No experiment tests IMoRe with the same encoder as NSPose, nor does any experiment adapt NSPose to the ViT feature space fairly. Consequently, the 3.1-6.2 point overall gap attributed to implicit program guidance could be substantially due to the stronger backbone, and the central claim that the reasoning architecture itself outperforms explicit neuro-symbolic execution is not yet isolated. The same issue applies to the HuMMan-QA results, where only the full IMoRe system is reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IMoRe, an implicit program-guided reasoning framework for human motion question answering. Instead of executing handcrafted neuro-symbolic modules as in NSPose, IMoRe uses a MAC-style iterative memory network whose reasoning steps are conditioned on structured program functions, together with a program-guided reading mechanism that selects from multi-level features of a pretrained motion ViT. The authors evaluate on the Babel-QA benchmark and introduce a new HuMMan-QA dataset. They report state-of-the-art results: IMoRe II reaches 64.0% overall on Babel-QA versus 57.8% for NSPose, and 73.0% versus 69.1% on HuMMan-QA. They also report robustness to predicted programs, an ablation study, and comparisons with video-language model baselines.","tokens_in":14147,"tokens_out":5114,"duration_ms":53275,"significance":"If validated, the paper makes a useful architectural contribution: replacing handcrafted explicit modules with program-conditioned implicit reasoning, while retaining interpretability through program structure and intermediate attention maps. The release of code and a new HuMMan-QA dataset, the inclusion of a predicted-program setting, and the addition of VLM baselines are all strengths. However, the central claim that implicit program-guided reasoning outperforms explicit neuro-symbolic execution is not yet isolated from two confounding factors: the motion encoder is not matched across comparisons, and the best IMoRe II result uses a test-time best-of-five selection rule not applied to baselines. These issues require additional experiments before the headline performance gain can be attributed to the reasoning architecture.","major_comments":[{"comment":"The ablation intended to control for the motion encoder does not isolate the reasoning contribution. Row A, 'ViT + NSPose', drops to 0.440 overall, below NSPose's native 0.578 in Table 1, and the paper interprets this as evidence that the gain comes from the reasoning module. However, NSPose's explicit modules were designed for the original joint-level encoder, and patch-based ViT features are likely incompatible with them; the row is therefore a strawman baseline. The paper provides no experiment running IMoRe with NSPose's encoder and no fair adaptation of NSPose to ViT features, so the 3.1–6.2 point gap in Table 1 may be substantially due to the stronger backbone. Please add an encoder-matched comparison or explicitly qualify the claim.","section":"§4.3, Table 3"},{"comment":"The headline IMoRe II result uses a test-time best-of-five selection rule: 'During validation and testing, we execute five runs for the same sequence and select the run with the highest logits scores.' This selection is not applied to any baseline or to IMoRe I, so the Table 1 comparison between IMoRe II (0.640) and NSPose (0.578) is not apples-to-apples. Since the selection is an oracle-like operation over random 224-frame windows, it can only help IMoRe II. Please report IMoRe II without highest-logit selection, and/or apply the same best-of-k procedure to baselines that can use it. In addition, the three-run averages are reported without standard deviations or significance tests; with gaps as small as 3.1 points, this leaves the reader unable to assess whether differences are reliable.","section":"§4, Implementation Details"},{"comment":"The predicted-program robustness claim is not fully supported. The text says only that 'we adapt the coarse-to-fine two-stage program generation network from [9]' to predict programs, with no details about training data, supervision, or predicted-program accuracy on Babel-QA. It is also unclear whether the NSPose* numbers in Table 1 use the same predicted programs as IMoRe I* and IMoRe II*. Without these details, the conclusion that implicit reasoning degrades only 1–2% while explicit reasoning degrades more under noisy programs cannot be evaluated. Please describe the program predictor, report its accuracy, and state whether both methods consume identical predicted programs.","section":"§4, Implementation Details; §4.1, Table 1"},{"comment":"The new HuMMan-QA dataset is a core contribution, but its label quality is not documented enough to support the generalization claim. The paper states that GPT-4o generated labels 'which were then meticulously refined and manually verified against the actual motion sequences by the authors.' No verification protocol, inter-annotator agreement, or per-question-type error analysis is provided. Since the labels for action, direction, and body part come from a model plus manual checking rather than from existing annotations, the benchmark's reliability needs quantitative support. Please include annotation statistics and, if possible, a small human-study comparison. Also clarify how ground-truth programs are defined for HuMMan-QA questions.","section":"§4, Datasets"}],"minor_comments":[{"comment":"The sentence preceding Eq. (3) says the attention uses 'the key and query', but it should say 'the key and value'; Eq. (3) itself correctly specifies K=hm, V=hm.","section":"§3.3, Eq. (3)"},{"comment":"Row D of Table 3 reports 0.000 for the body-part 'BTW' condition, while Table 1 reports 0.083 for IMoRe II in the same condition; these should be reconciled if Row D is intended to be the full IMoRe II model.","section":"Table 3 vs Table 1"},{"comment":"The affiliations use the number 1 for both the Institute of High-Performance Computing and the Centre for Frontier AI Research; the second affiliation should be numbered 2.","section":"Title page"},{"comment":"Figure 3 appears to contain repeated panels and inconsistent captions (for example, multiple 'Query Action (Relation: After)' blocks with the same question); please replace with a clean set of distinct qualitative examples.","section":"Figure 3"},{"comment":"The paper alternates between 'NS-Pose' and 'NSPose'; please use a single consistent name for the baseline.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The architecture is plausible and the empirical results are promising, but the central claim requires an encoder-matched comparison and a fair treatment of the IMoRe II test-time selection rule. The new dataset's label-quality documentation also needs strengthening. These are fixable with additional experiments and reporting, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is a workmanlike extension of MAC-style implicit reasoning to human motion QA. The new ingredients are conditioning the reasoning steps on program functions rather than question words, and a multi-level ViT reading mechanism that selects features per reasoning step. It also introduces HuMMan-QA, a new benchmark, and ships code and dataset. That is real value.\n\nThe strongest evidence is the ablation in Table 3. Within the same ViT backbone, implicit MAC gets 0.582, adding program guidance gets 0.607, and adding the multi-level feature selection gets 0.640. That cleanly isolates the contribution of each piece, and it is a fair comparison. The predicted-program results are also informative: the method degrades only 1-2% instead of the 10+ point drop NSPose shows, which supports the robustness argument. The supplementary attention maps give plausible qualitative support.\n\nThe soft spot is the comparison against NSPose. The paper claims superiority of implicit over explicit neuro-symbolic reasoning, but IMoRe uses a stronger, pretrained ViT encoder while NSPose uses its own. The attempted control, ViT+NSPose, drops to 0.440, which likely says more about NSPose's modules being incompatible with patch-based ViT features than about reasoning quality. So the headline 6-point gap is not fully attributable to the reasoning module. The internal ablation partly compensates, but the paper should be more careful in its claims. The best-of-five test-time selection for IMoRe II is another advantage the baselines do not get, and there are no error bars for the three runs. The HuMMan-QA labels come from GPT-4o plus manual checking, which is acceptable but should be released with the dataset for scrutiny.\n\nWho is this for? Researchers working on motion QA or on neuro-symbolic reasoning over video/motion. It is not a breakthrough, but it is a clean, reproducible step forward. I would send it to peer review, with the request that the authors reanalyze the ablation and either rerun NSPose in a fair setting or tone down the SOTA claim.","headline":"Solid architecture paper for motion QA, but the headline SOTA claim is confounded by the backbone; the internal ablation is the part to trust.","tokens_in":14710,"tokens_out":2026,"would_cite":true,"duration_ms":25893,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Implicit program-guided reasoning, not handcrafted modules, carries human motion question answering to new best accuracies.","keywords":["human motion question answering","program-guided reasoning","implicit reasoning","neuro-symbolic reasoning","Memory-Attention-Composition network","motion Vision Transformer","Babel-QA","HuMMan-QA"],"falsifier":"Run IMoRe and NSPose on Babel-QA with the same motion encoder and with programs replaced by random or empty programs; if IMoRe's margin over NSPose persists without meaningful program information, the reported gain is not driven by program guidance. A second check is to report IMoRe II's accuracy for a single randomly chosen 224-frame window instead of the best of five; if single-window accuracy falls much below 64.0%, part of the headline result comes from window selection rather than reasoning.","tokens_in":13720,"feed_emoji":"🧠","tokens_out":6740,"duration_ms":70554,"temperature":0.7,"pith_summary":"This paper argues that human motion question answering does not need handcrafted functional modules, and that implicit program-guided reasoning can do the job better. The authors propose IMoRe, a memory-based reasoning network that conditions every reasoning step on a structured program's functions rather than on soft attention over question words, and that reads from multiple layers of a pretrained motion Vision Transformer. On the Babel-QA benchmark its strongest setting reaches 64.0% overall accuracy, against 57.8% for the explicit neuro-symbolic baseline NSPose, and it retains most of that advantage when programs are predicted rather than given. The authors also introduce a HuMMan-QA dataset and report that IMoRe generalizes to it, reaching 73.0% versus 69.1% for NSPose. If correct, the contribution is a single adaptable reasoning architecture that removes the manual module design bottleneck while keeping the interpretability of program structure.","feed_headline":"Program-guided reasoning beats modular motion QA by 6 points","feed_subtitle":"A single memory network conditioned on program functions outscores explicit neuro-symbolic execution and carries to a new dataset.","key_machinery":"The load-bearing object is the implicit program-guided reasoning module, a MAC-style Memory-Attention-Composition cell with control, read, and write operations, but with the control signal taken directly from structured program functions instead of inferred by soft attention over question words. At step $i+1$ the module first attends to previous memory states using program function $P_{i+1}$ as the query, then reads from a knowledge pool of projected multi-level features $F'_m = \\{h'_1, \\dots, h'_M\\}$ extracted from six middle layers and the final layer of a pretrained motion Vision Transformer. The program's dependency structure, such as $\\text{relate}(\\text{before})$ depending on $\\text{filter}(\\text{left})$, tells the network which earlier memory states matter, and the program-guided reading mechanism chooses which feature level to attend to, so action queries can use high-level semantics while body-part queries use fine-grained local detail. In the IMoRe II variant, 224-frame windows are randomly sampled from each sequence and the best-scoring of five test windows is kept, a protocol the paper uses because the highest-logit segment should contain the queried concept. This replaces the handcrafted modules of explicit neuro-symbolic execution while preserving an interpretable step-by-step trace.","core_discovery":"The central claim is that symbolic program functions should serve as a conditioning signal for a unified implicit reasoning network, not as executable instructions for handcrafted modules. In IMoRe, the program P is read step by step: each reasoning step attends to the relevant previous memory states and to motion features selected from a multi-level knowledge pool built from six intermediate layers plus the final output of the motion ViT. This replaces MAC's soft attention over question words with explicit program guidance, and replaces NSPose's separate functional modules with one memory network. The paper reports that this design yields 64.0% overall accuracy on Babel-QA in the ground-truth program setting (IMoRe II), beats NSPose's 57.8%, and that with predicted programs the drop is roughly 2.5 points for IMoRe II whereas NSPose drops about 10.6 points. On the new HuMMan-QA dataset IMoRe reaches 73.0% versus 69.1% for NSPose, which the authors take as evidence that the approach generalizes across datasets. The paper also argues that implicit reasoning's resilience to noisy program predictions is a structural advantage over explicit execution, where one mistaken step can cascade.","pith_inferences":["Editorial extension: the same program-conditioned implicit reasoning recipe could apply to other temporal reasoning tasks, such as video QA or instruction following, whenever a structured program can be supplied.","Editorial extension: the best-of-five random-frame selection used in IMoRe II is an optimistic oracle; reporting single-window accuracy and the variance across the five runs would show how much of the 64.0% gain depends on that selection rather than on reasoning.","Editorial extension: a controlled noise experiment on programs, varying the fraction of randomly replaced functions, would directly test the paper's claim that implicit reasoning degrades gracefully while explicit execution cascades.","Editorial extension: the attention visualization of feature-level selection suggests a testable prediction—if lower ViT layers are removed from the knowledge pool, body-part accuracy should drop more than action accuracy."],"forward_implications":["If the central claim holds, new motion QA systems can be built without designing one module per operation type; a single reasoning stack conditioned on programs handles action, direction, and body-part questions.","Predicted-program robustness becomes a real deployment advantage: the paper reports only a 1-2% drop for IMoRe under predicted programs while the explicit baseline NSPose loses roughly ten points, implying the implicit network can absorb noisy control signals.","The multi-level reading mechanism implies that a single pretrained motion transformer can supply both the global action semantics and the local body-part cues needed for fine-grained reasoning, so feature-level selection is part of the reasoning itself.","The introduced HuMMan-QA dataset, with 2066 training, 524 validation, and 533 test questions generated with GPT-4o and manually verified, provides a second benchmark that future motion QA work can use to test generalization."],"supporting_citations":[{"why":"Defines the Babel-QA benchmark, the NSPose baseline with explicit handcrafted functional modules, and the direct feature-fusion baselines that IMoRe must beat.","marker":"[11]"},{"why":"Supplies the MAC Memory-Attention-Composition architecture whose control, read, and write units IMoRe reworks into program-conditioned reasoning.","marker":"[16]"},{"why":"Supplies the pretrained motion Vision Transformer with motion patches whose intermediate layers form the multi-level knowledge pool for program-guided reading.","marker":"[38]"},{"why":"Provides the HuMMan-MoGen motion dataset from which the paper constructs the new HuMMan-QA benchmark.","marker":"[39]"},{"why":"Supplies the coarse-to-fine program predictor used to generate predicted programs in the robustness experiments.","marker":"[9]"}],"fun_headline_variants":["Program-guided memory beats explicit neuro-symbolic QA","IMoRe: Program conditioning, not execution, for motion QA","Robust to noisy programs: IMoRe's memory reasoning wins","IMoRe survives noisy programs that break explicit QA","No handcrafted modules: IMoRe uses program-guided memory"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every question comes with a structured program P whose functions align with the reasoning steps, and that the pretrained motion transformer's layers genuinely separate high-level action semantics from fine-grained body-part detail; if either fails, the program signal and the reading mechanism lose their value.","fun_headline_variants_meta":{"raw":{"variants":["Program-guided memory beats explicit neuro-symbolic QA","IMoRe: Program conditioning, not execution, for motion QA","Robust to noisy programs: IMoRe's memory reasoning wins","IMoRe survives noisy programs that break explicit QA","No handcrafted modules: IMoRe uses program-guided memory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00192,"raw_usage":{"total_tokens":7536,"prompt_tokens":981,"completion_tokens":6555,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":6470}},"tokens_in":597,"tokens_out":6555,"duration_ms":49515,"temperature":1.0,"reasoning_tokens":6470,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:15:20.083207+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run IMoRe and NSPose on Babel-QA with the same motion encoder and with programs replaced by random or empty programs; if IMoRe's margin over NSPose persists without meaningful program information, the reported gain is not driven by program guidance. A second check is to report IMoRe II's accuracy for a single randomly chosen 224-frame window instead of the best of five; if single-window accuracy falls much below 64.0%, part of the headline result comes from window selection rather than reasoning.","supporting_citations":[{"cited_title":"Motion question answering via modular motion programs","cited_arxiv_id":null,"evidence_quote":"Defines the Babel-QA benchmark, the NSPose baseline with explicit handcrafted functional modules, and the direct feature-fusion baselines that IMoRe must beat."},{"cited_title":"Exploring vision transformers for 3d human motion-language models with motion patches","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained motion Vision Transformer with motion patches whose intermediate layers form the multi-level knowledge pool for program-guided reading."},{"cited_title":"Finemogen: Fine-grained spatio- temporal motion generation and editing","cited_arxiv_id":null,"evidence_quote":"Provides the HuMMan-MoGen motion dataset from which the paper constructs the new HuMMan-QA benchmark."}],"review_version":1}