{"id":"3dca35ee-53b2-40c2-b1a6-eba8d21b28da","arxiv_id":"2411.09020","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A robot actively pushes and pulls objects, combining vision and touch to estimate mass, center of mass, and friction with a learned graph-based filter.","lead":"The paper describes a robot that pushes or pulls an object, watches how the object moves, and feels contact forces to estimate the object's mass, center of mass, and friction. This matters because autonomous robots in homes, warehouses, or factories need to understand unknown objects before manipulating them safely.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed 'consistent' superiority is contradicted by Tables II–III, and Table IV daily-object predictions miss ground truth by large margins with small claimed variance.","rationale":"In good faith, the paper does build a plausible extension of the authors' prior differentiable filter: it uses real-robot experiments, compares against its own feedforward baseline, includes ablations of active action selection, and demonstrates three downstream uses. Those are genuine strengths. However, the central claim as stated is about consistent superiority over baselines and uniform/random action selection. That claim is contradicted by the paper's own Tables II and III in several individual-parameter and per-link rows, and by the abstract's emphasis on real-world generalization, which Table IV directly undermines with large daily-object errors and apparently overconfident predicted variances. I therefore do not identify the quasi-static planar assumption as the most load-bearing weakness; that assumption is explicitly enforced by the low 25 mm/s action speed and is a scope condition rather than an internal contradiction. The internal evidence mismatch is more decisive because it concerns whether the experiments actually support the claimed conclusion. A CONDITIONAL verdict remains appropriate: the framework may be sound after correcting overclaims and supplying raw data, but the current text cannot be taken as a reliable demonstration of consistent superiority. The lack of code, data, and unresolved cross-references further supports the conditional status, though those are secondary to the numerical inconsistency.","tokens_in":29989,"tokens_out":4960,"duration_ms":54299,"concrete_test":"Obtain the per-object, per-interaction prediction logs and re-tabulate Tables II, III, and IV using paired per-object NRMSE, reporting how many of the 40 heterogeneous and 40 articulated test configurations actually have A-GNN as the best method. If the cells quoted above reproduce, the Section IV.B.3 'consistently' claim must be weakened to aggregate Overall rows, and Table IV's sugar and olive-oil friction entries must be re-measured; if those entries persist, the daily-object generalization claim fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing weakness is internal: the paper's own tables do not support the universal form of the central claim. Section IV.B.3 states that 'the proposed A-GNN consistently estimates the properties of objects with the least NRMSE,' and the Discussion repeats that A-GNN 'outperforms' A-FF. Table II (heterogeneous, pushing) contradicts this: A-GNN f_l2 is 0.21±0.11 versus R-GNN 0.10±0.06 and U-GNN 0.15±0.09; CoM_l2 is 0.16±0.12 versus U-GNN 0.09±0.06; Overall_l2 is 0.16±0.08 versus R-GNN 0.11±0.03. Table III (articulated, pushing) shows A-GNN mass_l1 0.08 versus A-FF 0.04 and R-GNN 0.05, and CoM_l1 0.07 versus A-FF 0.05. Thus the claim is true only for some aggregate rows, not consistently. Table IV's daily-object generalization is also problematic: sugar friction predicted 0.12±0.1 vs GT 0.5, olive-oil friction predicted 0.5±0.12 vs GT 0.15, coffee mass predicted 1.0±0.06 vs GT 0.67, hammer mass predicted 0.7±0.25 vs GT 1.2. If '±' is a predicted standard deviation, the filter is overconfident by several sigma in the real-world setting highlighted in the abstract. The quasi-static planar assumption is a real scope limit, but the implemented 25 mm/s actions and planar setups make it less decisive than the mismatch between the stated claim and the reported numbers.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a predictive visuo-tactile interactive perception framework for inferring time-invariant physical properties (mass, center of mass, surface friction, joint friction) of planar rigid objects, including homogeneous, heterogeneous, and articulated objects. The pipeline comprises active superquadric-based shape perception with next-best-view selection, a dual differentiable filter whose process model is a graph neural network (GNN), learned heteroscedastic observation noise models, and an N-step information-gain criterion for selecting push or pull actions. The framework is evaluated in real-robot experiments on 120 configurable 3D-printed objects and 8 daily objects, and is compared against the authors' prior feedforward-filter baseline (A-FF), an analytical pushing baseline, and uniform/random action-selection ablations. The paper additionally reports applications to pose tracking, goal-driven pushing, and detection of a changed support-surface environment.","tokens_in":30325,"tokens_out":4661,"duration_ms":49284,"significance":"If the central claims held as stated, this would be a noteworthy advance: a single learned framework that infers multiple physical properties across three object classes using only push/pull interactions and visuo-tactile observations. The experimental effort is substantial: real-robot data, 120 configurable objects with independently measured ground truth, an analytical baseline, an ablation study, and three downstream applications. The paper also ships a concrete graph-based interaction model that is more general than the earlier feedforward model. However, the load-bearing comparative claim—that A-GNN 'consistently' achieves the least NRMSE—is contradicted by the paper's own tables, and the daily-object results show large discrepancies between predictions and ground truth relative to the reported uncertainties. These issues must be resolved before the contribution can be accepted.","major_comments":[{"comment":"The claim that 'the proposed A-GNN consistently estimates the properties of objects with the least NRMSE' is not supported by the reported numbers. In Table II (heterogeneous, pushing), A-GNN has fl2 = 0.21±0.11 versus R-GNN 0.10±0.06, CoM l2 = 0.16±0.12 versus U-GNN 0.09±0.06, and Overall l2 = 0.16±0.08 versus R-GNN 0.11±0.03. In Table III (articulated, pushing), A-GNN mass l1 = 0.08±0.09 versus A-FF 0.04±0.03 and R-GNN 0.05±0.03, and CoM l1 = 0.07±0.05 versus A-FF 0.05±0.02. The Discussion itself concedes that 'for some parameters, A-FF performs better than A-GNN.' The authors should either substantially soften the 'consistently' claim, report statistical significance tests over the object configurations, or identify the precise conditions under which A-GNN is superior.","section":"IV.B.3, Tables II–III"},{"comment":"The daily-object generalization results raise serious calibration concerns. For example, sugar friction is predicted as 0.12±0.1 against a ground truth of 0.5; olive-oil friction is predicted as 0.5±0.12 against 0.15; coffee mass is predicted as 1.0±0.06 against 0.67; hammer mass is predicted as 0.7±0.25 against 1.2. If the reported '±' values are posterior standard deviations, the filter is overconfident by several sigma in exactly the real-world setting highlighted in the abstract. If they are not posterior standard deviations, their meaning should be clarified. The manuscript should report a calibration measure for the predicted uncertainties and discuss these failures rather than presenting the daily-object results without error analysis.","section":"Table IV"},{"comment":"The definition of the N-step information gain is internally inconsistent. Equation (17) defines IG_N(π_i) ≈ −E[ln bel_i(τ_N) − ln bel_i(τ_0)], but Equation (18) equates this to D_KL[N_i(τ_N) || N_i(τ_0)] = E_{τ_N}[ln N(τ_N) − ln N(τ_0)]. As written, maximizing the quantity in Eq. (17) would minimize the KL divergence in Eq. (18). Since the N-step information-gain action selector is one of the paper's stated contributions, this sign inconsistency must be corrected, and the authors should confirm that the implemented criterion is the one reported in the experiments.","section":"III.E, Eqs. (17)–(18)"},{"comment":"The superiority claims for active action selection over uniform and random selection are not backed by statistical analysis. Many of the reported mean NRMSE differences are smaller than the reported standard deviations (e.g., Table II, Overall pushing: A-GNN 0.15±0.06 vs. R-GNN 0.15±0.05; Table III, Overall pushing: A-GNN 0.15±0.03 vs. R-GNN 0.15±0.02). Without per-configuration error distributions, paired tests, or confidence intervals, the reader cannot determine whether the active strategy improves inference. The authors should either provide such an analysis or restrict their claims to qualitative observations about training efficiency.","section":"IV.B.3, Figs. 12–14"}],"minor_comments":[{"comment":"There is a typo in the sentence beginning 'Thee homogeneous, heterogeneous and articulated...' — 'Thee' should be 'The'.","section":"III.A"},{"comment":"The text states 'We present both quantitative and quantitative results on shape perception in Section ?? and in the Appendix.' 'quantitative' appears twice and the section cross-reference is unresolved.","section":"III.B.3"},{"comment":"The sentence 'The error in the pulling interaction is lower than that of pulling due to the constrained movement...' compares pulling with itself; the second instance should presumably be 'pushing'.","section":"IV.B.4"},{"comment":"Equation (17) has a missing closing parenthesis; the expression 'ln(beli(ψτN, ϕτN) − ln(beli(ψτ0, ϕτ0)' is incomplete.","section":"III.E, Eq. (17)"},{"comment":"The notation for the active feedforward baseline is inconsistent: the text uses 'Active − FF' in one place and 'A − FF' elsewhere. Please standardize.","section":"IV.B.2 and Fig. 10"},{"comment":"The quasi-static planar assumption is a legitimate scope limit, but the paper should state explicitly in the conclusions that the results do not claim to extend to fast, dynamic, or out-of-plane interactions, especially since the daily-object experiments are presented as a real-world validation.","section":"III.A and IV.A"}],"recommendation":"major_revision","confidential_remarks":"The paper is a substantial extension of the authors' prior IROS 2023 work and the experimental effort is considerable. The main barrier is not the validity of the framework as a whole but the gap between the stated claims and the reported numbers, particularly the 'consistent' superiority claim and the calibration of uncertainties in the daily-object experiments. I would ask for a revised version that recalibrates the claims, adds statistical testing, corrects the information-gain sign issue, and addresses Table IV explicitly. No code or data repository is mentioned; requesting one would strengthen reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a real extension of the authors' IROS 2023 differentiable filter, and the real-robot experiments give it substance. But the paper's own tables do not support the central claim that A-GNN consistently gives the least NRMSE, and the daily-object generalization results show errors that are too large to wave off. I would send it to peer review, but I would expect a substantial revision.\n\nWhat is genuinely new: replacing the feedforward interaction model with a causal graph neural network, adding prehensile pulling alongside pushing, an active superquadric shape-perception module with next-best-view selection, and an N-step information-gain action selector. The experimental apparatus is a real point in its favor: 120 configurable 3D-printed objects, independent ground-truth characterization of mass, friction, CoM, and joint friction, plus a non-trivial analytical baseline for homogeneous pushing. The shape-perception results, the training-efficiency comparison, and the downstream pose-tracking/control demonstrations all look like honest engineering work. I also do not hold the authors' use of their own A-FF baseline against them; updating its visual model and comparing on equal footing is legitimate.\n\nThe soft spots are real and mostly internal. Section IV.B.3 says A-GNN consistently estimates properties with the least NRMSE, but Tables II and III contradict that in multiple cells: for heterogeneous pushing, R-GNN beats A-GNN on f_l2 and overall_l2, and for articulated pushing, A-FF beats A-GNN on mass_l1 and CoM_l1. The Discussion's quieter admission that A-FF sometimes does better is closer to the data. Table IV is more worrying: sugar friction is predicted as 0.12 ± 0.1 against a true value of 0.5, olive-oil friction as 0.5 ± 0.12 against 0.15, coffee mass as 1.0 ± 0.06 against 0.67. If those ± values are predicted standard deviations, the filter is overconfident by several sigma in the exact real-world setting the abstract highlights. That needs a discussion, a recalibration, or a weaker claim. The quasi-static planar assumption is a genuine scope limit, but the implemented 25 mm/s pushes and planar setups make it less decisive than the claim-data mismatch. There are also unfinished cross-references (\"Section ??\"), typos, and no code or data release, which makes independent verification harder.\n\nWho this is for: researchers working on interactive perception and tactile-based manipulation, especially those interested in differentiable filtering and active exploration. The framework is worth reading even if the headline claim needs to be trimmed.\n\nRecommendation: accept for review, but the referee should require the authors to either fix the consistency claim to match their tables or provide the missing evidence, and to address the Table IV overconfidence directly. Code or data would materially increase confidence.","headline":"A credible extension of the authors' own differentiable-filter line with real robot data, but the reported tables do not support the 'consistent' superiority claim and daily-object predictions show overconfidence; it deserves review but needs revision.","tokens_in":30860,"tokens_out":1714,"would_cite":false,"duration_ms":19868,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A predictive visuo-tactile framework lets a robot infer the mass, center of mass, and friction of unfamiliar planar objects through active pushing and pulling, reporting better accuracy than prior feedforward and non-active baselines.","keywords":["Visual and Tactile Sensing","Active Interactive Perception","Recursive Bayesian Filtering","Graph Neural Networks","Physical Property Inference","Superquadrics","Action Selection","Robotic Manipulation"],"falsifier":"Push the same object with identical mass, center of mass, and surface friction but with rotational inertia changed (for example, by moving internal weights outward), and increase push velocity; if the filter's predicted pose error or estimated parameter error grows with velocity or with the inertia change, the claim that $I_z$ does not affect the interaction and the quasi-static premise are falsified.","tokens_in":29765,"feed_emoji":"🤖","tokens_out":7481,"duration_ms":64704,"temperature":0.7,"pith_summary":"The paper claims that a robot can autonomously infer the physical properties of previously unseen planar objects—mass, center of mass, surface friction, and joint friction—by actively pushing or pulling them and combining vision with touch. The proposed predictive perception framework estimates object shape with superquadrics, learns a graph-neural-network model of the object-robot interaction, and uses that learned model inside a dual differentiable filter to track pose while estimating time-invariant parameters. An N-step information gain criterion selects the most informative next actions, improving data efficiency. In real-robot experiments across homogeneous, heterogeneous, and articulated objects, the framework reports lower normalized root-mean-square error than a feedforward baseline and than uniform or random action selection, and it demonstrates gains in pose tracking, goal-driven pushing, and detecting a change in the environment.","feed_headline":"Robots infer mass, friction, and center of mass by pushing and pulling","feed_subtitle":"A visuo-tactile predictive framework estimates unseen object properties with fewer interactions than prior baselines.","key_machinery":"The machinery is a dual differentiable filter with a causal graph neural network (GNN) as its learned process model: a recursive Bayesian filter (Unscented-Kalman-style) that maintains a joint Gaussian belief over per-link pose and twist and over time-invariant inertial and frictional parameters. The GNN represents the robot, the object's links, and the support table as nodes, with directed edges carrying contact-force information; a cause-effect message-passing scheme updates node and edge features so that the model predicts both the object's next pose and the tactile forces on the robot. The dual structure updates the parameter belief from observation likelihoods and then updates the pose belief conditionally on the parameters, while constrained Monte Carlo sampling keeps physically invalid sigma points out of the dynamics while preserving uncertainty. This same machinery provides the predictive distribution used by the N-step information gain action selector, and lets one learned model serve homogeneous, heterogeneous, and articulated objects.","core_discovery":"The central discovery is that a single learnable framework can estimate the inertial and frictional parameters of diverse planar rigid objects without prior knowledge of their properties, using simple non-prehensile push and prehensile pull interactions. The state is split into time-varying pose and twist per link plus time-invariant parameters; a dual differentiable filter maintains a joint Gaussian belief over both. The process model is a graph neural network whose directed edges encode the causal transfer of forces from robot to object to support table, and its node and edge updates predict both visual (projected point cloud) and tactile (contact force) observations. Active shape perception via Bayesian superquadric fitting provides initial shape and pose, and an N-step information gain action selector chooses the next push or pull. On a real robotic setup with 120 configurable objects and eight daily objects, the proposed approach A-GNN consistently achieves the lowest normalized root-mean-square error for the inferred parameters compared with the A-FF baseline and uniform or random action selection, and the estimated parameters improve downstream pose tracking and goal-driven pushing.","pith_inferences":["An implication the authors leave implicit is that the dual-filter-plus-GNN architecture could estimate time-varying properties, such as a shifting center of mass in a liquid container or a deformable object, by enlarging the state space, since the filter already separates time-varying pose from time-invariant parameters.","The finding that predicted tactile noise is much lower than visual noise during prehensile pulling suggests a learned sensor-weighting rule: when the robot arm occludes the camera, the filter should rely more on touch, and the noise models could be used to trade off modalities explicitly.","The quasi-static assumption and fixed push velocity mean the framework is currently tuned for slow, planar exploration; a testable extension is to let the N-step information gain also select push velocity, which would probe whether relaxing the $I_z$ omission extends the method to heavier or taller objects.","The causal graph representation's success in encoding force transmission suggests it could transfer to bimanual or multi-robot interactions, where contact forces pass through multiple agents before returning to the sensor."],"forward_implications":["A robot using this framework can begin with no knowledge of an object's mass, friction, or center of mass and converge on estimates through a small number of push or pull interactions, with the filter reporting uncertainty alongside each estimate.","The same graph representation handles single-link, multi-link rigid, and articulated objects, so the approach removes the need to know the number of links or a kinematic model in advance.","Active N-step action selection reduces the number of interactions needed to train the dynamics model compared with uniform or random exploration, which matters when robot time is expensive.","Estimated parameters improve pose tracking during interaction and reduce goal-reaching error in model-predictive push control, so the inferred properties are directly useful for downstream manipulation.","The learned observation likelihood can flag changes in the environment, such as a tilted support surface, enabling a robot to detect when its interaction model is out of distribution."],"supporting_citations":[{"why":"Supplies the baseline A-FF dual differentiable filter that this work extends and compares against.","marker":"[22]"},{"why":"Provides the Bayesian EMS superquadric recovery method used for active shape perception and next-best-view computation.","marker":"[25]"},{"why":"Provides the graph-network message-passing framework that the causal robot-object-table interaction model is built on.","marker":"[59]"},{"why":"Supplies the differentiable Unscented Kalman filter training methodology that underlies the dual filter.","marker":"[79]"},{"why":"Gives the analytical quasi-static pushing model used as the A-Analytical baseline for homogeneous objects.","marker":"[84]"}],"fun_headline_variants":["Pushing and pulling robots learn hidden object physics","Visuo-tactile AI estimates object properties with fewer pokes","Robot pokes and pulls to read mass and friction","Dual-filter GNN learns object physics from push-pull actions","Active push-pull explores hidden object properties"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes quasi-static planar interaction: pushes and pulls are slow enough that out-of-plane motion and rotational inertia do not affect the dynamics, and the friction between robot and object is approximately known from the tactile sensor.","fun_headline_variants_meta":{"raw":{"variants":["Pushing and pulling robots learn hidden object physics","Visuo-tactile AI estimates object properties with fewer pokes","Robot pokes and pulls to read mass and friction","Dual-filter GNN learns object physics from push-pull actions","Active push-pull explores hidden object properties"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000507,"raw_usage":{"total_tokens":2502,"prompt_tokens":1004,"completion_tokens":1498,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":1419}},"tokens_in":620,"tokens_out":1498,"duration_ms":10278,"temperature":1.0,"reasoning_tokens":1419,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:09:19.890327+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Push the same object with identical mass, center of mass, and surface friction but with rotational inertia changed (for example, by moving internal weights outward), and increase push velocity; if the filter's predicted pose error or estimated parameter error grows with velocity or with the inertia change, the claim that $I_z$ does not affect the interaction and the quasi-static premise are falsified.","supporting_citations":[{"cited_title":"Push to know!-visuo-tactile based active object parameter inference with dual differentiable filtering,","cited_arxiv_id":null,"evidence_quote":"Supplies the baseline A-FF dual differentiable filter that this work extends and compares against."},{"cited_title":"Robust and accurate superquadric recovery: A probabilistic approach,","cited_arxiv_id":null,"evidence_quote":"Provides the Bayesian EMS superquadric recovery method used for active shape perception and next-best-view computation."},{"cited_title":"Graph networks as learnable physics engines for inference and control,","cited_arxiv_id":null,"evidence_quote":"Provides the graph-network message-passing framework that the causal robot-object-table interaction model is built on."},{"cited_title":"How to train your differentiable filter,","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable Unscented Kalman filter training methodology that underlies the dual filter."},{"cited_title":"Manipulation and active sensing by pushing using tactile feedback","cited_arxiv_id":null,"evidence_quote":"Gives the analytical quasi-static pushing model used as the A-Analytical baseline for homogeneous objects."}],"review_version":1}