{"id":"6fa28532-845c-4f7d-ab51-58b8ef0c7595","arxiv_id":"2504.18719","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Vysics reconstructs occluded object geometry and inertial properties by fusing RGBD vision with contact-rich physics from a short robot interaction.","lead":"Vysics fuses an RGBD camera video with the physics of a robot pushing an object to reconstruct the object's full 3D shape, including parts the camera never sees. It builds a robot-ready model, shape plus mass properties, from a roughly ten-second video, which matters for robots that must manipulate unfamiliar objects in cluttered scenes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Contact-inferred geometry depends on hand-set friction coefficients (0.26 table, 0.15 robot) that are neither identified nor ablated; a sensitivity analysis is needed to confirm 'physible' geometry is not an artifact of these values.","rationale":"I agree with the reader's identification of the convex-hull/friction assumption as the weakest link, and I sharpen it to the hand-set friction coefficients. The paper's central claim—that contact-inferred 'physible' geometry improves both reconstruction accuracy and dynamics prediction—requires that the contact forces inferred by PLL are a faithful inverse rendering of the true physics. Two assumptions underpin this: (i) all contacts occur on the convex hull, and (ii) the friction coefficients are correct. Assumption (i) is a representational limit that likely holds for the seven approximately convex objects tested, so it threatens external validity more than the reported results. Assumption (ii) applies to every experiment, since the coefficients are fixed at 0.26/0.15 and the paper admits friction is not identifiable from the sliding motions. Because the PLL loss couples geometry, inertia, and friction, misspecified friction can be absorbed by shifting the support points, corrupting the supervision that drives Vysics' improvement over BundleSDF. This is exactly the sort of parameter-sensitivity failure that would make the headline numbers misleading even with perfect RGBD, and it is directly testable. I propose a friction sensitivity study as the decisive check. If geometry is stable under plausible friction variations, the concern is resolved and the paper's mechanism is credible. If not, the geometric and dynamics claims should be conditioned on the availability of accurate friction parameters. I do not change the reader's CONDITIONAL verdict, but the acceptance conditions should include this sensitivity analysis. The in-sample dynamics evaluation is a legitimate secondary concern about the strength of the 'prediction' claim, and the exclusion of BundleSDF tracking failures should be disclosed, but neither attacks the core mechanism as directly as the friction dependence.","tokens_in":17576,"tokens_out":10690,"duration_ms":107680,"concrete_test":"Run Vysics on at least 3 sessions per object with object-table mu in {0.15, 0.26, 0.40} and object-robot mu in {0.08, 0.15, 0.30} (full factorial or Latin hypercube), keeping all other settings and random seeds fixed. Report per-session chamfer distance, IoU, and the average displacement of the filtered support-point set P. If the IoU varies by more than 0.05 or the chamfer distance by more than 30% between any two settings, the hand-set friction is load-bearing and the reported improvements over BundleSDF are not robust; if the geometry is stable, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism infers occluded geometry from contact dynamics: PLL hypothesizes contact forces that explain the BundleSDF trajectory and returns support points (n_p, s_p) that then supervise the SDF via Eq. (8), (11), and the convexity loss (13). Those contact forces are only meaningful if the Coulomb friction coefficients are correct. The paper fixes mu_table=0.26 and mu_robot=0.15 for all objects and states in §V-A that sliding motions in the dataset make unique friction identification difficult. Under misspecified friction, the optimizer can trade off friction-cone violations against geometry: a higher true table friction would require different normal forces or lever arms, shifting the inferred support points and hyperplane constraints. Eq. (6) also pulls the DSF toward the visible mesh, so the final SDF is a blend of vision and potentially biased physics. If the bias is large, the claimed geometric improvement over vision-only baselines could be an artifact of the chosen friction values rather than of genuine contact information. The representational limit that the DSF is a convex hull (§III-B) compounds this: any contact with a concave region is mapped to the hull, further biasing the supervision. No ablation or sensitivity analysis over friction is reported, so the influence of this hand-set parameter on the headline chamfer/IoU numbers is unknown.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Vysics proposes to reconstruct a rigid object's geometry from a short RGBD video by fusing BundleSDF's vision-based tracking/reconstruction with PLL's contact-implicit dynamics learning. The pipeline first runs BundleSDF to obtain poses and a visible mesh, runs PLL (with a convex-hull DSF) to infer contact forces, support points, and inertia under both dynamics and vision losses, then runs BundleSDF a second time with support-point, hyperplane, and convexity losses derived from the PLL output. The method is evaluated on a new dataset of Franka Panda interactions under occlusion, reporting lower chamfer distance and higher IoU than vision-only baselines, and better open-loop dynamics rollouts than a BundleSDF-geometry baseline. The authors also release a URDF output and dataset.","tokens_in":17867,"tokens_out":6957,"duration_ms":70135,"significance":"The central idea is attractive and timely: contact observations during manipulation are an underused cue for completing occluded geometry, and the paper's formulation via support points and hyperplane constraints is clear and well motivated. If the reported geometry improvements hold, the work is a useful step beyond vision-only completion, with the practical advantage of requiring no pretraining or tactile sensors. The geometric evaluation against ground-truth meshes (chamfer distance and IoU) is an appropriate external check, and the new RGBD dataset is a valuable resource. The loss derivations in Eqs. (7)-(13) are mathematically sound under the stated convexity assumption. However, the dynamics-prediction evaluation currently appears to be an in-sample fit, and the hand-set friction coefficients are not subjected to sensitivity analysis; until these are addressed, the breadth of the claims should be considered provisional.","major_comments":[{"comment":"The dynamics-prediction evaluation compares open-loop simulations against the same BundleSDF trajectory that was used to train PLL and the second BundleSDF pass. Because the model is optimized to reproduce that exact trajectory, the reported reductions in pose error and time-before-divergence on that trajectory can reflect overfitting to the training signal rather than genuine predictive skill. Please evaluate on held-out interaction sessions for each object, or explicitly frame the metric as training-set fit and support the generalization claim with cross-validation.","section":"§V-B, §VI-B"},{"comment":"The 'physible' geometry that supervises the final SDF via Eqs. (8) and (11) is derived from PLL's hypothesized contact forces under Coulomb friction with hand-set coefficients mu_table=0.26 and mu_robot=0.15 for all objects. The paper notes in §V-A that friction is difficult to identify uniquely in this dataset, yet no sensitivity study or ablation over these values is reported. Under misspecified friction, the optimizer can trade friction-cone violations against geometry, shifting support points and hyperplane constraints and thus biasing the final shape. Please add a sensitivity analysis over a plausible range of friction values, or otherwise show that the chamfer/IoU results are insensitive to these choices.","section":"§V-A, §IV-B"},{"comment":"The DSF can only represent the convex hull, and the paper assumes contacts occur only on the object's convex hull. This assumption is central because any contact on a concave region maps to a support point on the hull rather than the true contact point, directly corrupting the supervision in Eqs. (8) and (11). The manuscript provides no quantitative check of whether the recorded robot/table contacts actually satisfy this condition for the seven objects. Please add an analysis of contact locations (e.g., from manual annotation or by comparing PLL's support points to the ground-truth mesh), or explicitly document the convex-hull assumption as an untested limitation with its likely effect on the results.","section":"§III-B, §VII"},{"comment":"The second BundleSDF run is described as 'BundleSDF runs again' with the new physics losses, but the paper does not state whether object poses are re-estimated in this run. If poses are re-estimated, the trajectory used to train PLL no longer matches the trajectory associated with the final SDF, and the dynamics evaluation becomes ambiguous; if poses are frozen, that should be stated explicitly and the implementation should be made reproducible. Please clarify this point.","section":"§IV, pipeline description"}],"minor_comments":[{"comment":"Table I and Figures 8-12 would benefit from reporting the number of sessions per object and per-metric standard deviations or confidence intervals; the current per-object 'all' averages do not indicate variance across sessions.","section":"Table I and Figures 8-12"},{"comment":"The 'visibility check' that defines the visible set V is not described; please specify how visible vertices are determined from the RGBD video.","section":"§IV-A"},{"comment":"The abstract and introduction state Vysics requires 'no fundamental priors,' yet the method uses a convexity assumption, hand-set friction coefficients, and a table-plane model; consider softening this phrasing.","section":"Abstract, §I"},{"comment":"The qualitative comparison in Figure 6 and the statement that generative models 'do not generate a complete shape' would be more compelling if at least one representative quantitative result was reported for these methods.","section":"§VI-A, Figure 6"},{"comment":"In Eq. (6), the loss is written as a norm without explicitly stating whether it is L1 or L2; please clarify the norm used in the implementation.","section":"Eq. (6)"},{"comment":"The dataset description in §V-A does not list the number of recorded sessions per object; please add this information for reproducibility.","section":"§V-A"}],"recommendation":"major_revision","confidential_remarks":"The paper's core contribution is sound and worth publishing after major revision. The two issues I would insist on in review are (a) a cross-validation or held-out evaluation for the dynamics claim and (b) a friction sensitivity study, because they directly bear on whether the reported improvements come from the physics signal rather than from fitting the evaluation trajectory or from the chosen parameters. The convex-hull contact analysis is also important. If the authors can provide these experiments, I would expect the manuscript to be ready."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"X,\n\nThe headline: Vysics is a clean, credible integration of vision-based SDF tracking (BundleSDF) and contact-implicit dynamics learning (PLL), with a genuinely new mutual-supervision loop. The geometric reconstruction results are the strongest part; the dynamics prediction evaluation is weaker than the paper suggests, because it is in-sample.\n\nWhat's actually new: the support-point loss (Eq. 8), hyperplane-constrained loss (Eq. 11), and convexity loss (Eq. 13) that feed PLL's contact-inferred support points back into the SDF, plus the continuous regression alternative to Song and Boularias' discrete geometry search. That is a real step beyond the components. The paper also releases a new RGBD dataset of a Franka interacting with occluded objects, which is useful.\n\nWhat the paper does well: the geometric claim is backed by chamfer/IoU comparisons against ground-truth meshes, and the improvement over vision-only baselines is consistent across sessions. The loss derivations are mathematically valid under the stated convex-hull assumption. The authors are honest about limitations: elasticity, tracking failures, sparse contact signal. That matters.\n\nSoft spots, in proportion: the dynamics prediction section is the weak link. The model is optimized on a BundleSDF trajectory, then 'predicted' on the same trajectory by replaying robot commands in simulation. That is in-sample fitting, not held-out prediction. The paper should include at least one held-out session with different interaction patterns, or clearly label the numbers as replay fits. Second, the friction coefficients are hand-set (0.26 table, 0.15 robot) and are not ablated. The stress-test concern is fair: the inferred support points depend on those values, and without a sensitivity study we don't know how much of the claimed gain comes from the chosen friction rather than the contact information itself. This is a minor-to-moderate issue given the paper explains why friction identification is hard, but it needs addressing. Third, sessions where BundleSDF lost track were excluded without a failure rate table; the reader should know how often the pipeline fails.\n\nThe central idea holds up: feeding contact-inferred geometry into an SDF is sound and the geometry results support it. The paper deserves a serious referee, with the expectation of a revision that adds held-out dynamics evaluation, friction sensitivity, and a tracking-failure report. I'd bring it to reading group and would cite it once it's in final form.","headline":"A credible vision-physics fusion with genuinely new mutual-supervision losses; geometry results are solid, but the dynamics evaluation is in-sample and friction sensitivity is unexamined.","tokens_in":18428,"tokens_out":2638,"would_cite":true,"duration_ms":24463,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vysics claims that a robot can reconstruct a heavily occluded object's full shape by fusing visible RGBD geometry with contact geometry inferred from the object's observed motion.","keywords":["object reconstruction","occlusion","contact-rich manipulation","signed distance function","deep support function","dynamics model learning","RGBD video","rigid body"],"falsifier":"Take a known object with a deep concavity, such as a mug's handle side or a bowl's inside, record an RGBD video where the robot's sphere deliberately contacts only the concave region so the object moves, run Vysics, and compare the reconstructed mesh to the ground truth; if the output replaces the concavity with a convex hull or the inferred contact point lies outside the true surface, the central claim is falsified. A cheaper check is to rerun the same video with intentionally wrong friction coefficients and observe whether the recovered geometry shifts systematically.","tokens_in":17364,"feed_emoji":"🤖","tokens_out":4071,"duration_ms":42944,"temperature":0.7,"pith_summary":"Vysics claims that a robot holding a short RGBD video of a heavily occluded object can recover not only the visible geometry but also the hidden shape, by treating the object's observed motion as evidence about where it was touched. It fuses a vision-based tracker and shape estimator with a contact-implicit dynamics learner: the dynamics learner proposes a convex \"physible\" geometry that explains the trajectory through hypothesized contacts, and both streams supervise a signed distance function. The paper reports that on a new dataset of a robot arm pushing, pivoting, and toppling seven everyday objects behind occluders, Vysics cuts average chamfer distance roughly in half relative to vision-only reconstruction and produces simulated rollouts that stay accurate longer. If right, this gives robots a way to build usable object models on the spot without pretraining, CAD models, or force and tactile sensors.","feed_headline":"Contact physics completes occluded object shapes","feed_subtitle":"Vysics fuses RGBD video with contacts inferred from robot motion, roughly halving reconstruction error versus vision alone.","key_machinery":"The central object is the deep support function, an input-convex neural network that represents the convex hull of a shape by returning, for any unit direction, the distance to the furthest surface point in that direction; its gradient is the support point itself. PLL uses this DSF to learn geometry from observed trajectories through an implicit contact-dynamics loss, while BundleSDF maintains a signed distance function from the RGBD video. Vysics bridges the two by adding a visible-geometry loss to the DSF training, and by converting DSF support points into three SDF supervision terms: a ray-based support point loss, a hyperplane-constrained lower bound on signed distance, and a convexity bias that connects sparse contact points to the visible surface.","core_discovery":"The central discovery claim is that occluded geometry is observable through dynamics: a trajectory of a rigid body interacting with known surfaces encodes its shape wherever contact occurred, and that contact-inferred geometry can be integrated with visual depth into a single signed distance function. Vysics claims that the visible and \"physible\" geometries are complementary—vision constrains the sides seen, contact dynamics constrains the sides touched—and that jointly optimizing a hybrid SDF under both sources yields higher geometric accuracy and better dynamics predictions than vision alone. This is demonstrated by feeding BundleSDF-estimated poses and meshes into PLL, filtering the hypothesized contact points by force magnitude, and then running BundleSDF a second time with additional support-point, hyperplane, and convexity losses derived from the contact geometry.","pith_inferences":["The loss machinery would support an active perception loop: a robot unsure of hidden geometry could poke or tap the object at selected locations and use the resulting motion to reduce uncertainty, a direction the paper lists as future work.","The convex-hull limitation implies the method would systematically misreport concave features such as handle holes or recessed grips if contacts occur there; extending the DSF to a nonconvex implicit contact representation is a natural next step.","Because friction coefficients are hand-set, the inferred geometry could absorb errors in friction; holding friction fixed at wrong values on a known object and measuring how the reconstructed hull shifts would quantify that coupling.","The two-round BundleSDF-PLL design suggests a fixed-point scheme in which pose and shape mutually refine; if that iteration converges, it could remove the dependence on the initial BundleSDF pose quality."],"forward_implications":["Occluded faces that make contact get reconstructed even when never visible, so interaction plans can trust the model where vision alone would fail.","The same pipeline yields inertial parameters and a URDF, so simulated rollouts can be used to test manipulation strategies before execution.","Because no pretraining or fiducials are needed, the method applies to novel objects immediately from a few seconds of RGBD data plus robot joint states.","Dynamics prediction quality, measured by pose-error duration and contact-activation overlap, improves over vision-only geometry even in chaotic open-loop rollouts."],"supporting_citations":[{"why":"Supplies the vision-based pose tracker and SDF shape estimator that produce the trajectory and visible geometry used by both streams.","marker":"[66]"},{"why":"Provides the underlying model-free 6D pose tracker on which BundleSDF builds.","marker":"[65]"},{"why":"Introduces simultaneous learning of contact and continuous dynamics, the method PLL uses to infer geometry and inertia from a trajectory.","marker":"[9]"},{"why":"Establishes the violation-based implicit contact loss that lets PLL learn from contact-rich trajectories without differentiable simulation.","marker":"[53]"},{"why":"Defines the deep support function representation whose convex-hull constraint shapes what the physics stream can recover.","marker":"[25]"},{"why":"Generates the semi-automatic object masks used to prepare the RGBD dataset.","marker":"[17]"}],"fun_headline_variants":["Occluded objects revealed by contact physics","Physics fills in occluded object shape","Contact-rich physics completes hidden geometry","Fusing vision and physics for occluded shape","Inferring occluded shape from motion and contact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole inference assumes that every motion the robot sees is caused by rigid-body contact with the flat table or the spherical gripper, and that those contacts only ever touch the object's convex outer surface; if a robot pokes a hollow or recessed part, the model will misattribute the cause.","fun_headline_variants_meta":{"raw":{"variants":["Occluded objects revealed by contact physics","Physics fills in occluded object shape","Contact-rich physics completes hidden geometry","Fusing vision and physics for occluded shape","Inferring occluded shape from motion and contact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"total_tokens":1273,"prompt_tokens":943,"completion_tokens":330,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":264}},"tokens_in":559,"tokens_out":330,"duration_ms":3254,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:12:01.609225+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a known object with a deep concavity, such as a mug's handle side or a bowl's inside, record an RGBD video where the robot's sphere deliberately contacts only the concave region so the object moves, run Vysics, and compare the reconstructed mesh to the ground truth; if the output replaces the concavity with a convex hull or the inferred contact point lies outside the true surface, the central claim is falsified. A cheaper check is to rerun the same video with intentionally wrong friction coefficients and observe whether the recovered geometry shifts systematically.","supporting_citations":[{"cited_title":"Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects","cited_arxiv_id":null,"evidence_quote":"Supplies the vision-based pose tracker and SDF shape estimator that produce the trajectory and visible geometry used by both streams."},{"cited_title":"Bundletrack: 6d pose tracking for novel objects without instance or category- level 3d models, 2021","cited_arxiv_id":null,"evidence_quote":"Provides the underlying model-free 6D pose tracker on which BundleSDF builds."},{"cited_title":"Si- multaneous learning of contact and continuous dynamics","cited_arxiv_id":null,"evidence_quote":"Introduces simultaneous learning of contact and continuous dynamics, the method PLL uses to infer geometry and inertia from a trajectory."},{"cited_title":"ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations","cited_arxiv_id":null,"evidence_quote":"Establishes the violation-based implicit contact loss that lets PLL learn from contact-rich trajectories without differentiable simulation."},{"cited_title":"Addressing Stiffness-Induced Challenges in Modeling and Identification for Rigid-Body Systems With Friction and Impacts","cited_arxiv_id":null,"evidence_quote":"Defines the deep support function representation whose convex-hull constraint shapes what the physics stream can recover."},{"cited_title":"Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model","cited_arxiv_id":null,"evidence_quote":"Generates the semi-automatic object masks used to prepare the RGBD dataset."}],"review_version":1}