Pith. sign in

REVIEW 4 major objections 6 minor 89 references

Predictive Visuo-Tactile Interactive Perception Framework for Object Properties Inference

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A predictive visuo-tactile framework lets a robot infer the mass, center of mass, and friction of unfamiliar planar objects through active pushing and pulling, reporting better accuracy than prior feedforward and non-active baselines.

desk verdict A credible extension of the authors' own differentiable-filter line with real robot data, but the reported tables do not support the 'consistent' superiority claim and daily-object predictions show overconfidence; it deserves review but needs revision. read the letter →

arxiv 2411.09020 v1 pith:ZRQX46MZ submitted 2024-11-13 cs.RO

Anirvan Dutta , Etienne Burdet , Mohsen Kaboli This is my paper · ORCID
classification cs.RO
keywords VisualandTactileSensingActiveInteractivePerceptionRecursiveBayesianFilteringGraphNeuralNetworksPhysicalPropertyInferenceSuperquadricsActionSelectionRoboticManipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a robot can autonomously infer the physical properties of previously unseen planar objects—mass, center of mass, surface friction, and joint friction—by actively pushing or pulling them and combining vision with touch. The proposed predictive perception framework estimates object shape with superquadrics, learns a graph-neural-network model of the object-robot interaction, and uses that learned model inside a dual differentiable filter to track pose while estimating time-invariant parameters. An N-step information gain criterion selects the most informative next actions, improving data efficiency. In real-robot experiments across homogeneous, heterogeneous, and articulated objects, the framework reports lower normalized root-mean-square error than a feedforward baseline and than uniform or random action selection, and it demonstrates gains in pose tracking, goal-driven pushing, and detecting a change in the environment.

What carries the argument

The machinery is a dual differentiable filter with a causal graph neural network (GNN) as its learned process model: a recursive Bayesian filter (Unscented-Kalman-style) that maintains a joint Gaussian belief over per-link pose and twist and over time-invariant inertial and frictional parameters. The GNN represents the robot, the object's links, and the support table as nodes, with directed edges carrying contact-force information; a cause-effect message-passing scheme updates node and edge features so that the model predicts both the object's next pose and the tactile forces on the robot. The dual structure updates the parameter belief from observation likelihoods and then updates the pose belief conditionally on the parameters, while constrained Monte Carlo sampling keeps physically invalid sigma points out of the dynamics while preserving uncertainty. This same machinery provides the predictive distribution used by the N-step information gain action selector, and lets one learned model serve homogeneous, heterogeneous, and articulated objects.

What would settle it

Push the same object with identical mass, center of mass, and surface friction but with rotational inertia changed (for example, by moving internal weights outward), and increase push velocity; if the filter's predicted pose error or estimated parameter error grows with velocity or with the inertia change, the claim that $I_z$ does not affect the interaction and the quasi-static premise are falsified.

Watch

Extended reading notes

Core claim

The central discovery is that a single learnable framework can estimate the inertial and frictional parameters of diverse planar rigid objects without prior knowledge of their properties, using simple non-prehensile push and prehensile pull interactions. The state is split into time-varying pose and twist per link plus time-invariant parameters; a dual differentiable filter maintains a joint Gaussian belief over both. The process model is a graph neural network whose directed edges encode the causal transfer of forces from robot to object to support table, and its node and edge updates predict both visual (projected point cloud) and tactile (contact force) observations. Active shape perception via Bayesian superquadric fitting provides initial shape and pose, and an N-step information gain action selector chooses the next push or pull. On a real robotic setup with 120 configurable objects and eight daily objects, the proposed approach A-GNN consistently achieves the lowest normalized root-mean-square error for the inferred parameters compared with the A-FF baseline and uniform or random action selection, and the estimated parameters improve downstream pose tracking and goal-driven pushing.

Load-bearing premise

The framework assumes quasi-static planar interaction: pushes and pulls are slow enough that out-of-plane motion and rotational inertia do not affect the dynamics, and the friction between robot and object is approximately known from the tactile sensor.

Editorial extensions

If this is right

  • A robot using this framework can begin with no knowledge of an object's mass, friction, or center of mass and converge on estimates through a small number of push or pull interactions, with the filter reporting uncertainty alongside each estimate.
  • The same graph representation handles single-link, multi-link rigid, and articulated objects, so the approach removes the need to know the number of links or a kinematic model in advance.
  • Active N-step action selection reduces the number of interactions needed to train the dynamics model compared with uniform or random exploration, which matters when robot time is expensive.
  • Estimated parameters improve pose tracking during interaction and reduce goal-reaching error in model-predictive push control, so the inferred properties are directly useful for downstream manipulation.
  • The learned observation likelihood can flag changes in the environment, such as a tilted support surface, enabling a robot to detect when its interaction model is out of distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the dual-filter-plus-GNN architecture could estimate time-varying properties, such as a shifting center of mass in a liquid container or a deformable object, by enlarging the state space, since the filter already separates time-varying pose from time-invariant parameters.
  • The finding that predicted tactile noise is much lower than visual noise during prehensile pulling suggests a learned sensor-weighting rule: when the robot arm occludes the camera, the filter should rely more on touch, and the noise models could be used to trade off modalities explicitly.
  • The quasi-static assumption and fixed push velocity mean the framework is currently tuned for slow, planar exploration; a testable extension is to let the N-step information gain also select push velocity, which would probe whether relaxing the $I_z$ omission extends the method to heavier or taller objects.
  • The causal graph representation's success in encoding force transmission suggests it could transfer to bimanual or multi-robot interactions, where contact forces pass through multiple agents before returning to the sensor.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a predictive visuo-tactile interactive perception framework for inferring time-invariant physical properties (mass, center of mass, surface friction, joint friction) of planar rigid objects, including homogeneous, heterogeneous, and articulated objects. The pipeline comprises active superquadric-based shape perception with next-best-view selection, a dual differentiable filter whose process model is a graph neural network (GNN), learned heteroscedastic observation noise models, and an N-step information-gain criterion for selecting push or pull actions. The framework is evaluated in real-robot experiments on 120 configurable 3D-printed objects and 8 daily objects, and is compared against the authors' prior feedforward-filter baseline (A-FF), an analytical pushing baseline, and uniform/random action-selection ablations. The paper additionally reports applications to pose tracking, goal-driven pushing, and detection of a changed support-surface environment.

Significance. If the central claims held as stated, this would be a noteworthy advance: a single learned framework that infers multiple physical properties across three object classes using only push/pull interactions and visuo-tactile observations. The experimental effort is substantial: real-robot data, 120 configurable objects with independently measured ground truth, an analytical baseline, an ablation study, and three downstream applications. The paper also ships a concrete graph-based interaction model that is more general than the earlier feedforward model. However, the load-bearing comparative claim—that A-GNN 'consistently' achieves the least NRMSE—is contradicted by the paper's own tables, and the daily-object results show large discrepancies between predictions and ground truth relative to the reported uncertainties. These issues must be resolved before the contribution can be accepted.

major comments (4)
  1. [IV.B.3, Tables II–III] The claim that 'the proposed A-GNN consistently estimates the properties of objects with the least NRMSE' is not supported by the reported numbers. In Table II (heterogeneous, pushing), A-GNN has fl2 = 0.21±0.11 versus R-GNN 0.10±0.06, CoM l2 = 0.16±0.12 versus U-GNN 0.09±0.06, and Overall l2 = 0.16±0.08 versus R-GNN 0.11±0.03. In Table III (articulated, pushing), A-GNN mass l1 = 0.08±0.09 versus A-FF 0.04±0.03 and R-GNN 0.05±0.03, and CoM l1 = 0.07±0.05 versus A-FF 0.05±0.02. The Discussion itself concedes that 'for some parameters, A-FF performs better than A-GNN.' The authors should either substantially soften the 'consistently' claim, report statistical significance tests over the object configurations, or identify the precise conditions under which A-GNN is superior.
  2. [Table IV] The daily-object generalization results raise serious calibration concerns. For example, sugar friction is predicted as 0.12±0.1 against a ground truth of 0.5; olive-oil friction is predicted as 0.5±0.12 against 0.15; coffee mass is predicted as 1.0±0.06 against 0.67; hammer mass is predicted as 0.7±0.25 against 1.2. If the reported '±' values are posterior standard deviations, the filter is overconfident by several sigma in exactly the real-world setting highlighted in the abstract. If they are not posterior standard deviations, their meaning should be clarified. The manuscript should report a calibration measure for the predicted uncertainties and discuss these failures rather than presenting the daily-object results without error analysis.
  3. [III.E, Eqs. (17)–(18)] The definition of the N-step information gain is internally inconsistent. Equation (17) defines IG_N(π_i) ≈ −E[ln bel_i(τ_N) − ln bel_i(τ_0)], but Equation (18) equates this to D_KL[N_i(τ_N) || N_i(τ_0)] = E_{τ_N}[ln N(τ_N) − ln N(τ_0)]. As written, maximizing the quantity in Eq. (17) would minimize the KL divergence in Eq. (18). Since the N-step information-gain action selector is one of the paper's stated contributions, this sign inconsistency must be corrected, and the authors should confirm that the implemented criterion is the one reported in the experiments.
  4. [IV.B.3, Figs. 12–14] The superiority claims for active action selection over uniform and random selection are not backed by statistical analysis. Many of the reported mean NRMSE differences are smaller than the reported standard deviations (e.g., Table II, Overall pushing: A-GNN 0.15±0.06 vs. R-GNN 0.15±0.05; Table III, Overall pushing: A-GNN 0.15±0.03 vs. R-GNN 0.15±0.02). Without per-configuration error distributions, paired tests, or confidence intervals, the reader cannot determine whether the active strategy improves inference. The authors should either provide such an analysis or restrict their claims to qualitative observations about training efficiency.
minor comments (6)
  1. [III.A] There is a typo in the sentence beginning 'Thee homogeneous, heterogeneous and articulated...' — 'Thee' should be 'The'.
  2. [III.B.3] The text states 'We present both quantitative and quantitative results on shape perception in Section ?? and in the Appendix.' 'quantitative' appears twice and the section cross-reference is unresolved.
  3. [IV.B.4] The sentence 'The error in the pulling interaction is lower than that of pulling due to the constrained movement...' compares pulling with itself; the second instance should presumably be 'pushing'.
  4. [III.E, Eq. (17)] Equation (17) has a missing closing parenthesis; the expression 'ln(beli(ψτN, ϕτN) − ln(beli(ψτ0, ϕτ0)' is incomplete.
  5. [IV.B.2 and Fig. 10] The notation for the active feedforward baseline is inconsistent: the text uses 'Active − FF' in one place and 'A − FF' elsewhere. Please standardize.
  6. [III.A and IV.A] The quasi-static planar assumption is a legitimate scope limit, but the paper should state explicitly in the conclusions that the results do not claim to extend to fast, dynamic, or out-of-plane interactions, especially since the daily-object experiments are presented as a real-world validation.

Circularity Check

0 steps flagged · score 2.0 of 10

No equation-level circularity: parameter inference is evaluated against independently measured ground truth on held-out objects, and the only circularity-adjacent issue is minor reliance on the authors' own prior formulations, which are re-derived and empirically compared rather than assumed.

full rationale

Walking the derivation chain, I find no step in which a predicted quantity is defined in terms of the target quantity or in which a fitted parameter is relabeled as a prediction. The dual differentiable filter (Section III.F) predicts poses and parameters from a learned graph-neural-network process model and a projective visual observation model; the update step (Eqs. 31-36) combines these predictions with actual observations using a likelihood, and the inferred parameters are compared with ground truth independently characterized by motion capture, tilt-table friction measurement, and pendulum-based joint-friction measurement (Section IV.A). The test objects are a held-out 10% of the 120 configurable objects, and the daily objects in Table IV were not used in training; the fact that several daily-object predictions are poor (e.g., sugar friction predicted 0.12 ± 0.1 versus GT 0.5) is evidence that the method is not circularly forced to match its inputs. The N-step information-gain action selection (Eqs. 17-18) uses the learned process model to choose actions, but the downstream parameter error is measured against ground truth, so the active strategy is evaluated empirically rather than defined into success. The main caveat is self-citation: the dual filter, constrained Monte Carlo sampling, and N-step formulation are attributed to the authors' prior work [22], and the primary learned baseline A-FF is that same prior method. However, the paper re-derives the equations in its own notation, presents ablations (U-GNN, R-GNN, A-Analytical), and does not invoke any uniqueness theorem or unverified self-citation to rule out alternatives. The discussion itself admits that A-FF is better for some parameters and that active selection is less effective for some object types, which weakens the 'consistent' claim but is a correctness/consistency issue, not circularity. Overall, the central inference chain is self-contained and externally grounded; score 2 reflects the minor, non-load-bearing self-citation pattern rather than any reduction of prediction to input.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The framework introduces no fitted physical constants; the numbers listed are design hyperparameters of the filter. The five axioms are modeling assumptions about dynamics and shape representation that bound the regime of validity. No invented physical entities such as new forces, particles, or mediators are introduced.

free parameters (4)
  • Process noise covariance Qt = user-defined per object type and interaction
    Used in Eq. 24 to add noise during the prediction step; magnitude directly controls belief width and estimation behavior.
  • Kernel shrinkage a = 0.01
    Used in Eqs. 32-33 (Gaussian Smooth Kernel) to update the parameter belief; choice affects convergence and variance.
  • Monte Carlo sigma points C = 100
    Number of sampled sigma points in Eq. 21 for constrained sampling; larger C improves approximation but increases compute.
  • N-step look-ahead horizon = 2.5 seconds
    Action selection look-ahead parameter used in the information gain criterion; fixed but user-chosen.
assumptions (5)
  • domain assumption The recursive Bayesian filter's Markov assumption (Eq. 16) holds for object-robot interaction.
    Gives the recursive belief update bel(st) = eta p(ot|st,at) integral p(st|st-1,at-1) bel(st-1) dst-1, which is central to the dual filter.
  • domain assumption Planar pushing and pulling are quasi-static; a 2D state with mass, COM, and friction suffices, and rotational inertia Iz can be ignored.
    Section III.A states Iz does not affect quasistatic interactions; Section III.E fixes velocity to respect the quasi-static assumption.
  • domain assumption The friction between the robot fingertip and the object (fr) is approximately known from the tactile sensor property.
    Section III.A treats fr as approximately known rather than estimating it, so inference targets only table friction and joint friction.
  • domain assumption Object shape can be represented by superquadrics with one tapering deformation per link.
    Active shape perception in Section III.B fits one superquadric per link; the Discussion explicitly excludes objects with cavities like cups and bowls.
  • domain assumption The GNN dynamics model trained on 80% of the 120 configurable objects transfers to held-out configurations and to the chosen daily objects.
    The train/test split in Section IV.B.2 assumes transfer; daily-object results in Table IV show this transfer is imperfect, especially for friction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predictive Visuo-Tactile Interactive Perception Framework for Object Properties Inference." pith.science (2026). https://pith.science/paper/ZRQX46MZ

@misc{pith2026241109020,
  author       = {Pith},
  title        = {Pith review of: Predictive Visuo-Tactile Interactive Perception Framework for Object Properties Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZRQX46MZ}},
  note         = {Machine review of arXiv:2411.09020}
}
abstract

Interactive exploration of the unknown physical properties of objects such as stiffness, mass, center of mass, friction coefficient, and shape is crucial for autonomous robotic systems operating continuously in unstructured environments. Precise identification of these properties is essential to manipulate objects in a stable and controlled way, and is also required to anticipate the outcomes of (prehensile or non-prehensile) manipulation actions such as pushing, pulling, lifting, etc. Our study focuses on autonomously inferring the physical properties of a diverse set of various homogeneous, heterogeneous, and articulated objects utilizing a robotic system equipped with vision and tactile sensors. We propose a novel predictive perception framework for identifying object properties of the diverse objects by leveraging versatile exploratory actions: non-prehensile pushing and prehensile pulling. As part of the framework, we propose a novel active shape perception to seamlessly initiate exploration. Our innovative dual differentiable filtering with Graph Neural Networks learns the object-robot interaction and performs consistent inference of indirectly observable time-invariant object properties. In addition, we formulate a $N$-step information gain approach to actively select the most informative actions for efficient learning and inference. Extensive real-robot experiments with planar objects show that our predictive perception framework results in better performance than the state-of-the-art baseline and demonstrate our framework in three major applications for i) object tracking, ii) goal-driven task, and iii) change in environment detection.

Figures

Figures reproduced from arXiv: 2411.09020 by the authors.

Figure 1
Figure 1. Overview of the proposed framework for visuo-tactile based interactive [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our proposed framework is presented in detail for interactively inferring the diverse objects using visuo-tactile sensing. The framework starts in the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Illustration of a few basic superquadric shapes with the proposed non [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Illustration of the shape perception approach. The entropy of each point calculated from Eq. 10 is mapped to 0-255 red channel of the sampled [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: a) Illustration of the proposed graph representation of an example articulated object with two links b) Novel graph propagation for updating the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the constrained Monte Carlo sampling compared with [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Experimental Setup utilized to validate the framework. a) Presents the configurable 3D printed object designed to be used as diverse objects of types: [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 9
Figure 9. Figure 9: Comparison of Active, Uniform and Random shape perception result [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Results on number of interaction required for training the models for [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 12
Figure 12. Figure 12: Parameter estimation error across multiple interactions for homo [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Parameter estimation error across multiple interactions for heteroge [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 15
Figure 15. Figure 15: Pose tracking result of the proposed approach with and without [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: MSE of goal-driven control for 3 trials on 8 objects performed each [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]
Figure 17
Figure 17. Figure 17: Setup for inducing a change in the learned environment by tilting the [PITH_FULL_IMAGE:figures/full_fig_p015_17.png]
Figure 18
Figure 18. Figure 18: Likelihood of visual and tactile observation comparing flat surface [PITH_FULL_IMAGE:figures/full_fig_p016_18.png]
Figure 19
Figure 19. Figure 19: Analytical model of planar pushing α = arctan(µr) (38) fl = R(−α)n (39) fr = R(α)n (40) ml = cpxfly − cpyflx (41) mr = cxfry − cyfrx (42) The rotation matrix R(α) represents a rotation about the z￾axis with an angle α. The contact point cp and the surface normal n are…
Figure 22
Figure 22. Figure 22: Examination of the filtering process of a representative trajectory for [PITH_FULL_IMAGE:figures/full_fig_p020_22.png]
Figure 23
Figure 23. Figure 23: Examination of the filtering process of a representative trajectory [PITH_FULL_IMAGE:figures/full_fig_p020_23.png]
Figure 24
Figure 24. Figure 24: Examination of the filtering process of a representative trajectory for [PITH_FULL_IMAGE:figures/full_fig_p020_24.png]
Figure 25
Figure 25. Figure 25: Examination of the filtering process of a representative trajectory for [PITH_FULL_IMAGE:figures/full_fig_p021_25.png]
Figure 26
Figure 26. Figure 26: Examination of the filtering process of a representative trajectory for [PITH_FULL_IMAGE:figures/full_fig_p021_26.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 69 canonical work pages

  1. [1]

    Densephysnet: Learning dense physical object representations via multi-step dynamic interactions,

    Z. Xu, J. Wu, A. Zeng, J. B. Tenenbaum, and S. Song, “Densephysnet: Learning dense physical object representations via multi-step dynamic interactions,” arXiv preprint arXiv:1906.03853 , 2019

    arXiv 1906

  2. [2]

    A review of tactile information: Perception and action through touch,

    Q. Li et al. , “A review of tactile information: Perception and action through touch,” IEEE Trans. on Rob. , vol. 36, no. 6, pp. 1619–1634, 2020

    2020

  3. [3]

    Active haptic perception in robots: a review,

    L. Seminara, P. Gastaldo, S. J. Watt, K. F. Valyear, F. Zuher, and F. Mastrogiovanni, “Active haptic perception in robots: a review,” Frontiers in neurorobotics, vol. 13, p. 53, 2019

    2019

  4. [4]

    Visuo-haptic object perception for robots: an overview,

    N. Navarro-Guerrero et al., “Visuo-haptic object perception for robots: an overview,” AuRo, vol. 47, no. 4, pp. 377–403, 2023

    2023

  5. [5]

    Active visuo-tactile interactive robotic perception for accurate object pose estimation in dense clutter,

    P. K. Murali, A. Dutta, M. Gentner, E. Burdet, R. Dahiya, and M. Kaboli, “Active visuo-tactile interactive robotic perception for accurate object pose estimation in dense clutter,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4686–4693, 2022

    2022

  6. [6]

    Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks,

    M. A. Lee, Y . Zhu, K. Srinivasan, P. Shah, S. Savarese, L. Fei- Fei, A. Garg, and J. Bohg, “Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks,” in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 8943–8950

    2019

  7. [7]

    Interactive perception: Leveraging action in perception and perception in action,

    J. Bohg, K. Hausman, B. Sankaran, O. Brock, D. Kragic, S. Schaal, and G. S. Sukhatme, “Interactive perception: Leveraging action in perception and perception in action,” IEEE Transactions on Robotics, vol. 33, no. 6, pp. 1273–1291, 2017

    2017

  8. [8]

    Revisiting active perception,

    R. Bajcsy, Y . Aloimonos, and J. Tsotsos, “Revisiting active perception,” Autonomous Robots, vol. 42, 02 2018

    2018

Show all 89 references
  1. [9]

    Neuro-inspired electronic skin for robots,

    F. Liu, S. Deswal, A. Christou, Y . Sandamirskaya, M. Kaboli, and R. Dahiya, “Neuro-inspired electronic skin for robots,” Science Robotics, vol. 7, no. 67, p. eabl7344, 2022

    2022

  2. [10]

    Shared visuo-tactile inter- active perception for robust object pose estimation,

    M. K. Prajval Kumar Murali, Bernd Porr, “Shared visuo-tactile inter- active perception for robust object pose estimation,” The International Journal of Robotics Research (IJRR) , pp. 1–28, 2024

    2024

  3. [11]

    Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties,

    M. Kaboli, R. Walker, and G. Cheng, “Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 2242–2247

    2016

  4. [12]

    Learning to poke by poking: Experiential learning of intuitive physics,

    P. Agrawal, A. V . Nair, P. Abbeel, J. Malik, and S. Levine, “Learning to poke by poking: Experiential learning of intuitive physics,” Advances in neural information processing systems , vol. 29, 2016

    2016

  5. [13]

    Tactile-based manipulation of deformable objects with dynamic center of mass,

    M. Kaboli et al., “Tactile-based manipulation of deformable objects with dynamic center of mass,” in ICHR. IEEE, 2016

    2016

  6. [15]

    Active tactile transfer learning for object discrimination in an unstructured environment using multimodal robotic skin,

    M. Kaboli, D. Feng, and G. Cheng, “Active tactile transfer learning for object discrimination in an unstructured environment using multimodal robotic skin,” International Journal of Humanoid Robotics , vol. 15, no. 01, p. 1850001, 2018

    2018

  7. [16]

    Data-driven grasp synthesis—a survey,

    J. Bohg, A. Morales, T. Asfour, and D. Kragic, “Data-driven grasp synthesis—a survey,” IEEE Transactions on Robotics , vol. 30, no. 2, pp. 289–309, 2014

    2014

  8. [17]

    Neuromorphic computing hardware and neural architectures for robotics,

    Y . Sandamirskaya, M. Kaboli, J. Conradt, and T. Celikel, “Neuromorphic computing hardware and neural architectures for robotics,” Science Robotics, vol. 7, no. 67, 2022

    2022

  9. [18]

    A tactile- based framework for active object learning and discrimination using multimodal robotic skin,

    M. Kaboli, D. Feng, K. Yao, P. Lanillos, and G. Cheng, “A tactile- based framework for active object learning and discrimination using multimodal robotic skin,” IEEE Robotics and Automation Letters, vol. 2, no. 4, pp. 2143–2150, 2017

    2017

  10. [19]

    Conditions for versatile learning, helmholtz’s unconscious inference, and the task of perception,

    H. Barlow, “Conditions for versatile learning, helmholtz’s unconscious inference, and the task of perception,” Vision research, vol. 30, no. 11, pp. 1561–1571, 1990

    1990

  11. [20]

    The free-energy principle: a unified brain theory?

    K. Friston, “The free-energy principle: a unified brain theory?” Nature reviews neuroscience, vol. 11, no. 2, pp. 127–138, 2010

    2010

  12. [21]

    Differentiable particle filters: End-to-end learning with algorithmic priors,

    R. Jonschkowski, D. Rastogi, and O. Brock, “Differentiable particle filters: End-to-end learning with algorithmic priors,” arXiv preprint arXiv:1805.11122, 2018

    2018 arXiv

  13. [22]

    Push to know!-visuo-tactile based active object parameter inference with dual differentiable filtering,

    A. Dutta, E. Burdet, and M. Kaboli, “Push to know!-visuo-tactile based active object parameter inference with dual differentiable filtering,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 3137–3144

    2023

  14. [23]

    Tactile-based active object discrimination and target object search in an unknown workspace,

    M. Kaboli, K. Yao, D. Feng, and G. Cheng, “Tactile-based active object discrimination and target object search in an unknown workspace,” Autonomous Robots, vol. 43, pp. 123–152, 2019

    2019

  15. [24]

    Active visuo-tactile point cloud registration for accurate pose estimation of objects in an unknown workspace,

    P. K. Murali, M. Gentner, and M. Kaboli, “Active visuo-tactile point cloud registration for accurate pose estimation of objects in an unknown workspace,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 2838–2844

    2021

  16. [25]

    Robust and accurate superquadric recovery: A probabilistic approach,

    W. Liu, Y . Wu, S. Ruan, and G. S. Chirikjian, “Robust and accurate superquadric recovery: A probabilistic approach,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 2676–2685

    2022

  17. [26]

    Primitive-based shape abstraction via nonparametric bayesian inference,

    Y . Wu et al. , “Primitive-based shape abstraction via nonparametric bayesian inference,” in ECCV. Springer, 2022, pp. 479–495

    2022

  18. [27]

    Grasp it like a pro 2.0: A data-driven approach exploiting basic shape decomposition and human data for grasping unknown objects,

    A. Palleschi et al. , “Grasp it like a pro 2.0: A data-driven approach exploiting basic shape decomposition and human data for grasping unknown objects,” IEEE T-RO, 2023

    2023

  19. [28]

    A review of three dimensional reconstruction tech- niques,

    J. Phang et al. , “A review of three dimensional reconstruction tech- niques,” Multimedia Tools & App. , vol. 80, no. 12, pp. 17 879–17 891, 2021

    2021

  20. [29]

    Completion and reconstruction with primitive shapes,

    R. Schnabel et al. , “Completion and reconstruction with primitive shapes,” in Computer Graphics Forum , vol. 28, no. 2. Wiley Online Library, 2009, pp. 503–512

    2009

  21. [30]

    Surface reconstruction from fitted shape primitives

    P. Jenke et al., “Surface reconstruction from fitted shape primitives.” in VMV. Citeseer, 2008, pp. 31–40

    2008

  22. [31]

    Mind the gap-robotic grasping under incomplete obser- vation,

    J. Bohg et al., “Mind the gap-robotic grasping under incomplete obser- vation,” in ICRA. IEEE, 2011, pp. 686–693

    2011

  23. [32]

    Shape from symmetry,

    S. Thrun and B. Wegbreit, “Shape from symmetry,” in ICCV, vol. 2. IEEE, 2005, pp. 1824–1831

    2005

  24. [33]

    Comprehensive review of deep learning-based 3d point cloud completion processing and analysis,

    B. Fei et al. , “Comprehensive review of deep learning-based 3d point cloud completion processing and analysis,” IEEE T-ITS, vol. 23, no. 12, pp. 22 862–22 883, 2022

    2022

  25. [34]

    Active vision,

    J. Aloimonos et al., “Active vision,” IJCV, vol. 1, pp. 333–356, 1988

    1988

  26. [35]

    A comparison of volumetric information gain metrics for active 3d object reconstruction,

    J. Delmerico et al. , “A comparison of volumetric information gain metrics for active 3d object reconstruction,” Autonomous Robots, 2018

    2018

  27. [36]

    View planning in robot active vision: A survey of systems, algorithms, and applications,

    R. Zeng et al. , “View planning in robot active vision: A survey of systems, algorithms, and applications,” Comp. Visual Media, vol. 6, pp. 225–245, 2020

    2020

  28. [37]

    Pc-nbv: A point cloud based deep network for efficient next best view planning,

    R. Zeng et al., “Pc-nbv: A point cloud based deep network for efficient next best view planning,” in IROS. IEEE, 2020, pp. 7050–7057

    2020

  29. [38]

    Supervised learning of the next-best-view for 3d object reconstruction,

    M. Mendoza et al. , “Supervised learning of the next-best-view for 3d object reconstruction,” Pattern Recog. Letters , vol. 133, pp. 224–231, 2020

    2020

  30. [39]

    Reducing uncertainty in undersampled mri reconstruc- tion with active acquisition,

    Z. Zhang et al., “Reducing uncertainty in undersampled mri reconstruc- tion with active acquisition,” in IEEE CVPR, 2019, pp. 2049–2058

    2019

  31. [40]

    Neu-nbv: Next best view planning using uncertainty estimation in image-based neural rendering,

    L. Jin et al. , “Neu-nbv: Next best view planning using uncertainty estimation in image-based neural rendering,” in IROS. IEEE, 2023, pp. 11 305–11 312

    2023

  32. [41]

    Next best view planning via reinforcement learning for scanning of arbitrary 3d shapes,

    S. Potapova et al., “Next best view planning via reinforcement learning for scanning of arbitrary 3d shapes,” Journal of Comm. Tech. & Elec. , vol. 65, pp. 1484–1490, 2020

    2020

  33. [42]

    Active mr k-space sampling with reinforcement learning,

    L. Pineda et al. , “Active mr k-space sampling with reinforcement learning,” in Medical Image Comp. & CAI 2020 . Springer, 2020, pp. 23–33

    2020

  34. [43]

    Ljung, System Identification

    L. Ljung, System Identification . Boston, MA: Birkh ¨auser Boston, 1998, pp. 163–173. [Online]. Available: https://doi.org/10.1007/ 978-1-4612-1768-8 11

    1998

  35. [44]

    Identification of the ten inertia parameters of a rigid body,

    M. Niebergall and H. Hahn, “Identification of the ten inertia parameters of a rigid body,” Nonlinear Dynamics, vol. 13, pp. 361–372, 1997

    1997

  36. [45]

    Estimation of inertial parameters of manipulator loads and links,

    C. G. Atkeson, C. H. An, and J. M. Hollerbach, “Estimation of inertial parameters of manipulator loads and links,” The International Journal of Robotics Research , vol. 5, no. 3, pp. 101–119, 1986

    1986

  37. [46]

    Parameter estimation and object gripping based on fingertip force/torque sensors,

    C. Wang, X. Zang, X. Zhang, Y . Liu, and J. Zhao, “Parameter estimation and object gripping based on fingertip force/torque sensors,” Measure- ment, vol. 179, p. 109479, 2021

    2021

  38. [47]

    Fast object inertial parameter identification for collaborative robots,

    P. Nadeau, M. Giamou, and J. Kelly, “Fast object inertial parameter identification for collaborative robots,”arXiv preprint arXiv:2203.00830, 2022

    2022 arXiv

  39. [48]

    Estimation of object inertia parameters on robot pushing operation,

    Y . Yu, T. Arima, and S. Tsujio, “Estimation of object inertia parameters on robot pushing operation,” in Proceedings of the 2005 IEEE Inter- national Conference on Robotics and Automation . IEEE, 2005, pp. 1657–1662

    2005

  40. [49]

    Estimation of mass and center of mass of graspless and shape-unknown object,

    Y . Yu, K. Fukuda, and S. Tsujio, “Estimation of mass and center of mass of graspless and shape-unknown object,” in Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No. 99CH36288C), vol. 4. IEEE, 1999, pp. 2893–2898

    1999

  41. [50]

    Center of mass and friction coefficient exploration of unknown object for a robotic grasping manip- ulation,

    Z. Zhao, X. Li, C. Lu, and Y . Wang, “Center of mass and friction coefficient exploration of unknown object for a robotic grasping manip- ulation,” in 2018 IEEE International Conference on Mechatronics and Automation (ICMA), 2018, pp. 2352–2357

    2018

  42. [51]

    Active mass estimation with haptic vision,

    S. Tanaka, T. Tanigawa, Y . Abe, M. Uejo, and H. T. Tanaka, “Active mass estimation with haptic vision,” in Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. , vol. 3. IEEE, 2004, pp. 256–261

    2004

  43. [52]

    Tactile-based object center of mass exploration and discrimination,

    K. Yao, M. Kaboli, and G. Cheng, “Tactile-based object center of mass exploration and discrimination,” in 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids) , 2017, pp. 876–881

    2017

  44. [53]

    In-hand object-dynamics inference using tactile fingertips,

    B. Sundaralingam and T. Hermans, “In-hand object-dynamics inference using tactile fingertips,” IEEE Transactions on Robotics , vol. 37, no. 4, pp. 1115–1126, 2021

    2021

  45. [54]

    Galileo: Perceiving physical object properties by integrating a physics engine with deep learning,

    J. Wu, I. Yildirim, J. J. Lim, B. Freeman, and J. Tenenbaum, “Galileo: Perceiving physical object properties by integrating a physics engine with deep learning,” Advances in neural information processing systems, vol. 28, 2015

    2015

  46. [55]

    A probabilistic model for planar sliding of objects with unknown material properties: Identification and robust planning,

    C. Song and A. Boularias, “A probabilistic model for planar sliding of objects with unknown material properties: Identification and robust planning,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2020, pp. 5311–5318

    2020

  47. [56]

    Learning to slide unknown objects with differentiable physics simulations,

    C. Song and A. Boularias, “Learning to slide unknown objects with differentiable physics simulations,” in Robotics science and systems , 2020

    2020

  48. [57]

    Estimating an object’s inertial param- eters by robotic pushing: a data-driven approach,

    N. Mavrakis, R. Stolkin, et al., “Estimating an object’s inertial param- eters by robotic pushing: a data-driven approach,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2020, pp. 9537–9544

    2020

  49. [58]

    Learning to simulate complex physics with graph networks,

    A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. Battaglia, “Learning to simulate complex physics with graph networks,” in International conference on machine learning . PMLR, 2020, pp. 8459–8468

    2020

  50. [59]

    Graph networks as learnable physics engines for inference and control,

    A. Sanchez-Gonzalez, N. Heess, J. T. Springenberg, J. Merel, M. Ried- miller, R. Hadsell, and P. Battaglia, “Graph networks as learnable physics engines for inference and control,” in International Conference on Machine Learning . PMLR, 2018, pp. 4470–4479

    2018

  51. [60]

    Predicting pushing action effects on spatial object relations by learning internal prediction models,

    F. Paus, T. Huang, and T. Asfour, “Predicting pushing action effects on spatial object relations by learning internal prediction models,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) , 2020, pp. 10 584–10 590

    2020

  52. [61]

    Object and relation centric representations for push effect prediction,

    A. E. Tekden, A. Erdem, E. Erdem, T. Asfour, and E. Ugur, “Object and relation centric representations for push effect prediction,” Robotics and Autonomous Systems , p. 104632, 2024

    2024

  53. [62]

    Dynamics learning with object- centric interaction networks for robot manipulation,

    J. Wang, C. Hu, Y . Wang, and Y . Zhu, “Dynamics learning with object- centric interaction networks for robot manipulation,” IEEE Access , vol. 9, pp. 68 277–68 288, 2021

    2021

  54. [63]

    Tactile sim-to-real policy transfer via real-to-sim image translation,

    A. Church, J. Lloyd, N. F. Lepora, et al. , “Tactile sim-to-real policy transfer via real-to-sim image translation,” in Conference on Robot Learning. PMLR, 2022, pp. 1645–1654

    2022

  55. [64]

    Reducing tactile sim2real domain gaps via deep texture generation networks,

    T. Jianu, D. F. Gomes, and S. Luo, “Reducing tactile sim2real domain gaps via deep texture generation networks,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 8305–8311

    2022

  56. [65]

    Active learning literature survey,

    B. Settles, “Active learning literature survey,” 2009

    2009

  57. [66]

    Active learning in robotics: A review of control principles,

    A. T. Taylor, T. A. Berrueta, and T. D. Murphey, “Active learning in robotics: A review of control principles,” Mechatronics, vol. 77, p. 102576, 2021

    2021

  58. [67]

    Tandem3d: Active tactile exploration for 3d object recognition,

    J. Xu, H. Lin, S. Song, and M. Ciocarlie, “Tandem3d: Active tactile exploration for 3d object recognition,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 10 401–10 407

    2023

  59. [68]

    Active object perception using bayesian classifiers and haptic exploration,

    T. Sun, H. Liu, and Z. Miao, “Active object perception using bayesian classifiers and haptic exploration,” Autonomous Robots, vol. 47, no. 1, pp. 19–36, 2023

    2023

  60. [69]

    T. Parr, G. Pezzulo, and K. J. Friston, Active inference: the free energy principle in mind, brain, and behavior . MIT Press, 2022

    2022

  61. [70]

    Object recognition using mechanical impact, viscoelasticity, and surface friction during interaction,

    P. Uttayopas, X. Cheng, J. Eden, and E. Burdet, “Object recognition using mechanical impact, viscoelasticity, and surface friction during interaction,” IEEE Transactions on Haptics , 2023

    2023

  62. [71]

    Probabilistic surface friction estimation based on visual and haptic measurements,

    T. N. Le, F. Verdoja, F. J. Abu-Dakka, and V . Kyrki, “Probabilistic surface friction estimation based on visual and haptic measurements,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 2838–2845, 2021

    2021

  63. [72]

    Coupled recursive estimation for online interactive perception of articulated objects,

    R. Mart ´ın-Mart´ın and O. Brock, “Coupled recursive estimation for online interactive perception of articulated objects,” The International Journal of Robotics Research , vol. 41, no. 8, pp. 741–777, 2022

    2022

  64. [73]

    Superquadrics and angle-preserving transformations,

    Barr, “Superquadrics and angle-preserving transformations,” IEEE Com- puter Graphics and Applications , vol. 1, no. 1, pp. 11–23, 1981

    1981

  65. [74]

    Manual of photogrammetry,

    J. C. McGlone, “Manual of photogrammetry,” (No Title), 2013

    2013

  66. [75]

    T3vip: Transformation-based 3D video prediction,

    I. Nematollahi, E. Rosete-Beas, S. M. B. Azad, R. Rajan, F. Hutter, and W. Burgard, “T3vip: Transformation-based 3D video prediction,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 4174–4181

    2022

  67. [76]

    Least-squares fitting of two 3-d point sets,

    K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-d point sets,” IEEE Transactions on pattern analysis and machine intelligence, no. 5, pp. 698–700, 1987

    1987

  68. [77]

    Probabilistic robotics,

    S. Thrun, “Probabilistic robotics,” Communications of the ACM, vol. 45, no. 3, pp. 52–57, 2002

    2002

  69. [78]

    Derivations for linear algebra and optimization,

    J. Duchi, “Derivations for linear algebra and optimization,” Berkeley, California, vol. 3, no. 1, pp. 2325–5870, 2007

    2007

  70. [79]

    How to train your differentiable filter,

    A. Kloss, G. Martius, and J. Bohg, “How to train your differentiable filter,” Autonomous Robots, vol. 45, no. 4, pp. 561–578, 2021

    2021

  71. [80]

    A generalized unscented transformation for probability distributions,

    D. Ebeigbe, T. Berry, M. M. Norton, A. J. Whalen, D. Simon, T. Sauer, and S. J. Schiff, “A generalized unscented transformation for probability distributions,” ArXiv, 2021

    2021

  72. [81]

    Robust gaussian filtering using a pseudo measurement,

    M. W ¨uthrich, C. G. Cifuentes, S. Trimpe, F. Meier, J. Bohg, J. Issac, and S. Schaal, “Robust gaussian filtering using a pseudo measurement,” in 2016 American Control Conference (ACC) . IEEE, 2016, pp. 3606– 3613

    2016

  73. [82]

    Combined parameter and state estimation in simulation-based filtering,

    J. Liu and M. West, “Combined parameter and state estimation in simulation-based filtering,” in Sequential Monte Carlo methods in prac- tice. Springer, 2001, pp. 197–223

    2001

  74. [83]

    The multivariate gaussian distribution,

    C. B. Do, “The multivariate gaussian distribution,” Section Notes, Lecture on Machine Learning, CS , vol. 229, 2008

    2008

  75. [84]

    Manipulation and active sensing by pushing using tactile feedback

    K. M. Lynch, H. Maekawa, and K. Tanie, “Manipulation and active sensing by pushing using tactile feedback.” in IROS, vol. 1, 1992, pp. 416–421

    1992

  76. [85]

    Contactile,

    Contactile, “Contactile,” https://contactile.com/, 2023, [Online; accessed 15092023]

    2023

  77. [86]

    Zed, “Zed,” https://www.stereolabs.com/products/zed-2, 2023, [Online; accessed 15092023]

    2023

  78. [87]

    Optitrack - motion capture systems,

    “Optitrack - motion capture systems,” https://optitrack.com/, (Accessed on 03/02/2023)

    2023

  79. [88]

    A survey of deep active learning,

    P. Ren, Y . Xiao, X. Chang, P.-Y . Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang, “A survey of deep active learning,” ACM computing surveys (CSUR), vol. 54, no. 9, pp. 1–40, 2021

    2021

  80. [89]

    Sample-efficient cross-entropy method for real-time planning,

    C. Pinneri, S. Sawant, S. Blaes, J. Achterhold, J. Stueckler, M. Rolinek, and G. Martius, “Sample-efficient cross-entropy method for real-time planning,” in Conference on Robot Learning . PMLR, 2021, pp. 1049– 1065

    2021

  81. [90]

    Mechanics and planning of manipulator pushing oper- ations,

    M. T. Mason, “Mechanics and planning of manipulator pushing oper- ations,” The International Journal of Robotics Research , vol. 5, no. 3, pp. 53–71, 1986. VI. A PPENDIX A. Analytical Model of pushing Objects with homogeneous properties have an analytical model of quasistatic ...

    1986

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.