Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bite acquisition succeeds more often when a robot picks a skill from calibrated tool capabilities and measured food properties rather than the food's category.

desk verdict Solid empirical result, but the property-refinement mechanism is confounded by the pose channel and unvalidated against ground truth; referee it, but demand a cleaner test. read the letter →

arxiv 2506.02353 v2 pith:73MPX3WW submitted 2025-06-03 cs.RO

Zhanxin Wu , Bo Ai , Tom Silver , Tapomayukh Bhattacharjee This is my paper
classification cs.RO
keywords robot-assistedfeedingbiteacquisitionskillaffordancesvisuo-hapticperceptionfoodphysicalpropertiestoolcalibrationassistiveroboticsaffordancelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that, for robot-assisted feeding, the right manipulation skill (skewer, scoop, and so on) is determined not by the food's category but by the joint food–utensil interaction, and that this interaction can be captured by combining an offline tool calibration with online estimates of three food properties: softness, moisture, and viscosity. The authors build SAVOR, which first reasons about a food's properties from an image using a vision-language model, then refines those estimates with SAVOR-Net, a network that fuses time-series RGB-D, force-torque, and pose data collected during contact. At planning time, a vision-language model selects among six skills using the calibration summary and the updated property estimates. On 20 single food items and 10 in-the-wild plates, SAVOR reaches 51.5% average attempt efficiency and 87.3% success within three attempts, a 13% improvement over the category-based state of the art. If correct, feeding robots could adapt mid-meal to foods that firm up or become moister, instead of repeatedly failing whenever an item breaks category expectations.

What carries the argument

The load-bearing object is the affordance decomposition: skill affordance equals tool affordance plus food affordance. Tool affordance is carried by a calibration dataset, natural-language summaries of per-skill success rates for a given utensil across five calibration foods. Food affordance is carried by the three property scores and by SAVOR-Net, a multimodal network with separate encoders for RGB, depth, force-torque, and pose, fused by a two-layer LSTM to predict softness, moisture, and viscosity log-probabilities. The identity that does the work is the update rule: start from vision-language commonsense priors, accept SAVOR-Net's refinement only when its confidence exceeds a threshold, and give the calibration summary plus updated properties to the vision-language planner on every re-selection. This lets the system switch skills, for example from skewer to scoop, when a food item turns out firmer than its appearance suggested.

What would settle it

Take two foods that receive identical estimated property vectors but different internal textures, for example poached chicken breast and firm tofu both scored softness 3, moisture 3, viscosity 2, and run SAVOR with the same fork. If their skill success rates diverge systematically across repeated trials, the three scalar properties are not a sufficient statistic for skill affordance and the planner's core input is missing interaction-relevant physics.

Watch

Extended reading notes

Core claim

The central claim is that skill affordances for bite acquisition decompose into tool affordances and food affordances, and that both can be estimated rather than looked up from a food category. Tool affordances are learned offline: the robot executes each of six skills on five foods (raw carrot, cooked carrot, soft tofu, nuts, and cheese) for about 20 minutes and records per-skill success rates; this calibration dataset is the implicit model of what the utensil can do. Food affordances are represented by three scalar properties, softness, moisture, and viscosity, each rated 1–5. These are initialized by commonsense reasoning from a single RGB-D image and updated online whenever an attempt fails: SAVOR-Net ingests the time series of RGB-D images, force-torque readings, and end-effector poses from the failed attempt, and emits updated property log-probabilities that are accepted only above a confidence threshold. A vision-language planner then re-selects a skill from the library of push, cut, skewer, dip, scoop, and twirl. The paper reports that this pipeline achieves the highest average attempt efficiency (51.5%) and 87.3% success within three attempts across ten in-the-wild dishes, outperforming category-based FLAIR, a vision-only VLM, SayCan, and an end-to-end policy, and that adaptivity shows in the jump from 53.2% first-attempt success to 77.2% second-attempt success as the robot updates its estimate after failed tries.

Load-bearing premise

The method assumes that three 1–5 scores for softness, moisture, and viscosity, together with a 20-minute calibration on five foods, capture enough of the physics of utensil–food interaction to pick the right skill.

Editorial extensions

If this is right

  • Replacing category-based skill selection with property-based selection raises bite acquisition success by 13% over FLAIR and raises success-within-three-attempts from 73.4% to 87.3%.
  • Tool calibration on five foods transfers to new utensils and foods; with calibration, success-within-three-attempts improves by 13–18% depending on the fork and prevents infeasible actions such as skewering firm steak with a plastic fork.
  • Online visuo-haptic refinement converts first-attempt failures into second-attempt successes: SR1 is 53.2%, SR2 rises to 77.2%, and SR3 reaches 87.3%.
  • Property estimation generalizes to unseen foods: SAVOR outperforms FLAIR on 70% of 20 single food items, including foods never seen during training.
  • The system handles temporally varying food properties, switching skills as steak cools or chicken nuggets firm up, which category-based policies cannot do.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same affordance decomposition could transfer to other contact-rich assistive manipulation tasks, such as cooking robots choosing between cutting and tearing, wherever tool capability and object property jointly decide success.
  • If the calibration result is robust, a practical consequence the authors do not state is that adapting to a new utensil requires only a short recalibration run, not a new food-modeling effort; menu changes are handled by online property updates.
  • A testable extension of the thresholded-update idea is active exploration: when the network is uncertain, the planner could deliberately choose a skill that maximizes expected information gain about the property vector rather than the most likely success.
  • The slippage noise noted in the limitations suggests a drop-in improvement: a contact-onset and contact-loss detector on the haptic stream before SAVOR-Net would reduce mis-estimates on oily surfaces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents SAVOR, a system for robot-assisted bite acquisition that selects among six manipulation skills (skewer, scoop, twirl, dip, cut, push) by combining offline tool calibration with online food property estimation. Tool affordances are captured by a calibration dataset recording success rates of each skill on five foods per utensil. Food affordances are represented by three physical properties (softness, moisture, viscosity) on a 1–5 scale; these are initialized by a vision-language model (GPT-4V) and refined after failed attempts by SAVOR-Net, a multimodal LSTM that consumes time-series RGB-D, force-torque, and end-effector pose data. A VLM planner uses the current property estimates and calibration history to choose the next skill. The system is evaluated on a physical Kinova arm with 20 single food items (10 unseen during training) and 10 in-the-wild plates, reporting an average attempt efficiency of 51.5% and SR3 of 87.3%, outperforming FLAIR, SayCan, an end-to-end policy, and several ablations.

Significance. If the claimed mechanism is valid, the paper is a meaningful advance over category-based bite-acquisition policies: it demonstrates that tool-specific calibration plus property-grounded skill selection can improve success rates and adapt within a meal as food properties change (e.g., cooling steak). The real-robot evaluation with multiple baselines, ablations, and statistical tests is a strength, and the proposed dataset, though not yet released, could be useful to the community. However, the central mechanistic claim—that SAVOR-Net learns food physical properties from visuo-haptic signals—is not supported by any direct validation of the property estimates, which leaves the interpretation of the headline results open to a simpler failure-to-switch explanation.

major comments (3)
  1. [§4.2, §4.3, Table 1] The paper never validates SAVOR-Net's predictions of softness, moisture, and viscosity against ground-truth labels or human annotations on held-out data. Since the input includes the end-effector pose trajectory Pt, which effectively encodes the skill that was executed (skewer versus scoop versus cut), the network can learn a skill-to-property mapping that justifies a pre-programmed 'switch after failure' behavior without estimating food physics. The reported improvements in Table 1 and Figure 5 would then be equally consistent with a direct failure-to-switch heuristic, leaving the headline empirical gain intact but weakening the paper's interpretation. Please report held-out property classification accuracy (with confusion matrices and per-property errors) and compare against a baseline that replaces SAVOR-Net with a fixed 'switch skill after one failure' rule.
  2. [Table 1, Figure 4, Figure 7] The 10 in-the-wild dishes appear to contain many food items that are in the SAVOR-Net training set (e.g., carrot, broccoli, chicken, mushroom, steak, tofu), so the 13-percentage-point AE gain over FLAIR in Table 1 may partly reflect training-set familiarity. The paper does include 10 unseen single items in Figure 7 and shows gains there, which mitigates the concern, but the main dish-level results should be stratified into seen versus unseen items, or the claim should be narrowed to the single-item generalization experiment.
  3. [§4.3] The update rule uses a confidence threshold θth: 'if its log probability is less than a threshold θth, the prediction is ignored.' The manuscript does not specify the threshold value, how it was chosen, or its sensitivity. Because this parameter controls how often SAVOR-Net updates the planner's belief, the reported results are under-specified; please provide the value (or a tuning procedure) and a sensitivity analysis over a reasonable range.
minor comments (6)
  1. [Abstract, Table 1] The abstract says 'improves bite acquisition success rate by 13%' but Table 1 shows a gain of 13.3 percentage points in AE (51.5% versus 38.2%). Please state this as percentage points or as a relative improvement to avoid ambiguity.
  2. [Contribution (iv)] The claimed dataset is not linked in the manuscript; if it is released, include a URL in the final version.
  3. [Appendix B.4.4] In the example prompt, there is an unmatched quote in the action list: `'skewer, 'scoop'`; also, 'viscosity' appears with a lowercase 'v' in one history summary, which is inconsistent with the rest of the text.
  4. [Table 2, §4.3] Adding the value of θth to the timing table would improve reproducibility.
  5. [Figure 5] The caption says 'SR3 using a plastic/metal fork for 10 in-the-wild dishes' but the main text reports SR3 with a plastic fork; please clarify which utensil is used in Table 1.
  6. [§4.3] The paper would benefit from an explanation of how SAVOR-Net's online updates are integrated over multiple timesteps within a single failed attempt; §4.3 describes updates for each t > 0 but does not specify how the final property estimate is formed from multiple ψt.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-level circularity identified: the tool-calibration, property-estimation, and skill-planning pipeline is not self-referential, although the unvalidated property outputs are a separate correctness concern.

full rationale

The paper's claim chain is: (i) offline calibration records actual skill success rates for a given utensil on five foods and is used directly as natural-language few-shot context for planning; (ii) SAVOR-Net is trained to map raw (I,F,P) time series to softness/moisture/viscosity scores, with no stated equation tying its output to the success objective; and (iii) a VLM planner combines the calibration summary, attempt history, and property estimates to select a skill. None of these steps is a fitted parameter renamed as a prediction. The tool affordance is not predicted: it is a measured calibration history (Section 4.1). The food-property output psi_t is a supervised network output, not a computed function of success (Section 4.2). The headline 13% improvement is an external comparison against FLAIR and other baselines on the same held-out dishes, and the paper additionally reports SR3 on 10 unseen single items in Figure 7, so the central result has independent empirical content. The skeptic's concern that P_t encodes the executed skill and that SAVOR-Net's property estimates are never validated against ground-truth properties is a substantive internal-validity and evaluation gap, but it is a hypothesis about a possible shortcut, not a demonstration that any equation or fitted value reduces to its own input by construction. The self-citations (e.g., [4] for the choice of properties) are not load-bearing: choosing softness, moisture, and viscosity is a modeling assumption, not a theorem imported from the authors' prior work. No circular step is established under the stated criteria.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim relies on the adequacy of the hand-coded property representation and on the generalization of a 5-food calibration set. No new physical entities are introduced. The confidence threshold is an unstated free parameter.

free parameters (1)
  • Confidence threshold theta_th for SAVOR-Net updates = Not reported
    Section 4.3: predictions with log probability below theta_th are ignored; the value is hand-chosen and affects whether property estimates are updated.
assumptions (3)
  • domain assumption Visuo-haptic observations (RGB-D, force-torque, end-effector pose) provide sufficient information to infer softness, moisture, and viscosity.
    Section 3 states this hypothesis directly.
  • domain assumption The 1-5 Likert property representation is sufficient for skill selection on the tested foods.
    The method's state space is defined by these scalar descriptors; no other physical quantities are modeled.
  • domain assumption GPT-4V provides reliable commonsense priors for food properties and skill selection from natural-language calibration summaries.
    The planner and initial property estimator are implemented with GPT-4V; the paper treats its outputs as reliable enough for control.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition." pith.science (2026). https://pith.science/paper/73MPX3WW

@misc{pith2026250602353,
  author       = {Pith},
  title        = {Pith review of: SAVOR: Skill Affordance Learning from Visuo-Haptic Perception for Robot-Assisted Bite Acquisition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/73MPX3WW}},
  note         = {Machine review of arXiv:2506.02353}
}
read the original abstract

Robot-assisted feeding requires reliable bite acquisition, a challenging task due to the complex interactions between utensils and food with diverse physical properties. These interactions are further complicated by the temporal variability of food properties-for example, steak becomes firm as it cools even during a meal. To address this, we propose SAVOR, a novel approach for learning skill affordances for bite acquisition-how suitable a manipulation skill (e.g., skewering, scooping) is for a given utensil-food interaction. In our formulation, skill affordances arise from the combination of tool affordances (what a utensil can do) and food affordances (what the food allows). Tool affordances are learned offline through calibration, where different utensils interact with a variety of foods to model their functional capabilities. Food affordances are characterized by physical properties such as softness, moisture, and viscosity, initially inferred through commonsense reasoning using a visually-conditioned language model and then dynamically refined through online multi-modal visuo-haptic perception using SAVOR-Net during interaction. Our method integrates these offline and online estimates to predict skill affordances in real time, enabling the robot to select the most appropriate skill for each food item. Evaluated on 20 single-item foods and 10 in-the-wild meals, our approach improves bite acquisition success rate by 13% over state-of-the-art (SOTA) category-based methods (e.g. use skewer for fruits). These results highlight the importance of modeling interaction-driven skill affordances for generalizable and effective robot-assisted bite acquisition. Website: https://emprise.cs.cornell.edu/savor/

Figures

Figures reproduced from arXiv: 2506.02353 by the authors.

Figure 1
Figure 1. We propose SAVOR, a method that combines tool affordances and food affordances to select the appropriate manipulation skill for robust bite acquisition. Abstract: Robot-assisted feeding requires reliable bite acquisition, a challeng￾ing task due to the complex interactions between utensils and food with diverse physical properties. These interactions are further complicated by the temporal variability of food proper… view at source ↗
Figure 2
Figure 2. SAVOR Framework. Before deployment, we perform an offline tool calibration to un￾derstand tool affordances. During deployment, we first use a visually-conditioned language model to estimate food physical properties and then refine it through online visuo-haptic perception. Integrating Vision and Haptics in Robotics. Vision-only manipulation has demonstrated effective￾ness in robotic domains such as semantic grasping… view at source ↗
Figure 3
Figure 3. (a) Skill library for bite acquisition. (b) SAVOR-Net model architecture. maining properties are scalar values from 1 to 5, similar to a 5-point Likert scale. For example, a softness score of 1 indicates very hard, while a score of 5 indicates very soft. We hypothesize that the visuo-haptic observations (I, F) provide sufficient information to infer these physical proper￾ties. The robot should use its estimates of t… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Experimental setup: 10 in-the-wild dishes. ∗ denotes food items unseen during training [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: (a) System setup. (b) SR3 using a plastic/metal fork for 10 in-the-wild dishes. (c) [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results on bite acquisition. The robot first attempts to skewer the food based on its initial property estimate but fails (step 2). Vision and haptic data from this attempt are processed by SAVOR-Net, refining the estimate with high confidence. A VLM planne…
Figure 7
Figure 7. Figure 7: Generalization performance on seen and unseen food items. We compare SAVOR and SOTA FLAIR across 20 food items. We show SR3 on 10 episodes per food item, where each episode allows up to 3 attempts. Asterisks (∗) indicate statistically significant differences (p < 0.05)…
Figure 8
Figure 8. Figure 8: Effect of food physical properties on utensil interactions. The robot skewers food items of varying softness (top) and viscosity (bottom). Soft tofu and low-viscosity mashed potatoes are successfully acquired, while firm tofu and high-viscosity mashed potatoes lead to …

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Human-Centered Shared Autonomy for Motor Planning, Learning, and Control Applications

    cs.HC 2025-06 unverdicted novelty 3.0 of 10

    A review chapter that organizes BCI, rehabilitation, and assistive robotics under a single adaptive-arbitration framework and illustrates it with the authors' own prior systems.

Reference graph

Works this paper leans on

49 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Global report on health equity for persons with disabilities

    World Health Organization. Global report on health equity for persons with disabilities . World Health Organization, 2022. ISBN 9789240063600. URL https://www.who.int/ publications/i/item/9789240063600

    arXiv 2022

  2. [2]

    Nanavati, P

    A. Nanavati, P. Alves-Oliveira, T. Schrenk, E. K. Gordon, M. Cakmak, and S. S. Srinivasa. Design principles for robot-assisted feeding in social contexts. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, HRI ’23, page 24–33, New York, NY , USA, 2023. Association for Computing Machinery. ISBN 9781450399647. doi: 10.1...

    work page arXiv 2023

  3. [3]

    R. K. Jenamani, P. Sundaresan, M. Sakr, T. Bhattacharjee, and D. Sadigh. Flair: Feeding via long-horizon acquisition of realistic dishes. arXiv preprint arXiv:2407.07561, 2024

    arXiv 2024

  4. [4]

    N. Ha, R. Ye, Z. Liu, S. Sinha, and T. Bhattacharjee. Repeat: A real2sim2real approach for pre-acquisition of soft food items in robot-assisted feeding. 2024

    work page 2024

  5. [5]

    Bhattacharjee, G

    T. Bhattacharjee, G. Lee, H. Song, and S. S. Srinivasa. Towards robotic feeding: Role of haptics in fork-based food manipulation. IEEE Robotics and Automation Letters , 4(2):1485– 1492, 2019

    work page 2019

  6. [6]

    R. K. Jenamani, D. Stabile, Z. Liu, A. Anwar, K. Dimitropoulou, and T. Bhattacharjee. Feel the bite: Robot-assisted inside-mouth bite transfer using robust mouth perception and physical interaction-aware control. In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, pages 313–322, 2024

    work page 2024

  7. [7]

    Nanavati, E

    A. Nanavati, E. K. Gordon, T. A. Kessler Faulkner, Y . R. Song, J. Ko, T. Schrenk, V . Nguyen, B. H. Zhu, H. Bolotski, A. Kashyap, S. Kutty, R. Karim, L. Rainbolt, R. Scalise, H. Song, R. Qu, M. Cakmak, and S. S. Srinivasa. Lessons learned from designing and evaluating a robot- assisted feeding system for out-of-lab use. InProceedings of the 2025 ACM/IEEE...

    work page 2025

  8. [8]

    Sundaresan, S

    P. Sundaresan, S. Belkhale, and D. Sadigh. Learning visuo-haptic skewering strategies for robot-assisted feeding. In 6th Annual Conference on Robot Learning , 2022. URL https: //openreview.net/forum?id=lLq09gVoaTE

    work page 2022

Show all 49 references
  1. [9]

    E. K. Gordon, X. Meng, T. Bhattacharjee, M. Barnes, and S. S. Srinivasa. Adaptive robot- assisted feeding: An online learning framework for acquiring previously unseen food items. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 9659–9...

    2020

  2. [10]

    E. K. Gordon, S. Roychowdhury, T. Bhattacharjee, K. Jamieson, and S. S. Srinivasa. Lever- aging post hoc context for faster learning in bandit settings with applications in robot-assisted feeding. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages ...

    2021

  3. [11]

    E. K. Gordon, A. Nanavati, R. Challa, B. H. Zhu, T. A. K. Faulkner, and S. Srinivasa. Towards general single-utensil food acquisition with human-informed actions. In Conference on Robot Learning, pages 2414–2428. PMLR, 2023

    2023

  4. [12]

    R. Feng, Y . Kim, G. Lee, E. K. Gordon, M. Schmittle, S. Kumar, T. Bhattacharjee, and S. S. Srinivasa. Robot-assisted feeding: Generalizing skewering strategies across food items on a plate. In The International Symposium of Robotics Research , pages 427–442. Springer, 2019

    2019

  5. [13]

    Sundaresan, J

    P. Sundaresan, J. Wu, and D. Sadigh. Learning sequential acquisition policies for robot-assisted feeding. In J. Tan, M. Toussaint, and K. Darvish, editors,Proceedings of The 7th Conference on Robot Learning, volume 229 ofProceedings of Machine Learning Research, pages 1282–129...

    2023

  6. [14]

    Y .-L. Tai, Y . C. Chiu, Y .-W. Chao, and Y .-T. Chen. Scone: A food scooping robot learning framework with active perception. In J. Tan, M. Toussaint, and K. Darvish, editors, Pro- ceedings of The 7th Conference on Robot Learning , volume 229 of Proceedings of Machine Learnin...

    2023

  7. [15]

    Grannen, Y

    J. Grannen, Y . Wu, S. Belkhale, and D. Sadigh. Learning bimanual scooping policies for food acquisition. In 6th Annual Conference on Robot Learning, 2022. URL https://openreview. net/forum?id=qDtbMK67PJG

    2022

  8. [16]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox. Cliport: What and where pathways for robotic manipu- lation. In Conference on robot learning, pages 894–906. PMLR, 2022

    2022

  9. [17]

    Brosnan and D.-W

    T. Brosnan and D.-W. Sun. Improving quality inspection of food products by computer vi- sion––a review. Journal of F ood Engineering , 61(1):3–16, 2004. ISSN 0260-8774. doi: https://doi.org/10.1016/S0260-8774(03)00183-3. URL https://www.sciencedirect.com/ science/article/pii/S...

    2004 doi

  10. [18]

    Z. He, B. Ai, Y . Liu, W. Wan, H. I. Christensen, and H. Su. Learning dexterous deformable object manipulation through cross-embodiment dynamics learning. In RSS Workshop on Dex- terous Manipulation: Learning and Control with Diverse Data , 2025

    2025

  11. [19]

    T. Tian, H. Li, B. Ai, X. Yuan, Z. Huang, and H. Su. Diffusion dynamics models with genera- tive state estimation for cloth manipulation. Conference on Robot Learning (CoRL) , 2025

    2025

  12. [20]

    H. Qi, B. Yi, S. Suresh, M. Lambeta, Y . Ma, R. Calandra, and J. Malik. General In-Hand Object Rotation with Vision and Touch. In Conference on Robot Learning (CoRL) , 2023

    2023

  13. [21]

    Suresh, H

    S. Suresh, H. Qi, T. Wu, T. Fan, L. Pineda, M. Lambeta, J. Malik, M. Kalakrishnan, R. Ca- landra, M. Kaess, J. Ortiz, and M. Mukadam. Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation. Science Robotics, page adl0628, 2024

    2024

  14. [22]

    B. Ai, S. Tian, H. Shi, Y . Wang, C. Tan, Y . Li, and J. Wu. Robopack: Learning tactile- informed dynamics models for dense packing. Robotics: Science and Systems (RSS) , 2024. URL https://arxiv.org/abs/2407.01418

    2024 arXiv

  15. [23]

    Bhattacharjee, G

    T. Bhattacharjee, G. Lee, H. Song, and S. S. Srinivasa. Towards robotic feeding: Role of haptics in fork-based food manipulation. IEEE Robotics and Automation Letters , 4(2):1485– 1492, 2019. doi:10.1109/LRA.2019.2894592

    2019

  16. [24]

    Yamaguchi and C

    A. Yamaguchi and C. G. Atkeson. Combining finger vision and optical tactile sensing: Reducing and handling errors while cutting vegetables. In 2016 IEEE-RAS 16th Inter- national Conference on Humanoid Robots (Humanoids) , pages 1045–1051, 2016. doi: 10.1109/HUMANOIDS.2016.7803400

    2016

  17. [25]

    Z. Xu, Z. Xian, X. Lin, C. Chi, Z. Huang, C. Gan, and S. Song. Roboninja: Learning an adaptive cutting policy for multi-material objects. In Proceedings of Robotics: Science and Systems (RSS), 2023

    2023

  18. [26]

    Heiden, M

    E. Heiden, M. Macklin, Y . S. Narang, D. Fox, A. Garg, and F. Ramos. DiSECt: A Differ- entiable Simulation Engine for Autonomous Robotic Cutting. In Proceedings of Robotics: Science and Systems, Virtual, July 2021. doi:10.15607/RSS.2021.XVII.067

    2021 doi

  19. [27]

    Zhang, M

    K. Zhang, M. Sharma, M. Veloso, and O. Kroemer. Leveraging multimodal haptic sensory data for robust cutting. In2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), pages 409–416. IEEE, 2019. 11

    2019

  20. [28]

    I. Lenz, R. A. Knepper, and A. Saxena. Deepmpc: Learning deep latent features for model predictive control. In Robotics: Science and Systems , volume 10, page 25. Rome, Italy, 2015

    2015

  21. [29]

    R. Ye, Y . Hu, Y . A. Bian, L. Kulm, and T. Bhattacharjee. Morpheus: a multimodal one- armed robot-assisted peeling system with human users in-the-loop. In 2024 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 9540–9547, 2024. doi: 10.1109/ICRA57147.20...

    2024

  22. [30]

    C. Dong, L. Yu, M. Takizawa, S. Kudoh, and T. Suehiro. Food peeling method for dual-arm cooking robot. pages 801–806, 01 2021. doi:10.1109/IEEECONF49454.2021.9382700

    2021

  23. [31]

    X. Luo, S. Jin, H.-J. Huang, and W. Yuan. An intelligent robotic system for perceptive pan- cake batter stirring and precise pouring. In 2024 IEEE/RSJ International Conference on In- telligent Robots and Systems (IROS) , pages 5970–5977, 2024. doi:10.1109/IROS58592.2024. 10802841

    2024

  24. [32]

    Bommasani, D

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N. S. Chat- terji, A. S. Chen, K. A. Creel, J. Davis, D. Demszky, C. Donahue, M. Doumbouya, E. Durmus, S...

    2021

  25. [34]

    Huang, F

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar, P. Sermanet, N. Brown, T. Jackson, L. Luu, S. Levine, K. Hausman, and B. Ichter. Inner monologue: Embodied reasoning through planning with language models. In arXiv prepr...

    2022 arXiv

  26. [35]

    J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser. Tidybot: Personalized robot assistance with large language models. Au- tonomous Robots, 2023

    2023

  27. [36]

    Z. Wu, B. Ai, and D. Hsu. Integrating common sense and planning with large language models for room tidying. In RSS 2023 Workshop on Learning for Task and Motion Planning , 2023

    2023

  28. [37]

    Singh, V

    I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 11523– 11530, 2023....

    2023

  29. [38]

    Driess, F

    D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y . Chebotar, P. Sermanet, D. Duckworth, S. Levine, V . Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence. Palm-e: An emb...

    2023 arXiv

  30. [39]

    C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y . Su. Llm-planner: Few- shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2023

    2023

  31. [40]

    Ichter, A

    B. Ichter, A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Ir- pan, E. Jang, R. Julian, D. Kalashnikov, S. Levine, Y . Lu, C. Parada, K. Rao, P. Sermanet, A. Toshev, V . Vanhoucke, F. Xia, T. Xiao, P. Xu, M. Yan, N. Brown, M. Ahn, O. Cortes, N. Sie...

    2022

  32. [41]

    Y . R. Wang, J. Duan, D. Fox, and S. S. Srinivasa. NEWTON: are large language models capable of physical reasoning? In Findings of the Association for Computational Linguistics (EMNLP), 2023

    2023

  33. [42]

    Q. Gao, X. Pi, K. Liu, J. Chen, R. Yang, X. Huang, X. Fang, L. Sun, G. Kishore, B. Ai, S. Tao, M. Liu, J. Yang, C.-J. Lai, C. Jin, J. Xiang, B. Huang, D. Danks, H. Su, T. Shu, Z. Ma, L. Qin, and Z. Hu. Do vision-language models have internal world models? towards an atomic eva...

    2025

  34. [43]

    L. Shi, Y . Liu, L. Zeng, B. Ai, Z. Hong, and H. Su. Learning adaptive dexterous grasping from single demonstrations. arXiv preprint arXiv:2503.20208, 2025

    2025 arXiv

  35. [44]

    J. Loo, Z. Wu, and D. Hsu. Open scene graphs for open-world object-goal navigation. The International Journal of Robotics Research, 2025

    2025

  36. [45]

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observ- able stochastic domains. Artif. Intell., 101(1-2):99–134, 1998. doi:10.1016/S0004-3702(98) 00023-X. URL https://doi.org/10.1016/S0004-3702(98)00023-X

    1998 doi

  37. [46]

    M. C. Gemici and A. Saxena. Learning haptic representation for manipulating deformable food objects. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 638–645, 2014. doi:10.1109/IROS.2014.6942626

    2014

  38. [47]

    Nanavati, R

    A. Nanavati, R. Challa, E. K. Gordon, and S. S. Srinivasa. A Dataset of Food Manipulation Strategies for Diverse Foods, 2022. URL https://doi.org/10.7910/DVN/C8SI1D

    2022 doi

  39. [48]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  40. [49]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, C. Li, J. Yang, H. Su, J. Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  41. [50]

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159, 2024. 13 Appendix A Additional Results A.1 Timing and Latency Analysis We present a...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.