Demonstrating the Octopi-1.5 Visual-Tactile-Language Model

Harold Soh; Kelvin Lin; Samson Yu

REVIEW 3 major objections 5 minor 2 cited by

Demonstrating the Octopi-1.5 Visual-Tactile-Language Model

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A touch-based language model uses text retrieval to name grasped objects at 96-100% accuracy on familiar items.

desk verdict Solid demo paper with a plausible but under-controlled RAG claim; the numbers in Table III are suspect because the retrieval database likely overlaps with the test objects. read the letter →

arxiv 2507.09985 v1 pith:YFCXHLHE submitted 2025-07-14 cs.RO cs.AI

Samson Yu , Kelvin Lin , Harold Soh This is my paper · ORCID

classification cs.ROcs.AI

keywords visual-tactile-languagemodelretrieval-augmentedgenerationtactileperceptionobjectidentificationGelSightTAC-02multi-partreasoningrobottouch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Octopi-1.5 is a visual-tactile-language model built on Qwen2-VL 7B that tries to answer a simple question: given only touch signals, what object is in your hand? Its central claim is that two modest additions — processing tactile frames from several parts of an object, and retrieving plain-text descriptions of similar objects from a database before answering — sharply improve tactile inference. On the planned demo tasks, these additions raise accuracy on familiar balls and fruits to 96-100 percent, give 73 percent on objects the system has not seen named, and reach 95 percent after a user teaches a new object by adding its sample to the database. If true, this matters because it points to a cheap way to extend a tactile model to new objects without retraining, and because the whole setup runs on a handheld gripper rather than a robot.

What carries the argument

Retrieval-augmented generation (RAG) over tactile embeddings is the load-bearing mechanism: the model averages the embeddings of the query's salient tactile frames, searches a database of known objects by cosine similarity, and injects the names and touch descriptions of the closest matches into the language prompt. A second mechanism is the multi-part tactile encoder, which processes frames from several parts of an object so descriptions and hardness/roughness rankings can be made per part. Together they turn the base VLM's commonsense reasoning into grounded guesses about what is being touched.

What would settle it

Remove every evaluation object (balls, fruits, hairbrush, microfiber cloth) and their near-duplicates from the RAG database, then rerun the Guessing Game; if unseen accuracy falls to the no-RAG level or the retrieved top-5 lists never contain the true label, the RAG advantage is label lookup, not tactile generalization.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a retrieval-augmented generation module plus multi-part tactile encoding turns a visual-tactile-language model from a poor guesser into a reliable identifier under touch-only conditions. Octopi-1.5 first encodes salient GelSight frames from one or more object parts with a fine-tuned CLIP encoder, feeds those tokens into Qwen2-VL 7B, and, when RAG is enabled, augments the prompt with the labels and tactile descriptions of the top-5 most similar objects found by cosine similarity over a database of tactile embeddings. In the Guessing Game, the model's accuracy on balls and fruits jumps from 44-58 percent without RAG to 96-100 percent with RAG, and from 41 percent to 73 percent on unseen objects; adding the new item to the database as a teaching step brings unseen-object accuracy to 95 percent. The paper also reports that the model can describe and rank different parts of the same object, and the accompanying TMI gripper combines a GelSight Mini and a TAC-02 sensor in a portable, robot-free demonstrator.

Load-bearing premise

The retrieval database is built from the same tactile datasets used to train and evaluate the model, and the paper never says the test balls, fruits, and unseen items were removed from it, so the accuracy gain may come from the database containing the correct answer rather than from tactile understanding.

Editorial extensions

If this is right

With RAG enabled, touch-only identification reaches 96-100% on familiar balls and fruits, so a handheld tactile interface without any robot can demonstrate object recognition in a few minutes.
New object-tactile pairings can be added by inserting samples into the retrieval database, and accuracy on previously unseen objects rises to 95% after such teaching.
Multi-part tactile input lets the model rank parts by hardness and roughness, supporting sorting tasks based on material property; sorting accuracy is 100% on balls and 93% on fruits.
The approach still struggles on fully unseen objects in sorting, with 43% accuracy, and roughness ranking remains error-prone, marking limits the authors flag as open.
Because only text is retrieved, tactile qualities that are hard to verbalize may be lost; the paper states that retrieving tactile embeddings directly would likely require retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported RAG gain may partly be label lookup: the database is built from the same datasets that supplied the evaluation objects, and the paper does not state that the unseen items were excluded from retrieval, so a clean held-out database test is required to separate memorization from generalization.
If the RAG database is made disjoint from test objects, the most direct testable prediction is that unseen-object accuracy drops toward the no-RAG level; this could be checked by logging the top-5 retrieved labels and seeing whether the true object's name appears.
The multi-part design suggests a natural extension to part-level property queries such as which side is rougher, and the TAC-02 pressure stream could provide dynamic cues such as weight or compliance that static GelSight frames miss, though the paper has no training data for it yet.
A VTLM that chains RAG with a manipulation policy, such as the vision-tactile-language-action model the authors say is in progress, could use the retrieved object's handling instructions to set grasp force, e.g. soft for fruits.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Octopi-1.5 is a reasonable incremental step in tactile-language models: a larger tactile encoder, explicit multi-part object reasoning, and a simple RAG module that retrieves text descriptions of similar objects. The TMI hardware looks practical, the code is out, and the planned demos seem feasible. I expect the live demonstrations to work. The real question is whether the RAG numbers in Table III mean anything. The retrieval database is built from the same datasets used to train the encoder (PhysiCLeAR, Hardness, ObjectFolder), and the paper never states that the test balls, fruits, or 'unseen' objects are excluded from that database. ObjectFolder-Real alone contains many household objects, so there is a real chance the query objects are literally in the index. If so, the top-5 retrieval is partly label lookup, and the jump from 56% to 96% on balls is not clean evidence of improved tactile inference. The teaching row is worse: adding samples to the database before evaluation makes the 95% score close to retrieval by construction. That said, this is a demo paper, and the authors are not overclaiming. They call the RAG 'experimental' and note limitations. The sorting results are honestly reported as prose, and the failure cases are highlighted. For a demo, the system is clearly described and likely implementable. The missing piece is a controlled evaluation: exclude all query objects from the retrieval set and report error bars. That is a straightforward fix, not a fatal flaw. I recommend sending this to peer review for the demo track. It is not a rigorous research contribution yet, but it is a legitimate system demonstration with a useful cautionary lesson about evaluation leakage. A serious referee can push the authors to either fix the protocol or soften the RAG claim. I would bring it to a reading group to discuss retrieval evaluation pitfalls, but I would not cite it in my own work yet. The thinking is clear and the limitations section is honest; the overlap issue is an oversight, not bad faith.

Referee Report

3 major / 5 minor

Summary. This demonstration paper presents Octopi-1.5, a visual-tactile-language model built on Qwen2-VL 7B, which extends the earlier Octopi system with a tactile encoder trained on PhysiCLeAR, Hardness, and ObjectFolder datasets, the ability to process tactile signals from multiple object parts, and a retrieval-augmented generation (RAG) module that augments tactile descriptions with text from similar objects. The paper also introduces the Tactile Manipulation Interface (TMI), a handheld gripper with GelSight and TAC-02 sensors, and outlines three proposed demonstrations: the Guessing Game, Sorting, and Free Interaction. Preliminary results in Table III report that RAG raises the Guessing Game accuracy of Octopi-1.5 to 96-100% on seen balls and fruits, 73% on unseen objects, and 95% after a 'teaching' step that adds new samples to the retrieval database.

Significance. If the RAG result is genuine, the paper demonstrates a computationally cheap way to improve tactile object identification and to add new object-tactile pairings at runtime, which is a useful capability for interactive robotics. The TMI hardware and the open code/CAD files are concrete contributions that others can build on. However, the headline accuracy gains rest on an evaluation that does not rule out a labeling confound: the RAG retrieval database appears to be the same training data that may contain the evaluation objects, and the 'teaching' condition explicitly inserts the query item into the database. The central claim is therefore currently under-supported; a controlled evaluation would materially strengthen the paper.

major comments (3)

[III-A and IV-A, Table III] The RAG module retrieves from 'an existing dataset of tactile embeddings' (Section III-A) that appears to be the same training datasets listed in Table I. The Guessing Game objects include common household items such as baseball, apple, orange, and hairbrush, which may also appear in ObjectFolder-Real or PhysiCLeAR. The paper never states that these evaluation objects, or near-duplicate samples of them, were excluded from the retrieval index. If they were not excluded, the top-5 retrieved items can contain the object's own label, so the accuracy jump in Table III (for example, Octopi-1.5 with RAG rising from 56.00% to 96.00% on Balls) is at least partly lookup rather than tactile generalization. The authors should state whether the retrieval database contained the evaluation objects and should report results with the query objects removed from the index.
[IV-A, Table III, 'Unseen (teaching)' column] The 'teaching' condition is described as 'adding new item samples to the dataset on the fly.' Because the new object's samples are inserted into the retrieval database before the post-teaching evaluation, a nearest-neighbor retrieval system will return the correct label almost by construction. The 95.12% post-teaching score therefore does not establish that Octopi-1.5 has learned a new object-tactile pairing; it demonstrates a database write. To support the on-the-fly learning claim, the authors should either show behavior beyond label retrieval (for example, follow-up reasoning about the new object that uses the retrieved text) or rephrase the claim as database augmentation rather than model learning.
[Table III and Section IV-A] The Guessing Game accuracies are reported as averages over 25, 26, and 41 samples, but no variance, confidence intervals, or per-category breakdowns are given. Since the central RAG claim rests entirely on these numbers, the authors should report standard errors or bootstrap intervals, and ideally a small confusion matrix, so that readers can assess whether the large differences are stable rather than the result of a few outlier trials.

minor comments (5)

[IV-B] The sorting task results (100% for balls, 93.18% for fruits, 43.33% for unseen objects) are reported only in prose; a table with the number of trials and the evaluation protocol would be more informative and would match the style used for the Guessing Game.
[III-A] The salient-frame heuristic ('top 10 frames with the largest differences') and the RAG 'top-5' selection are fixed constants; a sentence justifying these values or a small sensitivity study would help readers assess how dependent the results are on these choices.
[IV-A, Fig. 7] The example in Fig. 7 shows a long chain-of-thought explanation for the apple guess; please clarify whether this is the raw model output or part of the simplified chat interface that hides more complex prompts.
[III-B] The TAC-02 sensor is described in detail, but all quantitative experiments use the GelSight Mini; the paper should state explicitly whether TAC-02 data are used in any of the demonstrated tasks or only showcased as a hardware proof-of-concept.
[V] The limitations paragraph correctly notes that the RAG module is text-based, but it does not mention the possibility of retrieval-database overlap with the evaluation objects; this limitation should be acknowledged alongside the others.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'teaching' result in Table III is a database lookup by construction, and the paper does not document whether the 'unseen' evaluation objects are excluded from the retrieval index.

fitted input called prediction [Section IV-A (Guessing Game) and Table III]
"Simple RAG retrieval improves performance and "teaching" further boosts results by adding new item samples to the dataset on the fly. With RAG enabled, accuracy scores for Octopi-1.5 are high (96-100%) for the seen objects (Balls, Fruits) with good performance (73%) for unseen objects. Once teaching is enabled, performance rises to (95%)."

Teaching is defined as inserting the query object's own samples into the retrieval database. The RAG procedure then computes the query embedding, runs a cosine-similarity search over that database, and aggregates the top-5 retrieved object labels. The just-inserted samples are by construction the most similar to the query, so the correct label is guaranteed to enter the retrieved context. The reported 95% 'unseen (teaching)' accuracy therefore measures the success of retrieving the stored answer, not the VTLM's ability to infer or generalize a new object-tactile pairing. The 'learn new objects on-the-fly' claim reduces to storing the answer in memory and reading it back.

full rationale

The core model components are not circular: the tactile encoder is trained on PhysiCLeAR, Hardness, and ObjectFolder with contrastive and regression losses, then finetuned end-to-end with Qwen2-VL, and the no-RAG accuracy comparisons provide independent behavioral content. However, the central demonstration of on-the-fly learning rests on the teaching row, which is circular by construction because the evaluation object is added to the retrieval database before the accuracy is measured. The 73% accuracy on 'completely unseen objects' could have provided independent evidence of tactile generalization, but the paper never states that the hairbrush and microfiber objects are excluded from the 'existing dataset of tactile embeddings' built from the same training corpora; without that exclusion, part of the RAG gain may be label retrieval rather than inferred tactile understanding. This partial circularity of the headline RAG/teaching claim warrants a score of 6.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The system combines established components (Qwen2-VL, CLIP, contrastive learning, RAG) with several hand-picked design choices: 10 salient frames, top-5 retrieval, and a 0-10 annotation scale. The most consequential unstated assumption is the relationship between the retrieval database and the evaluation objects; if they overlap, the central RAG improvement can be explained by label retrieval. The human-annotation and CLIP-transfer assumptions are standard but unverified here.

free parameters (3)

k_salient_frames = 10
Tactile encoder selects the top 10 frames with largest difference from the previous frame; this hand-chosen threshold is not ablated and affects what tactile information reaches the model.
n_retrieved_objects = 5
RAG retrieves top-5 most similar objects from the embedding database; the number is fixed without ablation and directly determines how much retrieved text the VLM sees.
validation_objects = 6 unseen objects
The CLIP encoder checkpoint is selected by validation loss on a set of six unseen objects; the composition of this set shapes the encoder and is a design choice.

assumptions (5)

domain assumption CLIP visual representations transfer to GelSight tactile images after fine-tuning.
The tactile encoder starts from CLIP (L/14) and is trained with contrastive and regression losses, assuming tactile images are close enough to natural images for CLIP's prior to help. Invoked in Sec. III.A.
ad hoc to paper Largest frame-to-frame difference identifies salient tactile information.
The salient frame heuristic keeps the top 10 frames by difference to the preceding frame; there is no evidence that this captures the most informative contact events. Invoked in Sec. III.A.
domain assumption Mean embedding cosine similarity is a valid object-similarity measure for retrieval.
RAG averages tactile embeddings and searches by cosine similarity, presuming the embedding space is organized by object and part identity. Invoked in Sec. III.A.
domain assumption Human-annotated hardness and roughness scores are reliable ground truth.
The regression head is trained on these labels (Table II), so encoder quality and downstream tasks inherit the reliability and consistency of these annotations. Invoked in Sec. III.A.
ad hoc to paper The retrieval database does not contain the query objects used in evaluation.
Not stated in the paper. For seen objects the database likely includes them; if so, retrieval can return the correct label directly, undermining the RAG improvement claim. Relevant to Sec. IV.A and Table III.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Demonstrating the Octopi-1.5 Visual-Tactile-Language Model." pith.science (2026). https://pith.science/paper/YFCXHLHE

@misc{pith2026250709985,
  author       = {Pith},
  title        = {Pith review of: Demonstrating the Octopi-1.5 Visual-Tactile-Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFCXHLHE}},
  note         = {Machine review of arXiv:2507.09985}
}

read the original abstract

Touch is recognized as a vital sense for humans and an equally important modality for robots, especially for dexterous manipulation, material identification, and scenarios involving visual occlusion. Building upon very recent work in touch foundation models, this demonstration will feature Octopi-1.5, our latest visual-tactile-language model. Compared to its predecessor, Octopi-1.5 introduces the ability to process tactile signals from multiple object parts and employs a simple retrieval-augmented generation (RAG) module to improve performance on tasks and potentially learn new objects on-the-fly. The system can be experienced live through a new handheld tactile-enabled interface, the TMI, equipped with GelSight and TAC-02 tactile sensors. This convenient and accessible setup allows users to interact with Octopi-1.5 without requiring a robot. During the demonstration, we will showcase Octopi-1.5 solving tactile inference tasks by leveraging tactile inputs and commonsense knowledge. For example, in a Guessing Game, Octopi-1.5 will identify objects being grasped and respond to follow-up queries about how to handle it (e.g., recommending careful handling for soft fruits). We also plan to demonstrate Octopi-1.5's RAG capabilities by teaching it new items. With live interactions, this demonstration aims to highlight both the progress and limitations of VTLMs such as Octopi-1.5 and to foster further interest in this exciting field. Code for Octopi-1.5 and design files for the TMI gripper are available at https://github.com/clear-nus/octopi-1.5.

Figures

Figures reproduced from arXiv: 2507.09985 by the authors.

**Figure 2.** A provides a high-level [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 2.** (A) Octopi-1.5 model architecture. Octopi-1.5 is a fine-tuned Qwen2-VL 7B multimodal VLM with a tactile encoder and retrieval module. (B) The CLIP module of the tactile encoder is pre-trained before finetuning using contrastive and regression losses on PhysiCLeAR, Hardness, and ObjectFolder datasets. Please see main text for details. TABLE II ANNOTATOR SCORING GUIDANCE FOR HARDNESS AND ROUGHNESS. Property Score Desc… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: The outcome of a tactile description by Octopi-1.5 augmented with [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 5.** Figure 5: TMI fingers and CAD drawings with compartments for inserting the [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

**Figure 7.** Figure 7: User Interaction Example for the Guessing Game. [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

**Figure 8.** Figure 8: Example User Interaction for the Sorting Task. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

TacReasoner: A Dynamic Tactile-Language Framework for Interactive Reasoning in Real-World Scenarios
cs.AI 2026-07 conditional novelty 6.0 of 10

A dynamic-aware tactile encoder plus TouchCoT-10k chain-of-thought data lets a 7B model outperform larger tactile-language baselines on physical-property and real-world reasoning tasks.
LightTact: A Visual-Tactile Fingertip Sensor for Deformation-Independent Contact Sensing
cs.RO 2025-12 conditional novelty 6.0 of 10

A fingertip camera sensor uses a light-blocking wedge so that only true contact pixels brighten, enabling deformation-free contact detection with liquids, soft materials, and rigid objects.

Reference graph

Works this paper leans on

19 extracted references · 14 canonical work pages · cited by 2 Pith papers

[1]

Gelsight: High-resolution robot tactile sensors for esti- mating geometry and force

Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gelsight: High-resolution robot tactile sensors for esti- mating geometry and force. Sensors, 17(12):2762, 2017

2017
[2]

Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation

Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dave Stroud, Raymond Santos, Ahmad Byagowi, Gregg Kammerer, et al. Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation. IEEE Robotics and Automation Letters , 5(3):3838–3845, 2020

work page 2020
[3]

Octopi: Object property reasoning with large tactile-language models, 2024

Samson Yu, Kelvin Lin, Anxing Xiao, Jiafei Duan, and Harold Soh. Octopi: Object property reasoning with large tactile-language models, 2024

work page 2024
[4]

A touch, vision, and language dataset for multimodal alignment

Letian Fu, Gaurav Datta, Huang Huang, William Chung- Ho Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, and Ken Goldberg. A touch, vision, and language dataset for multimodal alignment. In International Conference on Machine Learning , 2024

work page 2024
[5]

Binding touch to everything: Learning unified multimodal tactile representations

Fengyu Yang, Chao Feng, Ziyang Chen, Hyoungseob Park, Daniel Wang, Yiming Dou, Ziyao Zeng, Xien Chen, Rit Gangopadhyay, Andrew Owens, et al. Binding touch to everything: Learning unified multimodal tactile representations. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 26340–26353, 2024

work page 2024
[6]

Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhi- hao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 , 2024

arXiv 2024
[7]

Retrieval-augmented generation for large language mod- els: A survey

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jin- liu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augmented generation for large language mod- els: A survey. arXiv preprint arXiv:2312.10997 , 2023

arXiv 2023
[8]

Universal manipulation interface: In- the-wild robot teaching without in-the-wild robots

Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In- the-wild robot teaching without in-the-wild robots. In Proceedings of Robotics: Science and Systems (RSS) , 2024

2024

Show all 19 references

[9]

Robotic tactile perception of object properties: A review

Shan Luo, Joao Bimbo, Ravinder Dahiya, and Hongbin Liu. Robotic tactile perception of object properties: A review. Mechatronics, 48:54–67, 2017

2017
[10]

Soft tactile sensing skins for robotics

Peter Roberts, Mason Zadan, and Carmel Majidi. Soft tactile sensing skins for robotics. Current Robotics Reports, 2:343–354, 2021

2021
[11]

A comprehensive review of robot intelligent grasping based on tactile percep- tion

Tong Li, Yuhang Yan, Chengshun Yu, Jing An, Yi- fan Wang, and Gang Chen. A comprehensive review of robot intelligent grasping based on tactile percep- tion. Robotics and Computer-Integrated Manufacturing , 90:102792, 2024

2024
[12]

A survey on multi- modal large language models

Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multi- modal large language models. National Science Review , 11(12):nwae403, 11 2024

2024
[13]

Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations

Ruohan Gao, Yen-Yu Chang, Shivani Mall, Li Fei-Fei, and Jiajun Wu. Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations. In Conference on Robot Learning (CoRL) , 2021

2021
[14]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

2021
[15]

Estimating object hardness with a gelsight touch sensor

Wenzhen Yuan, Mandayam A Srinivasan, and Edward H Adelson. Estimating object hardness with a gelsight touch sensor. In 2016 IEEE/RSJ International Confer- ence on Intelligent Robots and Systems (IROS) , pages 208–215. IEEE, 2016

2016
[16]

Judging llm-as- a-judge with mt-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuo- han Li, Dacheng Li, Eric Xing, et al. Judging llm-as- a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems , 36:46595– 46623, 2023

2023
[17]

Learning transferable visual models from natural lan- guage supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural lan- guage supervision. In International conference on ma- chine learning...

2021
[18]

Visual prompt tuning

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer, 2022

2022
[19]

Robust speech recognition via large-scale weak supervision

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In Inter- national conference on machine learning , pages 28492– 28518. PMLR, 2023

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Gelsight: High-resolution robot tactile sensors for esti- mating geometry and force

Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gelsight: High-resolution robot tactile sensors for esti- mating geometry and force. Sensors, 17(12):2762, 2017

2017

[2] [2]

Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation

Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dave Stroud, Raymond Santos, Ahmad Byagowi, Gregg Kammerer, et al. Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation. IEEE Robotics and Automation Letters , 5(3):3838–3845, 2020

work page 2020

[3] [3]

Octopi: Object property reasoning with large tactile-language models, 2024

Samson Yu, Kelvin Lin, Anxing Xiao, Jiafei Duan, and Harold Soh. Octopi: Object property reasoning with large tactile-language models, 2024

work page 2024

[4] [4]

A touch, vision, and language dataset for multimodal alignment

Letian Fu, Gaurav Datta, Huang Huang, William Chung- Ho Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, and Ken Goldberg. A touch, vision, and language dataset for multimodal alignment. In International Conference on Machine Learning , 2024

work page 2024

[5] [5]

Binding touch to everything: Learning unified multimodal tactile representations

Fengyu Yang, Chao Feng, Ziyang Chen, Hyoungseob Park, Daniel Wang, Yiming Dou, Ziyao Zeng, Xien Chen, Rit Gangopadhyay, Andrew Owens, et al. Binding touch to everything: Learning unified multimodal tactile representations. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 26340–26353, 2024

work page 2024

[6] [6]

Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhi- hao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 , 2024

arXiv 2024

[7] [7]

Retrieval-augmented generation for large language mod- els: A survey

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jin- liu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augmented generation for large language mod- els: A survey. arXiv preprint arXiv:2312.10997 , 2023

arXiv 2023

[8] [8]

Universal manipulation interface: In- the-wild robot teaching without in-the-wild robots

Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In- the-wild robot teaching without in-the-wild robots. In Proceedings of Robotics: Science and Systems (RSS) , 2024

2024