Pith. sign in

REVIEW 4 major objections 6 minor 33 references

Audio-Visual Contact Classification for Tree Structures in Agriculture

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A robot arm can classify whether it is touching a leaf, twig, trunk, or nothing by fusing contact-microphone audio with camera images, and the classifier transfers from a hand-held probe to a robot without retraining.

desk verdict Useful engineering for agricultural contact sensing, with a real dataset and a plausible fusion story, but the headline F1 is inconsistent, the window was chosen on the test set, and the auto-labeling is unvalidated, so treat the numbers as provisional. read the letter →

arxiv 2505.12665 v1 pith:HB2Q2OKB submitted 2025-05-19 cs.RO

Ryan Spears , Moonyoung Lee , George Kantor , Oliver Kroemer This is my paper · ORCID
classification cs.RO
keywords contactclassificationmultimodallearningaudio-visualfusionmicrophonesvibrotactilesensingtreestructuresagriculturalroboticszero-shottransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a robot can tell which part of a tree it is touching—leaf, twig, trunk, or nothing—by combining the vibrations picked up by a contact microphone with a camera view of the contact point. That distinction matters for pruning and harvesting, where the robot must know whether it is safe to keep pushing through foliage or should stop at a rigid branch, and vision alone is often occluded. The proposed classifier fuses two pretrained audio encoders and one visual encoder and is trained entirely on hand-held probe recordings; tested on a robot-mounted probe, it reports a multiclass F1 of 0.82, above the audio-only 0.53 and image-only 0.35 baselines. The paper's central bet is that contact-induced vibration carries enough material-specific structure to survive the change in hardware, as long as the audio is noise-filtered and the encoders carry internet-scale pretraining.

What carries the argument

The load-bearing mechanism is a fusion of three frozen pretrained encoders: the Audio Spectrogram Transformer (AST), which reads the 1024×128 mel-spectrogram for fine-grained spectral and temporal patterns; CLAP (Contrastive Language–Audio Pretraining), which projects the same spectrogram into a 512-dimensional semantic audio space; and a ViT-B/16 vision transformer, whose CLS token supplies 768 dimensions of visual context. Their embeddings are concatenated and fed to a lightweight transformer encoder followed by an MLP classification head. Two auxiliary mechanisms make the transfer work: spectral gating removes static, motor, and generator noise from the contact audio, and data augmentation injects pitch shifts, added noise, and sampled motor noise into probe recordings to mimic the robot's operating conditions. The 0.8-second audio window is empirically chosen as the best accuracy-latency trade-off.

What would settle it

Run the segmentation rule of Eqs. 1–3 on the collected trials and compare its contact intervals against force-torque sensor measurements on the same recordings, plus human review of the verification videos; if agreement is poor, or if single-class trials frequently contain visible contact with a second material, the ground-truth labels are unreliable and the transfer result is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that contact-microphone audio, converted to mel-spectrograms, encodes material-specific vibration signatures that let a robot classify leaf, twig, trunk, and ambient contacts, and that this ability transfers zero-shot from a human-held probe to a robot-mounted probe. Audio is the primary contact detector: audio-only models reach binary contact-detection F1 above 0.90, while vision contributes texture and shape cues that separate visually similar classes such as leaf and twig. The fused model concatenates AST, CLAP, and ViT-B/16 embeddings through a lightweight transformer, keeps the encoders frozen, and trains only the fusion head on hand-held probe data; on robot-collected test recordings it reports a multiclass F1 of 0.82 in the confusion-matrix summary (0.74 in the modality-comparison table) and binary F1 above 0.90. The paper also finds that pretraining the encoders is decisive: the same fusion architecture trained from scratch reaches only 0.52, whereas the pretrained version reaches 0.74–0.82.

Load-bearing premise

The ground-truth labels come from an automatic audio-amplitude thresholding rule, and the paper trusts that rule to mark exactly when the probe is touching the intended material; if that segmentation mislabels contact intervals, or if a 'leaf' trial accidentally brushes a twig, every reported F1 number inherits the error.

Editorial extensions

If this is right

  • An orchard robot can classify each contact in about 14 ms per one-second sample, fast enough for a reactive policy that decides whether to keep pushing or retreat.
  • Training data can be collected safely with a hand-held probe by a human, and the resulting model works on a robot-mounted probe with no robot-specific retraining.
  • Fusing audio and vision beats either modality alone for fine-grained classes (multiclass F1 0.82 vs 0.53 audio-only and 0.35 image-only), so cameras remain useful even when audio is the primary contact sensor.
  • Internet-scale pretraining of the encoders is essential in this domain; the fused model drops from 0.74–0.82 to 0.52 when trained from scratch.
  • A 0.8 s window balances accuracy and latency; shorter windows (0.3–0.5 s) trade a few accuracy points for faster reactions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper stops at classification, but its four output classes map directly onto a motion policy—leaves are pushable, twigs are bendable with care, trunks are hard obstacles, and ambient is free space—so the natural next step is a reactive controller that consumes these labels.
  • Because the pipeline depends on vibration propagation through the structure rather than on tree-specific geometry, the same hand-held-probe training scheme could plausibly transfer to other deformable-contact settings such as dense shrubs, vines, or post-harvest handling.
  • The t-SNE evidence that robot audio is less separable than probe audio suggests the fusion gain will vary with noise level; a direct test would be to occlude the camera during robot deployment and measure how much of the F1 advantage audio alone retains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper addresses contact classification for robot manipulation in orchard environments, distinguishing leaf, twig, trunk, and ambient contact from a contact microphone and an RGB camera mounted on a hand-held probe or a robot end-effector. The proposed model freezes three pretrained encoders (AST, CLAP, and ViT), concatenates their embeddings, and trains a lightweight transformer plus MLP head on hand-held probe data. The central empirical claim is zero-shot transfer to robot-mounted sensor data, with a reported multiclass F1 of 0.82 in the abstract and Fig. 8, or 0.74 in Table I, and the qualitative conclusion that audio-visual fusion outperforms audio-only (F1 0.53) and image-only (F1 0.35) baselines. The paper also contributes an open-source multisensory dataset and a set of ablations on pretraining and audio window length.

Significance. If the experimental claims are reliable, the paper makes a useful empirical contribution: it demonstrates that a fusion of pretrained audio and visual features can transfer across embodiments for a challenging agricultural contact-classification task, and it releases the dataset and code. The experimental setup of training on hand-held probe data and testing on robot data is a meaningful domain-shift test, and the qualitative finding that fusion beats unimodal baselines is consistent across Tables I and II. However, the significance is limited by an unvalidated labeling pipeline that generates the ground-truth labels, by an inconsistency between the headline F1 values, and by the absence of error bars. These issues directly affect the confidence in the central zero-shot transfer claim.

major comments (4)
  1. [Sec. IV-A, Eqs. (1)-(3)] The ground-truth contact/non-contact labels are produced by thresholding the smoothed audio amplitude (Eqs. (1)-(3)), and the same automatic labels are used for both training and test. The paper states in Sec. III-A that the ATI force-torque sensor verifies this annotation, but Sec. IV-A reports only qualitative video review and no agreement statistics. If the amplitude threshold misses quiet leaf contacts or flags motor/generator noise as contact, every multiclass and binary F1 value inherits that error, including the headline 0.82/0.74. The authors should report quantitative label-validation results (e.g., precision/recall of the automatic segmentation against the force-torque signal or against human annotations) and per-class segmentation quality before the headline classification numbers can be interpreted.
  2. [Abstract, Fig. 8, Table I, Sec. VI.D] The headline multiclass F1 is internally inconsistent: the abstract reports 0.82, Fig. 8 (left) states an F1 of 0.82, but Table I reports F1=0.74 for the pretrained audio-image DualAudio model that is the paper's main method. Section VI.D identifies 0.8 s as the best window based on an accuracy of 0.82, and Sec. V states this window was chosen from the ablation in Fig. 10. The authors need to specify exactly which dataset each number is computed on and to reconcile the discrepancy; if the 0.82 comes from a test-set-tuned window, it is not an unbiased estimate of the model's expected performance.
  3. [Sec. V, Fig. 10] The audio window length of 0.8 s is selected from the ablation in Fig. 10, which appears to be evaluated on the same robot test set used for the final comparisons in Tables I and II. Selecting a hyperparameter on the test set and then reporting performance on that same test set introduces optimistic bias. The authors should either use a validation split for window selection or explicitly state that Fig. 10 reports a separate test-set result and adjust the final reported accuracy accordingly.
  4. [Tables I and II] All results are point estimates with no error bars, confidence intervals, or number of random seeds. Given that the evaluation set is approximately 1,200 samples and the models are trained both from scratch and with pretraining, the stability of the reported differences (e.g., 0.74 vs. 0.53 in Table I) is unknown. The authors should report mean and standard deviation over at least three independent training runs, and they should also report the class distribution of the robot test set to allow the reader to assess whether the F1 scores are driven by class imbalance.
minor comments (6)
  1. [Sec. VI.A] The text states that audio-only attained an F1 score of 0.91 for binary detection in Table II, but Table II contains no 0.91 entry; it lists 0.92 for AST-pretrained and 0.80 for DualAudio-pretrained. The sentence and table need to be reconciled.
  2. [Sec. IV.C and Sec. V] The dataset description says the training dataset comprises approximately 7,300 samples, but Sec. V says the model was trained on approximately 3,500 samples. Please clarify whether 3,500 is the post-split training size, a filtered subset, or a typo, since the 80:20 split of 7,300 would give roughly 5,840 training samples.
  3. [Throughout] There are several typographical errors, including 'Evalulation' in Sec. VI.C, 'passivelycapture' in the Introduction, 'infront' in Sec. III.B, and 'generalization ability to zero-shot transfer' in the Conclusion; these should be corrected in a revision.
  4. [Fig. 7 and Sec. VI.A] Figure 7 shows probe-data confusion matrices and describes them as 'near-perfect accuracy,' but no quantitative numbers or sample counts are given. Please state whether these are training, validation, or test splits of the probe data and report the corresponding F1 values.
  5. [Sec. VI.A] The random-guess baseline F1 of 0.28 is mentioned without a definition; please specify the assumed class prior used to compute this baseline, since random performance depends on the class distribution.
  6. [Sec. IV.A, Eq. (3)] The notation in Eq. (3) uses 'C(t)=c' but c is not defined; it should be made explicit that c denotes a constant binary value (0 or 1) over each segment.

Circularity Check

2 steps flagged · score 6.0 of 10

Contact labels are generated by thresholding the same audio signal the model is trained on, and the 0.8 s window is tuned on the robot test set, so the headline F1s are partly circular.

  1. self definitional [Section IV-A, Eqs. (1)-(3); Section VI-A / Table II and Fig. 8]
    "We compute a smoothed moving average of audio amplitudes and dynamically determine a thresholds per sample that distinguishes background noise from active contact events. ... C(t) = ( 1, E(t)>T contact, 0, E(t)≤T contact ... C(t)∈{0,1} is the binary contact label; E(t) is the smoothed signal value."

    The binary contact ground truth C(t) is defined by thresholding the audio amplitude E(t), with the threshold itself computed from percentiles of the same E(t) signal. Therefore the binary contact-detection result (Table II, F1=0.94) measures how well the audio model recovers the threshold rule that generated its labels, not an independently verified contact event. The same audio-derived intervals assign the semantic labels leaf/twig/trunk/ambient in each trial, so the multiclass labels inherit this audio-derived timing. The paper states that a force-torque sensor verifies the annotation, but no verification statistics are reported. The prediction is therefore partly equivalent, by construction, to the labeling rule.

  2. fitted input called prediction [Section V first paragraph; Section VI-D / Fig. 10]
    "We select a 0.8 s window because our time-length ablation study identified it as the optimal trade-off between capturing contact dynamics and keeping inference latency low as shown in Figure 10. ... All results reported below are on robot data. ... Figure 10 ... showing that accuracy increases to 0.82 at 0.8 seconds before tapering off."

    The 0.8 s audio window is chosen by an accuracy sweep on the robot evaluation data (the paper says all results are on robot data), and the same robot test set is then used for the reported F1 scores. The headline 0.82 is the maximum of an in-sample window-length sweep, so it is a selected optimum rather than an independent held-out prediction. Evaluating multiple window lengths on the test set and reporting the best value is a form of fitting the evaluation protocol to the test set.

full rationale

The core embodiment-transfer experiment (train on hand-held probe, test on robot) has a genuinely independent train/test split, and the qualitative claim that fusion beats audio-only and image-only baselines does not depend on the labeling rule alone. No load-bearing self-citation was found: reference [2] is used for spectral gating, not to justify the central classification claim, and the pretrained encoders are external, independent support. However, two factors make the reported numbers partially circular. First, the binary contact ground truth is defined by thresholding the audio amplitude (Eq. 2), so the binary F1=0.94 largely measures whether the model can reproduce the threshold that produced its labels; the claimed force-torque verification is never quantified. Second, the 0.8 s window is selected by an accuracy sweep on the robot test set, so the headline 0.82 is a selected maximum rather than an unbiased prediction. These issues mainly affect the absolute performance numbers and the binary detection claim, while the relative ordering of modalities retains independent content; hence a mid-range score rather than a fully circular derivation.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central generalization result rests on the validity of the automatically generated contact labels, the assumption that vibrotactile signals carry material-specific information, and the transferability of pretrained audio and visual features. No new physical entities are introduced, so invented_entities is empty.

free parameters (5)
  • alpha_offset
    Contact-threshold offset fraction in Eq. (1); hand-tuned, value not reported. Determines what counts as contact in the ground truth.
  • beta_factor
    Non-contact factor in Eq. (1); hand-tuned, value not reported. Sets the hysteresis threshold for no-contact labels.
  • delta_min = 1 s
    Minimum segment duration in Eq. (3); text states 1 s. Filters short contact/no-contact intervals.
  • gamma_squeeze
    Maximum merge gap in Eq. (3); hand-tuned, value not reported. Merges adjacent contact intervals.
  • audio_window_length = 0.8 s
    Window duration selected using the robot test-set ablation (Fig. 10); directly affects all reported results.
assumptions (6)
  • domain assumption Contact-induced vibrations carry material-specific signals that generalize across embodiments after preprocessing.
    Stated in Sec. I and III; the entire transfer result depends on this.
  • domain assumption Automatically segmented audio amplitude intervals correspond to true physical contact events.
    Sec. IV-A; labels are derived from audio thresholds, with force-torque verification mentioned but not reported.
  • domain assumption Each trial involves interaction with a single object class, making per-trial labeling valid.
    Sec. III-B; if contacts hit multiple classes, labels would be noisy.
  • domain assumption Pretrained encoders (AST, CLAP, ViT) on internet-scale data provide transferable features for contact audio.
    Sec. V; Q3 results support this empirically, but it is assumed as prior.
  • domain assumption Spectral gating with a recorded robot noise reference removes embodiment-specific noise without removing contact signal.
    Sec. IV-B; uses pre-recorded reference from both probe and robot, including test-embodiment noise.
  • standard math Standard cross-entropy training and transformer fusion are appropriate for this classification task.
    Eq. (4); standard supervised learning assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Audio-Visual Contact Classification for Tree Structures in Agriculture." pith.science (2026). https://pith.science/paper/HB2Q2OKB

@misc{pith2026250512665,
  author       = {Pith},
  title        = {Pith review of: Audio-Visual Contact Classification for Tree Structures in Agriculture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HB2Q2OKB}},
  note         = {Machine review of arXiv:2505.12665}
}
read the original abstract

Contact-rich manipulation tasks in agriculture, such as pruning and harvesting, require robots to physically interact with tree structures to maneuver through cluttered foliage. Identifying whether the robot is contacting rigid or soft materials is critical for the downstream manipulation policy to be safe, yet vision alone is often insufficient due to occlusion and limited viewpoints in this unstructured environment. To address this, we propose a multi-modal classification framework that fuses vibrotactile (audio) and visual inputs to identify the contact class: leaf, twig, trunk, or ambient. Our key insight is that contact-induced vibrations carry material-specific signals, making audio effective for detecting contact events and distinguishing material types, while visual features add complementary semantic cues that support more fine-grained classification. We collect training data using a hand-held sensor probe and demonstrate zero-shot generalization to a robot-mounted probe embodiment, achieving an F1 score of 0.82. These results underscore the potential of audio-visual learning for manipulation in unstructured, contact-rich environments.

Figures

Figures reproduced from arXiv: 2505.12665 by the authors.

Figure 1
Figure 1. Robotic arm in an apple orchard. To distinguish safe versus unsafe [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Multi-modal sensor suite in two different embodiments. (Left) [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Audio-image pair of four contact classes. In both sensing modalities, [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Examples of correct (green) and incorrect (red) predictions from [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Overall system diagram of our classification framework. The audio input signal is pre-processed into mel spectrograms. Each sensing modality is [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Probe-data confusion matrices for contact classification. The vision [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Confusion matrices for (left) four-class contact classification and [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: The t-SNE analysis of extracted audio features from our 1-second [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 18 canonical work pages

  1. [1]

    From machine learning to robotics: Challenges and opportunities for embodied intelligence,

    N. Roy, I. Posner, T. Barfoot, P. Beaudoin, Y . Bengio, J. Bohg, O. Brock, I. Depatie, D. Fox, D. Koditscheket al., “From machine learning to robotics: Challenges and opportunities for embodied intelligence,”arXiv preprint arXiv:2110.15245, 2021

    arXiv

  2. [2]

    Sonicboom: Contact localization using array of microphones,

    M. Lee, U. Yoo, J. Oh, J. Ichnowski, G. Kantor, and O. Kroemer, “Sonicboom: Contact localization using array of microphones,”arXiv preprint arXiv:2412.09878, 2024

    arXiv 2024

  3. [3]

    Soft magnetic skin for continuous deformation sensing,

    T. Hellebrekers, O. Kroemer, and C. Majidi, “Soft magnetic skin for continuous deformation sensing,”Advanced Intelligent Systems, vol. 1, no. 4, p. 1900025, 2019

    work page 2019

  4. [4]

    Gelsight: High-resolution robot tactile sensors for estimating geometry and force,

    W. Yuan, S. Dong, and E. H. Adelson, “Gelsight: High-resolution robot tactile sensors for estimating geometry and force,”Sensors, vol. 17, no. 12, p. 2762, 2017

    2017

  5. [5]

    Reskin:versatile, replaceable, lasting tactile skins,

    R. Bhirangi, T. Hellebrekers, C. Majidi, and A. Gupta, “Reskin:versatile, replaceable, lasting tactile skins,” inCoRL, 2021

    work page 2021

  6. [6]

    That sounds right: Auditory self-supervision for dynamic robot manipulation,

    A. Thankaraj and L. Pinto, “That sounds right: Auditory self-supervision for dynamic robot manipulation,” inConference on Robot Learning. PMLR, 2023, pp. 1036–1049

    work page 2023

  7. [7]

    Maniwav: Learning robot manipulation from in-the-wild audio-visual data,

    Z. Liu, C. Chi, E. Cousineau, N. Kuppuswamy, B. Burchfiel, and S. Song, “Maniwav: Learning robot manipulation from in-the-wild audio-visual data,” in8th Annual Conference on Robot Learning, 2024

    2024

  8. [8]

    Occlusion reasoning for skeleton extraction of self-occluded tree canopies,

    C. H. Kim and G. Kantor, “Occlusion reasoning for skeleton extraction of self-occluded tree canopies,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 9580–9586

    work page 2023

Show all 33 references
  1. [9]

    Task-oriented active learning of model preconditions for inaccurate dynamics models,

    A. LaGrassa, M. Lee, and O. Kroemer, “Task-oriented active learning of model preconditions for inaccurate dynamics models,”arXiv, 2024

    2024

  2. [10]

    Visual 3D Reconstruction and Dynamic Simulation of Fruit Trees for Robotic Manipulation,

    F. Yandun, A. Silwal, and G. Kantor, “Visual 3D Reconstruction and Dynamic Simulation of Fruit Trees for Robotic Manipulation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 54–55

    2020

  3. [11]

    3d reconstruction-based seed counting of sorghum panicles for agricultural inspection,

    H. Freeman, E. Schneider, C. H. Kim, M. Lee, and G. Kantor, “3d reconstruction-based seed counting of sorghum panicles for agricultural inspection,” in2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 9594–9600

    2023

  4. [12]

    Learning to simulate tree-branch dynamics for manipulation,

    J. Jacob, T. Bandyopadhyay, J. Williams, P. Borges, and F. Ramos, “Learning to simulate tree-branch dynamics for manipulation,”IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 1748–1755, 2024

    2024

  5. [13]

    Towards robotic tree manipulation: Leveraging graph representations,

    C. H. Kim, M. Lee, O. Kroemer, and G. Kantor, “Towards robotic tree manipulation: Leveraging graph representations,” 2023

    2023

  6. [14]

    Gentle manipulation of tree branches: A contact-aware policy learning approach,

    J. Jacob, S. Cai, P. V . K. Borges, T. Bandyopadhyay, and F. Ramos, “Gentle manipulation of tree branches: A contact-aware policy learning approach,” in8th Annual Conference on Robot Learning, 2024

    2024

  7. [15]

    Sonicsense: Object perception from in-hand acoustic vibration,

    J. Liu and B. Chen, “Sonicsense: Object perception from in-hand acoustic vibration,”arXiv preprint arXiv:2406.17932, 2024

    2024 arXiv

  8. [16]

    Visual-auditory extrinsic contact estima- tion,

    X. Yi, J. Lee, and N. Fazeli, “Visual-auditory extrinsic contact estima- tion,”arXiv preprint arXiv:2409.14608, 2024

    2024

  9. [17]

    Play it by ear: Learning skills amidst occlusion through audio-visual imitation learning,

    M. Du, O. Y . Lee, S. Nair, and C. Finn, “Play it by ear: Learning skills amidst occlusion through audio-visual imitation learning,”arXiv preprint arXiv:2205.14850, 2022

    2022 arXiv

  10. [18]

    Learning audio feedback for estimating amount and flow of granular material,

    S. Clarke, T. Rhodes, C. G. Atkeson, and O. Kroemer, “Learning audio feedback for estimating amount and flow of granular material,” Proceedings of Machine Learning Research, vol. 87, 2018

    2018

  11. [19]

    Swoosh! rattle! thump!–actions that sound,

    D. Gandhi, A. Gupta, and L. Pinto, “Swoosh! rattle! thump!–actions that sound,”arXiv preprint arXiv:2007.01851, 2020

    2007 arXiv

  12. [20]

    The boom- box: Visual reconstruction from acoustic vibrations,

    B. Chen, M. Chiquier, H. Lipson, and C. V ondrick, “The boom- box: Visual reconstruction from acoustic vibrations,”arXiv preprint arXiv:2105.08052, 2021

    2021 arXiv

  13. [21]

    Leveraging multimodal haptic sensory data for robust cutting,

    K. Zhang, M. Sharma, M. Veloso, and O. Kroemer, “Leveraging multimodal haptic sensory data for robust cutting,” inIEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 2019

    2019

  14. [22]

    Playing with food: Learning food item representations through interactive explo- ration,

    A. Sawhney, S. Lee, K. Zhang, M. Veloso, and O. Kroemer, “Playing with food: Learning food item representations through interactive explo- ration,” inInternational Symposium on Experimental Robotics, 2020, pp. 309–322

    2020

  15. [23]

    Passive and active acoustic sensing for soft pneumatic actuators,

    V . Wall, G. Z ¨oller, and O. Brock, “Passive and active acoustic sensing for soft pneumatic actuators,”The International Journal of Robotics Research, vol. 42, no. 3, pp. 108–122, 2023

    2023

  16. [24]

    Poe: Acoustic soft robotic proprioception for omnidirectional end-effectors,

    U. Yoo, Z. Lopez, J. Ichnowski, and J. Oh, “Poe: Acoustic soft robotic proprioception for omnidirectional end-effectors,”arXiv preprint arXiv:2401.09382, 2024

    2024 arXiv

  17. [25]

    Sonicfinger: Pre-touch and contact detection tactile sensor for reactive pregrasping,

    S. Rupavatharam, C. Escobedo, D. Lee, C. Prepscius, L. Jackel, R. Howard, and V . Isler, “Sonicfinger: Pre-touch and contact detection tactile sensor for reactive pregrasping,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 12 556–12 562

    2023

  18. [26]

    Binding touch to everything: Learning unified multimodal tactile representations,

    F. Yang, C. Feng, Z. Chen, H. Park, D. Wang, Y . Dou, Z. Zeng, X. Chen, R. Gangopadhyay, A. Owenset al., “Binding touch to everything: Learning unified multimodal tactile representations,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 202...

    2024

  19. [27]

    Hearing touch: Audio-visual pretraining for contact-rich manipulation,

    J. Mejia, V . Dean, T. Hellebrekers, and A. Gupta, “Hearing touch: Audio-visual pretraining for contact-rich manipulation,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6912–6919

    2024

  20. [28]

    A lab- customized autonomous humanoid apple harvesting robot,

    X. Yu, Z. Fan, X. Wang, H. Wan, P. Wang, X. Zeng, and F. Jia, “A lab- customized autonomous humanoid apple harvesting robot,”Computers & Electrical Engineering, vol. 96, p. 107459, 2021

    2021

  21. [29]

    Bumblebee: A path towards fully autonomous robotic vine pruning

    A. Silwal, F. Yandun, A. K. Nellithimaru, T. Bates, and G. Kantor, “Bumblebee: A path towards fully autonomous robotic vine pruning.” Field Robotics, vol. 2, no. 1, pp. 1661–1696, 2022

    2022

  22. [30]

    Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires,

    T. Sainburg, M. Thielk, and T. Q. Gentner, “Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires,” PLoS computational biology, vol. 16, no. 10, p. e1008228, 2020

    2020

  23. [31]

    A biomimetic elastomeric robot skin using electrical impedance and acoustic tomogra- phy for tactile sensing,

    K. Park, H. Yuk, M. Yang, J. Cho, H. Lee, and J. Kim, “A biomimetic elastomeric robot skin using electrical impedance and acoustic tomogra- phy for tactile sensing,”Science Robotics, vol. 7, no. 67, p. eabm7187, 2022

    2022

  24. [32]

    Ast: Audio spectrogram trans- former,

    Y . Gong, Y .-A. Chung, and J. Glass, “Ast: Audio spectrogram trans- former,”arXiv preprint arXiv:2104.01778, 2021

    2021 arXiv

  25. [33]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (IC...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.