HapticCap: A Multimodal Dataset and Task for Understanding User Experience of Vibration Haptic Signals

Daniel Hershcovich; Guimin Hu; Hasti Seifi

REVIEW 3 major objections 5 minor 57 references

HapticCap: A Multimodal Dataset and Task for Understanding User Experience of Vibration Haptic Signals

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HapticCap introduces a 92,070-pair human-annotated dataset and a haptic-caption retrieval task that links vibration signals to user descriptions across sensory, emotional, and associative categories.

desk verdict A valuable new dataset and a plausible task, but the headline retrieval numbers are compromised by a likely signal-level train/test leak and a T5-based curation that favors the reported best model. read the letter →

arxiv 2507.13318 v1 pith:JTAXQ2KR submitted 2025-07-17 cs.CL

Guimin Hu , Daniel Hershcovich , Hasti Seifi This is my paper

classification cs.CL

keywords hapticcaptionsvibrationsignalshaptic-captionretrievalsupervisedcontrastivelearningmultimodaldatasetuserexperiencevibrotactilefeedbacksensorylanguage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces HapticCap, which it presents as the first fully human-annotated haptic-caption dataset for vibrations, with 92,070 haptic-text pairs covering sensory, emotional, and associative descriptions from 235 users. The authors propose a haptic-caption retrieval task: given a vibration signal, a model should retrieve the textual descriptions users would write for it. They build a supervised contrastive learning framework that aligns vibration representations with text representations and report a working baseline, with the T5 text encoder paired with the AST audio encoder performing best, especially when trained separately for each description category. The paper's claim is that this dataset and task make it possible to study how users verbally experience vibration signals and to build models that connect haptic signals to language.

What carries the argument

The load-bearing mechanism is supervised contrastive learning over haptic-text pairs: each pair receives a label that concatenates the haptic signal ID and the description category, so all captions for the same signal in the same category are positive examples. The haptic encoder is a pretrained audio model fine-tuned on vibrations, and the text encoder is a language model with its last layers fine-tuned; two linear projections map both into a shared space, and the contrastive loss pulls matching pairs together while pushing others apart. The AST spectrogram transformer works best as the haptic encoder, and T5 works best as the text encoder.

What would settle it

Check the released HapticCap splits for vibration IDs shared between the training and test sets; if any exist, recompute P@10, R@10, mAP@10, and nDCG@10 using a split that assigns all captions of a vibration to the same partition and compare.

Watch

Extended reading notes

Core claim

HapticCap contains 2,736 unique vibration signals, each described by at least 10 users from sensory, emotional, and associative perspectives, yielding 92,070 pairs after removing empty responses. The central discovery is that a supervised contrastive framework using a pretrained language model and a pretrained audio model can align these modalities well enough to retrieve the correct descriptions in the top 10, with a best P@10 of 16.66 when trained on all categories and higher scores when trained per category; filtering low-agreement captions improves all metrics by roughly 20%. Emotion captions are retrieved most accurately, associative captions least, matching inter-annotator agreement. The paper also finds that pretrained audio encoders transfer to vibrations better than a vibration-only encoder trained from scratch.

Load-bearing premise

The evaluation assumes that splitting haptic-text pairs into train, validation, and test sets keeps every vibration signal out of two splits at once, but the paper does not state that signals are disjoint; if the same vibration appears with different captions in both training and test, the retrieval scores are inflated.

Editorial extensions

If this is right

HapticCap gives a benchmark for haptic-caption retrieval, so future work can compare models on a fixed signal-to-description matching task.
Because category-specific training outperforms combined training, downstream haptic search systems should train separate matchers for sensory, emotional, and associative queries.
The low-agreement filtering step shows that removing captions that diverge from majority perception improves retrieval, making agreement filtering a reusable quality-control step for haptic caption data.
The success of pretrained audio encoders suggests audio models are a strong starting point for vibration representation, which could apply to other haptic tasks beyond retrieval.
The zero-shot results show sensory-trained models transfer to other categories better than emotion- or association-trained models, indicating a hierarchy among description types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not state whether the same vibration signal appears in both training and test splits when haptic-text pairs are partitioned; if signals overlap across splits, the reported retrieval numbers overstate true generalization, and a vibration-level split would be the honest check.
Because each original signal was augmented into eight variants, retrieval may be easier for near-duplicates; evaluating on the original 304 signals alone would test whether the model captures genuinely new sensations.
The dataset naturally supports haptic caption generation, mapping vibrations to new text, which the authors list as future work and which would let designers ask for a vibration by describing it.
The associative category being hardest suggests that metaphor and real-world reference need different supervision, such as external knowledge of objects and scenes, rather than more contrastive pairs alone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The dataset is the real contribution here, and it is a good one. HapticCap is the first large, fully human-annotated vibration-caption corpus: 92,070 pairs over 2,736 signals, with sensory, emotional, and associative descriptions from 235 users. The collection procedure is careful, the signal augmentation is validated by human raters (98.78% agreement), and the paper is refreshingly transparent about its limitations, including the maturity of the contrastive learning framework. That part deserves credit.

The evaluation, however, has problems that need to be fixed before the numbers mean much. The split is described as partitioning the haptic-text pair set, with no statement that signals are disjoint between train and test. With roughly 34 pairs per signal, a random pair-level split puts essentially every vibration in both training and testing. Since the contrastive label is haptic ID plus category, the model can succeed at retrieval by recognizing exact waveforms it already saw, rather than by generalizing to unseen signals. The paper's own motivation for generating eight variants per signal—to improve generalization—implies the test should contain new signals. The authors need to confirm the split is signal-disjoint; if it is not, the reported P@10/R@10 and the T5+AST ranking are inflated.

There is also a circularity issue with the agreement filter. Section 3.3 uses T5 embeddings to remove descriptions that disagree with the majority, and then the paper reports T5+AST as the best model on this filtered data. Notably, on the full unfiltered set, Llama+AST actually edges out T5+AST on the combined metrics. So the headline claim may be an artifact of curating the evaluation set with the same model family being evaluated. That is not fatal if the full-data results are reported as primary, but it does mean the current framing overstates the T5 advantage.

One more small thing: the "correct retrieval" definition in Section 4.1 is circular as written. It should say a description is correct if it belongs to the same haptic signal and category.

These are serious but fixable issues. The dataset contribution is independent of the benchmark numbers and should survive. For peer review, I would send it out, but with a strong request that the authors resolve the split granularity, release the data and code, and either use a model-agnostic agreement measure or report full-dataset results as the primary comparison.

Referee Report

3 major / 5 minor

Summary. This paper introduces HapticCap, a dataset of 92,070 haptic-text pairs over 2,736 vibration signals collected from 235 users, with human-written descriptions in sensory, emotional, and associative categories. It proposes a haptic-caption retrieval task and evaluates a supervised contrastive learning framework that combines language model text encoders with audio-model haptic encoders. Experiments compare BERT, T5, Mistral, and Llama against AST, Wav2Vec, and EnCodec, reporting that T5+AST performs best on the filtered medium/high-agreement subset, especially when models are trained per category. The paper also includes zero-shot cross-category experiments and dataset analyses.

Significance. If the evaluation is sound, HapticCap would be a valuable resource for haptics and multimodal language research: the 11-month human annotation effort, the three-category design, and the scale of the dataset are clear strengths. The paper also provides a concrete baseline framework and zero-shot generalization experiments, which are useful for future work. However, the central quantitative claims currently depend on unresolved evaluation issues: the train/test split granularity is ambiguous, the correctness definition in the task formalization is circular, and the agreement filter is built with the same model family that is later reported as best. These issues must be resolved before the reported rankings can be trusted.

major comments (3)

[4.3, Eq. (2)] The paper states 'We partition haptic-text pair set C into mutually exclusive train, validation, and test sets' and then evaluates with candidates d in Ctest, but it never states whether the 2,736 haptic signals are disjoint across splits. Since there are roughly 34 pairs per signal, a random pair-level 70/10/20 split puts essentially every vibration signal in both training and test. The contrastive label is haptic ID plus category, so training teaches the haptic encoder to map each exact waveform to its caption cluster; at test time, retrieval for that same waveform can succeed by waveform identity rather than by generalization to unseen vibrations. This inflates all reported metrics and can distort the model ranking, including the headline T5+AST result. Please re-run the experiments with signal-disjoint splits or report both pair-level and signal-level results explicitly.
[4.1] The definition of a correct prediction is circular as written: 'Prediction is counted as correct if the retrieved description is in the set of the top K most similar objects to a given haptic signal in semantics.' This defines correctness by membership in the retrieved set rather than by ground-truth haptic-text pairs. Metrics such as P@10 and R@10 require a labeled set of relevant descriptions for each query signal. Please replace this with a standard formalization, e.g., a retrieved description is relevant if and only if it is paired with the query haptic signal in the held-out data.
[3.3, Tables 2-3] The low-agreement filter is computed using T5 embeddings: descriptions are removed when their average cosine similarity to other participants' descriptions for the same signal and category is below 0.5. The paper then reports T5+AST as the best model on this filtered data and claims filtering improves performance by about 20%. Because the curation measure comes from the same model family that is later evaluated, the comparison on the filtered subset is confounded: T5-similar descriptions may be systematically easier for T5 to align. Notably, on the full unfiltered dataset in Table 2, Llama+AST achieves higher full-data P@10, R@10, mAP@10, and nDCG@10 than T5+AST (12.37 vs. 12.25; 13.67 vs. 13.19; 24.32 vs. 24.03; 0.4731 vs. 0.4610), so the reported T5 advantage is currently only visible on the T5-filtered subset. Please test robustness by re-running the filter with a different encoder (e.g., BERT) and by reporting unfiltered results as primary.

minor comments (5)

[Tables 2-3] Several table entries are incomplete or hard to read: Table 3 has missing values for WavVec/Emotion R@10 and WavVec/Association R@10, one Llama/WavVec/Associative R@10 entry reads '20.01/24.67' and looks inconsistent with neighboring columns, and the run-together numbers in Table 3 need proofreading.
[4.3] The symbol d is used both for the embedding dimension and for retrieved descriptions in Eq. (2), making the formalization difficult to parse; please use distinct notation.
[5.2 and Table 2 caption] The caption says 'Significance differences of p < 0.05 using t-test are indicated by *,' but no details are given on the number of runs, whether the test is paired, or how significance is computed across random seeds; please clarify.
[References and text] The reference list contains duplicate Israr et al. 2014a/2014b entries, and there are small typos such as 'softmax is sof tmaxfunction' and 'Lim et al. (2025) shows'; a careful proofreading pass is needed.
[Data Availability] The paper says the dataset will be released under CC BY-NC 4.0 but provides no release link or code repository; a Data Availability statement would help reviewers verify the split and reproducibility claims.

Circularity Check

2 steps flagged · score 6.0 of 10

Pair-level splits place the same vibration signals in training and test, and the T5-based agreement filter curates the benchmark with the evaluated text encoder, so the headline T5+AST retrieval numbers are partly circular.

fitted input called prediction [Section 4.3 (Eq. 2), Section 5.3 (data split), Section 3.1 (generalization motivation)]
"Suppose we partition haptic-text pair set C into mutually exclusive train, validation, and test sets, C = Ctrain ∪ Cvalid ∪ Ctest. ... For h ∈ Ctest: d = top-K_{d∈Ctest} sim(text(d), haptic(h)). ... The data is divided into 70% training, 10% validation, and 20% test subsets for the full dataset and the medium/high agreement subset."

HapticCap has 2,736 signals and 92,070 haptic-text pairs, about 34 pairs per signal; a 70/10/20 split at the pair level therefore places essentially every vibration waveform in both training and test. The contrastive label is the haptic ID plus category, so the haptic encoder is trained to map each exact waveform to its caption cluster. At test, Eq. (2) retrieves captions for that same waveform; the top-K scores can be driven by waveform identity and memorization, not by generalization to unseen vibrations. The paper states the augmented signals exist to improve the generalization of the haptic caption retrieval model, which requires signal-disjoint splits. The reported test predictions are consequently fitted on the very haptic inputs being queried.
self definitional [Section 3.3 (T5 agreement filter), Section 6.1 (filtered improvement), Section 5.3 (T5 text encoder)]
"We first encode the description into a representation vector by T5 (Raffel et al., 2020) and then calculate cosine similarity for every pair of descriptions from different participants on the same category and the same haptic signal. Finally, we filter out descriptions with an average similarity score of less than 0.5 to other participants. ... First, the model after filtering out low-agreement data performs better, improved by average around 20% on all metrics, indicating filtering is effective and helpful to performance."

The medium/high-agreement subset that carries the headline results is defined by T5: captions are kept only if their T5 cosine similarity to other captions for the same signal exceeds 0.5. The retrieval model's text encoder is T5-base, fine-tuned in the framework, and the emphasized result is T5+AST on this filtered subset, with filtering improving scores by about 20%. Thus the evaluation set is partly curated with the same model family that is then declared best; the filter preferentially retains captions that T5 can cluster, giving T5 a structural advantage over BERT, Llama, and Mistral on the filtered rows. The full-data rows are less affected, but the headline filtered comparisons are not neutral among text encoders.

full rationale

HapticCap itself is an independent, human-annotated dataset, and the full-data results are reported, so this is not a wholesale circular paper. The self-citations (Hu et al. 2024; Lim et al. 2025) are not load-bearing in the derivation chain: Hu et al. is a related-work comparison, and Lim et al.'s augmentation claim is supported by an external three-researcher validation in Section 3.3. The circularity is in the evaluation protocol. A 70/10/20 split of the pair set, with about 34 pairs per vibration, makes the same haptic signals appear in both training and test; because the contrastive label is the haptic ID plus category, the model can memorize each waveform and retrieve its held-out captions by identity, so Eq. (2) measures memorization rather than generalization to new vibrations. Separately, the medium/high-agreement subset used for the headline numbers is defined by T5 embedding similarity, and T5+AST is then reported as best on exactly that subset; the roughly 20% filtered improvement and the model ranking on filtered rows are partly an artifact of constructing the evaluation set with the evaluated text encoder. These two construction-level reductions affect the central quantitative claim, while the dataset and full-data comparisons retain independent content, giving a score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central evaluation is shaped by several hand-selected numbers: the 0.5 agreement threshold, the 8x augmentation factor, the 10-user annotation convention, and the hyperparameter grid results. These choices are not derived from theory and can affect the reported rankings. The principal domain assumptions concern T5-similarity as a proxy for caption agreement, audio-encoder transfer to haptics, and hardware consistency, all of which are plausible but not independently established. No new physical entities are introduced.

free parameters (4)

Agreement filtering threshold = cosine similarity = 0.5
Descriptions with average T5-embedding cosine similarity below 0.5 are filtered as low-agreement (Section 3.3). This threshold is chosen post hoc and affects the size and composition of the evaluation set, and the T5-based similarity measure may bias the evaluation against other text encoders.
Signal generation augmentation factor = 8 new signals per original signal
Each of 304 base signals is stretched, amplified, noise-injected, or combined to create 8 variants (Section 3.1). The factor 8 is an arbitrary choice that determines the dataset size and the proportion of near-duplicate signals.
Number of annotators per signal = 10 users (more for the first two signal sets)
Each vibration is described by at least 10 users (Section 3.2). The choice of 10 was made after initial analysis of variation in the first two groups and determines the number of relevant captions per query and the ceiling of retrieval metrics.
Model hyperparameters = alpha=1e-3, tau=0.1, n=3, m=2, K=10, kappa=100
Selected by grid search on a validation set (Section 5.3). These are standard model hyperparameters, but K and kappa directly shape the retrieval metric and similarity scaling.

assumptions (5)

domain assumption T5-embedding cosine similarity approximates human agreement in captions
Section 3.3 filters descriptions with average similarity below 0.5 as low-agreement, assuming embeddings capture perceptual agreement. This injects T5-specific geometry into the evaluation set.
domain assumption Pretrained audio models represent vibration signals sufficiently for haptic-text alignment
Section 4.2 uses AST, Wav2Vec, and EnCodec as haptic encoders, relying on the claim that audio and vibration share temporal and spectral features. Prior work supports overlap, but the perceptual equivalence is not complete.
domain assumption Meta Quest 3 and Pro controllers reproduce the intended vibration signals consistently
Data collection in Section 3.2 assumes the VR hardware renders signals faithfully and that wearing earplugs and headphones removes audio confounds across users and sessions.
domain assumption Generated signals preserve the haptic experience of the original source signals
Section 3.1 adds 8 stretching, amplifying, noise-injected, or combined variants per signal based on Lim et al. (2025), with human validation on only 5% of source signals. The 98.78% pass rate is from a small sample and a binary similar/different question.
standard math The supervised contrastive loss in Equation 1 is well-formed and appropriate
Equation 1 follows the standard supervised contrastive loss (Khosla et al., 2020) and is an established loss function for alignment tasks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HapticCap: A Multimodal Dataset and Task for Understanding User Experience of Vibration Haptic Signals." pith.science (2026). https://pith.science/paper/JTAXQ2KR

@misc{pith2026250713318,
  author       = {Pith},
  title        = {Pith review of: HapticCap: A Multimodal Dataset and Task for Understanding User Experience of Vibration Haptic Signals},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JTAXQ2KR}},
  note         = {Machine review of arXiv:2507.13318}
}

read the original abstract

Haptic signals, from smartphone vibrations to virtual reality touch feedback, can effectively convey information and enhance realism, but designing signals that resonate meaningfully with users is challenging. To facilitate this, we introduce a multimodal dataset and task, of matching user descriptions to vibration haptic signals, and highlight two primary challenges: (1) lack of large haptic vibration datasets annotated with textual descriptions as collecting haptic descriptions is time-consuming, and (2) limited capability of existing tasks and models to describe vibration signals in text. To advance this area, we create HapticCap, the first fully human-annotated haptic-captioned dataset, containing 92,070 haptic-text pairs for user descriptions of sensory, emotional, and associative attributes of vibrations. Based on HapticCap, we propose the haptic-caption retrieval task and present the results of this task from a supervised contrastive learning framework that brings together text representations within specific categories and vibrations. Overall, the combination of language model T5 and audio model AST yields the best performance in the haptic-caption retrieval task, especially when separately trained for each description category.

Figures

Figures reproduced from arXiv: 2507.13318 by the authors.

**Figure 2.** Illustration of haptic signal statistics: (a) dis [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** t-SNE visualization of haptic signal features [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Diversity of signal-caption pairs in Haptic [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Overview of supervised contrastive learning for haptic signals and user descriptions. The proposed [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Haptic signal statistics : (a) signal count on [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

**Figure 9.** Figure 9: Distribution of (a) gender and (b) native lan [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

**Figure 7.** Figure 7: Visualization of word distribution across sen [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 10.** Figure 10: Illustration of the data collection interface. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 40 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html wav2vec 2.0: A framework for self-supervised learning of speech representations . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing ...

work page 2020
[4]

Jagan K Balasubramanian, Bence L Kodak, and Yasemin Vardar. 2024. Sens3: Multisensory database of finger-surface interactions and corresponding sensations. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pages 262--277. Springer

work page 2024
[5]

Corentin Bernard, Jocelyn Monnoyer, Micha \"e l Wiertlewski, and S lvi Ystad. 2022. Rhythm perception is shared between audio and haptics. Scientific Reports, 12(1):4188

work page 2022
[6]

Ning Cheng, Changhao Guan, Jing Gao, Weihao Wang, You Li, Fandong Meng, Jie Zhou, Bin Fang, Jinan Xu, and Wenjuan Han. 2024 a . Touch100k: A large-scale touch-language-vision dataset for touch-centric multimodal representation. arXiv preprint arXiv:2406.03813

arXiv 2024
[7]

Ning Cheng, You Li, Jing Gao, Bin Fang, Jinan Xu, and Wenjuan Han. 2024 b . Towards comprehensive multimodal perception: Introducing the touch-language-vision dataset. arXiv preprint arXiv:2403.09813

arXiv 2024
[8]

Miller, Alex Olwal, and Sean Follmer

Inrak Choi, Heather Culbertson, Mark R. Miller, Alex Olwal, and Sean Follmer. 2017. https://doi.org/10.1145/3126594.3126599 Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality . In Proceedings of the ACM Symposium on User Interface Software and Technology ( UIST ) , UIST '17, pages 119--130, New York, NY, USA. Associ...

work page arXiv 2017

Show all 57 references

[9]

Gonzalez, and Sean Follmer

Inrak Choi, Yiwei Zhao, Eric J. Gonzalez, and Sean Follmer. 2021. https://doi.org/10.1109/TVCG.2020.3002245 Augmenting Perceived Softness of Haptic Proxy Objects Through Transient Vibration and Visuo - Haptic Illusion in Virtual Reality . IEEE Transactions on Visualization and...

2021
[10]

Tor-Salve Dalsgaard, Joanna Bergström, Marianna Obrist, and Kasper Hornbæk. 2022. A user-derived mapping for mid-air haptic experiences. International Journal of Human-Computer Studies, 168:102920

2022
[11]

Alexandre D \' e fossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2023. https://openreview.net/forum?id=ivCd8z8zR2 High fidelity neural audio compression . Trans. Mach. Learn. Res., 2023

2023
[12]

u ng \"o r, Antonio Kr \

Donald Degraen, Bruno Fruchard, Frederik Smolders, Emmanouil Potetsianakis, Seref G \"u ng \"o r, Antonio Kr \"u ger, and J \"u rgen Steimle. 2021. Weirding haptics: In-situ prototyping of vibrotactile feedback in virtual reality through vocalization. In The 34th Annual ACM sy...

2021
[13]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Associat...

2019 doi
[14]

Enes Selman Ege, Furkan Cetin, and Cagatay Basdogan. 2011. Vibrotactile feedback in steering wheel reduces navigation errors during gps-guided car driving. In IEEE World Haptics Conference (WHC), pages 345--348. IEEE

2011
[15]

Letian Fu, Gaurav Datta, Huang Huang, William Chung-Ho Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, and Ken Goldberg. 2024. A touch, vision, and language dataset for multimodal alignment. arXiv preprint arXiv:2402.13232

2024 arXiv
[16]

Zihao Fu, Wai Lam, Qian Yu, Anthony Man-Cho So, Shengding Hu, Zhiyuan Liu, and Nigel Collier. 2023. Decoder-only or encoder-decoder? interpreting language model as a regularized encoder-decoder. arXiv preprint arXiv:2304.04052

2023 arXiv
[17]

Gonzalo Garc \' a-Valle, Manuel Ferre, Jose Bre \ n osa, and David Vargas. 2017. Evaluation of presence in virtual environments: haptic vest and user’s haptic skills. IEEE Access, 6:7224--7233

2017
[18]

Yuan Gong, Yu - An Chung, and James R. Glass. 2021. https://doi.org/10.21437/INTERSPEECH.2021-698 AST: audio spectrogram transformer . In 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021, Brno, Czechia, August 30 - September 3, 202...

2021 doi
[19]

Guimin Hu, Zirui Zhao, Lukas Heilmann, Yasemin Vardar, and Hasti Seifi. 2024. Grounding emotional descriptions to electrovibration haptic signals. arXiv preprint arXiv:2411.02118

2024 arXiv
[20]

Vladimir Iashin and Esa Rahtu. 2020. Multi-modal dense video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 958--959

2020
[21]

Ali Israr, Siyan Zhao, Kaitlyn Schwalje, Roberta Klatzky, and Jill Lehman. 2014 a . Feel Effects : Enriching Storytelling With Haptic Feedback . ACM Transactions on Applied Perception (TAP), 11(3):1--17. Publisher: ACM New York, NY, USA

2014
[22]

Ali Israr, Siyan Zhao, Kaitlyn Schwalje, Roberta Klatzky, and Jill Lehman. 2014 b . Feel effects: enriching storytelling with haptic feedback. ACM Transactions on Applied Perception (TAP), 11(3):1--17

2014
[23]

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L \' e lio Renard Lavaud, Marie - Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, ...

2023 doi arXiv
[24]

Kyungeun Jung, Sangpil Kim, Seungjae Oh, and Sang Ho Yoon. 2024. Hapmotion: motion-to-tactile framework with wearable haptic devices for immersive vr performance experience. Virtual Reality, 28(1):13

2024
[25]

Oliver Beren Kaul and Michael Rohs. 2017. Haptichead: A spherical vibrotactile grid around the head for 3d guidance in virtual and augmented reality. In Proceedings of the ACM CHI Conference on Human Factors in Computing Systems, pages 3729--3740

2017
[26]

Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Advances in neural information processing systems, 33:18661--18673

2020
[27]

Jarrod Knibbe, Adrian Alsmith, and Kasper Hornbæk. 2018. https://doi.org/10.1145/3264928 Experiencing Electrical Muscle Stimulation . Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(3):1--14. Publisher: ACM New York, NY, USA

2018 doi
[28]

Kuchenbecker, Robert C

Katherine J. Kuchenbecker, Robert C. Parajon, and Margrit P. Maggio. 2017. https://doi.org/10.1097/SIH.0000000000000201 Evaluation of a Vibrotactile Simulator for Dental Caries Detection . Simulation in Healthcare, 12(3):148

2017 doi
[29]

Chungman Lim, Gyeongdeok Kim, Su-Yeon Kang, Hasti Seifi, and Gunhyuk Park. 2025. Can a machine feel vibrations?: Predicting roughness and emotional responses to vibration tactons via a neural network. IEEE Transactions on Haptics

2025
[30]

Xubo Liu, Qiushi Huang, Xinhao Mei, Haohe Liu, Qiuqiang Kong, Jianyuan Sun, Shengchen Li, Tom Ko, Yu Zhang, Lilian H Tang, et al. 2022. Visually-aware audio captioning with adaptive audio-visual attention. arXiv preprint arXiv:2210.16428

2022 arXiv
[31]

Karon E MacLean, Oliver S Schneider, and Hasti Seifi. 2017. Multisensory Haptic Interactions : Understanding the Sense and Designing for It . In The Handbook of Multimodal - Multisensor Interfaces : Foundations , User Modeling , and Common Modality Combinations , volume 1, pag...

2017
[32]

Kunkel, Magalie Lilavois, Alexei Wedmid, David I

William McMahan, Jamie Gewirtz, Dorsey Standish, Paul Martin, Jacquelyn A. Kunkel, Magalie Lilavois, Alexei Wedmid, David I. Lee, and Katherine J. Kuchenbecker. 2011. https://doi.org/10.1109/TOH.2011.31 Tool Contact Acceleration Feedback for Telerobotic Surgery . IEEE Transact...

2011 doi
[33]

Ron Mokady, Amir Hertz, and Amit H Bermano. 2021. Clipcap: Clip prefix for image captioning. arXiv preprint arXiv:2111.09734

2021 arXiv
[34]

Marianna Obrist, Sue Ann Seah, and Sriram Subramanian. 2013. Talking About Tactile Experiences . In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems , pages 1659--1668

2013
[35]

Gunhyuk Park, Seungmoon Choi, Kyunghun Hwang, Sunwook Kim, Jaecheon Sa, and Moonchae Joung. 2011. Tactile Effect Design and Evaluation for Virtual Buttons on a Mobile Device Touchscreen . In Proceedings of the International Conference on Human Computer Interaction with Mobile ...

2011
[36]

Bastian P \"a tzold, Andre Rochow, Michael Schreiber, Raphael Memmesheimer, Christian Lenz, Max Schwarz, and Sven Behnke. 2023. Audio-based roughness sensing and tactile feedback for haptic perception in telepresence. In 2023 IEEE International Conference on Systems, Man, and ...

2023
[37]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . J. Mach. Learn. Res., ...

2020
[38]

Williams, Cara M

Mike Salvato, Sophia R. Williams, Cara M. Nunez, Xin Zhu, Ali Israr, Frances Lau, Keith Klumb, Freddy Abnousi, Allison M. Okamura, and Heather Culbertson. 2022. https://doi.org/10.1109/TOH.2021.3129067 Data- Driven Sparse Skin Stimulation Can Convey Social Touch Information to...

2022
[39]

Oliver S Schneider and Karon E MacLean. 2016. Studying design process and example use with macaron, a web-based vibrotactile effect editor. In IEEE Haptics Symposium (Haptics), pages 52--58. IEEE

2016
[40]

Hasti Seifi, Matthew Chun, Colin Gallacher, Oliver Schneider, and Karon E MacLean. 2020. How do novice hapticians design? a case study in creating haptic learning environments. IEEE transactions on haptics, 13(4):791--805

2020
[41]

Hasti Seifi and Karon E MacLean. 2017. Exploiting haptic facets: Users' sensemaking schemas as a path to design and personalization of experience. International Journal of Human-Computer Studies, 107:38--61

2017
[42]

Hasti Seifi, Kailun Zhang, and Karon E MacLean. 2015. Vibviz: Organizing, visualizing and navigating vibration libraries. In 2015 IEEE World Haptics Conference (WHC), pages 254--259. IEEE

2015
[43]

Seim, Brandon Ritter, Thad E

Caitlyn E. Seim, Brandon Ritter, Thad E. Starner, Kara Flavin, Maarten G. Lansberg, and Allison M. Okamura. 2022. https://doi.org/10.1109/TNSRE.2022.3174808 Design of a Wearable Vibrotactile Stimulation Device for Individuals With Upper - Limb Hemiparesis and Spasticity . IEEE...

2022
[44]

Youjin Sung, Kevin John, Sang Ho Yoon, and Hasti Seifi. 2025. Hapticgen: Generative text-to-vibration model for streamlining haptic design. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1--24

2025
[45]

David Ternes and Karon E MacLean. 2008. Designing large sets of haptic icons with rhythm. In Haptics: Perception, Devices and Scenarios: 6th International Conference, EuroHaptics 2008 Madrid, Spain, June 10-13, 2008 Proceedings 6, pages 199--208. Springer

2008
[46]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie - Anne Lachaux, Timoth \' e e Lacroix, Baptiste Rozi \` e re, Naman Goyal, Eric Hambro, Faisal Azhar, Aur \' e lien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://doi.org/10.485...

2023 doi arXiv
[47]

Janet van der Linden, Rose Johnson, Jon Bird, Yvonne Rogers, and Erwin Schoonderwaldt. 2011. https://doi.org/10.1145/1978942.1979017 Buzzing to Play : Lessons Learned from an in The Wild Study of Real - Time Vibrotactile Feedback . In Proceedings of the SIGCHI Conference on Hu...

2011
[48]

Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3156--3164

2015
[49]

Leon Weber-Genzel, Siyao Peng, Marie-Catherine de Marneffe, and Barbara Plank. 2024. Varierr nli: Separating annotation error from human label variation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2...

2024
[50]

Wenhao Wu, Haipeng Luo, Bo Fang, Jingdong Wang, and Wanli Ouyang. 2023. Cap4video: What can auxiliary captions do for text-video retrieval? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10704--10713

2023
[51]

Fengyu Yang, Chenyang Ma, Jiacheng Zhang, Jing Zhu, Wenzhen Yuan, and Andrew Owens. 2022. Touch and go: Learning from human-collected vision and touch. arXiv preprint arXiv:2211.12498

2022 arXiv
[52]

Yongjae Yoo, Taekbeom Yoo, Jihyun Kong, and Seungmoon Choi. 2015. Emotional responses of tactile icons: Effects of amplitude, frequency, duration, and envelope. In 2015 IEEE World Haptics Conference (WHC), pages 235--240. IEEE

2015
[53]

Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. 2017. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17(12):2762

2017
[54]

Gyeore Yun, Minjae Mun, Jungeun Lee, Dong-Geun Kim, Hong Z Tan, and Seungmoon Choi. 2023 a . Generating Real - Time , Selective , and Multimodal Haptic Effects from Sound for Gaming Experience Enhancement . In Proceedings of the ACM SIGCHI Conference on Human Factors in Comput...

2023
[55]

Gyeore Yun, Minjae Mun, Jungeun Lee, Dong-Geun Kim, Hong Z Tan, and Seungmoon Choi. 2023 b . Generating real-time, selective, and multimodal haptic effects from sound for gaming experience enhancement. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Sys...

2023
[56]

Yiming Zhang, Hong Yu, Ruoyi Du, Zhanyu Ma, and Yuan Dong. 2022. Caption feature space regularization for audio captioning. arXiv preprint arXiv:2204.08409

2022 arXiv
[57]

Siyan Zhao, Zachary Schwemler, Adam Fritz, and Ali Israr. 2016. Stereo haptics: Designing haptic interactions using audio tools. In Proceedings of the TEI'16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, pages 778--781

2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html wav2vec 2.0: A framework for self-supervised learning of speech representations . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing ...

work page 2020

[4] [4]

Jagan K Balasubramanian, Bence L Kodak, and Yasemin Vardar. 2024. Sens3: Multisensory database of finger-surface interactions and corresponding sensations. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, pages 262--277. Springer

work page 2024

[5] [5]

Corentin Bernard, Jocelyn Monnoyer, Micha \"e l Wiertlewski, and S lvi Ystad. 2022. Rhythm perception is shared between audio and haptics. Scientific Reports, 12(1):4188

work page 2022

[6] [6]

Ning Cheng, Changhao Guan, Jing Gao, Weihao Wang, You Li, Fandong Meng, Jie Zhou, Bin Fang, Jinan Xu, and Wenjuan Han. 2024 a . Touch100k: A large-scale touch-language-vision dataset for touch-centric multimodal representation. arXiv preprint arXiv:2406.03813

arXiv 2024

[7] [7]

Ning Cheng, You Li, Jing Gao, Bin Fang, Jinan Xu, and Wenjuan Han. 2024 b . Towards comprehensive multimodal perception: Introducing the touch-language-vision dataset. arXiv preprint arXiv:2403.09813

arXiv 2024

[8] [8]

Miller, Alex Olwal, and Sean Follmer

Inrak Choi, Heather Culbertson, Mark R. Miller, Alex Olwal, and Sean Follmer. 2017. https://doi.org/10.1145/3126594.3126599 Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality . In Proceedings of the ACM Symposium on User Interface Software and Technology ( UIST ) , UIST '17, pages 119--130, New York, NY, USA. Associ...

work page arXiv 2017