Pith. sign in

REVIEW 3 major objections 3 minor 45 references

A new 34-hour tri-modal dataset collected with acoustic-sensing eyewear establishes the first open-vocabulary silent speech recognition benchmark, with a baseline CTC ResNet-34 reaching 26.3% word error rate on silently mouthed speech.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 00:10 UTC pith:Y7GZVEST

load-bearing objection A genuinely new open-vocabulary benchmark for acoustic-sensing silent speech, with one load-bearing assumption—silence—that the authors should verify before the WER is trusted. the 3 major comments →

arxiv 2608.00803 v1 pith:Y7GZVEST submitted 2026-08-01 cs.SD cs.HCcs.LG

SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces

Ruidong Zhang , Jiacheng Liu , Fran\c{c}ois Guimbreti\`ere , Cheng Zhang This is my paper
classification cs.SD cs.HCcs.LG
keywords silent speech interfaceacoustic sensingopen-vocabulary speech recognitionwearable computingFMCW echo profilestri-modal datasetspeech recognition dataseteyeglass form factor
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Silent speech interfaces that are socially unobtrusive have historically been stuck with small, closed vocabularies, while systems that handle open vocabularies require electrodes or chin-mounted probes. SoniSpeech is a 34-hour, 18,000-utterance dataset collected with acoustic-sensing eyewear that pairs inaudible ultrasound echo profiles of facial motion with voiced audio and video, recorded in both voiced and silently mouthed versions of the same sentences. The paper reports a baseline CTC-based ResNet-34 reaching 26.3% word error rate on open-vocabulary silent speech recognition, which it presents as the first benchmark showing that glasses-only silent speech recognition is computationally tractable. The dataset also exposes a large voiced-to-silent domain gap, and shows that combining voiced with silent training data improves silent recognition, suggesting both a challenge and a path forward.

Core claim

SoniSpeech is offered as the first large-scale, open-vocabulary, tri-modal dataset for wearable silent speech interfaces built around acoustic-sensing eyewear. The glasses emit inaudible FMCW chirps and record four differential echo profiles of facial deformation, synchronized with voiced audio and frontal video. Sentences are drawn from SODA, giving contemporary conversational English with 5,356 word types and complete phoneme coverage, recorded in paired voiced and silent modes by one speaker. The baseline—a ResNet-34 with CTC loss and a 1,000-unit SentencePiece tokenizer—reaches 26.3% word error rate on silent test speech when trained on voiced and silent data together, versus 33.7% silen

What carries the argument

The central object is the differential echo profile: time-range maps of reflected FMCW ultrasound, temporally differentiated to remove static reflections and cropped to 80 range bins, yielding 4-channel sequences at 200 Hz that carry the articulatory information. The accompanying construction is the paired voiced/silent corpus drawn from SODA, which makes the open-vocabulary benchmark possible. The baseline model couples a ResNet-34 encoder with 16× temporal downsampling and frequency-wise pooling to a CTC loss—connectionist temporal classification, a standard alignment-free sequence loss—and a 1,000-unit SentencePiece subword tokenizer that lets the decoder handle words unseen in training.

Load-bearing premise

The central benchmark depends on the paper's statement in Section 3.3 that silent-mode recordings were made by mouthing strictly silently without vocalization or whispering; if any voicing or whispering leaked into those sessions, the ultrasonic echo signal could carry voice cues that inflate the 26.3% word error rate.

What would settle it

Record the silent sessions with an independent high-sensitivity microphone or a contact laryngeal accelerometer, and check for voicing or whispering in the 0–18 kHz band of the aligned SoniSpeech recordings; any systematic voiced energy that follows the transcript would indicate the benchmark is inflated. A cheaper check is to measure spectral energy below the 18 kHz transmission band in the existing silent echo files and compare it with the voiced-mode files.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • An unobtrusive eyewear form factor can support open-vocabulary silent speech recognition: the 26.3% WER baseline, achieved without a language model, gives the community a concrete starting point.
  • More data helps: WER decreases steadily as training sessions are added and has not saturated at 140 sessions.
  • Voiced speech is a complementary, not substitute, training signal: adding voiced sessions to silent training cuts WER from 33.7% to 26.3%, while voiced-only training transfers poorly to silent evaluation (78.4% WER).
  • The dataset's three synchronized modalities enable cross-modal and self-supervised approaches such as contrastive learning, silent-to-speech synthesis, and multimodal speech enhancement.
  • By drawing on SODA, the corpus reflects contemporary conversational English—contractions, discourse markers, colloquialisms—closer to real deployment than 19th-century literary text.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the silent-mode recordings are genuinely free of vocalization, the 26.3% WER is a lower bound on what is achievable; adding a language model, self-supervised pretraining, or larger corpora should push the number substantially lower.
  • The voiced-silent gap suggests silent speech should be modeled as its own articulatory domain rather than as muffled voiced speech; domain-adaptation and contrastive objectives between paired voiced and silent samples are natural next tests.
  • Because the dataset is single-speaker and single-environment, the open-vocabulary claim is established for one person; multi-speaker generalization remains an open extension, though prior work on the same sensing hardware suggests robustness to movement and noise.
  • The tri-modal alignment makes silent-speech-as-privileged-information experiments possible: models could use voiced audio or video only at training time to improve silent-only inference.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. SoniSpeech proposes a large-scale, open-vocabulary, tri-modal dataset for wearable silent speech interfaces based on acoustic-sensing eyewear. The dataset contains 34 hours / 18,000 utterances of synchronized ultrasound echo profiles, audible audio, and frontal video from a single speaker, in both voiced and silent modes, with utterances drawn from the SODA dialogue corpus. The paper reports a CTC-based ResNet-34 baseline achieving 26.3% WER on silent-speech recognition, claiming this is the first open-vocabulary benchmark for unobtrusive acoustic-sensing SSI. The dataset is publicly released, and supplementary experiments examine data-scaling behavior and voiced-to-silent generalization.

Significance. If the reported WER and dataset validity hold, this is a valuable community resource: it is the first large-scale open-vocabulary corpus for a minimally-obtrusive SSI modality, and it includes paired voiced/silent data that enables study of the modality gap. The paper ships the dataset with a DOI, provides concrete hardware details, and describes a reproducible baseline with standard training pipelines (CTC, ResNet-34, SpecAugment). The data-scaling trend supports the motivation for larger datasets. The main contribution is empirical; the paper's value depends on the silent-mode recordings being genuinely silent and on the reliability of the single-run WER measurement.

major comments (3)
  1. [§3.3 (Data collection procedure)] The silent-mode recordings are described as 'mouthing strictly silently without vocalization or whispering,' but no quantitative verification is provided. If any subvocalization or whispered phonation occurred, mechanical vibration of the vocal folds could modulate the 18–39 kHz echo profiles even after band-pass filtering, inflating the 26.3% WER benchmark and undermining the central tractability claim. Since the audible-band audio is already captured, the paper should report energy, voice-activity detection, or spectral analysis on the silent-mode audio channel to demonstrate the absence of voicing. This is directly testable with the released data and is load-bearing for the main result.
  2. [§4.2 / §5.1 (Training details and Main results)] The baseline WER numbers (26.3% silent combined, etc.) are reported as single values with no variance, no random-seed variation, and no validation-based model-selection protocol. CTC training is described for 200 epochs, but there is no checkpoint-selection criterion or early-stopping rule. A single run can be unrepresentative for a benchmark claim. The authors should report mean±std over at least three independent training runs, or clarify the checkpoint selection and provide learning curves or validation-set based early stopping.
  3. [§5.2 (Data scaling)] Figure 2(d) shows a decreasing WER trend with more training sessions and the claim that the curve 'has not saturated' at 140 sessions. Without error bars or multiple runs at each scale point, the trend could be within run-to-run noise. Adding confidence intervals or significance tests over a few seeds at a subset of training sizes would make the scaling claim credible.
minor comments (3)
  1. [General] Several typos and formatting artifacts: 'V oiced' in Table 3 headings, 'uses uses fabricated' in §2.2, and inconsistent use of 'parallel' for the two modes. A copyedit pass is needed.
  2. [§5.1] The claim that silent speech has 'more deliberate mouth movements' is qualitative; the duration data (5.41 s vs. 5.26 s) partially supports this, but the discussion would benefit from a direct reference to the measured speaking-rate difference.
  3. [§1 / Table 1] The title 'first open-vocabulary baseline for acoustic-sensing wearable SSI' is clear, but the distinction between this and prior continuous-but-closed-vocabulary work (e.g., EchoSpeech) could be sharpened in the introduction to avoid ambiguity.

Circularity Check

0 steps flagged

No significant circularity: the 26.3% WER is an empirical measurement on held-out data, not derived from fitted parameters or self-citation chains.

full rationale

The paper's central claim--that open-vocabulary silent speech recognition is tractable with acoustic-sensing eyewear--rests on the reported 26.3% WER (Section 5.1). This is a direct empirical measurement on a held-out test set (1,000 test sentences disjoint from training, Section 3.2), produced by a standard ResNet-34+CTC baseline. There is no derivation chain in which a fitted parameter is renamed as a prediction, nor any self-referential definition that forces the result. The sensing hardware and Echo Profile feature extraction are adopted from prior work by the same group (e.g., [4]), but the baseline's performance does not assume the success of those prior systems; it is evaluated independently on newly collected data. The 'cross-modal mismatch' finding is an empirical observation explained post hoc by plausible articulatory and acoustic factors, not a conclusion that reduces to its inputs. The only notable concern--whether the 'silent' recordings truly contain no leaked voice cues (Section 3.3)--is a validity/assumption threat, not a circularity of the derivation. Self-citations appear throughout (e.g., EchoSpeech, HPSpeech, EarIO), but none are used as load-bearing justification for the central measured result; they provide context and method precedents. Therefore, the paper is not circular.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claim rests on the sensing modality faithfully capturing articulatory movements, the corpus being a valid open-vocabulary sample, and the speaker's silent mouthing being truly voiceless. These are domain assumptions grounded in prior work and stated limitations, not free parameters fitted to the result.

axioms (4)
  • domain assumption Echo profiles encode articulatory movements
    The FMCW echo processing from prior work is assumed to capture facial deformation; cited to [4].
  • domain assumption Single-speaker data suffices for a first benchmark
    Precedent from LJSpeech and Gaddy's corpus; acknowledged as a limitation.
  • domain assumption SODA corpus provides natural conversational English
    The filtering and normalization steps are trusted to produce a representative open-vocabulary corpus.
  • domain assumption Silent mouthing was performed without vocalization
    Data quality relies on the speaker's compliance; not independently verified.

pith-pipeline@v1.3.0-alltime-deepseek · 9801 in / 10001 out tokens · 106588 ms · 2026-08-05T00:10:44.834749+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces." pith.science (2026). https://pith.science/paper/Y7GZVEST

@misc{pith2026260800803,
  author       = {Pith},
  title        = {Pith review of: SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7GZVEST}},
  note         = {Machine review of arXiv:2608.00803}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Wearable silent speech interfaces (SSIs) are limited to small, closed vocabularies. Approaches achieving larger vocabularies require obtrusive hardware such as facial electrodes. We present SoniSpeech, the first large-scale, open-vocabulary, trimodal dataset for wearable SSI using acoustic-sensing eyewear. It contains 34 hours across 18,000 utterances with three synchronized modalities: ultrasound echo profiles, voiced audio, and frontal video, in both voiced and silent modes. The corpus draws from the SODA dialogue dataset, providing contemporary conversational English with 5,356 unique words and full phoneme coverage. A CTC-based ResNet-34 baseline achieves 26.3% word error rate (WER) on open-vocabulary silent speech recognition, the first benchmark for this task. Dataset is available at https://doi.org/10.7298/xjjr-9m85

Figures

Figures reproduced from arXiv: 2608.00803 by Cheng Zhang, Fran\c{c}ois Guimbreti\`ere, Jiacheng Liu, Ruidong Zhang.

Figure 1
Figure 1. Figure 1: Overview of SoniSpeech. (a) Hardware and signals. (a-1) Illustration of the hardware setup. (a-2) Configuration of trans￾mitted signals. (a-3) User wearing the SoniSpeech device. (a-4) Visualization of 4-channel differential Echo Profiles. (b) Illustration of dataset setup. SoniSpeech datasets includes 3 synchronized modalities: Echo Profiles as the representation of inaudible signals, audi￾ble voices and … view at source ↗
Figure 2
Figure 2. Figure 2: Statistics and results. (a) Distribution of ARPAbet phoneme frequency in the SoniSpeech corpus. (b) Distribution of utterance duration vs. number of words. (c) Number of OOV word types in the evaluation set as the number of training ses￾sions scales. (d) Performance curve as number of training ses￾sions scales [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 31 canonical work pages · 1 internal anchor

  1. [1]

    Introduction Silent speech interfaces (SSIs) offer transformative potential for accessible, private, and low-latency communication [2]. Yet, practical deployment is currently paralyzed by a fundamental compromise: systems are either capable of open-vocabulary recognition but physically obtrusive [3], or socially acceptable but confined to small, closed vo...

    work page

  2. [2]

    SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces

    Related work 2.1. Open-vocabulary wearable SSI The most successful open-vocabulary SSI efforts rely on high- fidelity but physically obtrusive sensors. Gaddy and Klein [8, 9, 3] created the seminal precedent: approximately 20 hours of 8-channel facial EMG from a single speaker sourced from Project Gutenberg texts. Ultrasound tongue imaging (UTI) of- fers ...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  3. [3]

    I’ll”, “don’t

    Dataset description 3.1. Sensing hardware The sensing platform uses an eyeglass frame equipped with two speakers (Ole Wolff OWR-05049T-38D) and two ultra- sound microphones (Syntiant SPH0641LU4H-1) mounted on the lower frame edges. The speakers emit inaudible FMCW chirps that travel across the face; facial deformations during speech alter the signal path,...

    work page 1920

  4. [4]

    Overview We build a baseline silent speech recognition system to facili- tate future research

    Baseline system 4.1. Overview We build a baseline silent speech recognition system to facili- tate future research. The baseline encoder is a ResNet-34 [32] adapted for sequence modeling. The input is a 4-channel differ- ential echo profile with 80 range bins along the frequency axis. The standard stem max-pool layer is removed to preserve tem- poral reso...

    work page

  5. [5]

    Main results Table 3 summarizes the performance for each configuration

    Experiments and results 5.1. Main results Table 3 summarizes the performance for each configuration. Silent-to-silent recognition.The silent-only model achieves 33.7% WER on open-vocabulary silent speech recog- nition. To our knowledge, this is the first time open-vocabulary WER has been reported for an unobtrusive acoustic-sensing SSI. At this level, the...

    work page

  6. [6]

    Discussion and conclusion We have introduced SoniSpeech, the first large-scale, open- vocabulary, trimodal dataset for minimally-obtrusive acoustic- sensing silent speech recognition. The baseline WER of 26.3% on silent speech demonstrates that open-vocabulary wearable SSI is a tractable problem, while the significant voiced-silent modality gap reveals a ...

    work page

  7. [7]

    The models used include: Gemini 3.0 Pro and Claude Opus 4.6

    Use of Generative AI Disclosure Generative AI has been used to improve writing of this paper. The models used include: Gemini 3.0 Pro and Claude Opus 4.6

    work page

  8. [8]

    Acknowledgments This research is supported by the National Science Foundation Grant No. 2239569. The lead author is partially supported by Qualcomm Innovation Fellowship. We would also like to thank the Information Science Department at Cornell University for providing support for this work

    work page

  9. [9]

    An improved model for voicing silent speech,

    ——, “An improved model for voicing silent speech,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), C. Zong, F. Xia, W. Li, and R. Navigli, Eds. Online: Association for Computational Linguistics, aug 2021, pp. 175–

    work page 2021

  10. [10]

    SODA: Million-scale dialogue distillation with social commonsense contextualization,

    H. Kim, J. Hessel, L. Jiang, P. West, X. Lu, Y . Yu, P. Zhou, R. Bras, M. Alikhani, G. Kim, M. Sap, and Y . Choi, “SODA: Million-scale dialogue distillation with social commonsense contextualization,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for ...

    work page 2023

  11. [11]

    Silent speech interfaces,

    B. Denby, T. Schultz, K. Honda, T. Hueber, J. Gilbert, and J. Brumberg, “Silent speech interfaces,”Speech Communication, vol. 52, no. 4, pp. 270–287, 2010, silent Speech Interfaces. [On- line]. Available: https://www.sciencedirect.com/science/article/ pii/S0167639309001307

    work page 2010

  12. [12]

    V oicing silent speech,

    D. M. Gaddy, “V oicing silent speech,” Ph.D. disser- tation, 2022, copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individ- ual underlying works; Last updated - 2023-03-08. [On- line]. Available: https://www.proquest.com/dissertations-theses/ voicing-silent-speech/docview/2726946028/se-2

    work page arXiv 2022

  13. [13]

    Echospeech: Continuous silent speech recognition on minimally-obtrusive eyewear powered by acoustic sensing,

    R. Zhang, K. Li, Y . Hao, Y . Wang, Z. Lai, F. Guimbreti `ere, and C. Zhang, “Echospeech: Continuous silent speech recognition on minimally-obtrusive eyewear powered by acoustic sensing,” inProceedings of the 2023 CHI Conference on Human Factors in Computing Systems, ser. CHI ’23. New York, NY , USA: Association for Computing Machinery, 2023. [Online]. Av...

    arXiv 2023

  14. [14]

    Tal: A synchronised multi-speaker corpus of ultrasound tongue imaging, audio, and lip videos,

    M. S. Ribeiro, J. Sanger, J.-X. Zhang, A. Eshky, A. Wrench, K. Richmond, and S. Renals, “Tal: A synchronised multi-speaker corpus of ultrasound tongue imaging, audio, and lip videos,” in 2021 IEEE Spoken Language Technology Workshop (SLT), 2021, pp. 1109–1116

    work page 2021

  15. [16]

    Data augmentation consists of SpecAugment [35] (2 fre- quency masks, 3 time masks) and concatenation of up to 3 ut- 100 102 104 Frequency ZH OY CH SH JH TH UH AW G HH ER V P AO EY Z F AA OW B NG DH Y W EH AE K M UW AY IY D L R S IH N AH T (a) Phoneme Distribution 5 10 15 20 25 Number of Words 2 4 6 8 10 12Duration (s) (b) Duration vs. Word Count Voiced Si...

    work page

  16. [17]

    The lj speech dataset,

    K. Ito and L. Johnson, “The lj speech dataset,” https://keithito. com/LJ-Speech-Dataset/, 2017

    work page 2017

  17. [18]

    Digital voicing of silent speech,

    D. Gaddy and D. Klein, “Digital voicing of silent speech,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), B. Webber, T. Cohn, Y . He, and Y . Liu, Eds. Online: Association for Computational Linguistics, Nov. 2020, pp. 5521–5530. [Online]. Available: https://aclanthology.org/2020.emnlp-main.445/

    work page 2020

  18. [19]

    Eario: A low-power acoustic sensing earable for continuously tracking detailed facial movements,

    K. Li, R. Zhang, B. Liang, F. Guimbreti `ere, and C. Zhang, “Eario: A low-power acoustic sensing earable for continuously tracking detailed facial movements,”Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 6, no. 2, Jul. 2022. [Online]. Available: https://doi.org/10.1145/3534621

    work page doi:10.1145/3534621 2022

  19. [20]

    SSR7000: A synchronized corpus of ultrasound tongue imaging for end-to-end silent speech recognition,

    N. Kimura, Z. Su, T. Saeki, and J. Rekimoto, “SSR7000: A synchronized corpus of ultrasound tongue imaging for end-to-end silent speech recognition,” inProceedings of the Thirteenth Language Resources and Evaluation Conference, N. Calzolari, F. B´echet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, J....

    work page 2022

  20. [21]

    Scaling law in neural data: Non-invasive speech decoding with 175 hours of eeg data,

    M. Sato, K. Tomeoka, I. Horiguchi, K. Arulkumaran, R. Kanai, and S. Sasai, “Scaling law in neural data: Non-invasive speech decoding with 175 hours of eeg data,”arXiv preprint arXiv:2407.07595, 2024

    Pith/arXiv arXiv 2024

  21. [22]

    A cross-modal approach to silent speech with llm-enhanced recognition,

    T. Benster, G. Wilson, R. Elisha, F. R. Willett, and S. Druckmann, “A cross-modal approach to silent speech with llm-enhanced recognition,”arXiv preprint arXiv:2403.05583, 2024

    Pith/arXiv arXiv 2024

  22. [23]

    Speechin: A smart necklace for silent speech recognition,

    R. Zhang, M. Chen, B. Steeper, Y . Li, Z. Yan, Y . Chen, S. Tao, T. Chen, H. Lim, and C. Zhang, “Speechin: A smart necklace for silent speech recognition,”Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 5, no. 4, Dec. 2022. [Online]. Available: https://doi.org/10.1145/3494987

    work page doi:10.1145/3494987 2022

  23. [24]

    A data-efficient and easy-to-use lip language interface based on wearable motion capture and speech movement reconstruction,

    S. Liu, T. Fawden, R. Zhu, G. G. Malliaras, and M. Bance, “A data-efficient and easy-to-use lip language interface based on wearable motion capture and speech movement reconstruction,” Science Advances, vol. 10, no. 26, p. eado9576, 2024. [Online]. Available: https://www.science.org/doi/abs/10.1126/ sciadv.ado9576

    work page 2024

  24. [25]

    Hpspeech: Silent speech interface for commodity headphones,

    R. Zhang, H. Chen, D. Agarwal, R. Jin, K. Li, F. Guimbreti `ere, and C. Zhang, “Hpspeech: Silent speech interface for commodity headphones,” inProceedings of the 2023 ACM International Symposium on Wearable Computers, ser. ISWC ’23. New York, NY , USA: Association for Computing Machinery, 2023, p. 60–65. [Online]. Available: https://doi.org/10.1145/359473...

    work page arXiv 2023

  25. [26]

    Earcommand:

    Y . Jin, Y . Gao, X. Xu, S. Choi, J. Li, F. Liu, Z. Li, and Z. Jin, “Earcommand: ”hearing” your silent speech commands in ear,”Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 6, no. 2, Jul. 2022. [Online]. Available: https://doi.org/10.1145/3534613

    doi:10.1145/3534613 2022

  26. [27]

    Rehearsse: Recognizing hidden-in-the-ear silently spelled expressions,

    X. Dong, Y . Chen, Y . Nishiyama, K. Sezaki, Y . Wang, K. Christofferson, and A. Mariakakis, “Rehearsse: Recognizing hidden-in-the-ear silently spelled expressions,” inProceedings of the 2024 CHI Conference on Human Factors in Computing Systems, ser. CHI ’24. New York, NY , USA: Association for Computing Machinery, 2024. [Online]. Available: https: //doi....

    arXiv 2024

  27. [28]

    Novel three-axis accelerometer-based silent speech in- terface using deep neural network,

    J. Kwon, H. Nam, Y . Chae, S. Lee, I. Y . Kim, and C.-H. Im, “Novel three-axis accelerometer-based silent speech in- terface using deep neural network,”Engineering Applications of Artificial Intelligence, vol. 120, p. 105909, 2023. [On- line]. Available: https://www.sciencedirect.com/science/article/ pii/S0952197623000933

    work page 2023

  28. [29]

    Spellring: Recognizing continuous fingerspelling in american sign language using a ring,

    H. Lim, N. A. Dang, D. Lee, T. C. Yu, J. Lu, F. M. Li, Y . Jin, Y . Ma, X. Bi, F. Guimbreti`ere, and C. Zhang, “Spellring: Recognizing continuous fingerspelling in american sign language using a ring,” inProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, ser. CHI ’25. New York, NY , USA: Association for Computing Machinery, 2025...

    arXiv 2025

  29. [30]

    Eyeecho: Continuous and low-power facial expression tracking on glasses,

    K. Li, R. Zhang, S. Chen, B. Chen, M. Sakashita, F. Guimbretiere, and C. Zhang, “Eyeecho: Continuous and low-power facial expression tracking on glasses,” inProceedings of the 2024 CHI Conference on Human Factors in Computing Systems, ser. CHI ’24. New York, NY , USA: Association for Computing Machinery, 2024. [Online]. Available: https://doi.org/10.1145/...

    arXiv 2024

  30. [31]

    An audio-visual corpus for speech perception and automatic speech recognition,

    M. Cooke, J. Barker, S. Cunningham, and X. Shao, “An audio-visual corpus for speech perception and automatic speech recognition,”The Journal of the Acoustical Society of America, vol. 120, no. 5, pp. 2421–2424, 11 2006. [Online]. Available: https://doi.org/10.1121/1.2229005

    work page doi:10.1121/1.2229005 2006

  31. [32]

    Speech rate and pausing in english: Comparing learners at different levels of proficiency with native speakers

    L.-F. Huang and T. Gr ´af, “Speech rate and pausing in english: Comparing learners at different levels of proficiency with native speakers.”Taiwan Journal of TESOL, vol. 17, no. 1, pp. 57–86, 2020

    work page 2020

  32. [33]

    Average speaking rate and words per minute,

    D. Barnard, “Average speaking rate and words per minute,” https://virtualspeech.com/blog/ average-speaking-rate-words-per-minute, VirtualSpeech, November 2022, accessed: 2026-02-22

    work page 2022

  33. [34]

    Sonicid: User identification on smart glasses with acoustic sensing,

    K. Li, D. Agarwal, R. Zhang, V . Gunda, T. Mo, S. Mahmud, B. Chen, F. Guimbreti ˇere, and C. Zhang, “Sonicid: User identification on smart glasses with acoustic sensing,”Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 8, no. 4, Nov

    work page

  34. [35]

    Specaugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” inInterspeech 2019, ser. interspeech 2019. ISCA, Sep. 2019, p. 2613–2617. [Online]. Available: http://dx.doi.org/10.21437/Interspeech.2019-2680

    doi:10.21437/interspeech.2019-2680 2019

  35. [36]

    Echoforce: Continuous grip force estimation from skin deformation using active acoustic sensing on a wristband,

    K. Mahmoodi, Y . Xie, T. Gemicioglu, C.-J. Lee, J. Kim, and C. Zhang, “Echoforce: Continuous grip force estimation from skin deformation using active acoustic sensing on a wristband,” inProceedings of the 2025 ACM International Symposium on Wearable Computers, ser. ISWC ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 53–60. [Online]...

    arXiv 2025

  36. [37]

    Ring-a-pose: A ring for continuous hand pose tracking,

    T. C. Yu, G. Hu, R. Zhang, H. Lim, S. Mahmud, C.-J. Lee, K. Li, D. Agarwal, S. Nie, J. Oh, F. Guimbreti `ere, and C. Zhang, “Ring-a-pose: A ring for continuous hand pose tracking,”Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 8, no. 4, Nov. 2024. [Online]. Available: https://doi.org/10.1145/3699741

    work page doi:10.1145/3699741 2024

  37. [38]

    Echowrist: Continuous hand pose tracking and hand-object interaction recognition using low-power active acoustic sensing on a wristband,

    C.-J. Lee, R. Zhang, D. Agarwal, T. C. Yu, V . Gunda, O. Lopez, J. Kim, S. Yin, B. Dong, K. Li, M. Sakashita, F. Guimbretiere, and C. Zhang, “Echowrist: Continuous hand pose tracking and hand-object interaction recognition using low-power active acoustic sensing on a wristband,” in Proceedings of the 2024 CHI Conference on Human Factors in Computing Syste...

    work page arXiv 2024

  38. [39]

    Watchhand: Enabling continuous hand pose tracking on off-the-shelf smartwatches,

    J. Kim, C.-J. Lee, H. Jung, T. Catherine Yu, R. Zhang, I. Oakley, and C. Zhang, “Watchhand: Enabling continuous hand pose tracking on off-the-shelf smartwatches,” inProceedings of the 2026 CHI Conference on Human Factors in Computing Systems, ser. CHI ’26. New York, NY , USA: Association for Computing Machinery, 2026. [Online]. Available: https: //doi.org...

    work page arXiv 2026

  39. [41]

    Actsonic: Recognizing everyday activities from inaudible acoustic wave around the body,

    S. Mahmud, V . Parikh, Q. Liang, K. Li, R. Zhang, A. Ajit, V . Gunda, D. Agarwal, F. Guimbretiere, and C. Zhang, “Actsonic: Recognizing everyday activities from inaudible acoustic wave around the body,”Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 8, no. 4, Nov. 2024. [Online]. Available: https://doi.org/10.1145/3699752

    doi:10.1145/3699752 2024

  40. [42]

    Munchsonic: Tracking fine- grained dietary actions through active acoustic sensing on eyeglasses,

    S. Mahmud, D. Agarwal, A. Ajit, Q. Liang, T. Viranda, F. Guimbretiere, and C. Zhang, “Munchsonic: Tracking fine- grained dietary actions through active acoustic sensing on eyeglasses,” inProceedings of the 2024 ACM International Symposium on Wearable Computers, ser. ISWC ’24. New York, NY , USA: Association for Computing Machinery, 2024, p. 96–103. [Onlin...

    doi:10.1145/3675095 2024

  41. [43]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    2016

  42. [44]

    SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,

    T. Kudo and J. Richardson, “SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,” inProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, E. Blanco and W. Lu, Eds. Brussels, Belgium: Association for Computational Linguistics, Nov. 2018, pp. 6...

    2018

  43. [45]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” inProceedings of the 23rd International Conference on Machine Learning, ser. ICML ’06. New York, NY , USA: Association for Computing Machinery, 2006, p. 369–376. [Online]. Available: https://...

    arXiv 2006

  44. [181]

    Available: https://aclanthology.org/2021.acl-short

    [Online]. Available: https://aclanthology.org/2021.acl-short. 23/

    work page 2021

  45. [2024]

    Available: https://doi.org/10.1145/3699734

    [Online]. Available: https://doi.org/10.1145/3699734

    work page doi:10.1145/3699734