Pith. sign in

REVIEW 3 major objections 6 minor 75 references

CapTune: Adapting Non-Speech Captions With Anchored Generative Models

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CapTune claims non-speech captions can be personalized for deaf and hard-of-hearing viewers—tuning detail, expressiveness, sound style, and genre fit—while anchored, creator-set bounds preserve editorial control.

desk verdict CapTune is a solid, clearly described HCI systems paper whose main risk—unverified LLM adherence to creator anchors—is real but openly acknowledged; the qualitative findings hold up, though the abstract overstates what was measured. read the letter →

arxiv 2508.19971 v1 pith:HVQDDIXQ submitted 2025-08-27 cs.HC

Jeremy Zhengqi Huang , Caluã de Lacerda Pataca , Liang-Yuan Wu , Dhruv Jain This is my paper
classification cs.HC
keywords non-speechcaptionsdeafandhardofhearingcaptionpersonalizationgenerativeAIlargelanguagemodelsanchoredtransformationaccessibilityvideo
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to replace the "one-size-fits-all" non-speech caption track with a dynamic one: captions that deaf and hard-of-hearing viewers can tune to their own needs without overriding what the caption author intended. CapTune does this with two tools—a Creator Tool in which authors mark a two-dimensional safe space for transformations, bounded by anchor captions along Level of Detail and Expressiveness, and a Viewer Client in which each viewer picks settings inside that space, plus a sound-representation style and optional genre alignment. The rewriting is done by GPT-4o, prompted with interpolation ratios that place the viewer's choice relative to the anchors and the current caption, together with audio-visual scene context extracted by a video language model. Evaluations with seven caption creators and twelve DHH participants found that creators felt they kept editorial control and viewers reported greater emotional and narrative engagement; the study also surfaced tensions between information richness and cognitive load, between interpretive and descriptive sound language, and the strongly context-dependent nature of caption preferences.

What carries the argument

The anchored transformation space: a two-dimensional space whose axes are Level of Detail and Expressiveness (each a 1–10 semantic scale), with its bounds set by creator-chosen lower and upper anchor captions. Transformation requests are converted into two ratios per axis—r (Eq. 2), the requested setting's position between the anchors, and δ (Eq. 3), the signed magnitude of change from the current caption—embedded in a structured prompt that instructs GPT-4o to interpolate among the original caption, the lower-anchor caption, and the upper-anchor caption, using audio-visual scene descriptions extracted by VideoLLaMA2 as context. This interpolation-between-examples mechanism is what keeps gen

What would settle it

A systematic accuracy and consistency audit: take a fixed caption set across genres, apply the transformation at every grid point in the creator-defined space, and have independent raters (including DHH viewers) judge whether each output stays within the anchor bounds and preserves the sound event's meaning. The paper's own reported failures—[Ice freezing sound] becoming [Chill ice cracking], and identical [Dolphin whistles] captions transformed differently in different scenes—are concrete starting points; if bound violations or meaning shifts are frequent even at moderate settings, the claim

Watch

Extended reading notes

Core claim

CapTune's central claim is that the text of non-speech captions—not just their visual styling—can be safely personalized by anchoring generative transformations to creator-defined boundaries. A caption author sets two anchor points: a lower anchor for the most minimal acceptable captions and an upper anchor for the most elaborate ones, on the Level of Detail and Expressiveness axes. The Viewer Client exposes only cells inside this space; when a viewer picks a setting, the system computes an interpolation ratio (Eq. 2) locating the choice between the two anchors and a change ratio (Eq. 3) measuring the shift from the current caption, using both to instruct GPT-4o to interpolate between the an

Load-bearing premise

The entire system rests on one assumption: that GPT-4o, given the anchor captions, the interpolation ratios, and the scene context, actually rewrites each caption so it stays within the creator's bounds and preserves the meaning of the original sound event—and the paper concedes in its limitations that outputs can be inconsistent or semantically inaccurate.

Editorial extensions

If this is right

  • Non-speech captions can be treated as co-authored media: the creator sets the safe range, the viewer personalizes within it, and the language model does the rewriting—offering a template for accessibility content that is neither fixed nor unconstrained.
  • Caption-customization interfaces can be built around interpolation between concrete anchor examples rather than abstract style rules, which creators in the study found intuitive.
  • Viewer preference is context-dependent (genre, scene pacing, viewing intent), so future systems should support scene-level or context-aware caption adaptation rather than a single global setting.
  • Design requirements follow directly: user profiles that retain preferences, preview-and-compare views, explainable transformation logic, and vocabulary simplification for ASL-first or non-native-English viewers.
  • The four-parameter scheme makes explicit a core trade-off: richer, more expressive captions heighten emotional engagement but raise cognitive load and can crowd out the viewer's own interpretation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The two-anchor, ratio-driven prompting recipe is domain-neutral: the same "human sets bounds with concrete examples, model interpolates between them" pattern could extend to audio description, simplified subtitles for language learners, or other constrained rewriting tasks—though the paper does not claim this.
  • The reported inconsistency of identical sound sources (e.g., dolphin whistles) across scenes hints at a testable invariant: at fixed parameter settings, a given sound event should yield the same transformed caption regardless of context; adding such a consistency check would strengthen the pipeline.
  • The interpretive-versus-descriptive tension participants voiced suggests a fifth axis or an explicit marking of inferred content (e.g., "warm purr" as interpretation), which the paper mentions as future work only in passing.
  • Short clips (2–8 minutes) leave open whether emotional-engagement gains persist across feature-length content, where fatigue and cross-scene narrative consistency become dominant factors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces CapTune, a system for customizing non-speech captions (NSI) in video for deaf and hard of hearing (DHH) viewers. Caption creators define a two-dimensional transformation space using anchor points for Level of Detail and Expressiveness; viewers then select preferences within that space, optionally toggling sound representation method and genre alignment. Transformations are generated by GPT-4o using prompts that encode interpolation ratios (Eqs. 2 and 3) relative to the creator's anchors and the current caption. The system was evaluated in two qualitative studies: seven caption creators used the Creator Tool, and twelve DHH participants used the Viewer Client. The paper reports that creators felt they retained creative control and that DHH viewers experienced enhanced emotional engagement, while also surfacing trade-offs between information richness and cognitive load, tensions between interpretive and descriptive sound representation, and context-dependent preferences. The authors acknowledge in Section 7.7 that the system can produce inconsistent or semantically inaccurate outputs and that no systematic metrics of caption transformation accuracy were collected.

Significance. If the central claims hold, CapTune is a useful contribution to accessible media: it moves beyond visual styling of captions to personalized transformation of caption text itself, grounded in a qualitative analysis of DHH viewers' expressed preferences. The system is open-sourced, and the evaluation is substantial for an HCI paper, including a three-level codebook with 168 third-level codes and interrater reliability of 0.74, which is a concrete strength. The design space (level of detail, expressiveness, sound representation, genre alignment) is well motivated and the findings on context-dependent preferences and cognitive load are valuable for future captioning systems. However, the two headline claims—'preserving creator intent' and 'enhancing viewers' emotional engagement'—are not equally supported. The first depends on an unverified assumption that GPT-4o transformations stay within creator-defined bounds and preserve sound-event semantics; the second rests entirely on self-report without a control or baseline. These gaps are acknowledged in the paper itself, but they are load-bearing for the abstract's claims.

major comments (3)
  1. [§4.3.3 and §7.7] The claim that CapTune 'preserves creator intent' is not supported by the evidence presented. Equations (2) and (3) only compute interpolation ratios; they do not constrain the LLM output. The transformation is a prompted call to GPT-4o with no verification that the output lies within the anchor-defined bounds or preserves the meaning of the original sound event. The authors concede in §7.7 that outputs 'can still produce inconsistent or semantically inaccurate outputs' and that no systematic metrics of caption transformation accuracy or consistency were collected. Participant data already show concrete violations: C2 (Section 5.3.2) flagged 'Anna exhales with a sigh of relief' as overly specific; P2 (Section 6.4.6) observed identical [Dolphin whistles] transformed inconsistently; P3 and P8 questioned interpretive overreach. The abstract's 'preserving creator intent' is therefore an over
  2. [§6.4.1 and §6.2] The claim that CapTune 'enhanced viewers' emotional engagement' rests on self-report from nine of twelve participants during a single-session, no-baseline, no-control qualitative study. Participants were introduced to the system and asked to explore it while watching short clips; there was no comparison with the original unmodified captions, no alternative condition (e.g., unconstrained LLM transformation, or a non-anchored personalization interface), and no measurement of engagement beyond interview statements. This design cannot rule out novelty effects or demand characteristics, and it does not support the directional claim of 'enhancing' engagement. A controlled comparison or at minimum a pre/post self-report with the original caption track as baseline would be needed to substantiate the wording in the abstract.
  3. [§4.3.3, §5.3.2, and §7.2] The qualitative findings about creators' 'creative control' cannot be attributed specifically to the anchored transformation mechanism, because no condition isolates it from generic LLM prompting. Creators interacted with a full interface that included sliders, previews, manual editing, and locking; any of these could produce the sense of agency that participants described. C2's and C6's concerns about semantic accuracy are evidence that the anchor mechanism did not reliably prevent over-interpretation. Without an ablation or a side-by-side comparison with unconstrained GPT-4o generation, the paper should not imply that anchoring is the mechanism responsible for the reported creative-control benefits; it can only claim that creators felt control in this particular system configuration.
minor comments (6)
  1. [Figure references] The text 'visualized in Figure 4.2' (Section 4.2.3) appears to reference a figure number incorrectly; check whether it should be a numbered figure, likely Figure 1 or a dedicated anchor-space figure.
  2. [Table 2] There is a typo in C6's entry: 'Professioal ads' should be 'Professional ads.'
  3. [§4.2.2] The description of how GPT-4o determines baseline Level of Detail and Expressiveness values is underspecified. State whether this is a one-time analysis of the whole caption file or per caption, and provide the exact prompt used, since the baseline affects all subsequent transformations.
  4. [§4.3.3] The prompt template refers to 'the [lower-anchor captions]' and 'the [upper-anchor captions]' without clarifying how multiple captions at an anchor are sampled or represented. If anchors are per-caption, clarify; if anchors are global, explain how a single pair of exemplars is chosen for each transformation.
  5. [§3.1] The Reddit-derived dataset is small (51 posts from 13 unique threads) and may be subject to self-selection. This is acceptable for a formative analysis, but the paper should acknowledge more explicitly that the four design opportunities are drawn from a narrow online sample.
  6. [§6.4.6] The inconsistency example of [Dolphin whistles] is an important data point, but the text does not state which parameter settings produced the differing outputs. Including that context would strengthen the finding and help future work reproduce or address the issue.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation is circular: the anchor equations are definitional mappings, and the unverified LLM-adherence claim is a validity gap rather than a reduction to inputs.

full rationale

The paper contains no derivation chain that reduces a predicted quantity to a fitted input. Equations (1)-(3) are definitional: Eq. (1) maps slider positions to parameter values, and Eqs. (2)-(3) compute interpolation ratios from the creator-set anchors and viewer-selected values. These quantities are designer/user-defined, not fit to outcome data, and the paper does not claim to predict them from the evaluation results. The abstract's claim that CapTune preserves 'creator intent' is an empirical assertion about GPT-4o adherence, not a consequence of the equations; Section 7.7 explicitly concedes that GPT-4o 'can still produce inconsistent or semantically inaccurate outputs' and that the evaluation 'do[es] not provide systematic metrics of caption transformation accuracy or consistency.' That is a validity/evidence limitation, not circularity. The qualitative findings are user reports, not quantities derived from the system's parameters, so there is no fitted-input-called-prediction step. Self-citations (e.g., refs. [21], [23], [35]) are contextual and do not carry the central claim; no uniqueness theorem or ansatz is imported from them. No specific circular step can be quoted or exhibited, so the score reflects only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numeric constants are fitted to data in this paper. The Level of Detail and Expressiveness baselines (e.g., 3 and 2 in Section 4.2.2) are UI calibration values inferred by GPT-4o, not fitted parameters tied to an outcome, so they are not listed as free parameters. No new physical or conceptual entities are postulated; the four customization dimensions are design constructs grounded in qualitative themes. The load-bearing assumptions are about LLM reliability, video model accuracy, self-report validity, and the representativeness of the Reddit corpus.

assumptions (4)
  • domain assumption GPT-4o transforms captions inside creator-defined bounds while preserving semantic meaning when prompted with anchor examples and interpolation ratios.
    Used throughout Section 4.3.3; no systematic accuracy or consistency metric is reported, and Section 7.7 concedes outputs can be inconsistent or semantically inaccurate.
  • domain assumption VideoLLaMA2 supplies accurate audio-visual descriptions of 5-second segments around each caption.
    Section 4.2.1 uses these descriptions as context for transformations; their accuracy is not evaluated.
  • domain assumption Self-report during a single session with short clips reflects real long-form viewing behavior and accessibility outcomes.
    Sections 5 and 6 rest on interview data; Section 7.7 lists short-form-only evaluation as a limitation.
  • domain assumption The four customization parameters capture the main axes of DHH caption preference diversity.
    Section 3 derives them from 51 posts in 13 Reddit threads; this is a small, platform-specific corpus, so generalizability is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CapTune: Adapting Non-Speech Captions With Anchored Generative Models." pith.science (2026). https://pith.science/paper/HVQDDIXQ

@misc{pith2026250819971,
  author       = {Pith},
  title        = {Pith review of: CapTune: Adapting Non-Speech Captions With Anchored Generative Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HVQDDIXQ}},
  note         = {Machine review of arXiv:2508.19971}
}
read the original abstract

Non-speech captions are essential to the video experience of deaf and hard of hearing (DHH) viewers, yet conventional approaches often overlook the diversity of their preferences. We present CapTune, a system that enables customization of non-speech captions based on DHH viewers' needs while preserving creator intent. CapTune allows caption authors to define safe transformation spaces using concrete examples and empowers viewers to personalize captions across four dimensions: level of detail, expressiveness, sound representation method, and genre alignment. Evaluations with seven caption creators and twelve DHH participants showed that CapTune supported creators' creative control while enhancing viewers' emotional engagement with content. Our findings also reveal trade-offs between information richness and cognitive load, tensions between interpretive and descriptive representations of sound, and the context-dependent nature of caption preferences.

Figures

Figures reproduced from arXiv: 2508.19971 by the authors.

Figure 1
Figure 1. Overview of the CapTune transformation system. (1) Non-speech caption transformations are mapped within a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Creator Tool interface for configuring the caption transformation space. The image shows (1) the overall interface [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Example of iterative caption transformation in CapTune. (1) The original caption, [Loud Thunder Sound], is provided by [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Viewer Client interface for customizing non-speech information in captions. The image shows (1) a screenshot of the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 54 canonical work pages

  1. [1]

    Access Intimacy: The Missing Link

    2011. Access Intimacy: The Missing Link. https://leavingevidence.wordpress.com/ 2011/05/05/access-intimacy-the-missing-link/

    work page 2011

  2. [2]

    Captioning Key

    2025. Captioning Key. https://dcmp.org/learn/captioningkey

    work page 2025

  3. [3]

    DaVinci Resolve | Blackmagic Design

    2025. DaVinci Resolve | Blackmagic Design . https://www.blackmagicdesign.com/ products/davinciresolve

    work page 2025

  4. [4]

    English (USA) Timed Text Style Guide

    2025. English (USA) Timed Text Style Guide . https://partnerhelp.netflixstudios. com/hc/en-us/articles/217350977-English-USA-Timed-Text-Style-Guide

    work page arXiv 2025

  5. [5]

    Expressive Captions: Google’s Next Generation Caption | Android

    2024. Expressive Captions: Google’s Next Generation Caption | Android . https: //www.android.com/accessibility/expressive-captions/

    work page 2024

  6. [6]

    2025. Gemini. https://deepmind.google/technologies/gemini/

    work page 2025

  7. [7]

    Listening Fatigue

    2025. Listening Fatigue. https://www.audiology.org/consumers-and-patients/ hearing-and-balance/fatigue/

    work page 2025

  8. [8]

    Medical and Social Models of Disability | Office of Developmental Primary Care

    2013. Medical and Social Models of Disability | Office of Developmental Primary Care. https://odpc.ucsf.edu/clinical/patient-centered-care/medical-and-social- models-of-disability

    work page 2013

Show all 75 references
  1. [9]

    Meet Claude

    2025. Meet Claude. https://www.anthropic.com/claude

    2025

  2. [10]

    PRA W 7.7.1 documentation

    2023. PRA W 7.7.1 documentation. https://praw.readthedocs.io/en/stable/

    2023

  3. [11]

    What Is American Sign Language (ASL)? | NIDCD

    2021. What Is American Sign Language (ASL)? | NIDCD . https://www.nidcd.nih. gov/health/american-sign-language

    2021

  4. [12]

    Abby Alepa. 2022. Stranger Things’ Captions: A Lesson in Language and De- tail. https://www.3playmedia.com/blog/stranger-things-captions-a-lesson-in- language-and-detail/

    2022

  5. [13]

    Oliver Alonzo, Hijung Valentina Shin, and Dingzeyu Li. 2022. Beyond Subtitles: Captioning and Visualizing Non-speech Sounds to Improve Accessibility of User-Generated Videos. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (New ...

    2022 doi

  6. [14]

    Meryl Alper. 2017. Giving Voice: Mobile Communication, Disability, and Inequal- ity. The MIT Press. https://direct.mit.edu/books/monograph/4490/Giving- VoiceMobile-Communication-Disability-and

    2017

  7. [15]

    Bradley Knox, and Todd Kulesza

    Saleema Amershi, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4 (2014), 105–120

    2014

  8. [16]

    Kathy Baxter, Catherine Courage, and Kelly Caine. 2015. Chapter 13 - Field Studies. In Understanding your Users (Second Edition) , Kathy Baxter, Catherine Courage, and Kelly Caine (Eds.). Morgan Kaufmann, 378–428. doi:10.1016/B978- 0-12-800232-2.00013-4

    2015 doi

  9. [17]

    Bennett and Daniela K

    Cynthia L. Bennett and Daniela K. Rosner. 2019. The Promise of Empathy: Design, Disability, and Knowing the "Other". InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2019-05-02) (CHI ’19). Association for Computing Machinery, 1...

    2019

  10. [18]

    Tara Bitran. 2022. Meet the Linguists Behind the Genius ‘Stranger Things’ Season 4 Captions. https://www.netflix.com/tudum/articles/stranger-things-season-4- captions

    2022

  11. [19]

    Danielle Bragg, Nicholas Huynh, and Richard E. Ladner. 2016. A Person- alizable Mobile Sound Detector App Design for Deaf and Hard-of-Hearing Users. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (Reno Nevada USA, 2016-10-23). ...

    2016

  12. [20]

    Cristina Cambra, Leal , Aurora, , and Núria Silvestre. 2010. How Deaf and Hearing Adolescents Comprehend a Televised Story. 12, 1 (2010), 34–

    2010

  13. [21]

    Caluã De Lacerda Pataca, Saad Hassan, Nathan Tinker, Roshan Lalintha Peiris, and Matt Huenerfauth. 2024. Caption Royale: Exploring the Design Space of Affective Captions from the Perspective of Deaf and Hard-of-Hearing Individuals. In Proceedings of the CHI Conference on Human...

    2024

  14. [22]

    Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019. Build it break it fix it for dialogue safety: Robustness from adversarial human attack. arXiv preprint arXiv:1908.06083 (2019)

    2019 arXiv

  15. [24]

    Gregory J. Downey. 2008. Closed Captioning. Johns Hopkins University Press. doi:10.1353/book.3337

    2008 doi

  16. [25]

    Upol Ehsan, Pradyumna Tambwekar, Larry Chan, Brent Harrison, and Mark Riedl

  17. [26]

    Michele Friedner, , and Stefan Helmreich. 2012. Sound Studies Meets Deaf Studies. 7, 1 (2012), 72–86. doi:10.2752/174589312X13173255802120 Publisher: Routledge _eprint: https://doi.org/10.2752/174589312X13173255802120

    2012 doi

  18. [27]

    Gajos, Daniel S

    Krzysztof Z. Gajos, Daniel S. Weld, and Jacob O. Wobbrock. 2010. Automatically generating personalized user interfaces with Supple. 174, 12 (2010), 910–950. doi:10.1016/j.artint.2010.05.005

    2010 doi

  19. [28]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith

  20. [29]

    Great-Typhoon. 2023. What should hearing folks keep in mind when creating subtitles? https://www.reddit.com/r/deaf/comments/142wszc/what_should_ hearing_folks_keep_in_mind_when/

    2023

  21. [30]

    Greg Guest, Kathleen M.MacQueen, and Emily E.Namey. 2012. Applied Thematic Analysis. SAGE Publications, Inc. doi:10.4135/9781483384436

    2012 doi

  22. [31]

    Gaole He, Abri Bharos, and Ujwal Gadiraju. 2024. To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems . doi:10.1145/3648188. 3675141

    2024 doi

  23. [32]

    Hoffmeister and Catherine L

    Robert J. Hoffmeister and Catherine L. Caldwell-Harris. 2014. Acquiring English as a second language via print: The task for deaf children. 132, 2 (2014), 229–242. doi:10.1016/j.cognition.2014.03.014

    2014 doi

  24. [33]

    Thomas K. Holcomb. 2013. Introduction to American Deaf Culture . OUP USA. Google-Books-ID: pySsIU0WFpQC

    2013

  25. [34]

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The Curious Case of Neural Text Degeneration. arXiv preprint arXiv:1904.09751 (2019)

    2019 arXiv

  26. [35]

    Jeremy Zhengqi Huang, Jaylin Herskovitz, Liang-Yuan Wu, Cecily Morrison, and Dhruv Jain. 2025. Weaving Sound Information to Support Real-Time Sensemaking of Auditory Environments: Co-Designing with a DHH User

    2025

  27. [36]

    W3C Web Accessibility Initiative (WAI). 2025. Captions/Subtitles. https://www. w3.org/WAI/media/av/captions/

    2025

  28. [37]

    Goodman, Rachel Grossman- Kahn, Hung Ngo, Aditya Kusupati, Ruofei Du, Alex Olwal, Leah Findlater, and Jon E

    Dhruv Jain, Khoa Huynh Anh Nguyen, Steven M. Goodman, Rachel Grossman- Kahn, Hung Ngo, Aditya Kusupati, Ruofei Du, Alex Olwal, Leah Findlater, and Jon E. Froehlich. 2022. ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users. In Pr...

    2022

  29. [38]

    Froehlich

    Dhruv Jain, Kelly Mack, Akli Amrous, Matt Wright, Steven Goodman, Leah Findlater, and Jon E. Froehlich. 2020. HomeSound: An Iterative Field Deployment of an In-Home Sound Awareness System for Deaf or Hard of Hearing Users. In Proceedings of the 2020 CHI Conference on Human Fac...

    2020

  30. [39]

    Dhruv Jain, Hung Ngo, Pratyush Patel, Steven Goodman, Leah Findlater, and Jon Froehlich. 2020. SoundWatch: Exploring Smartwatch-based Deep Learning Approaches to Support Sound Awareness for Deaf and Hard of Hearing Users. In The 22nd International ACM SIGACCESS Conference on C...

    2020

  31. [40]

    Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Kwan Yee Ng, Juntao Dai, Xuehai Pan, Aidan O’Gara, Yingshan Lei, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, S...

    2024 doi arXiv

  32. [41]

    Association for Computing Machinery, 1–16

    (CHI ’22). Association for Computing Machinery, 1–16. doi:10.1145/3491102. 3502020

    doi

  33. [42]

    Dr Paddy Ladd. 2008. Understanding Deaf Culture: In Search of Deafhood . Multi- lingual Matters. ASSETS ’25, October 26–29, 2025, Denver, CO, USA Huang et al

    2008

  34. [43]

    2003.Understanding Deaf Culture: In Search of Deafhood

    Paddy Ladd. 2003.Understanding Deaf Culture: In Search of Deafhood. Multilingual Matters. http://ebookcentral.proquest.com/lib/umichigan/detail.action?docID= 204124

    2003

  35. [44]

    Harlan Lane. 2002. Do Deaf People Have a Disability? 2, 4 (2002), 356–379. https://www.jstor.org/stable/26204820 Publisher: Gallaudet University Press

    2002

  36. [45]

    Sachin Kumar, Biswajit Paria, and Yulia Tsvetkov. 2022. Gradient-based con- strained sampling from language models. arXiv preprint arXiv:2205.12558 (2022)

    2022 arXiv

  37. [46]

    Mina Lee, Katy Ilonka Gero, John Joon Young Chung, Simon Buckingham Shum, Vipul Raheja, Hua Shen, Subhashini Venugopalan, Thiemo Wambsganss, David Zhou, Emad A Alghamdi, et al. 2024. A design space for intelligent and interactive writing assistants. In Proceedings of the 2024 ...

    2024

  38. [47]

    Scott K. Liddell. 2003. Grammar, Gesture, and Meaning in American Sign Language. Cambridge University Press. doi:10.1017/CBO9780511615054

    2003 doi

  39. [48]

    Hayes, and Devva Kasnitz

    Jennifer Mankoff, Gillian R. Hayes, and Devva Kasnitz. 2010. Disability studies as a source of critical inquiry for the field of assistive technology. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibil- ity (New York, NY, USA, 2010-10...

    2010

  40. [49]

    LEE, DEBORAH I

    DANIEL G. LEE, DEBORAH I. FELS, and JOHN PATRICK UDO. 2007. Emotive captioning. 5, 2 (2007), 11:11. doi:10.1145/1279540.1279551

    2007

  41. [50]

    Lloyd May, Alex Williams, Saad Hassan, Mark Cartwright, and Sooyeon Lee

  42. [51]

    doi:10.1179/146431510X12626982043606 Publisher: Routledge _eprint: https://doi.org/10.1179/146431510X12626982043606

    doi

  43. [52]

    2004.Hearing Loss: Determining Eligibility for Social Security Benefits

    National Research Council (US) Committee on Disability Determination for Individuals with Hearing Impairments. 2004.Hearing Loss: Determining Eligibility for Social Security Benefits . National Academies Press (US). http://www.ncbi. nlm.nih.gov/books/NBK207838/

    2004

  44. [53]

    Lloyd May, So Yeon Park, and Jonathan Berger. 2023. Enhancing Non-Speech Information Communicated in Closed Captioning Through Critical Design. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY, USA, 2023-10-22) (ASS...

    2023

  45. [54]

    Pop-Up Magazine. 2020. Artist Christine Sun Kim Rewrites Closed Captions | Pop-Up Magazine. https://www.youtube.com/watch?v=tfe479qL8hg

    2020

  46. [55]

    Raisa Rashid, Vy , Quoc, Hunt , Richard, , and Deborah I. Fels. 2008. Danc- ing with Words: Using Animated Text for Captioning. 24, 5 (2008), 505–

    2008

  47. [56]

    Mary L. McHugh. 2012. Interrater reliability: the kappa statistic. 22, 3 (2012), 276–282. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3900052/

    2012

  48. [57]

    Russell S. Rosen. 2007. Representations of Sound in American Deaf Literature. 12, 4 (2007), 552–565. doi:10.1093/deafed/enm010

    2007 doi

  49. [58]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35...

    2022

  50. [59]

    exciting music plays

    SamwellBarley. 2021. Deaf people of Reddit - How much do subtitles like "exciting music plays" or "menacing laugh" aid your understanding of what you’re watch- ing? https://www.reddit.com/r/AskReddit/comments/q5rhyx/deaf_people_of_ reddit_how_much_do_subtitles_like/

    2021

  51. [60]

    Wendy Sandler and Diane Lillo-Martin. 2006. Sign Language and Linguistic Universals. Cambridge University Press. doi:10.1017/CBO9781139163910

    2006 doi

  52. [61]

    Bigham, Frank Bentley, Joyce Chai, Zachary Lipton, Qiaozhu Mei, Rada Mihalcea, Michael Terry, Diyi Yang, Meredith Ringel Morris, Paul Resnick, and David Jurgens

    Hua Shen, Tiffany Knearem, Reshmi Ghosh, Kenan Alkiek, Kundan Krishna, Yachuan Liu, Ziqiao Ma, Savvas Petridis, Yi-Hao Peng, Li Qiwei, Sushrita Rakshit, Chenglei Si, Yutong Xie, Jeffrey P. Bigham, Frank Bentley, Joyce Chai, Zachary Lipton, Qiaozhu Mei, Rada Mihalcea, Michael T...

    2024 doi

  53. [62]

    Laria Reynolds and Kyle McDonell. 2021. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended abstracts of the 2021 CHI conference on human factors in computing systems . 1–7

    2021

  54. [63]

    J. P. Udo and D. I. Fels. 2010. The rogue poster-children of universal design: closed captioning and audio description. (2010). doi:10.1080/09544820903310691 Publisher: Taylor & Francis

    2010 doi

  55. [64]

    Savannah Salazar. 2022. Wet Writhing, Eldritch Gurgling: A Chat With the Stranger Things Subtitles Team . https://www.vulture.com/2022/07/stranger- things-subtitles-captions-team-interview.html

    2022

  56. [65]

    Vy and Deborah I

    Quoc V. Vy and Deborah I. Fels. 2010. Using Placement and Name for Speaker Identification in Captioning. In Computers Helping People with Special Needs (Berlin, Heidelberg, 2010), Klaus Miesenberger, Joachim Klaus, Wolfgang Zagler, and Arthur Karshmer (Eds.). Springer, 247–254...

    2010 doi

  57. [66]

    Fangzhou Wang, Hidehisa Nagano, Kunio Kashino, and Takeo Igarashi. 2017. Visualizing Video Sounds With Sound Word Animation to Enrich User Experience. 19, 2 (2017), 418–429. doi:10.1109/TMM.2016.2613641 Conference Name: IEEE Transactions on Multimedia

    2017

  58. [67]

    Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang

    Zijie J. Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang. 2021. Putting Humans in the Natural Language Processing Loop: A Survey.arXiv preprint arXiv:2103.04044 (2021)

    2021 arXiv

  59. [68]

    Wobbrock

    Kristen Shinohara and Jacob O. Wobbrock. 2011. In the shadow of misperception: assistive technology use and social interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2011- 05-07) (CHI ’11). Association for Computing ...

    2011

  60. [69]

    Sean Zdenek. 2015. Reading Sounds: Closed-Captioned Media and Popular Cul- ture. University of Chicago Press. https://press.uchicago.edu/ucp/books/book/ chicago/R/bo21933882.html

    2015

  61. [70]

    Mulrooney, and Miako Villanueva

    Clayton Valli, Ceil Lucas, Kristin J. Mulrooney, and Miako Villanueva. 2011. Linguistics of American Sign Language, 5th Ed.: An Introduction (fifth edition, completely revised and updated ed.). Gallaudet University Press

    2011

  62. [74]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  63. [76]

    Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A survey of controllable text generation using transformer-based pre-trained language models. Comput. Surveys 56, 3 (2023), 1–37. Adapting Non-Speech Captions with Anchored Generative Models ASSETS ’25, Oc...

    2023

  64. [519]

    doi:10.1080/10447310802142342 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/10447310802142342

    doi

  65. [2019]

    doi:10.48550/arXiv.1901.03729 arXiv:1901.03729 [cs]

    Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions. doi:10.48550/arXiv.1901.03729 arXiv:1901.03729 [cs]

    1901 doi arXiv

  66. [2020]

    arXiv preprint arXiv:2009.11462 (2020)

    RealToxicityPrompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462 (2020)

    2009 arXiv

  67. [2024]

    In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY, USA, 2024-10-27) (ASSETS ’24)

    Towards a Rich Format for Closed-Captioning. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY, USA, 2024-10-27) (ASSETS ’24). Association for Computing Machinery, 1–5. doi:10.1145/3663548.3688504

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.