Pith. sign in

REVIEW 3 major objections 6 minor 128 references

Uncertainty support for long LLM answers works better when systems help users interpret, evaluate, and act—not only show confidence scores.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:31 UTC pith:VJHSTG2F

load-bearing objection Solid HCI systems paper: multi-granular targets + TMIM stages beat sentence confidence cues on efficiency, false positives, and workload in a clean n=18 handoff study. the 3 major comments →

arxiv 2607.10604 v1 pith:VJHSTG2F submitted 2026-07-12 cs.HC

U-Lens: Supporting User Uncertainty Management in Long-Form LLM Responses

Yu Mei , Qingyue Zhuang , Jie Cai , Chang Liu , Zhi Zheng , Zhoutong Ye , Chun Yu , Yuanchun Shi This is my paper
classification cs.HC
keywords uncertainty managementlarge language modelshuman-AI interactionlong-form generationverification interfacesconfidence communicationinteractive systems
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long LLM answers bury unreliable claims, vague passages, and shaky relations inside otherwise fluent text, so readers often cannot tell what to check, why it matters, or what to do next. This paper argues that showing isolated confidence cues is not enough: people manage uncertainty as a three-stage process of locating concerns, judging them, and choosing responses. From a formative study it derives four design needs—multi-granular targets, evaluative explanations, action guidance, and prioritized interactive presentation—and builds U-Lens to turn uncertain spans into ranked, contextual inspection objects with explanations and next steps. In a limited-budget handoff verification study with 18 people, U-Lens cut wasted checks, sped verification, lowered workload, and raised perceived support across all three stages versus sentence-level support labels. The result reframes generative-AI uncertainty design around the user’s management process rather than model-centered text cues alone.

Core claim

Supporting user uncertainty management in long-form LLM responses requires organizing uncertain content into contextual multi-granular inspection targets, prioritizing them, and attaching evaluative explanations and response options—yielding better verification efficiency, effort allocation, lower workload, and stronger stage-level support than confidence-cue displays.

What carries the argument

U-Lens: a system that extracts unit- and relation-based uncertainty targets from generation signals and extractors, ranks them with a calibrated priority score, and links each target to adaptive evaluative explanation and response guidance under user-controlled filtering.

Load-bearing premise

The system’s priority scores, trained on generation signals and weak LLM-plus-search labels, sufficiently match the real errors people should spend a tight verification budget on.

What would settle it

Rerun the same limited five-check handoff task with materials where the calibrator systematically ranks true errors below non-errors; if false-positive checks, time, workload, and stage-support gains over the confidence-cue baseline disappear, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Uncertainty UIs for generative AI should treat multi-granular, contextual targets—not only sentence confidence labels—as first-class interaction objects.
  • Model-side signals are better used as inputs to prioritized, actionable hypotheses than as end-user scores alone.
  • Limited-budget verification workflows can cut wasted checks and handoff quality costs by scaffolding interpretation, evaluation, and decision together.
  • Response guidance should structure collaborative plans (what to check, how) while leaving final correctness judgment with the user.
  • Designers must expect a tradeoff: focused target lists reduce organization work but can raise oversight, autonomy, and coverage concerns.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same target-plus-guidance pattern could sit on top of citation and retrieval pipelines, managing residual uncertainty after sources are shown rather than only before.
  • Domains with relational or assumption-heavy text (code, technical docs, learning materials) may benefit more from relation-based targets than pure fact lists.
  • Longitudinal real-world use may reveal whether priority lists shrink holistic reading over time, a risk the lab handoff task only partly captures.
  • Calibrators tied to one open model’s generation signals may need re-fitting when the underlying LLM or language changes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper argues that uncertainty support for long-form LLM responses should move beyond isolated confidence cues to a user-centered process of interpretation, evaluation, and decision (framed via TMIM). A formative think-aloud study (n=12) across biography, event, and scientific explanation tasks yields four design guidelines (target representation, evaluative explanation, response guidance, interactive presentation). These are instantiated in U-Lens, which extracts multi-granular unit- and relation-based targets, prioritizes them with a generation-signal calibrator, and attaches explanations and action guidance in a three-region interface. A within-subjects evaluation (n=18) against a sentence-level low/medium/high support baseline in a five-credit handoff-verification task reports lower overall and action-to-save time, fewer false-positive checks, lower NASA-TLX mental demand/effort/frustration, higher handoff-note quality, and stronger perceived support across TMIM stages.

Significance. If the results hold under the stated scope, the paper makes a clear HCI contribution: it reframes uncertainty communication as workflow support rather than cue design, ties that framing to TMIM, and delivers a concrete system plus a controlled comparison with objective efficiency, effort-allocation, and workload measures. Strengths include a carefully counterbalanced within-subjects design, shared generation pipeline and unit-based calibrator across conditions (fairer than a no-support control), high inter-coder agreement on handoff and rationale quality (ICC ≈ 0.93–0.94), Holm-corrected NASA-TLX tests, and an explicit discussion of autonomy/coverage tradeoffs. The work is relevant to LLM verification, sensemaking, and evidence-based interfaces, and the design guidelines are reusable beyond the prototype.

major comments (3)
  1. [§6.1.1 Objective Task Outcomes] §6.1.1 / false-positive checks: The headline effort-allocation result (M=0.13 vs 1.75 false-positive checks, p<.01) is load-bearing for the claim that U-Lens improves verification efficiency and effort allocation. The manuscript defines false positives as checks on targets without actual errors but does not specify the ground-truth annotation protocol for evaluation materials (who labeled errors, at what granularity, how multi-granular U-Lens spans and relation targets were mapped to those labels, and how partial-span correctness was scored). Without that operationalization, the large FP gap is hard to audit or reproduce. Please add a precise coding scheme and, if possible, inter-annotator agreement for error labels.
  2. [§5.3 Conditions; §6.2] §5.3 Conditions and §4.2.1 / Appendix B: The baseline reuses the same unit-based calibrator, which is good for fairness, but U-Lens additionally supplies multi-granular targets, a ranked panel, detail-card explanations, and response guidance. The evaluation therefore tests the full system bundle against a minimal cue, not individual guidelines DG1–DG4. That is acceptable for a systems paper if framed as such, but several claims in §6.2 attribute stage-specific gains to particular mechanisms (priority list, detail card, recommended actions). Either add lightweight ablations/log analyses that isolate components, or revise the interpretation language so that causal credit is assigned only to the system as a whole.
  3. [Appendix B; §4.2.1] Appendix B / Table 7: The unit-based calibrator’s test AUROC is modest overall (.726) and weaker for sentence-level targets (.650), with risk prevalence varying sharply by type. Because prioritization and the baseline labels both depend on these scores, the paper should report how often high-priority targets actually contained errors in the study materials (precision@k / rank of true errors) and discuss sensitivity of the FP and inspection-precision results to calibrator quality. This does not invalidate the within-study contrast (shared scores), but it is needed to bound transfer of the efficiency claim beyond the Qwen3-8B / weak-label setup.
minor comments (6)
  1. [§3.1.2 Study Materials] §3.1.2 / materials: Clarify whether formative and evaluation materials were Chinese or English (Appendix B mentions Chinese long-form responses; main text examples are English). Language affects uncertainty localization and search behavior.
  2. [§6 Results] Figure 7–9: Report exact test statistics (e.g., Wilcoxon W/Z and effect sizes) alongside p-values for primary objective outcomes, not only means/SDs and significance stars.
  3. [§5.2 Study Materials and User Task] §5.2: The five-credit budget and forced spend are realistic for the handoff framing but should be justified more explicitly (pilot data or prior verification budgets) so readers can judge ecological validity.
  4. [§4.2.2 Uncertainty Communication] §4.1.3 / response guidance: State how the action-selection knowledge base is encoded (rules vs LLM prompt) and whether recommended actions were fixed offline for study materials.
  5. [§5 heading; throughout] Typos/consistency: “USER EV ALUATION” heading spacing; occasional mixed en-dash/hyphen in p-values; ensure Table 4 “LLM Skepticism” scale anchors are defined in text.
  6. [§2.3] Related work §2.3: A brief pointer to other process models of uncertainty (beyond TMIM) would help readers situate the choice without weakening the framing.

Circularity Check

0 steps flagged

No circular derivation: empirical HCI design-and-evaluation paper; outcomes are measured user behavior, not fitted quantities re-labeled as predictions.

full rationale

U-Lens is an HCI systems paper whose load-bearing claims rest on a formative think-aloud study, design guidelines derived from that study, a prototype that organizes uncertainty targets, and a controlled within-subjects user evaluation (n=18) against a confidence-cue baseline. The only fitted component is an offline class-balanced L2 logistic calibrator trained on web-search-assisted weak labels from 7,059 targets (120 train / 30 test responses) and used solely as a prioritization cue (S_i = 1 - P_theta); the paper does not present calibrator scores as scientific predictions of user outcomes, and the baseline reuses the same unit-based scores so relative study contrasts are not forced by the fit. Study metrics (task time, false-positive checks, NASA-TLX, stage-support ratings, handoff quality) are measured participant behavior and ratings that do not feed back into the model or guidelines. There is no self-definitional identity, no uniqueness theorem imported from overlapping authors, no ansatz smuggled via self-citation, and no renaming of a known result as a first-principles derivation. The derivation chain is therefore self-contained empirical work with no circular reduction of claim to input.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

As an HCI systems paper the load-bearing commitments are methodological and modeling choices rather than physical constants. The central empirical claim rests on the applicability of TMIM stages, the validity of the generation-signal calibrator as a prioritization proxy, the fairness of the confidence-cue baseline, and the ecological validity of the five-check handoff task on three knowledge domains.

free parameters (3)
  • priority threshold (user setting)
    User-adjustable cutoff that filters which targets appear; chosen interactively and affects which items enter the verification budget.
  • calibrator L2 regularization and class-balance weights
    Hyper-parameters of the logistic model that maps generation signals to support probability; selected via training-set CV and one-standard-error rule.
  • five verification credits budget
    Fixed experimental constraint that defines the limited-budget regime under which efficiency and false-positive metrics are measured.
axioms (4)
  • domain assumption Theory of Motivated Information Management’s three stages (interpretation, evaluation, decision) adequately describe user uncertainty management for long-form LLM responses.
    Used to structure formative analysis, design guidelines, and evaluation questionnaires (Sections 3 and 6).
  • domain assumption Generation-time token signals plus target type can be calibrated to produce useful priority scores for user inspection.
    Core of the uncertainty target construction layer (Section 4.2.1, Appendix B).
  • ad hoc to paper A sentence-level low/medium/high support label interface is a fair, representative baseline for prior cue-centered uncertainty communication.
    Chosen as the comparator in the within-subjects evaluation (Section 5.3).
  • domain assumption Web-search-assisted LLM judging yields sufficiently accurate weak labels for calibrator training.
    Labels for the 7,059-target calibration corpus (Section 4.2.1).
invented entities (2)
  • contextual multi-granular uncertainty targets (unit-based + relation-based) no independent evidence
    purpose: Turn distributed uncertainty into inspectable interaction objects that preserve original spans and relations.
    Central design object of U-Lens; extracted by rules and LLM extractors (Section 4.2.1).
  • U-Lens three-region interface (LLM Interaction View, Prioritized Uncertainty Panel, Uncertainty Detail Card) no independent evidence
    purpose: Operationalize the four design guidelines as a reading-to-action workflow.
    The concrete system artifact evaluated against the baseline (Section 4.1).

pith-pipeline@v1.1.0-grok45 · 35816 in / 3065 out tokens · 45992 ms · 2026-07-14T10:31:39.534054+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) are increasingly used to generate long-form answers for knowledge-intensive tasks, but users often struggle to decide which parts of a response deserve scrutiny, why they may be unreliable, and what to do next. Prior work on uncertainty communication has largely focused on making uncertainty visible through cues such as confidence scores, leaving less support for the broader process of managing uncertainty distributed across a long response. Through a formative study, we examine how users manage such uncertainty across three stages: interpretation, evaluation, and decision. Based on these insights, we derive design guidelines that address both stage-specific and cross-stage needs: uncertainty target representation, evaluative explanation, response guidance, and interactive presentation. We instantiate these guidelines in U-Lens, an uncertainty-management support system that organizes uncertain information in long-form responses into contextual inspection targets, prioritizes them for attention, and connects each target with evaluative context and response options. We evaluated U-Lens in a controlled within-subjects study with 18 participants, comparing it against a confidence-cue baseline. Our results show that U-Lens improved verification efficiency and effort allocation, lowered perceived workload, and strengthened perceived support across interpretation, evaluation, and decision stages. This work reframes uncertainty support for generative AI from presenting isolated, text-centered cues toward supporting the user-centered process of interpreting, evaluating, and acting on uncertain information.

Figures

Figures reproduced from arXiv: 2607.10604 by Chang Liu, Chun Yu, Jie Cai, Qingyue Zhuang, Yuanchun Shi, Yu Mei, Zhi Zheng, Zhoutong Ye.

Figure 1
Figure 1. Figure 1: Example of the think-aloud reading task. Participants highlighted text spans they perceived as uncertain from [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: U-Lens user interface. The main reading view overlays uncertainty targets in the generated response (DG1). The [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: shows an overview of the system architecture. The uncer￾tainty target construction layer constructs unit-based and relation￾based uncertainty targets identified in the formative study (Sec￾tion 3.2.1). The uncertainty communication layer transforms the extracted uncertainty targets into user-facing objects that include scores, labels, evaluation support, and response guidance. The pre￾sentation adaptation … view at source ↗
Figure 4
Figure 4. Figure 4: Calibrator training for unit-based uncertainty targets [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Evaluation study interfaces. (A) The confidence-cue baseline displays lightweight low, medium, or high estimated [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Evaluation study procedure. 6 RESULTS 6.1 Overall Task Outcomes 6.1.1 Objective Task Outcomes. Participants completed the overall task in less time with U-Lens (𝑀 = 22.32 minutes, 𝑆𝐷 = 7.56) than with the confidence-cue baseline (𝑀 = 28.81 minutes, 𝑆𝐷 = 7.38), 𝑝 < .05 (Figure 7A). Within each verification workflow, the average time from selecting an action to saving the resulting decision was also lower wi… view at source ↗
Figure 7
Figure 7. Figure 7: Objective task outcomes by condition. (A) Overall task time. (B) Time spent within each saved verification-budget [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: NASA-TLX workload ratings by condition. Compared with the baseline, U-Lens shifted participants’ ratings toward [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Perceived support for the uncertainty-management process. Participants rated U-Lens as more helpful than the [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

128 extracted references · 16 linked inside Pith

  1. [1]

    Ashraf Abdul, Christian Von Der Weth, Mohan Kankanhalli, and Brian Y Lim

  2. [2]

    InProceedings of the 2020 CHI conference on human factors in computing systems

    COGAM: measuring and moderating cognitive load in machine learning model explanations. InProceedings of the 2020 CHI conference on human factors in computing systems. 1–14

    2020

  3. [3]

    Walid A Afifi and Tamara D Afifi. 2009. Avoidance among adolescents in con- versations about their parents’ relationship: Applying the theory of motivated information management.Journal of social and personal relationships26, 4 (2009), 488–511

    2009

  4. [4]

    Walid A Afifi, Susan E Morgan, Michael T Stephenson, Chris Morse, Tyler Harrison, Tom Reichert, and Shawn D Long. 2006. Examining the decision to talk with family about organ donation: Applying the theory of motivated information management.Communication Monographs73, 2 (2006), 188–215

    2006

  5. [5]

    Walid A Afifi and Judith L Weiner. 2004. Toward a theory of motivated infor- mation management.Communication theory14, 2 (2004), 167–190

    2004

  6. [6]

    Walid A Afifi and Judith L Weiner. 2006. Seeking information about sexual health: Applying the theory of motivated information management.Human Communication Research32, 1 (2006), 35–57

    2006

  7. [7]

    Aisha Alansari and Hamzah Luqman. 2026. Large language models hallucina- tion: A comprehensive survey.Computer Science Review61 (2026), 100970

    2026

  8. [8]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations

    2023

  9. [9]

    Yiting Bai, Donghan Fu, and Lyn M van Swol. 2024. Research on interpersonal electronic surveillance in romantic relationships: Applying the theory of moti- vated information management.Asian Journal of Social Psychology27, 4 (2024), 712–726

    2024

  10. [10]

    Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. 2024. Linguistic calibration of long-form generations.arXiv preprint arXiv:2404.00474(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and ad- vances.Computational Linguistics48, 1 (2022), 207–219

    2022

  12. [12]

    Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, et al. 2021. Uncertainty as a form of transparency: Measuring, communicating, and using uncertainty. InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 401–413

    2021

  13. [13]

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258(2021)

    Pith/arXiv arXiv 2021

  14. [14]

    Dale E Brashers. 2001. Communication and uncertainty management.Journal of communication51, 3 (2001), 477–497

    2001

  15. [15]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in neural information processing systems33 (2020), 1877–1901

    2020

  16. [16]

    Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making.Proceedings of the ACM on Human-computer Interaction5, CSCW1 (2021), 1–21

    2021

  17. [17]

    Shiye Cao, Anqi Liu, and Chien-Ming Huang. 2024. Designing for appropriate reliance: The roles of AI uncertainty presentation, initial user decision, and user demographics in AI-assisted decision-making.Proceedings of the ACM on Human-Computer Interaction8, CSCW1 (2024), 1–32

    2024

  18. [18]

    Valerie Chen, Q Vera Liao, Jennifer Wortman Vaughan, and Gagan Bansal

  19. [19]

    Understanding the role of human intuition on reliance in human-AI decision-making with explanations.Proceedings of the ACM on Human-computer Interaction7, CSCW2 (2023), 1–32

    2023

  20. [20]

    Furui Cheng, Vilém Zouhar, Simran Arora, Mrinmaya Sachan, Hendrik Strobelt, and Mennatallah El-Assady. 2024. Relic: Investigating large language model responses using self-consistency. InProceedings of the 2024 CHI conference on human factors in computing systems. 1–18

    2024

  21. [21]

    Victoria Clarke and Virginia Braun. 2017. Thematic analysis.The journal of positive psychology12, 3 (2017), 297–298

    2017

  22. [22]

    Jeremy Cole, Michael Zhang, Dan Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. 2023. Selectively answering ambiguous questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 530–543

    2023

  23. [23]

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey

  24. [24]

    arXiv preprint arXiv:2309.08600(2023)

    Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600(2023)

    Pith/arXiv arXiv 2023

  25. [25]

    Mandeep K Dhami and David R Mandel. 2022. Communicating uncertainty using words and numbers.Trends in Cognitive Sciences26, 6 (2022), 514–526

    2022

  26. [26]

    Paramveer S Dhillon, Somayeh Molaei, Jiaqi Li, Maximilian Golub, Shaochun Zheng, and Lionel Peter Robert. 2024. Shaping human-AI collaboration: Varied scaffolding levels in co-writing with language models. InProceedings of the 2024 CHI conference on human factors in computing systems. 1–18

    2024

  27. [27]

    Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2024. Chain-of-verification reduces hallucina- tion in large language models. InFindings of the association for computational linguistics: ACL 2024. 3563–3578. U-Lens

    2024

  28. [28]

    Yifan Ding, Matthew Facciani, Ellen Joyce, Amrit Poudel, Sanmitra Bhat- tacharya, Balaji Veeramani, Sal Aguinaga, and Tim Weninger. 2025. Citations and trust in llm generated responses. InProceedings of the AAAI conference on artificial intelligence, Vol. 39. 23787–23795

    2025

  29. [29]

    Hyo Jin Do and Werner Geyer. 2025. Hide or Highlight: Understanding the Impact of Factuality Expression on User Trust. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 8. 785–797

    2025

  30. [30]

    Hyo Jin Do, Rachel Ostrand, Werner Geyer, Keerthiram Murugesan, Dennis Wei, and Justin Weisz. 2025. Highlight All the Phrases: Enhancing LLM Transparency through Visual Factuality Indicators. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 8. 772–784

    2025

  31. [31]

    Hyo Jin Do, Rachel Ostrand, Justin D Weisz, Casey Dugan, Prasanna Sattigeri, Dennis Wei, Keerthiram Murugesan, and Werner Geyer. 2024. Facilitating human-LLM collaboration through factuality scores and source attributions. arXiv preprint arXiv:2405.20434(2024)

    Pith/arXiv arXiv 2024

  32. [32]

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy.Nature630, 8017 (2024), 625–630

    2024

  33. [33]

    Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Cha- ganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al

  34. [34]

    InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Rarr: Researching and revising what language models say, using lan- guage models. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 16477–16508

  35. [35]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling large language models to generate text with citations. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 6465–6488

    2023

  36. [36]

    Katy Ilonka Gero, Chelse Swoopes, Ziwei Gu, Jonathan K Kummerfeld, and Elena L Glassman. 2024. Supporting sensemaking of large language model outputs at scale. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–21

    2024

  37. [37]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing.arXiv preprint arXiv:2305.11738(2023)

    Pith/arXiv arXiv 2023

  38. [38]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Nan Duan, Weizhu Chen, et al. 2024. Critic: Large language models can self-correct with tool-interactive critiquing. InInternational Conference on Learning Representations, Vol. 2024. 57734–57811

    2024

  39. [39]

    Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the- loop decision making.Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1–24

    2019

  40. [40]

    Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing.arXiv preprint arXiv:2305.01610(2023)

    Pith/arXiv arXiv 2023

  41. [41]

    Hendrik Heuer and Elena Leah Glassman. 2022. A comparative evaluation of interventions against misinformation: Augmenting the WHO checklist. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–21

    2022

  42. [42]

    Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, and Steven M Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. InProceedings of the 2019 CHI conference on human factors in computing systems. 1–13

    2019

  43. [43]

    Sungsoo Ray Hong, Jessica Hullman, and Enrico Bertini. 2020. Human factors in model interpretability: Industry practices, challenges, and needs.Proceedings of the ACM on Human-Computer Interaction4, CSCW1 (2020), 1–26

    2020

  44. [44]

    Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang. 2023. Decomposing uncertainty for large language models through input clarification ensembling.arXiv preprint arXiv:2311.08718(2023)

    Pith/arXiv arXiv 2023

  45. [45]

    Farnaz Jahanbakhsh and David R Karger. 2024. A Browser Extension for in- place Signaling and Assessment of Misinformation. InProceedings of the 2024 CHI conference on human factors in computing systems. 1–21

    2024

  46. [46]

    Farnaz Jahanbakhsh, Yannis Katsis, Dakuo Wang, Lucian Popa, and Michael Muller. 2023. Exploring the use of personalized AI for identifying misinfor- mation on social media. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–27

    2023

  47. [47]

    Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naa- man. 2023. Co-writing with opinionated language models affects users’ views. InProceedings of the 2023 CHI conference on human factors in computing systems. 1–15

    2023

  48. [48]

    Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language models for question answering.Transactions of the Association for Computational Linguistics9 (2021), 962–977

    2021

  49. [49]

    Hyoungwook Jin, Seonghee Lee, Hyungyu Shin, and Juho Kim. 2024. Teach ai how to code: Using large language models as teachable agents for program- ming education. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–28

    2024

  50. [50]

    Adam Tauman Kalai, Ofir Nachum, Santosh S Vempala, and Edwin Zhang. 2025. Why language models hallucinate.arXiv preprint arXiv:2509.04664(2025)

    Pith/arXiv arXiv 2025

  51. [51]

    Haoqiang Kang, Juntong Ni, and Huaxiu Yao. 2023. Ever: Mitigating hallucina- tion in large language models through real-time verification and rectification. arXiv preprint arXiv:2311.09114(2023)

    Pith/arXiv arXiv 2023

  52. [52]

    Elisa Kanter, Jennifer L Bevan, and Sam M Dorros. 2019. The use of online support groups to seek information about chronic illness: applying the theory of motivated information management.Communication Quarterly67, 1 (2019), 100–121

    2019

  53. [53]

    Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting interpretability: under- standing data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–14

    2020

  54. [54]

    I’m Not Sure, But

    Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jen- nifer Wortman Vaughan. 2024. " I’m Not Sure, But... ": Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust. InProceedings of the 2024 ACM conference on fairness, accountability, and trans- parency. 822–835

    2024

  55. [55]

    Sunnie SY Kim, Jennifer Wortman Vaughan, Q Vera Liao, Tania Lombrozo, and Olga Russakovsky. 2025. Fostering appropriate reliance on large language models: The role of explanations, sources, and inconsistencies. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19

    2025

  56. [56]

    Kai Kuang, Xiaoman Zhao, Iccha Basnyat, and Tianping He. 2022. From offline to online: Understanding Chinese single mothers’ uncertainty management in interpersonal and online contexts.Frontiers in Psychology13 (2022), 845760

    2022

  57. [57]

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664(2023)

    Pith/arXiv arXiv 2023

  58. [58]

    Mina Lee, Percy Liang, and Qian Yang. 2022. Coauthor: Designing a human- ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–19

    2022

  59. [59]

    Florian Leiser, Sven Eckhardt, Valentin Leuthe, Merlin Knaeble, Alexander Mäd- che, Gerhard Schwabe, and Ali Sunyaev. 2024. Hill: A hallucination identifier for large language models. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–13

    2024

  60. [60]

    Changmao Li and Jeffrey Flanigan. 2024. Rac: Efficient llm factuality correction with retrieval augmentation.arXiv preprint arXiv:2410.15667(2024)

    Pith/arXiv arXiv 2024

  61. [61]

    Jo-Yun Li, Jing Wen, Joon Kim, and Robert McKeever. 2020. Applying the theory of motivated information management to the context of conflicting online health information: implications for childhood vaccination communication with parents.International Journal of Strategic Communication14, 5 (2020), 330–347

    2020

  62. [62]

    Weitao Li, Junkai Li, Weizhi Ma, and Yang Liu. 2024. Citation-enhanced gener- ation for LLM-based chatbots. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1451–1466

    2024

  63. [63]

    Zhuoyan Li, Chen Liang, Jing Peng, and Ming Yin. 2024. The value, benefits, and concerns of generative ai-powered assistance in writing. InProceedings of the 2024 CHI conference on human factors in computing systems. 1–25

    2024

  64. [64]

    Q Vera Liao and Wai-Tat Fu. 2013. Beyond the filter bubble: interactive effects of perceived threat and topic involvement on selective exposure to information. InProceedings of the SIGCHI conference on human factors in computing systems. 2359–2368

    2013

  65. [65]

    Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: informing design practices for explainable AI user experiences. InProceedings of the 2020 CHI conference on human factors in computing systems. 1–15

    2020

  66. [66]

    Brian Y Lim and Anind K Dey. 2009. Assessing demand for intelligibility in context-aware applications. InProceedings of the 11th international conference on Ubiquitous computing. 195–204

    2009

  67. [67]

    Brian Y Lim and Anind K Dey. 2010. Toolkit to support intelligibility in context- aware applications. InProceedings of the 12th ACM international conference on Ubiquitous computing. 13–22

    2010

  68. [68]

    Brian Y Lim, Anind K Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. InProceedings of the SIGCHI conference on human factors in computing systems. 2119–2128

    2009

  69. [69]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words.arXiv preprint arXiv:2205.14334(2022)

    Pith/arXiv arXiv 2022

  70. [70]

    Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong, Jimmy Lin, Wen- tau Yih, and Xilun Chen. 2024. Flame: Factuality-aware alignment for large language models.Advances in Neural Information Processing Systems37 (2024), 115588–115614

    2024

  71. [71]

    Gabrielle Kaili-May Liu, Gal Yona, Avi Caciularu, Idan Szpektor, Tim GJ Rudner, and Arman Cohan. 2025. Metafaith: Faithful natural language uncertainty expression in LLMs. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 29600–29644

    2025

  72. [72]

    Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar, Niket Tandon, Marianna Apidianaki, Mrinmaya Sachan, and Chris Callison-Burch

  73. [73]

    InProceedings Mei et al

    Calibrating large language models with sample consistency. InProceedings Mei et al. of the AAAI Conference on Artificial Intelligence, Vol. 39. 19260–19268

  74. [74]

    Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. Selfcheckgpt: Zero- resource black-box hallucination detection for generative large language models. InProceedings of the 2023 conference on empirical methods in natural language processing. 9004–9017

    2023

  75. [75]

    Rhodes Massenon, Ishaya Gambo, Javed Ali Khan, Christopher Agbonkhese, and Ayed Alwadain. 2025. ” My AI is Lying to Me”: User-reported LLM hallucinations in AI mobile apps reviews.Scientific Reports15, 1 (2025), 30397

    2025

  76. [76]

    Yu Mei, Ziyao Zhang, Qingyang Wan, Shiyi Wang, Ge Wang, Jie Cai, Chun Yu, and Yuanchun Shi. 2026. Adapting AI to the Moment: Understanding the Dynamics of Parent-AI Collaboration Modes in Real-Time Conversations with Children.arXiv preprint arXiv:2603.27633(2026)

    arXiv 2026

  77. [77]

    Miriam J Metzger and Andrew J Flanagin. 2013. Credibility and trust of in- formation in online environments: The use of cognitive heuristics.Journal of pragmatics59 (2013), 210–220

    2013

  78. [78]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine- grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 12076–12100

    2023

  79. [79]

    Peya Mowar, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, and Jeffrey P Bigham

  80. [80]

    InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems

    Codea11y: Making ai coding assistants useful for accessible web develop- ment. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–15

    2025

Showing first 80 references.