Pith. sign in

REVIEW 4 major objections 7 minor 116 references

Reddit users report that LLM failures are the biggest risk, and each chatbot has a distinct risk fingerprint

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:59 UTC pith:YQDJAE43

load-bearing objection A useful descriptive map of Reddit-reported risks across seven chatbots, but the per-product 'risk fingerprints' rest on a chi-square test that ignores nested data. the 4 major comments →

arxiv 2509.08912 v1 pith:YQDJAE43 submitted 2025-09-10 cs.CY cs.HC

Towards Trustworthy AI: Characterizing User-Reported Risks across LLMs "In the Wild"

Lingyao Li , Renkai Ma , Zhaoqian Xue , Junjie Xiong This is my paper
classification cs.CY cs.HC
keywords LLM risksRedditNIST AI RMFuser-reported risksrisk fingerprintknowledge graphtrustworthy AIHCI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper analyzes user-reported risks of seven major LLM chatbots by mining Reddit discussions and mapping them onto the U.S. NIST AI Risk Management Framework. It finds that 'Valid and Reliable' failures dominate user complaints across all chatbots, but each product shows a unique secondary risk profile: GPT is associated with safety and fairness issues, Gemini with privacy, Claude with security and resilience, and open-source models like Qwen concentrate almost entirely on reliability. The paper argues that this user-centered view reveals a gap between the risks emphasized in technical research and the risks users actually experience, and that users cope through pragmatic, coerced, and voluntary trade-offs.

Core claim

The central claim is that user-reported risks of LLM chatbots in the wild are unevenly distributed and platform-specific. Using a knowledge graph built from 4,438 Reddit posts and 48,797 comments, the authors show that 'Valid and Reliable' risk is the most frequently mentioned category (58.39% of tagged risk mentions), while less frequent risks like 'Explainability' and 'Privacy' manifest as nuanced user trade-offs and more common risks like 'Fairness' are experienced as direct personal harms. Each of the seven chatbots exhibits a distinct 'risk fingerprint': GPT is overrepresented in 'Safe' and 'Fair' issues, Gemini in 'Privacy', Claude in 'Secure and Resilient', and Qwen almost entirely in

What carries the argument

The knowledge graph (KG) that connects LLM chatbots to NIST AI RMF categories and to granular risk types extracted via an LLM pipeline, visualized with D3.js. The KG serves as the analytical backbone, combining a top-down framework (NIST categories) with bottom-up topic modeling (BERTopic) to quantify risk prevalence and link it to qualitative user quotes.

Load-bearing premise

The paper assumes that the Reddit discussions scraped from 51 hand-selected subreddits, filtered by post popularity and comment length, are representative of what typical LLM users experience in the wild, rather than being skewed by Reddit demographics, subreddit norms, or the platform's top-post retrieval bias.

What would settle it

A direct test would be to run the same annotation pipeline on a different data source, such as app store reviews or a nationally representative survey of LLM users, and check whether 'Valid and Reliable' still dominates and whether the per-chatbot risk fingerprints (GPT safety, Gemini privacy, Claude security, Qwen reliability) replicate. If those profiles do not generalize, the paper's central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the paper is right, designers and policymakers should prioritize improving reliability and validity of LLM outputs over other risk categories, since that is what users report most.
  • Different chatbots have different risk profiles, so risk mitigation should be tailored per product rather than applied uniformly.
  • Users' lived experiences reveal that some risks are accepted as trade-offs, meaning safety and transparency interventions must respect users' pragmatic and voluntary choices.
  • The observed gap between system-centered and user-centered risk priorities suggests that technical risk research may be addressing issues that are less salient to everyday users.
  • The prevalence-based distinction between direct harms and trade-offs offers a way to triage which risks require immediate intervention versus which are matters of user preference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct extension of the risk-fingerprint idea is that platform-specific design choices (e.g., GPT's safety filters, Gemini's data practices, Claude's stability) causally shape which risks surface, not just which risks users happen to discuss.
  • The method of using an LLM to annotate LLM-related risks could introduce a self-confirming bias; a human-only coding on a larger sample would test whether the fingerprints persist.
  • The 'pragmatic, coerced, voluntary' trade-off taxonomy could be applied to other AI products (e.g., voice assistants, recommender systems) to predict when users will tolerate failures.
  • If Reddit's user base skews technical, the dominance of 'Valid and Reliable' might be inflated; a cross-platform replication (e.g., Twitter/X, Facebook, or app store reviews) would reveal whether the risk hierarchy is an artifact of the platform.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents a large-scale, mixed-methods study of user-reported risks with seven LLM chatbots by analyzing Reddit posts and comments from November 2022 through June 2025. The authors build an LLM-driven extraction pipeline, grounded in the NIST AI Risk Management Framework's seven risk characteristics, to tag 32,302 risk mentions across 4,438 posts and 48,797 comments. They combine this with BERTopic-based bottom-up clustering, an interactive knowledge graph, chi-square post-hoc residual analysis, and qualitative thematic analysis. The main claims are that user-reported risks are unevenly distributed across chatbots, that 'Valid and Reliable' dominates at 58.39% of tagged mentions, that each chatbot has a distinctive 'risk fingerprint' (e.g., GPT with Safe and Fair, Gemini with Privacy, Claude with Secure and Resilient), and that lower-frequency risks are experienced as trade-offs while higher-frequency risks are experienced as direct harms. The paper contributes an empirical mapping of user-perceived LLM risks and argues for user-centered AI risk management.

Significance. If the empirical claims hold, this is a timely and useful contribution to HCI and trustworthy-AI research. The paper addresses a genuine gap: most risk taxonomies are system-centered and lab-based, whereas this study draws on unsolicited user discourse across multiple chatbot products. The authors are appropriately transparent about their pipeline: they validate the GPT-4.1-mini annotation on 200 samples (NIST-category accuracy 0.87, Krippendorff's alpha 0.76, risk-type F1 0.83), make the knowledge graph publicly available, and provide detailed appendices with thematic codebooks. The qualitative quotes and thematic groupings give the findings concreteness. The main risk is that the central comparative claim—the 'risk fingerprint'—rests on a statistical analysis that does not account for the nested, non-independent structure of the data; this is fixable, but until addressed the fingerprints should be treated as provisional.

major comments (4)
  1. [Section 4.2, Table 2] The chi-square post-hoc analysis treats each of the 32,302 extracted risk mentions as an independent observation, but these mentions are nested within comments, threads, posts, and product-specific subreddits. The corpus has only 4,438 posts and 48,797 comments (Section 3.1); a single viral thread or the normal topic focus of r/Qwen_AI can generate many correlated mentions in one cell. The adjusted residuals and p-values are therefore anti-conservative, and the 'risk fingerprints' may reflect corpus structure rather than product-level user experience. Please re-analyze at the post or comment level (e.g., mixed-effects regression with random intercepts for thread and subreddit, or cluster-bootstrapped residuals) and report whether the Table 2 pattern survives.
  2. [Section 3.1, Table 1] The sampling design uses PRAW top posts from 51 hand-selected subreddits, including product-specific communities such as r/ChatGPT, r/ClaudeAI, and r/Qwen_AI. This creates a plausible confound: differences in subreddit norms and topics can produce cross-chatbot differences in risk mentions even if the products themselves do not differ. Section 7 acknowledges the top-post popularity bias but does not address subreddit confounding. Please add a sensitivity analysis excluding product-specific subreddits or controlling for subreddit membership; at minimum, quantify how much of the chi-square association is driven by these communities.
  3. [Section 3.2] The annotation pipeline uses GPT-4.1-mini, an OpenAI model, to classify every risk mention, including mentions about OpenAI's ChatGPT. Validation is reported only in aggregate (accuracy 0.87, alpha 0.76, risk-type F1 0.83). Given the highly imbalanced category distribution and the possibility of systematic labeling tendencies by the annotator model, aggregate metrics do not rule out differential annotation bias across chatbots or across NIST categories. Please report per-category precision/recall and, ideally, per-chatbot validation, or re-annotate a sample with a different model to assess whether the Table 2 fingerprints are stable across annotators.
  4. [Sections 5 and 6.3] The abstract and Section 6 claim that lower-prevalence risks (Explainability, Privacy) manifest as trade-offs while higher-prevalence risks (Fairness, Safety) are experienced as direct personal harms. This is a central empirical claim, but it is supported by a purposively sampled qualitative narrative rather than a systematic coding or analysis. Several categories contain both framings: Fairness includes both direct stereotyping harm and an academic-dishonesty social dilemma, and Privacy includes coerced trade-offs. If this is a substantive prevalence-dependent claim, it needs an operationalized comparison; otherwise it should be presented as an interpretive hypothesis rather than a finding.
minor comments (7)
  1. [Abstract] 'Platform-specific' is ambiguous because the data come from a single platform (Reddit). Consider 'product-specific' or 'chatbot-specific.'
  2. [Section 4.2, Table 2] The text says |r| ≥ 2 while Table 2's note says |r| > 1.96; align these. Also report the chi-square statistic, degrees of freedom, and p-value, not only the post-hoc residuals.
  3. [Section 4.2 / Section 5.7] The paper reports an average of 61.3% for 'Valid and Reliable' in Section 4.2 but 58.39% of total tagged mentions in Section 5.7 and Table 12. Clarify that the former is an unweighted mean across chatbot percentages and the latter is the overall share.
  4. [Figure 4] The numeric labels under each column are not explained in the caption or text. Add a legend or caption describing what these values represent.
  5. [Table 3] Several cells contain only 'high report volume' without the corresponding frequencies or residuals. Add exact values or point readers to Table 2.
  6. [Table 4] The Fair example contains a typo ('tras‘' instead of 'trash'). Also, 'Deepseek' in the table should be 'DeepSeek' for consistency.
  7. [Section 1] The introduction refers to 'ARF' but the rest of the paper uses 'AI RMF'; define and use one abbreviation consistently.

Circularity Check

0 steps flagged

No significant circularity; risk fingerprints are descriptive statistics from a human-validated annotation pipeline, not fitted predictions.

full rationale

The paper's central claims (prevalence of NIST risk categories, product-specific fingerprints) are derived by counting LLM-extracted risk mentions from Reddit. The extraction pipeline (GPT-4.1-mini) is validated against a 200-sample human-coded ground truth with Krippendorff's alpha = 0.76 for NIST category and F1 = 0.83 for risk type (Section 3.2). The statistical analysis (chi-square and adjusted residuals in Section 4.2) operates on the resulting contingency table; it is a descriptive post-hoc characterization, not a parameter fitted to a subset and then predicted on a closely related quantity. No equation in the paper reduces to its own input: the NIST taxonomy is a coding scheme, not derived from the data, and the frequencies are empirical outputs. Self-citations (e.g., [70], [71], [84], [99]) appear in related work and methods as background or examples of established practice; they are not load-bearing for the main result. The mild concern that the annotator (GPT-4.1-mini, an OpenAI model) might bias labels toward ChatGPT is a measurement validity threat, not circularity in the derivation chain; the human ground truth provides independent support. The pseudo-replication issue with chi-square is a statistical robustness concern, not a circularity concern. Overall, the derivation is self-contained and does not reduce to its inputs by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central prevalence claims rest on the NIST taxonomy, the representativeness of the PRAW-scraped Reddit corpus, and the fidelity of GPT-4.1-mini annotations. The paper validates the annotation on 200 samples but does not publicly release data or prompts, and it acknowledges platform and collection biases. No new physical or conceptual entities are invented.

free parameters (2)
  • BERTopic cluster count = not reported
    The optimal number of clusters is chosen by the elbow method on UMAP-reduced embeddings (Section 3.3). This choice shapes the RQ2 thematic grouping, though it does not directly affect the RQ1 prevalence percentages.
  • Minimum comment length threshold = 10 words
    A hand-set filter in Section 3.1. It excludes short comments and therefore changes the composition of the risk corpus and the frequencies used in the central claims.
axioms (4)
  • domain assumption The seven NIST AI RMF categories are an appropriate, exhaustive top-down taxonomy for user-reported LLM risks.
    Adopted in Section 3.2. Risks that do not fit neatly into these categories are folded in or dropped, and the resulting distribution is constrained by this taxonomy.
  • domain assumption Reddit discussions from 51 hand-selected subreddits, restricted to top posts via PRAW, represent 'in the wild' user risk experiences across LLM chatbots.
    Section 3.1 and Section 7 acknowledge demographic, recency, and popularity biases. If this corpus is unrepresentative, the cross-chatbot risk fingerprints could be artifacts.
  • domain assumption GPT-4.1-mini structured extractions are accurate enough to label the full dataset after validation on 200 samples.
    Section 3.2 reports validation metrics on 200 samples only. The annotator is itself an LLM whose priors may correlate with the systems being studied, and rare risk types may not be covered by the validation sample.
  • domain assumption Each extracted risk mention is an independent observation for the Chi-square test.
    Section 4.2. Multiple risk mentions from one post or comment and the nested subreddit structure violate independence, so p-values and residuals should be treated as directional diagnostics.

pith-pipeline@v1.3.0-alltime-deepseek · 29140 in / 13446 out tokens · 154585 ms · 2026-08-04T19:59:20.637765+00:00 · methodology

0 comments
read the original abstract

While Large Language Models (LLMs) are rapidly integrating into daily life, research on their risks often remains lab-based and disconnected from the problems users encounter "in the wild." While recent HCI research has begun to explore these user-facing risks, it typically concentrates on a singular LLM chatbot like ChatGPT or an isolated risk like privacy. To gain a holistic understanding of multi-risk across LLM chatbots, we analyze online discussions on Reddit around seven major LLM chatbots through the U.S. NIST's AI Risk Management Framework. We find that user-reported risks are unevenly distributed and platform-specific. While "Valid and Reliable" risk is the most frequently mentioned, each product also exhibits a unique "risk fingerprint;" for instance, user discussions associate GPT more with "Safe" and "Fair" issues, Gemini with "Privacy," and Claude with "Secure and Resilient" risks. Furthermore, the nature of these risks differs by their prevalence: less frequent risks like "Explainability" and "Privacy" manifest as nuanced user trade-offs, more common ones like "Fairness" are experienced as direct personal harms. Our findings reveal gaps between risks reported by system-centered studies and by users, highlighting the need for user-centered approaches that support users in their daily use of LLM chatbots.

Figures

Figures reproduced from arXiv: 2509.08912 by Junjie Xiong, Lingyao Li, Renkai Ma, Zhaoqian Xue.

Figure 1
Figure 1. Figure 1: The illustrative framework to implement this study. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Interactive KG visualization of user-reported LLM risks extracted from Reddit data. (a) The hierarchical relationships [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The distribution of user-reported LLM risks. (a) A stacked bar chart showing the percentage breakdown of reported [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: A categorical analysis of granular user-reported risks. The dot plot visualizes the frequency of specific risk sub [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

116 extracted references · 2 canonical work pages

  1. [1]

    Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 298–306

    2021

  2. [2]

    Mark S Ackerman. 2000. The intellectual challenge of CSCW: the gap between social requirements and technical feasibility.Human–Computer Interaction15, 2-3 (2000), 179–203

    2000

  3. [3]

    Alessandro Acquisti, Laura Brandimarte, and George Loewenstein. 2015. Privacy and human behavior in the age of information.Science347, 6221 (2015), 509– 514

    2015

  4. [4]

    Alessandro Acquisti and Jens Grossklags. 2005. Privacy and rationality in individual decision making.IEEE security & privacy3, 1 (2005), 26–33

    2005

  5. [5]

    NIST AI. 2023. Artificial intelligence risk management framework (AI RMF 1.0). URL: https://nvlpubs. nist. gov/nistpubs/ai/nist. ai(2023), 100–1

    2023

  6. [6]

    Rohan Ajwani, Shashidhar Reddy Javaji, Frank Rudzicz, and Zining Zhu. 2024. LLM-generated black-box explanations can be adversarially helpful.arXiv preprint arXiv:2405.06800(2024)

    Pith/arXiv arXiv 2024

  7. [7]

    Daniel Alexander Alber, Zihao Yang, Anton Alyakin, Eunice Yang, Sumedha Rai, Aly A Valliani, Jeff Zhang, Gabriel R Rosenbaum, Ashley K Amend-Thomas, David B Kurland, et al. 2025. Medical large language models are vulnerable to data-poisoning attacks.Nature Medicine31, 2 (2025), 618–626

    2025

  8. [8]

    Shahad Alkamli and Reham Alabduljabbar. 2024. Understanding privacy con- cerns in ChatGPT: A data-driven approach with LDA topic modeling.Heliyon 10, 20 (2024)

    2024

  9. [9]

    Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoe Juozapaitis, Evan Newman, Jed Irvine, Souti Chattopadhyay, Matthew Olson, Alan Fern, and Margaret Burnett. 2020. Mental models of mere mortals with explanations of reinforcement learning.ACM Transactions on Interactive Intelligent Systems (TiiS)10, 2 (2020), 1–37

    2020

  10. [10]

    Evan Bailyn. 2025. Top Generative AI Chatbots by Market Share – August

    2025

  11. [11]

    Yonatan Belinkov and James Glass. 2019. Analysis methods in neural lan- guage processing: A survey.Transactions of the Association for Computational Linguistics7 (2019), 49–72

    2019

  12. [12]

    Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big?. InProceedings of the 2021 ACM conference on fairness, accountability, and transparency. 610–623

    2021

  13. [13]

    Federico Bianchi and James Zou. 2024. Large language models are vulnerable to bait-and-switch attacks for generating harmful content.arXiv preprint arXiv:2402.13926(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Noémi Bontridder and Yves Poullet. 2021. The role of artificial intelligence in disinformation.Data & Policy3 (2021), e32

    2021

  15. [15]

    Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3Data-Driven Documents.IEEE Transactions on Visualization and Computer Graphics17, 12 (2011), 2301–2309. doi:10.1109/TVCG.2011.185

    doi:10.1109/tvcg.2011.185 2011

  16. [16]

    Samuel R Bowman. 2024. Eight things to know about large language models. Critical AI2, 2 (2024)

    2024

  17. [17]

    Anna Brown, Alexandra Chouldechova, Emily Putnam-Hornstein, Andrew Tobin, and Rhema Vaithianathan. 2019. Toward algorithmic accountability in public services: A qualitative study of affected community perspectives on algorithmic decision-making in child welfare services. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12

    2019

  18. [18]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In30th USENIX security symposium (USENIX Security 21). 2633–2650

    2021

  19. [19]

    Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. InProceedings of the 21th ACM SIGKDD inter- national conference on knowledge discovery and data mining. 1721–1730

    2015

  20. [20]

    Qijia Chen, Seyed Mahed Mousavi, Giuseppe Riccardi, and Giulio Jacucci. 2025. Investigating the Use and Perception of Blocking Feature in Social Virtual Reality Spaces: A Study on Discussion Forums.Proceedings of the ACM on Human-Computer Interaction9, 2 (2025), 1–19

    2025

  21. [21]

    Su-Yen Chen, HY Kuo, and Shu-Hao Chang. 2024. Perceptions of ChatGPT in healthcare: usefulness, trust, and risk.Frontiers in Public Health12 (2024), 1457131

    2024

  22. [22]

    Zenan Chen and Jason Chan. 2024. Large language model in creative work: The role of collaboration modality and user expertise.Management Science70, 12 (2024), 9101–9117. Characterizing User-Reported Risks Across LLM chatbots

    2024

  23. [23]

    Avishek Choudhury and Hamid Shamszare. 2023. Investigating the impact of user trust on the adoption and use of ChatGPT: survey analysis.Journal of Medical Internet Research25 (2023), e47184

    2023

  24. [24]

    Gabriele Civitarese, Michele Fiori, Priyankar Choudhary, and Claudio Bettini

  25. [25]

    Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A Smith. 2021. All that’s’ human’is not gold: Evaluating human evaluation of generated text.arXiv preprint arXiv:2107.00061(2021)

    Pith/arXiv arXiv 2021

  26. [26]

    Large language models are zero-shot recognizers for activities of daily living.ACM Transactions on Intelligent Systems and Technology16, 4 (2025), 1–32

    2025

  27. [27]

    Dibble, Tilo Hartmann, and Sarah F

    Jayson L. Dibble, Tilo Hartmann, and Sarah F. Rosaen. 2016. Parasocial In- teraction and Parasocial Relationship: Conceptual Clarification and a Critical Assessment of Measures.Human Communication Research42, 1 (1 2016), 21–44. doi:10.1111/HCRE.12063

    work page doi:10.1111/hcre.12063 2016

  28. [28]

    Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models.arXiv preprint arXiv:2304.05335(2023)

    Pith/arXiv arXiv 2023

  29. [29]

    Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of inter- pretable machine learning.arXiv preprint arXiv:1702.08608(2017)

    Pith/arXiv arXiv 2017

  30. [30]

    Hyo Jin Do, Ha-Kyung Kong, Pooja Tetali, Jaewook Lee, and Brian P Bailey

  31. [31]

    ChatGPT Europol. 2023. The impact of large language models on law enforce- ment.Europa. Eu(2023)

    2023

  32. [32]

    Feuston, Alex S

    Jessica L. Feuston, Alex S. Taylor, and Anne Marie Piper. 2020. Conformity of Eating Disorders through Content Moderation.Proceedings of the ACM on Human-Computer Interaction4, CSCW1 (5 2020). doi:10.1145/3392845

    work page doi:10.1145/3392845 2020

  33. [33]

    ET Online. 2025. Top AI tools of 2025: Is ChatGPT still leading or is Gemini, Grok, DeepSeek taking over? https://economictimes.indiatimes.com/news/ international/us/top-ai-tools-of-2025-is-chatgpt-still-leading-or-is-gemini- grok-deepseek-taking-over/articleshow/123552050.cms

    arXiv 2025

  34. [34]

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. 2024. Bias and fairness in large language models: A survey.Computational Linguistics50, 3 (2024), 1097–1179

    2024

  35. [35]

    Shubham Gandhi, Manasi Patwardhan, Lovekesh Vig, and Gautam Shroff. 2024. Budgetmlagent: A cost-effective llm multi-agent system for automating machine learning tasks. InProceedings of the 4th International Conference on AI-ML Systems. 1–9

    2024

  36. [36]

    Casey Fiesler. 2019. Ethical considerations for research involving (speculative) public data.Proceedings of the ACM on Human-Computer Interaction3, GROUP (2019), 1–13

    2019

  37. [37]

    Nina Gerber, Paul Gerber, and Melanie Volkamer. 2018. Explaining the privacy paradox: A systematic review of literature investigating privacy attitude and behavior.Computers & security77 (2018), 226–261

    2018

  38. [38]

    Josh A Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. 2023. Generative language models and automated influ- ence operations: Emerging threats and potential mitigations.arXiv preprint arXiv:2301.042461 (2023)

    Pith/arXiv arXiv 2023

  39. [39]

    Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models.arXiv preprint arXiv:2009.11462(2020)

    Pith/arXiv arXiv 2020

  40. [40]

    Johanna Gunawan, Amogh Pradeep, David Choffnes, Woodrow Hartzog, and Christo Wilson. 2021. A comparative study of dark patterns across web and mobile modalities.Proceedings of the ACM on Human-Computer Interaction5, CSCW2 (2021), 1–29

    2021

  41. [41]

    Don’t Forget the Teachers

    Emma Harvey, Allison Koenecke, and Rene F Kizilcec. 2025. " Don’t Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Education. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19

    2025

  42. [42]

    Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class- based TF-IDF procedure.arXiv preprint arXiv:2203.05794(2022)

    Pith/arXiv arXiv 2022

  43. [43]

    Lyndsey Hipgrave, Jessie Goldie, Simon Dennis, and Amanda Coleman. 2025. Balancing risks and benefits: clinicians’ perspectives on the use of generative AI chatbots in mental healthcare.Frontiers in Digital Health7 (2025), 1606291

    2025

  44. [44]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems43, 2 (2025), 1–55

    2025

  45. [45]

    Ilka Hein and Sarah Diefenbach. 2025. Towards a Comprehensive View on Tech- nology Transparency: A Cross-Technology Investigation of Psychological and Design Factors Around Users’ Transparency Need and Perception.International Journal of Human–Computer Interaction(2025), 1–18

    2025

  46. [46]

    Sarthak Jain and Byron C Wallace. 2019. Attention is not explanation.arXiv preprint arXiv:1902.10186(2019)

    Pith/arXiv arXiv 2019

  47. [47]

    Ammar Abdulrahman Jairoun, Sabaa Saleh Al-Hemyari, Moyad Shahwan, Tariq Al-Qirim, and Monzer Shahwan. 2024. Benefit–risk assessment of chatgpt appli- cations in the field of diabetes and metabolic illnesses: a qualitative study.Clini- cal Medicine Insights: Endocrinology and Diabetes17 (2024), 11795514241235514

    2024

  48. [48]

    Lujain Ibrahim, Luc Rocher, and Ana Valdivia. 2024. Characterizing and mod- eling harms from interactions with design patterns in AI interfaces.arXiv preprint arXiv:2404.11370(2024)

    Pith/arXiv arXiv 2024

  49. [49]

    Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung

  50. [50]

    Yongnam Jung, Cheng Chen, Eunchae Jang, and S Shyam Sundar. 2024. Do we trust Chatgpt as much as Google search and Wikipedia?. InExtended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–9

    2024

  51. [51]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation.ACM computing surveys55, 12 (2023), 1–38

    2023

  52. [52]

    I’m Not Sure, But

    Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jen- nifer Wortman Vaughan. 2024. " I’m Not Sure, But... ": Examining the Impact of Large Language Models’ Uncertainty Expression on User Reliance and Trust. InProceedings of the 2024 ACM conference on fairness, accountability, and trans- parency. 822–835

    2024

  53. [53]

    InFindings of the Association for Computational Linguistics: EMNLP 2023

    Towards mitigating LLM hallucination via self reflection. InFindings of the Association for Computational Linguistics: EMNLP 2023. 1827–1843

    2023

  54. [54]

    Thomas C King, Nikita Aggarwal, Mariarosaria Taddeo, and Luciano Floridi

  55. [55]

    Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022. Deduplicating training data mitigates privacy risks in language models. InInternational Conference on Machine Learning. PMLR, 10697–10707

    2022

  56. [56]

    If the machine is as good as me, then what use am I?

    Charlotte Kobiella, Yarhy Said Flores López, Franz Waltenberger, Fiona Draxler, and Albrecht Schmidt. 2024. " If the machine is as good as me, then what use am I?"–How the use of ChatGPT changes young professionals’ perception of productivity and accomplishment. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–16

    2024

  57. [57]

    Yoonsu Kim, Jueon Lee, Seoyoung Kim, Jaehyuk Park, and Juho Kim. 2024. Understanding users’ dissatisfaction with chatgpt responses: Types, resolving tactics, and the effect of knowledge level. InProceedings of the 29th International Conference on Intelligent User Interfaces. 385–404

    2024

  58. [58]

    Hannah R Lawrence, Renee A Schneider, Susan B Rubin, Maja J Matarić, Daniel J McDuff, and Megan Jones Bell. 2024. The opportunities and risks of large language models in mental health.JMIR Mental Health11, 1 (2024), e59479

    2024

  59. [59]

    Joseph Lee, Tianqi Shang, Jae Young Baik, Duy Duong-Tran, Shu Yang, Lingyao Li, and Li Shen. 2025. Investigating LLMs in Clinical Triage: Promising Capabil- ities, Persistent Intersectional Biases.arXiv preprint arXiv:2504.16273(2025)

    Pith/arXiv arXiv 2025

  60. [60]

    René F Kizilcec. 2016. How much information? Effects of transparency on trust in an algorithmic interface. InProceedings of the 2016 CHI conference on human factors in computing systems. 2390–2395

    2016

  61. [61]

    Min Kyung Lee, Ji Tae Kim, and Leah Lizarondo. 2017. A Human-Centered Approach to Algorithmic Services: Considerations for Fair and Motivating Smart Community Service Management that Allocates Donations to Non-Profit Organizations.Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(2017). doi:10.1145/3025453

    doi:10.1145/3025453 2017

  62. [62]

    Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereo- types in large language models. InProceedings of the ACM collective intelligence conference. 12–24

    2023

  63. [63]

    Lingyao Li, Jiayan Zhou, Zhenxiang Gao, Wenyue Hua, Lizhou Fan, Huizi Yu, Loni Hagen, Yongfeng Zhang, Themistocles L Assimes, Libby Hemphill, et al

  64. [64]

    Megan Li, Wendy Bickersteth, Ningjing Tang, Jason Hong, Lorrie Cranor, Hong Shen, and Hoda Heidari. 2025. A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents.arXiv preprint arXiv:2505.22073(2025)

    Pith/arXiv arXiv 2025

  65. [65]

    Cha, Shashank Ojha, and Daniel Kusbit

    Min Kyung Lee, Anuraag Jain, Hae J.I.N. Cha, Shashank Ojha, and Daniel Kusbit

  66. [66]

    Q Vera Liao and Jennifer Wortman Vaughan. 2023. Ai transparency in the age of llms: A human-centered research roadmap.arXiv preprint arXiv:2306.01941 10 (2023)

    Pith/arXiv arXiv 2023

  67. [67]

    Aristidis Likas, Nikos Vlassis, and Jakob J Verbeek. 2003. The global k-means clustering algorithm.Pattern recognition36, 2 (2003), 451–461. Lingyao Li, Renkai Ma, Zhaoqian Xue, and Junjie Xiong

    2003

  68. [68]

    Lingyao Li, Runlong Yu, Qikai Hu, Bowei Li, Min Deng, Yang Zhou, and Xi- aowei Jia. 2025. From Pixels to Places: A Systematic Benchmark for Evaluat- ing Image Geolocalization Ability in Large Language Models.arXiv preprint arXiv:2508.01608(2025)

    Pith/arXiv arXiv 2025

  69. [69]

    Minqian Liu, Zhiyang Xu, Xinyi Zhang, Heajun An, Sarvech Qadir, Qi Zhang, Pamela J Wisniewski, Jin-Hee Cho, Sang Won Lee, Ruoxi Jia, et al. 2025. LLM can be a dangerous persuader: Empirical study of persuasion safety in large language models.arXiv preprint arXiv:2504.10430(2025)

    Pith/arXiv arXiv 2025

  70. [70]

    How advertiser-friendly is my video?

    Renkai Ma and Yubo Kou. 2021. “How advertiser-friendly is my video?”: YouTu- ber’s Socioeconomic Interactions with Algorithmic Content Moderation.PACM on Human Computer Interaction5, CSCW2 (2021), 1–26. doi:10.1145/3479573

    doi:10.1145/3479573 2021

  71. [71]

    Renkai Ma, Yue You, Xinning Gui, and Yubo Kou. 2023. How Do Users Experi- ence Moderation?: A Systematic Literature Review.PACM on Human-Computer Interaction, Vol. 7, No. CSCW27, CSCW2 (2023). doi:10.1145/3610069

    doi:10.1145/3610069 2023

  72. [72]

    Tianshi Li, Sauvik Das, Hao-Ping Lee, Dakuo Wang, Bingsheng Yao, and Zhip- ing Zhang. 2024. Human-centered privacy research in the age of large language models. InExtended Abstracts of the CHI Conference on Human Factors in Com- puting Systems. 1–4

    2024

  73. [73]

    Arunesh Mathur, Gunes Acar, Michael J Friedman, Eli Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019. Dark patterns at scale: Findings from a crawl of 11K shopping websites.Proceedings of the ACM on human-computer interaction3, CSCW (2019), 1–32

    2019

  74. [74]

    Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform man- ifold approximation and projection for dimension reduction.arXiv preprint arXiv:1802.03426(2018)

    Pith/arXiv arXiv 2018

  75. [75]

    Bowen Liu, Boao Xiao, Xutong Jiang, Siyuan Cen, Xin He, and Wanchun Dou

  76. [76]

    Adversarial Attacks on Large Language Model-Based System and Mit- igating Strategies: A Case Study on ChatGPT.Security and Communication Networks2023, 1 (2023), 8691095

    2023

  77. [77]

    Savvas Petridis, Nicholas Diakopoulos, Kevin Crowston, Mark Hansen, Keren Henderson, Stan Jastrzebski, Jeffrey V Nickerson, and Lydia B Chilton. 2023. Anglekindling: Supporting journalistic angle ideation with large language mod- els. InProceedings of the 2023 CHI conference on human factors in computing systems. 1–16

    2023

  78. [78]

    2024.PRA W: The Python Reddit API Wrapper

    PRAW Development Team. 2024.PRA W: The Python Reddit API Wrapper. https: //github.com/praw-dev/praw

    2024

  79. [79]

    Nicholas Proferes, Naiyan Jones, Sarah Gilbert, Casey Fiesler, and Michael Zimmer. 2021. Studying reddit: A systematic overview of disciplines, approaches, methods, and ethics.Social Media+ Society7, 2 (2021), 20563051211019004

    2021

  80. [80]

    Rhodes Massenon, Ishaya Gambo, Javed Ali Khan, Christopher Agbonkhese, and Ayed Alwadain. 2025. ” My AI is Lying to Me”: User-reported LLM hallucinations in AI mobile apps reviews.Scientific Reports15, 1 (2025), 30397

    2025

Showing first 80 references.