Pith. sign in

REVIEW 2 major objections 7 minor 211 references

What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research

T0 review · 2 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A synthesis of 161 empirical studies finds that responsible AI practice in industry has matured in awareness, professionalization, and tool adoption, while gaps in training, support, and tailored interventions persist.

desk verdict A solid, comprehensive review of 161 empirical RAI studies with an open database, but the abstract's 'meaningful progress' conclusion rests on a temporal comparison that is weaker than it looks. read the letter →

arxiv 2608.10431 v1 pith:XLZF7D3V submitted 2026-08-11 cs.HC cs.AI

Wesley Hanwen Deng , Agathe Balayn , Andrew Selbst , Jason I. Hong , Motahhare Eslami , Kenneth Holstein , Hanna Wallach , Jennifer Wortman Vaughan
show 1 more author
Solon Barocas
This is my paper · ORCID
classification cs.HCcs.AI
keywords responsibleAIindustrypracticeempiricalliteraturereviewfairnessgovernancepractitionerchallengesRAIinterventionsthematicsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper synthesizes six years (2019-2025) of empirical research on how industry practitioners actually do responsible AI (RAI) work, drawing on 161 studies that used interviews, surveys, workshops, and other methods. It claims that the literature as a whole shows two simultaneous trends: practitioners' awareness of RAI has grown, RAI roles are becoming professionalized, and toolkits, guidelines, and policies are being adopted; yet the same practitioners still face limited training, uneven organizational support, and interventions poorly matched to their workflows. The authors argue that the field now has enough accumulated evidence to move from cataloging problems toward designing and field-testing RAI interventions as end-to-end sociotechnical systems embedded in real workflows.

What carries the argument

The load-bearing mechanism is a systematically assembled and manually annotated corpus of 161 empirical papers, combined with a temporal comparison. The authors marked each paper for descriptive features (methods, participants) and findings, then clustered findings thematically into four progress themes (awareness, professionalization, intervention adoption, external engagement) and four challenge themes (knowledge gaps, organizational dynamics, tailoring of interventions, engagement barriers). The temporal comparison between studies published before and after roughly 2022 is what carries the 'progress' claims; the prevalence counts across the corpus (e.g., more than 120 papers reporting knowledge gaps) carry the 'persistent challenges' claims.

What would settle it

A matched-sample replication: take the same survey or interview protocol used in a 2019-2021 study (for example, the fairness-related practitioner survey), recruit a comparable sample of practitioners from the same roles and regions in 2024-2025, and check whether awareness and formalized RAI practices are actually higher. If awareness and practice levels match the earlier results, the paper's temporal progress claim would be refuted. A second check would compare the pre-2022 and post-2022 subsets of the corpus after excluding studies from EU-regulated contexts; if the progress signal disappears, regulation-driven geography, not general maturation, explains the trend.

Watch

Extended reading notes

Core claim

The central claim is that, across 161 empirical studies, the evidence supports a reading of industry RAI as both progressing and persistently challenged. Earlier studies (roughly 2019-2022) describe ad hoc, advocacy-driven fairness work, practitioners unaware of RAI concepts, and tools rarely used; later studies (2023 onward) report practitioners who can articulate RAI's importance, dedicated RAI roles with formal responsibilities, routine fairness testing and documentation, and active adoption and customization of toolkits and guidelines. The same corpus, however, consistently documents that more than 120 papers mention inadequate RAI knowledge and training, that organizational resource constraints remain the most salient barrier, and that RAI interventions are often too abstract or output-focused for the domains, applications, and pipeline stages where practitioners work. The paper's contribution is a consolidated, practice-grounded account of this literature, organized into progress areas and persistent challenges, with implications for researchers, practitioners, and policymakers.

Load-bearing premise

The paper's 'progress' conclusions depend on comparing studies published before and after about 2022 as if they measure the same population; if the later studies recruited different roles, company sizes, or regions, the apparent improvement could be an artifact of sampling rather than a real change.

Editorial extensions

If this is right

  • If the synthesis is correct, future RAI research should shift from documenting known problems to co-designing and evaluating interventions with practitioners in real workflows.
  • Organizations should treat RAI capability as infrastructure: dedicated roles, ongoing cross-role training, and integration into existing pipelines rather than parallel compliance work.
  • Regulators should prioritize implementability and substantive accountability over procedural check-box compliance, and use anticipated regulation as a signal that shapes organizational capacity-building.
  • Practitioners' external engagement with users, domain experts, and annotators needs formal infrastructures, not ad hoc outreach, to be meaningful and sustainable.
  • The persistence of the same challenges across roles and years suggests that isolated tool design will not solve RAI; attention must move to organizational incentives and supply-chain accountability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the temporal progress is real, a testable prediction follows: matched-sample replications of early studies (for example, the 2019 fairness survey) should show higher baseline awareness and more formalized practices; as of this review, only one such direct comparison exists.
  • The review's geographic skew implies that its 'progress' story may be specific to Western, often US/UK/European, technology hubs; the same literature may not yet support claims about RAI maturation in Asian or Global South contexts.
  • The emphasis on generative AI as under-studied suggests an extension: empirical studies of frontier labs and LLM supply chains may reveal that awareness and professionalization claims do not carry over to foundation-model development, where accountability is fragmented across organizations.
  • A practical extension would be to treat the open-source database as a living resource, re-running the temporal analysis at later cutoff dates to see whether the post-2022 trends continue or plateau.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. This manuscript reports a systematic literature review of 161 empirical studies (published 2019 through August 2025) that engage industry practitioners in responsible AI (RAI) work. The authors searched computer science and specialized databases with keyword queries and citation snowballing, then manually annotated each paper for methods, findings, and implications. The synthesis is organized around four areas of reported progress (increased RAI awareness, professionalization of RAI roles and practices, wider adoption of policies/processes/toolkits, and initial external stakeholder engagement) and four persistent challenges (insufficient RAI knowledge and training, organizational resource constraints and misalignment, lack of tailored interventions, and structural barriers to external engagement). The paper concludes with implications for researchers, practitioners, and policymakers, and releases an open-source database of the annotated corpus.

Significance. If the synthesis is correct, this is the most comprehensive and practice-grounded account of empirical RAI-in-industry research to date, consolidating a dispersed body of work across HCI, software engineering, and AI ethics venues. The paper's strengths include a transparent search and annotation protocol, an open-source database that supports verification and reuse, explicit acknowledgment of search truncation (Section 3.1) and reliance on self-reported data, and a clear separation between synthesized findings and forward-looking recommendations. The review also identifies concrete research gaps (e.g., understudied generative-AI contexts, limited geographic diversity, narrow role coverage) that are well supported by the corpus. The central claims about 'meaningful progress' versus persistent challenges are plausible and broadly consistent with the cited literature, but the temporal progress claims rest on a comparability assumption that the manuscript does not adequately defend, and several quantitative prevalence statements lack the supporting breakdowns needed to assess their robustness.

major comments (2)
  1. [Sections 5.1.1, 5.2.1, 5.3, 5.4; Abstract] The claim that awareness, professionalization, and adoption of RAI interventions have increased over time is inferred by comparing studies published before and after roughly 2022, but the earlier and later corpora differ systematically in composition. The early period contains 34 papers (2019–2022) versus 107 from 2023 onward, and Section 5.2 itself documents that recent studies increasingly report dedicated RAI roles, while many post-2023 studies deliberately recruit RAI specialists (e.g., Madaio et al., 2024; Smith et al., 2025) or practitioners who opted into RAI work. Such samples would report higher awareness and more formalized practices regardless of any sector-wide shift. The review does not stratify its temporal comparisons by participant recruitment criteria, role, organization size, or geography, nor does it test whether the observed pattern survives such stratification. Because the abstract's headline 'meaningful progress' conclusion depends on these temporal contrasts, the authors should either provide a compositional analysis (e.g., a table showing role and recruitment distributions per period) or substantially soften the progress claims and explicitly frame them as suggestive evidence conditional on sample comparability. The current Section 6.5 limitations paragraph does not mention this threat.
  2. [Sections 6.1, 6.2, 6.3] The review repeatedly anchors its 'persistent challenges' narrative in prevalence counts (e.g., 'more than 120 papers' for lack of expertise, 'more than 120 papers' for lack of prioritization, 'more than 100 papers' for untailored interventions, 'nearly all papers' for organizational support), but it never reports the exact numerators or a table of thematic frequencies broken down by year, role, or method. Since the open-source database could support such counts, the manuscript should include a supplementary table with these numbers and the criteria used to define each theme. Without this, the reader cannot assess the robustness of the prevalence claims or the proportionality of the synthesis relative to the underlying evidence.
minor comments (7)
  1. [Section 4] The text 'around 80% fo the paper' contains a typo; it should read 'of the papers.'
  2. [Section 4] The interview participant statistics (mean = 27, SD = 25.79, range = 12–55) are mathematically inconsistent: for a bounded variable with range 12–55, the maximum possible standard deviation is (55−12)/2 = 21.5. Please verify all descriptive statistics and correct any data entry or reporting errors.
  3. [Section 6.2.1] In the list of four motivators, the third item reads 'attending to the demand on RAI from end users []' with an empty citation bracket; a supporting reference should be supplied or the bracket removed.
  4. [Section 7.2.2] The citation placeholder '[87?]' should be resolved to the intended reference (e.g., Hurst et al., GPT-4o system card) or removed.
  5. [Section 6 (introductory paragraph)] The sentence 'Figure 2 provides an overview of the identified challenges and opportunities, mapped to the practices we observed' refers to Figure 2, but Figure 2 is the paper review process diagram in Section 3.1; the overview described here appears to correspond to Figure 1.
  6. [Section 3.1] The first-300-results truncation is acknowledged, but the authors do not estimate how many potentially relevant papers might have been missed by this procedure; a brief sensitivity note would help readers calibrate the corpus's completeness.
  7. [Section 3.1] The seed set was assembled from the authors' own knowledge, and all authors have previously published papers meeting the inclusion criteria. The manuscript does not report how many of the 45 seed papers or of the final 161 are authored or co-authored by the review team. Given the visible presence of the authors' own work in the corpus (e.g., refs [55,57,83,119,120,121,174]), a disclosure of the self-citation count would increase transparency.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the review synthesizes 161 external empirical studies and does not derive its conclusions from fitted parameters or self-cited definitions.

full rationale

This paper is a literature review rather than a quantitative derivation. Its central claims (increased RAI awareness, professionalization of RAI roles, growing adoption of RAI interventions) are synthesized from 161 distinct empirical studies; the review does not fit a parameter to a subset of data and then rename that fit as a prediction. The seed set described in Section 3.1 was assembled from the authors' own knowledge and includes some of their prior papers, and self-citations such as [55,56,57,83,146] appear throughout the synthesis. However, these self-citations are used as ordinary empirical evidence within a much larger corpus that was expanded through keyword searches and snowballing, and the main conclusions are also supported by many studies with no author overlap (e.g., [7,40,73,119,174]). No uniqueness theorem is imported from the authors' prior work, and no definition is constructed so that a later 'finding' holds by definition. The temporal comparisons in Section 5 (e.g., Section 5.1.1 and Section 5.2) may be subject to sampling-composition threats, but that is a validity concern about the underlying studies, not circularity: the earlier and later papers are distinct inputs to the review, not outputs of the review. Because no circular step can be exhibited with a specific reduction of a conclusion to an input, the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

This is a literature review; it introduces no fitted parameters or hypothetical entities. Its load-bearing assumptions are about corpus representativeness and the validity of synthesizing self-reported data.

assumptions (3)
  • domain assumption The 161-paper corpus is representative of the empirical RAI literature.
    The review's prevalence and trend claims rely on corpus coverage, which is shaped by seed papers, keyword searches, and truncation to the first 300 results in specialized databases (Section 3.1).
  • domain assumption Self-reported practitioner experiences in interviews and surveys accurately reflect real RAI practices.
    The synthesis aggregates participants' accounts as evidence of practice, though self-reports can be aspirational or biased (Sections 5 and 6).
  • domain assumption Qualitative thematic analysis across heterogeneous studies yields reliable cross-study themes.
    The authors clustered annotations into themes in a multi-step process (Section 3.3), which depends on researcher judgment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research." pith.science (2026). https://pith.science/paper/XLZF7D3V

@misc{pith2026260810431,
  author       = {Pith},
  title        = {Pith review of: What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XLZF7D3V}},
  note         = {Machine review of arXiv:2608.10431}
}
read the original abstract

Responsible AI (RAI) has become a central concern for technology companies, regulators, and the public. How industry practitioners interpret, implement, and sustain RAI work directly shapes the design and deployment of AI systems. As empirical scholarship examining RAI practices in industry has rapidly expanded, findings are dispersed across studies that focus on different roles, organizational contexts, and interventions. This work synthesizes current knowledge through a literature review of 161 empirical studies spanning six years, each engaging industry practitioners via interviews, surveys, workshops, ethnographies, and other methods. Our synthesis reveals both meaningful progress and persistent challenges in industry RAI practice. Practitioner awareness has increased, RAI activities have become more professionalized, and interventions such as toolkits and guidelines are more widely adopted. At the same time, practitioners continue to face substantial barriers, including limited training, uneven organizational support, and a lack of interventions tailored to day-to-day work practices. By consolidating and organizing these findings, we provide a more complete account of industry RAI than any single study to date. We conclude by discussing implications for RAI researchers, practitioners seeking to adopt effective practices, and policymakers aiming to ground governance efforts in the realities of industry contexts.

Figures

Figures reproduced from arXiv: 2608.10431 by the authors.

Figure 1
Figure 1. An overview of the main findings of our review of 161 empirical works on industry RAI practice. We [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An overview of our paper review process research on industry RAI practices, we followed a literature review process inspired by prior work [67, 112, 117, 175], as described below. 3.1 Collecting Publications To develop our corpus of papers, we first generated an initial corpus using a mix of a manually assembled collection of seed papers (a “seed set” of relevant papers already known to the authors), keyword searche… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

211 extracted references · 62 canonical work pages

  1. [1]

    Bogdan Adamyk, Vladlena Benson, Oksana Adamyk, Bożena Fraczek, and Anitha Chinnaswamy. 2024. Barriers to Trust in AI: a Study of the Explainability Technologies Adoption in Banks. In2024 14th International Conference on Advanced Computer Information Technologies (ACIT). IEEE, 230–234

    2024

  2. [2]

    Mamia Agbese, Rahul Mohanani, Arif Khan, and Pekka Abrahamsson. 2023. Implementing AI ethics: Making sense of the ethical requirements. InProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 62–71

    2023

  3. [3]

    Khlood Ahmad. 2021. Human-centric requirements engineering for artificial intelligence software systems. In2021 IEEE 29th International Requirements Engineering Conference (RE). IEEE, 468–473

    2021

  4. [4]

    IBM Resaerch Trusted AI. 2021. AIF360 API. (2021). https://aif360.mybluemix.net/

    2021

  5. [5]

    IBM Resaerch Trusted AI. 2021. AIX360 API. (2021). https://aix360.mybluemix.net/

    2021

  6. [6]

    Meta AI. 2022. System cards. (2022). https://ai.meta.com/tools/system-cards/

    2022

  7. [7]

    Muhammad Azeem Akbar, Arif Ali Khan, Sajjad Mahmood, Saima Rafi, and Selina Demi. 2024. Trustworthy artificial intelligence: A decision-making taxonomy of potential challenges.Software: Practice and Experience54, 9 (2024), 1621–1650

    2024

  8. [8]

    Pouria Akbarighatar. 2024. Operationalizing responsible AI principles through responsible AI capabilities.AI and Ethics(2024), 1–15

    2024

Show all 211 references
  1. [9]

    Pouria Akbarighatar, Ilias O Pappas, Polyxeni Vassilakopoulou, and Sandeep Purao. 2024. Responsible AI Principles: Findings from an Empirical Study on Practitioners’ Perceptions. (2024)

    2024

  2. [10]

    Anwar Al-Hwsali, Balqes Alsaadi, Nima Abdi, Shaza Khatab, Mahmood Alzubaidi, Barry Solaiman, and Mowafa Househ. 2023. Scoping review: legal and ethical principles of artificial intelligence in public health.Healthcare Transformation with Informatics and Artificial Intelligence...

    2023

  3. [11]

    Sanna J Ali, Angèle Christin, Andrew Smart, and Riitta Katila. 2023. Walking the walk of AI ethics: Organizational challenges and the individualization of risk among ethics entrepreneurs. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 217–226

    2023

  4. [12]

    Marianna Anagnostou, Olga Karvounidou, Chrysovalantou Katritzidaki, Christina Kechagia, Kyriaki Melidou, Eleni Mpeza, Ioannis Konstantinidis, Eleni Kapantai, Christos Berberidis, Ioannis Magnisalis, et al. 2022. Characteristics and challenges in the industries towards responsi...

    2022

  5. [13]

    Marc M Anderson and Karën Fort. 2024. Evaluating the acceptability of ethical recommendations in industry 4.0: an ethics by design approach.AI & SOCIETY(2024), 1–15

    2024

  6. [14]

    McKane Andrus, Elena Spitzer, Jeffrey Brown, and Alice Xiang. 2021. What we can’t measure, we can’t understand: Challenges to demographic data procurement in the pursuit of fairness. InProceedings of the 2021 ACM conference on fairness, accountability, and transparency. 249–260

    2021

  7. [15]

    Laura Arbelaez Ossa, Giorgia Lorenzini, Stephen R Milford, David Shaw, Bernice S Elger, and Michael Rost. 2024. Integrating ethics in AI development: a qualitative study.BMC Medical Ethics25, 1 (2024), 10

    2024

  8. [16]

    Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities an...

    2020

  9. [17]

    Zahra Ashktorab, Benjamin Hoover, Mayank Agarwal, Casey Dugan, Werner Geyer, Hao Bang Yang, and Mikhail Yurochkin. 2023. Fairness evaluation in text classification: Machine learning practitioner perspectives of individual and group fairness. InProceedings of the 2023 CHI Confe...

    2023

  10. [18]

    Narges Ashtari, Ryan Mullins, Crystal Qian, James Wexler, Ian Tenney, and Mahima Pushkarna. 2023. From discovery to adoption: Understanding the ml practitioners’ interpretability journey. InProceedings of the 2023 ACM Designing Interactive Systems Conference. 2304–2325

    2023

  11. [19]

    Tita Alissa Bach, Magnhild Kaarstad, Elizabeth Solberg, and Aleksandar Babic. 2025. Insights into suggested Respon- sible AI (RAI) practices in real-world settings: a systematic literature review.AI and Ethics(2025), 1–48

    2025

  12. [20]

    Josef Baker-Brunnbauer. 2021. Management perspective of ethics in artificial intelligence.AI and Ethics1, 2 (2021), 173–181

    2021

  13. [21]

    Agathe Balayn, Mireia Yurrita, Fanny Rancourt, Fabio Casati, and Ujwal Gadiraju. 2025. Unpacking Trust Dynamics in the LLM Supply Chain: An Empirical Exploration to Foster Trustworthy LLM Production & Use. InProceedings of the 2025 CHI Conference on Human Factors in Computing ...

    2025

  14. [22]

    Fairness Toolkits, A Checkbox Culture?

    Agathe Balayn, Mireia Yurrita, Jie Yang, and Ujwal Gadiraju. 2023. “Fairness Toolkits, A Checkbox Culture?” On the Factors that Fragment Developer Practices in Handling Algorithmic Harms. InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. 482–495

    2023

  15. [23]

    2015.Privacy on the ground: driving corporate behavior in the United States and Europe

    Kenneth A Bamberger and Deirdre K Mulligan. 2015.Privacy on the ground: driving corporate behavior in the United States and Europe. MIT Press

    2015

  16. [24]

    Oshri Bar-Gil, Tom Ron, and Ofir Czerniak. 2024. AI for the people? Embedding AI ethics in HR and people analytics projects.Technology in Society77 (2024), 102527

    2024

  17. [25]

    2023.Fairness and machine learning: Limitations and opportunities

    Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023.Fairness and machine learning: Limitations and opportunities. MIT press

    2023

  18. [26]

    Abhishek Behl, Brinda Sampat, Vijay Pereira, and Charbel José Chiappetta Jabbour. 2023. The role played by responsible artificial intelligence (RAI) in improving supply chain performance in the MSME sector: an empirical inquiry.Annals of Operations Research(2023), 1–30

    2023

  19. [27]

    Richard Benjamins, Alberto Barbado, and Daniel Sierra. 2019. Responsible AI by design in practice.arXiv preprint arXiv:1909.12838(2019)

    2019 arXiv

  20. [28]

    Everybody knows what a pothole is

    SJ Bennett, Benedetta Catanzariti, and Fabio Tollon. 2025. “Everybody knows what a pothole is”: representations of work and intelligence in AI practice and governance.AI & SOCIETY(2025), 1–12

    2025

  21. [29]

    Glen Berman, Nitesh Goyal, and Michael Madaio. 2024. A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–24

    2024

  22. [30]

    James Bessen, Stephen Michael Impink, and Robert Seamans. 2022. The cost of ethical AI development for AI startups. InProceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 92–106

    2022

  23. [31]

    Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Allison Woodruff, Christine Luu, Pierre Kreitmann, Jonathan Bischof, and Ed H Chi. 2019. Putting fairness principles into practice: Challenges, metrics, and improvements. InProceedings of the 2019 AAAI/ACM Conference on AI, Ethi...

    2019

  24. [32]

    Aditya Bhattacharya, Simone Stumpf, Robin De Croon, and Katrien Verbert. 2025. Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems. InProceedings of the 2025 CHI Conference on Human Factors in Computing S...

    2025

  25. [33]

    Sarah Bird. 2020. Fairlearn API. https://fairlearn.github.io/v0.5.0/api_reference/fairlearn.datasets.html

    2020

  26. [34]

    2020.Fairlearn: A toolkit for assessing and improving fairness in AI

    Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. 2020.Fairlearn: A toolkit for assessing and improving fairness in AI. Technical Report MSR-TR-2020-32. Microsoft. https://www.microsoft.com/en-...

    2020

  27. [35]

    Edyta Bogucka, Marios Constantinides, Sanja Šćepanović, and Daniele Quercia. 2024. Co-designing an AI impact assessment report template with AI practitioners and AI compliance experts. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7. 168–180

    2024

  28. [36]

    Karen Boyd. 2022. Designing up with value-sensitive design: Building a field guide for ethical ML development. In Proceedings of the 2022 ACM conference on fairness, accountability, and transparency. 2069–2082

    2022

  29. [37]

    Karen L Boyd. 2021. Datasheets for datasets help ML engineers notice and understand ethical issues in training data. Proceedings of the ACM on Human-Computer Interaction5, CSCW2 (2021), 1–27

    2021

  30. [38]

    Jude Browne, Eleanor Drage, and Kerry McInerney. 2024. Tech workers’ perspectives on ethical issues in AI development: Foregrounding feminist approaches.Big Data & Society11, 1 (2024), 20539517231221780

    2024

  31. [39]

    Anders Bruun, Niels Van Berkel, Dimitrios Raptis, and Effie LC Law. 2025. Coordination Mechanisms in AI Develop- ment: Practitioner Experiences on Integrating UX Activities. InProceedings of the 2025 CHI Conference on Human , Vol. 1, No. 1, Article . Publication date: August 2...

    2025

  32. [40]

    Jacques Bughin. 2024. Doing versus saying: responsible AI among large firms.AI & SOCIETY(2024), 1–13

    2024

  33. [41]

    Jiyoo Chang and Christine Custis. 2022. Understanding implementation challenges in machine learning documentation. InProceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–8

    2022

  34. [42]

    Matthew Charlesworth and David A Sewry. 2004. South African IT industry professionals’ ethical awareness: an exploratory study. InACM International Conference Proceeding Series, Vol. 75. 269–273

    2004

  35. [43]

    Larissa Chazette, Jil Klünder, Merve Balci, and Kurt Schneider. 2022. How can we develop explainable systems? insights from a literature review and an interview study. InProceedings of the International Conference on Software and System Processes and International Conference o...

    2022

  36. [44]

    Zhilong Chen, Jinghua Piao, Xiaochong Lan, Hancheng Cao, Chen Gao, Zhicong Lu, and Yong Li. 2022. Practitioners versus users: A value-sensitive evaluation of current industrial recommender system design.Proceedings of the ACM on Human-Computer Interaction6, CSCW2 (2022), 1–32

    2022

  37. [45]

    Robert Cinca, Enrico Costanza, and Mirco Musolesi. 2025. Practitioners and Bias in Machine Learning: A Study.ACM Transactions on Interactive Intelligent Systems15, 2 (2025), 1–28

    2025

  38. [46]

    Jennifer Cobbe, Michael Veale, and Jatinder Singh. 2023. Understanding accountability in algorithmic supply chains. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1186–1197

    2023

  39. [47]

    Christina Cociancig, Hendrik Heuer, and Andreas Breiter. 2024. AI ethics unwrapped: an empirical investigation of ethical principles in collaborative ideation processes.AI and Ethics(2024), 1–14

    2024

  40. [48]

    Marios Constantinides, Edyta Bogucka, Daniele Quercia, Susanna Kallio, and Mohammad Tahaei. 2024. RAI guidelines: Method for generating responsible AI guidelines grounded in regulations and usable by (non-) technical roles. Proceedings of the ACM on Human-Computer Interaction8...

    2024

  41. [49]

    Nicholas Kluge Corrêa, Camila Galvão, James William Santos, Carolina Del Pino, Edson Pontes Pinto, Camila Barbosa, Diogo Massmann, Rodrigo Mambrini, Luiza Galvão, Edmund Terem, et al. 2023. Worldwide AI ethics: A review of 200 guidelines and recommendations for AI governance.P...

    2023

  42. [50]

    Sasha Costanza-Chock, Inioluwa Deborah Raji, and Joy Buolamwini. 2022. Who Audits the Auditors? Recommenda- tions from a field scan of the algorithmic auditing ecosystem. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1571–1583

    2022

  43. [51]

    Keeley Crockett, Edwin Colyer, Luciano Gerber, and Annabel Latham. 2021. Building trustworthy AI solutions: A case for practical solutions for small businesses.IEEE Transactions on Artificial Intelligence4, 4 (2021), 778–791

    2021

  44. [52]

    Sauvik Das, Cori Faklaris, Jason I Hong, and Laura A Dabbish. 2022. The security & privacy acceptance framework (spaf).Annals of Science and Technology Policy5, 1-2 (2022), 1–143

    2022

  45. [53]

    Smruti Ranjan Das, Prithu Sarkar, Shripada Patil, Ritesh Sharma, Shikha Aggarwal, and Melanie Lourens. 2023. Artificial intelligence in human resource management: Transforming business practices. In2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, El...

    2023

  46. [54]

    Ronnie de Souza Santos, Luiz de Lima, Maria Teresa Baldassarre, and Rodrigo Spínola. 2024. Preliminary Insights on Industry Practices for Addressing Fairness Debt. InProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 566–571

    2024

  47. [55]

    Wesley Hanwen Deng, Boyuan Guo, Alicia Devrio, Hong Shen, Motahhare Eslami, and Kenneth Holstein. 2023. Understanding practices, challenges, and opportunities for user-engaged algorithm auditing in industry practice. In Proceedings of the 2023 CHI Conference on Human Factors i...

    2023

  48. [56]

    Wesley Hanwen Deng, Manish Nagireddy, Michelle Seng Ah Lee, Jatinder Singh, Zhiwei Steven Wu, Kenneth Holstein, and Haiyi Zhu. 2022. Exploring how machine learning practitioners (try to) use fairness toolkits. InProceedings of the 2022 ACM Conference on Fairness, Accountabilit...

    2022

  49. [57]

    Wesley Hanwen Deng, Nur Yildirim, Monica Chang, Motahhare Eslami, Kenneth Holstein, and Michael Madaio. 2023. Investigating practices and opportunities for cross-functional collaboration around AI fairness in industry practice. InProceedings of the 2023 ACM Conference on Fairn...

    2023

  50. [58]

    Shipi Dhanorkar, Christine T Wolf, Kun Qian, Anbang Xu, Lucian Popa, and Yunyao Li. 2021. Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle. InProceedings of the 2021 ACM Designing Interactive Sys...

    2021

  51. [59]

    Di Di. 2023. Ethical ambiguity and complexity: tech workers’ perceptions of big data ethics in China and the US. Information, Communication & Society26, 5 (2023), 957–973

    2023

  52. [60]

    Brandon Dominique, Kaoutar El Maghraoui, David Piorkowski, and Lorraine Herger. 2023. FactSheets for Hardware- Aware AI Models: A Case Study of Analog In Memory Computing AI Models. In2023 IEEE International Conference on Software Services Engineering (SSE). IEEE, 148–158

    2023

  53. [61]

    Eleanor Drage, Kerry McInerney, and Jude Browne. 2024. Engineers on responsibility: feminist approaches to who’s responsible for ethical AI.Ethics and Information Technology26, 1 (2024), 4. , Vol. 1, No. 1, Article . Publication date: August 2026. RAI Practices in Industry 33

    2024

  54. [62]

    2020.Working law: Courts, corporations, and symbolic civil rights

    Lauren B Edelman. 2020.Working law: Courts, corporations, and symbolic civil rights. University of Chicago Press

    2020

  55. [63]

    Upol Ehsan, Q Vera Liao, Michael Muller, Mark O Riedl, and Justin D Weisz. 2021. Expanding explainability: Towards social transparency in ai systems. InProceedings of the 2021 CHI conference on human factors in computing systems. 1–19

    2021

  56. [64]

    Upol Ehsan, Q Vera Liao, Samir Passi, Mark O Riedl, and Hal Daumé III. 2024. Seamful xai: Operationalizing seamful design in explainable ai.Proceedings of the ACM on Human-Computer Interaction8, CSCW1 (2024), 1–29

    2024

  57. [65]

    Salma Elsayed-Ali, Sara E Berger, Vagner Figueredo De Santana, and Juana Catalina Becerra Sandoval. 2023. Respon- sible & Inclusive Cards: An online card tool to promote critical reflection in technology industry work practices. In Proceedings of the 2023 CHI Conference on Hum...

    2023

  58. [66]

    Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets.Commun. ACM64, 12 (2021), 86–92

    2021

  59. [67]

    Uğur Genç and Himanshu Verma. 2024. Situating Empathy in HCI/CSCW: A Scoping Review.Proceedings of the ACM on Human-Computer Interaction8, CSCW2 (2024), 1–37

    2024

  60. [68]

    Pouria Akbari Ghatar, Ilias O Pappas, and Polyxeni Vassilakopoulou. 2023. Practices for Responsible AI: Findings from Interviews with Experts.. InAMCIS

    2023

  61. [69]

    Sabrina Goellner, Marina Tropmann-Frick, and Bostjan Brumen. 2024. Responsible artificial intelligence: A structured literature review.arXiv preprint arXiv:2403.06910(2024)

    2024 arXiv

  62. [70]

    Tricia A Griffin, Brian P Green, and Jos VM Welie. 2024. The ethical wisdom of AI developers.AI and Ethics(2024), 1–11

    2024

  63. [71]

    Lakshitha Gunasekara, Nicole El-Haber, Swati Nagpal, Harsha Moraliyage, Zafar Issadeen, Milos Manic, and Daswin De Silva. 2025. A Systematic Review of Responsible Artificial Intelligence Principles and Practice.Applied System Innovation8, 4 (2025), 97

    2025

  64. [72]

    Marisela Gutierrez Lopez and Susan Halford. 2025. Explaining machine learning practice: Findings from an engaged science and technology studies project.Information, Communication & Society28, 4 (2025), 616–632

    2025

  65. [73]

    Umm-e Habiba, Mohammad Kasra Habib, Justus Bogner, Jonas Fritzsch, and Stefan Wagner. 2025. How do ML practitioners perceive explainability? an interview study of practices and challenges.Empirical Software Engineering 30, 1 (2025), 18

    2025

  66. [74]

    Emily Hadley, Alan Blatecky, and Megan Comfort. 2024. Investigating algorithm review boards for organizational responsible artificial intelligence governance.AI and Ethics(2024), 1–11

    2024

  67. [75]

    Erika Halme, Marianna Jantunen, Ville Vakkuri, Kai-Kristian Kemell, and Pekka Abrahamsson. 2024. Making ethics practical: User stories as a way of implementing ethical consideration in Software Engineering.Information and Software Technology167 (2024), 107379

    2024

  68. [76]

    Vanessa Aisyahsari Hanschke, Dylan Rees, Merve Alanyali, David Hopkinson, and Paul Marshall. 2024. Data Ethics Emergency Drill: A Toolbox for Discussing Responsible AI for Industry Teams. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–17

    2024

  69. [77]

    Maria Hartikainen, Kaisa Väänänen, and Thomas Olsson. 2023. Towards a human-centred artificial intelligence maturity model. InExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–7

    2023

  70. [78]

    Maria Hedlund and Erik Persson. 2025. Distribution of responsibility for AI development: expert views.AI & SOCIETY (2025), 1–13

    2025

  71. [79]

    Amy K Heger, Liz B Marquis, Mihaela Vorvoreanu, Hanna Wallach, and Jennifer Wortman Vaughan. 2022. Understand- ing machine learning practitioners’ data documentation perceptions, needs, challenges, and desiderata.Proceedings of the ACM on Human-Computer Interaction6, CSCW2 (20...

    2022

  72. [80]

    Christine Hine and Payam Barnaghi. 2024. Ethics and Artificial Intelligence in the Interdisciplinary Collaborations of Smart Care.Science, Technology, & Human Values(2024), 01622439241302519

    2024

  73. [81]

    Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, and Steven M Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. InProceedings of the 2019 CHI conference on human factors in computing systems. 1–13

    2019

  74. [82]

    Tomasz Hollanek, Yulu Pi, Cosimo Fiorini, Virginia Vignali, Dorian Peters, and Eleanor Drage. 2025. A Toolkit for Compliance, a Toolkit for Justice: Drawing on Cross-sectoral Expertise to Develop a Pro-justice EU AI Act Toolkit. In Proceedings of the 2025 ACM Conference on Fai...

    2025

  75. [83]

    Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. 2019. Improving fairness in machine learning systems: What do industry practitioners need?. InProceedings of the 2019 CHI conference on human factors in computing systems. 1–16

    2019

  76. [84]

    Sungsoo Ray Hong, Jessica Hullman, and Enrico Bertini. 2020. Human factors in model interpretability: Industry practices, challenges, and needs.Proceedings of the ACM on Human-Computer Interaction4, CSCW1 (2020), 1–26

    2020

  77. [85]

    Aspen Hopkins and Serena Booth. 2021. Machine learning practices outside big tech: How resource constraints challenge responsible development. InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. , Vol. 1, No. 1, Article . Publication date: August 2026. 34...

    2021

  78. [86]

    Aspen Hopkins, Sarah H Cen, Isabella Struckman, Andrew Ilyas, Luis Videgaray, and Aleksander Mądry. 2025. Ai supply chains: An emerging ecosystem of ai actors, products, and services. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 8. 1266–1277

    2025

  79. [87]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card.arXiv preprint arXiv:2410.21276(2024)

    2024 arXiv

  80. [88]

    Javier Camacho Ibáñez and Mónica Villas Olmeda. 2022. Operationalising AI ethics: how are companies bridging the gap between practice and principles? An exploratory study.Ai & Society37, 4 (2022), 1663–1687

    2022

  81. [89]

    Nanna Inie, Jonathan Stray, and Leon Derczynski. 2023. Summon a demon and bind it: A grounded theory of llm red teaming in the wild.arXiv preprint arXiv:2311.06237(2023)

    2023 arXiv

  82. [90]

    Maurice Jakesch, Zana Buçinca, Saleema Amershi, and Alexandra Olteanu. 2022. How different groups prioritize ethical values for responsible AI. Inproceedings of the 2022 ACM conference on fairness, accountability, and transparency. 310–323

    2022

  83. [91]

    Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines.Nature machine intelligence1, 9 (2019), 389–399

    2019

  84. [92]

    Emma Kallina, Thomas Bohné, and Jatinder Singh. 2025. Stakeholder Participation for Responsible AI Develop- ment: Disconnects Between Guidance and Current Practice. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 1060–1079

    2025

  85. [93]

    Shivani Kapania, Stephanie Ballard, Alex Kessler, and Jennifer Wortman Vaughan. 2025. Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipeline. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 45–60

    2025

  86. [94]

    Shivani Kapania, Alex S Taylor, and Ding Wang. 2023. A hunt for the snark: Annotator diversity in data practices. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15

    2023

  87. [95]

    Harmanpreet Kaur, Matthew R Conrad, Davis Rule, Cliff Lampe, and Eric Gilbert. 2024. Interpretability Gone Bad: The Role of Bounded Rationality in How Practitioners Understand Machine Learning.Proceedings of the ACM on Human-Computer Interaction8, CSCW1 (2024), 1–34

    2024

  88. [96]

    Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan

  89. [97]

    Gunay Kazimzade and Milagros Miceli. 2020. Biased priorities, biased outcomes: three recommendations for ethics- oriented data annotation practices. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 71–71

    2020

  90. [98]

    2007.Guidelines for performing systematic literature reviews in software engineering

    Staffs Keele et al. 2007.Guidelines for performing systematic literature reviews in software engineering. Technical Report. Technical report, ver. 2.3 ebse technical report. ebse

    2007

  91. [99]

    Arif Ali Khan, Muhammad Azeem Akbar, Mahdi Fahmideh, Peng Liang, Muhammad Waseem, Aakash Ahmad, Mahmood Niazi, and Pekka Abrahamsson. 2023. AI ethics: an empirical study on the views of practitioners and lawmakers.IEEE Transactions on Computational Social Systems10, 6 (2023), ...

    2023

  92. [100]

    Arif Ali Khan, Sher Badshah, Peng Liang, Muhammad Waseem, Bilal Khan, Aakash Ahmad, Mahdi Fahmideh, Mahmood Niazi, and Muhammad Azeem Akbar. 2022. Ethics of AI: A systematic literature review of principles and challenges. InProceedings of the 26th international conference on e...

    2022

  93. [101]

    Maira Khanam, Saima Rafi, and Muhammad Azeem Akbar. 2023. Understandings of Ethics in DevOps Teams: A Preliminary Investigation. InProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 394–397

    2023

  94. [102]

    Ramaravind Kommiya Mothilal, Shion Guha, and Syed Ishtiaque Ahmed. 2024. Towards a non-ideal methodological framework for responsible ML. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–17

    2024

  95. [103]

    Apoorva Nalini Pradeep Kumar, Justus Bogner, Markus Funke, and Patricia Lago. 2024. Balancing Progress and Responsibility: A Synthesis of Sustainability Trade-Offs of AI-Based Systems. In2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C). IEEE, 207–214

    2024

  96. [104]

    Steven Labkoff, Bilikis Oladimeji, Joseph Kannry, Anthony Solomonides, Russell Leftwich, Eileen Koski, Amanda L Joseph, Monica Lopez-Gonzalez, Lee A Fleisher, Kimberly Nolen, et al. 2024. Toward a responsible future: recommen- dations for AI-enabled clinical decision support.J...

    2024

  97. [105]

    M-C Laï, M Brian, and M-F Mamzer. 2020. Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.Journal of translational medicine18 (2020), 1–13

    2020

  98. [106]

    It’s everybody’s role to speak up... but not everyone will

    Caitlin M Lancaster, Kelsea Schulenberg, Christopher Flathmann, Nathan J McNeese, and Guo Freeman. 2024. “It’s everybody’s role to speak up... but not everyone will”: Understanding AI professionals’ perceptions of accountability , Vol. 1, No. 1, Article . Publication date: Aug...

    2024

  99. [107]

    Marinka Lanne, Mika Nieminen, and Jaana Leikas. 2025. Organisational tensions in introducing socially sustainable AI.AI & SOCIETY(2025), 1–21

    2025

  100. [108]

    I Don’t Know If We’re Doing Good. I Don’t Know If We’re Doing Bad

    Hao-Ping Hank Lee, Lan Gao, Stephanie Yang, Jodi Forlizzi, and Sauvik Das. 2024. "I Don’t Know If We’re Doing Good. I Don’t Know If We’re Doing Bad": Investigating How Practitioners Scope, Motivate, and Conduct Privacy Work When Developing{AI}Products. In33rd USENIX Security S...

    2024

  101. [109]

    Michelle Seng Ah Lee and Jat Singh. 2021. The landscape and gaps in open source fairness toolkits. InProceedings of the 2021 CHI conference on human factors in computing systems. 1–13

    2021

  102. [110]

    Michelle Seng Ah Lee and Jatinder Singh. 2021. Risk identification questionnaire for detecting unintended bias in the machine learning development lifecycle. InProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 704–714

    2021

  103. [111]

    Q Vera Liao, Hariharan Subramonyam, Jennifer Wang, and Jennifer Wortman Vaughan. 2023. Designerly under- standing: Information needs for model transparency to support design ideation for AI-powered user experience. In Proceedings of the 2023 CHI conference on human factors in ...

    2023

  104. [112]

    Gustavo Lopez and Luis A Guerrero. 2017. Awareness supporting technologies used in collaborative systems: a systematic literature review. InProceedings of the 2017 ACM conference on computer supported cooperative work and social computing. 808–820

    2017

  105. [113]

    Qinghua Lu, Liming Zhu, Xiwei Xu, Jon Whittle, David Douglas, and Conrad Sanderson. 2022. Software engineering for responsible AI: An empirical study and operationalised patterns. InProceedings of the 44th International Conference on Software Engineering: Software Engineering ...

    2022

  106. [114]

    Qinghua Lu, Liming Zhu, Xiwei Xu, Jon Whittle, Didar Zowghi, and Aurelie Jacquet. 2024. Responsible AI pattern catalogue: A collection of best practices for AI governance and engineering.Comput. Surveys56, 7 (2024), 1–35

    2024

  107. [115]

    Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions.Advances in neural information processing systems30 (2017)

    2017

  108. [116]

    Stephen MacDonell, Martin Shepperd, Barbara Kitchenham, and Emilia Mendes. 2010. How reliable are systematic reviews in empirical software engineering?IEEE Transactions on Software Engineering36, 5 (2010), 676–687

    2010

  109. [117]

    accessibility research

    Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, and Leah Findlater. 2021. What do we mean by “accessibility research”? A literature survey of accessibility papers in CHI and ASSETS from 1994 to 2019. In Proceedings of the 2021 CHI Conference on Human Fa...

    2021

  110. [118]

    Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. 2022. Assessing the fairness of ai systems: Ai practitioners’ processes, challenges, and needs for support.Proceedings of the ACM on Human-Computer Interaction6, CSCW1 (2022), 1–26

    2022

  111. [119]

    Michael Madaio, Shivani Kapania, Rida Qadri, Ding Wang, Andrew Zaldivar, Remi Denton, and Lauren Wilcox. 2024. Learning about Responsible AI On-The-Job: Learning Pathways, Orientations, and Aspirations. InProceedings of the 2024 ACM Conference on Fairness, Accountability, and ...

    2024

  112. [120]

    Michael A Madaio, Jingya Chen, Hanna Wallach, and Jennifer Wortman Vaughan. 2024. Tinker, Tailor, Configure, Customize: The Articulation Work of Contextualizing an AI Fairness Checklist.Proceedings of the ACM on Human- Computer Interaction8, CSCW1 (2024), 1–20

    2024

  113. [121]

    Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-designing checklists to understand organizational challenges and opportunities around fairness in AI. InProceedings of the 2020 CHI conference on human factors in computing systems. 1–14

    2020

  114. [122]

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning.ACM computing surveys (CSUR)54, 6 (2021), 1–35

    2021

  115. [123]

    Jacob Metcalf, Emanuel Moss, et al. 2019. Owning ethics: Corporate logics, silicon valley, and the institutionalization of ethics.Social Research: An International Quarterly86, 2 (2019), 449–476

    2019

  116. [124]

    Milagros Miceli, Martin Schuessler, and Tianling Yang. 2020. Between subjectivity and imposition: Power dynamics in data annotation for computer vision.Proceedings of the ACM on Human-Computer Interaction4, CSCW2 (2020), 1–25

    2020

  117. [125]

    Matti Minkkinen, Markus Philipp Zimmer, and Matti Mäntymäki. 2023. Co-shaping an ecosystem for responsible AI: Five types of expectation work in response to a technological frame.Information Systems Frontiers25, 1 (2023), 103–121

    2023

  118. [126]

    Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. InProceedings of the conference on fairness, accountability, and transparency. 220–229

    2019

  119. [127]

    Surbhi Mittal, Kartik Thakral, Richa Singh, Mayank Vatsa, Tamar Glaser, Cristian Canton Ferrer, and Tal Hassner

  120. [128]

    Jakob Mökander and Luciano Floridi. 2023. Operationalising AI governance through ethics-based auditing: an industry case study.AI and Ethics3, 2 (2023), 451–468

    2023

  121. [129]

    Jessica Morley, Luciano Floridi, Libby Kinsey, and Anat Elhalal. 2020. From what to how: an initial review of publicly available AI ethics tools, methods and research to translate principles into practices.Science and engineering ethics26, 4 (2020), 2141–2168

    2020

  122. [130]

    Jessica Morley, Libby Kinsey, Anat Elhalal, Francesca Garcia, Marta Ziosi, and Luciano Floridi. 2023. Operationalising AI ethics: barriers, enablers and next steps.AI & SOCIETY(2023), 1–13

    2023

  123. [131]

    Yuri Nakao, Lorenzo Strappelli, Simone Stumpf, Aisha Naseer, Daniele Regoli, and Giulia Del Gamba. 2023. Towards responsible AI: A design space exploration of human-centered artificial intelligence user interfaces to investigate fairness.International Journal of Human–Computer...

    2023

  124. [132]

    2023.Artificial Intelligence Risk Management Framework (AI RMF 1.0)

    National Institute of Standards and Technology. 2023.Artificial Intelligence Risk Management Framework (AI RMF 1.0). Technical Report NIST AI 100-1. NIST. https://doi.org/10.6028/NIST.AI.100-1

    2023 doi

  125. [133]

    Thanh Nguyen, Maria Teresa Baldassarre, Luiz Fernando de Lima, and Ronnie de Souza Santos. 2024. From literature to practice: Exploring fairness testing tools for the software industry adoption. InProceedings of the 18th ACM/IEEE International Symposium on Empirical Software E...

    2024

  126. [134]

    Ariadne A Nichol, Meghan Halley, Carole Federico, Mildred K Cho, and Pamela L Sankar. 2024. Moral engagement and disengagement in health care AI development.AJOB Empirical Bioethics15, 4 (2024), 291–300

    2024

  127. [135]

    Ariadne A Nichol, Pamela L Sankar, Meghan C Halley, Carole A Federico, and Mildred K Cho. 2023. Developer perspectives on potential harms of machine learning predictive analytics in health care: Qualitative analysis.Journal of Medical Internet Research25 (2023), e47609

    2023

  128. [136]

    NIST. 2023. Artificial Intelligence Risk Management Framework. https://www.nist.gov/itl/ai-risk-management- framework

    2023

  129. [137]

    Marie Oldfield. 2024. Technical challenges and perception: does AI have a PR issue?AI and Ethics4, 4 (2024), 975–995

    2024

  130. [138]

    Lauren Olson, Ricarda Anna-Lena Fischer, Florian Kunneman, and Emitzá Guzmán. 2025. Who Speaks for Ethics? How Demographics Shape Ethical Advocacy in Software Development. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 2847–2862

    2025

  131. [139]

    Will Orr and Jenny L Davis. 2020. Attributions of ethical responsibility by Artificial Intelligence practitioners. Information, Communication & Society23, 5 (2020), 719–735

    2020

  132. [140]

    Ricardo Ortega-Bolaños, Joshua Bernal-Salcedo, Mariana Germán Ortiz, Julian Galeano Sarmiento, Gonzalo A Ruz, and Reinel Tabares-Soto. 2024. Applying the ethics of AI: a systematic review of tools for developing and assessing AI-based systems.Artificial Intelligence Review57, ...

    2024

  133. [141]

    Guilherme Palumbo, Davide Carneiro, and Victor Alves. 2024. Objective metrics for ethical AI: a systematic literature review.International Journal of Data Science and Analytics(2024), 1–21

    2024

  134. [142]

    Aastha Pant, Rashina Hoda, Simone V Spiegler, Chakkrit Tantithamthavorn, and Burak Turhan. 2024. Ethics in the age of AI: An analysis of AI practitioners’ awareness and challenges.ACM Transactions on Software Engineering and Methodology33, 3 (2024), 1–35

    2024

  135. [143]

    Emmanouil Papagiannidis, Ida Merete Enholm, Chirstian Dremel, Patrick Mikalef, and John Krogstie. 2023. Toward AI governance: Identifying best practices and potential barriers and outcomes.Information Systems Frontiers25, 1 (2023), 123–141

    2023

  136. [144]

    Hyanghee Park, Daehwan Ahn, Kartik Hosanagar, and Joonhwan Lee. 2022. Designing fair AI in human resource management: Understanding tensions surrounding algorithmic evaluation and envisioning stakeholder-centered solutions. InProceedings of the 2022 CHI conference on human fac...

    2022

  137. [145]

    European Parliament. 2023. EU AI Act: first regulation on artificial intelligence.Topics of European Parliament (2023). https://www.europarl.europa.eu/topics/en/article/20230601STO93804/eu-ai-act-first-regulation-on-artificial- intelligence

    2023

  138. [146]

    Samir Passi and Solon Barocas. 2019. Problem formulation and fairness. InProceedings of the conference on fairness, accountability, and transparency. 39–48

    2019

  139. [147]

    Samir Passi and Steven J Jackson. 2018. Trust in data science: Collaboration, translation, and accountability in corporate data science projects.Proceedings of the ACM on human-computer interaction2, CSCW (2018), 1–28

    2018

  140. [148]

    Sarah Pink, Emma Quilty, John Grundy, and Rashina Hoda. 2024. Trust, artificial intelligence and software practitioners: an interdisciplinary agenda.AI & SOCIETY(2024), 1–14

    2024

  141. [149]

    Erich Prem. 2023. From ethical AI frameworks to tools: a review of approaches.AI and Ethics3, 3 (2023), 699–716

    2023

  142. [150]

    Joaquin Quiñonero Candela, Yuwen Wu, Brian Hsu, Sakshi Jain, Jennifer Ramos, Jon Adams, Robert Hallman, and Kinjal Basu. 2023. Disentangling and operationalizing ai fairness at linkedin. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1213–1228

    2023

  143. [151]

    Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices.Proceedings of the ACM on , Vol. 1, No. 1, Article . Publication date: August 2026. RAI ...

    2021

  144. [152]

    Anka Reuel, Patrick Connolly, Kiana Jafari Meimandi, Shekhar Tewari, Jakub Wiatrak, Dikshita Venkatesh, and Mykel Kochenderfer. 2025. Responsible AI in the Global Context: Maturity Model and Survey. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Tran...

    2025

  145. [153]

    Why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should i trust you?" Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135–1144

    2016

  146. [154]

    Brianna Richardson, Jean Garcia-Gathright, Samuel F Way, Jennifer Thom, and Henriette Cramer. 2021. Towards fairness in practice: A practitioner-oriented rubric for evaluating Fair ML Toolkits. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13

    2021

  147. [155]

    Shalaleh Rismani and AJung Moon. 2023. What does it mean to be a responsible AI practitioner: An ontology of roles and skills. InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. 584–595

    2023

  148. [156]

    Shalaleh Rismani, Renee Shelby, Andrew Smart, Edgar Jatho, Joshua Kroll, AJung Moon, and Negar Rostamzadeh

  149. [157]

    Arlette Danielle Román Almánzar, David Joachim Grüning, and Laura Marie Edinger-Schons. 2024. ‘It wasn’t me’: the impact of social responsibility and social dominance attitudes on AI programmers’ moral imagination (intention to correct bias).AI and Ethics(2024), 1–15

    2024

  150. [158]

    Lorenn P Ruster and Jenny L Davis. 2025. The Gaps that Never Were: Reconsidering Responsible AI’s Principle-Practice Problem. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 350–360

    2025

  151. [159]

    Mark Ryan, Eleni Christodoulou, Josephina Antoniou, and Kalypso Iordanou. 2024. An AI ethics ‘David and Goliath’: value conflicts between large tech companies and their employees.AI & SOCIETY39, 2 (2024), 557–572

    2024

  152. [160]

    Seamus Ryan, Camille Nadal, and Gavin Doherty. 2023. Integrating fairness in the software design process: An interview study with hci and ml experts.IEEE Access11 (2023), 29296–29313

    2023

  153. [161]

    Malak Sadek, Emma Kallina, Thomas Bohné, Céline Mougenot, Rafael A Calvo, and Stephen Cave. 2025. Challenges of responsible AI in practice: scoping review and recommended actions.AI & SOCIETY40, 1 (2025), 199–215

    2025

  154. [162]

    Malak Sadek and Celine Mougenot. 2024. Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews.International Journal of Human–Computer Interaction(2024), 1–18

    2024

  155. [163]

    Everyone wants to do the model work, not the data work

    Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. Inproceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15

    2021

  156. [164]

    Nithya Sambasivan and Rajesh Veeraraghavan. 2022. The deskilling of domain expertise in AI development. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–14

    2022

  157. [165]

    Conrad Sanderson, David Douglas, Qinghua Lu, Emma Schleiger, Jon Whittle, Justine Lacey, Glenn Newnham, Stefan Hajkowicz, Cathy Robinson, and David Hansen. 2023. AI ethics principles in practice: perspectives of designers and developers.IEEE Transactions on Technology and Soci...

    2023

  158. [166]

    Jayshree Sarathy, Sophia Song, Audrey Haque, Tania Schlatter, and Salil Vadhan. 2023. Don’t look at the data! how differential privacy reconfigures the practices of data science. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19

    2023

  159. [167]

    Daniel S Schiff, Stephanie Kelley, and Javier Camacho Ibáñez. 2024. The emergence of artificial intelligence ethics auditing.Big Data & Society11, 4 (2024), 20539517241299732

    2024

  160. [168]

    Bianca GS Schor, Chris Norval, Ellen Charlesworth, and Jatinder Singh. 2024. Mind the gap: Designers and standards on algorithmic system transparency for users. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–16

    2024

  161. [169]

    Andrew D. Selbst. 2021. An institutional view of algorithmic impact assessments.Harvard Journal of Law and Technology35 (2021), 117–191

    2021

  162. [170]

    Ali Akbar Septiandri, Marios Constantinides, Mohammad Tahaei, and Daniele Quercia. 2023. WEIRD FAccTs: How western, educated, industrialized, rich, and democratic is FAccT?. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 160–171

    2023

  163. [171]

    ai ethics

    Mona Sloane and Janina Zakrzewski. 2022. German ai start-ups and “ai ethics”: Using a social practice lens for assessing and implementing socio-technical innovation. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 935–947

    2022

  164. [172]

    Jessie J Smith, Lex Beattie, and Henriette Cramer. 2023. Scoping fairness objectives and identifying fairness metrics for recommender systems: The practitioners’ perspective. InProceedings of the ACM Web Conference 2023. 3648–3659

    2023

  165. [173]

    Jessie J Smith, Anas Buhayh, Anushka Kathait, Pradeep Ragothaman, Nicholas Mattei, Robin Burke, and Amy Voida

  166. [174]

    Jessie J Smith, Michael Madaio, Robin Burke, and Casey Fiesler. 2025. Pragmatic Fairness: Evaluating ML Fairness Within the Constraints of Industry. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. 628–638

    2025

  167. [175]

    Evropi Stefanidi, Marit Bentvelzen, Paweł W Woźniak, Thomas Kosch, Mikołaj P Woźniak, Thomas Mildner, Stefan Schneegass, Heiko Müller, and Jasmin Niess. 2023. Literature reviews in HCI: A review of reviews. InProceedings of the 2023 CHI Conference on Human Factors in Computing...

    2023

  168. [176]

    Hariharan Subramonyam, Jane Im, Colleen Seifert, and Eytan Adar. 2022. Solving separation-of-concerns problems in collaborative design of human-AI systems through leaky abstractions. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–21

    2022

  169. [177]

    InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

    The many faces of fairness: Exploring the institutional logics of multistakeholder microlending recommendation. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1652–1663. , Vol. 1, No. 1, Article . Publication date: August 2026. 38 Deng et al

    2023

  170. [178]

    M Thenral and Arunkumar Annamalai. 2021. Challenges of building, deploying, and using AI-enabled telepsychiatry platforms for clinical practice among urban Indians: a qualitative study.Indian Journal of Psychological Medicine43, 4 (2021), 336–342

    2021

  171. [179]

    Emmanuelle Vaast. 2025. Experiencing and Addressing the Moral Ambivalence of Developing Digital Technology: Insights from Artificial Intelligence Developers. (2025)

    2025

  172. [180]

    This is just a prototype

    Ville Vakkuri, Kai-Kristian Kemell, Marianna Jantunen, and Pekka Abrahamsson. 2020. “This is just a prototype”: How ethics are ignored in software startup-like environments. InInternational Conference on Agile Software Development. Springer International Publishing Cham, 195–210

    2020

  173. [181]

    Google People + AI Team. 2020. What-If Tool. (2020). https://pair-code.github.io/what-if-tool/

    2020

  174. [182]

    Ville Vakkuri, Kai-Kristian Kemell, Joni Kultanen, Mikko Siponen, and Pekka Abrahamsson. 2019. Ethically aligned design of autonomous systems: Industry viewpoint and an empirical study.arXiv preprint arXiv:1906.07946(2019)

    2019 arXiv

  175. [183]

    Ville Vakkuri, Kai-Kristian Kemell, Joel Tolvanen, Marianna Jantunen, Erika Halme, and Pekka Abrahamsson. 2022. How do software companies deal with artificial intelligence ethics? A gap analysis. InProceedings of the 26th International Conference on Evaluation and Assessment i...

    2022

  176. [184]

    Elmira Van den Broek, Anastasia Sergeeva, and Marleen Huysman. 2020. Hiring algorithms: An ethnography of fairness in practice. In40th international conference on information systems, ICIS 2019. Association for Information Systems, 1–9

    2020

  177. [185]

    Ville Vakkuri, Kai-Kristian Kemell, Joni Kultanen, and Pekka Abrahamsson. 2020. The current state of industrial practice in artificial intelligence ethics.Ieee Software37, 4 (2020), 50–57

    2020

  178. [186]

    Gianmario Voria, Stefano Lambiase, Maria Concetta Schiavone, Gemma Catolino, and Fabio Palomba. 2024. From Expectation to Habit: Why Do Software Practitioners Adopt Fairness Toolkits?arXiv preprint arXiv:2412.13846(2024)

    2024 arXiv

  179. [187]

    Gianmario Voria, Giulia Sellitto, Carmine Ferrara, Francesco Abate, Andrea De Lucia, Filomena Ferrucci, Gemma Catolino, and Fabio Palomba. 2025. Fairness-aware practices from developers’ perspective: A survey.Information and Software Technology182 (2025), 107710

    2025

  180. [188]

    2021.Industry unbound: The inside story of privacy, data, and corporate power

    Ari Ezra Waldman. 2021.Industry unbound: The inside story of privacy, data, and corporate power. Cambridge University Press

    2021

  181. [189]

    Oleksandra Vereschak, Fatemeh Alizadeh, Gilles Bailly, and Baptiste Caramiaux. 2024. Trust in AI-assisted decision making: Perspectives from those behind the system and those for whom the decision is made. InProceedings of the 2024 CHI Conference on Human Factors in Computing ...

    2024

  182. [190]

    Ding Wang, Shantanu Prabhat, and Nithya Sambasivan. 2022. Whose AI Dream? In search of the aspiration in data annotation.. InProceedings of the 2022 CHI conference on human factors in computing systems. 1–16

    2022

  183. [191]

    Fredrik Wang, Ilias Pappas, and Polyxeni Vassilakopoulou. 2024. Operationalizing Algorithmic Fairness: Empirical Study and Framework Proposal. (2024)

    2024

  184. [192]

    Qiaosi Wang, Michael Madaio, Shaun Kane, Shivani Kapania, Michael Terry, and Lauren Wilcox. 2023. Designing responsible ai: Adaptations of ux practice to meet responsible ai challenges. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16

    2023

  185. [193]

    Angelina Wang, Teresa Datta, and John P Dickerson. 2024. Strategies for increasing corporate responsible AI prioritization. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7. 1514–1526

    2024

  186. [194]

    Rik Wehrens, Marthe Stevens, Johanna Kostenzer, Anne Marie Weggelaar, and Antoinette De Bont. 2023. Ethics as discursive work: the role of ethical framing in the promissory future of data-driven healthcare technologies.Science, Technology, & Human Values48, 3 (2023), 606–634

    2023

  187. [195]

    License to Critique

    David Gray Widder, Laura Dabbish, James D Herbsleb, and Nikolas Martelaro. 2024. Power and Play: Investigat- ing"License to Critique"in Teams’ AI Ethics Discussions.Proceedings of the ACM on Human-Computer Interaction8, CSCW2 (2024), 1–23. , Vol. 1, No. 1, Article . Publicatio...

    2024

  188. [196]

    AI supply chain

    David Gray Widder and Dawn Nafus. 2023. Dislocated accountabilities in the “AI supply chain”: Modularity and developers’ notions of responsibility.Big Data & Society10, 1 (2023), 20539517231177620

    2023

  189. [197]

    Zijie J Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio. 2024. Farsight: Fostering responsible ai awareness during ai application prototyping. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–40

    2024

  190. [198]

    Amy A Winecoff and Elizabeth Anne Watkins. 2022. Artificial concepts of artificial intelligence: institutional compliance and resistance in AI startups. InProceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 788–799

    2022

  191. [199]

    Jessica Woodgate and Nirav Ajmeri. 2024. Macro ethics principles for responsible AI systems: Taxonomy and directions.Comput. Surveys56, 11 (2024), 1–37

    2024

  192. [200]

    It Might be Technically Impressive, But It’s Practically Useless to us

    Qing Xiao, Xianzhe Fan, Felix Marvin Simon, Bingbing Zhang, and Motahhare Eslami. 2025. "It Might be Technically Impressive, But It’s Practically Useless to us": Motivations, Practices, Challenges, and Opportunities for Cross- Functional Collaboration around AI within the News...

    2025

  193. [201]

    Ethical AI

    David Gray Widder, Dawn Nafus, Laura Dabbish, and James Herbsleb. 2022. Limits and possibilities for “Ethical AI” in open source: A study of deepfakes. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2035–2046

    2022

  194. [202]

    Khensani Xivuri and Hosanna Twinomurinzi. 2023. How AI developers can assure algorithmic fairness.Discover Artificial Intelligence3, 1 (2023), 27

    2023

  195. [203]

    Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Viégas. 2023. Investigating how practitioners use human-ai guidelines: A case study on the people+ ai guidebook. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13

    2023

  196. [204]

    Chien Wen Yuan, Nanyi Bi, Ya-Fang Lin, and Yuen-Hsien Tseng. 2023. Contextualizing user perceptions about biases for human-centered explainable artificial intelligence. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15

    2023

  197. [205]

    Bianca Helena Ximenes and Geber L Ramalho. 2021. Concrete ethical guidelines and best practices in machine learning development. In2021 IEEE International Symposium on Technology and Society (ISTAS). IEEE, 1–8

    2021

  198. [206]

    Jianlong Zhou, Fang Chen, Adam Berry, Mike Reed, Shujia Zhang, and Siobhan Savage. 2020. A survey on ethical principles of AI and implementations. In2020 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 3010–3017

    2020

  199. [207]

    responsi- ble ai

    Markus Philipp Zimmer, Matti Minkkinen, and Matti Mäntymäki. 2022. Responsible Artificial Intelligence Systems Critical considerations for business model design.Scandinavian Journal of Information Systems34, 2 (2022), 4. A APPENDIX A.1 Keyword search [[Abstract: practices] OR ...

    2022

  200. [209]

    Wenbin Zhang. 2024. AI fairness in practice: Paradigm, challenges, and prospects.Ai Magazine45, 3 (2024), 386–395

    2024

  201. [2020]

    In Proceedings of the 2020 CHI conference on human factors in computing systems

    Interpreting interpretability: understanding data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–14

    2020

  202. [2023]

    In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

    From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–18

    2023

  203. [2024]

    On responsible machine learning datasets emphasizing fairness, privacy and regulatory norms with examples in biometrics and healthcare.Nature Machine Intelligence6, 8 (2024), 936–949. , Vol. 1, No. 1, Article . Publication date: August 2026. 36 Deng et al

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.