Voices in the Loop: Mapping Participatory AI
Pith reviewed 2026-05-19 21:19 UTC · model grok-4.3
The pith
The paper builds a living atlas of participatory AI initiatives that maps their global distribution, typical participation stages, and a governance system for making community input the default in AI systems.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We construct an open atlas of participatory AI initiatives by harmonizing records from prior trustworthy AI corpora with additional audited cases. The atlas reveals corpus-level patterns in geography, participation tiers, lifecycle loci, organizational forms, and documentation gaps while operationalizing a governance framework through versioned releases, record-linked issue channels, schema feedback workflows, and options for redaction or restricted disclosure.
What carries the argument
The participatory AI atlas, a harmonized open repository that enables discovery, vetting, geocoding, provenance tracking, versioned publication, and community-driven updates through issue and annotation channels.
If this is right
- Documented initiatives remain concentrated in a small number of countries.
- Participation is coded most often at problem formulation, evaluation, and governance rather than model development or training.
- The atlas supports comparative research, policy learning, and community scrutiny through a living inventory.
- Versioned releases and linked feedback channels allow the resource to be updated, contested, and reused over time.
Where Pith is reading between the lines
- The atlas could serve as a base layer for tracking how participation practices evolve when new cases from additional regions are incorporated.
- Policy efforts might draw on the participation-stage patterns to design requirements that push involvement earlier into technical development phases.
- Researchers could extend the protocol to include quantitative measures of participation depth or outcome influence beyond the current tiers.
Load-bearing premise
The harmonized records from existing corpora plus additional audited cases form a sufficiently complete and unbiased sample of participatory AI initiatives worldwide, with documentation gaps that do not systematically distort the reported geographic and participation patterns.
What would settle it
Identification of a substantial number of participatory AI initiatives in currently underrepresented countries or regions that significantly shifts the reported geographic concentration or participation stage distributions.
Figures
read the original abstract
Participatory approaches to artificial intelligence are increasingly documented across public, civic, and humanitarian settings, but evidence about how participation is organized remains fragmented. This paper reports on the construction of an open repository and interactive atlas of participatory AI initiatives, using records harmonized from Maga~na and Shilton's Trustworthy AI corpus, and additional audited cases from research and practice. We contribute three elements. First, we specify a reproducible protocol for discovery, vetting, harmonization, geocoding, provenance tracking, and release-based publication of participatory AI records. Second, we report corpus-level patterns in geography, participation tiers, lifecycle loci, organizational form, verification status, and remaining documentation gaps. Documented initiatives remain concentrated in a small number of countries, while participation is most often coded at problem formulation, evaluation, and governance rather than model development or training. Third, we show how the atlas operationalizes a design and governance framework for participatory-by-default AI infrastructures through versioned releases, record-linked issue and annotation channels, schema feedback workflows, and redaction or restricted-disclosure requests. The atlas is intended to support comparative research, policy learning, and community scrutiny through a living inventory that can be updated, contested, and reused.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes the construction of an open repository and interactive atlas of participatory AI initiatives. It harmonizes records from the Magaña and Shilton Trustworthy AI corpus with additional audited cases. The authors specify a reproducible protocol for discovery, vetting, harmonization, geocoding, provenance tracking, and release-based publication. They report corpus-level patterns regarding geography, participation tiers, lifecycle loci, organizational form, verification status, and documentation gaps. Finally, they illustrate how the atlas operationalizes a design and governance framework for participatory-by-default AI infrastructures via versioned releases, record-linked channels, schema feedback, and redaction requests.
Significance. If the reported patterns hold, this work provides a significant contribution by creating a living, open inventory that can facilitate comparative research, policy learning, and community scrutiny in participatory AI. The emphasis on a reproducible protocol, provenance tracking, versioned releases, and mechanisms for updates, contestation, and feedback represents a concrete strength that supports more inclusive AI design and governance. The paper explicitly credits the value of open data practices and community involvement in building such resources.
major comments (2)
- [Abstract (protocol contribution)] The reproducible protocol for discovery, vetting, harmonization, geocoding, and provenance tracking is described at a high level in the abstract but lacks explicit enumeration of search sources, inclusion/exclusion rules, or quantitative checks for language/visibility bias. This directly affects the reliability of the reported geographic concentrations and participation patterns.
- [Abstract (corpus-level patterns contribution)] The corpus-level patterns in geography, participation tiers, lifecycle loci, and documentation gaps are reported without quantitative details, error estimates, or verification steps. The central claim that these patterns reflect actual distributions rather than visibility biases rests on the unverified assumption that the harmonized Magaña-Shilton corpus plus audited cases form a sufficiently complete and unbiased sample.
minor comments (2)
- The abstract contains 'Maga~na' which appears to be a typographical rendering of 'Magaña'; verify and correct spelling consistency across the manuscript and references.
- [Atlas operationalization section] The description of the interactive atlas features and record-linked issue channels could be expanded with a brief example or figure to clarify how they operationalize the governance framework.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed comments, which help clarify the presentation of our protocol and findings. We address each major comment below, indicating planned revisions to strengthen the manuscript.
read point-by-point responses
-
Referee: [Abstract (protocol contribution)] The reproducible protocol for discovery, vetting, harmonization, geocoding, and provenance tracking is described at a high level in the abstract but lacks explicit enumeration of search sources, inclusion/exclusion rules, or quantitative checks for language/visibility bias. This directly affects the reliability of the reported geographic concentrations and participation patterns.
Authors: We agree that the abstract presents the protocol at a summary level for brevity. The full manuscript details the protocol in the Methods section, specifying sources (starting from the Magaña-Shilton corpus with additional targeted searches), inclusion criteria focused on documented participatory elements, and steps for harmonization, geocoding, and provenance. To directly address the concern, we will revise the abstract to enumerate key search sources and core inclusion/exclusion rules in one additional sentence, and add a dedicated paragraph on bias considerations with available quantitative checks such as language distribution across records. revision: yes
-
Referee: [Abstract (corpus-level patterns contribution)] The corpus-level patterns in geography, participation tiers, lifecycle loci, and documentation gaps are reported without quantitative details, error estimates, or verification steps. The central claim that these patterns reflect actual distributions rather than visibility biases rests on the unverified assumption that the harmonized Magaña-Shilton corpus plus audited cases form a sufficiently complete and unbiased sample.
Authors: The manuscript reports specific patterns with supporting counts (e.g., geographic concentration and lifecycle stage distributions) drawn from the harmonized corpus of audited cases. We acknowledge that formal error estimates are not provided and that visibility bias cannot be fully ruled out. In revision we will expand the Results section with explicit percentages, counts per category, and a clearer description of the auditing and verification process. We will also revise the abstract and discussion to frame the patterns as observations from the current corpus rather than claims of unbiased global distributions, while noting the living nature of the atlas as a mechanism for future completeness improvements. revision: partial
- A fully quantitative verification that the corpus is free of visibility bias is not feasible given the decentralized and often undocumented nature of participatory AI initiatives; the auditing process described provides the strongest practical check available.
Circularity Check
No circularity: descriptive mapping from external corpus with new protocol
full rationale
The paper specifies a protocol for harmonizing records from an external corpus (Magaña and Shilton) plus audited cases, then reports observed patterns in geography and participation without any mathematical derivations, fitted parameters, or predictions. No equations, self-citations, or ansatzes reduce claims to inputs by construction; the atlas and framework are presented as operational outputs of the described processes rather than tautological restatements. The work is self-contained as a reproducible inventory construction and descriptive analysis.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Participatory AI initiatives can be identified, categorized by participation tier and lifecycle stage, and geocoded from existing corpora and additional audited sources with acceptable documentation gaps.
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We specify a reproducible protocol for discovery, vetting, harmonization, geocoding, provenance tracking, and release-based publication of participatory AI records.
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Sherry R. Arnstein. 1969. A Ladder of Citizen Participation.Journal of the American Institute of Planners35, 4 (1969), 216–224. doi:10.1080/01944366908977225
-
[2]
Emily M. Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science.Transactions of the Association for Computational Linguistics6 (2018), 587–604. doi:10.1162/tacl_a_00041
-
[3]
Aleks Berditchevskaia, Eirini Malliaraki, and Kathy Peach. 2021. Participatory AI for Humanitarian Innovation. Nesta Briefing Paper. https://media.nesta.org.uk/documents/Nesta_Participatory_AI_for_humanitarian_innovation_Final.pdf Accessed 2026-01-14
work page 2021
-
[4]
Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022. Power to the People? Opportunities and Challenges for Participatory AI. InProceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization(Arlington, VA, USA)(EAAMO ’22). Association for Com...
-
[5]
Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, Kenneth V
Rick Bonney, Caren B. Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, Kenneth V. Rosenberg, and Jennifer Shirk. 2009. Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy.BioScience59, 11 (2009), 977–984. doi:10.1525/bio. 2009.59.11.9
work page doi:10.1525/bio 2009
-
[6]
Geoffrey C. Bowker and Susan Leigh Star. 1999.Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA
work page 1999
-
[7]
Stephanie Russo Carroll, Ibrahim Garba, Oscar L. Figueroa-Rodríguez, Jarita Holbrook, Raymond Lovett, Simeon Materechera, Mark Parsons, Kay Raseroka, Desi Rodriguez-Lonebear, Robyn Rowe, Rodrigo Sara, Jennifer D. Walker, Jane Anderson, and Maui Hudson
-
[8]
The CARE Principles for Indigenous Data Governance.Data Science Journal19, 1 (2020), 43. doi:10.5334/dsj-2020-043
- [9]
-
[10]
City of Helsinki. 2020. AI Register. Municipal website. https://ai.hel.fi/en/ai-register/ Accessed 2026-01-14
work page 2020
-
[11]
2020.Design Justice: Community-Led Practices to Build the Worlds We Need
Sasha Costanza-Chock. 2020.Design Justice: Community-Led Practices to Build the Worlds We Need. MIT Press, Cambridge, MA. FAccT ’26, June 25–28, 2026, Montreal, QC, Canada Mushkani
work page 2020
-
[12]
Bradley Cousins and Elizabeth Whitmore
J. Bradley Cousins and Elizabeth Whitmore. 1998. Framing Participatory Evaluation.New Directions for Evaluation1998, 80 (1998), 5–23. doi:10.1002/ev.1114
-
[13]
Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. 2023. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. InProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (Boston, MA, USA)(EAAMO ’23). Association for Computing Machinery, New York, NY, USA, A...
-
[14]
Digital Forum Lab and Eurocities. 2022. Algorithmic Transparency Standard. https://standard.algorithmictransparency.org/ Accessed 2026-02-16
work page 2022
-
[15]
Catherine D’Ignazio and Lauren F. Klein. 2020.Data Feminism. MIT Press, Cambridge, MA
work page 2020
-
[16]
2018.Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Virginia Eubanks. 2018.Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin’s Press, New York, NY
work page 2018
-
[17]
Batya Friedman and David G. Hendry. 2019.Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press, Cambridge, MA
work page 2019
-
[18]
Archon Fung. 2006. Varieties of Participation in Complex Governance.Public Administration Review66, s1 (2006), 66–75. doi:10.1111/j. 1540-6210.2006.00667.x
work page doi:10.1111/j 2006
-
[19]
Archon Fung and Mark E. Warren. 2011. The Participedia Project: An Introduction.International Public Management Journal14, 3 (2011), 341–362. doi:10.1080/10967494.2011.618309
-
[20]
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford
-
[21]
URL https://cacm.acm.org/research/ datasheets-for-datasets/
Datasheets for Datasets.Commun. ACM64, 12 (2021), 86–92. doi:10.1145/3458723
-
[22]
2019.Chasing Innovation: Making Entrepreneurial Citizens in Modern India
Lilly Irani. 2019.Chasing Innovation: Making Entrepreneurial Citizens in Modern India. Princeton University Press, Princeton, NJ
work page 2019
-
[23]
Peter-Lucas Jones. 2025. Kaitiaki: Closing the Door on Open Indigenous Data.International Journal on Digital Libraries26, 1 (2025), 1. doi:10.1007/s00799-025-00410-2
-
[24]
Anna Kawakami, Amanda Coston, Haiyi Zhu, Hoda Heidari, and Kenneth Holstein. 2024. The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder, Early-stage Deliberations Around Public Sector AI Proposals. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA...
-
[25]
2007.Guidelines for Performing Systematic Literature Reviews in Software Engineering
Barbara Kitchenham and Stuart Charters. 2007.Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report EBSE-2007-01. Keele University and Durham University Joint Report
work page 2007
-
[26]
Rob Kitchin. 2017. Thinking Critically About and Researching Algorithms.Information, Communication & Society20, 1 (2017), 14–29
work page 2017
-
[27]
2016.Indigenous Data Sovereignty: Toward an Agenda
Tahu Kukutai and John Taylor (Eds.). 2016.Indigenous Data Sovereignty: Toward an Agenda. ANU Press, Canberra
work page 2016
-
[28]
Julien Landry and Bettina von Lieres. 2022. Strengthening Democracy Through Knowledge Mobilization: Participedia – A Citizen-Led Global Platform for Transformative and Democratic Learning.Journal of Transformative Education20, 3 (2022), 206–225. doi:10.1177/ 15413446221103191
work page 2022
-
[29]
Gianna Leoni, Lee Steven, Keith T¯ureiti, Keoni Mahelona, Peter-Lucas Jones, and Suzanne Duncan. 2024. Solving Failure Modes in the Creation of Trustworthy Language Technologies. InProceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024, Maite Melero, Sakriani Sakti, and Claudia Soria (Eds.). ELR...
work page 2024
-
[30]
Maria Isabel Magaña and Katie Shilton. 2025. Frameworks, Methods and Shared Tasks: Connecting Participatory AI to Trustworthy AI Through a Systematic Review of Global Projects. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, NY, USA, 2166–2179. doi:10.1145/3715275.3732148
-
[31]
2015.Digital Humanitarians: How Big Data Is Changing the Face of Humanitarian Response
Patrick Meier. 2015.Digital Humanitarians: How Big Data Is Changing the Face of Humanitarian Response. CRC Press, Boca Raton, FL
work page 2015
-
[32]
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. InProceedings of the Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, NY, USA, 220–229. doi:10.1145/3287560.3287596
-
[33]
Rashid Mushkani. 2026. Measuring What Matters: The AI Pluralism Index. doi:10.48550/arXiv.2510.08193 Proceedings of the International Association for Safe & Ethical AI (IASEAI), 2026
-
[34]
Rashid Mushkani, Hugo Berard, Allison Cohen, and Shin Koseki. 2025. Position: The Right to AI. InProceedings of the 42nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267), Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (Eds.). PMLR, ...
work page 2025
-
[35]
Tina Nabatchi. 2012. Putting the “Public” Back in Public Values Research: Designing Participation to Identify and Respond to Values. Public Administration Review72, 5 (2012), 699–708. doi:10.1111/j.1540-6210.2012.02544.x
-
[36]
New York City Automated Decision Systems Task Force. 2019. Automated Decision Systems Task Force Report. Report. https: //www.nyc.gov/site/adstaskforce/index.page Accessed 2026-01-14
work page 2019
-
[37]
Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , location =
Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Voices in the Loop FAccT ’26, June 25–28, 2026, Montreal, QC, Canada Auditing. InProceedings of the 2...
-
[38]
Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. 2018. Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. https://ainowinstitute.org/wp-content/uploads/2023/04/aiareport2018.pdf Accessed 2026-01-14
work page 2018
-
[39]
Toni Robertson and Jesper Simonsen. 2012. Participatory Design: An Introduction. InRoutledge International Handbook of Participatory Design, Jesper Simonsen and Toni Robertson (Eds.). Routledge, New York, NY, 1–17
work page 2012
-
[40]
1993.Participatory Design: Principles and Practices
Douglas Schuler and Aki Namioka (Eds.). 1993.Participatory Design: Principles and Practices. Lawrence Erlbaum Associates, Hillsdale, NJ
work page 1993
-
[41]
Model Cards for Model Reporting,
Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. InProceedings of the Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, NY, USA, 59–68. doi:10.1145/3287560.3287598
-
[42]
2012.Routledge International Handbook of Participatory Design
Jesper Simonsen and Toni Robertson (Eds.). 2012.Routledge International Handbook of Participatory Design. Routledge, New York, NY
work page 2012
-
[43]
Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano. 2022. Participation Is not a Design Fix for Machine Learning. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization(Arlington, VA, USA)(EAAMO ’22). Association for Computing Machinery, New York, NY, USA, Article 1. doi:10.1145/3551624.3555285
-
[44]
Susan Leigh Star. 1999. The Ethnography of Infrastructure.American Behavioral Scientist43, 3 (1999), 377–391
work page 1999
-
[45]
Susan Leigh Star and Karen Ruhleder. 1996. Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces. Information Systems Research7, 1 (1996), 111–134
work page 1996
-
[46]
UK Central Digital and Data Office. 2023. Algorithmic Transparency Recording Standard. GOV.UK guidance. https://www.gov.uk/ government/collections/algorithmic-transparency-recording-standard-hub Accessed 2026-01-14
work page 2023
-
[47]
Maranke Wieringa. 2020. What to Account for When Accounting for Algorithms: A Systematic Literature Review on Algorithmic Accountability. InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, NY, USA, 1–18. FAccT ’26, June 25–28, 2026, Montreal, QC, Canada Mushkani A Supplementar...
work page 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.