REVIEW 3 major objections 5 minor 29 references
We Are AI: Taking Control of Technology
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper introduces a free, five-module course that teaches responsible AI to non-experts through peer-led learning circles, and reports that participants' self-rated understanding of AI rose from about 4 to about 6 on a 10-point scale.
desk verdict A genuinely useful open resource for public RAI education, but the empirical claims in the paper run ahead of the evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the learning circle, a pedagogical model in which small groups of people voluntarily come together to learn a topic collectively, without a traditional teacher. In this course, facilitators organize meetings and guide discussion using prepared materials but are not required to have deep AI expertise, which makes the format replicable. The train-the-trainer design is embedded: participants are encouraged and, as observed in the two offerings, actually do step up to facilitate subsequent circles, allowing the course to scale with minimal resources. The course's modular structure—videos, discussion prompts, hands-on activities, and a multilingual comic series—keeps the barrier to entry low and the format engaging.
What would settle it
A credible test would give participants a short objective quiz on AI concepts before and after the course, alongside a matched control group that does not take the course; the central claim would be undercut if quiz gains were small or no larger than the control group's gains during the same window.
Extended reading notes
Core claim
The central claim is that a carefully scaffolded, discussion-based learning circle can close the gap between technical AI expertise and public understanding without requiring expert instructors. The paper shows that over five 90-minute sessions covering what AI is, learning from data, ethical tradeoffs, bias, and civic action, librarians and professional staff increased their self-reported understanding of AI and responsible AI. In Summer 2023 the weighted average rose from 4.08 to 6.33, and in Fall 2023 from 4.19 to 5.89, with two-thirds of respondents saying the course improved their understanding substantially. Participants also absorbed key concepts—mentioning bias, transparency, and regulation in their post-course definitions—and reported intentions to teach others, demonstrating the train-the-trainer mechanism. The paper's claim is that this model is sustainable and scalable: anyone can run the course with the public materials, and learners naturally become future facilitators.
Load-bearing premise
The evaluation treats participants' self-reported understanding on a 1–10 scale as a valid measure of real learning, and assumes the gains came from the course rather than from the heavy media coverage of AI during the same period.
Editorial extensions
If this is right
- If the model works, public libraries, community organizations, and universities can offer responsible-AI education with little more than internet access and a willing facilitator.
- Learners who complete the course will be able to critique AI applications in hiring, education, and law enforcement using concepts like bias, transparency, and stakeholder impact.
- The train-the-trainer effect means each cohort can seed the next, making the course self-propagating rather than dependent on a fixed set of instructors.
- The authors plan to add modules on generative AI, tailored to librarians, administrators, and educators, addressing the gap participants flagged.
- The openly available materials, including the comic series in English, Spanish, and Ukrainian, support independent study beyond the course itself.
Reading between the lines
- Because the evidence is self-reported, an objective knowledge test or a control group not exposed to the course could determine whether the gains reflect real learning or are inflated by participants' desire to improve—an extension the paper does not attempt.
- The course's deliberately low technical ceiling may limit how far it can take motivated learners; a follow-on intermediate course, which the authors acknowledge, would be a natural test of whether the model retains its benefits at higher levels.
- The model may transfer to other policy-adjacent literacies, such as data privacy or algorithmic governance, for which scaffolded peer-led materials could be built—an inference extending beyond the paper's stated scope.
- If the train-the-trainer dynamic generalizes beyond librarians, the course could become a decentralized public-infrastructure approach to responsible-AI literacy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces 'We Are AI: Taking Control of Technology,' a five-module, open-access course on Responsible AI designed for non-technical audiences and delivered in a peer-learning 'learning circle' format. It describes the course rationale, materials (including a multilingual comic series), the train-the-trainer mechanism, and three iterations: a Spring 2021 pilot with Queens Public Library, a Summer 2023 in-person offering to 15 NYU librarians/staff, and a Fall 2023 online offering to 26 NYU Abu Dhabi professionals. The authors report pre/post self-assessed understanding of AI (Figures 4 and 5), with weighted averages rising from 4.08 to 6.33 and from 4.19 to 5.89, plus qualitative quotes from participants, and they draw lessons learned for future iterations. The Introduction claims the course 'demonstrate[s] positive learning outcomes and strong participant engagement'; the Conclusion more cautiously acknowledges the need for rigorous evaluation.
Significance. If the central interpretation were supported, the paper would offer a useful, low-cost, scalable model for raising AI and Responsible AI literacy among non-experts, with concrete open materials (course website, facilitator guide, trilingual comics) and a plausible train-the-trainer pathway. The pedagogical design—intentional scaffolding, case studies, peer facilitation—is well-motivated by the learning-circles literature, and the paper's candor about gaps (generative AI coverage, facilitator scaffolding, cultural contextualization) adds practical value. The empirical evidence, however, is suggestive rather than demonstrative: self-report measures, small matched samples (12 and 18), no control group, and no inferential statistics. The paper's lasting value is as a detailed design/resource report; the evaluation claims will need to be substantially tempered or strengthened before publication.
major comments (3)
- [Introduction; Learning and Engagement around Ethical AI] The Introduction's claim that the paper 'report[s] on three iterations of the course, demonstrating positive learning outcomes and strong participant engagement' is not supported by the evidence presented in 'Learning and Engagement around Ethical AI.' The only quantitative outcome is a pre/post self-rating of understanding of AI on a 10-point scale (Figures 4 and 5), completed by 12 (Summer 2023) and 18 (Fall 2023) matched participants. No objective knowledge assessment, control or comparison group, or inferential statistic is reported; the word 'significantly' appears in the text without p-values, confidence intervals, or effect sizes. Self-reported understanding is at best a proxy for confidence rather than competence, and the Summer/Fall 2023 period coincides with intense public discourse about generative AI, so the observed gains cannot be unambiguously attributed to the course. The authors should either provide appropriate statistical analyses and a consideration of confounds, or rephrase the claim as reporting preliminary, self-reported indications of learning.
- [Teaching RAI in a Peer Learning Setting, 'Course timeline' paragraph] The manuscript says it reports on 'three iterations,' but outcome data are presented only for the second and third iterations. The Spring 2021 iteration is described in the 'Course timeline' paragraph without any evaluation measures; the subsequent quantitative and qualitative results concern only Summer 2023 and Fall 2023. The authors should clarify which iterations have formal evaluation data, and avoid implying that the first iteration contributed to the reported 'positive learning outcomes.'
- [Conclusion and Next Steps] The Conclusion states: 'Additionally, there is a need to rigorously evaluate the effectiveness of educational methodologies for these audiences.' This statement is in direct tension with the 'demonstrate' language used in the Introduction. Because the paper's own concluding assessment concedes that rigorous evaluation is still needed, the empirical claims in the Introduction and abstract should be calibrated to match, e.g., by describing the results as encouraging preliminary evidence rather than demonstrated outcomes.
minor comments (5)
- [Abstract] Typo 'hile' should be 'while' in 'benefiting diverse stakeholders hile controlling'; also 'We are AIto' is missing a space before 'to an active and engaged group'.
- [Learning and Engagement around Ethical AI] The phrase 'yielding an 85.7% completion rate between pre- and post-survey)' has a stray closing parenthesis; also 'weighted average' is used but the weighting scheme is never defined, and Figures 4 and 5 do not show paired pre/post responses, making the reported averages hard to evaluate.
- [Conclusion and Next Steps] The sentence beginning 'emerging, we are developing a module...' is a fragment; it should read something like 'As these technologies are emerging, we are developing a module...'.
- [Lessons Learned] In 'Need for additional scaffolding...', the sentence 'Another noted the importance for a good learning experience' seems to be missing a noun (e.g., 'the importance of the learning circle for a good learning experience').
- [Teaching RAI in a Peer Learning Setting] The 'Course timeline' paragraph says 'All iterations of our course targeted librarians as the primary audience,' yet the Fall 2023 sample includes administrators, designers, and technologists (Table 1); consider acknowledging this composition or adjusting the description.
Circularity Check
No significant circularity: the outcome claims rest on empirical survey observations, not on a derivation from the paper's own inputs.
full rationale
The paper's central claim — that three iterations of the course demonstrate positive learning outcomes and strong participant engagement — is supported by pre- and post-course self-assessment survey data (Figures 4 and 5, with weighted averages rising from 4.08 to 6.33 in Summer 2023 and from 4.19 to 5.89 in Fall 2023) and by qualitative participant quotes. These are empirical observations about a specific course offering, not mathematical consequences derived from an input. No parameter is fitted to a subset of the data and then renamed as a prediction; no uniqueness theorem from the authors' prior work is invoked to forbid alternatives; and no formal derivation reduces to its own assumptions by construction. The authors' self-citations, such as Lewis and Stoyanovich (2022), Domínguez Figaredo and Stoyanovich (2023), and Bell, Nov, and Stoyanovich (2023b), provide background and situate the course within existing RAI education efforts, but they are not load-bearing for the reported outcome claim. The main weakness — that self-reported Likert ratings are used as a proxy for learning, without an objective knowledge test, control group, or inferential statistics — is an empirical-evidence limitation rather than a circularity. The paper itself concedes that 'there is a need to rigorously evaluate the effectiveness of educational methodologies,' which further indicates that the outcome claim is presented as an open empirical finding rather than as an artifact of the paper's definitions or prior work.
Assumptions & free parameters
assumptions (3)
- domain assumption Learning circles increase sustained learning and engagement.
- domain assumption Self-reported ratings of understanding measure learning.
- domain assumption Train-the-trainer facilitation maintains course quality.
Cite this review
Pith. "Pith review of We Are AI: Taking Control of Technology." pith.science (2026). https://pith.science/paper/DFDCNSHJ
@misc{pith2026250608117,
author = {Pith},
title = {Pith review of: We Are AI: Taking Control of Technology},
year = {2026},
howpublished = {\url{https://pith.science/paper/DFDCNSHJ}},
note = {Machine review of arXiv:2506.08117}
}
read the original abstract
Responsible AI (RAI) is the science and practice of ensuring the design, development, use, and oversight of AI are socially sustainable--benefiting diverse stakeholders while controlling the risks. Achieving this goal requires active engagement and participation from the broader public. This paper introduces "We are AI: Taking Control of Technology," a public education course that brings the topics of AI and RAI to the general audience in a peer-learning setting. We outline the goals behind the course's development, discuss the multi-year iterative process that shaped its creation, and summarize its content. We also discuss two offerings of We are AI to an active and engaged group of librarians and professional staff at New York University, highlighting successes and areas for improvement. The course materials, including a multilingual comic book series by the same name, are publicly available and can be used independently. By sharing our experience in creating and teaching We are AI, we aim to introduce these resources to the community of AI educators, researchers, and practitioners, supporting their public education efforts.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Ali, M.; Sapiezynski, P.; Bogen, M.; Korolova, A.; Mislove, A.; and Rieke, A. 2019. Discrimination through optimization: How Facebook 's ad delivery can lead to skewed outcomes. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW): 1--30. ArXiv:1904.02095 [cs]
arXiv 2019
-
[4]
Barocas, S.; and Selbst, A. D. 2016. Big data's disparate impact. Calif. L. Rev., 104: 671
work page 2016
-
[5]
Bell, A.; Nov, O.; and Stoyanovich, J. 2023 a . The algorithmic transparency playbook. The algorithmic transparency playbook course site. [Online; accessed on 2-March-2023]
work page 2023
-
[6]
Bell, A.; Nov, O.; and Stoyanovich, J. 2023 b . Think about the stakeholders first! Toward an algorithmic transparency playbook for regulatory compliance. Data & Policy, 5: e12
work page 2023
-
[7]
Chen, C.; Feng, S.; Sharma, A.; and Tan, C. 2023. Machine Explanations and Human Understanding . In 2023 ACM Conference on Fairness , Accountability , and Transparency , 1--1. Chicago IL USA: ACM. ISBN 9798400701924
work page 2023
-
[8]
Collay, M., ed. 1998. Learning circles: creating conditions for professional development. Thousand Oaks, Calif: Corwin Press. ISBN 9780803966758 9780803966765
work page 1998
Show all 29 references
-
[9]
Delgado, F.; Yang, S.; Madaio, M.; and Yang, Q. 2023. The Participatory Turn in AI Design : Theoretical Foundations and the Current State of Practice . In Equity and Access in Algorithms , Mechanisms , and Optimization , 1--23. Boston MA USA: ACM. ISBN 9798400703812
2023
-
[10]
Domínguez Figaredo, D.; and Stoyanovich, J. 2023. Responsible AI literacy: A stakeholder-first approach. Big Data & Society, 10(2): 20539517231219958
2023
-
[11]
Friedman, B.; and Nissenbaum, H. 1996. Bias in Computer Systems. ACM Trans. Inf. Syst. , 14(3): 330--347
1996
-
[12]
W.; Wallach, H.; III, H
Gebru, T.; Morgenstern, J.; Vecchione, B.; Vaughan, J. W.; Wallach, H.; III, H. D.; and Crawford, K. 2021. Datasheets for datasets. Communications of the ACM, 64(12): 86--92
2021
-
[13]
H.; Bau, D.; Yuan, B
Gilpin, L. H.; Bau, D.; Yuan, B. Z.; Bajwa, A.; Specter, M.; and Kagal, L. 2018. Explaining Explanations : An Overview of Interpretability of Machine Learning . In DSAA 2018
2018
-
[14]
Güngör, H. 2020. Creating Value with Artificial Intelligence : A Multi -stakeholder Perspective . Journal of Creating Value, 6(1): 72--85
2020
-
[15]
Hackney, R.; McMaster, T.; and Harris, A. 2003. Using Cases as a Teaching Tool an IS Education . Journal of Information Systems Education, 14(3): 229
2003
-
[16]
Jagadish, H. 2023. Data science ethics. Data science ethics course site. [Online; accessed on 2-March-2023]
2023
-
[17]
Kitchin, R.; and Lauriault, T. 2014. Towards Critical Data Studies : Charting and Unpacking Data Assemblages and Their Work
2014
-
[18]
M.; Young, M.; Katell, M.; Lee, J
Krafft, P. M.; Young, M.; Katell, M.; Lee, J. E.; Narayan, S.; Epstein, M.; Dailey, D.; Herman, B.; Tam, A.; Guetler, V.; Bintz, C.; Raz, D.; Jobe, P. O.; Putz, F.; Robick, B.; and Barghouti, B. 2021. An Action - Oriented AI Policy Toolkit for Technology Audits by Community Ad...
2021
-
[19]
Kreber, C. 2001. Learning Experientially through Case Studies ? A Conceptual Analysis . Teaching in Higher Education, 6(2): 217--228
2001
-
[20]
Lewis, A.; and Stoyanovich, J. 2022. Teaching Responsible Data Science: Charting New Pedagogical Territory. Int. J. Artif. Intell. Educ., 32(3): 783--807
2022
-
[21]
V.; and Wortman Vaughan, J
Liao, Q. V.; and Wortman Vaughan, J. 2024. AI Transparency in the Age of LLMs : A Human - Centered Research Roadmap . Harvard Data Science Review, Special Issue 4
2024
-
[22]
Lima, G.; and Cha, M. 2020. Responsible AI and Its Stakeholders
2020
-
[23]
Marcinkevičs, R.; and Vogt, J. E. 2023. Interpretable and explainable machine learning: A methods‐centric overview with concrete examples. WIREs Data Mining and Knowledge Discovery, 13(3): e1493
2023
-
[24]
Miller, G. J. 2022. Stakeholder roles in artificial intelligence projects. Project Leadership and Society, 3: 100068
2022
-
[25]
D.; and Gebru, T
Mitchell, M.; Wu, S.; Zaldivar, A.; Barnes, P.; Vasserman, L.; Hutchinson, B.; Spitzer, E.; Raji, I. D.; and Gebru, T. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency - FAT * '19 , 220--229. ACM Press. ISBN ...
2019
-
[26]
T.; Singh, S.; and Guestrin, C
Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016. ” Why Should I Trust You ?”: Explaining the predictions of any classifier. In KDD '16 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 1135--1144. San Francisco, CA, USA: ACM
2016
-
[27]
J.; and West, T
Stoyanovich, J.; Van Bavel, J. J.; and West, T. V. 2020. The imperative of interpretable machines. Nature Machine Intelligence, 2(4): 197--199
2020
-
[28]
Stoyanovich, J.; Yang, K.; and Jagadish, H. V. 2018. Online Set Selection with Fairness and Diversity Constraints. In B \" o hlen, M. H.; Pichler, R.; May, N.; Rahm, E.; Wu, S.; and Hose, K., eds., Proceedings of the 21st International Conference on Extending Database Technolo...
2018
-
[29]
Wassermann, S. 1993. Getting Down to Cases . Learning To Teach with Case Studies . Teachers College Press, Columbia University, 1234 Amsterdam Avenue, New York, NY 10027. ISBN 9780807732915. ERIC Number: ED369783
1993
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.