{"id":"b0fd72e7-cfd9-41c6-9772-2daccd0aa322","arxiv_id":"2506.08117","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The authors present a free, open-source course on responsible AI for non-experts and report pre/post survey evidence of improved self-rated understanding in two librarian and staff cohorts.","lead":"We Are AI is an open, five-module course that teaches non-technical adults about artificial intelligence and responsible AI through small peer-learning groups called learning circles. This paper describes the course and reports two offerings where participants' self-rated understanding of AI rose after completion.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper overclaims: 'demonstrated learning' rests on self-reported confidence with no control or objective test, and outcome data exist for only two of the three cited iterations.","rationale":"The reader's weakest assumption correctly identifies self-reported understanding and the lack of causal control as the core vulnerability. I agree with that diagnosis, but a stress-test also surfaces two concrete, verifiable overclaims the reader did not emphasize. First, the Introduction asserts 'significantly higher' ratings without any statistical analysis; in a scientific paper, 'significantly' implies a test, and none is reported or derivable from the presented figures. Second, the claim covers 'three iterations,' yet the evaluation section presents outcome data only for the Summer 2023 and Fall 2023 offerings; the Spring 2021 iteration is described but has no survey, quiz, or engagement metrics, so the 'three iterations' claim is internally contradicted. The paper's own conclusion explicitly calls for rigorous evaluation as future work, undercutting the 'demonstrate' wording. These issues do not change the recommendation: the course is a useful, openly available resource, and the qualitative feedback plus the train-the-trainer outcomes are promising, so the CONDITIONAL verdict stands. The authors should soften the claim to 'report positive participant experiences in two full-scale offerings' and commit to validated, controlled evaluation before claiming demonstrated learning outcomes.","tokens_in":10245,"tokens_out":14416,"duration_ms":177340,"concrete_test":"Conduct one additional iteration with two additions: (a) an objective AI-literacy knowledge quiz aligned to the five module learning objectives, and (b) a no-treatment control group of comparable librarians, both assessed at the same pre/post times as the existing self-report; if the treated group does not outperform the control on the objective quiz, or if self-reported gains are not corroborated by objective quiz gains, the claim of demonstrated learning is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, stated in the Introduction rather than the abstract, is that the course 'demonstrate[s] positive learning outcomes and strong participant engagement.' The only quantitative evidence is self-rated 'understanding of AI' on a 1–10 scale, collected pre/post in Figures 4–5 (Summer 2023: 4.08→6.33; Fall 2023: 4.19→5.89). No objective knowledge assessment, control group, or inferential statistic is reported; the word 'significantly' appears without p-values, standard deviations, or effect sizes. Self-reported understanding is a proxy for confidence, not competence, and the Summer/Fall 2023 period coincides with intense public discourse on generative AI, so gains cannot be attributed to the course. The paper's own conclusion concedes 'there is a need to rigorously evaluate the effectiveness of educational methodologies,' directly undercutting the 'demonstrate' language. Additionally, the Introduction cites 'three iterations' with positive outcomes, but outcome data are presented only for the second and third iterations; the first (Spring 2021) is described in a paragraph with no evaluation measures. At best, the data support participant satisfaction and self-perceived improvement, not demonstrated learning.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'We Are AI: Taking Control of Technology,' a five-module, open-access course on Responsible AI designed for non-technical audiences and delivered in a peer-learning 'learning circle' format. It describes the course rationale, materials (including a multilingual comic series), the train-the-trainer mechanism, and three iterations: a Spring 2021 pilot with Queens Public Library, a Summer 2023 in-person offering to 15 NYU librarians/staff, and a Fall 2023 online offering to 26 NYU Abu Dhabi professionals. The authors report pre/post self-assessed understanding of AI (Figures 4 and 5), with weighted averages rising from 4.08 to 6.33 and from 4.19 to 5.89, plus qualitative quotes from participants, and they draw lessons learned for future iterations. The Introduction claims the course 'demonstrate[s] positive learning outcomes and strong participant engagement'; the Conclusion more cautiously acknowledges the need for rigorous evaluation.","tokens_in":10396,"tokens_out":5349,"duration_ms":59653,"significance":"If the central interpretation were supported, the paper would offer a useful, low-cost, scalable model for raising AI and Responsible AI literacy among non-experts, with concrete open materials (course website, facilitator guide, trilingual comics) and a plausible train-the-trainer pathway. The pedagogical design—intentional scaffolding, case studies, peer facilitation—is well-motivated by the learning-circles literature, and the paper's candor about gaps (generative AI coverage, facilitator scaffolding, cultural contextualization) adds practical value. The empirical evidence, however, is suggestive rather than demonstrative: self-report measures, small matched samples (12 and 18), no control group, and no inferential statistics. The paper's lasting value is as a detailed design/resource report; the evaluation claims will need to be substantially tempered or strengthened before publication.","major_comments":[{"comment":"The Introduction's claim that the paper 'report[s] on three iterations of the course, demonstrating positive learning outcomes and strong participant engagement' is not supported by the evidence presented in 'Learning and Engagement around Ethical AI.' The only quantitative outcome is a pre/post self-rating of understanding of AI on a 10-point scale (Figures 4 and 5), completed by 12 (Summer 2023) and 18 (Fall 2023) matched participants. No objective knowledge assessment, control or comparison group, or inferential statistic is reported; the word 'significantly' appears in the text without p-values, confidence intervals, or effect sizes. Self-reported understanding is at best a proxy for confidence rather than competence, and the Summer/Fall 2023 period coincides with intense public discourse about generative AI, so the observed gains cannot be unambiguously attributed to the course. The authors should either provide appropriate statistical analyses and a consideration of confounds, or rephrase the claim as reporting preliminary, self-reported indications of learning.","section":"Introduction; Learning and Engagement around Ethical AI"},{"comment":"The manuscript says it reports on 'three iterations,' but outcome data are presented only for the second and third iterations. The Spring 2021 iteration is described in the 'Course timeline' paragraph without any evaluation measures; the subsequent quantitative and qualitative results concern only Summer 2023 and Fall 2023. The authors should clarify which iterations have formal evaluation data, and avoid implying that the first iteration contributed to the reported 'positive learning outcomes.'","section":"Teaching RAI in a Peer Learning Setting, 'Course timeline' paragraph"},{"comment":"The Conclusion states: 'Additionally, there is a need to rigorously evaluate the effectiveness of educational methodologies for these audiences.' This statement is in direct tension with the 'demonstrate' language used in the Introduction. Because the paper's own concluding assessment concedes that rigorous evaluation is still needed, the empirical claims in the Introduction and abstract should be calibrated to match, e.g., by describing the results as encouraging preliminary evidence rather than demonstrated outcomes.","section":"Conclusion and Next Steps"}],"minor_comments":[{"comment":"Typo 'hile' should be 'while' in 'benefiting diverse stakeholders hile controlling'; also 'We are AIto' is missing a space before 'to an active and engaged group'.","section":"Abstract"},{"comment":"The phrase 'yielding an 85.7% completion rate between pre- and post-survey)' has a stray closing parenthesis; also 'weighted average' is used but the weighting scheme is never defined, and Figures 4 and 5 do not show paired pre/post responses, making the reported averages hard to evaluate.","section":"Learning and Engagement around Ethical AI"},{"comment":"The sentence beginning 'emerging, we are developing a module...' is a fragment; it should read something like 'As these technologies are emerging, we are developing a module...'.","section":"Conclusion and Next Steps"},{"comment":"In 'Need for additional scaffolding...', the sentence 'Another noted the importance for a good learning experience' seems to be missing a noun (e.g., 'the importance of the learning circle for a good learning experience').","section":"Lessons Learned"},{"comment":"The 'Course timeline' paragraph says 'All iterations of our course targeted librarians as the primary audience,' yet the Fall 2023 sample includes administrators, designers, and technologists (Table 1); consider acknowledging this composition or adjusting the description.","section":"Teaching RAI in a Peer Learning Setting"}],"recommendation":"major_revision","confidential_remarks":"The paper is best viewed as an experience/design report rather than an evaluation study. The self-citation pattern is dense but not inappropriate for a continuing line of work on responsible AI education. My main concern for the editor is that the 'demonstrate' language, if left unchanged, could mislead readers about the strength of the evidence; revision should bring the claims in line with the reported data. The paper fits a journal that publishes case studies in computing education or AI ethics education."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nShort version: this is a useful experience report introducing an open, peer-learning course on responsible AI for non-experts, with comics and facilitator guides in three languages. The course itself is a real contribution. But the paper's claim to \"demonstrate positive learning outcomes\" is not supported by the data. The evaluation is a small pre/post self-assessment with no control group, no objective knowledge test, and no statistical analysis. Read it for the resource, not for the evidence.\n\nWhat's new: RAI courses for non-technical audiences are scarce, and this one is designed around learning circles with a train-the-trainer mechanism, which is a sensible way to scale. The materials are public, modular, and the comics are a nice touch. The paper documents three iterations and gives honest lessons learned.\n\nWhere it's soft: The quantitative evidence consists of self-rated understanding of AI going from about 4 to 6 on a 10-point scale in two cohorts (n=12 and n=18). That's it. The word \"significantly\" appears without any inferential statistics. Self-reported understanding tracks confidence, not competence, and the summer/fall 2023 period coincided with heavy media coverage of generative AI, so the gains can't be cleanly attributed to the course. The paper's own conclusion admits rigorous evaluation is still needed, which undercuts the \"demonstrated\" language in the introduction. Also, the intro says \"three iterations\" with positive outcomes, but outcome data are only reported for the second and third; the first is described without evaluation measures.\n\nNone of this sinks the resource. The materials are worth adopting, and the train-the-trainer observations are suggestive. But the empirical claims need to be scaled back to \"participants reported increased confidence and engagement,\" with the positive anecdotes clearly labeled as such. Future work should include a knowledge test, a control or comparison group, and proper statistical reporting.\n\nWho it's for: educators, librarians, and community organizers working on AI literacy. They should definitely know about this course. As a paper, it deserves a serious referee, but the authors should be pushed to revise the claims and strengthen the evaluation before acceptance.\n\nRecommendation: send to peer review with a request for major revision, mainly on the evidence claims. The course itself is the contribution.","headline":"A genuinely useful open resource for public RAI education, but the empirical claims in the paper run ahead of the evidence.","tokens_in":10956,"tokens_out":2642,"would_cite":true,"duration_ms":26308,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces a free, five-module course that teaches responsible AI to non-experts through peer-led learning circles, and reports that participants' self-rated understanding of AI rose from about 4 to about 6 on a 10-point scale.","keywords":["responsible AI literacy","learning circles","train-the-trainer","public engagement with AI","non-expert stakeholders","open educational resources","AI ethics education"],"falsifier":"A credible test would give participants a short objective quiz on AI concepts before and after the course, alongside a matched control group that does not take the course; the central claim would be undercut if quiz gains were small or no larger than the control group's gains during the same window.","tokens_in":10024,"feed_emoji":"🎓","tokens_out":4367,"duration_ms":45475,"temperature":0.7,"pith_summary":"The paper introduces \"We Are AI: Taking Control of Technology,\" a free five-module course that teaches responsible AI to people without technical backgrounds through small peer-led groups called learning circles. It argues that this format, with prepared materials and facilitators who need not be experts, produces measurable gains: participants' self-rated understanding of AI rose from about 4.1 to about 6.1 on a 10-point scale across two offerings, alongside reports of strong engagement. The authors' broader claim is that this low-cost, sustainable model—where learners can become future facilitators—offers a realistic path to raising public AI literacy and giving non-experts agency over AI in their lives. A sympathetic reader would take the course design and the qualitative feedback as the core contribution, with the survey numbers as supporting evidence for the model's promise.","feed_headline":"A 5-week learning circle raises non-experts' AI understanding","feed_subtitle":"Self-rated understanding jumped from about 4 to about 6 on a 10-point scale for librarians and staff.","key_machinery":"The load-bearing mechanism is the learning circle, a pedagogical model in which small groups of people voluntarily come together to learn a topic collectively, without a traditional teacher. In this course, facilitators organize meetings and guide discussion using prepared materials but are not required to have deep AI expertise, which makes the format replicable. The train-the-trainer design is embedded: participants are encouraged and, as observed in the two offerings, actually do step up to facilitate subsequent circles, allowing the course to scale with minimal resources. The course's modular structure—videos, discussion prompts, hands-on activities, and a multilingual comic series—keeps the barrier to entry low and the format engaging.","core_discovery":"The central claim is that a carefully scaffolded, discussion-based learning circle can close the gap between technical AI expertise and public understanding without requiring expert instructors. The paper shows that over five 90-minute sessions covering what AI is, learning from data, ethical tradeoffs, bias, and civic action, librarians and professional staff increased their self-reported understanding of AI and responsible AI. In Summer 2023 the weighted average rose from 4.08 to 6.33, and in Fall 2023 from 4.19 to 5.89, with two-thirds of respondents saying the course improved their understanding substantially. Participants also absorbed key concepts—mentioning bias, transparency, and regulation in their post-course definitions—and reported intentions to teach others, demonstrating the train-the-trainer mechanism. The paper's claim is that this model is sustainable and scalable: anyone can run the course with the public materials, and learners naturally become future facilitators.","pith_inferences":["Because the evidence is self-reported, an objective knowledge test or a control group not exposed to the course could determine whether the gains reflect real learning or are inflated by participants' desire to improve—an extension the paper does not attempt.","The course's deliberately low technical ceiling may limit how far it can take motivated learners; a follow-on intermediate course, which the authors acknowledge, would be a natural test of whether the model retains its benefits at higher levels.","The model may transfer to other policy-adjacent literacies, such as data privacy or algorithmic governance, for which scaffolded peer-led materials could be built—an inference extending beyond the paper's stated scope.","If the train-the-trainer dynamic generalizes beyond librarians, the course could become a decentralized public-infrastructure approach to responsible-AI literacy."],"forward_implications":["If the model works, public libraries, community organizations, and universities can offer responsible-AI education with little more than internet access and a willing facilitator.","Learners who complete the course will be able to critique AI applications in hiring, education, and law enforcement using concepts like bias, transparency, and stakeholder impact.","The train-the-trainer effect means each cohort can seed the next, making the course self-propagating rather than dependent on a fixed set of instructors.","The authors plan to add modules on generative AI, tailored to librarians, administrators, and educators, addressing the gap participants flagged.","The openly available materials, including the comic series in English, Spanish, and Ukrainian, support independent study beyond the course itself."],"supporting_citations":[{"why":"Defines the learning circle model that the course is built on.","marker":"(Collay 1998)"},{"why":"Supplies the stakeholder-first AI literacy framing that motivates targeting non-experts.","marker":"(Domínguez Figaredo and Stoyanovich 2023)"},{"why":"Prior university RAI course that the authors contrast with the open learning-circle format.","marker":"(Lewis and Stoyanovich 2022)"},{"why":"Provides the pre-existing, technical, and emergent bias taxonomy used in Module 4.","marker":"(Friedman and Nissenbaum 1996)"},{"why":"Grounds the case-study pedagogy used throughout the modules.","marker":"(Kreber 2001)"},{"why":"Supports the stakeholder-first perspective that shapes course goals.","marker":"(Bell, Nov, and Stoyanovich 2023b)"}],"fun_headline_variants":["Learning circle lifts non-experts' AI savvy","5-week course boosts AI understanding in librarians","Peer-led circle raises AI literacy for non-techies","Non-experts gain AI insight via 5-week circle","Librarians up AI understanding after 5-week circle"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation treats participants' self-reported understanding on a 1–10 scale as a valid measure of real learning, and assumes the gains came from the course rather than from the heavy media coverage of AI during the same period.","fun_headline_variants_meta":{"raw":{"variants":["Learning circle lifts non-experts' AI savvy","5-week course boosts AI understanding in librarians","Peer-led circle raises AI literacy for non-techies","Non-experts gain AI insight via 5-week circle","Librarians up AI understanding after 5-week circle"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1231,"prompt_tokens":905,"completion_tokens":326,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":251}},"tokens_in":521,"tokens_out":326,"duration_ms":4658,"temperature":1.0,"reasoning_tokens":251,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:18:46.742466+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A credible test would give participants a short objective quiz on AI concepts before and after the course, alongside a matched control group that does not take the course; the central claim would be undercut if quiz gains were small or no larger than the control group's gains during the same window.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the learning circle model that the course is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the stakeholder-first AI literacy framing that motivates targeting non-experts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior university RAI course that the authors contrast with the open learning-circle format."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pre-existing, technical, and emergent bias taxonomy used in Module 4."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the case-study pedagogy used throughout the modules."}],"review_version":1}