Pith. sign in

REVIEW 4 major objections 4 minor 59 references

Thinking Like a Scientist: Can Interactive Simulations Foster Critical AI Literacy?

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Interactive simulations can teach critical AI literacy and transfer to new AI scenarios.

desk verdict A mostly well-designed preregistered study whose central claim doesn't survive the uncalibrated pre/post items; the real finding is that interactive Explorables are at least as good as static, not that they're better. read the letter →

arxiv 2507.21090 v1 pith:USINA3FE submitted 2025-06-25 cs.HC cs.AI

Yiling Zhao , Audrey Michal , Nithum Thain , Hari Subramonyam This is my paper · ORCID
classification cs.HCcs.AI
keywords criticalAIliteracyinteractivesimulationExplorableexplanationsscientificdiscoverylearningknowledgetransferalgorithmicfairnesseducationhuman-computerinteraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether interactive simulations can teach people to think critically about AI systems. In a controlled study with over 600 participants, it compares interactive 'Explorable' tutorials against static PDFs and a no-instruction control across four AI topics. The authors report that the interactive condition produced significant learning gains, stronger transfer to unfamiliar AI scenarios, and more consistent self-reported confidence gains. They conclude that interactive, inquiry-driven formats are an effective way to build critical AI literacy, while cautioning that raw engagement metrics do not predict learning.

What carries the argument

The central object is the 'Explorable': a web-based interactive article that combines explanatory text with adjustable parameters, dynamic visualizations, and real-time feedback, letting learners manipulate inputs and observe AI outputs. The paper builds on scientific discovery learning (SDL), the idea that learners acquire deeper understanding by actively testing hypotheses and observing results, which the authors argue supports the observed gains in critical AI literacy and transfer.

What would settle it

A replication that counterbalances the scenario items across pre- and post-tests and equates item difficulty with item response theory would falsify the transfer claim if the interactive condition no longer outperformed the no-instruction control on non-target questions.

Watch

Extended reading notes

Core claim

The paper's central claim is that interactive simulations—Explorable explanations that let users adjust parameters, test hypotheses, and observe AI behavior in real time—enhance critical AI literacy. The evidence comes from a preregistered controlled study in which participants who used the Explorables improved on scenario-based assessments of AI issues, showed comparable or better performance on non-target transfer questions than controls, and reported increased confidence in their AI literacy. The authors attribute this to scientific discovery learning: engagement through experimentation and direct observation fosters conceptual understanding and transfer. They also find that the amount of interaction does not predict learning, suggesting that the quality of engagement matters more than its quantity.

Load-bearing premise

The learning and transfer measures rely on scenario questions that were not item-calibrated, and the pre- and post-tests used different question sets, so the observed gains could reflect item difficulty differences or practice effects rather than the intervention.

Editorial extensions

If this is right

  • AI literacy instruction should incorporate interactive, inquiry-driven materials, since interactive engagement produced consistent learning gains across topics.
  • Passive exposure (reading or no instruction) may not be enough to build transferable critical thinking about AI, as the no-instruction control showed weaker generalization to new scenarios.
  • Educational tool designs should prioritize meaningful interaction over raw activity, because interaction counts were not reliable predictors of learning.
  • Topics like large language models may require additional scaffolding when taught interactively, since performance on that topic declined.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors do not pursue is to use the same Explorables as a reusable critical-literacy curriculum, swapping in topic-specific cases rather than building new materials each time.
  • A testable follow-up is to capture richer process data—such as screen recordings or verbal protocols—to identify which interaction moments actually drive transfer, since scroll counts are weak predictors.
  • The interaction-quality finding suggests that adaptive systems could intervene when a learner is merely clicking without experimenting, potentially boosting learning gains beyond those reported here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This preregistered online experiment (n≈605) compares interactive 'Explorable' tutorials, static PDF tutorials, and a no-tutorial Basic Control across four AI topics (Worldview, Diversity, Fairness, LLM). Learning is measured with scenario-based multiple-choice questions (pre/post, target/non-target) plus self-reported AI literacy items from the MAILS scale. The paper claims that interactive simulations effectively enhance AI literacy across topics, support knowledge transfer to new scenarios, and increase self-reported confidence, while engagement quantity alone does not predict learning.

Significance. If substantiated, the paper would provide rare controlled evidence on whether interactive, inquiry-driven tutorials improve critical AI literacy beyond static materials, which would be a useful contribution to AI education. Strengths include the preregistered design, random assignment, multiple AI topics, use of real-world scenarios from the AI Incident Database, logging of user interactions, and transparent reporting of null between-condition regressions in the results. However, the central effectiveness and transfer claims are not currently supported by the primary analyses: the within-subject gains rest on non-equivalent pre/post item sets, the Basic Control also shows large gains, and the transfer analysis treats non-significance as evidence of transfer. The abstract and conclusions therefore overstate the findings.

major comments (4)
  1. [§4.1, Tables 3 and 4] The within-subject pre-to-post gains are not interpretable as learning because no scenario item appears in both pre-test and post-test within a topic; for example, in the Fairness group the pre-test uses Question Set 2 and the post-test uses Question Set 1 (Table 3). The Basic Control group, which received no instruction, improved on Fairness overall from 0.39 to 0.75 (p<.001) (Table 4), demonstrating that the post-test items are easier or that substantial practice effects are present. Since the between-condition OLS regression found no significant condition effects (Static β=-0.01, p=.642; Basic β=-0.04, p=.127), the abstract's claim that 'interactive simulations effectively enhance AI literacy across topics' is not supported by the primary analysis; item calibration or equating is needed before these gains can be attributed to the intervention.
  2. [§4.2, Table 1] The generalization analysis uses non-significance (p>0.05) as evidence of transfer, stating that 'effective generalization is indicated by statistically similar performance across target and non-target questions.' With roughly 50 participants per condition, these comparisons have low power, and no equivalence bounds, effect sizes, or item-difficulty controls are reported. The non-target questions are drawn from different topics and are uncalibrated, so equal mean scores may reflect item difficulty rather than conceptual transfer; therefore the claim that Explorables 'support greater knowledge transfer' is not established.
  3. [§4.1, Table 4] The LLM topic showed significant declines in the Explorable condition (overall 0.76 to 0.66, p=.01) and the Static condition (0.79 to 0.68, p=.01), with a non-significant decline in Basic Control (0.69 to 0.63, p=.11). This contradicts the abstract's 'across topics' generalization and should be acknowledged as a topic-dependent boundary condition, with the abstract and conclusion claims adjusted accordingly.
  4. [§5.4 Limitations] The Limitations section does not mention the non-equivalence of pre- and post-test items or the Basic Control's large gains on Fairness, even though these directly threaten the central effectiveness claim. The omitted limitation should be added, and the conclusions should be revised to reflect the null between-condition regression and the measurement confound.
minor comments (4)
  1. [Abstract and §3] The abstract states '605 participants' while §3 reports n=612 recruited, and the final valid samples per condition range from 47 to 51; please reconcile these numbers and provide a participant flow diagram.
  2. [Table 1] Table 1 presents significance with asterisks but no numeric p-values or test statistics for most cells, and the caption does not fully define all abbreviations; consider adding a supplementary table with means, standard deviations, and effect sizes.
  3. [§4.2] The phrase 'statistically similar performance (p>0.05)' should be replaced by proper equivalence-testing language or an explicit power analysis, since a non-significant difference is not evidence of similarity.
  4. [§3, Topic Selection] The text refers to 'Incident 375' for the Amazon recruiting scenario while footnote 5 links to incident 37; please verify and correct the incident identifiers.

Circularity Check

0 steps flagged · score 1.0 of 10

No notable circularity: central claims rest on external outcome measures and preregistered analyses, not on fitted parameters or self-citation chains.

full rationale

The paper's central claims are empirical rather than derivational. Learning outcomes are measured with the Meta AI Literacy Scale (an external, published instrument) and scenario questions adapted from the AI Incident Database, not defined in terms of the intervention. No parameter is fitted to a subset of data and then renamed as a prediction; the regression and correlation analyses are reported as exploratory or confirmatory statistical tests, and the paper honestly reports that between-condition OLS differences were not statistically significant. The 'generalization' criterion (p > 0.05 for target vs. non-target items) is a stated analytic choice, not a result smuggled in from the inputs. The Explorables are Google PAIR artifacts and one author is from Google Research, but the citation of the artifacts is descriptive (identifying the stimuli), not load-bearing evidence for the effectiveness claim; the evaluation is external to the lab and preregistered. The more serious concern identified by skeptical review is that pre-test and post-test use different, non-overlapping scenario question sets (Table 3 and Appendix D), and the Basic Control group's large Fairness gain (0.39 to 0.75, p < .001, Table 4) suggests item difficulty differences could confound the reported learning gains. That is a genuine internal-validity and item-calibration threat, but it is not a circularity: the outcome measure is not equivalent to the treatment by construction, and no claim reduces to its own input. Accordingly, the circularity score is 1, reflecting only a minor self-referential element (evaluating the authors' affiliated artifacts) that does not carry the argument.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The study's claims rest on the validity of the AI literacy measures and on standard statistical assumptions. No invented entities or fitted free parameters are introduced; the analysis uses preregistered thresholds.

assumptions (3)
  • domain assumption The Meta AI Literacy Scale (MAILS) items are valid indicators of self-assessed AI literacy.
    Used as the self-report outcome in pre/post tests (Section 3).
  • domain assumption The four AI scenario question sets are comparable in difficulty and measure the same construct of critical AI literacy.
    Generalization analysis compares target vs non-target accuracy; no difficulty calibration is reported (Section 3.1, Appendix D).
  • standard math Ordinary least squares regression assumptions hold for the binary outcome scores (proportions).
    OLS used to predict post-test scores and non-target scores from pre-test and condition (Section 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Thinking Like a Scientist: Can Interactive Simulations Foster Critical AI Literacy?." pith.science (2026). https://pith.science/paper/USINA3FE

@misc{pith2026250721090,
  author       = {Pith},
  title        = {Pith review of: Thinking Like a Scientist: Can Interactive Simulations Foster Critical AI Literacy?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/USINA3FE}},
  note         = {Machine review of arXiv:2507.21090}
}
read the original abstract

As AI systems shape individual and societal decisions, fostering critical AI literacy is essential. Traditional approaches, such as blog articles, static lessons, and social media discussions, often fail to support deep conceptual understanding and critical engagement. This study examines whether interactive simulations can help learners think like a scientist by engaging them in hypothesis testing, experimentation, and direct observation of AI behavior. In a controlled study with 605 participants, we assess how interactive AI tutorials impact learning of key concepts such as fairness, dataset representativeness, and bias in language models. Results show that interactive simulations effectively enhance AI literacy across topics, supporting greater knowledge transfer and self-reported confidence, though engagement alone does not predict learning. This work contributes to the growing field of AI literacy education, highlighting how interactive, inquiry-driven methodologies can better equip individuals to critically engage with AI in their daily lives.

Figures

Figures reproduced from arXiv: 2507.21090 by the authors.

Figure 1
Figure 1. Explorable Illustration: Datasets Have Worldviews [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. AI Literacy Learning Gains Across Conditions [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Different user scroll patterns p = 0.024). These findings suggest that while overall engagement does not uni￾formly predict learning outcomes, the quality and type of engagement in specific tasks is meaningfully related to performance [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Correlation: Activity Counts vs Score Increment [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Demographic Distributions of Participants in the Explorable Condition [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Scroll Pattern Heatmaps and Cluster Centroids [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 58 canonical work pages

  1. [1]

    In: 2022 InternationalConferenceonSustainableComputingandDataCommunicationSys- tems (ICSCDS)

    Alam, A.: A digital game based learning approach for effective curriculum trans- action for teaching-learning of artificial intelligence and machine learning. In: 2022 InternationalConferenceonSustainableComputingandDataCommunicationSys- tems (ICSCDS). pp. 69–74. IEEE (2022)

    work page 2022

  2. [2]

    In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

    Anik, A.I., Bunt, A.: Data-centric explanations: explaining training data of ma- chine learning systems to promote transparency. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. pp. 1–13 (2021)

    work page 2021

  3. [3]

    In: Extended abstracts of the 2020 CHI conference on human factors in computing systems

    Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., Jonge- jan, J., Pitaru, A., Chen, A.: Teachable machine: Approachable web-based tool for exploring machine learning classification. In: Extended abstracts of the 2020 CHI conference on human factors in computing systems. pp. 1–8 (2020)

    work page 2020

  4. [4]

    Carolus, A., Koch, M., Straka, S., Latoschik, M.E., Wienrich, C.: Mails – meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta- competencies (2023)

    work page 2023

  5. [5]

    International Journal of STEM Education10(1), 29 (2023)

    Casal-Otero, L., Catala, A., Fernández-Morante, C., Taboada, M., Cebreiro, B., Barro, S.: Ai literacy in k-12: a systematic literature review. International Journal of STEM Education10(1), 29 (2023)

    work page 2023

  6. [6]

    Educational psychologist49(4), 219–243 (2014)

    Chi, M.T., Wylie, R.: The icap framework: Linking cognitive engagement to active learning outcomes. Educational psychologist49(4), 219–243 (2014)

    2014

  7. [7]

    Review of educational research68(2), 179–201 (1998)

    De Jong, T., Van Joolingen, W.R.: Scientific discovery learning with computer simulations of conceptual domains. Review of educational research68(2), 179–201 (1998)

    work page 1998

  8. [8]

    Machine Learning and Knowledge Extraction 6(2), 1114–1125 (2024)

    De Silva, D., Jayatilleke, S., El-Ayoubi, M., Issadeen, Z., Moraliyage, H., Mills, N.: The human-centred design of a universal module for artificial intelligence literacy in tertiary education institutions. Machine Learning and Knowledge Extraction 6(2), 1114–1125 (2024)

    work page 2024

Show all 59 references
  1. [9]

    Dignum, V.: Responsible artificial intelligence: how to develop and use AI in a responsible way, vol. 2156. Springer (2019)

    2019

  2. [10]

    Goeser, P.T., Hamza-Lup, F.G., Johnson, W.M., Scharfer, D.: View: A virtual interactive web-based learning environment for engineering (2018)

    2018

  3. [11]

    https://pair.withgoogle.com/explorables/ (2025), accessed: 2025-01-22

    Google: Explorables - google pair. https://pair.withgoogle.com/explorables/ (2025), accessed: 2025-01-22

    2025

  4. [12]

    Springer (2014)

    Griffin, P., Care, E.: Assessment and teaching of 21st century skills: Methods and approach. Springer (2014)

    2014

  5. [13]

    Academic Medicine77(3), 263–265 (2002)

    Hallgren, R.C., Parkhurst, P.E., Monson, C.L., Crewe, N.M.: An interactive, web- based tool for learning anatomic landmarks. Academic Medicine77(3), 263–265 (2002)

    2002

  6. [14]

    In: International Conference on Artificial Intelligence in Education

    Hidalgo, C.: Agency in ai and education policy: European resolution three on har- nessing the potential for ai in and through education. In: International Conference on Artificial Intelligence in Education. pp. 319–327. Springer (2024)

    2024

  7. [15]

    In: Proceedings of the 2019 CHI conference on human factors in computing systems

    Hitron, T., Orlev, Y., Wald, I., Shamir, A., Erel, H., Zuckerman, O.: Can children understand machine learning concepts? the effect of uncovering black boxes. In: Proceedings of the 2019 CHI conference on human factors in computing systems. pp. 1–11 (2019)

    2019

  8. [16]

    In: 2019 IEEE blocks and beyond workshop (B&B)

    Jatzlau, S., Michaeli, T., Seegerer, S., Romeike, R.: It’s not magic after all–machine learning in snap! using reinforcement learning. In: 2019 IEEE blocks and beyond workshop (B&B). pp. 37–41. IEEE (2019) Interactive Simulations for Critical AI Literacy 15

    2019

  9. [17]

    In: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V

    Kasinidou, M., Kleanthous, S., Orphanou, K., Otterbacher, J.: Educating com- puter science students about algorithmic fairness, accountability, transparency and ethics. In: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1. ...

    2021

  10. [18]

    International Journal of Child-Computer Interaction34, 100539 (2022)

    Kaspersen, M.H., Bilstrup, K.E.K., Van Mechelen, M., Hjort, A., Bouvin, N.O., Pe- tersen, M.G.: High school students exploring machine learning and its societal im- plications: Opportunities and challenges. International Journal of Child-Computer Interaction34, 100539 (2022)

    2022

  11. [19]

    In: International Conference on Artificial Intelligence in Education

    Kim, H.J., So, H.J., Suh, Y.J.: Examining the impact of flipped learning for de- veloping young job seekers’ ai literacy. In: International Conference on Artificial Intelligence in Education. pp. 817–823. Springer (2023)

    2023

  12. [20]

    In:Proceedingsofthe2020CHIconferenceonhumanfactorsincomputingsystems

    Long,D.,Magerko,B.:Whatisailiteracy?competenciesanddesignconsiderations. In:Proceedingsofthe2020CHIconferenceonhumanfactorsincomputingsystems. pp. 1–16 (2020)

    2020

  13. [21]

    Journal of research on com- puting in education33(1), 55–76 (2000)

    Mioduser, D., Nachmias, R., Lahav, O., Oren, A.: Web-based learning environ- ments: Current pedagogical and technological state. Journal of research on com- puting in education33(1), 55–76 (2000)

    2000

  14. [22]

    Royal Society Open Science12(1), 241482 (2025)

    Newman-Griffis, D.: Ai thinking: a framework for rethinking artificial intelligence in practice. Royal Society Open Science12(1), 241482 (2025)

    2025

  15. [23]

    Computers and Education: Artificial Intelligence2, 100041 (2021)

    Ng, D.T.K., Leung, J.K.L., Chu, S.K.W., Qiao, M.S.: Conceptualizing ai literacy: An exploratory review. Computers and Education: Artificial Intelligence2, 100041 (2021)

    2021

  16. [24]

    Academic Medicine76(5), 550 (2001)

    PARKER, M.J., SEIFTER, J.L.: An interactive, web-based learning environment for pathophysiology. Academic Medicine76(5), 550 (2001)

    2001

  17. [25]

    AI perception and AI literacy requirements (November 01, 2024) (2024)

    Popa, D.M.: Critical exploratory investigation of ai consumption, ai perception and ai literacy requirements. AI perception and AI literacy requirements (November 01, 2024) (2024)

    2024

  18. [26]

    https://www.prolific.com/ (2025), accessed: 2025-01-22

    Prolific: Prolific - participant recruitment for online research. https://www.prolific.com/ (2025), accessed: 2025-01-22

    2025

  19. [27]

    Raub, M.: Bots, bias and big data: artificial intelligence, algorithmic bias and disparate impact liability in hiring practices. Ark. L. Rev.71, 529 (2018)

    2018

  20. [28]

    Journal of surgical research134(1), 102–106 (2006)

    Servais, E.L., LaMorte, W.W., Agarwal, S., Moschetti, W., Mallipattu, S.K., Moul- ton, S.L.: Teaching surgical decision-making: an interactive, web-based approach. Journal of surgical research134(1), 102–106 (2006)

    2006

  21. [29]

    In: International Conference on Artificial Intelligence in Education

    Shved, E., Bumbacher, E., Mejia-Domenzain, P., Kapur, M., Käser, T.: Teach- ing and measuring multidimensional inquiry skills using interactive simulations. In: International Conference on Artificial Intelligence in Education. pp. 482–496. Springer (2024)

    2024

  22. [30]

    In: International Conference on Artificial Intelligence in Education

    Sneirson, M., Chai, J., Howley, I.: A learning approach for increasing ai literacy via xai in informal settings. In: International Conference on Artificial Intelligence in Education. pp. 336–343. Springer (2024)

    2024

  23. [31]

    artificial intelligence

    Wan, X., Zhou, X., Ye, Z., Mortensen, C.K., Bai, Z.: Smileycluster: supporting accessible machine learning in k-12 scientific discovery. In: Proceedings of the in- teraction design and children conference. pp. 23–35 (2020) 16 Y. Zhao et al. A Participant Demographics Fig.5: De...

    2020

  24. [32]

    This system was trained using resumes submitted by all applicants over the past 10 years

    Imagine a leading technology company using an AI system that automati- cally sifts through resumes to identify the most promising candidates. This system was trained using resumes submitted by all applicants over the past 10 years. Which group of candidates is most likely to b...

  25. [33]

    Considering this, what is a more equitable approach to using AI in the recruiting process? Select all that apply

    Given that the tech industry is predominantly male, and the majority of the resumes used for training came from men, the recruitment system has started to show a preference for male candidates over female candidates. Considering this, what is a more equitable approach to using...

  26. [34]

    This algorithm assigns each cou- ple a ’Red’ or ’Green’ rating

    Imagine the National Home Office deploying an algorithm to detect sham marriages, where couples get married to avoid immigration law, rather than because they have a genuine relationship. This algorithm assigns each cou- ple a ’Red’ or ’Green’ rating. A ’Red’ rating suggests t...

  27. [35]

    What measures could be implemented to refine the algorithm and ensure it more accurately and equitably identifies sham marriages, without perpetu- ating biases or overlooking evolving patterns? Select all that apply. Interactive Simulations for Critical AI Literacy 21 □Regular...

  28. [36]

    Imagine a leading technology and online retail company using an algorithm to oversee and evaluate the performance of contract delivery drivers. Drivers are assessed based on a range of variables, including on-time performance, how well the package is hidden from street view, a...

  29. [37]

    □The idea of increasing the frequency of evaluations to weekly, assuming that more data points will automatically lead to fairer assessments

    What steps can be taken to enhance the fairness and accuracy of the al- gorithmic evaluation system for contract drivers, ensuring it accounts for external factors and provides equitable assessments? Select all that apply. □The idea of increasing the frequency of evaluations t...

  30. [38]

    A network of 22 acoustic listening devices, fine-tuned to distinguish gunshots from other sounds, has been set up to cover neighbor- hoods with the highest crime rates

    Imagine a police department using a computerized monitoring system to locate gunshots. A network of 22 acoustic listening devices, fine-tuned to distinguish gunshots from other sounds, has been set up to cover neighbor- hoods with the highest crime rates. Identify the FALSE st...

  31. [39]

    □Focus solely on the technological aspects to improve the system’s accu- racy and efficiency

    Considering the concerns around the deployment of acoustic listening devices to detect gunshots in certain neighborhoods, what steps could be taken to address the fairness and effectiveness issues identified? Select all that apply. □Focus solely on the technological aspects to...

  32. [40]

    What potential issues could arise from this scenario? Select all that apply

    Imagine a lawyer employing a large language model like ChatGPT to draft a legal brief for court, wherein the model cites previous cases. What potential issues could arise from this scenario? Select all that apply. □AI-Enhanced Persuasion: The AI’s use of language is so persuas...

  33. [41]

    □Use LLMs for initial case law research to identify relevant precedents before thorough verification by legal professionals

    Considering the scenario where a lawyer might use a large language model (LLM) like ChatGPT to draft a legal brief for court, which includes citing non-existing cases, how could LLMs be more effectively and appropriately utilized in this context? Select all that apply. □Use LL...

  34. [42]

    Despite its promising capabilities, the company decommissioned the model just three days after its launch

    Imagine a leading technology company has developed a large language model designedtogeneratesummariesofscientificliterature.Thismodelwastrained on a vast and carefully curated corpus of humanity’s scientific knowledge, comprising over 48 million papers, textbooks, lecture note...

  35. [43]

    □Implement stringent quality control measures, including expert review processes, to verify the accuracy and relevance of the content generated by the model before publication

    Given the scenario of the decommissioned large language model designed for generating scientific literature, how could large language models be more effectively and responsibly used to assist in scientific research? Select all that apply. □Implement stringent quality control m...

  36. [44]

    Before a person steps into the full-body scan- ner,aTSAofficermustregistertheperson’sgenderbypressingapinkbutton for female or a blue button for male

    Imagine the Transportation Security Administration (TSA) uses an image- processing body scanner at airport checkpoints, to detect potential threats that are not necessarily metal. Before a person steps into the full-body scan- ner,aTSAofficermustregistertheperson’sgenderbypres...

  37. [45]

    □Automate the gender identification process using facial recognition tech- nology to determine a passenger’s gender, thereby eliminating the need for manual selection by TSA agents

    Considering the limitations and challenges posed by the current binary gen- der selection system (pink button for female, blue button for male) used in TSA’s image-processing body scanners at airport checkpoints, what al- ternative labeling or identification method would be mo...

  38. [46]

    What potential issues do you see with this system? Select all that apply

    Imagine a prestigious university using an algorithm to assess applicants for a PhDintherespectedcomputersciencedepartment.Thesoftwarewastrained using details from previously accepted students, aiming to teach the system to identify candidates the school would likely favor and ...

  39. [47]

    □Train the model to rely solely on quantitative metrics such as GPA and standardized test scores when selecting top candidates to avoid potential biases related to diversity

    Given the potential issues identified with using a machine learning algorithm for assessing PhD applicants based on data from previously accepted stu- dents, what would be a better approach for the university to take to address these concerns while still leveraging technology ...

  40. [48]

    How do you describe yourself? □Male □Female □Non-binary / third gender □Prefer to self-describe □Prefer not to say

  41. [49]

    In which country do you currently reside?

  42. [50]

    What is the highest level of education you have completed? □Some high school or less □High school diploma or GED □Some college, but no degree □Associates or technical degree □Bachelor’s degree □Graduate or professional degree (MA, MS, MBA, PhD, JD, MD, DDS etc.) □Prefer not to say

  43. [51]

    Choose one or more races that you consider yourself to be □White or Caucasian □Black or African American □American Indian/Native American or Alaska Native □Asian □Native Hawaiian or Other Pacific Islander Interactive Simulations for Critical AI Literacy 27 □Other □Prefer not to say

  44. [53]

    Explorable Group Target AI Question Set F Post-Test Questionnaire

  45. [54]

    Explorable Group Target AI Question Set

  46. [55]

    Non-target AI Question Set 1

  47. [56]

    Non-target AI Question Set 2

  48. [57]

    Non-target AI Question Set 3

  49. [58]

    Reinforcement Learning

    To ensure you’re paying attention, please select "Reinforcement Learning" from the list below. ⃝Supervised Learning ⃝Unsupervised Learning ⃝Reinforcement Learning ⃝Transfer Learning Note: The order of these question sets (items 1-5) was randomized for each participant

  50. [59]

    To what extent do you agree or disagree with the following statements: AI Literacy Scale (2)

  51. [60]

    Zhao et al

    What other questions you might have but were not answered by the ex- plorable explanation? 28 Y. Zhao et al. G Tables Table 4: Performance Metrics Across Conditions Condition AI Topic Identify Issue Propose Solutions Overall Pre Post P-value Pre Post P-value Pre Post P-value E...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.