REVIEW 3 major objections 2 minor 6 references
How do machines learn? Evaluating the AIcon2abs method
T0 review · 3 major / 2 minor · reviewed 2026-05-24 · grok-4.3
Pith's one-line read The AIcon2abs method using WiSARD raises participant satisfaction with understanding machine learning processes in a short hands-on course.
desk verdict Satisfaction ratings after one 6-hour session do not show objective gains in ML understanding. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The WiSARD weightless neural network, which supports intuitive simulation of training and classification through physical or low-resource activities without needing internet.
What would settle it
A controlled pre-post test on machine learning concepts showing no greater improvement for course participants than for a matched group that received no instruction.
Extended reading notes
Core claim
AIcon2abs enables users to visualize and interact with machine learning by embodying the WiSARD algorithm in hands-on activities, observing incremental accuracy gains from minimal data and inspecting generated mental images that highlight essential data features. Testing in a course with 34 participants produced high satisfaction ratings that indicate the method achieved its intended educational outcomes.
Load-bearing premise
Positive satisfaction ratings from a single short course with self-selected participants reflect lasting gains in conceptual understanding of machine learning.
Editorial extensions
If this is right
- The method can reach K-12 students and other non-technical groups across age ranges.
- It functions in settings without reliable internet because WiSARD runs locally from small data sets.
- Participants can directly observe accuracy improving example by example.
- Generated mental images make visible which features the system has extracted from the data.
Reading between the lines
- If the satisfaction results hold in repeated trials, the format could be adapted for larger online or in-person workshops.
- Similar low-tech simulations might extend to teaching other AI topics such as decision trees or reinforcement learning.
- Follow-up assessments weeks later would clarify whether conceptual gains persist beyond the immediate session.
- Direct comparison with lecture-only formats would isolate the contribution of the embodied activities.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the AIcon2abs educational method, which uses the WiSARD weightless neural network to enable hands-on visualization of ML training and classification processes without requiring internet or large datasets. It evaluates the method via a six-hour remote course with 34 self-selected Brazilian participants (including children and adolescents), employing a mixed-method pre-experiment that includes hypothesis testing plus qualitative phenomenological analysis, and reports that nearly all participants gave positive ratings indicating high satisfaction with outcomes related to increased ML understanding.
Significance. If the evaluation were to demonstrate objective gains in conceptual understanding of ML processes, the method would represent a useful contribution to accessible AI education, particularly its emphasis on a simple, interpretable algorithm suitable for resource-limited settings and diverse age groups. The hands-on, incremental learning design using minimal examples is a clear strength for demystifying black-box aspects of ML.
major comments (3)
- [Abstract] Abstract: The central claim that the results demonstrate achievement of the 'intended outcomes' of increasing understanding of machine learning is not supported by the reported data, which consists solely of satisfaction ratings and phenomenological analysis; no pre/post objective knowledge measures, control condition, or retention assessment are described.
- [Methods] Methods (mixed-method pre-experiment description): The abstract states that hypothesis testing was included, but provides no details on the specific hypotheses, instruments for assessing conceptual understanding, statistical power, data exclusion criteria, or inter-coder reliability for the qualitative component, rendering the design's ability to support the understanding claim unverifiable.
- [Results] Results and Discussion: The sample of 34 self-selected participants in a single six-hour session (including 5 children) lacks a control group or comparison to alternative teaching methods, so positive ratings cannot be attributed specifically to gains in ML conceptual understanding rather than engagement or novelty effects.
minor comments (2)
- [Abstract] Abstract: The ethics approval statement references CEP-HUCFF-UFRJ but omits the protocol number and approval date.
- [Introduction] Introduction: Repetitive phrasing appears when describing WiSARD's ability to learn from minimal data and generate mental images.
Simulated Author's Rebuttal
We thank the referee for these constructive comments on the evaluation design and claims. We agree that the current data do not support strong assertions about objective gains in ML understanding and will revise the manuscript to align claims with the satisfaction ratings and phenomenological findings actually reported.
read point-by-point responses
-
Referee: [Abstract] Abstract: The central claim that the results demonstrate achievement of the 'intended outcomes' of increasing understanding of machine learning is not supported by the reported data, which consists solely of satisfaction ratings and phenomenological analysis; no pre/post objective knowledge measures, control condition, or retention assessment are described.
Authors: We agree. The reported data consist of post-session satisfaction ratings and phenomenological analysis; no objective pre/post knowledge tests were administered. We will revise the abstract to state that participants reported high satisfaction with the method and that the phenomenological analysis provided insights into their experiences, removing any claim that the results demonstrate increased conceptual understanding of machine learning. revision: yes
-
Referee: [Methods] Methods (mixed-method pre-experiment description): The abstract states that hypothesis testing was included, but provides no details on the specific hypotheses, instruments for assessing conceptual understanding, statistical power, data exclusion criteria, or inter-coder reliability for the qualitative component, rendering the design's ability to support the understanding claim unverifiable.
Authors: The abstract's reference to hypothesis testing was intended to describe an exploratory analysis of the satisfaction scores. However, we acknowledge that the manuscript provides none of the requested methodological details. We will revise the abstract and methods section to remove the reference to hypothesis testing and to clarify that the instruments captured satisfaction and qualitative experience rather than direct measures of conceptual understanding. revision: yes
-
Referee: [Results] Results and Discussion: The sample of 34 self-selected participants in a single six-hour session (including 5 children) lacks a control group or comparison to alternative teaching methods, so positive ratings cannot be attributed specifically to gains in ML conceptual understanding rather than engagement or novelty effects.
Authors: We accept this limitation. The study was designed as a preliminary feasibility evaluation in a naturalistic remote setting rather than a controlled experiment. We will revise the Results and Discussion sections to state explicitly that positive ratings indicate satisfaction but cannot be attributed specifically to gains in conceptual understanding, as opposed to engagement or novelty, and to recommend controlled comparisons in future work. revision: yes
Circularity Check
No circularity: empirical evaluation of educational intervention
full rationale
The paper reports results from a mixed-methods evaluation of an educational method using participant satisfaction ratings and phenomenological analysis after a single session. No equations, derivations, fitted parameters, or predictions appear. The central claim rests on the collected data rather than reducing to self-citation or definitional equivalence. Prior work is cited only to introduce the method being evaluated; the evaluation itself is independent.
Assumptions & free parameters
Cite this review
Pith. "Pith review of How do machines learn? Evaluating the AIcon2abs method." pith.science (2026). https://pith.science/paper/QXF7SXSP
@misc{pith2026240107386,
author = {Pith},
title = {Pith review of: How do machines learn? Evaluating the AIcon2abs method},
year = {2026},
howpublished = {\url{https://pith.science/paper/QXF7SXSP}},
note = {Machine review of arXiv:2401.07386}
}
read the original abstract
This study expands on previous work that introduced the AIcon2abs method (AI from Concrete to Abstract: Demystifying Artificial Intelligence to the general public), an innovative approach designed to increase public understanding of machine learning (ML) across diverse age groups, including K-12 students, and aims to evaluate its effectiveness. AIcon2Abs employs the WiSARD algorithm, a weightless neural network known for its simplicity, and user accessibility. WiSARD does not require Internet, making it ideal for non-technical users and resource-limited environments. This method enables participants to intuitively visualize and interact with ML processes through engaging, hands-on activities, as if they were the algorithms themselves. The method allows users to intuitively visualize and understand the internal processes of training and classification through practical activities. Once WiSARDs functionality does not require an Internet connection, it can learn effectively from a minimal dataset, even from a single example. This feature enables users to observe how the machine improves its accuracy incrementally as it receives more data. Moreover, WiSARD generates mental images representing what it has learned, highlighting essential features of the classified data. AIcon2abs was tested through a six-hour remote course with 34 Brazilian participants, including 5 children, 5 adolescents, and 24 adults. Data analysis was conducted from two perspectives: a mixed-method pre-experiment (including hypothesis testing), and a qualitative phenomenological analysis. Nearly all participants rated AIcon2abs positively, with the results demonstrating a high degree of satisfaction in achieving the intended outcomes. This research was approved by the CEP-HUCFF-UFRJ Research Ethics Committee.
Reference graph
Works this paper leans on
-
[1]
AIcon2abs Empirical Evaluation Data
AIcon2abs (2023). AIcon2abs Empirical Evaluation Data. Mendeley Data, V1. https://doi.org/10.17632/ch6ys3gz82.1 Aleksander, I., De Gregorio, M., França, F.M.G., Lima, P.M.V., & Morton, H, (2009), A brief introduction to weightless neural systems. Proceedings of ESANN 2009, 299–305. Almeida, V. (2018, August 8). Brasil precisa de estratégia para inteligênc...
-
[2]
Academia Brasileira de Ciências
MCTIC lança consulta pública para a Estratégia Brasileira de Inteligência Artificial (Transl.: MCTIC launches public consultation for the Brazilian Artificial Intelligence Strategy). Academia Brasileira de Ciências. Retrieved January 28, 2024, from http://www.abc.org.br/2019/12/21/mctic- lanca-consulta-publica-para-a-estrategia-brasileira-de-inteligencia-...
work page 2024
-
[3]
https://doi.org/10.1063/1.2810904 Fleming, N.D. (1995). I'm different; not dumb. Modes of presentation (VARK) in the tertiary classroom. Proceedings of the 1995 Annual Conference of the Higher Education Research and Development Society of Australasia (HERDSA), 308–313. Gregor, S., & Jones, D. (2007). The anatomy of a design theory. Journal of the Associat...
-
[4]
https://doi.org/10.3390/educsci1402017 Kaner, C., Bach, J., & Pettichord, B. (2001). Lessons learned in software testing. John Wiley & Sons. Kirk, R.E. (1978). Introductory statistics. Brooks/Cole Publishing Company. Maharaj, S., Koon, M.K., Gookool, C., Taylor, M., Birbal, R., & Mohammed, N. (2023). Cambridge Primary Computing Learner's Book - Stage
-
[5]
Hodder Education. Medina, E.M. (2004). Beyond the ballot box: Computer science education and social responsibility. ACM SIGCSE Bulletin, 36(4), 7–10. https://doi.org/10.1145/1041624.1041626 Mitchell, T.M. (1997). Machine intelligence. McGraw-Hill. Mitchell, T.M. (2006). The discipline of machine learning . Carnegie Mellon University, School of Computer Sc...
-
[6]
Reprinted in Pearson, E.A. (Ed). (1948). Karl Pearson’s Early Statistical Papers, Cambridge University Press. Pett, M. A. (2015). Nonparametric statistics for health care research: Statistics for small samples and unusual distributions. Sage Publications. Piaget, J. (1950). The psychology of intelligence. London Routledge & Kegan Paul. Pimentel, M., Filip...
Reviewed May 24, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.