REVIEW
A Multi-level Analysis of Factors Associated with Student Performance: A Machine Learning Approach to the SAEB Microdata
Machine Learning · Submitted 25 Oct 2025 · 73 references
Pith's verdict: Solid data-engineering exercise and a confirmed known result, but the headline metrics and SHAP ranking rest on an invalid random split with school-level features, so the numbers aren't trustworthy as submitted.
Why Pith says this
Highlight any text to comment on it.
Referee report
The paper
The paper claims
A school's average socioeconomic level, not individual traits, is the strongest predictor of a student's SAEB exam performance, this machine-learning study concludes.
The reading
What carries the argument
The carrying mechanism is the multi-level table: four SAEB microdata files (student socioeconomic questionnaire, teacher profile, school indicators, principal questionnaire) joined by school and class IDs to make one hierarchical dataset. On top of that sits a Random Forest classifier, and the evidence for the systemic claim comes from SHAP (Shapley Additive Explanations), a model-agnostic method that attributes each prediction to its input features, which lets the authors rank the school's average socioeconomic level first.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that the school's ecosystem — captured chiefly by the school's average socioeconomic level, the share of teachers with adequate training, and student participation rate — predicts student proficiency more strongly than any student-level characteristic. The authors demonstrate this by training four tree-based ensembles on a feature set refined from 98 candidates to 44 via the Boruta algorithm and then to 17 with specialist input, finding Random Forest's 90.2% accuracy and 96.7% AUC dominate untuned gradient boosting, and then showing with SHAP that school-level variables sit at the top of the importance ranking.
Load-bearing premise
The load-bearing premise is that randomly splitting students into train and test while keeping school-level aggregates as features does not inflate accuracy; if the model is only ever tested on students from schools it already saw in training, school-level memorization could masquerade as a systemic effect.
Editorial extensions
If this is right
- If the systemic claim holds, funding formulas and teacher-allocation policies should weight school socioeconomic context heavily.
- The model's balanced precision and recall across above- and below-average students means it can serve as a diagnostic, flagging which students are likely to underperform and which school-level drivers push them there.
- The dominance of school-level indicators over individual teacher experience and degree implies that improving school composition and participation may yield larger gains than isolated professional-development programs.
- Applying the same SHAP pipeline to later SAEB editions would let policymakers trace whether investments in teacher training shift feature importance over time.
Reading between the lines
- Inference: because the data split randomly assigns students rather than schools, and school-level aggregates are features, part of the reported accuracy may come from the model recognizing individual schools; a leave-one-school-out validation is the test that would tell whether the systemic finding generalizes to unseen schools.
- Inference: the school socioeconomic level is itself an average of students' socioeconomic scores, so the paper's ranking partly measures the predictive power of aggregating individual background up to the school level, not necessarily a separate institutional effect.
- Inference: a natural extension the paper does not run — a multilevel or hierarchical model keeping school as a random effect — would quantify how much variance lies between schools versus within them, cleanly separating composition from context.
- Inference: if school-level predictions prove robust out-of-school, the same four-layer design could be ported to other large-scale assessments, such as state-level exams, to guide decentralized education policy.
Original abstract
Identifying the factors that influence student performance in basic education is a central challenge for formulating effective public policies in Brazil. This study introduces a multi-level machine learning approach to classify the proficiency of 9th-grade and high school students using microdata from the System of Assessment of Basic Education (SAEB). Our model uniquely integrates four data sources: student socioeconomic characteristics, teacher professional profiles, school indicators, and principal management profiles. A comparative analysis of four ensemble algorithms confirmed the superiority of a Random Forest model, which achieved 90.2% accuracy and an Area Under the Curve (AUC) of 96.7%. To move beyond prediction, we applied Explainable AI (XAI) using SHAP, which revealed that the school's average socioeconomic level is the most dominant predictor, demonstrating that systemic factors have a greater impact than individual characteristics in isolation. The primary conclusion is that academic performance is a systemic phenomenon deeply tied to the school's ecosystem. This study provides a data-driven, interpretable tool to inform policies aimed at promoting educational equity by addressing disparities between schools.
References
How the literature uses it
What this paper does with the arXiv papers it cites
Discussion
Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.