REVIEW 4 major objections 5 minor 29 references
Analysis of Public Schools Educational Performance Based on Causal Models and Hierarchical Clustering
T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read This paper argues that clustering schools on causally relevant features, rather than all available census variables, produces typologies that separate educational performance more sharply and are geometrically more coherent.
desk verdict A plausible application with an unsupported causal claim: the clustering improvement may just be a smaller, outcome-correlated feature set, and the causal graph was chosen post hoc. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Markov blanket of the target variable: within a fitted causal directed acyclic graph, the set of a variable's parents, children, and co-parents, which is the minimal set that screens the target off from all other variables. The paper uses the Markov blanket as a causally grounded feature selector: after fitting a linear non-Gaussian acyclic causal model (DirectLiNGAM), it retains only the blanket variables for clustering. The clustering itself is hierarchical agglomerative clustering with Euclidean distance and minimum-variance linkage, producing eight clusters from the selected feature space.
What would settle it
Re-run the clustering pipeline with random subsets of the same size as the causal feature set, many times. If random subsets frequently match or beat the causal scenario's silhouette (about 0.35) and ANOVA effect size (eta-squared about 0.13), then the causal interpretation is not needed to explain the improvement.
Extended reading notes
Core claim
The paper's central claim is that clustering schools on the Markov blanket of the Saeb score, rather than on all 33 census variables, produces school typologies that are more compact, better separated, and more strongly associated with measured performance. Using a causal graph estimated by DirectLiNGAM and extracting the Markov blanket of the performance target, the authors built an eight-cluster hierarchical solution and compared it with a baseline using all variables. The causal solution wins on every quantitative criterion the paper reports: larger ANOVA F and non-parametric test statistics, larger effect sizes, homogeneity of variances, better silhouette score, and lower Davies-Bouldin
Load-bearing premise
The claim stands or falls on whether the causal graph the authors chose really represents how school characteristics cause performance; if that graph is wrong, the causal features are just another arbitrary subset and the observed advantages could be an artifact of using fewer variables.
Editorial extensions
If this is right
- If the causal scenario is correct, schools grouped by causal features are more homogeneous in performance and more distinct across groups, making cluster membership a stronger signal for diagnosing school effectiveness.
- The causal clusters satisfy the homogeneity-of-variances assumption that the baseline violates, so comparisons of mean performance across clusters are statistically more trustworthy.
- Moderate overlap between the two partitions (ARI 0.338, AMI 0.470) implies that causal selection reorganizes schools rather than reproducing existing typologies, so policy conclusions would differ between the two schemes.
- Because the causal feature space yields better geometric quality, the typologies are better suited as targets for interventions tailored to distinct school profiles.
- The authors' stated next step, replicating the analysis nationwide, would test whether the causal blanket found here generalizes beyond their dataset.
Reading between the lines
- Inference: the paper does not run an ablation against random subsets of the same size; if random subsets reproduced the silhouette and effect-size gains, the advantage would be attributable to dimension reduction rather than to causal validity.
- Inference: the authors selected the 'most coherent' DAG among four causal-discovery algorithms after inspecting the results, so the causal feature set is partly a matter of judgment; checking Markov blankets from the other algorithms would show how much the typology improvement depends on that choice.
- Inference: the weighted Saeb score blends elementary and high school stages, and if causal mechanisms differ by stage, stage-specific blankets might yield even sharper typologies and different policy levers.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares two hierarchical clustering pipelines for Brazilian public schools: a baseline using all 33 census-derived variables and a 'causal' scenario using only the Markov blanket of a weighted Saeb performance score, where the blanket is extracted from a DirectLiNGAM causal DAG. The authors report that the causal scenario yields higher silhouette score (0.1211 to 0.3496), lower Davies–Bouldin index (2.0120 to 1.2040), larger ANOVA F (10.29 to 16.97), larger eta-squared (0.085 to 0.133), and larger Kruskal–Wallis statistic (67.48 to 109.76), concluding that causal feature selection improves the discriminative capacity, performance association, and internal consistency of school typologies. The manuscript also includes structural similarity metrics (ARI, AMI, Cramér's V) between the two partitions.
Significance. If the central claim were established, the paper would contribute a practical template for integrating causal discovery with clustering in educational data, with potential policy relevance for identifying school typologies that align with performance. Strengths include the use of multiple complementary validation metrics, clearly reported effect sizes and test statistics, and a transparent comparison framework. However, the current design does not support the causal interpretation because the improvement could arise from reducing the feature set size or from selecting features by their association with the outcome, rather than from causal structure per se. The paper would be strengthened by including a same-size non-causal baseline, external validation, and robustness checks across causal discovery algorithms. These are feasible fixes within the manuscript's scope, so the result is defensible only after substantial revision.
major comments (4)
- [§4.3, Tables 3 and 4] The central comparison is confounded by feature-set size. The baseline uses all 33 variables, while the causal scenario uses only the Markov blanket of the Saeb score, which is likely a smaller subset. In hierarchical clustering with Euclidean distance and Ward linkage, adding many weakly relevant or noisy coordinates can flatten distance ratios and degrade geometric indices; restricting to a smaller feature set can improve silhouette and Davies–Bouldin even when the features are chosen by simple correlation with the target. To support the claim that causality—rather than dimensionality—drives the improvement, the authors must report a control clustering using the same number of features selected at random or by a purely associational criterion (e.g., top-k by absolute correlation with Saeb). Without this control, the improvements in Tables 3 and 4 are not attributable to causal structur
- [§4.2, §3.2, Eq. (1)] The Markov blanket is described as 'the theoretically optimal feature set for predicting T' and is extracted using the same Saeb outcome variable used to evaluate cluster-performance association. This creates a selection loop: features are chosen for their statistical relation to the outcome, and then cluster-outcome association is measured on the same data. The better ANOVA/Kruskal–Wallis and effect sizes in the causal scenario are expected from predictive feature selection, not from causal structure. An external or held-out evaluation—e.g., estimating the Markov blanket on one subsample and evaluating clustering on another—would mitigate this circularity. The current design cannot distinguish 'causal' from 'outcome-correlated, smaller feature set.'
- [§5.1, §4.2] The causal DAG is selected because it 'exhibited the most coherent structure' among DAGMA, FCI, GES, and DirectLiNGAM, but no formal criterion or sensitivity analysis is provided. This after-the-fact selection is vulnerable to confirmation bias and does not quantify the uncertainty in the recovered graph. The authors should report the Markov blankets obtained from the other algorithms, the stability of the subsequent clustering results, and the reason DirectLiNGAM's output is preferred (e.g., agreement with established educational findings, stability across subsamples). Without this, the 'causal' scenario is effectively one arbitrary feature subset among many possible ones.
- [§3.1, §5.1] DirectLiNGAM relies on linearity, non-Gaussian disturbances, acyclicity, and causal sufficiency (no latent confounders). The paper does not test or discuss these assumptions for the educational dataset, where latent constructs (e.g., school management quality, community engagement) are plausible. If any assumption fails, the estimated DAG—and hence the Markov blanket—may not be causally valid, making the 'causal' scenario an associational feature subset. At minimum, the authors should acknowledge this threat and provide diagnostic checks (e.g., testing pairwise linearity, comparing LiNGAM with a non-linear method) or a robustness analysis.
minor comments (5)
- [§4.2 title] The section title 'Casual Features' appears to be a typo for 'Causal Features'.
- [Table 1] The table uses Portuguese dimension names (e.g., 'Infraestrutura Básica', 'Recursos e Espaços'). English translations or a bilingual table would improve accessibility.
- [References] References [25] and [26] appear to duplicate the same paper (Weinstein and Blei, 'Hierarchical Causal Models'). Please consolidate or clarify.
- [§5.5] The sentence 'the inclusion of causal features does not merely generate an alternative partition' is imprecise; the comparison is between feature sets, not 'inclusion' of features. Rewording would avoid confusion.
- [Data availability] The School Census is described as not public and used under institutional access, but no data availability statement or replication details (e.g., software versions, hyperparameters for DirectLiNGAM) are provided. Adding these would strengthen reproducibility.
Circularity Check
The causal scenario's features are the Markov blanket of the target outcome; evaluating cluster-target association on the same outcome makes the claimed performance improvement a selection-on-the-outcome artifact rather than evidence for causal structure.
-
fitted input called prediction
[Section 3.2, Section 4.2, Section 5.3 (Tables 3–4)]
"The Markov blanket of a variable provides a principled answer to the question: which variables are sufficient—and necessary—to predict a given target? ... After estimating the candidate DAGs, we extracted the Markov blanket of the target variable, which provides the minimal set of variables that carry causal information about the outcome. This subset was then used as the causal feature set for the clustering analysis."
The causal features are precisely the Markov blanket of the weighted Saeb score, which the paper defines as the optimal set for predicting that same target. The causal scenario is then evaluated by measuring how well cluster membership explains the weighted Saeb score (ANOVA η², Kruskal–Wallis) and by geometric indices. Since the features were selected using the outcome variable, the improved outcome association is a direct artifact of outcome-based feature selection, not evidence that the DirectLiNGAM graph is correct. The baseline uses all 33 variables, so the comparison is between an outcome-selected low-dimensional set and a noisy high-dimensional set; a same-size correlational selection would likely produce the same pattern. The conclusion 'clustering based on causal features improves
full rationale
The paper's derivation chain is not circular in the sense of reusing its own fitted equations as predictions. The clustering itself is computed from the selected features rather than directly from the outcome, and the geometric-quality claims (silhouette, Davies–Bouldin) are not tautological. However, the central comparative claim is undermined by an outcome-selection loop: the causal feature set is the Markov blanket of the weighted Saeb score, which by the paper's own definition is the theoretically optimal set for predicting that same outcome. The causal scenario is then evaluated by how well cluster membership explains that same outcome (η², Kruskal–Wallis) and by geometric indices—but the baseline uses all 33 variables, so the comparison cannot separate 'causal' from 'fewer, outcome-selected features.' The improved performance association is thus a built-in consequence of feature selection on the dependent variable, not evidence for causal structure. No self-citation chain appears load-bearing. The paper itself acknowledges subjectivity in causal feature identification, which further weakens the causal interpretation but is not itself circularity. Score 6 reflects partial circularity: one 'prediction' (better outcome discrimination) reduces by construction to the way the feature set was chosen.
Assumptions & free parameters
free parameters (3)
- Number of clusters k =
8
- Overcrowding threshold =
0.75 quantile per stage
- Choice of causal discovery algorithm =
DirectLiNGAM chosen among DAGMA, FCI, GES, DirectLiNGAM
assumptions (4)
- domain assumption DirectLiNGAM's structural equation model is correctly specified: linear relations, non-Gaussian independent disturbances, acyclic graph, and no latent confounders (causal sufficiency).
- domain assumption The selected DAG (Figure 1) is the true causal structure among the 33 school variables.
- domain assumption Euclidean distance with Ward linkage is appropriate for the mixed binary and min-max-scaled continuous features.
- standard math Markov blanket characterization theorem: in a faithful DAG, the Markov blanket consists of parents, children, and co-parents.
Cite this review
Pith. "Pith review of Analysis of Public Schools Educational Performance Based on Causal Models and Hierarchical Clustering." pith.science (2026). https://pith.science/paper/CR7IHIPU
@misc{pith2026260714124,
author = {Pith},
title = {Pith review of: Analysis of Public Schools Educational Performance Based on Causal Models and Hierarchical Clustering},
year = {2026},
howpublished = {\url{https://pith.science/paper/CR7IHIPU}},
note = {Machine review of arXiv:2607.14124}
}
read the original abstract
The increasing availability of large-scale educational datasets has expanded the use of quantitative methods for investigating school performance. However, institutional heterogeneity among schools and the structural complexity of educational data pose substantial challenges to traditional statistical modeling approaches. This study investigates the existence of school typologies based on structural, pedagogical, and demographic characteristics, and examines how these typologies relate to performance in the Brazilian Basic Education Assessment System (Saeb). Using data from the Brazilian School Census and Saeb, data preprocessing and normalization procedures are applied followed by hierarchical clustering to identify groups of schools with similar structural profiles. After the identification of these typologies, causal analysis techniques are employed to investigate potential causal relationships between school characteristics and educational outcomes. The results reveal the presence of distinct school profiles and statistically significant differences in average performance among them. The causal analysis provides insights into the structural and contextual factors that may influence educational performance, contributing to a better understanding of the mechanisms associated with school effectiveness.
Figures
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the 36th Interna- tional Conference on Neural Information Processing Systems
Bello, K., Aragam, B., Ravikumar, P.: Dagma: learning dags via m-matrices and a log-determinant acyclicity characterization. In: Proceedings of the 36th Interna- tional Conference on Neural Information Processing Systems. NIPS ’22, Curran Associates Inc., Red Hook, NY, USA (2022) Causal Models and Hierarchical Clustering 15
2022
-
[2]
Chickering, D.M.: Optimal structure identification with greedy search. J. Mach. Learn. Res.3(null), 507–554 (Mar 2003). https://doi.org/10.1162/153244303321897717, https://doi.org/10.1162/153244303321897717
-
[3]
Colombo, D., Maathuis, M.H.: Order-independent constraint-based causal struc- ture learning. J. Mach. Learn. Res.15(1), 3741–3782 (Jan 2014)
2014
-
[4]
Education Sciences12(1) (2022)
García-Jiménez, J., Torres-Gordillo, J.J., Rodríguez-Santero, J.: Factors as- sociated with school effectiveness: Detection of high- and low-efficiency schools through hierarchical linear models. Education Sciences12(1) (2022). https://doi.org/10.3390/educsci12010059, https://www.mdpi.com/2227- 7102/12/1/59
-
[5]
Frontiers in GeneticsV ol- ume 10 - 2019(2019)
Glymour, C., Zhang, K., Spirtes, P.: Review of causal discov- ery methods based on graphical models. Frontiers in GeneticsV ol- ume 10 - 2019(2019). https://doi.org/10.3389/fgene.2019.00524, https://www.frontiersin.org/journals/genetics/articles/10.3389/fgene.2019.00524
arXiv 2019
-
[6]
The Economic Journal113(485), F64–F98 (2003)
Hanushek, E.A.: The failure of input-based schooling policies. The Economic Journal113(485), F64–F98 (2003). https://doi.org/https://doi.org/10.1111/1468- 0297.00099, https://onlinelibrary.wiley.com/doi/abs/10.1111/1468-0297.00099
arXiv 2003
-
[7]
In: Proceedings of the 38th International Conference on Neural Information Processing Systems
Kim, K., Kim, J., Wasserman, L.A., Kennedy, E.H.: Hierarchical and density-based causal clustering. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. NIPS ’24, Curran Associates Inc., Red Hook, NY, USA (2024)
2024
-
[8]
In: Proceedings of the Thirteenth International Conference on International Conference on Machine Learning
Koller, D., Sahami, M.: Toward optimal feature selection. In: Proceedings of the Thirteenth International Conference on International Conference on Machine Learning. p. 284–292. ICML’96, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1996)
1996
Show all 29 references
-
[9]
In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence
Meek, C.: Causal inference and causal explanation with background knowledge. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence. p. 403–410. UAI’95, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1995)
1995
-
[10]
Applied Sciences12(19), 9467 (2022)
Nafuri, M., Fikri, A., Sani, N.S., Zainudin, N.F.A., Rahman, A.H.A., Aliff, M.: Clustering analysis for classifying student academic performance in higher educa- tion. Applied Sciences12(19), 9467 (2022). https://doi.org/10.3390/app12199467, https://www.mdpi.com/2076-3417/12/19/9467
2022 doi
-
[11]
NIPS ’20, Curran Associates Inc., Red Hook, NY, USA (2020)
Ng, I., Ghassami, A., Zhang, K.: On the role of sparsity and dag constraints for learninglineardags.In:Proceedingsofthe34thInternationalConferenceonNeural Information Processing Systems. NIPS ’20, Curran Associates Inc., Red Hook, NY, USA (2020)
2020
-
[12]
Intelligent Data Analysis20, 933–951 (2016)
Oviedo, B., Moral, S., Puris, A.: A hierarchical clustering method: Appli- cations to educational data. Intelligent Data Analysis20, 933–951 (2016). https://doi.org/10.3233/IDA-160839
2016 doi
-
[13]
Morgan Kaufmann (1988)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann (1988)
1988
-
[14]
Cambridge University Press, 2nd edn
Pearl, J.: Causality: Models, Reasoning, and Inference. Cambridge University Press, 2nd edn. (2009)
2009
-
[15]
Pellet, J.P., Elisseeff, A.: Using markov blankets for causal structure learning. J. Mach. Learn. Res.9, 1295–1342 (Jun 2008)
2008
-
[16]
Pertanika Journal of Social Sciences and Humanities28(4), 3003–3024 (2020)
Prasertcharoensuk, T., Chaiwan, J., Tang, K.N., Makmee, P.: A causal relationship model of primary public school students’ achievement: A multiple group analysis. Pertanika Journal of Social Sciences and Humanities28(4), 3003–3024 (2020). https://doi.org/10.47836/pjssh.28.4.29...
2020 doi
-
[17]
Al-Ishlah: Jurnal Pendidikan15(2), 2481–2497 (2023)
Rajagukguk, S.A., Fudholi, D.H., Pratama, A.R.: A data-driven causal modelling analysis of socio-economic factors and its impact on student’s performance: A case study of a junior high school in bali. Al-Ishlah: Jurnal Pendidikan15(2), 2481–2497 (2023). https://doi.org/10.3544...
2023 doi
-
[18]
So- ciology of Education59(1), 1–17 (1986), http://www.jstor.org/stable/2112482
Raudenbush, S., Bryk, A.S.: A hierarchical model for studying school effects. So- ciology of Education59(1), 1–17 (1986), http://www.jstor.org/stable/2112482
1986
-
[19]
Shimizu, S., Hoyer, P.O., Hyvärinen, A., Kerminen, A.: A linear non-gaussian acyclic model for causal discovery. J. Mach. Learn. Res.7, 2003–2030 (Dec 2006)
2003
-
[20]
Shimizu, S., Inazumi, T., Sogawa, Y., Hyvärinen, A., Kawahara, Y., Washio, T., Hoyer, P.O., Bollen, K.: Directlingam: A direct method for learning a linear non- gaussian structural equation model. J. Mach. Learn. Res.12(null), 1225–1248 (Jul 2011)
2011
-
[21]
MIT Press, 2nd edn
Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. MIT Press, 2nd edn. (2000)
2000
-
[22]
In: Proceedings of the Eleventh Conference on Uncer- tainty in Artificial Intelligence
Spirtes, P., Meek, C., Richardson, T.: Causal inference in the presence of latent variables and selection bias. In: Proceedings of the Eleventh Conference on Uncer- tainty in Artificial Intelligence. p. 499–506. UAI’95, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (1995)
1995
-
[23]
In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (AISTATS)
Tsamardinos, I., Aliferis, C.F.: Towards principled feature selection: Relevancy, fil- ters and wrappers. In: Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics (AISTATS). pp. 300–307 (12 2003)
2003
-
[24]
ACM Comput
Vowels, M.J., Camgoz, N.C., Bowden, R.: D’ya like dags? a survey on struc- ture learning and causal discovery. ACM Comput. Surv.55(4) (Nov 2022). https://doi.org/10.1145/3527154, https://doi.org/10.1145/3527154
2022 doi
-
[25]
Journal of Machine Learn- ing Research27, 1–73 (2026)
Weinstein, E.N., Blei, D.M.: Hierarchical causal models. Journal of Machine Learn- ing Research27, 1–73 (2026)
2026
-
[26]
Journal of Machine Learn- ing Research27(37), 1–73 (2026), http://jmlr.org/papers/v27/25-0899.html
Weinstein, E.N., Blei, D.M.: Hierarchical causal models. Journal of Machine Learn- ing Research27(37), 1–73 (2026), http://jmlr.org/papers/v27/25-0899.html
2026
-
[27]
ACM Comput
Yu, K., Guo, X., Liu, L., Li, J., Wang, H., Ling, Z., Wu, X.: Causality-based feature selection: Methods and evaluations. ACM Comput. Surv.53(5) (Sep 2020). https://doi.org/10.1145/3409382, https://doi.org/10.1145/3409382
2020 doi
-
[28]
Artificial Intelligence172(16), 1873–1896 (2008)
Zhang, J.: On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias. Artificial Intelligence172(16), 1873–1896 (2008). https://doi.org/https://doi.org/10.1016/j.artint.2008.08.001, https://www.sciencedirect.com/scie...
2008 doi
-
[29]
In: Proceedings of the 32nd International Con- ference on Neural Information Processing Systems
Zheng, X., Aragam, B., Ravikumar, P., Xing, E.P.: Dags with no tears: continuous optimization for structure learning. In: Proceedings of the 32nd International Con- ference on Neural Information Processing Systems. p. 9492–9503. NIPS’18, Curran Associates Inc., Red Hook, NY, U...
2018
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.