{"id":"e1436f75-3301-47c5-8652-058b5a43ed86","arxiv_id":"2509.25353","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using DEA and SHAP on PISA 2022, the study finds a cognitive efficiency gap of about 0.10 and a non-cognitive gap of 0.045 favoring private schools across nine Latin American countries, with public schools showing more heterogeneous efficiency.","lead":"Private schools in nine Latin American countries appear more efficient than public schools at converting resources into test scores and reported soft skills, using PISA 2022 data. The paper combines data envelopment analysis with interpretable machine learning to identify which school and student factors drive low and high efficiency.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The private-public efficiency gap rests on separate DEA frontiers whose comparability is never tested; a pooled-frontier re-estimation should settle whether the 0.10 gap is real or mechanical.","rationale":"The paper's headline is a comparison between two groups, but the identification comes from separate DEA frontiers estimated within each group. Sections 3.1 and 4.1 treat private and public schools as having different technologies and estimate VRS frontiers for each type. Section 5.1 then reads the difference in average bias-corrected scores as evidence that private schools are more efficient. This inference requires the two frontier estimates to be on a comparable scale. The paper provides no test of frontier equality; Section 4.1 simply states that different technologies are used and reports VRS-versus-CRS tests. The unequal sample sizes (about 1,548 public schools versus 486 private schools) make the comparability issue concrete: even with a common true technology, the public-sector frontier estimated from over three times as many DMUs will be estimated more precisely and can lie beyond the private frontier, mechanically lowering public schools' measured efficiency. So the gap in Tables 3 and 4 could be an artifact of how the frontier is estimated rather than of school-type productivity. This is exactly the sort of specification point that a pooled-frontier robustness check can settle. The paper's bootstrap and outlier checks are good internal robustness, but they do not address this between-group comparability concern. If the pooled-frontier re-estimation reproduces the gap, the central claim is supported; if not, the paper would need to be reinterpreted as documenting within-group dispersion, not private-school superiority. The SES-as-input choice compounds this: if SES is an environmental endowment rather than a managed input, the private frontier's high-input/high-output position is partly inherited, and the efficiency gap is not a managerial difference. Because the reader already identified this concern and made the verdict conditional, my recommendation is unchanged: the paper should be accepted only after this pooled-frontier test is run and reported.","tokens_in":24824,"tokens_out":6921,"duration_ms":74710,"concrete_test":"Re-estimate the cognitive and non-cognitive DEA models once on a single pooled frontier containing both school types (same inputs and outputs), and compute bias-corrected mean efficiency separately for public and private schools; additionally run a bootstrap test of equality of the separate private and public frontiers (e.g., Simar and Wilson). If the pooled-frontier gap is materially smaller than 0.10/0.045, or if the equality-of-frontiers test cannot reject a common technology, then Tables 3 and 4 are specification artifacts and the central claim fails. A useful secondary check is to repeat the pooled analysis with family SES treated as a non-discretionary variable to see how much of the gap is due to SES-as-input.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 estimates four separate VRS DEA frontiers—one per school type and outcome—and Section 5.1 then compares the average bias-corrected scores across school types. This comparison is only meaningful if private and public schools operate under the same, or at least commensurable, technology. That condition is assumed, never tested. With separate frontiers, the reported 'gap' measures each sector's average distance to its own estimated frontier; a larger public sample (1,584 or 1,548) and a smaller private sample (486) imply different finite-sample frontier estimates even under a common technology, so a mechanical gap can arise from noise and sample size alone. Moreover, if the frontiers actually differ, the scalar gap does not identify which technology is more productive; it only says public schools are more internally dispersed around their own best practice. The SES-as-input choice is part of the same specification risk: if SES is an environmental factor rather than a school-controlled input, the gap is even less interpretable as technical efficiency. This is the load-bearing assumption for the headline 0.10/0.045 result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Using PISA 2022 data on 2,034 schools in nine Latin American countries, the paper estimates school-level technical efficiency separately for private and public schools, with cognitive outputs (math, reading, science) and a non-cognitive output (a soft-skills composite), via output-oriented VRS Data Envelopment Analysis with bootstrap bias correction. It reports a mean bias-corrected efficiency gap of about 0.10 in cognitive efficiency (0.865 vs. 0.768) and 0.045 in non-cognitive efficiency (0.685 vs. 0.640), with lower dispersion among private schools, and interprets these as private-sector technical superiority. In a second stage, the paper dichotomizes efficiency at the regional mean and uses gradient-boosted trees with SHAP values to rank 42 school and student covariates associated with high efficiency, profiling the highest-efficiency private school and the lowest-efficiency public school. The paper claims to be the first post-pandemic regional LAC study of school-type efficiency gaps and to extend DEA-plus-interpretable-ML to cross-country education efficiency.","tokens_in":25062,"tokens_out":5055,"duration_ms":51455,"significance":"If the headline gap withstands specification scrutiny, the paper would supply timely regional evidence on private-public school efficiency after the pandemic, and its combination of DEA with SHAP-based second-stage explanations is a useful addition to the emerging DEA-ML literature. The analysis is careful in several respects that deserve credit: bias-corrected radial efficiency with bootstrap confidence intervals, a robustness check for outliers (Appendix C), stochastic dominance comparisons, and detailed tables reporting country-level efficiency scores. The central empirical claim, however, rests on two untested specification choices: separate production frontiers for private and public schools, and the treatment of average family SES as a school input. Because the estimated gap is defined as the difference between each sector's distance to its own frontier, it is not yet established that the gap reflects a real technological or managerial advantage rather than a mechanical consequence of sample size, frontier estimation noise, or the SES-as-input assumption.","major_comments":[{"comment":"The main private-public efficiency gap is computed from four separate VRS DEA frontiers, one per school type and outcome, without any test of whether private and public schools operate under a common technology. With separate frontiers, the reported gap of 0.10 (cognitive) and 0.045 (non-cognitive) measures each sector's average distance to its own estimated best practice; under a common technology, different sample sizes (1,584 public vs. 486 private) and finite-sample noise can alone produce a mechanical gap, while if the technologies genuinely differ, the scalar gap does not identify which technology is more productive. Please add a pooled-frontier re-estimation or a formal test of technology equality (for example, a bootstrap test of the equality of the two frontiers, or a meta-frontier analysis with a technology gap ratio) and report whether the gap persists in that specification.","section":"Section 4.1 and Section 5.1"},{"comment":"Average family SES is included as one of the four DEA inputs, and the summary statistics show a 1.12 difference in SES between private and public schools. If SES is an environmental or non-discretionary factor rather than a school-controlled resource, then treating it as a regular input can bias the efficiency scores in favor of high-SES private schools, because the frontier can 'explain' higher outputs by the more favorable student mix. The paper should report a robustness analysis that either omits SES from the input set or treats it as an external/non-discretionary variable; without this, the headline 0.10 cognitive gap remains dependent on a contestable modeling assumption.","section":"Section 3.1 and Table 1"},{"comment":"There is a numerical inconsistency in the reported sample sizes. Section 3 states that the working sample contains 2,034 schools, of which 1,548 are public and 486 are private, but Tables 3 and 4 report N = 1,584 public schools and the country-level public-school rows sum to 1,584, giving a total of 2,070. Please reconcile these numbers and clarify which sample is used for the DEA estimation and which is used for the second-stage analysis; if 36 additional schools enter the DEA sample, the estimates and standard errors would need to be re-checked.","section":"Section 3 and Tables 3-4"},{"comment":"The stochastic dominance tests are reported with p-values of 1.0000 and 0.9450 for the first-order tests, and the null hypothesis is written as 'bθ_private ⪯_s bθ_public'. As written, acceptance of this null does not transparently establish that private schools dominate public schools; the notation is ambiguous about whether '⪯' orders the CDFs (so private CDF below public CDF means private dominance) or orders the efficiency levels. Please restate the null in terms of the cumulative distribution functions and explain whether a failure to reject in these tests is being treated as affirmative evidence in favor of private dominance, since failure to reject a null is not normally evidence for it.","section":"Section 5.1.1 and Table 5"}],"minor_comments":[{"comment":"There are several typographical errors, including 'INLATIN AMERICA—ACOMBINEDDEAAND', '˘interpreable' in Section 4.2, and 'loosing around lost around 0.9-1.1 years' in the Introduction; these should be corrected throughout.","section":"Title, Abstract, Introduction"},{"comment":"The VRS-versus-CRS tests are said to be 'available from the author upon request'; for a central modeling choice, the tests and their p-values should be reported in the appendix or a table rather than left to request.","section":"Section 4.1 and Note 5"},{"comment":"The binary dependent variable is defined as efficiency above the regional mean, separately within each school type, so the second-stage results describe predictors of being above the sector-specific average, not predictors of reaching an absolute efficiency standard. This should be stated more explicitly in the text, and the SHAP-based comparisons between private and public schools should be interpreted accordingly.","section":"Section 4.2 and Figure 3"},{"comment":"The outlier robustness check is a welcome addition, but the country-level proportional differences in Table C1 are large in some cases (e.g., -9.53% for Dominican Republic non-cognitive private efficiency and +3.01% for Chile), so the statement that the estimates 'are not as different' should be softened or the country-level outlier sensitivity should be discussed directly.","section":"Appendix C and Table C1"},{"comment":"Table 3 notes state that all private-public differences are statistically significant at 10%, while the text and Figure 1c report that four non-cognitive country gaps are not statistically significant; the table notes should be aligned with the reported significance pattern.","section":"Tables 3 and 4 notes"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for an applied economics journal and the empirical design is mostly standard, but the headline gap is built on untested separate-frontier and SES-as-input assumptions. These are fixable with additional tests, so I would not reject, but the revision needs to include pooled-frontier or technology-equality evidence and a clear reconciliation of the sample-size discrepancy. If those tests show the gap disappears, the contribution should be reframed as a descriptive within-sector efficiency comparison rather than a private-public productivity advantage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know the headline result before reading: private schools look more efficient than public schools in LAC (cognitive gap ~0.10, non-cognitive ~0.045), but that gap is not a clean estimate of productivity differences. The paper estimates separate DEA frontiers for each school type, then compares average distances to each own frontier. That comparison is only meaningful if the technologies are the same, and the author never tests that. With 1,548 public schools and 486 private schools, finite-sample noise alone can produce a mechanical gap, because larger samples tend to push the frontier outward and drag down mean scores. The stochastic dominance tests suffer the same issue: they compare distributions of scores measured against different benchmarks.\n\nCredit where it's due: this is the first post-pandemic regional study for LAC using PISA 2022, and including non-cognitive efficiency is a genuine extension beyond prior work. The DEA implementation is careful—bootstrap bias correction, outlier robustness, clearly reported scores. The second-stage SHAP analysis is competently executed, though binarizing efficiency at the regional mean throws away information.\n\nSoft spots, in proportion: the separate-frontier assumption is load-bearing, not minor. The treatment of school-average family SES as a controllable input is also questionable; SES is an environmental factor, not a resource a principal can allocate, and including it as a standard input biases the efficiency interpretation. There's a notation/interpretation error in the stochastic dominance tests: the null is written as private ⪯ public, the p-values accept that null, yet the text concludes private schools are more efficient—one of those two things is wrong. And there is no replication code or data-cleaning script provided, which matters for a paper whose numbers will be cited by ministries.\n\nNone of this makes the paper worthless. It's a serious empirical contribution that asks the right question with relevant new data. But the central numeric claim needs a pooled-frontier or metafrontier re-estimation, plus a test of technology equality, before I'd trust the 0.10 gap. The SES treatment needs to be defended or changed to a non-discretionary input.\n\nWho is this for? Education policy economists and LAC ministries. It deserves a serious referee, but the referee should demand a considerably revised version. I would not cite the headline gap as it stands.","headline":"The first post-pandemic regional private-public school efficiency benchmark for LAC, but the headline 0.10 gap is not identified as stated because the DEA frontiers are estimated separately by school type and never compared on a common technology.","tokens_in":25573,"tokens_out":2979,"would_cite":false,"duration_ms":31230,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Private schools in Latin America turn resources into learning more efficiently than public schools, the paper estimates.","keywords":["school efficiency","private and public schools","data envelopment analysis","interpretable machine learning","SHAP","Latin America","PISA 2022"],"falsifier":"Re-estimate the DEA with all schools pooled into one frontier and move average family SES from the input set to a non-discretionary environment variable. If the bias-corrected cognitive efficiency gap between private and public schools drops below roughly 0.05 or reverses, the reported gap is an artifact of the modeling choice; if it stays near 0.10, the paper's central claim is supported.","tokens_in":24634,"feed_emoji":"🎓","tokens_out":10750,"duration_ms":84204,"temperature":0.7,"pith_summary":"The paper sets out to measure whether private secondary schools in Latin America use their resources more efficiently than public schools, and to identify what pushes a school toward the top of the efficiency ladder. Using PISA 2022 data on 2,034 schools in nine countries, it estimates output-oriented Data Envelopment Analysis frontiers separately for each school type, for both cognitive outcomes (math, reading, science) and a non-cognitive well-being index. The headline result is a bias-corrected cognitive efficiency score of 0.865 for private schools versus 0.768 for public schools—a gap of nearly 0.10—and a non-cognitive gap of 0.045 (0.685 versus 0.640), with tighter dispersion among private schools. The second stage uses gradient-boosted trees and SHAP values to rank 42 possible drivers of high efficiency. A sympathetic reader would care because, if the gap is real, public schools could raise cognitive output by roughly a quarter without additional inputs, and the paper points to concrete levers such as school climate, repetition, and pandemic-era homework barriers.","feed_headline":"Efficiency gap favors private schools in Latin America by 10 points","feed_subtitle":"PISA 2022 data on 2,034 schools show the private sector wins on both test scores and soft skills.","key_machinery":"The carrying machinery is output-oriented Data Envelopment Analysis (DEA) under variable returns to scale—a nonparametric linear-programming method that builds a best-practice frontier from the observed schools and scores each school by how far it falls short of the maximum output attainable from its inputs—run separately for private and public schools and for cognitive and non-cognitive outputs, with 2,000-repetition bootstrap bias correction. Efficiency is then converted into a binary indicator (above or below the regional mean) and modelled with gradient-boosted trees, an ensemble of decision trees that captures nonlinearities and interactions, with SHAP values used to rank and sign the contribution of 42 student-, school-, and COVID-related covariates. The DEA scores carry the efficiency claim; the SHAP analysis carries the attribution of which factors separate high- from low-efficiency schools.","core_discovery":"On the paper's own terms, the central discovery is a private-school efficiency advantage that holds across the whole distribution, not just at the mean. Bias-corrected technical efficiency for cognitive outputs is 0.865 for private schools and 0.768 for public schools; for the non-cognitive output it is 0.685 versus 0.640. Stochastic dominance tests accept first- and second-order dominance of private over public efficiency profiles, meaning an education planner who values higher efficiency and lower between-school dispersion would prefer the private-school profile. Within the region the cognitive gap is positive in every country, ranging from about 0.08 to 0.11, while non-cognitive gaps are positive in all nine countries though not statistically significant in four. The interpretable-machine-learning stage shows that high-efficiency private schools are characterized by more books and computers at home, little student paid work, and high school autonomy, while low-efficiency public schools are marked by poor climate, high repetition and truancy, intense paid work, few books, and COVID-era homework barriers. The paper also reports a positive correlation between cognitive and non-cognitive efficiency that is stronger for public schools.","pith_inferences":["The paper's separate-frontier design leaves open whether a single pooled technology would shrink the 0.10 cognitive gap; testing that specification directly would show how much of the gap is genuine production efficiency rather than sector-specific technology.","Because the PISA soft-skills score is a school average over self-reported personality domains, the non-cognitive gap may reflect reporting or composition; disaggregating the index by domain would test whether the private advantage is uniform across traits like empathy, perseverance, and emotional control.","The cross-sectional design cannot establish that autonomy, books, or homework support cause efficiency; a pilot that transfers the high-efficiency private-school profile to public schools and tracks cognitive and non-cognitive efficiency would be the natural test of the paper's policy story.","Future waves of PISA or a value-added panel of schools could separate the persistent efficiency gap from selection into private schooling, since the current data contain only one cross-section."],"forward_implications":["Public schools' average cognitive efficiency of 0.768 implies they could increase learning outcomes by about 23 percent while holding inputs constant, if the frontier estimate is correct.","Because private schools also dominate in non-cognitive efficiency by 0.045, the private advantage is not confined to test scores; it extends to the well-being/soft-skills output measured in PISA.","Lower dispersion in private-school efficiency (IQR 0.083 versus 0.117 for cognitive outcomes) means the private advantage is not driven by a few outliers; the whole distribution is shifted and compressed.","The SHAP ranking suggests different policy levers for the two sectors: in private schools the high-efficiency profile is tied to home educational resources and autonomy, while in public schools the low-efficiency profile is tied to repetition, truancy, paid work, and poor climate.","The positive correlation between cognitive and non-cognitive efficiency, stronger for public schools, implies that policies improving soft skills could also help close the cognitive efficiency gap."],"supporting_citations":[{"why":"Provides the PISA 2022 data from which the school-level inputs, outputs, and COVID-era covariates are built.","marker":"OECD, 2023a,b"},{"why":"Defines the separate-frontier DEA comparison by school type for LAC using PISA-D pre-pandemic data and finds a 6% private-school gap that this paper extends regionally.","marker":"Delprato and Antequera, 2021a"},{"why":"Compares LAC school efficiency before and after COVID with PISA 2018 and 2022, supplying the pre/post framing and the private-school effect the paper revisits.","marker":"Delprato and Antequera, 2025"},{"why":"Guides the selection of inputs and outputs in the education production function used in the DEA specification.","marker":"De Witte and López-Torres, 2017"},{"why":"Provides cross-country PISA 2012 evidence that private schools are more efficient in developing countries, the comparative baseline the paper checks for post-pandemic LAC.","marker":"Agasisti and Zoido, 2019"},{"why":"Supplies the teradial routines used to compute bias-corrected DEA efficiency scores and their bootstrap confidence intervals.","marker":"Badunenko and Mozharovskyi, 2016"},{"why":"Introduces the additive SHAP attribution method used to rank the determinants of high efficiency.","marker":"Lundberg and Lee, 2017"},{"why":"Gives the gradient-boosted-trees algorithm fitted in the second-stage machine-learning analysis of efficiency determinants.","marker":"Chen and Guestrin, 2016"}],"fun_headline_variants":["Private schools 10 points more efficient in Latin America","Private school efficiency lead: 10 points in Latin America","Study: private schools top public on efficiency in all Latin America","Hybrid DEA-ML finds private school efficiency edge in Latin America","Latin America: private schools more efficient by 10 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a school's average family wealth is an input the school can control and that private and public schools each have their own production technology; if either premise fails, the estimated efficiency gap is a modeling artifact.","fun_headline_variants_meta":{"raw":{"variants":["Private schools 10 points more efficient in Latin America","Private school efficiency lead: 10 points in Latin America","Study: private schools top public on efficiency in all Latin America","Hybrid DEA-ML finds private school efficiency edge in Latin America","Latin America: private schools more efficient by 10 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001751,"raw_usage":{"total_tokens":6983,"prompt_tokens":1079,"completion_tokens":5904,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":5820}},"tokens_in":695,"tokens_out":5904,"duration_ms":37625,"temperature":1.0,"reasoning_tokens":5820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:42:45.621929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-estimate the DEA with all schools pooled into one frontier and move average family SES from the input set to a non-discretionary environment variable. If the bias-corrected cognitive efficiency gap between private and public schools drops below roughly 0.05 or reverses, the reported gap is an artifact of the modeling choice; if it stays near 0.10, the paper's central claim is supported.","supporting_citations":[{"cited_title":", author L \\'o pez-Torres, L","cited_arxiv_id":null,"evidence_quote":"Guides the selection of inputs and outputs in the education production function used in the DEA specification."},{"cited_title":", author Zoido, P","cited_arxiv_id":null,"evidence_quote":"Provides cross-country PISA 2012 evidence that private schools are more efficient in developing countries, the comparative baseline the paper checks for post-pandemic LAC."},{"cited_title":", author Mozharovskyi, P","cited_arxiv_id":null,"evidence_quote":"Supplies the teradial routines used to compute bias-corrected DEA efficiency scores and their bootstrap confidence intervals."}],"review_version":1}