Pith. sign in

REVIEW 5 major objections 6 minor 25 references

Bias Analysis of AI Models for Undergraduate Student Admissions

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Models trained on six years of admissions data create persistent bias against non-white and first-generation students, incorrectly rejecting them more often despite high accuracy and passing fairness metrics.

desk verdict A transparent, credible case study that undermines itself with an arbitrary bias threshold, missing uncertainty quantification, and a direct contradiction between the single-split and aggregate gender results. read the letter →

arxiv 2412.02528 v1 pith:3EGBLLXN submitted 2024-12-03 cs.AI

classification cs.AI
keywords machinelearningbiasdetectionfairnessmetricspredictivemodelingtest-optionaladmissionscollegealgorithmichighereducation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that machine-learning models trained on real undergraduate admissions data carry a hidden form of bias: they misclassify students in different sensitive groups at different rates, even when overall accuracy is high and standard fairness metrics look fine. Using six years of admissions records from a large urban university, the authors built linear support-vector-machine models predicting direct admission to the School of Science, then compared accuracy, specificity, and sensitivity across gender, race/ethnicity, and first-generation status. They report that all three models incorrectly admit white students more often than non-white students, and that models for the test-required cohort and the simulated larger test-optional dataset incorrectly reject non-white and first-generation students more often than their counterparts. These gaps exceed a 5 percent threshold and persist across up to 500 random train/test splits, while overall accuracy stays near 80 to 89 percent. The matter is worth caring about because test-optional admissions policies and AI-assisted admission decisions are spreading, and bias that is invisible to common fairness metrics could still harm students if such models are deployed.

What carries the argument

The argument is carried by a linear support vector machine that classifies each applicant as a direct admit or not, using features such as GPA, standardized test scores, gender, race/ethnicity, first-generation status, residency, and campus-application count. Bias is declared when the difference in accuracy, specificity, or sensitivity between subgroups of a sensitive variable exceeds 5 percent, a threshold the authors state is chosen somewhat subjectively. To test persistence, each model is retrained over 10, 100, and 500 random two-thirds/one-thirds splits with proportionate stratified sampling on the sensitive variable, and mean subgroup metrics are compared. Three fairness metrics, Brier Score, Balance for the Negative Class, and Balance for the Positive Class, are computed alongside, and the paper uses the known incompatibility of fairness criteria to explain why these metrics mostly pass while specificity and sensitivity gaps fail.

What would settle it

Re-analyze the same data with statistical significance testing on the subgroup specificity and sensitivity differences across the 500 repeated splits, for example by computing confidence intervals or p-values for each gap. If the intervals for the reported race, gender, and first-generation gaps include zero, or the effect sizes are trivially small, then the claimed biases would not be established; conversely, tight intervals far from zero would confirm them.

Watch

Extended reading notes

Core claim

The central discovery is that the bias in the models is not a fluke of one train/test split and is not visible from overall accuracy or from the Brier Score and balance metrics. For race, every cohort model predicts white students will be incorrectly admitted more than non-white students, a specificity gap above 5 percent, and the test-required and larger test-optional models also predict non-white students will be incorrectly rejected more than white students, a sensitivity gap above 5 percent. For first-generation status, the same two models predict first-generation students will be incorrectly rejected more often and non-first-generation students incorrectly admitted more often than their counterparts. For gender, the larger test-optional model predicts women will be incorrectly admitted more than men. The paper treats these uneven error rates as evidence that the predictive models create new bias relative to sensitive variables, and it argues that fairness metrics can be simultaneously satisfied while such group-level error bias exists because the models are highly accurate.

Load-bearing premise

The entire bias verdict rests on a 5 percent difference in error rates between subgroups, a cutoff the authors admit is chosen subjectively, and no statistical significance test is applied to determine whether the observed gaps could arise by chance.

Editorial extensions

If this is right

  • If the central claim is right, a model can look fair by standard aggregate metrics yet still systematically mis-admit or mis-reject protected groups, so admissions AI should be audited for subgroup specificity and sensitivity, not just overall accuracy.
  • The finding that standardized test scores dominate the test-required model implies that making tests optional can change admission outcomes for a large number of students, including more women, non-white students, and first-generation students.
  • The persistence of the gaps over hundreds of random splits means the bias is not an artifact of a particular training set, so mitigation efforts must address the data or the model rather than the split.
  • The simultaneous satisfaction of all three fairness metrics in most conditions supports the theoretical point that one fairness definition can conceal another, and suggests fairness reports should include error-rate breakdowns.
  • The larger simulated test-optional dataset, while useful for evaluating models, should not be read as a prediction of what a true test-optional applicant pool would look like; the authors explicitly caution against that inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper uses historical admission decisions as ground truth, the model's uneven error rates may partly reflect bias in the original human and policy decisions, not just algorithmic bias; the method cannot cleanly separate the two, so the word 'new bias' is stronger than what the design can prove.
  • A statistically grounded threshold, such as a 95 percent confidence interval that excludes zero, might change which of the reported gaps count as bias; the 5 percent rule is exploratory, and the headline list of biases should be read with that caveat.
  • The same subgroup-specific specificity and sensitivity audit could be applied to other high-stakes classifiers where accuracy is high and common fairness metrics look acceptable, such as hiring screens, loan approval, or clinical triage tools.
  • A testable extension would be to retrain the models with reweighted or resampled training data designed to equalize specificity and sensitivity across subgroups and then measure how much overall accuracy must be sacrificed to remove the reported gaps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper analyzes bias in linear support vector machine models trained on six years of undergraduate admissions data from a School of Science at a large urban research university. The authors compare three cohorts (test-required, test-optional, and a larger simulated test-optional dataset) and evaluate model predictions for differences in accuracy, specificity, and sensitivity across gender, race/ethnicity, and first-generation status, using a 5% difference threshold to declare bias. They also report Brier scores and balance-based fairness metrics, and repeat random train/test splits to assess the persistence of observed biases. The main claims are that the models show specificity bias against white students in all three cohorts, sensitivity bias against non-white and first-generation students in the test-required and larger test-optional cohorts, and a gender specificity bias in the larger test-optional cohort that emerges only in the aggregate analysis.

Significance. The paper tackles a socially important and timely question, and it has several strengths: it uses a real institutional dataset, the authors are transparent about the subjective nature of the 5% threshold, and they make a genuine effort to assess stability via repeated random splits (up to 500 trials). The observed disparities in error patterns are policy-relevant and merit further investigation. However, the statistical support for the persistence claims is currently incomplete, and the internal inconsistency between Sections 4.2 and 4.3 weakens confidence in the presented results. If the authors add proper uncertainty quantification, resolve the contradiction, and clarify the evaluation protocol, the paper could be a useful contribution to the literature on bias in AI-based admissions.

major comments (5)
  1. [Section 3, Tables 7-9] The definition of bias as a difference of 5% or more in subgroup accuracy, specificity, or sensitivity is explicitly acknowledged as subjective, yet all headline findings are operationalized through this threshold. The aggregate results in Tables 7-9 report only mean values over 10, 100, and 500 random splits, with no standard deviations, confidence intervals, exceedance rates, or hypothesis tests. As a result, the claim that biases persist over up to 500 splits is not statistically supported; for example, the gender specificity difference in the larger test-optional model is 0.056-0.057, only slightly above the threshold, and it is possible that many individual splits fall below it. I recommend reporting the distribution of differences across splits, including the proportion of trials exceeding the threshold, and a sensitivity analysis showing how findings change for alternative thresholds.
  2. [Section 4.2 vs. Section 4.3, Tables 4 and 7] There is a direct contradiction between the single-split and aggregate analyses for gender. Section 4.2 and Table 4 state that "None of the predictive models where Gender is the sensitive variable show bias," but Section 4.3 and Table 7 report a Specificity bias in the larger test-optional model, with differences of 0.057, 0.057, and 0.056 across 10, 100, and 500 trials. The paper must reconcile these statements; as written, the aggregate result invalidates the earlier claim, and the discrepancy is not discussed.
  3. [Section 3 and Section 4.3] The evaluation protocol is ambiguous. The text says that for each trial two-thirds of the data is randomly selected for training and that each model is validated using five-fold cross-validation, but it does not state whether the reported Specificity, Sensitivity, and fairness metrics are computed on the held-out one-third test set, on cross-validation folds, or on the training set. This is essential for interpreting Tables 4-9 and for replicating the analysis. Please clarify the exact computation of the reported metrics.
  4. [Section 3 and Section 4.1] The Methods section promises, for each of the three groups, three analyses (GPA included/test scores excluded, test scores included/GPA excluded, both included), each repeated for three sensitive variables. Section 4.1, however, reports only three scenarios: test-required with both GPA and test scores, test-optional with GPA only, and the full dataset with GPA only. The remaining six planned analyses are neither presented nor explicitly omitted with justification. This mismatch between the stated design and the reported results should be resolved.
  5. [Section 4.3, fairness metrics] The claim that specificity/sensitivity bias not reflected in the fairness metrics "underscores an important limitation of bias and fairness metrics" is overstated. All metrics are evaluated using the same arbitrary 5% threshold, and the fairness metrics (Brier score, balance for positive/negative class) measure calibration rather than error-rate disparities, so differences in their behavior are expected. The discussion should be reframed as a comparison of threshold-based error rates with calibration-based metrics, rather than a general limitation of fairness metrics.
minor comments (6)
  1. [Table 2] Table 2 is difficult to read because the three variable-importance lists are interleaved in a single block; consider separating them into three panels.
  2. [Section 4.2] Section 4.2 uses the phrase "significant differences" in overall accuracies without any statistical testing; please either perform significance tests or use neutral wording such as "observed differences."
  3. [Section 3, Tables 4-6] The paper does not report base rates or class proportions for the subgroups within each cohort, which would help contextualize the specificity/sensitivity differences in Tables 4-9.
  4. [Section 3] The definitions of Balance for the Negative Class and Balance for the Positive Class are informal; please provide formal mathematical definitions for reproducibility.
  5. [Abstract and Section 4.1] The abstract's mention of "various scenarios" is broader than the three scenarios actually reported in Section 4.1; align the abstract and results.
  6. [Section 2] Several references in the Related Work section are incomplete (the citation is shown as "0"); these should be fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical measurement study whose bias findings are computed from held-out predictions and do not reduce to fitted inputs or self-citations.

full rationale

The paper's central claims are empirical measurements of a trained SVM on admissions data, not a derivation chain in which an output is defined in terms of its inputs. Bias is operationalized as a >=5% subgroup difference in accuracy, specificity, or sensitivity, and the reported values are computed from models evaluated on held-out test portions under repeated random splits. The subjective 5% threshold is a definitional choice and a statistical robustness limitation, but it is not circular: the threshold does not by construction force any particular subgroup difference to exceed it, and the paper explicitly reports cases where no bias was found. The self-citation to [1] appears only as context for the extension and is not used to justify the present findings, so it is not load-bearing. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no external empirical pattern is repackaged as a new result. The main legitimate concerns are statistical (absence of confidence intervals or hypothesis tests around the 5% threshold), which belong under correctness risk rather than circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The analysis introduces no new entities or forces. The only hand-chosen parameter is the 5% bias threshold. The main domain assumptions concern the validity of historical admission decisions as ground truth and the simulation of a larger test-optional dataset by deleting test scores.

free parameters (1)
  • Bias threshold = 0.05
    A difference of 5% or more in accuracy, specificity, or sensitivity between subgroups is declared biased. The authors state this is 'selected somewhat subjectively' in Section 3.
assumptions (4)
  • domain assumption The historical admission decisions in the dataset are correct ground-truth labels for evaluating model errors.
    The entire bias analysis compares model predictions to the recorded direct-admit decisions, treating those decisions as ground truth.
  • domain assumption Removing test scores from the full dataset simulates a larger test-optional dataset.
    Section 4.2 acknowledges this ignores policy-driven changes in applicant composition, but the bias persistence claim for the larger dataset depends on this constructed dataset.
  • domain assumption The linear SVM is a suitable model class for measuring bias in this admissions setting.
    Section 3 states that other models could improve accuracy but are beyond scope; the bias results are specific to this model family.
  • standard math Standard five-fold cross-validation and stratified sampling yield stable performance estimates.
    These are standard ML practices, though the paper does not report variance for all subgroup metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bias Analysis of AI Models for Undergraduate Student Admissions." pith.science (2026). https://pith.science/paper/3EGBLLXN

@misc{pith2026241202528,
  author       = {Pith},
  title        = {Pith review of: Bias Analysis of AI Models for Undergraduate Student Admissions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3EGBLLXN}},
  note         = {Machine review of arXiv:2412.02528}
}
read the original abstract

Bias detection and mitigation is an active area of research in machine learning. This work extends previous research done by the authors to provide a rigorous and more complete analysis of the bias found in AI predictive models. Admissions data spanning six years was used to create an AI model to determine whether a given student would be directly admitted into the School of Science under various scenarios at a large urban research university. During this time, submission of standardized test scores as part of an application became optional which led to interesting questions about the impact of standardized test scores on admission decisions. We developed and analyzed AI models to understand which variables are important in admissions decisions, and how the decision to exclude test scores affects the demographics of the students who are admitted. We then evaluated the predictive models to detect and analyze biases these models may carry with respect to three variables chosen to represent sensitive populations: gender, race, and whether a student was the first in his or her family to attend college. We also extended our analysis to show that the biases detected were persistent. Finally, we included several fairness metrics in our analysis and discussed the uses and limitations of these metrics.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 13 canonical work pages

  1. [1]

    Kelly Van Busum and Shiaofen Fang. 2023. Analysis of AI Models for Student Admissions: A Case Study. Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing (SAC ’23), March 2023, 17-22. DOI: https://doi.org/10.1145/3555776.3577743

  2. [2]

    Hammer, Tenora D

    Leslie B. Hammer, Tenora D. Grigsby, and Steven Woods. 1998. The Conflicting Demands of Work, Family, and School Among Students at an Urban University. The Journal of Psychology 132, 1 (1998), 220-226. DOI: https://doi.org/10.1080/00223989809599161

  3. [3]

    Harper, Edward J

    Shaun R. Harper, Edward J. Smith, and Charles H. F. Davis, III. 2018. A Critical Race Case Analysis of Black Undergraduate Student Success at an Urban University. Urban Education 53, 1 (2018), 3-25. DOI: https://doi.org/10.1177/0042085916668956

  4. [4]

    Luisa Sotomayor, Derya Tarhan, Marcelo Vieta, Shelagh McCartney, and Aida Mas. 2022. When students are house-poor: Urban universities, student marginality, and the hidden curriculum of student housing. Cities 124 (May 2022), 103572. DOI: https://doi.org/10.1016/j.cities.2022.103572

  5. [5]

    Nuria Rodriquez-Planas. 2022. Hitting where it hurts most: COVID-19 and low-income urban college students. Econ Ed Rev (Apr. 2022), 102233. DOI: https://doi.org/10.1016/j.econedurev.2022.102233

  6. [6]

    Jon Edelman. 2022. Survey: Test-Optional is Appealing to Minority Students. Retrieved Oct. 12, 2022 from https://www.diverseeducation.com/students/article/15292751/survey-finds-testoptional-policies-a-significant-motivator-for- minority-college-applicants

  7. [7]

    Christopher T. Bennett. 2022. Untested Admissions: Examining Changes in Application Behavior and Student Demographics Under Test-Optional Policies. American Educational Research Journal 59, 1 (Feb. 2022), 180-216. DOI: https://doi.org/10.3102/00028312211003526

  8. [8]

    Belasco, Kelly O

    Andrew S. Belasco, Kelly O. Rosinger, and James C. Hearn. 2015. The Test-Optional Movement at America’s Selective Liberal Arts Colleges: A Boon for Equity or Something Else? Educational Evaluation and Policy Analysis 37, 2 (June 2015), 206-223. DOI: https://doi.org/10.3102/0162373714537350

Show all 25 references
  1. [9]

    Zhi Liu and Nikhil Garg. 2021. Test-optional Policies: Overcoming Strategic Behavior and Informational Gaps. In Proceedings of the ACM Equity and Access in Algorithms, Mechanisms, and Optimization (EEAAMO ’21), Oct. 5-9, 2021, ACM Inc., New York, NY, 1-13. DOI: https://doi.org...

  2. [10]

    Austin Waters and Risto Miikkulainen. 2014. GRADE: Machine Learning Support for Graduate Admissions. AI Magazine 35, 1 (Spring 2014), 64-75. DOI: https://doi.org/10.1609/aimag.v35i1.2504

  3. [11]

    Young and Marcos D

    Nicholas T. Young and Marcos D. Caballero. 2019. Using machine learning to understand physics graduate school admissions. arXiv: 1907.01570. Retrieved from https://arxiv.org/abs/1907.01570

  4. [12]

    Joseph Jamison. 2017. Applying Machine Learning to Predict Davidson College’s Admissions Yield. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’17), Mar. 8-11, 2017, Seattle, Washington. ACM Inc., New York, NY, 765-766. DOI: htt...

  5. [13]

    Acharya, Asfia Armaan, and Aneeta S

    Mohan S. Acharya, Asfia Armaan, and Aneeta S. Antony. A Comparison of Regression Models for Prediction of Graduate Admissions. In 2019 International Conference on Computational Intelligence in Data Science (ICCIDS) , Feb. 21- 23, 2019, Chennai, India. IEEE, 1-5. DOI: 10.1109/I...

  6. [14]

    Ch. V. Raghavendran, Ch. Pavan Venkata Vamsi, T. Veerraju, and Ravi Kishore Veluri. 2021. Predicting Student Admissions Rate into University Using Machine Learning Models. In: D. Bhattacharyya and N. Thirupathi Rao, Machine Intelligence and Soft Computing. Advances in Intellig...

  7. [15]

    Alvero, Noah Arthurs, Anthony Lising Antonio, Benjamin W

    A.J. Alvero, Noah Arthurs, Anthony Lising Antonio, Benjamin W. Domingue, Ben Gebre -Medhin, Sonia Giebel, and Mitchell L. Stevens. 2020. AI and Holistic Review: Informing Human Reading in College Admissions. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society ...

  8. [16]

    Barbara Martinez Neda, Yue Zeng, and Sergio-Gago-Masague. 2021. Using Machine Learning in Admissions: Reducing Human and Algorithmic Bias in the Selection Process. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE ’21), Mar. 13-20, 2021. ...

  9. [17]

    Brian d'Alessandro, Cathy O'Neil, and Tom LaGatta. 2017. Conscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification. Big Data 5, 2 (Jun. 2017), 120-134. DOI: http://doi.org/10.1089/big.2016.0048

  10. [18]

    David Danks and Alex John London. 2017. Algorithmic Bias in Autonomous Systems. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI ‘17) , Aug. 19-25, 2017. Melbourne, Australia, 4691-4697. DOI: https://doi.org/10.24963/ijcai.2017/654

  11. [19]

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2019. A Survey on Bias and Fairness in Machine Learning. arXiv: 1908.09635. Retrieved from https://arxiv.org/abs/1908.09635

  12. [20]

    Simon Caton and Christian Haas. 2020. Fairness in Machine Learning: A Survey. arXiv: 2010.04053. Retrieved from https://arxiv.org/abs/2010.04053

  13. [21]

    Frank Marcinkowski, Kimon Kieslich, Christopher Starke, and Marco Lünich. 2020. Implications of AI (un-)fairness in higher education admissions: the effects of perceived AI (un-)fairness on exit, voice and organizational reputation. In Proceedings of the 2020 Conference on Fai...

  14. [22]

    Nima Kordzadeh and Maryam Ghasemaghaei. 2021. Algorithmic Bias: review, synthesis, and future research directions. European Journal of Information Systems (Jun. 2021), 388-409. DOI: https://doi.org/10.1080/0960085X.2021.1927212

  15. [23]

    Bahar Memarian and Tenzin Doleck. 2023. Fairness, Accountability, Transparency, and Ethics (FATE) in Artificial Intelligence (AI) and higher education: A systematic review. Computers and Education: Artificial Intelligence 5 (2023). DOI: https://doi.org/10.1016/j.caeai.2023.100152

  16. [24]

    Lorenzo Belenguer. 2022. AI bias: exploring discriminatory algorithmic decision -making models and the application of possible machine-centric solutions adapted from the pharmaceutical industry. AI Ethics 2, 4 (2022), 771-787. DOI: https://doi.org/10.1007/s43681-022-00138-8

  17. [25]

    Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent Trade -Offs in the Fair Determination of Risk Scores. arXiv: 1609.05807v1. Retrieved from https://arxiv.org/pdf/1609.05807v1.pdf

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.