REVIEW 3 major objections 5 minor 45 references
Algorithmic Bias in Lending: Evidence from a Fintech Audit
T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read A profit-based audit finds that a fintech lender’s race- and gender-blind underwriting model misprices loans by group, giving Black borrowers and men relatively favorable terms.
desk verdict Profit-gap result is strong, but the miscalibration mechanism rests on a proxy model; still deserves serious refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the annualized internal rate of return (IRR) of a group’s aggregated loan cash flows—the interest rate that makes the present value of repayments equal the principal disbursed, interpreted as realized profit per group. The paper’s test compares these group IRRs; a lender that prices risk accurately should earn similar returns across similarly priced groups. To diagnose the cause of gaps, the paper reconstructs the lender’s underwriting model by training a gradient-boosted tree on the same proprietary features without race or gender, checks its calibration against realized defaults, and then contrasts it with a model that includes race and gender. A second diagnostic com
What would settle it
Disclose the lender’s actual internal risk scores and default outcomes; if, conditional on the real score, Black borrowers do not default at higher rates than White borrowers (or women at lower rates than men), the miscalibration claim is false. Alternatively, if a separate audit using the lender’s true model finds no group profit gaps, the test’s diagnosis collapses.
Extended reading notes
Core claim
The central claim is that profit disparities across demographic groups, measured by annualized IRR on aggregate loan cash flows, serve as an outcome-based signal of discriminatory pricing under U.S. fair lending law. The paper reports that, in this fintech’s portfolio, loans to Black borrowers and men are less profitable than loans to other groups; because the lender sets higher target returns for riskier borrowers, this is opposite to what risk aversion alone would predict. The paper attributes the gap to miscalibration of the lender’s risk score: a race- and gender-blind model trained on the lender’s own features underestimates default risk for Black borrowers and overestimates it for wome
Load-bearing premise
The key assumption is that the reconstructed race- and gender-blind model, trained on the same proprietary features, behaves like the lender’s actual withheld underwriting model, so the miscalibration attributed to the lender is really the lender’s miscalibration.
Editorial extensions
If this is right
- If the profit-based test is adopted, regulators can screen lenders using only application demographics and repayment histories, without access to proprietary risk models.
- The paper’s APR-conditioned default curves offer a direct way to detect miscalibration in any lender’s pricing.
- Correcting miscalibration by including race and gender would raise APRs for Black borrowers by about 0.8 points and lower approval rates for Black applicants by about 3 percentage points, according to the paper’s estimates.
- The same method can generalize to other credit markets and products, as the paper notes, though the direction and magnitude of disparities may vary.
- The findings imply that facially neutral algorithmic underwriting can produce disparate impact under current law even when no protected attribute is used.
Reading between the lines
- Editorial inference: The profit-gap test could be repurposed as a continuous monitoring tool, tracking changes in group IRR over time to spot drift in model calibration before regulatory complaints arise.
- Editorial inference: The miscalibration pattern—underestimating risk for historically marginalized groups—may reflect systematic bias in training labels or historical lending data; if so, similar audits in other fintechs should find analogous patterns, a testable extension across lenders.
- Editorial inference: The paper’s conclusion suggests a policy tension: rather than adding protected attributes to models (legally suspect), lenders might alternatively recalibrate risk models on outcomes after excluding demographic-sensitive proxies, or regulators could require calibration audits, though the paper does not propose these.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a profit-based measure of lending discrimination, operationalized as the annualized internal rate of return (IRR) on loans, applied to about 80,000 funded personal loans from a major U.S. fintech platform. The authors report that loans to Black borrowers and men yield lower profits than loans to other groups, implying relatively favorable pricing for these groups. They then attempt to trace this profitability gap to miscalibration in the platform's underwriting model, arguing that a reconstructed race- and gender-blind XGBoost model underestimates default risk for Black borrowers and overestimates it for women. They support this with APR-conditional default curves and a counterfactual analysis ruling out strategic shopping, and they show that an explicitly race- and gender-aware model would reduce the disparities, illustrating a trade-off between fairness notions.
Significance. If the main claims hold, this is a valuable contribution to empirical fair-lending research and algorithmic auditing. The profit-based IRR test is simple, transparent, and avoids common pitfalls such as omitted-variable bias in regression-based tests and inframarginality in default-rate comparisons. The paper uses a unique proprietary dataset with lender target returns, repayment histories, and the full feature set used by the underwriting model, and it includes robustness checks with alternative demographic imputation. The practical relevance is high because the method requires only repayment data and demographics, making it feasible for regulators. However, the strength of the contribution depends critically on the miscalibration attribution, which is currently supported only indirectly through a proxy model and APR-conditional comparisons.
major comments (3)
- [Section 5, Fig. 4 and Fig. 2] The central claim that the lender's underwriting model is miscalibrated for Black and women borrowers rests on a reconstructed 'blind' XGBoost model that the authors themselves acknowledge is a proxy ('our analysis is inherently limited since we do not have access to the actual, proprietary risk model'). The lender's actual risk score is withheld, yet the paper observes the lender's target return for each funded loan, which is a direct output of the internal model via the cumulative-loss-rate curve in Fig. 2. The target returns are only used as an aggregate control for risk aversion in Fig. 3; they are never used to test calibration directly. A straightforward test would invert the target-return curve to derive the lender's predicted cumulative loss rate for each loan and compare it with realized cumulative loss by group. This would directly confirm or refute the miscalibration mechanism
- [Section 5, Fig. 5] The APR-conditional default analysis, which is the only evidence aimed at the lender's internal model (rather than the proxy), assumes APR is approximately monotone in the lender's internal risk score and not confounded by other pricing factors. APR in this market varies with the federal funds rate (the authors' own APR model in the same section includes the federal rate F_i), loan amount, origination costs, and potentially applicant-specific factors. If these factors are correlated with race or gender, group differences in default rates at a fixed APR can arise even when the lender's risk score is perfectly calibrated. To make this test convincing, the authors should adjust APR for these covariates or provide evidence that APR is a sufficient statistic for the internal risk score. As presented, Fig. 5 does not establish that the lender's own model is miscalibrated in the claimed directi
- [Section 6, Eq. (3)] The counterfactual IRR model used to rule out strategic shopping is fitted on funded loans: IRR_i = α R_A,i + γ LoanAmount_i + β APR_i. Applying this model to all approved applicants assumes the relationship between APR, risk score, loan amount, and IRR is the same for applicants who accept the offer and those who shop for better terms or decline. If shopping behavior is correlated with the unobserved error in this linear model, the counterfactual IRR gaps in Fig. 8 could be biased, weakening the conclusion that shopping does not explain the profit disparities. At a minimum, the model should be validated on held-out data or compared with an alternative specification that includes observable applicant characteristics. This concern is secondary to the miscalibration evidence but is load-bearing for ruling out an important alternative explanation.
minor comments (5)
- [Figure 6 caption] The caption states 'reconstructed race- and gender-blind risk scores' but the figure describes the race- and gender-aware model; this appears to be a typo and should be corrected.
- [Figure 2 caption] The caption says 'the proportion of the principle that the lender expects to lose'; 'principle' should be 'principal.'
- [Section 5, last paragraph] The counterfactual approval/APR changes in Fig. 7 are labeled 'for illustrative purposes only,' but the framing could be read as recommending a legally impermissible aware model. Consider adding an explicit sentence that the authors do not endorse using protected attributes in underwriting and that the figure is only intended to illustrate the mechanics of miscalibration.
- [Section 5, Fig. 4] The calibration figures for the proxy XGBoost would benefit from reporting standard performance metrics (e.g., AUC, Brier score) and the number of loans per group, so readers can assess whether the apparent miscalibration is driven by small samples or model underfitting.
- [Section 3] The description of IRR as ranging from -100% to 'arbitrarily large' is correct, but the convention for immediate defaults and prepayments is only briefly explained. Consider moving the technical details of the cash-flow construction from Section 6 to Section 3, since they affect the main estimates.
Circularity Check
No significant circularity; the profit gaps come from realized cash flows, and the miscalibration analysis is out-of-sample rather than a refit of the target quantity.
full rationale
The core profit test (Section 3, Fig. 1) is computed from realized loan cash flows and group weights, not from the lender's risk model or the paper's fitted models, so it is externally grounded. The miscalibration claim (Section 5) is explicitly a reconstruction: the authors 'evaluate the calibration of our own (race- and gender-blind) risk model trained with the same set of proprietary features used by the lender’s internal model,' using 5-fold cross-validation. The race/gender calibration gaps in Fig. 4 are therefore out-of-sample predictions, not in-sample fits. The APR-conditional default comparison in Fig. 5 uses an observable APR and realized defaults as an indirect check on the lender's internal calibration; this is a proxy-validity argument, not a definitional circle. The aware-model correction (Figs. 6–7) is labeled 'for illustrative purposes only' and is not used as the identification of the profit gap. The paper also states its key limitation directly: 'our analysis is inherently limited since we do not have access to the actual, proprietary risk model the lender employs.' That is a validity limitation, not circularity. Self-citations (e.g., [16], [23], [38]) support background concepts like inframarginality and fairness and are not load-bearing for the empirical derivation. No equation or fitted parameter is shown to reduce to the quantity it is claimed to predict.
Assumptions & free parameters
free parameters (7)
- XGBoost hyperparameters for blind and aware risk models =
not reported
- Approval model coefficients (alpha, beta) =
not reported
- APR model coefficients (alpha, beta, gamma) =
not reported
- Counterfactual IRR model coefficients (alpha, gamma, beta) =
not reported
- GAM smoothing parameters =
not reported
- Immediate default IRR convention =
-100%
- Recovery assumption R_t =
0
assumptions (6)
- domain assumption In a competitive marketplace of risk-neutral lenders, loans are priced to achieve the same expected return for every borrower.
- domain assumption Under U.S. fair lending law, the only acceptable justification for pricing disparities is creditworthiness.
- domain assumption BISG probability estimates provide valid proxies for race/ethnicity and gender.
- domain assumption The proprietary feature set provided to the authors is the same full set used by the lender’s internal model.
- ad hoc to paper The authors’ reconstructed blind XGBoost model approximates the lender’s internal underwriting model.
- domain assumption APR is approximately monotone in the lender’s internal risk score.
Cite this review
Pith. "Pith review of Algorithmic Bias in Lending: Evidence from a Fintech Audit." pith.science (2026). https://pith.science/paper/LXMPJR2J
@misc{pith2026251220753,
author = {Pith},
title = {Pith review of: Algorithmic Bias in Lending: Evidence from a Fintech Audit},
year = {2026},
howpublished = {\url{https://pith.science/paper/LXMPJR2J}},
note = {Machine review of arXiv:2512.20753}
}
read the original abstract
Algorithmic lending has transformed the consumer credit landscape, with machine learning models commonly facilitating underwriting decisions. To comply with fair lending laws, these algorithms exclude legally protected characteristics, such as race and gender. Yet algorithmic underwriting can still inadvertently favor certain groups, prompting concerns about whether lending algorithms exhibit discriminatory behavior. Using proprietary loan-level data from a major U.S. fintech platform, we audit lending decisions across approximately 80,000 personal loans. We find that loans made to men and Black borrowers yielded lower profits than loans to other groups, suggesting that men and Black borrowers benefited from relatively favorable pricing. We trace these disparities to miscalibration in the platform's underwriting model, which overestimates risk for women and underestimates risk for Black borrowers. We then show that one could correct this miscalibration -- and the corresponding disparities -- by including race and gender in underwriting models, illustrating a tension between competing notions of fairness.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Akinwumi, J
M. Akinwumi, J. Merrill, L. Rice, K. Saleh, and M. Yap. An ai fair policy lending agenda for the federal financial regulators, 2021. Brookings Center on Regulation and Markets
2021
-
[2]
Does credit scoring produce a disparate impact?Real Estate Economics, 40:S65–S114, 2012
Robert B Avery, Kenneth P Brevoort, and Glenn Canner. Does credit scoring produce a disparate impact?Real Estate Economics, 40:S65–S114, 2012
2012
-
[3]
Outcome tests of racial disparities in police practices.Justice Research and Policy, 4(1-2):131–142, 2002
Ian Ayres. Outcome tests of racial disparities in police practices.Justice Research and Policy, 4(1-2):131–142, 2002
2002
-
[4]
Credit access in the united states
Trevor J Bakker, Stefanie DeLuca, Eric A English, James S Fogel, Nathaniel Hendren, and Daniel Herbst. Credit access in the united states. Technical report, National Bureau of Economic Research, 2025
2025
-
[5]
Barocas, M
S. Barocas, M. Hardt, and A. Narayanan.Fairness and machine learning. 2019. URL http://fairmlbook.org
2019
-
[6]
Big data’s disparate impact.Calif
Solon Barocas and Andrew D Selbst. Big data’s disparate impact.Calif. L. Rev., 104: 671, 2016
2016
-
[7]
R. Bartlett, A. Morse, R. Stanton, and N. Wallace. Consumer-lending discrimination in the fintech era.J Financ Econ, 143(1):30–56, 2022. doi: 10.1016/j.jfineco.2021.05.031
-
[8]
Becker.The economics of discrimination
G.S. Becker.The economics of discrimination. University of Chicago Press, 2nd edition,
Show all 45 references
-
[9]
G.S. Becker. The evidence against banks doesn’t prove bias, 1993. In Bloomberg Businessweek (Online) (No. 3315, p. 18), Bloomberg Finance LP
1993
-
[10]
Bhutta and A
N. Bhutta and A. Hizmo. Do minorities pay more for mortgages?Rev Financ Stud, 34(2):763–789, 2020
2020
-
[11]
Bhutta, A
N. Bhutta, A. Hizmo, and D. Ringo. How much does racial bias affect mortgage lend- ing? Evidence from human and algorithmic credit decisions.The Journal of Finance, LXXX(WP 24-09), 2024
2024
-
[12]
How costly is noise? data and disparities in consumer credit.arXiv preprint arXiv:2105.07554, 2021
Laura Blattner and Scott Nelson. How costly is noise? data and disparities in consumer credit.arXiv preprint arXiv:2105.07554, 2021
2021 arXiv
-
[13]
Chohlas-Wood, M
A. Chohlas-Wood, M. Coots, S. Goel, and J. Nyarko. Designing equitable algorithms. Nature Computational Science, 3(7):601–610, 2023. 21
2023
-
[14]
Using publicly available information to proxy for unidentified race and ethnicity: A methodology and assessment
Consumer Financial Protection Bureau. Using publicly available information to proxy for unidentified race and ethnicity: A methodology and assessment. Technical report, Consumer Financial Protection Bureau, Washington, D.C., 2014
2014
-
[15]
Con- sent Order In the Matter of Ally Financial Inc
Consumer Financial Protection Bureau and United States Department of Justice. Con- sent Order In the Matter of Ally Financial Inc. and Ally Bank, 2013. Administrative Proceeding File No. 2013-CFPB-0010
2013
-
[16]
Corbett-Davies, J.D
S. Corbett-Davies, J.D. Gaebler, H. Nilforoshan, R. Shroff, and S. Goel. The mea- sure and mismeasure of fairness.J Mach Learn Res, 24(312):1–117, 2023. URL http://jmlr.org/papers/v24/22-1511.html
2023
-
[17]
Di Maggio and V
M. Di Maggio and V. Yao. Fintech borrowers: Lax screening or cream-skimming? Review of Financial Studies, 34(10):4565–4618, 2020. doi: 10.1093/rfs/hhaa142
2020 doi
-
[18]
Di Maggio, D
M. Di Maggio, D. Ratnadiwakara, and D. Carmichael. Invisible primes: Fintech lending with alternative data. Working Paper 29840, National Bureau of Economic Research,
-
[19]
Dobbie, A
W. Dobbie, A. Liberman, D. Paravisini, and V. Pathania. Measuring bias in consumer lending.Rev Econ Stud, 88(6):2799–2832, 2021. doi: 10.1093/restud/rdaa078
2021 doi
-
[20]
Elliott, A
M.N. Elliott, A. Fremont, P.A. Morrison, P. Pantoja, and N. Lurie. A new method for estimating race/ethnicity and associated disparities where administrative records lack self-reported race/ethnicity.Health Serv Res, 43:1772–1736, 2008
2008
-
[21]
Elliott, P.A
M.N. Elliott, P.A. Morrison, A. Fremont, D.F. McCaffrey, P. Pantoja, and N. Lurie. Using the census bureau’s surname list to improve estimates of race/ethnicity and associated disparities.Health Serv Outcomes Res Methodol, 9:69–83, 2009
2009
-
[22]
Predictably unequal? The effects of machine learning on credit markets, 2017
A Fuster, P Goldsmith-Pinkham, T Ramadorai, and A Walther. Predictably unequal? The effects of machine learning on credit markets, 2017
2017
-
[23]
Gaebler and Sharad Goel
Johann D. Gaebler and Sharad Goel. A simple, statistically robust test of discrimina- tion.Proceedings of the National Academy of Sciences, 122(10):e2416348122, 2025
2025
-
[24]
Giacoletti, R
M. Giacoletti, R. Heimer, and E.G. Yu. Using high-frequency evaluations to esti- mate discrimination: Evidence from mortgage loan officers, 2023. Available at SSRN 3795547
2023
-
[25]
T. B. Gillis. The input fallacy.Minnesota Law Review, 106(3):1175, 2022. ISSN 0026-5535. 22
2022
-
[26]
price discrimination
Talia B. Gillis. “price discrimination” discrimination. Working paper, Columbia Law School; draft (Nov 2024), available on SSRN, 2024
2024
-
[27]
Discrimination in lending markets: Status and the intersections of gender and race.Social Psychology Quarterly, 79(1):81–93, 2016
Sarah K Harkness. Discrimination in lending markets: Status and the intersections of gender and race.Social Psychology Quarterly, 79(1):81–93, 2016
2016
-
[28]
Credit scoring in the era of big data.Yale JL & Tech., 18:148, 2016
Mikella Hurley and Julius Adebayo. Credit scoring in the era of big data.Yale JL & Tech., 18:148, 2016
2016
-
[29]
The fairness of credit scoring models.Management Science, 2024
Christophe Hurlin, Christophe P´ erignon, and S´ ebastien Saurin. The fairness of credit scoring models.Management Science, 2024
2024
-
[30]
J. Jung, S. Corbett-Davies, J. Gaebler, R. Shroff, and S. Goel. Measuring disparate impact in human and machine decisions.Proceedings of the National Academy of Science, 2026. (forthcoming)
2026
-
[31]
Kleinberg, J
J. Kleinberg, J. Ludwig, S. Mullainathan, and C. R. Sunstein. Discrimination in the age of algorithms.Journal of Legal Analysis, 10:113–174, 2018
2018
- [32]
-
[33]
Are credit scores gender-neutral? ev- idence of mis-calibration from alternative and traditional borrowing data.Journal of Behavioral and Experimental Finance, 47:101081, 2025
Zilong Liu and Hongyan Liang. Are credit scores gender-neutral? ev- idence of mis-calibration from alternative and traditional borrowing data.Journal of Behavioral and Experimental Finance, 47:101081, 2025. ISSN 2214-6350. doi: https://doi.org/10.1016/j.jbef.2025.101081. URL h...
2025
-
[34]
Munnell, G.M.B
A.H. Munnell, G.M.B. Tootell, L.E. Browne, and J. McEneaney. Mortgage lending in boston: Interpreting hmda data.Am Econ Rev, 86:25–53, 1996
1996
-
[35]
K.A. Park. Measuring risk and access to mortgage credit with new disclosure data.J Struct Finance, 26(4):53–72, 2021
2021
-
[36]
Implementing anti-discrimination policies in statistical profiling models.American Economic Journal: Economic Policy, 3(3):206– 231, 2011
Devin G Pope and Justin R Sydnor. Implementing anti-discrimination policies in statistical profiling models.American Economic Journal: Economic Policy, 3(3):206– 231, 2011
2011
-
[37]
Testing for racial bias in business credit scores
Alicia Robb and David T Robinson. Testing for racial bias in business credit scores. Small Business Economics, 50(3):429–443, 2018. 23
2018
-
[38]
The problem of inframarginal- ity in outcome tests for discrimination.Annals of Applied Statistics, 11:1193–1216, 2017
Camelia Simoiu, Sam Corbett-Davies, and Sharad Goel. The problem of inframarginal- ity in outcome tests for discrimination.Annals of Applied Statistics, 11:1193–1216, 2017
2017
-
[39]
Gender, risk assessment, and sanctioning: The cost of treating women like men.Law and human behavior, 40 (5):580, 2016
Jennifer Skeem, John Monahan, and Christopher Lowenkamp. Gender, risk assessment, and sanctioning: The cost of treating women like men.Law and human behavior, 40 (5):580, 2016
2016
-
[40]
Winnie. F. Taylor. The ecoa and disparate impact theory: A historical perspective. Journal of Law and Policty, 26(2):575–635, 2018
2018
-
[41]
Congress
U.S. Congress. Fair Housing Act, 1968
1968
-
[42]
Congress
U.S. Congress. Equal Credit Opportunity Act, 1974
1974
-
[43]
Assessing fair lending risks using race/ethnicity proxies.Management Science, 64(1):178–197, 2018
Yan Zhang. Assessing fair lending risks using race/ethnicity proxies.Management Science, 64(1):178–197, 2018. 24 Appendix BISG Robustness Check In line with recommendations from the Consumer Financial Protection Bureau, our main results were obtained by using the BISG probabil...
2018
-
[1971]
Originally published in 1957
1957
-
[2022]
URLhttp://www.nber.org/papers/w29840
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.