{"id":"61df8170-b59d-4a4d-8511-5a229d5a0fd9","arxiv_id":"2412.05832","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Machine learning models for substance use treatment length of stay show selection-rate disparities by race, region, substance, diagnosis, and payment source in 2019 US discharge data.","lead":"Researchers trained machine learning models to predict how long patients stay in substance use treatment, then checked the predictions for bias across racial, medical, and financial groups. They found that race, US region, type of substance, diagnosis, and payment source show the largest fairness gaps in both inpatient and outpatient settings.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Demographic parity ratios are never benchmarked against actual long-stay rates, so the claimed unfairness may simply reflect legitimate differences in LOS.","rationale":"The reader's weakest assumption is that selection-rate parity is the correct fairness standard and that lower selection rates might reflect clinically appropriate shorter stays rather than unfair treatment. This is exactly the load-bearing issue in the paper. The reported fairness metrics quantify disparities in predicted positive outcomes, but they never establish that those disparities exceed, or even differ from, disparities in the actual long-stay outcomes the model was trained to predict. Since the model is trained on historical TEDS-D LOS, its predicted rates can mirror existing differences in treatment duration that are correlated with race, region, diagnosis, and payment source. Calling these differences 'unfairness' requires showing that similarly situated patients with similar clinical needs receive systematically different predictions, or that the model amplifies outcome disparities present in the labels. Neither is provided. The proposed test—comparing model DPR to actual-label DPR—would directly distinguish inherited disparity from model-introduced disparity and is feasible with the public TEDS-D data. The verdict remains CONDITIONAL: the empirical audit is potentially valuable, but the central interpretive claim requires this additional baseline analysis.","tokens_in":28198,"tokens_out":2816,"duration_ms":32545,"concrete_test":"Recompute on the same 2019 TEDS-D test sets the actual long-stay prevalence (Y=1) and the corresponding actual-label demographic parity ratio (DPR_actual) for each variable in Tables 3 and 4, especially race for inpatient and payment source for outpatient. Compare DPR_actual with the model DPR values. If DPR_actual is approximately equal to the model DPR, the disparities are inherited from historical LOS and do not by themselves demonstrate model-introduced unfairness; if the model DPR is substantially lower than DPR_actual after stratifying by diagnosis and region, the bias claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that race, region, diagnosis, and payment source are 'primary indicators of unfairness' rests on selection-rate disparities, but the paper never compares model selection rates to the actual long-stay rates in the test data. In Section 3.4, fairness is assessed by computing P(Yhat=1 | A=a) and applying the 80% rule, yet the target Y is itself a binarized LOS. If some groups genuinely have lower long-stay prevalence—due to case mix, clinical need, or regional practice—then a well-calibrated model will by construction produce disparate selection rates, and DPR < 0.8 is not evidence of model-introduced bias. The paper even states in Section 2.2 that differences in resource use based on medical needs are acceptable, but the fairness analysis never conditions on medical need or compares predicted to actual rates. Notably, the paper's own definition of selection rate in Section 3.4 ('ratio of predicted positive outcomes to ground truth positive outcomes') is not the standard selection rate used elsewhere, and the reported metrics never include actual positive rates by subgroup. Without a baseline comparison to actual LOS or an adjustment for clinical need, the headline claim does not follow from the reported metrics.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops machine learning classifiers to predict binarized length of stay (LOS) for inpatient (>30 days) and outpatient (>90 days) substance use disorder (SUD) treatment using the 2019 TEDS-D dataset. After feature selection by majority vote and training random forest, LightGBM, XGBoost, and TabNet, the authors assess fairness with FairLearn metrics (selection rate, demographic parity ratio, equalized odds difference, false positive/negative rate differences). They report that race, U.S. region, substance use at discharge, SUD diagnosis, and payment source exhibit the largest demographic parity violations, identify specific subgroups with demographic parity ratios below 0.80, and propose model-adjustment and policy mitigation strategies. The central claim is that the observed selection-rate disparities constitute model bias with implications for discharge decisions and health equity.","tokens_in":28406,"tokens_out":4304,"duration_ms":45397,"significance":"The topic is timely and relevant: predictive models for SUD treatment LOS are an underexplored fairness domain, and the paper uses a large nationwide public dataset with a transparent pipeline. The authors are to be credited for using standard tools (FairLearn), reporting group- and subgroup-level metrics, and connecting their results to distributive-justice and socio-relational framing. The study also makes falsifiable, concrete claims about specific subgroups (e.g., Alaska Native, barbiturate users, self-pay patients). However, the central inference from selection-rate disparity to unfairness is not yet supported. The analysis never compares model selection rates to actual long-stay rates by subgroup, nor does it condition on clinical need, despite the paper's own acknowledgement that need-based differences are acceptable. If the authors add the missing baseline comparisons and uncertainty quantification, the paper could become a useful fairness audit; in its current form, the evidence for the headline claim is incomplete.","major_comments":[{"comment":"The central claim that race, U.S. region, substance use, diagnosis, and payment source are “primary indicators of unfairness” is not supported because the demographic parity ratios are never benchmarked against actual long-stay rates by subgroup. A well-calibrated model will produce DPR values below 0.80 whenever subgroup base rates differ, and Section 2.2 explicitly allows differences in resource use based on medical needs. The manuscript should report the actual (ground-truth) long-stay prevalence for each subgroup, compare predicted selection rates to those base rates, and ideally show that the disparities persist after conditioning on clinical need or after calibration. Without such a baseline, the reported DPR values (e.g., 0.19 for inpatient race under random forest in Table 3) do not by themselves demonstrate model-introduced bias.","section":"Section 3.4 and Appendix D (Tables D2–D12)"},{"comment":"Several subgroups identified as “disadvantaged” have very small sample sizes: e.g., Attention Deficit/disruptive behavior disorders (n=31, Table D2), Asian/Pacific Islander inpatient (n=502, Table D4), barbiturates outpatient (n=721, Table D9), and U.S. territories inpatient (n=352, Table D8). The reported selection rates are point estimates without confidence intervals, and the min/max demographic parity ratio is highly sensitive to small denominators. The claim that these small subgroups are underserved needs uncertainty quantification, such as bootstrap confidence intervals, or smoothing/empirical Bayes shrinkage, before policy conclusions are drawn.","section":"Section 4.3 and Appendix D"},{"comment":"The definition of selection rate is internally inconsistent. The text says selection rate is “the ratio of predicted positive outcomes to ground truth positive outcomes,” but Eq. (C1) computes (TP+FP)/(P+N), which is the predicted positive rate. The results in Tables 3–4 and Appendix D are consistent with the formula, not with the textual definition. This needs to be corrected, because the “ratio to ground truth” wording implies a comparison to actual rates that is never performed; the actual ground-truth positive rates are not reported anywhere.","section":"Section 3.4 and Appendix C (Eq. C1)"},{"comment":"The identification of disadvantaged subgroups is conditional on two imported thresholds: the LOS binarization thresholds (>30 days inpatient, >90 days outpatient) and the 0.80 demographic parity ratio rule. Neither choice is subjected to sensitivity analysis. The LOS thresholds are based on prior work and a partner provider, and the 80% rule is borrowed from employment disparate-impact guidance. A robustness check varying the LOS thresholds and the DPR threshold would clarify whether the main findings are stable or artifacts of these choices.","section":"Sections 3.1.2 and 3.4"}],"minor_comments":[{"comment":"The phrase “ratio of predicted positive outcomes to ground truth positive outcomes” should be replaced with “predicted positive rate” (or the metric should be redefined accordingly) to match the actual computation.","section":"Abstract and Section 3.4"},{"comment":"Accuracy, precision, recall, and F1 are reported to only two decimal places, so the differences between random forest and LightGBM are not visible (both show 0.81/0.77 or 0.81/0.76). Reporting additional digits or other metrics would make the model-selection step more transparent.","section":"Table 2"},{"comment":"The red (and green) dots are described as outliers, but the colors appear to be assigned by the 0.80 demographic parity threshold, not by a statistical outlier rule. Please clarify the criterion used to mark these points.","section":"Figures 2 and 3"},{"comment":"The text states that “the subgroups identified by Asian/Pacific Islander, Native Americans, and multiple racial backgrounds show the lowest selection rates,” but Appendix Table D4 shows Alaska Native with the lowest selection rate for inpatients. Please make the summary consistent with the appendix tables.","section":"Section 5.2"},{"comment":"The optimal LOS thresholds are said to be determined in consultation with prior literature and “a partner SUD treatment provider.” If the provider input influenced the threshold choice, more detail should be provided; otherwise the reliance on an unnamed partner should be disclosed or removed.","section":"Section 3.1.2"},{"comment":"Several reference entries contain formatting artifacts (e.g., “e.a.” instead of “et al.”) and missing page numbers; please clean up the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the paper is a solid, well-organized fairness audit that is potentially publishable, but the main conclusion goes beyond what the reported metrics can support. The missing baseline comparison (predicted selection rates vs. actual long-stay rates) is the key substantive gap. I would ask the authors to add that analysis and confidence intervals before reconsidering. I do not see any ethical or provenance concerns beyond the statistical issues noted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know this paper is a competent, reproducible fairness audit of ML models for substance-use-disorder length-of-stay, but its headline claim is under-supported. The fairness metrics are never benchmarked against actual long-stay rates, so the reported disparities may simply reflect base-rate differences across groups.\n\nWhat's actually new is empirical: subgroup-level selection-rate parity tables for inpatient and outpatient SUD LOS predictions on TEDS-D 2019. They use public data, standard FairLearn metrics, group and subgroup levels, and four model families with a reasonable feature-selection pipeline. The measurements look reproducible, the data work is solid, and the citation pattern is normal for this line of research. The paper is honestly described as a first application to this domain.\n\nThe soft spot is the central one. In Section 3.4, selection rate is defined as 'the ratio of predicted positive outcomes to ground truth positive outcomes.' That is not the standard selection rate. If it means P(Ŷ=1 | A=a), then DPR below 0.8 only tells you groups are predicted positive at different rates. The paper never compares those predicted rates to the actual long-stay rates in the test data. In Section 2.2 they accept differences in resource use based on medical need, but the fairness analysis never conditions on need. Without that baseline, the 80% rule can flag a group as disadvantaged when its actual long-stay rate is low. The claim that race, region, diagnosis, and payment source are 'primary indicators of unfairness' is a demographic-parity statement, not evidence of model-introduced bias. This is fixable, but it is load-bearing.\n\nMinor issues: no confidence intervals for the fairness metrics, and some subgroups are tiny (Alaska Native n=638, Asian/Pacific Islander n=502), so the extreme ratios are noisy. The LOS thresholds come from prior self-cited work and a partner provider; no sensitivity analysis. The mitigation section discusses techniques but does not test any; calling it 'we provide bias mitigation strategies' overstates.\n\nWho is this for? Readers wanting a template for a standard fairness audit on public SUD data, and people who want concrete subgroup-level parity numbers. The paper deserves a serious referee. I would send it to peer review with a request for major revision: add a baseline comparison to actual long-stay rates, fix the selection-rate definition, and add error bars or at least flag small cell sizes.","headline":"Competent, reproducible fairness audit of SUD length-of-stay predictions, but the headline claim of unfairness rests on demographic-parity ratios that are never benchmarked against actual long-stay rates.","tokens_in":28915,"tokens_out":3322,"would_cite":false,"duration_ms":32939,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that machine learning models predicting substance use treatment length of stay systematically under-select Alaska Native, self-pay, and cannabis-abuse patients for longer stays, framing these disparities as unfairness…","keywords":["algorithmic fairness","substance use disorder","length of stay","demographic parity","health disparities","machine learning","TEDS-D","policy mitigation"],"falsifier":"Compare the models' predicted long-stay rates against actual clinically indicated stays and post-discharge outcomes (e.g., readmission within 90 days) for each subgroup. If under-selected groups show no higher readmission or relapse, and if adjusting for measured clinical need removes the demographic parity gaps, the claim that these disparities constitute unfairness would fail.","tokens_in":28008,"feed_emoji":"⚖️","tokens_out":5034,"duration_ms":43165,"temperature":0.7,"pith_summary":"Using 2019 national U.S. treatment discharge records, the paper builds machine learning classifiers that predict whether a substance use disorder patient will have a long stay (more than 30 days inpatient or more than 90 days outpatient). It then audits these models for fairness and claims that race, U.S. region, primary substance, diagnosis, and payment source are the main sources of unfairness. The paper argues that groups such as Alaska Native patients, patients paying out of pocket or without insurance, and patients with cannabis abuse diagnoses are systematically under-predicted for longer stays, which could lead to premature discharge and worse recovery. If correct, this would mean that deploying such models without mitigation could encode existing health disparities into discharge decisions.","feed_headline":"ML predicts shorter stays for Alaska Native and self-pay SUD patients","feed_subtitle":"A 2019 national dataset shows wide parity gaps across race, region, diagnosis, and payment, prompting mitigation and policy fixes.","key_machinery":"The machinery is the binary length-of-stay classification task (long stay defined as over 30 days inpatient or over 90 days outpatient) paired with group and subgroup fairness metrics computed with FairLearn: selection rate, demographic parity ratio (with the 80 percent disparate impact threshold), equalized odds difference, and false positive/negative rate differences. Feature selection uses a majority vote across LASSO, decision tree, random forest, and XGBoost to choose 28 variables; the fairness audit then compares subgroup selection rates and identifies those below 80 percent of the highest rate as disadvantaged.","core_discovery":"The central discovery is that two well-performing classifiers (random forest and LightGBM) trained on the 2019 TEDS-D data produce large and systematic differences in the rate at which they predict long-term stays across demographic, medical, and financial groups. For inpatient race, the demographic parity ratio falls to 0.19 (random forest), and subgroup selection rates span from 0.17 for Alaska Native patients to 0.83 for Asian/Pacific Islander patients. The authors interpret these disparities as unfairness under distributive justice and socio-relational fairness, arguing they would deny adequate treatment duration to vulnerable groups.","pith_inferences":["A natural next test is to check whether under-selected groups actually have higher readmission or relapse rates when discharged earlier; if not, the parity gap may reflect clinically appropriate shorter stays rather than bias.","The paper's reliance on demographic parity alone leaves open the possibility that adjusting for clinical need (diagnosis, severity, co-occurring conditions) would shrink or eliminate the disparities; such a risk-adjusted audit is a testable extension.","The regional disparities suggest that a single national model may be less fair than regionally tuned models; evaluating fairness after fine-tuning on local data is a concrete follow-up."],"forward_implications":["If deployed without mitigation, these models would systematically recommend shorter stays for already-marginalized groups, potentially increasing relapse risk.","The identified variables (race, region, substance, diagnosis, payment) should be monitored in any length-of-stay prediction model used for substance use disorder discharge decisions.","The paper's group-level and subgroup-level audit procedure can be adapted to other treatment settings and other resource-allocation decisions.","Mitigation strategies, from data reweighting to threshold adjustment, are proposed to bring selection-rate parity within the 80 percent guideline."],"supporting_citations":[{"why":"Demonstrated racial bias in a hospital resource-management algorithm, motivating the need for fairness audits in health care ML.","marker":"Obermeyer et al., 2019"},{"why":"Formalized equality of opportunity and equalized odds, which the paper uses as fairness metrics.","marker":"Hardt et al., 2016"},{"why":"Supplied the disparate impact / 80% rule framework applied to identify disadvantaged subgroups.","marker":"Barocas & Selbst, 2016"},{"why":"Defined the distributive justice approaches (equal outcomes, equal performance, equal allocation) that structure the fairness evaluation.","marker":"Rajkomar, 2018"},{"why":"Provided Fairlearn, the toolkit used to compute the fairness metrics.","marker":"Bird et al., 2020"},{"why":"Clarified nuances among fairness metrics, guiding the paper's choice of selection rate, demographic parity, equalized odds, and rate differences.","marker":"Castelnovo et al., 2022"}],"fun_headline_variants":["Bias in SUD length-of-stay predictions shortchanges Alaska Native patients","ML models show racial bias in substance use treatment stays","Study: predictive models underrate stays for Alaska Native patients","Fairness gaps in AI-driven substance use discharge timing","Alaska Native patients get shorter predicted SUD stays in ML model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that selection-rate parity is the correct fairness standard for length-of-stay predictions, so that a group whose predicted long-stay rate is below 80 percent of the highest group's rate is being unfairly treated.","fun_headline_variants_meta":{"raw":{"variants":["Bias in SUD length-of-stay predictions shortchanges Alaska Native patients","ML models show racial bias in substance use treatment stays","Study: predictive models underrate stays for Alaska Native patients","Fairness gaps in AI-driven substance use discharge timing","Alaska Native patients get shorter predicted SUD stays in ML model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000502,"raw_usage":{"total_tokens":2449,"prompt_tokens":938,"completion_tokens":1511,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":1426}},"tokens_in":554,"tokens_out":1511,"duration_ms":9354,"temperature":1.0,"reasoning_tokens":1426,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:17:45.046322+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the models' predicted long-stay rates against actual clinically indicated stays and post-discharge outcomes (e.g., readmission within 90 days) for each subgroup. If under-selected groups show no higher readmission or relapse, and if adjusting for measured clinical need removes the demographic parity gaps, the claim that these disparities constitute unfairness would fail.","supporting_citations":[{"cited_title":", Powers, B","cited_arxiv_id":null,"evidence_quote":"Demonstrated racial bias in a hospital resource-management algorithm, motivating the need for fairness audits in health care ML."},{"cited_title":"APACrefauthors \\ 2018","cited_arxiv_id":null,"evidence_quote":"Defined the distributive justice approaches (equal outcomes, equal performance, equal allocation) that structure the fairness evaluation."}],"review_version":1}