Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Human-AI Co-design for Clinical Prediction Models

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an iterative human-in-the-loop framework, HACHI, lets clinicians steer a language-model agent to find yes/no concepts in notes that yield interpretable models outperforming standard scores and one-shot brainstorming.

desk verdict A well-written, plausible framework for human-LLM co-design of interpretable clinical prediction models, with strong AKI evidence but a circular TBI evaluation that undercuts the headline generalizability claim. read the letter →

arxiv 2601.09072 v1 pith:CAYSQSS5 submitted 2026-01-14 cs.AI cs.CLstat.ME

Jean Feng , Avni Kothari , Patrick Vossler , Andrew Bishara , Lucas Zier , Newton Addo , Aaron Kornblith , Yan Shuo Tan
show 1 more author
Chandan Singh
This is my paper · ORCID
classification cs.AIcs.CLstat.ME
keywords clinicalpredictionmodelsconceptbottlenecklargelanguagehuman-in-the-loopelectronichealthrecordsnotesinterpretabilitytraumaticbraininjury
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an iterative collaboration between clinicians and a large-language-model agent can turn unstructured clinical notes into small, transparent prediction models that outperform standard instruments. The workflow, called HACHI, repeatedly has the agent propose candidate concepts as yes/no questions, annotate notes, and fit a linear model, while clinicians review results each round and refine prompts, data, and objectives. In two retrospective tasks—pediatric head trauma and postoperative kidney injury—the loop raised accuracy, surfaced predictors absent from common models, and improved performance across sites and time periods at a cost of roughly 1-2 hours of expert time per round. A sympathetic reader would care because this is a concrete path from black-box LLM automation to interpretable bedside tools.

What carries the argument

The central mechanism is a concept bottleneck defined by yes/no questions: each concept is a single human-readable question, and the model is a linear combination of those binary answers. The AI agent runs a greedy hill-climbing loop—extract keyphrases from notes, fit a bag-of-words logistic model to find outcome-associated phrases, ask the LLM to propose candidate yes/no questions, have the LLM annotate every note for each candidate, evaluate validation AUC, and replace weak concepts—while the human team steers the next round by editing the prompts, the dataset, the coefficient-sign constraint, or the sample weights. This division of labor is what carries the argument: the agent explores an

What would settle it

Take one learned concept such as 'normal gait' or 'sleep apnea,' have two clinicians independently annotate a random sample of 100-200 notes, compare their answers with the LLM's, and check whether disagreements are associated with the outcome; low agreement or outcome-correlated errors would indicate the reported AUC gains are artifacts of annotation bias rather than true predictive signal.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that human oversight in the loop changes what the AI agent learns, not just how well it performs: the clinicians caught a leaked 'brain bleed' concept that inflated the first model, removed transferred patients with prior CT results, forced concept wording to refer to patient attributes rather than note style, required coefficient signs to match clinical priors, and reweighted sites to fix a 3:1 imbalance. The final TBI model used five yes/no questions (loss of consciousness, altered mental status, headache, head trauma, normal gait) and improved cross-campus AUCs from 0.93/0.71 to 0.93/0.80 while outperforming the standard rule at matched s

Load-bearing premise

The load-bearing premise is that the language model's yes/no answers about the notes are accurate enough to serve as the ground-truth labels used to fit and choose concepts; if those annotations are noisy or biased, the learned concepts and reported AUCs are not trustworthy.

Editorial extensions

If this is right

  • If HACHI works as reported, a small clinical team with EHR notes and a compliant LLM can produce a locally valid, fully interpretable prediction model in about 3-4 rounds and 1-2 expert hours per round.
  • The learned concepts—e.g., normal gait in pediatric head trauma, or minimally invasive surgery and sleep apnea in AKI—become candidate predictors for prospective validation in larger, multi-site cohorts.
  • At inference, a HACHI model requires only a clinician answering 5-10 yes/no questions during an encounter; no LLM is needed at the bedside, which lowers deployment barriers.
  • Because the learning loop exposes the human team to the agent's intermediate annotations and errors, it also exposes data leakage, documentation-style artifacts, and site imbalances that pure automated feature discovery would miss.
  • HACHI's procedure is not tied to its defaults: swapping the lasso model for sparse integer scoring or rule lists is a stated extension, which would preserve interpretability while potentially improving calibration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a natural ablation would separate the value of the human feedback loop from the value of the agent's search—if clinicians only approve or reject the final concept list rather than steering prompts, performance should drop measurably if the human input is doing the real work.
  • Editorial inference: the design treats the LLM's yes/no annotations as ground truth, so chart-reviewing a subsample of annotations and checking whether disagreements correlate with the outcome would clarify how much of the reported AUC advantage is genuine signal versus annotation bias.
  • Editorial inference: the same outer/inner loop should transfer to other high-stakes domains with unstructured text, such as eligibility decisions, discharge planning, or social-work notes, where a small expert team could steer concept discovery in the same way.
  • Editorial inference: the claimed human cost of 1-2 hours per round excludes dataset preparation, team meetings, and code updates, so a fuller time accounting is needed before comparing HACHI's efficiency to traditional model-building pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces HACHI, an iterative human-in-the-loop framework in which an LLM-based AI agent proposes, annotates, and selects yes/no clinical concepts from unstructured notes, while a clinical AI team reviews each round and provides feedback through prompt edits, dataset modifications, or changes to the learning procedure. The framework is evaluated on two retrospective UCSF case studies: pediatric traumatic brain injury (TBI) and postoperative acute kidney injury (AKI). The authors report that HACHI outperforms standard instruments (PECARN, Kheterpal) and a one-shot OpenEvidence-based brainstorming baseline, and that it improves generalizability across sites (TBI) and across time periods (AKI). The paper includes an open-source implementation and a web-based audit interface.

Significance. If the evaluation concerns are resolved, HACHI is a meaningful contribution to interpretable clinical prediction modeling: it operationalizes the iterative human-expert/LLM co-design loop, makes the learned concepts inspectable, and publishes reusable code. The AKI arm provides a genuine temporally disjoint test set (Period 2), which is a real strength, and the paper is transparent about many limitations, including the retrospective design and dependence on LLM extraction accuracy. However, the load-bearing claim of 'improved generalizability across clinical sites' rests almost entirely on the TBI arm, which lacks a true held-out test set; the site-level results are validation-set numbers obtained after iterative feedback and reweighting on the same data. The absence of a direct comparison to BC-LLM, the closest AI-only baseline, also leaves the specific benefit of human feedback unquantified. These issues are fixable and the core framework is plausible, but the current evidence is not sufficient to support the abstract's strong claims.

major comments (3)
  1. [Sec. 2.1, Fig. 1b, Table 1] The TBI arm has no independent test set. The four HACHI rounds are evaluated on the same validation partition used for concept selection, and Round 4's campus reweighting was introduced after observing the Round 3 campus AUC gap (0.93 vs 0.71). The reported final AUCs (0.91 overall; 0.93 and 0.80 by campus) are therefore validation-set numbers, not held-out estimates. This makes the abstract's claim that HACHI 'improves model generalizability across clinical sites' unsupported for the site dimension. The AKI arm's temporally disjoint Period 2 set is the right template; the TBI arm should be re-analyzed with a true holdout (e.g., temporal split or nested validation) or the claim must be explicitly downgraded to validation-only, exploratory evidence.
  2. [Sec. 2.1, Sec. 2.2, Sec. 3] The closest baseline, BC-LLM [28], is described as the most related method but is never empirically compared. HACHI's central contribution is the human-in-the-loop component over an AI-only agentic loop. Without a BC-LLM comparison or an ablation with human feedback removed, the observed improvements cannot be attributed to the co-design process rather than to the underlying LLM-driven greedy concept search. Please add this comparison or narrow the claim to 'HACHI, including its human feedback, outperforms...'.
  3. [Sec. 4.1 Step 3, Discussion limitations] The framework treats LLM yes/no concept annotations as ground-truth features for fitting and selecting the CPMs, but annotation accuracy is not measured on either dataset. The Discussion acknowledges this dependency, but a concrete reliability check is needed: for example, clinician audit of a random sample of concept annotations with agreement statistics, or a sensitivity analysis using a different extraction model. Without this, both the reported AUCs and the claim that HACHI 'surfaces new clinically relevant concepts' remain vulnerable to annotation noise or bias.
minor comments (5)
  1. [Sec. 4.1, Step 4] The text says 'Repeat steps 2-4 until convergence'; the loop should say 'repeat steps 2-3' since step 4 is the iteration itself.
  2. [Introduction, reference [1]] Typo: 'SOF A' should be 'SOFA'.
  3. [Fig. 1b] The figure uses 'Hospital 1' and 'Hospital 2' while the text and Table 1 refer to Oakland and Mission Bay campuses. Use consistent labels for readability.
  4. [Table A2] Several confidence intervals appear implausible (e.g., sensitivity 0.942 with CI (0.942, 1.000); specificity lower bounds of 0.000). Recalculate or omit the CIs if they are derived from a small bootstrap.
  5. [Sec. 4.1, Implementation details] The statement 'In ablation studies, we did not find significant improvement...' is not accompanied by any results or supplement reference. Provide the ablation or cite the supplement.

Circularity Check

1 steps flagged · score 6.0 of 10

TBI site-generalizability claim is circular: sample weights were chosen from validation-set campus AUCs, and the same validation AUCs are then reported as evidence of improved generalizability.

  1. fitted input called prediction [Section 2.1 (TBI Case Study), Round 3→4; Table 1; Fig. 1b]
    "the team assessed the generalizability of the model across the two Oakland and Mission Bay campuses. This revealed a surprising gap in performance: the AUCs at the Oakland and Mission Bay campuses were 0.93 and 0.71, respectively. ... So forRound 4, the team decided to update the learning procedure to allow for sample weights, so that the two campuses could be equally weighted. Round 4’s CPM not only achieved a better overall AUC of 0.91, but also better campus-specific AUCs of 0.93 and 0.80 at Oakland and Mission Bay, respectively."

    The sample-weight modification was introduced in direct response to campus-specific AUCs computed on the validation partition, and the final AUCs (0.91 overall; 0.93/0.80 by campus) are then reported on the same partition as evidence that HACHI 'improves model generalizability across clinical sites.' No separate held-out test set is described for the TBI arm, so the reweighting is a fitted input and the 'predicted' site generalizability is measured on the same data used to fit it. The reported improvement is therefore at least partly forced by the selection procedure rather than independently confirmed. The AKI arm's temporally disjoint Period 2 test set provides independent content, which is why the circularity is only partial.

full rationale

The paper contains no definitional derivation or load-bearing self-citation: the AI-guided procedure is adapted from the authors' BC-LLM work, but that citation supports methodology rather than a uniqueness claim, and the core contribution is empirical. The main circularity risk is in evaluation. In the TBI arm, all rounds use the same train/validation split, and human feedback decisions—removing cases with prior CT results, constraining concept signs, and introducing campus sample weights—were motivated by observed validation results. The final campus-specific AUCs are then reported on that same validation set as evidence of 'generalizability across clinical sites,' which is effectively fitting to the evaluation data and calling the result a prediction. The AKI arm, by contrast, includes a genuinely temporally disjoint Period 2 test set and external comparators (Kheterpal, OpenEvidence), so the temporal-generalization claim has independent content. The PECARN/OpenEvidence comparisons in TBI are also evaluated on the same validation set used to tune HACHI, further biasing the comparison. Overall, this is partial circularity concentrated in the TBI site-generalizability claim, not a fully circular derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; its load-bearing assumptions are domain-level (LLM extraction accuracy, label validity, validation-set AUC as selection criterion) plus standard statistical model choices. The free parameters listed are human/model-selection choices that directly shape the reported CPMs.

free parameters (4)
  • number of concepts k = 5 (TBI), 10 (AKI)
    Set by the clinical team to match PECARN/Kheterpal model sizes; directly determines model complexity and the space of concepts searched (Sec. 2.1-2.2).
  • number of inner-loop iterations = 10
    Used for all experiments; the paper says an ablation found no significant gain with more iterations but reports no details (Sec. 4.1).
  • TBI Round 4 campus reweighting = equal weights for Oakland and Mission Bay
    Introduced after observing site-specific AUCs of 0.93 vs 0.71; this post-hoc adjustment is directly responsible for the reported generalizability gain (Sec. 2.1).
  • TBI Round 3 sign constraint on coefficients = coefficient sign must match LLM's clinical prior
    Added to enforce clinical intuition; changes the concept-selection search and the final model (Sec. 2.1 Round 3).
assumptions (5)
  • domain assumption LLM annotations of yes/no concepts from clinical notes are sufficiently accurate to act as features for model fitting and selection.
    The entire concept-evaluation step uses LLM annotations as ground truth; the paper relies on refs 21/65 rather than measuring extraction accuracy on these data (Sec. 3 limitations, Sec. 4.1 Step 3).
  • domain assumption Outcome labels (ICD-10 TBI diagnosis; KDIGO creatinine-based AKI) are correct and clinically meaningful.
    Cases/controls are defined by administrative codes and lab criteria; label noise or coding artifacts would bias AUC estimates (Sec. 4.3).
  • domain assumption Validation-set AUC is a valid criterion for choosing between concepts and rounds.
    AUC on the validation split is used to greedily select concepts and to decide round-to-round changes; for TBI, the same split is used for final reporting, making the assumption load-bearing (Sec. 4.1, Sec. 2.1).
  • standard math Lasso-penalized logistic regression on binary concepts is an appropriate model class.
    Used throughout as the statistical workhorse; a standard model choice, not a novel claim (Sec. 4.1).
  • standard math Bag-of-words keyphrase associations identify candidate concepts worth exploring.
    The proposal step uses ridge/lasso on keyphrase features; a heuristic feature-selection assumption (Sec. 4.1 Steps 1-2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Human-AI Co-design for Clinical Prediction Models." pith.science (2026). https://pith.science/paper/CAYSQSS5

@misc{pith2026260109072,
  author       = {Pith},
  title        = {Pith review of: Human-AI Co-design for Clinical Prediction Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CAYSQSS5}},
  note         = {Machine review of arXiv:2601.09072}
}
read the original abstract

Developing safe, effective, and practically useful clinical prediction models (CPMs) traditionally requires iterative collaboration between clinical experts, data scientists, and informaticists. This process refines the often small but critical details of the model building process, such as which features/patients to include and how clinical categories should be defined. However, this traditional collaboration process is extremely time- and resource-intensive, resulting in only a small fraction of CPMs reaching clinical practice. This challenge intensifies when teams attempt to incorporate unstructured clinical notes, which can contain an enormous number of concepts. To address this challenge, we introduce HACHI, an iterative human-in-the-loop framework that uses AI agents to accelerate the development of fully interpretable CPMs by enabling the exploration of concepts in clinical notes. HACHI alternates between (i) an AI agent rapidly exploring and evaluating candidate concepts in clinical notes and (ii) clinical and domain experts providing feedback to improve the CPM learning process. HACHI defines concepts as simple yes-no questions that are used in linear models, allowing the clinical AI team to transparently review, refine, and validate the CPM learned in each round. In two real-world prediction tasks (acute kidney injury and traumatic brain injury), HACHI outperforms existing approaches, surfaces new clinically relevant concepts not included in commonly-used CPMs, and improves model generalizability across clinical sites and time periods. Furthermore, HACHI reveals the critical role of the clinical AI team, such as directing the AI agent to explore concepts that it had not previously considered, adjusting the granularity of concepts it considers, changing the objective function to better align with the clinical objectives, and identifying issues of data bias and leakage.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Data-Efficient Adaptation of LLMs via Attention Head Reweighting

    cs.LG 2026-07 conditional novelty 5.0 of 10

    Learning a single scalar per attention head lets LLMs adapt to few-shot text classification better than LoRA, with 200–1000x fewer trainable parameters.

Reference graph

Works this paper leans on

70 extracted references · 7 linked inside Pith · cited by 1 Pith paper

  1. [28]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025)

    Feng, J., Kothari, A., Zier, L., Singh, C., Tan, Y.S.: Bayesian concept bottleneck models with LLM priors. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2025). https://openreview.net/forum?id=oXSkzIXgbk

    2025

  2. [1]

    JAMA 334(23), 2090–2103 (2025)

    Ranzani, O.T., Singer, M., Salluh, J.I.F., Shankar-Hari, M., Pilcher, D., Berger-Estilita, J., Coop- ersmith, C.M., Juffermans, N.P., Laffey, J., Reinikainen, M., Neto, A.S., Tavares, M., Timsit, J.-F., Arias Lopez, M.D.P., Arulkumaran, N., Aryal, D., Azoulay, E., Celi, L.A., Chaudhuri, D., De Lange, D., De Waele, J., Dos Santos, C.C., Du, B., Einav, S., ...

  3. [2]

    The Lancet 374(9696), 1160–1170 (2009)

    Kuppermann, N., Holmes, J.F., Dayan, P.S., Hoyle, J.D., Atabaki, S.M., Holubkov, R., Nadel, F.M., Monroe, D., Stanley, R.M., Borgialli, D.A.,et al.: Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. The Lancet 374(9696), 1160–1170 (2009)

    2009

  4. [3]

    a practical scale

    Teasdale, G., Jennett, B.: Assessment of coma and impaired consciousness. a practical scale. Lancet2(7872), 81–84 (1974)

    1974

  5. [4]

    a review and suggested modifications of methodological standards

    Laupacis, A., Sekar, N., Stiell, I.G.: Clinical prediction rules. a review and suggested modifications of methodological standards. JAMA277(6), 488–494 (1997)

    1997

  6. [5]

    Nature Machine Intelligence1(5), 206–215 (2019)

    Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence1(5), 206–215 (2019)

    2019

  7. [6]

    ICMLabs/2007.04612(2020)

    Koh, P.W., Nguyen, T., Tang, Y.S., Mussmann, S., Pierson, E., Kim, B., Liang, P.: Concept bottleneck models. ICMLabs/2007.04612(2020)

    arXiv 2007

  8. [7]

    Wells, P.S., Anderson, D.R., Rodger, M., Ginsberg, J.S., Kearon, C., Gent, M., Turpie, A.G., Bormanis, J., Weitz, J., Chamberlain, M., Bowie, D., Barnes, D., Hirsh, J.: Derivation of a simple 14 clinical model to categorize patients probability of pulmonary embolism: increasing the models utility with the SimpliRED D-dimer. Thromb. Haemost.83(3), 416–420 (2000)

    2000

Show all 70 references
  1. [8]

    Chest149(3), 631–638 (2016)

    Chung, F., Abdullah, H.R., Liao, P.: STOP-bang questionnaire: A practical approach to screen for obstructive sleep apnea. Chest149(3), 631–638 (2016)

    2016

  2. [9]

    In: StatPearls

    McLendon, K., Goyal, A., Attia, M.: Deep venous thrombosis risk factors. In: StatPearls. StatPearls Publishing, Treasure Island (FL) (2025)

    2025

  3. [10]

    In: StatPearls

    Jain, S., Margetis, K., Iverson, L.M.: Glasgow coma scale. In: StatPearls. StatPearls Publishing, Treasure Island (FL) (2025)

    2025

  4. [11]

    BMJ353, 2416 (2016)

    Damen, J.A.A.G., Hooft, L., Schuit, E., Debray, T.P.A., Collins, G.S., Tzoulaki, I., Lassale, C.M., Siontis, G.C.M., Chiocchia, V., Roberts, C., Schl¨ ussel, M.M., Gerry, S., Black, J.A., Heus, P., Schouw, Y.T., Peelen, L.M., Moons, K.G.M.: Prediction models for cardiovascular...

    2016

  5. [12]

    BMJ369, 1328 (2020)

    Wynants, L., Van Calster, B., Collins, G.S., Riley, R.D., Heinze, G., Schuit, E., Bonten, M.M.J., Dahly, D.L., Damen, J.A.A., Debray, T.P.A., Jong, V.M.T., De Vos, M., Dhiman, P., Haller, M.C., Harhay, M.O., Henckaerts, L., Heus, P., Kammer, M., Kreuzberger, N., Lohmann, A., L...

    2020

  6. [13]

    JAMA Netw

    Feng, Y., Wang, A.Y., Jun, M., Pu, L., Weisbord, S.D., Bellomo, R., Hong, D., Gallagher, M.: Characterization of risk prediction models for acute kidney injury: A systematic review and meta-analysis. JAMA Netw. Open6(5), 2313359 (2023)

    2023

  7. [14]

    Os, H.J.A., Kanning, J.P., Wermer, M.J.H., Chavannes, N.H., Numans, M.E., Ruigrok, Y.M., Zwet, E.W., Putter, H., Steyerberg, E.W., Groenwold, R.H.H.: Developing clinical prediction models using primary care electronic health record data: The impact of data preparation choices ...

    2022

  8. [15]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Ramaswamy, V.V., Kim, S.S.Y., Fong, R., Russakovsky, O.: Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10932–10941 (2023)

    2023

  9. [16]

    NPJ Digit

    Obra, J.K., Singh, C., Watkins, K., Feng, J., Obermeyer, Z., Kornblith, A.: Potential for algorithmic bias in clinical decision instrument development. NPJ Digit. Med., 1–7 (2025)

    2025

  10. [17]

    World Neurosurg.134, 311–322 (2020)

    Mattei, T.A., Teasdale, G.M.: The story of the development and adoption of the glasgow coma scale: Part I, the early years. World Neurosurg.134, 311–322 (2020)

    2020

  11. [18]

    Nature (2023)

    Jiang, L.Y., Liu, X.C., Nejatian, N.P., Nasir-Moin, M., Wang, D., Abidin, A., Eaton, K., Riina, H.A., Laufer, I., Punjabi, P., Miceli, M., Kim, N.C., Orillac, C., Schnurman, Z., Livia, C., Weiss, H., Kurland, D., Neifert, S., Dastagirzada, Y., Kondziolka, D., Cheung, A.T.M., Y...

    2023

  12. [19]

    Seinen, T.M., Fridgeirsson, E.A., Ioannou, S., Jeannetot, D., John, L.H., Kors, J.A., Markus, 15 A.F., Pera, V., Rekkas, A., Williams, R.D., Yang, C., Mulligen, E.M., Rijnbeek, P.R.: Use of unstructured text in prognostic clinical prediction models: a systematic review. J. Am....

    2022

  13. [20]

    Seinen, T.M., Kors, J.A., Mulligen, E.M., Rijnbeek, P.R.: Using structured codes and free- text notes to measure information complementarity in electronic health records: Feasibility and validation study. J. Med. Internet Res.27(1), 66910 (2025)

    2025

  14. [21]

    Empirical Methods in Natural Language Processing, 1998–2022 (2022)

    Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., Sontag, D.: Large language models are few-shot clinical information extractors. Empirical Methods in Natural Language Processing, 1998–2022 (2022)

    1998

  15. [22]

    NPJ Digit

    Yang, X., Chen, A., PourNejatian, N., Shin, H.C., Smith, K.E., Parisien, C., Compas, C., Martin, C., Costa, A.B., Flores, M.G., Zhang, Y., Magoc, T., Harle, C.A., Lipori, G., Mitchell, D.A., Hogan, W.R., Shenkman, E.A., Bian, J., Wu, Y.: A large language model for electronic h...

    2022

  16. [23]

    npj Digital Medicine7(1), 1–14 (2024)

    Guevara, M., Chen, S., Thomas, S., Chaunzwa, T.L., Franco, I., Kann, B.H., Moningi, S., Qian, J.M., Goldstein, M., Harper, S., Aerts, H.J.W.L., Catalano, P.J., Savova, G.K., Mak, R.H., Bit- terman, D.S.: Large language models to identify social determinants of health in electr...

    2024

  17. [24]

    International Conference on Learning Representations (2023) [cs.LG]

    Oikarinen, T., Das, S., Nguyen, L.M., Weng, T.-W.: Label-free concept bottleneck models. International Conference on Learning Representations (2023) [cs.LG]

    2023

  18. [25]

    In: The 2023 Conference on Empirical Methods in Natural Language Processing (2023)

    McInerney, D.J., Young, G., Meent, J.-W., Wallace, B.C.: CHiLL: Zero-shot custom interpretable feature extraction from clinical notes with large language models. In: The 2023 Conference on Empirical Methods in Natural Language Processing (2023)

    2023

  19. [26]

    Advances in Neural Information Processing Systems37, 124137–124162 (2025)

    Benara, V., Singh, C., Morris, J., Antonello, R., Stoica, I., Huth, A., Gao, J.: Crafting inter- pretable embeddings for language neuroscience by asking llms questions. Advances in Neural Information Processing Systems37, 124137–124162 (2025)

    2025

  20. [27]

    In: European Conference on Computer Vision, pp

    Kim, J., Wang, Z., Qiu, Q.: Constructing concept-based models to mitigate spurious correlations with minimal human effort. In: European Conference on Computer Vision, pp. 137–153 (2024). Springer

    2024

  21. [29]

    arXiv preprint arXiv:2310.19660 (2023)

    Ludan, J.M., Lyu, Q., Yang, Y., Dugan, L., Yatskar, M., Callison-Burch, C.: Interpretable- by-design text classification with iteratively generated concept bottleneck. arXiv preprint arXiv:2310.19660 (2023)

    2023 arXiv

  22. [30]

    In: Proceedings of the CHI Conference on Human Factors in Computing Systems, vol

    Subramonyam, H., Pea, R., Pondoc, C., Agrawala, M., Seifert, C.: Bridging the gulf of envision- ing: Cognitive challenges in prompt based interactions with LLMs. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, vol. 31, pp. 1–19. ACM, New York, NY, ...

    2024

  23. [31]

    Kothari, A., Vossler, P., Digitale, J., Forouzannia, M., Rosenberg, E., Lee, M., Bryant, J., Molina, M., Marks, J., Zier, L., Feng, J.: When the domain expert has no time and the LLM developer 16 has no clinical expertise: Real-world lessons from LLM co-design in a safety-net ...

    2026

  24. [32]

    Langlois, J.A., Rutland-Brown, W., Thomas, K.E.: Traumatic brain injury in the united states: emergency department visits, hospitalizations, and deaths (2006)

    2006

  25. [33]

    Coronado, V.G., Xu, L., Basavaraju, S.V., McGuire, L.C., Wald, M.M., Faul, M.D., Guzman, B.R., Hemphill, J.D., Disease Control, C., (CDC), P., et al.: Surveillance for traumatic brain injury-related deaths: United states, 1997-2007 (2011)

    1997

  26. [34]

    Pediatric radiology32(4), 228–231 (2002)

    Brenner, D.J.: Estimating cancer risks from pediatric ct: going from the qualitative to the quantitative. Pediatric radiology32(4), 228–231 (2002)

    2002

  27. [35]

    New England journal of medicine357(22), 2277–2284 (2007)

    Brenner, D.J., Hall, E.J.: Computed tomography—an increasing source of radiation exposure. New England journal of medicine357(22), 2277–2284 (2007)

    2007

  28. [36]

    Annals of emergency medicine64(2), 145–152 (2014)

    Easter, J.S., Bakes, K., Dhaliwal, J., Miller, M., Caruso, E., Haukoos, J.S.: Comparison of pecarn, catch, and chalice rules for children with minor head injury: a prospective cohort study. Annals of emergency medicine64(2), 145–152 (2014)

    2014

  29. [37]

    The Lancet Child & Adolescent Health8(5), 339–347 (2024)

    Holmes, J.F., Yen, K., Ugalde, I.T., Ishimine, P., Chaudhari, P.P., Atigapramoj, N., Badawy, M., McCarten-Gibbs, K.A., Nielsen, D., Sage, A.C.,et al.: Pecarn prediction rules for ct imaging of children presenting to the emergency department with blunt abdominal or minor head t...

    2024

  30. [38]

    Yen, K., Kuppermann, N., Lillis, K., Monroe, D., Borgialli, D., Kerrey, B.T., Sokolove, P.E., Ellison, A.M., Cook, L.J., Holmes, J.F., Intra-abdominal Injury Study Group for the Pediatric Emergency Care Applied Research Network (PECARN): Interobserver agreement in the clinical...

    2013

  31. [39]

    Hurt, R.T., Stephenson, C.R., Gilman, E.A., Aakre, C.A., Croghan, I.T., Mundi, M.S., Ghosh, K., Edakkanambeth Varayil, J.: The use of an artificial intelligence platform OpenEvidence to augment clinical decision-making for primary care physicians. J. Prim. Care Community Healt...

    2025

  32. [40]

    Anesthesiology104(1), 65–72 (2006)

    Wijeysundera, D.N., Karkouti, K., Beattie, W.S., Rao, V., Ivanov, J.: Improving the identification of patients at risk of postoperative renal failure after cardiac surgery. Anesthesiology104(1), 65–72 (2006)

    2006

  33. [41]

    Journal of the American Society of Nephrology16(1), 162–168 (2005)

    Thakar, C.V., Arrigain, S., Worley, S., Yared, J.-P., Paganini, E.P.: A clinical score to predict acute renal failure after cardiac surgery. Journal of the American Society of Nephrology16(1), 162–168 (2005)

    2005

  34. [42]

    Circulation95(4), 878–884 (1997)

    Chertow, G.M., Lazarus, J.M., Christiansen, C.L., Cook, E.F., Hammermeister, K.E., Grover, F., Daley, J.: Preoperative renal risk stratification. Circulation95(4), 878–884 (1997)

    1997

  35. [43]

    Jr: Development and validation of an acute kidney injury risk index for patients undergoing general surgery: results from a national data set: Results from a national data set

    Kheterpal, S., Tremper, K.K., Heung, M., Rosenberg, A.L., Englesbe, M., Shanks, A.M., Camp- bell, D.A. Jr: Development and validation of an acute kidney injury risk index for patients undergoing general surgery: results from a national data set: Results from a national data se...

    2009

  36. [44]

    KDIGO clinical practice guideline for acute kidney injury

    Kellum, J.A., Lameire, N., Aspelin, P., Barsoum, R.S., Burdmann, E.A., Goldstein, S.L., Her- zog, C.A., Joannidis, M., Kribben, A., Levey, A.S., MacLeod, A.M., Mehta, R.L., Murray, P.T., Naicker, S., Opal, S.M., Schaefer, F., Schetz, M., Uchino, S.: Kidney disease: Improving g...

    2011

  37. [45]

    Lancet Digit Health2(9), 489–492 (2020)

    Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., Celi, L.A.: The myth of generalisability in clinical research and machine learning in health care. Lancet Digit Health2(9), 489–492 (2020)

    2020

  38. [46]

    Patel, A., Rao, D., Kothary, A., McKeown, K., Callison-Burch, C.: Learning interpretable style embeddings via prompting LLMs. Proc. Conf. Empir. Methods Nat. Lang. Process. (2023) [cs.CL]

    2023

  39. [47]

    In: The Thirteenth International Conference on Learning Representations (2025)

    Sun, Y., Huang, Q., Tang, Y., Tung, A.K.H., Yu, J.: A general framework for producing inter- pretable semantic text embeddings. In: The Thirteenth International Conference on Learning Representations (2025). https://openreview.net/forum?id=23uY3FpQxc

    2025

  40. [48]

    Nature625(7995), 468–475 (2024)

    Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M.P., Dupont, E., Ruiz, F.J., Ellenberg, J.S., Wang, P., Fawzi, O.,et al.: Mathematical discoveries from program search with large language models. Nature625(7995), 468–475 (2024)

    2024

  41. [49]

    arXiv preprint arXiv:2506.13131 (2025)

    Novikov, A., V˜ u, N., Eisenberger, M., Dupont, E., Huang, P.-S., Wagner, A.Z., Shirobokov, S., Kozlovskii, B., Ruiz, F.J., Mehrabian, A., et al.: Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131 (2025)

    2025 arXiv

  42. [50]

    arXiv preprint arXiv:2210.01848 (2022)

    Singh, C., Morris, J.X., Aneja, J., Rush, A.M., Gao, J.: Explaining patterns in data with language models via interpretable autoprompting. arXiv preprint arXiv:2210.01848 (2022)

    2022 arXiv

  43. [51]

    Simon and Schuster, ??? (2021)

    Monarch, R.M.: Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-centered AI. Simon and Schuster, ??? (2021)

    2021

  44. [52]

    Artificial Intelligence Review56(4), 3005–3054 (2023)

    Mosqueira-Rey, E., Hern´ andez-Pereira, E., Alonso-R ´ ıos, D., Bobes-Bascar´ an, J., Fern´ andez-Leal, ´A.: Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review56(4), 3005–3054 (2023)

    2023

  45. [53]

    arXiv preprint arXiv:2012.02898 (2020)

    Lage, I., Doshi-Velez, F.: Learning interpretable concept-based models with human feedback. arXiv preprint arXiv:2012.02898 (2020)

    2012 arXiv

  46. [54]

    NPJ digital medicine8(1), 629 (2025)

    Brewster, R.C., Tse, G., Fan, A.L., Elborki, M., Newell, M., Gonzalez, P., Hoq, A., Chang, C., Chowdhury, M., Geeti, A.,et al.: Evaluating human-in-the-loop strategies for artificial intelligence-enabled translation of patient discharge instructions: a multidisciplinary analys...

    2025

  47. [55]

    In: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pp

    Sivaraman, V., Vaishampayan, A., Li, X., Buck, B.R., Ma, Z., Boyce, R.D., Perer, A.: Tempo: Helping data scientists and domain experts collaboratively specify predictive modeling tasks. In: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pp. 1–18....

    2025

  48. [56]

    In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pp

    Gao, J., Gebreegziabher, S.A., Choo, K.T.W., Li, T.J.-J., Perrault, S.T., Malone, T.W.: A tax- onomy for human-llm interaction modes: An initial exploration. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2024)

    2024

  49. [57]

    In: Forty-second International Conference on Machine Learning

    Wu, S., Galley, M., Peng, B., Cheng, H., Li, G., Dou, Y., Cai, W., Zou, J., Leskovec, J., Gao, 18 J.: Collabllm: From passive responders to active collaborators. In: Forty-second International Conference on Machine Learning

  50. [58]

    In: Proceedings of the Aaai Conference on Artificial Intelligence, vol

    Chauhan, K., Tiwari, R., Freyberg, J., Shenoy, P., Dvijotham, K.: Interactive concept bottleneck models. In: Proceedings of the Aaai Conference on Artificial Intelligence, vol. 37, pp. 5948–5955 (2023)

    2023

  51. [59]

    Machine Learning102(3), 349–391 (2016)

    Ustun, B., Rudin, C.: Supersparse linear integer models for optimized medical scoring systems. Machine Learning102(3), 349–391 (2016)

    2016

  52. [60]

    In: International Conference on Machine Learning, pp

    Yang, H., Rudin, C., Seltzer, M.: Scalable bayesian rule lists. In: International Conference on Machine Learning, pp. 3921–3930 (2017). PMLR

    2017

  53. [61]

    arXiv preprint arXiv:2509.26433 (2025)

    Grari, V., Arni, T., Laugel, T., Lamprier, S., Zou, J., Detyniecki, M.: Act: Agentic classification tree. arXiv preprint arXiv:2509.26433 (2025)

    2025 arXiv

  54. [62]

    In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp

    Singh, C., Morris, J., Rush, A.M., Gao, J., Deng, Y.: Tree prompting: Efficient task adaptation without fine-tuning. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 6253–6267 (2023)

    2023

  55. [63]

    arXiv preprint arXiv:2410.06352 (2024)

    Ragkousis, A., Parbhoo, S.: Tree-based leakage inspection and control in concept bottleneck models. arXiv preprint arXiv:2410.06352 (2024)

    2024 arXiv

  56. [64]

    Journal of Open Source Software6(61), 3192 (2021) https://doi.org/10.21105/ joss.03192

    Singh, C., Nasseri, K., Tan, Y.S., Tang, T., Yu, B.: imodels: a python package for fitting inter- pretable models. Journal of Open Source Software6(61), 3192 (2021) https://doi.org/10.21105/ joss.03192

    2021

  57. [65]

    Scientific reports15(1), 6179 (2025)

    Kornblith, A.E., Singh, C., Innes, J.C., Chang, T.P., Adelgais, K.M., Holsti, M., Kim, J., McClain, B., Nishijima, D.K., Rodgers, S.,et al.: Analyzing patient perspectives with large language mod- els: a cross-sectional study of sentiment and thematic classification on excepti...

    2025

  58. [66]

    In: Mechanistic Interpretability Workshop at NeurIPS 2025

    Shi, T., Yan, G., Oikarinen, T., Weng, T.-W.: Multimodal concept bottleneck models. In: Mechanistic Interpretability Workshop at NeurIPS 2025

    2025

  59. [67]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Pang, W., Ke, X., Tsutsui, S., Wen, B.: Integrating clinical knowledge into concept bottle- neck models. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 243–253 (2024). Springer 19 A Extended Data T able A1:Complete list of conce...

    2024

  60. [68]

    The primary descriptor (exact or paraphrased from the note)

  61. [69]

    Synonyms or alternative phrasings

  62. [70]

    protective factor

    Broader generalizations that capture similar concepts Requirements: - Each descriptor should be $\leq$3 words - The descriptor should state distinguishing features. - List specific terms, synonyms, and increasingly general concepts. However, do not state overly general concept...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.