Pith. sign in

REVIEW 3 major objections 6 minor 39 references

Can human clinical rationales improve the performance and explainability of clinical text classification models?

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Human clinical rationales improve transformer classifiers less than adding the same number of full pathology reports, and sufficiency-based preselection is unreliable.

desk verdict A well-controlled empirical negative result on rationales for clinical text classification, but the headline recommendation is overgeneralized because the rationale sample is biased toward hard cases. read the letter →

arxiv 2507.21302 v1 pith:YRC2567Z submitted 2025-07-28 cs.CL

Christoph Metzner , Shang Gao , Drahomira Herrmannova , Heidi A. Hanson This is my paper
classification cs.CL
keywords clinicaltextclassificationhumanrationalesextractivetransformermodelsexplainabilitysufficiencymetricprimarycancersiteelectronicpathologyreports
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether human-created clinical rationales—short highlighted spans that explain why a pathology report supports a particular primary cancer site diagnosis—are worth using as extra training data for transformer-based clinical text classifiers. It compares models trained on about 96,700 rationale snippets against models trained on the same number of additional full pathology reports, holding all other training data fixed. The central finding is that rationales yield smaller performance gains than full reports: adding rationales improved accuracy by up to about 0.5 percentage points, while adding the same number of full reports improved accuracy by about 0.8 to 1.8 points, and rationale-supplemented models never beat the report-supplemented control. Rationales did improve the overlap between a model's attention and human-highlighted tokens, but adding full reports gave a comparable overlap at the token level. The paper therefore concludes that if accuracy is the goal, annotation effort should go to labeling new reports, and rationales should be treated mainly as a modest explainability aid.

What carries the argument

The central object is the extractive human rationale: a highlighted span, averaging about 13.6 words and covering roughly 4 percent of the report text, that an oncology specialist selected as justifying the primary-site diagnosis; each rationale is treated as an independent training sample alongside full reports. The load-bearing comparison is equal-sized supplementation: 96,679 added rationales versus 96,679 added full reports with matched class distributions and no overlapping tumor cases. The main scoring mechanism for rationale quality is sufficiency, $\mathit{Suff}(x,\hat{y},\alpha)=1-\max(0,p(\hat{y}|x)-p(\hat{y}|x,\alpha))$, with binary token mask $\alpha$, used to pre-filter rationales at thresholds 0.2 and 0.817, alongside comprehensiveness, $\mathit{Comp}(x,\hat{y},\alpha)=\max(0,p(\hat{y}|x)-p(\hat{y}|x,1-\alpha))$, used to explain why rationales underperform as training input. Explainability is measured as average token-level rationale coverage, i.e., the fraction of high-attention tokens (above the 90th, 95th, or 98th percentile of attention scores) that overlap human rationale tokens, computed for all models on 2,446 rationale-annotated test reports. All experiments use a Clinical Longformer encoder, either in its baseline form or extended with a deformable phrase-level attention mechanism that supplies word- and phrase-level attention scores for the coverage analysis.

What would settle it

A reader could settle the central claim by taking a random sample of all autocoded reports, including those that passed quality control, having specialists write rationales for the same diagnosis, and repeating the controlled comparison of the same number of added rationales versus the same number of added full reports; if the rationale-augmented model matches or exceeds the additional-report model in accuracy or macro-F1, the paper's central conclusion fails. A second check would be finding any sufficiency threshold that consistently improves both accuracy and F1 across many classes and both model architectures, which would refute the claim that sufficiency preselection is unreliable.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a comparative negative result: with a fixed annotation budget, a full-report label carries more training value than a rationale label. Using 96,679 extractive rationales as independent training samples moved the baseline Clinical Longformer's accuracy from 0.885 to 0.888 and the DPLA-extended model from 0.900 to 0.904, while adding the same number of full reports moved accuracy to 0.903 and 0.908 respectively; macro-F1 followed the same ordering, and the confidence intervals show the report supplement as the only consistently significant gain over the reports-only baseline. In low-resource simulations, adding 100 to 1,000 rationales for ten common classes frequently decreased overall accuracy and produced erratic class-wise changes with no consistent benefit from sufficiency-filtered rationales. The explainability analysis, computed as average token-level rationale coverage from attention scores on 2,446 test reports, shows that rationales help a model attend to rationale-like tokens only when combined with full reports, and that additional full reports achieve similar token-level coverage. The paper's stated conclusion is that when the goal is optimizing accuracy, annotation resources should be spent on labeling more reports rather than creating rationales, with rationales relegated to a secondary explainability benefit.

Load-bearing premise

The load-bearing premise is that the rationale collection process fairly represents what rationales would deliver in practice; because rationales were created only for reports that failed automated quality-control or data-origin checks, the rationale set is a biased sample, and a random-sample collection could change the comparison.

Editorial extensions

If this is right

  • If the goal is maximum accuracy for a fixed annotation budget, new full-report labels dominate rationale extraction: in the paper's largest comparison, 96,679 rationales raised baseline accuracy from 0.885 to 0.888, while 96,679 extra full reports raised it to 0.903.
  • Rationale supplementation is not a reliable low-resource strategy: adding 100 to 1,000 rationales for frequent classes often lowered overall accuracy, and class-wise F1 changed inconsistently across classes and models.
  • The sufficiency metric should not be used as a general rationale-quality filter, because thresholds that helped one model or class hurt another, so pre-filtering rationales by sufficiency does not give a dependable performance gain.
  • If explainability is measured as attention overlap with human rationales, training on reports plus rationales can improve plausibility, but similar token-level overlap can be obtained by adding more full reports, so rationales are not necessary for plausibility.
  • For deployment in automated medical coding, the practical recommendation follows directly: spend scarce annotation resources on labeling new, unlabeled pathology reports, and treat rationales as an optional byproduct when they cost nothing extra.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's acknowledged sampling bias cuts both ways: rationales came only from reports that failed automated quality-control or data-origin checks, so the rationale pool overrepresents difficult or atypical cases, and a random-sample rationale collection could shrink, widen, or even reverse the gap against the additional-report control.
  • An equal-sample-count comparison is not an equal-cost comparison; if rationales are produced as a free byproduct of quality control, even a small performance or explainability gain may justify keeping them, and the real policy question is marginal cost per accuracy point.
  • Because sufficiency is computed from model prediction probabilities, it is entangled with class frequency and model confidence; a cleaner test of rationale quality would remove the rationale tokens and measure the change in prediction independent of confidence, or use inter-annotator agreement on whether the highlighted span justifies the diagnosis.
  • Because the average rationale covers only about 4 percent of report text, the low-resource performance drops may reflect information starvation rather than low rationale quality; a follow-up that supplements rationales with surrounding context would separate these two explanations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper asks whether human-created clinical rationales (extractive highlights from electronic pathology reports) can improve the performance and explainability of transformer-based multiclass classifiers that assign primary cancer site codes. Using 99,125 rationales drawn from 89,718 SEER reports plus 128,649 full reports, the authors compare, for two Clinical Longformer variants, training on reports only, reports supplemented with all rationales, reports supplemented with rationale subsets filtered by two sufficiency thresholds, and reports supplemented with an equal number of additional full reports as a control. They also test low-resource class-wise settings, evaluate sufficiency/comprehensiveness as faithfulness metrics, and measure attention overlap with rationales as an explainability proxy. The headline findings are that rationales provide small and inconsistent gains, that additional reports consistently outperform rationales, that sufficiency-based preselection does not reliably help, and that rationales improve explainability only slightly relative to additional reports. The authors recommend prioritizing the annotation of new reports over the creation of rationales.

Significance. If the central comparison is valid, the paper makes a practically important contribution: it suggests that annotation budgets for clinical text classification should be spent on labeling new documents rather than on collecting rationale annotations, and it quantifies the magnitude of the trade-off on a large, real-world dataset. The study has notable strengths: the two training-regimen families are evaluated on the same held-out test set; the additional-report control is matched in size and class distribution to the rationale set; 95% confidence intervals are provided for the headline metrics in Tables 2 and B.2; and the explainability analysis aggregates over multiple random seeds. The paper also honestly reports a major data-collection limitation in Section 6. However, that limitation is precisely the main threat to the central claim: because rationales were only collected for reports that failed QC or drew specialist disagreement, the comparison against a randomly selected report control may reflect case difficulty rather than the value of rationales as an annotation type. The sufficiency-threshold analysis is also affected by post hoc threshold selection.

major comments (3)
  1. [§3.1.2, §6] The rationale training set is not representative of the general population of reports: rationales were created only for reports that failed the initial quality control and a registry-specific data origin check, and in practice when a specialist disagreed with the API's top-ranked diagnosis. The additional-reports control was randomly selected and matched only on class distribution. The headline result that additional reports outperform rationales is therefore confounded by case difficulty: the rationale set is enriched for atypical or hard reports, while the control is in-distribution. Section 6 acknowledges this as a limitation, but it is not a peripheral issue—it directly affects the central claim and the practical recommendation that annotation effort should be redirected from rationales to new reports. Please either add a matched control consisting of reports that also failed QC (or are otherwise matched on predicted difficulty), or substantially qualify the conclusion.
  2. [§3.4, §4.1, Table 2] Sufficiency thresholds are selected from the same data used for evaluation. The 0.817 threshold is defined as the 90th percentile of rationales leading to false predictions in preliminary experiments with the same model family, and the 0.2 threshold appears to be chosen after evaluating a range of thresholds. Using these thresholds to argue that sufficiency is 'unreliable' for preselection is therefore to some extent circular: the threshold has been tuned to the false-prediction distribution of the same model class. The conclusion in §5.1 that sufficiency is not an ideal preselection metric would be stronger if thresholds were fixed in advance or derived on a held-out set, and the sensitivity of the conclusions to threshold choice should be reported.
  3. [§4.2, Table 3] Table 3 reports class-wise F1-scores for m ∈ {100, 500, 1000} added rationales without confidence intervals or significance tests. The narrative in §5.1 that the low-resource regime shows 'inconsistent' and 'decreased' performance rests substantially on these point estimates (e.g., kidney C64 dropping to 0.432 with 1000 rationales but remaining around 0.54 with 500 and 0.52 with 100). Given the small absolute differences and the absence of uncertainty quantification, the apparent inconsistency may be within sampling noise. Please provide confidence intervals, or at least state explicitly that Table 3 is exploratory, and adjust the low-resource conclusions accordingly.
minor comments (6)
  1. [§4.1, Table 2] The asterisks denoting 'significant performance improvements' are not tied to a described significance test. Specify the procedure (e.g., bootstrap or normal-approximation z-test) or replace 'significant' with a CI-based description such as 'non-overlapping confidence intervals'.
  2. [§3.1.2, Table A.4] Section 3.1.2 reports 89,718 reports with 99,125 rationales, while Table A.4 lists the population of reports with rationales as 172,120. Clarify whether the larger number includes histology-related rationales or another earlier subset, and reconcile the discrepancy.
  3. [Table A.4] The header contains a typo: 'Pancrease' should be 'Pancreas'.
  4. [§2, §3.1.2] There are minor typographical errors throughout, including 'thesufficiency metric' in Section 2 and 'succesfully' in Section 3.1.2; a careful proofreading pass is recommended.
  5. [§5.2, Figure 1] The claim that reports supplemented with rationales achieve only 'slightly better' explainability than reports supplemented with additional reports is based on visual inspection of error bars. Report a statistical comparison of the coverage ratios across training regimens, or phrase the claim more cautiously.
  6. [§5.1] The phrase 'we agree with [8] observation' should read 'we agree with the observation of [8]'.

Circularity Check

1 steps flagged · score 2.0 of 10

Core comparison is empirical and not circular; minor self-referentiality in the sufficiency-based rationale preselection, where the 0.817 threshold is derived from the same model family's false predictions.

  1. fitted input called prediction [Section 3.4 (Evaluation of Rationales), applied in Tables 2 and B.2 and interpreted in Section 5.1]
    "We experimented with a sufficiency threshold of 0.2, which removes the least sufficient rationales, and a threshold of 0.817 represents the 90% percentile of rationales with false predictions for falsely predicted reports; removing roughly 88% of the rationales that led to false predictions in preliminary experiments."

    The 0.817 threshold is fitted to false predictions produced by the same CLF model family in preliminary runs, and the sufficiency scores themselves are computed from prediction probabilities of the same experimental models, as stated in Section 5.1: 'We computed the sufficiency and comprehensiveness scores by using the prediction probabilities generated from the experiments that compared model training on rationales, full reports, and complements.' The paper then uses this model-fitted threshold to filter training rationales and concludes that sufficiency-based preselection 'does not consistently improve performance.' The evaluation is therefore partly self-referential: the quality filter is calibrated on the very model family whose later performance is used to validate the filter, so…

full rationale

The central claim—that models trained on human clinical rationales are consistently outperformed by models trained on additional full reports—is an empirical comparison, not a derivation. It rests on held-out test accuracy and macro-F1 comparisons across training regimens, so it is not equivalent to its inputs by construction. The additional-reports control is matched only on class distribution and is confounded by the acknowledged selection bias in how rationales were collected (Section 6: rationales were created for reports that 'failed the initial quality control and a registry-specific data origin check'), but confounding is an external-validity threat, not circularity. The self-citations to the authors' prior CLF-DPLA architecture and SEER data-processing work are not load-bearing in a circular way: the DPLA architecture is described in Appendix A.2, and the comparison is run internally on both architectures. The only genuine circularity signal is in the sufficiency preselection experiments: the 0.817 threshold is extracted from the same model family's false predictions and sufficiency is defined via model prediction probabilities, making the subsequent evaluation of sufficiency-based filtering partly self-referential. This is a minor methodological loop in a secondary analysis and does not compromise the paper's main empirical finding, so the overall circularity score is low.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The study is empirical, so the ledger is mostly domain assumptions about data quality and measurement. No invented entities. Two sufficiency thresholds are data-derived free parameters; the maximum rationale length is a hand-chosen preprocessing value. The main comparison itself does not depend on fitted constants, which is why the circularity burden is low.

free parameters (3)
  • Sufficiency threshold 0.2 = 0.2
    Chosen in Section 3.4 as the threshold that achieved the best performance improvements across tested thresholds; this is data-derived from the same task, making the 'sufficient rationales' experiment partially post hoc.
  • Sufficiency threshold 0.817 = 0.817
    Defined in Section 3.4 as the 90th percentile of rationales with false predictions from preliminary experiments; this is a data-derived threshold rather than a pre-registered value.
  • Maximum rationale length (128 word tokens) = 128 tokens
    Set as mean plus two standard deviations and used to drop long rationales; a hand-chosen preprocessing value that could affect the composition of the rationale training set.
assumptions (5)
  • domain assumption ICD-O-3 primary cancer site labels assigned by certified tumor registrars are correct ground truth.
    The paper uses these labels as targets for training and evaluation without external verification; label noise would affect all comparisons, but there is no reason to expect differential effects across regimens.
  • domain assumption Human rationales are plausible explanations for the primary cancer site diagnosis.
    Section 3.4 states that because rationales were generated by subject matter experts, they are assumed highly plausible; this underpins the use of token-level overlap with rationales as an explainability measure.
  • domain assumption Attention scores are a valid measure of token importance for explainability.
    Section 4.3 uses attention score percentiles to compute rationale coverage; the paper acknowledges the controversy around attention as explanation (citation 36). If attention is not a faithful importance measure, the explainability conclusion is weakened.
  • ad hoc to paper Training on rationales as independent samples is a valid way to integrate rationale supervision.
    The authors chose this approach after preliminary experiments (Appendix B.1) showed token-level supervision and attention masking performed worse; this is a modeling choice specific to this study, and the conclusion about rationales may not generalize to other integration methods, as the paper itself notes.
  • standard math Normal approximation for 95% confidence intervals on accuracy and F1-macro.
    Used in Equation A.2-A.3; valid for large test sets but not stated whether it accounts for class imbalance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can human clinical rationales improve the performance and explainability of clinical text classification models?." pith.science (2026). https://pith.science/paper/YRC2567Z

@misc{pith2026250721302,
  author       = {Pith},
  title        = {Pith review of: Can human clinical rationales improve the performance and explainability of clinical text classification models?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YRC2567Z}},
  note         = {Machine review of arXiv:2507.21302}
}
read the original abstract

AI-driven clinical text classification is vital for explainable automated retrieval of population-level health information. This work investigates whether human-based clinical rationales can serve as additional supervision to improve both performance and explainability of transformer-based models that automatically encode clinical documents. We analyzed 99,125 human-based clinical rationales that provide plausible explanations for primary cancer site diagnoses, using them as additional training samples alongside 128,649 electronic pathology reports to evaluate transformer-based models for extracting primary cancer sites. We also investigated sufficiency as a way to measure rationale quality for pre-selecting rationales. Our results showed that clinical rationales as additional training data can improve model performance in high-resource scenarios but produce inconsistent behavior when resources are limited. Using sufficiency as an automatic metric to preselect rationales also leads to inconsistent results. Importantly, models trained on rationales were consistently outperformed by models trained on additional reports instead. This suggests that clinical rationales don't consistently improve model performance and are outperformed by simply using more reports. Therefore, if the goal is optimizing accuracy, annotation efforts should focus on labeling more reports rather than creating rationales. However, if explainability is the priority, training models on rationale-supplemented data may help them better identify rationale-like features. We conclude that using clinical rationales as additional training data results in smaller performance improvements and only slightly better explainability (measured as average token-level rationale coverage) compared to training on additional reports.

Figures

Figures reproduced from arXiv: 2507.21302 by the authors.

Figure 1
Figure 1. Average token-level rationale coverage across different attention score percentiles for DPLA’s word- and [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Evaluation of rationale faithfulness decomposed into sufficiency and comprehensiveness for the CLF-BS and [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 33 canonical work pages

  1. [1]

    Ai-driven clinical decision support systems: an ongoing pursuit of potential

    Malek Elhaddad and Sara Hamam. Ai-driven clinical decision support systems: an ongoing pursuit of potential. Cureus, 16(4), 2024

    work page 2024

  2. [2]

    Teach me to explain: A review of datasets for explainable natural language processing

    Sarah Wiegreffe and Ana Marasovi´c. Teach me to explain: A review of datasets for explainable natural language processing. arXiv preprint arXiv:2102.12060, 2021

    arXiv 2021

  3. [3]

    Evaluating and Characterizing Human Rationales

    Samuel Carton, Anirudh Rathore, and Chenhao Tan. Evaluating and characterizing human rationales. arXiv preprint arXiv:2010.04736, 2020

    work page Pith review arXiv 2010

  4. [4]

    Knife: Distilling meta- reasoning knowledge with free-text rationales

    Aaron Chan, Zhiyuan Zeng, Wyatt Lake, Brihi Joshi, Hanjie Chen, and Xiang Ren. Knife: Distilling meta- reasoning knowledge with free-text rationales. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML, 2023

    work page 2023

  5. [5]

    Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning

    Yue Yu, Jiaming Shen, Tianqi Liu, Zhen Qin, Jing Nathan Yan, Jialu Liu, Chao Zhang, and Michael Bender- sky. Explanation-aware soft ensemble empowers large language model in-context learning. arXiv preprint arXiv:2311.07099, 2023

    work page Pith review arXiv 2023

  6. [6]

    Large language models are clinical reasoners: Reasoning- aware diagnosis framework with prompt-generated rationales

    Taeyoon Kwon, Kai Tzu-iunn Ong, Dongjin Kang, Seungjun Moon, Jeong Ryong Lee, Dosik Hwang, Beomseok Sohn, Yongsik Sim, Dongha Lee, and Jinyoung Yeo. Large language models are clinical reasoners: Reasoning- aware diagnosis framework with prompt-generated rationales. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18417–18...

    work page 2024

  7. [7]

    Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine

    Thomas Savage, Ashwin Nayak, Robert Gallo, Ekanath Rangan, and Jonathan H Chen. Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. NPJ Digital Medicine, 7(1):20, 2024

    work page 2024

  8. [8]

    annotator rationales

    Omar Zaidan, Jason Eisner, and Christine Piatko. Using “annotator rationales” to improve machine learning for text categorization. In Human language technologies 2007: The conference of the North American chapter of the association for computational linguistics; proceedings of the main conference, pages 260–267, 2007

    work page 2007

Show all 39 references
  1. [9]

    Rationale-augmented convolutional neural networks for text clas- sification

    Ye Zhang, Iain Marshall, and Byron C Wallace. Rationale-augmented convolutional neural networks for text clas- sification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volu...

    2016

  2. [10]

    Deriving machine attention from human rationales

    Yujia Bao, Shiyu Chang, Mo Yu, and Regina Barzilay. Deriving machine attention from human rationales. arXiv preprint arXiv:1808.09367, 2018

    2018 arXiv

  3. [11]

    Do human rationales improve machine explanations? arXiv preprint arXiv:1905.13714, 2019

    Julia Strout, Ye Zhang, and Raymond J Mooney. Do human rationales improve machine explanations? arXiv preprint arXiv:1905.13714, 2019

    1905 arXiv

  4. [12]

    Fine-grained sentiment analysis with faithful attention

    Ruiqi Zhong, Steven Shao, and Kathleen McKeown. Fine-grained sentiment analysis with faithful attention. arXiv preprint arXiv:1908.06870, 2019

    1908 arXiv

  5. [13]

    Marta: Leveraging human rationales for explainable text classification

    Ines Arous, Ljiljana Dolamic, Jie Yang, Akansha Bhardwaj, Giuseppe Cuccu, and Philippe Cudré-Mauroux. Marta: Leveraging human rationales for explainable text classification. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 5868–5876, 2021

    2021

  6. [14]

    Human-like explanation for text classification with limited attention supervision

    Dongyu Zhang, Cansu Sen, Jidapa Thadajarassiri, Thomas Hartvigsen, Xiangnan Kong, and Elke Rundensteiner. Human-like explanation for text classification with limited attention supervision. In 2021 ieee international conference on big data (big data), pages 957–967. IEEE, 2021

    2021

  7. [15]

    Learning with rationales for document classification

    Manali Sharma and Mustafa Bilgic. Learning with rationales for document classification. Machine Learning, 107:797–824, 2018

    2018

  8. [16]

    Rationale production to support clinical decision-making

    Niall Taylor, Lei Sha, Dan W Joyce, Thomas Lukasiewicz, Alejo Nevado-Holgado, and Andrey Kormilitzin. Rationale production to support clinical decision-making. arXiv preprint arXiv:2111.07611, 2021

    2021 arXiv

  9. [17]

    Explain and predict, and then predict again

    Zijian Zhang, Koustav Rudra, and Avishek Anand. Explain and predict, and then predict again. In Proceedings of the 14th ACM international conference on web search and data mining, pages 418–426, 2021

    2021

  10. [18]

    Evaluating explanations: How much do explanations from the teacher aid students? Transactions of the Association for Computational Linguistics, 10:359–375, 2022

    Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C Lipton, Graham Neubig, and William W Cohen. Evaluating explanations: How much do explanations from the teacher aid students? Transactions of the Association for Computational Linguis...

    2022

  11. [19]

    Rationalization for explainable nlp: a survey

    Sai Gurrapu, Ajay Kulkarni, Lifu Huang, Ismini Lourentzou, and Feras A Batarseh. Rationalization for explainable nlp: a survey. Frontiers in Artificial Intelligence, 6:1225093, 2023

    2023

  12. [20]

    Exploring the trade-off between model performance and explanation plausibility of text classifiers using human rationales

    Lucas E Resck, Marcos M Raimundo, and Jorge Poco. Exploring the trade-off between model performance and explanation plausibility of text classifiers using human rationales. arXiv preprint arXiv:2404.03098, 2024. 12

    2024 arXiv

  13. [21]

    Contrastive explanations for model interpretability

    Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel, Yanai Elazar, Yejin Choi, and Yoav Goldberg. Contrastive explanations for model interpretability. arXiv preprint arXiv:2103.01378, 2021

    2021 arXiv

  14. [22]

    https://seer.cancer.gov/data-software/

    National Cancer Institute seer. https://seer.cancer.gov/data-software/. Accessed: 2022-07-05

    2022

  15. [23]

    Eraser: A benchmark to evaluate rationalized nlp models

    Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. Eraser: A benchmark to evaluate rationalized nlp models. arXiv preprint arXiv:1911.03429, 2019

    1911 arXiv

  16. [24]

    Machine learning and deep learning tools for the automated capture of cancer surveillance data

    Elizabeth Hsu, Heidi Hanson, Linda Coyle, Jennifer Stevens, Georgia Tourassi, and Lynne Penberthy. Machine learning and deep learning tools for the automated capture of cancer surveillance data. JNCI Monographs, 2024(65):145–151, 2024

    2024

  17. [25]

    SEER*DMS User Manual: Chapter 14 - Annotation Tasks, 2020

    Surveillance, Epidemiology, and End Results (SEER) Program. SEER*DMS User Manual: Chapter 14 - Annotation Tasks, 2020. Accessed: 2024-08-14

    2020

  18. [26]

    A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports

    Madhumita Sushil, Travis Zack, Divneet Mandair, Zhiwei Zheng, Ahmed Wali, Yan-Ning Yu, Yuwei Quan, Dmytro Lituiev, and Atul J Butte. A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology rep...

    2024

  19. [27]

    Clinical-longformer and clinical- bigbird: Transformers for long clinical sequences

    Yikuan Li, Ramsey M Wehbe, Faraz S Ahmad, Hanyin Wang, and Yuan Luo. Clinical-longformer and clinical- bigbird: Transformers for long clinical sequences. arXiv preprint arXiv:2201.11838, 2022

    2022 arXiv

  20. [28]

    Attention mechanisms in clinical text classification: A comparative evaluation

    Christoph S Metzner, Shang Gao, Drahomira Herrmannova, Elia Lima-Walton, and Heidi A Hanson. Attention mechanisms in clinical text classification: A comparative evaluation. IEEE Journal of Biomedical and Health Informatics, 2024

    2024

  21. [29]

    Deformable phrase level attention: A flexible approach for improving ai based medical coding

    Christoph Simon Metzner, Shang Gao, Drahomira Herrmannova, John Gounley, and Heidi Hanson. Deformable phrase level attention: A flexible approach for improving ai based medical coding. Available at SSRN 4864687, 2024

    2024

  22. [30]

    Text classification algorithms: A survey

    Kamran Kowsari, Kiana Jafari Meimandi, Mojtaba Heidarysafa, Sanjana Mendu, Laura Barnes, and Donald Brown. Text classification algorithms: A survey. Information, 10(4):150, 2019

    2019

  23. [31]

    Limitations of transformers on clinical text classification

    Shang Gao, Mohammed Alawad, M Todd Young, John Gounley, Noah Schaefferkoetter, Hong Jun Yoon, Xiao- Cheng Wu, Eric B Durbin, Jennifer Doherty, Antoinette Stroup, et al. Limitations of transformers on clinical text classification. IEEE journal of biomedical and health informati...

    2021

  24. [32]

    Deep learning–based text classification: a comprehensive review

    Shervin Minaee, Nal Kalchbrenner, Erik Cambria, Narjes Nikzad, Meysam Chenaghlu, and Jianfeng Gao. Deep learning–based text classification: a comprehensive review. ACM computing surveys (CSUR), 54(3):1–40, 2021

    2021

  25. [33]

    Automatic classification of cancer pathology reports: a systematic review

    Thiago Santos, Amara Tariq, Judy Wawira Gichoya, Hari Trivedi, and Imon Banerjee. Automatic classification of cancer pathology reports: a systematic review. Journal of Pathology Informatics, 13:100003, 2022

    2022

  26. [34]

    Model evaluation, model selection, and algorithm selection in machine learning

    Sebastian Raschka. Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808, 2018

    2018 arXiv

  27. [35]

    Rethinking cooperative rationalization: Introspective extraction and complement control

    Mo Yu, Shiyu Chang, Yang Zhang, and Tommi S Jaakkola. Rethinking cooperative rationalization: Introspective extraction and complement control. arXiv preprint arXiv:1910.13294, 2019

    1910 arXiv

  28. [36]

    Is attention explanation? an introduction to the debate

    Adrien Bibal, Rémi Cardon, David Alfter, Rodrigo Wilkens, Xiaoou Wang, Thomas François, and Patrick Watrin. Is attention explanation? an introduction to the debate. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2022

  29. [37]

    Competency problems: On finding and removing artifacts in language data

    Matt Gardner, William Merrill, Jesse Dodge, Matthew E Peters, Alexis Ross, Sameer Singh, and Noah A Smith. Competency problems: On finding and removing artifacts in language data. arXiv preprint arXiv:2104.08646, 2021

    2021 arXiv

  30. [38]

    Towards faithful model explanation in nlp: A survey

    Qing Lyu, Marianna Apidianaki, and Chris Callison-Burch. Towards faithful model explanation in nlp: A survey. Computational Linguistics, pages 1–67, 2024

    2024

  31. [39]

    What to learn, and how: Toward effective learning from rationales

    Samuel Carton, Surya Kanoria, and Chenhao Tan. What to learn, and how: Toward effective learning from rationales. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1075–1088, 2022. 13 A Materials and Methods A.1 Data This work considers human-based ...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.