Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Learn from A Rationalist: Distilling Intermediate Interpretable Rationales

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes REKD, a distillation method that lets a small rationale-extraction model learn both the feature selections and the predictions of a larger teacher, and reports that this substantially improves student accuracy under ident

desk verdict A plausible distillation recipe for rationale extraction with real gains, but the headline numbers rest on a single above-average teacher per dataset; referee it, but ask for teacher-seed robustness and significance tests. read the letter →

arxiv 2601.22531 v2 pith:F7G3ZNGA submitted 2026-01-30 cs.LG cs.AI

Jiayi Dai , Randy Goebel This is my paper
classification cs.LGcs.AI
keywords rationaleextractionknowledgedistillationinterpretable-by-designselect-predictarchitectureGumbel-SoftmaxtemperatureannealingvisiontransformerBERT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rationale extraction is a way to make neural networks interpretable by design: a generator network picks a small subset of input features and a predictor makes the decision using only those features. Training these two networks together is hard, and the paper argues the difficulty hurts smaller, less capable networks most, creating a chicken-and-egg loop: the selector needs the predictor's guidance to learn what matters, and the predictor needs good selections to learn its task. This paper proposes REKD, which adds a third signal: a larger teacher RE model contributes its own feature selections and prediction probabilities, so the student imitates a 'rationalist' while still exploring on its own. The reported numbers are concrete—on CIFAR-10 with a 15% feature budget, ViT-Small accuracy rises from 0.889 to 0.968 and ViT-Tiny from 0.797 to 0.936, with similar gains on CIFAR-100 and IMDB. This matters because small, deployable, faithful-by-design models could approach or exceed larger teachers instead of trading interpretability for accuracy.

What carries the argument

The central object is the binary feature mask (which input patches or tokens are kept) produced by the generator. REKD attaches two knowledge-distillation losses to it: a per-feature KL divergence between the teacher's and student's Gumbel-Softmax selection distributions, and a KL divergence between their temperature-scaled class predictions. The same exponential temperature schedule that anneals the Straight-Through Gumbel-Softmax estimator—a differentiable relaxation that lets binary selection be trained by backpropagation—also scales the distillation losses, turning training into a curriculum from soft teacher guidance to sharp selections.

What would settle it

Run REKD on CIFAR-10 with each of the ten seeds 2026–2035 as the ViT-Base teacher, keeping the student setup fixed; if the average ViT-Small REKD accuracy across teacher seeds is not above the 0.889 RE baseline, the reported gain partly reflects the chosen teacher seed rather than the method.

Watch

Extended reading notes

Core claim

REKD distills a large teacher rationale-extraction model into a small student RE model. The student must match the teacher's per-feature selection distribution and softened class probabilities while still optimizing its own select-predict objective. Because feature masks are architecture-independent, no projection layers are needed; the shared Gumbel-Softmax annealing temperature creates a soft-to-hard curriculum. With a 15% rationale budget on CIFAR-10, ViT-Small accuracy rises from 0.889 to 0.968 and ViT-Tiny from 0.797 to 0.936, with analogous gains on CIFAR-100 and IMDB. Pure imitation (α=0) is worse than mixed training, so exploration remains useful.

Load-bearing premise

The experiments assume one teacher run per dataset is representative—the paper's Training Setup fixes the teacher seed without stating how it was chosen, and its Conclusion notes all distillation is ViT→ViT or BERT→BERT—so an unrepresentative teacher draw or a cross-architecture failure would shrink the reported gains.

Editorial extensions

If this is right

  • Small rationale-extraction students can reach or exceed the average accuracy of a large teacher under the same sparse-feature budget, so faithful-by-design models need not forfeit accuracy.
  • Because the distillation target is the feature mask, teacher and student need no projection layers or matched hidden dimensions, which simplifies transfer across model sizes.
  • The student's own exploration is necessary: pure supervised imitation (α=0) underperforms, so teacher guidance should supplement rather than replace the select-predict loop.
  • The same shared-temperature distillation could extend to any discrete latent structure trained with Gumbel-Softmax, such as relational graph selection—the paper names this as a next step.
  • In compute-limited deployments where interpretability matters, a REKD student offers faithful explanations with a much smaller accuracy penalty than prior RE models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the method is as architecture-agnostic as the paper suggests, the same teacher rationales should transfer from a vision transformer to a ResNet-based RE student; this cross-architecture experiment is the paper's most direct untested extension.
  • The single-teacher-per-dataset protocol means part of the reported gap may be teacher luck; averaging student accuracy over many teacher seeds would tell how much of the 0.079/0.139 gain is method rather than draw.
  • The α=0 finding implies teacher rationales alone do not determine a full student policy; this predicts that adding controlled exploration noise to the distillation objective should help even with a fixed teacher—an experiment not run in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes REKD (Rationale Extraction with Knowledge Distillation), a training objective for rationale-extraction (RE) models that adds two distillation terms to the standard RE loss: a KL divergence between the teacher's and student's Gumbel-Softmax feature-selection distributions, and a temperature-scaled KL divergence between teacher and student prediction distributions. These are combined with the student's own RE loss via a scalar weight α, with the KD temperature tied to the Gumbel-Softmax annealing schedule. The method is evaluated on CIFAR-10, CIFAR-100, and IMDB using ViT and BERT backbones of varying sizes, comparing a student trained with REKD against the same student trained with standard RE under the same rationale-ratio budget. The headline results are large accuracy gains on CIFAR-10 (ViT-Small 0.889→0.968, ViT-Tiny 0.797→0.936), smaller but consistent gains on CIFAR-100 and IMDB, and lower student variance. The paper also discusses the 'chicken-and-egg' difficulty of joint rationale/predictor learning and argues that predictive performance under a rationale budget is a more objective evaluation metric than human-annotation alignment.

Significance. If the reported gains are robust, REKD is a simple and potentially useful contribution: it provides a concrete way to transfer interpretable feature-selection knowledge from a strong RE model to a weaker student, with an architecture-agnostic interface at the selection layer. The paper is generally clearly written, the method is well specified, and the experimental setup is reproducible in many respects: all seeds are listed, models and datasets are public, and the appendices give hyperparameter-tuning details. The repeated runs with standard deviations are a strength, and the reported variance reduction is a nice additional result. However, the central empirical claim — that REKD 'significantly improves' students — is currently conditioned on a single, possibly favorable, teacher run per dataset. The student-level standard deviations do not capture teacher variability, and no statistical significance tests are reported. The paper would be considerably stronger if the experiments were repeated over multiple teacher draws or if a clear pre-specified teacher-selection policy were provided. The architecture-agnostic claim is also stronger than what is tested, since all distillation is

major comments (3)
  1. The central comparison uses a single teacher run per dataset, but the paper does not state how the teacher seeds were selected. For CIFAR-10 the teacher is seed 2029 with accuracy 0.969, above the mean 0.964; for CIFAR-100 it is seed 2031 with accuracy 0.848, above the mean 0.830. For IMDB the teacher seed is not reported at all. If these teacher seeds were chosen after inspecting test accuracy, then the reported REKD gains are not the expected improvement over a randomly chosen teacher, and the student standard deviations (e.g., .006 for ViT-Small on CIFAR-10) do not bound the uncertainty in the headline claim. Please clarify the teacher-seed selection rule, and ideally rerun with multiple teacher draws (or provide teacher-aware confidence intervals) so the gains can be attributed to the method rather than to a favorable teacher.
  2. The abstract and Section 4 repeatedly use the word 'significantly' to describe the improvements (e.g., 'REKD significantly improves the predictive performance of the student RE models'). No statistical significance tests are reported, only means and standard deviations over 10 student seeds with a single teacher. Even setting aside the teacher-seed issue, paired or unpaired tests (e.g., bootstrap or permutation tests over the 10 student seeds) are needed to support the word 'significantly', and the tests should account for teacher variability if multiple teachers are used.
  3. The claim that the REKD student 'slightly outperforms the average of the teacher ViT Base models' (0.968 vs. 0.964) is misleading in context. The teacher actually used has accuracy 0.969, which is above the teacher average; the REKD student average 0.968 is below that specific teacher. Comparing against the average of all teacher runs is not an apples-to-apples comparison when the distillation signal comes from a particular above-average teacher. Please compare against the actual teacher used, or report results averaged over multiple teacher draws.
minor comments (5)
  1. There is a typo: 'bert-base-uncasesd' should be 'bert-base-uncased'. Also, the quoted model labels are inconsistent in formatting, and the label for 'prajjwal1/bert-small' appears to be missing its closing quotation mark.
  2. The α tuning tables are only for ViT models on CIFAR, but the text states that α=0.3 is then used for all models, including BERTs on IMDB. Please clarify whether α was tuned on IMDB as well, or whether it was transferred from the vision experiments. If transferred, this should be stated explicitly as a design choice, and its effect on the IMDB results should be discussed.
  3. In the IMDB description, the 5,000-sample split is called 'testing', but later the paper refers to a 'developing' set for model selection. Please clarify the terminology: is this split used as a validation set, with final evaluation on the original test set? The same applies to the CIFAR splits, where 'developing' is used.
  4. The α=0 case is described as 'KD without RE', but the objective still includes the rationale-distillation term that supervises the generator. Consider clarifying that α=0 means 'without the student's own RE task loss', not 'without any rationale supervision'.
  5. The abstract and contribution list call the method 'neural-model agnostic', but all experiments distill within the same architecture family (ViT→ViT, BERT→BERT). The conclusion acknowledges this limitation, but the earlier statements should be softened or supported by at least one cross-architecture experiment (e.g., ViT→ResNet or BERT→LSTM) to fully justify the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: REKD is an empirical distillation scheme; the central improvement claim is evaluated on held-out test data and does not reduce to its training objective or to a self-citation.

full rationale

The paper's derivation chain (Eqs. 1-11) defines RE and KD losses and combines them into L_REKD = α L_RE + (1-α) L_KD; no step defines a target result in terms of itself, and no fitted parameter is renamed as a prediction. The headline claim—that REKD improves student RE accuracy—is tested on held-out test sets (CIFAR 10/100, IMDB) against RE baselines, with hyperparameters α and λ_select chosen by development loss, which is standard model selection rather than circularity. Self-citations (Dai et al. 2022; Dai et al. 2025) provide background context or support secondary observations; they are not load-bearing for the central claim, and no uniqueness theorem or ansatz is imported from them. The Conclusion honestly limits the claim to same-architecture distillation; this is a scope limitation, not a circular step. The concern that single teacher seeds may have been selected after inspecting test accuracy is a validity/selection-bias issue, not a definitional circularity, and cannot be established from the manuscript text.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a handful of hyperparameters (α, λselect, ptarget, λR, τ0/τK) plus the assumption that teacher rationales transfer within architecture families and that the dev RE loss is a valid selector. The teacher-seed choice is the most fragile unmodeled factor.

free parameters (6)
  • ptarget (target rationale ratio) = 0.15 for ViT/CIFAR, 0.10 for BERT/IMDB; 0.095 for BERT-Small RE baseline
    Desired rationale ratio in the selection loss (Eq. 3); chosen by hand and varied in Figure 2.
  • λselect (selection-loss weight) = per-model values in Appendix C.1 (e.g., 1e-3–5e-2)
    Tuned separately for each base model to keep the rationale ratio near ptarget on the dev split.
  • α (RE/KD weight) = 0.3
    Chosen as the value with lowest dev RE loss in Appendix C.2, then applied to all models including BERTs.
  • λR (rationale-distillation weight) = 0.5
    Set as a constant hyperparameter for the generator-distillation term in Eq. (9).
  • τ0, τK (Gumbel temperatures) = 5, 0.1
    Initial and final temperatures in the annealing schedule (Eq. 10); chosen by hand.
  • Teacher seed choice = 2029 (CIFAR-10), 2031 (CIFAR-100), undisclosed (IMDB)
    One teacher seed per dataset is used for all student runs; the CIFAR seeds are above the teacher mean and the selection criterion is not stated.
assumptions (5)
  • standard math Straight-through Gumbel-Softmax yields usable gradient estimates for the discrete mask M (Eq. 1–2, Appendix B).
    The method relies on the ST estimator to train the generator through the argmax; this is a standard approximation from Jang et al. 2017.
  • domain assumption Teacher rationales are transferable knowledge across models of the same architecture family.
    REKD assumes that matching the teacher's Gumbel-Softmax distributions improves student selection; this core premise is tested only within BERT/ViT families.
  • domain assumption The dev-set RE loss is a valid model-selection criterion.
    Hyperparameters α and λselect and the early-stopped checkpoint are chosen by lowest LRE on a 20% dev split (Appendix C.1/C.2).
  • domain assumption CIFAR-100 coarse labels and upscaled 224x224 images preserve the relevant classification task.
    The paper uses 20 coarse classes instead of 100 and upscales 32x32 images; these changes could affect conclusions.
  • ad hoc to paper The chosen teacher seeds are representative of teacher performance.
    The paper uses one teacher seed per dataset without random selection or justification; for CIFAR these seeds are above the reported teacher mean.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learn from A Rationalist: Distilling Intermediate Interpretable Rationales." pith.science (2026). https://pith.science/paper/F7G3ZNGA

@misc{pith2026260122531,
  author       = {Pith},
  title        = {Pith review of: Learn from A Rationalist: Distilling Intermediate Interpretable Rationales},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F7G3ZNGA}},
  note         = {Machine review of arXiv:2601.22531}
}
read the original abstract

Because of the pervasive use of deep neural networks (DNNs), especially in high-stakes domains, the interpretability of DNNs has received increased attention. The general idea of rationale extraction (RE) is to provide an interpretable-by-design framework for DNNs via a select-predict architecture where two neural networks learn jointly to perform feature selection and prediction, respectively. Given only the remote supervision from the final task prediction, the process of learning to select subsets of features (or rationales) requires searching in the space of all possible feature combinations, which is computationally challenging and even harder when the base neural networks are not sufficiently capable. To improve the predictive performance of RE models that are based on less capable or smaller neural networks (i.e., the students), we propose REKD (Rationale Extraction with Knowledge Distillation) where a student RE model learns from the rationales and predictions of a teacher (i.e., a rationalist) in addition to the student's own RE optimization. This structural adjustment to RE aligns well with how humans could learn effectively from interpretable and verifiable knowledge. Because of the neural-model agnostic nature of the method, any black-box neural network could be integrated as a backbone model. To demonstrate the viability of REKD, we conduct experiments with multiple variants of BERT and vision transformer (ViT) models. Our experiments across language and vision classification datasets (i.e., IMDB movie reviews, CIFAR 10 and CIFAR 100) show that REKD significantly improves the predictive performance of the student RE models.

Figures

Figures reproduced from arXiv: 2601.22531 by the authors.

Figure 1
Figure 1. The schematic of REKD. The student RE model learns from the teacher’s supervision on feature selections and predictions in addition to its own RE exploration via Straight-Through Gumbel-Softmax. The red arrows indicate the gradient flows and the red 1 represents the Straight-Through estimation, i.e., ∂M ∂S ≈ 1. The definitions for the computation and the variables are in Section 2. Then, we define a selection loss t… view at source ↗
Figure 2
Figure 2. Rationale ratio vs. accuracy for ViT Base, Small and Tiny on CIFAR 10 by varying ptarget from 5% to 75%. The x-axis is the actual rationale ratio and the y-axis is the accuracy. The target rationale ratio ptarget is used as the label for each data point. The label “cls” represents the classification result. Each data point is an average of 10 runs. See [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of rationale extraction for ViT Base on CIFAR 10, CIFAR 100 and BERT Base on IMDB movie reviews (arranged from top to bottom). 14 [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 7 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    M., Confalonieri, R., Guidotti, R., Del Ser , J., Díaz-Rodríguez, N., and Herrera, F

    Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser , J., Díaz-Rodríguez, N., and Herrera, F. Explainable artificial intelligence (xai): What we know and what is left to attain trustworthy artificial intelligence. Information Fusion, 99: 0 101805, 2023. ISSN 1566-2535. doi:https://doi.org/10.101...

    arXiv 2023

  3. [3]

    Neural machine translation by jointly learning to align and translate, 2016

    Bahdanau, D., Cho, K., and Bengio, Y. Neural machine translation by jointly learning to align and translate, 2016. URL https://arxiv.org/abs/1409.0473

    arXiv 2016

  4. [4]

    Interpretable neural predictions with differentiable binary variables

    Bastings, J., Aziz, W., and Titov, I. Interpretable neural predictions with differentiable binary variables. In Korhonen, A., Traum, D., and M \`a rquez, L. (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.\ 2963--2977, Florence, Italy, July 2019. Association for Computational Linguistics. doi:10.18653/v1...

    doi:10.18653/v1/p19-1284 2019

  5. [5]

    Estimating or propagating gradients through stochastic neurons for conditional computation, 2013

    Bengio, Y., Léonard, N., and Courville, A. Estimating or propagating gradients through stochastic neurons for conditional computation, 2013. URL https://arxiv.org/abs/1308.3432

    arXiv 2013

  6. [6]

    Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, ...

    arXiv 2020

  7. [7]

    Ai in finance: Challenges, techniques, and opportunities

    Cao, L. Ai in finance: Challenges, techniques, and opportunities. ACM Comput. Surv., 55 0 (3), February 2022. ISSN 0360-0300. doi:10.1145/3502289. URL https://doi.org/10.1145/3502289

    doi:10.1145/3502289 2022

  8. [8]

    What to learn, and how: T oward effective learning from rationales

    Carton, S., Kanoria, S., and Tan, C. What to learn, and how: T oward effective learning from rationales. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Findings of the Association for Computational Linguistics: ACL 2022, pp.\ 1075--1088, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi:10.18653/v1/2022.findings-acl.86. U...

    work page doi:10.18653/v1/2022.findings-acl.86 2022

Show all 45 references
  1. [9]

    Interactive rationale extraction for text classification

    Dai, J., Kim, M.-Y., and Goebel, R. Interactive rationale extraction for text classification. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022, 2022. URL https://openreview.net/forum?id=zaJsDuwwdlJ

    2022

  2. [10]

    T., Mitchell, J

    Dai, J., Kim, M.-Y., Sutton, R. T., Mitchell, J. R., Goebel, R., and Baumgart, D. C. Comparative analysis of natural language processing methodologies for classifying computed tomography enterography reports in crohn's disease patients. npj Digital Medicine, 8 0 (1): 0 324, Ma...

    2025

  3. [11]

    BERT : Pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT : Pre-training of deep bidirectional transformers for language understanding. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Conference of the North A merican Chapter of the Association for ...

    2019 doi

  4. [12]

    F., Lehman, E., Xiong, C., Socher, R., and Wallace, B

    DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., and Wallace, B. C. ERASER : A benchmark to evaluate rationalized NLP models. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (eds.), Proceedings of the 58th Annual Meeting of the Association fo...

    2020 doi

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on...

    2021

  6. [14]

    I., Benta Hasan, S., Sultana Prity, F., Ahmed, R., Ahmed, F., Hossen, M

    Fahad, N., Rabbi, R. I., Benta Hasan, S., Sultana Prity, F., Ahmed, R., Ahmed, F., Hossen, M. J., Liew, T. H., Sayeed, M. S., and Ong Michael Goh, K. Generative AI in clinical (2020-2025): a mini-review of applications, emerging trends, and clinical challenges. Front. Digit. H...

    2020

  7. [15]

    Ghasemi Madani, M. R. and Minervini, P. REFER : An end-to-end rationale extraction framework for explanation regularization. In Jiang, J., Reitter, D., and Deng, S. (eds.), Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pp.\ 587--602, Si...

    2023 doi

  8. [16]

    Deep residual learning for image recognition, 2015

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition, 2015. URL https://arxiv.org/abs/1512.03385

    2015 arXiv

  9. [17]

    Smartphone-based clinical diagnostics: towards democratization of evidence-based health care

    Hernández-Neuta, I., Neumann, F., Brightmeyer, J., Ba Tis, T., Madaboosi, N., Wei, Q., Ozcan, A., and Nilsson, M. Smartphone-based clinical diagnostics: towards democratization of evidence-based health care. Journal of Internal Medicine, 285 0 (1): 0 19--39, 2019. doi:https://...

    2019 doi

  10. [18]

    Distilling the knowledge in a neural network, 2015

    Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network, 2015. URL https://arxiv.org/abs/1503.02531

    2015 arXiv

  11. [19]

    Annealing knowledge distillation

    Jafari, A., Rezagholizadeh, M., Sharma, P., and Ghodsi, A. Annealing knowledge distillation. In Merlo, P., Tiedemann, J., and Tsarfaty, R. (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp.\ 249...

    2021 doi

  12. [20]

    Jain, S., Wiegreffe, S., Pinter, Y., and Wallace, B. C. L earning to faithfully rationalize by construction. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.\ 4459--4...

    2020 doi

  13. [21]

    Categorical reparameterization with gumbel-softmax

    Jang, E., Gu, S., and Poole, B. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=rkE3y85ee

    2017

  14. [22]

    MARE : Multi-aspect rationale extractor on unsupervised rationale extraction

    Jiang, H., Duan, J., Qu, Z., and Wang, J. MARE : Multi-aspect rationale extractor on unsupervised rationale extraction. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 11734--11...

    2024 doi

  15. [23]

    Neural relational inference for interacting systems, 2018

    Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. Neural relational inference for interacting systems, 2018. URL https://arxiv.org/abs/1802.04687

    2018 arXiv

  16. [24]

    Learning multiple layers of features from tiny images

    Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto, Ontario, 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

    2009

  17. [25]

    Rationalizing neural predictions

    Lei, T., Barzilay, R., and Jaakkola, T. Rationalizing neural predictions. In Su, J., Duh, K., and Carreras, X. (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp.\ 107--117, Austin, Texas, November 2016. Association for Computati...

    2016 doi

  18. [26]

    D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., Jiang, R., Khosravi, H., Lecue, F., Malgieri, G., Páez, A., Samek, W., Schneider, J., Speith, T., and Stumpf, S

    Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., Jiang, R., Khosravi, H., Lecue, F., Malgieri, G., Páez, A., Samek, W., Schneider, J., Speith, T., and Stumpf, S. Explainable artificial intelligenc...

    2024

  19. [27]

    Lundberg, S. M. and Lee, S.-I. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 4768–4777, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964

    2017

  20. [28]

    L., Daly, R

    Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., and Potts, C. Learning word vectors for sentiment analysis. In Lin, D., Matsumoto, Y., and Mihalcea, R. (eds.), Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language ...

    2011

  21. [29]

    D., and Aruleba, K

    Mienye, E., Jere, N., Obaido, G., Mienye, I. D., and Aruleba, K. Deep learning in finance: A survey of applications and techniques. AI, 5 0 (4): 0 2066--2091, 2024. ISSN 2673-2688. doi:10.3390/ai5040101. URL https://www.mdpi.com/2673-2688/5/4/101

    doi

  22. [30]

    The Principia: Mathematical Principles of Natural Philosophy

    Newton, I. The Principia: Mathematical Principles of Natural Philosophy. University of California Press, Berkeley, CA, 1999. ISBN 978-0520088177. Original work published 1687

    1999

  23. [31]

    GPT -4 technical report, 2024

    OpenAI. GPT -4 technical report, 2024. URL https://arxiv.org/abs/2303.08774

    2024 arXiv

  24. [32]

    An information bottleneck approach for controlling conciseness in rationale extraction

    Paranjape, B., Joshi, M., Thickstun, J., Hajishirzi, H., and Zettlemoyer, L. An information bottleneck approach for controlling conciseness in rationale extraction. In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceedings of the 2020 Conference on Empirical Methods in ...

    2020 doi

  25. [33]

    why should i trust you?

    Ribeiro, M. T., Singh, S., and Guestrin, C. "why should i trust you?": Explaining the predictions of any classifier, 2016. URL https://arxiv.org/abs/1602.04938

    2016 arXiv

  26. [34]

    E., Chassang, A., Gatta, C., and Bengio, Y

    Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., and Bengio, Y. Fitnets: Hints for thin deep nets, 2015. URL https://arxiv.org/abs/1412.6550

    2015 arXiv

  27. [35]

    R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D

    Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV), pp.\ 618--626, 2017. doi:10.1109/ICCV.2017.74

    2017 doi

  28. [36]

    Axiomatic attribution for deep networks

    Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, pp.\ 3319–3328. JMLR.org, 2017

    2017

  29. [37]

    The bitter lesson

    Sutton, R. The bitter lesson. http://www.incompleteideas.net/IncIdeas/BitterLesson.html, March 2019. Accessed: 2026-01-25

    2019

  30. [38]

    Training data-efficient image transformers & distillation through attention, 2021

    Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. Training data-efficient image transformers & distillation through attention, 2021. URL https://arxiv.org/abs/2012.12877

    2021 arXiv

  31. [39]

    Well-read students learn better: On the importance of pre-training compact models, 2019

    Turc, I., Chang, M.-W., Lee, K., and Toutanova, K. Well-read students learn better: On the importance of pre-training compact models, 2019. URL https://arxiv.org/abs/1908.08962

    2019 arXiv

  32. [40]

    Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn., 8 0 (3–4): 0 229–256, May 1992. ISSN 0885-6125. doi:10.1007/BF00992696. URL https://doi.org/10.1007/BF00992696

    1992 doi

  33. [41]

    Xie, S. M. and Ermon, S. Reparameterizable subset sampling via continuous relaxations, 2021. URL https://arxiv.org/abs/1901.10517

    2021 arXiv

  34. [42]

    Yu, M., Zhang, Y., Chang, S., and Jaakkola, T. S. Understanding interlocking dynamics of cooperative rationalization. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/foru...

    2021

  35. [43]

    DARE : Disentanglement-augmented rationale extraction

    Yue, L., Liu, Q., Du, Y., An, Y., Wang, L., and Chen, E. DARE : Disentanglement-augmented rationale extraction. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=6OhjECfqt2

    2022

  36. [44]

    Towards faithful explanations: Boosting rationalization with shortcuts discovery

    Yue, L., Liu, Q., Du, Y., Wang, L., Gao, W., and An, Y. Towards faithful explanations: Boosting rationalization with shortcuts discovery. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=uGtfk2OphU

    2024

  37. [45]

    and Komodakis, N

    Zagoruyko, S. and Komodakis, N. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Sks9_ajex

    2017

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.