Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Compressed Models are NOT Trust-equivalent to Their Large Counterparts

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Compressed BERT variants are not trust-equivalent to BERT-base: even when accuracy is nearly identical, their top-10 LIME/SHAP features overlap only partially and their confidence calibration differs.

desk verdict Useful caution about accuracy parity not implying trust-equivalence, but the interpretability-alignment evidence needs a self-overlap baseline before it can carry the central claim. read the letter →

arxiv 2508.13533 v1 pith:XHUADQTZ submitted 2025-08-19 cs.CL cs.LG

Rohit Raj Rai , Chirag Kothari , Siddhesh Shelke , Amit Awekar This is my paper
classification cs.CLcs.LG
keywords modelcompressiontrust-equivalenceinterpretabilityalignmentcalibrationsimilarityLIMESHAPBERTnaturallanguageinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models are routinely compressed before deployment, and compressed variants often match the original's accuracy. This paper argues that accuracy parity is not enough: a compressed model can be trusted in the same way as its large counterpart only if it bases predictions on the same input features and assigns confidence with the same reliability. Testing BERT-base against four compressed variants (Distil-BERT, BERT-Medium, BERT-Mini, BERT-Tiny) on natural-language-inference and paraphrase-identification tasks, the paper finds the overlap of top-10 LIME/SHAP features never reaches 90% and peaks around 67%. Calibration profiles also diverge, with the smallest model sometimes better calibrated (lower ECE) but worse on Brier score. The paper concludes that compressed models are not trust-equivalent drop-in replacements, and that deployment decisions must go beyond performance parity.

What carries the argument

The central object is the two-dimensional trust-equivalence framework. A model M1 is trust-equivalent to M2 if both conditions hold: (1) interpretability alignment — both models base predictions on the same salient input features, operationalized as Jaccard similarity between unordered top-K LIME and SHAP feature sets (K=10); (2) calibration similarity — both exhibit comparable probability reliability, operationalized by ECE, MCE, Brier score, and reliability diagrams. The framework deliberately compares equivalence, not quality: it does not rate which model is better.

What would settle it

Compute LIME/SHAP top-10 Jaccard between BERT-base and a copy of BERT-base initialized and fine-tuned with a different random seed. If self-agreement is also low (e.g., around 0.6 rather than near 0.9), the reported alignment gap would reflect explainer instability rather than a genuine difference in decision processes between large and compressed models.

Watch

Extended reading notes

Core claim

The paper claims that compressed models fail to maintain trust-equivalence with their large counterparts, defined along two dimensions: interpretability alignment and calibration similarity. Interpretability alignment, measured by Jaccard similarity of top-10 LIME/SHAP feature sets, never reaches 90% and peaks at about 67% (BERT-Medium on NLI), meaning compressed models select roughly one-third of their salient features differently from BERT-base. Calibration similarity, measured by ECE, MCE, Brier score, and reliability diagrams, also differs: for example, on NLI, BERT-Tiny has ECE 0.018 versus BERT-base's 0.057, but BERT-Tiny's Brier score is 0.29 versus 0.16. The paper concludes that accu

Load-bearing premise

The load-bearing premise is that LIME/SHAP top-10 feature sets faithfully capture each model's decision process and that Jaccard overlap between those sets is a valid measure of decision-process similarity; the paper does not validate this and even cites known instabilities of these explainers.

Editorial extensions

If this is right

  • Performance parity is insufficient for safe substitution in high-stakes settings; trust-equivalence metrics should become part of model evaluation before deployment.
  • Knowledge-distilled models like Distil-BERT track the large model's calibration more closely than architectural-reduction models, suggesting the compression paradigm influences trust-equivalence.
  • Extreme compression can improve ECE while worsening Brier score, forcing an explicit trade-off between calibration and accuracy when choosing a compressed replacement.
  • Compressed models' confidence distributions shift toward lower confidence, so confidence thresholds tuned on the large model may not transfer to the compressed model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If LIME/SHAP attributions are unstable, the reported Jaccard gaps may be inflated by explainer noise; measuring each explainer's self-agreement across random seeds would separate decision-process divergence from measurement noise.
  • The framework is demonstrated on encoder-only text models; applying it to decoder-only LLMs or image and audio models would test whether compression-paradigm effects generalize across modalities.
  • Trust-equivalence could become an explicit training objective: compression methods could add feature-attribution and calibration-matching losses rather than only imitating the teacher's outputs.
  • If the trend holds, model cards for compressed models should report trust-equivalence relative to the uncompressed original, not just accuracy and latency.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a two-dimensional framework for evaluating whether a compressed model is 'trust-equivalent' to its large counterpart: (1) interpretability alignment, measured as average Jaccard overlap of top-10 LIME/SHAP feature sets between models, and (2) calibration similarity, measured via ECE, MCE, Brier score, and reliability diagrams. Experiments on NLI (SNLI) and paraphrase identification (QQP) compare BERT-base with Distil-BERT, BERT-Medium, BERT-Mini, and BERT-Tiny. The reported interpretability alignment values are 0.37–0.67, and calibration profiles differ across models, with BERT-Tiny showing lower ECE but higher Brier score than BERT-base. The authors conclude that compressed models are not trust-equivalent to BERT-base even when accuracy is close.

Significance. If validated, the proposed framework would be a useful practical tool for model replacement decisions, moving beyond accuracy parity. The two dimensions—feature-based decision similarity and probability reliability—are well motivated and the experimental setup covers two representative compression paradigms across two complementary tasks. However, the paper's central negative result currently rests on a measurement of interpretability alignment that lacks a null baseline or stability control, given the acknowledged stochasticity of LIME/SHAP. The calibration evidence is mixed and is not presented as a direct similarity measure. The work is also partly reproducible (detailed training setup, public datasets), though the promised code link is a placeholder. Overall, the idea is sound and the empirical direction is worth publishing after the evidentiary gaps are addressed.

major comments (3)
  1. [Section 3.2, Table 3] The 'low interpretability alignment' claim is load-bearing for the paper's main conclusion, but the Jaccard values (0.372–0.669) are presented without any baseline. Since the paper itself notes that 'LIME and SHAP have known instabilities' (Section 1), the observed cross-model overlap may reflect explainer noise rather than genuine differences in decision processes. The authors should provide (a) a self-overlap baseline: Jaccard between two independent explanation runs of the same model on the same instances; (b) a random-feature baseline: expected overlap between top-K sets chosen uniformly at random; and (c) confidence intervals or a significance test comparing cross-model overlap to these baselines. Without such controls, the statement that 'even the best case is 67%, so one-third of the features differ' (Section 3.2) is uninterpretable.
  2. [Section 3.3, Table 6 and Figure 2] The paper claims 'calibration similarity' and 'profiles differ markedly,' but the metrics reported—ECE, MCE, Brier score—are each model's own calibration error, not a distance or similarity to BERT-base's calibration profile. For example, BERT-Tiny's ECE (0.018) is lower than BERT-base's (0.057), while its Brier score (0.29) is worse; the conclusion that these are 'not trust-equivalent' is asserted rather than measured. The authors should define a quantitative similarity criterion, e.g., absolute difference in ECE per bin, area between reliability curves, or a statistical test comparing the two models' reliability diagrams. As written, the calibration evidence is mixed and does not uniformly support the headline claim.
  3. [Section 3, Tables 3–6 and Figure 1] All reported metrics are point estimates with no variance or significance testing. No standard deviations, confidence intervals, or hypothesis tests are given for the interpretability alignment or calibration measures. The threshold of 90% for 'high' alignment is introduced without justification, and the Figure 1 claim that trends 'remain the same' is based on visual inspection only. The authors should add uncertainty estimates (e.g., bootstrapping across test instances or multiple fine-tuning seeds) and either predefine a failure threshold or compare against a null distribution.
minor comments (5)
  1. [Formatting] The manuscript contains ACM template artifacts: 'Conference acronym ’XX', '© 2018', 'Received 20 February 2007', and a 'Link to be provided in the camera-ready version' for code. These must be cleaned before submission.
  2. [Section 3.3] There is an inconsistency: the reliability-diagram discussion says 'all models generally exhibit overconfidence,' while the Brier-score paragraph says 'compressed models are generally under-confident.' Please clarify which regime applies under which metric.
  3. [Table 2] The highlighted top-3 words are not visible in the plain-text table. If this table is important for intuition, consider presenting it as a figure with colored/bold tokens.
  4. [References] Reference [1] is cited for LIME/SHAP instability, but the claim is broad; adding a second reference (e.g., on explainer robustness in NLP) would strengthen the acknowledgment.
  5. [Section 3.3] Typo: 'trus-equivalence' should be 'trust-equivalence'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the trust-equivalence verdict is an empirical measurement under an explicitly stated operational definition.

full rationale

The paper's claim is an empirical negative result, not a derivation. Trust-equivalence is explicitly defined in Section 1 as two conditions: same salient input features (measured by LIME/SHAP Jaccard) and same probability reliability (measured by ECE/MCE/Brier/reliability diagrams). The experiments independently measure these quantities on fine-tuned BERT-family models; no parameter is fitted to the target conclusion. The low interpretability-alignment numbers (Table 3) and differing calibration metrics (Table 6) are direct observations, and the conclusion 'compressed models fail to maintain trust-equivalence' follows by applying the stated definition to those observations. This is a conditional operationalization, not a circular derivation. The only self-citation is reference [16] (Rai, Pal, Awekar), used in Related Work to note that prior work showed behavioral differences; that sentence is also supported by external references [13,26,27], and the present work's own experiments carry the argument, so the self-citation is not load-bearing. Concerns about LIME/SHAP instability and lack of a self-overlap baseline are validity threats to the proxy, not circularity: the paper explicitly acknowledges the instability and still measures the chosen quantity. No equation reduces to another by construction, and no fitted parameter is renamed as a prediction. Hence no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central empirical findings rest on the assumptions that explainer attributions are faithful, that the Jaccard alignment score captures decision-process similarity, that max-softmax confidence is a meaningful reliability measure, and that the chosen model family and tasks represent the broader claim. No parameters are fitted to force the conclusion; K is a chosen hyperparameter with a robustness check.

free parameters (1)
  • K (number of top features) = 10
    Chosen for the main interpretability alignment results; the paper reports trends are stable for K=1..10, so it is a manual hyperparameter rather than a fitted constant.
assumptions (4)
  • domain assumption LIME and SHAP attributions approximate the features a model actually uses for a prediction.
    Section 3.2 uses top-K LIME/SHAP features as the decision process; known instability is cited in [1] but not mitigated.
  • domain assumption Jaccard similarity over unordered top-K sets is a meaningful scalar of interpretability alignment.
    Section 3.2 defines alignment this way; no external validation that this matches human or behavioral agreement.
  • domain assumption Maximum softmax probability is a valid confidence estimate for assessing calibration.
    Section 3.3 uses the max softmax probability as confidence; standard practice, but ignores other uncertainty sources.
  • domain assumption The chosen tasks (SNLI, QQP) and BERT-family variants are representative enough for the general conclusion.
    The conclusion generalizes to compressed models broadly from a single architecture family and two text tasks; stated as future work in Section 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Compressed Models are NOT Trust-equivalent to Their Large Counterparts." pith.science (2026). https://pith.science/paper/XHUADQTZ

@misc{pith2026250813533,
  author       = {Pith},
  title        = {Pith review of: Compressed Models are NOT Trust-equivalent to Their Large Counterparts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XHUADQTZ}},
  note         = {Machine review of arXiv:2508.13533}
}
read the original abstract

Large Deep Learning models are often compressed before being deployed in a resource-constrained environment. Can we trust the prediction of compressed models just as we trust the prediction of the original large model? Existing work has keenly studied the effect of compression on accuracy and related performance measures. However, performance parity does not guarantee trust-equivalence. We propose a two-dimensional framework for trust-equivalence evaluation. First, interpretability alignment measures whether the models base their predictions on the same input features. We use LIME and SHAP tests to measure the interpretability alignment. Second, calibration similarity measures whether the models exhibit comparable reliability in their predicted probabilities. It is assessed via ECE, MCE, Brier Score, and reliability diagrams. We conducted experiments using BERT-base as the large model and its multiple compressed variants. We focused on two text classification tasks: natural language inference and paraphrase identification. Our results reveal low interpretability alignment and significant mismatch in calibration similarity. It happens even when the accuracies are nearly identical between models. These findings show that compressed models are not trust-equivalent to their large counterparts. Deploying compressed models as a drop-in replacement for large models requires careful assessment, going beyond performance parity.

Figures

Figures reproduced from arXiv: 2508.13533 by the authors.

Figure 1
Figure 1. Our Interpretability Alignment results remain valid even with variation in the value of [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Reliability Diagrams for NLI and PI tasks [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 20 canonical work pages

  1. [1]

    David Alvarez-Melis and Tommi S Jaakkola. 2018. On the robustness of inter- pretability methods. arXiv preprint arXiv:1806.08049 (2018). https://arxiv.org/ pdf/1806.08049

    arXiv 2018

  2. [2]

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016). https://arxiv.org/abs/1606.06565

    arXiv 2016

  3. [3]

    Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica (2016). https://www.propublica.org/article/ machine-bias-risk-assessments-in-criminal-sentencing

    work page 2016

  4. [4]

    https://dl.acm.org/doi/10.5555/3157382.3157584

    Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems (2016). "https://dl.acm.org/doi/10.5555/3157382.3157584"

    arXiv 2016

  5. [5]

    Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning

    work page

  6. [6]

    Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science (2017). https://www.science.org/doi/10.1126/science.aal4230

    doi:10.1126/science.aal4230 2017

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . https://aclanthology.org/N19-1423.pdf

    work page 2019

  8. [8]

    Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, He- len M Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks. nature (2017). https://www.nature.com/ articles/nature21056

    work page 2017

Show all 33 references
  1. [9]

    Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR) (2018). https://dl.acm.org/doi/10. 1145/3236009

    2018

  2. [10]

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning . https://proceedings.mlr.press/v70/guo17a/guo17a.pdf

    2017

  3. [11]

    Manish Gupta and Puneet Agrawal. 2022. Compression of deep learning models for text: A survey. ACM Transactions on Knowledge Discovery from Data (TKDD) (2022). https://dl.acm.org/doi/full/10.1145/3487045

    2022 doi

  4. [12]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015). https://arxiv.org/abs/ 1503.02531

    2015 arXiv

  5. [13]

    Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome

  6. [14]

    Zhuo Li, Hengyi Li, and Lin Meng. 2023. Model compression for deep neural networks: A survey. Computers (2023). https://www.mdpi.com/2073-431X/12/3/ 60

    2023

  7. [15]

    Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems . https: //dl.acm.org/doi/10.5555/3295222.3295230

    2017

  8. [16]

    Rohit Raj Rai, Rishant Pal, and Amit Awekar. 2024. Compressed models are NOT miniature versions of large models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management . https://dl.acm.org/doi/ abs/10.1145/3627673.3679888

    2024

  9. [17]

    Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, et al

  10. [18]

    Why Should I Trust You?

    Marco Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. https://aclant...

    2016

  11. [19]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Dis- tilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108 (2019). https://arxiv.org/abs/1910.01108

    2019 arXiv

  12. [20]

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. 2021. Training data-efficient image transformers & distillation through attention. In International conference on machine learning . https://proceedings.mlr.press/v139/touvro...

    2021

  13. [21]

    Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962 (2019). https://arxiv.org/abs/1908.08962

    2019 arXiv

  14. [22]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems (2017). https: //dl.acm.org/doi/10.5555/3295222.3295349

    2017

  15. [23]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  16. [24]

    Kan Wu, Houwen Peng, Zhenghong Zhou, Bin Xiao, Mengchen Liu, Lu Yuan, Hong Xuan, Michael Valenzuela, Xi Stephen Chen, Xinggang Wang, et al

  17. [25]

    Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. 2014. Zipformer: A faster and better encoder for automatic speech recognition. InThe Twelfth International Conference on Learning Representations. https://openre...

    2014

  18. [26]

    Shaokai Ye, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin

  19. [27]

    Xiaoyong Yuan and Lan Zhang. 2022. Membership inference attacks and defenses in neural network pruning. In 31st USENIX Security Symposium (USENIX Security 22). https://www.usenix.org/conference/usenixsecurity22/presentation/yuan- xiaoyong

    2022

  20. [28]

    Xingjian Zhang, Siwei Wen, Wenjun Wu, and Lei Huang. 2025. Tinyllava-video- r1: Towards smaller lmms for video reasoning. arXiv preprint arXiv:2504.09641 (2025). https://arxiv.org/abs/2504.09641 Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    2025 arXiv

  21. [31]

    model compression, or both?

    Adversarial robustness vs. model compression, or both?. In Pro- ceedings of the IEEE/CVF international conference on computer vision . https://openaccess.thecvf.com/content_ICCV_2019/papers/Ye_Adversarial_ Robustness_vs._Model_Compression_or_Both_ICCV_2019_paper.pdf

  22. [2015]

    In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

    A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. https://aclanthology.org/D15-1075/#

    2015

  23. [2017]

    arXiv preprint arXiv:1711.05225 (2017)

    Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017). https://arxiv.org/abs/1711.05225

    2017 arXiv

  24. [2019]

    https://arxiv.org/abs/1911.05248

    What do compressed deep neural networks forget? arXiv preprint arXiv:1911.05248 (2019). https://arxiv.org/abs/1911.05248

    1911 arXiv

  25. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Tinyclip: Clip distillation via affinity mimicking and weight inheritance. In Proceedings of the IEEE/CVF International Conference on Computer Vision . https://openaccess.thecvf.com/content/ICCV2023/html/Wu_TinyCLIP_CLIP_ Distillation_via_Affinity_Mimicking_and_Weight_Inherita...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.