REVIEW 3 major objections 5 minor 28 references
Calibrating LLMs with Information-Theoretic Evidential Deep Learning
T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read This paper introduces IB-EDL, adding an information bottleneck to evidential deep learning; across six datasets and three 7-8B LLMs it reports the lowest calibration error and NLL, cutting ECE several-fold versus standard baselines.
desk verdict Useful empirical calibration method, but the information-theoretic lower-bound story is mathematically wrong as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the EDL pipeline $x \to \tilde{e} \to e \to \alpha \to \pi \to y$ with the pre-evidence $\tilde{e}$ chosen as the stochastic bottleneck $z$. The LLM's final linear head is split into two parts, one predicting the mean $\mu$ and one predicting the variance $\sigma$ of a factorized Gaussian $p(z|x) = \mathcal{N}(z; \mu, \operatorname{diag}(\sigma))$, with $\sigma$ passed through SoftPlus. Sampling during training uses the reparameterization trick, and the analytic KL to a standard normal prior produces the $\ell^2$ penalty that counteracts the EDL tendency toward huge evidence values.
What would settle it
Train IB-EDL on a synthetic binary classification task where the true conditional $p(y|x)$ is known and the base model is deliberately miscalibrated; if the training objective is not a lower bound on an empirically estimated $I(Z,Y)$ in early epochs, the theoretical claim fails in that regime. A simpler check: replace the KL term with plain $\ell^2$ weight decay on the output head; if the ECE and NLL gains disappear, the information-bottleneck machinery is not what is doing the work.
Extended reading notes
Core claim
The central discovery is that the information bottleneck can be applied to evidential deep learning by choosing the pre-evidence $\tilde{e}$ as the bottleneck variable $z$. Because the EDL pipeline $\tilde{e} \to e \to \alpha \to \pi \to y$ is deterministic except for the Dirichlet draw, the conditional $p(y|z)$ is computable directly from $\alpha = \operatorname{SoftPlus}(\tilde{e}) + 1$, so the lower bound $I(Z,Y) \ge \mathbb{E}[\log p(y|z)]$ is used without learning an approximate decoder $q(y|z)$. The resulting objective is $L^{\text{IB-MSE}}(\theta) + \beta L^{\text{IB-Info}}(\theta)$, where $L^{\text{IB-Info}}$ is a Gaussian KL divergence whose closed form is $\tfrac{1}{2}\mathbb{E}[\|\mu\|^2 + \|\sigma\|^2 - 2\sum_j \log \sigma_j]$, imposing $\ell^2$ regularization on the mean pre-evidence. The paper also proves that variational-Bayes EDL methods are a special case of IB-EDL when the bottleneck variable is $\pi$ and the prior is a Dirichlet with the target's evidence removed.
Load-bearing premise
The derivation assumes the model's own probability estimate for each answer, computed from the Dirichlet parameters, is the true conditional probability, an assumption that only holds for an already calibrated model — the very thing the training is meant to produce.
Editorial extensions
If this is right
- IB-EDL improves in-distribution calibration without multiple forward passes: on the six reported datasets it reaches the lowest ECE and NLL, with accuracy on par with or above MAP training.
- OOD detection improves under both max-probability and uncertainty-mass scores, including under a large shift from common-sense reasoning to math questions.
- Fine-tuning with 30% label noise is more robust: IB-EDL reports the highest accuracy among EDL baselines in most settings.
- The regularization costs less than 2% extra parameters and GFLOPs relative to the pretrained LLM, so the calibration gain does not trade away inference efficiency.
- The framework unifies several existing EDL methods as special cases, giving a common lens for comparing their objectives.
Reading between the lines
- Because the KL regularizer $\tfrac{1}{2}\mathbb{E}[\|\mu\|^2 + \|\sigma\|^2 - 2\sum_j \log \sigma_j]$ does not reference the label space, the same IB-EDL mechanism could be transplanted to regression or generative heads, not just classification logits.
- The lower-bound claim is only guaranteed if the model's $p(y|z)$ equals the true conditional; in regimes where it does not, the objective may still act as an effective regularizer — a distinction that could be probed by directly estimating $I(Z,Y)$ on a synthetic task.
- The $\ell^2$ pull on the mean pre-evidence suggests a possible failure mode: datasets where the true posterior is genuinely sharp could be under-confident under IB-EDL, since the regularizer uniformly discourages large evidence.
- The unifying special-case result hints that other uncertainty layers (e.g., temperature-scaled logits or feature-space bottlenecks) could be designed by choosing different bottleneck variables $z$ and priors $r(z)$.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes IB-EDL, a method that regularizes evidential deep learning (EDL) with an information bottleneck (IB) to calibrate fine-tuned large language models. The authors choose the pre-evidence (logits) as the IB stochastic variable, model its distribution as a diagonal Gaussian whose mean and variance are predicted by the LLM, and train with an MSE-based EDL loss plus a KL penalty toward a standard normal prior. Experiments on Llama2-7B, Llama3-8B, and Mistral-7B across six multiple-choice datasets report consistent improvements in ECE and NLL over MAP, MC-Dropout, Deep Ensembles, Laplace-LoRA, and several EDL baselines, together with strong OOD detection and label-noise robustness, at less than 2% computational overhead.
Significance. If the empirical results hold, the paper delivers a practical single-forward-pass calibration method for fine-tuned LLMs, with extensive comparisons, released code, and consistent gains across models and datasets. The low-overhead aspect is valuable for deployment. However, the theoretical framing is a central part of the contribution and is not sound as written: the claimed guaranteed lower bound on mutual information is undermined by a mathematical sign error and by a mismatch between the derived objective and the objective actually optimized. These issues do not necessarily invalidate the empirical findings, but they require substantial revision before the paper's theoretical claims can be accepted.
major comments (3)
- [Section 3.1, Eq. (11)] The statement that term (ii) is 'not necessarily non-negative' is mathematically incorrect. Under the assumed Markov chain X-Z-Y, term (ii) equals E_{p(x)p(z|x)}[KL(p(y|z) || q(y|z))], which is always non-negative by Gibbs' inequality. Replacing the true p(y|z) with a variational q(y|z) in Eq. (9) therefore yields a standard variational lower bound; it does not undermine the bound. This invalidates the paper's stated motivation for choosing z = pre-evidence to 'skip' q(y|z) and 'ensure' a valid lower bound.
- [Section 3.2, Eq. (15)] The actual training objective used in all experiments, LIB-MSE in Eq. (15), is not the log-likelihood term LIB-NLL in Eq. (12) for which the lower-bound derivation in Eqs. (9)-(10) is given. No inequality is shown that the expected Brier score (LIB-MSE) is a lower bound on I(Z,Y), so the objective optimized in practice is not the IB objective whose validity is claimed. The paper should either derive an appropriate bound for the MSE surrogate or explicitly reframe IB-EDL as an empirical regularizer rather than a guaranteed information-theoretic lower bound.
- [Section 3.2, Eq. (12)] The lower-bound derivation substitutes the model's Dirichlet predictive distribution alpha_y/alpha_0 for the true conditional p(y|z) in Eq. (9). The 'skip q' argument removes only the variational approximation; it does not remove the gap between the model's predictive distribution and the true data-generating conditional. The claimed 'valid lower bound of I(Z,Y)' therefore holds only under the assumption that the model's predictive distribution already coincides with the true conditional, which is exactly the property that calibration training is meant to achieve. This assumption is not stated or tested, and it is a load-bearing gap in the theoretical contribution.
minor comments (5)
- [Figure 1] The axis labels in Figure 1 are garbled (e.g., 'Acc(racy', 'D fferen( β'), which makes the ablation plot difficult to read.
- [Appendix E.3] There is an unresolved placeholder reference '??' for the MMLU dataset citation in the OOD-detection section.
- [Sections 3.2 and 3.3] The notation p(y|z) is used both for the marginal predictive distribution in Eq. (12) and for the distribution inside the expectation in Eq. (13); this conflation should be clarified to avoid confusion about what is being bounded.
- [Section 3.2] The justification for the standard Gaussian prior on pre-evidence relies on cited observations about late-layer activations, but the paper does not provide quantitative evidence for the claimed range of pre-evidences; a histogram or other empirical check would strengthen this choice.
- [Algorithm 1] At inference the paper averages K sampled pre-evidences and then computes alpha from the average, which is not the same as propagating the full predictive distribution through the nonlinear SoftPlus; this approximation should be stated explicitly and justified.
Circularity Check
The main in-distribution calibration results are independent of the method's own outputs, but the OOD calibration generalization claim in Section 4.3 is supported by per-test-set fitted post-hoc values in Table 12.
-
fitted input called prediction
[Appendix D, Appendix E.4 (Table 12), and Section 4.3.]
"In the calibration experiment on OOD test sets presented in Table 12, we use ζ = −1.0 for OBQA→ ARC-C, ζ = 3.0 for OBQA→ ARC-E, and ζ = −5.0 for OBQA→ CSQA. ... Its calibration performance also generalizes well to OOD datasets (see Appendix E.4)."
The claim that IB-EDL's calibration generalizes to OOD datasets is evidenced by Table 12, where IB-EDL's ECE and NLL are produced after applying the post-hoc adjustment α_j ← α_j − ζ·σ_j with a distinct scalar ζ selected per OOD test set. The paper reports the per-test-set ζ values rather than a single validation-selected value, so the resulting ECE and NLL reflect in-sample tuning to the target test sets rather than an independent out-of-sample prediction. The apparent OOD calibration improvement is therefore forced by the fitted ζ, making this specific generalization result reduce to its own input.
full rationale
The central in-distribution calibration results (Tables 1, 2, and 3) compare IB-EDL against external baselines on standard benchmarks, with the regularization weight β tuned on validation data in the usual way; those results are not circular. The only identified circular step is in the secondary OOD calibration claim (Appendix E.4/Table 12), where separate post-hoc parameters ζ are chosen per OOD test set and the resulting ECE/NLL are then presented as evidence of generalization. That is a fitted-input-called-prediction issue. There is also a minor self-citation (Zhang et al. 2021) supporting the Gaussian prior, but it is not load-bearing; the method's core empirical claim stands independently. The theoretical lower-bound derivation in Section 3.2 contains a questionable identification of the model's own predictive distribution with the true conditional p(y|z) and the sign discussion around Eq. (11), but these are correctness concerns rather than circularity: the main results do not derive their numbers from that bound. Overall circularity is partial and confined to a supporting OOD experiment, so it does not invalidate the primary calibration comparisons.
Assumptions & free parameters
free parameters (2)
- beta (IB regularization weight) =
Values per model/dataset in Table 5, ranging from 1e-3 to 1e-6
- K (number of sampled pre-evidences) =
20
assumptions (4)
- domain assumption The Markov chain X->Z->Y holds for the chosen latent variable Z.
- domain assumption The pre-evidences z are Gaussian with diagonal covariance, with prior N(0,1).
- domain assumption p(y|z) in the lower bound can be identified with the model's predictive distribution alpha_y/alpha_0.
- standard math Non-negativity of KL divergence and the reparameterization trick are used without proof.
Cite this review
Pith. "Pith review of Calibrating LLMs with Information-Theoretic Evidential Deep Learning." pith.science (2026). https://pith.science/paper/GO5SA7T6
@misc{pith2026250206351,
author = {Pith},
title = {Pith review of: Calibrating LLMs with Information-Theoretic Evidential Deep Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/GO5SA7T6}},
note = {Machine review of arXiv:2502.06351}
}
read the original abstract
Fine-tuned large language models (LLMs) often exhibit overconfidence, particularly when trained on small datasets, resulting in poor calibration and inaccurate uncertainty estimates. Evidential Deep Learning (EDL), an uncertainty-aware approach, enables uncertainty estimation in a single forward pass, making it a promising method for calibrating fine-tuned LLMs. However, despite its computational efficiency, EDL is prone to overfitting, as its training objective can result in overly concentrated probability distributions. To mitigate this, we propose regularizing EDL by incorporating an information bottleneck (IB). Our approach IB-EDL suppresses spurious information in the evidence generated by the model and encourages truly predictive information to influence both the predictions and uncertainty estimates. Extensive experiments across various fine-tuned LLMs and tasks demonstrate that IB-EDL outperforms both existing EDL and non-EDL approaches. By improving the trustworthiness of LLMs, IB-EDL facilitates their broader adoption in domains requiring high levels of confidence calibration. Code is available at https://github.com/sandylaker/ib-edl.
Figures
Reference graph
Works this paper leans on
-
[1]
In fact, the choice of the prior r(π ) is not unique
The condition in Proposition 1 is that the latent variable z = π , and the prior r(z) = r(π ) = Dir (π ; y⊙ α + (1− y)). In fact, the choice of the prior r(π ) is not unique. F or example, Chen et al. (2018) present three options. The correctness of Proposition 1 remains unaffected by the choice of prior . Proposition 1 uses one exemplary prior r(π ) = Di...
work page 2018
-
[5]
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan
URL https://arxiv.org/abs/2407.21783. Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Dee p ensembles: A loss landscape per- spective. arXiv preprint arXiv:1912.02757 ,
arXiv 1912
-
[7]
Understanding the effect of noi se in llm training data with algorithmic chains of thought
13 Published as a conference paper at ICLR 2025 Alex Havrilla and Maia Iyer. Understanding the effect of noi se in llm training data with algorithmic chains of thought. arXiv preprint arXiv:2402.04004 ,
arXiv 2025
-
[9]
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig
URL https://arxiv.org/abs/2310.06825. Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. Ho w can we know when language models know? on the calibration of language models for quest ion answering. Transactions of the Association for Computational Linguistics , 9:962–977,
-
[10]
Diederik P . Kingma and Max Welling. Auto-Encoding V ariatio nal Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings,
work page 2014
-
[13]
Can a suit of armor conduct elec- tricity? a new dataset for open book question answering
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct elec- tricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pp. 2381–2391,
work page 2018
-
[15]
Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Xingcheng Fu, Chen g Ji, and S Y u Philip
URL https://arxiv.org/abs/2402.06160. Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Xingcheng Fu, Chen g Ji, and S Y u Philip. Graph structure learning with variational information bottleneck. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pp. 4165–4174,
-
[16]
Commonsenseqa: A question answering challenge targeting commonsense knowledge
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa tional Linguistics: Human Lan- guage T echnologies, V olume 1 (Long and Short Papers), pp. 4149–4158,
work page 2019
Show all 28 references
-
[17]
If the calibration curve lies below the optimal diagonal line, it indicates that the model is ove rconfident
on the training or validation set, which plots accuracy against confidence for binned samples. If the calibration curve lies below the optimal diagonal line, it indicates that the model is ove rconfident. In this case, we set ζ > 0 to encourage greater uncertainty and improve ca...
2024
-
[19]
Information robust dirichlet networks for predictive uncertainty estimation, April 8
15 Published as a conference paper at ICLR 2025 Theodoros Tsiligkaridis. Information robust dirichlet networks for predictive uncertainty estimation, April 8
2025
-
[20]
17/064,046
US Patent App. 17/064,046. Song Wang, Zhen Tan, Ruocheng Guo, and Jundong Li. Noise-rob ust fine-tuning of pretrained language models via external guidance. In Findings of the Association for Computational Lin- guistics: EMNLP 2023 , pp. 12528–12540, 2023a. Yibin Wang, Haizhou ...
2023 arXiv
-
[22]
To believe or not to believe your llm
Y asin Abbasi Y adkori, Ilja Kuzborskij, Andr´ as Gy¨ orgy, and Csaba Szepesv´ ari. To believe or not to believe your llm. arXiv preprint arXiv:2406.02543 ,
-
[23]
(2017) as follows: I(Z,X ) = ∫ p(x, z) logp(z|x) p(z) dzdx = Ep(z|x)Ep(x)[logp(z|x)]− Ep(z)[logp(z)]
16 Published as a conference paper at ICLR 2025 A D ERIVATION OF THE VARIATIONAL BOUNDS Upper bound of I(Z,X ): We reproduce the derivation steps from Alemi et al. (2017) as follows: I(Z,X ) = ∫ p(x, z) logp(z|x) p(z) dzdx = Ep(z|x)Ep(x)[logp(z|x)]− Ep(z)[logp(z)]. (16) Given ...
2017
-
[24]
(17) Plugging Eq
Therefore, Ep(z)[logp(z)]≥ Ep(z)[logr(z)]. (17) Plugging Eq. (17) into Eq. (16), we obtain I(Z,X ) = Ep(z|x)Ep(x)[logp(z|x)]− Ep(z)[logp(z)] ≤ Ep(z|x)Ep(x)[logp(z|x)]− Ep(z)[logr(z)] ≤ Ep(z|x)Ep(x)[logp(z|x)]− Ep(x)Ep(z|x) [logr(z)] = Ep(x)Ep(z|x) [ logp(z|x) r(z) ] = Ep(x) [D...
2020
-
[26]
LoRA hyperparameters: We applied LoRA (Hu et al.,
, Llama3-8B (Dubey et al., 2024), and Mistral-7B-v0.1 (Jiang et al., 2023). LoRA hyperparameters: We applied LoRA (Hu et al.,
2024
-
[27]
lora only
library. For all models, LoRA adaptors were applied to the q proj, v proj, and lm head modules. Additionally, we used Dropout with a dropout rate o f p = 0.1, LoRAα = 16, rankr = 8, and set bias = "lora only". Training details: For the MAP baseline and all EDL methods, all mod...
2016
-
[1968]
Uncertainty estimation by fisher information-based evidential deep learning
11 Published as a conference paper at ICLR 2025 Danruo Deng, Guangyong Chen, Y ang Y u, Furui Liu, and Pheng-Ann Heng. Uncertainty estimation by fisher information-based evidential deep learning. In International Conference on Machine Learning, pp. 7596–7616. PMLR,
2025
-
[1999]
Llama 2: Open founda- tion and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amja d Almahairi, Y asmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bho sale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 ,
-
[2008]
A scal able laplace approximation for neural networks
Hippolyt Ritter, Aleksandar Botev, and David Barber. A scal able laplace approximation for neural networks. In 6th international conference on learning representations , ICLR 2018-conference track proceedings, volume
2018
-
[2016]
Evidential concept embedding models: Towards reliable concept explanations f or skin disease diagnosis
Yibo Gao, Zheyao Gao, Xin Gao, Y uanye Liu, Bomin Wang, and Xiahai Zhuang. Evidential concept embedding models: Towards reliable concept explanations f or skin disease diagnosis. arXiv preprint arXiv:2406.19130,
-
[2017]
Mixlora : Enhancing large language models fine-tuning with lora-based mixture of experts, 2024
Dengchun Li, Yingzi Ma, Naizheng Wang, Zhengmao Y e, Zhiyuan Cheng, Yinghao Tang, Y an Zhang, Lei Duan, Jie Zuo, Cal Y ang, and Mingjie Tang. Mixlora : Enhancing large language models fine-tuning with lora-based mixture of experts, 2024 . 14 Published as a conference paper at I...
2024
-
[2018]
Think you have solved question answering? t ry arc, the ai2 reasoning challenge
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashis h Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? t ry arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 ,
-
[2019]
Deep learning and the inf ormation bottleneck principle
Naftali Tishby and Noga Zaslavsky. Deep learning and the inf ormation bottleneck principle. 2015 IEEE Information Theory W orkshop (ITW), pp. 1–5,
2015
-
[2020]
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R´ emi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in ...
2020
-
[2021]
RACE: Large-scale ReAding comprehension dataset from examinations
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Y ang, and Eduard Hovy. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pp. 785–794. Association for Computational Linguis-...
2017
-
[2022]
Andrew Jesson, Nicolas Beltran-V elez, Quentin Chu, Sweta K arlekar, Jannik Kossen, Y arin Gal, John P Cunningham, and David Blei
URL https://openreview.net/forum?id=nZeVKeeFYf9. Andrew Jesson, Nicolas Beltran-V elez, Quentin Chu, Sweta K arlekar, Jannik Kossen, Y arin Gal, John P Cunningham, and David Blei. Estimating the hallucina tion rate of generative ai. arXiv preprint arXiv:2406.07457,
-
[2023]
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Y ang, Angela Fan, A nirudh Goyal, Anthony Hartshorn, Aobo Y ang, Archi Mitra, Archie Sravankumar, Art em Korenev, Arthur Hinsvark, Arun Rao, Aston...
2025
-
[2024]
A vari ational dirichlet framework for out-of-distribution detection
Wenhu Chen, Yilin Shen, Hongxia Jin, and William Wang. A vari ational dirichlet framework for out-of-distribution detection. arXiv preprint arXiv:1811.07308 ,
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.