Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

Entropy-Based Non-Invasive Reliability Monitoring of Convolutional Neural Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adversarial inputs shift CNN activation entropy immediately, allowing 90% detection without retraining.

desk verdict Plausible idea, unsupported evidence: the decision rule contradicts the reported results and the headline numbers are fitted on ten test batches. read the letter →

arxiv 2508.21715 v1 pith:3XHSQ3NU submitted 2025-08-29 cs.CV cs.AIcs.CRcs.ITeess.IVmath.IT

Amirhossein Nazeri , Wael Hafez This is my paper
classification cs.CVcs.AIcs.CRcs.ITeess.IVmath.IT
keywords AdversarialrobustnessConvolutionalneuralnetworksEntropymonitoringNon-invasiveself-diagnosticsOut-of-distributiondetectionActivationFGSMVGG-16
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a single scalar—the Shannon entropy of post-ReLU activations in a CNN layer—carries a usable fingerprint of adversarial input. On VGG-16 with FGSM attacks on ImageNet, clean and adversarial batches separate enough that a fixed threshold catches 90% of adversarial batches while missing 20% and falsely flagging none of the clean batches at the first convolutional layer. The monitor is non-invasive: it reads activations through forward hooks, so a deployed CNN could gain self-diagnostic ability without retraining, architecture changes, or extra forward passes. A deeper layer gives a weaker but still useful signal, with an 80% detection rate.

What carries the argument

The load-bearing object is batch-wise layer entropy: for a chosen layer, take the activation tensor for a batch of 16 images, flatten it, apply ReLU to keep only positive values, place values into hand-tuned non-uniform bins, normalize the histogram, and compute H = -Σ p log2 p. The bin edges are chosen to maximize clean-versus-adversarial separation, and the resulting scalar is compared against a fixed threshold. This single scalar carries the argument because it moves in opposite directions in the two monitored layers, and the early-layer shift creates a usable decision boundary. The parallel monitoring architecture—forward hooks on features.0 and classifier.3—makes the measurement possibl

What would settle it

Run the same fixed bin edges and threshold on VGG-16 batches attacked with PGD at comparable strength, or compute the monitor per image instead of per 16-image batch; either experiment would show whether the 7% clean-adversarial gap is a stable signature or an artifact of the chosen attack and batch averaging.

Watch

Extended reading notes

Core claim

The paper's central discovery is that adversarial perturbations leave an immediate information-theoretic trace in a CNN's intermediate activations, before the final misclassification occurs. By computing Shannon entropy over flattened, ReLU-masked activation histograms at the first convolutional layer (features.0) and the second fully connected layer (classifier.3) on batches of 16 images, the authors find that FGSM-adversarial batches from ImageNet shift early-layer entropy upward by about 7% (from roughly 5.05–5.12 bits to 5.14–5.20 bits), while pre-classification layer entropy drops from roughly 4.27–4.32 bits to 4.05–4.22 bits. A fixed threshold set between the distributions detects adve

Load-bearing premise

The load-bearing premise is that hand-tuned histogram bins and a fixed threshold, calibrated on 18 clean and 20 adversarial batches from one attack on one architecture, yield an entropy separation that stays stable for other images, other attacks, and other CNN architectures without retuning.

Editorial extensions

If this is right

  • Deployed CNNs could be retrofitted with adversarial detection by hooking a single early convolutional layer, with no retraining or accuracy loss.
  • The 7% early-layer entropy increase can serve as an interpretable, threshold-based early-warning signal that fires before the model's final misclassification.
  • The dual-layer pattern—entropy up in the first conv layer, down in the FC layer—gives two complementary detection signals that could be combined to reduce error below either layer alone.
  • Because adversarial examples retain high softmax confidence, confidence-based monitoring would miss them; entropy-based monitoring offers a signal that does not depend on output confidence.
  • The approach shifts reliability assessment from output statistics to information flow, opening a practical path between information-bottleneck theory and fielded model monitoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 90% figure is a batch-level rate over 16-image batches; per-image entropy distributions would overlap more, so the metric likely overstates single-image detection sensitivity.
  • The bin edges and threshold were calibrated on the same clean and adversarial batches they later separate; the 7% shift should be treated as a demonstrated effect on this dataset, not yet a general law, until tested on other attacks and architectures.
  • If the early-layer shift is caused by high-frequency gradient noise, then input smoothing or JPEG compression may erase the entropy signature and defeat the monitor—a testable weakness.
  • The same activation-entropy recipe could plausibly flag natural corruptions and domain shifts, not just adversarial attacks, but the paper does not yet demonstrate that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript proposes a non-invasive adversarial-input detector for pre-trained CNNs based on Shannon entropy of ReLU activations at two VGG-16 layers: the first convolutional layer (features.0) and the second fully-connected layer (classifier.3). Entropies are computed batch-wise from flattened activations using hand-designed adaptive binning, and detection thresholds are chosen on 18 clean and 20 adversarial training batches and evaluated on 5 clean and 5 adversarial held-out batches. The paper reports 90% detection accuracy with 0% false positives at the conv layer, 80% accuracy at the FC layer, and claims that adversarial inputs 'consistently shift activation entropy by 7%,' with 'complete separation between clean and adversarial entropy distributions.' The central conclusion is that CNNs inherently encode distribution shifts in their activation entropy.

Significance. If the claims were correct, the approach would be practically attractive because it requires no retraining, no architectural modification, and only forward-hook activation reads. The implementation is concrete enough to reproduce, and the authors are candid in §VII.C about the narrow scope (FGSM only, VGG-16 only, batch-wise decisions). However, the headline claims are undermined by internal contradictions and by the small, calibration-fitted evaluation. The stated decision rule is opposite to the reported early-layer entropy shift; 'complete separation' is contradicted by the paper's own overlap range and by a 20% false-negative rate; and the bin edges and thresholds are explicitly optimized on the same data that defines the separation. The held-out test set is only ten batches, so the reported 90% accuracy is nine correct batch-level decisions. As presented, the evidence does not support the paper's central claim that CNNs 'inherently encode distribution shifts' in activation entropy.

major comments (3)
  1. [§IV.D and §VI.D] The detection rule is contradicted by the reported entropy direction. §IV.D states: 'if the computed entropy S_test for a test batch falls below the threshold τ, the batch is classified as adversarial.' For the first convolutional layer, §VI.D reports clean entropy in 5.05–5.12 bits and adversarial entropy in 5.14–5.20 bits, with τ*=5.1200. Under the stated rule, many or all clean batches would fall at or below the threshold and be flagged adversarial, making the reported 0% false-positive rate impossible at that layer. A rule of S_test > τ would be needed for the early layer, but then the reported 20% FNR is inconsistent with the stated adversarial range. Additionally, §III.B assumes layer-wise entropies 'should generally increase' for out-of-distribution inputs, but §VI.E reports a decrease at the second FC layer. These contradictions mean the threshold, decision rule, and reported dis
  2. [Abstract, §VI.D, Table II] The abstract claims 'complete separation between clean and adversarial entropy distributions,' but §VI.D explicitly describes 'minimal overlap between distributions (occurring only in the narrow range of 5.12-5.14 bits),' and Table II reports FNR=20% at the first convolutional layer. A 20% false-negative rate means at least one adversarial test batch fell on the clean side of the threshold, so the distributions are not completely separated. The '7% shift' in the abstract and §VII.A is also inconsistent with the numbers in §VI.D: 0.07 bits relative to a baseline of about 5.1 bits is roughly 1.4%, not 7%; §VI.D itself reports '35% relative shift' relative to the 0.20-bit total range. These are different quantities, and the abstract's 7% is not derivable from the presented data.
  3. [§IV.C, §V.C, §IV.D] The detector is fitted, not independently predictive. Algorithm 2 states that bin edges are 'optimized to maximize discrimination between clean and adversarial entropy distributions,' and §IV.D selects τ* by minimizing FNR+FPR on the training/validation batches. Because the entropy measurements themselves depend on bin edges chosen to separate the two classes, reporting 'complete separation' as evidence that CNNs inherently encode distribution shifts is circular. The evaluation uses only 5 clean and 5 adversarial test batches, so the headline 90% accuracy is 9/10 batch-level decisions and carries large sampling uncertainty. The paper should reframe the claim as a small fitted demonstration and validate on many more batches, multiple architectures, and additional attack types before making the 'inherently encode' assertion.
minor comments (5)
  1. [Abstract / Table II] The abstract says false positive and false negative rates are 'below 20%,' but Table II reports FNR=20% at both layers and FPR=20% at the FC layer. Use 'at most 20%' or update the numbers.
  2. [§I.C] The introduction states 'By analyzing over 50,000 ImageNet images across multiple architectures...' and claims '75-120% higher sensitivity than confidence-based measures.' The experimental section uses 368 images on VGG-16 only, and no confidence-based baseline comparison is reported. These statements should be removed or supported by the experiments in the paper.
  3. [§V.B] The FGSM equation is missing: the text reads 'Fast Gradient Sign Method (FGSM) [3]: where x represents...' with no displayed formula. Please insert the actual equation.
  4. [Fig. 3 / Fig. 4] The captions say 'across monitored CNN layers' for both figures, but the text indicates Fig. 3 is the first convolutional layer and Fig. 4 is the second FC layer. Make the captions layer-specific.
  5. [§IV.B, Algorithm 1] Algorithm 1 applies MAX(0, A_l_plus) after flattening. The text says 'only positive activations are retained, corresponding to post-ReLU values.' If both monitored layers are already post-ReLU, clarify why the ReLU mask is applied, and specify whether features.0 and classifier.3 are indeed post-activation in the PyTorch hooks.

Circularity Check

2 steps flagged · score 6.0 of 10

Binning and thresholds are fit to maximize the clean/adversarial separation that is then reported as an inherent CNN signature; held-out evaluation is tiny and internally contradicted.

  1. fitted input called prediction [Section V.C (Adaptive Binning), Algorithm 2; results in Section VI.D]
    "We employ non-uniform bin sizes optimized to maximize discrimination between clean and adversarial entropy distributions. The bin edges are empirically determined through analysis of activation value distributions demonstrated in Algorithm 2."

    The entropy values used to report the '7% shift' and 'complete separation' are computed from histograms whose bin edges were explicitly optimized to maximize the same clean-vs-adversarial separation. Thus the observed separation in Figures 3-4 is partly the objective of the bin-fitting procedure, not an independent measurement of an inherent activation signature. The abstract's conclusion that 'CNNs inherently encode distribution shifts in their activation patterns' therefore rests on a fitted preprocessing choice. The held-out test batches provide some independent evidence, but the headline separation magnitude is not a prediction; it is the optimization target.

  2. fitted input called prediction [Section IV.D decision rule vs Section VI.D ranges and Table II]
    "if the computed entropy S_test for a test batch falls below the threshold τ, the batch is classified as adversarial; otherwise, it is classified as clean. ... The threshold value of 5.1200 bits effectively separates clean and adversarial distributions at this layer. ... adversarial samples ... values ranging from 5.14 to 5.20 bits"

    At the early convolutional layer, the paper's own numbers put clean entropy in 5.05-5.12 and adversarial in 5.14-5.20, while the rule says 'below τ = adversarial' with τ = 5.1200. That combination would classify nearly all clean batches as adversarial, making the reported 0% FPR impossible. The reported 90% detection and 0% FPR therefore do not follow from a coherent measured entropy shift; they are artifacts of a threshold fitted to label the training batches as separated. This is the fitted-input-called-prediction pattern: the threshold is the fitted parameter, and the claimed separation is the fitting outcome rather than a validated directional signature.

full rationale

The central empirical claim is partially circular. The binning used to define activation entropy is openly optimized to maximize clean-vs-adversarial discrimination (V.C), and the same separation is then presented as evidence that CNNs 'inherently encode distribution shifts.' A held-out test of 5+5 batches is reported, which prevents this from being fully definitional, but the sample is tiny and the decision rule in IV.D is inconsistent with the early-layer entropy ordering in VI.D (adversarial entropy is higher, yet 'below threshold' is called adversarial). This internal contradiction, together with the acknowledged FNR=20% and the 'minimal overlap' in 5.12-5.14 that contradicts the abstract's 'complete separation,' shows the reported signature is a fitted artifact rather than a stable natural law. The self-citations to the authors' Entanglement Learning framework (refs [28], VII.D, VIII) are rhetorical and not load-bearing; they do not supply any step of the derivation. Overall, the 90% detection figure is a small held-out result, but the headline separation and 7% shift reduce substantially to the bin/threshold fitting procedure, so partial circularity is present. Score 6.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on substantial fitted calibration: hand-tuned bin edges, validation-tuned thresholds, and baseline distributions from 288 images. The paper also assumes FGSM on VGG-16 is representative, and that batch-level aggregation will transfer to deployment. No independent theory or external benchmark supports the generality.

free parameters (4)
  • Adaptive bin edges (features.0, classifier.3) = features.0: 0.0..7.0 with 39 edges; classifier.3: 0.0..7.0 with 23 edges (Algorithm 2)
    Algorithm 2 states bin edges are "empirically determined" and "optimized to maximize discrimination" between clean and adversarial entropy; entropy values and separation depend directly on these hand-chosen edges.
  • Detection thresholds tau* = 5.1200 (conv), 4.1800 (FC)
    Section IV.D selects tau* by minimizing FPR+FNR on validation data; the resulting accuracies are reported on a 10-batch test set, so the threshold is a fitted parameter.
  • Baseline entropy distributions and validation set = 18 clean batches (288 images) and 20 adversarial batches (320 images)
    Baselines and validation statistics are estimated from the same small ImageNet subset; the detector is calibrated to these batches.
  • Batch size and monitored layers = M=16; layers features.0 and classifier.3
    Entropy is computed per batch, so changing batch size changes histograms and thresholds; the two monitored layers are chosen by hand without a systematic search.
assumptions (6)
  • domain assumption Shannon entropy computed from flattened, ReLU-masked, adaptively binned activations approximates the network's true information flow.
    Section III.A defines H(X) via arbitrary binning; the "entropy signature" is only as meaningful as the binning.
  • domain assumption Layer-wise entropies systematically deviate and generally increase on out-of-distribution inputs.
    Section III.B: "We further assume that the layer-wise entropies should generally increase in the event of out-of-distribution inputs." This assumption is used to justify the detector.
  • domain assumption VGG-16 trained on ImageNet is representative of CNNs in general.
    The abstract concludes "CNNs inherently encode distribution shifts" from a single architecture; Section VII.C admits broader architectures are untested.
  • domain assumption FGSM with eps=0.2 is representative of adversarial perturbations.
    Section VII.C limits validation to FGSM and calls for PGD, C&W, and patch attacks; the generality of the entropy signature is assumed.
  • domain assumption Batch-level aggregation produces a per-batch detector that transfers to per-image or streaming deployment.
    Section VII.C: "current batch-wise decision framework needs extension to per-frame streaming thresholds."
  • domain assumption The 368-image ImageNet subset is sufficient to establish stable baseline distributions.
    Section V.A uses 18 batches for baselines and 5+5 test batches; no confidence intervals are given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Entropy-Based Non-Invasive Reliability Monitoring of Convolutional Neural Networks." pith.science (2026). https://pith.science/paper/3XHSQ3NU

@misc{pith2026250821715,
  author       = {Pith},
  title        = {Pith review of: Entropy-Based Non-Invasive Reliability Monitoring of Convolutional Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3XHSQ3NU}},
  note         = {Machine review of arXiv:2508.21715}
}
read the original abstract

Convolutional Neural Networks (CNNs) have become the foundation of modern computer vision, achieving unprecedented accuracy across diverse image recognition tasks. While these networks excel on in-distribution data, they remain vulnerable to adversarial perturbations imperceptible input modifications that cause misclassification with high confidence. However, existing detection methods either require expensive retraining, modify network architecture, or degrade performance on clean inputs. Here we show that adversarial perturbations create immediate, detectable entropy signatures in CNN activations that can be monitored without any model modification. Using parallel entropy monitoring on VGG-16, we demonstrate that adversarial inputs consistently shift activation entropy by 7% in early convolutional layers, enabling 90% detection accuracy with false positives and false negative rates below 20%. The complete separation between clean and adversarial entropy distributions reveals that CNNs inherently encode distribution shifts in their activation patterns. This work establishes that CNN reliability can be assessed through activation entropy alone, enabling practical deployment of self-diagnostic vision systems that detect adversarial inputs in real-time without compromising original model performance.

Figures

Figures reproduced from arXiv: 2508.21715 by the authors.

Figure 1
Figure 1. VCNN monitoring framework: activations from the first convolutional layer (feature maps) and the second fully connected layer (neurons) are extracted to compute entropy and mutual information. These information metrics provide non￾invasive profiling of feature transformations across the network, enabling early detection of deviations in representation quality. Information metrics are calculated asynchronously from t… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    A^4D is a classifier- and attack-agnostic zero-shot adversarial attack detector based on CLIP embedding shifts that claims SOTA performance.

  2. A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    A^4D detects adversarial attacks in an attack- and classifier-agnostic way by measuring non-arbitrary shifts in CLIP embedding space from prompt-based similarity scores.

  3. The Informational Cost of Agency: A Bounded Measure of Interaction Efficiency for Deployed Reinforcement Learning

    cs.AI 2026-03 unverdicted novelty 6.0 of 10

    Bipredictability P measures shared predictability in RL interaction loops with a universal upper bound of 0.5 that agency suppresses to 0.33, operationalized via an Information Digital Twin for faster detection of cou...

  4. The Informational Cost of Agency: A Bounded Measure of Interaction Efficiency for Deployed Reinforcement Learning

    cs.AI 2026-03 unverdicted novelty 5.0 of 10

    Introduces Bipredictability P with a provable bound P ≤ 0.5 from entropy subadditivity, showing responsive agency imposes an informational cost by suppressing P to ~0.33, validated across RL agents and other systems, ...

Reference graph

Works this paper leans on

27 extracted references · 19 canonical work pages · cited by 2 Pith papers

  1. [1]

    These information metrics provide non-invasive profiling of feature transformations across the network, enabling early detection of deviations in representation quality

    VCNN monitoring framework: activations from the first convolutional layer (feature maps) and the second fully connected layer (neurons) are extracted to compute entropy and mutual information. These information metrics provide non-invasive profiling of feature transformations across the network, enabling early detection of deviations in representation qua...

    work page 2012

  2. [2]

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. In International Conference on Learning Representations

    work page 2014

  3. [3]

    Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations

    work page 2019

  4. [4]

    Ovadia, Y., et al. (2019). Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems

    work page 2019

  5. [5]

    W., et al

    Koh, P. W., et al. (2021). WILDS: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning

    work page 2021

  6. [6]

    Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning

    work page 2016

  7. [7]

    Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural networks. In International Conference on Machine Learning

    work page 2015

  8. [8]

    Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems,

    work page 2018

Show all 27 references
  1. [9]

    Towards deep learning models resistant to adversarial attacks

    Madry, Aleksander, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. "Towards deep learning models resistant to adversarial attacks." arXiv preprint arXiv:1706.06083 (2017)

    2017 arXiv

  2. [10]

    Unlike existing methods, our framework operates in parallel to the original network, requiring no architectural modifications, retraining, or multiple forward passes

    [11], our approach measures entropy and mutual information between network layers, capturing how information propagates through the model during inference. Unlike existing methods, our framework operates in parallel to the original network, requiring no architectural modificat...

    2017

  3. [11]

    Energy-based out-of-distribution detection

    Liu, Weitang, Xiaoyun Wang, John Owens, and Yixuan Li. "Energy-based out-of-distribution detection." Advances in neural information processing systems 33 (2020): 21464-21475

    2020

  4. [12]

    Shwartz-Ziv, R., & Tishby, N. (2017). Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810

    2017 arXiv

  5. [13]

    R., Shintre, S., & Gardner, A

    Feinman, R., Curtin, R. R., Shintre, S., & Gardner, A. B. (2017). Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410

    2017 arXiv

  6. [15]

    Sun, Y., Ming, Y., Zhu, X., & Li, Y. (2021). Out-of-distribution detection with deep neural networks: A survey and experimental evaluation. arXiv preprint arXiv:2110.11334

    2021 arXiv

  7. [16]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. "Simple and scalable predictive uncertainty estimation using deep ensembles." Advances in neural information processing systems 30 (2017)

    2017

  8. [17]

    Evidential deep learning to quantify classification uncertainty

    Sensoy, Murat, Lance Kaplan, and Melih Kandemir. "Evidential deep learning to quantify classification uncertainty." Advances in neural information processing systems 31 (2018)

    2018

  9. [18]

    Predictive uncertainty estimation via prior networks

    Malinin, Andrey, and Mark Gales. "Predictive uncertainty estimation via prior networks." Advances in neural information processing systems 31 (2018)

    2018

  10. [21]

    Benchmarking bayesian deep learning on diabetic retinopathy detection tasks

    Band, Neil, Tim GJ Rudner, Qixuan Feng, Angelos Filos, Zachary Nado, Michael W. Dusenberry, Ghassen Jerfel, Dustin Tran, and Yarin Gal. "Benchmarking bayesian deep learning on diabetic retinopathy detection tasks." arXiv preprint arXiv:2211.12717 (2022)

    2022 arXiv

  11. [22]

    Deep learning and the information bottleneck principle

    Tishby, Naftali, and Noga Zaslavsky. "Deep learning and the information bottleneck principle." In 2015 ieee information theory workshop (itw), pp. 1-5. Ieee,

    2015

  12. [23]

    Estimating information flow in deep neural networks

    Goldfeld, Ziv, Ewout van den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, and Yury Polyanskiy. "Estimating information flow in deep neural networks." arXiv preprint arXiv:1810.05728 (2018)

    2018 arXiv

  13. [24]

    Scalable mutual information estimation using dependence graphs

    Noshad, Morteza, Yu Zeng, and Alfred O. Hero. "Scalable mutual information estimation using dependence graphs." In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2962-2966. IEEE,

    2019

  14. [25]

    The information bottleneck problem and its applications in machine learning

    Goldfeld, Ziv, and Yury Polyanskiy. "The information bottleneck problem and its applications in machine learning." IEEE Journal on Selected Areas in Information Theory 1, no. 1 (2020): 19-38

    2020

  15. [26]

    Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379-423

    1948

  16. [28]

    (2025, May)

    Hafez, W., Nazeri, A., Reid, C., & Elshami, S. (2025, May). Entanglement Learning: An Information-Theoretic Framework for Adaptive Convolutional Neural Networks. In 2025 IEEE Conference on Artificial Intelligence (CAI) (pp. 777-782). IEEE

    2025

  17. [2016]

    arXiv preprint arXiv:1610.02136

    A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136

    arXiv

  18. [2017]

    arXiv preprint arXiv:1702.04267

    On detecting adversarial perturbations. arXiv preprint arXiv:1702.04267

    arXiv

  19. [2019]

    Journal of Statistical Mechanics: Theory and Experiment, 2019(12), p.124020

    On the information bottleneck theory of deep learning. Journal of Statistical Mechanics: Theory and Experiment, 2019(12), p.124020

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.