Pith. sign in

REVIEW 2 major objections 5 minor 43 references

Does the Data Processing Inequality Reflect Practice? On the Utility of Low-Level Tasks

T0 review · 2 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that for any finite number of training samples, a pre-classification linear projection that preserves class separation strictly improves a plug-in classifier that converges to the Bayes-optimal rule — so the data processin

desk verdict Useful theory for why preprocessing can help finite-sample classifiers, but the headline exact-N claim overreaches: the theorems prove it for approximate probabilities without uniform error control. read the letter →

arxiv 2512.21315 v2 pith:BKXGNFNZ submitted 2025-12-24 cs.LG cs.CVstat.ML

Roy Turgeman , Tom Tirer This is my paper
classification cs.LGcs.CVstat.ML MSC 62H3060F0594A15
keywords dataprocessinginequalitylow-leveltasksplug-inclassifierGaussianmixturemodeldimensionalityreductionfinite-sampleclassificationBerry–Esseenapproximationdenoisingandencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes on a genuine puzzle: the data processing inequality says information cannot be increased by processing observations, and for the optimal Bayes classifier this provably means preprocessing never improves accuracy — yet in practice, denoising or encoding images before classifying them routinely does help. The paper's claim is that the inequality describes an idealized, infinite-data classifier, while any classifier built from a finite training set carries estimation error that preprocessing can reduce. In a high-dimensional binary Gaussian model, it proves that a linear projection which preserves the class-separation direction and lowers the dimension strictly decreases the error probability of a plug-in classifier that converges to the Bayes-optimal rule, for every finite sample size and every signal-to-noise ratio. It further shows the relative gain peaks at an intermediate sample size, grows with the amount of dimension reduction and with training imbalance, and is largest when the data are clean and labels scarce. If right, this supplies a rigorous reason low-level tasks help high-level ones and gives quantitative predictions verified on noisy image benchmarks.

What carries the argument

The engine is a semi-orthogonal linear projection A∈R^{k×d} with AA^T=I_k and ∥Aμ∥=∥μ∥: it keeps the one-dimensional signal direction exactly while attenuating all orthogonal directions, so the distance between the class means is preserved but the variance of the estimated means shrinks by the factor k/d. Around it, the paper builds closed-form approximations p̂_x and p̂_z for the plug-in classifier's error before and after processing (Theorems 2 and 4), which turn the comparison into a monotonicity statement about the Q-function's argument — the argument is strictly larger after projection, so the error is strictly smaller. Theorem 3 shows A can be learned from unlabeled data via the leadin

What would settle it

Fix concrete small dimensions (for example d=50, k=25), a modest SNR, and N=1 or 2, and compute the exact plug-in error probabilities before and after the paper's projection by Monte Carlo; if any such configuration shows the exact error after processing equal to or larger than before, the literal finite-N claim fails, since at small d the O(1/√d) approximation error can exceed the gap the proof relies on.

Watch

Extended reading notes

Core claim

Working in a high-dimensional binary Gaussian mixture model with equal class priors, the authors study the plug-in rule that assigns a test point to the class whose estimated mean is nearest. This rule converges to the Bayes-optimal classifier as the training set grows and has the best possible estimation variance at any finite sample size. They construct a linear map A that reduces the dimension from d to k while exactly preserving the norm of the class-mean difference (∥Aμ∥=∥μ∥), so the signal-to-noise ratio S is untouched but the variance of the estimated means drops by the factor k/d. Via high-dimensional central-limit (Berry–Esseen) approximations of the error probabilities before and a

Load-bearing premise

The strict-improvement theorems are proven for the approximate error probabilities p̂_x and p̂_z, which match the true probabilities only to O(1/√d) and O(1/√k) by a generalized Berry–Esseen argument (Theorem 2); the paper gives no uniform bound ensuring those approximation errors are smaller than the p̂_x−p̂_z gap it proves, so the abstract's "any finite N" claim holds in the high-dimensional reading rather than as a literal fixed-d statement.

Editorial extensions

If this is right

  • Preprocessing helps exactly when the classifier is data-limited: the gain is provably positive for every finite N and vanishes as N→∞, placing the data processing inequality at the infinite-sample limit rather than in contradiction with practice.
  • The benefit is achievable without labels: the projection that provably improves classification can be estimated from unlabeled data, so the mechanism is available to self-supervised pipelines.
  • The relative gain is largest at an intermediate training-set size, and its maximum increases with the signal-to-noise ratio — so data processing is most valuable for clean, well-separated data with scarce labels.
  • Imbalanced training sets and stronger dimension reduction (larger d−k) both increase the asymptotic gain, giving testable quantitative predictions for when denoising or encoding will help most.
  • The predicted trends appear in practical settings without distribution shift: denoising and self-supervised encoding on noisy image benchmarks reproduce the non-monotonic efficiency curves and the SNR ordering predicted by the theory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mechanism binds through estimator variance rather than the Gaussian likelihood, so a similar variance-reduction argument should extend to any plug-in classifier whose decision boundary depends continuously on estimated parameters, such as high-dimensional logistic regression or linear discriminant analysis.
  • The paper leaves open the optimal-processing question: among all maps preserving the discriminant direction, is the variance-minimizing linear projection optimal, and can nonlinear denoising beat it at small N by also shrinking bias? The Q-function-argument comparison suggests a direct way to test this.
  • Tasks that need spatial information, such as object detection or segmentation, should show smaller gains from aggressive encoding but larger gains from restoration that preserves geometry — a falsifiable prediction using the same efficiency metric.
  • Because the gain arises from estimation error rather than distribution shift, the authors' conclusion implies out-of-distribution settings should show even larger gains; quantifying that with the same S, N, γ, and d−k parameters is a natural extension of the paper's qualitative remark.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies binary classification under a two-component Gaussian mixture model, with a plug-in nearest-centroid classifier trained on labeled samples and a linear dimensionality-reduction pre-processing A that preserves the class-separation direction. Using a generalized Berry–Esseen argument, the authors derive approximate error probabilities p̂x and p̂z before and after processing, prove that p̂x > p̂z for balanced training data (and under conditions for imbalanced data), analyze the relative efficiency of the processing, and support the theory with simulations and deep-learning experiments on CIFAR-10 and Mini-ImageNet. The central message is that low-level pre-processing can improve a strong finite-sample classifier even though the Bayes-optimal classifier cannot be improved, seemingly contradicting the data-processing inequality.

Significance. If the central claim holds for exact error probabilities, the paper makes a substantive point: finite-sample variance reduction in estimating class means can overcome the data-processing inequality for plug-in classifiers. The derivations are self-contained and parameter-free, and the Berry–Esseen analysis is nontrivial; the simulations match the derived formulas, and the practical experiments exhibit the predicted non-monotone efficiency curves. The main caveat is the gap between the approximate probabilities p̂x,p̂z and the true probabilities px,pz, which currently prevents the abstract's literal finite-N claim from being supported.

major comments (2)
  1. [Abstract; §3.2.1 (Theorem 5, Theorem 2, Eq. (48), Eq. (60), Theorem 10)] The abstract's 'any finite number of training samples' is not established for exact error probabilities. Theorem 5 proves p̂x > p̂z for the approximate probabilities of Theorems 2 and 4, which are only accurate to O(1/√d) and O(1/√k). In the proof of Theorem 2, Eq. (48) sets D = σ⁴/N·((1+γ)/γ + (1+γ²)/(2γ²)·1/N), and Eq. (60) bounds the Berry–Esseen error by C0C/(D^{3/2}√d). For γ=1 this bound grows like N^{3/2}/√d, so the approximation is not uniform in N for fixed d. Theorem 10 (Eq. (169)) gives p̂x−p̂z = Θ(1/N_T) for large N_T. Hence for fixed d,k and large N, the approximation error can dominate the gap, and p̂x>p̂z does not imply px>pz. The assertion is therefore only supported in the high-dimensional limit d,k→∞ with N fixed. Please either provide a uniform bound or restate the theorems as high-dimensional asymptotic statements.
  2. [§3.2.2 (Definition 1, Theorems 7–8)] The efficiency η in Eq. (13) is defined using p̂x and p̂z, and Theorems 7 and 8 analyze this approximate efficiency. The claims about maximal gain and its dependence on S, γ, and N are therefore for the approximate quantities. Without a quantitative bound between p̂ and the exact px,pz, the corresponding statements in the abstract and in Section 3.3 about actual accuracy are not formally proven. Please state clearly which results are proved for exact probabilities and which only for the Berry–Esseen approximations.
minor comments (5)
  1. [Theorem 4] The sentence 'The approximate probability of error of the processed data, p̂x(error), admits an expression similar to the one obtained for the raw data, p̂z(error)' swaps the two terms; it should read p̂z(error) for the processed data and p̂x(error) for the raw data.
  2. [Eq. (10)] The balanced-case expression appears to be Q(√S / sqrt(d/(2NS)+d/(4N²S)+1/N+1)) if specialized from Eq. (9), but Eq. (10) as printed shows √S multiplied by a square-root expression. Please check and correct the typo.
  3. [Theorem 8 proof] In the proof, the notation r = d/k is used in the approximations (133)–(134), but r is introduced only in the theorem statement; please define it before use.
  4. [Theorem 3] The learnability statement claims that A can be learned 'to arbitrary accuracy' from unlabeled samples, but the proof only shows consistency as the number of unlabeled samples m→∞; no rate or finite-sample guarantee is given. Please state this as an asymptotic guarantee rather than a finite-sample one.
  5. [Section 4] The practical experiments are presented as corroborating trends; this is fine, but the reader should be told explicitly that the theory is for a two-class Gaussian setup with a specific linear projection, and that the transfer to deep models is by analogy rather than by theorem.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the theoretical derivation is self-contained and parameter-free.

full rationale

The paper's central claim is derived from a fully specified GMM model (Eq. 3-4) and a plug-in nearest-mean classifier (Eq. 6), with no parameter fitted to the quantity being predicted. Theorems 2 and 4 obtain closed-form approximations p̂_x and p̂_z via a generalized Berry-Esseen argument, and Theorem 5 then proves p̂_x > p̂_z by direct comparison of Q-function arguments; the inequality is a mathematical consequence, not an assumption. The preprocessing matrix A is constructed (Theorem 3) to satisfy AA^T=I_k and ∥Aµ∥=∥µ∥, and the gain shown in Theorem 5 follows from the derived formulas, not from positing the conclusion. Theorem 4 legitimately reuses Theorem 2 with d replaced by k because z=Ax remains a GMM with the same S. The self-citations (e.g., Kothapalli & Tirer 2025, Garber & Tirer 2024/2025) are contextual and not load-bearing; Theorem 1 itself is proved in Appendix A.1. The acknowledged distinction between the approximate probabilities used in the proofs and the exact finite-N claim in the abstract is a rigor/correctness concern about controlling O(1/√d), O(1/√k) errors uniformly in N, not an instance of defining or fitting the target into the input. The empirical sections are external validations, not inputs to the derivation. Overall, the derivation chain is self-contained and contains no step where a prediction is equivalent by construction to a fitted input or to a self-citation.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No constants are fitted to data. S, N, γ, d, and k are model inputs; σ and μ are data-distribution parameters; k is a design choice. The theoretical efficiency η is computed from derived formulas, not calibrated against the experiments before deriving the theorems.

assumptions (5)
  • domain assumption Data follow a two-component Gaussian mixture with equal covariance σ²I_d, equal priors, and symmetric means μ_2=-μ_1 (Eqs. 3-4).
    This defines the entire theoretical scope and makes the Bayes classifier and plug-in nearest-mean classifier tractable.
  • domain assumption The entries of μ are bounded by a universal constant and σ is independent of d.
    Used to make the Berry–Esseen ratio O(1/√d) uniform; excludes models with a few very large or growing entries of μ.
  • standard math Generalized Berry–Esseen theorem (Feller) applies to the independent per-dimension summands y_i.
    This is the backbone of Theorem 2's approximation p̂_x(error), which all later inequalities build on.
  • domain assumption The O(1/√d) and O(1/√k) approximation errors can be ignored when converting p̂ inequalities into claims about exact error probabilities.
    Theorems 5-8 are stated for p̂; the abstract's 'any finite N' claim treats the approximation as exact, but no uniform bound on the error gap is provided.
  • domain assumption The direction μ can be recovered arbitrarily well from unlabeled samples via the top eigenvector of Σ=σ²I+μμᵀ.
    Theorem 3 relies on a spectral gap and on 'sufficiently many' unlabeled samples; finite-sample accuracy of the learned A is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Does the Data Processing Inequality Reflect Practice? On the Utility of Low-Level Tasks." pith.science (2026). https://pith.science/paper/BKXGNFNZ

@misc{pith2026251221315,
  author       = {Pith},
  title        = {Pith review of: Does the Data Processing Inequality Reflect Practice? On the Utility of Low-Level Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKXGNFNZ}},
  note         = {Machine review of arXiv:2512.21315}
}
read the original abstract

The data processing inequality is an information-theoretic principle stating that the information content of a signal cannot be increased by processing the observations. In particular, it suggests that there is no benefit in enhancing the signal or encoding it before addressing a classification problem. This assertion can be proven to be true for the case of the optimal Bayes classifier. However, in practice, it is common to perform "low-level" tasks before "high-level" downstream tasks despite the overwhelming capabilities of modern deep neural networks. In this paper, we aim to understand when and why low-level processing can be beneficial for classification. We present a comprehensive theoretical study of a binary classification setup, where we consider a classifier that is tightly connected to the optimal Bayes classifier and converges to it as the number of training samples increases. We prove that for any finite number of training samples, there exists a pre-classification processing that improves the classification accuracy. We also explore the effect of class separation, training set size, and class balance on the relative gain from this procedure. We support our theory with an empirical investigation of the theoretical setup. Finally, we conduct an empirical study where we investigate the effect of denoising and encoding on the performance of practical deep classifiers on benchmark datasets. Specifically, we vary the size and class distribution of the training set, and the noise level, and demonstrate trends that are consistent with our theoretical results.

Figures

Figures reproduced from arXiv: 2512.21315 by the authors.

Figure 1
Figure 1. The theoretical setup. Efficiency of the data processing procedure versus the number of [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Noisy CIFAR-10 and pre-classification denoising. Efficiency versus [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Noisy Mini-ImageNet and pre-classification encoding. Efficiency versus [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Training and testing error as a function of epochs for (a) noisy data and (b) denoised data. [PITH_FULL_IMAGE:figures/full_fig_p049_4.png]
Figure 5
Figure 5. Figure 5: Practical deep learning setup with noisy CIFAR-10 and SURE-based denoiser. Efficiency [PITH_FULL_IMAGE:figures/full_fig_p050_5.png]
Figure 6
Figure 6. Figure 6: Practical deep learning setup with noisy CIFAR-10 and MSE-based denoiser. Efficiency of [PITH_FULL_IMAGE:figures/full_fig_p051_6.png]
Figure 7
Figure 7. Figure 7: Practical deep learning setup with noisy Mini-ImageNet and pre-classification encoding. [PITH_FULL_IMAGE:figures/full_fig_p052_7.png]
Figure 8
Figure 8. Figure 8: Clean and noisy Mini-ImageNet images. (a) Clean image. (b) Image with Gaussian noise [PITH_FULL_IMAGE:figures/full_fig_p053_8.png]
Figure 9
Figure 9. Figure 9: Noisy CIFAR-10 and pre-classification encoding. Efficiency versus Ntrain [PITH_FULL_IMAGE:figures/full_fig_p054_9.png]
Figure 10
Figure 10. Figure 10: The theoretical setup. Efficiency of the data processing procedure versus the number of [PITH_FULL_IMAGE:figures/full_fig_p054_10.png]
Figure 11
Figure 11. Figure 11: Extended simulation of the theoretical setup. Efficiency of the data processing versus the [PITH_FULL_IMAGE:figures/full_fig_p055_11.png]
Figure 12
Figure 12. Figure 12: The theoretical setup. Efficiency of the data processing procedure versus the number of [PITH_FULL_IMAGE:figures/full_fig_p055_12.png]
Figure 13
Figure 13. Figure 13: Extended empirical verification - per Ntrain, A is learned from 50,000 unlabeled examples. Presented for (a) S = 0.752 , (b) S = 1 and (c) S = 1.5 2 . processing by A : R d → R k , the empirical means obey Aµbj ∼ N  Aµj , σ 2 Nj Ik  , where the semi-orthonormality A…
Figure 14
Figure 14. Figure 14: Extended empirical verification - per Ntrain, A is learned from 5,000,000 unlabeled examples. Presented for (a) S = 0.752 , (b) S = 1 and (c) S = 1.5 2 . That is, A preserves the separation quality of the classes, while improving the estimation quality of µ. 57 [PITH…
Figure 15
Figure 15. Figure 15: Visualization of the effect of A : R 2 → R. Note that the empirical means (green diamonds) are closer to the true means (black ‘X’s) after the operation A. The distance between the empirical means of the different classes remains similar. 58 [PITH_FULL_IMAGE:figures/…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 1 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    Dz Co f^ z eOza F zެ ޳

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

  5. [5]

    Pattern recognition and machine learning

    Christopher M Bishop. Pattern recognition and machine learning. Springer, 2006

    2006

  6. [6]

    Risk bounds for over-parameterized maximum margin classification on sub-gaussian mixtures

    Yuan Cao, Quanquan Gu, and Mikhail Belkin. Risk bounds for over-parameterized maximum margin classification on sub-gaussian mixtures. Advances in Neural Information Processing Systems, 34: 0 8407--8418, 2021

    2021

  7. [7]

    Smote: synthetic minority over-sampling technique

    Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16: 0 321--357, 2002

    2002

  8. [8]

    Elements of information theory

    Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999

    1999

Show all 43 references
  1. [9]

    Is image super-resolution helpful for other vision tasks? In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.\ 1--9

    Dengxin Dai, Yujian Wang, Yuhua Chen, and Luc Van Gool. Is image super-resolution helpful for other vision tasks? In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.\ 1--9. IEEE, 2016

    2016

  2. [10]

    A model of double descent for high-dimensional binary linear classification

    Zeyu Deng, Abla Kammoun, and Christos Thrampoulidis. A model of double descent for high-dimensional binary linear classification. Information and Inference: A Journal of the IMA, 11 0 (2): 0 435--495, 2022

    2022

  3. [11]

    An introduction to probability theory and its applications, Volume 2, volume 2

    William Feller. An introduction to probability theory and its applications, Volume 2, volume 2. John Wiley & Sons, 1991

    1991

  4. [12]

    Introduction to statistical pattern recognition

    Keinosuke Fukunaga. Introduction to statistical pattern recognition. Elsevier, 2013

    2013

  5. [13]

    Entropy and mutual information in models of deep neural networks

    Marylou Gabri \'e , Andre Manoel, Cl \'e ment Luneau, Nicolas Macris, Florent Krzakala, Lenka Zdeborov \'a , et al. Entropy and mutual information in models of deep neural networks. Advances in neural information processing systems, 31, 2018

    2018

  6. [14]

    Sliced mutual information: A scalable measure of statistical dependence

    Ziv Goldfeld and Kristjan Greenewald. Sliced mutual information: A scalable measure of statistical dependence. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021

    2021

  7. [15]

    Task-driven super resolution: Object detection in low-resolution images

    Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita. Task-driven super resolution: Object detection in low-resolution images. In Neural Information Processing: 28th International Conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8--12, 2021, Proceedings, Part V 28,...

    2021

  8. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  9. [17]

    Benchmarking neural network robustness to common corruptions and perturbations

    Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019

    2019

  10. [18]

    Learning deep representation for imbalanced classification

    Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang. Learning deep representation for imbalanced classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 5375--5384, 2016

    2016

  11. [19]

    An information-theoretic framework for deep learning

    Hong Jun Jeon and Benjamin Van Roy. An information-theoretic framework for deep learning. Advances in Neural Information Processing Systems, 35: 0 3279--3291, 2022

    2022

  12. [20]

    Fundamentals of statistical signal processing: estimation theory

    Steven M Kay. Fundamentals of statistical signal processing: estimation theory. Prentice-Hall, Inc., 1993

    1993

  13. [21]

    Can kernel methods explain how the data affects neural collapse? Transactions on Machine Learning Research, 2025

    Vignesh Kothapalli and Tom Tirer. Can kernel methods explain how the data affects neural collapse? Transactions on Machine Learning Research, 2025. ISSN 2835-8856

    2025

  14. [22]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  15. [23]

    Fifo: Learning fog-invariant features for foggy scene segmentation

    Sohyun Lee, Taeyoung Son, and Suha Kwak. Fifo: Learning fog-invariant features for foggy scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 18911--18921, 2022

    2022

  16. [24]

    Detection-friendly dehazing: Object detection in real-world hazy scenes

    Chengyang Li, Heng Zhou, Yang Liu, Caidong Yang, Yongqiang Xie, Zhongbo Li, and Liping Zhu. Detection-friendly dehazing: Object detection in real-world hazy scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 0 (7): 0 8284--8295, 2023

    2023

  17. [25]

    Autobalance: Optimized loss functions for imbalanced data

    Mingchen Li, Xuechen Zhang, Christos Thrampoulidis, Jiasi Chen, and Samet Oymak. Autobalance: Optimized loss functions for imbalanced data. Advances in Neural Information Processing Systems, 34: 0 3163--3177, 2021

    2021

  18. [26]

    When image denoising meets high-level vision tasks: A deep learning approach

    Ding Liu, Bihan Wen, Xianming Liu, Zhangyang Wang, and Thomas S Huang. When image denoising meets high-level vision tasks: A deep learning approach. In 27th International Joint Conference on Artificial Intelligence, IJCAI 2018, pp.\ 842--848. International Joint Conferences on...

    2018

  19. [27]

    On the classification-distortion-perception tradeoff

    Dong Liu, Haochen Zhang, and Zhiwei Xiong. On the classification-distortion-perception tradeoff. Advances in Neural Information Processing Systems, 32, 2019

    2019

  20. [28]

    Ditch the denoiser: Emergence of noise robustness in self-supervised learning from data curriculum

    Wenquan Lu, Jiaqi Zhang, Hugues Van Assel, and Randall Balestriero. Ditch the denoiser: Emergence of noise robustness in self-supervised learning from data curriculum. arXiv preprint arXiv:2505.12191, 2025

    2025

  21. [29]

    Does haze removal help cnn-based image classification? In Proceedings of the European conference on computer vision (ECCV), pp.\ 682--697, 2018

    Yanting Pei, Yaping Huang, Qi Zou, Yuhang Lu, and Song Wang. Does haze removal help cnn-based image classification? In Proceedings of the European conference on computer vision (ECCV), pp.\ 682--697, 2018

    2018

  22. [30]

    On the information bottleneck theory of deep learning

    Andrew M Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan D Tracey, and David D Cox. On the information bottleneck theory of deep learning. Journal of Statistical Mechanics: Theory and Experiment, 2019 0 (12): 0 124020, 2019

    2019

  23. [31]

    Opening the black box of deep neural networks via information

    Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810, 2017

    2017 arXiv

  24. [32]

    Dual directed capsule network for very low resolution image recognition

    Maneet Singh, Shruti Nagpal, Richa Singh, and Mayank Vatsa. Dual directed capsule network for very low resolution image recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 340--349, 2019

    2019

  25. [33]

    Training deep learning based denoisers without ground truth data

    Shakarim Soltanayev and Se Young Chun. Training deep learning based denoisers without ground truth data. Advances in neural information processing systems, 31, 2018

    2018

  26. [34]

    Urie: Universal image enhancement for visual recognition in the wild

    Taeyoung Son, Juwon Kang, Namyup Kim, Sunghyun Cho, and Suha Kwak. Urie: Universal image enhancement for visual recognition in the wild. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part IX 16, pp.\ 749--765. Springer, 2020

    2020

  27. [35]

    Estimation of the mean of a multivariate normal distribution

    Charles M Stein. Estimation of the mean of a multivariate normal distribution. The annals of Statistics, pp.\ 1135--1151, 1981

    1981

  28. [36]

    Deep learning and the information bottleneck principle

    Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pp.\ 1--5. Ieee, 2015

    2015

  29. [37]

    Matching networks for one shot learning

    Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016

    2016

  30. [38]

    Binary classification of gaussian mixtures: Abundance of support vectors, benign overfitting, and regularization

    Ke Wang and Christos Thrampoulidis. Binary classification of gaussian mixtures: Abundance of support vectors, benign overfitting, and regularization. SIAM Journal on Mathematics of Data Science, 4 0 (1): 0 260--284, 2022

    2022

  31. [39]

    Denoising masked autoencoders help robust classification

    QuanLin Wu, Hang Ye, Yuntian Gu, Huishuai Zhang, Liwei Wang, and Di He. Denoising masked autoencoders help robust classification. In The Eleventh International Conference on Learning Representations, 2023

    2023

  32. [40]

    A theory of usable information under computational constraints

    Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. A theory of usable information under computational constraints. In International Conference on Learning Representations, 2020

    2020

  33. [41]

    Enhancing the performance of convolutional neural networks on quality degraded datasets

    Jonghwa Yim and Kyung-Ah Sohn. Enhancing the performance of convolutional neural networks on quality degraded datasets. In 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp.\ 1--8. IEEE, 2017

    2017

  34. [42]

    Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising

    Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26 0 (7): 0 3142--3155, 2017

    2017

  35. [43]

    Anomaly detection with robust deep autoencoders

    Chong Zhou and Randy C Paffenroth. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp.\ 665--674, 2017

    2017

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.