Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Gibbs randomness-compression proposition

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a lossy compression process is equivalent to directed randomness that preserves information content, and it supports the claim with a theorem and experiments on iterative neural-network pruning.

desk verdict The DTC pruning method is a genuinely new trick with decent empirical promise, but the Gibbs-entropy proposition is unsupported and should not be cited as established. read the letter →

arxiv 2505.23869 v5 pith:YUGIEL2D submitted 2025-05-29 stat.ML cs.LG

M Süzen This is my paper
classification stat.MLcs.LG MSC 68T0794A1794A29
keywords Gibbsentropylossycompressionrandomnessneuralnetworkpruningcompressedsensingdualtomographictrain-compresslotterytickethypothesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish a formal link between randomness and lossy compression: it claims that an iterative compression process, when each step changes performance and entropy only slightly, is equivalent to 'directed randomness' that preserves information content. The connection is made through Gibbs entropy computed on the measurement vectors produced by a new neural-network pruning scheme called Dual Tomographic Compression (DTC). The author argues that if this proposition holds, the Gibbs entropy of a learned model's measurement vectors becomes a computable proxy for how much compression the model can tolerate. Supporting evidence is a set of MNIST experiments in which test accuracy and Gibbs entropy move together over compress-train cycles, with Pearson correlations of 0.9174 (DTC) and 0.9412 (random pruning). A sympathetic reader would care because the proposition would turn an abstract statistical-mechanics analogy into a quantitative tool for guiding compression-aware training.

What carries the argument

The carrying objects are (1) the measurement vector $y_i$, obtained by an inverse compressed-sensing step (called 'inverse' because the weights are already known and a hypothetical measurement is generated from them), and (2) the Gibbs entropy $G_i = -\sum_k p_k \log_2 p_k$ computed from the normalized histogram of $y_i$. In DTC, weight rays $w_r^p + w_r^q$ are formed by dual $\ell^1$-minimization reconstructions from previous and next layers, then neurons are clipped at quantiles corresponding to the target sparsity; the entropy of the measurement vectors evolves across cycles and is compared with the pruned network's test accuracy. The proof of Theorem 5.1 is a logical argument that small per-cycle changes make both functions non-increasing and comonotonic, hence highly correlated.

What would settle it

Recompute the Pearson correlation between $f_i$ and $G_i$ using only the early compression cycles in which test accuracy rises; if the high correlation disappears or becomes negative, the comonotonic claim fails. A second check is to find a compress-train run where performance improves substantially while Gibbs entropy keeps falling, which would directly contradict the proposition.

Watch

Extended reading notes

Core claim

On the paper's own terms, the core claim is Theorem 5.1: for a sequential lossy compression process with small per-cycle changes in learning performance ($|f_i - f_{i-1}| < \varepsilon_i$) and Gibbs entropy ($|G_i - G_{i-1}| < \delta_i$), the functions $f_i(s_i)$ and $G_i(s_i)$ are highly correlated. In the abstract this is stated more strongly: 'a lossy compression process is equivalent to directed randomness that preserves information content.' The mechanism is that each compression cycle produces a measurement vector $y_i$ from the network weights through a randomized algorithm; normalizing the histogram of $y_i$ gives a distribution whose Gibbs entropy $G_i$ decreases as complexity is removed, while the network's test performance $f_i$ serves as the probe of how much information survived. The paper reads the high empirical correlations as confirmation that these two quantities are comonotonic: they move together monotonically across compression levels.

Load-bearing premise

The proof requires that the network's test performance never increases from one compression cycle to the next, but the paper's own Table 1 shows accuracy rising from 0.9663 at 10% sparsity to 0.9735 at 41% sparsity in the early DTC cycles, so this monotonicity premise is not satisfied by the reported data.

Editorial extensions

If this is right

  • If the proposition holds, the Gibbs entropy of a model's measurement vectors is a computable proxy for how much compression the model can tolerate without retraining a separate evaluator.
  • DTC's train-compress procedure can prune whole neurons during training, so sparse subnetworks are reached faster than full-training-then-pruning pipelines, supporting an accelerated version of the lottery ticket hypothesis.
  • Random pruning showing a higher correlation than DTC suggests the randomized mechanism itself, not the tomographic reconstruction, drives the entropy-performance link.
  • Because the theorem applies layer-by-layer to any hidden layer mapped to weight rays, the same entropy-based probe could transfer to convolutional or transformer-style layers.
  • The inverse compressed-sensing formulation generalizes classic compressed sensing to settings where the signal is already known, so hypothetical measurements can be generated from any learned weight structure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own Table 1 shows test accuracy rising in the early compression cycles (DTC from 0.9663 at 10% sparsity to 0.9735 at 41% sparsity), so the theorem's monotonicity premise is violated on its reported data; the high whole-trajectory correlations likely reflect the shared downward trend rather than a per-step comonotonic law.
  • A sharper test of the proposition would compute per-cycle correlations between the stepwise changes in performance and entropy; the proposition predicts these stepwise changes stay proportional even when the global trend is non-monotonic.
  • The reported Pearson correlations would be more convincing against a trivial baseline: correlation with sparsity or model size alone, which would also be very high on these runs.
  • Probing over-parameterized networks where pruning improves generalization (test accuracy rising while entropy falls) would provide a strong out-of-sample check of the equivalence between compression and directed randomness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a 'Gibbs randomness-compression proposition' claiming that lossy compression is equivalent to directed randomness that preserves information content, formalized through a claimed high correlation between learning performance f_i and Gibbs entropy G_i of measurement vectors over iterative compress-train cycles. It introduces Dual Tomographic Compression (DTC), an inverse compressed-sensing based neuron-pruning method, and tests it on an MNIST MLP against magnitude and random pruning. The central theoretical statement is Theorem 5.1 in Section 5.1, and the empirical support is a reported Pearson correlation of 0.9174 (DTC) and 0.9412 (random pruning) between performance and entropy over sparsity levels.

Significance. If the proposition were sound, it would offer a computable statistical-mechanics proxy for how much compression a trained model can tolerate, which would be a notable conceptual bridge between deep-learning pruning and information theory. The DTC method is concrete and the paper provides Jupyter notebooks for reproducibility, which is commendable. However, the central theorem is not proved, and the empirical correlation is confounded by the common dependence of both quantities on the sparsity schedule. As it stands, the paper does not establish the proposed relationship between randomness and compression.

major comments (4)
  1. [Section 5.1, proof of Theorem 5.1] The proof asserts that f_i and G_i are 'non-increasing monotonically (comonotonic) due to quasi-static decreases' and concludes that they are 'highly correlated.' Neither step is valid: small successive changes |f_i-f_{i-1}|<ε_i and |G_i-G_{i-1}|<δ_i do not imply monotonicity, and monotonicity does not imply high Pearson correlation. For example, with s_i evenly spaced, f_i=s_i and G_i=s_i^10 are comonotonic with arbitrarily small successive changes as M grows, yet their correlation is about 0.66. More importantly, Table 1 contradicts the monotonicity premise: DTC test accuracy rises from 0.9663 at 10.16% sparsity to 0.9735 at 41.41% sparsity before declining, and similar early rises occur for magnitude and random pruning. The theorem's premise fails on the paper's own data, and the conclusion does not follow from the stated assumptions.
  2. [Section 5 and Abstract] The proposition's stronger statement that 'a lossy compression process is equivalent to directed randomness that preserves information content' is never given a precise meaning. No information-theoretic equivalence, invariance, or converse is proved; the only formal statement is a correlation claim, and 'directed randomness' is not defined beyond the informal description. The abstract and Section 7 therefore overstate what Theorem 5.1 and the experiments establish.
  3. [Algorithm 1, steps 5-10, and Section 6.2] The Gibbs entropy G_i is computed on the same measurement vectors y_j that the DTC procedure uses to rank and clip neurons, and f_i is the test accuracy of the resulting pruned network. Both sequences are therefore functions of the same pruning decisions and of the sparsity schedule s_i. The reported Pearson correlations (0.9174 for DTC, 0.9412 for random pruning) may largely reflect this common dependence on sparsity rather than a general relationship between randomness and compression; the random-pruning comparison does not control for this confound because both f_i and G_i decline with sparsity. Without a control or a partial-correlation analysis, the empirical evidence does not discriminate the proposition from a trivial shared-trend effect.
  4. [Section 6.2] The evidence for the proposition consists of two Pearson correlations, but the paper does not state over how many points they are computed, whether they are correlations between f_i and G_i across cycles or as functions of s_i, or what the uncertainty is (no multiple seeds, confidence intervals, or significance tests are reported). Given the small number of compression cycles and the strong monotone trends in both quantities, these point estimates are insufficient to support 'an almost perfect relationship.'
minor comments (4)
  1. [Section 5.1] There is a typo: 'We known that functions' should read 'We know that functions.'
  2. [Table 1 and Figures 1-5] The quantity labeled 'Sparsity' in Table 1 is plotted as 'Percent Neurons removed' in the figures; the paper should state explicitly whether these values are the fraction removed or the fraction remaining, since the proposition depends on the direction of s_i.
  3. [Section 6.3] Section 6.3 identifies binning, penalty term, and compression frequency as open items; the manuscript should specify the histogram bin count k and the L1 penalty λ used in the experiments, as these are free parameters in the definition of G_i and in DTC.
  4. [Section 4.3] The complexity comparison assumes that full train-compress without DTC requires N·M optimizer calls and that M ≫ L, but this is not argued; the comparison should state the assumptions under which DTC's overhead is lower.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 5.1's proof gap is a non-sequitur, not a reduction of the conclusion to the assumptions, and the cited self-work is background rather than load-bearing.

full rationale

I find no circular step in the claimed derivation chain. The central claim is an empirical correlation between Gibbs entropy G_i over measurement vectors y_j (Algorithm 1, step 10) and learning performance f_i after pruning (step 11), reported as Pearson correlations 0.9174 and 0.9412 for DTC and random pruning. G_i and f_i are not defined in terms of one another: G_i is a histogram entropy of the compressed-sensing measurement vector, while f_i is test accuracy of the pruned network, which additionally depends on training and initialization. Their shared dependence on the same weights and measurement vector does not make high correlation true by construction, since two functions of the same variable can be arbitrarily correlated or uncorrelated. The only proof-like step, Theorem 5.1, is logically invalid rather than circular: it asserts monotonic comonotonicity 'due to quasi-static decreases' without deriving it from the stated small-step assumptions, and Table 1 shows early accuracy increases that contradict monotonicity. An invalid inference is a correctness risk, not a circularity. The self-citations [12,16-18] support background remarks about entropy production and physics-machine-learning interplay, and no uniqueness or external theorem from those works is invoked to force the proposition. The paper also explicitly lists as limitations the need for more experiments, binning studies, and other architectures, which is consistent with an empirically motivated claim rather than a definitional tautology. Therefore no circularity is established, and the score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central proposition rests on several unexamined modeling choices: the histogram binning of measurement vectors, the L1 penalty in the CS reconstruction, the hand-chosen sparsity schedule, and the identification of histogram entropy with directed randomness. None of these is independently grounded or tested for sensitivity. The theorem's monotonicity premise is contradicted by the paper's own Table 1.

free parameters (4)
  • Histogram bin count k for Gibbs entropy
    Gibbs entropy G_i = -sum_k p_k log2 p_k requires discretizing measurement vectors into k bins; k is never reported, yet entropy values and the claimed correlations depend on it.
  • L1 penalty lambda in inverse compressed sensing
    The objective min ||Theta w_r - y||^2 + lambda ||w_r||_1 uses an unstated lambda, and DTC's pruning decisions depend on it.
  • Sparsity schedule s_i = 0.1016, 0.1914, 0.3477, ..., 0.9824
    The sequence of pruning ratios is chosen by hand and reported without derivation; the observed correlations may depend on this schedule.
  • Compression cycle frequency = every 200 batches of size 512
    The length of the train-compress cycle is a hand-chosen hyperparameter; the paper notes it is limited by compute budget and not optimized.
assumptions (4)
  • domain assumption Learning performance f_i(s_i) is non-increasing monotonically during the compress-train cycles.
    Used in the proof of Theorem 5.1; contradicted by Table 1, where accuracy increases up to about 41% sparsity for all three methods.
  • ad hoc to paper The histogram entropy of measurement vectors y = Phi w is a valid measure of 'directed randomness' of the compression process.
    Identified in Section 5 without derivation from SKC algorithmic entropy or from statistical mechanics; no independent justification is given.
  • domain assumption Standard compressed sensing guarantees, such as sparsity or RIP conditions, apply when reconstructing sparse projections of deep learning weight matrices.
    Section 4 applies L1 minimization to weight-layer vectors without verifying sparsity, incoherence, or restricted isometry conditions.
  • domain assumption Learned weights directly affect both f_i and G_i through the same measurement vectors, which mechanistically connects the two.
    Stated in Section 5.1 as 'we known that functions f_i and G_i are mechanistically connected'; this is an assumed causal link, not proven.
invented entities (2)
  • weight rays (w_r)
    purpose: Sparse reconstructed projections of layer weight matrices, used to rank and clip neurons in DTC.
    Introduced in Section 3; it has no falsifiable handle outside the DTC procedure itself and depends entirely on the paper's own definitions.
  • directed randomness
    purpose: Conceptual name for the assumed entropy decline that accompanies information-preserving lossy compression.
    Defined only in the Abstract and Section 5; no independent operational definition is given beyond the post hoc correlation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gibbs randomness-compression proposition." pith.science (2026). https://pith.science/paper/YUGIEL2D

@misc{pith2026250523869,
  author       = {Pith},
  title        = {Pith review of: Gibbs randomness-compression proposition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YUGIEL2D}},
  note         = {Machine review of arXiv:2505.23869}
}
read the original abstract

A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a lossy compression process. In building this connection, we use a performance of a learning task as a probe of compression in iterative compress-train cycles. This can be thought as iterative coarse-graining from statistical mechanics perspective using thermodynamic efficiency as a probe. We formulate this connection via comonotonic relationship within a very small decrease in compression ratio and the performance. We have showcase the validity of this proposition with a canonical vision task in deep learning with three different model compression processes as {\it a baseline model}. We use the following, simpler to more complex model compression approaches: (1) random pruning,(2) magnitude pruning, and (3) a more complex compression by using dual tomographic compression, which utilizes compressed sensing in dual fashion which is introduced as a new method. We use remaining weights of deep learning network as a measurement vector where we measure the Gibbs entropy. We show case the idea that there is an inherent computable connection between compression probed by performance and randomness from an entropy measure on the learned model.

Figures

Figures reproduced from arXiv: 2505.23869 by the authors.

Figure 1
Figure 1. DTC accuracy over different sparsity levels in trai [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. DTC accuracy over different sparsity levels in trai [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Weight-rays, total of reconstructed weight over d [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Previous layer Gibbs entropy of computed measure [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Next layer Gibbs entropy of computed measurements [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Wigner Cat Phases: A finely tunable system for exploring the transition to quantum chaos

    quant-ph 2025-12 reject novelty 3.0 of 10

    Repeatedly padding eigenvalues from mixed-size GOE matrices produces heavy-tailed level statistics that the paper labels 'Wigner Cat Phases.'

Reference graph

Works this paper leans on

55 extracted references · 52 canonical work pages · cited by 1 Pith paper

  1. [1]

    R. J. Solomonoff, A formal theory of inductive inference, Parts I and II, Information and Con- trol 7 (1964) 1–22, 224–254. doi:10.1016/S0019- 9958(64)90223-2. URL https://doi.org/10.1016/S0019-9958(64)90223-2

    doi:10.1016/s0019- 1964

  2. [2]

    A. N. Kolmogorov, Three approaches to the quantitative definition of information, Problems of Information Transmission 1 (1) (1965) 1– 7, english translation: Problems of Information Transmission, 1, 1–7, 1965

    work page 1965

  3. [3]

    G. J. Chaitin, On the length of programs for computing finite binary sequences, Journal of the ACM 13 (1966) 547–569

    work page 1966

  4. [4]

    G. J. Chaitin, Randomness and mathematical proof, Scientific American 232 (5) (1975) 47–53

    work page 1975

  5. [5]

    C. H. Bennett, The thermodynamics of compu- tation—a review, International Journal of The- oretical Physics 21 (1982) 905–940

    work page 1982

  6. [6]

    W. H. Zurek, Algorithmic randomness and phys- ical entropy, Physical Review A 40 (1989) 4731– 4751

    work page 1989

  7. [7]

    Machta, Entropy, information, and computa- tion, American Journal of Physics 67 (12) (1999) 1074–1077

    J. Machta, Entropy, information, and computa- tion, American Journal of Physics 67 (12) (1999) 1074–1077

    work page 1999

  8. [8]

    E. T. Jaynes, Gibbs vs. Boltzmann entropies, American Journal of Physics 33 (5) (1965) 391– 398

    work page 1965

Show all 55 references
  1. [9]

    E. T. Jaynes, Information theory and statistical mechanics, Physical Review 106 (4) (1957) 620

    1957

  2. [10]

    Buonsante, R

    P. Buonsante, R. Franzosi, A. Smerzi, On the dispute between Boltzmann and Gibbs entropy, Annals of Physics 375 (2016) 414–434

    2016

  3. [11]

    Rondoni, E

    L. Rondoni, E. Cohen, Gibbs entropy and ir- reversible thermodynamics, Nonlinearity 13 (6) (2000) 1905

    2000

  4. [12]

    Süzen, Understanding entropy production via a thermal zero-player game, arXiv preprint arXiv:2503.03769 (2025)

    M. Süzen, Understanding entropy production via a thermal zero-player game, arXiv preprint arXiv:2503.03769 (2025). 8

    2025 arXiv

  5. [13]

    Sherrington, S

    D. Sherrington, S. Kirkpatrick, Solvable model of a spin-glass, Physical review letters 35 (26) (1975) 1792

    1975

  6. [14]

    Hopfield, Neural networks and physical sys- tems with emergent collective computational abilities., Proceedings of the national academy of sciences 79 (8) (1982) 2554–2558

    J. Hopfield, Neural networks and physical sys- tems with emergent collective computational abilities., Proceedings of the national academy of sciences 79 (8) (1982) 2554–2558

    1982

  7. [15]

    D. H. Ackley, G. E. Hinton, T. J. Sejnowski, A learning algorithm for boltzmann machines, Cognitive science 9 (1) (1985) 147–169

    1985

  8. [16]

    Süzen, Cooperative phenomenon: Statistical physics origins of connectionist learning, HAL Open Archive (2022)

    M. Süzen, Cooperative phenomenon: Statistical physics origins of connectionist learning, HAL Open Archive (2022). URL https://hal.science/hal-03650339

    2022

  9. [17]

    Süzen, H-theorem do-conjecture, arXiv preprint arXiv:2310.01458 (2023)

    M. Süzen, H-theorem do-conjecture, arXiv preprint arXiv:2310.01458 (2023)

    2023 arXiv

  10. [18]

    Süzen, In-context learning as a new kind of symbolic-automl: Lyapunov conjecture for cots, HAL Open Science (2024)

    M. Süzen, In-context learning as a new kind of symbolic-automl: Lyapunov conjecture for cots, HAL Open Science (2024). URL https://hal.science/hal-04850283

    2024

  11. [19]

    Compagner, Definitions of randomness, American Journal of Physics 59 (8) (1991) 700– 705

    A. Compagner, Definitions of randomness, American Journal of Physics 59 (8) (1991) 700– 705

    1991

  12. [20]

    O’Neill, PCG: A Family of Sim- ple Fast Space-Efficient Statistically Good Algo- rithms for Random Number Generation (2014)

    Melissa E. O’Neill, PCG: A Family of Sim- ple Fast Space-Efficient Statistically Good Algo- rithms for Random Number Generation (2014)

    2014

  13. [21]

    Chang, B

    W. Chang, B. Fang, X. Yun, S. Wang, X. Yu, Randomness testing of compressed data, arXiv preprint arXiv:1001.3485 (2010)

    2010 arXiv

  14. [22]

    M. Li, P. Vitányi, An introduction to Kol- mogorov complexity and its applications, Vol. 3, Springer, 2008

    2008

  15. [23]

    W. H. Zurek, Algorithmic information content, church—turing thesis, physical entropy, and maxwell’s demon, in: Information Dynamics, Springer, 1991, pp. 245–259

    1991

  16. [24]

    Wagner, The role of randomness in darwinian evolution, Philosophy of Science 79 (1) (2012) 95–119

    A. Wagner, The role of randomness in darwinian evolution, Philosophy of Science 79 (1) (2012) 95–119

    2012

  17. [25]

    D. L. Donoho, Compressed sensing, IEEE Trans- actions on information theory 52 (4) (2006) 1289–1306

    2006

  18. [26]

    E. J. Candès, et al., Compressive sampling, in: Proceedings of the international congress of mathematicians, Vol. 3, Madrid, Spain, 2006, pp. 1433–1452

    2006

  19. [27]

    E. J. Candes, T. Tao, Near-optimal signal recov- ery from random projections: Universal encod- ing strategies?, IEEE transactions on informa- tion theory 52 (12) (2006) 5406–5425

    2006

  20. [28]

    R. G. Baraniuk, Compressive sensing [lecture notes], IEEE signal processing magazine 24 (4) (2007) 118–121

    2007

  21. [29]

    Y. C. Eldar, G. Kutyniok, Compressed sensing: theory and applications, Cambridge University Press, 2012

    2012

  22. [30]

    S. A. Janowsky, Pruning versus clipping in neu- ral networks, Physical Review A 39 (12) (1989) 6600

    1989

  23. [31]

    LeCun, J

    Y. LeCun, J. Denker, S. Solla, Optimal brain damage, Advances in neural information pro- cessing systems 2 (1989)

    1989

  24. [32]

    Shwartz Ziv, Y

    R. Shwartz Ziv, Y. LeCun, To compress or not to compress—self-supervised learning and infor- mation theory: A review, Entropy 26 (3) (2024) 252

    2024

  25. [33]

    Shani, D

    C. Shani, D. Jurafsky, Y. LeCun, R. Shwartz- Ziv, From tokens to thoughts: How llms and humans trade compression for meaning, arXiv preprint arXiv:2505.17117 (2025)

    2025

  26. [34]

    LeCun, Y

    Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444

    2015

  27. [35]

    Schmidhuber, Deep learning in neural net- works: An overview, Neural networks 61 (2015) 85–117

    J. Schmidhuber, Deep learning in neural net- works: An overview, Neural networks 61 (2015) 85–117

    2015

  28. [36]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in Neural Information Processing Systems 30 (2017). 9

    2017

  29. [37]

    Sohl-Dickstein, E

    J. Sohl-Dickstein, E. Weiss, N. Mah- eswaranathan, S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynam- ics, in: International Conference on Machine Learning, PMLR, 2015, pp. 2256–2265

    2015

  30. [38]

    Menghani, Efficient deep learning: A survey on making deep learning models smaller, faster, and better, ACM Computing Surveys 55 (12) (2023) 1–37

    G. Menghani, Efficient deep learning: A survey on making deep learning models smaller, faster, and better, ACM Computing Surveys 55 (12) (2023) 1–37

    2023

  31. [39]

    K. Zhu, F. Hu, Y. Ding, W. Zhou, R. Wang, A comprehensive review of network pruning based on pruning granularity and pruning time per- spectives, Neurocomputing (2025) 129382

    2025

  32. [40]

    Hoefler, D

    T. Hoefler, D. Alistarh, T. Ben-Nun, N. Dryden, A. Peste, Sparsity in deep learning: Pruning and growth for efficient inference and training in neu- ral networks, Journal of Machine Learning Re- search 22 (241) (2021) 1–124

    2021

  33. [41]

    Y. Han, G. Huang, S. Song, L. Yang, H. Wang, Y. Wang, Dynamic neural networks: A survey, IEEE transactions on pattern analysis and ma- chine intelligence 44 (11) (2021) 7436–7456

    2021

  34. [42]

    Tatarnikova, A

    T. Tatarnikova, A. Raskopina, Optimization of neural networks through semi-structured activa- tion pruning, in: 2025 Wave Electronics and its Application in Information and Telecommunica- tion Systems (WECONF), IEEE, 2025, pp. 1–5

    2025

  35. [43]

    Frankle, M

    J. Frankle, M. Carbin, The lottery ticket hypoth- esis: Finding sparse, trainable neural networks, in: International Conference on Learning Repre- sentations, 2018

    2018

  36. [44]

    S. Han, J. Pool, J. Tran, W. Dally, Learning both weights and connections for efficient neu- ral network, Advances in neural information pro- cessing systems 28 (2015)

    2015

  37. [45]

    Learning-compression

    M. A. Carreira-Perpinán, Y. Idelbayev, “Learning-compression” algorithms for neu- ral net pruning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8532–8541

    2018

  38. [46]

    Zimmer, C

    M. Zimmer, C. Spiegel, S. Pokutta, Compression aware training of neural networks using Frank- Wolfe, Mathematical Optimization for Machine Learning: Proceedings of the MATH+ Thematic Einstein Semester 2023 (2025) 137

    2025

  39. [47]

    A. H. Gadhikar, S. Mukherjee, R. Burkholz, Why random pruning is all we need to start sparse, in: International Conference on Machine Learning, PMLR, 2023, pp. 10542–10570

    2023

  40. [48]

    K. Zhen, H. D. Nguyen, F.-J. Chang, A. Mouchtaris, A. Rastrow, Sparsification via compressed sensing for automatic speech recog- nition, in: ICASSP 2021-2021 IEEE Interna- tional Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), IEEE, 2021, pp. 6009– 6013

    2021

  41. [49]

    Y. Wu, M. Rosca, T. Lillicrap, Deep compressed sensing, in: International Conference on Ma- chine Learning, PMLR, 2019, pp. 6850–6860

    2019

  42. [50]

    A. L. Machidon, V. Pejović, Deep learning for compressive sensing: a ubiquitous systems per- spective, Artificial Intelligence Review 56 (4) (2023) 3619–3658

    2023

  43. [51]

    LeCun, L

    Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (2002) 2278–2324

    2002

  44. [52]

    O’Donoghue, E

    B. O’Donoghue, E. Chu, N. Parikh, S. Boyd, Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding, Journal of Optimization Theory and Applications 169 (3) (2016) 1042–1068

    2016

  45. [53]

    S. K. Kumar, On weight initialization in deep neural networks, arXiv preprint arXiv:1704.08863 (2017)

    2017 arXiv

  46. [54]

    Paszke, S

    A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in pytorch, in: NIPS-W, 2017. 10

    2017

  47. [55]

    Diamond, S

    S. Diamond, S. Boyd, CVXPY: A Python- embedded modeling language for convex opti- mization, Journal of Machine Learning Research 17 (83) (2016) 1–5. 11

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.