Pith. sign in

REVIEW 4 major objections 6 minor 28 references

This paper claims that a graph-curvature score computed from a trained network and a small calibration set ranks connections by functional importance, so curvature can drive pruning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:37 UTC pith:XJ3UXBE4

load-bearing objection A genuinely novel activation-aware curvature score for pruning, but the headline result may be dead-ReLU detection plus inverse-magnitude ranking in disguise; worth refereeing, not yet worth citing. the 4 major comments →

arxiv 2601.16366 v2 pith:XJ3UXBE4 submitted 2026-01-22 cs.LG cs.SC

Post-Training Neural Network Pruning using Graph Curvature

Shuhang Tan , Jayson Sia , Paul Bogdan , Radoslav Ivanov This is my paper
classification cs.LG cs.SC
keywords neural curvatureOllivier-Ricci curvaturepost-training pruningconnection importance rankingactivation-aware curvaturegraph-based data flow analysiscross-layer pruningcalibration data efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that the bottlenecks and redundancies of a trained neural network can be read off from a geometric quantity: the Ollivier-Ricci curvature of a graph whose nodes are neurons and whose edge weights encode both the learned weight magnitudes and the activation values observed on a handful of input examples. Negative-curvature edges, it argues, are the network's critical data-flow bridges—removing them first collapses accuracy—while positive-curvature edges can be pruned in large numbers with little effect. This gives a single cross-layer importance score for every connection, one that avoids the layer-collapse failure of magnitude-based pruning and identifies more removable parameters than existing gradient- or conservation-based pruning baselines. The practical payoff would be a cheap, data-efficient way to build a data-flow map of a trained model, which the paper points toward robustness analysis, model repair, and focusing retraining on underused connections.

Core claim

On its own terms, the paper's central discovery is that Ollivier-Ricci curvature, adapted to be activation-dependent, separates neural connections into two functionally distinct classes: edges whose curvature is negative act as bottlenecks whose removal quickly degrades test accuracy, and edges whose curvature is positive can be removed without immediate harm. The adaptation works by building a 'neural graph' from the network (one node per neuron, one edge per connection, edge cost equal to the inverse weight magnitude), replacing the graph's static neighbor distributions with distributions derived from neuron activations for each calibration example, and scaling edge costs by how much the a

What carries the argument

Neural curvature: the limit, as the alpha-parameter goes to 1, of a modified alpha-Ollivier-Ricci curvature computed on the neural graph, where alpha leaves probability mass on the source and target nodes. The graph's edge costs are inverse weight magnitudes, neighbor distributions are built from min-max normalized, reciprocated, then exponentially smoothed absolute activations, and the edge cost is inflated (to infinity for ReLU dead neurons) according to the saturation of the activation function. This mechanism turns a static structural graph into a per-example data-flow geometry, and taking the minimum over calibration examples gives a global cross-layer importance ranking.

Load-bearing premise

The ranking inherits everything from the chosen graph encoding: edge cost is the inverse absolute weight and neighbor distributions are a hand-built transform of activations, and the paper does not validate that this encoding measures actual information transmission; if a plausible alternative encoding changes the curvature ordering, the bottleneck semantics rest on convention.

What would settle it

Run the same pruning experiment on the same trained models with two edge-cost choices—inverse weight magnitude versus absolute weight magnitude, or a sign-aware cost—and check whether the set of negative-curvature edges and the pruning curves remain approximately the same. If the curvature ranking flips under this substitution, the claim that curvature identifies information-flow bottlenecks is not robust to the encoding; if it does not, the geometric signal is stable. A complementary check is to measure edge-level data transmission directly and test whether inverse-weight edges actually carry

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, pruning a trained network by removing the most positive-curvature edges first should preserve accuracy far longer than removing negative-curvature edges first, as demonstrated across MNIST, CIFAR-10, and CIFAR-100 models with both ReLU and Tanh activations.
  • The curvature score provides a global ranking across layers, so pruning decisions are not distorted by differing weight scales between layers, which is the mechanism the paper gives for avoiding layer collapse.
  • A very small calibration set, on the order of one example per label, appears sufficient to separate important from unimportant edges, making the method practical when data are scarce.
  • The same ranking can serve as a data-flow map for the network: negative-curvature edges identify the paths the model actually relies on, which the paper proposes as a starting point for robustness analysis and model repair.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit: if the edge cost is changed from inverse weight magnitude to, say, absolute weight or a sign-aware cost, the curvature ordering may shift; comparing pruning curves under these weightings would separate the geometric signal from the modelling convention.
  • The negative-edges-as-bottlenecks view suggests an attack-oriented use: adversarial perturbations might be most effective when targeted at edges with the most negative curvature, and defending those edges could be a cheap robustness intervention—this is not tested in the paper.
  • The method's cross-layer normalization is the likely source of its advantage over magnitude pruning; the paper's per-layer ablation shows the two agree within a layer, implying curvature's main added value is not local importance but a common scale across layers.
  • The same curvature machinery could be carried to other architectures, such as transformers or recurrent networks, by defining appropriate neighbor distributions and edge costs; whether the bottleneck semantics survive attention's softmax coupling is an open question.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a graph-curvature-based method for ranking connections in trained feedforward NNs. It constructs a directed weighted neural graph (Def. 5) with edge costs equal to inverse weight magnitudes, defines activation-dependent neighbor distributions (Def. 6), modifies the edge cost to account for activation saturation/deadness (Def. 7), and introduces a "neural curvature" via the limiting α-Ricci curvature (Def. 8). Edges are ranked by the minimum curvature over a small calibration set, and the ranking is evaluated in edge-removal experiments on MNIST, CIFAR-10, and CIFAR-100 against magnitude pruning, SNIP, and SynFlow. The central claim is that negative-curvature edges are bottlenecks and carry the main data flow, while positive-curvature edges are unimportant; the method is claimed to identify more unimportant edges than prior pruning methods.

Significance. If the central claim is robust, this would be a useful contribution: it provides a geometrically motivated, data-dependent importance score that can be computed on layered DAGs with a dynamic-programming shortest-path implementation, and it shows promising results across several architectures, activations, and training schemes. The paper is honest about several limitations, noting comparable performance to magnitude pruning on MNIST and for weight-decayed models, and it includes an ablation of calibration-set size. The method's data efficiency (one example per label is often sufficient) is also a strength. However, the significance is currently conditional: the experimental evidence does not yet separate the proposed curvature signal from simpler signals embedded in the construction, and the notation/definition of the neural distributions is under-specified in a way that affects reproducibility.

major comments (4)
  1. [§7.3.3 and Defs. 5/7/8, Prop. 1] The central claim that negative-neural-curvature edges are bottlenecks is not isolated from simpler confounds. Definition 7 assigns infinite cost to edges incident to ReLU-dead neurons, and Definition 8/Proposition 1 map those edges to curvature 1–2, i.e., the highest values. Since ranking is descending, all ReLU-dead edges are removed first, then edges with small |W| (Def. 5). The ablation in §7.3.3 only compares against static Lin et al. Ricci; it does not include a control that removes ReLU-dead edges first and then prunes by inverse magnitude, nor a control using per-layer normalized magnitude. Section 7.3.2 concedes that within a layer the ordering behaves like magnitude pruning, so the remaining claimed advantage is cross-layer normalization. Without a control that uses activation-deadness and inverse magnitude alone, the abstract's statement that negative-ORC edges are bottlenecks
  2. [Def. 8 and Eq. (7)] The definition of the α-neural distribution is ambiguous. Equation (3) defines m^α_x using graph weights w_n, while Definition 6 defines an activation-based probability vector. The text says m^α_{v_i,l}(x) and m^α_{v_j,l+1}(x) are 'calculated according to Definition 6 and equation 3', but it does not specify how the two are combined. If Equation (3) is used literally with graph weights, the activation data enter only through the edge cost dσ, which would weaken the claimed data dependence. If the Definition 6 vector replaces the weight-proportional distribution in Eq. (3), this must be stated explicitly. This ambiguity affects the implementation of every curvature value and must be resolved before the method can be reproduced.
  3. [Def. 6, Step 1] The per-layer normalization (n_i - n_min)/(n_max - n_min) is undefined whenever n_max = n_min, which occurs not only when all activations are zero (the case discussed in the footnote) but also when all |n_i| in a layer are equal to the same positive constant. The footnote only addresses the all-zero case. If the implementation adds a small constant to the denominator in all cases, that should be stated; otherwise a well-defined fallback (e.g., a uniform distribution) is needed.
  4. [§7.3.2 / Fig. 5] The paper claims that the method's advantage over magnitude pruning comes from 'effective normalization across layers'. This is a load-bearing claim, because within-layer behavior is admitted to match magnitude pruning. However, no experiment directly tests this normalization hypothesis. A natural control is per-layer normalized magnitude pruning (e.g., dividing weights by the maximum |W| in each layer before global ranking). If that control matches the neural-curvature curve, the cross-layer signal reduces to a standard normalization and the geometric interpretation is weakened. Please include this control and, if possible, report the curvature distribution over layers to substantiate the claim.
minor comments (6)
  1. [Title/Abstract] The arXiv title is 'Post-Training Neural Network Pruning using Graph Curvature' while the body title is 'Analyzing Neural Network Information Flow Using Differential Geometry'. Please align the title and the abstract with the paper's actual scope.
  2. [Def. 1] In Eq. (1), d(u',v') is described as 'the cost from u' to v', e.g., d(u,v)=w(u,v)'. For nodes that are not directly connected by an edge, this cost must be a shortest-path distance in the weighted graph. Please state this explicitly, since the implementation section relies on shortest-path distances.
  3. [Sec. 6] The choice α=0.9 is justified only by 'empirical observations'. Please provide a sensitivity analysis or at least a statement of how curvature rankings change for α in {0.5, 0.7, 0.9, 0.99}. This is especially important because the theoretical definition uses the limit α→1.
  4. [Sec. 7.1 / Model description] The text says results are 'equally effective across all considered combinations', but CIFAR-100 is evaluated with only one model (ReLU). Please temper the generalization claim or add additional CIFAR-100 configurations.
  5. [Reproducibility] No code or data are provided. Given the number of ad-hoc choices in the pipeline (normalization, edge costs, α, min aggregation), releasing code would substantially increase confidence in the reported results.
  6. [Example 1] The vector for m_n5 appears to have a typo: '0.2 0.2 0.2 0.2 0 0 .2 0 0 0 0 0' contains a stray '.2'. Please reformat and verify the entries sum to 1.

Circularity Check

0 steps flagged

No significant circularity; neural curvature is evaluated against external pruning baselines rather than fitted to pruning outcomes.

full rationale

The derivation chain is self-contained: the neural graph is defined from NN weights (Def. 5, w = 1/|W|) and activations (Defs. 6 and 7), then curvature (Def. 8) is used to rank edges, and the ranking is tested by removing edges and measuring test accuracy against external baselines (magnitude, SNIP, SynFlow). No parameter is fitted to the removal outcome, so the claim that removing negative-ORC edges degrades accuracy while positive-ORC edges do not (Sec. 7.3.1) is an empirical finding, not a tautology. The neural-graph construction is attributed to self-citations (Tan et al. 2024; Xiao et al. 2024), but Def. 5 gives the full construction, so those citations are not load-bearing. The paper itself flags the inverse-weight choice as unvalidated ('Other weightings will be considered in future work', Sec. 5.2) and concedes within-layer similarity to magnitude pruning (Sec. 7.3.2); a confound between curvature and inverse-magnitude/dead-ReLU signals is possible, but a confound is a validity threat, not circularity. Prop. 1 maps dead-ReLU edges to curvature 2, which makes a flat initial pruning region unsurprising, but this is an experimental artifact risk rather than a prediction fitted by construction. No uniqueness theorem or machine-checked claim is imported from self-citations. Overall, no prediction reduces by construction to an input; score 2 reflects minor non-load-bearing self-citations, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new particles, forces, or conserved quantities are introduced. 'Neural curvature' (Def 8) is a named metric constructed from existing curvature theory, not an entity with independent falsifiable handles. The method has no fitted parameters targeting pruning accuracy, but it has several hand-selected modeling choices (alpha, activation normalization, inverse-weight cost, min-aggregation) that fully determine the rankings; the curvature value is a convention, not a quantity derived from first principles. The main axiom is the analogy that ORC bottleneck semantics transfer to NN graphs with this cost model.

free parameters (4)
  • alpha (alpha-Ricci mixing) = 0.9
    Chosen by 'empirical observations' (Section 6: 'the alpha-curvature does not change significantly beyond that point') as a numerical stand-in for the alpha->1 limit in Def 8. Scales curvature values and could change cross-edge ordering.
  • Activation normalization pipeline = min-max -> reciprocal -> exp(-x^2) normalized kernel; epsilon unspecified
    Def 6: three hand-selected steps whose only stated justification is intuition ('large neurons get a larger density'). Shapes the neighbor distributions used in every curvature value.
  • Graph edge cost function w = 1/|W| = inverse weight magnitude
    Def 5: 'the larger the weight, the more data is sent'. The paper itself flags 'Other weightings will be considered in future work', so the cost model is unvalidated and load-bearing.
  • Aggregation rule: min over examples = min_x kappa_N(e,x)
    Algorithm 1, lines 4-8: the minimum curvature over calibration examples defines the ranking, so a single outlier example can dominate an edge's score. The ablation shows one-example-per-label works, but min versus mean is never compared.
axioms (4)
  • standard math Ollivier-Ricci curvature and the alpha->1 limit, including the theorem that h(alpha) = kappa_alpha/(1-alpha) is increasing and bounded, are valid for directed weighted graphs.
    Defs 1-4 and the limit in Def 8 rely on Ollivier (2009) and Lin et al. (2011) without re-derivation.
  • domain assumption Negative curvature indicates a bottleneck edge in a directed layered graph, transferring the network-science intuition (road, biological, social graphs) to neural networks.
    The whole approach rests on this analogy (Section 1, Section 5). The paper validates it empirically for three of thirteen models via Fig 4 but offers no proof or independent information-flow measurement.
  • domain assumption Removing edges without retraining and observing test-accuracy change is a valid operationalization of 'connection importance'.
    Section 7.1 evaluation protocol. Importance is defined by one-shot removal on the full model; conclusions may not transfer to fine-tuned or iterative pruning.
  • standard math For infinite-cost edges (dead ReLU or saturated paths), curvature converges to 1 (input/output layers) or 2 (hidden layers) as in Prop 1.
    Prop 1's transport-plan decomposition assumes the (2alpha-1) mass from u to v must traverse (u,v), which holds in a layered DAG; the proof is internally consistent.

pith-pipeline@v1.3.0-alltime-deepseek · 18812 in / 26296 out tokens · 261471 ms · 2026-08-03T08:37:30.916754+00:00 · methodology

0 comments
read the original abstract

This paper provides a fresh view of the neural network (NN) pruning problem through the lens of graph theory. To achieve effective pruning, we aim to identify the main NN data flows and the corresponding NN connections that are most and least important for the performance of the full model. Unlike the standard approach to NN data flow analysis, which is based on information theory, we employ the notion of graph curvature, specifically Ollivier-Ricci curvature (ORC). ORC has been successfully used to identify important graph edges in various domains such as road traffic analysis, biological networks, and social networks. In particular, edges with negative ORC are considered bottlenecks and are therefore critical to the graph's overall connectivity, whereas positive-ORC edges are less essential. We use this intuition for NNs to (1) construct a graph induced by the NN structure and introduce the notion of neural curvature (NC) based on ORC; (2) calculate curvatures based on activation patterns for a set of input examples; and (3) demonstrate that NC can be used to rank edges according to their importance for overall NN functionality. We evaluate our method through pruning experiments on a variety of small and medium size models trained on three image datasets: MNIST, CIFAR-10, and CIFAR-100. The results indicate that our method can identify a larger number of unimportant edges compared to existing pruning methods.

Figures

Figures reproduced from arXiv: 2601.16366 by Jayson Sia, Paul Bogdan, Radoslav Ivanov, Shuhang Tan.

Figure 1
Figure 1. Figure 1: ORC illustration on an undirected graph with unit edge weights. Red-colored edges (left) have a positive [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Approach overview. Given a trained NN fθ and a calibration set D, we aim to identify the importance of NN connections, respectively. Specifically, we 1) construct a neural graph Gfθ induced by the NN architecture (including an the input layer); 2) for each edge (u, v) ∈ Gfθ and each example x ∈ D, we calculate the neural curvature κN (u, v, x), as defined in Definition 8; 3) finally we order the whole edge… view at source ↗
Figure 3
Figure 3. Figure 3: Edge removal evaluation on MNIST, CIFAR-10, and CIFAR-100. Each subfigure shows a comparison of our [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Edge removal evaluation on MNIST and CIFAR-10 using our neural curvature algorithm. Each subfigure [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation experiment on per-layer edge removal. Each subfigure shows the edge removal analysis for each [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ablation experiment for edge removal with different neural curvature calculation modules (VGG9-lite, CE [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation experiment for edge removal using different numbers of calibration examples (VGG9-lite, CE [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Full model edge removal comparison between magnitude-based and ours (VGG9-lite, WD ReLU, on CIFAR [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Full model edge removal comparison between magnitude-based and ours (VGG9-lite, WD Tanh, on CIFAR [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 20 linked inside Pith

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [7]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635,

    Pith/arXiv arXiv

  3. [9]

    Flow of information in feed-forward deep neural networks

    Pejman Khadivi, Ravi Tandon, and Naren Ramakrishnan. Flow of information in feed-forward deep neural networks. arXiv preprint arXiv:1603.06220,

    Pith/arXiv arXiv

  4. [12]

    Pruning filters for efficient convnets

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710,

    Pith/arXiv arXiv

  5. [14]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083, 2017a. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706....

    Pith/arXiv arXiv

  6. [15]

    Ricci curvature of the internet topology

    Chien-Chun Ni, Y u-Y ao Lin, Jie Gao, Xianfeng David Gu, and Emil Saucan. Ricci curvature of the internet topology. In 2015 IEEE conference on computer communications (INFOCOM) , pp. 2758–2766. IEEE,

    2015

  7. [16]

    Sosp: Efficiently capturing global correlations by second-order structured pruning

    Manuel Nonnenmacher, Thomas Pfeil, Ingo Steinwart, and David Reeb. Sosp: Efficiently capturing global correlations by second-order structured pruning. arXiv preprint arXiv:2110.11395,

    Pith/arXiv arXiv

  8. [17]

    What matters in the structured pruning of generative language models? arXiv preprint arXiv:2302.03773,

    Michael Santacroce, Zixin Wen, Y elong Shen, and Y uanzhi Li. What matters in the structured pruning of generative language models? arXiv preprint arXiv:2302.03773,

    Pith/arXiv arXiv

  9. [18]

    Opening the black box of deep neural networks via information

    Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810,

    Pith/arXiv arXiv

  10. [22]

    Deep learning and the information bottleneck principle

    Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pp. 1–5. Ieee,

    2015

  11. [25]

    Analyzing multi-head self-attention: Spe- cialized heads do the heavy lifting, the rest can be pruned

    Elena V oita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Spe- cialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418,

    Pith/arXiv arXiv 1905

  12. [26]

    Mixed-curvature multi-relational graph neural network for knowledge graph completion

    Shen Wang, Xiaokai Wei, Cicero Nogueira Nogueira dos Santos, Zhiguo Wang, Ramesh Nallapati, Andrew Arnold, Bing Xiang, Philip S Y u, and Isabel F Cruz. Mixed-curvature multi-relational graph neural network for knowledge graph completion. In Proceedings of the web conference 2021 , pp. 1761–1771,

    2021

  13. [27]

    Ex- ploring neuron interactions and emergence in llms: From the multifractal analysis perspective

    Xiongye Xiao, Chenyu Zhou, Heng Ping, Defu Cao, Y axing Li, Yizhuo Zhou, Shixuan Li, and Paul Bogdan. Ex- ploring neuron interactions and emergence in llms: From the multifractal analysis perspective. arXiv preprint arXiv:2402.09099,

    Pith/arXiv arXiv

  14. [28]

    The ricci curvature on directed graphs

    Taiki Y amada. The ricci curvature on directed graphs. arXiv preprint arXiv:1602.07779,

    Pith/arXiv arXiv

  15. [29]

    Mingyang Zhang, Xinyi Y u, Jingtao Rong, and Linlin Ou

    URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ d201587e3a84fc4761eadc743e9b3f35-Paper-Datasets_and_Benchmarks.pdf. Mingyang Zhang, Xinyi Y u, Jingtao Rong, and Linlin Ou. Graph pruning for model compression.Applied Intelligence, 52(10):11244–11256,

    2022

  16. [1946]

    Modeling information flow through deep neural networks

    Ahmad Chaddad, Behnaz Naisiri, Marco Pedersoli, Eric Granger, Christian Desrosiers, and Matthew Toews. Modeling information flow through deep neural networks. arXiv preprint arXiv:1712.00003,

    Pith/arXiv arXiv

  17. [1998]

    Snip: Single-shot network pruning based on connection sensitivity

    Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340,

    Pith/arXiv arXiv

  18. [2000]

    The llm surgeon

    Tycho FA van der Ouderaa, Markus Nagel, Mart V an Baalen, Y uki M Asano, and Tijmen Blankevoort. The llm surgeon. arXiv preprint arXiv:2312.17244,

    Pith/arXiv arXiv

  19. [2009]

    URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR. pdf. Y ann LeCun, Léon Bottou, Y oshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE , 86(11):2278–2324,

    2009

  20. [2013]

    Analyzing neural network robustness using graph curvature

    Shuhang Tan, Jayson Sia, Paul Bogdan, and Radoslav Ivanov. Analyzing neural network robustness using graph curvature. arXiv:1312.6199,

    Pith/arXiv arXiv

  21. [2015]

    The information bottleneck method

    20 Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004057,

    Pith/arXiv arXiv

  22. [2018]

    Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, et al

    URL https://arxiv.org/abs/1610.02136. Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, et al. Emma: End-to-end multimodal model for autonomous driving. arXiv preprint arXiv:2410.23262,

    Pith/arXiv arXiv

  23. [2019]

    On network science and mutual information for explaining deep neural networks

    Brian Davis, Umang Bhatt, Kartikeya Bhardwaj, Radu Marculescu, and José MF Moura. On network science and mutual information for explaining deep neural networks. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 8399–8403. IEEE,

    2020

  24. [2020]

    Everybody prune now: Structured pruning of llms with only forward passes

    Lucio Dery, Steven Kolawole, Jean-François Kagy, Virginia Smith, Graham Neubig, and Ameet Talwalkar. Everybody prune now: Structured pruning of llms with only forward passes. arXiv preprint arXiv:2402.05406,

    arXiv

  25. [2021]

    Curvdrop: A ricci curvature based approach to prevent graph neural networks from over-smoothing and over-squashing

    Y ang Liu, Chuan Zhou, Shirui Pan, Jia Wu, Zhao Li, Hongyang Chen, and Peng Zhang. Curvdrop: A ricci curvature based approach to prevent graph neural networks from over-smoothing and over-squashing. In Proceedings of the ACM Web Conference 2023, pp. 221–230,

    2023

  26. [2022]

    A simple and effective pruning approach for large language models

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695,

    Pith/arXiv arXiv

  27. [2023]

    Where is the information in a deep neural network? arXiv preprint arXiv:1905.12213,

    Alessandro Achille, Giovanni Paolini, and Stefano Soatto. Where is the information in a deep neural network? arXiv preprint arXiv:1905.12213,

    Pith/arXiv arXiv 1905

  28. [2024]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

    Pith/arXiv arXiv 2010