Pith. sign in

REVIEW 4 major objections 4 minor 45 references

A graph-reading reinforcement learning agent can learn which channels to prune directly from data, and the resulting masks match or beat weight-magnitude pruning at the same FLOPs budget.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 10:13 UTC pith:OWQC4RY7

load-bearing objection A plausible incremental RL-pruning idea with a good policy analysis, but the action space is underspecified (ResNet masks irreproducible) and the experiments overstate the case. the 4 major comments →

arxiv 2509.10526 v1 pith:OWQC4RY7 submitted 2025-09-04 cs.LG cs.AI

Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning

Dieter Balemans , Thomas Huybrechts , Jan Steckel , Siegfried Mercelis This is my paper
classification cs.LG cs.AI
keywords neural network pruningreinforcement learninggraph attention networkchannel pruningconstrained Markov decision processself-competition rewardAutoMLmodel compression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that neural-network pruning decisions can be learned rather than prescribed. It replaces the standard layer-by-layer heuristics—cut whichever channels have the smallest weight magnitudes—with a reinforcement learning agent that observes the whole network as a graph, encodes it with a Graph Attention Network, and makes binary keep-or-kill choices over individual channels while respecting a resource budget such as a FLOPs target. The claimed payoff is task-specific pruning: learned masks that diverge from magnitude masks most in deep, task-specific layers, and accuracies at a given compression level that match or beat traditional and AutoML baselines on CIFAR-10, CIFAR-100, and ImageNet. A self-competition reward—comparing each episode against the agent's own exponential moving average—first pushes the network to the target compression, then rewards accuracy recovery, without Lagrangian penalty parameters. If the claims hold, the framework removes hand-designed importance metrics and per-layer manual tuning from the pruning pipeline.

Core claim

A pruning agent can learn channel importance from data when it observes the whole network as a graph and acts with binary channel-level decisions. Nodes are layers with structural, computational, and weight features; edges carry connection type (regular, skip, residual). A GAT encoder with global attention aggregation feeds a PPO policy that prunes channel groups sequentially (n=1: one combinatorial step). The authors report masks matching or beating traditional pruning at equal FLOPs budgets on VGG-16, ResNet-56, MobileNet-V2 over CIFAR-10/100 and ImageNet; learned masks overlap magnitude masks early but diverge deeper; and on ResNet-56 at an 80% accuracy target the learned policy removes 6

What carries the argument

The load-bearing mechanism is a graph observation space paired with a channel-grouped binary action space in a constrained Markov decision process. Channels are split into n groups; each step the agent's binary mask chooses which channels in the current group survive. n=1 reduces the task to a single combinatorial pruning step; n>1 makes it sequential, with the graph updated after each group. A Graph Attention Network encoder with global attention aggregation maps the graph to one embedding consumed by the PPO policy and value networks, and a self-competition reward—comparing the episode's accuracy and FLOPs against exponential moving averages of past episodes—alternates between a compressio

Load-bearing premise

The framework assumes every channel of the network can be lined up in one fixed order, split into equal-sized groups, and pruned group by group while the network always stays valid—but the paper never specifies that order or how shortcut connections constrain the pruning.

What would settle it

Reproduce the ResNet-56 experiment using only the paper's description: if two different channel orderings for the same group partition yield different FLOPs counts or different accuracies, the action space is not well-defined and the reported numbers pin down no single reproducible method. Failing that, evaluate a trained G-SCC policy without fine-tuning at the same FLOPs budget on a held-out architecture: if it does not beat magnitude-based pruning, the central advantage claim falls.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Channel importance becomes a learned function of whole-network topology, so a new architecture or dataset can be pruned without hand-designing importance metrics or per-layer ratios.
  • The same agent and reward structure can target FLOPs, parameters, memory, or latency, or flip to the 'minimize resources above an accuracy floor' mode, without redesigning the pipeline.
  • Self-competition removes penalty-term tuning for constraint satisfaction, simplifying deployment across different compression budgets.
  • Learned masks diverge from magnitude masks most in deep, task-specific layers, implying magnitude-based pruning leaves structurally redundant channels in place.
  • As the paper's own conclusion states, each learned policy is specific to its architecture–dataset–compression triple; the reported gains are per-deployment rather than a universal mask.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: an ablation that swaps the graph embedding for AMC's eleven hand-crafted layer features, keeping the rest of the pipeline fixed, would isolate how much of the gain comes from the global observation; the paper does not report such an ablation.
  • Inference: because the channel ordering, group assignment, and residual-connection handling are left unspecified, reproducing the ResNet results requires choices the text does not pin down; making the group assignment randomized or learned during training is a testable robustness extension.
  • Inference: the n=1 configuration is a general single-shot binary-mask optimizer, so the same machinery could be applied outside pruning—e.g., to sparse-training or input-selection masks—where the graph encoder would play a smaller role.
  • Inference: comparing the EMA self-competition reward against a fixed, non-adaptive target would separate the benefit of the adaptive reference from the benefit of the phase-structured reward itself.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes G-SCC, a reinforcement-learning framework for structured channel pruning in which a PPO agent selects binary channel-pruning masks on the basis of a graph observation of the full network, encoded by a GAT. The authors formulate pruning as a CMDP with a self-competition reward designed to satisfy a FLOPs constraint while maximizing accuracy. Experiments on CIFAR-10, CIFAR-100, and ImageNet compare G-SCC with hand-crafted pruning methods, prior AutoML methods, and an evolutionary (CMA-ES) baseline. The paper also analyzes the learned masks against magnitude-based masks using Jaccard, cosine, and Hamming similarity. The main claims are that the graph-based global observation space and data-driven binary actions remove reliance on hand-crafted importance metrics and that the method achieves state-of-the-art or competitive results across models and datasets.

Significance. If the method works as described, it would be a useful contribution to automated structured pruning: a graph-based, global state representation with a discrete action space is a natural way to address the layer-local heuristics of earlier work such as AMC, and the self-competition reward is an interesting alternative to Lagrangian constraint handling. The paper also provides a direct comparison against an evolutionary search baseline, which is commendable for calibrating what the RL component adds. However, the significance is currently limited by three issues that affect the central claims: the action space is not well specified for residual networks, the observation still contains l1-norm importance features despite the 'no hand-crafted metrics' claim, and the experimental comparisons are not sufficiently controlled to support the stated superiority.

major comments (4)
  1. [§2.2.2–2.2.3, Eq. (3)] The action space is not well-defined for residual architectures. The paper defines A = {a1, ..., a_{C/n}} with 'each value in the mask correspond[ing] to a channel in the target neural network,' but for ResNet-56, pruning a channel in one convolutional layer forces alignment of the corresponding channel in the residual/skip path and in downstream layers. The paper never states the channel ordering, how groups are formed when layers have different channel counts, how the GAT state indicates which group is currently being pruned, or how the transition function in §2.2.3 handles invalid masks that break residual alignment. If the environment silently corrects masks, the policy's output is not the executed action; if it does not, episodes can terminate in shape mismatch. Since ResNet-56 is a core experimental model (Tables 2–3, Figures 3, 5, 6), the reported results, including the 69% FLOPs
  2. [Abstract, §1, §2.2.1] The central claim that the method 'eliminates the reliance on predefined importance metrics such as l1-norm weights' is contradicted by the observation space. In §2.2.1, node features explicitly include 'the l1-norm of each of the individual channels in the layer,' and edge features include 'the l1-norm of the output channel activations of the source layer.' These are exactly hand-crafted importance features. The agent may use them in a learned way, but the statement that the framework moves 'away from predefined scoring functions' is overstated. The authors should either remove the claim, clarify that l1-norm is only an input feature rather than a fixed scoring rule, or provide an ablation showing the method works without these features.
  3. [Tables 1–4] The experimental evidence is not sufficiently controlled to support the comparative claims. (i) No error bars, seeds, or variance estimates are reported anywhere, so it is impossible to tell whether differences such as 93.67 vs. 93.14 in Table 2 are significant. (ii) Methods are compared at different FLOPs ratios (e.g., Table 3: G-SCC at 40% vs. AGSPRL at 70%; Table 1: G-SCC at 40% vs. CMA-ES at 50% on VGG-16 CIFAR-100), so the ranking may reflect the target ratio rather than method quality. (iii) Fine-tuning protocols differ across baselines; the paper states this itself in the discussion following Table 4. (iv) In Table 1, CMA-ES outperforms G-SCC (n=1) on VGG-16 CIFAR-10 (93.55 vs. 92.69) and ResNet-56 CIFAR-10 (90.57 vs. 90.21), which weakens the claim that the RL agent is a better combinatorial optimizer. These issues need to be addressed, at minimum by reporting multiple seeds, mat
  4. [§2.2.4, Eq. (7)] The Markovianity claim for the self-competition reward is not justified. The reward in Eq. (6) depends on A_EMA and F_EMA, which are global running averages updated over the course of training. These quantities are not part of the observation space O defined in §2.2.1, so the reward is not a function of the current state-action pair alone; the MDP is effectively non-stationary. The argument that 'it does not inherently rely on a sequence of prior states within each individual scenario' does not address this. Additionally, Eq. (7) is malformed: the text says the indicator function outputs 1 if the condition is true and −1 otherwise, which is not a standard indicator, and the conditions F_EMA≤0 and F_ep≤0 are inconsistent with F representing FLOPs (always positive). The authors should correct Eq. (7) and either provide a proper treatment of the non-stationary reward or explicitly frame sel
minor comments (4)
  1. [§2.2.2] It is not specified what happens when the total channel count C is not divisible by the group size n, nor how layers with different numbers of channels are mapped to the fixed-size action vector. State whether groups are padded, masked, or assigned per layer.
  2. [Figure 2] The y-axis is labeled 'Top-1 Accuracy (%)' but the values shown are in [0.91, 0.93] and [0.80, 0.95], suggesting these are proportions. Use consistent units (percent or fraction).
  3. [Figures 4–6] The pruning-mask visualizations report values such as '0.25 %' and '0.88 %' under each layer name; if these represent sparsity fractions or percentages, the formatting is confusing. Define the quantity and units in the caption.
  4. [§2.3.1] The GAT encoder is described as using 'a standard of three message passing operations,' but no ablation or sensitivity analysis is provided for this choice. Since it is a hyperparameter of the method, state whether the results are robust to it.

Circularity Check

0 steps flagged

No significant circularity. Mild self-citation of prior work [29] as a comparison baseline, but the central claims are empirically evaluated and not derived from the method's own fitted quantities.

full rationale

The paper's derivation chain is not circular. The reward function (Eq. 6) uses exponential moving averages of the agent's own accuracy and FLOPs as an internal training signal; the reported accuracies are measured after pruning, not predicted from the reward. The observation space (Sec. 2.2.1) includes l1-norm channel features, which weakens the narrative that the method eliminates predefined importance metrics, but this is an input feature, not a fitted parameter renamed as a prediction. The learned mask is a function of all node and edge features, and Sec. 3.2 explicitly shows the learned masks differ from magnitude-based masks (low Jaccard/cosine similarity in deeper layers), so the result is not equivalent to the l1-norm input by construction. The only self-citation is the authors' prior SCC work [29], used as a baseline for comparison; it is not invoked as a load-bearing justification or uniqueness theorem. All main comparisons are against external methods (AMC, FPGM, EagleEye, etc.) on standard benchmarks. Therefore no step reduces to its own inputs by definition.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The ledger contains three free parameters or unstated training choices, with the channel-grouping assumption being the most structurally load-bearing. No new physical or conceptual entities are introduced; the self-competition reward is a training mechanism, not an invented entity.

free parameters (3)
  • n (channel group count) = 4 (default; n=1 in some experiments)
    Controls episode length and action granularity; chosen by hand. Section 3.3 reports that n=1 converges faster and n=4 was used for most experiments, but no systematic tuning protocol is given.
  • EMA decay factors for accuracy and FLOPs = not reported
    The self-competition reward in Eq (6) depends on exponential moving averages AccEMA and FEMA; the decay hyperparameters are never specified, so the reward signal as implemented is not fully defined.
  • Target FLOPs ratio Ftarget = per experiment (e.g., 50%, 80% removed)
    Input constraint set by user, not fitted. Listed for completeness because it directly shapes the reward and masks in each reported result.
axioms (4)
  • domain assumption A neural network can be faithfully represented as a DAG with layer nodes and edge features, and padding node features with zeros preserves enough information for pruning decisions.
    The observation space in §2.2.1 assumes this; padding to max channel count may distort the message passing for layers with few channels.
  • domain assumption Pruning a channel group in the flat channel ordering always yields a valid network, including skip and residual connections.
    Action space §2.2.2 does not specify channel ordering or structural constraints; for ResNet this is non-trivial.
  • ad hoc to paper The self-competition reward satisfies the Markov property; PPO with this non-stationary reward converges to a good policy.
    Argued in §2.2.4, but EMA history is not part of the state, so the reward is not a function of the current state alone.
  • standard math Standard PPO and GAT operations are used as in cited references.
    Equations (9)-(12) rely on standard GAT and attention aggregation.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 22589 in / 14041 out tokens · 118940 ms · 2026-08-05T10:13:05.333555+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning." pith.science (2026). https://pith.science/paper/OWQC4RY7

@misc{pith2026250910526,
  author       = {Pith},
  title        = {Pith review of: Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OWQC4RY7}},
  note         = {Machine review of arXiv:2509.10526}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents a novel approach to neural network pruning by integrating a graph-based observation space into an AutoML framework to address the limitations of existing methods. Traditional pruning approaches often depend on hand-crafted heuristics and local optimization perspectives, which can lead to suboptimal performance and inefficient pruning strategies. Our framework transforms the pruning process by introducing a graph representation of the target neural network that captures complete topological relationships between layers and channels, replacing the limited layer-wise observation space with a global view of network structure. The core innovations include a Graph Attention Network (GAT) encoder that processes the network's graph representation and generates a rich embedding. Additionally, for the action space we transition from continuous pruning ratios to fine-grained binary action spaces which enables the agent to learn optimal channel importance criteria directly from data, moving away from predefined scoring functions. These contributions are modelled within a Constrained Markov Decision Process (CMDP) framework, allowing the agent to make informed pruning decisions while adhering to resource constraints such as target compression rates. For this, we design a self-competition reward system that encourages the agent to outperform its previous best performance while satisfying the defined constraints. We demonstrate the effectiveness of our approach through extensive experiments on benchmark datasets including CIFAR-10, CIFAR-100, and ImageNet. The experiments show that our method consistently outperforms traditional pruning techniques, showing state-of-the-art results while learning task-specific pruning strategies that identify functionally redundant connections beyond simple weight magnitude considerations.

Figures

Figures reproduced from arXiv: 2509.10526 by Dieter Balemans, Jan Steckel, Siegfried Mercelis, Thomas Huybrechts.

Figure 1
Figure 1. Figure 1: Overview of the self-competition environment where the PPO agent interacts with the pruning environment. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Trajectories of the agent during training. The graphs show the trajectory of the agent in resource constrained [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Trajectory of the agent during training. The graphs show the trajectory of the agent in accuracy guarantee [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Analysis of pruning policies for VGG-16 on CIFAR-10 with 50% removed FLOPs. (a-b) Comparison of [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Analysis of pruning policies for ResNet-56 on CIFAR-10 with 80% accuracy guaranteed. (a-b) Comparison [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pruning masks of ResNet-56 on CIFAR-10 with 80% accuracy guaranteed. (a-b) Visual representation of [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 24 canonical work pages · 11 internal anchors

  1. [1]

    Automated machine learning: past, present and future,

    M. Baratchi, C. Wang, S. Limmer, J. N. van Rijn, H. Hoos, T. B ¨ack, and M. Olhofer, “Automated machine learning: past, present and future,” Artificial intelligence review , vol. 57, no. 5, pp. 1–88, 18 Apr. 2024. [Online]. Available: https://link.springer.com/article/10.1007/s10462-024-10726-1 2

    work page doi:10.1007/s10462-024-10726-1 2024

  2. [2]

    NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications

    T.-J. Yang, A. Howard, B. Chen, X. Zhang, A. Go, M. Sandler, V . Sze, and H. Adam, “NetAdapt: Platform-aware neural network adaptation for mobile applications,”arXiv [cs.CV], 9 Apr. 2018. [Online]. Available: http://arxiv.org/abs/1804.03230 2, 16, 17

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  3. [3]

    EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning

    B. Li, B. Wu, J. Su, G. Wang, and L. Lin, “EagleEye: Fast sub-net evaluation for efficient neural network pruning,” arXiv [cs.CV], 5 Jul. 2020. [Online]. Available: http://arxiv.org/abs/2007.02491 2, 16, 17

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  4. [4]

    MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning

    Z. Liu, H. Mu, X. Zhang, Z. Guo, X. Yang, T. K.-T. Cheng, and J. Sun, “MetaPruning: Meta learning for automatic neural network channel pruning,” arXiv [cs.CV], 25 Mar. 2019. [Online]. Available: http://arxiv.org/abs/1903.10258 2, 16, 17

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  5. [5]

    AutoPruner: An end-to-end trainable filter pruning method for efficient deep model inference,

    J.-H. Luo and J. Wu, “AutoPruner: An end-to-end trainable filter pruning method for efficient deep model inference,” arXiv [cs.CV], 22 May 2018. [Online]. Available: http://arxiv.org/abs/1805.08941 2, 16, 17

    Pith/arXiv arXiv 2018

  6. [6]

    AutoSlim: Towards one-shot architecture search for channel numbers,

    J. Yu and T. Huang, “AutoSlim: Towards one-shot architecture search for channel numbers,” arXiv [cs.CV], 27 Mar. 2019. [Online]. Available: http://arxiv.org/abs/1903.11728 2, 16, 17

    Pith/arXiv arXiv 2019

  7. [7]

    AutoCompress: An automatic DNN structured pruning framework for ultra-high compression rates,

    N. Liu, X. Ma, Z. Xu, Y . Wang, J. Tang, and J. Ye, “AutoCompress: An automatic DNN structured pruning framework for ultra-high compression rates,” Proceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence , vol. 34, no. 04, pp. 4876–4883, 3 Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/...

    work page 2020

  8. [8]

    AMC: AutoML for model compression and acceleration on mobile devices,

    Y . He, J. Lin, Z. Liu, H. Wang, L.-J. Li, and S. Han, “AMC: AutoML for model compression and acceleration on mobile devices,” in Computer Vision – ECCV 2018, ser. Lecture notes in computer science. Cham: Springer International Publishing, 2018, pp. 815–832. [Online]. Available: http://dx.doi.org/10.1007/978-3-030-01234-2 48 2, 3, 16, 17

    doi:10.1007/978-3-030-01234-2 2018

  9. [9]

    A novel filter-level deep convolutional neural network pruning method based on deep reinforcement learning,

    Y . Feng, C. Huang, L. Wang, X. Luo, and Q. Li, “A novel filter-level deep convolutional neural network pruning method based on deep reinforcement learning,” Applied sciences (Basel, Switzerland) , vol. 12, no. 22, p. 11414, 10 Nov. 2022. [Online]. Available: https://www.mdpi.com/2076-3417/12/22/11414 2

    work page 2022

  10. [10]

    Neural Network Pruning Through Constrained Reinforcement Learning

    S. Malik, M. U. Haider, O. Iqbal, and M. Taj, “Neural network pruning through constrained reinforcement learning,” arXiv [cs.CV], 16 Oct. 2021. [Online]. Available: http://arxiv.org/abs/2110.08558 2, 16

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  11. [11]

    Runtime neural pruning,

    J. Lin, Y . Rao, J. Lu, and J. Zhou, “Runtime neural pruning,” Neural Information Processing Systems, pp. 2181–2191, 2017. [Online]. Available: https://www.semanticscholar.org/paper/88cd4209db62a34d9cba0b9cbe9d45d1e57d21e5 2, 16, 17

    work page 2017

  12. [12]

    Learning to Prune Filters in Convolutional Neural Networks

    Q. Huang, K. Zhou, S. You, and U. Neumann, “Learning to prune filters in convolutional neural networks,” arXiv [cs.CV], 23 Jan. 2018. [Online]. Available: http://arxiv.org/abs/1801.07365 2

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  13. [13]

    Deterministic policy gradient algorithms,

    D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. A. Riedmiller, “Deterministic policy gradient algorithms,” International Conference on Machine Learning , vol. 32, no. 1, pp. 387–395, 21 Jun. 2014. [Online]. Available: https://proceedings.mlr.press/v32/silver14.pdf 2

    work page 2014

  14. [14]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv [cs.LG], 19 Jul. 2017. [Online]. Available: http://arxiv.org/abs/1707.06347 2, 7

    Pith/arXiv arXiv 2017

  15. [15]

    MuZero with self-competition for rate control in VP9 video compression,

    A. Mandhane, A. Zhernov, M. Rauh, C. Gu, M. Wang, F. Xue, W. Shang, D. Pang, R. Claus, C.-H. Chiang, C. Chen, J. Han, A. Chen, D. J. Mankowitz, J. Broshear, J. Schrittwieser, T. Hubert, O. Vinyals, and T. Mann, “MuZero with self-competition for rate control in VP9 video compression,” arXiv [eess.IV] , 14 Feb. 2022. [Online]. Available: http://arxiv.org/ab...

    Pith/arXiv arXiv 2022

  16. [16]

    Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning

    S. Yu, A. Mazaheri, and A. Jannesari, “Topology-aware network pruning using multi-stage graph embedding and reinforcement learning,” in International Conference on Machine Learning , 2021. [Online]. Available: https: //arxiv.org/pdf/2102.03214.pdf 3

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  17. [17]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Li `o, and Y . Bengio, “Graph attention networks,” arXiv [stat.ML], 30 Oct. 2017. [Online]. Available: http://arxiv.org/abs/1710.10903 3, 7

    Pith/arXiv arXiv 2017

  18. [18]

    How attentive are graph attention networks?

    S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?” arXiv [cs.LG], 30 May 2021. [Online]. Available: http://arxiv.org/abs/2105.14491 3, 7

    Pith/arXiv arXiv 2021

  19. [19]

    Altman, Constrained Markov decision processes: Stochastic modeling

    E. Altman, Constrained Markov decision processes: Stochastic modeling. Boca Raton: Routledge, 13 Dec. 2021. [Online]. Available: https://www-sop.inria.fr/members/Eitan.Altman/TEMP/h.pdf 3

    work page 2021

  20. [20]

    Simon, Evolutionary optimization algorithms

    D. Simon, Evolutionary optimization algorithms. Nashville, TN: John Wiley & Sons, 13 Jun. 2013. 5

    work page 2013

  21. [21]

    Graph Matching Networks for Learning the Similarity of Graph Structured Objects

    Y . Li, C. Gu, T. Dullien, O. Vinyals, and P. Kohli, “Graph matching networks for learning the similarity of graph structured objects,” arXiv [cs.LG], 29 Apr. 2019. [Online]. Available: http://arxiv.org/abs/1904.12787 8

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  22. [22]

    Exact solutions to the nonlinear dynamics of learning in deep linear neural networks,

    A. M. Saxe, J. L. McClelland, and S. Ganguli, “Exact solutions to the nonlinear dynamics of learning in deep linear neural networks,” arXiv [cs.NE], 20 Dec. 2013. [Online]. Available: http://arxiv.org/abs/1312.6120 8 19 Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning A PREPRINT

    Pith/arXiv arXiv 2013

  23. [23]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv [cs.LG], 9 Sep. 2016. [Online]. Available: http://arxiv.org/abs/1609.02907 8

    Pith/arXiv arXiv 2016

  24. [24]

    Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,

    J. S. Bridle, “Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,” in Neurocomputing. Berlin, Heidelberg: Springer Berlin Heidelberg, 1990, pp. 227–236. [Online]. Available: https://doi.org/10.1007/978-3-642-76153-9 28 9

    work page doi:10.1007/978-3-642-76153-9 1990

  25. [25]

    Learning representations by back-propagating errors,

    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, Oct. 1986. [Online]. Available: http://dx.doi.org/10.1038/323533a0 9

    doi:10.1038/323533a0 1986

  26. [26]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 22 Dec. 2014. [Online]. Available: http://arxiv.org/abs/1412.6980 9

    Pith/arXiv arXiv 2014

  27. [27]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv [cs.CV], 4 Sep

    work page

  28. [28]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, “Learning multiple layers of features from tiny images,” 2009. [On- line]. Available: https://www.semanticscholar.org/paper/Learning-Multiple-Layers-of-Features-from-Tiny-Krizhevsky/ 5d90f06bb70a0a3dced62413346235c02b1aa086 10, 16

    work page 2009

  29. [29]

    Resource-aware neural network pruning using constrained reinforcement learning and self-competition,

    D. Balemans, T. Huybrechts, J. Steckel, and S. Mercelis, “Resource-aware neural network pruning using constrained reinforcement learning and self-competition,” in 2025 6th International Conference on Artificial Intelligence, Robotics and Control (AIRC). IEEE, 7 May 2025, pp. 526–534. [Online]. Available: https://ieeexplore.ieee.org/document/11077487/ 11, 16, 17

    work page arXiv 2025

  30. [30]

    Numerical optimization of computer models,

    A. Tripathy, “Numerical optimization of computer models,” The Journal of the Operational Research Society, vol. 33, no. 12, pp. 1166–1166, Dec. 1982. [Online]. Available: http://dx.doi.org/10.1057/jors.1982.238 15

    work page doi:10.1057/jors.1982.238 1982

  31. [31]

    Completely derandomized self-adaptation in evolution strategies,

    N. Hansen and A. Ostermeier, “Completely derandomized self-adaptation in evolution strategies,” Evolutionary computation, vol. 9, no. 2, pp. 159–195, 2001. [Online]. Available: http://dx.doi.org/10.1162/106365601750190398 15

    doi:10.1162/106365601750190398 2001

  32. [32]

    The CMA evolution strategy: A tutorial,

    N. Hansen, “The CMA evolution strategy: A tutorial,” arXiv [cs.LG] , 4 Apr. 2016. [Online]. Available: http://arxiv.org/abs/1604.00772 15

    Pith/arXiv arXiv 2016

  33. [33]

    ImageNet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition . IEEE, Jun. 2009, pp. 248–255. [Online]. Available: https://ieeexplore.ieee.org/document/5206848 16

    arXiv 2009

  34. [34]

    Structured Probabilistic Pruning for Convolutional Neural Network Acceleration

    H. Wang, Q. Zhang, Y . Wang, and H. Hu, “Structured probabilistic pruning for convolutional neural network acceleration,” arXiv [cs.LG], 19 Sep. 2017. [Online]. Available: http://arxiv.org/abs/1709.06994 16, 17

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  35. [35]

    Pruning filters for efficient ConvNets,

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient ConvNets,” arXiv [cs.CV], no. 2016, pp. 1–13, 30 Aug. 2016. [Online]. Available: http://arxiv.org/abs/1608.08710 16, 17

    Pith/arXiv arXiv 2016

  36. [36]

    Soft filter pruning for accelerating deep convolutional neural networks,

    Y . He, G. Kang, X. Dong, Y . Fu, and Y . Yang, “Soft filter pruning for accelerating deep convolutional neural networks,” arXiv [cs.CV], 21 Aug. 2018. [Online]. Available: http://arxiv.org/abs/1808.06866 16

    Pith/arXiv arXiv 2018

  37. [37]

    Filter pruning via geometric median for deep convolutional neural networks acceleration,

    Y . He, P. Liu, Z. Wang, Z. Hu, and Y . Yang, “Filter pruning via geometric median for deep convolutional neural networks acceleration,” arXiv [cs.CV], 1 Nov. 2018. [Online]. Available: http://arxiv.org/abs/1811.00250 16

    Pith/arXiv arXiv 2018

  38. [38]

    DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation

    X. Ning, T. Zhao, W. Li, P. Lei, Y . Wang, and H. Yang, “DSA: More efficient budgeted pruning via differentiable sparsity allocation,” arXiv [cs.CV], 5 Apr. 2020. [Online]. Available: http://arxiv.org/abs/2004.02164 16

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  39. [39]

    Provable Filter Pruning for Efficient Neural Networks

    L. Liebenwein, C. Baykal, H. Lang, D. Feldman, and D. Rus, “Provable filter pruning for efficient neural networks,” arXiv [cs.LG], 17 Nov. 2019. [Online]. Available: http://arxiv.org/abs/1911.07412 16

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  40. [40]

    Automatic group-based structured pruning for deep convolutional networks,

    H. Wei, Z. Wang, G. Hua, J. Sun, and Y . Zhao, “Automatic group-based structured pruning for deep convolutional networks,” IEEE access: practical innovations, open solutions , vol. 10, pp. 128 824–128 834, 2022. [Online]. Available: http://dx.doi.org/10.1109/ACCESS.2022.3227619 16, 17

    work page arXiv 2022

  41. [41]

    To prune, or not to prune: exploring the efficacy of pruning for model compression,

    M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compression,” arXiv [stat.ML], 5 Oct. 2017. [Online]. Available: http://arxiv.org/abs/1710.01878 18

    Pith/arXiv arXiv 2017

  42. [42]

    Learning Efficient Convolutional Networks through Network Slimming

    Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, and C. Zhang, “Learning efficient convolutional networks through network slimming,” arXiv [cs.CV], 22 Aug. 2017. [Online]. Available: http://arxiv.org/abs/1708.06519 18

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  43. [43]

    A comprehensive overview of large language models,

    H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian, “A comprehensive overview of large language models,”arXiv [cs.CL], 12 Jul. 2023. [Online]. Available: http://arxiv.org/abs/2307.06435 18

    Pith/arXiv arXiv 2023

  44. [44]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv [cs.CV], 22 Oct. 2020. [Online]. Available: http://arxiv.org/abs/2010.11929 18 20

    Pith/arXiv arXiv 2020

  45. [2014]

    Available: http://arxiv.org/abs/1409.1556 10

    [Online]. Available: http://arxiv.org/abs/1409.1556 10

    Pith/arXiv arXiv