Pith. sign in

REVIEW 4 major objections 6 minor 48 references

Scalable iterative pruning of large language and vision models using block coordinate descent

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Iterative block-wise pruning matches or beats one-shot baselines on LLMs by solving small quadratic binary problems.

desk verdict A real engineering contribution that generalizes CBS to iterative block-wise pruning with weight-fixing and tabu search, reporting large gains over Wanda — plausible and worth refereeing, but the single-run, validation-tuned evaluation needs code and seeds before the numbers can be trusted. read the letter →

arxiv 2411.17796 v1 pith:7FJN5R7W submitted 2024-11-26 cs.LG math.OCquant-ph

Gili Rosenberg , J. Kyle Brubaker , Martin J. A. Schuetz , Elton Yechao Zhu , Serdar Kad{i}oğlu , Sima E. Borujeni , Helmut G. Katzgraber This is my paper · ORCID
classification cs.LGmath.OCquant-ph
keywords neuralnetworkpruningblockcoordinatedescentCombinatorialBrainSurgeonlargelanguagemodelsHessianapproximationquadraticconstrainedbinaryoptimizationmodelcompressionweightscoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces iterative Combinatorial Brain Surgeon (iCBS), a pruning method that starts from a one-shot pruning such as Wanda or magnitude pruning and repeatedly improves it by solving a small quadratic binary optimization problem over a block of weights using block coordinate descent. The central claim is that this iterative, block-wise procedure scales to large models such as Mistral-7b and DeiT and achieves higher validation accuracy at the same density than one-shot baselines such as Wanda. The authors report that only a small fraction of weights needs to be optimized (0.07 for DeiT and 0.002 for Mistral-7b) to obtain these gains. If correct, this shows that Hessian-based pruning is feasible for large language models, at the cost of substantially more compute, and that the per-block optimization could be offloaded to hardware accelerators or quantum solvers.

What carries the argument

The carrying object is the per-block quadratic constrained binary optimization problem of Eq. (8): binary variables $x_i$ indicate whether weight $i$ is pruned, and the objective combines a linear gradient term, a quadratic Hessian term, and a ridge term, with a cardinality constraint that exactly $k$ of the $n$ selected weights are pruned. The block Hessian is approximated from per-sample gradients as $H_B \approx \frac{1}{n} A^T A$, so only the Hessian entries for the selected block are needed. Weight-scoring methods (Wanda, gradient, magnitude) choose which weights are fixed as always pruned or kept and which weights enter each block, while a tabu list prevents re-selecting the same weights; the block solver is a constrained simulated annealing routine that searches only feasible solutions.

What would settle it

Re-run the Mistral-7b experiment at density 30 percent with fresh random batches from C4 used for gradient and Hessian estimation; if the reported accuracy gain over per-output Wanda does not reproduce across independent batches, or if a randomly scrambled Hessian of the same scale produces the same gain, the load-bearing role of the local quadratic model is falsified.

Watch

Extended reading notes

Core claim

Starting from a cheap one-shot pruning, iCBS repeatedly selects a block of $n$ candidate weights using weight-scoring methods and a tabu list, estimates the block gradient and Hessian from a small data batch via $H_B \approx \frac{1}{n} A^T A$, solves a quadratic constrained binary optimization problem that decides which weights in the block to prune or un-prune, and applies the solution. The central discovery is that this local second-order correction, repeated over blocks, improves the pruned model's accuracy on Fashion-MNIST, ImageNet-1K, and seven language-modeling tasks compared with Wanda, magnitude, and gradient baselines at the same densities, despite optimizing over only a small fraction of the weights. The improvement is largest at low densities (up to +21.6 percent accuracy for the Garment Classifier at density 10 percent, +14.2 percent for DeiT at density 20 percent, and +7.7 percent for Mistral-7b at density 30 percent) and shrinks as density approaches the unpruned model.

Load-bearing premise

The method assumes that the quadratic loss approximation built from a small batch of pruning data (batch size 16 for Mistral-7b) is accurate enough that solving the local binary problem improves the model's accuracy on the validation tasks.

Editorial extensions

If this is right

  • Hessian-based second-order pruning is not limited to small networks; it can improve large language models and vision transformers at the same density as one-shot baselines.
  • One-shot pruning results serve as a warm start, and iterative block solves refine them without ever forming the full $N \times N$ Hessian.
  • The method exposes a quality-time tradeoff: more epochs, more steps, or larger blocks cost more compute but buy accuracy, which one-shot methods do not offer.
  • Because each block problem is a small quadratic constrained binary program, faster specialized solvers, including quantum or quantum-inspired hardware, could directly enable larger blocks and better pruning.
  • The gains come from optimizing only a small fraction of weights, suggesting that the hard decisions are concentrated among weights that cheap scoring cannot confidently classify.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the fixing and selection steps imply most weights are easy to classify as keep or prune, so expensive optimization only needs to touch a small 'disputed' boundary; this supports a two-stage view of pruning where cheap scoring handles the bulk and optimization handles the margin.
  • Editorial extension: applying iCBS on top of SparseGPT instead of Wanda could combine SparseGPT's weight-reconstruction benefits with iCBS's iterative rebalancing.
  • Editorial extension: using richer selection scores, such as activation variance or range rather than mean activation, is a natural and testable way to improve which weights enter each block.
  • Editorial extension: the reported Mistral-7b runtime of 7.9 days suggests that, without algorithmic or hardware speedups, iCBS is currently a post-processing tool for one-shot pruning rather than a routine training-time method.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes iCBS, an iterative block-coordinate-descent pruning method that extends the Combinatorial Brain Surgeon to large models. Starting from an initial one-shot pruning (Wanda or magnitude), it fixes extreme-score weights, selects blocks by weight scores, and at each step solves a quadratic constrained binary optimization problem derived from a second-order Taylor expansion of the loss (Eq. 8) using a batch-estimated gradient and Hessian (Eq. 9). The authors report validation-accuracy gains over Wanda/magnitude baselines on Fashion-MNIST (Garment Classifier), ImageNet (DeiT-base), and seven-task LM evaluation (Mistral-7b), with only a small fraction of weights optimized (Table III), at the cost of substantially more compute (Table IV).

Significance. If the reported gains hold, iCBS is a meaningful advance in optimization-based pruning: it shows that a Hessian-based combinatorial objective can be applied to 7B-parameter models by block decomposition, and it explicitly trades compute for one-shot pruning quality. The derivation of the block objective from Eq. (4) via Eq. (7) is transparent and algebraically consistent, and the modular design (weight scoring, fixing, tabu, plug-in QCBO solver) is a useful contribution. However, the empirical support is currently not fully controlled: the main comparisons are single runs, hyperparameters were selected on the same validation metric that is reported, and the Hessian is approximated from very small batches; these issues should be resolved before the central claim can be considered established.

major comments (4)
  1. [Section V.A / Table V] For the Garment Classifier and DeiT, up to 100 hyperparameter configurations were tuned with Ray Tune on the validation accuracy that is later reported in Figs. 3-4, and the final parameters were obtained by averaging over the ~20 best configurations. Since iCBS has many more free parameters than Wanda (num_epochs, num_steps, block_size, num_restarts, batch sizes, alpha, lambda, tabu_frac, fix_frac_prune, fix_frac_keep), the reported improvements of +21.6%, +14.2%, and +7.7% (Fig. 6) may be inflated by validation-set overfitting. Please provide a tuner-free or nested-evaluation protocol: tune on a separate validation split, report test-set curves, or at least compare against Wanda under the same tuning budget.
  2. [Section V.B, Figs. 4 and 5] The iCBS curves appear to be single runs with no error bars, and the caption of Fig. 3 states that error bars are included only for the baselines. Because iCBS involves random batch draws, random layer selection, and stochastic simulated-annealing restarts, run-to-run variance is expected. Without confidence intervals or multiple seeds, the claimed gains over Wanda cannot be distinguished from noise. Please report means and standard deviations over at least 3-5 independent runs (or a justified deterministic protocol), for all methods and all plotted densities.
  3. [Section IV.A, Eq. (9), and Algorithm 1] The block Hessian is estimated as H_B ≈ (1/n) A^T A using n = batch_size_pruning samples; for Mistral-7b this is n=16, giving a rank-at-most-16 approximation to the 4096×4096 Hessian of each block. The paper gives no evidence that this batch-local quadratic model is sufficiently accurate to improve the true validation objective, nor any ablation on batch_size_pruning, block size, or number of solver restarts. Please add such ablations, or a direct check that solutions of Eq. (8) consistently reduce the validation loss, to support the claimed mechanism behind the gains.
  4. [Abstract and Section V.B, Table IV] The abstract highlights a quality-time tradeoff, but the paper only reports total runtime at density 50% (Table IV) and convergence curves (Figs. 4b and 5b) without a systematic accuracy-vs-cost comparison. Please quantify the tradeoff by varying num_steps, num_epochs, and block_size and plotting accuracy against pruning time, since this tradeoff is presented as a principal advantage over one-shot pruning.
minor comments (6)
  1. [Section IV.A, Eq. (9)] The symbol n is used for the block size in Eq. (8) and for the batch size in Eq. (9); please use distinct notation to avoid ambiguity.
  2. [Table III] The number of weights listed as "Optimized" is an upper bound, since blocks can revisit weights after tabu-list expiration; please state this explicitly in the caption or text.
  3. [Fig. 3 caption] The caption says error bars are included for all baselines, but iCBS is not a baseline; please clarify that the iCBS curve is a single run with no error bars.
  4. [Algorithm 1] The pseudocode uses num_steps_per_epoch while the text and Table V use num_steps; please align the terminology.
  5. [Appendix A] The constrained simulated annealing solver is described only by a reference; please provide the temperature schedule, number of sweeps, and stopping criteria to make the experiments reproducible.
  6. [General] The manuscript does not state whether source code and exact experimental configurations will be released; please add an availability statement, which is particularly important given the many implementation details in Appendix B.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: iCBS's pruning objective is a transparent Taylor/QCBO derivation, and its reported gains over Wanda are measured against external baselines rather than forced by construction.

full rationale

The derivation chain is self-contained and does not reduce to its own inputs. Eqs. (2)-(6) expand the loss in a second-order Taylor series around the trained weights, define binary pruning variables, and arrive at the CBS-style QCBO; Eqs. (7)-(8) are an algebraic substitution that generalizes the one-shot problem to per-block iterations. The block Hessian estimate H_B ≈ (1/n) A^T A in Eq. (9) is an explicit approximation attributed to Ref. [26], an external 1993 result, and is not a re-statement of the paper's target claim. The central empirical claim, that iCBS achieves higher validation accuracy than Wanda at matched density, is a measured benchmark result: iCBS is initialized from the best Wanda variant, but the improvement is not definitionally guaranteed, and the paper even reports cases (Mistral-7b at 50% density) where iCBS does not improve on the baseline. The self-citations present in the paper are not load-bearing: Ref. [19] is historical context for block coordinate descent in the quantum community, and Ref. [28] is the specific simulated-annealing block solver implementation used as a component; neither is invoked as a uniqueness theorem, nor does either smuggle in the central pruning result. Hyperparameter tuning on the validation set (up to 100 Ray Tune configurations for the small/medium models) is a legitimate correctness/selection-effect concern, but it is not circularity: the reported accuracies are measurements of the tuned configuration, not predictions defined in terms of the fitted values. No fitted parameter is renamed as a prediction, and no known result is repackaged as new by redefinition. Overall the paper's central derivation is independent of its inputs, so the circularity score is 0.

Assumptions & free parameters 15 free parameters · 6 assumptions · 0 invented entities

The central claim depends on a long chain of modeling assumptions: a truncated quadratic loss model, a cheap approximate Hessian from small batches, heuristic weight-fixing and selection, and a heuristic block solver. The paper provides empirical evidence for three models but no theoretical guarantees, so the ledger records the main unverified premises.

free parameters (15)
  • num_epochs = 10 (Garment, Mistral), 12 (DeiT)
    Outer-loop count; selected to observe convergence without excessive evaluation time.
  • num_steps = 300 (Garment, Mistral), 500 (DeiT)
    Inner-loop steps per epoch; selected to produce observable validation changes.
  • init_method = Magnitude per layer (Garment), Wanda per layer (DeiT), Wanda per output (Mistral)
    Initial one-shot pruning method; chosen as the best baseline per model.
  • selection_method = Gradient per layer (all models)
    Block selection scorer; authors report experiments showed gradient works best.
  • block_size (n) = 1024 (Garment, DeiT), 4096 (Mistral)
    Number of weights per block; limited by solver runtime and observed validation gains.
  • num_restarts = 10 (Garment, DeiT), 20 (Mistral)
    Parallel starts for the simulated annealing solver; chosen as the smallest number with visible improvements.
  • batch_size_evaluation = 4096 (Garment), 64 (DeiT), 1 (Mistral)
    Validation batch size; chosen for speed and GPU memory.
  • batch_size_pruning = 2000 (Garment), 1024 (DeiT), 16 (Mistral)
    Samples per pruning step for gradient and Hessian estimation; central to estimate quality.
  • batch_size_calibration = 4096 (Garment), 4096 (DeiT), 128 (Mistral)
    Samples for Wanda and gradient initial scoring; for LLMs set to 128 as in Ref. [7].
  • max_batch_size = None (Garment), 64 (DeiT), 1 (Mistral)
    Sub-batch limit for forward and backward passes; chosen for memory and speed.
  • grad_multiplier (alpha) = 0.75
    Gradient term weight in Eq. (8); tuned via hyperparameter optimization.
  • ridge_multiplier (lambda) = 0.001
    Ridge term weight in Eq. (8); tuned via hyperparameter optimization.
  • tabu_frac = 0.40
    Tabu list length as a fraction of layer weight count; tuned.
  • fix_frac_prune = 0.42
    Fraction of lowest-score initially pruned weights permanently fixed as pruned; tuned.
  • fix_frac_keep = 0.35
    Fraction of highest-score initially kept weights permanently fixed as kept; tuned.
assumptions (6)
  • domain assumption The loss function is locally quadratic so the third-order term in Eq. (2) can be neglected.
    Section III.B relies on O(delta_w^3) being small relative to the quadratic term, although this is not verified for LLM weight blocks.
  • domain assumption The block Hessian can be approximated by H_B ≈ (1/n) A^T A from per-sample gradients.
    Section IV.A, Eq. (9); this cheap estimate underlies every per-block decision.
  • domain assumption A small random data batch, as small as 16 samples for Mistral, gives gradient and Hessian estimates representative enough to improve validation accuracy.
    Section IV.B 'Estimate the gradient and Hessian'; unverified across data batches for LLMs.
  • ad hoc to paper Weights with extreme scores can be safely fixed as pruned or kept without combinatorial balancing.
    Section IV.B 'Fixing weights'; the fractions fix_frac_prune and fix_frac_keep are tuning choices, not derived guarantees.
  • domain assumption Weight-scoring selection plus tabu search directs optimization to the weights that matter.
    Section IV.B 'Selection' and 'Tabu list'; heuristic rationale inspired by extremal optimization, with no performance guarantee.
  • domain assumption The simulated annealing solver returns solutions close enough to optimal for each block that local decisions improve the global objective.
    Section IV.B 'Solve the per-block optimization problem'; the solver from Ref. [28] is not analyzed for optimality gap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalable iterative pruning of large language and vision models using block coordinate descent." pith.science (2026). https://pith.science/paper/7FJN5R7W

@misc{pith2026241117796,
  author       = {Pith},
  title        = {Pith review of: Scalable iterative pruning of large language and vision models using block coordinate descent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7FJN5R7W}},
  note         = {Machine review of arXiv:2411.17796}
}
read the original abstract

Pruning neural networks, which involves removing a fraction of their weights, can often maintain high accuracy while significantly reducing model complexity, at least up to a certain limit. We present a neural network pruning technique that builds upon the Combinatorial Brain Surgeon, but solves an optimization problem over a subset of the network weights in an iterative, block-wise manner using block coordinate descent. The iterative, block-based nature of this pruning technique, which we dub ``iterative Combinatorial Brain Surgeon'' (iCBS) allows for scalability to very large models, including large language models (LLMs), that may not be feasible with a one-shot combinatorial optimization approach. When applied to large models like Mistral and DeiT, iCBS achieves higher performance metrics at the same density levels compared to existing pruning methods such as Wanda. This demonstrates the effectiveness of this iterative, block-wise pruning method in compressing and optimizing the performance of large deep learning models, even while optimizing over only a small fraction of the weights. Moreover, our approach allows for a quality-time (or cost) tradeoff that is not available when using a one-shot pruning technique alone. The block-wise formulation of the optimization problem enables the use of hardware accelerators, potentially offsetting the increased computational costs compared to one-shot pruning methods like Wanda. In particular, the optimization problem solved for each block is quantum-amenable in that it could, in principle, be solved by a quantum computer.

Figures

Figures reproduced from arXiv: 2411.17796 by the authors.

Figure 1
Figure 1. FIG. 1: Pruning neural network (NN) weights. An example fully connected (feedforward) NN with three input nodes, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2: Schematic illustration of the per-block pruning process in iCBS. The [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3: Results for the Garment Classifier model on the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: FIG. 4: Results for the DeiT model on the ImageNet-1K dataset. These plots shows the dependence of the post [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5: Results for the Mistral-7b model pruned using the C4 dataset and validated using the LM Evaluation Harness. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6: Improvement in validation accuracy of iCBS over [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 24 canonical work pages

  1. [1]

    No pruning

    on the density for various types of pruning – the baselines and our pruner iCBS. The horizontal line labeled “No pruning” shows the validation accuracy of the unpruned model. Error bars are included for all baselines except magnitude (since it is deterministic) and show the standard deviation over 30 random repetitions. images accurately, out of 1000 clas...

    work page

  2. [2]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine 12 Learning Research, 23(120):1–39, 2022

    work page 2022

  3. [3]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    arXiv 2001

  4. [4]

    A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  5. [5]

    Block pruning for faster transformers

    François Lagunas, Ella Charlaix, Victor Sanh, and Alexander M Rush. Block pruning for faster transformers. arXiv preprint arXiv:2109.04838 , 2021

    arXiv 2021

  6. [6]

    The optimal BERT surgeon: Scalable and accurate second-order pruning for large language models

    Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, and Dan Alistarh. The optimal BERT surgeon: Scalable and accurate second-order pruning for large language models. arXiv preprint arXiv:2203.07259 , 2022

    arXiv 2022

  7. [7]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 , 2018

    arXiv 2018

  8. [8]

    A simple and effective pruning approach for large language models.arXiv preprint arXiv:2306.11695, 2023

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models.arXiv preprint arXiv:2306.11695, 2023

    arXiv 2023

Show all 48 references
  1. [9]

    SparseGPT: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. SparseGPT: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning , pages 10323–10337. PMLR, 2023

    2023

  2. [10]

    A user’s guide to tabu search

    Fred Glover, Eric Taillard, and Eric Taillard. A user’s guide to tabu search. Annals of operations research , 41 (1):1–28, 1993

    1993

  3. [11]

    Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    arXiv

  4. [12]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning , pages 10347–10357. PMLR, 2021

    2021

  5. [13]

    Optimal brain damage.Advances in neural information processing systems, 2, 1989

    Yann LeCun, John Denker, and Sara Solla. Optimal brain damage.Advances in neural information processing systems, 2, 1989

    1989

  6. [14]

    Optimal brain surgeon and general network pruning

    Babak Hassibi, David G Stork, and Gregory J Wolff. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks , pages 293–299. IEEE, 1993

    1993

  7. [15]

    Thecombinatorialbrainsurgeon: Pruning weights that cancel one another in neural networks

    Xin Yu, Thiago Serra, Srikumar Ramalingam, and ShandianZhe. Thecombinatorialbrainsurgeon: Pruning weights that cancel one another in neural networks. In International Conference on Machine Learning , pages 25668–25683. PMLR, 2022

    2022

  8. [16]

    Learning both weights and connections for efficient neural network

    Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015

    2015

  9. [17]

    What is the state of neural network pruning? Proceedings of machine learning and systems, 2:129–146, 2020

    Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? Proceedings of machine learning and systems, 2:129–146, 2020

    2020

  10. [18]

    Woodfisher: Efficient second-order approximation for neural network compression

    Sidak Pal Singh and Dan Alistarh. Woodfisher: Efficient second-order approximation for neural network compression. Advances in Neural Information Processing Systems, 33:18098–18109, 2020

    2020

  11. [19]

    Fast as CHITA: Neural network pruning with combinatorial optimization

    Riade Benbaki, Wenyu Chen, Xiang Meng, Hussein Hazimeh, Natalia Ponomareva, Zhe Zhao, and Rahul Mazumder. Fast as CHITA: Neural network pruning with combinatorial optimization. InInternational Conference on Machine Learning, pages 2031–2049. PMLR, 2023

    2023

  12. [20]

    Building an iterative heuristic solver for a quantum annealer

    Gili Rosenberg, Mohammad Vazifeh, Brad Woods, and Eldad Haber. Building an iterative heuristic solver for a quantum annealer. Computational Optimization and Applications, 65:845–869, 2016

    2016

  13. [21]

    From local to global ground states in Ising spin glasses

    Ilia Zintchenko, Matthew B Hastings, and Matthias Troyer. From local to global ground states in Ising spin glasses. Physical Review B, 91(2):024201, 2015

    2015

  14. [22]

    Partitioning optimization problems for hybrid classical

    Michael Booth, Steven P Reinhardt, and Aidan Roy. Partitioning optimization problems for hybrid classical. quantum execution. Technical Report, pages 01–09, 2017

    2017

  15. [23]

    The Llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  16. [24]

    A quantum approximate optimization algorithm.arXiv preprint arXiv:1411.4028, 2014

    Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. A quantum approximate optimization algorithm.arXiv preprint arXiv:1411.4028, 2014

    2014 arXiv

  17. [25]

    Quantum approximate optimization with hard and soft constraints

    Stuart Hadfield, Zhihui Wang, Eleanor G Rieffel, Bryan O’Gorman, Davide Venturelli, and Rupak Biswas. Quantum approximate optimization with hard and soft constraints. In Proceedings of the Second International Workshop on Post Moores Era Supercomputing , pages 15–21, 2017

    2017

  18. [26]

    Q-CHOP: Quantum constrained hamiltonian optimization

    Michael A Perlin, Ruslan Shaydulin, Benjamin P Hall, Pierre Minssen, Changhao Li, Kabir Dubey, Rich Rines, Eric R Anschuetz, Marco Pistoia, and Pranav Gokhale. Q-CHOP: Quantum constrained hamiltonian optimization. arXiv preprint arXiv:2403.05653 , 2024

    2024

  19. [27]

    Optimal brain surgeon: Extensions and performance comparisons

    Babak Hassibi, David Stork, and Gregory Wolff. Optimal brain surgeon: Extensions and performance comparisons. Advances in neural information processing systems , 6, 1993

    1993

  20. [28]

    Extremal optimization

    Stefan Boettcher. Extremal optimization. New optimization algorithms in physics , pages 227–251, 2004

    2004

  21. [29]

    Thomas Häner, Kyle E. C. Booth, Sima E. Borujeni, and Elton Yechao Zhu. Solving QUBOs with a quantum- amenable branch and bound method. arXiv preprint arXiv:2407.20185, 2024

    2024 arXiv

  22. [30]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  23. [31]

    BoolQ: Exploring the surprising difficulty of natural yes/noquestions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty of natural yes/noquestions. arXiv preprint arXiv:1905.10044, 2019

    1905 arXiv

  24. [32]

    GLUE: Amulti-task benchmark and analysis platform for natural language understanding

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. GLUE: Amulti-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461 , 2018

    2018 arXiv

  25. [33]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830 , 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830 , 2019

    1905 arXiv

  26. [34]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM , 64(9):99–106, 2021. 13

    2021

  27. [35]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  28. [36]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018

    2018 arXiv

  29. [37]

    Fashion- MNIST:anovelimagedatasetforbenchmarkingmachine learning algorithms

    Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion- MNIST:anovelimagedatasetforbenchmarkingmachine learning algorithms. arXiv preprint arXiv:1708.07747 , 2017

    2017 arXiv

  30. [38]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  31. [39]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research , 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research , 21(140):1–67, 2020

    2020

  32. [40]

    A framework for few-shot language model evaluation, 12 2023

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2023

  33. [41]

    Tune: A research platform for distributed model selection and training

    Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Stoica. Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118 , 2018

    2018 arXiv

  34. [42]

    Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures

    James Bergstra, Daniel Yamins, and David Cox. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In International conference on machine learning, pages115–

  35. [43]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  36. [44]

    Maxwell’s demon at work: Efficient pruning by leveraging saturation of neurons

    Simon Dufort-Labbé, Pierluca D’Oro, Evgenii Nikishin, Razvan Pascanu, Pierre-Luc Bacon, and Aristide Baratin. Maxwell’s demon at work: Efficient pruning by leveraging saturation of neurons. arXiv preprint arXiv:2403.07688, 2024

    2024

  37. [45]

    Not all tickets are equal and we know it: Guiding pruning with domain-specific knowledge

    Intekhab Hossain, Jonas Fischer, Rebekka Burkholz, and John Quackenbush. Not all tickets are equal and we know it: Guiding pruning with domain-specific knowledge. arXiv preprint arXiv:2403.04805 , 2024

    2024 arXiv

  38. [46]

    Quantum neural network compression

    Zhirui Hu, Peiyan Dong, Zhepeng Wang, Youzuo Lin, Yanzhi Wang, and Weiwen Jiang. Quantum neural network compression. In Proceedings of the 41st IEEE/ACM International Conference on Computer- Aided Design, pages 1–9, 2022

    2022

  39. [47]

    Qadaprune: Adaptive parameter pruning for training variational quantum circuits

    Ankit Kulshrestha, Xiaoyuan Liu, Hayato Ushijima- Mwesigwa, Bao Bach, and Ilya Safro. Qadaprune: Adaptive parameter pruning for training variational quantum circuits. arXiv preprint arXiv:2408.13352 , 2024

    2024 arXiv

  40. [48]

    gradient accumulation

    Valerie Sarge, Michael Andersch, Lynsey Fabel, Paulius Micikevicius, and John Tran. Tips for optimizing gpu performance using tensor cores, 2019. Appendix A: Experimental parameters The parameter values used for each of the experiments are presented in Table V. The parameters ...

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.