Pith. sign in

REVIEW 4 major objections 4 minor 36 references

Starting Positions Matter: A Study on Better Weight Initialization for Neural Network Quantization

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The choice of random weight initializer significantly affects how much accuracy a CNN loses to quantization, and a graph hypernetwork trained on quantized graphs (GHN-QAT) can predict robust weights for unseen architectures in one forward…

desk verdict GHN-QAT is a new and promising idea, but the simulation-only evaluation and missing fine-tuning experiments mean the headline claims are not yet fully supported. read the letter →

arxiv 2506.10463 v1 pith:OKYD6TFQ submitted 2025-06-12 cs.CV cs.AIeess.IV

Stone Yun , Alexander Wong This is my paper · ORCID
classification cs.CVcs.AIeess.IV
keywords neuralnetworkquantizationweightinitializationgraphhypernetworkquantization-awaretraininglow-bitprecisionefficientCNNCIFAR-10post-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that where a CNN starts in weight space — the random initialization — shapes not only its final 32-bit floating-point accuracy but also how much accuracy it loses when the trained weights and activations are quantized to low-bit integers for efficient inference. Across several common CNN building blocks trained on CIFAR-10, the same architecture can lose anywhere from under 1% to over 30% accuracy under 8-bit quantization depending solely on the initializer, and the layerwise ranges that result explain much of the difference. It then proposes a remedy: a graph hypernetwork, a message-passing network that predicts all parameters of a CNN directly from its computational graph, finetuned on quantized CNN graphs (called GHN-QAT) so that a single forward pass yields weights that are already quantization-robust. If the claim holds, initialization becomes a first-class design variable for low-bit models, and the expensive process of training or fine-tuning for quantization can be largely amortized into one hypernetwork.

What carries the argument

The central object is the Graph Hypernetwork (GHN), a message-passing network that treats a CNN as a graph of layers and predicts every trainable parameter in a single forward pass. The paper inherits GHN-2's channel-wise weight tiling and differentiable parameter normalization, which keep predicted layerwise distributions compact and channel-aligned, and finetunes the GHN on randomly generated DARTS-style efficient CNN graphs. GHN-QAT adds quantization simulation into the hypernetwork's training loop (straight-through estimator for most bitwidths, additive uniform noise for 2-bit), so the predicted parameters are optimized against quantized inference error. Quantization is modelled by the affine uniform map $Q=\lfloor \mathrm{clamp}(R,\min,\max)/s \rceil+Z$ applied to BatchNorm-folded weights, following standard integer-only inference practice.

What would settle it

Run a GHN-QAT-predicted W4/A4 CNN through bit-exact integer-only inference on a fixed-point accelerator or a bit-accurate software simulator and compare CIFAR-10 top-1 accuracy to the reported ~52.5% mean (and ~26.3% for W2/A2). If the on-device number falls far below the simulated number, the load-bearing assumption fails; if instead a GHN-QAT initialization trained with the same QAT budget does not beat a good random initialization, the practical benefit claim fails.

Watch

Extended reading notes

Core claim

The central claim is that quantization robustness is strongly inherited from the initial weight distribution, and that this dependence can be exploited by learning to initialize. The paper demonstrates the dependence with a sweep of Glorot, He, and generic uniform/normal initializers across regular, depthwise-separable, residual, and inverted-bottleneck blocks, showing large spreads in quantized accuracy that track layerwise weight and activation ranges. It then shows that a GHN finetuned in float32 on a dataset of 250,000 randomly generated mobile-friendly CNN graphs (GHN-Q) already predicts parameters that survive W8/A8 quantization with almost no loss (70.9% vs 71.1% float32 mean top-1 on in-distribution test graphs), and that finetuning the GHN with quantization simulated in the loop (GHN-QAT) raises mean top-1 to 60.2% at W4/A8, 52.5% at W4/A4, and 26.3% at W2/A2, the last being better than random chance. A separately finetuned BN-Free model shows that out-of-distribution failure can be corrected by specializing the GHN to a design space.

Load-bearing premise

The pipeline assumes that quantization simulated during training (straight-through gradient estimates or added uniform noise) matches true fixed-point inference closely enough, and that a hypernetwork trained on randomly generated small-network blueprints transfers to architectures it never saw.

Editorial extensions

If this is right

  • Quantization studies that omit the initializer are under-specified: the same architecture, data, and training recipe can land at very different quantized accuracies depending on the starting distribution.
  • GHN-QAT-predicted parameters could initialize quantization-aware training directly, removing the usual first step of training to convergence in float32 and then quantizing.
  • Encoding bitwidth and quantization scheme into the CNN graph could let one GHN serve multiple precisions, removing the need for a separate finetuned model per bitwidth.
  • The same parameter-prediction machinery could accelerate quantized neural architecture search by producing fast, quantized accuracy estimates for candidate architectures.
  • Specializing the GHN to a target design space (as done for BatchNorm-free networks) is an effective way to close prediction gaps for out-of-distribution architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication the authors leave implicit is that initializer rankings for quantization robustness may be training-budget dependent: at short budgets the starting point likely dominates, while very long fine-tuning could erode the differences seen at 200 epochs.
  • Their evidence that compact, channel-aligned distributions quantize well suggests a testable design principle for any initialization: keep inter-channel distributional mismatch and dynamic range small, not just variance scaled to fan-in/fan-out.
  • Since 2-bit training required switching from straight-through estimation to additive noise, the paper points to the quantization-noise model itself as a tunable ingredient; encoding both bitwidth and noise model into the graph could enable one GHN to serve mixed-precision networks.
  • All experiments are at CIFAR-10 scale and image classification; whether GHN-QAT transfers to higher-resolution inputs or transformer-like blocks is untested, and a positive result there would make the approach a general way to produce quantized model families.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies how weight initialization affects the quantized accuracy of CNNs. It first presents an empirical study on VGG-like and MobileNet-like CNN blocks, finding that the choice of random initializer (uniform vs normal, Glorot vs He vs modified Glorot, different scales) changes both fp32 accuracy and simulated 8-bit quantized accuracy. It then proposes GHN-Q and GHN-QAT, graph hypernetwork variants finetuned on a generated mobile-friendly CNN graph space, with the goal of predicting quantization-robust parameters for unseen architectures in a single forward pass. The GHN-QAT results are reported on CIFAR-10 for W8/A8, W4/A8, W4/A4, and W2/A2, including out-of-distribution Deep, Wide, and BN-Free test splits, and the paper reports that GHN-QAT improves over the float-pretrained GHN-Q baseline and achieves better-than-random accuracy at 2-bit weight/activation quantization.

Significance. The problem addressed here is relevant and underexplored: initialization is a standard design choice in floating-point training but is rarely considered as a variable for quantization robustness. The paper is commendably transparent about several limitations, including the use of simulated quantization rather than fixed-point inference, the instability of SimQuant for 2-bit training, and the need for checkpoint selection in the BN-Free experiments. The out-of-distribution evaluation across Deep, Wide, and BN-Free architectures is a strength, as is the inclusion of multiple bitwidth settings. If the claims are confirmed with proper baselines, repeated runs, and hardware validation, GHN-QAT could be a useful amortized alternative to per-model quantization-aware training. As it stands, the significance is limited by the absence of standard comparison baselines and by uncertainty in the magnitude of the reported effects.

major comments (4)
  1. [§3.1, Tables 1–2] The central empirical claim that initialization significantly affects quantized accuracy rests on single training runs: the setup reports one model per initialization/architecture combination and does not state that multiple seeds or repetitions were used. I recommend repeating the headline comparisons with multiple seeds and reporting mean ± std, since the observed differences in QUINT8 accuracy (e.g., the 30-point drop for DWS_Conv_With_BN_HeNorm versus RandUni_Large, or the 21-point drop for Basic_Residual_With_BN_RandUni_Large) need to be separated from run-to-run training noise.
  2. [§4.4, Tables 3–5] The GHN-QAT evaluation lacks the baselines needed to support the abstract's claims. The tables compare GHN-QAT only against the GHN-Q float-finetuned model; there is no comparison to (i) randomly initialized CNNs trained with the same QAT recipe, (ii) standard post-training quantization of normally trained CNNs, or (iii) QAT starting from standard initializers. I ask for at least one such baseline per bitwidth setting so that 'better-than-random accuracy' and 'significant accuracy improvements' are quantified relative to existing practice rather than only to the authors' own float-pretrained GHN.
  3. [§4.2, §4.4] All accuracy numbers in Tables 3–5 are produced by simulated quantization (Quantize nodes for evaluation, plus NoiseQuant for W2/A2 training) rather than by integer-only fixed-point inference. Since the strongest W2/A2 result is obtained by training against additive uniform noise, the paper should demonstrate transfer to at least one representative fixed-point implementation (e.g., via TFLite or an integer kernel) for W8/A8 and W4/A4, and ideally for W2/A2, before claiming that these quantized accuracy gains will materialize on device.
  4. [§4.5, Table 5] The BN-Free W4/A4 result is reported from a checkpoint at Epoch 8 after training diverged, with the meta-batchsize and clipping scheme changed relative to the protocol in §4.2. Please state how this checkpoint was selected (validation set, held-out criterion, or early stopping rule) and report the accuracy trajectory across training epochs. Without this information, the improvement over the OOD BN-Free result in Table 4 cannot be distinguished from selective reporting of a favorable checkpoint.
minor comments (4)
  1. [Throughout] There are several typos and formatting inconsistencies, including 'ommitted' and 'decibed', and inconsistent numeric formatting between Tables 1 and 2 (percent values in Table 1 versus fractions in Table 2).
  2. [§4.5] Section 4.5 says the batch size was reduced to 32 from 64, but §4.2 specifies a training batch size of 32; please clarify which value is correct.
  3. [§4.4] The phrase 'better-than-random accuracy' is used without a definition; specify whether it means better than 10% chance-level accuracy, better than random initialization under the same QAT protocol, or some other baseline.
  4. [Eq. (5)] Equation (5) uses clamping bounds (a, b) but the text does not define how these relate to the quantization range or stepsize; adding explicit definitions would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; GHN-QAT is trained and evaluated on held-out quantized graphs, and the main quantitative claims are measured, not derived from their own inputs.

full rationale

The paper's derivation chain is self-contained with respect to the claims it actually establishes. The random-initialization study (Section 3) directly trains and quantizes CNNs under different init schemes and observes accuracy differences; no fitted parameter is renamed as a prediction. The GHN-QAT pipeline (Sections 4.3-4.4) finetunes a Graph Hypernetwork on the ConvNets-250K quantized graph dataset and evaluates on held-out in-distribution and OOD (Deep/Wide/BN-Free) graphs; this is a standard meta-learning train/test split, not a reduction of the evaluation to the training objective. The W2/A2 result is explicitly trained with NoiseQuant (Eq. 5) and evaluated with Quantize() simulation, so the headline number is not the training loss by construction. Self-citations ([7], [14], [3]) are used for motivation, analysis framework, and explanation, but the main results do not reduce to those works; no uniqueness theorem or ansatz is imported from the authors' prior papers. The manuscript itself flags a limitation at Section 4.2: 'we only quantize the weights and activations with a Quantize() operator ... instead of running fully fixed-point inference ... the simulated quantization generally correlates well with on-device accuracy [5, 34, 35].' This is an external-validity assumption, not circularity: the simulated-quantization claim is derived and evaluated within the paper, while the transfer to on-device fixed-point arithmetic rests on cited prior evidence. Section 4.5 also notes post-hoc checkpoint selection ('the W4/A4 GHN-QAT testing was done with a checkpoint from Epoch 8'), which is a stability/selection concern rather than a circular step. The central derivation is independent of its inputs, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the method is a finetuned version of an existing GHN. The load-bearing assumptions are domain assumptions about simulation fidelity, transfer of the pretrained GHN, and graph coverage, plus a few hand-chosen hyperparameters that are not fitted to the main results but do affect the reported numbers.

free parameters (3)
  • top/bottom 1% percentile clipping threshold = 1%
    Chosen post hoc for BN-Free W4/A4 and W4/A8 after absolute-range clipping caused unstable training or worse accuracy (Section 4.5.1).
  • C in Modified Glorot Uniform = 36 and 1296
    Chosen by hand to represent medium and large initialization ranges (Section 3, Eq. 8).
  • meta-batch-size for W4/A4 BN-Free finetuning = 16 (batch-size reduced to 32)
    Increased from 4/64 to stabilize training; reported in Section 4.5.1.
assumptions (5)
  • domain assumption Quantization simulation correlates with on-device accuracy
    Invoked in Section 4.2 to justify evaluating SimQuant/NoiseQuant instead of real fixed-point inference; cited to refs [5,34,35].
  • domain assumption GHN-2 pretrained on ImageNet/DeepNets-1M transfers to efficient CNN graphs
    The paper finetunes a public GHN-2 checkpoint from [32] and assumes it provides a good starting point for ConvNets-250K (Section 4.2).
  • domain assumption Randomly generated DARTS-like graphs cover the efficient-CNN design space
    ConvNets-250K is generated by sampling connections and operations from a reduced DARTS space (Section 4.1); generalization to unseen architectures is assumed.
  • domain assumption CIFAR-10 is a sufficient benchmark for quantization robustness
    All experiments use CIFAR-10; no other dataset is tested (Sections 3 and 4).
  • standard math Uniform affine quantization equations (Eqs. 1-3) are standard
    Used to define the quantization mapping and are standard in the field.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Starting Positions Matter: A Study on Better Weight Initialization for Neural Network Quantization." pith.science (2026). https://pith.science/paper/OKYD6TFQ

@misc{pith2026250610463,
  author       = {Pith},
  title        = {Pith review of: Starting Positions Matter: A Study on Better Weight Initialization for Neural Network Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OKYD6TFQ}},
  note         = {Machine review of arXiv:2506.10463}
}
read the original abstract

Deep neural network (DNN) quantization for fast, efficient inference has been an important tool in limiting the cost of machine learning (ML) model inference. Quantization-specific model development techniques such as regularization, quantization-aware training, and quantization-robustness penalties have served to greatly boost the accuracy and robustness of modern DNNs. However, very little exploration has been done on improving the initial conditions of DNN training for quantization. Just as random weight initialization has been shown to significantly impact test accuracy of floating point models, it would make sense that different weight initialization methods impact quantization robustness of trained models. We present an extensive study examining the effects of different weight initializations on a variety of CNN building blocks commonly used in efficient CNNs. This analysis reveals that even with varying CNN architectures, the choice of random weight initializer can significantly affect final quantization robustness. Next, we explore a new method for quantization-robust CNN initialization -- using Graph Hypernetworks (GHN) to predict parameters of quantized DNNs. Besides showing that GHN-predicted parameters are quantization-robust after regular float32 pretraining (of the GHN), we find that finetuning GHNs to predict parameters for quantized graphs (which we call GHN-QAT) can further improve quantized accuracy of CNNs. Notably, GHN-QAT shows significant accuracy improvements for even 4-bit quantization and better-than-random accuracy for 2-bits. To the best of our knowledge, this is the first in-depth study on quantization-aware DNN weight initialization. GHN-QAT offers a novel approach to quantized DNN model design. Future investigations, such as using GHN-QAT-initialized parameters for quantization-aware training, can further streamline the DNN quantization process.

Figures

Figures reproduced from arXiv: 2506.10463 by the authors.

Figure 1
Figure 1. General Macroarchitecture of the CNN. For our analysis we use a fixed macro-architecture so that we can isolate the interactions between various weight initialization strategies and a few different convolutional layer choices. We train four variations of this macro-architecture determined by the type of conv-block used at each layer: Regular_Conv_With_BN, Regular_Conv_No_BN, DWS_Conv_With_BN, and DWS_Conv_No_BN. The… view at source ↗
Figure 2
Figure 2. General Macroarchitecture of the “Complex Block” CNN. Similar to the first study, we fix the macro-architecture and then vary the type of Complex Block used (i.e., Basic Residual vs. Inverted Bottleneck Residual). The very first convolution layer stays fixed for all architectures, but follows the With/Without BatchNorm behaviour of the rest of the layers. 5 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Layerwise plots of selected Simple-Conv-Block networks. Includes both Regular-Conv and DWS￾Conv. All Weights plots are using BN-Folded Weights 7 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Layerwise plots for selected Basic-Residual-style networks. All Weights plots are using BN-Folded Weights 3.2 Discussion We can see in both Tables 1 and 2 that besides affecting the final FP32 accuracy of a given CNN architecture, the weights initialization strategy al…
Figure 5
Figure 5. Figure 5: Layerwise plots for selected MbNetv2-style networks. All Weights plots are using BN-Folded Weights [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Overlaid plots of range and precision for MbNetv2_Conv_With_BN_He_Uni. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: GHN-QAT finetuned on ConvNets-250K. We generate a large number of CNN graphs [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 28 canonical work pages

  1. [2]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018, in Salt Lake City, USA, 18–22 June, 2018

    work page 2018

  2. [3]

    FactorizeNet: Progressive Depth Factorization for Efficient Network Architecture Exploration Under Quantization Constraints

    S. Yun and A. Wong, “Factorizenet: Progressive depth factorization for efficient network architecture exploration under quantization constraints,”CoRR, vol. abs/2011.14586, 2020

    work page Pith review arXiv 2011

  3. [4]

    Data-free quantization through weight equalization and bias correction,

    M. Nagel, M. Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” in2019 IEEE/CVF International Conference on Computer Vision (ICCV). Los Alamitos, CA, USA: IEEE Computer Society, nov 2019, pp. 1325–1334

    work page 2019

  4. [5]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. G. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” CoRR, vol. abs/1712.05877, 2017

    arXiv 2017

  5. [6]

    Convolutional neural networks using logarithmic data representation,

    D. Miyashita, E. H. Lee, and B. Murmann, “Convolutional neural networks using logarithmic data representation,”CoRR, vol. abs/1603.01025, 2016

    arXiv 2016

  6. [7]

    Yun and A

    S. Yun and A. Wong, “Do all mobilenets quantize poorly? gaining insights into the effect of quantization on depthwise separable convolutional networks through the eyes of multi-scale distributional dynamics,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2021, pp. 2447–2456

    work page 2021

  7. [8]

    Understanding the difficulty of training deep feedforward neural networks,

    X. Glorot and Y . Bengio, “Understanding the difficulty of training deep feedforward neural networks,” inJMLR, ser. Proceedings of Machine Learning Research, Y . W. Teh and M. Titter- ington, Eds., vol. 9. Chia Laguna Resort, Sardinia, Italy: JMLR Workshop and Conference Proceedings, 13–15 May 2010, pp. 249–256

    work page 2010

  8. [9]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,

    K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1026–1034

    work page 2015

Show all 36 references
  1. [10]

    How to start training: The effect of initialization and architecture,

    B. Hanin and D. Rolnick, “How to start training: The effect of initialization and architecture,” 2018. 15

    2018

  2. [11]

    Parameter prediction for unseen deep architectures,

    B. Knyazev, M. Drozdzal, G. W. Taylor, and A. Romero-Soriano, “Parameter prediction for unseen deep architectures,” inAdvances in Neural Information Processing Systems, 2021

    2021

  3. [12]

    Graph hypernetworks for neural architecture search,

    C. Zhang, M. Ren, and R. Urtasun, “Graph hypernetworks for neural architecture search,” in International Conference on Learning Representations, 2019

    2019

  4. [13]

    Can we scale transformers to predict parameters of diverse imagenet models?

    B. Knyazev, D. Hwang, and S. Lacoste-Julien, “Can we scale transformers to predict parameters of diverse imagenet models?” inInternational Conference on Machine Learning, 2023

    2023

  5. [14]

    An analysis framework for the quantization-aware design of efficient, low-power convolutional neural networks,

    Yun, Stone, “An analysis framework for the quantization-aware design of efficient, low-power convolutional neural networks,” Master’s thesis, University of Waterloo, 2022

    2022

  6. [15]

    Compressing deep convolutional networks using vector quantization,

    Y . Gong, L. Liu, M. Yang, and L. D. Bourdev, “Compressing deep convolutional networks using vector quantization,”CoRR, vol. abs/1412.6115, 2014

    2014 arXiv

  7. [16]

    Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks,

    S. Jain, A. Gural, M. Wu, and C. Dick, “Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks,” inProceedings of Machine Learning and Systems, I. Dhillon, D. Papailiopoulos, and V . Sze, Eds., vol. 2, 2020, pp. 112–128, in Aus...

    2020

  8. [17]

    Accurate and efficient 2-bit quantized neural networks,

    J. Choi, S. Venkataramani, V . V . Srinivasan, K. Gopalakrishnan, Z. Wang, and P. Chuang, “Accurate and efficient 2-bit quantized neural networks,” inProceedings of Machine Learning and Systems, A. Talwalkar, V . Smith, and M. Zaharia, Eds., vol. 1, 2019, pp. 348–359, in Stanf...

    2019

  9. [18]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” inProceedings of the 32nd International Conference on International Conference on Machine Learning - V olume 37, ser. ICML’15 in Lille, France, 6–11 July, 2...

    2015

  10. [19]

    Nice: Noise injection and clamping estimation for neural network quantization,

    C. Baskin, E. Zheltonozhkii, T. Rozen, N. Liss, Y . Chai, E. Schwartz, R. Giryes, A. M. Bronstein, and A. Mendelson, “Nice: Noise injection and clamping estimation for neural network quantization,”Mathematics, vol. 9, no. 17, 2021

    2021

  11. [20]

    Differentiable model compression via pseudo quantiza- tion noise,

    A. Défossez, Y . Adi, and G. Synnaeve, “Differentiable model compression via pseudo quantiza- tion noise,”TMLR, 2022

    2022

  12. [21]

    Hypernetworks,

    D. Ha, A. Dai, and Q. Le, “Hypernetworks,” 2016

    2016

  13. [22]

    The graph neural network model,

    F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,”IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009

    2009

  14. [23]

    Gated graph sequence neural networks,

    Y . Li, D. Tarlow, M. Brockschmidt, and R. S. Zemel, “Gated graph sequence neural networks,” inProceedings of the 4th International Conference on Learning Representations (ICLR), 2016

    2016

  15. [24]

    Neural message passing for quantum chemistry,

    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” inProceedings of the 34th International Conference on Machine Learning - V olume 70, ser. ICML’17. JMLR.org, 2017, p. 1263–1272

    2017

  16. [25]

    A gentle introduction to graph neural networks,

    B. Sanchez-Lengeling, E. Reif, A. Pearce, and A. B. Wiltschko, “A gentle introduction to graph neural networks,”Distill, 2021, https://distill.pub/2021/gnn-intro

    2021

  17. [26]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” CoRR, vol. abs/1704.04861, 2017

    2017 arXiv

  18. [27]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,”CoRR, vol. abs/1512.03385, 2015

    2015 arXiv

  19. [28]

    DARTS: Differentiable architecture search,

    H. Liu, K. Simonyan, and Y . Yang, “DARTS: Differentiable architecture search,” inInternational Conference on Learning Representations, 2019

    2019

  20. [29]

    MobileBERT: a compact task- agnostic BERT for resource-limited devices,

    Z. Sun, H. Yu, X. Song, R. Liu, Y . Yang, and D. Zhou, “MobileBERT: a compact task- agnostic BERT for resource-limited devices,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds....

    2020

  21. [30]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Kolesnikov, A. Dosovitskiy, D. Weissenborn, G. Heigold, J. Uszkoreit, L. Beyer, M. Minderer, M. Dehghani, N. Houlsby, S. Gelly, T. Unterthiner, and X. Zhai, “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021. 16

    2021

  22. [31]

    Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search,

    B. Wu, X. Dai, P. Zhang, Y . Wang, F. Sun, Y . Wu, Y . Tian, P. Vajda, Y . Jia, and K. Keutzer, “Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search,” in2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, p...

    2019

  23. [32]

    Facebookresearch/ppuda: Code for parameter prediction for unseen deep architectures (neurips 2021),

    B. Knyazev, M. Drozdzal, G. Taylor, and A. Romero-Soriano, “Facebookresearch/ppuda: Code for parameter prediction for unseen deep architectures (neurips 2021),” Oct 2021. [Online]. Available: https://github.com/facebookresearch/ppuda

    2021

  24. [33]

    Krizhevsky,Learning Multiple Layers of Features from Tiny Images, Apr 2009

    A. Krizhevsky,Learning Multiple Layers of Features from Tiny Images, Apr 2009. [Online]. Available: https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf

    2009

  25. [34]

    Quantizing deep convolutional networks for efficient inference: A whitepa- per,

    R. Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepa- per,”CoRR, vol. abs/1806.08342, 2018

    2018 arXiv

  26. [35]

    Aimet quantization simulation,

    Qualcomm, “Aimet quantization simulation,” 2020. [Online]. Available: https://quic.github.io/ aimet-pages/releases/latest/user_guide/quantization_sim.html

    2020

  27. [36]

    Overcoming oscillations in quantization-aware training,

    M. Nagel, M. Fournarakis, Y . Bondarenko, and T. Blankevoort, “Overcoming oscillations in quantization-aware training,” inProceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. ...

    2022

  28. [37]

    Esrgan: Enhanced super-resolution generative adversarial networks,

    X. Wang, K. Yu, S. Wu, J. Gu, Y . Liu, C. Dong, Y . Qiao, and C. Change Loy, “Esrgan: Enhanced super-resolution generative adversarial networks,” inProceedings of the European Conference on Computer Vision (ECCV) Workshops, September 2018. 17

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.