Pith. sign in

REVIEW 5 major objections 5 minor 43 references

SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Quantizing teacher features removes the loss floor in label-free quantization-aware training by projecting them onto the student's own quantization lattice, giving the student targets it can actually reach.

desk verdict Simple student-aware projection trick that mostly works; the theory is a tautology, but the empirical results make it a useful label-free QAT method. read the letter →

arxiv 2608.10709 v1 pith:EICVEFMS submitted 2026-08-11 cs.LG

HyeonJun Lee , Hyeonsik Jo , Jinwoo Chung , Jangho Kim This is my paper · ORCID
classification cs.LG
keywords Quantization-awaretrainingKnowledgedistillationLabel-freeQATStudent-awareprojectionFeatureLow-bitquantizationSelf-supervisedlearningNeuralnetworkcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that when a quantized student is trained by knowledge distillation from a full-precision teacher, using the teacher's raw features as targets creates an irreducible floor on the distillation loss, because the student cannot represent values outside its quantization lattice. SQuaT removes this floor by running teacher features through the student's own quantization function, projecting them onto the discrete lattice the student actually uses. This makes every target attainable, and Theorem 1 proves that the best achievable expected squared distance to such a target is zero. Empirically, label-free quantization-aware training with this student-aware projection consistently beats logit-only and teacher-calibrated baselines, with the largest gains at 1- and 2-bit widths, and is robust across architectures and loss functions.

What carries the argument

The central object is the student-aware projection $\Pi_{\phi_{S,\ell}}(z)$, defined as the student's forward quantization path applied to teacher features: normalize $z$ by the student's learned range $[a^-_\ell, a^+_\ell]$, round onto the $b$-bit uniform grid, and apply the type-specific linear mapping. It carries the argument by guaranteeing that the projected teacher feature lies in the set of outputs the student can actually produce, so the squared-error distillation loss over that target can be driven to zero; all empirical gains are attributed to this alignment.

What would settle it

Train a linear probe on the projected teacher features $\tilde{f}_{T,\ell} = \Pi_{\phi_{S,\ell}}(f_{T,\ell})$ at a distillation layer and compare its label-prediction accuracy with a probe on the raw teacher features $f_{T,\ell}$; if the projected features are near chance while the raw features are informative, the student-aware projection has discarded the teacher's signal and cannot be the source of the reported accuracy gains.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is that the distributional mismatch between full-precision teacher features and low-bit student features creates an unattainable residual. For a fixed student quantization map $\Pi$, the expected squared distance from any attainable student output to the teacher feature is at least $\mathbb{E}\|\Pi(f_T)-f_T\|^2$, and this lower bound grows as teacher values fall outside the student's clipping range. SQuaT eliminates the bound by replacing the target $f_T$ with the student-quantized target $\Pi(f_T)$, which lies in the student's attainable set, making zero expected residual reachable. The paper treats this projection as a paradigm shift from teacher-aware to student-aware projection and validates it across CNN, ViT, and transformer architectures, multiple loss functions, and bit-widths from 1 to 8.

Load-bearing premise

The load-bearing premise is that the student's learned quantization range, which is optimized for the student's own activation distribution, is also a suitable range for projecting teacher features; if teacher feature values fall mostly outside that range, the projected targets would discard much of the teacher's knowledge.

Editorial extensions

If this is right

  • In label-free settings, a student trained with SQuaT can match or exceed the accuracy of supervised QAT baselines in several configurations, including surpassing the full-precision teacher under 4-bit quantization on CIFAR-10 and CIFAR-100.
  • The largest gains are concentrated in extreme low-bit regimes (1- and 2-bit), where the representational mismatch between teacher and student is most severe and the lower bound is largest.
  • The method works across CNNs, Vision Transformers, and BERT, and with L1, L2, KL, and cosine feature losses, so the benefit is tied to the projection principle rather than a specific architecture or loss.
  • No teacher-side calibration pretraining is needed, removing the additional overhead required by teacher-quantizer-based approaches such as QFD.
  • Models trained with SQuaT at INT8 on an edge device achieve a 6.49x throughput speedup over FP32 inference while maintaining competitive accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same 'project into the student's representable set' principle should transfer to any distillation target with a constrained output space, such as binary or ternary embeddings, hashed features, or learned embedding tables, not just uniform quantizers.
  • The theorem bounds the squared-error loss, not task accuracy; the observed gains suggest the loss floor was indeed harmful, but a direct test would be to correlate the measured lower bound with final accuracy across layers and bit-widths.
  • The method's reliance on the student's learned range could be a liability early in training, when the range is still changing; an inference is that a warm-up or range-regularization schedule could stabilize or further improve SQuaT.
  • Because the projection uses the student's quantization parameters, the teacher signal itself becomes a function of the student's state, coupling the target distribution to training dynamics; this may reduce the information content of targets compared to full-precision features, so the net gain depends on how much teacher signal survives rounding.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes SQuaT, a label-free quantization-aware training (QAT) method combined with knowledge distillation (KD). The key idea is to quantize teacher intermediate features using the student's own quantization parameters, thereby projecting the teacher's full-precision features onto the student's quantization lattice before applying a feature-level distillation loss. The authors argue that prior teacher-aware projection methods leave an unattainable residual, creating an irreducible lower bound on the distillation loss, and that this bound is removed because SQuaT targets are attainable by construction. The paper proves a theorem to this effect (Theorem 1, Section 3.3) and presents experiments on CIFAR-10, CIFAR-100, ImageNet, Tiny-ImageNet, DeiT-Tiny, and BERT/GLUE, comparing against SQAKD and other baselines, with additional ablations on loss functions, projection schemes, and on-device throughput.

Significance. If the central claim is validated, student-aware projection is a simple and appealing idea that could improve label-free QAT in extreme low-bit regimes. The paper is strong in empirical breadth: it covers CNN and Transformer architectures, vision and NLP, multiple bit widths, several loss functions, and reports error bars and source code availability, which are concrete strengths. However, the theoretical contribution is elementary: Theorem 1 states a projection property that holds by definition, and it does not establish that the projected teacher features retain useful knowledge. The empirical evidence is the main support for the method, but it has gaps (e.g., no feature-loss ablation to zero or constant target, and one hyperparameter selected using test accuracy). The stress-test concern about the student's clipping range discarding teacher signal therefore lands directly and needs to be addressed.

major comments (5)
  1. [Section 3.3, Eq. (13)-(14)] Theorem 1 is a tautology rather than a substantive result. Since G is defined as the image of the student-aware projection and the target is chosen to be the projection of f_T onto G, part (b) holds by construction (select h = Pi(f_T)). The theorem does not address whether Pi(f_T) retains teacher knowledge, and the claimed 'elimination of the lower bound' is a direct consequence of defining the target as the projection. The paper must provide quantitative evidence that teacher features are not largely clipped by the student's learned range [a^-_l, a^+_l]. Please report, per distillation layer and bit width, the fraction of teacher feature components that fall inside the student's clipping range, or an equivalent coverage statistic. Without this, the projection could discard the teacher's signal and the method could still minimize the feature loss.
  2. [Section 3.2.1, Eq. (7) and Algorithm 1] Jointly optimizing the student's quantization parameters phi_S with the feature loss L_feat permits a degenerate solution: the student can shrink its clipping range so that both the student's quantized features and the projected teacher features collapse toward a small set of values, driving L_feat toward zero without transferring teacher knowledge. Theorem 1 does not protect against this collapse. Please add an analysis or experiment showing that the learned ranges remain informative, for example by reporting final a^-/a^+ values relative to teacher feature quantiles, or by evaluating a variant in which phi_S is frozen during feature distillation.
  3. [Section 4.5, Table 8] The sensitivity analysis selects lambda_feat = 10 based on the best test accuracy on CIFAR-100 with ResNet-32 at 4-bit, and this value is then used for the corresponding headline row in Table 2. This is test-set overfitting. The hyperparameter should be chosen on a held-out validation split, or the paper should demonstrate that the relative ranking of SQuaT versus SQAKD is stable across a range of lambda_feat values.
  4. [Section 4.4, Table 7] The projection-scheme comparison in Table 7 always includes the logit-level KD loss and does not include a baseline with lambda_feat = 0 or with a constant/uninformative feature target. As a result, the accuracy improvement attributed to student-aware projection could be partially or entirely due to the presence of any auxiliary feature loss, or to implicit regularization, rather than to the specific projection mechanism. Please add a no-feature-loss baseline and a constant-target baseline to isolate the effect.
  5. [Section 4.2, Tables 1-2] The claim that SQuaT 'surpasses the corresponding pretrained full-precision models' rests on margins of 0.01 and 0.04 percentage points on CIFAR-10 (Table 1, W4A4), with reported standard deviations of 0.01-0.03. These differences are not convincingly beyond noise. Please either provide significance tests or soften the claim to avoid overstating the result.
minor comments (5)
  1. [Tables 5 and 11] There are typos in table headers: 'R TE' should be 'RTE' in Table 5, and 'Accuarcy' should be 'Accuracy' in Table 11.
  2. [Figure 3] The learning curves in Figure 3 would be easier to interpret with explicit axis labels, a legend identifying the three projection schemes in each panel, and a note on which bit width each panel corresponds to.
  3. [Algorithm 1] Algorithm 1 updates phi_S for all layers L, not only the distillation layers K. Please clarify whether quantization parameters outside K are trained during feature alignment, and if so, how this interacts with the logit loss.
  4. [Section 3.1.2, Eq. (5)] The approximation sign in Eq. (5) and the hyperparameter eta are introduced without a precise statement of how they relate to the EWGS method; a short clarifying sentence or a pointer to the EWGS formulation would help.
  5. [Section 4.4, Table 7] For the Teacher-aware baseline, the number of calibration epochs and the calibration data used are not specified; adding this detail would make the comparison reproducible.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 1(b) removes the lower bound only by construction: the SQuaT target is defined as the projection onto the student's attainable set, so the zero infimum follows from the definition of the loss; the empirical ablations are independent and not circular.

  1. self definitional [Section 3.3 Theorem 1(b), with the target defined in Section 3.2.1, Eqs. (6)-(8)]
    "Let the target be ΠϕS,ℓ(fT,ℓ). Choosing h = ΠϕS,ℓ(fT,ℓ) ∈ GϕS,ℓ achieves zero error pointwise, which implies infh∈GϕS,ℓ E∥h − ΠϕS,ℓ(fT,ℓ)∥2^2 = 0."

    The headline claim that SQuaT 'removes this lower bound' is asserted as a theorem, but it is an immediate consequence of how the objective is defined: Eq. (7) sets the target to the student-aware projection ~fT,ℓ = Π_φS,ℓ(f_T,ℓ), and G_φS,ℓ is defined as Im(Π_φS,ℓ), so ~fT,ℓ ∈ G by construction. Any target chosen from G — including a constant — yields infimum zero, so Theorem 1(b) is a tautology of the target definition. The theorem never establishes that Π_φS(f_T) retains teacher knowledge: the load-bearing assumption that the student's learned clipping range is also a good projection range for teacher features is untested (Figure 1 is qualitative; no coverage statistic is reported), and Table 7 keeps logit KD in every row, with no constant-target or feature-loss-zero control.

full rationale

The paper's empirical core is self-contained and not circular: SQuaT is evaluated against SQAKD and supervised EWGS across CIFAR-10/100, ImageNet-1K, Tiny-ImageNet, DeiT-Tiny, and BERT/GLUE, at multiple bit-widths, with ablations over loss functions, quantizers, and projection schemes (Tables 1-11, Figs. 2-3). These are external benchmarks, and the accuracy gains are measured outcomes, not fitted parameters renamed as predictions. Self-citations (Back et al. 2025, Kim et al. 2023) are related-work credits and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The circularity is confined to the theoretical framing in Section 3.3: Theorem 1(a) restates that, for the rounding-to-nearest uniform quantizer, the closest point in the attainable set is the quantization of the teacher feature — a definitional fact about projections; Theorem 1(b) asserts the SQuaT objective's infimum is zero because the target was placed in the attainable set by definition. Consequently, the paper's central theoretical contribution — 'theoretically eliminates this lower bound' — reduces to the definition of the target, i.e., pattern 1 (self_definitional). The paper also omits the controls that would give the projection independent content: no quantitative coverage of teacher features by [a^-, a^+], no ablation dropping L_feat or substituting a constant target, and all Table 7 rows include logit KD, so the projection's contribution is not isolated. Because the headline theoretical claim reduces by construction while the experimental evaluation is genuinely independent, the appropriate score is 6 (partial circularity).

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical entities, fields, or conserved quantities are introduced. The method reuses the student's existing quantization parameters and introduces no ad hoc variables beyond ordinary hyperparameters.

free parameters (1)
  • lambda_feat (feature loss weight) = 10 (CIFAR-100 ResNet-32 4-bit); 1 elsewhere
    Selected by evaluating test accuracy across values in Table 8, then used in the main results Table 2. This is a test-set-based hyperparameter choice.
assumptions (4)
  • domain assumption Feature-level distillation transfers useful knowledge from teacher to student.
    Motivates L_feat in Eq. (9). Standard KD assumption; the paper does not prove it for quantized students.
  • standard math The attainable set G is exactly the image of the student's forward quantization map.
    Used in Theorem 1; true by definition in Section 3.3.
  • domain assumption The unlabeled data distribution D matches the distribution on which the teacher performs well.
    The method trains on unlabeled data D; if D is out-of-distribution, the teacher features are meaningless.
  • domain assumption The EWGS gradient approximation in Eq. (5) is a valid way to train the quantization parameters.
    Adopted from Lee et al. (2021); the paper relies on it for updating phi_S.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features." pith.science (2026). https://pith.science/paper/EICVEFMS

@misc{pith2026260810709,
  author       = {Pith},
  title        = {Pith review of: SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EICVEFMS}},
  note         = {Machine review of arXiv:2608.10709}
}
read the original abstract

Quantization-Aware Training (QAT) enables the deployment of quantized models with minimal accuracy degradation. However, in practical scenarios, training labels are often unavailable due to privacy, copyright, or cost constraints. Knowledge Distillation (KD) is a common approach to address this challenge, but we observe that prior work combining QAT with KD suffers from a fundamental limitation: during distillation, the range mismatch between the teacher and the quantized student model induces an unattainable residual, resulting in an irreducible lower bound on the distillation loss. Motivated by this observation, we propose SQuaT (Student-Aware Quantized Teacher Features), a label-free QAT framework with KD that theoretically eliminates this lower bound by applying the student's quantization parameters to quantize the teacher's features during distillation. Through comprehensive experiments across diverse settings, we demonstrate that SQuaT consistently outperforms strong baselines, with particularly pronounced gains in extreme low-bit (e.g., 1- and 2-bit) settings. Furthermore, extensive evaluations across various model design choices show that our approach does not rely on specific architectural assumptions, making it broadly applicable across diverse architectures and quantization settings. The source code is available at https://github.com/lcdbsa522/SQuaT.

Figures

Figures reproduced from arXiv: 2608.10709 by the authors.

Figure 1
Figure 1. Distribution of feature maps for the FP teacher and 4-bit student in ResNet-20 on CIFAR-10. 1 INTRODUCTION Over the past decade, Deep Neural Networks (DNNs) have emerged as a dominant modeling paradigm in ar￾tificial intelligence, achieving remarkable performance across a wide range of domains, including computer vi￾sion and natural language processing. However, these advances have increasingly relied on larger mode… view at source ↗
Figure 2
Figure 2. Overall framework of SQuaT. By projecting teacher features onto the student’s quantization lattice [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Training curves of feature distillation loss and Top-1 test accuracy under different bit-width settings. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 11 canonical work pages

  1. [1]

    Advances in neural information processing systems , volume=

    Bootstrap your own latent-a new approach to self-supervised learning , author=. Advances in neural information processing systems , volume=

  2. [2]

    International conference on machine learning , pages=

    A simple framework for contrastive learning of visual representations , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  3. [3]

    arXiv preprint arXiv:2106.08295 , year=

    A white paper on neural network quantization , author=. arXiv preprint arXiv:2106.08295 , year=

    arXiv

  4. [4]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Quantization and training of neural networks for efficient integer-arithmetic-only inference , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  5. [5]

    arXiv preprint arXiv:1503.02531 , year=

    Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=

    arXiv

  6. [6]

    International Conference on Artificial Intelligence and Statistics , pages=

    Self-Supervised Quantization-Aware Knowledge Distillation , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2024 , organization=

    work page 2024

  7. [7]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Network quantization with element-wise gradient scaling , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    work page

  8. [8]

    arXiv preprint arXiv:1308.3432 , year=

    Estimating or propagating gradients through stochastic neurons for conditional computation , author=. arXiv preprint arXiv:1308.3432 , year=

    arXiv

Show all 43 references
  1. [9]

    arXiv preprint arXiv:1810.00861 , year=

    Proxquant: Quantized neural networks via proximal operators , author=. arXiv preprint arXiv:1810.00861 , year=

    arXiv

  2. [10]

    Advances in Neural Information Processing Systems , volume=

    Metaquant: Learning to quantize by learning to penetrate non-differentiable quantization , author=. Advances in Neural Information Processing Systems , volume=

  3. [11]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Differentiable soft quantization: Bridging full-precision and low-bit neural networks , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  4. [12]

    arXiv preprint arXiv:1911.12491 , year=

    Qkd: Quantization-aware knowledge distillation , author=. arXiv preprint arXiv:1911.12491 , year=

    1911 arXiv

  5. [13]

    arXiv 2014 , author=

    Fitnets: Hints for thin deep nets. arXiv 2014 , author=. arXiv preprint arXiv:1412.6550 , year=

    2014 arXiv

  6. [14]

    arXiv preprint arXiv:1612.03928 , year=

    Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer , author=. arXiv preprint arXiv:1612.03928 , year=

    arXiv

  7. [15]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    A comprehensive overhaul of feature distillation , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  8. [16]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    A gift from knowledge distillation: Fast optimization, network minimization and transfer learning , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  9. [17]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Knowledge transfer via distillation of activation boundaries formed by hidden neurons , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  10. [18]

    arXiv preprint arXiv:1902.08153 , year=

    Learned step size quantization , author=. arXiv preprint arXiv:1902.08153 , year=

    1902 arXiv

  11. [19]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Learning to quantize deep networks by optimizing quantization intervals with task loss , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  12. [20]

    arXiv preprint arXiv:1805.06085 , year=

    Pact: Parameterized clipping activation for quantized neural networks , author=. arXiv preprint arXiv:1805.06085 , year=

    arXiv

  13. [21]

    arXiv preprint arXiv:1905.11452 , volume=

    Differentiable quantization of deep neural networks , author=. arXiv preprint arXiv:1905.11452 , volume=. 2019 , publisher=

    1905 arXiv

  14. [22]

    arXiv preprint arXiv:1909.13144 , year=

    Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks , author=. arXiv preprint arXiv:1909.13144 , year=

    1909 arXiv

  15. [23]

    Learning multiple layers of features from tiny images.(2009) , author=

    2009

  16. [24]

    2009 IEEE conference on computer vision and pattern recognition , pages=

    Imagenet: A large-scale hierarchical image database , author=. 2009 IEEE conference on computer vision and pattern recognition , pages=. 2009 , organization=

    2009

  17. [25]

    CS 231N , volume=

    Tiny imagenet visual recognition challenge , author=. CS 231N , volume=

  18. [26]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Deep residual learning for image recognition , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  19. [27]

    arXiv preprint arXiv:1409.1556 , year=

    Very deep convolutional networks for large-scale image recognition , author=. arXiv preprint arXiv:1409.1556 , year=

    arXiv

  20. [28]

    Advances in neural information processing systems , volume=

    Pytorch: An imperative style, high-performance deep learning library , author=. Advances in neural information processing systems , volume=

  21. [29]

    The annals of mathematical statistics , pages=

    A stochastic approximation method , author=. The annals of mathematical statistics , pages=. 1951 , publisher=

    1951

  22. [31]

    arXiv preprint arXiv:1412.6980 , year=

    Adam: A method for stochastic optimization , author=. arXiv preprint arXiv:1412.6980 , year=

    arXiv

  23. [32]

    arXiv preprint arXiv:1711.05101 , year=

    Decoupled weight decay regularization , author=. arXiv preprint arXiv:1711.05101 , year=

    arXiv

  24. [33]

    arXiv preprint arXiv:1608.03983 , year=

    Sgdr: Stochastic gradient descent with warm restarts , author=. arXiv preprint arXiv:1608.03983 , year=

    arXiv

  25. [34]

    arXiv preprint arXiv:1510.00149 , year=

    Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding , author=. arXiv preprint arXiv:1510.00149 , year=

    arXiv

  26. [35]

    Expert Systems with Applications , volume=

    Magnitude attention-based dynamic pruning , author=. Expert Systems with Applications , volume=. 2025 , publisher=

    2025

  27. [36]

    Proceedings of the 31st ACM International Conference on Multimedia , pages=

    Finding efficient pruned network via refined gradients for pruned weights , author=. Proceedings of the 31st ACM International Conference on Multimedia , pages=

  28. [37]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Quantized feature distillation for network quantization , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  29. [38]

    arXiv preprint arXiv:2010.11929 , year=

    An image is worth 16x16 words: Transformers for image recognition at scale , author=. arXiv preprint arXiv:2010.11929 , year=

    2010 arXiv

  30. [39]

    International conference on machine learning , pages=

    Training data-efficient image transformers & distillation through attention , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  31. [40]

    Bert: Pre-training of deep bidirectional transformers for language understanding , author=. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) , pages=

    2019

  32. [41]

    Proceedings of the 2018 EMNLP workshop BlackboxNLP: Analyzing and interpreting neural networks for NLP , pages=

    GLUE: A multi-task benchmark and analysis platform for natural language understanding , author=. Proceedings of the 2018 EMNLP workshop BlackboxNLP: Analyzing and interpreting neural networks for NLP , pages=

    2018

  33. [42]

    International conference on machine learning , pages=

    Oscillation-free quantization for low-bit vision transformers , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  34. [43]

    arXiv preprint arXiv:1605.04711 , year=

    Ternary weight networks , author=. arXiv preprint arXiv:1605.04711 , year=

    arXiv

  35. [44]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Q-bert: Hessian based ultra low precision quantization of bert , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.