REVIEW 5 major objections 5 minor 1 cited by
Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning
T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Mixed-precision quantization policies can be searched on a small proxy dataset — CIFAR10 is 0.5% of ImageNet — and transferred to large datasets at equal accuracy, if the search steers toward flat loss minima.
desk verdict Novel transfer claim for MPQ policy search, but theoretical support is shaky and the efficiency comparison is under-specified; worth refereeing if the authors supply wall-clock budgets and code. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the surrogate gap $h(\theta) = L_p(\theta) - L(\theta)$, the difference between the loss at weights perturbed by a radius-$\rho$ step along the gradient and the loss at the current weights; through the identity $\sigma_{\max} \approx 2h(\theta)/\rho^2$ it serves as a cheap, computable measure of how sharply the loss rises around the minimum, and minimizing it is what the paper claims makes the policy portable. Around that object, the machinery has three parts: sharpness-aware minimization that descends on the perturbed loss $L_p$ instead of $L$; implicit gradient alignment, derived by Taylor expansion, that keeps $\nabla L_p(\theta)$ consistent with $\nabla L(\theta)$ so the surrogate-gap reduction does not fight the accuracy objective; and an adaptive radius $\rho = \min(\rho_{\max}, \phi/\ln(h(\theta)+1))$ that keeps the surrogate gap measurable as the landscape flattens. This objective is embedded in a differentiable MPQ supernet whose learnable bitwidth-selection weights are trained against a combined loss of accuracy, complexity in BOPs, and the sharpness regularizer.
What would settle it
Search the same ResNet18 under ASGA on two proxies engineered to converge to the same sharpness $\sigma_{\max}$ but very different distance to ImageNet — say CIFAR10 and a grayscale synthetic-noise dataset of equal size. The paper's mechanism predicts both policies transfer with near-equal ImageNet Top-1; a gap larger than the roughly one-point spread the paper reports across its own proxies would falsify sharpness as the transfer carrier. A minimal reproduction check: the CIFAR10-searched 3MP policy should reach about 67.9% ImageNet Top-1, 1.5 points above the 66.4% of the ImageNet-searched EdMIPS baseline.
Extended reading notes
Core claim
The paper's central claim is that the sharpness of the loss landscape at the searched solution governs whether a mixed-precision quantization policy transfers from a small proxy dataset to a large target dataset. ASGA wraps a differentiable MPQ search (a supernet with learnable per-layer bitwidth weights) in a sharpness objective: it perturbs the quantized weights by radius $\rho$ along the gradient direction, forming the perturbed loss $L_p(\theta)$, and minimizes the surrogate gap $h(\theta) = L_p(\theta) - L(\theta)$, which tracks the sharpness measure $\sigma_{\max} \approx 2h(\theta)/\rho^2$. Because $\nabla L_p(\theta)$ and $\nabla L(\theta)$ can point against each other, the method reformulates the objective as $\min_\theta L(\theta + (\rho/\|\nabla L(\theta)\| - \mu)\nabla L(\theta))$ to align the two gradients implicitly, and sets $\rho = \min(\rho_{\max}, \phi/\ln(h(\theta)+1))$ so the perturbation shrinks when the landscape is steep and grows as it flattens. The claimed results: proxy-searched policies match or beat policies searched on the target data itself — 67.9% versus 66.4% ImageNet Top-1 for ResNet18 at 3 mixed bits with 35 fewer search epochs, and similar gains for ResNet50, MobileNet-V2, and VOC detection — while a random 60K-image subset of ImageNet searched without ASGA lands 1.6 points below the CIFAR10-searched ASGA policy.
Load-bearing premise
The load-bearing premise is that a flat loss landscape achieved on the small proxy dataset carries over to guarantee small generalization error on a different, much larger target dataset, even though the formal bound in the paper only covers the proxy's own data distribution.
Editorial extensions
If this is right
- Search and deployment are decoupled: a policy found on a small proxy dataset deploys on a large target dataset after weight-only adjustment, with no large-scale quantization-aware policy search — the paper reports 150%, 127%, and 113% search-efficiency gains for ResNet18, ResNet50, and MobileNet-V2.
- Flatness buys accuracy, not just transfer: ASGA-searched policies match or beat target-searched baselines at the same bitwidths, such as 67.9% versus 66.4% Top-1 for ResNet18 and 74.9% versus 74.6% for ResNet50 on ImageNet.
- The transfer extends to other tasks: 3MP/4MP policies searched on CIFAR10 for VGG16-SSD and ResNet18-Faster R-CNN reach the same mAP on VOC as target-searched policies while using fewer search epochs.
- Proxy quality becomes measurable: across CIFAR10, Flowers, and Food, ASGA lowers the sharpness measure $\sigma_{\max}$, and the proxy with the lowest sharpness yields the best transferred ImageNet accuracy, so flatness can act as a selection signal for choosing a proxy.
- The sharpness regularizer is nearly free: ASGA reuses the already-computed gradient, so its per-step cost equals SAM's, while the adaptive radius shortens the search schedule (65–76 epochs versus 69–81 for fixed radii on CIFAR10).
Reading between the lines
- A stronger reading of the paper's premise, which it does not test, is that proxy-to-target transfer should survive even severe domain shifts — for example, searching on a grayscale or texture-only dataset for a natural-image target — if flatness is the whole story; running that experiment would separate flatness from class-similarity as the transfer driver.
- The adaptive-radius schedule is a specific, falsifiable design choice: $\rho$ grows exactly as $h(\theta)$ shrinks, which means the method starts near ordinary SAM and widens its neighborhood as training proceeds; ablating the schedule against a fixed final $\rho$ from step one would isolate whether the speedup comes from the schedule or from the flatness objective.
- The same recipe plausibly extends beyond bitwidths to other hardware-aware search objectives — latency, energy, or kernel-shape choices — because the transferable signal is the curvature of the loss landscape, not the quantization operator itself; that extension is implied by the paper's framing but not demonstrated.
- The paper's Lemma 1 bounds generalization on the proxy's own distribution; a natural continuation is an out-of-distribution bound that quantifies how proxy sharpness controls target risk, which would convert the empirical transfer results into a theorem-backed guarantee.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses the cost of mixed-precision quantization (MPQ) policy search by proposing to search on a small proxy dataset (e.g., CIFAR10, Flowers, Food) and transfer the resulting layer-wise bitwidth policy to a large target dataset (ImageNet, VOC). The proposed method, ASGA, adds a sharpness-aware term to differentiable MPQ search, uses an adaptive perturbation radius, and performs implicit gradient-direction alignment. The manuscript presents two theoretical lemmas (a PAC-Bayes style generalization bound and a convergence bound) as justification, and reports experiments on ResNet-18/50, MobileNet-V2, and object detection backbones, claiming accuracy comparable to or better than baselines while reducing search epochs and achieving up to 1.5x speedup.
Significance. The motivating problem is well chosen and practically valuable: decoupling MPQ policy search from the target dataset would make quantization far cheaper, especially for large models and privacy-sensitive scenarios. The paper includes useful experiments across multiple architectures and tasks, and the sharpness-aware formulation for MPQ search is a plausible and novel direction. If the transfer claim were firmly established, this would be a solid contribution. However, the theoretical support is currently not valid as stated, and the empirical protocol does not yet isolate the transfer mechanism from target-dataset fine-tuning. I therefore view the significance as conditional on substantial revision.
major comments (5)
- [Section 2.4.2, Eqs. (14)-(19)] Lemma 2's bound is not meaningful as stated. The term Θ = ρ²β²(3βγ0 − √T) + γ0βM becomes negative once T > 9β²γ0², so for large T the claimed upper bound in Eq. (14) can be negative while the left-hand side is nonnegative. Additionally, the proof in Eq. (15) analyzes the standard SAM update θ_{t+1} = θ_t − γ∇L_B(θ_p) with θ_p = θ + ρ∇L(θ)/‖∇L(θ)‖, not the ASGA objective in Eq. (10) with update direction (ρ/‖∇L(θ)‖ − μ)∇L(θ) and adaptive ρ. The convergence claim therefore does not apply to the proposed algorithm. Please either correct the analysis to the actual ASGA update and provide a valid nonnegative bound, or remove Lemma 2 and present the convergence behavior as an empirical observation.
- [Section 2.4.1 and Supplementary A.4] Lemma 1 does not establish the cross-dataset transfer claim. The PAC-Bayes bound in Eqs. (24)-(26) is over the proxy samples x_i and the weight posterior τ; the target distribution D_val and the quantized policy Q never appear in the bound. Moreover, since L_p(θ) = L(θ) + h(θ) with h(θ) ≥ 0 by construction, the statement that minimizing h tightens the bound is a restatement of the definition L_p ≥ L rather than an independent result. To support the headline claim, the authors need an explicit connection between sharpness on the proxy and generalization on a different target distribution (for example, a domain-adaptation style bound that quantifies distribution shift), or they should explicitly frame the transfer phenomenon as an empirical hypothesis.
- [Section 3.1 and Tables 1-2] The experimental protocol does not currently distinguish transfer due to flatness from target-dataset fine-tuning. The text states that after policy search the model is quantized and fine-tuned on the target dataset (ImageNet or VOC), so the reported accuracy includes substantial target-data training. To support the claim that the searched policy generalizes, please report complete wall-clock time including both search and fine-tuning, add a random-policy baseline with the same bitwidth budget and identical target fine-tuning, and compare against a policy search that does not use sharpness information. Without these controls, the reported accuracy gains could arise from the fine-tuning stage rather than from a transferable flat landscape.
- [Tables 1-4 and Figure 6] The empirical results are reported as single runs without error bars or number of seeds. Many differences are small (0.1–1.5 Top-1 points), so it is not possible to judge whether the claimed improvements are statistically significant. Please run multiple seeds for the main tables and for the proxy-dataset and ρ ablations, and report mean ± standard deviation (or confidence intervals) and the number of runs.
- [Equations (9)-(12)] The objective notation min_θ(L(θ), L_p(θ), h(θ)) in Eq. (9) is not a scalar objective; Eq. (12) reduces to min_θ (L(θ) + λL_comp(θ) + εL(θ + (ρ/‖∇L(θ)‖ − μ)∇L(θ))), and it is unclear how h(θ) is exactly optimized or how the gradient-alignment term relates to the final loss actually used for backpropagation. Please define the final scalar loss precisely, state the exact update rule, and explain the role of each hyperparameter (μ, ε, φ, ρ_max, ρ0) in the optimization.
minor comments (5)
- [Supplementary A.4] The section title 'Proof of Lamma 1' should be 'Proof of Lemma 1', and there are several typographical errors in the supplementary material ('reefers', 'mehohds', 'quanitzed', 'SDG' instead of SGD).
- [Eq. (4)] The notation ∥Bθ∥ and the sums over j and k are ambiguous; please define the cardinality of the candidate bitwidth sets and clarify that p_{l,θ}^j and p_{l,a}^k are the weight and activation softmax probabilities for layer l.
- [Figure 6] The caption of Figure 6(b) appears to contain garbled control characters in the rendered text; the heatmap description should be cleaned and the color-map legend should be explained.
- [Section 3.3.3 and Table 4] The comparison between 'subset of ImageNet without ASGA' and 'CIFAR10 with ASGA' varies two factors at once (proxy dataset and use of ASGA); a 2×2 ablation would be cleaner and would better support the conclusion that ASGA is responsible for the improvement.
- [Table 5] In the notation table, σ_max is described as 'the upper limit of the value of ρ', but in Section 2.2 it is defined as the dominant Hessian eigenvalue; please correct the table entry to be consistent with the main text.
Circularity Check
Partial circularity: Lemma 1 is a definitional restatement of the ASGA objective, while the central CIFAR10-to-ImageNet transfer claim is externally benchmarked and not circular.
-
self definitional
[Section 2.4.1 (Lemma 1) and Supplementary Material A.4 (Proof of Lemma 1)]
"where h(θ) =Lp(θ)−L (θ) denotes the surrogate gap. ... Here,L(θ) can be considered as a local minimum at this point, and then Lp(θ) , which is defined in Eq. (6), is always greater thanL(θ). As a result, the following inequality holds: Eθ∼τ ExiLi(θ)≤h(θ) +L(θ) +R, ... which implies that minimizing h(θ) is expected to achieve a tighter upper bound of the generalization performance."
The surrogate gap h(θ) is defined as Lp(θ)−L(θ), so the displayed inequality Eθ∼τ ExiLi(θ) ≤ h(θ)+L(θ)+R is obtained by substituting Lp = L+h into the PAC-Bayes bound E L_i ≤ L + R. The lemma's conclusion that minimizing h tightens the bound is therefore an algebraic consequence of the definition of h, not an independent mathematical result showing that sharpness minimization controls generalization. The paper's gloss that 'ASGA effectively reduces the upper bound' restates the ASGA objective, which explicitly minimizes h(θ), rather than providing independent theoretical support. Moreover, Supplementary A.4 concedes that Lp(θ)=h(θ)+L(θ) holds only when ρ equals ρ_true, so the equality underlying the lemma is not exactly the objective actually minimized (min(Lp,h)).
full rationale
The central empirical claim—that a mixed-precision policy searched on CIFAR10 transfers to ImageNet—is tested on external target data, so it cannot be reduced to the training objective by construction. No load-bearing self-citation chain exists: the SAM and PAC-Bayes machinery is cited to Foret et al. and McAllester, not to the authors' own prior work, and the adaptive-ρ and gradient-alignment components are novel and ablated. The main circularity is confined to Lemma 1 and its proof: h(θ) is defined as Lp(θ)−L(θ), so the bound Eθ∼τ ExiLi(θ) ≤ h(θ)+L(θ)+R and the claim that minimizing h tightens the bound merely restate the definition of h and the ASGA objective. The supplementary admission that Lp = h + L holds only when ρ = ρ_true further separates the theorem from the implemented objective. Because the theory is offered as validation but the headline transfer result is independently benchmarked, a partial-circularity score of 4 is appropriate.
Assumptions & free parameters
free parameters (6)
- Initial perturbation radius rho0 =
0.1
- ASGA loss weight epsilon =
0.1
- Complexity coefficient lambda =
not reported numerically
- Gradient-alignment step mu =
not reported
- Adaptive radius scale phi and cap rho_max =
not reported
- Gaussian noise scale b =
not reported
assumptions (4)
- standard math PAC-Bayes generalization bound (McAllester)
- standard math beta-smoothness and bounded gradient variance (Assumptions 1 and 2)
- ad hoc to paper Flat minima on the proxy dataset transfer to the target dataset
- domain assumption Differentiable MPQ supernet relaxation is a faithful surrogate for the discrete bitwidth search
Cite this review
Pith. "Pith review of Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning." pith.science (2026). https://pith.science/paper/374RULZI
@misc{pith2026250504877,
author = {Pith},
title = {Pith review of: Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning},
year = {2026},
howpublished = {\url{https://pith.science/paper/374RULZI}},
note = {Machine review of arXiv:2505.04877}
}
read the original abstract
Mixed Precision Quantization (MPQ) has become an essential technique for optimizing neural network by determining the optimal bitwidth per layer. Existing MPQ methods, however, face a major hurdle: they require a computationally expensive search for quantization policies on large-scale datasets. To resolve this issue, we introduce a novel approach that first searches for quantization policies on small datasets and then generalizes them to large-scale datasets. This approach simplifies the process, eliminating the need for large-scale quantization fine-tuning and only necessitating model weight adjustment. Our method is characterized by three key techniques: sharpness-aware minimization for enhanced quantization generalization, implicit gradient direction alignment to handle gradient conflicts among different optimization objectives, and an adaptive perturbation radius to accelerate optimization. Both theoretical analysis and experimental results validate our approach. Using the CIFAR10 dataset (just 0.5\% the size of ImageNet training data) for MPQ policy search, we achieved equivalent accuracy on ImageNet with a significantly lower computational cost, while improving efficiency by up to 150% over the baselines.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs
A training-free compression pipeline that updates calibration activations layer by layer and re-allocates ranks iteratively improves compressed LLM accuracy by up to a few points on five zero-shot tasks.
Reference graph
Works this paper leans on
-
[1]
Food-101– mining discriminative components with random forests
Bossard, L., Guillaumin, M., and Van Gool, L. Food-101– mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European con- ference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part VI 13, pp. 446–461. Springer,
2014
-
[6]
Single path one-shot neural architecture search with uniform sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y ., and Sun, J. Single path one-shot neural architecture search with uniform sampling. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pp. 544–560. Springer,
2020
-
[7]
Habi, H. V ., Jennings, R. H., and Netzer, A. Hmq: Hardware friendly mixed precision quantization block for cnns. In Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16, pp. 448–463. Springer,
work page 2020
-
[10]
Simplified pac-bayesian margin bounds
McAllester, D. Simplified pac-bayesian margin bounds. In Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27,
work page 2003
-
[13]
A., Tiedemann, S., Kemp, T., and Nakamura, A
Uhlich, S., Mauch, L., Yoshiyama, K., Cardinaux, F., Garcia, J. A., Tiedemann, S., Kemp, T., and Nakamura, A. Dif- ferentiable quantization of deep neural networks. arXiv preprint arXiv:1905.11452, 2(8),
arXiv 1905
-
[14]
Mixed precision quantization of convnets via dif- ferentiable neural architecture search
Wu, B., Wang, Y ., Zhang, P., Tian, Y ., Vajda, P., and Keutzer, K. Mixed precision quantization of convnets via dif- ferentiable neural architecture search. arXiv preprint arXiv:1812.00090,
-
[15]
Search what you want: Barrier panelty nas for mixed precision quantization
Yu, H., Han, Q., Li, J., Shi, J., Cheng, G., and Fan, B. Search what you want: Barrier panelty nas for mixed precision quantization. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16, pp. 1–16. Springer,
work page 2020
-
[17]
Surrogate gap minimization improves sharpness-aware training
Zhuang, J., Gong, B., Yuan, L., Cui, Y ., Adam, H., Dvornek, N., Tatikonda, S., Duncan, J., and Liu, T. Surrogate gap minimization improves sharpness-aware training. arXiv preprint arXiv:2203.08065,
Show all 20 references
-
[18]
Conventional methods require the consistency of datasets for bitwidth search and model deployment, while our GMPQ-ASGA searches the optimal quantization policy on small datasets and generalizes it to large-scale datasets. A.2. Details of Adaptive Sharpness-Aware Gradient Align...
2023
-
[19]
This shows that ASGA effectively reduces the upper bound of the generalization error during the MPQ process
denotes confidence level, and R = 4 q KL(τ∥ζ) + log 2m a /m. This shows that ASGA effectively reduces the upper bound of the generalization error during the MPQ process. Proof. According to the PAC-Bayeian theory (McAllester, 2003), we can employ Probably Approximately Correct...
2003
-
[20]
14 Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning Details of models
with 20 categories, containing about 1.6K training samples and 5K validation samples. 14 Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning Details of models. We employ four representative network architectur...
2016
-
[2003]
and Zisserman, A
Nilsback, M.-E. and Zisserman, A. Automated flower clas- sification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. 722–729. IEEE,
2008
-
[2010]
Sharpness-aware minimization for efficiently improving generalization
Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412,
2010 arXiv
-
[2016]
An adaptive policy to employ sharpness-aware minimization
Jiang, W., Yang, H., Zhang, Y ., and Kwok, J. An adaptive policy to employ sharpness-aware minimization. arXiv preprint arXiv:2304.14647,
-
[2018]
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee,
2009
-
[2020]
K., McKinstry, J
Esser, S. K., McKinstry, J. L., Bablani, D., Appuswamy, R., and Modha, D. S. Learned step size quantization. arXiv preprint arXiv:1902.08153,
1902 arXiv
-
[2021]
I.-J., Srini- vasan, V ., and Gopalakrishnan, K
Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srini- vasan, V ., and Gopalakrishnan, K. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085,
-
[2022]
and Zisserman, A
Simonyan, K. and Zisserman, A. Very deep convolu- tional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,
-
[2023]
S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P
Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836,
-
[2024]
Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients
Zhou, S., Wu, Y ., Ni, Z., Zhou, X., Wen, H., and Zou, Y . Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160,
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.