{"id":"6f6f2136-d457-4e47-8ea2-17d0f5253f0d","arxiv_id":"2505.04877","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A sharpness-aware, gradient-aligned search finds mixed-precision quantization policies on small proxy datasets that transfer to ImageNet with matching accuracy and faster convergence.","lead":"This paper proposes a way to choose per-layer bitwidths for quantized neural networks by searching on a small dataset like CIFAR10 and then reusing that policy on a large one like ImageNet. The method adds sharpness-aware minimization to mixed-precision quantization search and claims equal accuracy with up to 150% less search cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proxy flatness is not shown to transfer: Lemma 1 bounds only proxy-distribution generalization, so the CIFAR10-to-ImageNet transfer claim rests on an unproven premise.","rationale":"The reader's weakest_assumption is exactly the premise I would attack: flatness on the proxy is asserted to transfer to the target, and Lemma 1 does not establish it. I agree with that identification. I did not find a separate, stronger objection, though I note the cost claim should also be checked with wall-clock timings rather than convergence epochs; that is an additional reason to keep the verdict conditional rather than to accept. The paper deserves credit for attempting a theoretical framing and for comparing multiple proxy datasets and target tasks, but the theory is same-distribution and the experiments lack error bars and do not control for target fine-tuning. Therefore the central transfer claim is not yet supported, and the existing CONDITIONAL verdict is appropriate.","tokens_in":19815,"tokens_out":7941,"duration_ms":83716,"concrete_test":"Sample 20 random ResNet18 MPQ policies; for each, measure ASGA sharpness h(theta) on CIFAR10 and final ImageNet Top-1 after identical quantization-aware fine-tuning (same epochs, LR schedule, and seeds). Compute Spearman's rank correlation between proxy sharpness and target accuracy. If the correlation is not significantly negative, or if the ASGA-searched policy from Table 1 is not in the flattest quartile, then proxy flatness is not the driver of ImageNet transfer and the central claim lacks support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim has two coupled premises: (1) a flat proxy loss landscape transfers to a different large dataset, and (2) the reported speedup is a fair computational comparison. Premise (1) is the more fundamental and is not established. Lemma 1 (Sec. 2.4.1) bounds E_{theta~tau}E_{x_i}L_i(theta) <= L_p(theta)+R over the proxy distribution; the bound involves only the proxy samples x_i and the weight posterior tau, not the target distribution D_val and not the searched bitwidth policy Q. Minimizing sharpness on CIFAR10 is therefore not guaranteed to control anything on ImageNet. The proof in Supp. A.4 itself concedes that L_p(theta)=h(theta)+L(theta) only holds when rho equals rho_true, so the implemented objective min(L_p,h) is not the quantity the theorem analyzes. The empirical support is also not isolating: Section 3.1 states that after search the model is quantized and fine-tuned on the target dataset, so reported ImageNet accuracy includes target-data training that can mask poor transfer; Table 3 and Figure 6 report single runs with no error bars. If proxy flatness is not the operative mechanism, the method reduces to random policy search plus target fine-tuning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the cost of mixed-precision quantization (MPQ) policy search by proposing to search on a small proxy dataset (e.g., CIFAR10, Flowers, Food) and transfer the resulting layer-wise bitwidth policy to a large target dataset (ImageNet, VOC). The proposed method, ASGA, adds a sharpness-aware term to differentiable MPQ search, uses an adaptive perturbation radius, and performs implicit gradient-direction alignment. The manuscript presents two theoretical lemmas (a PAC-Bayes style generalization bound and a convergence bound) as justification, and reports experiments on ResNet-18/50, MobileNet-V2, and object detection backbones, claiming accuracy comparable to or better than baselines while reducing search epochs and achieving up to 1.5x speedup.","tokens_in":20135,"tokens_out":5606,"duration_ms":56533,"significance":"The motivating problem is well chosen and practically valuable: decoupling MPQ policy search from the target dataset would make quantization far cheaper, especially for large models and privacy-sensitive scenarios. The paper includes useful experiments across multiple architectures and tasks, and the sharpness-aware formulation for MPQ search is a plausible and novel direction. If the transfer claim were firmly established, this would be a solid contribution. However, the theoretical support is currently not valid as stated, and the empirical protocol does not yet isolate the transfer mechanism from target-dataset fine-tuning. I therefore view the significance as conditional on substantial revision.","major_comments":[{"comment":"Lemma 2's bound is not meaningful as stated. The term Θ = ρ²β²(3βγ0 − √T) + γ0βM becomes negative once T > 9β²γ0², so for large T the claimed upper bound in Eq. (14) can be negative while the left-hand side is nonnegative. Additionally, the proof in Eq. (15) analyzes the standard SAM update θ_{t+1} = θ_t − γ∇L_B(θ_p) with θ_p = θ + ρ∇L(θ)/‖∇L(θ)‖, not the ASGA objective in Eq. (10) with update direction (ρ/‖∇L(θ)‖ − μ)∇L(θ) and adaptive ρ. The convergence claim therefore does not apply to the proposed algorithm. Please either correct the analysis to the actual ASGA update and provide a valid nonnegative bound, or remove Lemma 2 and present the convergence behavior as an empirical observation.","section":"Section 2.4.2, Eqs. (14)-(19)"},{"comment":"Lemma 1 does not establish the cross-dataset transfer claim. The PAC-Bayes bound in Eqs. (24)-(26) is over the proxy samples x_i and the weight posterior τ; the target distribution D_val and the quantized policy Q never appear in the bound. Moreover, since L_p(θ) = L(θ) + h(θ) with h(θ) ≥ 0 by construction, the statement that minimizing h tightens the bound is a restatement of the definition L_p ≥ L rather than an independent result. To support the headline claim, the authors need an explicit connection between sharpness on the proxy and generalization on a different target distribution (for example, a domain-adaptation style bound that quantifies distribution shift), or they should explicitly frame the transfer phenomenon as an empirical hypothesis.","section":"Section 2.4.1 and Supplementary A.4"},{"comment":"The experimental protocol does not currently distinguish transfer due to flatness from target-dataset fine-tuning. The text states that after policy search the model is quantized and fine-tuned on the target dataset (ImageNet or VOC), so the reported accuracy includes substantial target-data training. To support the claim that the searched policy generalizes, please report complete wall-clock time including both search and fine-tuning, add a random-policy baseline with the same bitwidth budget and identical target fine-tuning, and compare against a policy search that does not use sharpness information. Without these controls, the reported accuracy gains could arise from the fine-tuning stage rather than from a transferable flat landscape.","section":"Section 3.1 and Tables 1-2"},{"comment":"The empirical results are reported as single runs without error bars or number of seeds. Many differences are small (0.1–1.5 Top-1 points), so it is not possible to judge whether the claimed improvements are statistically significant. Please run multiple seeds for the main tables and for the proxy-dataset and ρ ablations, and report mean ± standard deviation (or confidence intervals) and the number of runs.","section":"Tables 1-4 and Figure 6"},{"comment":"The objective notation min_θ(L(θ), L_p(θ), h(θ)) in Eq. (9) is not a scalar objective; Eq. (12) reduces to min_θ (L(θ) + λL_comp(θ) + εL(θ + (ρ/‖∇L(θ)‖ − μ)∇L(θ))), and it is unclear how h(θ) is exactly optimized or how the gradient-alignment term relates to the final loss actually used for backpropagation. Please define the final scalar loss precisely, state the exact update rule, and explain the role of each hyperparameter (μ, ε, φ, ρ_max, ρ0) in the optimization.","section":"Equations (9)-(12)"}],"minor_comments":[{"comment":"The section title 'Proof of Lamma 1' should be 'Proof of Lemma 1', and there are several typographical errors in the supplementary material ('reefers', 'mehohds', 'quanitzed', 'SDG' instead of SGD).","section":"Supplementary A.4"},{"comment":"The notation ∥Bθ∥ and the sums over j and k are ambiguous; please define the cardinality of the candidate bitwidth sets and clarify that p_{l,θ}^j and p_{l,a}^k are the weight and activation softmax probabilities for layer l.","section":"Eq. (4)"},{"comment":"The caption of Figure 6(b) appears to contain garbled control characters in the rendered text; the heatmap description should be cleaned and the color-map legend should be explained.","section":"Figure 6"},{"comment":"The comparison between 'subset of ImageNet without ASGA' and 'CIFAR10 with ASGA' varies two factors at once (proxy dataset and use of ASGA); a 2×2 ablation would be cleaner and would better support the conclusion that ASGA is responsible for the improvement.","section":"Section 3.3.3 and Table 4"},{"comment":"In the notation table, σ_max is described as 'the upper limit of the value of ρ', but in Section 2.2 it is defined as the dominant Hessian eigenvalue; please correct the table entry to be consistent with the main text.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is an ICML 2025 camera-ready version, which may affect the range of revisions the authors can make; however, as a referee I must judge the scientific content. The central transferability claim is plausible but currently rests on flawed theory and an experimental design that does not isolate the proposed mechanism. If the authors can provide corrected theoretical statements (or remove the invalid ones), add random-policy and no-ASGA controls, and report repeated-seed statistics, the paper could become a solid contribution. I would not recommend acceptance in the present form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The interesting bit is empirical: they claim a mixed-precision policy searched on CIFAR10 transfers to ImageNet with accuracy matching direct search on ImageNet, at a fraction of the cost. If that holds, it's a real practical gain for privacy-sensitive and compute-limited MPQ. The cross-dataset transfer experiment is not in the cited literature, so that's genuinely new.\n\nWhere it's good: the writing is clear, the method is simple (add a sharpness penalty to the differentiable MPQ objective), and they run a broad set of baselines and three proxy datasets. The ablation showing that a random ImageNet subset does worse than CIFAR10 with their method, if reliable, is a nice piece of evidence that something about the proxy choice matters.\n\nNow the soft spots. The theory doesn't do the work they want. Lemma 1 is essentially the definition h = Lp - L plus a PAC-Bayes bound; it only says flatness on the proxy controls generalization on the proxy. It doesn't touch the target distribution, so the central transfer claim rests on an unproven premise. Lemma 2's bound contains a term that goes negative for large T, which makes the claimed upper bound meaningless; more importantly, the proof analyzes a standard SAM update (Eq 15), not the ASGA objective in Eq 10. These aren't minor typos; they're load-bearing in the paper's framing, though not necessarily in the empirical result.\n\nThe empirical comparison also needs care. They report convergence epochs, not wall-clock time, and the baseline epoch counts may not include the same fine-tuning or initialization costs. Single runs, no error bars. And they fine-tune on the target dataset after search, so the reported ImageNet accuracy includes target-data training that could mask poor transfer. The efficiency claim of 150% speedup is therefore plausible but not yet demonstrated.\n\nMy overall take: the core empirical claim is interesting and testable; the theory is weak and should be either fixed or heavily de-emphasized. This is a paper a serious referee should engage with, but as presented it needs significant revision: wall-clock budgets, multiple seeds, released code, and a more honest treatment of what the theory does and doesn't show. I wouldn't cite it in my own work yet, but I'd bring it to a reading group to discuss the proxy-transfer question.","headline":"Novel transfer claim for MPQ policy search, but theoretical support is shaky and the efficiency comparison is under-specified; worth refereeing if the authors supply wall-clock budgets and code.","tokens_in":20651,"tokens_out":2065,"would_cite":false,"duration_ms":19573,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mixed-precision quantization policies can be searched on a small proxy dataset — CIFAR10 is 0.5% of ImageNet — and transferred to large datasets at equal accuracy, if the search steers toward flat loss minima.","keywords":["mixed-precision quantization","sharpness-aware minimization","transferable quantization policy","loss landscape sharpness","surrogate gap","differentiable architecture search","proxy dataset transfer","model compression"],"falsifier":"Search the same ResNet18 under ASGA on two proxies engineered to converge to the same sharpness $\\sigma_{\\max}$ but very different distance to ImageNet — say CIFAR10 and a grayscale synthetic-noise dataset of equal size. The paper's mechanism predicts both policies transfer with near-equal ImageNet Top-1; a gap larger than the roughly one-point spread the paper reports across its own proxies would falsify sharpness as the transfer carrier. A minimal reproduction check: the CIFAR10-searched 3MP policy should reach about 67.9% ImageNet Top-1, 1.5 points above the 66.4% of the ImageNet-searched EdMIPS baseline.","tokens_in":19638,"feed_emoji":"⚡","tokens_out":19906,"duration_ms":158168,"temperature":0.7,"pith_summary":"The paper claims that the expensive dataset-specific part of mixed-precision quantization — searching for the right per-layer bitwidth assignment — can be done once on a tiny proxy dataset and then transferred to a large target dataset at equal accuracy. Its mechanism for making the search transferable is to steer it toward flat minima of the loss landscape, because flat regions tolerate quantization noise well and the paper argues flatness is a dataset-independent property. The proposed method, ASGA, minimizes a surrogate gap between the perturbed and unperturbed losses, implicitly aligns their gradient directions to avoid conflicting updates, and adapts the perturbation radius as the landscape flattens. With CIFAR10 — about 0.5% of the ImageNet training data — as the proxy, the paper reports equivalent (sometimes higher) Top-1 accuracy on ImageNet for ResNet18, ResNet50, and MobileNet-V2 at up to 150% better search efficiency, and the same policies also transfer to object-detection backbones on VOC.","feed_headline":"A tiny proxy dataset finds ImageNet quantization policies 150% faster","feed_subtitle":"A sharpness-aware search on 0.5% of ImageNet's data matches full-dataset accuracy, no large-scale fine-tuning.","key_machinery":"The load-bearing object is the surrogate gap $h(\\theta) = L_p(\\theta) - L(\\theta)$, the difference between the loss at weights perturbed by a radius-$\\rho$ step along the gradient and the loss at the current weights; through the identity $\\sigma_{\\max} \\approx 2h(\\theta)/\\rho^2$ it serves as a cheap, computable measure of how sharply the loss rises around the minimum, and minimizing it is what the paper claims makes the policy portable. Around that object, the machinery has three parts: sharpness-aware minimization that descends on the perturbed loss $L_p$ instead of $L$; implicit gradient alignment, derived by Taylor expansion, that keeps $\\nabla L_p(\\theta)$ consistent with $\\nabla L(\\theta)$ so the surrogate-gap reduction does not fight the accuracy objective; and an adaptive radius $\\rho = \\min(\\rho_{\\max}, \\phi/\\ln(h(\\theta)+1))$ that keeps the surrogate gap measurable as the landscape flattens. This objective is embedded in a differentiable MPQ supernet whose learnable bitwidth-selection weights are trained against a combined loss of accuracy, complexity in BOPs, and the sharpness regularizer.","core_discovery":"The paper's central claim is that the sharpness of the loss landscape at the searched solution governs whether a mixed-precision quantization policy transfers from a small proxy dataset to a large target dataset. ASGA wraps a differentiable MPQ search (a supernet with learnable per-layer bitwidth weights) in a sharpness objective: it perturbs the quantized weights by radius $\\rho$ along the gradient direction, forming the perturbed loss $L_p(\\theta)$, and minimizes the surrogate gap $h(\\theta) = L_p(\\theta) - L(\\theta)$, which tracks the sharpness measure $\\sigma_{\\max} \\approx 2h(\\theta)/\\rho^2$. Because $\\nabla L_p(\\theta)$ and $\\nabla L(\\theta)$ can point against each other, the method reformulates the objective as $\\min_\\theta L(\\theta + (\\rho/\\|\\nabla L(\\theta)\\| - \\mu)\\nabla L(\\theta))$ to align the two gradients implicitly, and sets $\\rho = \\min(\\rho_{\\max}, \\phi/\\ln(h(\\theta)+1))$ so the perturbation shrinks when the landscape is steep and grows as it flattens. The claimed results: proxy-searched policies match or beat policies searched on the target data itself — 67.9% versus 66.4% ImageNet Top-1 for ResNet18 at 3 mixed bits with 35 fewer search epochs, and similar gains for ResNet50, MobileNet-V2, and VOC detection — while a random 60K-image subset of ImageNet searched without ASGA lands 1.6 points below the CIFAR10-searched ASGA policy.","pith_inferences":["A stronger reading of the paper's premise, which it does not test, is that proxy-to-target transfer should survive even severe domain shifts — for example, searching on a grayscale or texture-only dataset for a natural-image target — if flatness is the whole story; running that experiment would separate flatness from class-similarity as the transfer driver.","The adaptive-radius schedule is a specific, falsifiable design choice: $\\rho$ grows exactly as $h(\\theta)$ shrinks, which means the method starts near ordinary SAM and widens its neighborhood as training proceeds; ablating the schedule against a fixed final $\\rho$ from step one would isolate whether the speedup comes from the schedule or from the flatness objective.","The same recipe plausibly extends beyond bitwidths to other hardware-aware search objectives — latency, energy, or kernel-shape choices — because the transferable signal is the curvature of the loss landscape, not the quantization operator itself; that extension is implied by the paper's framing but not demonstrated.","The paper's Lemma 1 bounds generalization on the proxy's own distribution; a natural continuation is an out-of-distribution bound that quantifies how proxy sharpness controls target risk, which would convert the empirical transfer results into a theorem-backed guarantee."],"forward_implications":["Search and deployment are decoupled: a policy found on a small proxy dataset deploys on a large target dataset after weight-only adjustment, with no large-scale quantization-aware policy search — the paper reports 150%, 127%, and 113% search-efficiency gains for ResNet18, ResNet50, and MobileNet-V2.","Flatness buys accuracy, not just transfer: ASGA-searched policies match or beat target-searched baselines at the same bitwidths, such as 67.9% versus 66.4% Top-1 for ResNet18 and 74.9% versus 74.6% for ResNet50 on ImageNet.","The transfer extends to other tasks: 3MP/4MP policies searched on CIFAR10 for VGG16-SSD and ResNet18-Faster R-CNN reach the same mAP on VOC as target-searched policies while using fewer search epochs.","Proxy quality becomes measurable: across CIFAR10, Flowers, and Food, ASGA lowers the sharpness measure $\\sigma_{\\max}$, and the proxy with the lowest sharpness yields the best transferred ImageNet accuracy, so flatness can act as a selection signal for choosing a proxy.","The sharpness regularizer is nearly free: ASGA reuses the already-computed gradient, so its per-step cost equals SAM's, while the adaptive radius shortens the search schedule (65–76 epochs versus 69–81 for fixed radii on CIFAR10)."],"supporting_citations":[{"why":"Supplies SAM, the perturbed-loss flat-minimum optimizer that ASGA adapts to quantization policy search.","marker":"Foret et al., 2020"},{"why":"Provides EdMIPS, the differentiable mixed-precision supernet search that ASGA builds on and the main baseline it is compared against.","marker":"Cai & Vasconcelos, 2020"},{"why":"Supplies the surrogate-gap minimization and gradient-conflict analysis that motivates ASGA's gradient alignment.","marker":"Zhuang et al., 2022"},{"why":"The PAC-Bayesian theorem used to prove Lemma 1, the generalization bound sharpness minimization tightens.","marker":"McAllester, 2003"},{"why":"Gives the sharpness-aware gradient-matching Taylor expansion used to derive ASGA's implicit gradient-alignment objective.","marker":"Wang et al., 2023"},{"why":"GMPQ, the attribution-rank transferable-policy method the paper compares against and positions ASGA as an alternative.","marker":"Wang et al., 2021"},{"why":"HAQ, the hardware-aware quantizer whose roughly 72 GPU hours of ImageNet search motivates decoupling search from the target dataset.","marker":"Wang et al., 2019"},{"why":"SEAM, the class-margin transferable-policy baseline the paper re-implements and compares against in the ImageNet results.","marker":"Tang et al., 2023"}],"fun_headline_variants":["Tiny proxy set finds ImageNet MPQ policies, 150% faster","Sharpness-aware MPQ: search CIFAR10, match ImageNet, 150% speedup","MPQ policy transfer from 0.5% data, 150% efficiency gain","Loss landscape alignment makes quantization portable, 150% faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a flat loss landscape achieved on the small proxy dataset carries over to guarantee small generalization error on a different, much larger target dataset, even though the formal bound in the paper only covers the proxy's own data distribution.","fun_headline_variants_meta":{"raw":{"variants":["Tiny proxy set finds ImageNet MPQ policies, 150% faster","Sharpness-aware MPQ: search CIFAR10, match ImageNet, 150% speedup","MPQ policy transfer from 0.5% data, 150% efficiency gain","Loss landscape alignment makes quantization portable, 150% faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001017,"raw_usage":{"total_tokens":4359,"prompt_tokens":1079,"completion_tokens":3280,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":3193}},"tokens_in":695,"tokens_out":3280,"duration_ms":29121,"temperature":1.0,"reasoning_tokens":3193,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:18:57.566419+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Search the same ResNet18 under ASGA on two proxies engineered to converge to the same sharpness $\\sigma_{\\max}$ but very different distance to ImageNet — say CIFAR10 and a grayscale synthetic-noise dataset of equal size. The paper's mechanism predicts both policies transfer with near-equal ImageNet Top-1; a gap larger than the roughly one-point spread the paper reports across its own proxies would falsify sharpness as the transfer carrier. A minimal reproduction check: the CIFAR10-searched 3MP policy should reach about 67.9% ImageNet Top-1, 1.5 points above the 66.4% of the ImageNet-searched EdMIPS baseline.","supporting_citations":[{"cited_title":"Simplified pac-bayesian margin bounds","cited_arxiv_id":null,"evidence_quote":"The PAC-Bayesian theorem used to prove Lemma 1, the generalization bound sharpness minimization tightens."}],"review_version":1}