{"id":"6f408d36-6022-4985-9da8-cc6785ef58e9","arxiv_id":"2412.05551","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A gradient-disorder trigger that selectively freezes task gradients of quantizer scale factors improves quantized domain-generalization accuracy, including near-lossless 4-bit results on DomainNet.","lead":"Researchers combined two known training tricks, flat-minimum domain generalization and low-bit quantization, and found they fight each other inside the model. They introduce a method that detects when a quantizer's scaling-factor gradients conflict and selectively freezes one gradient stream, improving out-of-distribution accuracy at 3 and 4 bits.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own tables contradict each other for the identical GAQAT configuration, so the claimed 4.4-point gain on PACS is not backed by stable measurements.","rationale":"The reader's weakest assumption concerns the unproven mechanism and the entanglement of the freezing threshold r and interval K with validation-based tuning. I agree that the mechanism lacks theoretical support, but the more load-bearing problem is that the paper's own measurements are internally inconsistent for what should be the same experimental configuration. Table 4, Table 5, and Table 6 present different test accuracies for the 4-bit PACS setting with r=0.30 and K=350; the largest per-domain swing is 3.89 points, and the average differs by 1.67 points. The central claim is a 4.40-point improvement over SAGM+LSQ, so the reported effect is not clearly larger than the paper's own run-to-run variability. Since no code or seeds are available, a reader cannot determine which table reflects the true performance. This does not force rejection of the underlying idea—the mechanism could still work—but it does force a conditional stance: the empirical evidence must be stabilized and reconciled before the claim can be relied upon. The manuscript's stated limitations are reasonable and do not address this inconsistency. I therefore keep the reader's CONDITIONAL verdict, while noting that the reason for conditionality should explicitly include the table contradiction and the need for seeded, reproducible runs.","tokens_in":14686,"tokens_out":8090,"duration_ms":71153,"concrete_test":"Run the exact PACS 4-bit configuration used in Table 4 (r=0.30, K=350, same MoCo-v2 initialization and ERM fine-tuning baseline) for at least 5 seeds, and report per-domain and average test accuracies with seed-level standard deviations. Then check whether Tables 4, 5 (K=350), and 6 (r=0.30) are reproducible from that single run, and compare the mean and 95% confidence interval against SAGM+LSQ run under the same protocol. If the seed-to-seed standard deviation exceeds about 1.5 points or the confidence interval includes the SAGM+LSQ mean, the claimed 4.40-point advantage is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that GAQAT's selective freezing of task gradients for low-disorder scale factors improves OOD accuracy (Table 2, PACS 4-bit 61.33% vs 56.93% for SAGM+LSQ). However, the paper never reports the same configuration twice with the same result. Table 4's 'Ours' row explicitly uses freeze steps=350 and threshold r=0.30, giving average test accuracy 59.33% (Photo 67.51, Sketch 62.37). Table 5 row 350 reproduces exactly those numbers, but Table 6 row r=0.30, which should be the same configuration under the stated protocol of fixing one hyperparameter while varying the other, reports average test accuracy 57.66% (Photo 63.62, Cartoon 56.45) — a 1.67-point discrepancy driven by large per-domain swings (Photo −3.89, Cartoon −2.77). The SAGM+LSQ baseline also differs between Table 2 (avg 56.93) and Table 4 (avg 56.27). Since the reported gain over SAGM+LSQ is 4.40 points, and identical settings within the paper differ by up to 3.89 points on a single domain and 1.67 points on the average, the headline effect is comparable to the paper's own internal variability. No code or seeds are provided to determine which table is correct. Assumption 3.1 (low gradient disorder implies a suboptimal equilibrium that is safe to freeze) is also unproven, but the internal contradiction is the more immediate obstacle to relying on the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GAQAT, a quantization-aware training framework for domain generalization under low bit-widths (4-bit and 3-bit). It observes that applying SAGM, a flatness-aware DG method, to LSQ quantized models degrades OOD accuracy, and attributes this to conflicting gradients between the task loss and the smoothness loss on quantizer scale factors. The paper defines a 'gradient disorder' metric, and proposes a dynamic selective freezing strategy that, every K steps, freezes the task-gradient update of scale factors whose disorder falls below a threshold r, while continuing smoothness-gradient updates. Experiments on PACS and DomainNet report consistent gains over LSQ and SAGM+LSQ, plus ablations testing freeze steps, threshold r, and reversed freezing. The conclusion claims up to 4.5% improvement over direct DG-QAT integration on PACS and near-lossless 4-bit performance on DomainNet.","tokens_in":15041,"tokens_out":6855,"duration_ms":59375,"significance":"If the empirical claims are reliable, this is a practically relevant contribution: it addresses a real deployment gap by extending flatness-based DG methods to low-precision quantized models. The diagnostic experiments (perturbing trained scale factors, visualizing gradient conflicts) are a useful way to motivate the method, and the ablation direction—freezing low-disorder scales helps while freezing high-disorder scales hurts—is a meaningful sanity check. The paper also follows the DomainBed protocol and uses MoCo-v2 pretrained weights to avoid the test-data leakage risk discussed by Yu et al. However, the central evidence is empirical, and the manuscript currently contains internal numerical contradictions for the exact reported configuration. Without code, seeds, or standard deviations, the claimed gains, some of which are as small as 0.09 points on DomainNet 3-bit, cannot be independently evaluated. The core idea is plausible and worth investigating further, but the present version does not provide stable measurements to support its headline claim.","major_comments":[{"comment":"The same GAQAT configuration is reported with different numbers. Table 4's 'Ours' row, described as freeze steps=350 and threshold r=0.30, reports average test accuracy 59.33% (Art 48.20, Cartoon 59.22, Photo 67.51, Sketch 62.37). Table 5's '350' row reproduces exactly these numbers, which is consistent. However, Table 6's 'r=0.30' row, which under the stated protocol of fixing freeze steps and varying the threshold should be the identical configuration, reports average test accuracy 57.66% (Art 48.20, Cartoon 56.45, Photo 63.62, Sketch 62.37). The per-domain difference is up to 3.89 points (Photo). Furthermore, the main result in Table 2 for 'Ours' 4-bit on PACS (average 61.33%, Photo 69.46, Sketch 67.40) does not match any configuration in the ablation tables, whose best average is 59.33%. The paper provides no explanation for these discrepancies. Since the claimed improvement over SAGM+LSQ in Table 2 is 4.40 points, and the internal variability for identical settings is 1.67 points on the average and up to 3.89 points on a single domain, the headline effect is comparable to the manuscript's own inconsistency. The authors must reconcile these tables, report the exact hyperparameters used for Table 2, and provide code or seeds so the reader can determine which numbers are correct.","section":"Section 4.3, Tables 4, 5, and 6"},{"comment":"The SAGM+LSQ baseline is not stable across tables for the same 4-bit PACS setting. Table 2 reports SAGM+LSQ average test accuracy 56.93% with Sketch 61.45, while Table 4 reports SAGM+LSQ average 56.27% with Sketch 58.81. This is a 0.66-point difference in the average and a 2.64-point difference in the Sketch domain. Because the tables are claimed to be based on the same experimental setup, this variation further undermines the reliability of the reported comparisons and indicates that run-to-run variability is not controlled or reported.","section":"Table 2 vs. Table 4"},{"comment":"The formal definition of gradient disorder is mathematically incomplete. Equation (2) writes δ = (1/K) 1(sgn(S1) ≠ sgn(S2)), where S1 and S2 are sequences of gradients. As written, the indicator function on two sequences is not defined and there is no summation over the K−1 consecutive comparisons described in the prose ('the proportion of steps where the gradient direction is opposite to that of the previous step'). The correct expression should be δ = (1/(K−1)) Σ_{j=1}^{K−1} 1(sgn(g_j) ≠ sgn(g_{j+1})). This is not a cosmetic issue: Algorithm 1 and all downstream results depend on this quantity, and the current formula cannot be implemented as stated. The authors should fix the equation and state whether the sign is taken element-wise over gradient components and then averaged, or applied to the scalar loss gradient.","section":"Section 3.3, Eq. (2)"},{"comment":"The empirical claims lack the statistical support needed for the small margins reported. No code, random seeds, or standard deviations are provided. On DomainNet 3-bit, the reported gain of Ours over SAGM+LSQ is only 39.53% vs. 39.44% (0.09 points), and on PACS 4-bit the ablation tables show that varying the threshold r from 0.30 to 0.32 changes average accuracy by 1.29 points, while varying freeze steps from 300 to 400 changes it by 0.16 points. Given the internal contradictions documented above, the reader cannot determine whether the advertised gains are larger than run-to-run noise. The authors should report mean and standard deviation over at least three seeds, and release code to reproduce the key tables.","section":"Section 4.1 and Section 4.3"}],"minor_comments":[{"comment":"Assumption 3.1 is phrased as a mathematical assumption but is actually an empirical hypothesis supported only by the qualitative evidence in Figures 3 and 4. There is no formal statement of what 'impact' means or under what conditions it holds. The authors should relabel it as a 'Hypothesis' or provide a more precise statement and evidence.","section":"Section 3.3, Assumption 3.1"},{"comment":"There are numerous typographical errors that should be corrected: 'Shareness-aware minimization' in the Introduction, 'Quantizaion-aware training' in Section 5.2, 'Morever' in Section 3.2, and 'DoaminNet' in the Figure 5 caption. The figure caption also states 'every 2111 steps', which appears to be a typo for the evaluation interval.","section":"Throughout"},{"comment":"The symbol K is used both for the number of steps in the disorder window (Definition 3.1) and for the evaluation interval in Algorithm 1. These are conceptually different quantities and should be denoted separately, e.g., K_w for the window and K_eval for the interval.","section":"Section 3.3, Algorithm 1 and Definition 3.1"},{"comment":"The paper uses test-domain validation for model selection across all methods. This is an oracle-like selection that uses target-domain labels; while it follows DomainBed practice, it should be explicitly acknowledged as a limitation because it can inflate absolute accuracies and may interact with the per-domain hyperparameter sensitivity the authors themselves observe.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistencies in the ablation tables are the immediate obstacle to publication. I would ask the authors to provide the exact configuration for Table 2, reconcile Tables 4-6, and release code and seeds. The idea is interesting and the paper fits a CV/ML venue, but the central empirical claim is not currently reproducible from the information in the manuscript. I do not recommend rejection because the discrepancies may be fixable by a rerun and a careful reporting of results, but acceptance is not possible without these fixes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, short take: the paper has a real idea worth talking about, but the headline PACS numbers don't survive contact with the paper's own tables.\n\nThe new thing here is the combination of flatness-aware DG (SAGM) with low-bit QAT (LSQ), plus the specific diagnosis: the task loss and the smoothness loss give conflicting gradients for the learned quantizer scale factors, and that conflict destabilizes training. The gradient-disorder metric (sign flips over K steps) is simple and reasonable, and the selective freezing of task gradients for low-disorder scales while keeping smoothness-gradient updates is a sensible way to break the conflict. The ablation direction supports the mechanism: freezing low-disorder scales helps, freezing high-disorder scales hurts. The perturbation experiment in Table 1 is a nice check that converged-looking scales can be suboptimal for OOD. DomainNet results are consistent and show near-lossless 4-bit and small gains over the SAGM+LSQ baseline. That part I buy.\n\nThe soft spot is serious. The same GAQAT configuration (freeze steps 350, threshold 0.30) appears in three places with three different 4-bit PACS averages: Table 2 reports 61.33, Table 4/5 reports 59.33, and Table 6 reports 57.66. The SAGM+LSQ baseline also changes between Table 2 (56.93) and Table 4 (56.27). The claimed 4.4-point gain over the baseline is therefore not reproducible from the paper's own numbers; the spread across identical settings (up to about 4 points on Photo, about 2 points on the average) is comparable to the effect size. There is no code, no seeds, no standard deviation, and model selection uses test-domain validation, which they disclose but which is known to inflate DG numbers. The threshold and freeze interval are tuned per dataset and bit width, so we can't tell how much of the benefit comes from the mechanism and how much from per-dataset hyperparameter selection. Assumption 3.1 is plausible but unproven; the paper would be stronger if it were framed as a hypothesis to test rather than a given.\n\nWho is this for? Someone working at the intersection of QAT and DG, or doing edge deployment of domain-robust models, will find the problem framing and the gradient-disorder idea worth engaging. But as it stands, I would not cite the PACS numbers, and I'd want code and a rerun before relying on the central claim. It deserves a serious referee—the idea is good enough to warrant a revision that fixes the numbers and adds seeds—but the current version shouldn't be accepted as is.","headline":"A plausible QAT-for-DG recipe with a useful gradient-disorder freezing idea, but the PACS numbers are internally inconsistent across tables and the headline gain is not stable.","tokens_in":15571,"tokens_out":3015,"would_cite":false,"duration_ms":23229,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GAQAT claims that quantizer scale factors in low-bit training receive conflicting task and smoothness gradients, and that dynamically freezing the task-gradient component whenever its direction is too consistent stabilizes training and…","keywords":["domain generalization","quantization-aware training","sharpness-aware minimization","gradient conflict","scale factor freezing","gradient disorder","low-bit quantization","out-of-distribution accuracy"],"falsifier":"Run the 4-bit PACS protocol with GAQAT's exact threshold and interval, but replace the disorder ranking with randomly chosen scale factors frozen in the same counts; if the random-freeze control reaches or exceeds 61.33 percent average test accuracy, then low gradient disorder is not the active ingredient and the claim that the signal identifies harmful equilibria would be falsified.","tokens_in":14484,"feed_emoji":"🧊","tokens_out":9942,"duration_ms":85262,"temperature":0.7,"pith_summary":"Low-bit models trained on source domains usually lose the out-of-domain generalization that full-precision sharpness-aware methods provide. This paper claims the reason is a conflict inside the quantizer: the scale factors that set quantization step sizes receive two gradients, one from the classification task and one from the flatness or smoothness objective, and in many layers these point in opposite directions, canceling out and driving scales to bad equilibria. GAQAT defines gradient disorder as the rate at which a scale's task gradient flips sign, and whenever that rate falls below a threshold, it freezes only the task-gradient update while letting the smoothness gradient keep working. The result is more stable quantized training, smoother loss surfaces, and 3-bit and 4-bit accuracies that beat direct quantized integration of sharpness-aware training by up to about 4.5 points on PACS, with 4-bit DomainNet nearly matching full-precision accuracy. If correct, this makes flatness-based domain generalization practical on the low-bit models that edge devices actually run.","feed_headline":"Frozen scale gradients recover quantized domain generalization","feed_subtitle":"GAQAT's 4-bit PACS accuracy hits 61.33 percent; DomainNet nearly matches full precision.","key_machinery":"The machinery has two parts. The first is the LSQ-style quantizer scale factor $s$, a learnable step size that maps continuous weights and activations to discrete levels; in GAQAT it is the only parameter that receives two gradient streams. The second is gradient disorder, defined over $K$ training steps as the fraction of adjacent steps in which the task gradient $g_{\\text{task}}$ changes sign: $\\delta = \\tfrac{1}{K} \\sum_{j} \\mathbf{1}[\\operatorname{sgn}(g_j) \\ne \\operatorname{sgn}(g_{j+1})]$. A low $\\delta$ means the task gradient direction is consistent; the paper interprets this as a signal that the scale has reached a suboptimal equilibrium where $g_{\\text{task}}$ and $g_{\\text{smooth}}$ cancel. Every $K$ steps, scales with $\\delta$ below a threshold $r$ are marked frozen for the next $K$ steps, meaning only $g_{\\text{smooth}}$ updates them, while all other scales continue to receive both gradients. This selective freezing is the active mechanism that stabilizes quantized weight updates and flattens the loss surface.","core_discovery":"The paper's central claim is that the gradient conflict at quantizer scale factors, not the quantization itself, is what destroys out-of-domain generalization when sharpness-aware objectives are added to quantization-aware training. Each scale factor $s$ receives a task gradient $g_{\\text{task}}$ from the empirical risk and a smoothness gradient $g_{\\text{smooth}}$ from the perturbed loss; in certain layers the two are opposite and nearly equal in magnitude, so the accumulated update stalls while the scale sits at a point that looks converged. The paper shows this is a bad convergence by perturbing converged scales by plus or minus 10 to 20 percent: on some layers, out-of-domain accuracy improves, meaning the optimizer had parked the scale at a suboptimal equilibrium. GAQAT therefore computes, over $K$ steps, the proportion of sign changes in $g_{\\text{task}}$ and freezes $g_{\\text{task}}$ for any scale whose disorder is below a threshold $r$, while still updating that scale with $g_{\\text{smooth}}$, and re-evaluates the freeze set every $K$ steps. With this mechanism, the 4-bit PACS average test accuracy rises to 61.33 percent from 56.93 percent for SAGM+LSQ, and 4-bit DomainNet reaches 40.74 percent versus 40.00 percent for SAGM+LSQ and 40.95 percent for full-precision ERM.","pith_inferences":["A natural next test is to apply the same disorder-triggered freezing to other quantizer parameters, such as clipping bounds or per-channel scales, or to other multi-objective training setups like distillation or adversarial robustness where scale factors also receive competing gradients.","Because low disorder is observable online without target-domain labels, it could serve as an early-stopping or per-layer freezing signal during deployment-time adaptation, not only during initial quantization-aware training.","A causal check beyond the paper's ablations would compare GAQAT with random freezing of the same number of scales; if random selection matches the 61.33 percent PACS result, then the specific information in gradient disorder is not necessary for the gain."],"forward_implications":["On 4-bit PACS, GAQAT reaches 61.33 percent average test accuracy, 4.40 points above SAGM+LSQ and 2.35 points above LSQ alone; on 3-bit it reaches 57.13 percent, 4.55 points above SAGM+LSQ.","On 4-bit DomainNet, the model reaches 40.74 percent average accuracy, within 0.21 points of the 40.95 percent full-precision ERM baseline, so flatness-based domain generalization can be made nearly lossless at low bit width.","The conflict is stronger in 4-bit than in 3-bit quantization; the paper's gradient plots show task gradients dominating at 3-bit, which is why a lighter freeze schedule works there.","Freezing task gradients on scales with high disorder, or never unfreezing, both underperform GAQAT, indicating that the direction of the disorder signal and periodic re-evaluation matter, not just freezing per se."],"supporting_citations":[{"why":"Supplies the sharpness-aware gradient matching (SAGM) objective, empirical risk, perturbed loss, and surrogate gap, that GAQAT plugs into the quantizer.","marker":"Wang et al., 2023"},{"why":"Supplies the LSQ-style learnable scale factor s and quantization-aware training setup whose gradient conflict GAQAT diagnoses.","marker":"Esser et al., 2019"},{"why":"Introduces sharpness-aware minimization and the perturbation radius rho that underlies the smoothness loss and the SAGM+LSQ baseline.","marker":"Foret et al., 2020"},{"why":"Provides the claim that learnable scale factors amplify weight oscillations under quantization, grounding the instability mechanism.","marker":"Liu et al., 2023"},{"why":"Shows that freezing and unfreezing parts of the quantizer overcomes oscillation, the family of strategies GAQAT adapts to scale-factor task gradients.","marker":"Nagel et al., 2022"},{"why":"Cited as a gradient-freezing approach for quantization that motivates selectively disabling updates at converged scales.","marker":"Tang et al., 2024"},{"why":"Defines the benchmark protocol, data splits, and model selection used for the PACS and DomainNet comparisons.","marker":"Gulrajani & Lopez-Paz, 2020"},{"why":"Supplies the evaluation protocol and the warning against supervised-pretrained weight leakage that dictates the contrastive pretrained initialization.","marker":"Yu et al., 2024b"}],"fun_headline_variants":["Freeze scale gradients to recover quantized DG","Resolving scale-gradient conflicts in quantized DG","Selective scale-gradient freeze improves quantized DG","Scale-gradient freeze stabilizes quantized domain generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a low gradient disorder, a task gradient with a consistent sign over K steps, marks a scale factor that has reached a harmful suboptimal equilibrium, so stopping its task-gradient updates will improve generalization; the paper states this as an assumption rather than proving it, and the disorder threshold r and freeze interval K are selected per dataset and bit width on validation data.","fun_headline_variants_meta":{"raw":{"variants":["Freeze scale gradients to recover quantized DG","Resolving scale-gradient conflicts in quantized DG","Selective scale-gradient freeze improves quantized DG","Scale-gradient freeze stabilizes quantized domain generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001175,"raw_usage":{"total_tokens":4972,"prompt_tokens":1174,"completion_tokens":3798,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":790,"completion_tokens_details":{"reasoning_tokens":3739}},"tokens_in":790,"tokens_out":3798,"duration_ms":29287,"temperature":1.0,"reasoning_tokens":3739,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:35:57.806113+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the 4-bit PACS protocol with GAQAT's exact threshold and interval, but replace the disorder ranking with randomly chosen scale factors frozen in the same counts; if the random-freeze control reaches or exceeds 61.33 percent average test accuracy, then low gradient disorder is not the active ingredient and the claim that the signal identifies harmful equilibria would be falsified.","supporting_citations":[],"review_version":1}