{"id":"70518084-9583-4197-b3ea-7bc8b09bc7f0","arxiv_id":"2605.25469","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"JacQuant learns a diagonal or block-diagonal Jacobian surrogate to replace STE in QAT, with convergence proofs and higher accuracy than STE at ≤2 bits on LLMs.","lead":"JacQuant replaces the Straight-Through Estimator in quantization-aware training with a learned lightweight surrogate of local parameter sensitivity. This approach aims to stabilize ultra-low-bit LLM training while adding negligible runtime cost.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader correctly flagged the surrogate-approximation assumption as weakest and noted the abstract-only limitation. No additional load-bearing technical gap emerges from the provided material that would alter the UNVERDICTED verdict.","tokens_in":1699,"tokens_out":198,"duration_ms":19484,"concrete_test":"Reproduce the LLM benchmark tables using the exact hyper-parameters, group sizes, and variance-reduced optimizer settings reported in the experimental section; confirm whether accuracy deltas and runtime overheads hold under identical random seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract presents a scoped empirical claim (higher accuracy than STE QAT at ≤2 bits on LLMs, negligible overhead) together with theory limited to code-preserving phases and a calibration argument linking sensitivity to output fidelity. No internal inconsistency, unsupported assumption, or mismatch between stated scope and claim is visible from the given text.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces JacQuant, a QAT framework that replaces the Straight-Through Estimator with a learned lightweight (diagonal or block-diagonal) surrogate of local parameter sensitivity. The surrogate is integrated into standard variance-reduced optimizers without altering forward quantizer behavior. Convergence is proved for non-convex objectives on code-preserving phases, with linear rates under the PL condition; a calibration argument relates the surrogate to end-to-end output fidelity. Empirically, the method yields higher accuracy than STE-based QAT on LLM benchmarks at ≤2 bits while adding negligible runtime cost under practical group sizes.","tokens_in":1775,"tokens_out":220,"duration_ms":23092,"significance":"If the surrogate accurately captures sensitivity and the scoped theoretical results hold, the work supplies a practical, drop-in alternative to STE that improves training stability and fidelity alignment in ultra-low-bit LLM quantization. The negligible overhead and compatibility with common quantizers strengthen the practical contribution; the convergence analysis and calibration argument provide theoretical grounding within the stated scope.","major_comments":[],"minor_comments":[],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. No specific major comments were raised in the report.","responses":[],"tokens_in":1231,"tokens_out":47,"duration_ms":9276,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core contribution is a data-driven surrogate for the model's local sensitivity that gets plugged into standard variance-reduced optimizers instead of relying on the straight-through estimator. They keep the forward pass unchanged and train the surrogate to be diagonal or block-diagonal so the extra cost stays low. On the theory side they show convergence for non-convex objectives and linear rates under the PL condition, plus a calibration step that ties the surrogate back to output fidelity.\n\nWhat stands out is the empirical scope: consistent accuracy improvements over STE-based QAT on LLM benchmarks at ≤2 bits, with runtime numbers showing the overhead is negligible at practical group sizes. The method is presented as drop-in, which is useful for people already running QAT pipelines.\n\nThe soft spots are mostly around verification. The convergence claims are limited to code-preserving phases, so it is not clear how much of a full training run this covers. The calibration argument that links the learned sensitivity to end-to-end fidelity is stated but would need a close look at the derivation to confirm it is not just restating the fitting objective. Because the surrogate is trained on the same data the model sees, there is always a risk that any reported gain partly reflects how well the surrogate was tuned rather than a fundamental improvement in gradient quality. The abstract does not compare against other learned surrogate or straight-through alternatives in detail, so the novelty relative to that literature is hard to judge from the summary alone.\n\nThis is a paper for researchers working on ultra-low-bit quantization for large models who already know the STE pain points. The combination of scoped theory and targeted experiments is enough to justify sending it to referees; the claims are narrow enough that a review can focus on whether the surrogate actually delivers independent signal and whether the runtime numbers hold under broader conditions.","headline":"JacQuant replaces STE with a learned lightweight Jacobian surrogate for QAT gradients, adds convergence theory scoped to code-preserving phases, and reports accuracy gains at 2 bits and below on LLMs with small overhead.","tokens_in":2268,"tokens_out":446,"would_cite":false,"duration_ms":15242,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"JacQuant replaces the straight-through estimator with a learned diagonal Jacobian surrogate to stabilize quantization-aware training and reach higher accuracy on LLMs at two bits and below.","keywords":["quantization-aware training","straight-through estimator","Jacobian surrogate","low-bit quantization","LLM compression","variance-reduced optimization","non-convex convergence"],"falsifier":"On a small model where the true local Jacobian is computed exactly by automatic differentiation, replace JacQuant's learned surrogate with a version whose entries differ by more than a small constant factor and observe whether the accuracy advantage over STE disappears.","tokens_in":2604,"feed_emoji":"🔢","tokens_out":704,"duration_ms":32172,"temperature":0.7,"pith_summary":"Quantization-aware training typically relies on the straight-through estimator to pass gradients through non-differentiable quantizers, but this creates brittleness near bin boundaries and poor alignment with the final low-precision model. JacQuant instead learns a lightweight diagonal or block-diagonal surrogate of the model's local sensitivity to parameter changes and substitutes it into standard variance-reduced optimizers. The surrogate is data-driven, inexpensive to maintain, and leaves the forward quantizer unchanged. The paper proves convergence for non-convex objectives and linear rates under the PL condition, plus a calibration link between the surrogate and end-to-end output fidelity. On LLM benchmarks at ≤2 bits the method outperforms STE-based QAT while adding negligible runtime cost under practical group sizes.","feed_headline":"Learned diagonal surrogate beats STE at ≤2-bit LLM QAT","feed_subtitle":"A lightweight sensitivity model stabilizes training and raises accuracy with negligible overhead while leaving the forward quantizer untouch","key_machinery":"A learned diagonal or block-diagonal Jacobian surrogate that approximates the local sensitivity of model output to weight changes and is inserted into variance-reduced optimizers.","core_discovery":"JacQuant learns a data-driven diagonal or block-diagonal approximation to the Jacobian of the model's output with respect to its parameters and uses this surrogate in place of the straight-through estimator during the backward pass, enabling stable training of ultra-low-bit models without any modification to the forward quantization operation.","pith_inferences":["The inexpensive nature of the diagonal surrogate suggests it could be recomputed periodically during long training runs to track distribution shifts.","Similar learned sensitivity surrogates might be applied to other non-differentiable operations such as structured pruning or dynamic mixed-precision allocation.","Because the method requires no change to the forward quantizer, it could be combined with existing quantization libraries without code changes.","The calibration link between surrogate and output fidelity may extend to measuring how well other compression techniques preserve model behavior."],"forward_implications":["Higher accuracy than STE-based QAT across LLM benchmarks at ≤2 bits.","Negligible added runtime cost under practical group sizes on various models.","Convergence guarantees for non-convex objectives and linear rates under the PL condition.","Drop-in compatibility with common weight and activation quantizers that leaves the forward pass unchanged.","A simple calibration argument relates the learned sensitivity directly to end-to-end output fidelity."],"fun_headline_variants":["JacQuant bypasses STE with learned Jacobian surrogates","Diagonal surrogate enables STE-free ultra-low-bit LLM QAT","Jacobian approximation stabilizes QAT at 2 bits and below","Learned sensitivity surrogate for STE-free low-bit QAT"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The learned surrogate accurately approximates the model's local sensitivity to parameter changes and can be safely used inside standard variance-reduced optimizers without altering forward quantizer behavior.","fun_headline_variants_meta":{"raw":{"variants":["JacQuant bypasses STE with learned Jacobian surrogates","Diagonal surrogate enables STE-free ultra-low-bit LLM QAT","Jacobian approximation stabilizes QAT at 2 bits and below","Learned sensitivity surrogate for STE-free low-bit QAT"]},"model":"grok-4.3","cost_usd":0.004686,"raw_usage":{"total_tokens":2301,"prompt_tokens":638,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":46862000,"prompt_tokens_details":{"text_tokens":638,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1598,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":638,"tokens_out":65,"duration_ms":18251,"temperature":1.0,"reasoning_tokens":1598,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T22:51:27.393105+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"On a small model where the true local Jacobian is computed exactly by automatic differentiation, replace JacQuant's learned surrogate with a version whose entries differ by more than a small constant factor and observe whether the accuracy advantage over STE disappears.","supporting_citations":[],"review_version":1}