{"id":"25042ffc-cef0-487c-b4a7-71b027f908c9","arxiv_id":"2501.06736","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"ZOQO combines zero-order sign-based gradient estimation with quantized noise and a quantized learning rate so that parameter updates stay on a fixed quantization grid.","lead":"This paper introduces ZOQO, a training method that uses quantized parameters and zero-order gradient-sign estimates from forward passes, avoiding full-precision backpropagation. It reports competitive results in quantized black-box adversarial attacks and in low-bit LoRA fine-tuning of a 1.3B language model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The update direction relies on sign(ℓ(x+u)−ℓ(x−u)) being informative on a discrete, piecewise-constant loss; this is untested and has no theory, and a concrete failure appears in Table I (MNIST b=4 natural: 0.74 vs 0.32).","rationale":"The reader's weakest_assumption is the same one I would stress: the sign of a two-point loss difference is the entire learning signal, and the quantization grid makes the landscape non-smooth. I agree with the conditional verdict rather than escalating because the paper explicitly frames convergence as future work and because the method's performance on LLM fine-tuning is non-trivial; however, that performance is not enough to retire the concern. The MNIST b=4 cell (Table I) is the sharpest existing evidence that the sign estimate can be uninformative, and the absence of error bars means even the favorable numbers could be seed luck. The memory and loss caveats (hash-table loss not implemented, toy memory model) are secondary but reinforce that the system has not yet been demonstrated as described. If the concrete test shows sign agreement well above chance and reproducible performance, the conditional verdict could later be upgraded; if not, the central claim would need to be rejected or heavily scoped.","tokens_in":9633,"tokens_out":9182,"duration_ms":97599,"concrete_test":"Run the MNIST b=4 naturally-trained RandSign condition from Table I for 10 seeds, and in each run log the agreement between the Algorithm-1 per-coordinate update sign and the sign of the coordinate-wise finite-difference gradient of the quantized loss (computed on a random 5% coordinate subset every 100 steps). If ZOQO's failure rate stays near 0.7 while FP stays near 0.3, or if sign agreement is at or below 0.5 in the b=4 regime, the central 'competitive performance' claim is not supported; if both are favorable (agreement clearly above 0.5 and failure near 0.3), the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"ZOQO's central mechanism is the coordinate-wise update sign = sign(ℓ(x+u)−ℓ(x−u)) · sign(u), with u drawn from a finite quantization grid B. For this to optimize, the scalar loss difference must carry reliable directional information about the quantized loss. But the loss restricted to the quantized parameter grid is neither smooth nor continuous; the ZO-SignSGD guarantees in [1] assume a smooth objective with Gaussian perturbations, and Section V explicitly defers convergence theory. The quantization step s can exceed the intended perturbation scale µ: m = max(⌊µ/s⌋,1) forces u = ±s when µ<s, so the estimator no longer approximates a local gradient. A single loss comparison then gates every coordinate of the update, so one spurious sign can move the whole iterate in a bad direction. Table I supplies a plausible instance: on naturally trained MNIST at b=4, RandSign+ZOQO has failure rate 0.74 versus 0.32–0.33 for FP and post-quantized baselines—a departure from 'minor degradation.' Without seeds, error bars, or a direct measurement of sign fidelity, the empirical support for the central claim is thin. None of this proves the method is broken; the LLM results suggest the sign can carry signal in some settings. The point is that the load-bearing assumption is currently unverified and there is at least one regime where it appears to fail.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ZOQO, a zero-order quantized optimization method for training models with quantized parameters and updates. The algorithm adapts ZO-SignSGD by replacing Gaussian perturbations with noise sampled from a quantized grid B and by rounding the learning rate to a multiple of the quantization scale s, so that parameter updates remain on the quantized grid. The authors evaluate ZOQO in two settings: black-box adversarial attacks on MNIST, CIFAR-10, and ImageNet, and LoRA fine-tuning of OPT-1.3b on SST2, together with a toy-model memory simulation. The central claim is that ZOQO achieves competitive performance compared with full-precision methods while eliminating the need for full-precision first-order gradient and parameter-update calculations.","tokens_in":9865,"tokens_out":5388,"duration_ms":51947,"significance":"If the empirical claims hold, ZOQO offers a simple and potentially useful recipe for fully quantized, forward-only training in memory-constrained settings. The paper's strengths include the clean specification of Algorithm 1, the comparison against several quantization baselines, and the encouraging LLM fine-tuning results at 4-bit quantization in Table IV. The authors also acknowledge in Section V that convergence theory is deferred. However, the evidence is not yet conclusive: the key sign-estimation mechanism is unverified on discrete loss landscapes, no repeated trials or error bars are reported, and the memory benefit is only simulated on a toy model. These gaps are fixable and should be addressed before publication.","major_comments":[{"comment":"The update direction is determined by sign(ℓ(x+u)-ℓ(x-u))·sign(u) with a single quantized perturbation u. Because quantization makes the loss piecewise constant and non-smooth, the ZO-SignSGD convergence guarantees from [1], which assume a smooth objective and Gaussian perturbations, do not apply; Section V explicitly defers convergence theory. The common scalar factor sign(ℓ(x+u)-ℓ(x-u)) multiplies every coordinate update, so a single erroneous comparison can flip all coordinate signs. Table I provides a concrete regime of concern: on naturally trained MNIST at b=4, RandSign+ZOQO attains a failure rate of 0.74 versus 0.32-0.33 for the FP and post-quantized baselines. The authors should either provide a convergence or bias analysis for the discrete perturbation model, or add direct measurements of sign fidelity (e.g., agreement with the sign of the true gradient or with multi-query estimates) across bit budgets.","section":"Section III, Eq. (1) and Algorithm 1"},{"comment":"None of the experimental tables report error bars, number of seeds, or per-run variance. The text gives evaluation set sizes but not how often the attack or fine-tuning procedure was repeated. Differences that are central to the claims, such as the MNIST b=4 natural RandSign failure rate 0.74 in Table I and the LLM b=4 ZOQO result in Table IV, cannot be distinguished from noise. Please report the mean and standard deviation (or at least multiple seeds) for each reported number and state the exact number of independent runs.","section":"Tables I-IV"},{"comment":"The memory claim is supported only by a simulation on a three-layer fully connected toy model (Table V) and by the formula M = 3bd + 2b_FP in Section III. The formula omits activations, optimizer states, loss/auxiliary tensors, and implementation-specific allocations, and the toy simulation is not the model used in any of the experiments. To support the memory advantage as a general claim, the authors should either measure peak memory on the actual MNIST/CIFAR-10/LLM experiments or clearly delimit the claim as an idealized parameter-storage analysis.","section":"Section IV, Memory Efficiency; Section III memory formula"},{"comment":"The LoRA quantization range is set to twice the initial minimal and maximal parameter values, with no ablation or justification. Since s determines both the noise grid B (through m) and the quantized learning rate ηq (Eq. (3)), this range multiplier directly influences the algorithm's behavior and the reported LLM results. Please provide an ablation over this multiplier or a principled rule for setting it.","section":"Section IV, LLM fine-tuning setup"}],"minor_comments":[{"comment":"In the introduction, the method is called 'Zero-Order Quatized Optimization'; correct the spelling to 'Quantized'.","section":"Section I"},{"comment":"The update expression contains a stray closing brace: 'xi = clamp(...)(xi−1 − ηq · sign(∇f) });' — remove the extra brace.","section":"Algorithm 1, line 9"},{"comment":"The text says 'the discritization of the distribution'; 'discritization' should be 'discretization'.","section":"Section III"},{"comment":"The header '8-bit / 4-bit quantization' combined with slash-separated entries under 'Post quant.' and 'ZOQO' is ambiguous; label the subcolumns explicitly for each bit budget.","section":"Table II"},{"comment":"The table mixes bit budgets in one row for ZOQO (b=(8/4)) while other methods have separate rows; reformat so each bit budget is a separate row or use explicit subcolumns.","section":"Table IV"},{"comment":"In the formula M = 3bd + 2b_FP, define the units of bd and state the value of b_FP used in the simulation; as written, 'bd' and 'bFP' are ambiguous.","section":"Section III, memory formula"},{"comment":"The convention for sign(ui) when ui = 0 is undefined; specify that sign(0) = 0 or exclude zero from the sampled noise.","section":"Section III, notation"}],"recommendation":"major_revision","confidential_remarks":"I recommend major revision rather than rejection: the core algorithm is cleanly specified and the LLM fine-tuning results are encouraging, but the missing variance reporting and the unvalidated sign-estimation mechanism are essential to support the central claim. I did not find circularity in the evaluation; the free parameters are design choices rather than fitted constants. The paper would also benefit from releasing code and exact hyperparameters to improve reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: ZOQO is a real, new variant — ZO-SignSGD with quantized noise and a quantized learning rate — and on the LLM fine-tuning task it looks like it works reasonably. But the paper's central mechanism, using the sign of a single loss difference on a piecewise-constant quantized loss as an update direction, is untested and untheorized, and Table I contains a concrete cell (MNIST b=4 natural, RandSign: 0.74 vs 0.32) where it clearly fails. That doesn't sink the paper, but it should push the authors to add variance, code, and a direct measurement of sign fidelity.\n\nThe main new thing is the combination itself. I checked the citations: ZO-SignSGD doesn't quantize updates, and quantization-aware training doesn't use zero-order sign estimates. The algorithm is simple and cleanly specified. The LLM results (b=8 within ~1.4 points of full-precision, b=4 best among their baselines) are plausible evidence that the sign can carry signal in some settings. The ablation framing is also honest: the baselines with partial quantization are reasonably interpreted as failed versions of the full recipe.\n\nThe soft spots are real but not fatal. The stress-test is right that the loss restricted to a quantized grid is neither smooth nor continuous, and the ZO-SignSGD guarantees don't transfer. Section V defers convergence theory explicitly, so that's not a hidden flaw, but it does mean the method's load-bearing assumption is currently unverified. The MNIST b=4 natural cell is the clearest counterexample: a failure rate of 0.74 versus 0.32–0.33 for FP baselines is not 'minor degradation.' The absence of error bars and seeds is also a problem, especially since the differences between b=8 and b=4 in several cells are small enough that run-to-run variance could matter. The memory claim is only simulated on a toy model, and the paper's assertion that it 'eliminates the need for full precision calculations' is overstated — the loss itself is still FP, and the hash-table approximation is mentioned but not implemented.\n\nOverall: the paper deserves serious referee attention. The idea is well-motivated, the experiments are within-subfield, and the central gap is unverified rather than disproven. I would recommend conditional accept after the authors release code, report variance across at least three seeds, address the 0.74 vs 0.32 cell explicitly, and soften the full-precision-free language. The paper would be useful for people working on edge training or low-resource adversarial testing.","headline":"A genuine new ZO/quantized variant with a plausible LLM result, but the sign estimator on quantized losses is unverified and one table cell shows it can fail.","tokens_in":10448,"tokens_out":2072,"would_cite":true,"duration_ms":18915,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ZOQO trains quantized models with forward-pass-only gradient signs and quantized updates, matching full-precision accuracy within a small margin.","keywords":["zero-order optimization","quantization","black-box adversarial attacks","large language model fine-tuning","LoRA","sign-based gradient estimation","low-resource training"],"falsifier":"Measure, on a small quantized network, the correlation between the ZOQO update direction sign(ℓ(x+u)−ℓ(x−u))·sign(u) and the true gradient sign over many random quantized weights; if the correlation is statistically indistinguishable from zero on the same tasks where ZOQO appears to work, the update is not estimating gradient sign and the performance would have to come from another mechanism.","tokens_in":9373,"feed_emoji":"⚙️","tokens_out":5884,"duration_ms":54461,"temperature":0.7,"pith_summary":"This paper seeks to show that neural networks can be trained with every step—weights, injected noise, and parameter updates—kept on a low-bit quantization grid, using only two forward passes per update and never computing a full-precision gradient. The proposed method, ZOQO, estimates the sign of the gradient from the sign of the loss difference between two quantized perturbations and rescales the learning rate to the quantization grid so that updates stay quantized. If correct, model training becomes feasible in low-resource settings where floating-point arithmetic is costly or unavailable, and quantized models remain vulnerable to black-box adversarial attacks mounted from the same constrained setting. The paper supports the claim experimentally with black-box attacks on MNIST, CIFAR-10, and ImageNet, and with LoRA fine-tuning of a 1.3-billion-parameter language model, reporting accuracy close to full-precision training at 8 bits and competitive accuracy at 4 bits.","feed_headline":"All-quantized training matches full precision on key tasks","feed_subtitle":"ZOQO replaces gradients with loss comparisons, fine-tunes a 1.3B LLM at 8 bits with a ~1.4-point accuracy drop.","key_machinery":"The load-bearing mechanism is the sign of the loss difference between two symmetrical quantized perturbations, combined with a quantization-grid learning rate. Let s be the uniform quantization scale. Noise u is sampled from the discrete set B={−ms,…,0,…,ms} with m=max{⌊μ/s⌋,1}, and the gradient sign is estimated as sign(ℓ(x+u)−ℓ(x−u))·sign(u). The effective learning rate ηq=max{⌊η/s⌋,1}s ensures every parameter update is a multiple of s, so parameters never leave the quantized set. The loss values themselves are computed in floating point, but the paper notes that only their comparison is needed, not exact values.","core_discovery":"The central discovery is that replacing Gaussian noise with discrete quantized noise in a sign-based zero-order optimizer, and snapping the learning rate to multiples of the quantization scale, keeps the entire optimization trajectory on the quantized grid without collapsing performance. Concretely, the gradient sign is approximated as sign(ℓ(x+u)−ℓ(x−u))·sign(u), where u is drawn from the discrete set {−ms,…,ms}, and the update is x←x−ηq·sign(∇̂f) with ηq a multiple of the grid scale s. This requires no backward pass, no full-precision parameter storage, and no full-precision update arithmetic. In the reported experiments ZOQO attains an SST2 accuracy of 89.68% at 8 bits versus 91.06% for full-precision ZO-SignSGD with post-training quantization, and 64.34% at 4 bits, the best among the compared quantized variants; it also keeps black-box attack failure rates close to full-precision attacks, while cutting peak update memory from 903.71 MB to 371.21 MB on a toy model.","pith_inferences":["Editorial inference: because only the sign of the loss difference is used, ZOQO could in principle be driven by any binary comparison oracle, such as human preference judgments, without needing exact loss values.","Editorial inference: if the sign estimate remains informative in the nondifferentiable quantized landscape, the method should extend to other high-dimensional tasks such as reinforcement learning or distributed optimization, but this requires the deferred convergence theory to identify which loss landscapes satisfy that condition.","Editorial inference: the uniform-quantization assumption is the main simplification; adaptive or per-layer quantization, which the paper itself mentions as future work, would likely reduce the accuracy gap at 4 bits."],"forward_implications":["If ZOQO holds up, training a quantized model no longer requires a full-precision copy of weights or gradients, removing a major memory bottleneck in on-device learning.","Black-box adversarial attacks can be launched in a fully quantized environment, meaning quantized deployments cannot assume safety from low-resource attackers.","LoRA-style fine-tuning of LLMs can be performed with forward passes only and quantized updates, at 8-bit accuracy within about 1.5 points of full-precision fine-tuning.","Lower bit budgets shift the advantage toward fully quantized training, where ZOQO outperforms partial-quantization baselines that leave the learning rate or noise in full precision.","Memory savings scale with parameter dimension, since ZOQO stores 3bd bits for parameters and noise instead of an additional full-precision gradient of size b_FP d."],"supporting_citations":[{"why":"Supplies the ZO-SignSGD update whose fixed step size and sign-only updates ZOQO adapts to the quantized setting.","marker":"[1]"},{"why":"Provides the zero-order LLM fine-tuning benchmark and hyperparameters used for the SST2 LoRA experiments.","marker":"[11]"},{"why":"Provides the SignHunter black-box attack implementation that ZOQO integrates for sign-based attacks.","marker":"[35]"},{"why":"Serves as the no-gradient Simple attack baseline that ZOQO is compared against in the quantized attack setting.","marker":"[37]"},{"why":"Supplies the ZO-Adam baseline compared in the LLM fine-tuning experiments.","marker":"[38]"},{"why":"Represents prior quantized training that still relies on higher-precision gradient calculations, the contrast motivating fully quantized updates.","marker":"[27]"}],"fun_headline_variants":["ZOQO sidesteps backward passes and full-precision memory","ZOQO cuts memory 59% and keeps accuracy within 1.4 points","Quantized zero-order optimizer trains without gradients at 4 bits","All-quantized fine-tuning: 89.7% SST2 at 8 bits, no backward pass"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that comparing the losses at two neighboring quantized points tells you which direction lowers the loss, even though quantization makes the loss function piecewise constant and the comparison can be flat or noisy; the paper defers a proof of this to future work.","fun_headline_variants_meta":{"raw":{"variants":["ZOQO sidesteps backward passes and full-precision memory","ZOQO cuts memory 59% and keeps accuracy within 1.4 points","Quantized zero-order optimizer trains without gradients at 4 bits","All-quantized fine-tuning: 89.7% SST2 at 8 bits, no backward pass"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000743,"raw_usage":{"total_tokens":3288,"prompt_tokens":891,"completion_tokens":2397,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":2310}},"tokens_in":507,"tokens_out":2397,"duration_ms":18706,"temperature":1.0,"reasoning_tokens":2310,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:51:07.261256+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on a small quantized network, the correlation between the ZOQO update direction sign(ℓ(x+u)−ℓ(x−u))·sign(u) and the true gradient sign over many random quantized weights; if the correlation is statistically indistinguishable from zero on the same tasks where ZOQO appears to work, the update is not estimating gradient sign and the performance would have to come from another mechanism.","supporting_citations":[{"cited_title":"signsgd via zeroth-order oracle,","cited_arxiv_id":null,"evidence_quote":"Supplies the ZO-SignSGD update whose fixed step size and sign-only updates ZOQO adapts to the quantized setting."},{"cited_title":"Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark,","cited_arxiv_id":null,"evidence_quote":"Provides the zero-order LLM fine-tuning benchmark and hyperparameters used for the SST2 LoRA experiments."},{"cited_title":"Sign bits are all you need for black- box attacks,","cited_arxiv_id":null,"evidence_quote":"Provides the SignHunter black-box attack implementation that ZOQO integrates for sign-based attacks."},{"cited_title":"Simple black-box adversarial attacks,","cited_arxiv_id":null,"evidence_quote":"Serves as the no-gradient Simple attack baseline that ZOQO is compared against in the quantized attack setting."},{"cited_title":"Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization,","cited_arxiv_id":null,"evidence_quote":"Supplies the ZO-Adam baseline compared in the LLM fine-tuning experiments."}],"review_version":1}