{"id":"0c4da38b-70f9-4d4d-b4bb-64a9b9bc2d6c","arxiv_id":"2504.20314","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"PeZO makes zeroth-order learning hardware-friendly by reusing a small pool of uniform random perturbations rescaled to match Gaussian length statistics, achieving accuracy close to MeZO with far fewer random number generators.","lead":"The paper shows that zeroth-order training of large language models, which normally needs a huge number of random numbers, can be made hardware-friendly by reusing a small pool of random perturbations and scaling them to mimic Gaussian noise. This is interesting because it could allow energy-efficient fine-tuning of AI models directly on FPGAs and other edge devices.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hardware scaling LUT in §3.2 contradicts Eq. (3): a 2^b-entry table indexed by one RNG output cannot represent the full-perturbation modulus, so the reported resource savings may not implement the evaluated algorithm.","rationale":"PeZO's central novelty is replacing Gaussian perturbations with uniform perturbations rescaled by the ratio of expected Gaussian modulus to actual uniform modulus. The empirical accuracy results (Tables 4–5) suggest this rescaling works in software for the tested NLP tasks. However, the paper's headline hardware savings rest entirely on the adaptive scaling being implementable in a few LUTs/FFs. The described 2^b-entry LUT, addressed by one RNG output, is exponentially too small to capture the modulus of an n-tuple per cycle, and analytically a single output cannot determine the full modulus. This is not a question of whether the method generalizes to vision; it is a question of whether the evaluated algorithm and the synthesized hardware are the same object. The reader's weakest assumption focused on cross-domain generalization; I agree that is a concern, but the hardware/algorithm mismatch is more load-bearing because it threatens the validity of the resource claims. A conditional acceptance with a request for RTL or a correct description of the scaling unit is the appropriate outcome.","tokens_in":14881,"tokens_out":13055,"duration_ms":134443,"concrete_test":"Request or reconstruct the RTL for the on-the-fly scaling unit and trace the LUT address generation. If the address is a single RNG output, construct a counterexample: two clock cycles with equal first-RNG value but different remaining n-1 values; ||u_i||_2 differs, yet the LUT returns the same scale factor, violating Eq. (3). Alternatively, run the on-the-fly configuration (n=32, b=8 or 14) with a correct full-vector modulus computation and compare the resulting scale factors against the LUT; if the mismatch changes the perturbation norms materially, the accuracy results in Tables 4–5 cannot be attributed to the described hardware. This check settles whether the resource savings claim is tied to a realizable implementation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (3) scales the full uniform perturbation u_i (dimension d, the model parameter count) by s_i = E||\\hat u_i||_2 / ||u_i||_2, so s_i depends on the L2 norm of the entire perturbation vector. The on-the-fly hardware description in §3.2 and Figure 2 instead pre-computes s_i for '2^b possible combinations (length of one cycle)' and stores these in a 2^b-entry LUT, addressed by the output of a single RNG (the one selected by the RNG pointer). With n=32 RNGs per clock cycle and b-bit outputs, one cycle is an n-tuple of b-bit values: the number of distinct cycle moduli is (2^b)^n, not 2^b, and the modulus is not determined by any single RNG's output. Unless the perturbation consists of a single scalar or all RNGs emit identical values, a 2^b-entry LUT cannot provide the correct scale factor for Eq. (3). The paper thus leaves the core hardware scaling mechanism unspecified or incorrect; the tiny resource counts in Table 6 (32 LUTs, 449 FFs, 1 BRAM) appear to describe a design that does not implement the algorithm whose accuracy is reported in Tables 4–5.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PeZO, a perturbation-efficient zeroth-order optimization framework for on-device training. It argues that ZO training, although memory-efficient, is impractical on FPGA/ASIC because generating many Gaussian random numbers is resource-intensive. PeZO replaces Gaussian perturbations with uniformly distributed perturbations rescaled by an adaptive modulus-scaling factor, and introduces two random-number reuse strategies: pre-generation (a fixed pool of random numbers) and on-the-fly generation (a small array of uniform RNGs). The authors report accuracy results fine-tuning RoBERTa, OPT, and Llama models on few-shot NLP benchmarks, comparing against MeZO, and report FPGA resource and power numbers showing large reductions in LUTs, FFs, and power for the random-number-generation subsystem. The conclusion is that PeZO enables feasible ZO on-device training without compromising accuracy.","tokens_in":15158,"tokens_out":5642,"duration_ms":60143,"significance":"If the hardware implementation actually matches the evaluated algorithm, the paper identifies a real and underappreciated bottleneck in ZO training and offers a practical route around it. The reuse strategies and the empirical characterization of pool size, RNG count, and bit-width are useful design guidance. The paper is primarily empirical rather than theoretical, and no code or formal analysis is provided, but the breadth of models, datasets, and the concrete FPGA implementation are strengths. The central value depends on validating that the hardware scaling mechanism genuinely realizes Eq. (3) and on tempering the accuracy claims to match the reported tables.","major_comments":[{"comment":"The on-the-fly scaling implementation as described cannot implement Eq. (3). Eq. (3) scales the full perturbation vector u_i by a factor s_i = E||u_hat_i||_2 / ||u_i||_2, whose value depends on the L2 norm of the entire n-dimensional vector produced in one clock cycle. With n RNGs each outputting b bits, the number of distinct cycle-level moduli is (2^b)^n, not 2^b, and the modulus is not determined by any single RNG output. The text says there are \"2^b possible combinations\" and uses the current output of the RNG pointed to by the pointer as the LUT address. A 2^b-entry LUT can encode scale factors for one scalar output, but not for the n-tuple that appears in Eq. (3), unless n=1 or all RNGs emit identical values. The resource counts in Table 6 (32 LUTs, 449 or 512 FFs, 1 BRAM) may therefore be for a design that does not evaluate the algorithm whose accuracy appears in Tables 4–5. Please provide the actual hardware scaling structure (e.g., a tree of sum-of-squares accumulators over all RNG outputs, or an explicitly justified approximation), re-validate accuracy with that exact design, and update resource/power numbers accordingly.","section":"§3.2, Eq. (3), Figure 2"},{"comment":"The statement that with k=256 \"the accuracy gap between PeZO and the baseline still narrows to within 0.5% across all tasks\" is not supported by the data. For RoBERTa-base on-the-fly generation, RTE is 60.3 vs. MeZO's 65.7 and TREC is 78.4 vs. 81.4; for RoBERTa-large on-the-fly, RTE is 66.4 vs. 72.2 and TREC is 84.2 vs. 90.8. These gaps are 3.0–6.6 points. The abstract's claim \"without compromising training performance\" is therefore too strong for several on-the-fly configurations. Please qualify the central claim by specifying which reuse strategy and which k settings are meant, and revise the abstract and conclusion accordingly. The conclusion that PeZO is competitive in many settings may still be defensible, but the current blanket wording is contradicted by the paper's own tables.","section":"§4.2, Table 4"},{"comment":"The BP-based and MeZO rows for RoBERTa-base with k=16 are identical in all five columns (88.1, 40.3, 55.7, 56.6, 59.6). This appears to be a copy/paste or reporting error, since the same table and Table 5 show nontrivial differences between BP-based and MeZO elsewhere. Because MeZO is the central baseline, these numbers should be corrected and the affected comparisons re-checked.","section":"Table 4, RoBERTa-base k=16"}],"minor_comments":[{"comment":"The caption says all reported numbers are averaged accuracy with standard deviation, but Table 5 does not include standard deviations for OPT/Llama models. Please either add standard deviations or state that they are omitted for those models.","section":"Table 4 caption and Table 5"},{"comment":"The phrase \"yield(ing) 2^b unique random numbers per cycle\" is imprecise: each RNG produces one b-bit value, so the RNG array produces n b-bit values per cycle; the number of possible output combinations is (2^b)^n, not 2^b. Please clarify the wording to avoid confusion with the LUT-size argument in §3.2.","section":"§3.1 and Figure 1"},{"comment":"The pre-generation row lists \"Ours Pre-gen - 16 8 2.104 700\" with no LUT value, while the on-the-fly rows report 32 LUTs but different FF counts (449 for RoBERTa, 512 for OPT). Please clarify what the dash means and why the FF counts differ between the two on-the-fly rows.","section":"Table 6"},{"comment":"The latency comparison for LLaMA2-7B would be more informative if the number of random numbers per attention layer update (4×4096×4096) were explicitly derived or a pointer to the layer configuration given, since the reader cannot verify the factor of four from the text.","section":"§2.3"}],"recommendation":"major_revision","confidential_remarks":"The duplicate BP/MeZO row in Table 4 and the §3.2 LUT-vs-Eq. (3) contradiction should be resolved before publication. The LUT issue in particular affects the paper's core hardware-resource claim: if the implemented scaling module does not realize the evaluated perturbation, the reported LUT/FF/power savings do not establish the paper's thesis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe thing to know: this paper identifies a genuinely overlooked hardware cost in ZO training — generating thousands of Gaussian random numbers per step — and the random-reuse strategies are sensible and empirically workable. But the hardware scaling mechanism in §3.2 is internally inconsistent, and the 'without compromising performance' claim is not supported by their own tables.\n\nWhat's new: the combination of random number reuse with modulus matching to replace Gaussian with uniform perturbations appears new. The paper gives a useful empirical study of pool sizes, RNG counts, and bit widths, plus real FPGA synthesis numbers. That's more than most ML-for-hardware papers do.\n\nThe soft spots. First, the LUT description in §3.2 contradicts Eq. (3). Eq. (3) scales by the modulus of the full perturbation vector. On-the-fly, the perturbation is built from n=32 RNG outputs per cycle, so the number of distinct moduli is (2^b)^32, not 2^b. A 2^b-entry LUT addressed by one RNG's output cannot encode that. The resource numbers in Table 6 may describe a design that doesn't implement the algorithm whose accuracy is reported. This is load-bearing. Second, the accuracy comparisons are cherry-picked in the abstract: on-the-fly loses 4–6 points on several k=256 tasks (e.g., SST-5, RTE), so 'without compromising training performance' is too strong. Third, Table 4 has identical BP-based and MeZO rows for RoBERTa-base k=16 — a reporting error that erodes trust. Fourth, no comparisons to parameter-efficient ZO baselines (Sparse MeZO, etc.), no code or RTL released, and the 'first' claim is softened by their own citation [14].\n\nWho this is for: anyone working on on-device ZO training, FPGA-based fine-tuning, or perturbation-efficient optimizers. The reuse strategies themselves are worth knowing.\n\nRecommendation: engage with it, but treat the hardware scaling description as the main thing to fix. A serious referee should ask for a corrected LUT design or an explanation of how the scaling is actually computed, and for the performance claims to be reworded. With those changes, the paper could be solid. As is, conditional.","headline":"A real hardware bottleneck, sensible reuse ideas, and actual FPGA synthesis, but the core scaling LUT contradicts the paper's own equation and the performance claims are softer than the abstract suggests.","tokens_in":15723,"tokens_out":3654,"would_cite":false,"duration_ms":35472,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PeZO claims that zeroth-order optimization becomes feasible for on-device training once Gaussian per-weight perturbations are replaced by a small reused pool of uniform random numbers rescaled to the expected Gaussian modulus.","keywords":["zeroth-order optimization","on-device training","FPGA implementation","random number generation","Gaussian perturbation","uniform perturbation","language model fine-tuning","hardware-efficient training"],"falsifier":"Run PeZO on a non-NLP task, such as few-shot image classification on CIFAR-10 with a standard convolutional model, and compare final accuracy against the same optimizer using fresh Gaussian perturbations; a drop materially larger than the Gaussian baseline's run-to-run variation would show the equivalence does not generalize. A cheaper probe is to compute the mean-squared error of PeZO's rescaled-uniform gradient estimate versus the Gaussian estimate on one fixed model and batch, since large bias would explain any task class that fails.","tokens_in":14660,"feed_emoji":"🔢","tokens_out":8419,"duration_ms":79133,"temperature":0.7,"pith_summary":"Zeroth-order (ZO) optimization estimates gradients from function values, which promises memory savings for training on devices, but it carries a hidden hardware cost: every weight needs a fresh Gaussian random number at every step. This paper argues that this cost, rather than the forward passes, is what makes ZO training infeasible on FPGAs and ASICs, and proposes PeZO to remove it. PeZO replaces costly per-weight Gaussian perturbations with a small pool of reused uniform random numbers, adaptively rescaled to the modulus a Gaussian perturbation would have. On few-shot NLP fine-tuning benchmarks, the authors report accuracy competitive with MeZO, the representative ZO baseline, while cutting random-number-generation logic (LUTs and flip-flops) by 48.6% and 12.7%, and power by up to 86%. If that empirical equivalence holds, ZO becomes a realistic memory-efficient training path for edge hardware.","feed_headline":"A 4,096-number pool cuts FPGA training cost by up to 86%","feed_subtitle":"Zeroth-order fine-tuning of language models stays accurate with far fewer random-number generators.","key_machinery":"PeZO's load-bearing mechanism is random-number reuse plus adaptive modulus scaling. Reuse works by concatenating numbers from a pre-generated pool (size $2^{12}$) or from a small array of uniform RNGs, with circular-shift operators that vary the ordering so perturbations do not fall into regular patterns; this cuts the unique random numbers needed per clock cycle from roughly the parameter count to $2^{12}$ or $2^5$. Adaptive modulus scaling replaces Gaussian draws with uniform draws by rescaling each perturbation $u_i$ to the expected Gaussian modulus $\\mathbb{E}\\|\\hat{u}_i\\|_2$, with the scaling factor stored in a look-up table and rounded to a power of two so the scaling itself is just bitwise shifts. Together these remove Gaussian random-number generators from the hardware design, while the low intrinsic dimensionality of language-model fine-tuning provides the intuition for why heavily reused perturbations can still find good updates.","core_discovery":"The paper's central claim is that the two obstacles to hardware ZO — the large quantity of random numbers and the Gaussian distribution — can be dissolved without hurting accuracy. With pre-generation reuse, as few as $2^{12}$ pre-stored random numbers suffice; with on-the-fly reuse, 32 uniform random-number generators suffice. The adaptive modulus scaling step rescales each uniform perturbation to the expected Euclidean norm of a same-dimensional Gaussian perturbation, with the factor computed in advance and looked up from a small table so hardware only performs bit-shifts. Across RoBERTa-base/large, OPT-1.3B/2.7B, and Llama3-1B/3B on standard few-shot classification tasks, the accuracy stays competitive with MeZO's ideal Gaussian perturbation, and in the $k=256$ regime the gap narrows to within 0.5%. Hardware synthesis on a ZCU102 FPGA shows the random-number-generation block dropping from 133,120 LUTs and 4.474 W to 32 LUTs and 0.608–2.104 W, which the paper presents as making ZO optimization feasible for on-device training.","pith_inferences":["An unstated test is outside NLP: the paper's own limitation section concedes that few-shot computer-vision ZO lacks the prompt-style augmentation that NLP provides, so if reused uniform perturbations fail on vision, the hardware gains apply only to text-domain fine-tuning.","The results suggest a cost-geometry principle worth testing: what matters for ZO gradient estimation may be the modulus and symmetry of the perturbation distribution rather than its exact Gaussian shape; a rigorous bias-variance comparison of the rescaled-uniform estimator against the Gaussian estimator would settle this.","Reuse could compound with parameter-efficient fine-tuning: updating fewer weights needs fewer perturbations, which would cut the random-number demand even further; the paper notes LoRA-style methods but does not quantify this combination."],"forward_implications":["ZO fine-tuning can be implemented on FPGAs and ASICs with on-chip BRAM and a few dozen uniform generators, making memory-efficient on-device training concrete.","The two reuse strategies give a designer a hardware-accuracy trade-off: on-the-fly reuse is better on small-data tasks, pre-generation reuse is better on larger-data, harder settings.","RNG bit-width can stay low — 8 bits for RoBERTa-class models and 14 bits for OPT-class models — with RNG shifting extending the effective randomness cycle.","PeZO plugs into any ZO method that uses the standard two-point perturbation estimator, so future ZO improvements can inherit the hardware savings.","With random-number generation cut to 32 LUTs and 449–512 FFs at 0.61–2.10 W, the perturbation block is no longer the bottleneck; inference compute becomes the limiting cost."],"supporting_citations":[{"why":"supplies the MeZO baseline whose ideal per-weight Gaussian perturbation PeZO must match in accuracy while using far fewer random numbers.","marker":"[24]"},{"why":"establishes that language-model fine-tuning has low intrinsic dimensionality, the motivation for letting many weights share a single perturbation.","marker":"[1]"},{"why":"TreeGRNG provides the efficient Gaussian random-number generator used as the hardware baseline that PeZO's resource savings are measured against.","marker":"[7]"},{"why":"documents the area-efficient table-Hadamard GRNG's 0.7% FPGA cost, evidence that hundreds of parallel GRNGs are infeasible.","marker":"[34]"},{"why":"documents a precision-oriented GRNG consuming 6.6% FFs and 10% DSPs, evidence for the hardware cost of Gaussian generation.","marker":"[17]"},{"why":"supports the claim that easier optimization settings sit on a low-dimensional manifold, explaining why on-the-fly reuse with fewer unique numbers can win in small-data regimes.","marker":"[29]"}],"fun_headline_variants":["PeZO slashes FPGA random-number cost by 86%","2^12 random numbers make ZO training hardware-friendly","PeZO: 32 generators beat 133k LUTs with same accuracy","Zero-order on-device training without Gaussian overhead","PeZO cuts FPGA power 86% for on-device ZO training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a small set of reused uniform random numbers, rescaled to the expected Gaussian modulus, produces gradient estimates statistically close enough to fresh Gaussian perturbations to preserve accuracy across diverse models and tasks; the paper supports this only empirically on a few few-shot NLP benchmarks and provides no theoretical analysis.","fun_headline_variants_meta":{"raw":{"variants":["PeZO slashes FPGA random-number cost by 86%","2^12 random numbers make ZO training hardware-friendly","PeZO: 32 generators beat 133k LUTs with same accuracy","Zero-order on-device training without Gaussian overhead","PeZO cuts FPGA power 86% for on-device ZO training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000134,"raw_usage":{"total_tokens":1167,"prompt_tokens":999,"completion_tokens":168,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":79}},"tokens_in":615,"tokens_out":168,"duration_ms":2162,"temperature":1.0,"reasoning_tokens":79,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:32:28.540123+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run PeZO on a non-NLP task, such as few-shot image classification on CIFAR-10 with a standard convolutional model, and compare final accuracy against the same optimizer using fresh Gaussian perturbations; a drop materially larger than the Gaussian baseline's run-to-run variation would show the equivalence does not generalize. A cheaper probe is to compute the mean-squared error of PeZO's rescaled-uniform gradient estimate versus the Gaussian estimate on one fixed model and batch, since large bias would explain any task class that fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the MeZO baseline whose ideal per-weight Gaussian perturbation PeZO must match in accuracy while using far fewer random numbers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TreeGRNG provides the efficient Gaussian random-number generator used as the hardware baseline that PeZO's resource savings are measured against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"documents the area-efficient table-Hadamard GRNG's 0.7% FPGA cost, evidence that hundreds of parallel GRNGs are infeasible."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"documents a precision-oriented GRNG consuming 6.6% FFs and 10% DSPs, evidence for the hardware cost of Gaussian generation."},{"cited_title":"Learning to Guide Random Search","cited_arxiv_id":"2004.12214","evidence_quote":"supports the claim that easier optimization settings sit on a low-dimensional manifold, explaining why on-the-fly reuse with fewer unique numbers can win in small-data regimes."}],"review_version":1}