Pith. sign in

REVIEW 4 major objections 7 minor 38 references

ZOQO: Zero-Order Quantized Optimization

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ZOQO trains quantized models with forward-pass-only gradient signs and quantized updates, matching full-precision accuracy within a small margin.

desk verdict A genuine new ZO/quantized variant with a plausible LLM result, but the sign estimator on quantized losses is unverified and one table cell shows it can fail. read the letter →

arxiv 2501.06736 v1 pith:4QLMDNOA submitted 2025-01-12 cs.LG cs.CL

Noga Bar , Raja Giryes This is my paper · ORCID
classification cs.LGcs.CL
keywords zero-orderoptimizationquantizationblack-boxadversarialattackslargelanguagemodelfine-tuningLoRAsign-basedgradientestimationlow-resourcetraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to show that neural networks can be trained with every step—weights, injected noise, and parameter updates—kept on a low-bit quantization grid, using only two forward passes per update and never computing a full-precision gradient. The proposed method, ZOQO, estimates the sign of the gradient from the sign of the loss difference between two quantized perturbations and rescales the learning rate to the quantization grid so that updates stay quantized. If correct, model training becomes feasible in low-resource settings where floating-point arithmetic is costly or unavailable, and quantized models remain vulnerable to black-box adversarial attacks mounted from the same constrained setting. The paper supports the claim experimentally with black-box attacks on MNIST, CIFAR-10, and ImageNet, and with LoRA fine-tuning of a 1.3-billion-parameter language model, reporting accuracy close to full-precision training at 8 bits and competitive accuracy at 4 bits.

What carries the argument

The load-bearing mechanism is the sign of the loss difference between two symmetrical quantized perturbations, combined with a quantization-grid learning rate. Let s be the uniform quantization scale. Noise u is sampled from the discrete set B={−ms,…,0,…,ms} with m=max{⌊μ/s⌋,1}, and the gradient sign is estimated as sign(ℓ(x+u)−ℓ(x−u))·sign(u). The effective learning rate ηq=max{⌊η/s⌋,1}s ensures every parameter update is a multiple of s, so parameters never leave the quantized set. The loss values themselves are computed in floating point, but the paper notes that only their comparison is needed, not exact values.

What would settle it

Measure, on a small quantized network, the correlation between the ZOQO update direction sign(ℓ(x+u)−ℓ(x−u))·sign(u) and the true gradient sign over many random quantized weights; if the correlation is statistically indistinguishable from zero on the same tasks where ZOQO appears to work, the update is not estimating gradient sign and the performance would have to come from another mechanism.

Watch

Extended reading notes

Core claim

The central discovery is that replacing Gaussian noise with discrete quantized noise in a sign-based zero-order optimizer, and snapping the learning rate to multiples of the quantization scale, keeps the entire optimization trajectory on the quantized grid without collapsing performance. Concretely, the gradient sign is approximated as sign(ℓ(x+u)−ℓ(x−u))·sign(u), where u is drawn from the discrete set {−ms,…,ms}, and the update is x←x−ηq·sign(∇̂f) with ηq a multiple of the grid scale s. This requires no backward pass, no full-precision parameter storage, and no full-precision update arithmetic. In the reported experiments ZOQO attains an SST2 accuracy of 89.68% at 8 bits versus 91.06% for full-precision ZO-SignSGD with post-training quantization, and 64.34% at 4 bits, the best among the compared quantized variants; it also keeps black-box attack failure rates close to full-precision attacks, while cutting peak update memory from 903.71 MB to 371.21 MB on a toy model.

Load-bearing premise

The method assumes that comparing the losses at two neighboring quantized points tells you which direction lowers the loss, even though quantization makes the loss function piecewise constant and the comparison can be flat or noisy; the paper defers a proof of this to future work.

Editorial extensions

If this is right

  • If ZOQO holds up, training a quantized model no longer requires a full-precision copy of weights or gradients, removing a major memory bottleneck in on-device learning.
  • Black-box adversarial attacks can be launched in a fully quantized environment, meaning quantized deployments cannot assume safety from low-resource attackers.
  • LoRA-style fine-tuning of LLMs can be performed with forward passes only and quantized updates, at 8-bit accuracy within about 1.5 points of full-precision fine-tuning.
  • Lower bit budgets shift the advantage toward fully quantized training, where ZOQO outperforms partial-quantization baselines that leave the learning rate or noise in full precision.
  • Memory savings scale with parameter dimension, since ZOQO stores 3bd bits for parameters and noise instead of an additional full-precision gradient of size b_FP d.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because only the sign of the loss difference is used, ZOQO could in principle be driven by any binary comparison oracle, such as human preference judgments, without needing exact loss values.
  • Editorial inference: if the sign estimate remains informative in the nondifferentiable quantized landscape, the method should extend to other high-dimensional tasks such as reinforcement learning or distributed optimization, but this requires the deferred convergence theory to identify which loss landscapes satisfy that condition.
  • Editorial inference: the uniform-quantization assumption is the main simplification; adaptive or per-layer quantization, which the paper itself mentions as future work, would likely reduce the accuracy gap at 4 bits.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces ZOQO, a zero-order quantized optimization method for training models with quantized parameters and updates. The algorithm adapts ZO-SignSGD by replacing Gaussian perturbations with noise sampled from a quantized grid B and by rounding the learning rate to a multiple of the quantization scale s, so that parameter updates remain on the quantized grid. The authors evaluate ZOQO in two settings: black-box adversarial attacks on MNIST, CIFAR-10, and ImageNet, and LoRA fine-tuning of OPT-1.3b on SST2, together with a toy-model memory simulation. The central claim is that ZOQO achieves competitive performance compared with full-precision methods while eliminating the need for full-precision first-order gradient and parameter-update calculations.

Significance. If the empirical claims hold, ZOQO offers a simple and potentially useful recipe for fully quantized, forward-only training in memory-constrained settings. The paper's strengths include the clean specification of Algorithm 1, the comparison against several quantization baselines, and the encouraging LLM fine-tuning results at 4-bit quantization in Table IV. The authors also acknowledge in Section V that convergence theory is deferred. However, the evidence is not yet conclusive: the key sign-estimation mechanism is unverified on discrete loss landscapes, no repeated trials or error bars are reported, and the memory benefit is only simulated on a toy model. These gaps are fixable and should be addressed before publication.

major comments (4)
  1. [Section III, Eq. (1) and Algorithm 1] The update direction is determined by sign(ℓ(x+u)-ℓ(x-u))·sign(u) with a single quantized perturbation u. Because quantization makes the loss piecewise constant and non-smooth, the ZO-SignSGD convergence guarantees from [1], which assume a smooth objective and Gaussian perturbations, do not apply; Section V explicitly defers convergence theory. The common scalar factor sign(ℓ(x+u)-ℓ(x-u)) multiplies every coordinate update, so a single erroneous comparison can flip all coordinate signs. Table I provides a concrete regime of concern: on naturally trained MNIST at b=4, RandSign+ZOQO attains a failure rate of 0.74 versus 0.32-0.33 for the FP and post-quantized baselines. The authors should either provide a convergence or bias analysis for the discrete perturbation model, or add direct measurements of sign fidelity (e.g., agreement with the sign of the true gradient or with multi-query estimates) across bit budgets.
  2. [Tables I-IV] None of the experimental tables report error bars, number of seeds, or per-run variance. The text gives evaluation set sizes but not how often the attack or fine-tuning procedure was repeated. Differences that are central to the claims, such as the MNIST b=4 natural RandSign failure rate 0.74 in Table I and the LLM b=4 ZOQO result in Table IV, cannot be distinguished from noise. Please report the mean and standard deviation (or at least multiple seeds) for each reported number and state the exact number of independent runs.
  3. [Section IV, Memory Efficiency; Section III memory formula] The memory claim is supported only by a simulation on a three-layer fully connected toy model (Table V) and by the formula M = 3bd + 2b_FP in Section III. The formula omits activations, optimizer states, loss/auxiliary tensors, and implementation-specific allocations, and the toy simulation is not the model used in any of the experiments. To support the memory advantage as a general claim, the authors should either measure peak memory on the actual MNIST/CIFAR-10/LLM experiments or clearly delimit the claim as an idealized parameter-storage analysis.
  4. [Section IV, LLM fine-tuning setup] The LoRA quantization range is set to twice the initial minimal and maximal parameter values, with no ablation or justification. Since s determines both the noise grid B (through m) and the quantized learning rate ηq (Eq. (3)), this range multiplier directly influences the algorithm's behavior and the reported LLM results. Please provide an ablation over this multiplier or a principled rule for setting it.
minor comments (7)
  1. [Section I] In the introduction, the method is called 'Zero-Order Quatized Optimization'; correct the spelling to 'Quantized'.
  2. [Algorithm 1, line 9] The update expression contains a stray closing brace: 'xi = clamp(...)(xi−1 − ηq · sign(∇f) });' — remove the extra brace.
  3. [Section III] The text says 'the discritization of the distribution'; 'discritization' should be 'discretization'.
  4. [Table II] The header '8-bit / 4-bit quantization' combined with slash-separated entries under 'Post quant.' and 'ZOQO' is ambiguous; label the subcolumns explicitly for each bit budget.
  5. [Table IV] The table mixes bit budgets in one row for ZOQO (b=(8/4)) while other methods have separate rows; reformat so each bit budget is a separate row or use explicit subcolumns.
  6. [Section III, memory formula] In the formula M = 3bd + 2b_FP, define the units of bd and state the value of b_FP used in the simulation; as written, 'bd' and 'bFP' are ambiguous.
  7. [Section III, notation] The convention for sign(ui) when ui = 0 is undefined; specify that sign(0) = 0 or exclude zero from the sampled noise.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ZOQO's design choices and empirical claims are self-contained, with no fitted parameter or self-citation chain that reduces a result to its inputs.

full rationale

The paper contains no derivation that equates a prediction with a fitted input. The method is defined by Algorithm 1, which combines the existing ZO-SignSGD update with quantized noise and a quantized learning rate. The noise distribution is constructed from the quantization grid B and the rescaled learning rate eta_q = max(floor(eta/s),1)s; these are design choices stated explicitly, not parameters fitted to the evaluation data. The central claims are supported by empirical comparisons on adversarial attacks and LLM fine-tuning, using external baselines and benchmarks: the attack experiments use the code of SignHunter [35] and the LLM experiments use the ZO benchmark [11] with the same hyperparameters. No self-citation by the authors is load-bearing: the cited ZO-SignSGD method is prior work by other authors, and its convergence theory is not invoked to prove ZOQO's performance; in fact, Section V explicitly defers convergence theory to future work. The absence of a proof that sign(loss(x+u)-loss(x-u)) is informative on a piecewise-constant quantized loss is a correctness risk, not circularity, because the assumption is an unverified premise rather than a result derived from itself. The factor-two range extension for LoRA initialization is a stated implementation choice, not a fitted constant renamed as a prediction. Overall, the paper's claims are validated by held-out empirical evaluation and are not circular.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The method introduces no new particles or physical entities. Its empirical claims rest on standard zero-order estimation plus two unproven domain assumptions: sign information survives quantization, and the fixed quantization range covers the trajectory. The only hand-set constant is the factor-2 LoRA range extension.

free parameters (1)
  • LoRA quantization range multiplier = 2
    To accommodate LoRA's random initialization, the paper extends the quantization range to twice the initial minimal and maximal values (Section IV). This hand-selected factor affects the grid and the optimization trajectory.
assumptions (4)
  • domain assumption The sign of the loss difference between two quantized perturbations remains informative for optimization when the loss is piecewise constant.
    ZOQO builds on ZO-SignSGD [1] without proving convergence for quantized weights; Section V lists convergence theory as future work.
  • domain assumption The initial quantization range [Rmin,Rmax] and scale s remain valid for the whole training run, and clamped parameters outside the range are not needed.
    Algorithm 1 clamps x+, x-, and updated parameters to [Rmin,Rmax]. The LoRA experiment's factor-2 range extension is an adjustment for this assumption.
  • domain assumption The discretized distribution D(B) over quantized noise levels approximates the Gaussian N(0,mu^2) well enough to preserve the zero-order sign estimate.
    Section III says a uniform approximation may deviate and harm performance, so the Gaussian discretization is load-bearing.
  • domain assumption Loss values, despite being FP scalars, can be computed or approximated efficiently on edge devices, for example via hash tables.
    The paper argues forward logits are quantized while the loss stays FP; no implementation of the hash-table approximation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ZOQO: Zero-Order Quantized Optimization." pith.science (2026). https://pith.science/paper/4QLMDNOA

@misc{pith2026250106736,
  author       = {Pith},
  title        = {Pith review of: ZOQO: Zero-Order Quantized Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4QLMDNOA}},
  note         = {Machine review of arXiv:2501.06736}
}
read the original abstract

The increasing computational and memory demands in deep learning present significant challenges, especially in resource-constrained environments. We introduce a zero-order quantized optimization (ZOQO) method designed for training models with quantized parameters and operations. Our approach leverages zero-order approximations of the gradient sign and adapts the learning process to maintain the parameters' quantization without the need for full-precision gradient calculations. We demonstrate the effectiveness of ZOQO through experiments in fine-tuning of large language models and black-box adversarial attacks. Despite the limitations of zero-order and quantized operations training, our method achieves competitive performance compared to full-precision methods, highlighting its potential for low-resource environments.

Figures

Figures reproduced from arXiv: 2501.06736 by the authors.

Figure 1
Figure 1. QAT and ZOQO update steps. Unlike QAT, ZOQO does not use FP operations, making it efficient for resource-constrained setting. Algorithm 1 ZOQO: Zero-Order Quantized Optimization 1: Input: LR η, initial parameters x0, bit budget b, ZO step size µ 2: Initial Quantization: Quantization scale: s = x0max−x0min 2b−1 3: Quantize initial parameters: x q 0 = Q(x0) (Eq. (2)) 4: Compute quantized learning rate: ηq = c(η) (Eq. … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 30 canonical work pages

  1. [1]

    signsgd via zeroth-order oracle,

    S. Liu, P.-Y . Chen, X. Chen, and M. Hong, “signsgd via zeroth-order oracle,” in International Conference on Learning Representations, 2019

    work page 2019

  2. [2]

    A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications,

    S. Liu, P.-Y . Chen, B. Kailkhura, G. Zhang, A. O. Hero III, and P. K. Varshney, “A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications,” IEEE Signal Processing Magazine , vol. 37, no. 5, pp. 43–54, 2020

    2020

  3. [3]

    Online convex optimization in the bandit setting: gradient descent without a gradient,

    A. D. Flaxman, A. T. Kalai, and H. B. McMahan, “Online convex optimization in the bandit setting: gradient descent without a gradient,” in Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms, 2005, pp. 385–394

    work page 2005

  4. [4]

    Stochastic first-and zeroth-order methods for nonconvex stochastic programming,

    S. Ghadimi and G. Lan, “Stochastic first-and zeroth-order methods for nonconvex stochastic programming,” SIAM journal on optimization , pp. 2341–2368, 2013

    work page 2013

  5. [5]

    Optimal rates for zero-order convex optimization: The power of two function evaluations,

    J. C. Duchi, M. I. Jordan, M. J. Wainwright, and A. Wibisono, “Optimal rates for zero-order convex optimization: The power of two function evaluations,” IEEE Transactions on Information Theory , 2015

    work page 2015

  6. [6]

    Random gradient-free minimization of convex functions,

    Y . Nesterov and V . Spokoiny, “Random gradient-free minimization of convex functions,” Foundations of Computational Mathematics, pp. 527– 566, 2017

    work page 2017

  7. [7]

    Deepzero: Scaling up zeroth- order optimization for deep model training,

    A. Chen, Y . Zhang, J. Jia, J. Diffenderfer, K. Parasyris, J. Liu, Y . Zhang, Z. Zhang, B. Kailkhura, and S. Liu, “Deepzero: Scaling up zeroth- order optimization for deep model training,” in The Twelfth International Conference on Learning Representations , 2024

    work page 2024

  8. [8]

    Zeroth- order optimization with trajectory-informed derivative estimation,

    Y . Shu, Z. Dai, W. Sng, A. Verma, P. Jaillet, and B. K. H. Low, “Zeroth- order optimization with trajectory-informed derivative estimation,” in The Eleventh International Conference on Learning Representations , 2023

    work page 2023

Show all 38 references
  1. [9]

    Zeroth-order fine- tuning of LLMs with extreme sparsity,

    W. Guo, J. Long, Y . Zeng, Z. Liu, X. Yang, Y . Ran, J. R. Gardner, O. Bastani, C. D. Sa, X. Yu, B. Chen, and Z. Xu, “Zeroth-order fine- tuning of LLMs with extreme sparsity,” in 2nd Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Reso...

    2024

  2. [10]

    Fine-tuning language models with just forward passes,

    S. Malladi, T. Gao, E. Nichani, A. Damian, J. D. Lee, D. Chen, and S. Arora, “Fine-tuning language models with just forward passes,” Advances in Neural Information Processing Systems, vol. 36, pp. 53038– 53075, 2023

    2023

  3. [11]

    Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark,

    Y . Zhang, P. Li, J. Hong, J. Li, Y . Zhang, W. Zheng, P.-Y . Chen, J. D. Lee, W. Yin, M. Hong, et al., “Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark,” in Forty-first International Conference on Machine Learning , 2024

    2024

  4. [12]

    Variance-reduced zeroth-order methods for fine-tuning language models,

    T. Gautam, Y . Park, H. Zhou, P. Raman, and W. Ha, “Variance-reduced zeroth-order methods for fine-tuning language models,” in Forty-first International Conference on Machine Learning , 2024

    2024

  5. [13]

    Zeroth-order stochastic variance reduction for nonconvex optimization,

    S. Liu, B. Kailkhura, P.-Y . Chen, P. Ting, S. Chang, and L. Amini, “Zeroth-order stochastic variance reduction for nonconvex optimization,” Advances in Neural Information Processing Systems , vol. 31, 2018

    2018

  6. [14]

    A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization,

    H. Cai, Y . Lou, D. McKenzie, and W. Yin, “A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization,” in International Conference on Machine Learning . PMLR, 2021, pp. 1193–1203

    2021

  7. [15]

    Efficient on-chip learning for optical neural networks through power-aware sparse zeroth-order optimization,

    J. Gu, C. Feng, Z. Zhao, Z. Ying, R. T. Chen, and D. Z. Pan, “Efficient on-chip learning for optical neural networks through power-aware sparse zeroth-order optimization,” in Proceedings of the AAAI conference on artificial intelligence, 2021, vol. 35

    2021

  8. [16]

    Stochastic zeroth- order optimization in high dimensions,

    Y . Wang, S. Du, S. Balakrishnan, and A. Singh, “Stochastic zeroth- order optimization in high dimensions,” in International conference on artificial intelligence and statistics . PMLR, 2018, pp. 1356–1365

    2018

  9. [17]

    Zeroth-order (non)-convex stochastic optimization via conditional gradient and gradient updates,

    K. Balasubramanian and S. Ghadimi, “Zeroth-order (non)-convex stochastic optimization via conditional gradient and gradient updates,” Advances in Neural Information Processing Systems , vol. 31, 2018

    2018

  10. [18]

    Zeroth-order regular- ized optimization (zoro): Approximately sparse gradients and adaptive sampling,

    H. Cai, D. McKenzie, W. Yin, and Z. Zhang, “Zeroth-order regular- ized optimization (zoro): Approximately sparse gradients and adaptive sampling,” SIAM Journal on Optimization , 2022

    2022

  11. [19]

    On the convergence of prior-guided zeroth-order optimization algorithms,

    S. Cheng, G. Wu, and J. Zhu, “On the convergence of prior-guided zeroth-order optimization algorithms,” Advances in Neural Information Processing Systems, vol. 34, pp. 14620–14631, 2021

    2021

  12. [20]

    Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,

    P.-Y . Chen, H. Zhang, Y . Sharma, J. Yi, and C.-J. Hsieh, “Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,” in Proceedings of the 10th ACM workshop on artificial intelligence and security , 2017, pp. 15–26

    2017

  13. [21]

    Black-box adversarial at- tacks with limited queries and information,

    A. Ilyas, L. Engstrom, A. Athalye, and J. Lin, “Black-box adversarial at- tacks with limited queries and information,” in International conference on machine learning . PMLR, 2018, pp. 2137–2146

    2018

  14. [22]

    Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks,

    C.-C. Tu, P. Ting, P.-Y . Chen, S. Liu, H. Zhang, J. Yi, C.-J. Hsieh, and S.-M. Cheng, “Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks,” in Proceedings of the AAAI conference on artificial intelligence , 2019, pp. 742–749

    2019

  15. [23]

    On the design of black-box adversarial examples by leveraging gradient-free optimization and operator splitting method,

    P. Zhao, S. Liu, P.-Y . Chen, N. Hoang, K. Xu, B. Kailkhura, and X. Lin, “On the design of black-box adversarial examples by leveraging gradient-free optimization and operator splitting method,” in Proceed- ings of the IEEE/CVF International Conference on Computer Vision , 201...

    2019

  16. [24]

    How to robustify black-box ml models? a zeroth-order optimization perspective,

    Y . Zhang, Y . Yao, J. Jia, J. Yi, M. Hong, S. Chang, and S. Liu, “How to robustify black-box ml models? a zeroth-order optimization perspective,” in International Conference on Learning Representations , 2022

    2022

  17. [25]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. ...

    2018

  18. [26]

    A survey of quantization methods for efficient neural network infer- ence,

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neural network infer- ence,” in Low-Power Computer Vision , pp. 291–326. Chapman and Hall/CRC, 2022

    2022

  19. [27]

    Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,

    S. Zhou, Y . Wu, Z. Ni, X. Zhou, H. Wen, and Y . Zou, “Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,” arXiv preprint arXiv:1606.06160 , 2016

    2016 arXiv

  20. [28]

    Bnn+: Improved binary network training,

    S. Darabi, M. Belbahri, M. Courbariaux, and V . P. Nia, “Bnn+: Improved binary network training,” 2018

    2018

  21. [29]

    Dif- ferentiable soft quantization: Bridging full-precision and low-bit neural networks,

    R. Gong, X. Liu, S. Jiang, T. Li, P. Hu, J. Lin, F. Yu, and J. Yan, “Dif- ferentiable soft quantization: Bridging full-precision and low-bit neural networks,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 4852–4861

    2019

  22. [30]

    Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm,

    Z. Liu, B. Wu, W. Luo, X. Yang, W. Liu, and K.-T. Cheng, “Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 722–737

    2018

  23. [31]

    Binaryconnect: Training deep neural networks with binary weights during propagations,

    M. Courbariaux, Y . Bengio, and J.-P. David, “Binaryconnect: Training deep neural networks with binary weights during propagations,” Ad- vances in neural information processing systems , vol. 28, 2015

    2015

  24. [32]

    Binarized neural networks,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Binarized neural networks,” Advances in neural information processing systems, vol. 29, 2016

    2016

  25. [33]

    Xnor-net: Imagenet classification using binary convolutional neural networks,

    M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi, “Xnor-net: Imagenet classification using binary convolutional neural networks,” in European conference on computer vision . Springer, 2016, pp. 525–542

    2016

  26. [34]

    Deep reinforcement learning from human preferences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems , vol. 30, 2017

    2017

  27. [35]

    Sign bits are all you need for black- box attacks,

    A. Al-Dujaili and U.-M. O’Reilly, “Sign bits are all you need for black- box attacks,” in International conference on learning representations , 2020

    2020

  28. [36]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083 , 2017

    2017 arXiv

  29. [37]

    Simple black-box adversarial attacks,

    C. Guo, J. Gardner, Y . You, A. G. Wilson, and K. Weinberger, “Simple black-box adversarial attacks,” in International conference on machine learning. PMLR, 2019, pp. 2484–2493

    2019

  30. [38]

    Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization,

    X. Chen, S. Liu, K. Xu, X. Li, X. Lin, M. Hong, and D. Cox, “Zo-adamm: Zeroth-order adaptive momentum method for black-box optimization,” Advances in neural information processing systems , vol. 32, 2019

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.