Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Quantized continuous-control policies match full precision using just 2–3 bits per weight, and run as integer-only networks on small FPGAs with microsecond latency and microjoule energy per action.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 23:08 UTC pith:OUCVG3NV

load-bearing objection Useful empirical study of low-bit QAT for RL policies, but the headline '2–3 bits suffice' rests on a loose parity rule with significant selection pressure. the 2 major comments →

arxiv 2511.07046 v4 pith:OUCVG3NV submitted 2025-11-10 cs.LG cs.AI

Learning Quantized Continuous Controllers for Integer Hardware

Fabian Kresse , Christoph H. Lampert This is my paper
classification cs.LG cs.AI
keywords quantization-aware trainingreinforcement learningcontinuous controlinteger-only inferenceFPGAlow-bitwidth quantizationpolicy compressionembedded control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that continuous-control reinforcement learning policies can be trained, using quantization-aware training, to run entirely with integer arithmetic while matching the control quality of full-precision (FP32) policies. The key result is that weights and internal activations can be reduced to 2 or 3 bits without losing FP32-level returns, as long as the input state is quantized separately with sufficient precision. If true, learned controllers can be deployed on small, low-power FPGAs—avoiding costly floating-point pipelines—and achieve inference latencies in the microsecond range and energy consumption in the microjoule range per action. The paper also demonstrates that these quantized policies remain competitive under injected input noise, often outperforming the FP32 baseline.

Core claim

The paper demonstrates that quantization-aware training (QAT) of continuous-control policies yields integer-only networks that stay within one standard deviation of the FP32 baseline's mean return across five simulated locomotion tasks, using as few as 2 or 3 bits for weights and internal activations when input precision is set separately. The authors insert quantize/de-quantize (QDQ) operations at inputs, weights, after every ReLU, and before the output, using straight-through estimators so gradients flow through the non-differentiable rounding during training. A three-stage selection rule—first core bitwidth, then hidden-layer width, then input bitwidth—automatically finds a minimal config

What carries the argument

The central mechanism is the QDQ (quantize/de-quantize) layer: each FP32 value is scaled, rounded, clipped to an integer grid, then scaled back to FP32. Inserting QDQ after every ReLU and before the output forces the network to respect integer arithmetic during training while keeping the standard FP32 training loop. The straight-through estimator lets gradients pass through rounding as if it were the identity, so the network learns to be robust to quantization error. At deployment, all scales are fixed, the network performs integer-only matrix products with requantization implemented via lookup tables, and the only remaining floating-point operation is the initial input quantization.

Load-bearing premise

The load-bearing premise is that 'competitive' means a quantized model's mean return falls within one standard deviation of the FP32 baseline on the same tasks and evaluation seeds; the final bitwidth choices are selected with precisely that rule, so a more demanding parity definition could invalidate the claim that 2–3 bits suffice.

What would settle it

Take the selected 2- and 3-bit policies and evaluate them on held-out seeds, slightly perturbed dynamics, and non-Gaussian input noise, comparing mean returns to the FP32 baseline with a 95% confidence interval. If the quantized models' mean returns fall outside that interval on any task, the claim that low-bit policies match FP32 quality is not transferable beyond the original selection rule.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, low-bit integer-only policies can replace FP32 controllers on embedded hardware with order-of-magnitude reductions in latency and energy per action, making them suitable for microsecond-scale control loops.
  • Input bitwidth is a separate and sometimes dominant bottleneck: on one task, 8-bit inputs were needed to retain FP32 parity even though the core ran at 3 bits, so input precision must be chosen deliberately.
  • The finding that output quantization has little effect on performance suggests that the final hyperbolic tangent mapping can remain at 8-bit or be replaced by a lookup table without weakening control quality.
  • Quantized policies trained with QAT show equal or better tolerance to injected input noise than FP32 policies, which argues for their robustness in real-world settings with sensor noise.
  • The automatic three-stage selection rule—core bitwidth, hidden width, input bitwidth—provides a practical recipe for generating hardware-friendly policies without exhaustive search.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The headline claim that '2–3 bits suffice' is calibrated to a permissive FP32-parity rule (mean within one standard deviation of the FP32 baseline). A stricter statistical criterion, such as a 95% confidence interval or a minimum acceptable reward drop, could shift the selected bitwidths upward.
  • Because the policies were trained and evaluated on clean simulated states, their apparent noise robustness may not transfer to real sensor noise, which often has structured, non-Gaussian components. The paper's own hypothesis about real-world robustness remains untested.
  • The comparison against a 4-bit/8-bit reference, while reasonable, does not establish that 2–3 bit policies are uniquely optimal; an equally strong baseline at 3-bit with a different architecture or training schedule might close the gap.
  • The results suggest that quantization itself acts as an implicit regularizer, but a direct test—e.g., comparing the effective Lipschitz constants or state-action margins of FP32 and quantized policies—would clarify the mechanism behind the observed robustness gains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a quantization-aware training (QAT) pipeline for continuous-control reinforcement learning policies that are deployable as integer-only neural networks on FPGAs. The method quantizes inputs, weights, internal activations, and outputs during training (using straight-through estimators), then synthesizes the selected policies to an Artix-7 FPGA using the FINN compiler. Across five MuJoCo tasks with SAC and DDPG, the authors report that policies competitive with FP32 baselines can be obtained with as few as 3 or even 2 bits for internal weights/activations, provided the input bitwidth is chosen carefully. They also report that the selected quantized policies are more robust to input noise than FP32 policies, and that the synthesized designs achieve microsecond-scale latencies and microjoule-scale energy per action.

Significance. If the central claim holds, the paper provides a practically relevant result: low-bit integer-only policies can replace FP32 controllers on resource-constrained embedded hardware without losing control quality, with substantial latency and energy benefits. The empirical design has clear strengths: 10 training seeds, 1,000 rollouts per model, five tasks, and two RL algorithms; the bitwidth and width sweeps are systematic; and the hardware synthesis procedure is transparently scoped. The paper also makes a useful contribution by separating the effect of input, output, and core quantization and by providing a concrete learning-to-hardware pipeline. However, the headline bitwidth claim is selected using a permissive FP32-parity criterion on the same tasks used for evaluation, so the current evidence does not independently establish that 2–3 bits suffice. The robustness finding is interesting but statistically preliminary.

major comments (2)
  1. [§3.1–§3.2, Table 1] The FP32-parity selection rule is too permissive and is used both to define success and to select the final configurations. With 10 training seeds, the standard error of the mean is σ/√10 ≈ 0.316σ; accepting any quantized mean inside the FP32 mean ±1σ band means a policy whose true mean is 0.5σ below FP32 passes with probability roughly 0.9 (≈0.87 if both means are treated as random). Because §3.2 searches over core bitwidths, hidden widths (16–256), and input bitwidths, the entries in Table 1 are selected as the smallest configurations that pass a multiple-comparison filter; no held-out seeds or tasks are used to validate them. The claim that '2–3 bits suffice' is therefore partly a selection outcome. Please validate the selected configurations on held-out seeds (or tasks), report confidence intervals on the FP32 difference, or apply a multiple-testing correction to the parity criterion
  2. [§3.4, Table 3] The hardware results are clearly scoped to the integer-only core: the initial FP32 input quantization and the final tanh lookup are excluded, and power figures come from the Xilinx Power Estimator rather than from measurement. This is stated in §3.4, but the abstract and §5 present the latency/energy numbers without this caveat. Please qualify these claims (e.g., 'for the synthesized integer-only core') so that readers do not infer end-to-end measured performance. This is a presentation issue rather than a technical error, but it matters for the deployment claim.
minor comments (5)
  1. [§3.3] Typo: 'perputations' should be 'perturbations'. Also, the axes in Figure 2 read 'Enviroment Step' (should be 'Environment Step').
  2. [Eq. (1), §2.2] The notation 's qs' is ambiguous and q_s is defined only after the formula. Use parentheses, e.g., QDQ_b(x;s) = s · q_s · Q_b(x;s), and define q_s before Eq. (1).
  3. [Table 3] The reference label '8-4-8' is not defined in the caption. State that it denotes input–core–output bitwidths and that the output remains fixed at 8-bit for all selected models (as described in §3.1).
  4. [Abstract / §5] The abstract says 'on the target hardware' without noting that the latency/energy numbers are estimates for the integer core only. Add a brief qualifying phrase.
  5. [§4] The introduction motivates QAT over PTQ, but no PTQ baseline is run on these tasks. A PTQ comparison on the same small MuJoCo policies would strengthen the motivation; at minimum, the paper should acknowledge that the PTQ comparison relies on literature results.

Circularity Check

0 steps flagged

No circularity: the low-bitwidth claim is an empirical search outcome under an explicitly stated parity rule, not a fit or definitional equivalence.

full rationale

The paper's central claim is empirical, not a derived equivalence. In §3.1 the authors explicitly define when a quantized model is considered to match FP32: 'We consider a quantized model to match FP32 returns if its mean return lies within the FP32 band.' In §3.2 they then 'choose the smallest core quantization precision' that satisfies that rule, and similarly select hidden width and input bitwidth. The resulting headline ('as few as 3 or even only 2 bits per weight') is therefore a report of the minimum feasible configuration found by the search, not a quantity that was fitted from the rule and then re-presented as an independent prediction. It is informative because the minimum could have been larger; the empirical content is that such small configurations exist and pass the chosen criterion. The FP32-parity rule's permissiveness (mean ±1 standard deviation over 10 models) is a legitimate statistical concern about selection bias and about how strong the word 'competitive' is, but that is a validity issue, not a circularity issue: the criterion is not secretly defined in terms of the final bitwidths, and no equation in the paper is constructed so that the conclusion is its own input. The QAT/QDQ machinery is standard and externally implemented (Brevitas, CleanRL, FINN); there is no load-bearing self-citation, no imported uniqueness theorem, and no renaming of a known result under new coordinates. Stated limitations, such as the Discussion's caveat that experiments are simulation-only and the FPGA section's exclusion of the input-quantization and tanh stages, are scoping statements rather than circular steps. Overall, the derivation chain is self-contained and the empirical evaluation is transparent enough that no circularity is present.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The paper contributes an empirical pipeline; its central claim rests mainly on the QDQ/STE training recipe (standard in supervised QAT), the FP32-parity selection rule, and the FINN synthesis assumptions. No new entities are introduced.

free parameters (5)
  • Selected config: Humanoid (h, b_core, b_in) = h=16, b_core=3, b_in=4
    Chosen in §3.2 by the FP32-parity rule on the Humanoid task; used for Table 1, Figures 2/3, and Table 3 hardware results.
  • Selected config: Walker2d (h, b_core, b_in) = h=128, b_core=2, b_in=3
    Chosen in §3.2 by the FP32-parity rule on the Walker2d task; used for the reported results and hardware synthesis.
  • Selected config: Ant (h, b_core, b_in) = h=64, b_core=2, b_in=3
    Chosen in §3.2 by the FP32-parity rule on the Ant task; used for the reported results and hardware synthesis.
  • Selected config: HalfCheetah (h, b_core, b_in) = h=256, b_core=3, b_in=8
    Chosen in §3.2 by the FP32-parity rule on the HalfCheetah task; used for the reported results and hardware synthesis.
  • Selected config: Hopper (h, b_core, b_in) = h=16, b_core=2, b_in=6
    Chosen in §3.2 by the FP32-parity rule on the Hopper task; used for the reported results and hardware synthesis.
axioms (6)
  • standard math MDP/RL formulation and SAC/DDPG actor-critic objectives
    Section 2.1; standard background.
  • domain assumption QDQ with straight-through estimator provides usable gradients for low-bit policies
    Section 2.2; not proven, empirical; the whole QAT result depends on it.
  • domain assumption MuJoCo continuous-control tasks are a valid proxy for real embedded control workloads
    Sections 3 and 5; simulation only, acknowledged by the authors.
  • ad hoc to paper FP32 parity defined as mean return within ±1 std of FP32 baseline on the same tasks is a sufficient definition of 'competitive'
    §3.1/3.2; used to select all final configurations.
  • ad hoc to paper The excluded FP32 input normalization/quantization and tanh lookup are negligible for the end-to-end hardware claim
    §3.4 states hardware results are for the integer portion only.
  • domain assumption FINN out-of-context synthesis with Xilinx Power Estimator at 100 MHz approximates real on-chip performance
    §3.4; power is estimated, not measured.

pith-pipeline@v1.3.0-alltime-deepseek · 12129 in / 15347 out tokens · 157206 ms · 2026-08-03T23:08:05.696151+00:00 · methodology

0 comments
read the original abstract

Deploying continuous-control reinforcement learning policies on embedded hardware requires meeting tight latency and power budgets. Small FPGAs can deliver these, but only if costly floating-point pipelines are avoided. We study quantization-aware training (QAT) of policies for integer inference and we present a learning-to-hardware pipeline that automatically selects low-bit policies and synthesizes them to an Artix-7 FPGA. Across five MuJoCo tasks, we obtain policy networks that are competitive with full precision (FP32) policies but require as few as 3 or even only 2 bits per weight, and per internal activation value, as long as input precision is chosen carefully. On the target hardware, the selected policies achieve inference latencies on the order of microseconds and consume microjoules per action, favorably comparing to a quantized reference. Last, we observe that the quantized policies exhibit increased input noise robustness compared to the floating-point baseline.

Figures

Figures reproduced from arXiv: 2511.07046 by Christoph H. Lampert, Fabian Kresse.

Figure 1
Figure 1. Figure 1: Reward vs. bitwidth for full-precision (FP32) baselines (shaded region indicates one [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evaluation reward across training time steps for our environments with SAC. Shaded [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Robustness to observation input noise. Reward vs. noise level, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Return vs. hidden width for SAC under the minimal FP32-matching [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Return vs. input quantization for SAC under the configuration from Table [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: DDPG; Reward vs. bitwidth for full-precision (FP32) baselines (shaded region indicates [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control

    cs.LG 2025-12 conditional novelty 6.0

    Logic-gate circuits trained with gradient descent can match neural-network policies on most MuJoCo continuous-control tasks and run on FPGAs in a few clock cycles.

Reference graph

Works this paper leans on

38 extracted references · 3 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Efficient reinforcement learning by discovering neural pathways

    Samin Yeasar Arnob, Riyasat Ohib, Sergey Plis, Amy Zhang, Alessandro Sordoni, and Doina Precup. Efficient reinforcement learning by discovering neural pathways. Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  2. [2]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.342, 2013

    2013

  3. [3]

    Low-precision reinforcement learning: Running soft actor-critic in half precision

    Johan Bj \"o rck, Xiangyu Chen, Christopher De Sa, Carla P Gomes, and Kilian Weinberger. Low-precision reinforcement learning: Running soft actor-critic in half precision. In International Conference on Machine Learning (ICML), 2021

    2021

  4. [4]

    FINN-R : An end-to-end deep-learning framework for fast exploration of quantized neural networks

    Michaela Blott, Thomas B Preu er, Nicholas J Fraser, Giulio Gambardella, Kenneth O’brien, Yaman Umuroglu, Miriam Leeser, and Kees Vissers. FINN-R : An end-to-end deep-learning framework for fast exploration of quantized neural networks. ACM Transactions on Reconfigurable Technology and Systems (TRETS), 2018

    2018

  5. [5]

    Quantization-aware training for multi-agent reinforcement learning

    Nikolaos Chandrinos, Michalis Amasialidis, Manos Kirtas, Konstantinos Tsampazis, Nikolaos Passalis, and Anastasios Tefas. Quantization-aware training for multi-agent reinforcement learning. In European Signal Processing Conference (EUSIPCO), 2024

    2024

  6. [6]

    Combining STDP and binary networks for reinforcement learning from images and sparse rewards

    S \'e rgio F Chevtchenko and Teresa B Ludermir. Combining STDP and binary networks for reinforcement learning from images and sparse rewards. Neural Networks, 2021

    2021

  7. [7]

    Continuous control with action quantization from demonstrations

    Robert Dadashi, L \'e onard Hussenot, Damien Vincent, Sertan Girgin, Anton Raichuk, Matthieu Geist, and Olivier Pietquin. Continuous control with action quantization from demonstrations. In International Conference on Machine Learning (ICML), 2022

    2022

  8. [8]

    Magnetic control of tokamak plasmas through deep reinforcement learning

    Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de las Casas, Craig Donner, Leslie Fritz, Cristian Galperti, Andrea Huber, James Keeling, Maria Tsimpoukelli, Jackie Kay, Antoine Merle, Jean-Marc Moret, Seb Noury, Federico Pesamosca, David Pfau, Olivier...

    2022

  9. [9]

    Towards low-latency high-bandwidth control of quadrotors using event cameras

    Rika Sugimoto Dimitrova, Mathias Gehrig, Dario Brescianini, and Davide Scaramuzza. Towards low-latency high-bandwidth control of quadrotors using event cameras. In IEEE International Conference on Robotics and Automation (ICRA), 2020

    2020

  10. [10]

    Xilinx/brevitas, 2025

    Giuseppe Franco, Alessandro Pappalardo, and Nicholas J Fraser. Xilinx/brevitas, 2025. URL https://doi.org/10.5281/zenodo.3333552

    doi:10.5281/zenodo.3333552 2025

  11. [11]

    A survey of quantization methods for efficient neural network inference

    Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. In Low-power computer vision (LPCV). 2022

    2022

  12. [12]

    Quantization-aware pruning criterion for industrial applications

    Yoonhee Gil, Jong-Hyeok Park, Jongchan Baek, and Soohee Han. Quantization-aware pruning criterion for industrial applications. IEEE Transactions on Industrial Electronics, 2021

    2021

  13. [13]

    The state of sparse training in deep reinforcement learning

    Laura Graesser, Utku Evci, Erich Elsen, and Pablo Samuel Castro. The state of sparse training in deep reinforcement learning. In International Conference on Machine Learning (ICML), 2022

    2022

  14. [14]

    Just round: Quantized observation spaces enable memory efficient learning of dynamic locomotion

    Lev Grossman and Brian Plancher. Just round: Quantized observation spaces enable memory efficient learning of dynamic locomotion. In IEEE International Conference on Robotics and Automation (ICRA), 2023

    2023

  15. [15]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning (ICML), 2018

    2018

  16. [16]

    Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and João G.M. Araújo. CleanRL : High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research (JMLR), 2022

    2022

  17. [17]

    Neural network compression for reinforcement learning tasks

    Dmitry A Ivanov, Denis A Larionov, Oleg V Maslennikov, and Vladimir V Voevodin. Neural network compression for reinforcement learning tasks. Scientific Reports, 2025

    2025

  18. [18]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  19. [19]

    Binarized p-network: Deep reinforcement learning of robot control from raw images on FPGA

    Yuki Kadokawa, Yoshihisa Tsurumine, and Takamitsu Matsubara. Binarized p-network: Deep reinforcement learning of robot control from raw images on FPGA . IEEE Robotics and Automation Letters, 2021

    2021

  20. [20]

    Scalable deep reinforcement learning for vision-based robotic manipulation

    Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforcement learning for vision-based robotic manipulation. In Conference on Robot Learning (CORL), 2018

    2018

  21. [21]

    An open approach to autonomous vehicles

    Shinpei Kato, Eijiro Takeuchi, Yoshio Ishiguro, Yoshiki Ninomiya, Kazuya Takeda, and Tsuyoshi Hamada. An open approach to autonomous vehicles. IEEE Micro, 2015

    2015

  22. [22]

    Champion-level drone racing using deep reinforcement learning

    Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias M \"u ller, Vladlen Koltun, and Davide Scaramuzza. Champion-level drone racing using deep reinforcement learning. Nature, 2023

    2023

  23. [23]

    Qua RL : Quantization for fast and environmentally sustainable reinforcement learning

    Srivatsan Krishnan, Maximilian Lam, Sharad Chitlangia, Zishen Wan, Gabriel Barth-Maron, Aleksandra Faust, and Vijay Janapa Reddi. Qua RL : Quantization for fast and environmentally sustainable reinforcement learning. Transactions on Machine Learning Research (TMLR), 2022

    2022

  24. [24]

    Kochenderfer

    Christopher Lazarus and Mykel J. Kochenderfer. Deep binary reinforcement learning for scalable verification. arXiv preprint arXiv:2203.05704, 2022

    Pith/arXiv arXiv 2022

  25. [25]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In International Conference on Learning Representations (ICLR), 2016

    2016

  26. [26]

    The impact of quantization and pruning on deep reinforcement learning models

    Heng Lu, Mehdi Alemi, and Reza Rawassizadeh. The impact of quantization and pruning on deep reinforcement learning models. arXiv preprint arXiv:2407.04803, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    Network sparsity unlocks the scaling potential of deep reinforcement learning

    Guozheng Ma, Lu Li, Zilin Wang, Li Shen, Pierre-Luc Bacon, and Dacheng Tao. Network sparsity unlocks the scaling potential of deep reinforcement learning. In International Conference on Machine Learning (ICML), 2025

    2025

  28. [28]

    NVIDIA H100 Data Sheet , 2025

    NVIDIA . NVIDIA H100 Data Sheet , 2025. URL https://resources.nvidia.com/en-us-gpu-resources/h100-datasheet-24306 . Accessed: 2025-11-06

    2025

  29. [29]

    Fully onboard AI -powered human-drone pose estimation on ultralow-power autonomous flying nano- UAV s

    Daniele Palossi, Nicky Zimmerman, Alessio Burrello, Francesco Conti, Hanna M \"u ller, Luca Maria Gambardella, Luca Benini, Alessandro Giusti, and J \'e r \^o me Guzzi. Fully onboard AI -powered human-drone pose estimation on ultralow-power autonomous flying nano- UAV s. IEEE Internet of Things Journal, 2021

    2021

  30. [30]

    Adaptive dataflow and precision optimization for deep learning on configurable hardware architectures

    Gulnaz Rati, Rafael Mendes, and Aisha Noor. Adaptive dataflow and precision optimization for deep learning on configurable hardware architectures. 2025

    2025

  31. [31]

    Control delay in reinforcement learning for real-time dynamic systems: A memoryless approach

    Erik Schuitema, Lucian Bu s oniu, Robert Babu s ka, and Pieter Jonker. Control delay in reinforcement learning for real-time dynamic systems: A memoryless approach. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2010

    2010

  32. [32]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2018

    2018

  33. [33]

    Sim-to-real: Learning agile locomotion for quadruped robots

    Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: Learning agile locomotion for quadruped robots. In Robotics Science and Systems (RSS), 2018

    2018

  34. [34]

    RL x2: Training a sparse deep reinforcement learning model from scratch

    Yiqin Tan, Pihe Hu, Ling Pan, Jiatai Huang, and Longbo Huang. RL x2: Training a sparse deep reinforcement learning model from scratch. In IEEE International Conference on Robotics and Automation (ICRA), 2023

    2023

  35. [35]

    Fraser, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre, and Kees Vissers

    Yaman Umuroglu, Nicholas J. Fraser, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre, and Kees Vissers. FINN : A framework for fast, scalable binarized neural network inference. In ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2017

    2017

  36. [36]

    Using neuroevolved binary neural networks to solve reinforcement learning environments

    Raul Valencia, Chiu-Wing Sham, and Oliver Sinnen. Using neuroevolved binary neural networks to solve reinforcement learning environments. In IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), 2019

    2019

  37. [37]

    A survey of FPGA -based robotic computing

    Zishen Wan, Bo Yu, Thomas Yuang Li, Jie Tang, Yuhao Zhu, Yu Wang, Arijit Raychowdhury, and Shaoshan Liu. A survey of FPGA -based robotic computing. IEEE Circuits and Systems Magazine, 2021

    2021

  38. [38]

    A modular robotic arm control stack for research: Franka-interface and FrankaPy

    Kevin Zhang, Mohit Sharma, Jacky Liang, and Oliver Kroemer. A modular robotic arm control stack for research: Franka-interface and FrankaPy . arXiv preprint arXiv:2011.02398, 2020

    Pith/arXiv arXiv 2011