Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Massively parallel GPU simulation lets reinforcement-learning trading agents sample up to 1,746 times faster, and ensembling those agents trims drawdown and lifts Sharpe.

desk verdict Speedup measurement is the real contribution; ensemble claims are not supported as stated. read the letter →

arxiv 2501.10709 v1 pith:XR3RZZWR submitted 2025-01-18 cs.CE cs.AIstat.ML

Nikolaus Holzer , Keyi Wang , Kairong Xiao , Xiao-Yang Liu Yanglet This is my paper · ORCID
classification cs.CEcs.AIstat.ML
keywords financialreinforcementlearningensemblemethodsmassivelyparallelsimulationGPUvectorizedenvironmentsstocktradingcryptocurrencypolicyinstabilitysamplingbottleneck
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that two obstacles to using reinforcement learning for trading, policy instability and slow data collection, can be attacked together by running thousands of market simulations in parallel on one GPU and combining several trained agents into an ensemble. It reports sampling speedups up to 1,746 times in a Bitcoin task and, in both stock and crypto experiments, ensembles that reduce maximum drawdown by up to 4.17% and improve Sharpe ratio by up to 0.21 versus individual agents. The work matters because it suggests that the robustness of ensembling becomes affordable once sampling is no longer the bottleneck.

What carries the argument

The load-bearing mechanism is the vectorized environment (VecEnv), a container that holds N independent sub-environments and applies the same reset, step, and reward operations to all of them simultaneously on the GPU, storing states, actions, and rewards as tensors of shape T by N by D. This turns the Monte Carlo gradient estimate over N trajectories into a single batched computation, avoiding CPU-GPU transfers. On top of that, the ensemble layer either softmax-weights agents' action probabilities according to their validation Sharpe ratios (discarding very low-Sharpe agents) or takes a majority vote, and a KL-divergence penalty in each agent's loss encourages the policies to diverge.

What would settle it

Run the same stock and crypto tasks with many random seeds (e.g., 20 or more), recompute the rolling-window Sharpe-based weights each time, and compare the distribution of maximum drawdown and Sharpe improvements against the single best agent. If the ensemble's advantage over the best individual agent frequently reverses sign or the 4.17% drawdown reduction falls within the run-to-run noise, the robustness claim would not survive.

Watch

Extended reading notes

Core claim

The central claim is that a GPU-vectorized trading environment removes the sampling bottleneck and makes ensemble training practical, while the resulting ensembles are steadier than any single agent. The paper builds vectorized market environments using PyTorch's vmap so that every sub-environment steps, rewards, and stores samples as GPU tensors, then trains PPO, SAC, and DDPG agents for stocks and DQN-style agents for crypto, combining them by Sharpe-weighted averaging of action probabilities (stocks) or majority voting (crypto). The reported numbers are concrete: 8,813.81 samples per second versus 184.63 for the stock task (47.73 times), and 114,885.98 versus 65.79 for the crypto task (1,746 times), with the best ensemble achieving a 62.60% cumulative return in stocks and a 0.28 Sharpe with a 0.73% drawdown in crypto.

Load-bearing premise

The reported improvements rest on Sharpe ratios computed from 5-day validation windows used to weight the agents, and on single-run results; if those Sharpe estimates are noisy or the runs depend heavily on random seeds, the ensemble's apparent advantage could be a selection artifact rather than true robustness.

Editorial extensions

If this is right

  • If the speedup transfers beyond the two tasks tested, sampling-heavy reinforcement learning methods for finance become tractable on a single GPU, enabling larger state spaces and more frequent retraining.
  • Ensemble aggregation reduces the variance of cumulative returns across rolling windows, implying that a Sharpe-weighted ensemble could be more reliable for deployment than picking the single best-performing algorithm.
  • Storing samples entirely in GPU memory removes the PCIe transfer bottleneck, so increasing the number of parallel environments does not incur the usual communication cost and may scale further with larger GPUs.
  • The diversity-inducing KL penalty and varied training data could make ensembles of same-architecture agents more robust than the paper's specific PPO/SAC/DDPG mix, which is a directly testable extension.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's single-run results do not establish that Sharpe-based softmax weighting beats equal weighting; a natural follow-up is to compare those weighting schemes across multiple validation windows and seeds.
  • The 1,746-fold speedup is measured on a single-asset task with simple environment logic; more complex, multi-asset or higher-frequency environments will likely see smaller but still substantial gains.
  • Section 6.5's zero-knowledge-proof proposal, if realized, would address the incentive problem of publishing profitable trading models, but it is a speculation separate from the experimental results.
  • A stronger test of the robustness claim would be to report the distribution of Sharpe and drawdown improvements over many random seeds, since the paper's own Figure 1 shows that single runs can be misleading.
Share X Bluesky LinkedIn Reddit HN

Formalized claims in Lean

  1. Claim #1: The central claim is that a GPU-vectorized trading environment removes the sampling bottleneck and makes ensemble training practical, while the resulting ensembles are steadier than any single agent. The paper builds vectorized market environments using PyTorch's vmap so that every sub-environment steps, rewards, and stores samples as GPU tensors, then trains PPO, SAC, and DDPG agents for stocks a

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a GPU-vectorized market environment for FinRL-style trading, claiming a sampling speedup of up to 1,746x with 2,048 parallel environments on a single A100 GPU. It then evaluates ensemble methods for stock trading (PPO/SAC/DDPG combined by weighted averaging) and cryptocurrency trading (DQN variants combined by majority voting), with KL-divergence diversity regularization. Stock experiments use rolling 30-day training, 5-day validation, and 5-day testing windows on 30 Dow Jones stocks; crypto experiments use second-level BTC LOB data with a roughly 45-minute out-of-sample window. The abstract and conclusion state that the ensembles outperform some individual agents, with maximum drawdown reduced by up to 4.17% and Sharpe ratio improved by up to 0.21.

Significance. The speedup claim is an engineering contribution that, if reproducible, is useful for FinRL training: the measurement methodology in Section 6.2 is clear and the scaling behavior in Figure 4 is plausible. The ensemble-performance claim is the other headline contribution, but as presented it is not statistically supported: results are single-run, the ensemble composition is described inconsistently, and one section of the text directly contradicts the reported table. The paper does not supply code, repeated-seed statistics, or confidence intervals, so the central ensemble claim cannot currently be verified. The authors do provide a concrete benchmark setup and a detailed description of the vectorized environment, which are strengths.

major comments (5)
  1. [Section 6.3, Table 2] The text states that 'the ensemble method has the highest cumulative return, Sharpe Ratio, and risk-adjusted returns,' but Table 2 shows PPO with cumulative return 63.37%, Sharpe 1.55, and Sortino 2.44, all higher than Ensemble-1's 62.60%, 1.48, and 2.34. This internal contradiction directly undermines the abstract's claim that ensembles outperform individual agents.
  2. [Section 5.2.1 (Validation bullet), Tables 2 and 3] No repeated seeds or confidence intervals are reported. The ensemble weights are derived from Sharpe ratios computed on 5-day rolling validation windows, which are extremely noisy, and the headline 'up to 4.17% MDD reduction' and 'up to 0.21 Sharpe improvement' are best-case single-run differences selected from the tables. Without repeated-seed statistics, the claimed superiority of the ensembles over individual agents is not established.
  3. [Section 6.4, Table 3] The cryptocurrency out-of-sample test is only 45 minutes long (04/19 09:09:22 to 09:54:22), and all three ensemble sizes produce identical metrics. This is too short and too degenerate to support the statement that 'the ensemble models outperform all individual agents in all metrics,' and the near-identical results suggest the different ensemble sizes are not meaningfully distinct.
  4. [Section 5.1] The sentence 'Agents are also trained on different stocks from the test set for the stock trading task' is ambiguous and potentially inconsistent with the 'There are no leaks of future information' claim in Section 6.1. If 'different stocks' means stocks outside the test set, the description needs to explain how the agents then trade the test stocks; if it means training on data from the test period, that is leakage. This ambiguity must be resolved before any performance conclusion can be accepted.
  5. [Section 6.3 (ensemble composition)] The composition of Ensemble 1 is described inconsistently within the same section: the first description says it contains '1 PPO, 1 SAC, and 1 DDPG agents,' while a later paragraph says 'Ensemble 1 consists of 1 PPO and 1 SAC agents.' This makes the experimental setup irreproducible and blurs whether DDPG contributes to the reported drawdown and Sharpe improvements.
minor comments (5)
  1. [Throughout] There are several typos, e.g., 'Experiement' in the Section 6.1 heading, 'Traning' in Section 5.2.1, 'Parallelim' in Section 4.2.1, 'developping' and 'inmediately' in Section 6.5.
  2. [Section 6.2 vs Section 6.1] The stock data date range differs between sections: Section 6.1 says 01/01/2021 to 12/01/2023, while Section 6.2 says January 1, 2020 to January 1, 2023. These should be reconciled.
  3. [Table 3 caption] The caption says the data is 'split into out-of-sample data for training and in-sample data for testing,' but the body text says the opposite (in-sample training, out-of-sample testing). The caption should be corrected.
  4. [Figure 5 and Section 6.3] The description of Ensemble 1's performance is inconsistent: one passage says 'superior performance from Sep 2022 to Oct 2023,' another says 'during steps 450 to 700.' The figure axes should be labeled and the narrative should use one reference frame.
  5. [Section 6.5] The zero-knowledge proof discussion is unrelated to the substantive experiments and appears to be a position statement; it could be removed or moved to a separate discussion without affecting the technical content.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the GPU speedup is an external measurement and the ensemble results are holdout benchmarks; the strongest concerns are statistical robustness and internal consistency, not circularity.

full rationale

The paper's two central claims are not derived from self-referential inputs. The 1,746x sampling-speed improvement is an externally measured wall-clock/throughput comparison (Section 6.2, Fig. 4) between 2,048 parallel GPU environments and a single environment; it is not obtained by fitting a parameter and then predicting a related quantity. The ensemble performance claims in Tables 2 and 3 come from test-window backtests after training and validation. Although Section 5.2.1 weights agents by Sharpe ratios computed on 5-day validation windows, the reported test-window Sharpe, drawdown, and return metrics are different quantities from those validation fits, so the ensemble results are not forced by construction. The Condorcet/CLT argument in Eq. (1) is a standard probability theorem, not a definition of the result. The paper does cite the authors' own FinRL framework [18-22], but these citations are framework references and do not supply an unverified uniqueness theorem or a fitted ansatz on which the quantitative claims depend. What should be flagged are correctness and robustness concerns: Section 6.3 claims 'the ensemble method has the highest cumulative return, Sharpe Ratio, and risk-adjusted returns' even though Table 2 shows PPO with a higher Sharpe ratio (1.55 vs. 1.48) and higher cumulative return (63.37% vs. 62.60%); the crypto out-of-sample test spans only about 45 minutes (04/19 09:09:22-09:54:22); and Section 5.1 says agents are 'trained on different stocks from the test set,' which creates a possible data-leakage ambiguity. These are statistical and reporting flaws, not circularity. Accordingly, the circularity score is 2: minor non-load-bearing self-citation, no circular step.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities; VecEnv and SubEnv are software abstractions. The central speedup claim relies on the standard assumption that parallel environments are independent and that larger samples reduce gradient variance; the ensemble claim relies on unvalidated diversity regularization and unspecified market-fidelity parameters.

free parameters (3)
  • lambda (KL divergence regularization constant)
    Introduced in Eq. (5) to enforce agent diversity; no value, range, or sensitivity analysis is given, and the paper never ablates it.
  • random price perturbation range for dataset augmentation = -1% to 1%
    In Section 5.1, prices are randomly perturbed by up to +/-1% to create diverse training datasets; the range is chosen by hand and not justified.
  • ensemble composition sizes = 1, 5, 10 agents per type
    The three ensemble sizes are chosen arbitrarily in Sections 6.3 and 6.4; no search or analysis of the effect of ensemble size is reported.
assumptions (5)
  • standard math Condorcet jury theorem applies to RL agents: if each agent has probability p > 0.5 of correct trading decision, ensemble accuracy tends to 1.
    Used in Section 3.3.1 to motivate ensembles; the CLT calculation is correct, but the mapping to neural-network policies is unverified.
  • domain assumption Deep RL policy networks in FinRL typically have accuracy above 0.5.
    Asserted without evidence in Section 3.3.1; load-bearing for the Condorcet argument, though the empirical results do not test it.
  • standard math Increasing N parallel trajectories reduces gradient-estimation variance per the CLT in Eq. (4).
    The CLT is standard for i.i.d. samples, but RL trajectories are non-stationary and correlated; the paper does not address this gap.
  • domain assumption The simulated market environments with transaction costs, slippage, turbulence threshold, and stop-loss faithfully represent real trading.
    Section 3.2 mentions these constraints but never specifies their values; the central trading results depend on the simulator's fidelity.
  • domain assumption The 30 Dow Jones stocks (2020-2023) and the 45-minute BTC out-of-sample window are representative for evaluating trading strategies.
    Results are reported on a single short crypto test window and one stock universe; Section 6.1 supplies no justification of representativeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024." pith.science (2026). https://pith.science/paper/XR3RZZWR

@misc{pith2026250110709,
  author       = {Pith},
  title        = {Pith review of: Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XR3RZZWR}},
  note         = {Machine review of arXiv:2501.10709}
}
abstract

Reinforcement learning has demonstrated great potential for performing financial tasks. However, it faces two major challenges: policy instability and sampling bottlenecks. In this paper, we revisit ensemble methods with massively parallel simulations on graphics processing units (GPUs), significantly enhancing the computational efficiency and robustness of trained models in volatile financial markets. Our approach leverages the parallel processing capability of GPUs to significantly improve the sampling speed for training ensemble models. The ensemble models combine the strengths of component agents to improve the robustness of financial decision-making strategies. We conduct experiments in both stock and cryptocurrency trading tasks to evaluate the effectiveness of our approach. Massively parallel simulation on a single GPU improves the sampling speed by up to $1,746\times$ using $2,048$ parallel environments compared to a single environment. The ensemble models have high cumulative returns and outperform some individual agents, reducing maximum drawdown by up to $4.17\%$ and improving the Sharpe ratio by up to $0.21$. This paper describes trading tasks at ACM ICAIF FinRL Contests in 2023 and 2024.

Figures

Figures reproduced from arXiv: 2501.10709 by the authors.

Figure 1
Figure 1. Performance deviation for different RL algorithms [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Producer-Consumer model for RL. 4 MASSIVELY PARALLEL SIMULATION 4.1 Simulation Phase for Gradient Estimate To estimate ∇𝐽 (𝜃) in (3), we can use the Monte Carlo method [25]: ∇𝐽 (𝜃) = 1 𝑁 ∑︁ 𝑁 𝑖=1 ∑︁ 𝑇 𝑡=1 𝑅(𝜏 (𝑖) ) ∇𝜃 log 𝜋𝜃 (𝑎 (𝑖) 𝑡 |𝑠 (𝑖) 𝑡 ), (4) Where 𝑁 trajectories are used. The Law of Large Numbers guaran￾tees that as the sample size 𝑁 increases, the estimation of ∇𝐽 (𝜃) will converge to its expected value. Ac… view at source ↗
Figure 3
Figure 3. Ensemble methods. fetches the data on the CPU for policy updating. PCIe allows for communication between the CPU and GPU but has limited band￾width, making frequent large data transfers a significant bottleneck. This problem can be solved by storing and processing data samples in tensors on the GPU. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Samples per second for the stock trading task and the cryptocurrency trading task. NVIDIA A100 GPU is used. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Cumulative returns of different strategies for the stock trading task and cryptocurrency trading task. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024

    q-fin.TR 2025-02 reject novelty 4.0 of 10

    The paper claims that fine-tuning LLaMA-3.2-3B with market-feedback prompts reduces volatility of LLM trading signals, but supplies no metrics to support the claim.

Reference graph

Works this paper leans on

38 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    Selim Amrouni, Aymeric Moulin, Jared Vann, Svitlana Vyetrenko, Tucker Balch, and Manuela Veloso. 2022. ABIDES-gym: gym environments for multi-agent dis- crete event simulation and application to financial markets. In ACM International Conference on AI in Finance (ICAIF ’21) . New York, NY, USA

    work page 2022

  2. [2]

    Leo Breiman. 1996. Bagging predictors. Machine Learning 24, 2 (1996), 123–140

    work page 1996

  3. [3]

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schul- man, Jie Tang, and Wojciech Zaremba. 2016. OpenAI gym. arXiv preprint arXiv:1606.01540 (2016)

    arXiv 2016

  4. [4]

    Stephanie CY Chan, Samuel Fishman, Anoop Korattikara, John Canny, and Ser- gio Guadarrama. 2020. Measuring the Reliability of Reinforcement Learning Algorithms. International Conference on Learning Representations (2020)

    work page 2020

  5. [5]

    Schapire

    Yoav Freund and Robert E. Schapire. 1996. Experiments with a new boosting al- gorithm. In Proceedings of the Thirteenth International Conference on International Conference on Machine Learning (ICML’96) . San Francisco, CA, USA, 148–156

    work page 1996

  6. [6]

    Sascha Yves Frey, Kang Li, Peer Nagy, Silvia Sapora, Christopher Lu, Stefan Zohren, Jakob Foerster, and Anisoara Calinescu. 2023. JAX-LOB: a GPU- accelerated limit order book simulator to unlock large scale reinforcement learn- ing for trading. In ACM International Conference on AI in Finance (ICAIF ’23) . New York, NY, USA, 583–591

    work page 2023

  7. [7]

    Scott Fujimoto, Herke van Hoof, and David Meger. 2018. Addressing function ap- proximation error in actor-critic methods. In International Conference on Machine Learning

    work page 2018

  8. [8]

    Ganaie, Minghui Hu, A.K

    M.A. Ganaie, Minghui Hu, A.K. Malik, M. Tanveer, and P.N. Suganthan. 2022. Ensemble deep learning: a review. Eng. Appl. Artif. Intell. 115, C (2022), 18 pages

    work page 2022

Show all 38 references
  1. [9]

    Jingyi Gu, Wenlu Du, A M Muntasir Rahman, and Guiling Wang. 2023. Margin trader: a reinforcement learning framework for portfolio management with margin and constraints. In ACM International Conference on AI in Finance (ICAIF ’23). New York, NY, USA, 9 pages

    2023

  2. [10]

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning , Vol. 80. PMLR, 1861–1870

    2018

  3. [11]

    Ben Hambly, Renyuan Xu, and Huining Yang. 2023. Recent advances in rein- forcement learning in finance. Mathematical Finance 33, 3 (2023), 437–503

    2023

  4. [12]

    Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double Q-Learning. In AAAI Conference on Artificial Intelligence (AAAI’16). 2094–2100

    2016

  5. [13]

    Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. 2018. Deep reinforcement learning that matters. In AAAI Conference on Artificial Intelligence (AAAI’18) . 8 pages

    2018

  6. [14]

    Liu Jing and Yuncheol Kang. 2024. Automated cryptocurrency trading approach using ensemble deep reinforcement learning: Learn to understand candlesticks. Expert Syst. Appl. 237 (2024), 20 pages

    2024

  7. [15]

    Schulman John, Wolski Filip, Dhariwal Prafulla, Radford Alec, and Klimov Oleg

  8. [16]

    Zura Kakushadze. 2016. 101 formulaic alphas. arXiv preprint arXiv:1601.00991 (2016)

    2016 arXiv

  9. [17]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2016. Continuous control with deep reinforcement learning. In International Conference on Learning Representations, ICLR

    2016

  10. [18]

    Xiao-Yang Liu, Ziyi Xia, Jingyang Rui, Jiechao Gao, Hongyang Yang, Ming Zhu, Christina Wang, Zhaoran Wang, and Jian Guo. 2022. FinRL-Meta: Market en- vironments and benchmarks for data-driven financial reinforcement learning. Advances in Neural Information Processing Systems (...

    2022

  11. [19]

    Xiao-Yang Liu, Ziyi Xia, Hongyang Yang, Jiechao Gao, Daochen Zha, Ming Zhu, Christina Dan Wang, Zhaoran Wang, and Jian Guo. 2024. Dynamic datasets and market environments for financial reinforcement learning. Machine Learning - Nature (2024)

    2024

  12. [20]

    Xiao-Yang Liu, Zhuoran Xiong, Shan Zhong, Hongyang Yang, and Anwar Walid

  13. [21]

    Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. 2020. FinRL: a deep reinforcement learning ACM ICAIF ’24, Nov. 14–15, 2025, Brooklyn, NY Nikolaus Holzer, Keyi Wang, Kairong Xiao, and Xiao-Yang Liu Yanglet library for aut...

    2020

  14. [22]

    Xiao-Yang Liu, Hongyang Yang, Jiechao Gao, and Christina Dan Wang. 2022. FinRL: deep reinforcement learning framework to automate trading in quantita- tive finance. ACM International Conference on AI in Finance (2022)

    2022

  15. [23]

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. 2021. Isaac gym: high performance GPU based physics simulation for robot learning. In Proceedings of the N...

    2021

  16. [24]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-level control through deep reinforcement learning. Nature 518 (2015), 529–533

    2015

  17. [25]

    Shakir Mohamed, Mihaela Rosca, Michael Figurnov, and Andriy Mnih. 2020. Monte Carlo gradient estimation in machine learning. J. Mach. Learn. Res. 21, 1 (2020), 62 pages

    2020

  18. [26]

    Fernando Perez-Cruz. 2008. Kullback-Leibler divergence estimation of continuous distributions. In IEEE International Symposium on Information Theory. 1666–1670

    2008

  19. [27]

    Andrew Bagnell

    Jan Peters and J. Andrew Bagnell. 2010. Policy gradient methods. Springer US, Boston, MA, 774–776

    2010

  20. [28]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: your language model is secretly a reward model. In Advances in Neural Information Processing Systems, Vol. 36. 53728–53741

    2023

  21. [29]

    Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. 2022. Learning to walk in minutes using massively parallel deep reinforcement learning.Proceedings of the 5th Conference on Robot Learning (2022), 91–100

    2022

  22. [30]

    Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems , Vol. 12

    1999

  23. [31]

    Edoardo Vittori, Michele Trapletti, and Marcello Restelli. 2021. Option hedging with risk averse reinforcement learning. In ACM International Conference on AI in Finance (ICAIF ’20) . New York, NY, USA, 8 pages

    2021

  24. [32]

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas. 2016. Dueling network architectures for deep reinforcement learning. In International Conference on International Conference on Machine Learning (ICML’16, Vol. 48). 1995–2003

    2016

  25. [33]

    M. A. Wiering and H. van Hasselt. 2008. Ensemble algorithms in reinforcement learning. Trans. Sys. Man Cyber. Part B 38, 4 (2008), 930–936

    2008

  26. [34]

    Hongyang Yang, Xiao-Yang Liu, Shan Zhong, and Anwar Walid. 2021. Deep reinforcement learning for automated stock trading: an ensemble strategy. In ACM International Conference on AI in Finance (ICAIF ’20) . New York, NY, USA

    2021

  27. [35]

    Zihao Zhang, Stefan Zohren, and Stephen Roberts. 2019. Deep reinforcement learning for trading. Journal of Financial Data Science (2019)

    2019

  28. [36]

    Wenshuai Zhao, Jorge Peña Queralta, and Tomi Westerlund. 2020. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. InIEEE Symposium Series on Computational Intelligence (SSCI) . 737–744

    2020

  29. [2017]

    arXiv preprint arXiv:1707.06347 (2017)

    Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  30. [2018]

    NeurIPS Workshop on Challenges and Opportunities for AI in Financial Services (2018)

    Practical deep reinforcement learning approach for stock trading. NeurIPS Workshop on Challenges and Opportunities for AI in Financial Services (2018)

    2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.