Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards

Caglar Gulcehre; Daniil Karzanov; Marcin Detyniecki; Mikhail Terekhov; Rub\'en Garz\'on; Thomas Raffinot

REVIEW 4 major objections 5 minor 45 references

Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A PPO agent trained against a hindsight-optimal Oracle repeatedly beats the 60/40 benchmark in out-of-sample return.

desk verdict A worthwhile RL portfolio-overlay pipeline undermined by a misnamed reward and missing error bars; the test-phase result is suggestive, not demonstrated. read the letter →

arxiv 2502.02619 v1 pith:CPRIHQ56 submitted 2025-02-04 q-fin.PM cs.LGq-fin.RM

Daniil Karzanov , Rubén Garzón , Mikhail Terekhov , Caglar Gulcehre , Thomas Raffinot , Marcin Detyniecki This is my paper · ORCID

classification q-fin.PMcs.LGq-fin.RM MSC 91G1068T07

keywords deepreinforcementlearningProximalPolicyOptimizationregretrewardcircularblockbootstraptransactioncostschedulerportfoliorebalancing60/40benchmarkmaximumdrawdown

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a deep reinforcement learning overlay, built on Proximal Policy Optimization (PPO), can improve a traditional 60/40 stocks/bonds portfolio by dynamically rebalancing among equity, balanced, and bond sleeves. The agent is trained with a negative Sharpe-based regret reward that penalizes its return shortfall relative to a hindsight-optimal (Oracle) allocation computed from forward-looking returns and a transaction-cost penalty. To make the policy generalize, the authors train on circular block bootstrap synthetic data and a transaction cost scheduler that ramps fees during training. They report that the full configuration outperforms the 60/40 benchmark in annual return across all three test phases, and lowers maximum drawdown relative to the benchmark in two of the three test phases. If the claim is right, a cost-aware, hindsight-trained policy can serve as a practical enhancement layer on top of an existing allocation strategy.

What carries the argument

The central object is the negative Sharpe regret reward in Eq. (8), $\text{Reward}_t = -\bar{\mu}_{t+n}^{t} (w^* - w_t)'$, which measures the agent's return shortfall against the Oracle allocation $w^*$ defined in Eq. (9) as the maximizer of a forward-looking Sharpe ratio over the next $n = 14$ business days, computed with a covariance window from $t-3n$ to $t+3n$ and penalized by the transaction cost term $\text{TC}_{\text{train}}(t)\,\|w - w_{t-1}\|_1$. The reward makes every allocation comparable to the best hindsight choice and pushes the policy toward the Oracle's behavior. The supporting machinery is PPO with clipping, a circular block bootstrap applied every 10 episodes with block sizes of 70-90% of the training set, and a curriculum-style transaction cost scheduler that raises fees from zero to 0.0025 during training. The bootstrap acts as a regularizer that prevents the agent from memorizing non-reproducible strategies, and the scheduler lets the agent first learn allocation signal before facing frictions.

What would settle it

Permute the daily returns of each asset independently, destroying temporal structure while preserving cross-sectional correlations, and run the full training pipeline on the shuffled data; if the regret agent still produces out-of-sample returns comparable to those in Table 1, the edge is an artifact of the training procedure rather than predictive signal in the state features.

Watch

Extended reading notes

Core claim

The paper's central claim is that the combination of the negative Sharpe regret reward, circular block bootstrap training, and a transaction cost scheduler yields a PPO agent that generalizes out-of-sample and beats the 60/40 benchmark. In the test rows of Table 1, the regret agent's annual returns are 0.064, 0.128, and -0.007 against benchmark returns of 0.056, 0.105, and -0.026, and it improves on the benchmark's maximum drawdown in two of the three test phases. The paper reads this as evidence that regret-shaped rewards using Oracle knowledge during training transfer better than rewards that directly optimize the displayed return metric, which the ablation shows overfits to training.

Load-bearing premise

The out-of-sample edge rests on the assumption that the state features — rolling 40-day and 60-day returns, VIX, high-yield spread, and MOVE — carry stable predictive information for the next 14 business days; if markets are efficient or the return process shifts materially across periods, the reported outperformance is noise or overfitting.

Editorial extensions

If this is right

A regret-trained PPO overlay can add return relative to a passive 60/40 allocation across distinct market regimes (pre-pandemic, pandemic, post-pandemic) while remaining within reasonable drawdown bounds.
The combination of transaction cost scheduling and block bootstrap synthetic data is what makes the regret reward transfer; removing either component degrades validation and test performance.
Reward functions that directly maximize the reported return metric tend to overfit the training window and fail out-of-sample, suggesting that hindsight-based reward shaping is a more robust training signal.
The pipeline, being agnostic to the specific assets, can be applied to other sequential allocation problems such as supply chain resource allocation or bandwidth distribution, as the paper notes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported edge depends on the Oracle's 14-day look-ahead and 42-day covariance window; varying n and testing whether outperformance survives would isolate whether the signal is short-term momentum or a longer-horizon effect.
Since the bootstrap only reorders historical blocks, it does not generate genuinely new regimes; an inference is that the policy may not adapt to a crisis unlike any in the training window, and a test on post-2024 data would probe this.
The paper averages over 20 independent agents but reports only the mean performance; an inference is that the individual policies vary substantially, and a practitioner could use the cross-agent variance as a confidence measure for each rebalancing decision.
MDD is not part of the reward, and the paper's MDD results are better than the benchmark in two of three phases but worse in the pandemic phase; a natural extension is to add a drawdown penalty to the regret reward to make the drawdown improvement more consistent.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a plausible RL overlay for a 60/40 book that deserves a proper referee but only if the authors are pushed to confront the statistics. The new bits are the transaction-cost curriculum, the circular block bootstrap during training, and a regret reward computed against a hindsight oracle. The combination is genuinely not in the cited literature. Credit also for testing against two published reward baselines and for a candid "What Didn't Work" section. That is real engineering care.

The soft spots are the usual load-bearing kind. First, the reward in Eq. (8) is not Sharpe-based; it is linear in the agent weights, −μ̄(w*−w). The covariance shows up only in the oracle w*, so calling it "negative Sharpe regret" overstates what the agent optimizes. Second, the headline test-phase numbers have no confidence intervals or significance tests. Twenty seeds are trained, so the authors clearly have the data to give us error bars; they just don't. The gaps in Table 1 (0.064 vs 0.056, 0.128 vs 0.105, −0.007 vs −0.026) are small against annualized equity volatility, and with three phases and several metrics you would expect some positives by chance. Figure 3 shows distributions but no numeric summary, so we cannot tell whether the 60/40 benchmark sits inside the agent's spread. Third, the out-of-sample transfer rests entirely on the Section 4.2 features being predictive of 14-day returns, and there is no feature significance, no lagged-feature or noise-feature control, and no evidence that an overfit policy memorized training regimes. And there is a text-to-table slip: the paper says the agent outperforms everywhere except "validation in the pre-pandemic phase," but Table 1 shows that phase (valid Phase 1) is actually an outperformance, while valid Phase 2 (pandemic) is the underperformance.

None of this kills the idea. A re-run with significance tests, a test of the features' predictive power, and a corrected reward description would answer most of it. The paper is for RL-finance researchers and practitioners who want a concrete overlay recipe; they should read it as a proposal, not a result.

For peer review: yes, send it out. It is worth referee time. Require the authors to provide error bars, a feature-validity check, and a fixed Section 5 text before acceptance.

Referee Report

4 major / 5 minor

Summary. This paper proposes a PPO-based overlay that dynamically rebalances a 60/40 equity/bond portfolio among three strategies (developed-market equity, the 60/40 portfolio, and global government bonds). Training uses a future-looking 'regret' reward: the agent is rewarded for matching the forward return of a hindsight-optimal (Oracle) allocation, with a transaction-cost curriculum and circular-block-bootstrap synthetic data. The authors compare their regret PPO against the static 60/40 benchmark and two DRL baselines across three train-validation-test phases (1996-2024), reporting that regret PPO beats 60/40 on annual return in all three test phases and on maximum drawdown in two of three. They also present an ablation study and a section on failed approaches.

Significance. If the reported result holds, the paper would make a useful empirical contribution: a practical DRL overlay that adds value to a standard 60/40 allocation while explicitly handling transaction frictions and using a hindsight-informed reward without leaking future information at inference. The paper has real strengths: the environment and reward are specified precisely (Eqs. 7-9), Algorithm 1 gives a complete training loop, the ablation in Fig. 1 is informative, the evaluation uses 20 seeds, and the authors are transparent about approaches that failed. The central claim is empirical, so circularity concerns are limited to the legitimate use of hindsight during training. However, the current evidence for the central claim is incomplete: the reported outperformance lacks statistical inference, and the mechanism by which the policy transfers from training features to test periods is not demonstrated.

major comments (4)

[§5.1, Table 1 and Fig. 3] The paper's central claim—that the regret PPO agent outperforms the 60/40 benchmark on annual return in all three test phases—rests on point estimates only. No standard deviations, confidence intervals, or significance tests are reported for the 20 seeds, even though the text says 20 independent agents were trained. The Phase 1 and Phase 3 test-return gaps (0.064 vs 0.056 and -0.007 vs -0.026, respectively) are small relative to annualized equity volatility, and with three phases and multiple metrics some favorable comparisons are expected by chance. Figure 3 plots distributions but does not report numeric quantiles, so the reader cannot tell whether the benchmark lies inside the agent's return or MDD distribution. The authors should report 95% confidence intervals for the entries in Table 1, a bootstrap or permutation test of the mean against the benchmark per test phase, and the fraction of seeds that beat the benchmark.
[§4.2, §4.4 and §5.1] Because the reward is set to zero at test time (Section 4.4), the policy can only outperform by transferring predictive signal from the observation features in Section 4.2 to the next 14-day returns. The paper provides no evidence that this transfer exists: there is no regression of forward returns on the features, no noise-feature or lagged-feature control, no feature-ablation study, and no analysis of distribution shift across the three phases. The reported test-phase outperformance could therefore arise from memorizing training-period patterns that coincide with the test windows. I would like to see at least one control experiment in which a feature is replaced by an identically distributed noise series, with a report of how test performance degrades; alternatively, an out-of-sample predictive regression for the 14-day forward return using the Section 4.2 covariates.
[§4.4, Eq. (8)] Equation (8) defines Reward_t = - mean return * (w* - w_t), which is linear in the agent's action w_t. For a fixed w*, maximizing expected cumulative reward is equivalent to maximizing expected portfolio return; the Sharpe ratio enters only through the definition of w* in Eq. (9). Thus the term 'Regret-based Sharpe reward function' overstates the role of risk adjustment: the agent is not penalized for the variance of its own allocations except through the oracle target. This may explain the inconsistent MDD results in Table 1. The authors should either define the reward as a function of the Sharpe-ratio regret, or explicitly reframe the contribution as a return-regret reward with a Sharpe-based oracle and adjust the claims and comparisons accordingly.
[§4.6 and Table 2] Several per-phase design choices are reported without sensitivity analysis: the TC scheduler shape is changed to be more concave in Phases 2 and 3, the entropy schedule is set per phase, and the model is selected from the Pareto front with a preference for less risk-averse models. These choices are validated on the validation set, but because they are made separately for each phase and are not held fixed or varied in the reported experiments, the test results conflate method performance with per-phase tuning. The authors should either fix a single protocol for all phases or provide a sensitivity analysis showing that the Table 1 conclusions are robust to these choices.

minor comments (5)

[§5.1] The list of performance measures mentions the Sortino Ratio, but Table 1 does not report Sortino; either add the column or remove the bullet.
[§4.5 and Table 2] Section 4.5 states that block sizes of 70-90% of the training set performed best, but Table 2 omits the exact block size used in the reported experiments; specify it.
[Algorithm 1 and §4.5] Algorithm 1 selects synthetic data with a Bernoulli(0.7) draw every 10 episodes, whereas Section 4.5 describes a deterministic alternation between 10 episodes of real data and 10 episodes of synthetic data; reconcile the description.
[Table 4] The end date of each validation period equals the start date of the corresponding test period; this is presumably intentional but should be stated explicitly to avoid the appearance of overlap.
[Table 2 and Fig. 4] Table 2 lists alpha values of 1 and 0.45 for the transaction-cost convexity, while Fig. 4 shows several other alpha values; clarify which alpha is used in each phase and whether the other curves are exploratory.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an external backtest; the oracle target is a training-time hindsight signal, and the test evaluation is independent of it.

full rationale

The paper's central claim is an empirical backtest comparison against the external 60/40 benchmark and two literature baselines. The Oracle allocation w* in Eq. (9) is computed from future returns and a covariance window that includes future data, and it is used only in the training reward of Eq. (8). The paper explicitly states that at inference time the reward is set to zero and no future information is used, and that train, validation, and test periods do not overlap. The reported test-phase outperformance therefore cannot reduce by construction to the Oracle's hindsight knowledge; it depends on whether the observation features in Section 4.2 generalize across phases, which is an empirical assumption rather than a circular derivation. No load-bearing argument relies on a self-citation, an imported uniqueness theorem, or an ansatz smuggled in via citation. The only mild concern—whether the small test-phase return gaps are statistically robust given 20 seeds—is a question of evidence strength, not circularity. Accordingly, no circular step is present and the score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The 'Oracle agent' w* is a mathematical target defined in Eq. (9) and used only to compute training rewards; it is not a postulated external object with independent evidence. The free parameters are primarily lookahead horizon, transaction cost magnitude and schedule, bootstrap block size, and entropy schedule, all chosen by hand or exploratory search.

free parameters (5)

Oracle lookahead n (n_steps_foresee) = 14 business days
Horizon for the expected forward-return vector in the reward, Eq. (8)-(9). Chosen by hand and directly sets how far into the future the hindsight oracle looks.
Evaluation transaction cost TC_eval = 0.0025
Per-trade transaction cost applied after the ramp in Eq. (7). The authors describe it as a fair value for traditional brokers and large institutional traders; it directly affects returns and allocation decisions.
TC scheduler shape = alpha = 1 in Phase 1, alpha = 0.45 in Phases 2 and 3; ramp limit S = 100 * episode_length
Convexity parameter and ramp duration of the cost curriculum in Eq. (7) and Figure 4, selected through an exploratory search rather than a principled derivation.
Circular block bootstrap block size = 70-90% of the original training set
Chosen after experiments in Section 4.5; larger blocks preserve long-range temporal dependencies better, but the exact block length is not fixed or reported per run.
Entropy coefficient start value = 0.00005, annealed to 0 over first 10% of episodes
Exploration regularization in the PPO loss, Eq. (10). The annealing schedule is an implementation choice that affects generalization and policy diversity.

assumptions (5)

domain assumption Historical return process is sufficiently stationary or repeatable that a policy trained on real and block-bootstrapped data generalizes to future holdout periods.
The entire evaluation in Table 1 assumes that patterns learned in each training window carry over to the validation and test windows. Section 4.5 motivates the bootstrap as a regularizer to support this transfer.
domain assumption The state features defined in Section 4.2 (rolling 40/60-day returns and volatilities, VIX, high-yield spread, MOVE) have predictive content for the next 14-day future returns.
The reward in Eq. (8) trains the agent to maximize hindsight return; without predictive signal in the features, the policy cannot beat 60/40 out of sample.
ad hoc to paper A curriculum over transaction costs in Eq. (7) converges to a policy that remains good under full costs, with TC_eval = 0.0025.
The scheduler is introduced to prevent signal loss but is not derived from any optimality principle; its benefit is supported only by the ablation in Figure 1.
standard math PPO with the declared hyperparameters converges to a stable and useful policy in this nonstationary financial environment.
Assumes standard convergence and stability properties of clipped PPO (Eq. 6) in a setting with hindsight-shaped rewards and bootstrap-generated data; the paper does not analyze convergence.
domain assumption There is no data leakage between training and test periods within each phase, despite overlapping calendar windows across phases.
Table 4 shows that some test dates in one phase appear in training or validation of another phase. The authors assert no overlap within each split; if leakage exists, the test results would be inflated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards." pith.science (2026). https://pith.science/paper/CPRIHQ56

@misc{pith2026250202619,
  author       = {Pith},
  title        = {Pith review of: Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CPRIHQ56}},
  note         = {Machine review of arXiv:2502.02619}
}

read the original abstract

This paper introduces a novel agent-based approach for enhancing existing portfolio strategies using Proximal Policy Optimization (PPO). Rather than focusing solely on traditional portfolio construction, our approach aims to improve an already high-performing strategy through dynamic rebalancing driven by PPO and Oracle agents. Our target is to enhance the traditional 60/40 benchmark (60% stocks, 40% bonds) by employing the Regret-based Sharpe reward function. To address the impact of transaction fee frictions and prevent signal loss, we develop a transaction cost scheduler. We introduce a future-looking reward function and employ synthetic data training through a circular block bootstrap method to facilitate the learning of generalizable allocation strategies. We focus on two key evaluation measures: return and maximum drawdown. Given the high stochasticity of financial markets, we train 20 independent agents each period and evaluate their average performance against the benchmark. Our method not only enhances the performance of the existing portfolio strategy through strategic rebalancing but also demonstrates strong results compared to other baselines.

Figures

Figures reproduced from arXiv: 2502.02619 by the authors.

**Figure 1.** The evolution of accumulated (financial) return over training. Ablation study: removal of specific model components [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Example of allocation during the testing period of [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Distribution of returns and MDDs for the PPO [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: Example of allocation and return dynamics by Diff. Sharpe (1) during the testing period of phase 3. 2022-01 2022-04 2022-07 2022-10 2023-01 2023-04 2023-07 2023-10 2024-01 time 0.0 0.2 0.4 0.6 0.8 1.0 position DevEquity 60/40 Govies 2022-01 2022-04 2022-07 2022-10 2023…

**Figure 4.** Figure 4: Transaction cost schedule example. 2022-01 2022-04 2022-07 2022-10 2023-01 2023-04 2023-07 2023-10 2024-01 time 0.0 0.2 0.4 0.6 0.8 1.0 position DevEquity 60/40 Govies 2022-01 2022-04 2022-07 2022-10 2023-01 2023-04 2023-07 2023-10 2024-01 X axis 0.05 0.04 0.03 0.02 0.…

**Figure 7.** Figure 7: Multi-Body CNN allocation example. We can observe noisy allocation behaviors. Introducing regularization to smooth the allocation line could enhance performance. For instance, adding a term to the reward function that penalizes sharp allocation changes—only to revert …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 35 canonical work pages

[1]

Saud Almahdi and Steve Y Yang. 2017. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Expert Systems with Applications 87 (2017), 267– 279

work page 2017
[2]

Henrik Andersson, Henrik Scholtz, and Jiakun Zheng. 2023. Measuring regret theory in the health and financial domain. (2023)

work page 2023
[3]

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning. 41–48

2009
[4]

Eric Benhamou, David Saltiel, Jean-Jacques Ohana, and Jamal Atif. 2021. De- tecting and adapting to crisis pattern with context based Deep Reinforcement Learning. In 2020 25th International Conference on Pattern Recognition (ICPR) . IEEE, 10050–10057

work page 2021
[5]

Eric Benhamou, David Saltiel, Sandrine Ungari, and Abhishek Mukhopadhyay

work page
[6]

Kiran Bisht and Arun Kumar. 2020. Deep reinforcement learning based multi- objective systems for financial trading. In 2020 5th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE) . IEEE, 1–6

work page 2020
[7]

Ivo Blohm, Torben Antretter, Charlotta Sirén, Dietmar Grichnik, and Joakim Win- cent. 2020. It’s a Peoples Game, Isn’t It?! A Comparison Between the Investment Returns of Business Angels and Machine Learning Algorithms. Entrepreneurship Theory and Practice 46 (2020), 1054 – 1091. https://api.semanticscholar.org/ CorpusID:224959426

work page 2020
[8]

Alessio Brini and Daniele Tantari. 2023. Deep reinforcement trading with pre- dictable returns. Physica A: Statistical Mechanics and its Applications 622 (2023), 128901

work page 2023

Show all 45 references

[9]

Jurgita Černevičien˙e and Audrius Kabašinskas. 2022. Review of multi-criteria decision-making methods in finance using explainable artificial intelligence. Frontiers in artificial intelligence 5 (2022), 827584

2022
[10]

Alexei Chekhlov, Stanislav Uryasev, and Michael Zabarankin. 2005. Drawdown measure in portfolio optimization.International Journal of Theoretical and Applied Finance 8, 01 (2005), 13–58

2005
[11]

Federico Cornalba, Constantin Disselkamp, Davide Scassola, and Christopher Helf. 2024. Multi-objective reward generalization: improving performance of Deep Reinforcement Learning for applications in single-asset trading. Neural Computing and Applications 36, 2 (2024), 619–637

2024
[12]

Richard A Davis, Keh-Shin Lii, and Dimitris N Politis. 2011. Remarks on some non- parametric estimates of a density function. Selected Works of Murray Rosenblatt (2011), 95–100

2011
[13]

Marcos M López de Prado. 2020. Machine learning for asset managers . Cambridge University Press

2020
[14]

Yi Ding, Weiqing Liu, Jiang Bian, Daoqiang Zhang, and Tie-Yan Liu. 2018. Investor- imitator: A framework for trading knowledge extraction. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 1310–1319

2018
[15]

Joost Driessen, Pascal J Maenhout, and Grigory Vilkov. 2009. The price of correlation risk: Evidence from equity options. The Journal of Finance 64, 3 (2009), 1377–1406

2009
[16]

predict, then optimize

Adam N Elmachtoub and Paul Grigas. 2022. Smart “predict, then optimize”. Management Science 68, 1 (2022), 9–26

2022
[17]

Justin Fu, Katie Luo, and Sergey Levine. 2018. Learning Robust Rewards with Adversarial Inverse Reinforcement Learning. (2018). arXiv:1710.11248 [cs.LG] https://arxiv.org/abs/1710.11248

2018 arXiv
[18]

Simon Gilchrist and Egon Zakrajšek. 2012. Credit spreads and business cycle fluctuations. American economic review 102, 4 (2012), 1692–1720

2012
[19]

Igor Halperin, Jiayu Liu, and Xiao Zhang. [n. d.]. Combining Reinforcement Learning and Inverse Reinforcement Learning for Asset Allocation Recommen- dations. arXiv:2201.01874 [cs, q-fin] http://arxiv.org/abs/2201.01874

arXiv
[20]

Ben Hambly, Renyuan Xu, and Huining Yang. 2023. Recent advances in rein- forcement learning in finance. Mathematical Finance 33, 3 (2023), 437–503

2023
[21]

Jonathan Ho and Stefano Ermon. 2016. Generative adversarial imitation learning. Advances in neural information processing systems 29 (2016)

2016
[22]

Zhengyao Jiang, Dixing Xu, and Jinjun Liang. 2017. A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059 (2017)

2017 arXiv
[23]

Daniil Karzanov. 2023. Headline-Driven Classification and Local Interpretation for Market Outperformance and Low-Risk Stock Prediction. Computational Economics (2023), 1–20

2023
[24]

Vasileios Kochliaridis, Eleftherios Kouloumpris, and Ioannis Vlahavas. 2023. Com- bining deep reinforcement learning with technical analysis and trend monitoring on cryptocurrency markets. Neural Computing and Applications 35, 29 (2023), 21445–21462

2023
[25]

Allison Koenecke and Amita Gajewar. 2020. Curriculum learning in deep neural networks for financial forecasting. In Mining Data for Financial Applications: 4th ECML PKDD Workshop, MIDAS 2019, Würzburg, Germany, September 16, 2019, Revised Selected Papers 4 . Springer, 16–31

2020
[26]

Chung I Lu. 2023. Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation. arXiv preprint arXiv:2307.07694 (2023)

2023 arXiv
[27]

Giorgio Lucarelli and Matteo Borrotti. 2019. A deep reinforcement learning approach for automated cryptocurrency trading. In Artificial Intelligence Appli- cations and Innovations: 15th IFIP WG 12.5 International Conference, AIAI 2019, Hersonissos, Crete, Greece, May 24–26, 20...

2019
[28]

Giorgio Lucarelli and Matteo Borrotti. 2020. A deep Q-learning portfolio man- agement framework for the cryptocurrency market. Neural Computing and Applications 32 (2020), 17229–17244

2020
[29]

Jayanta Mandi, Peter J Stuckey, Tias Guns, et al. 2020. Smart predict-and-optimize for hard combinatorial optimization problems. In Proceedings of the AAAI Con- ference on Artificial Intelligence, Vol. 34. 1603–1610

2020
[30]

Harry Markowitz. 1952. Portfolio Selection. The Journal of Finance 7, 1 (1952), 77–91. http://www.jstor.org/stable/2975974

1952
[31]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. nature 518, 7540 (2015), 529–533

2015
[32]

Bernardo K Pagnoncelli, Domingo Ramírez, Hamed Rahimian, and Arturo Ci- fuentes. 2023. A synthetic data-plus-features driven approach for portfolio optimization. Computational Economics 62, 1 (2023), 187–204

2023
[33]

José-Manuel Peña, Fernando Suárez, Omar Larré, Domingo Ramírez, and Arturo Cifuentes. 2024. A modified CTGAN-plus-features-based method for optimal asset allocation. Quantitative Finance 24, 3-4 (2024), 465–479

2024
[34]

Uta Pigorsch and Sebastian Schäfer. 2022. High-dimensional stock portfolio trad- ing with deep reinforcement learning. In 2022 IEEE Symposium on Computational Intelligence for Financial Engineering and Economics (CIFEr) . IEEE, 1–8

2022
[35]

Mélanie Rey. 2015. Copula models in machine learning . Ph. D. Dissertation. University_of_Basel

2015
[36]

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov
[37]

Kevin Sheppard. 2023. ARCH: Autoregressive Conditional Heteroskedasticity models in Python. https://github.com/bashtage/arch Version 5.3.1

2023
[38]

David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershel- vam, Marc Lanctot, et al . 2016. Mastering the game of Go with deep neural networks and tree search. nature 529, 7587 (2...

2016
[39]

Srijan Sood, Kassiani Papasotiriou, Marius Vaiciulis, and Tucker Balch. 2023. Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization. FinPlan 2023, 2023 (2023), 21

2023
[40]

Sonam Srivastava et al. 2020. Deep reinforcement learning for asset allocation in us equities. arXiv preprint arXiv:2010.04404 (2020)

2020 arXiv
[41]

Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An intro- duction. MIT press

2018
[42]

Robert E Whaley. 2000. The investor fear gauge. Journal of portfolio management 26, 3 (2000), 12

2000
[43]

Jimmy Ming-Tai Wu, Sheng-Hao Lin, Jia-Hao Syu, and Mu-En Wu. 2022. Em- bedded draw-down constraint reward function for deep reinforcement learning. Applied Soft Computing 125 (2022), 109150. APPENDIX Table 2: Hyperparameter values. Hyperparameter Description Value 𝛾 Reward dis...

2022
[2017]

arXiv preprint arXiv:1707.06347 (2017)

Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

2017 arXiv
[2020]

arXiv preprint arXiv:2010.09108 (2020)

Bridging the gap between Markowitz planning and deep reinforcement learning. arXiv preprint arXiv:2010.09108 (2020)

2020 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Saud Almahdi and Steve Y Yang. 2017. An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Expert Systems with Applications 87 (2017), 267– 279

work page 2017

[2] [2]

Henrik Andersson, Henrik Scholtz, and Jiakun Zheng. 2023. Measuring regret theory in the health and financial domain. (2023)

work page 2023

[3] [3]

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning. 41–48

2009

[4] [4]

Eric Benhamou, David Saltiel, Jean-Jacques Ohana, and Jamal Atif. 2021. De- tecting and adapting to crisis pattern with context based Deep Reinforcement Learning. In 2020 25th International Conference on Pattern Recognition (ICPR) . IEEE, 10050–10057

work page 2021

[5] [5]

Eric Benhamou, David Saltiel, Sandrine Ungari, and Abhishek Mukhopadhyay

work page

[6] [6]

Kiran Bisht and Arun Kumar. 2020. Deep reinforcement learning based multi- objective systems for financial trading. In 2020 5th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE) . IEEE, 1–6

work page 2020

[7] [7]

Ivo Blohm, Torben Antretter, Charlotta Sirén, Dietmar Grichnik, and Joakim Win- cent. 2020. It’s a Peoples Game, Isn’t It?! A Comparison Between the Investment Returns of Business Angels and Machine Learning Algorithms. Entrepreneurship Theory and Practice 46 (2020), 1054 – 1091. https://api.semanticscholar.org/ CorpusID:224959426

work page 2020

[8] [8]

Alessio Brini and Daniele Tantari. 2023. Deep reinforcement trading with pre- dictable returns. Physica A: Statistical Mechanics and its Applications 622 (2023), 128901

work page 2023