Pith. sign in

REVIEW 3 major objections 5 minor 72 references

AlphaZeroBeta: Deep Reinforcement Learning for Market-Neutral Portfolios

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A deep reinforcement learning policy can beat convex optimizers at market-neutral investing across seven equity indices.

desk verdict Solid, transparent DRL-for-portfolios paper whose headline Sharpe result is undercut by a baseline-constraint asymmetry the paper misdescribes. read the letter →

arxiv 2607.18001 v1 pith:C74EM6DL submitted 2026-07-20 q-fin.PM q-fin.CP

Boris Belyakov This is my paper
classification q-fin.PMq-fin.CP MSC 91G1068T07 PACS 89.65.Gh
keywords Market-neutralportfolioDeepreinforcementlearningRecurrentPPOCNN-GRUDollarneutralitySharperatioWalk-forwardbacktestingoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a deep reinforcement learning agent, trained with a composite reward that balances risk-adjusted excess return, benchmark correlation, and transaction costs, can construct market-neutral portfolios that beat classical convex optimization baselines. The agent learns directly from market data and enforces dollar neutrality by construction — every weight vector is centered and projected to have gross exposure at most one — so no factor model or beta estimate is needed. Across 22 walk-forward out-of-sample slices (2014–2024) on seven equity indices, the reported result is a higher Sharpe ratio in every market, benchmark correlations within ±0.15 of zero, and drawdowns uniformly shallower than buy-and-hold. A sympathetic reader would care because the claim, if true, means an end-to-end learned policy can replace hand-specified factor hedges with a single objective that encodes the constraints directly.

What carries the argument

The load-bearing mechanism is the reward function R_t = (r_p − r_m)/σ_p − λ_1·Corr(r_p, r_m) − λ_2·Σ|Δw_i|, evaluated with a CNN-GRU encoder feeding a Recurrent PPO actor-critic. After the policy outputs a weight vector, a centering step subtracts the cross-sectional mean and an L1-projection caps gross exposure at one, making dollar neutrality a hard constraint rather than a soft objective. The reward is the only training signal; it simultaneously drives alpha, discourages benchmark comovement, and limits turnover.

What would settle it

Re-run the walk-forward protocol with the convex baselines constrained to the same dollar-neutral set (sum of weights = 0, gross exposure ≤ 1); if AlphaZeroBeta's Sharpe advantage largely disappears, the claim of superior learning is an artifact of unequal constraints.

Watch

Extended reading notes

Core claim

The paper proposes AlphaZeroBeta, a deep RL system that learns dollar-neutral portfolios end-to-end, and claims that over 22 out-of-sample walk-forward windows spanning 2014–2024 on seven equity indices it achieves higher Sharpe ratios than buy-and-hold, maximum-Sharpe, and minimal-correlation baselines while keeping benchmark correlation within ±0.15 of zero. Drawdowns are uniformly shallower than buy-and-hold and shallower than the decorrelation baseline in six of seven markets. The author attributes the improvement to the composite reward — risk-adjusted excess return minus a correlation penalty minus a turnover penalty — combined with a hard projection that forces the sum of weights to z

Load-bearing premise

The comparison to the convex baselines is assumed fair even though AlphaZeroBeta caps gross exposure at one while the baselines can hold gross exposure up to the number of assets, so the higher Sharpe ratios could reflect tighter risk limits rather than superior stock selection.

Editorial extensions

If this is right

  • Market-neutral portfolios can be built without factor models or beta estimates; the learned policy encodes neutrality structurally.
  • The same hyperparameters transfer across universes from 30 to 2,200 stocks, suggesting the method scales without per-index tuning.
  • The correlation penalty, not RL alone, drives the improvement: removing it raises benchmark correlations into the 0.4–0.6 range and deepens drawdowns (Appendix B).
  • The factor attribution implies the policy behaves as a moderate trend follower with negative reversal loading, so the realized alpha is associated with momentum exposure rather than unexplained skill.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the baselines were restricted to the same dollar-neutral, gross-exposure-one constraint set, the Sharpe gap could narrow; that experiment would test whether the superiority is alpha or constraint asymmetry.
  • The momentum and reversal loadings suggest a large part of the return stream might be a well-known long-short momentum premium; comparing directly against a factor-mimicking momentum portfolio would clarify the incremental contribution.
  • The fixed 2025-02-01 SSE composite universe is the paper's own stated survivorship caveat; using time-varying membership would tell whether the China result survives.
  • A live simulation with realistic market impact and execution latency would test whether the roughly 0.56-per-rebalance turnover remains profitable after more aggressive slippage.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes AlphaZeroBeta, a deep reinforcement learning framework for market-neutral portfolio construction. The agent uses a CNN-GRU encoder with Recurrent PPO and a composite reward (Eq. 8) combining risk-adjusted excess return, a benchmark-correlation penalty, and a turnover penalty. Dollar neutrality is enforced at every rebalance by centering the policy output and projecting onto the L1 ball of radius one (Section 3.5, Listing D.4.2). The method is evaluated in a 22-fold walk-forward protocol over 2014–2024 on seven equity indices, comparing against buy-and-hold, a maximum-Sharpe portfolio, and a minimum-correlation portfolio. The paper reports that AlphaZeroBeta achieves higher Sharpe ratios in all markets, near-zero correlations with the benchmarks, and competitive drawdowns, supported by factor regressions, an RL ablation, and feature-attribution diagnostics.

Significance. If the empirical claims hold, the paper contributes a scalable DRL framework for market-neutral portfolios, with useful engineering details: multi-resolution CNN-GRU encoding, a 2,200-asset universe, detailed pseudocode, a transparent transaction-cost/borrow-fee schedule, and an architectural ablation. The walk-forward design with 22 non-overlapping test windows and nine seeds is a strength, as are the factor-attribution results in Table 5 and the candor about the fixed SSE constituent snapshot. However, the central comparison in Table 4 is not yet established because the baselines are not compared under matched constraints, and the reported significance lacks proper accounting for the non-independence of the seed dimension.

major comments (3)
  1. [Section 4.3, Eqs. (10)–(11), Table 4] The claim that the comparison is 'leverage-neutral in terms of gross exposure' is incorrect. MxSharpe and Decorr impose only sum(w_i)=1 and -1<=w_i<=1, so their gross exposure can be as large as O(N) (e.g., about N-1 for N assets), while AlphaZeroBeta enforces sum(w_i)=0 and ||w||_1<=1 (Section 3.5, Listing D.4.2). Thus the baselines can take substantially larger long/short positions, and the higher Sharpe ratios in Table 4 may reflect looser gross-exposure limits rather than superior alpha. The paper must either report realized gross exposure for every method and demonstrate that the constraint asymmetry does not drive the results, or rerun the baselines with matched constraints (e.g., dollar-neutral and gross exposure <=1).
  2. [Table 4 and Section 6.1] The headline claim that AlphaZeroBeta 'achieves the highest Sharpe ratio in all markets' is based on point estimates with no formal significance test that accounts for the data structure. The reported dispersion for AlphaZeroBeta is computed over 198 samples, but these are not independent: nine seeds share the same 22 market paths, so the effective number of independent observations is at most 22. A paired test over the 22 folds, a bootstrap over folds, or a Deflated Sharpe Ratio (the authors cite Bailey et al. [7] but do not apply it) is needed to establish that the Sharpe gaps are not within sampling noise, particularly for the smaller gaps such as FTSE (0.94 vs. 0.37) and GDAXI (0.86 vs. 0.11).
  3. [Section 3.7/Eq. (8), Listing D.4.2, Section 6] The reported near-zero correlations in Table 4 are largely a structural consequence of the hard-coded centering and L1 projection, not an emergent property discovered by the agent. The paper should more sharply distinguish between neutrality that is enforced by construction and the residual beta that the correlation penalty actually influences. The sentence in Section 6 claiming that 'the soft correlation penalty in the reward suffices to neutralize residual beta without requiring a hard zero-correlation constraint' is misleading, since a hard dollar-neutral projection is already present. Table 5's factor betas are the appropriate evidence for residual neutrality and should be positioned as such; Table 4 correlations should be interpreted as a feasibility check on the constraint, not as a standalone achievement.
minor comments (5)
  1. [Section 6, paragraph on rebalancing] The text states that 'all four strategies rebalance at the same daily frequency,' but Index B&H is defined in Eq. (9) as holding the initial allocation unchanged, with costs only on membership changes. This inconsistency should be corrected to avoid confusion about cost comparability.
  2. [Appendix D.2.2, Table D5] The correlation penalty lambda_1 and turnover penalty lambda_2 are said to be chosen by manual optimization on pilot experiments. Since the same values are then used across all seven markets, there is a possibility of selection over markets. A brief sensitivity analysis around the chosen values would strengthen the claim that the results are not knife-edge.
  3. [Section 5.2.1] Typo: 'A WSp3.2xlarge' should read 'AWS p3.2xlarge'.
  4. [Figure 3] The caption reports R2 ≈ 0.97, while the text says the cross-seed mean is ≈ 0.95 with most runs above 0.97. These numbers should be reconciled.
  5. [Table 3 and Section 6.4] The fixed 2025-02-01 SSE constituent snapshot is acknowledged as a survivorship caveat; this is appropriate, but the factor attribution for 000001.SS in Table 5 inherits the same caveat and should be flagged in the table caption or the surrounding text.

Circularity Check

1 steps flagged · score 4.0 of 10

Near-zero beta is substantially hard-coded by the dollar-neutral projection and correlation penalty, so the reported low correlations are partly by construction rather than an independent discovery.

  1. self definitional [Section 3.5 / Listing D.4.2; Section 3.7 Eq. (8); Section 5.3.2 Eq. (13); Section 6 Table 4]
    "we subtract the cross-sectional mean of the action so that P_i wi = 0 before the l1 projection ... This combination enforces dollar neutrality at each rebalance ... Rt = rp(t)−rm(t)/σp(t) − λ1 · Corr(rp(t), rm(t)) − λ2 Σ_i |Δwi(t)|"

    The action space is forced to be dollar-neutral (sum of weights equals zero) and the reward function directly penalizes the correlation between portfolio and benchmark returns. The evaluation metric 'correlation with the benchmark' (Eq. 13) is thus minimized by construction during training and at every deployment step. Reporting near-zero correlations as evidence of learned 'non-trivial return structures' or 'strong orthogonality' (Section 6) presents a design constraint and a training objective as an empirical discovery. The result is a direct consequence of the chosen projection and reward, not an independent prediction.

full rationale

The paper's main claim—higher out-of-sample Sharpe ratios relative to convex baselines—is an empirical result that is not circular: the policy is trained on in-sample data and evaluated on non-overlapping out-of-sample windows, and the ablation with λ1 = 0 provides a meaningful contrast showing that the neutrality penalty reduces correlation and drawdowns. However, the 'near-zero beta/correlation' headline is substantially predetermined. The model's action space is projected to be dollar-neutral (Σ w = 0) and the reward function (Eq. 8) explicitly penalizes the same correlation metric that is later reported as a success (Eq. 13). The paper itself notes 'neutrality is hard-coded rather than emergent' (Section 4.3), yet Section 6 interprets the low correlations as confirming 'that our strategy learns non-trivial return structures'. This is a partial self-definitional circularity: the outcome is built into the constraints and objective. The additional issue that the convex baselines are not leverage-neutral (they can take gross exposure >1) is a fairness/validity concern about the comparison, not a circularity, so it does not enter the circularity score. Overall, the central Sharpe-ratio comparison retains independent content, giving a score of 4.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a long empirical pipeline: look-ahead-free data, a neutrality proxy, a biased but disclosed SSE universe, and assumed statistical significance. No new physical or theoretical entities are introduced.

free parameters (3)
  • lambda_1 (correlation penalty) = 0.5
    Chosen by manual optimization on pilot experiments (Table D5, Section D.2.2); it directly determines how much neutrality is forced, so the near-zero correlation result is sensitive to this choice.
  • lambda_2 (turnover penalty) = 0.001
    Also pilot-tuned; it controls trading frequency and post-cost net returns, and no sensitivity analysis is reported.
  • PPO/architecture hyperparameters = lr=3e-4, clip=0.20, entropy=0.01, hidden=512, agent_window=100, vol_window=60
    Hand-selected or taken from standard defaults (Table D5). The reported Sharpe and stability can depend on these choices, and no systematic sensitivity analysis is provided.
assumptions (5)
  • domain assumption Historical data from Bloomberg/FMP plus the stated alignment rules (60-day fundamental lag, announcement-date revisions, local calendars) are free of look-ahead bias.
    Central to walk-forward validity; Section 4.2 sets out the rules, but the actual vendor data cannot be audited from the paper.
  • domain assumption Dollar neutrality (sum of weights zero and gross exposure <= 1) is an adequate operationalization of market neutrality.
    Section 3.5 and Listing D.4.2. The factor regressions in Table 5 support near-zero market beta, but the constraint itself is only a proxy for true beta neutrality.
  • ad hoc to paper The fixed 2025-02-01 SSE constituent snapshot is an acceptable proxy for the historical SSE universe.
    Acknowledged in Section 4.2.4 and Table 3; this introduces survivorship bias into one of the seven headline universes.
  • domain assumption The absence of significance tests is acceptable because the walk-forward out-of-sample design and multiple seeds control overfitting.
    The paper assumes the mean Sharpe gaps in Table 4 are not sampling noise; no confidence intervals or hypothesis tests are provided.
  • domain assumption Locally constructed Fama-French-like factors for non-US markets are correctly time-aligned and specified.
    Section 6.4.1 describes the construction; the reported significant alphas depend on these factor definitions and alignment rules.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AlphaZeroBeta: Deep Reinforcement Learning for Market-Neutral Portfolios." pith.science (2026). https://pith.science/paper/C74EM6DL

@misc{pith2026260718001,
  author       = {Pith},
  title        = {Pith review of: AlphaZeroBeta: Deep Reinforcement Learning for Market-Neutral Portfolios},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C74EM6DL}},
  note         = {Machine review of arXiv:2607.18001}
}
read the original abstract

Market-neutral portfolios aim to generate consistent returns while offsetting systematic market risk. Traditional approaches based on factor models or convex optimization often underperform during market regime shifts or when structural assumptions break down. We propose AlphaZeroBeta, a deep reinforcement learning framework designed to deliver benchmark-relative alpha (excess returns) with near-zero beta (market neutrality). AlphaZeroBeta combines a composite reward function that balances risk-adjusted excess return, benchmark correlation, and transaction costs with a CNN-GRU policy trained end-to-end via Recurrent PPO and evaluated through a rolling walk-forward protocol. Backtests covering 2014-2024 across seven equity indices show that the model achieves higher Sharpe ratios than the baselines while maintaining near-zero benchmark correlations and competitive drawdowns.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

72 extracted references · 10 linked inside Pith

  1. [7]

    H., Borwein, J

    Bailey, D. H., Borwein, J. M., Lopez de Prado, M., & Zhu, Q. J. (2017). The probability of backtest overfitting.Journal of Computational Finance, 20(4), 39–69

    2017

  2. [1]

    Almahdi, S., & Yang, S. Y. (2017). An adaptive portfolio trading system: A risk–return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown.Expert Systems with Applications, 87, 267–279

    2017

  3. [2]

    Anderson, R., et al. (2024). Long-history PCA in a dynamic factor model with weak loadings.Berkeley Center for Risk Management Research Working Paper

    2024

  4. [3]

    S., Frazzini, A., & Pedersen, L

    Asness, C. S., Frazzini, A., & Pedersen, L. H. (2012). Leverage aversion and risk parity.Financial Analysts Journal, 68(1), 47–59. 54

    2012

  5. [4]

    S., Frazzini, A., & Pedersen, L

    Asness, C. S., Frazzini, A., & Pedersen, L. H. (2019). Quality minus junk.Review of Accounting Studies, 24(1), 34–112

    2019

  6. [5]

    Avellaneda, M., & Lee, J.-H. (2010). Statistical arbitrage in the US equities market. Quantitative Finance, 10(7), 761–782

    2010

  7. [6]

    Bai, Y., Gao, Y., Wan, R., Zhang, S., & Song, R. (2024). A review of reinforcement learning in financial applications.arXiv preprintarXiv:2411.12746

    arXiv 2024

  8. [8]

    Barigozzi, M., & Hallin, M. (2024). The dynamic, the static, and the weak: Factor models and the analysis of high-dimensional time series.arXiv preprint arXiv:2407.10653

    arXiv 2024

Show all 72 references
  1. [9]

    R., Boduroglu, I

    Baronyan, S. R., Boduroglu, I. I., & Sener, E. (2010). Investigation of stochastic pairs trading strategies under different volatility regimes.Manchester School, 78(1), 114–134

    2010

  2. [10]

    Black, F. (1972). Capital market equilibrium with restricted borrowing.Journal of Business, 45(3), 444–455

    1972

  3. [11]

    Blitz, D. (2025). Caveats of Simple Factor Timing Strategies.SSRN Scholarly Paper No. 5256938

    2025

  4. [12]

    Bloomberg L.P. (2025). Bloomberg Terminal financial data [Data set]. Retrieved February 1, 2025, from https://www.bloomberg.com/professional

    2025

  5. [13]

    B¨ uhler, H., Gonon, L., Teichmann, J., & Wood, B. (2019). Deep hedging. Quantitative Finance, 19(8), 1271–1291

    2019

  6. [14]

    Y., Lo, A

    Campbell, J. Y., Lo, A. W., & MacKinlay, A. C. (1997).The Econometrics of Financial Markets. Princeton University Press

    1997

  7. [15]

    Carhart, M. M. (1997). On persistence in mutual fund performance.Journal of Finance, 52(1), 57–82

    1997

  8. [16]

    Charpentier, A., ´Elie, R., & Remlinger, C. (2023). Reinforcement learning in economics and finance.Computational Economics, 62(1), 425–462

    2023

  9. [17]

    Chekhlov, A., Uryasev, S., & Zabarankin, M. (2005). Drawdown measure in port- folio optimization.International Journal of Theoretical and Applied Finance, 8(1), 13–58

    2005

  10. [18]

    Cho, K., Van Merri¨ enboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. InProceedings of the 2014 55 Conference on Empirical Methods in Natural La...

    2014

  11. [19]

    Choudhary, H., Orra, A., Sahoo, K., & Thakur, M. (2025). Risk-adjusted deep reinforcement learning for portfolio optimization: A multi-reward approach. International Journal of Computational Intelligence Systems, 18, Article 126

    2025

  12. [20]

    V., & Salakhutdinov, R

    Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019). Transformer-XL: Attentive language models beyond a fixed-length context. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2978–2988

    2019

  13. [21]

    Fama, E. F. (1970). Efficient capital markets: A review of theory and empirical work.Journal of Finance, 25(2), 383–417

    1970

  14. [22]

    F., & French, K

    Fama, E. F., & French, K. R. (1993). Common risk factors in the returns on stocks and bonds.Journal of Financial Economics, 33(1), 3–56

    1993

  15. [23]

    F., & French, K

    Fama, E. F., & French, K. R. (2015). A five-factor asset pricing model.Journal of Financial Economics, 116(1), 1–22

    2015

  16. [24]

    Feghali, G., & Villalon, D. (2015). Building a better equity market neutral strategy.AQR White Paper, AQR Capital Management, LLC

    2015

  17. [25]

    Financial Modeling Prep. (2025). Financial Modeling Prep API: aggre- gated financial data [Data set]. Retrieved February 1, 2025, from https://financialmodelingprep.com

    2025

  18. [26]

    Fischer, T. G. (2018). Reinforcement learning in financial markets: A survey.F AU Discussion Papers in Economics, No. 12/2018

    2018

  19. [27]

    Ganesan, G. (2011). A subspace approach to portfolio analysis: A focus on equity investments.IEEE Signal Processing Magazine, 28(5), 49–60

    2011

  20. [28]

    Gaˇ sperov, B., Beguˇ si´ c, S.,ˇSimovi´ c, P., & Kostanjˇ car, Z. (2021). Reinforcement learning approaches to optimal market making.Mathematics, 9(21), 2689

    2021

  21. [29]

    N., & Rouwenhorst, K

    Gatev, E., Goetzmann, W. N., & Rouwenhorst, K. G. (2006). Pairs trading: Performance of a relative-value arbitrage rule.Review of Financial Studies, 19(3), 797–827

    2006

  22. [30]

    C., & Kahn, R

    Grinold, R. C., & Kahn, R. N. (2000).Active Portfolio Management. McGraw-Hill

    2000

  23. [31]

    M., Xu, R., & Yang, H

    Hambly, B. M., Xu, R., & Yang, H. (2023). Recent advances in reinforcement learning in finance.Mathematical Finance, 33(3), 437–503. 56

    2023

  24. [32]

    R., & Liu, Y

    Harvey, C. R., & Liu, Y. (2015). Backtesting.Journal of Portfolio Management, 42(1), 13–28

    2015

  25. [33]

    Stockholm: Nordic Business Media AB (HedgeNordic Special Report)

    HedgeNordic (2015).Market Neutral Strategies: The Key to Alpha in any Market Direction. Stockholm: Nordic Business Media AB (HedgeNordic Special Report)

    2015

  26. [34]

    Huang, G., Zhou, X., & Song, Q. (2020). Deep reinforcement learning for long- short portfolio optimization.arXiv preprintarXiv:2012.13773

    2020 arXiv

  27. [35]

    Jegadeesh, N. (1990). Evidence of predictable behavior of security returns. Journal of Finance, 45(3), 881–898

    1990

  28. [36]

    Jegadeesh, N., & Titman, S. (1993). Returns to buying winners and selling losers: Implications for stock market efficiency.Journal of Finance, 48(1), 65–91

    1993

  29. [37]

    Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem.arXiv preprintarXiv:1706.10059

    2017 arXiv

  30. [38]

    E., & Lo, A

    Khandani, A. E., & Lo, A. W. (2007). What happened to the quants in August 2007?Journal of Investment Management, 5(4), 5–54

    2007

  31. [39]

    E., & Lo, A

    Khandani, A. E., & Lo, A. W. (2011). What happened to the quants in August 2007? Evidence from factors and transactions data.Journal of Financial Markets, 14(1), 1–46

    2011

  32. [40]

    Koratamaddi, P., Wadhwani, K., Gupta, M., & Sanjeevi, S. G. (2021). Market sentiment-aware deep reinforcement learning approach for stock portfolio allocation. Engineering Science and Technology, an International Journal, 24(4), 848–859

    2021

  33. [41]

    A., & Huck, N

    Krauss, C., Do, X. A., & Huck, N. (2017). Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500.European Journal of Operational Research, 259(2), 689–702

    2017

  34. [42]

    Krogh, A., & Hertz, J. A. (1992). A simple weight decay can improve generaliza- tion. InAdvances in Neural Information Processing Systems,4, 950–957

    1992

  35. [43]

    Li, C.-L., & Hsieh, C.-H. (2023). On unified adaptive Black-Litterman mean- variance portfolio management.arXiv preprintarXiv:2307.03391

    2023 arXiv

  36. [44]

    Liang, Z., Chen, H., Zhu, J., Jiang, K., & Li, Y. (2018). Adversarial deep reinforcement learning in portfolio management.arXiv preprintarXiv:1808.09940

    2018 arXiv

  37. [45]

    Lim, E. Q. Y., Cao, Q., & Quek, C. (2022). Dynamic portfolio rebalancing through reinforcement learning.Neural Computing and Applications, 34(9), 7125–7139

    2022

  38. [46]

    Lo, A. W. (2002). The statistics of Sharpe ratios.Financial Analysts Journal, 58(4), 36–52. 57

    2002

  39. [47]

    Markowitz, H. (1952). Portfolio selection.Journal of Finance, 7(1), 77–91

    1952

  40. [48]

    Mnih, V., et al. (2016). Asynchronous methods for deep reinforcement learning. InProceedings of the 33rd International Conference on Machine Learning (ICML), 1928–1937

    2016

  41. [49]

    Moody, J., & Saffell, M. (2001). Learning to trade via direct reinforcement.IEEE Transactions on Neural Networks, 12(4), 875–889

    2001

  42. [50]

    Orra, A., Choudhary, H., Sharma, A., & Thakur, M. (2025). Enhancing deep reinforcement learning for stock trading: A reward shaping approach via expert feedback.Knowledge and Information Systems, 67(11), 11075–11094

    2025

  43. [51]

    Parisotto, E., et al. (2020). Stabilizing transformers for reinforcement learning. InProceedings of the 37th International Conference on Machine Learning, PMLR 119, 7487–7498

    2020

  44. [52]

    market neutral

    Patton, A. J. (2009). Are “market neutral” hedge funds really market neutral? Review of Financial Studies, 22(7), 2295–2330

    2009

  45. [53]

    F., & Sharpe, W

    Perold, A. F., & Sharpe, W. F. (1988). Dynamic strategies for asset allocation. Financial Analysts Journal, 44(1), 16–27

    1988

  46. [54]

    Pleines, M., Pallasch, M., Zimmer, F., & Preuss, M. (2022). Generalization, mayhems and limits in recurrent proximal policy optimization.arXiv preprint arXiv:2205.11104

    2022 arXiv

  47. [55]

    Pleines, M., Pallasch, M., Zimmer, F., & Preuss, M. (2025). Memory Gym: Towards endless tasks to benchmark memory capabilities of agents.Journal of Machine Learning Research, 26(6), 1–40

    2025

  48. [56]

    Prechelt, L. (1998). Early stopping — but when? In G. B. Orr & K. R. M¨ uller (Eds.),Neural Networks: Tricks of the Trade(pp. 55–69). Springer

    1998

  49. [57]

    Pricope, T.-V. (2021). Deep reinforcement learning in quantitative algorithmic trading: A review.arXiv preprintarXiv:2106.00123

    2021 arXiv

  50. [58]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms.arXiv preprintarXiv:1707.06347

    2017 arXiv

  51. [59]

    Sharpe, W. F. (1966). Mutual fund performance.Journal of Business, 39(1), 119–138

    1966

  52. [60]

    Sharpe, W. F. (1994). The Sharpe ratio.Journal of Portfolio Management, 21(1), 49–58

    1994

  53. [61]

    Shiller, R. J. (1981). Do stock prices move too much to be justified by subsequent changes in dividends?American Economic Review, 71(3), 421–436. 58

    1981

  54. [62]

    Spooner, T., Fearnley, J., Savani, R., & Koukorinis, A. (2018). Market making via reinforcement learning. InProceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 434–442

    2018

  55. [63]

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting.Journal of Machine Learning Research, 15(56), 1929–1958

    2014

  56. [64]

    Stable-Baselines Team. (2025). sb3-contrib: RecurrentPPO (Version 2.7.0) [Com- puter software]. Retrieved February 1, 2025, from https://pypi.org/project/sb3- contrib/

    2025

  57. [65]

    S., & Barto, A

    Sutton, R. S., & Barto, A. G. (2018).Reinforcement Learning: An Introduction (2nd ed.). MIT Press

    2018

  58. [66]

    A., Meade, N., & Beasley, J

    Valle, C. A., Meade, N., & Beasley, J. E. (2014). Market neutral portfolios. Optimization Letters, 8(7), 1961–1984

    2014

  59. [67]

    C.-W., & Ho, J.-M

    Wu, M.-E., Syu, J.-H., Lin, J. C.-W., & Ho, J.-M. (2021). Portfolio management system in equity market neutral using reinforcement learning.Applied Intelligence, 51(11), 8119–8131

    2021

  60. [68]

    Yang, H., Liu, X.-Y., Zhong, S., & Walid, A. (2020). Deep reinforcement learning for automated stock trading: An ensemble strategy. InProceedings of the 1st ACM International Conference on AI in Finance (ICAIF ’20), 1–8

    2020

  61. [69]

    Ye, Y., Pei, H., Wang, B., Chen, P.-Y., Zhu, Y., Xiao, J., & Li, B. (2020). Reinforcement-learning based portfolio management with augmented asset move- ment prediction states. InProceedings of the 34th AAAI Conference on Artificial Intelligence, 1112–1119

    2020

  62. [70]

    Ye, J., et al. (2024). From factor models to deep learning: machine learning in reshaping empirical asset pricing.arXiv preprintarXiv:2403.06779

    2024 arXiv

  63. [71]

    Zhang, Z., Zohren, S., & Roberts, S. (2020). Deep learning for portfolio optimiza- tion.Journal of Financial Data Science, 2(4), 8–20

    2020

  64. [72]

    Zhang, Z., Zohren, S., & Roberts, S. (2020). Deep reinforcement learning for trading.Journal of Financial Data Science, 2(2), 25–40. 59

    2020

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.