Pith. sign in

REVIEW 3 major objections 5 minor 58 references

Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Information

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DERL, an end-to-end reinforcement-learning framework with dynamically updated market embeddings, outperforms predict-then-optimize and index benchmarks in a 30-year backtest of top-500 U.S. stocks, especially in high-volatility periods.

desk verdict A genuinely novel DERL framework with a clean rolling-window design, but the headline Sharpe depends on an unverified turnover/cost assumption and undisclosed hyperparameter choices. read the letter →

arxiv 2501.17992 v1 pith:5F4OKTST submitted 2025-01-29 q-fin.PM cs.LG

Jinghai He , Cheng Hua , Chunyang Zhou , Zeyu Zheng This is my paper · ORCID
classification q-fin.PMcs.LG
keywords portfolioallocationreinforcementlearninggenerativeautoencoderonlinemeta-learningdynamicembeddingvolatilitytimingtransactioncostsSharperatio
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an end-to-end reinforcement-learning agent can solve high-dimensional, non-stationary portfolio allocation when the raw market state is first compressed into a transition-aware embedding. It claims that this Dynamic Embedding Reinforcement Learning (DERL) framework beats a machine-learning predict-then-optimize strategy and value- and equal-weighted benchmarks in a 1993-2022 out-of-sample test on the top 500 U.S. stocks, with the largest edge in high-VIX periods. If the result holds, a long-only, no-leverage daily-rebalanced portfolio can earn roughly 7.5% per year in risk-adjusted return that standard factor models do not explain, and the mechanism is a learned tendency to cut market exposure when volatility rises.

What carries the argument

The key machinery is the dynamic embedding: a Wasserstein autoencoder whose encoder $\Gamma_\phi(z_s|s)$ maps a raw state to a low-dimensional latent variable $z_s$, trained so that the decoder $G_\theta(\hat{s}'|z_s,a)$ reconstructs the next state from the latent state and action, thereby encoding the transition $P(s'|s,a)$. Online meta-learning modifies the encoder every 42 days with a new-data update plus a squared-error penalty pulling it toward the base parameters, so the embedding keeps track of changing market dynamics. A TD3 actor-critic agent then learns a policy $\pi(a|z_s)$ on the embedded states, with the Sharpe ratio as reward and no shorting or leverage.

What would settle it

Compute the daily turnover implied by DERL's weight series and re-run the backtest at 0.2% and 0.5% per-side transaction costs including a market-impact proxy; if the Sharpe ratio and alpha collapse, the reported edge is a transaction-cost artifact rather than a portfolio-selection effect.

Watch

Extended reading notes

Core claim

The central discovery is that compressing the state before learning the policy is what makes RL work on noisy market data. DERL projects the 15,506-dimensional price-volume/technical state to a 500-dimensional latent variable with a generative autoencoder trained to reconstruct the next day's state from the latent state and the chosen action; the encoder is then updated every 42 days by online meta-learning. On out-of-sample daily rebalancing of the top 500 stocks from 1993 to 2022, the resulting portfolio has an annualized Sharpe ratio of 1.04 and Sortino ratio of 1.62 net of 0.1% transaction costs, versus 0.53-0.69 for the alternative strategies, with significant positive skewness and a market beta near 0.64. Fama-French three- and four-factor regressions leave a daily alpha above 0.03%, about 7.5% annualized, and the portfolio lowers its market beta from about 0.75 in low-VIX periods to about 0.62 in high-VIX periods, which the paper reads as volatility timing.

Load-bearing premise

The net-return figures assume transaction costs of 0.1% of total trade value on every daily rebalance, and the paper reports no turnover; if actual round-trip costs or market impact on 500 large-cap names are higher, the 7.5% annualized alpha could shrink or disappear.

Editorial extensions

If this is right

  • Over the full 1993-2022 backtest, DERL reports an annualized Sharpe ratio of 1.04 and Sortino ratio of 1.62 net of 0.1% costs, beating the two-step PTO, value-weighted and equal-weighted benchmarks.
  • The outperformance is concentrated in high-volatility regimes: when VIX is above its median, DERL keeps a positive Sharpe ratio while the benchmarks go negative.
  • Standard Fama-French three- and four-factor models leave a daily alpha above 0.03%, so the returns are not explained by size, value, market or momentum exposure.
  • Ablation results attribute the edge to the components: removing both embedding and meta-learning drops the Sharpe ratio from 1.04 to about 0.50, and next-state embeddings beat current-state embeddings.
  • Lasso analysis of the generated weights shows the agent follows a 7-day short-term reversal and 14-day momentum pattern and tilts toward low-beta stocks, with less market exposure in stressed periods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication of the paper's volatility-timing story is that adding a VIX-regime feature to a simpler rule-based strategy, such as scaling market exposure inversely to VIX, should capture part of DERL's high-volatility edge; the paper does not report that comparison.
  • Because the paper does not report turnover, the 0.1% cost assumption is unverified; an out-of-sample replication that measures daily turnover and applies institutional large-cap cost estimates including market impact is the cleanest way to see whether the 7.5% alpha is tradable.
  • The same encoder-decoder reconstruction trick could be transferred to other high-dimensional sequential allocation problems, such as intraday execution or multi-asset allocation, but the paper's evidence is limited to daily top-500 U.S. equity rebalancing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DERL, a portfolio allocation framework that combines a Wasserstein autoencoder for state embedding, fully online meta-learning for dynamic encoder updates, and the TD3 reinforcement learning algorithm to output daily portfolio weights. Using top-500 U.S. stocks over 1993–2022 with rolling 42-day validation windows, the authors report that DERL achieves higher out-of-sample Sharpe and Sortino ratios than a two-step predict-then-optimize MLP, value-weighted, and equal-weighted portfolios, with daily factor-model alphas above 0.03% and stronger performance in high-VIX periods. Ablation studies attribute the gains to the embedding and meta-learning components, and lasso regressions interpret the policy as short-term reversal, 14-day momentum, low-beta tilts, and volatility timing.

Significance. If the empirical claims are robust, the paper makes a useful contribution to the RL-based portfolio allocation literature by demonstrating an end-to-end framework with dynamic embeddings and online meta-learning. The design is attractive: it inputs only price–volume and technical indicators, avoids shorting and leverage, and uses a forward-looking validation protocol in which embeddings and RL parameters are updated only on data before each validation window. The ablation study across TD3, DDPG, and A2C, the factor regressions with Newey–West statistics, and the lasso-based interpretation of the learned policy are strengths. The paper's central quantitative claim, however, depends on an unverified transaction-cost assumption and on the reproducibility of a number of undisclosed hyperparameters, so the current evidence is not yet load-bearing for the headline Sharpe ratios and alphas.

major comments (3)
  1. [§3.1.2, Tables 1–2] The net-of-cost performance claims are not robustly established because the paper fixes the transaction cost rate at 0.1% of total trade value but never reports portfolio turnover, gross returns, or a cost-drag decomposition. For a daily-rebalanced long-only portfolio of 500 stocks with softmax action weights, daily turnover can easily be tens of percent; since the headline daily alpha in Table 2 is 0.0003 (0.03%), a one-way daily turnover of roughly 30% would consume the entire alpha at a 0.1% cost, and the breakeven turnover drops to about 15% if the realistic one-way cost is 0.2%. The reward function includes transaction costs, so the agent may have learned a low-turnover policy, but without turnover statistics or a sensitivity analysis over cost rates, the out-of-sample Sharpe ratios in Table 1 and the factor alphas in Table 2 cannot be independently verified. Please report gross and net annualized returns, the distribution of daily one-way turnover, and results under at least 0.05%, 0.1%, 0.2%, and 0.5% cost rates.
  2. [§3.2, Algorithm 1, Eq. (2)] Several hyperparameters that materially affect the reported results are either undisclosed or selected without a described protocol. The reward lookahead k in Eq. (2) is never specified; the TD3 exploration noise variance σ^2 and the target-policy smoothing clipping level c in Algorithm 1 are not given; and the kernel scale d_z in the WAE MMD loss is not assigned a numeric value (the displayed formula uses d_z^2 but the text does not say whether d_z equals the embedding dimension or is a separate tuning parameter). In addition, the embedding dimension dim(Z)=500 is said to have been chosen after testing values from 50 to 2000, but the selection criterion (validation performance? in-sample reconstruction?) is not described. If the final test period was used in any way to select these hyperparameters, the out-of-sample results would be optimistic. Please provide a complete table of hyperparameters, including k, σ^2, c, the kernel scale, and the embedding-selection protocol, and state explicitly which data were used for any model selection.
  3. [§3.1.2, Figure 4] The rolling-window training schedule is not fully specified. The text states that the first training window starts on January 1 of 1990, 1995, 2000, 2005, 2010, and 2015, and that each validation window is 42 days, but it does not define the length of the training window that precedes each validation window, nor how trs_j and tre_j relate to the validation windows. This makes it impossible to tell whether the RL agent is retrained on a rolling 42-day window, on the entire history since the segment start, or on some other interval, and it also affects the interpretation of the online meta-learning updates. Please define the exact calendar schedule for training and validation windows, including the length of each training window and the frequency of RL parameter updates.
minor comments (5)
  1. [Throughout] There are several typos and formatting issues, including 'F ormulation' in the §2.1.1 heading, 'disccusions' near the end of the introduction, 'Factor analyisis' in the Table 2 title, and 'VIX¡17.91' in Panel C1 of Table 1 (should be '<').
  2. [§3.1] The raw state dimension is stated to be 15,506 for the top-500 universe, but the derivation (31 indicators per stock plus two market variables) is not shown; please clarify the count of technical indicators per stock and any additional state components.
  3. [§3.5, Eq. (18)] The text says that the meta-learning contribution is 'expected to be insignificant for the first 42 days of each training segment' and reports a t-statistic of -0.22, but the sentence appears to contradict the fact that meta-learning updates occur every 42 days; please clarify whether the first 42 days after each update or the first 42 days of each segment is intended, and how the subsample t-statistic of 2.04 was computed.
  4. [References] The reference list includes several works cited in the E-Companion (e.g., Goodfellow et al. 2014, Arjovsky et al. 2017) but not all are cited in the main text; please ensure consistent citation formatting between the main text and the E-Companion.
  5. [§EC.1.2] The statement that 'the target network is updated every five trading days' in the main text conflicts with the delayed-policy-update frequency d in Algorithm 1, whose value is not given; please specify d explicitly and reconcile with the target-network update schedule.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported out-of-sample Sharpe is partly a training target because the reward horizon in Eq. (2) crosses the training/validation boundary; hyperparameter and transaction-cost concerns are secondary and non-circular.

  1. fitted input called prediction [Section 2.1.1, Eq. (2); Section 3.1.2 rolling-window backtesting and Figure 4 note]
    "rt = μt/σt, where μt = 1/k Σ_{i=t}^{t+k−1} R_i and σt = sqrt(1/(k−1) Σ_{i=t}^{t+k−1} (R_i − μt)^2) are the mean and standard deviation of the realized portfolio return in the following k days after taking action at ... After updating the encoder Γϕj and the RL agent, we conduct backtesting in the jth validation window."

    The reward for a training action at day t uses portfolio returns over days t+1 through t+k. The rolling design places the validation window immediately after the training window (Figure 4 note: valsi = trei). For actions in the last k days of a training window, the 'following k days' are the first k days of the validation window. Those same validation-window returns are later used to compute the reported out-of-sample Sharpe, Sortino, and factor alphas in Tables 1-2. The paper never states that boundary rewards are truncated or that k is capped at the remaining training days. Thus the 'out-of-sample' evaluation is not a clean forward test: the validation returns enter the RL training objective through Eq. (2), so the reported performance is partly fitted by construction.

full rationale

The central derivation chain is otherwise self-contained: the DERL policy is updated on data before each 42-day validation window, the factor alphas are regression statistics rather than constrained predictions, and the ablation comparisons are genuine forward contrasts. The load-bearing circular step is the reward-horizon boundary overlap: because r_t is defined over the following k days and each validation window abuts the training window, the first k days of every validation window are used in the training rewards of the preceding training actions, so the reported Sharpe ratios are not strictly out-of-sample. The transaction-cost assumption (0.1% per trade) and the unreported turnover are separate external premises that affect robustness but are not circular. The hyperparameter choice of embedding size 500 is reported as robust across sizes 300-600, so it is not a fitted-input-called-prediction. No self-citation is load-bearing; references to Cong et al. (2020), Gu et al. (2020), and Rajasegaran et al. (2022) are external prior work.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claim is empirical, so the main cost elements are hyperparameters chosen without a fully disclosed protocol (especially embedding size 500, which was selected after a 50-2000 search), and economic assumptions (0.1% transaction cost, top-500 universe, Sharpe utility). No new physical entities are introduced.

free parameters (8)
  • Embedding dimension dim(Z) = 500
    Chosen after testing sizes 50 to 2000 (Section 3.2). Selection protocol not disclosed, so the 30-year backtest may have influenced this choice.
  • FOML learning rates alpha1, beta1, alpha2, beta2 = 0.0001, 0.001, 0.0005, 0.005
    Set in Section 3.2; no sensitivity analysis reported.
  • TD3 learning rate, soft-update tau, discount gamma = 0.0002, 0.005, 0.999
    Set in Section 3.2; standard choices but still tuned for this problem.
  • WAE regularization lambda and MMD kernel scale d_z = lambda=2; d_z unspecified
    lambda set in Section 3.2; the inverse multiquadratics kernel in Section 2.4.1 contains d_z, but its numeric value is not given.
  • Validation/update window length |U| = 42 days
    Section 3.1.2; determines how often embeddings and the RL agent are updated.
  • Transaction cost rate = 0.1% per trade
    Section 3.1.2; a key economic assumption for net returns, with no turnover reporting.
  • Reward lookahead k in Eq. (2) = unspecified
    The Sharpe ratio reward uses the next k daily returns, but k is never defined in the paper.
  • TD3 action noise variance sigma^2 and clipping level c = unspecified
    Algorithm 1 references these, but the values are not reported.
assumptions (6)
  • domain assumption The financial market is a Markov decision process with transition P(s'|s,a) that can be approximated by a generative encoder-decoder decomposition (Eq. 8).
    Section 2.2.2; the whole embedding approach rests on this decomposition without a formal guarantee.
  • standard math Bellman optimality and existence of an optimal stationary policy.
    EC.1.1, Proposition EC.1; standard RL theory.
  • domain assumption The Sharpe ratio over a fixed lookahead window is the right utility for an investor.
    Section 2.1.1; used as reward, but it ignores path-dependent preferences and is measured with a short window.
  • domain assumption Fama-French three- and four-factor models are the correct null models for explaining returns.
    Table 2; alpha is computed against these factors only, not against more flexible factor models or other risk adjustments.
  • domain assumption CRSP returns, VIX, and the top-500 universe are free of survivorship and data errors.
    Section 3.1; the 30-year backtest depends on the integrity of the data.
  • domain assumption Top-500 stocks at the start of each 5-year period are known and tradeable.
    Section 3.1.1; reconstitution at the end of the prior year avoids lookahead.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Information." pith.science (2026). https://pith.science/paper/5F4OKTST

@misc{pith2026250117992,
  author       = {Pith},
  title        = {Pith review of: Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5F4OKTST}},
  note         = {Machine review of arXiv:2501.17992}
}
read the original abstract

We develop a portfolio allocation framework that leverages deep learning techniques to address challenges arising from high-dimensional, non-stationary, and low-signal-to-noise market information. Our approach includes a dynamic embedding method that reduces the non-stationary, high-dimensional state space into a lower-dimensional representation. We design a reinforcement learning (RL) framework that integrates generative autoencoders and online meta-learning to dynamically embed market information, enabling the RL agent to focus on the most impactful parts of the state space for portfolio allocation decisions. Empirical analysis based on the top 500 U.S. stocks demonstrates that our framework outperforms common portfolio benchmarks and the predict-then-optimize (PTO) approach using machine learning, particularly during periods of market stress. Traditional factor models do not fully explain this superior performance. The framework's ability to time volatility reduces its market exposure during turbulent times. Ablation studies confirm the robustness of this performance across various reinforcement learning algorithms. Additionally, the embedding and meta-learning techniques effectively manage the complexities of high-dimensional, noisy, and non-stationary financial data, enhancing both portfolio performance and risk management.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 50 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "" FUNCTION format.date year ...

  3. [3]

    Journal of Financial Markets 5(1):31--56

    Amihud Y (2002) Illiquidity and stock returns: Cross-section and time-series effects. Journal of Financial Markets 5(1):31--56

    work page 2002

  4. [4]

    Anderer A, Bastani H, Silberholz J (2022) Adaptive clinical trial designs with surrogates: When should we bother? Management Science 68(3):1982--2002

    work page 2022

  5. [5]

    Review of Financial Studies 32(7):2890--2919

    Ao M, Li Y, Zheng X (2019) Approaching mean-variance efficiency for large portfolios. Review of Financial Studies 32(7):2890--2919

    work page 2019

  6. [6]

    Proceedings of the 34th International Conference on Machine Learning - Volume 70, 214–223, ICML'17 (JMLR.org)

    Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning - Volume 70, 214–223, ICML'17 (JMLR.org)

    work page 2017

  7. [7]

    economic restrictions: Evidence from stock return predictability

    Avramov D, Cheng S, Metzker L (2023) Machine learning vs. economic restrictions: Evidence from stock return predictability. Management Science 69(5):2587--2619

    work page 2023

  8. [8]

    Management Science 64(3):1136--1154

    Ban GY, El Karoui N, Lim AE (2018) Machine learning and portfolio optimization. Management Science 64(3):1136--1154

    work page 2018

Show all 58 references
  1. [9]

    Management Science 67(5):2964--2984

    Bastani H (2021) Predicting with proxies: Transfer learning in high dimension. Management Science 67(5):2964--2984

    2021

  2. [10]

    Management Science 68(3):1865--1881

    Bastani H, Simchi-Levi D, Zhu R (2022) Meta dynamic pricing: Transfer learning across experiments. Management Science 68(3):1865--1881

    2022

  3. [11]

    Available at SSRN 4344837

    Bryzgalova S, DeMiguel V, Li S, Pelger M (2023) Asset-pricing factors with economic targets. Available at SSRN 4344837

    2023

  4. [12]

    The Review of Economic Studies 60(1):1--34, ISSN 00346527, 1467937X, ://www.jstor.org/stable/2297810

    Campbell JY, Kyle AS (1993) Smart money, noise trading and stock price behaviour. The Review of Economic Studies 60(1):1--34, ISSN 00346527, 1467937X, ://www.jstor.org/stable/2297810

    1993

  5. [13]

    Journal of Finance 52(1):57--82

    Carhart MM (1997) On persistence in mutual fund performance. Journal of Finance 52(1):57--82

    1997

  6. [14]

    Management Science 70(2):714--750

    Chen L, Pelger M, Zhu J (2023) Deep learning in asset pricing. Management Science 70(2):714--750

    2023

  7. [15]

    Interpretable, and Data-driven Approach to Analyzing Unstructured Information (September 1, 2019)

    Cong LW, Liang T, Zhang X (2019) Textual factors: A scalable, interpretable, and data-driven approach to analyzing unstructured information. Interpretable, and Data-driven Approach to Analyzing Unstructured Information (September 1, 2019)

    2019

  8. [16]

    SSRN Electronic Journal https://www.semanticscholar.org/paper/AlphaPortfolio\

    Cong LW, Tang K, Wang J, Zhang Y (2020) Alphaportfolio: Direct construction through deep reinforcement learning and interpretable ai. SSRN Electronic Journal https://www.semanticscholar.org/paper/AlphaPortfolio\

    2020

  9. [17]

    The Journal of Financial Data Science 3(1):28--42

    Cong LW, Tang K, Wang J, Zhang Y (2021) Deep sequence modeling: Development and applications in asset pricing. The Journal of Financial Data Science 3(1):28--42

    2021

  10. [18]

    Review of Financial Studies 33(5):2180--2222

    DeMiguel V, Martin-Utrera A, Nogales FJ, Uppal R (2020) A transaction-cost perspective on the multitude of firm characteristics. Review of Financial Studies 33(5):2180--2222

    2020

  11. [19]

    Journal of Financial and Quantitative Analysis 48(6):1813--1845

    DeMiguel V, Plyakha Y, Uppal R, Vilkov G (2013) Improving portfolio selection using option-implied volatility and skewness. Journal of Financial and Quantitative Analysis 48(6):1813--1845

    2013

  12. [20]

    Available at SSRN

    Duan J, Pelger M, Xiong R (2022) Target pca: Transfer learning large dimensional panel data. Available at SSRN

    2022

  13. [21]

    Journal of Business Finance and Accounting 22(1-2):1--18

    Fama EF (1965) Investigations of nonstationarity in prices. Journal of Business Finance and Accounting 22(1-2):1--18

    1965

  14. [22]

    Journal of political Economy 96(2):246--273

    Fama EF, French KR (1988) Permanent and temporary components of stock prices. Journal of political Economy 96(2):246--273

    1988

  15. [23]

    Journal of Financial Economics 33(1):3--56, ISSN 0304-405X, ://dx.doi.org/https://doi.org/10.1016/0304-405X(93)90023-5

    Fama EF, French KR (1993) Common risk factors in the returns on stocks and bonds. Journal of Financial Economics 33(1):3--56, ISSN 0304-405X, ://dx.doi.org/https://doi.org/10.1016/0304-405X(93)90023-5

    1993 doi

  16. [24]

    Journal of Financial Economics 116:1--22

    Fama EF, French KR (2015) A five-factor asset pricing model. Journal of Financial Economics 116:1--22

    2015

  17. [25]

    Review of Financial Studies 33(5):2326--2377

    Freyberger J, Neuhierl A, Weber M (2020) Dissecting characteristics nonparametrically. Review of Financial Studies 33(5):2326--2377

    2020

  18. [26]

    International conference on machine learning, 1587--1596 (PMLR)

    Fujimoto S, Hoof H, Meger D (2018) Addressing function approximation error in actor-critic methods. International conference on machine learning, 1587--1596 (PMLR)

    2018

  19. [27]

    Advances in neural information processing systems 27

    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Advances in neural information processing systems 27

    2014

  20. [28]

    INFORMS Journal on Computing 21(2):178--192

    Gosavi A (2009) Reinforcement learning: A tutorial survey and recent advances. INFORMS Journal on Computing 21(2):178--192

    2009

  21. [29]

    The Review of Financial Studies 33(5):2223--2273

    Gu S, Kelly B, Xiu D (2020) Empirical asset pricing via machine learning. The Review of Financial Studies 33(5):2223--2273

    2020

  22. [30]

    Journal of Econometrics 222(1):429--450

    Gu S, Kelly B, Xiu D (2021) Autoencoder asset pricing models. Journal of Econometrics 222(1):429--450

    2021

  23. [31]

    arXiv preprint arXiv:2112.04553

    Hambly B, Xu R, Yang H (2021) Recent advances in reinforcement learning in finance. arXiv preprint arXiv:2112.04553

    2021 arXiv

  24. [32]

    Review of Finance 25(1):1--41

    Hou K, Mo H, Xue C, Zhang L (2021) An augmented q-factor model with expected growth. Review of Finance 25(1):1--41

    2021

  25. [33]

    Management Science

    Huang JZ, Shi Z (2022) Machine-learning-based return predictors and the spanning controversy in macro-finance. Management Science

    2022

  26. [34]

    The Journal of Finance 78(6):3193--3249

    Jiang J, Kelly B, Xiu D (2023) ( R e-) I mag (in) ing price trends. The Journal of Finance 78(6):3193--3249

    2023

  27. [35]

    Journal of Financial Economics 134(3):501--524

    Kelly BT, Pruitt S, Su Y (2019) Characteristics are covariances: A unified model of risk and return. Journal of Financial Economics 134(3):501--524

    2019

  28. [36]

    SSRN working paper

    Kelly BT, Xiu D (2023) Financial machine learning. SSRN working paper

    2023

  29. [37]

    2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings

    Kingma DP, Welling M (2014) Auto-Encoding Variational Bayes . 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings

    2014

  30. [38]

    arXiv preprint arXiv:1509.02971

    Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971

    2015 arXiv

  31. [39]

    Marketing Science

    Liu X (2022) Dynamic coupon targeting using batch deep reinforcement learning: An application to livestream shopping. Marketing Science

    2022

  32. [40]

    Machine Learning - Springer Nature

    Liu XY, Xia Z, Yang H, Gao J, Zha D, Zhu M, Wang CD, Wang Z, Guo J (2024) Dynamic datasets and market environments for financial reinforcement learning. Machine Learning - Springer Nature

    2024

  33. [41]

    ACM International Conference on AI in Finance (ICAIF)

    Liu XY, Yang H, Gao J, Wang CD (2021) FinRL : Deep reinforcement learning framework to automate trading in quantitative finance. ACM International Conference on AI in Finance (ICAIF)

    2021

  34. [42]

    The Journal of Finance 7(1):77--91, ://dx.doi.org/https://doi.org/10.1111/j.1540-6261.1952.tb01525.x

    Markowitz H (1952) Portfolio selection*. The Journal of Finance 7(1):77--91, ://dx.doi.org/https://doi.org/10.1111/j.1540-6261.1952.tb01525.x

    1952

  35. [43]

    International conference on machine learning, 2391--2400 (PMLR)

    Mescheder L, Nowozin S, Geiger A (2017) Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks. International conference on machine learning, 2391--2400 (PMLR)

    2017

  36. [44]

    Michaud RO (1989) The markowitz optimization enigma: Is 'optimized' optimal? Financial Analysts Journal 45(1):31--42

    1989

  37. [45]

    International conference on machine learning, 1928--1937 (PMLR)

    Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. International conference on machine learning, 1928--1937 (PMLR)

    2016

  38. [46]

    Econometrica 79(4):973--1026

    Park A, Sabourian H (2011) Herding and contrarian behavior in financial markets. Econometrica 79(4):973--1026

    2011

  39. [47]

    Operations Research 70(6):3601--3628

    Qu G, Wierman A, Li N (2022) Scalable reinforcement learning for multiagent networked systems. Operations Research 70(6):3601--3628

    2022

  40. [48]

    ://openreview.net/forum?id=THMafOyRVpE

    Rajasegaran J, Finn C, Levine S (2022) Fully online meta-learning without task boundaries. ://openreview.net/forum?id=THMafOyRVpE

    2022

  41. [49]

    Public finance quarterly 65:168--195, ://dx.doi.org/10.35551/PFQ_2020_2_2

    Salahuddin S, Kashif M, Rehman M (2020) Time varying stock market integration and diversification opportunities within emerging and frontier markets. Public finance quarterly 65:168--195, ://dx.doi.org/10.35551/PFQ_2020_2_2

    2020 doi

  42. [50]

    (2016) Mastering the game of go with deep neural networks and tree search

    Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al. (2016) Mastering the game of go with deep neural networks and tree search. nature 529(7587):484--489

    2016

  43. [51]

    Xing EP, Jebara T, eds., Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, 387--395 (Bejing, China: PMLR)

    Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms. Xing EP, Jebara T, eds., Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, 387--395 (Bejin...

    2014

  44. [52]

    (2017) Mastering the game of go without human knowledge

    Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A, et al. (2017) Mastering the game of go without human knowledge. nature 550(7676):354--359

    2017

  45. [53]

    The Journal of Finance 35(4):883--896

    Sunder S (1980) Stationarity of market risk: Random coefficients tests for individual stocks. The Journal of Finance 35(4):883--896

    1980

  46. [54]

    (1998) Introduction to reinforcement learning

    Sutton RS, Barto AG, et al. (1998) Introduction to reinforcement learning

    1998

  47. [55]

    International Conference on Learning Representations, ://openreview.net/forum?id=HkL7n1-0b

    Tolstikhin I, Bousquet O, Gelly S, Schoelkopf B (2018) Wasserstein auto-encoders. International Conference on Learning Representations, ://openreview.net/forum?id=HkL7n1-0b

    2018

  48. [56]

    Management Science

    Wang W, Li B, Luo X, Wang X (2022) Deep reinforcement learning for sequential targeting. Management Science

    2022

  49. [57]

    arXiv preprint arXiv:1908.09357

    Whitney W, Agarwal R, Cho K, Gupta A (2019) Dynamics-aware embeddings. arXiv preprint arXiv:1908.09357

    2019 arXiv

  50. [58]

    Xiao SC (2020) Do noisy stock prices impede real efficiency? Management Science 66(12):5990--6014, ://dx.doi.org/10.1287/mnsc.2019.3422

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.