REVIEW 5 major objections 5 minor 14 references
Multimodal Deep Reinforcement Learning for Portfolio Optimization
T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A deep RL trader using SEC and news sentiment beats most S&P100 benchmarks in backtests.
desk verdict A careful EIIE re-implementation with sentiment channels whose own tables contradict the paper's headline claim that combining data sources improves performance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the state tensor $S=(S^*,\delta)$ that adds alternative-data channels to historical prices: for each asset and each day in the lookback window, the tensor stores high, low, and close prices together with a SEC sentiment score (Loughran–McDonald word proportions) and a news sentiment score from FinBERT mapped through the $\tanh(\text{positive}/\text{negative}/\text{neutral})$ embedding of Equation (1). The policy is the Ensemble of Identical Independent Evaluators (EIIE) from the reference framework, where one CNN or RNN feature extractor scores each asset independently and a softmax converts scores into portfolio weights; the MLP baseline applies the same tensor to a single fully connected network. Rewards are either period profit or the Differential Sharpe Ratio, and transaction costs enter through the fixed-point update factor $\mu_t$. Sparse sentiment dates are forward-filled by exponential decay $y=a(1-\gamma)^t$ with $\gamma\approx0.8$.
What would settle it
Retrain the best configuration (CNN EIIE, combined data, profit reward) with the news and SEC channels replaced by constant zeros, and separately with the same channels filled with random noise matched to the observed sentiment distribution; if test-period net profit, Sharpe ratio, and Sortino ratio stay within noise of the reported 0.166, 0.554, and 0.665, then the sentiment channels are not carrying the claimed signal.
Extended reading notes
Core claim
The paper's central discovery, stated on its own terms, is that the best-performing model across all three experiment sets is the CNN EIIE model trained on the combined dataset (price, SEC sentiment, and news sentiment) with the Profit reward. In the 2018–2019 out-of-sample test this agent reports a net profit of 0.166, Sharpe ratio of 0.554, Sortino ratio of 0.665, and max drawdown of 0.195, which beats the S&P500 index as well as the OLMAR and WMAMR benchmarks; only the Equal Buy-and-Hold portfolio has higher net profit and risk-adjusted ratios. The paper interprets this as evidence that multimodal company sentiment enriches the state representation, that SEC data in particular is regular and reliable enough to help learning, and that the choice of reward function matters more than model complexity once alternative data is present.
Load-bearing premise
The load-bearing premise is that exponential-decay forward-filling turns sparse SEC filings and news headlines into a faithful daily sentiment signal, because most of the sentiment values the agent sees are imputed rather than measured.
Editorial extensions
If this is right
- A practical takeaway is that adding regularly reported alternative data (SEC sentiment) to a compact EIIE policy improves returns under both profit and Differential Sharpe rewards.
- The profit reward is a more reliable training signal than the Differential Sharpe Ratio for these RL agents, so practitioners can expect better out-of-sample results from the simpler reward.
- CNN and RNN EIIE policies are preferable to a full-tensor MLP because their smaller parameter counts reduce overfitting on the price-plus-sentiment state.
- Combining news with SEC data gives the strongest configuration, while news alone is too sparse and irregular to help, especially under the harder reward.
- All trained agents in the comparison outperform the OLMAR and WMAMR online portfolio benchmarks over the test period.
Reading between the lines
- Beyond the paper's claims: the reported gains could be an artifact of the forward-filling rule, since the median ticker has actual news on only 905 trading dates; retraining with unfilled or shuffled sentiment would isolate the true signal.
- The sentiment embedding in Equation (1) is an ad hoc ratio that ignores the absolute level of neutral probability; a calibrated alternative such as the raw FinBERT positive-minus-negative probability would test how much the embedding form matters.
- The conclusions are drawn from one 2018–2019 test window with 1% transaction costs; extending the same architecture to other periods, universes, or cost levels would show whether the recipe generalizes.
- A testable extension is to use the same state tensor with a learned attention or gating layer over the sentiment channels; if the model learns to ignore sentiment on days after long gaps, that would corroborate the paper's interpretation that data regularity drives the improvement.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a deep reinforcement learning framework for S&P100 portfolio optimization that augments price history with sentiment features derived from SEC filings and news headlines, using EIIE-style CNN, RNN, and MLP policies under profit and Differential Sharpe rewards. The authors report backtests over a 2018–2019 test period and claim that the CNN EIIE model trained on combined price+SEC+news data with a profit reward is the best-performing model across all experiments, and that it exceeds OLMAR and WMAMR benchmarks. The paper also describes data processing, an exponential-decay forward-filling rule for sparse sentiment observations, and a transaction-cost-aware environment.
Significance. If the central claim were supported, the paper would provide a moderately useful incremental demonstration that multimodal sentiment channels can be integrated into EIIE-style RL portfolio policies. The manuscript has several strengths: it carefully describes the data pipeline, implements a nontrivial transaction-cost model from prior work, and explicitly acknowledges data sparsity limitations. However, the reported evidence does not support the headline claim. The paper's own tables show that a price-only CNN with profit reward outperforms the combined-data CNN on net profit, Sharpe, and Sortino ratios, and that the simple Equal Buy-and-Hold benchmark beats every trained RL policy. The selected comparison table omits the price-only configuration that contradicts the claim. Without multiple seeds or error bars, the small apparent differences among RL variants cannot be distinguished from noise. The forward-filling rule for news sentiment, which supplies the majority of the multimodal state entries for most tickers, is a load-bearing assumption that is not validated. These issues undermine the paper's central contribution as presented.
major comments (5)
- [§4.4, Table 11 vs. Table 7] The claim in §4.4 that 'our best-performing model across all three sets of experiments is the CNN EIIE model on the combined dataset using Profit reward' is contradicted by the paper's own numbers. Table 7 reports that the price-only CNN with Profit reward achieves net profit 0.166717, Sharpe ratio 0.561471, and Sortino ratio 0.676669, whereas Table 11 reports for the SEC+News CNN with Profit reward net profit 0.166081, Sharpe ratio 0.553644, and Sortino ratio 0.664706. The price-only model outperforms the combined model on all three metrics, so the stated 'best-performing model' is not the best according to the reported results.
- [§4.5, Table 12] Table 12, which is used to support the headline comparison of 'best strategies,' omits the price-only CNN with Profit reward from Table 7. Including that row would show that the selected SEC+News CNN is not the best trained policy, and the text's statement that the SEC+News CNN Profit policy has the highest net profit, Sharpe, and Sortino excluding Equal Buy-and-Hold is factually incorrect given the paper's own Table 7 values. The comparison must include all configurations, especially those that outperform the configuration being highlighted.
- [§4.2–§4.5, Tables 5–12] Every trained RL policy reported in Tables 6–11 has lower net profit and Sharpe ratio than the simple Equal Buy-and-Hold benchmark, which achieves net profit 0.181138 and Sharpe 0.595225. The abstract's claim that the approach 'delivers superior portfolio performance' and 'outperform standard benchmarks' is therefore not supported; the trained agents only outperform OLMAR and WMAMR, which are themselves weak in this test period. The benchmark comparison should be presented in full, and the conclusion should be revised to acknowledge that the method does not beat the equal-weight buy-and-hold baseline.
- [§2.2.3 and Eq. (2)] The multimodal state tensor is largely constructed by exponential-decay forward-filling of sparse sentiment observations. News coverage exists for only 84 of 100 tickers, and the median ticker has reports on only 905 trading dates out of the roughly 2,500 trading days in the full period. Eq. (2) with a tuned parameter γ≈0.8 imputes sentiment scores for the vast majority of state entries, and the paper itself notes in §2.2.3 that 'coverage is still incomplete.' Because the imputation rule determines most of the sentiment channel, the comparisons between price-only, SEC-only, and combined configurations reflect properties of the filling rule as much as the information content of the underlying news data. The paper should provide sensitivity analysis over the imputation rule (e.g., different decay factors, no imputation) before attributing performance differences to the data sources.
- [§4, all tables] No error bars, confidence intervals, or multiple-seed results are reported for any RL policy. Differences between configurations—for example, SEC+News CNN (0.166081) versus SEC-only CNN (0.162534) under Profit reward—are small and may be within run-to-run variance, especially given that policy-gradient training is stochastic. The paper should report mean and standard deviation over at least several independent training runs, or otherwise provide statistical evidence that the claimed improvement is not noise.
minor comments (5)
- [Table 11 caption] The caption of Table 11 says 'Strategies with Combined Data (DiffSharpe Reward)' but the surrounding text in §4.4 states that the table reports results for the Profit reward. The caption should be corrected to 'Profit Reward' to match the content.
- [§2.2.3, Table 3] The statistic 'Minimum Observations 1' is ambiguous: it could mean one reporting date or one headline. Please clarify whether the count is of reporting dates or individual headlines.
- [Reference [10]] The reference contains a typo: 'finaincial' should be 'financial.'
- [§4.3, Table 9] The text states 'The CNN and RNN policies using the Differential Sharpe ratio are among the strongest contenders,' but Table 9 shows Profit-reward results for SEC data. The intended reference is likely Table 8; please correct the cross-reference.
- [§3.6] The notation for the EIIE functions is inconsistent: the text uses fα, gβ, and γ, but also refers to fθ1 and trainable parameters α, β, and γ without defining θ1. Please harmonize the notation.
Circularity Check
No significant circularity: the RL pipeline, reward functions, and sentiment embeddings are externally defined, and no prediction reduces to a fitted input or self-citation.
full rationale
The paper's derivation chain is self-contained in the circularity sense. The state tensor is assembled from CRSP prices, Loughran-McDonald SEC sentiment proportions, and FinBERT news sentiment via Equation (1); the reward functions (profit and Differential Sharpe Ratio) are defined externally, and the EIIE policy architecture is adopted from prior work [3,4]. The only tuned quantity is the exponential-decay hyperparameter gamma in Equation (2), which is conventional hyperparameter fitting and not a prediction target. No load-bearing claim is justified solely by a self-citation, and no uniqueness theorem is imported from the authors' own prior work. The paper itself flags a data limitation in Section 2.2.3 ('our coverage is still incomplete'), which is a soundness caveat rather than a circular step. A separate, non-circular correctness concern remains: Section 4.4's claim that the combined-data CNN is 'best-performing across all three sets of experiments' is contradicted by the paper's own Table 7, where the price-only CNN with Profit reward has higher net profit (0.166717), Sharpe (0.561471), and Sortino (0.676669) than the combined-data CNN in Table 11 (0.166081, 0.553644, 0.664706), and every trained policy underperforms Equal Buy-and-Hold (0.181138, 0.595225). This is an evaluation-selection or reporting issue, not circularity.
Assumptions & free parameters
free parameters (3)
- sentiment decay factor gamma =
~0.8
- transaction cost rate =
1%
- state lookback window H =
not reported
assumptions (4)
- domain assumption The S&P100 trading environment can be modeled as a deterministic MDP whose transitions are fully known from historical data.
- domain assumption FinBERT probabilities and Loughran-McDonald word proportions are valid sentiment signals for news and SEC filings.
- ad hoc to paper Exponential decay with gamma=0.8 reconstructs unobserved daily sentiment between reports.
- domain assumption The EIIE policy class from [4] and the state-augmentation scheme from [3] transfer unchanged to S&P100 equities and to additional sentiment channels.
Cite this review
Pith. "Pith review of Multimodal Deep Reinforcement Learning for Portfolio Optimization." pith.science (2026). https://pith.science/paper/FNPBHP6D
@misc{pith2026241217293,
author = {Pith},
title = {Pith review of: Multimodal Deep Reinforcement Learning for Portfolio Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/FNPBHP6D}},
note = {Machine review of arXiv:2412.17293}
}
read the original abstract
We propose a reinforcement learning (RL) framework that leverages multimodal data including historical stock prices, sentiment analysis, and topic embeddings from news articles, to optimize trading strategies for SP100 stocks. Building upon recent advancements in financial reinforcement learning, we aim to enhance the state space representation by integrating financial sentiment data from SEC filings and news headlines and refining the reward function to better align with portfolio performance metrics. Our methodology includes deep reinforcement learning with state tensors comprising price data, sentiment scores, and news embeddings, processed through advanced feature extraction models like CNNs and RNNs. By benchmarking against traditional portfolio optimization techniques and advanced strategies, we demonstrate the efficacy of our approach in delivering superior portfolio performance. Empirical results showcase the potential of our agent to outperform standard benchmarks, especially when utilizing combined data sources under profit-based reward functions.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Srijan Sood, Kassiani Papasotiriou, Marius Vaiciulis, and Tucker Balch. Deep Reinforce- ment Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization
-
[2]
Junkyu Jang and NohYoon Seong. Deep reinforcement learning for stock portfolio optimization by connecting with modern portfolio theory. Expert Systems with Applications, 218:119556, May 2023
work page 2023
-
[3]
Yunan Ye, Hengzhi Pei, Boxin Wang, Pin-Yu Chen, Yada Zhu, Jun Xiao, and Bo Li. Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Pre- diction States, February 2020. arXiv:2002.05780 [cs, q-fin, stat]
arXiv 2020
-
[4]
A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem, July 2017
Zhengyao Jiang, Dixing Xu, and Jinjun Liang. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem, July 2017. arXiv:1706.10059 [cs, q-fin] version: 2
arXiv 2017
-
[5]
Qiang Song, Anqi Liu, and Steve Y . Yang. Stock portfolio selection using learning-to-rank algorithms with news sentiment. Neurocomputing, 264:20–28, November 2017
work page 2017
-
[6]
MAPS: Multi-agent Rein- forcement Learning-based Portfolio Management System
Jinho Lee, Raehyun Kim, Seok-Won Yi, and Jaewoo Kang. MAPS: Multi-agent Rein- forcement Learning-based Portfolio Management System. In Proceedings of the Twenty- Ninth International Joint Conference on Artificial Intelligence, pages 4520–4526, July 2020. arXiv:2007.05402 [cs]
arXiv 2020
-
[7]
Wharton Data Research Services. CRSP daily stocks, 2010-2024
work page 2010
-
[8]
SEC.gov | how to read a 10-k/10-q
Show all 14 references
-
[9]
SEC.gov | EDGAR | company filings
-
[10]
Daily finaincial news for 6000+ stocks, 2020
Miguel Aenille. Daily finaincial news for 6000+ stocks, 2020. Re- treived 2024-05-05 from https://www.kaggle.com/datasets/miguelaenlle/ massive-stock-news-analysis-db-for-nlpbacktests/data
2020
-
[11]
Moody and Lizhong Wu
J. Moody and Lizhong Wu. Optimization of trading systems and portfolios. In Proceedings of the IEEE/IAFE 1997 Computational Intelligence for Financial Engineering (CIFEr), pages 300–307
1997
-
[12]
Reinforcement learning for trading systems and portfolios: Immediate vs future rewards
John Moody, Matthew Saffell, Yuansong Liao, and Lizhong Wu. Reinforcement learning for trading systems and portfolios: Immediate vs future rewards. In Apostolos-Paul N. Refenes, Andrew N. Burgess, and John E. Moody, editors, Decision Technologies for Computational Finance: Pro...
-
[13]
Bin Li and Steven C. H. Hoi. On-line portfolio selection with moving average reversion
-
[14]
∞X t=0 γtRt s0 = s # Qπ(s, a) = E π
Li Gao and Weiguo Zhang. Weighted moving average passive aggressive algorithm for online portfolio selection. In 2013 5th International Conference on Intelligent Human-Machine Systems and Cybernetics, volume 1, pages 327–330. 16 Appendix A Reinforcement Learning Overview The r...
2013
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.