{"id":"2972ae88-60cb-450e-8771-958ee9129dc0","arxiv_id":"2507.18560","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A hierarchical RL portfolio optimizer using FinBERT sentiment and market indicators reports 26% annualized return and Sharpe 1.2 on a 2018-2024 backtest, beating equal-weight and S&P 500 benchmarks.","lead":"This paper builds a three-tier reinforcement learning system for portfolio management that combines FinBERT news sentiment with financial indicators, reporting a 26% annualized return and a Sharpe ratio of 1.2 in a 2018-2024 backtest. It is worth reading as a test of whether hierarchical RL and lightweight LLM sentiment can beat simple benchmarks, though the backtest omits transaction costs and statistical rigor.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transaction costs are excluded and turnover is unreported, so the 26% gross return cannot yet support the claim of outperforming passive benchmarks; realistic frictions could consume most or all of the 12.8pp edge over the S&P 500.","rationale":"I agree with the reader's CONDITIONAL verdict and with the identification of transaction-cost exclusion as a serious gap. I isolate transaction costs rather than the Algorithm 2 lookahead oracle as the single most load-bearing concern because the lookahead is confined to the training period: at test time the super-agent only maps meta-agent weight vectors to a final allocation, so the 2018–2024 evaluation is not directly contaminated by the oracle. If the oracle-trained policy generalizes at all, the gross result is at least internally coherent. By contrast, the cost omission directly affects the headline number: the reported 26.0% is a gross return in a strategy that rebalances 14 assets monthly, and the paper gives no turnover data from which the reader could assess net performance. The S&P 500 benchmark is effectively a passive hold, so comparing gross active returns to passive benchmark returns is apples-to-oranges. This is not an accusation of wrongdoing; the paper explicitly acknowledges the limitation. But because the central claim is stated as an outperformance result, and because costs can plausibly be on the order of the entire 12.8pp edge, the claim is not yet established. I do not raise the malformed code links as load-bearing here, since even perfect code access would not remove the need for turnover and net-return reporting. The paper does have positive elements: a temporal train/test split, seed-averaged base agents, and a modular reproducibility design. These support the framework as a research prototype, but not the quantitative headline as stated. Verdict stays CONDITIONAL/UNCHANGED: the missing cost analysis is addressable and should be a condition for accepting the performance claim.","tokens_in":10038,"tokens_out":5115,"duration_ms":58945,"concrete_test":"Obtain the super-agent's monthly allocation matrices for 2018–2024 from the provided Colab notebooks, and compute one-way turnover for month t as T_t = 0.5 * sum_i |w_{i,t} - w_{i,t-1}|, using drift-adjusted weights for the holdings at t-1. Annualize average monthly turnover, then recompute annualized ROI and Sharpe after subtracting a monthly cost of c * T_t * portfolio_value_t for c = 50 bps and c = 100 bps. If the net annualized ROI with c = 100 bps remains above the S&P 500's 13.2% and net Sharpe remains above 0.63, the transaction-cost objection is answered; otherwise the headline claim should be re-scoped to gross returns.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that HARLF achieves a 26.0% annualized ROI and Sharpe 1.2 in 2018–2024, beating equal-weight (7.5%) and S&P 500 (13.2%) benchmarks (Table 2, Section 8). The most load-bearing unsupported step is the explicit exclusion of transaction costs in Section 4.3: 'frequent trading incurs costs (excluded here for simplicity).' The portfolio has 14 assets, is rebalanced monthly, and the hierarchical agents output continuous weight vectors, so turnover is likely substantial and is never reported. At conservative one-way costs of 50–100 basis points on global indices and commodities, the 12.8 percentage-point gross edge over S&P 500 could shrink by more than half or vanish entirely. Because no net-of-cost figures or turnover statistics are provided, the headline result is not yet comparable to buy-and-hold benchmarks. This concern is independent of whether Algorithm 2's in-sample lookahead oracle generalizes to the test period; even a perfectly generalizing policy has unverified economic value once frictions are included. Section 9 acknowledges the omission, but acknowledgment does not quantify the effect.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HARLF, a three-tier hierarchical reinforcement learning framework for portfolio optimization. Base agents (PPO, SAC, DDPG, TD3) process either quantitative financial indicators or FinBERT-based sentiment scores; meta-agents, implemented as small PyTorch networks, aggregate the base-agent recommendations within each modality; and a super-agent combines the two meta-agent outputs into final monthly portfolio weights. The agents are trained on 2003–2017 data and backtested on 2018–2024 data across 14 global equity and commodity instruments under long-only, no-leverage, monthly-rebalancing constraints. The headline result is a 26.0% annualized ROI and Sharpe ratio of 1.2 for the super-agent, versus 13.2% for the S&P 500 and 7.5% for an equal-weighted portfolio. Three Google Colab notebooks are provided for reproducibility.","tokens_in":10410,"tokens_out":3547,"duration_ms":36684,"significance":"If the claimed results are robust, the framework would be a useful practical contribution: it demonstrates a scalable way to combine lightweight sentiment models with hierarchical reinforcement learning, and it provides a concrete decomposition of performance across base agents, meta-agents, and the super-agent. The paper's explicit delivery of reproducible notebooks and its clear statement of portfolio constraints are strengths. However, the central economic claim is not yet established: the reported returns are gross of transaction costs, no statistical uncertainty is reported, and the super-agent's training target relies on an in-sample lookahead oracle that could inflate apparent performance. The comparison with prior work in Table 4 mixes incompatible settings and provides no basis for a state-of-the-art claim.","major_comments":[{"comment":"The paper explicitly excludes transaction costs ('frequent trading incurs costs (excluded here for simplicity)') while making the central claim that the super-agent's 26.0% ROI outperforms the S&P 500 by 12.8 percentage points. Since the portfolio has 14 assets, uses continuous weight vectors, and is rebalanced monthly, turnover is likely substantial, and realistic frictions could consume a large fraction or all of the reported edge. Please report portfolio turnover, net-of-cost returns under several one-way cost assumptions (e.g., 10, 50, and 100 basis points), and the breakeven cost per trade that eliminates the outperformance. Without these numbers, the headline result is not comparable to the buy-and-hold benchmarks.","section":"§4.3 and Table 2"},{"comment":"Algorithm 2 selects the super-agent's training target as the base-agent action with the highest lookahead reward over a future window H: w*_t = arg max over the sum of rewards from t to t+H. This is an in-sample oracle that can systematically select actions that look good only in hindsight, and the imitation policy trained on these targets may not generalize to the test period. The paper does not report H, does not justify why this procedure is free of lookahead bias, and does not compare it against a purely backward-looking training target or an out-of-sample model-selection rule. Please clarify the exact procedure, report H, and provide an ablation or alternative validation that does not use future information when constructing training targets.","section":"§7, Algorithm 2"},{"comment":"All performance figures are annualized over a single 2018–2024 test window and, for base agents, only the median over 5 seeds is reported. Given that annualized volatility is around 20% and the sample is only 7 years, a 26% versus 13.2% difference need not be statistically meaningful. No confidence intervals, per-seed values, or significance tests are provided. Please report the full distribution of outcomes across seeds, bootstrap or subsample confidence intervals for ROI and Sharpe, and a formal test of whether the super-agent's excess return over the benchmarks is distinguishable from noise.","section":"§8, Table 3"},{"comment":"Table 4 compares the proposed agents with DQN/DDPG/PPO from [Espiga-Fernández et al., 2024] and CNN/RNN/LSTM from [Jiang et al., 2017], but the asset universes, test periods, rebalancing rules, and cost assumptions are not stated and almost certainly differ. The sentence 'our super agent seems to have surpassed the current state of the art' is therefore unsupported. Either remove Table 4 and the state-of-the-art claim, or provide a controlled comparison using the same data, period, and evaluation protocol for all methods.","section":"§8, Table 4"}],"minor_comments":[{"comment":"The training-period start date is inconsistent: the abstract says 2000–2017, while Sections 3.1 and 6.2 say 2003–2017. Please correct this.","section":"Abstract, §3.1, §6.2"},{"comment":"The text has several typographical issues, including 'How to Aboard the Time Specific Data Collection Issue?' and 'Scrape the 10 first article for each links.' These should be cleaned up before publication.","section":"§3.2, Algorithm 1"},{"comment":"Algorithm 2 is titled 'Training Super-Agent using PyTorch' but its Ensure line says 'Return Trained Meta-agent.' The terminology should be made consistent with the surrounding text, which distinguishes the super-agent from the meta-agents.","section":"§7, Algorithm 2"},{"comment":"The sentence 'Early pioneers like Moody and Saffell [Denget al., 2016]' cites [Deng et al., 2016] for work by Moody and Saffell; the reference seems mistargeted and should be corrected to the original Moody and Saffell work.","section":"§2.2"},{"comment":"The tables pack many columns without clear row separation, which makes them hard to read. Please reformat the tables to distinguish rows for each agent and benchmark.","section":"§8, Tables 2 and 3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the architecture is genuinely a new stack: three tiers of agents—base RL agents on separate data and NLP pipelines, MLP meta-aggregators, and a super-agent trained to imitate the best base-agent action under a lookahead oracle. That design is clean, modular, and clearly described, and the authors are honest that this is a hierarchical blending of known components. Second, the headline result—26% annualized ROI and Sharpe 1.2 on 2018–2024—is gross of transaction costs, and turnover is never reported. The paper explicitly says in Section 4.3 that frequent trading incurs costs but they are excluded for simplicity. With 14 assets, monthly rebalancing, and continuous weight outputs, turnover is likely high enough that a realistic 50–100 bp per side could eat most of the 12.8-point edge over the S&P 500. That is not a hypothetical: it is the single largest gap between what the abstract promises and what the backtest actually shows.\n\nWhat the paper does well: the separation into data-driven and sentiment-driven meta-agents is a sensible way to make the RL hierarchy interpretable, and the super-agent's oracle-imitation training (Algorithm 2) is at least explicitly stated rather than hidden. The training/test split (2003–2017 / 2018–2024) is reasonable, and the authors volunteer several limitations (transaction costs, asynchronous data, adversarial scenarios). The FinBERT choice is backed by a legitimate lightweight-model result, not just self-citation. They also provide Colab notebooks, though the URLs in the manuscript are malformed and need fixing.\n\nWhere it is soft, in order of severity. (1) No net-of-cost returns, no turnover statistic, no break-even cost analysis. This is fatal for the performance claim as stated. (2) No error bars or significance tests; we only see single backtests and median base-agent results over five seeds. (3) The super-agent's lookahead oracle in Algorithm 2 selects the best future action in the training period; the test-period claim assumes this imitation policy generalizes, which is plausible but untested. (4) The SOTA comparison in Table 4 mixes different test periods and asset universes, so \"beats CNN-RL\" is not a controlled claim. These are all addressable, and none of them make the architecture worthless.\n\nWho this is for: people working on RL portfolio optimization or FinLLM pipelines who want a concrete hierarchical baseline to compare against, and reviewers who want a case study in how backtest claims can be undermined by missing frictions. It does not contribute new theory or a verified result, but it is a coherent system paper.\n\nMy recommendation: yes, send it to peer review—not because the current evidence supports the headline, but because the architecture is solid, the flaws are specific and fixable, and a serious referee process could force the authors to provide the cost analysis and statistical rigor the paper needs. A desk reject would lose a useful data point for the community; an acceptance without major revision would mislead readers.","headline":"A plausible three-tier RL+FinBERT architecture with an interesting but unverified 26% gross-return claim; transaction costs and turnover are the load-bearing missing pieces, and they can be fixed in revision.","tokens_in":10823,"tokens_out":1894,"would_cite":false,"duration_ms":21752,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91G10","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A three-tier reinforcement-learning portfolio manager that fuses FinBERT news sentiment with price metrics reports 26% annualized return and a 1.2 Sharpe ratio on out-of-sample 2018–2024 data.","keywords":["hierarchical reinforcement learning","portfolio optimization","FinBERT sentiment","large language models in finance","multi-agent reinforcement learning","backtesting","quantitative finance"],"falsifier":"Run the released backtest with realistic transaction costs on every monthly rebalance of the 14 assets and with a rolling walk-forward scheme in which the super-agent is retrained only on data before each test month; if the 26.0% return and 1.2 Sharpe drop to benchmark levels, the imitation-policy generalization claim fails. A second decisive check is an ablation that shuffles the FinBERT sentiment scores across months: if the NLP meta-agent retains its 20.5% edge under shuffled labels, the sentiment channel is not the active ingredient.","tokens_in":9801,"feed_emoji":"📈","tokens_out":11859,"duration_ms":108105,"temperature":0.7,"pith_summary":"The paper aims to show that a three-tier hierarchical reinforcement-learning agent can beat standard portfolio benchmarks by fusing two data modalities: numeric market indicators and sentiment scores extracted from financial news by FinBERT, a lightweight language model fine-tuned for financial text. Base agents specialize in one modality, meta-agents learn to combine each modality's recommended allocations, and a top-level super-agent merges the two views into a monthly portfolio. Trained on 2003–2017 and backtested on 2018–2024, the stack reports 26.0% annualized return with a 1.2 Sharpe ratio, versus 7.5% and 13.2% for equal-weighted and S&P 500 benchmarks. The authors read this as evidence that hierarchical aggregation and LLM-driven sentiment both add out-of-sample value.","feed_headline":"Hierarchical RL portfolio agent posts 26% annual return","feed_subtitle":"Stacked reinforcement agents with FinBERT news sentiment beat equal-weight and S&P 500 on 2018–2024 data.","key_machinery":"The load-bearing mechanism is supervised imitation of the best lower-level decision, implemented in Algorithm 2. In each training month, the super-agent observes the concatenated weight vectors proposed by the two meta-agents, simulates each candidate portfolio $H$ months ahead, labels the candidate with the highest future reward as the target $w^*$, and fits its network by mean-squared error to those labels. This converts the top of the hierarchy into a regression-to-best-action problem rather than a separately trained RL policy; at test time the super-agent balances a data-driven view of the market with a sentiment-based 'conviction' view. The base layer uses four standard continuous-control RL algorithms (PPO, SAC, DDPG, TD3), and the NLP channel uses FinBERT, a BERT variant fine-tuned on financial text, with monthly sentiment scores computed from the mean of positive minus negative classification probabilities over that month's news articles.","core_discovery":"The central claim is that a three-layer hierarchy—base RL agents, modality-specific meta-agents, and a final super-agent—produces better risk-adjusted portfolio allocations than flat single-agent baselines. On monthly data for 14 global equity indices and commodities, base agents using four continuous-control reinforcement-learning algorithms propose weights from either price-derived statistics or FinBERT sentiment scores; a softmax-output network for each modality learns to aggregate those proposals; and the super-agent, fed the concatenated meta-agent weight vectors, is trained to imitate whichever candidate action achieved the highest simulated reward over a future window $H$ in the training period. In the 2018–2024 test window the NLP meta-agent earns 20.5% annualized ROI at 1.2 Sharpe, the metrics meta-agent 14.7% at 0.8, and the super-agent 26.0% at 1.2, above both benchmarks and above the annualized returns of the RL comparisons listed in Table 4 (whose Sharpe ratios range from 0.7 to 1.3).","pith_inferences":["The paper's result split implies the sentiment channel is a major driver, but the design never isolates the two ingredients: a crossed ablation (sentiment on/off times hierarchy on/off) would quantify how much of the 26% comes from each.","Because the best-action label in Algorithm 2 uses a lookahead reward over $H$ future steps inside the training period, the method is susceptible to hindsight selection; a natural stress test is to lag the news-to-rebalance alignment and retrain with a rolling window, which the paper lists as future work rather than testing.","With only 84 monthly test observations in one 2018–2024 window, the 1.2 Sharpe ratio has wide sampling uncertainty; a bootstrap over monthly returns would show whether the super-agent is statistically distinguishable from benchmarks at 0.57–0.63.","The benchmark set omits simple rules such as equal-risk contribution or momentum, so an outside reader cannot tell how much of the edge comes from the hierarchy versus the commodity-heavy asset universe; adding those baselines is a cheap way to localize the claimed improvement."],"forward_implications":["If the reported out-of-sample numbers hold, the hierarchy itself adds value: the super-agent's 26.0% beats the metrics-only meta-agent's 14.7% and the sentiment-only meta-agent's 20.5%, so combining the two views is more than simple averaging.","The sentiment channel carries signal: the NLP meta-agent achieves 20.5% at the same 16.0% volatility as the metrics meta-agent's 14.7%, consistent with the paper's claim that FinBERT news sentiment improves risk-adjusted returns.","The lightweight-LLM design is practical for monthly rebalancing: sentiment can be computed offline, and the reinforcement-learning policies act on monthly observation vectors rather than high-frequency data.","The architecture is extensible: additional base agents or data modalities can be inserted below the meta-agents without changing the super-agent's training rule, which only consumes the concatenated weight proposals.","The regression-to-best-action training rule implies that the top-level policy inherits only what lower-level agents propose, so information discarded at the base layer is inaccessible to the final allocation."],"supporting_citations":[{"why":"Supplies FinBERT, the financial-text sentiment model that generates the NLP scores feeding the base and meta agents.","marker":"[Araci, 2019]"},{"why":"Establishes that media sentiment affects stock prices, the empirical basis for adding a news-sentiment channel.","marker":"[Tetlock, 2007]"},{"why":"Provides the deep-RL portfolio-management formulation this work builds on and the CNN-RL/RNN-RL/LSTM-RL baselines used for comparison.","marker":"[Jiang et al., 2017]"},{"why":"Supports choosing a lightweight model by showing fine-tuned compact models can match or exceed larger LLMs on financial classification.","marker":"[Lefort et al., 2024]"},{"why":"Supplies the reference implementations of the four continuous-control algorithms used as base agents.","marker":"[Raffin et al., 2021]"},{"why":"Provides the DQN, DDPG, and PPO portfolio-optimization results that serve as recent baselines in Table 4.","marker":"[Espiga-Fernández et al., 2024]"},{"why":"Supports the claim that news-driven RL outperforms price-only trading, the pattern the NLP meta-agent is said to reproduce.","marker":"[Xu and Zhou, 2018]"}],"fun_headline_variants":["Three-tier RL with FinBERT sentiment posts 26% annual","Hierarchical RL and LLM sentiment achieve 26% yearly returns","Stacked RL agents and news sentiment beat S&P 500","HARLF: 26% return via hierarchical RL and sentiment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the assumption that the super-agent's policy—learned by copying whichever lower-level agent would have earned the highest reward over a future window in the 2003–2017 training period—keeps working on 2018–2024 data, and that the paper's deliberately excluded monthly trading costs would not erase the 26% gain.","fun_headline_variants_meta":{"raw":{"variants":["Three-tier RL with FinBERT sentiment posts 26% annual","Hierarchical RL and LLM sentiment achieve 26% yearly returns","Stacked RL agents and news sentiment beat S&P 500","HARLF: 26% return via hierarchical RL and sentiment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0005,"raw_usage":{"total_tokens":2421,"prompt_tokens":893,"completion_tokens":1528,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":1455}},"tokens_in":509,"tokens_out":1528,"duration_ms":11575,"temperature":1.0,"reasoning_tokens":1455,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:10:01.906944+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released backtest with realistic transaction costs on every monthly rebalance of the 14 assets and with a rolling walk-forward scheme in which the super-agent is retrained only on data before each test month; if the 26.0% return and 1.2 Sharpe drop to benchmark levels, the imitation-policy generalization claim fails. A second decisive check is an ablation that shuffles the FinBERT sentiment scores across months: if the NLP meta-agent retains its 20.5% edge under shuffled labels, the sentiment channel is not the active ingredient.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that media sentiment affects stock prices, the empirical basis for adding a news-sentiment channel."},{"cited_title":"Stable-baselines3: Reliable reinforcement learning implementations.Journal of Machine Learning Research, 22(268):1–8,","cited_arxiv_id":null,"evidence_quote":"Supplies the reference implementations of the four continuous-control algorithms used as base agents."},{"cited_title":"A systematic approach to portfolio optimization: A comparative study of reinforcement learning agents, market signals, and investment horizons","cited_arxiv_id":null,"evidence_quote":"Provides the DQN, DDPG, and PPO portfolio-optimization results that serve as recent baselines in Table 4."},{"cited_title":"Exciton-Peierls mechanism and universal many-body gaps in carbon nanotubes","cited_arxiv_id":"1807.05589","evidence_quote":"Supports the claim that news-driven RL outperforms price-only trading, the pattern the NLP meta-agent is said to reproduce."}],"review_version":2}