{"id":"ad80e7ba-eaee-48f6-aa94-76e6ebce0e03","arxiv_id":"2607.16028","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A DDPG agent using news sentiment and an excess-return reward beat buy-and-hold on Tesla (54.96% vs 16.45%) and Bitcoin (1.58% vs -34.27%) in the 2025-26 FinMMEval test window.","lead":"This paper tests four reinforcement-learning agents that make daily buy, sell, or hold decisions on Tesla and Bitcoin using prices, technical indicators, and news sentiment from LLaMA. The best agent beat buy-and-hold on both assets in the 2025–26 test period, but the authors also show validation rankings failed to predict Bitcoin performance.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Alpha reward's variance-reduction claim is false for flat/short positions, and no ablation supports the 'reduces overfitting' contribution; the paper's own train/test gap indicates overfitting persists.","rationale":"The reader's weakest assumption concerns the single cost-free test path, which is a real limitation for the empirical headline. But the more fundamental problem is that the central methodological contribution—alpha reward reduces overfitting—is unsupported: the paper's own equivalence theorem removes any objective-level difference, the variance-reduction property fails for flat and short positions, and no ablation isolates the alpha reward. The empirical cross-algorithm comparison cannot substitute for a raw-reward baseline because all algorithms use the alpha reward. The paper also reports enormous training returns, suggesting overfitting persists. These issues do not change the conditional verdict: the system description is valuable, but the contribution claims need either supporting ablations or softening. Properties 1 and 2 of the alpha reward are correct, and the authors are transparent about their selection protocol and limitations.","tokens_in":14560,"tokens_out":11842,"duration_ms":124905,"concrete_test":"Run a matched ablation: train DDPG and DQL with the raw log-return reward (no alpha subtraction) using identical hyperparameters, random episode starts, seeds, and validation-Sharpe selection, and compare test CR/SR and the train-vs-test gap across at least 10 seeds. If the raw-reward runs are statistically indistinguishable from the alpha-reward runs, the paper's central claim that the alpha reward reduces overfitting is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.4's Property 2 states that under the price-taker assumption the alpha reward and raw log-return reward differ by a policy-independent constant and share optimal policies. Consequently, the alpha reward cannot reshape the objective; the only mechanism offered for 'reduces overfitting' is the variance-reduction Property 3. That property is not generally valid in this discrete-action setting. For a flat position, log(V_t/V_{t-1})=0 and r_t^alpha=-m_t, so the alpha reward has variance Var(m_t) while raw reward has variance 0. For a short position, to first order r_t^alpha≈-2m_t versus r_t^raw≈-m_t, so subtracting the market return increases variance unless the agent is almost always long. The paper reports no ablation separating alpha reward from random episode starts; Section 7 lists exactly this ablation as future work. Meanwhile Section 6.1 reports training returns of 114,462% (TSLA) and 256,989% (BTC), evidence that severe overfitting persists. Thus the central contribution is not established, and the headline empirical comparison rests on one cost-free test path with no confidence intervals.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a system for CLEF 2026 FinMMEval Task 3, formulating daily long/flat/short trading as a discrete-action MDP. It compares PG, PPO, DQL, and DDPG on a feature set of technical indicators, calendar encodings, and LLaMA 3.2-based news sentiment. The methodological contribution is an 'alpha reward' r_t = log(V_t/V_{t-1}) - log(c_t/c_{t-1}) combined with random episode starts, claimed to reduce overfitting and align training with the evaluation metric of beating buy-and-hold. On the test set, DDPG achieves the best reported performance (TSLA CR 54.96%, SR 1.44; BTC CR 1.58%, SR 0.23), while DQL was selected a priori for the live endpoint based on validation Sharpe ratio. The paper also documents a large validation-to-test generalization gap, attributed to regime shift.","tokens_in":14842,"tokens_out":7251,"duration_ms":74053,"significance":"The alpha-reward telescoping identity (Eq. 4) and the equivalence of optimal policies with raw log-return reward (Property 2) are correctly stated under the price-taker assumption. The paper is transparent about its a-priori endpoint selection and about the train/test gap, and the reproducibility plan is detailed. However, the central claim that the alpha reward 'reduces overfitting' is not established: the variance-reduction argument is not valid for the full discrete action set, and no ablation isolates the reward change from random episode starts. The headline empirical results rest on one cost-free test path without confidence intervals. These issues are load-bearing for the paper's main contribution, though they are fixable with additional experiments.","major_comments":[{"comment":"The variance-reduction/overfitting-reduction mechanism is not valid for the full action set. For a flat position, r_t^alpha = -m_t, so Var(r^alpha)=Var(m_t) while the raw reward has zero variance. For a short position, to first order r_t^alpha ≈ -2m_t versus r_t^raw ≈ -m_t, so subtracting the market return doubles the market-noise variance. Property 3 holds only if the policy is almost always long. No ablation separates the alpha reward from random episode starts; §7 lists such an ablation as future work. Moreover, §6.1 reports training returns of 114,462% (TSLA) and 256,989% (BTC), indicating severe overfitting persists. The statement that the alpha reward 'reduces overfitting' is therefore unsupported as written.","section":"§4.4, Property 3; §1 Contribution 2"},{"comment":"The headline comparison rests on a single realized test path. No multiple seeds, bootstrap confidence intervals, or seed-variance information are reported. Evaluation is cost-free, as transaction costs are applied only during training, and episodes always begin from the first day of the period. Consequently the ranking DDPG > DQL > buy-and-hold, and even the sign of BTC returns, may change under realistic inference-time costs or a different test path. The paper should report uncertainty estimates (e.g., repeated seeds or resampling) and qualify the 'most robust' characterization of DDPG.","section":"§5.2 and Table 4"},{"comment":"The hyperparameter search runs 180 trials per algorithm-asset pair, but the random seed protocol for final training and evaluation is not described. With a single seed, the differences between algorithms (DQL vs DDPG on TSLA, or PG vs PPO on BTC) may be within optimization noise. Please state the number of seeds used and, ideally, report the mean and spread over seeds, or explicitly treat the reported numbers as single-run observations.","section":"§5.1 and §5.2"},{"comment":"The paper correctly identifies the bull-market validation window and bear-market test window as the source of the generalization gap, and admits walk-forward cross-validation was not implemented due to computational cost. This is an honest limitation, but it means the empirical conclusions are specific to one temporal split and one test path. The claims about 'robust performance' should be narrowed accordingly, or supported by at least a small walk-forward or regime-shift analysis.","section":"§6.2"}],"minor_comments":[{"comment":"'articles published after 16:00 UTC (i.e., after the U.S. equity market close)' is factually incorrect; the U.S. close is approximately 20:00/21:00 UTC. The shift is conservative, not look-ahead, but the text should be corrected to avoid confusion about the alignment rule.","section":"§4.1"},{"comment":"Rows 9 and 10 use EMA10/c_{t-1} and EMA50/c_{t-1} as formulas, while the text says each EMA is normalized by the current close. Please align notation.","section":"Table 2"},{"comment":"The heading 'log-probability extraction' is a misnomer: the method extracts logits and applies a restricted softmax. Consider renaming to 'logit-based probability extraction'.","section":"§4.3"},{"comment":"The sentence 'model selection uses the Sharpe ratio computed on the raw portfolio value series' is ambiguous; clarify that 'raw' means the unadjusted portfolio value, not the raw reward.","section":"§5.2"},{"comment":"Captions describe portfolio values but the y-axis appears to be in dollars; please state the assumed initial capital for the plots.","section":"Figures 3 and 4"}],"recommendation":"major_revision","confidential_remarks":"This is a competition system paper with modest theoretical novelty, but it is within the scope of a CLEF working note. The main risk is empirical: single-seed, single-path, cost-free evaluation combined with an unsupported overfitting-reduction claim. I recommend major revision rather than rejection because the theoretical core is sound, the limitations are disclosed, and the required fixes (ablation, seeds, uncertainty estimates) are feasible within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: this is a competent, honest system description. The genuinely useful bits are the head-to-head test results on FinMMEval Task 3 (DDPG 54.96% vs B&H 16.45% on TSLA; 1.58% vs -34.27% on BTC) and the clear demonstration that validation Sharpe fails when the regime flips. The alpha reward is mathematically just excess log return, and its formal properties are correct but essentially tautological under the price-taker assumption. The paper does not oversell it: Property 2 explicitly states optimal policies coincide with the raw reward.\n\nThe empirical protocol is careful: chronological splits, no look-ahead in article alignment, a priori endpoint selection, and honest reporting of the generalization gap, including those absurd 114,462% training returns. The future-work section is unusually candid, explicitly naming the ablation that would be needed to support the 'reduces overfitting' claim.\n\nThe soft spots are real. The central contribution claim — alpha reward plus random episode starts reduces overfitting — is never tested against a raw-reward baseline. The variance-reduction Property 3 is not generally true: for flat positions the alpha reward has variance Var(m_t) while raw reward has zero variance; for shorts it roughly doubles the market variance. And the test results rest on a single cost-free path with no confidence intervals, so the ordering of DDPG vs DQL vs B&H could plausibly change. These are not fatal for a working-notes paper, but they mean the contribution should be framed more modestly.\n\nVerdict: worth sending to peer review as a working-notes system description. The referee should request either ablations and multiple seeds or a softened contribution claim. I'd read it if I were doing DRL trading, and I'd cite it for the regime-shift finding, not for the alpha reward.","headline":"Solid, honest competition write-up with a useful regime-shift result; the alpha reward is mathematically trivial and its overfitting-reduction claim is unsubstantiated, but the paper deserves a referee as working notes.","tokens_in":15360,"tokens_out":1447,"would_cite":true,"duration_ms":14817,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The central claim is that replacing the raw log-return reward with a market-relative alpha reward — the day's portfolio growth minus the day's market growth — aligns training with beating buy-and-hold and, paired with random episode starts,","keywords":["deep reinforcement learning","alpha reward","algorithmic trading","buy-and-hold","sentiment analysis","DDPG","market regime shift","excess return"],"falsifier":"Evaluate the same trained policies on the test window with the 0.2% transaction cost applied at each position change (as it is during training) instead of zero cost. If DDPG's TSLA cumulative return drops below buy-and-hold, or its BTC return turns negative, the claim that the alpha-reward DDPG beats buy-and-hold is refuted. Alternatively, train a raw-log-return agent with identical seeds and hyperparameters; theory says the optimal policies coincide, so if the alpha-reward agent's test returns are not statistically distinguishable from the raw-reward agent's across several seeds, the variance","tokens_in":14437,"feed_emoji":"📈","tokens_out":6204,"duration_ms":53931,"temperature":0.7,"pith_summary":"The paper tries to establish that a trading agent should optimize excess return over the market rather than absolute return. To this end it introduces an alpha reward, which subtracts the daily log return of the asset from the daily log return of the agent's portfolio. The paper proves three properties of this reward: summed over an episode it equals log-alpha (the log of the ratio of portfolio growth to market growth); under the price-taker assumption it shares its optimal policies with the raw log-return reward; and it acts as a variance-reducing control variate. Empirically, a DDPG agent using this reward achieves a 54.96% cumulative return on TSLA versus 16.45% for buy-and-hold, and a 1.58% return on BTC versus -34.27% for buy-and-hold. The paper also documents a large validation-to-test generalization gap, concluding that validation-based model selection is fragile when the validation and test periods are in different market regimes.","feed_headline":"Alpha reward beats buy-and-hold and rescues a BTC bear market","feed_subtitle":"Training traders on excess return over the market yields 54.96% on TSLA and +1.58% on BTC vs -34.27%.","key_machinery":"The central object is the alpha reward, a market-relative reward defined as the difference between the daily log portfolio return and the daily log market return. It does three jobs: (1) it telescopes so that the episode return equals log-alpha, the log of the ratio of final portfolio value to final buy-and-hold value; (2) under the price-taker assumption it leaves the set of optimal policies unchanged relative to the raw log-return reward, since the subtracted market term is policy-independent; and (3) it acts as a control variate, stripping market-driven variance from the reward signal. The other machinery is random episode start dates, which forces the agent to experience diverse market c","core_discovery":"The paper's central discovery is the alpha reward: at each day t, r_t = log(V_t/V_{t-1}) - log(c_t/c_{t-1}), where V_t is the portfolio value and c_t is the asset's closing price. With this reward a buy-and-hold strategy receives zero reward every step, so the agent is trained explicitly to outperform the market. The reward telescopes over an episode to the log of the terminal outperformance ratio alpha_T, shares its optimal policies with the raw log-return objective whenever market dynamics do not depend on the policy (the price-taker assumption), and reduces the variance of the gradient estimate because the market component is subtracted out. In the paper's experiments, DDPG trained with t","pith_inferences":["Because the alpha reward shares optimal policies with the raw log-return reward, a direct ablation comparing two otherwise identical agents — one trained with each reward — would isolate whether the empirical gains come from variance reduction, and this comparison should be run across multiple seeds.","The paper evaluates without transaction costs at test time; re-running the same checkpoints with the 0.2% cost applied at inference would test whether the reported outperformance survives realistic frictions, which the authors themselves flag as a limitation.","The sentiment score's marginal contribution is untested; a natural follow-up is to retrain with sentiment features zeroed out, or replaced by a lexicon baseline, to see whether the results are driven by price-derived indicators rather than news.","The regime-shift finding suggests a practical selection rule: choose the model with the best validation Sharpe across multiple non-overlapping windows spanning both bull and bear phases, rather than a single recent window."],"forward_implications":["If the alpha reward is adopted, training objective and evaluation criterion coincide: an agent is graded on outperforming buy-and-hold, not on absolute profit, so a profitable market does not mask a bad policy.","The policy-equivalence result implies that any performance gain from the alpha reward in practice comes from variance reduction and better credit assignment, not from a changed optimum; algorithms that struggle with high-variance rewards should benefit most.","DDPG, a continuous-action actor-critic with replay, proved more stable than recurrent on-policy methods (PG, PPO) and value-based DQL, suggesting that off-policy stability matters more than temporal feature processing for this task.","Sharpe-ratio-based model selection on a single bull-market validation window does not guarantee test performance in a bear market; the paper's own results imply that walk-forward or regime-diverse validation is needed for reliable selection.","The framework should transfer to other long/flat/short trading tasks, since the alpha reward is asset-agnostic and needs only the asset's daily return as a benchmark."],"fun_headline_variants":["Alpha-reward RL nets 54.96% on TSLA, beats BTC buy-and-hold","RL trading with alpha reward turns TSLA into 54.96% gain","Alpha-reward DDPG beats buy-and-hold on TSLA and BTC","Excess-return RL turns BTC bear into +1.58%, TSLA up 54.96%","Alpha reward: train RL to beat market, not just predict it"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline results rest on a single realized test path evaluated without transaction costs, so if that path is unrepresentative or costs are reintroduced at inference, the ordering of agents and even the sign of the BTC return could change.","fun_headline_variants_meta":{"raw":{"variants":["Alpha-reward RL nets 54.96% on TSLA, beats BTC buy-and-hold","RL trading with alpha reward turns TSLA into 54.96% gain","Alpha-reward DDPG beats buy-and-hold on TSLA and BTC","Excess-return RL turns BTC bear into +1.58%, TSLA up 54.96%","Alpha reward: train RL to beat market, not just predict it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000986,"raw_usage":{"total_tokens":4090,"prompt_tokens":884,"completion_tokens":3206,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":628,"completion_tokens_details":{"reasoning_tokens":3094}},"tokens_in":628,"tokens_out":3206,"duration_ms":21549,"temperature":1.0,"reasoning_tokens":3094,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T21:32:57.406416+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the same trained policies on the test window with the 0.2% transaction cost applied at each position change (as it is during training) instead of zero cost. If DDPG's TSLA cumulative return drops below buy-and-hold, or its BTC return turns negative, the claim that the alpha-reward DDPG beats buy-and-hold is refuted. Alternatively, train a raw-log-return agent with identical seeds and hyperparameters; theory says the optimal policies coincide, so if the alpha-reward agent's test returns are not statistically distinguishable from the raw-reward agent's across several seeds, the variance","supporting_citations":[],"review_version":1}