{"id":"2647d3a4-ea1d-45b0-8d25-8595bd8a12ab","arxiv_id":"2507.03350","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A regression-based financial news sentiment model, used as the only trading signal on Dow Jones 30 stocks, backtests to a 50.63% cumulative return and 23.67% alpha over 28 months, but the result relies on in-sample threshold tuning and a single market period.","lead":"This paper backtests three news-sentiment models as standalone trading signals on Dow Jones 30 stocks over 28 months and reports that a regression-based model earned a 50.63% cumulative return, beating a buy-and-hold benchmark. It matters as a practical test of whether sentiment analysis alone can generate trading alpha, though the result rests on an in-sample backtest with tuned parameters.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never explains how articles are assigned to the daily sentiment window; if timestamps are missing or inferred from crawl times, the 50.63% return and 23.67% alpha may be timing artifacts.","rationale":"The strongest claim in the paper is that a sentiment-only strategy built on RoBERTa+Transfs generated a 50.63% cumulative return and 23.67% alpha over 28 months. For that claim to hold, the backtest must be temporally well-posed: each news article must be assigned to the correct pre-market window so that the buy/sell/neutral order is based only on information available before the market opens. Section 3.1.1 says the articles are 'raw and contain no meta-information,' which directly conflicts with the daily aggregation described in Section 3.1.3. This is not a minor implementation detail; if article dates are missing or inferred from crawl order, the entire signal-generation process is ill-defined and the backtest cannot be validated. The reader's weakest assumption identifies exactly this issue, and the paper provides no counter-evidence: no timestamp extraction procedure, no date distribution, no sensitivity analysis, and no code or data release. Confirming this concern does not require re-deriving the financial metrics; it requires only that the authors make the timestamp provenance explicit. Because this concern undermines the central claim, the reader's REJECT verdict remains appropriate. Other problems—in-sample threshold selection (Section 3.1.4), the unequal capital deployment of the equal-value strategy versus the fully invested Buy&Hold benchmark, and internal inconsistencies in Table 1 (e.g., a 26.96% cumulative return cannot correspond to a 23.44% annual compound return over 28 months)—further weaken the paper, but the timestamp issue is the most load-bearing because it threatens the validity of every number in the backtest. No independent support (released code, reproducible pipeline, machine-checked proof, or out-of-sample validation) is offered to offset this gap. I therefore see no reason to alter the reader's rejection, and I agree that the timestamp assumption is the weakest point in the argument.","tokens_in":11208,"tokens_out":4940,"duration_ms":55840,"concrete_test":"Obtain from the authors the exact timestamp or date-assignment procedure for each article in the corpus. Re-run the Section 3.1.3 daily aggregation and the Section 3.1.4 trading rules under three conditions: (i) true publication timestamps, (ii) crawl/ingestion timestamps, and (iii) timestamps shifted by one trading day. If condition (i) does not reproduce the reported 50.63% cumulative return and 23.67% alpha, or if conditions (ii) or (iii) change the cumulative return materially, the headline result is a timing artifact rather than evidence of sentiment alpha.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1.1 states that the collected articles 'are raw and contain no meta-information,' yet Section 3.1.3 aggregates 'sentiment scores derived from articles published from the previous market open day to the current market open day (i.e., 9:30 a.m. local time).' The paper gives no method for obtaining publication timestamps or for assigning each of the 1,332,446 articles to a specific trading window. Without timestamp provenance, the central mechanism of the strategy—producing a daily sentiment score before the market opens—is undefined. If timestamps were absent, or if they were inferred from crawl dates or file ordering, the backtest could mix look-ahead signals (news that arrived after the market open used in the same day's decision) with stale signals (old news used days later). Either error can produce spurious profits, especially in a high-frequency daily strategy. Figure 3 shows a time axis for article volume, so some date information exists, but its source is never stated and no reproducibility check is provided. This is the load-bearing step in the pipeline: every reported return, drawdown, and ratio in Table 1 depends on correctly dated news. The absence of a timestamp description makes the headline result non-reproducible and the central claim unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a backtesting framework in which sentiment scores from three BERT-based models (FinBERT, DualGCN, and RoBERTa+Transfs) are computed for news articles on Dow Jones 30 stocks between January 2020 and April 2022, aggregated into a daily score per stock, and converted into buy, neutral, or sell orders using thresholds. The authors compare the resulting equal-value portfolio with a Buy&Hold benchmark and report that the regression model RoBERTa+Transfs achieves a 50.63% cumulative return and a 23.67% alpha with lower drawdown and volatility than Buy&Hold. The paper interprets this as evidence that news sentiment can generate profitable, low-risk trading strategies.","tokens_in":11524,"tokens_out":7402,"duration_ms":83982,"significance":"If the results were valid, the paper would provide useful evidence on the practical value of continuous sentiment scores relative to discrete classification, and its comparison of three models on a large news corpus is a sensible design. The paper also cites the look-ahead bias literature, showing awareness of backtest pitfalls. However, the current manuscript does not establish the central claim because of missing timestamp provenance, in-sample threshold selection, and a benchmark comparison that is distorted by idle cash; the reported metrics also contain internal inconsistencies. These issues are load-bearing rather than cosmetic.","major_comments":[{"comment":"The paper states that articles 'are raw and contain no meta-information' (Section 3.1.1) but then defines the daily sentiment score using 'articles published from the previous market open day to the current market open day' (Section 3.1.3). The manuscript never explains how publication timestamps were obtained for 1,332,446 articles or how each article was assigned to a 24-hour trading window. Without this provenance, the backtest is not reproducible and may contain look-ahead or stale signals; every return in Table 1 depends on this step. Figure 3 plots article volume over time, so some date information exists, but its source is not stated.","section":"Section 3.1.1 and 3.1.3"},{"comment":"The thresholds SELL_SIGNAL=40 and BUY_SIGNAL=60 are selected by testing 45/55, 40/60, and 35/65 on the same backtest period. The reported 50.63% cumulative return and 23.67% alpha are therefore conditional on parameters fitted in-sample. There is no held-out period, walk-forward analysis, or sensitivity check to show that the strategy would have performed similarly out of sample. The central claim of alpha generation is not supported without such validation.","section":"Section 3.1.4"},{"comment":"The 'Annual Compound Return' column in Table 1 is inconsistent with the paper's own definitions and with the cumulative returns. For Buy&Hold, a 26.96% cumulative return over 28 months implies a CAGR of about 10.7%, not 23.44%; the 23.44% figure equals the product of the three calendar-year returns reported in Section 4.2 (9.98%, 18.24%, -5.07%). Similarly, RoBERTa+Transfs' 65.26% is the compounded three-year return, not the annual compound return defined by Eq. (2). The table's metric names need to be corrected or the computations need to be revised.","section":"Table 1 and Appendix C, Eq. (2)"},{"comment":"The equal-value strategy leaves most capital idle ('most of the time, our money will be idle'), so the sentiment portfolios are not fully invested. Their lower volatility, lower drawdown, and high Sharpe and Sortino ratios partly reflect holding cash, not superior signal quality. Comparing the cumulative return of a partially invested strategy with a fully invested Buy&Hold benchmark and calling the difference 'alpha' is not an apples-to-apples comparison. The authors should report a fully invested variant, such as scaling orders to deploy all capital, and report the average gross exposure of the sentiment strategy.","section":"Section 3.1.4 and Table 1"},{"comment":"Alpha is defined as the simple difference between portfolio return and benchmark return. This is not the usual risk-adjusted alpha; it ignores beta, timing, and statistical significance. Given the idle-cash issue, the reported alpha of 23.67% is not evidence of trading skill. A regression-based alpha with a significance test, or at least an information ratio, is needed to support the claim that the sentiment strategy generates positive risk-adjusted returns.","section":"Section 4.2"}],"minor_comments":[{"comment":"The header contains a typo: 'Annual V olatility' should be 'Annual Volatility'.","section":"Table 1"},{"comment":"The conclusion writes 'alpha of 23,67%' with a comma as the decimal separator, which is inconsistent with the rest of the paper.","section":"Section 5"},{"comment":"The paper does not state whether code and data are available. Given the backtest's dependence on a proprietary news corpus, a detailed data description or reproducibility appendix is essential.","section":"General"},{"comment":"The claim that floating-point predictions are superior is based on a single regression model compared with two classification models; confounding factors such as training data and model architecture are not controlled, so the comparison is only suggestive.","section":"Section 4.2"},{"comment":"Some references are incomplete or informal, such as Bird et al. (2023), described only as 'Rapport interne', and Glasserman & Lin (2023), which lacks publication venue details.","section":"References"},{"comment":"The discussion of monthly returns is qualitative; a table of yearly returns or monthly return statistics would make it easier to verify the claims about 2020, 2021, and 2022 performance.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and timely question, but the empirical evidence is currently not convincing. The most serious concern is the missing timestamp provenance: if the authors cannot document how each article was assigned to a daily trading window, the backtest is not reproducible and the headline result should not be published. The in-sample threshold selection and the idle-cash benchmark comparison also require substantial reanalysis. I would be willing to revisit a revised version that provides out-of-sample validation, a fully invested comparison, and corrected metrics, but the current manuscript is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely worth doing: it compares two classification sentiment models and one regression model in a solo-sentiment trading backtest on a large corpus of Dow Jones 30 news, and it reports that the continuous-valued model beats the discrete ones. That comparison is a legitimate extension of prior work, and the finding that continuous scores outperform hard classes is plausible and worth testing. The corpus is large, the pipeline is described step by step, and the authors cite Glasserman and Lin on look-ahead bias, so they are aware of the methodological hazards.\n\nBut the central result is not currently supported. The most serious problem is timing. Section 3.1.1 says the articles are raw and contain no meta-information, yet Section 3.1.3 aggregates sentiment over articles published between the previous and current market open. Without publication timestamps, the daily sentiment score is undefined. If timestamps were inferred from crawl dates or file order, the backtest could be mixing look-ahead and stale signals, which can manufacture spurious profits. The paper never explains this, and Figure 3 only shows a date axis for volume, not the source of the dates.\n\nSecond, the BUY_SIGNAL and SELL_SIGNAL thresholds were chosen by testing on the same backtest period (Section 3.1.4). That is in-sample tuning, and the reported alpha is conditional on that choice. Third, the comparison is skewed: the sentiment strategy uses an equal-value approach that leaves cash idle, so its lower drawdown and volatility are partly a cash-drag artifact, and the 'alpha' is just the difference in cumulative returns, not a risk-adjusted measure.\n\nFourth, the metrics in Table 1 are internally inconsistent. A 50.63% cumulative return over 28 months implies an annual compound return of roughly 19%, not the reported 65.26%. The annual return of 23.15% is also incompatible with a 65.26% compound figure. These numbers cannot all be correct, which undermines confidence in the other reported metrics.\n\nThis is a paper with a useful question and a plausible hypothesis, but the evidence as presented does not establish the claim. The timestamp provenance issue is load-bearing and may be unfixable if the data are truly raw. Still, I think this deserves peer review rather than desk rejection, because a serious referee can force the authors to clarify the timing, move threshold selection out-of-sample, and fix the metrics. The topic matters to financial NLP, and the classification-versus-regression comparison is worth getting right. My own verdict would be reject with major revision, but it is a paper worth engaging with.","headline":"A well-motivated comparison of sentiment models for trading that is undermined by missing timestamp provenance, in-sample threshold tuning, and internally inconsistent performance metrics.","tokens_in":12035,"tokens_out":2223,"would_cite":false,"duration_ms":26789,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sentiment-only trading strategy generated 50.63% over 28 months on Dow 30 stocks, the paper reports.","keywords":["sentiment analysis","trading strategy","financial backtesting","alpha generation","Dow Jones 30","FinBERT","RoBERTa+Transfs","regression sentiment"],"falsifier":"Recover true publication timestamps for the 1,332,446 articles and rerun the daily aggregation and orders; if the 50.63% return and 23.67% alpha cannot be reproduced with correctly timestamped articles, the headline result was an artifact of the missing meta-information. A companion test is to shuffle the aggregated sentiment scores across stocks and days and check whether the strategy's returns still exceed the shuffled distribution.","tokens_in":11083,"feed_emoji":"📈","tokens_out":8465,"duration_ms":93820,"temperature":0.7,"pith_summary":"The paper asks whether news sentiment alone, without price or technical data, can drive profitable trading decisions. It backtests daily sentiment scores for Dow Jones 30 stocks from January 2020 to April 2022, using three BERT-based models to turn news articles into buy, neutral, or sell orders. The regression model RoBERTa+Transfs reached a 50.63% cumulative return over 28 months, beating the 26.96% Buy&Hold benchmark and producing a 23.67% alpha with a much smaller maximum drawdown. The paper's conclusion is that continuous sentiment scores are a viable standalone input for alpha generation and that three-class sentiment labels are too coarse to beat the benchmark. If right, this makes sentiment analysis a practical component of investment strategies rather than just an accuracy benchmark task.","feed_headline":"Sentiment-only trading beat Buy&Hold by 23.67% in backtest","feed_subtitle":"A continuous sentiment model returned 50.63% on Dow 30 stocks over 28 months with lower maximum drawdown than the index.","key_machinery":"The load-bearing mechanism is the daily per-asset sentiment aggregation and the threshold-triggered order rule. Each morning, sentiment scores from articles published between the previous and current market open are averaged for each Dow 30 company and rescaled to a 0-100 range; aggregated scores above 60 generate buy orders, scores below 40 generate sell orders, and scores in between generate neutral or position-closing orders. Orders are executed at the opening price with equal dollar amounts and a 0.05% commission. The regression model RoBERTa+Transfs supplies continuous values in [-1, +1], and the paper argues that this continuous resolution is what lets the thresholds separate tradable signals, unlike the three-class outputs of FinBERT and DualGCN.","core_discovery":"The paper's central discovery is that a sentiment-based trading system can generate positive alpha when sentiment is predicted as a continuous value. Over the backtest period, the RoBERTa+Transfs regression model produced a 50.63% cumulative return versus 26.96% for Buy&Hold, an alpha of 23.67%, and a maximum drawdown of -5.27% versus -32.38% for the index. The paper attributes the gap to the floating-point predictions, which capture degrees of positivity and negativity, while the two classification models (FinBERT and DualGCN) collapse sentiment into three labels and end up with negative alpha (-1.73% and -11.47%). The intended message is that news sentiment, aggregated daily and used as the sole criterion for market entry, can create a profitable strategy with relatively low risk on large-cap stocks.","pith_inferences":["A natural control the paper does not run is to shuffle sentiment signals across days and stocks; if shuffled strategies produce similar returns, the headline alpha is a property of the backtest rule rather than of sentiment content.","Because the corpus is described as lacking publication timestamps, a follow-up study should date each article and re-run the backtest; until then, the 50.63% return should be read as conditional on the daily-window assignment actually used.","The 40/60 thresholds were selected by inspecting the same backtest period, so the reported alpha likely includes selection bias; an out-of-sample threshold choice is a direct robustness test.","If continuous sentiment works this well on Dow 30 megacaps, the natural next test is high-momentum or small-cap stocks, where news coverage is sparser and sentiment signals may be noisier."],"forward_implications":["Under the paper's claims, an NLP model's sentiment score is a tradable signal on its own, without price, volume, or macroeconomic inputs.","Regression-style sentiment scoring should be preferred over three-class classification in trading pipelines because the extra granularity changes whether alpha appears.","The reported risk metrics suggest that sentiment-driven daily rebalancing can cut maximum drawdown relative to passive holding, even during the Covid crash and the 2022 selloff.","The same aggregation-plus-threshold evaluation protocol can be applied to any new sentiment model to decide whether its scores are actionable before deployment."],"supporting_citations":[{"why":"Supplies the analytical framework for comparing sentiment-analysis methods in finance that the three-model design builds on.","marker":"(Mishev et al., 2020)"},{"why":"Provides the FinBERT pre-trained model used as the foundational classification model and as the tokenizer for DualGCN.","marker":"(Araci, 2019)"},{"why":"Defines the RoBERTa+Transfs regression model and its training procedure, which produces the continuous sentiment scores behind the best result.","marker":"(Linhares Pontes & Benjannet, 2021)"},{"why":"Provides the SemEval-2017 task #5 financial headlines with fine-grained sentiment labels used to train the DualGCN model.","marker":"(Cortis et al., 2017)"},{"why":"Supplies the BERT-CRF entity recognizer that maps articles to Dow 30 company names, making per-asset sentiment aggregation possible.","marker":"(Souza et al., 2019)"},{"why":"An earlier sentiment-driven trading strategy that the paper positions as related work and extends by evaluating multiple sentiment models over a longer period.","marker":"(Kazemian et al., 2016)"},{"why":"Identifies look-ahead and distraction biases in sentiment backtests, which motivates checking the daily-window timing assumptions.","marker":"(Glasserman & Lin, 2023)"}],"fun_headline_variants":["Sentiment regression yields 50.63% return, beating Buy&Hold by 23.67%","Continuous sentiment, not labels, drives alpha: 23.67% over Buy&Hold","Sentiment-only trading with lower drawdown beats index by 23.67%","Regression sentiment model: 50.63% return, -5.27% max drawdown","Alpha from news sentiment: regression beats classification, 23.67%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole strategy rests on assigning each article to the correct daily window between the previous market open and the current market open, but the corpus is described as raw and lacking publication timestamps, so that assignment has no documented basis.","fun_headline_variants_meta":{"raw":{"variants":["Sentiment regression yields 50.63% return, beating Buy&Hold by 23.67%","Continuous sentiment, not labels, drives alpha: 23.67% over Buy&Hold","Sentiment-only trading with lower drawdown beats index by 23.67%","Regression sentiment model: 50.63% return, -5.27% max drawdown","Alpha from news sentiment: regression beats classification, 23.67%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000778,"raw_usage":{"total_tokens":3398,"prompt_tokens":859,"completion_tokens":2539,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":2425}},"tokens_in":475,"tokens_out":2539,"duration_ms":19688,"temperature":1.0,"reasoning_tokens":2425,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:13:15.269187+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recover true publication timestamps for the 1,332,446 articles and rerun the daily aggregation and orders; if the 50.63% return and 23.67% alpha cannot be reproduced with correctly timestamped articles, the headline result was an artifact of the missing meta-information. A companion test is to shuffle the aggregated sentiment scores across stocks and days and check whether the strategy's returns still exceed the shuffled distribution.","supporting_citations":[],"review_version":1}