{"id":"9e0b7703-f250-448c-8523-ab2c0e4e9328","arxiv_id":"2412.06794","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Topic-specific news sentiment for markets and politics is reported to improve NIFTY 50 index prediction, but the analysis has inconsistent baselines, no significance testing, and no released code or data.","lead":"This paper tests whether sentiment from different news topics (markets, politics, sports) can predict next-day NIFTY 50 index moves, using VADER on article bodies and DistilBERT on headlines. It reports that market and politics news sentiment improve ridge regression accuracy, but the evidence is inconsistent and lacks statistical significance tests.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Weekend forward-fill duplicates Friday closes as target values, so the RMSE gains in Tables 1 and 3 do not establish a significant sentiment impact on actual NIFTY 50 movement.","rationale":"The central claim requires evidence that topic-specific sentiment features improve prediction of real one-day-ahead NIFTY 50 movement beyond OHLC history. The forward-fill step in Section IV violates that target definition by assigning Friday's Close to Saturday and Sunday rows, which are then scored as successful predictions when the model copies the previous Close. Because the same leakage is present in both the with-sentiment and without-sentiment models, it can distort the RMSE comparison in either direction; the paper does not report any significance test that would let a reader tell whether the observed gains are real. The absence of code and data makes it impossible to verify the numbers, and the paper's own conclusion is framed in causal 'impact' language that the predictive design cannot support. The reader's weakest assumption identifies exactly this forward-fill issue; I agree and would keep the REJECT verdict unchanged, though the confidence is low because a re-running of the analysis on trading days only could in principle show that the central claim survives.","tokens_in":680,"tokens_out":645,"duration_ms":111521,"concrete_test":"Re-run the full pipeline on trading days only: aggregate Saturday and Sunday sentiment into the next Monday's features and drop all weekend rows from training and test. Recompute the Ridge and Elastic Net RMSE comparisons from Tables 1 and 3 on this cleaned target, and form a block-bootstrap confidence interval for the with-versus-without sentiment RMSE difference. If the sentiment advantage disappears or falls within the interval, the weekend forward-fill is driving the claimed impact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV left-joins the sentiment dataset with the NIFTY 50 dataset and forward-fills the missing weekend Open/High/Low/Close values. This puts Saturday and Sunday rows in training and test with Close equal to the preceding Friday's Close. On those rows the lag-1 Close feature is exactly equal to the target, and the target is not a one-day-ahead market movement. A ridge model whose lag-1 coefficient is penalized toward zero can recruit weekend sentiment features as a proxy for the repeated Friday Close, so the with-sentiment RMSE advantage in Table 3 may be an artifact of the duplicated target rather than evidence that market or politics news moves the index. Separately, Tables 1 and 3 report only point RMSE values; there are no confidence intervals, permutation tests, or standard errors, so the word 'significant' in the abstract and conclusion is not statistically supported. The top-coefficient reading in Tables 2 and 4 also has no uncertainty quantification.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether topic-specific sentiment scores from news articles (VADER for full news texts, DistilBERT for headlines) can improve one-day-ahead prediction of the NIFTY 50 index. After scraping roughly 400,000 Economic Times articles from January 2021 to February 2024, the authors group sentiment by 22 topics, create lag-3 and lag-5 feature sets, and compare linear regression, Ridge, Lasso, and Elastic Net with and without sentiment features. They report RMSE values in Tables 1 and 3 and, from the top ridge coefficients in Tables 2 and 4, conclude that market and politics news have significant impact on index movement.","tokens_in":7142,"tokens_out":4358,"duration_ms":41225,"significance":"If established, the topic-specific sentiment decomposition would be a useful extension of news-driven index prediction, suggesting that political and market news should be weighted separately rather than pooled into an aggregate sentiment score. The paper's strengths are the large scraped corpus, the simple URL-based topic extraction, and the use of two sentiment models. However, the empirical support for the headline claim rests on point RMSE differences and unquantified ridge coefficients, and the manuscript contains no code or data availability statement. As detailed below, the weekend forward-filling and absence of significance testing mean the current evidence does not support the conclusion as stated.","major_comments":[{"comment":"The left join of the sentiment dataset with the NIFTY dataset, followed by forward-filling of Open, High, Low, and Close, places Saturday and Sunday rows in the training and test sets with Close equal to the preceding Friday's Close. On those rows, the lag-1 Close feature is exactly equal to the target, and the target is not a genuine one-day-ahead market movement. A ridge model can therefore recruit weekend sentiment features as a proxy for the repeated Friday Close, so the with-sentiment RMSE advantage in Tables 1 and 3 may be an artifact of the duplicated target rather than evidence that sentiment moves the index. The authors should repeat the analysis excluding weekend sentiment rows or aligning each news day's sentiment to the next trading day's return, and report whether the RMSE comparisons survive.","section":"Section IV, data preparation paragraph"},{"comment":"The paper uses the word 'significant' throughout (abstract, Section V) but reports only point RMSE values. There are no confidence intervals, permutation tests, or forecast-accuracy tests (e.g., Diebold-Mariano). Similarly, Tables 2 and 4 present the top ridge coefficients without standard errors, bootstrap intervals, or stability analysis; ranking point estimates of regularized coefficients does not establish that market and politics topics have a larger impact. Without uncertainty quantification, the central claim is not statistically supported.","section":"Abstract and conclusions vs. Tables 1 and 3"},{"comment":"The evidence is internally inconsistent. The 'without sentiment' baselines differ between the two tables for the same model (Ridge: 203.78 vs. 206.01; Linear: 134.85 vs. 137.76), and the VADER-based Elastic Net with sentiment gives RMSE 395.56 versus 174.81 without sentiment, which directly contradicts the text's assertion that regularized regression with sentiment scores 'produced better results' than without. The authors need to reconcile these numbers and specify exactly which baseline configuration is used in each comparison, and they should discuss why sentiment sometimes degrades performance.","section":"Tables 1 and 3"},{"comment":"The conclusion that market and politics news have 'significant impact' is derived by reading the signs and magnitudes of coefficients from the same fitted model that produced the RMSE. This is not an independent test of impact; a coefficient in a high-dimensional regularized regression is not a causal effect, and no ablation or external validation (e.g., removing or permuting one topic's sentiment and measuring the change in prediction error) is provided. The authors should supply a falsifiable test of topic-level impact rather than relying on the fitted coefficient ranking.","section":"Section IV, top-feature interpretation"}],"minor_comments":[{"comment":"The smoothing parameter alpha is said to have a typical value of 15, but no source is given; please provide a citation to the VADER formula or state the exact implementation used.","section":"Equation (1)"},{"comment":"The text says 'Total number of data points in each dataset was above 1000' but the period spans more than three years; given the September 2023 removal, the exact count should be stated precisely for reproducibility.","section":"Section IV, dataset description"},{"comment":"The figures are referenced in the text but no axis labels or caption-level RMSE values are provided, so the reader cannot verify the claimed close match from the figures.","section":"Figures 1 and 2"},{"comment":"The manuscript does not state whether the data or code will be made available; a data and code availability statement is needed for reproducibility.","section":"General"},{"comment":"There are several grammatical errors, including 'sentiment scores of news items of different other topics' in the abstract and 'the study was done on NIFTY 50 index to understand the impact' in Section I; these do not affect the science but should be corrected.","section":"Abstract and Section I"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the weekend forward-fill, which could invalidate the RMSE comparison entirely. If a reanalysis excluding weekend rows reverses the finding, the paper's contribution would not be established. Given the absence of code and the small number of tables, the authors should be asked to make the preprocessing code available and to rerun the entire pipeline on a trading-day calendar before the paper can be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Thanks for the note. My short take: this paper has a substantial dataset and a reasonable topic-sentiment pipeline, but the weekend forward-fill in Section IV undermines the main result, and 'significant' appears in the abstract without any statistical support.\n\nWhat's genuinely useful here: the authors scraped 400k+ Economic Times articles over three years, extracted 22 topics from URLs, and produced daily topic-sentiment features using VADER on full text and DistilBERT on headlines. That is real work. The decision to use regularized linear models makes sense for 78-130 features, and the tables suggest the pipeline runs end to end. The observation that market and politics topics have the largest ridge coefficients is interesting, if it holds up.\n\nThe problem is that it doesn't hold up under scrutiny. The left join of weekend news to the NIFTY dataset, followed by forward-filling Friday's Open/High/Low/Close, creates rows where the lag-1 Close feature is exactly equal to the target. Any ridge model can exploit that repeated value; sentiment features may just serve as a proxy for the duplicated Friday Close. So the RMSE improvements in Tables 1 and 3 are not evidence that market or politics news moves the index. This is a structural flaw, not a tuning issue.\n\nThe statistics are also thin. The abstract and conclusion use 'significant' without standard errors or confidence intervals, and the RMSE differences across models are point estimates only. On top of that, the 'without sentiment' baselines aren't consistent between tables: Linear Regression without sentiment is 134.85 in Table 1 but 137.76 in Table 3, and Elastic Net goes from 174.81 to 175.88. Either the datasets differ in ways not described, or the comparison isn't apples to apples. The authors also claim 'no such prior work' on this index, but the related work section is too thin to support that.\n\nWho is this for? Someone interested in topic-specific sentiment features for stock prediction might find the data-cleaning and feature-construction approach a useful starting point, but the current results don't demonstrate a real effect. I wouldn't cite this as evidence. If the authors re-run the analysis without the forward-fill (e.g., mapping weekend news to Monday's features only) and add proper significance testing, the core question could be answered properly. As submitted, I'd reject it. It's marginal for peer review; the flaw is fixable, but the current version doesn't support the headline claim.","headline":"A serious data-collection effort on topic-sentiment for NIFTY 50, but the weekend forward-fill and missing significance tests mean the 'impact' claim doesn't stand.","tokens_in":7590,"tokens_out":3526,"would_cite":false,"duration_ms":31361,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Topic-split news sentiment sharpens NIFTY 50 next-day forecasts.","keywords":["VADER","DistilBERT","Sentiment Analysis","NIFTY 50","Topic-based Sentiment","Ridge Regression","Stock Index Prediction","Web Scraping"],"falsifier":"Re-run the analysis with weekend news sentiments carried to the next trading day (Monday) before aligning with NIFTY data, instead of forward-filling Friday's OHLC values onto weekend rows, and check whether the RMSE advantage of sentiment features survives; if the advantage disappears or flips, the reported improvement is an artifact of the alignment.","tokens_in":6783,"feed_emoji":"📈","tokens_out":7172,"duration_ms":53502,"temperature":0.7,"pith_summary":"The paper claims that topic-specific news sentiment carries predictive information for the next-day movement of the NIFTY 50 stock index, and that the effect is not confined to market news: political and national news also appear among the strongest predictors. It tests this by scraping over 400,000 Economic Times articles from January 2021 to February 2024, scoring full articles with VADER and headlines with DistilBERT, then training linear models on lagged sentiment features. The central comparison is RMSE of predicting the closing value with and without sentiment features; Ridge regression with DistilBERT headline sentiments reduces test RMSE from 206.01 to 128.74. The authors read the largest fitted coefficients as evidence that 'market' and 'politics_and_nation' topics have the greatest influence on index movement. The study is explicitly limited to NIFTY 50 and warns against generalizing the findings to other indices or stocks.","feed_headline":"Topic-split news sentiment sharpens NIFTY 50 next-day forecasts","feed_subtitle":"Ridge regression with DistilBERT headline scores cut test RMSE from 206 to 129; market and politics news lead the coefficients.","key_machinery":"The central mechanism is a topic-conditioned sentiment pipeline: each day's news articles are assigned to one of 22 topics by parsing the article URL, headline and full-text sentiments are scored separately (DistilBERT for headlines, VADER for full text), daily per-topic scores are left-joined to NIFTY 50 OHLC data with forward-fill for weekends, and lagged versions of all features are fed to regularized linear regression. The comparative machinery is RMSE on a held-out test period with and without sentiment features, with the largest positive Ridge coefficients used to attribute influence to specific topics.","core_discovery":"On the paper's own terms, the discovery is that topic-conditioned sentiment scores measurably improve one-day-ahead forecasts of the NIFTY 50 closing value, and that the improvement is largest when headline sentiments come from DistilBERT rather than VADER. In the best configuration (Ridge regression, lag 3, DistilBERT headline scores), the test RMSE drops from 206.01 without sentiment features to 128.74 with them. The fitted coefficients place market lag features and politics_and_nation lag features among the top positive contributors, which the authors take as evidence that both market news and political/national news significantly affect index movement. The paper frames this as closing a gap: prior work used overall sentiment or stock-specific news, while this study disaggregates sentiment by 22 news topics.","pith_inferences":["If the forward-fill alignment is corrected, the reported RMSE gap may shrink, because weekend rows pair unchanged Friday closes with weekend sentiment, giving the model a near-free 'no change' anchor that the no-sentiment baseline does not have.","The method of inferring significance from the magnitude of Ridge coefficients, without confidence intervals or a significance test, leaves open the possibility that other topics with smaller coefficients are equally predictive; an ablation that removes market/politics topics while retraining would test the attribution.","The same topic-conditioned pipeline could be extended to other indices and to intraday data, where weekend alignment issues disappear and the impact of political news on market open gaps could be measured directly."],"forward_implications":["Topic-specific news sentiment, not just overall sentiment, directly improves one-day-ahead forecasts of NIFTY 50 closes, with the largest gains from headline-level DistilBERT scores.","News about markets and about politics/nation carry the strongest positive influence on index movement, as shown by the top Ridge coefficients.","Regularized linear models such as Ridge with lag-3 sentiment features produce better test RMSE than no-sentiment baselines, while Lasso performs poorly in this setting.","The findings apply to NIFTY 50 in the study period and should not be generalized to other indices or stocks without fresh analysis."],"supporting_citations":[{"why":"Supplies the VADER lexicon-based sentiment analyzer used to score the full text of each news article.","marker":"[15]"},{"why":"Supplies DistilBERT, the transformer model used to score the sentiment of news headlines.","marker":"[18]"},{"why":"Defines BERT, the model from which DistilBERT is distilled, used to justify DistilBERT's contextual understanding.","marker":"[17]"},{"why":"An earlier sentiment-aware stock prediction method that used overall sentiment, which this study claims to extend with topic-level scores.","marker":"[3]"},{"why":"An earlier study incorporating news sentiment with stock prices for prediction, serving as a baseline that this study builds on.","marker":"[4]"},{"why":"Establishes the role of investor sentiment in stock market volatility, the theoretical motivation for including news sentiment.","marker":"[1]"}],"fun_headline_variants":["Topic-split news sentiment improves NIFTY 50 forecasts","Politics and market news drive NIFTY 50 sentiment edge","DistilBERT topic sentiment beats VADER for NIFTY 50","News topics beyond markets predict NIFTY 50 moves"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on forward-filling Friday's 'Open', 'High', 'Low', and 'Close' values onto Saturday and Sunday, so that weekend news sentiment is aligned with an unchanged Friday target, which can create predictive signal that would not exist if news were matched to the next trading day's actual movement.","fun_headline_variants_meta":{"raw":{"variants":["Topic-split news sentiment improves NIFTY 50 forecasts","Politics and market news drive NIFTY 50 sentiment edge","DistilBERT topic sentiment beats VADER for NIFTY 50","News topics beyond markets predict NIFTY 50 moves"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1695,"prompt_tokens":878,"completion_tokens":817,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":744}},"tokens_in":494,"tokens_out":817,"duration_ms":18443,"temperature":1.0,"reasoning_tokens":744,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:55:42.372547+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the analysis with weekend news sentiments carried to the next trading day (Monday) before aligning with NIFTY data, instead of forward-filling Friday's OHLC values onto weekend rows, and check whether the RMSE advantage of sentiment features survives; if the advantage disappears or flips, the reported improvement is an artifact of the alignment.","supporting_citations":[{"cited_title":"Vader: A parsimonious rule-based model for sentiment analysis of social media text,","cited_arxiv_id":null,"evidence_quote":"Supplies the VADER lexicon-based sentiment analyzer used to score the full text of each news article."},{"cited_title":"Sentiment -aware stock market prediction: A deep learning method,","cited_arxiv_id":null,"evidence_quote":"An earlier sentiment-aware stock prediction method that used overall sentiment, which this study claims to extend with topic-level scores."},{"cited_title":"Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong,","cited_arxiv_id":null,"evidence_quote":"An earlier study incorporating news sentiment with stock prices for prediction, serving as a baseline that this study builds on."},{"cited_title":"Stock market volatility, excess returns, and the role of investor sentiment,","cited_arxiv_id":null,"evidence_quote":"Establishes the role of investor sentiment in stock market volatility, the theoretical motivation for including news sentiment."}],"review_version":1}