{"id":"92dbe1cc-1e40-481a-9fcf-eb3d6de76378","arxiv_id":"1909.00344","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Using machine learning on two volatile stocks over three months, the paper finds that stock price movements predict news sentiment direction more strongly than news predicts stocks.","lead":"This paper tests whether stock prices drive financial news or news drives stock prices, using three months of data on North Korea-related stocks and Tesla. It reports that stock price changes predict news sentiment better than news sentiment predicts stock prices.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported correlations cannot support the paper's directional claim: no lead-lag test is performed, and preprocessing choices were selected on the same stock data, making Table 1 an in-sample comparison.","rationale":"The reader's weakest-assumption concerns the sentiment measure being data-tuned; my concern is broader and, I think, more load-bearing: even with a perfect sentiment measure, the experimental design does not test the paper's directional, lead-lag claim. The stockToArticle and articleToStock comparisons are not lagged prediction experiments, and the model-selection choices are made on the same target data, so the reported correlations cannot support 'bigger impact' or 'responded before.' This does not change the reader's rejection, since the rejection already follows from the absence of a valid causal or lead-lag test. The paper does contain some genuine effort in data collection and in running five classifiers, but no independent verification, confidence intervals, or out-of-sample procedure is provided. The central claim requires a temporal test that the presented table does not supply.","tokens_in":5080,"tokens_out":3840,"duration_ms":36871,"concrete_test":"Re-run the pipeline with the sentiment lexicon and X-window fixed before any contact with the target data (e.g., use the UIC list alone and a single lag structure such as stock price change on day t-1 predicting sentiment change on day t, and vice versa), evaluated with blocked time-series cross-validation. Then compute lagged correlations with bootstrap confidence intervals for stock(t-k) -> sentiment(t) and sentiment(t-k) -> stock(t) for k=1,...,5. If the stockToArticle advantage over articleToStock disappears or is within noise under this fixed, out-of-sample design, the directional conclusion is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is 'stock prices have a bigger impact on the news contents than news does on stock prices,' with the conclusion that stocks respond to social issues before articles do. The load-bearing inference is from Table 1's correlations to this directional asymmetry. That inference requires that the stockToArticle and articleToStock correlations be comparable measures of predictive influence with a temporal ordering. Neither condition is met. In the 'Dataset Generation and Machine Learning' section, the chosen X ranges include [base date - 5 days, base date + 1 day], so the stockToArticle model can use stock prices one day in the future to predict same-day sentiment; neither split is a clean lagged prediction. The reported correlations are contemporaneous associations of same-day change rates, not evidence that prices moved before articles. Moreover, the comparison is contaminated by selection: the synonym-similarity method, the sentiment lexicon, and the X-window length were each chosen after measuring correlation with the same stock-price data, and the chosen window is the one with the highest average correlation (0.129, vs -0.016 and -0.093). Table 1 therefore reports post-selection, in-sample correlations with no confidence intervals or out-of-sample evaluation. The directional conclusion would require a fixed, pre-specified measure and lagged cross-correlation or Granger-style analysis; as presented, the abstract's 'bigger impact' claim does not follow.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes the relationship between stock price changes and financial news sentiment for North Korea-related stocks and Tesla, Inc. over roughly three months in 2019. Using daily stock prices and sentiment scores computed from keyword-filtered news articles, the authors apply five machine learning models (GPs, linear regression, MLP, SMOreg, random forest) in both regression and classification settings, and report correlations between predicted and actual change rates. From these correlations, shown in Table 1, the paper concludes that stock prices have a bigger impact on news contents than news has on stock prices, and that stock prices respond to social issues before news articles do.","tokens_in":5380,"tokens_out":2772,"duration_ms":23852,"significance":"If the central claim were supported, the finding would be noteworthy because it contradicts the common assumption that financial news drives stock prices, and it would suggest that prices incorporate information before published news. The paper is also commendable for using a transparent and reproducible data-collection pipeline, multiple classifiers, and public APIs. However, the reported correlations do not constitute a valid test of directional influence, the preprocessing choices are selected on the same data used for evaluation, and no uncertainty quantification is provided. As presented, the evidence is too weak to support the abstract's conclusion.","major_comments":[{"comment":"The X ranges include both past and future values: [base date - 5 days, base date + 1 day]. For stockToArticle, X includes stock prices one day after the sentiment date; for articleToStock, X includes articles one day after the stock date. These experiments therefore contain look-ahead information and cannot support a directional or lead-lag conclusion. The only clean experiments are the \"base date\" rows, which are contemporaneous associations, not evidence that one series moves before the other. The claim that stock prices 'responded before' news requires a proper lagged cross-correlation or Granger-style analysis, which is absent.","section":"Dataset Generation and Machine Learning; Table 1"},{"comment":"Several key methodological choices are selected based on in-sample correlation with the same stock-price data: the cosine similarity measure is chosen because it yielded 0.129 versus -0.068 for WordNet; the UIC sentiment list is chosen because it yielded 0.129 versus 0.018 for UIC+Stanford; and the X-window [base-5, base+1] is chosen because it yielded the highest average correlation (0.129, versus -0.016 and -0.093). No separate validation set or nested cross-validation is used. Consequently, the correlations reported in Table 1 are post-selection, in-sample statistics, and the apparent advantage of stockToArticle over articleToStock may be an artifact of tuning to the same data.","section":"Candidate Extraction and Synonym Detection; Important Term Identification and Sentiment Analysis; Dataset Generation…"},{"comment":"The test set consists of the final approximately 34% of a roughly three-month dataset, i.e., around 22 trading days per asset. No confidence intervals, significance tests, or effect-size measures are reported for the correlations in Table 1. Many reported averages (e.g., 0.129, -0.016, 0.014) are close to zero and would not be statistically distinguishable from noise at this sample size. Without uncertainty quantification, the claim that stock prices have a 'bigger impact' than news is not supported.","section":"Results and Analysis"},{"comment":"The one-day shift added to North Korea article dates is an arbitrary alignment choice that directly affects the computed correlations and any implied causal ordering. The manuscript does not test whether the results are robust to this shift or to alternative alignments. Since the directional claim depends on timing, this unvalidated alignment is a load-bearing assumption.","section":"News and Stock Data Collection"}],"minor_comments":[{"comment":"The sentence 'we filtered out an article if the keyword was mentioned less than once' is imprecise; presumably the intended meaning is that articles with zero keyword mentions were filtered out.","section":"Preprocessing"},{"comment":"The paper states that articles with Levenshtein similarity above 80% were removed, but Levenshtein distance is not a normalized similarity measure; a more precise description of the similarity definition would improve reproducibility.","section":"Preprocessing"},{"comment":"The statement 'stock prices responded to social issues before the articles do' is a stronger causal-temporal claim than the experiments can support; it would be better phrased as a hypothesis for future work.","section":"Concluding Remarks"},{"comment":"Figure 2 is described as comparing stock price and sentiment index variation, but the caption and text do not indicate the time axis or units clearly; adding axis labels and a legend would improve readability.","section":"Figure 2"}],"recommendation":"reject","confidential_remarks":"The paper is not yet at the standard for a serious journal. The core conclusion is based on correlations that are not valid evidence of directional influence, and the reported results are likely inflated by selection on the same data. A full re-analysis with pre-specified methods, proper lagged tests, and significance testing would be needed to assess the claim; such a revision is beyond a normal minor-revision cycle."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper asks a genuinely useful question—whether news moves stocks or stocks move news—and it's nice to see both directions tested on real articles for two volatile assets. But the core result is not supported. The X-range for both directions includes base date plus one future day, so the 'stockToArticle' comparison isn't actually a lead-lag test. Both directions can peek at the future. And the preprocessing was tuned on the same stock-price data: the synonym-similarity method, the sentiment lexicon, and the input window were each chosen because they gave higher correlations on the same sample. Table 1 therefore reports post-selection, in-sample numbers, with no confidence intervals and a test set of about 22 days.\n\nThe paper's strongest point is the setup: collecting articles via NewsAPI, filtering by keywords, using multiple classifiers, and reporting both directions is a reasonable starting point. The authors also acknowledge the short sample. But the overreach is in the abstract and conclusion: they say stock prices have 'a bigger impact' on news, when all they have is a set of contemporaneous correlations, some of which are higher in one direction. The classification results look inflated—stockToArticle correlations of 0.8+ on Tesla—because the models are almost certainly memorizing the contemporaneous relationship with future data in the features.\n\nI don't think this is a case of bad faith; it reads like an exploratory exercise that got over-interpreted. But as it stands, the central claim doesn't follow. A reader could salvage it by redoing the analysis with a proper lagged design, pre-specifying the preprocessing, and reporting significance. As is, it's not a serious candidate for acceptance. If the authors can fix the look-ahead and selection issues, the question is worth a revisit. For now, I'd treat it as a cautionary example rather than a substantive result.","headline":"A sensible research question undermined by look-ahead bias and post-selection; the directional conclusion doesn't follow from the data.","tokens_in":5882,"tokens_out":4324,"would_cite":false,"duration_ms":38821,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that, for the North Korea-related and Tesla stocks studied, stock prices have a larger impact on financial-news sentiment than news sentiment has on stock prices.","keywords":["stock price prediction","news sentiment analysis","financial news","time series analysis","sentiment lexicon","machine learning classifiers","Tesla stock","North Korea-related stocks"],"falsifier":"Run the same five-model comparison on a rebuilt dataset with all available articles for the same dates rather than the top 25, or apply a Granger-causality test to the daily series: the paper's claim predicts that stock prices at date t help predict sentiment at date t+1 while sentiment at t does not help predict prices at t+1. If the reverse pattern appears, the directional conclusion fails.","tokens_in":4900,"feed_emoji":"📈","tokens_out":8733,"duration_ms":79434,"temperature":0.7,"pith_summary":"Individual investors often treat financial news as a guide to the stock market, but this paper asks whether the arrow points the other way: do prices move first and the news merely catch up? For roughly three months of articles about North Korea-related stocks and Tesla, the authors score each day's news tone by counting positive minus negative words, then compare two mirrored tasks with five machine-learning models. Predicting news sentiment from price changes and trading volume works far better than predicting price changes from sentiment and article count, with average classification correlations of 0.455 and 0.826 for the two stocks versus near-zero or negative correlations in the reverse direction. The paper concludes that stock prices have the bigger impact on news contents and that prices responded to social issues before the articles did.","feed_headline":"Stock prices shape news more than news shapes prices","feed_subtitle":"Five-model comparison on Tesla and North Korea stocks finds sentiment follows price moves, not vice versa.","key_machinery":"The central machinery is a mirrored pair of prediction tasks on the same daily time series. In article-to-stock, the independent variables are the change in sentiment and the number of articles, and the target is the stock price change; in stock-to-article, the independent variables are the price change and trading volume, and the target is the sentiment change. Both tasks are scored by the correlation between predicted and actual outcomes across five models. The sentiment score itself is produced by counting positive words minus negative words in adjective-noun candidates extracted from the top 25 keyword-filtered articles per day, after TF-IDF term weighting. The asymmetry between the two directions is what carries the argument.","core_discovery":"On its own terms, the discovery is a directional asymmetry in time-series prediction. The authors built a daily sentiment index from news articles about North Korea and Tesla, used five learning algorithms (Gaussian processes, linear regression, multilayer perceptron, support-vector regression, and random forest) to test both article-to-stock and stock-to-article prediction, and found that the stock-to-article direction consistently gave higher correlations, especially in classification. In the Tesla classification results, predicting sentiment direction from prices and volume reached an average correlation of 0.826, while predicting price direction from sentiment had a negative average. For North Korea the corresponding numbers were 0.455 and -0.077. The paper therefore concludes that stock prices have a greater impact on news sentiment than news sentiment has on stock prices, and that stock prices respond to social issues before the articles do.","pith_inferences":["An untested implication of the asymmetry is that prices aggregate dispersed private information faster than reporting can; if so, the gap should widen for unexpected events and shrink for scheduled announcements, which could be checked with an event-study design.","A natural extension is to rerun the same comparison on a broad market index over a longer period with Granger-causality or lead-lag tests; if the asymmetry persists, news-sentiment indices would serve as forecasts of reporting rather than of markets.","Because the sentiment measure was chosen after comparing its correlation with prices (0.129 vs 0.018), a replication that selects sentiment methods on a training period only would test how much of the asymmetry is an artifact of that selection."],"forward_implications":["News-sentiment-based stock prediction is not supported for volatile, event-driven stocks; the paper finds sentiment does not reliably predict the direction of next-day price moves.","Price movements and trading volume are useful for forecasting near-future news tone, so market data can be treated as an early signal of what will be reported.","Including past and future article data in the feature set can make stock prediction worse, so simply adding more news history is not a safe improvement.","Article count and trading volume help regression models but matter little for classifying direction, so their role depends on the prediction task."],"supporting_citations":[{"why":"Represents the common prior approach of predicting stock movements from news sentiment, which this paper contrasts with the reverse direction.","marker":"Kalyani et al., 2016"},{"why":"Supplies the sentiment word list used to score the tone of extracted article candidates.","marker":"Hu and Liu, 2004"},{"why":"Supplies the lexicon-based sentiment scoring formula of counting positive words minus negative words.","marker":"Taj et al., 2019"}],"fun_headline_variants":["Stock prices steer news sentiment, not the reverse","News follows stocks, study of Tesla and North Korea shows","In market-news tussle, prices dominate sentiment","Stock moves predict headlines better than headlines predict stocks","Financial news trails behind stock price shifts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis stands or falls on the assumption that counting positive and negative words in the top 25 keyword-filtered articles per day yields a sentiment score that faithfully represents the news content relevant to these stocks.","fun_headline_variants_meta":{"raw":{"variants":["Stock prices steer news sentiment, not the reverse","News follows stocks, study of Tesla and North Korea shows","In market-news tussle, prices dominate sentiment","Stock moves predict headlines better than headlines predict stocks","Financial news trails behind stock price shifts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000436,"raw_usage":{"total_tokens":2151,"prompt_tokens":808,"completion_tokens":1343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":1270}},"tokens_in":424,"tokens_out":1343,"duration_ms":10020,"temperature":1.0,"reasoning_tokens":1270,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:54:37.314185+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five-model comparison on a rebuilt dataset with all available articles for the same dates rather than the top 25, or apply a Granger-causality test to the daily series: the paper's claim predicts that stock prices at date t help predict sentiment at date t+1 while sentiment at t does not help predict prices at t+1. If the reverse pattern appears, the directional conclusion fails.","supporting_citations":[],"review_version":1}