{"id":"c24d5c71-ec0f-4bf4-a945-6fc095368d02","arxiv_id":"2507.20535","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"FTS-Text-MoE is a sparse Mixture-of-Experts transformer that aligns daily text summaries with price series; it reports a higher Sharpe ratio in a backtest, but the accuracy advantage is inconsistent and the evaluation protocol is underspecified.","lead":"The paper introduces a mixture-of-experts model that combines daily stock prices with summarized news and tweets to predict future prices and to build a portfolio. The reported backtest returns look strong, but the paper omits a clear train/test split, error bars, and transaction costs, and its accuracy tables do not consistently beat the baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline claims of superior returns/Sharpe rest on an evaluation protocol that never appears: no train/test split, evaluation window, or baseline setting is stated, so the reported 0.1347 return and 1.0818 Sharpe are not verifiable as out-of-sample.","rationale":"The paper's central empirical claim can only be true if the reported metrics come from a valid out-of-sample backtest. I searched the full text for any statement of how the data are split: there is none. Appendix A.3 gives training hyperparameters but no train/validation/test boundaries; Section 4.5 reports portfolio returns without stating the evaluation period, the rebalancing rule, the risk-free rate, or whether test-period news/tweets were excluded from training. If the model was trained and evaluated on the same window, the 'predictions' are in-sample fits and the Sharpe/return numbers are meaningless. The baselines are uncontrolled too—nothing says Chronos/Moirai are zero-shot rather than fine-tuned, so Table 1 mixes training regimes. The abstract's 'superior accuracy' is also contradicted by Table 1 (e.g., Consumer Defensive: 0.5700 vs Chronos 0.1946; Real Estate: 0.3550 vs 0.2282), and several rows with identical 'Ours ts' and 'Ours ts&news' entries suggest uncontrolled ablations. Because the evaluation protocol is the single load-bearing assumption and it is both absent and inconsistent with the reported numbers, the reader's REJECT verdict stands. A 'conditional accept pending protocol' might be defensible if the only gap were reporting, but the internal contradictions in Table 1 and the lack of code/data make REJECT the appropriate verdict.","tokens_in":16964,"tokens_out":7035,"duration_ms":72898,"concrete_test":"Request the exact data partition (e.g., train through 2022-12-31, validate 2023-01-01 to 2024-06-30, test 2024-07-01 to 2025-01-19), retrain FTS-Text-MoE on the training split only, and recompute Table 2 on the test window with Chronos and Moirai run zero-shot under an identical protocol and a stated risk-free rate. If the test-window Sharpe drops below 0.3 or the leading gainers in Figure 4 change materially, the reported results are not valid out-of-sample predictions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—superior investment returns and Sharpe ratio (abstract; Table 2)—depends entirely on the reported metrics being computed out-of-sample. Appendix A.3 ('Training Configuration') specifies only training steps, batch size, and optimizer settings; it never defines a train/validation/test split, the evaluation window, or whether the model is evaluated on data used in training. Section 4.5 reports portfolio returns with no dates, no statement of the rebalancing period, no risk-free rate for Eq. 17, and no exclusion of test-period text or prices from training. If the model is trained and scored on the same 20-year window, the 'predictions' are in-sample reconstructions and the Sharpe/return numbers are artifacts. The baselines are also uncontrolled: nothing states whether Chronos and Moirai are used zero-shot or fine-tuned, so Table 1 compares a trained model against possibly untrained models without an equivalent protocol. Furthermore, Table 1 contradicts the abstract's 'superior accuracy' claim: in Consumer Defensive the best FTS-Text-MoE MSE is 0.5700 vs Chronos 0.1946; in Real Estate it is 0.3550 vs 0.2282. Several rows show identical MSE/MAE for 'Ours ts' and 'Ours ts&news' (Consumer Defensive, Utilities, Energy), which is inconsistent with the claim that text inputs improve accuracy. The Limitations section discusses lag and misinformation but never addresses lookahead leakage. The load-bearing condition—valid out-of-sample evaluation—is unstated and unsupported by the paper's evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FTS-Text-MoE, a decoder-only Mixture-of-Experts Transformer for stock price forecasting that combines numerical time series with daily news and tweet summaries via point-wise token embeddings, and uses multi-resolution prediction heads to forecast at several horizons. The authors report improved MSE/MAE over Chronos and Moirai in some sectors, and report a portfolio with higher cumulative return and Sharpe ratio than the baselines. They also describe an extension of the FNSPID news dataset to January 2025. The paper's central claims are that FTS-Text-MoE 'outperforms baseline methods in terms of investment returns and Sharpe ratio' and demonstrates 'superior accuracy'.","tokens_in":17257,"tokens_out":4226,"duration_ms":45713,"significance":"If the empirical claims were established, the paper would contribute a useful sparse architecture for multimodal financial forecasting, a cleaned text-summarization pipeline, and an updated news dataset. The architectural description is detailed, and the proposed alignment of text summaries with price series is plausible. However, the evidence presented does not currently support the headline claims: Table 1 is internally inconsistent with the 'superior accuracy' statement, and the portfolio results in Table 2 are not accompanied by a defined out-of-sample evaluation protocol. The paper's practical significance is therefore contingent on a substantial revision of the evaluation methodology.","major_comments":[{"comment":"The claim that FTS-Text-MoE shows 'superior accuracy' is contradicted by Table 1. In Consumer Defensive the best FTS-Text-MoE MSE is 0.5700 versus 0.1946 for Chronos and 0.1533 for Moirai; in Utilities it is 0.2797 versus 0.2517 for Moirai; in Energy it is 0.4196 versus 0.2094 for Chronos and 0.3204 for Moirai; in Technology it is 0.8688 versus 0.8681 for Chronos and 0.5858 for Moirai; in Real Estate it is 0.3892 versus 0.2282 for Chronos; and in Industrials it is 0.5829 versus 0.4708 for Chronos and 0.2126 for Moirai. The paper should either provide an aggregate, statistically valid comparison that supports the abstract's claim or substantially weaken the claim.","section":"Abstract and §4.2.1, Table 1"},{"comment":"The central result—overall return 0.1347 and Sharpe ratio 1.0818—is not verifiable as an out-of-sample result. Appendix A.3 states only training steps, batch size, and optimizer settings; it never specifies a train/validation/test split, the evaluation period, the rebalancing frequency, the risk-free rate used in Eq. (17), or whether Chronos and Moirai are used zero-shot, fine-tuned, or retrained. Without a defined temporal split and a statement that test-period text and price data were excluded from training, the reported numbers could be in-sample reconstructions. Section 6 discusses lag and misinformation but does not address lookahead leakage, which is a load-bearing omission.","section":"§4.5, Table 2, and Appendix A.3"},{"comment":"The baseline comparison is not controlled. The paper never states whether Chronos and Moirai are evaluated in their pretrained form or fine-tuned on the same data, nor whether they receive the same context length and forecast horizon. The text in §4.2.2 concedes that FTS-Text-MoE 'falls short of surpassing Chronos and Moirai in some industries and scenarios,' which conflicts with the unqualified superiority claims in the abstract and conclusion. Without a calibrated protocol, the relative performance cannot be interpreted.","section":"§4.1 and §4.2.2"},{"comment":"The data alignment appears inconsistent. Tweet data cover only 2020–2022, while news data begin as early as 2009 and, for some stocks, only begin in 2024 or 2023 (e.g., AAPL news from 2022, META news from 2024). The paper does not explain how missing text is handled at time steps without news or tweets, nor why 'Ours ts' and 'Ours ts&news' produce identical MSE/MAE in Consumer Defensive, Utilities, and Energy rows of Table 1. These identical values suggest the text embedding may be zero or absent for those periods, which undermines the claim that text inputs improve accuracy.","section":"Appendix B.2–B.3 and §3.1"},{"comment":"The claimed computational-efficiency advantage is not measured. The paper states that only 50M of 113M parameters are active, but no runtime, FLOPs, throughput, or energy comparison is reported against dense baselines or against Chronos/Moirai. Since reducing computational cost is one of the stated motivations, the absence of any efficiency experiment leaves this claim unsupported.","section":"§3.2 and §5"}],"minor_comments":[{"comment":"There is a typo in the first sentence: 'tex-tual' should be 'textual'.","section":"Abstract"},{"comment":"The notation MA for masked attention is used in Eq. (2) but not defined in the text; please define it explicitly.","section":"§3.2, Eq. (2)"},{"comment":"The ranking function score(x,c) is never defined, despite being load-bearing for the selection of the 'top' news and tweets; a precise definition or reference is needed.","section":"§3.1 and Algorithm 1"},{"comment":"The Sharpe ratio formula uses R_f but no value of the risk-free rate is reported anywhere in the paper; specifying R_f and the evaluation dates is necessary for reproducibility.","section":"Appendix B.7, Eq. (17)"},{"comment":"The columns labeled 'Daily Tweet Count' and 'Daily News Count' appear to contain total message counts over the whole period, not daily counts; the labels should be corrected.","section":"Tables 4 and 5"},{"comment":"There is an orphan superscript '3' after 'significant fluctuations' with no corresponding footnote; please remove it or add the note.","section":"§4.2.2"},{"comment":"The word 'Explainable' in the title is never substantiated: Figure 4 shows top gainers/losers but no explanation mechanism, attribution, or interpretability analysis is presented.","section":"Title and §4.4"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jakob—\n\nQuick take on arXiv:2507.20535 (FTS-Text-MoE). The paper proposes a sparse mixture-of-experts transformer decoder that aligns daily stock prices with summary embeddings of news and tweets, plus multi-resolution prediction heads. All pieces are borrowed from Time-MoE, Chronos, Moirai, and SBERT; the new bit is the specific point-wise text alignment and the updated NASDAQ news dataset (extended to Jan 2025). That dataset update is a real, reproducible contribution—they give counts and dates by ticker, and they were transparent about the crawling fixes. The architecture section is also clear enough that someone could reimplement it.\n\nThe load-bearing claim, though, is the abstract's: 'superior accuracy' and better returns/Sharpe based on Table 1 and Table 2. That claim is not supported by the evidence as written. No train/validation/test split is stated anywhere. Appendix A.3 gives only training steps, batch size, optimizer hyperparameters. No evaluation window, no dates for the backtest, no risk-free rate for the Sharpe formula in Eq. 17, no transaction costs. Table 2's 0.1347 return and 1.0818 Sharpe are therefore uninterpretable—could be in-sample reconstructions for all the paper discloses.\n\nTable 1 also doesn't consistently back 'superior accuracy.' In Consumer Defensive the best FTS-Text-MoE MSE is 0.5700 versus Chronos's 0.1946; in Real Estate it's 0.3550 versus 0.2282. The paper itself admits in 4.2.2 that it falls short in some industries—an honest admission, but it contradicts the abstract. Several rows show identical MSE/MAE for 'Ours ts' and 'Ours ts&news' (Consumer Defensive, Utilities, Energy), which weakens the claim that text inputs improve accuracy. And the title promises 'Explainable' predictions, but there is no explanation analysis anywhere—no SHAP, no attribution, no case study. The Limitations section talks about lag and misinformation but never mentions lookahead leakage, which is the first thing that should be addressed.\n\nOne more gap: tweet data spans only 2020–2022 (Appendix B.2), while prices span 20 years. The paper never says how they handle dates without tweets or news, nor does it specify the actual evaluation period for Table 1. That ambiguity alone makes the headline results non-reproducible.\n\nThe idea is not crazy—it's a reasonable incremental combination—and the dataset update is useful. But the evaluation is so under-specified that I can't treat the central empirical claim as established. If this crosses my desk, I'd desk-reject it for lack of a defined split and unverifiable backtest; with a proper temporal split, error bars, and released code/data, it could come back as a solid applied paper.\n\nLet's not put it on the reading-group agenda. —M.","headline":"The architecture is clearly described and the dataset update is real, but the paper's central claim of superior returns and Sharpe rests on an evaluation protocol that never appears—no train/test split, no baseline setup, no dates—so the results are unverifiable as out-of-sample.","tokens_in":17873,"tokens_out":3573,"would_cite":false,"duration_ms":35318,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FTS-Text-MoE claims that a sparse mixture-of-experts transformer aligned with tweet and news summaries beats Chronos and Moirai on stock returns and Sharpe ratio.","keywords":["stock price prediction","mixture of experts","financial time series","tweet sentiment","news summarization","multi-horizon forecasting","sparse transformer","portfolio optimization"],"falsifier":"Rerun the exact experiment with a hard temporal cutoff: train only on data before a fixed date, tune nothing on data after it, and evaluate on the later period; if the Sharpe ratio and MSE advantages over Chronos and Moirai vanish or invert, the reported edge is not predictive. A cheaper check is to inspect whether any predicted day's input text was published after that day's close.","tokens_in":16696,"feed_emoji":"📈","tokens_out":4687,"duration_ms":42488,"temperature":0.7,"pith_summary":"This paper tries to establish that stock price forecasts improve when a sparse mixture-of-experts transformer consumes both historical prices and short factual summaries of news and social-media posts about each stock. The proposed model, FTS-Text-MoE, aligns text and price embeddings point by point and predicts the next price value (not just the direction), with prediction heads at several horizons. The authors report lower mean squared error on many industry sectors when tweet summaries are added, and a portfolio built from its positive predictions returns 0.1347 with a Sharpe ratio of 1.0818, well above the Chronos and Moirai baselines. The significance would be a cheaper, explainable way to make text a first-class input to financial time-series forecasting, without relying on prompt-based LLM calls.","feed_headline":"Mixture-of-experts forecaster beats Chronos and Moirai with tweets","feed_subtitle":"FTS-Text-MoE pairs prices with summarized news and X posts, reporting higher returns and Sharpe ratio than the leading baselines.","key_machinery":"The load-bearing mechanism is the FTS-Text-MoE decoder: a decoder-only Transformer whose feedforward layers are replaced by a Mixture of Experts with eight experts per layer, top-2 routing, and one shared expert, so only about 50M of 113M parameters are active. Text is reduced to 1-2 factual summaries per day, ranked by relevance to the company, embedded with an SBERT sentence encoder, then mean-pooled with SwiGLU-embedded price points so each time step carries both modalities. A multi-resolution prediction head with horizons {1, 8, 32, 64} plus greedy concatenation lets the same model forecast arbitrary-length sequences.","core_discovery":"The central claim is that coupling text to numbers through point-aligned summaries and a sparse MoE decoder is enough to materially improve next-value stock prediction and downstream portfolio metrics. Concretely, the paper shows that adding summarized X (Twitter) comments reduces MSE and MAE across most of eleven industry sectors relative to price-only and news-only inputs, that the model outperforms Chronos and Moirai on cumulative returns (0.1347 versus 0.0079 and -0.0077 in its portfolio setup), and that its Sharpe ratio reaches 1.0818 against 0.3320 for Moirai and 0.1223 for Chronos. The authors interpret the gains as the model capturing short-term volatility with a more proactive strategy rather than the conservative smoothing of the baselines.","pith_inferences":["A strict temporal backtest with transaction costs could change the Sharpe gap; the paper's portfolio is rebalanced with no trading-cost adjustment, and the aggressive concentration it documents would erode under realistic slippage.","The same model should be testable on other text-heavy assets (cryptocurrencies, commodities) and on intraday data, where the lag between news publication and price impact is shorter.","The ranking pipeline that selects one summary per day could be the true source of signal; an ablation that replaces summaries with random tweets or with sentiment scores alone would isolate whether factual content or mere volume carries the gain."],"forward_implications":["If text-aligned sparse MoE genuinely forecasts next values, portfolio strategies can move from prompt-based LLM calls to a single lightweight model that produces multi-horizon predictions in near real time.","The multi-resolution head means one model can serve both short-horizon traders (1-8 days) and longer-horizon allocators (32-64 days), removing the need for separate models per horizon.","Because the model predicts numeric values rather than direction, it plugs directly into mean-variance and risk-parity portfolio optimizers that need return forecasts.","The reported cost reduction (50M active parameters out of 113M) suggests text-augmented forecasting can run on conventional hardware, not just large inference clusters."],"supporting_citations":[{"why":"Chronos baseline, the discretized 'language of time series' model whose performance the paper must beat.","marker":"(Ansari et al.)"},{"why":"Moirai baseline, universal forecasting transformer with multi-granularity patching.","marker":"(Woo et al., 2024)"},{"why":"Time-MoE, source of the sparse MoE decoder design and the multi-resolution prediction head idea.","marker":"(Shi et al., 2024)"},{"why":"SwiGLU activation used for the time-token embedding.","marker":"(Shazeer, 2020)"},{"why":"Switch Transformers, basis for the single-expert routing and auxiliary load-balancing loss.","marker":"(Fedus et al., 2022)"},{"why":"Channel-independence principle that the paper follows to decompose multivariate series into univariate series.","marker":"(Nie et al.)"},{"why":"Sharpe ratio statistic used for annualizing and interpreting risk-adjusted return.","marker":"(Lo, 2002)"}],"fun_headline_variants":["Tweet-summarized MoE beats top baselines in returns, Sharpe","Sparse MoE with tweets: Sharpe 1.08 vs 0.33 for Moirai","Point-aligned tweets lift MoE stock forecasts above baselines","Tweets add edge to MoE predictor: better returns, Sharpe","MoE + tweets: better returns and Sharpe than rivals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result stands or falls on the assumption that the reported MSE, returns, and Sharpe ratio are computed on a genuine out-of-sample test set with no lookahead; the paper trains for 10,000 steps but never states how the data is split into training, validation, and test periods, nor whether the Chronos and Moirai baselines are used zero-shot.","fun_headline_variants_meta":{"raw":{"variants":["Tweet-summarized MoE beats top baselines in returns, Sharpe","Sparse MoE with tweets: Sharpe 1.08 vs 0.33 for Moirai","Point-aligned tweets lift MoE stock forecasts above baselines","Tweets add edge to MoE predictor: better returns, Sharpe","MoE + tweets: better returns and Sharpe than rivals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001144,"raw_usage":{"total_tokens":4733,"prompt_tokens":919,"completion_tokens":3814,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":3717}},"tokens_in":535,"tokens_out":3814,"duration_ms":30490,"temperature":1.0,"reasoning_tokens":3717,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:42:45.859530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the exact experiment with a hard temporal cutoff: train only on data before a fixed date, tune nothing on data after it, and evaluate on the later period; if the Sharpe ratio and MSE advantages over Chronos and Moirai vanish or invert, the reported edge is not predictive. A cheaper check is to inspect whether any predicted day's input text was published after that day's close.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Moirai baseline, universal forecasting transformer with multi-granularity patching."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SwiGLU activation used for the time-token embedding."}],"review_version":2}