{"id":"2647f490-aa8a-4d6d-936d-1316c4608abc","arxiv_id":"2412.12516","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"On US equities from 2020 to 2023, the standard Momentum Transformer earns 4.14% annual returns with Sharpe 1.12, beating momentum but only slightly beating long-only.","lead":"This paper applies a machine learning trading model, the Momentum Transformer, to US stocks instead of the futures and indices it was designed for. It finds the model beats a classic momentum benchmark but barely beats simply holding the market, because stocks are more volatile.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Self-admitted survivorship-bias risk is never resolved: the universe selection ('top 5 by market cap within SIC') is not shown to be point-in-time, so all reported returns and Sharpe ratios may be inflated by excluding delisted stocks.","rationale":"The reader's weakest_assumption (transaction costs) is well-taken and the paper indeed never specifies a cost model, despite the abstract claiming cost-aware performance. However, I identify a more fundamental threat: the paper's own self-report that the data may suffer from survivorship bias, with no documented correction. Sections 3.1 and 4 describe the leakage fix and the universe rule, but never explain how the universe is selected in a point-in-time manner. A static 'top 5 by market cap within SIC' selection is a classic survivorship-bias design if the cap is measured at the end of the backtest. This threatens the validity of every number in Table 1, including the long-only baseline that is the key comparison for the conclusion that the TFT adds little beyond long-only. If the universe is instead already point-in-time, my concern is resolved by the proposed test; hence the reader's conditional verdict remains appropriate, now conditioned on demonstrating a point-in-time universe. I also credit the paper for a walk-forward design, transparent reporting of all tested variants, and openly discussing limitations and code availability; these are genuine strengths. But the survivorship question is precisely the kind of self-referential limitation that must be weighed before accepting the empirical claim, and the paper's own text does not retire it.","tokens_in":7177,"tokens_out":10787,"duration_ms":97180,"concrete_test":"Reconstruct the universe point-in-time: for the start of each year from 2017 to 2022, select the five largest stocks per SIC code using market capitalization as of the last trading day of the prior year, from CRSP data that includes delisted and subsequently delisted companies, and carry their returns until delisting. Re-run the walk-forward backtest for TFT252,4, long-only, and classical momentum with this universe. If the TFT's 4-year average Sharpe relative to long-only moves by more than ~0.2 (currently 1.12 vs 0.57) or if the TFT's return edge over long-only flips from +0.10% to negative, the survivorship concern is confirmed and the central claim fails in its current form.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's abstract and Section 3.1 acknowledge the 'Seven Sins' and list 'possibly even survivorship bias' as a challenge, but the only fix described is for the multi-instance data leakage. The methodology (Section 4) says the final dataset is the 'top 5 companies by market capitalization within specific ranges of SIC codes,' without stating whether this ranking is computed at each rebalance date using only information available then, or once over the full sample. If the latter, companies that were large at the end of 2023 but did not exist or were small in 2020 are included, while companies that were large in 2020 but later delisted are excluded. This directly biases the long-only baseline (4.04% return, 0.57 Sharpe) and the TFT (4.14% return, 1.12 Sharpe) upward, and because the TFT is long-biased and volatility-managed, the bias affects the two strategies differentially. The central claim that the Momentum Transformer 'outperforms momentum' and 'roughly matches long-only' is therefore not decidable from the reported numbers. The transaction-cost gap identified by the reader is real but secondary: even with zero costs, a survivor-biased universe does not represent a tradeable strategy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper extends the Momentum Transformer architecture of Wood et al. to US equities, testing it on a portfolio of large-cap stocks drawn from CRSP/Compustat. The authors train a Temporal Fusion Transformer with an LSTM encoder and attention, using daily close prices to produce position sizes, with a walk-forward expanding-window design: initial training on 2017–2019, then out-of-sample testing on each subsequent year from 2020 to 2023. They report that the vanilla TFT from the original paper achieves an average annual return of 4.14% and a Sharpe ratio of 1.12, compared with −1.07% and −0.18 for classical momentum and 4.04% and 0.57 for long-only. Variants with longer input windows, more attention heads, and change-point detection are also tested, and the authors find that these modifications generally do not improve on the vanilla TFT.","tokens_in":7437,"tokens_out":5013,"duration_ms":44194,"significance":"If the reported results are reliable, the paper would provide a modest but useful extension of the Momentum Transformer to individual equities, with the notable caveat that the Sharpe ratio is lower than in the original futures-based study. Strengths include the walk-forward out-of-sample design, a publicly available GitHub repository, and an honest discussion of data-leakage pitfalls. The paper also makes clear that its equity implementation underperforms the original futures results in risk-adjusted terms. However, the central empirical claim is undermined by two unresolved issues that the paper itself acknowledges or gestures toward: the universe selection is not demonstrated to be point-in-time, and no transaction costs are modeled despite the abstract's claim. In addition, the best model is selected from the same out-of-sample window used for the headline result, and no uncertainty quantification is provided. These issues are load-bearing because they directly affect the comparison between the momentum transformer and the baselines.","major_comments":[{"comment":"The universe selection is not shown to be point-in-time. The paper says the final dataset is the 'top 5 companies by market capitalization within specific ranges of SIC codes,' but it does not state whether this ranking is computed at each rebalance date using only information available then, or once over the full sample. If the ranking uses end-of-sample market caps, the backtest will include stocks that grew large only later and exclude stocks that were large early and then delisted, thereby inflating all reported returns and Sharpe ratios. The authors acknowledge in the Abstract and Section 3.1 that survivorship bias is a possible issue, yet the methodology section provides no fix. This directly biases the comparison between the TFT (4.14% return, 1.12 Sharpe) and the long-only baseline (4.04% return, 0.57 Sharpe), which is the paper's central result. The authors must either demonstrate that the CRSP/Compustat data are point-in-time and that the universe is reformulated at each rebalance, or re-run the analysis with a point-in-time universe.","section":"Section 4 (Data) and Abstract"},{"comment":"The abstract claims the model can 'enhance performance in scenarios accounting for transaction costs,' but no transaction cost model is ever specified. Section 5.1 reports returns, volatilities, Sharpe ratios, and drawdowns computed from close-to-close returns with no allowance for commissions, bid-ask spreads, slippage, or market impact. Since the strategy rebalances daily and trades relatively liquid but volatile individual equities, realistic costs (for example, 5–10 basis points per side plus impact) could materially reduce or entirely eliminate the reported 4.14% annual return and 1.12 Sharpe. The paper must either introduce a transparent cost model and report net-of-cost results, or remove the transaction-cost claim from the abstract.","section":"Abstract and Section 5.1"},{"comment":"The best-performing model is selected from the same 2020–2023 out-of-sample window that is used to evaluate it. The paper tests multiple variants (input windows of 252 and 378, attention heads of 4 and 6, with and without change-point detection) and then reports the vanilla TFT252,4 as 'the overall best performing model.' Because this selection is made after inspecting the test-period results, the reported performance is likely inflated by selection bias. The authors should pre-specify the model before seeing the test window, adjust for multiple comparisons (for example, with a Bonferroni or false-discovery-rate correction), or present the distribution of all variants' performances without highlighting a single one as the headline.","section":"Section 5.1 (Results) and Table 1"},{"comment":"No confidence intervals, standard errors, or statistical tests accompany any of the reported returns or Sharpe ratios. With only four annual observations per strategy, the difference between the TFT Sharpe ratio (1.12) and the long-only Sharpe ratio (0.57) may be well within sampling variation. This is especially important because the paper's qualitative conclusion that the TFT 'outperforms momentum' is based on a comparison with a baseline that has a negative Sharpe. The authors should provide block-bootstrap or other uncertainty quantification for the key metrics, or explicitly frame all claims as descriptive rather than inferential.","section":"Table 1 and Section 5.1"}],"minor_comments":[{"comment":"The sentence 'Once found and fixed, our model outputs were proper, and the data point-in-time' is too strong given that the Abstract and Section 3.1 itself list survivorship bias as a remaining possible challenge. The text should distinguish clearly between the multi-instance data-leakage issue that was fixed and the survivorship-bias issue that is not addressed.","section":"Section 3.1"},{"comment":"Table 1 is difficult to parse because each strategy's metrics are packed into a single line with repeated column headers. Consider reformatting as a conventional table with one row per strategy and separate columns for Return, Volatility, Sharpe, Downside Risk, Sortino, Maximum Drawdown, Calmar, % Positive Returns, and Profit/Loss Ratio.","section":"Table 1"},{"comment":"The description of the original Momentum Transformer architecture is terse and would benefit from explicit references to equations or a clearer diagram pointer, especially for the gated residual network and multi-head attention components.","section":"Section 2"},{"comment":"The list of SIC code ranges is incomplete and inconsistently formatted; for example, 'SIC0100-0199' is given for Agriculture, Forestry and Fishing, but other sectors do not follow the same 'SIC' prefix convention, and the list is missing some ranges. Clarify and standardize these ranges.","section":"Section 4"},{"comment":"Reference [2] is listed as 'Find oneabout MACD' , which is not a complete citation. The authors should supply a proper reference for the MACD indicator.","section":"References"},{"comment":"The numbering of the final sections is inconsistent: 'Section 6' appears both for Discussion & Future Work and for Acknowledgement, and 'Section 7' appears both for Summary and for References. Renumber the non-reference sections correctly.","section":"Section 6/7"},{"comment":"The GitHub link is appreciated, but the paper should also state the software environment and key hyperparameters (learning rate, batch size, number of LSTM layers, number of training epochs) so that the experiments are reproducible without inspecting code.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a transparent empirical study with a sensible walk-forward design and open code. The two most serious technical problems are the unresolved point-in-time universe issue (explicitly acknowledged as a possible survivorship bias) and the absence of any transaction-cost model despite a direct abstract claim. Both are addressable with additional data work, which is why I recommend major revision rather than rejection. The paper may also need to temper its claims about 'outperformance' given the absence of confidence intervals and the multiple-testing problem. Depending on the journal's scope, this could be more suited to a practitioner-oriented venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is an honest, walk-forward extension of the Momentum Transformer to individual US equities, and the headline result—vanilla TFT Sharpe 1.12 versus long-only 0.57 and classical momentum -0.18—is plausible but not established. The survivorship-bias question is the load-bearing issue, and the paper does not answer it.\n\nWhat is actually new: the transfer to a 45-stock portfolio of top-5-by-market-cap names per SIC sector, plus the hyperparameter variants. The negative result that longer windows and more attention heads do not help is a legitimate datapoint. The authors are transparent about the leakage bug they fixed and about the points where they fell short. Code is on GitHub.\n\nSoft spots, in order of severity. First, universe selection. Section 4 says the data include the top 5 companies by market capitalization within SIC ranges, but never states that the ranking is computed point-in-time. If the ranking uses end-of-sample market cap, the universe is survivor-biased, which inflates both the long-only and the TFT results. Because the model is long-biased and volatility-managed, the inflation is not uniform across strategies. The paper claims in Section 3.1 that the data are point-in-time after fixing the multi-instance stock issue, but the universe construction is not part of that fix. That mismatch needs explicit resolution. Second, the abstract says the model enhances performance in scenarios accounting for transaction costs, but there is no cost model, no cost-adjusted returns, and no turnover report. With daily rebalancing across roughly 45 stocks, costs could easily erase the 0.10 percentage point annual return edge over long-only. Third, the best model was selected from the same 2020–2023 window used to report performance, and there are no confidence intervals, so the gap between TFT and long-only is within noise. These are addressable issues, not fatal to the enterprise.\n\nWho it is for: quants deciding whether to bother applying this architecture to single-stock portfolios, and instructors wanting a compact case study in test-set selection and data pitfalls. As submitted, the central comparison is undecidable from the reported numbers. I would tell the authors to fix the survivorship question and provide turnover/cost analysis before treating the Sharpe as real. If they do, a short note is worth publishing. For now, I would not cite it, though it deserves a look in a reading group as a cautionary example.","headline":"Honest extension of the Momentum Transformer to single-stock portfolios, but the headline Sharpe is not decidable until the point-in-time universe and transaction costs are addressed.","tokens_in":7946,"tokens_out":3259,"would_cite":false,"duration_ms":30500,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Applying the Momentum Transformer to US equities, the best configuration—the original 252-step, 4-head TFT without changepoint detection—achieves 4.14% average annual return and a 1.12 Sharpe over 2020-2023, beating classical momentum and…","keywords":["momentum","temporal fusion transformer","deep learning","equities","sharpe ratio","changepoint detection","LSTM","point-in-time data"],"falsifier":"Run the same TFT strategy on the same 2020-2023 equity universe with a realistic per-trade cost of 5 to 15 basis points applied to daily rebalancing; if the cost-adjusted Sharpe falls below that of long-only (0.57), the claimed enhancement over the baseline disappears.","tokens_in":6979,"feed_emoji":"📈","tokens_out":3593,"duration_ms":31557,"temperature":0.7,"pith_summary":"The paper tries to establish that the Momentum Transformer, a Temporal Fusion Transformer designed for futures and indices, can be transplanted to individual US equities and still beat classical time-series momentum. Over a four-year out-of-sample test (2020-2023), the best configuration returns 4.14% annually with a 1.12 Sharpe, compared with -1.07% and -0.18 for the momentum baseline. The same model barely outperforms a simple long-only portfolio (4.04% return, 0.57 Sharpe), which the authors attribute to the higher volatility and covariance of equities. The paper also documents data-leakage pitfalls in equity backtests and how they were corrected to achieve point-in-time data.","feed_headline":"Momentum Transformer nets 4.14% a year on US stocks","feed_subtitle":"Applying the futures-tuned TFT to equities beats classic momentum and long-only from 2020 to 2023, before trading costs.","key_machinery":"The central object is the Temporal Fusion Transformer (TFT), a hybrid architecture combining an LSTM encoding layer, a variable selection network, gated linear units, gated residual networks, and multi-head attention over a rolling window of price-derived features. The attention mechanism gives the model direct access to all prior timesteps in the window, allowing it to learn both short-term and long-term dependencies and to size positions by directly optimizing the Sharpe ratio through a negative-Sharpe loss function. A separate changepoint-detection (CPD) script supplies regime-shift features, but the paper's best result comes from the TFT without CPD, meaning the attention mechanism itself carries the performance.","core_discovery":"The core claim is that a decoder-only Temporal Fusion Transformer with an LSTM encoding layer, variable selection, gated residual networks, and multi-head attention can be applied to US equities and outperforms classical time-series momentum. The paper reports that the vanilla TFT from the original paper—using a 252-timestep window and four attention heads, without changepoint detection—produced an average annual return of 4.14% and an average annual Sharpe of 1.12 from 2020 to 2023. This beats the classical momentum baseline (-1.07% return, -0.18 Sharpe) and slightly exceeds long-only (4.04% return, 0.57 Sharpe). The paper further finds that lengthening the lookback window to 378 timesteps or increasing attention heads to six hurt the vanilla TFT, while more attention heads improved the changepoint-detection variant, suggesting the CPD features are noisy for equities.","pith_inferences":["Extending the test period beyond 2023 could clarify whether the 2020-2023 window—which includes the Covid shock, rising rates, and inflation—biased results against momentum-based strategies and compressed the model's apparent advantage.","A natural testable extension is to integrate a transaction-cost model directly into the Sharpe objective; the paper claims the architecture enhances performance in cost-aware scenarios but never specifies a cost model or reports cost-adjusted returns.","The sector-diversified but high-covariance stock portfolio suggests that a long-short or market-neutral variant might isolate the model's alpha from broad market exposure, which would sharpen the comparison with long-only.","The data-leakage episode (multiple share classes per company leaking future values) is a cautionary template for equity momentum backtests; any replication must ensure point-in-time identifiers to avoid inflated Sharpe ratios."],"forward_implications":["If the result holds, deep momentum networks can be extended to equities, but the higher volatility of individual stocks compresses the Sharpe advantage; futures and indices remain more suitable for this architecture.","The finding that more attention heads helped only the CPD variant suggests that changepoint features on equities are noisy and require extra model capacity to interpret, which could guide future architecture choices.","The near-parity with long-only returns implies that the model's edge over the momentum baseline may come less from momentum timing and more from volatility reduction, as the drawdown analysis shows the TFT is less impacted by market downturns.","For practical deployment, the daily-rebalancing result depends on trading at or near the close with no transaction costs, so real-world returns would need a cost model to preserve the edge."],"supporting_citations":[{"why":"Supplies the base TFT architecture and training methodology that the paper extends from futures and indices to equities.","marker":"[6]"},{"why":"Provides the LSTM deep momentum network with changepoint detection that the TFT builds on and that the paper compares against.","marker":"[4]"},{"why":"Establishes the deep momentum network approach of directly optimizing the Sharpe ratio that underlies the model's objective.","marker":"[3]"},{"why":"Defines the classical time-series momentum effect that serves as the baseline strategy to beat.","marker":"[1]"},{"why":"Lists the data-leakage, look-ahead, and survivorship biases that the paper says it corrected to achieve point-in-time data.","marker":"[11]"}],"fun_headline_variants":["Momentum Transformer beats classic momentum on US stocks","Attention-based TFT yields 4.14% annual return, Sharpe 1.12","TFT model on equities outperforms momentum and long-only","Momentum Transformer applies to stocks, beats baselines","Decoder-only TFT nets 4.14% on stocks, besting long-only"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported returns assume trades are executed at or near closing prices with no transaction costs, slippage, or market impact, and the paper never specifies a cost model or reports cost-adjusted figures.","fun_headline_variants_meta":{"raw":{"variants":["Momentum Transformer beats classic momentum on US stocks","Attention-based TFT yields 4.14% annual return, Sharpe 1.12","TFT model on equities outperforms momentum and long-only","Momentum Transformer applies to stocks, beats baselines","Decoder-only TFT nets 4.14% on stocks, besting long-only"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001008,"raw_usage":{"total_tokens":4244,"prompt_tokens":911,"completion_tokens":3333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":3240}},"tokens_in":527,"tokens_out":3333,"duration_ms":22704,"temperature":1.0,"reasoning_tokens":3240,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:59:41.367806+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same TFT strategy on the same 2020-2023 equity universe with a realistic per-trade cost of 5 to 15 basis points applied to daily rebalancing; if the cost-adjusted Sharpe falls below that of long-only (0.57), the claimed enhancement over the baseline disappears.","supporting_citations":[{"cited_title":"The variations we added with theexpanded window and increased attention headsnegatively impacted the non-CPD TFT","cited_arxiv_id":null,"evidence_quote":"Supplies the base TFT architecture and training methodology that the paper extends from futures and indices to equities."}],"review_version":1}