{"id":"5ea6eaf5-5e2c-4627-8927-4797b6b695f6","arxiv_id":"2412.09880","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Fine-tuning TimesFM on log-transformed financial price data improves directional accuracy and mock-trading Sharpe ratios over the base model.","lead":"This paper fine-tunes the TimesFM foundation model on financial price data and reports improved directional accuracy and mock-trading returns. It shows that a general-purpose time series model can be adapted to markets with a simple fine-tuning recipe, and the authors release code and weights.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline Sharpe of 1.68 rests on a single overlapping year with zero costs and no significance testing; it should not be treated as an established result without overlap-adjusted inference and cost sensitivity.","rationale":"The reader's conditional verdict is appropriate. My stress-test focuses on the most quantitative part of the strongest claim, the 1.68 Sharpe. The single-year and zero-cost concerns are amplified by the overlapping-horizon structure, which the reader did not explicitly mention. This is a load-bearing issue because the headline number is the paper's most striking evidence of practical value; if it is within noise, the paper still has the accuracy improvements, but those are also reported without significance tests. I therefore recommend no change to the conditional verdict: the paper should be accepted only if the authors provide overlap-adjusted inference and a cost sensitivity analysis. I also note Section VI's admission that the model does not consistently beat AR1, which reinforces that the practical outperformance claim is overstated; this supports the conditional, not a rejection, because the core fine-tuning improvement over original TimesFM is plausible and partly supported by the loss curves and code release.","tokens_in":11843,"tokens_out":11698,"duration_ms":130541,"concrete_test":"Recompute the market-neutral trading strategy from the released code on the same 2023 test set, computing daily returns and then the annualized Sharpe with Newey-West standard errors using lag 127 (the number of overlapping days), and repeat with a 10 basis point round-trip transaction cost per trade. If the overlap-adjusted t-statistic falls below 2 or the net return after costs is no longer positive, the headline Sharpe claim is not supported. Additionally, rerun the same strategy on out-of-sample years 2019-2022 and 2024 with models trained only on data before each year to check stability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline market-neutral result (annualized Sharpe 1.68, 3.6% annual return) is computed from a single out-of-sample year, 2023, using a horizon of h=128 days. The daily PnL series is not close to i.i.d.: each day's return contains the same 127 open layers of overlapping positions, so the effective number of independent holding periods is only about 252/128 ≈ 2. Under a naive i.i.d. reading, a one-year Sharpe of 1.68 corresponds to a t-statistic of about 1.68; once the overlap is accounted for with Newey-West standard errors at lag 127, the t-statistic is far smaller. The paper reports no significance tests, error bars, or block-bootstrap confidence intervals. The zero-cost assumption is also material: Table III lists a 'neutral cost' of 0.60% at h=128, meaning a 60 basis point round-trip cost would eliminate the 3.6% annual return, and real trading costs are nonzero. Without overlap-adjusted inference or a cost sensitivity analysis, the claim that the strategy 'achieves' a 1.68 Sharpe is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes continual pre-training of the TimesFM time-series foundation model on a large corpus of financial price data, using a log-MSE loss and a random masking scheme to stabilize training. The fine-tuned model is evaluated on price-direction accuracy, Macro F1, and two mock trading strategies (basic and market-neutral) across S&P500 stocks, TOPIX500 stocks, currencies, and cryptocurrencies. The central claims are that fine-tuning improves prediction accuracy over both the original TimesFM and a chance-rate baseline, and that a market-neutral strategy on S&P500 stocks achieves an annualized Sharpe ratio of 1.68 and 3.6% annual returns in a zero-cost mock trading setting. The authors release code and model weights for reproducibility.","tokens_in":12051,"tokens_out":9330,"duration_ms":86929,"significance":"If the central claims hold, the paper would provide a useful demonstration that a general-purpose time-series foundation model can be adapted to financial price prediction through continual pre-training, with potential practical application in trading strategy construction. The paper has concrete strengths: it uses a temporal holdout (2023 onward) for testing to avoid look-ahead bias, proposes a principled log-transformation of the loss, publishes code and model weights, and reports multiple performance metrics across several markets. However, the statistical evidence is currently incomplete: the accuracy/F1 improvements are asserted without significance tests, and the headline trading result relies on a single out-of-sample year, overlapping positions, and zero-cost assumptions. The paper's own discussion also concedes that the proposed model does not consistently beat a simple AR1 model, which weakens the broad claims made in the abstract.","major_comments":[{"comment":"The text states that the fine-tuned model \"outperforms random chance on all prediction horizon, hinting at statistically significant performance\" and that fine-tuning \"significantly improves\" accuracy, but no significance tests, confidence intervals, or error bars are reported anywhere in the paper. The accuracy and Macro F1 comparisons are single point estimates computed on one test set; because this is the central claim of the paper, the authors should provide formal paired tests (e.g., McNemar's test for the accuracy classifications, or a bootstrap over series/days with cluster-robust standard errors) and report effect sizes with uncertainties. Without this, the word \"significant\" is not supported.","section":"Section V-B and Figures 3-4"},{"comment":"The headline result (annualized Sharpe 1.68 and 3.6% annual returns at h=128) is computed from a single out-of-sample year (2023) using a daily PnL series with strongly overlapping positions: each day's return contains 127 open layers of prior trades, so the effective number of independent holding periods is roughly 252/128 ≈ 2. No Newey-West adjustment, block-bootstrap, or other overlap-aware inference is reported, and the paper does not test whether the Sharpe is statistically distinguishable from zero. The evaluation also assumes zero trading costs; Table III's \"neutral cost\" of 0.60% at h=128 implies that a round-trip cost of 60 basis points would eliminate the entire 3.6% return. Please add overlap-adjusted confidence intervals for the Sharpe and a cost-sensitivity analysis over realistic round-trip costs (e.g., 5, 10, 20, 50 bp), and present the result as a single-year observation rather than an expected performance estimate.","section":"Section V-D, Table III, and Table IV"},{"comment":"The reported size of the continual pre-training dataset is inconsistent: the abstract says 100 million time points, Section III-A twice refers to \"80M time points\", Section IV-A says \"more than 100K time series and 90M time points\", and the sum of Table I is approximately 82 million time points. The authors must reconcile these numbers and state a single, correct figure in all locations, since the dataset description is a central part of the method.","section":"Abstract, Section III-A, Section IV-A, and Table I"},{"comment":"The paper's own Discussion admits that \"we are unable to ascertain consistently better performance over just a simple AR1 model,\" and Table IV confirms that AR1 outperforms the proposed model on Currencies (Sharpe 0.88 vs 0.25) and Crypto Daily (0.17 vs 0.26). The abstract's claim that the fine-tuned model \"outperforms various benchmarks in terms of returns, sharpe ratio, max drawdown and trading cost\" is therefore not supported as stated. The authors should qualify the claim to the specific markets and horizons where the improvement holds, and provide a statistical comparison against AR1 (e.g., a paired test on the PnL series) to justify any comparative statement.","section":"Section VI and Abstract/Table IV"}],"minor_comments":[{"comment":"The model name is misspelled as \"TimeFM\" in the abstract; the correct name is \"TimesFM.\"","section":"Abstract and throughout"},{"comment":"The notation \"1/(h−1)T\" is ambiguous; it should be written as 1/((h−1)T) to indicate the order size per asset.","section":"Section IV-C3"},{"comment":"The text says \"using a horizon length of H\" where it should be \"h\"; also, the \"Neutral Cost\" entries are negative for some horizons (e.g., -0.006 for h=4), which should be explained since a negative neutral cost means the strategy loses money even at zero cost.","section":"Section V-D and Table III"},{"comment":"There are several typos and grammatical errors: \"siginificantly\" in the Figure 1 caption, \"most oftenly\" in Section I, \"dailu\" in Section IV-C3, and \"seris\" and \"repitition\" in Section VI. A thorough proofread is needed.","section":"Sections I, IV-C3, and VI"},{"comment":"The repository URL is typeset as \"timesfm fin\" with a space; the correct project name should be given so that the reproducibility link is usable.","section":"Section V-D"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and interesting question and the central idea is plausible, but the current evidence does not support the strength of the claims made. The lack of significance testing, the overlap issue in the PnL series, the inconsistent dataset-size reports, and the overstatement relative to AR1 need to be addressed before the paper can be considered for publication. I recommend major revision with the expectation that the authors add proper inference, cost sensitivity, and a more measured abstract."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful core of this paper is simple and reproducible: continual pre-training of TimesFM on log-transformed price data with random masking, with code and weights released. The fine-tuned model beats vanilla TimesFM and a chance baseline on directional accuracy and macro F1 across horizons, and that part of the claim is believable. The training loss curves and the consistency of the accuracy results across seven horizons support it. The paper is also honest about its own limitations—it concedes it does not consistently beat AR1, notes the training data is dominated by hourly crypto and stock prices, and raises catastrophic forgetting. Those are real issues, and it flags them itself.\n\nThe soft spots are statistical, not conceptual. The headline Sharpe of 1.68 on S&P500 comes from a single out-of-sample year, 2023, with a 128-day horizon. The daily PnL series has heavy overlap: each day's return contains the same 127 open layers, so the effective number of independent holding periods is about two. Under a naive i.i.d. reading a one-year Sharpe of 1.68 corresponds to a t-statistic of roughly 1.68; with Newey-West or block bootstrap it is far smaller. The paper reports no confidence intervals, no significance tests, and no cost sensitivity—even though its own Table III shows a 60 basis point round-trip cost would eliminate the 3.6% annual return. Real trading costs are nonzero. So the claim that the strategy 'achieves' a 1.68 Sharpe is not established. The data-size inconsistency is minor but sloppy: abstract says 100M time points, Section III-A says 80M, and Table I sums to about 82M.\n\nI agree with the reader's conditional verdict and with the stress-test note. The central directional-accuracy result is plausible and likely reproducible, but the financial-performance claims are over-sold. On S&P500 the fine-tuned model's Sharpe is 1.68 versus AR1's 1.58—essentially a tie—and on currencies AR1 is clearly better. The paper's own discussion admits this.\n\nWho is this for? Researchers working on foundation models for finance will find the recipe and released artifacts useful, and the paper is a reasonable baseline for future work. It deserves a serious referee, not a desk reject. A referee should ask for overlap-adjusted inference, cost sensitivity, a multi-year test period, and corrected data counts. My own verdict: the fine-tuned model probably does improve on vanilla TimesFM for price prediction, but the Sharpe number should be treated as suggestive until the statistics are redone.","headline":"Fine-tuning TimesFM on financial data is a clean, reproducible empirical recipe, and the accuracy gains over the vanilla model look real, but the headline Sharpe 1.68 is not established: it rests on one overlapping year of zero-cost mock trading with no significance tests.","tokens_in":12591,"tokens_out":1644,"would_cite":false,"duration_ms":18313,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning the TimesFM time-series foundation model on financial price data turns it from a predictor that often misses chance into one that beats chance and an AR(1) baseline.","keywords":["financial fine-tuning","time series foundation model","TimesFM","price prediction","continual pre-training","market-neutral trading","Sharpe ratio","log-transformed MSE loss"],"falsifier":"Evaluate the fine-tuned model on 2024–2025 data with realistic transaction costs and a 128-step market-neutral strategy: if the net annualized Sharpe ratio is not positive while directional accuracy stays above chance, the central claim of a durable fine-tuning edge fails.","tokens_in":11610,"feed_emoji":"📈","tokens_out":8626,"duration_ms":81337,"temperature":0.7,"pith_summary":"Fine-tuning the pre-trained time-series foundation model TimesFM on financial price data changes it from a model that often underperforms random chance at price prediction into one that consistently beats chance and the original model across prediction horizons from 2 to 128 steps. The authors argue that the failure of the original model stems from the irregular, non-stationary character of price series, which differs sharply from the regular, seasonal time series used in pre-training. Their fine-tuning recipe — continual pre-training on roughly 100 million price points with a log-transformed mean-squared-error loss and random context masking — is what carries the improvement. They support this with mock trading: a market-neutral strategy based on 128-step predictions earns a 1.68 annualized Sharpe ratio and 3.6% annual returns on S&P500 stocks in a zero-cost setting, the strongest result across the markets tested.","feed_headline":"Fine-tuned TimesFM beats chance and AR(1) on price prediction","feed_subtitle":"Continual pre-training on 90 million price points lifts prediction accuracy and hits a 1.68 Sharpe in S&P500 trading.","key_machinery":"The load-bearing mechanism is the decoder-only TimesFM transformer, which reads price history in patches of 32 time points and predicts the next 128 points, with random masking during training so that it learns to forecast from any context length between 128 and 512. On top of this, the paper applies a log transform to prices before computing the mean-squared-error loss, turning the loss into something close to percentage error and preventing the rare crash events that otherwise produce NaN loss and destabilize training. The model is continually pre-trained from TimesFM's public checkpoint using SGD with warmup and cosine decay on a dataset dominated by hourly cryptocurrencies and stocks, and the whole run completes in under an hour on eight V100 GPUs.","core_discovery":"The paper's central claim is that a general-purpose time-series foundation model can be repurposed into a financial price predictor by continued pre-training on price data, and that the resulting model outperforms both the unmodified TimesFM and a chance-level classifier on directional accuracy and macro F1 at every tested horizon (2, 4, 8, 16, 32, 64, 128). In a zero-cost mock trading exercise on S&P500 stocks, the market-neutral version of the strategy reaches an annualized Sharpe ratio of 1.68, annual returns of 3.6%, and a maximum drawdown of about -0.1% at the 128-step horizon. The same model is the only one among the fine-tuned model, original TimesFM, a random model, and an AR(1) model to achieve positive Sharpe ratios in all four markets tested (S&P500, TOPIX500, currencies, daily crypto), although it trails AR(1) on currencies and crypto.","pith_inferences":["Beyond the paper: because the loss is computed on log prices, the model effectively optimizes percentage error; this may make the fine-tuned checkpoint transfer more readily to assets whose price scales differ by orders of magnitude without reweighting, a property the paper does not test directly.","Beyond the paper: the random-masking scheme means the model is trained to forecast from arbitrary context lengths between 128 and 512; an ablation that fixes context length could reveal whether this flexibility is what drives the gains over the original model, or whether the data alone does.","Beyond the paper: since the fine-tuning set is dominated by hourly cryptocurrency and stock data, the same recipe on a granularity-balanced or synthetic-augmented dataset — directions the paper lists as future work — could plausibly close the gap with AR(1) on currencies and crypto, which is a testable prediction.","Beyond the paper: the reported 2023-only evaluation leaves open whether the 1.68 Sharpe is a momentum-style edge or a year-specific beta; applying the market-neutral strategy to 2024 and 2025 data with costs would discriminate between these."],"forward_implications":["The vanilla TimesFM cannot be used as-is for financial price prediction: it underperforms a chance-rate model at four of seven prediction horizons.","Continual pre-training on price data with a log-transformed MSE loss and random masking yields consistent accuracy and macro-F1 gains over the original model at every horizon from 2 to 128.","A market-neutral trading strategy using the 128-step predictions achieves a 1.68 annualized Sharpe ratio and 3.6% annual returns on S&P500 stocks in zero-cost mock trading.","The fine-tuned model is the only method among those compared with positive Sharpe ratios in every market tested, although it trails AR(1) on currencies and crypto.","The full fine-tuning run takes under an hour on eight V100 GPUs, showing that adapting a 200-million-parameter foundation model to a new domain can be cheap when the dataset is modest."],"supporting_citations":[{"why":"Supplies the pre-trained TimesFM model, its decoder-only architecture, and the continual pre-training recipe that the paper adapts with a log-MSE loss and random masking.","marker":"[14]"},{"why":"Defines the AR(1) autoregressive baseline that the fine-tuned model is compared against in mock trading and accuracy tests.","marker":"[2]"},{"why":"Raises doubts about whether large language model backbones help time-series forecasting, which motivates using the purpose-built TimesFM instead of an LLM-based forecaster.","marker":"[13]"},{"why":"The Darts benchmark is one of the evaluation suites on which the original TimesFM's strong forecasting performance is established.","marker":"[29]"},{"why":"The Monash archive is another standard benchmark used to position TimesFM as a strong general forecaster before fine-tuning.","marker":"[30]"}],"fun_headline_variants":["Fine-tuned TimesFM beats chance and AR(1) in market tests","TimesFM fine-tuned for finance beats AR(1) and chance","Fine-tuning TimesFM on prices outperforms chance and AR(1)","Price-predicting TimesFM beats chance and AR(1) after tuning","Fine-tuned TimesFM reaches 1.68 Sharpe trading S&P500"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that one out-of-sample year (2023), the chosen asset universe and horizons, and zero-cost mock trading fairly represent how the model would actually perform when traded or applied to other periods, and that continual pre-training dominated by hourly crypto and stock prices transfers to daily instruments.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuned TimesFM beats chance and AR(1) in market tests","TimesFM fine-tuned for finance beats AR(1) and chance","Fine-tuning TimesFM on prices outperforms chance and AR(1)","Price-predicting TimesFM beats chance and AR(1) after tuning","Fine-tuned TimesFM reaches 1.68 Sharpe trading S&P500"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001492,"raw_usage":{"total_tokens":5974,"prompt_tokens":918,"completion_tokens":5056,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":4958}},"tokens_in":534,"tokens_out":5056,"duration_ms":39964,"temperature":1.0,"reasoning_tokens":4958,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:37:05.230213+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the fine-tuned model on 2024–2025 data with realistic transaction costs and a 128-step market-neutral strategy: if the net annualized Sharpe ratio is not positive while directional accuracy stays above chance, the central claim of a durable fine-tuning edge fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the AR(1) autoregressive baseline that the fine-tuned model is compared against in mock trading and accuracy tests."}],"review_version":1}