{"id":"74119984-16c0-4bf6-be8e-13dd31659e90","arxiv_id":"2506.06288","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"The paper reports that a time-series transformer pretrained on public and proprietary financial data becomes competitive on financial tasks after fine-tuning, while zero-shot general forecasting remains behind MOIRAI.","lead":"This paper trains a time-series transformer, Delphyne, on public data plus proprietary financial data and reports benchmark results for forecasting, probabilistic prediction, and anomaly detection. A reader might care because it tests whether adding financial data to a general pretraining corpus helps models adapt to finance tasks after fine-tuning.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Monash fine-tuned aggregate is computed on a subset of datasets and compared to MOIRAI's full-benchmark aggregate, so the 'rivaling MOIRAI' claim may not survive a matched-subset comparison.","rationale":"The reader's conditional verdict is appropriate, but the specific weakest assumption identified is likely incorrect. The paper's phrase 'partition out the same forecast length as the test set' (Sec. 5.2) indicates a validation split of equal horizon carved from the training portion of each Monash series, not use of the actual test set. The genuine load-bearing weakness is that the fine-tuned Monash aggregate (0.536 for Delphyne-A-FT) is computed only over datasets with sufficient training length, while the MOIRAI aggregate (0.58) is computed over the full Monash benchmark. Because datasets excluded from fine-tuning include cases where Delphyne's zero-shot performance is much worse than MOIRAI (Sunspot, Pedestrian Counts), the headline comparison is unmatched and the central claim of competitive general performance is not directly supported. This is fixable by recomputing aggregates on a common subset, so the paper should remain conditional rather than being rejected. The missing code, weights, and proprietary financial data further support conditionality, as independently reproducing the financial task results is not currently possible. The financial experiments themselves (Sec. 5.1) use temporally separated splits and show consistent improvements after fine-tuning, which gives partial independent support for the finance-specific part of the central claim. Overall, the concern is substantive but not destructive: a matched-subset reanalysis could confirm or refute the headline comparison, and no internal inconsistency was found in the core architecture or training procedure.","tokens_in":40613,"tokens_out":7633,"duration_ms":75696,"concrete_test":"Restrict both aggregations to the identical set of Monash datasets that have Delphyne-A-FT entries in Table 17 (i.e., exclude M1 Monthly, M3 Monthly, M3 Other, Tourism Quarterly, CIF 2016, Vehicle Trips, Carparts, Hospital), recompute the geometric mean of MAE/Naive for Delphyne-A-FT and MOIRAI, and compare. If MOIRAI's matched-subset aggregate is equal to or below 0.536, the 'rivaling best model' claim fails; if Delphyne still wins, the concern resolves. Additionally, record the number of fine-tuning gradient steps or epochs used for early stopping to test the 'few fine-tuning steps' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Sec. 5.2 the headline Monash result is the aggregate normalized MAE: Delphyne-A-FT 0.536 vs MOIRAI 0.58 (Fig. 6, Tables 16-17). These numbers are not computed over the same datasets. Appendix B.1 states that several Monash datasets are excluded from fine-tuning because their training data is shorter than 20 time steps after splitting test/validation; Table 17 shows no FT entry for M1 Monthly, M3 Monthly, M3 Other, Tourism Quarterly, CIF 2016, Vehicle Trips, Carparts, and Hospital. Delphyne's 0.536 is therefore a geometric mean over a subset, while the MOIRAI 0.58 aggregate covers all 28 datasets. On the excluded datasets Delphyne's zero-shot MAE is often worse than MOIRAI (e.g., Sunspot 3.51 vs 0.08, Pedestrian Counts 52.99 vs 23.17), so dropping them can only help Delphyne. The abstract's claim of competitive performance on publicly available datasets and the Sec. 5.2 claim of rivaling MOIRAI rest on this unmatched comparison. The reader's stated concern about using the test horizon as validation appears to be a misreading: 'partition out the same forecast length as the test set' (Sec. 5.2) describes carving a validation set of equal horizon from the training split, not test leakage. The real problem is the unmatched dataset subset. Separately, 'few fine-tuning steps' is never quantified; Sec. 5.2 reports only early stopping, and Fig. 2's x-axis is pre-training steps.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Delphyne, an encoder-only transformer pretrained on the LOTSA public corpus and on proprietary financial data, with any-variate attention, missing-data and forecast masks, and a mixture-of-Student-T output distribution. The authors document a negative transfer effect when financial and non-financial data are mixed, argue that fine-tuning is the remedy, and evaluate Delphyne on Monash short-term forecasting, long-horizon out-of-distribution forecasting, probabilistic forecasting, anomaly detection, and four financial tasks (stock risk, volatility, intraday bars, and company nowcasting). The central claim is that Delphyne is competitive with existing foundation and full-shot models after few fine-tuning steps on public benchmarks and superior on financial tasks.","tokens_in":40926,"tokens_out":7613,"duration_ms":72035,"significance":"If the claims are supported, the paper would make a meaningful contribution: it provides evidence that a time-series foundation model can incorporate proprietary financial data and recover from negative transfer through fine-tuning, and it gives a systematic study of design choices (masking, context length, variate handling, output distribution). Strengths of the paper include clean temporal separations in the financial experiments (pretraining data end in 2019 while evaluation windows start in 2021), consistent fine-tuning gains across most benchmarks, and a thorough ablation suite on synthetic data. The main weaknesses are that several headline comparisons are not computed on matched data subsets, the 'few fine-tuning steps' claim is not quantified, and some broad performance claims are contradicted by the paper's own tables. These issues are fixable but currently overstate the evidence.","major_comments":[{"comment":"The headline Monash comparison is computed on unmatched dataset subsets. Table 17 has no fine-tuned entries for M1 Monthly, M3 Monthly, M3 Other, Tourism Quarterly, CIF 2016, Vehicle Trips, Carparts, and Hospital, so the Delphyne-A-FT aggregate 0.536 is a geometric mean over the remaining datasets, whereas the MOIRAI 0.58 aggregate in Table 16 covers all 28 datasets. On several of the excluded datasets Delphyne-A-ZS is much worse than MOIRAI (e.g., Sunspot 3.51 vs. 0.08; Pedestrian Counts 52.99 vs. 23.17), so dropping them can only improve Delphyne's aggregate. The sentence in Sec. 5.2 that fine-tuned Delphyne is 'rivaling the best model, MOIRAI' is not supported by the current tables; please report a matched-subset comparison and state the number of datasets in each aggregate.","section":"Sec. 5.2, Tables 16-17"},{"comment":"The claim of adaptation with 'few fine-tuning steps' and 'minimal iterations' is never quantified. Sec. 5.2 reports only that early stopping is used, Fig. 2's x-axis is pre-training steps, and no experiment reports the number of gradient updates, epochs, or wall-clock time for fine-tuning the main models. Without this information, the 'few fine-tuning steps' qualifier in the abstract is unsupported. Please report fine-tuning cost (e.g., steps or epochs) for the Monash, long-horizon, and financial experiments, or remove the qualifier.","section":"Abstract and Sec. 2"},{"comment":"The probabilistic-forecasting claim is broader than the evidence. After fine-tuning, Delphyne-A's CRPS on Electricity is 0.140, compared with 0.048-0.065 for full-shot baselines and 0.055 for MOIRAI zero-shot; on Solar the fine-tuned CRPS is 1.306, compared with 0.420-0.518 for baselines and 0.419 for MOIRAI zero-shot. The statement that fine-tuned Delphyne 'achieves the best results across various datasets' is inaccurate for these datasets. Please restrict the claim to the datasets where it holds, or add a discussion of why fine-tuning does not close the gap on Electricity and Solar.","section":"Sec. 5.4 and Table 12"},{"comment":"The financial evaluations are performed on proprietary data of the same types that appear in Delphyne's pretraining corpus (company consumer transactions, intraday bars, stock returns, and related fields). While the temporal split is clean (pretraining ends in 2019 and evaluation starts in 2021), the 'superior performances on various financial tasks' claim is an in-domain result; the comparison methods (MOIRAI, MOMENT, TTM, PatchTST) did not have access to this proprietary pretraining data, so the comparison is not controlled for pretraining data. Please state this limitation explicitly and, if possible, add an external financial benchmark that is not part of the pretraining corpus.","section":"Sec. 5.1 and Appendix B.2"}],"minor_comments":[{"comment":"The caption says 'MSE results' but the table reports MAE; please correct the caption.","section":"Table 10 caption"},{"comment":"MOIRAI is cited as (Wang et al., 2024a), but the correct reference is Woo et al. (2024); the current citation points to a graph negative-transfer paper.","section":"Appendix J, Table 29"},{"comment":"The blank entries in Table 1 should be marked as not applicable rather than left empty, to avoid ambiguity about whether the measurement was not performed.","section":"Table 1"},{"comment":"The sampling probabilities for LOTSA sum to approximately 85.1 and those for finance to approximately 14.2, not the stated 85 and 15; please reconcile the rounding.","section":"Table 6"},{"comment":"The sentence 'We partition out the same forecast length as the test set, used as validation data for finetuning the model' can be misread as using test data for validation; please clarify that the validation set is carved from the training split.","section":"Sec. 5.2"},{"comment":"The x-axis of Fig. 2 is described as training steps but the axis label is missing; please add a label and, if possible, indicate the fine-tuning step count on the same axis.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is an engineering paper with one genuinely interesting idea—negative transfer in pretrained time-series models, demonstrated with synthetic GARCH vs wavelet data—and a set of financial evaluations that are mostly clean. The Monash comparison, though, is not apples-to-apples, and the abstract overstates the zero-shot story.\n\nWhat's good: the synthetic negative-transfer experiments (both the direct pretraining comparison and the Bayesian MCMC version) are the most novel part, and they make a real point: mixing finance and non-finance data hurts zero-shot performance, and fine-tuning recovers it. The finance tasks use temporally separated splits, and fine-tuned Delphyne-A beats MOIRAI, PatchTST, GARCH and the provider baseline on stock NLL, bars volume, and nowcasting. The ablations on context length, masking ratio, any-variate attention, and output distribution are useful and honestly reported; some are on synthetic data, but they inform design choices.\n\nWhere it gets soft. The headline Monash number in Fig. 6 compares Delphyne-A-FT's aggregate (0.536) to MOIRAI's 0.58, but they are not computed over the same datasets. Appendix B.1 and Table 17 show Delphyne has no fine-tuned entries for M1 Monthly, M3 Monthly, M3 Other, Tourism Quarterly, CIF 2016, Vehicle Trips, Carparts, and Hospital—datasets where its zero-shot is often much worse than MOIRAI (Sunspot 3.51 vs 0.08, Pedestrian Counts 52.99 vs 23.17). Dropping those can only help the aggregate, so the 'rivaling MOIRAI' claim rests on a matched-subset comparison that is never shown. That's the main flaw, and it's fixable: report the aggregate over the common subset. The reader's validation-leak concern is a misread; 'partition out the same forecast length as the test set' describes carving a validation set from the training split, not test leakage.\n\nAlso, the abstract says 'competitive performance ... with few fine-tuning steps,' but zero-shot CRPS on Electricity and Solar is far behind MOIRAI (0.159 vs 0.055; 0.905 vs 0.419), and fine-tuning on Solar actually makes it worse (1.306). The paper does admit these weaknesses in Sec 5.3, but the abstract doesn't reflect them. 'Few fine-tuning steps' is never quantified—early stopping is reported, not steps. And no code, weights, or proprietary data are released, so the trained model has to be taken on faith.\n\nVerdict: the negative-transfer story and the finance results are worth refereeing. A serious referee should ask for the matched-subset Monash comparison and a more measured abstract. I'd send it to review, and I'd cite the negative-transfer synthetic study if I were working on time-series foundation models.","headline":"Negative-transfer study is the real contribution; the Monash aggregate is computed on a biased subset and the abstract overpromises.","tokens_in":41489,"tokens_out":2741,"would_cite":true,"duration_ms":24898,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A time-series transformer pre-trained on general and financial data matches foundation and dedicated models after a few fine-tuning steps, and beats them on finance tasks.","keywords":["time-series foundation models","negative transfer","fine-tuning","financial time series","probabilistic forecasting","any-variate attention","nowcasting","mixture of Student-t distributions"],"falsifier":"Re-run the Monash fine-tuning evaluation with validation and hyperparameter selection confined to the training split of each dataset (or to a time-based split that never touches the test horizon), and check whether Delphyne-A-FT still posts an aggregate normalized MAE near 0.536 and still rivals MOIRAI; if the clean-protocol numbers degrade materially, the 'competitive after few fine-tuning steps' claim loses its support. A supporting check is to replicate the financial tasks on years after 2023, well beyond the end-2019 pre-training cutoff, to test whether fine-tuned finance gains persist under drift.","tokens_in":40351,"feed_emoji":"📈","tokens_out":20657,"duration_ms":162433,"temperature":0.7,"pith_summary":"This paper tries to establish that the value of a pre-trained time-series model lies in fine-tuning, not zero-shot transfer, and that a model pre-trained on general plus financial data can therefore serve both worlds. It documents a negative-transfer effect—mixing financial data into the pre-training corpus degrades zero-shot forecasts on general benchmarks, in synthetic experiments (GARCH versus wavelet generators) and in the full model alike—and proposes Delphyne, an encoder transformer with any-variate attention, a missing-data mask alongside the forecast mask, and a mixture of Student-$t$ output distribution. The paper claims that after a few fine-tuning steps Delphyne becomes competitive with existing foundation and full-shot models on public benchmarks, and that it beats them on financial tasks such as stock-return risk, intraday volume, and company-revenue nowcasting. If correct, this reframes what a time-series foundation model is for: not immediate zero-shot generalization, but fast adaptation from small amounts of task data.","feed_headline":"Adding finance data hurts zero-shot; fine-tuning restores the edge","feed_subtitle":"The model's value appears after adaptation: one transformer handles general benchmarks and wins key finance tasks.","key_machinery":"The central mechanism is an encoder transformer built to tolerate the messiness of financial series: all variates are flattened and given identity embeddings, series are right-padded, split into patches of size 32, and a missing-data mask runs alongside the forecast mask so that irregular sampling, multiple frequencies, and contemporaneous (nowcasting) inputs do not misalign the data. Attention is any-variate attention with rotary position embeddings, and the forecast is a mixture of Student-$t$ distributions. Around this architecture sits the paper's main conceptual mechanism: the demonstration and removal of negative transfer. Using synthetic GARCH-versus-wavelet models, a Bayesian MCMC posterior analysis, and the Delphyne-A versus Delphyne-L comparison, the paper argues that heterogeneous pre-training data degrades zero-shot forecasts, and that fine-tuning is what 'unlearns' that bias—making the fine-tuning protocol, not the zero-shot prior, the load-bearing part of the system.","core_discovery":"On the paper's own terms, the central claim is that pre-training a time-series transformer jointly on the public LOTSA corpus and on financial data—stocks, ETFs, currencies, commodities, and intraday bars—produces a model whose zero-shot performance on general tasks is worse than that of a model trained on LOTSA alone, yet whose fine-tuned performance matches or beats the best available baselines. Delphyne-A (LOTSA plus finance) shows higher zero-shot MAE than Delphyne-L (LOTSA only) on ETTh2 and on the Monash archive, but after fine-tuning the two converge, and Delphyne-A-FT posts the best numbers on the financial tasks: next-day stock-return risk likelihood (NLL 1.741), 78-step intraday bar-volume error (MSE 0.551), and nowcast MAE of 0.071 on company revenue growth, with quantile-coverage statistics close to optimal. The paper reads these results as evidence that the negative transfer induced by heterogeneous pre-training data is real and inherent—confirmed by GARCH-versus-wavelet synthetic models and a Bayesian MCMC posterior analysis—and that fine-tuning is the mechanism that 'unlearns' the pre-training bias.","pith_inferences":["A testable extension the paper leaves open: give the finance-only model (Delphyne-F) a much larger fine-tuning budget on financial tasks and check whether the general-domain prior in Delphyne-A eventually becomes irrelevant or actively harmful, which would sharpen the question of which pre-training data actually pays for itself.","The missing-data mask suggests an imputation use the paper does not develop: evaluate Delphyne on genuinely irregular calendars (holidays, half-days, multi-frequency lags) where left-padding and zero-filling baselines should degrade, and measure whether the mask preserves forecast calibration.","Because the financial pre-training data stops at end-2019, the reported finance results cover 2020-2023; a natural stress test is whether the fine-tuned advantage survives rolling windows after 2023, when the pre-training distribution is older.","The negative-transfer diagnostic could be repurposed: a practitioner facing a new domain could read zero-shot degradation as a cheap signal of domain mismatch, and use its size to decide how much fine-tuning data to collect."],"forward_implications":["A single pre-trained model covering general plus financial data can serve both worlds: after a short fine-tuning phase it reaches an aggregate 0.536 normalized MAE on the Monash archive and rivals the best foundation model on several datasets.","On financial tasks, fine-tuning delivers the wins: Delphyne beats the Student-$t$ GARCH baseline in next-day return likelihood, the fine-tuned Delphyne variants post the best errors on intraday bar volume and company-revenue nowcasting, and coverage statistics sit close to optimal.","Zero-shot results systematically understate a model trained on heterogeneous data, so evaluations of time-series foundation models should report fine-tuned performance; the negative-transfer penalty is a property of the data mixture, not a failure of the architecture.","Pre-training choices such as longer context lengths and lighter masking ratios improve downstream few-shot adaptation, so these hyperparameters should be selected with the fine-tuning regime in mind.","The same fine-tuned model also handles probabilistic forecasting (CRPS/MSIS) and anomaly detection, indicating that one foundation model can cover forecasting, uncertainty quantification, and monitoring after light adaptation."],"supporting_citations":[{"why":"Supplies the LOTSA pre-training corpus, the MOIRAI baseline that Delphyne is compared against, and the any-variate attention mechanism Delphyne adopts.","marker":"(Woo et al., 2024)"},{"why":"Defines negative transfer, the central phenomenon the paper documents in time-series pre-training and counters through fine-tuning.","marker":"(Wang et al., 2019)"},{"why":"Provides the Monash archive, the main general benchmark whose zero-shot and fine-tuning results support the competitive-performance claim.","marker":"(Godahewa et al., 2021)"},{"why":"Gives the GARCH data generator for the synthetic negative-transfer experiments and the Student-t GARCH baseline for financial risk tasks.","marker":"(Bollerslev, 1986)"},{"why":"Grounds the Bayesian view of pre-training as posterior inference, which the MCMC experiment uses to show negative transfer is inherent to the data mixture.","marker":"(Müller et al., 2022)"},{"why":"Introduces patching of time series into tokens, which Delphyne adopts, and provides the PatchTST full-shot baseline on several benchmarks.","marker":"(Nie et al., 2023)"},{"why":"Sets up the stock-return risk-analysis task and the GARCH-with-Student-t benchmark used for the financial evaluations.","marker":"(Tepelyan & Gopal, 2023)"},{"why":"Provides the TimesFM zero-shot baseline used in the long-term out-of-distribution forecasting comparison.","marker":"(Das et al., 2024)"},{"why":"Supplies the MOMENT baseline, the linear-probing results, and the anomaly-detection evaluation setup adopted for the UCR archive.","marker":"(Goswami et al., 2024)"}],"fun_headline_variants":["Negative transfer from finance data, fine-tune to fix","Zero-shot suffers, fine-tuning wins: Delphyne's story","Finance pre-training hurts zero-shot; fine-tune for wins","Delphyne: fine-tuning beats the pre-training bias"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the Monash fine-tuning protocol: validation for early stopping and hyperparameter selection is carved out of the same hold-out test horizon that later produces the reported scores, so if that protocol leaks test information, the fine-tuned gains on public benchmarks—including the aggregate 0.536 normalized MAE—are optimistic.","fun_headline_variants_meta":{"raw":{"variants":["Negative transfer from finance data, fine-tune to fix","Zero-shot suffers, fine-tuning wins: Delphyne's story","Finance pre-training hurts zero-shot; fine-tune for wins","Delphyne: fine-tuning beats the pre-training bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000458,"raw_usage":{"total_tokens":2327,"prompt_tokens":1004,"completion_tokens":1323,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":1253}},"tokens_in":620,"tokens_out":1323,"duration_ms":8922,"temperature":1.0,"reasoning_tokens":1253,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:09:22.681994+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Monash fine-tuning evaluation with validation and hyperparameter selection confined to the training split of each dataset (or to a time-based split that never touches the test horizon), and check whether Delphyne-A-FT still posts an aggregate normalized MAE near 0.536 and still rivals MOIRAI; if the clean-protocol numbers degrade materially, the 'competitive after few fine-tuning steps' claim loses its support. A supporting check is to replicate the financial tasks on years after 2023, well beyond the end-2019 pre-training cutoff, to test whether fine-tuned finance gains persist under drift.","supporting_citations":[{"cited_title":"Unified Training of Universal Time Series Forecasting Transformers","cited_arxiv_id":null,"evidence_quote":"Supplies the LOTSA pre-training corpus, the MOIRAI baseline that Delphyne is compared against, and the any-variate attention mechanism Delphyne adopts."},{"cited_title":"A decoder-only foundation model for time-series forecasting","cited_arxiv_id":null,"evidence_quote":"Provides the TimesFM zero-shot baseline used in the long-term out-of-distribution forecasting comparison."}],"review_version":1}