{"id":"c029beea-603c-4ba4-b221-8a88b45b07bf","arxiv_id":"2505.18442","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A learnable fusor that reads meta-features of an input time series and weights 13 pre-trained forecasters per sample outperforms each individual model on most benchmark samples, including zero-shot settings.","lead":"A new framework called TimeFuse learns to combine predictions from many different forecasting models, choosing for each input time series which models to trust most, based on 24 cheap statistical features. The paper shows the combined forecaster usually beats the best single model on standard benchmarks, and works even on datasets not seen during training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Meta-feature sufficiency is the load-bearing assumption: the linear fusor cannot exploit sample-level distinctions not encoded in the 24 hand-crafted descriptors, and the paper never measures how much of the oracle ranking is recoverable.","rationale":"The reader's conditional verdict is fair. The empirical study is broad, the code is released, and the comparison against static ensembles in Figure 3 and Table 10 is a genuine strength. The single largest structural assumption is that 24 meta-features plus a linear fusor can realize the sample-level adaptivity that motivates the paper. Without a per-sample oracle comparison, the reported gains could be mostly an ensemble effect rather than evidence of adaptive selection; the adaptive component might be contributing little. This does not invalidate the paper, but it warrants the conditional verdict and a specific follow-up experiment. I agree with the reader's identification of the same assumption, and the proposed test makes it falsifiable in a single pass over the existing meta-training data.","tokens_in":35652,"tokens_out":10072,"duration_ms":92393,"concrete_test":"Use a held-out split of the meta-training sets. For each sample x, compute the oracle best-model index b*(x)=argmin_i L(f_i(x),y) and the oracle fused loss L*=min_{w in simplex} L(sum_i w_i f_i(x),y). Train the TimeFuse linear fusor and a non-linear model (e.g., gradient-boosted trees) on the 24 meta-features to predict b*(x) or the optimal weights. Compare three numbers: the loss of TimeFuse, the loss of the globally best single model, and L*. If TimeFuse captures less than half of the gap between the global-best loss and L*, the meta-features recover only a minority of the per-sample adaptivity, and the central claim is weakened. If it captures most of that gap, the sufficiency concern does not land. This directly tests the reader's weakest assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central promise of TimeFuse—sample-level adaptive fusion in Eq. (1)/(2) of Section 3.2—is only as strong as the information available to the fusor. The fusor is a linear map from the 24 meta-features of Section 3.1/Table 1 to a softmax weight vector, so any determinant of 'which model wins on this sample' that is not one of these descriptors is invisible to it. The paper's evidence for sufficiency is indirect: Table 6 shows each feature group contributes and that the 24 features roughly match TSFEL's 165 features, but matching a larger hand-crafted set does not establish that the features capture the per-sample ranking. The paper never compares fused predictions with the per-sample oracle (best base model or optimal weights), so it is unknown how much of the available adaptivity is actually recovered. Appendix D's own admission that validation-strong but test-weak models can hurt fusion under distribution shift reinforces that the learned mapping is not reliably capturing test-time optimality. Because the zero-shot generalization claim also depends on these features transferring to unseen datasets, this is the load-bearing point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TimeFuse, a framework for sample-level adaptive fusion of heterogeneous time-series forecasting models. Given a model zoo of independently trained base forecasters, TimeFuse extracts 24 hand-crafted meta-features from each input series and trains a linear fusor to output softmax weights over the base models, minimizing Huber loss on held-out validation predictions. The fusor is meta-trained jointly across datasets, enabling zero-shot application to unseen datasets. Experiments cover 16 benchmarks and 13 base models for long- and short-term forecasting, with ablations, efficiency measurements, comparisons to static ensembles and AutoML baselines, and a zero-shot evaluation. The main reported findings are that no single base model wins on all samples and that TimeFuse achieves near-universal improvements, with average MSE reductions of 3.61%–11.77% over top individual baselines.","tokens_in":35949,"tokens_out":5694,"duration_ms":49537,"significance":"If its claims hold, TimeFuse is a practically useful and conceptually clean contribution: it decouples fusor training from base-model training, is architecture-agnostic, interpretable through the learned weights, and inexpensive at inference time. The empirical study is extensive, including 16 datasets, 13 base models, ablations over feature groups, comparison with AutoGluon and a foundation model, and an explicit zero-shot protocol. The release of code is a further strength. The main risk is that the headline claims—'sample-level adaptive fusion', 'consistently outperforms', and 'near-universal improvement'—are not yet fully supported by the evidence as presented, because the paper lacks an oracle bound on the achievable fusion gain, reports no repeated runs or significance tests, and contains several results that are ties or losses (e.g., zero-shot on ETTm2).","major_comments":[{"comment":"The central claim of sample-level adaptive fusion is not directly supported, because the fusor is a linear function of the 24 hand-crafted meta-features and the paper never measures how much of the per-sample model ranking is recoverable from those features. Table 6 shows that each feature group contributes and that the 24-feature set roughly matches TSFEL's 165 features, but neither result establishes that the features capture which base model is best for a given sample. The absence of an oracle comparison (per-sample best base model or optimal fusion weights) leaves open the possibility that TimeFuse operates largely as a dataset-level or coarse ensemble rather than a genuinely sample-adaptive mechanism. I recommend adding an oracle upper bound and reporting rank-recovery statistics, such as agreement between fusor-selected models and the per-sample best model.","section":"Section 3.1–3.2, Eq. (2)"},{"comment":"Several headline improvements are within the noise of a single split, and at least one zero-shot result is a loss. In Table 2, ETTm2 MSE is 0.274 for TimeFuse, TimeXer, and TimeMixer, and Weather MSE is 0.240 versus 0.241; in Table 5, zero-shot TimeFuse on ETTm2 is worse than the best individual model on both MSE (0.1721 versus 0.1712) and MAE (0.2568 versus 0.2560). No repeated runs, confidence intervals, or paired significance tests are reported, so the claimed average reductions of 3.61%–11.77% and the statement that TimeFuse consistently outperforms individual models are not statistically grounded. Please add multi-seed results with variance and, where appropriate, paired tests over test samples.","section":"Section 4.1, Tables 2 and 5"},{"comment":"The limitation discussion concedes that models that are strong on the meta-training (validation) set but weak on the test set can hurt fusion, and that excluding such models can improve results. This is directly relevant to the zero-shot generalization claim, which rests on the transferability of the validation-to-test mapping. Because the fusor is trained on validation loss and applied at test time, the paper should quantify when the learned mapping transfers (e.g., by comparing validation and test rankings of base models per dataset) and should compare TimeFuse against a dataset-identity-only baseline or per-dataset static weights to show that the per-sample variation in Figure 4 is predictive rather than noise.","section":"Appendix D and Section 3.2"}],"minor_comments":[{"comment":"The sentence 'Formally, let Xin ∈ R^{Tin×d} denote an input time series...' is duplicated in consecutive paragraphs; please remove one occurrence.","section":"Section 2"},{"comment":"The metric formulas for MSE, MAE, RMSE, and MAPE omit normalization by the number of samples and series elements; as written, MSE is a sum rather than a mean, which conflicts with the reported magnitudes.","section":"Appendix A.2"},{"comment":"The header contains the typo 'Forcast' (should be 'Forecast'), and the paper inconsistently uses both 'TimeFuse' and 'TIMEFUSE'; please standardize the naming.","section":"Table 11"},{"comment":"The header layout is confusing because 'Normal', 'Zero-shot', and 'Best Individual' are interleaved with repeated 'TIMEFUSE' entries; clearer column grouping or captions would help readability.","section":"Table 5"},{"comment":"The visualization of learned fusor weights is referenced in the text but does not appear in the provided manuscript; please ensure the actual figure is included.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple and the experimental breadth is a real strength. My recommendation is driven by the gap between the strong claims and the evidence: no oracle bound, no significance testing, and already visible ties/losses in the reported tables. These issues are fixable within the manuscript's scope, so I view major revision rather than rejection as the appropriate outcome."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on TimeFuse. It's a clean, honest empirical paper: a linear gating network that maps 24 meta-features to weights over 13 pre-trained forecasters, trained on validation data and evaluated zero-shot across datasets. The gating idea is old—this is essentially stacking with a feature-conditioned gater—but the specific package, with the heterogeneous model zoo, cross-task meta-training, and the thorough comparison against static ensembles and AutoML baselines, is a legitimate contribution. The results are mostly consistent: TimeFuse beats the best individual model on almost every benchmark, with real gains on short-term tasks and small but non-trivial gains on long-term ones. The code release and use of standard TSLib baselines are real pluses.\n\nThe soft spots are the ones the reader flagged. No repeated runs or significance tests; several long-term gains are within noise, e.g., Weather 0.240 vs 0.241. Zero-shot loses on ETTm2, though wins elsewhere. And the load-bearing assumption—that the 24 meta-features capture which base model wins on a given sample—is never directly tested. The paper shows each feature group contributes and that 24 features roughly match the 165-feature TSFEL set, but that's indirect. They never compare against the per-sample oracle (best base model or optimal weights), so we don't know how much of the available adaptivity is actually recovered. The appendix's admission that validation-strong models can hurt under distribution shift reinforces this concern. That said, this is a limitation, not a fatal flaw. The features are reasonable, and the ablation suggests they carry signal. The central claim, that sample-level adaptive fusion helps, is plausibly true and well-supported by the breadth of experiments.\n\nWho is this for? Anyone working in time-series forecasting who wants a cheap plug-in ensemble layer. It's not a field-reorganizing result, but it's a useful engineering contribution with an honest empirical evaluation. It deserves serious peer review. I'd send it out, with the request that the authors add oracle comparisons and ideally some measure of variance.","headline":"A solid, useful empirical paper on sample-level gating for heterogeneous forecasters—old idea, clean execution, with the meta-feature sufficiency assumption left untested.","tokens_in":36420,"tokens_out":2087,"would_cite":true,"duration_ms":17684,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TimeFuse claims that no single forecasting model wins on every sample, and that a simple per-sample linear fusion of 13 models, guided by 24 meta-features, outperforms the best individual model on up to 95.1% of test samples across long…","keywords":["time series forecasting","model ensemble","adaptive fusion","meta-features","zero-shot generalization","sample-level analysis"],"falsifier":"A concrete test: on a held-out set, find pairs of samples with nearly identical meta-feature vectors but different best base models. If such pairs are common, train a nonlinear fusor (or one with access to raw inputs) and show it beats the linear fusor on those pairs; that would refute the sufficiency of the 24-feature linear representation. The gap between TimeFuse and the oracle that always selects the best base model measures how much the meta-features leave on the table.","tokens_in":35516,"feed_emoji":"📈","tokens_out":6460,"duration_ms":51480,"temperature":0.7,"pith_summary":"The paper claims that no single state-of-the-art forecasting model wins on every test sample; each model tops a notable fraction of samples. It introduces TimeFuse, a framework that fuses a zoo of 13 heterogeneous models by learning per-input weights from a 24-dimensional meta-feature description of the input series. The fusor is a single-layer linear network trained on validation data across tasks, and it delivers near-universal gains: better than the best individual model on up to 95.1% of samples, with average MSE reductions of 3.61%–11.77% over top baselines on long-term benchmarks. The significance is that better forecasting can come from combining existing models adaptively rather than designing a new architecture. TimeFuse also beats static ensembles and shows zero-shot transfer to datasets never seen in meta-training.","feed_headline":"Adaptive fusion beats the best time-series model on 95% of samples","feed_subtitle":"A single-layer fusor learns per-input weights from 24 meta-features and cuts MSE by up to 11.8%.","key_machinery":"The load-bearing mechanism is the fusor: a softmax-normalized single-layer linear network that maps a 24-dimensional meta-feature vector to weights over the k base models, with the fused output being the weighted sum of base model predictions. The meta-features are crafted from four families—statistical (mean, skewness, kurtosis), temporal (autocorrelation, stationarity, rate of change, AR(1) coefficients), spectral (dominant frequency, spectral entropy, spectral variation), and multivariate (covariance and cross-correlation statistics)—and the paper shows this 24-feature set matches the performance of a 165-variable feature set from a time-series feature extraction library. The fusor is trained with Huber loss on validation-set triplets (meta-features, base predictions, ground truth), decoupled from base model training, with batch-level mixing and oversampling across tasks to support cross-task and zero-shot generalization.","core_discovery":"The central discovery is that sample-level adaptive fusion consistently outperforms the best single model across a wide range of forecasting tasks. On seven long-term benchmarks (ETT, Weather, Electricity, Traffic) and short-term benchmarks (PEMS, EPF), the fused forecast beats the task-specific best individual model consistently, with average MSE reductions of 3.61% relative to TimeXer, 6.88% relative to TimeMixer, 11.77% relative to PAttn, and 8.39% relative to iTransformer. On the PEMS traffic datasets the fused model reduces MAPE by roughly 10–20% over the top three individual models, and in zero-shot evaluation (fusor trained on all other datasets) it still outperforms the best individual model in most cases. The learned weights are interpretable, reflecting known model strengths: low-stationarity inputs are routed toward Non-stationary Transformer and spectrally complex inputs toward TimeMixer.","pith_inferences":["Because the fusor is linear in 24 meta-features, a natural next test is whether a nonlinear fusor on the same features, or a fusor that sees raw inputs, closes the remaining gap to the oracle that always picks the best base model.","The validation-set dependence implies a concrete failure mode: if a deployment distribution shifts away from validation, the paper's own distribution-shift discussion suggests pruning or re-weighting models may be needed, and the framework could be paired with test-time adaptation.","The meta-feature descriptions are dataset-agnostic, so TimeFuse could act as a portable controller across heterogeneous forecasters, including statistical, neural, and foundation models, using the same 24-feature interface."],"forward_implications":["The fusor's accuracy improves monotonically as more diverse base models are added to the zoo, so the framework can absorb future model advances without re-architecting the fusion.","Static ensembles (mean or median over a validation-selected top-k) are consistently beaten, removing the need to pick an ensemble size per dataset.","Zero-shot fusion trained on other datasets still beats the best individual model in most cases, so a single fusor can serve new forecasting tasks without per-dataset training.","The learned fusion weights give a per-model diagnostic: they reveal which temporal properties each base model handles well, which could guide future model design."],"supporting_citations":[{"why":"Supplies the 165-variable TSFEL feature set that the paper's 24 meta-features match in performance, validating the compact feature design.","marker":"Barandas et al. (2020)"},{"why":"Provides the empirical evaluation of time-series feature sets that motivates the choice of meta-features for characterizing input series.","marker":"Henderson & Fulcher (2021)"},{"why":"Provides the standard toolkit and official model configurations used to implement all 13 base models, ensuring fair comparison and reproducibility.","marker":"Wang et al. (2024b)"},{"why":"Supplies the optimal settings and benchmark results used in the sample-level inspection showing that no single model tops all test samples.","marker":"Wu et al. (2023)"},{"why":"Introduces the robust loss used to train the fusor, preventing outlying individual models from destabilizing the learned fusion weights.","marker":"Huber (1992)"}],"fun_headline_variants":["Adaptive fusion beats the best single model in time-series forecasting","Per-sample fusion of models improves forecasting over any single model","Meta-features guide adaptive fusion to beat top forecasters","TimeFuse: sample-level fusion consistently outperforms best model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that carries the method is that the 24 hand-crafted meta-features capture everything needed to know which base model will be most accurate on a given input; since the fusor is linear, any two inputs with similar meta-feature vectors receive similar weights, so differences between samples that live outside this feature space are invisible to the fusion.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive fusion beats the best single model in time-series forecasting","Per-sample fusion of models improves forecasting over any single model","Meta-features guide adaptive fusion to beat top forecasters","TimeFuse: sample-level fusion consistently outperforms best model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00028,"raw_usage":{"total_tokens":1660,"prompt_tokens":945,"completion_tokens":715,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":646}},"tokens_in":561,"tokens_out":715,"duration_ms":5986,"temperature":1.0,"reasoning_tokens":646,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:30:22.461430+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: on a held-out set, find pairs of samples with nearly identical meta-feature vectors but different best base models. If such pairs are common, train a nonlinear fusor (or one with access to raw inputs) and show it beats the linear fusor on those pairs; that would refute the sufficiency of the 24-feature linear representation. The gap between TimeFuse and the oracle that always selects the best base model measures how much the meta-features leave on the table.","supporting_citations":[],"review_version":1}