{"id":"729a7e53-684a-4868-b905-da18a078da4a","arxiv_id":"2602.16224","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"APTF reweights training samples by loss-based predictability buckets and uses an amortization model to stabilize the estimates, improving accuracy across TSF and TSC baselines.","lead":"The paper presents a training-time reweighting method for time-series models: hard-to-predict samples get lower loss weights, so the model focuses on clean patterns while still learning from noisy ones. The authors report consistent accuracy gains across forecasting and classification baselines on standard benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Short-term TSF gains may rest on merged train/val/test sets; the central claim needs clean held-out Fund evaluation.","rationale":"The reader's weakest assumption focuses on HPL's use of loss magnitude as a predictability measure. That is a real theoretical risk, but the more immediate threat to the central claim is the evaluation protocol described in §4.1: if the training/validation/test sets are literally merged for the Fund datasets, the short-term TSF results cannot support any generalization claim. The reader's rationale does flag the merged-splits issue, but it is not the identified weakest assumption, hence 'partial.' My proposed test would settle whether the short-term results are artifacts. If they are, the abstract's quantitative claims shrink to long-term/TSC only, and the 'general framework' claim should be narrowed. This reinforces, rather than overturns, the reader's conditional verdict, so the verdict remains UNCHANGED.","tokens_in":20304,"tokens_out":6057,"duration_ms":58783,"concrete_test":"Inspect the released code's data loader for the Fund datasets and verify whether any test-set indices are used in the training loop. Then re-run Table 1 using a clean chronological split per fund series (e.g., first 70% train, next 10% validation, last 20% test), training APTF and the eight baselines with identical budgets, and report mean ± std WMAPE over the same four seeds. If the APTF-vs-baseline differences on Fund1–3 do not reproduce (particularly the TimeMixer 87.91 vs 89.96 and Scaleformer 96.74 vs 109.01 entries), the short-term TSF component of the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 states: 'In short-term forecasting, we merged the training, validation, and test sets across all fund datasets for model training and evaluation because they share similar TS patterns.' Read literally, this means the Fund test partitions were used in training, so Table 1's WMAPE improvements (e.g., TimeMixer Fund1 87.91 vs 89.96; Scaleformer Fund2 96.74 vs 109.01) are not evidence of generalization. The abstract's 3%–15.36% short-term range depends on these numbers. Even a less literal reading leaves the evaluation protocol undefined: no separate held-out test split is described for Fund1–3, and standard deviations are deferred to a 'full version.' Because the central claim is that APTF is a general, model-agnostic training wrapper, one contaminated evaluation pillar is enough to undermine the headline. The long-term TSF (Table 2) and UCR TSC (Table 3) results use standard splits and remain independent evidence, but they do not rescue the short-term claims as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Amortized Predictability-aware Training Framework (APTF), a training-time wrapper for time series forecasting (TSF) and classification (TSC). APTF identifies 'low-predictability' samples as those with high loss under the model being trained, groups them into loss-ordered buckets (Hierarchical Predictability-aware Loss, HPL), assigns decreasing weights to lower-predictability buckets, and gradually expands penalization across training stages to address the 'predictability evolution' issue. A second 'amortization model' is added to mutualize predictability estimates and reduce bias. The paper claims consistent accuracy improvements on eleven TSF baselines (2%–9.79% on long-term, 3%–15.36% on short-term tasks) and five TSC baselines on 128 UCR datasets, supported by ablations, comparisons to Co-teaching, WaveBound, ARL, and SPL, loss-landscape analysis, and released code.","tokens_in":20622,"tokens_out":13995,"duration_ms":129137,"significance":"If the empirical claims hold, APTF is a practically useful model-agnostic training wrapper: it changes only the loss, not the inference procedure, and it is tested across a broad model zoo (eleven TSF and five TSC models) and 128 UCR datasets. The paper has several concrete strengths: public code; standard 7:1:2 splits for long-term TSF and original splits for UCR TSC; win/tie summaries over 128 datasets; ablations isolating HPL and the amortization model (Tables 4–5); hyperparameter sensitivity analysis; and a flatter-loss-landscape argument for generalization. The central weakness is the short-term Fund evaluation in §4.1, which as written merges the training, validation, and test sets; Table 1 and the abstract's short-term improvement range therefore cannot be read as evidence of generalization. The long-term TSF and TSC pillars remain independent evidence, but they do not rescue the short-term claims as reported. The verdict is therefore conditional on a clean re-evaluation and on fixing the internal inconsistencies in Algorithm 2 and Table 3.","major_comments":[{"comment":"The short-term TSF evaluation protocol is invalid as written: 'In short-term forecasting, we merged the training, validation, and test sets across all fund datasets for model training and evaluation because they share similar TS patterns.' Taken literally, the test partitions were used in training, so the WMAPE improvements in Table 1 (e.g., TimeMixer Fund1 87.91 vs 89.96; Scaleformer Fund2 96.74 vs 109.01) are not evidence of generalization. If a pooled-split design was intended, the held-out split is nowhere defined, and the sentence must be rewritten. Since §4.2.1 and the abstract's 3%–15.36% short-term range rest on these numbers, this is a load-bearing flaw. Please re-run the Fund experiments with a proper held-out split and report per-seed standard deviations, or remove the short-term claims.","section":"§4.1, Table 1"},{"comment":"Algorithm 2 as printed cannot implement the described method. The loop 'for i = 1 to S' returns unconditionally in both branches ('return L, eL' at i=1 and 'return L/G, eL/G' at i>1), so the function exits during the first stage and the hierarchical multi-bucket-group computation for later stages—the core of HPL in §3.1.4 and Figure 2(c)—is unreachable. The listing implements only a single-bucket-group loss and contradicts the surrounding text. Please correct the pseudocode: remove the premature returns, make the stage loop or per-stage invocation explicit, define how bucket groups {B_j} and weight groups {W_j} are constructed and expanded across stages, and specify how the amortization model's bucket groups (E-hat, W-hat) are computed and trained; the one-step-delay mechanism in §3.2 is currently not formally specified.","section":"Algorithm 2"},{"comment":"The core identification mechanism is circular by construction: 'low predictability' is defined as high loss of the model being trained, and that same loss is then reweighted to produce the next update. No independent predictability measure is used; the claim that loss values 'reliably reflect sample predictability in early training' (§1) is asserted rather than demonstrated, and the reweighting alters the loss ordering it relies on. This is not fatal to the paper's main empirical claim—external test accuracy is the evidence—but it is a correctness risk and currently unsupported. A concrete fix: on a subset of datasets, report the stability of bucket assignments across seeds and across the source vs. amortization model, or correlate the identified low-predictability samples with an external noisiness measure. The per-batch bucketing (N = batch size, §3.1.1) also makes the ordering noisy a","section":"§3.1.1–§3.1.3"},{"comment":"Table 3's Win/Tie statistics are internally inconsistent. The OSCNN row reports 165 wins and 40 ties on 128 UCR datasets, which is impossible, and no other row's win+tie sum equals 128 (row sums range from 102 to 111), so the number of losses is undefined; the Average row (75/31) does not reconcile either. Either the column meanings or the counts are wrong. Because the TSC claim is that APTF 'consistently' improves five baselines, the counts are part of the evidence and must be corrected: report win/tie/loss numbers that sum to 128 for each model.","section":"Table 3"}],"minor_comments":[{"comment":"Typo: 'for example, in the in the forecasting task'.","section":"§3.1.1"},{"comment":"Naming inconsistency: the TSC baseline is called FCN in §4.1 but FCNet in Table 3.","section":"§4.1 vs Table 3"},{"comment":"The text states that 'average metrics and standard deviations are reported' for four seeds, but Tables 1–2 and most appendix tables omit standard deviations; the same paragraph defers them to a 'full version paper.' Please either include the variances or state explicitly which tables are complete, since several gains are small (e.g., NLinear ETTh2: 0.380 vs 0.383 in Table 2).","section":"§4.1, Tables 1–2"},{"comment":"The hyperparameter sensitivity discussion admits that the settings used in the main experiments are not optimal: 'the maximum number of buckets in the experiments is set to 9, which is not optimal' and 'the better epoch interval ε is 3 or 4, while used in the paper is 2.' This raises a question of whether the reported improvements are representative; please clarify whether these suboptimal choices apply evenly to baselines and APTF, and reconcile the text with Figure 4, which shows better performance at K>9.","section":"§4.5"},{"comment":"The WaveBound comparison on Fund datasets yields extreme WMAPE values (e.g., TimeMixer-WaveBound 206.9 vs 89.96 baseline), which suggests the baseline may not have been tuned; since the Fund evaluation is also subject to the merged-set problem (see main comment 1), the comparison is hard to interpret as reported.","section":"Table 6"},{"comment":"The 'first framework' claim would benefit from a qualifier ('to our knowledge') and a positioning paragraph against the broader noise-robust and sample-reweighting literature beyond Co-teaching, self-paced learning, ARL, and WaveBound.","section":"§1, Related Work"},{"comment":"The abstract reports improvements over 'eleven TSF and five TSC models,' but Tables 1–2 cover eight TSF baselines and Tables 9–10 cover three additional ones; aggregating these into the abstract ranges would be clearer if the per-model improvement used to compute the 2%–9.79% and 3%–15.36% ranges were reproduced in a summary table.","section":"Abstract, §4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper reads as a strong conference draft rather than a finished journal submission: several load-bearing numbers are deferred to a 'full version,' Table 3 contains impossible counts, and Algorithm 2's pseudocode returns before executing its own core loop. The short-term evaluation issue in §4.1 is the single most important decision point: if the merged-set sentence is literal, the Fund experiments must be redone; if it is a wording accident, the authors must provide the held-out protocol and numbers. I would not reject on the basis of the long-term TSF and UCR evidence, which uses standard splits and looks plausible, but the paper as submitted does not yet meet the bar. The authors' self-citations [41]–[44] are relevant prior work and are not a concern, though the editor may wish to confirm no overlap with those already-published results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is a model-agnostic training wrapper that reweights samples by loss buckets, and it shows consistent gains on standard-split long-term forecasting and UCR classification. The short-term fund evaluation, though, looks contaminated: Section 4.1 says they merged training, validation, and test sets across the fund datasets, so Table 1 and the abstract's 3%–15.36% short-term range are not evidence of generalization. The stress-test note is right. Even if the merge is a misstatement, the evaluation protocol is undefined: no held-out split is described for Fund1–3, and standard deviations are deferred to a \"full version.\" That is a load-bearing flaw for the short-term claims, not a minor one.\n\nWhat is actually new: the hierarchical bucket-group loss with progressive stage expansion, plus the amortization model (co-training style) for predictability estimation, is not in the cited Co-teaching, WaveBound, self-paced learning, or adaptive robust loss literature. The paper does well by testing across eleven TSF and five TSC baselines, and the long-term TSF results on ETT, Weather, Exchange, Electricity, and Traffic use standard 7:1:2 splits and show consistent, modest improvements. The TSC results on 128 UCR datasets also show small but consistent accuracy gains. Those two pillars are independent evidence that the framework can help.\n\nSoft spots beyond the fund issue: Algorithm 2 is internally inconsistent—it returns inside the loop and the loop structure doesn't make sense, so the pseudocode needs a rewrite. The amortization model is underspecified: what it trains on, how the one-step delay works, and how it avoids learning the same biased signal as the source model are all vague. The circularity concern—defining predictability by the loss of the model being trained, then reweighting that same loss—is real but not fatal, because the long-term/TSC results are measured on external test accuracy. Still, the authors should address the risk that loss ordering gets corrupted by the reweighting itself.\n\nBottom line: the paper is for applied time-series researchers looking for a plug-in training method. It deserves a serious referee, but the short-term evaluation must be redone with clean held-out splits, and the algorithm and amortization details need to be specified properly. Accept for peer review with a clear expectation of major revision.","headline":"The long-term and UCR results make the core idea worth a look, but the short-term Fund evaluation contaminates train and test, so the headline range needs a redo.","tokens_in":21039,"tokens_out":1913,"would_cite":false,"duration_ms":18926,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A model-independent training wrapper that down-weights low-predictability samples during training is claimed to improve time series forecasting and classification across sixteen existing models.","keywords":["time series forecasting","time series classification","time series analysis","deep learning techniques","noise robust learning","loss reweighting","predictability-aware training"],"falsifier":"One concrete test: on a synthetic dataset where each sample's noise level is known exactly, train the same model with APTF, with random bucket assignments using identical weights, and with ordinary training. If random buckets match APTF's test-loss gains, then loss-based ordering is not the active ingredient; if APTF hurts on clean datasets without low-predictability samples, the down-weighting is harming rather than helping.","tokens_in":20223,"feed_emoji":"📈","tokens_out":5912,"duration_ms":49917,"temperature":0.7,"pith_summary":"This paper claims that time series models—both forecasting and classification—can be made more accurate and better generalizing by changing only how each training sample's loss is weighted, not the model itself. The proposed approach sorts samples by their current loss into buckets, treats high-loss samples as low-predictability, and progressively down-weights them while retaining a hierarchical memory of earlier bucket partitions. An auxiliary 'amortization' model reduces the source model's bias in this sorting by passing its own bucket assignments to the source one step later. On eleven forecasting and five classification models, the paper reports consistent error reductions, averaging 2–9.79% on long-term and 3–15.36% on short-term forecasting tasks. If true, any existing time series model could be boosted at training time without any change at inference.","feed_headline":"Training wrapper cuts forecast error on 16 time-series models","feed_subtitle":"Down-weighting low-predictability samples during training improves forecasting and classification with zero inference overhead.","key_machinery":"The load-bearing mechanism is HPL, the hierarchical predictability-aware loss: a bucketing scheme that orders training samples by loss magnitude, assigns monotonically decreasing weights to higher-loss buckets, and, at each training stage, computes the loss as an average over several bucket groups from previous stages together with the current one. The amortization model—a second network that supplies the source model with its own bucket assignments with a one-step delay—complements HPL by absorbing the source's bias in estimating predictability.","core_discovery":"The paper's central claim is that a sample's current training loss is a serviceable ordering of its predictability, and that reweighting the loss accordingly—down-weighting high-loss (low-predictability) samples and up-weighting low-loss ones—improves convergence and generalization. The hierarchical predictability-aware loss (HPL) implements this by partitioning each batch into K loss-ordered buckets, assigning decreasing weights to higher-loss buckets, and, as training stages advance, averaging over multiple bucket groups so that earlier fine-grained partitions are not discarded. The amortization model is a second network that computes its own bucket assignments and feeds them to the source","pith_inferences":["Because the ordering signal is just the loss, the same idea could be transferred to other sequence tasks (speech, event logs) and to anomaly detection, where predictability is a natural target.","A direct test of the causal mechanism would be to compare APTF against random bucket assignments with the same weights; if random buckets perform as well, the loss-based ordering is not what drives the gains.","The paper's own appendix notes that full per-dataset standard deviations and per-horizon results are omitted, so the consistency claim is not fully checkable from the text alone.","The paper also states its default hyperparameters are not the tuned optimum (e.g., epoch interval 2 vs. a better 3–4), so the reported gains likely underestimate the method's ceiling."],"forward_implications":["If the claim holds, any existing TSF or TSC model can be improved by wrapping its loss with HPL, with no change to inference or architecture.","The gains span very different architectures—Transformers, CNNs, and linear models—so the mechanism is orthogonal to model design.","HPL alone yields most of the gain; the amortization model adds a further 1.4–2.1% reduction, so the core effect is the loss reweighting.","The paper reports flatter loss landscapes after training with APTF, meaning the improved test accuracy is accompanied by better generalization, not just better fit."],"fun_headline_variants":["Loss-weighting trick improves 16 TS models","Down-weight noisy samples for better forecasts","APTF: training framework reweights noisy samples","Cut forecast error by reweighting low-predictability"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that a sample's current loss value is a faithful ordering of its true predictability, even after reweighting modifies those losses; if overfitting or the reweighting itself corrupts that ordering, HPL could systematically underweight informative samples.","fun_headline_variants_meta":{"raw":{"variants":["Loss-weighting trick improves 16 TS models","Down-weight noisy samples for better forecasts","APTF: training framework reweights noisy samples","Cut forecast error by reweighting low-predictability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000571,"raw_usage":{"total_tokens":2527,"prompt_tokens":726,"completion_tokens":1801,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":1754}},"tokens_in":470,"tokens_out":1801,"duration_ms":12612,"temperature":1.0,"reasoning_tokens":1754,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:36:09.463399+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete test: on a synthetic dataset where each sample's noise level is known exactly, train the same model with APTF, with random bucket assignments using identical weights, and with ordinary training. If random buckets match APTF's test-loss gains, then loss-based ordering is not the active ingredient; if APTF hurts on clean datasets without low-predictability samples, the down-weighting is harming rather than helping.","supporting_citations":[],"review_version":1}