{"id":"7750e260-3d1f-4fa9-b208-91295e213871","arxiv_id":"2608.01857","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CosDir aligns the first-difference vectors of predictions and targets via cosine similarity, improving directional accuracy while preserving MSE.","lead":"A new loss function called CosDir trains time series forecasters to predict the direction of change, not just the value. It adds a scale-invariant cosine term to the standard error loss, improving directional accuracy across 15 models and 13 datasets without degrading point accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's small-move claim contradicts Figure 4: gains concentrate on large moves, undermining the stated mechanism.","rationale":"The reader's weakest_assumption focuses on the surrogate gap between cosine similarity and per-step DA. That is a valid theoretical concern, but it is not load-bearing because DA is directly measured in every experiment; the empirical improvements are real regardless of surrogate tightness. The more serious issue is the contradiction between the abstract's 'small moves' narrative and Figure 4, which shows the gains concentrate on large moves. This undermines the paper's stated central contribution and mechanism, even though the headline empirical claim (CosDir improves DA) is well supported by extensive experiments with significance tests. I recommend keeping the reader's CONDITIONAL verdict, but for a more specific reason: the authors should either provide evidence of small-move DA improvement or revise the abstract and motivation to accurately describe where the gains come from.","tokens_in":38393,"tokens_out":7293,"duration_ms":72481,"concrete_test":"Recompute the decile analysis of Figure 4(a) across all five financial datasets and at least three backbone families (e.g., DLinear, PatchTST, TimesNet) using the released code. For each seed, compute the DA gain (MSE+CosDir minus MSE) within the smallest |Δy| decile and test whether the mean gain is significantly positive (paired one-sided test, p<0.01). If the gain in the smallest decile is not significant, the abstract's claim that CosDir improves small-move direction is unsupported and should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central motivation, stated in the abstract and introduction, is that MSE-trained forecasters fail on the direction of small moves and that CosDir re-injects learning signal exactly where MSE neglects it. However, the only direct evidence, Figure 4(a), shows that both MSE and MSE+CosDir achieve near-chance DA on the smallest-move decile, and CosDir's improvement grows monotonically with move size, being largest on the largest moves. The text explicitly says 'CosDir does not manufacture signal on the near-random smallest moves.' This is an internal inconsistency: the loss's advertised mechanism (scale-invariant directional gradient on small moves) does not translate into improved DA for small moves. The claimed small-move benefit is not supported, and the paper's significance is reduced to a generic directional loss rather than a targeted fix for the stated failure mode. This is not merely a narrative issue; it calls into question whether the abstract's claims about small moves are accurate and whether the mechanism explanation in Appendix C (gradient equalization) has any observable consequence for the primary metric.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CosDir, an auxiliary loss that adds a scale-invariant cosine penalty between the horizon difference vectors of the prediction and the target to a base MSE loss, and CosDir-UW, an extension that learns the direction–magnitude mixing ratio via uncertainty weighting. The authors claim that MSE-trained forecasters systematically mispredict the direction of small moves, that CosDir re-injects directional gradient where MSE is blind, and that the method improves directional accuracy (DA) across 13 datasets and 15 backbones while preserving magnitude accuracy. The paper reports over 100K experiments, significance tests, per-seed standard deviations, comparisons with several alternative losses, ablations, and a payoff analysis.","tokens_in":38658,"tokens_out":7305,"duration_ms":80719,"significance":"If the reported gains are reliable, the paper offers a simple, backbone-agnostic plug-in loss for improving directional accuracy in time series forecasting, supported by an unusually broad empirical study: 13 datasets, 15 backbones, up to six horizons, multiple random seeds, and comparisons with shape, frequency, and directional losses. The per-seed standard deviations and paired Wilcoxon tests strengthen the empirical core, and the release of code is a positive feature. However, the paper's stated failure mode—small moves—is not supported by its own Figure 4, and the ablation does not cleanly isolate scale invariance as the source of the gains. These issues affect the central narrative and the mechanistic explanation, though the core DA improvement appears defensible.","major_comments":[{"comment":"The central motivation is that MSE-trained forecasters fail on the direction of small moves and that CosDir's scale invariance re-injects learning signal exactly there. However, Figure 4(a) shows that both MSE and MSE+CosDir are near chance on the smallest-move decile, and CosDir's advantage grows monotonically with move size. The text explicitly states: 'CosDir does not manufacture signal on the near-random smallest moves.' This is an internal inconsistency. Either provide additional evidence that small-move DA improves (e.g., decile analysis aggregated across datasets/backbones), or revise the abstract, introduction, and conclusion to state that the gains concentrate on larger, decision-relevant moves. As written, the paper's stated mechanism and its primary evidence disagree.","section":"Abstract; Introduction; Analysis 'Where the gains come from' (Fig. 4)"},{"comment":"CosDir optimizes cosine similarity of the entire horizon difference vectors, while DA is a per-step sign agreement. The paper offers no bound or empirical analysis relating these two quantities. A forecast can achieve high cosine alignment while still having sign errors on small steps, so optimizing Eq. (2) need not improve Eq. (1). This surrogate gap is especially consequential given Figure 4(a) shows no DA improvement on small moves. Please add a formal or empirical analysis of the surrogate gap (e.g., a per-step decomposition of the gradient, or a synthetic example where the cosine surrogate and DA diverge), or substantially weaken the claim that CosDir is a tight surrogate for directional accuracy.","section":"Eq. (2) vs. Eq. (1)"},{"comment":"The ablation is intended to show that scale invariance is the source of CosDir's gains. Yet the sign-classification baseline (BCE, Eq. (7)) is not scale-invariant in the sense emphasized by the paper, but it nearly matches CosDir: average DA 57.76 vs. 57.98 (Table 7). The claim that 'the gain comes from scale invariance rather than from adding a directional term' is therefore not supported. The paper should either include a scale-invariant sign surrogate (e.g., BCE applied to the normalized predicted difference) or reinterpret the ablation: adding any directional term helps, and scale invariance provides only a small additional gain.","section":"Table 7; Appendix E, Eq. (7)"}],"minor_comments":[{"comment":"The stabilizer ε is introduced but its value and sensitivity are never reported. Please state the value used and, if possible, include a sensitivity check or a sentence on why it is benign.","section":"Eq. (2)"},{"comment":"The claim that CosDir-UW exposes 'no loss hyperparameter' is overstated: the regularization coefficient 1/2 in Eq. (3), the initial values of s1 and s2, and the same ε remain modeling choices that can affect the learned ratio. Please qualify the claim.","section":"Eq. (3); 'Hyperparameter-free' claims"},{"comment":"Spearman ρ=0.60 is a moderate correlation, not a tight 'match.' The text says the learned ratio 'matches' or 'tracks' the per-dataset tuned λ; please soften this to 'correlates moderately' or provide a quantitative equivalence test.","section":"Figure 6(b); 'From a fixed to an adaptive mixing ratio'"},{"comment":"The statement that p<10^-3 'stays significant under a Bonferroni correction over the cells reported in each table' needs the number of tests and the resulting corrected threshold. Please report them.","section":"Appendix J"},{"comment":"The notation 'MSE/(MSE+CosDir)/(MSE+CosDir-UW)' is easy to misread. Consider adding a sample row or clarifying in the caption that each cell contains three values in that order.","section":"Table 5 caption"},{"comment":"The Traffic H=720 cell for TiDE is reported as '–'. State explicitly why this cell is missing (e.g., the model did not produce output for this configuration).","section":"Table 41"},{"comment":"The claim of 'over 100K experiments' should be broken down (main results, ablations, hyperparameter sweeps) so readers can verify the count and understand the computational scale.","section":"Experimental protocol"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is unusually thorough, and the core DA improvements appear real. The main problem is that the paper's framing overclaims the small-move mechanism, and the ablation does not isolate scale invariance as cleanly as claimed. I would not reject: the issues can be fixed by adding aggregated decile evidence, analyzing the surrogate gap, and revising the narrative. If the authors cannot provide such evidence, the paper should be reframed as a generic directional loss with gains concentrated on larger moves."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: CosDir (cosine similarity on first-difference vectors) is a simple, scale-invariant, plug-in loss that empirically improves directional accuracy across 13 datasets and 15 backbones while preserving MSE. The experiments are extensive and careful: significance tests, per-seed standard deviations, ablations against non-scale-invariant variants, comparison with shape/frequency losses, and robustness checks. The UW variant that learns the direction/magnitude weight is a nice touch, and the evidence that learned lambda tracks tuned lambda across datasets is persuasive. This is a real contribution to forecasting loss design, and the code is released.\n\nThe soft spots are real but mostly fixable. The biggest: the abstract and intro say the motivation is that MSE fails on small moves and CosDir re-injects gradient there, but Figure 4 shows CosDir's gain grows monotonically with move size and is near zero on the smallest decile. The paper's own analysis text says it does not manufacture signal on smallest moves. That undercuts the stated mechanism and should be rewritten — the loss improves DA overall, which is valuable, but not for the reason advertised. Second, the paper optimizes cosine similarity but evaluates per-step sign agreement; the gap between the surrogate and the metric is unquantified. Empirically the DA improvement shows up, but a formal or empirical analysis of the alignment would tighten the claim. Third, the financial datasets are promised but not released, which matters because those are the primary domain; the general benchmarks are public but the financial ones are constructed from Yahoo data. In an empirical paper this size, data release is important.\n\nThe citation pattern looks fair: prior losses compared, and the novelty claim (no prior direction-aware scale-invariant loss) holds up against the cited work. The math in Appendix C is straightforward and matches the scale-invariance claim.\n\nWho is this for: anyone training forecasting models for decision-driven tasks (risk, finance) and researchers studying loss functions. It deserves a serious referee; the main issues are presentation and reproducibility, not a load-bearing flaw. My recommendation: send to peer review, but require the authors to reconcile the small-move narrative with Figure 4 and release the financial datasets.","headline":"A genuinely useful plug-in directional loss with rock-solid empirical breadth, but the abstract's small-move story is contradicted by the paper's own Figure 4.","tokens_in":39128,"tokens_out":2160,"would_cite":true,"duration_ms":24580,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MSE-trained forecasters systematically mispredict the direction of small moves; adding a scale-invariant cosine loss on difference vectors—CosDir—consistently improves directional accuracy across 13 datasets and 15 backbones while preservin","keywords":["time series forecasting","directional accuracy","cosine similarity loss","scale-invariant loss","uncertainty weighting","loss function","deep learning","direction of change"],"falsifier":"On a fixed benchmark, train the same 15 backbones with MSE and MSE+CosDir and compute DA within the decile of smallest true |Δy| steps: if CosDir does not improve (or worsens) DA in that decile relative to MSE, the claimed mechanism—that scale invariance re-injects directional signal where MSE's gradient vanishes—is contradicted. A second check would compare CosDir against a direct sign-classification auxiliary on a synthetic series with balanced up/down labels: if the direct sign term beats CosDir on DA, the cosine surrogate is not the tight direction surrogate the paper claims.","tokens_in":38330,"feed_emoji":"📈","tokens_out":5021,"duration_ms":54821,"temperature":0.7,"pith_summary":"This paper argues that standard magnitude losses such as MSE are blind to whether a forecast moves up or down, especially on small moves, because their gradients vanish as move size shrinks. The authors propose CosDir, a plug-in loss term that computes the cosine similarity between the predicted and true step-to-step difference vectors over the forecast horizon, penalizing angular disagreement regardless of scale. Across more than 100K experiments on 13 datasets and 15 forecasting backbones, adding CosDir to MSE raises directional accuracy (DA) by about one to two percentage points on average without degrading MSE, and an uncertainty-weighted variant CosDir-UW learns the direction–magnitude mixing ratio per dataset. If correct, the work identifies a neglected axis of forecasting quality—direction of change—and offers a simple, architecture-agnostic way to train for it.","feed_headline":"Cosine loss fixes the direction errors MSE ignores","feed_subtitle":"Adding a scale-invariant cosine term to MSE raises directional accuracy on 13 datasets without hurting magnitude accuracy.","key_machinery":"The central object is the horizon difference vector: for each channel, the H-length vector of step-to-step changes Δŷ and Δy, anchored at the last observed value. CosDir adds the penalty 1 − cos(Δŷ, Δy) to the base loss (e.g., MSE), averaged over channels. Because cosine similarity depends only on orientation, the penalty is invariant to rescaling the whole window; its gradient carries a 1/‖Δŷ‖ prefactor, so small-move windows receive a directional update rather than a vanishing one. CosDir-UW replaces the fixed λ with two learnable log-variances and regularizer (s1+s2)/2, making the effective directional weight λ_eff = e^{s1-s2} adapt to the dataset.","core_discovery":"Adding a scale-invariant cosine term between the predicted and true step-to-step difference vectors raises directional accuracy across 15 backbones and 13 datasets, with gains concentrated on large, decision-relevant moves. The paper argues that MSE-trained models neglect direction because the squared-error gradient is dominated by large-amplitude points, leaving small moves unlearned. CosDir's normalized cosine term keeps a directional gradient at every scale without altering the magnitude objective. Its uncertainty-weighted variant, CosDir-UW, learns the mixing ratio per dataset through two log-variances, yielding an effective weight λ_eff = e^{s1-s2} that tracks a tuned fixed λ.","pith_inferences":["A straightforward but untested consequence is that the same cosine-on-differences term could regularize probabilistic forecasting, where one would align sampled or quantile trajectories rather than point forecasts; the paper lists probabilistic extension as future work.","Because CosDir's gradient equalizes across windows by a 1/‖Δŷ‖ factor, it may also act as a helpful regularizer for trend-change detection in anomaly or regime-shift settings, where sign of change is the label.","The paper optimizes a continuous cosine surrogate for a discrete sign-agreement metric; a direct synthetic test with known sign labels would show how tight the surrogate is, since high cosine alignment can coexist with sign errors on small steps.","CosDir-UW's λ_eff could be repurposed as a measure of directional predictability of a dataset, potentially informing when direction-aware training is worthwhile."],"forward_implications":["Adding CosDir to an MSE base improves DA on 93% of (backbone, dataset) cells across 13 datasets and 15 backbones, with average DA rising from 55.60% to 57.10% (CosDir) and 57.32% (CosDir-UW).","Directional gains concentrate on the largest true moves—the ones that drive decisions—and a direction-following strategy based on CosDir accumulates more payoff than one based on MSE, even after transaction costs.","CosDir-UW's learned balance is a dataset property: 96% of the variance in the learned log-weight lies between datasets, and its median tracks a per-dataset tuned fixed λ (Spearman ρ = 0.60), so no hyperparameter sweep is needed.","CosDir does not trade off magnitude for direction; on financial datasets it keeps MSE at or slightly below the MSE-only baseline, whereas shape- and frequency-focused auxiliary losses degrade MSE.","CosDir is complementary to existing structure-aware losses: combining it with a frequency-domain loss (FreDF) gives the best DA among the combinations tested."],"fun_headline_variants":["CosDir loss: MSE's blind spot fixed","Direction-aware loss boosts forecast accuracy","Cosine term targets small move direction","MSE misses direction, CosDir adds it","CosDir: direction accuracy without magnitude cost"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that cosine similarity between the horizon difference vectors is a valid and sufficiently tight surrogate for the per-step sign-agreement metric DA; the paper does not quantify the gap, and a prediction can score high cosine alignment while still getting small steps' signs wrong.","fun_headline_variants_meta":{"raw":{"variants":["CosDir loss: MSE's blind spot fixed","Direction-aware loss boosts forecast accuracy","Cosine term targets small move direction","MSE misses direction, CosDir adds it","CosDir: direction accuracy without magnitude cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000794,"raw_usage":{"total_tokens":3330,"prompt_tokens":741,"completion_tokens":2589,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":2525}},"tokens_in":485,"tokens_out":2589,"duration_ms":21619,"temperature":1.0,"reasoning_tokens":2525,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T19:26:53.488809+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a fixed benchmark, train the same 15 backbones with MSE and MSE+CosDir and compute DA within the decile of smallest true |Δy| steps: if CosDir does not improve (or worsens) DA in that decile relative to MSE, the claimed mechanism—that scale invariance re-injects directional signal where MSE's gradient vanishes—is contradicted. A second check would compare CosDir against a direct sign-classification auxiliary on a synthetic series with balanced up/down labels: if the direct sign term beats CosDir on DA, the cosine surrogate is not the tight direction surrogate the paper claims.","supporting_citations":[],"review_version":1}