{"id":"32a8a327-120e-4199-8926-ad21d99f334b","arxiv_id":"2602.16220","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SEMixer combines random-mask patch interactions with progressive adjacent-scale mixing and reports improved MSE/MAE on common long-term forecasting benchmarks.","lead":"This paper introduces SEMixer, a lightweight neural network that forecasts long-term time series by mixing patterns across multiple time scales. It reports accuracy gains over 12 baselines on 10 public datasets and a third-place result in a wireless-network forecasting challenge, using less memory than comparison models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RAM's inference-time operation (Eq. 7) is a fixed global sum, not an ensemble of diverse interactions; the claimed mechanism is unsupported by the formulation.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: Eq. 7 reduces RAM to a fixed global average/sum, contradicting the claimed 'diverse interactions' mechanism. I agree with this reading. This concern is load-bearing because RAM is half of the proposed architecture and is central to the ablation analysis that claims superiority over self-attention. If RAM is just global pooling, the paper's contribution is significantly reframed: it becomes an MLP-Mixer with global context injection, which is a known technique, and an ablation comparing 'random attention' against standard attention is comparing a global non-parametric operation against learned attention. The empirical claims of 5-15% improvement might still hold, but the claimed mechanism would not, and the paper must be revised to either justify the training-time randomness as a regularizer or position RAM as a simple global-context module. Thus the reader's CONDITIONAL verdict remains correct: the authors need to fix the mechanism description and reframe the contribution. I see no reason to move to ACCEPT or REJECT based on this analysis alone; the concern is substantive but not definitively fatal unless the concrete test shows no benefit from random sampling, which would further degrade the claimed novelty. My response is limited to one concern, as requested, and I focus on this because it affects the central argument's coherence.","tokens_in":25610,"tokens_out":3039,"duration_ms":35797,"concrete_test":"Implement a deterministic global-context block that replaces RAM: compute the mean of patch embeddings (or sum), pass it through a single linear layer to dimension D, and add it to every patch embedding, preserving the same residual structure and hyperparameters. Train and evaluate on ETTh1, Weather, and Traffic with horizons 96/192/336/720. If the resulting MSE matches SEMixer's reported values within ±0.1% (e.g., ETTh1 0.400 vs. 0.400-0.401), the random-mask training is irrelevant and the 'diverse interactions' claim is refuted. If SEMixer is materially better, the random masks do contribute beyond global pooling.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claimed contribution of RAM is that it 'captures diverse time-patch interactions during training and aggregates them via dropout ensemble at inference' (Abstract, Sec. 3.2). But Eq. 5 randomizes the interaction matrix, and Eq. 7 replaces the ensemble with (1-p)*A where A is an all-ones matrix. Since for any binary mask M, (M ⊙ A) X = M X, the average over masks M~Bernoulli(0.15) converges to 0.15 * (all-ones) X. Thus Eq. 7 is exactly the expected value of the random ensemble: for each patch, it computes 0.15 times the sum over all patches. This is a deterministic global-pooling operation, not a diverse interaction pattern. The 'large number of interaction patterns learned via random sampling' are integrated into a single vector that is identical for every patch; no diversity remains at inference. Consequently, the performance gains attributed to RAM (Table 4, w/o RAM +5% MSE) are mechanistically explained only by the injection of a global context vector into each patch, likely via the residual design. The claim that RAM 'learns diverse interactions' is therefore not supported by the equations; it is a mischaracterization of a standard global-context trick. This undermines the novelty claim and the meaningfulness of the RAM-vs-SAM ablations, though the empirical numbers may still stand.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SEMixer, a lightweight multiscale MLP-Mixer architecture for long-term time series forecasting. Two components are introduced: Random Attention Mechanism (RAM), which samples random binary patch-interaction masks during training and aggregates them at inference via a dropout-style ensemble, and Multiscale Progressive Mixing Chain (MPMC), which stacks temporal mixing blocks over pairwise adjacent scales. The model is evaluated on ten public benchmarks, longer-horizon settings, and the 2025 CCF AlOps Challenge, where the authors report third place. The central claims are that RAM enhances patch semantics more effectively and efficiently than self-attention, and that MPMC improves multiscale mixing while reducing memory cost.","tokens_in":25948,"tokens_out":4739,"duration_ms":41555,"significance":"If the results are taken at face value, SEMixer is a simple, efficient, and strong multiscale forecaster. The code is public, ablations cover multiple variants, multiple seeds are reported, and the longer-horizon experiments (Table 2, Table 5) go beyond the standard 96/192/336/720 setup. The MPMC design is a reasonable engineering contribution, and the RAM idea, when correctly characterized, may still be a useful global-context mechanism. However, the current manuscript has several load-bearing reporting and conceptual issues that must be addressed before the claims can be accepted.","major_comments":[{"comment":"The baseline entries for ILI and Traffic are duplicated. For ILI, TSMixer, DLinear, PatchTST, and TimeMixer are listed as 2.799/2.758/2.898/2.752, which are identical to Deform.TST/TimeXer/ModernTCN/Pathformer. The correct values in Table 10 give TimeMixer an average ILI MSE of about 2.51, not 2.75. For Traffic, the duplicated TimeMixer value 0.412 is actually Pathformer's; Table 10 shows TimeMixer's true average is about 0.390, which is slightly better than SEMixer's 0.388, reversing the comparison on that dataset. The main results table therefore misrepresents the relative performance on two of ten datasets and directly supports the paper's '5–15% lower MSE' claim. The authors must correct Table 1 using the appendix numbers and re-verify all headline improvements.","section":"Table 1 (ILI and Traffic rows)"},{"comment":"The paper states in the abstract that SEMixer 'achieves third place' in the 2025 CCF AlOps Challenge, while Sec. 4.2.2 says 'SEMixer attains the lowest prediction error among all competitors' and Table 3 reports the best MSE among the listed methods. If the error is truly the lowest, the model would be first place, not third. The likely resolution is that Table 3 compares only the methods re-run by the authors, not the official leaderboard entries, but this is not stated. This contradiction undermines the competition claim and needs explicit clarification.","section":"Abstract and Sec. 4.2.2"},{"comment":"The central mechanism claim for RAM is not supported by the equations. In Eq. (5), the sampled interaction matrix is M ⊙ A, where A is all-ones. Since E[M] = (1-p)J (J all-ones), the dropout ensemble in Eq. (7) is exactly the expected value: the operation becomes (1-p) times the sum over all patches, i.e., a global average pooling per feature. This is identical for every patch; no diversity remains at inference. The claim that RAM 'learns diverse interactions' and integrates them via a dropout ensemble is therefore a mischaracterization of a standard global-context operation. The performance gains attributed to RAM (Table 4) could be explained by global pooling plus residual connections rather than by any learned diversity. The authors should either reformulate RAM so that diverse interaction patterns survive at inference (e.g., learnable non-uniform weights, multiple masks, or per-head","section":"Sec. 3.2, Eq. (5)–(7)"},{"comment":"The claim 'SEMixer achieves superior long-term forecasting performance, with 5–15% lower MSE than existing models' is an overgeneralization. Using the correct Table 1 entries, several datasets show improvements well below 5% (e.g., Solar Energy ~0%, ETTh1 ~2%, Electricity ~1%), and with the duplicated entries corrected, Traffic shows no improvement over TimeMixer. The percentage range should be recomputed on corrected numbers and stated per dataset, or the claim should be removed.","section":"Sec. 4.2.1 and Table 1"}],"minor_comments":[{"comment":"The 'Training stage of RAM' paragraph is repeated nearly verbatim twice. Please remove the duplicate.","section":"Sec. 3.2"},{"comment":"The patch count formula N_s = floor((n-L_s)/K_s)+2 should be justified. The standard formula is +1 for non-overlapping patches; the +2 appears to imply an extra padding or a different convention. Please clarify.","section":"Sec. 3.1, Eq. (3)"},{"comment":"It is unclear whether the optimal input length was also searched for SEMixer or only for the baselines. The text says 'we also search for the optimal input length ... for each baseline', and the Figure 1 caption says SEMixer adjusts only input length. A precise statement of the selection protocol for SEMixer and each baseline is needed to guarantee fairness.","section":"Sec. 4.1.3 and Figure 1 caption"},{"comment":"The table reports only six methods; the list of 'advanced methods' compared is incomplete relative to the 12 baselines in Table 1. State why FiLM, Scaleformer, and others are omitted from the competition comparison.","section":"Sec. 4.2.2, Table 3"},{"comment":"There are several typos and formatting inconsistencies, e.g., 'Mutiscale' in Sec. 3.1, the use of 'bX_d' in Sec. 4.1.3 where a dimension name is expected, and the incomplete 'w/ FA' column in Table 4 (missing values for some rows).","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's underlying empirical work may be sound, but the main table contains duplicated baseline numbers that reverse a conclusion on Traffic, the competition ranking is contradictory, and the RAM mechanism is over-claimed. These are fixable in revision. I would not recommend rejection because the architecture and experiments are potentially useful once the presentation is corrected. However, the authors should also address the conceptual gap between the RAM description and its actual mathematical form, as this affects the novelty narrative."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a decent empirical paper with a real but modest contribution, and it is not ready as written. The architecture—MLP-Mixer plus random binary masks during training and a scaled all-ones interaction at inference—does give good MSE numbers, and the progressive adjacent-scale mixing (MPMC) is a sensible way to keep memory down. If the results reproduce, the model is useful for long-horizon forecasting.\n\nWhat is genuinely new: the specific combination of RAM and MPMC, evaluated on 10 public datasets plus a 21GB competition set, with code released. The ablation study is more thorough than most: removing RAM/MPMC, replacing RAM with standard and efficient attention variants, and noise robustness all support the empirical claims. The 5–15% MSE improvement over strong baselines is credible, not paradigm-shifting.\n\nThe soft spots are real. The stress-test is right: Eq. 7 is exactly the expectation over Bernoulli masks, so at inference RAM reduces to multiplying by a fixed all-ones matrix—i.e., global average pooling scaled by (1-p). Calling that 'diverse interactions' is a mischaracterization. The training-time random masks may act as a form of dropout and help regularize, and the global context injection likely explains the ablation gains, but the paper needs to say that. This is a framing problem, not a fatal math error.\n\nThe main results table has duplicated baseline rows for ILI and Traffic (e.g., Deform.TST values identical to TSMixer, TimeXer identical to DLinear). That's a serious presentational error even if the appendix has distinct numbers. The competition claim is also internally inconsistent: third place in the AlOps Challenge, then Table 3 says SEMixer has the lowest error among 'all competitors'—the table only includes six models run by the authors, so it cannot support that phrase. The input-length search protocol (choosing each model's best from a list) should be described precisely; otherwise the comparison is hard to audit.\n\nThe result is worth a serious referee. The empirical work is substantial, the code link is there, and the core architectural idea—cheap global context plus progressive multiscale mixing—is worth testing. But I would send it back for major revision: fix the tables, reconcile the competition wording, reframe RAM's mechanism honestly, and cite/position against TimeMixer and dropout/edge-drop work more carefully. I wouldn't cite it in my own work until those corrections are made.","headline":"SEMixer is a plausible, efficient multiscale forecasting architecture with strong benchmark numbers, but the RAM mechanism is a global-pooling trick in disguise and the main table has duplicated baselines.","tokens_in":26439,"tokens_out":3712,"would_cite":false,"duration_ms":39685,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SEMixer claims that a lightweight MLP-Mixer with random masking and progressive adjacent-scale mixing forecasts long-horizon time series with 5–15% lower error than existing models while using less memory.","keywords":["time series forecasting","multiscale modeling","MLP-Mixer","random attention mechanism","dropout ensemble","long-term forecasting","patch-based models","efficient deep learning"],"falsifier":"Run SEMixer with RAM replaced at inference by exact global average pooling (the all-ones matrix scaled by 1−p, without any learned masks) and compare MSE to the full RAM; if accuracy is unchanged, the random-mask training story is falsified and the gain is just global context plus residual.","tokens_in":25467,"feed_emoji":"📈","tokens_out":4583,"duration_ms":42740,"temperature":0.7,"pith_summary":"This paper tries to establish that an MLP-Mixer architecture, augmented with two techniques, can be the best practical choice for long-term time series forecasting: it should be more accurate than transformer-, CNN-, and linear-based multiscale models, and also cheaper. The first technique, Random Attention Mechanism (RAM), replaces self-attention with randomly sampled binary interaction masks during training and a dropout-style ensemble at inference to enrich what each time patch represents. The second, Multiscale Progressive Mixing Chain (MPMC), mixes only adjacent temporal scales in a fine-to-coarse chain instead of all scales jointly, which the paper argues reduces memory and avoids semantic gaps between distant scales. If right, this matters because it points to a lower-cost, longer-input route to forecasting accuracy, corroborated by wins on ten public benchmarks and a large real wireless-network operations dataset.","feed_headline":"Random-mask mixing cuts forecast error by up to 15%","feed_subtitle":"A lightweight MLP-Mixer with progressive adjacent-scale mixing beats 12 baselines across 10 datasets and a real wireless challenge.","key_machinery":"The two load-bearing pieces are (1) RAM, a random attention mechanism that samples 0–1 Bernoulli masks over a full all-ones interaction matrix during training and, at inference, replaces the ensemble of masks by the all-ones matrix scaled by (1−p) — a dropout-style approximation; and (2) MPMC, a multiscale progressive mixing chain that feeds pairwise concatenations of adjacent scales through RAM+MLP-Mixer blocks in fine-to-coarse order. RAM's job is to give each time patch richer semantics than the raw patch alone, while MPMC's job is to fuse scales without jointly processing all of them.","core_discovery":"The central claim is that a deliberately simple, attention-free MLP-Mixer can outperform 12 advanced forecasting baselines on long-horizon prediction when its patched inputs are semantically enriched by random interaction masks and fused through a progressive adjacent-scale chain. On the paper's own terms, RAM learns a large number of diverse patch-interaction patterns during training and integrates them at inference through a dropout-ensemble approximation, giving MLP-Mixer richer patch semantics than standard self-attention. MPMC then stacks these mixing blocks so each scale is only ever concatenated with its neighbor, moving from finest to coarsest resolution, which the authors say both r","pith_inferences":["Editorial inference: At inference, RAM's all-ones matrix times a scaled version of the input is mathematically equivalent to global average pooling of the patch columns; so the empirical gains attributed to 'learned diverse interactions' may actually come from adding a global context term plus the residual connection. A direct ablation replacing RAM with exact global pooling would settle this.","Editorial inference: The random binary masks during training act as a strong regularizer or denoiser on the interaction matrix; this insight could transfer to other layer types (e.g., convolutions or linear layers) as a general train-time augmentation.","Editorial inference: MPMC's adjacent-scale-only mixing is a hierarchical fusion strategy that could be applied to other backbones, not just MLP-Mixer, potentially extending the efficiency and noise-robustness benefits to attention- and convolution-based forecasters.","Editorial inference: The paper's own limitation paragraph concedes that RAM assumes meaningful cross-patch correlations; where those correlations are weak, the residual connection must carry the load, suggesting a boundary condition on where SEMixer's gains persist."],"forward_implications":["If SEMixer is right, very long historical inputs can be exploited for forecasting without the memory blow-up that sinks transformer-based multiscale models, because MPMC never processes all scales at once.","Standard self-attention's role in patch-based forecasting can be replaced by a cheaper random-mask plus dropout mechanism, with accuracy gains rather than losses, according to the ablation tables.","The progressive fine-to-coarse chain is a concrete recipe for aligning scales that differ in semantics, which should make multiscale forecasting more robust to non-adjacent-scale noise.","The method's consistency across 10 datasets and a real competition dataset suggests the combination generalizes beyond the specific benchmarks.","The noise-injection experiments imply the architecture degrades gracefully when test data are corrupted, with MPMC suppressing noise through shared cross-scale structure."],"fun_headline_variants":["Attention-free MLP-Mixer beats 12 baselines on long-term forecasts","Random attention masks enhance MLP-Mixer for time series","Multiscale mixing chain wins third place in wireless forecasting challenge","Progressive adjacent-scale mixing improves long-horizon prediction","Lightweight mixer outperforms heavy forecasters with random masks"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The semantic-enhancement benefit of RAM rests on the assumption that averaging the learned random masks at inference is faithfully reproduced by a single scaled all-ones matrix — but that matrix collapses to global average pooling, so the 'diverse patch interactions' claimed at inference are not actually being computed.","fun_headline_variants_meta":{"raw":{"variants":["Attention-free MLP-Mixer beats 12 baselines on long-term forecasts","Random attention masks enhance MLP-Mixer for time series","Multiscale mixing chain wins third place in wireless forecasting challenge","Progressive adjacent-scale mixing improves long-horizon prediction","Lightweight mixer outperforms heavy forecasters with random masks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1410,"prompt_tokens":742,"completion_tokens":668,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":582}},"tokens_in":486,"tokens_out":668,"duration_ms":6600,"temperature":1.0,"reasoning_tokens":582,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:36:24.376112+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SEMixer with RAM replaced at inference by exact global average pooling (the all-ones matrix scaled by 1−p, without any learned masks) and compare MSE to the full RAM; if accuracy is unchanged, the random-mask training story is falsified and the gain is just global context plus residual.","supporting_citations":[],"review_version":1}