{"id":"805dcfef-47de-471a-9354-00704f279c27","arxiv_id":"2505.12544","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Alternator++ adds trainable noise-prediction networks and a noise-matching loss to Alternators, but the proposed training target is ill-defined and the reported improvements are mixed.","lead":"Alternator++ is a time-series model that replaces the fixed noise terms in Alternators with learned noise-prediction networks and a diffusion-style noise-matching loss. The paper reports wins on density estimation, imputation, and forecasting, but the training objective is described inconsistently and the empirical gains are largely within noise.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Noise-matching targets in Eq. 10 are independent Gaussian draws, so the optimal noise networks are zero and Alternator++'s claimed mechanism is unsupported.","rationale":"The reader's weakest assumption identifies exactly the load-bearing flaw: the noise-matching targets in Eq. 10 are not the noises that produced the trajectories, and as written they are independent of the networks' inputs. I agree with that assessment and add that the contradiction is not merely a text/algorithm mismatch. Even under Algorithm 1's generative interpretation, the additive noise at time t is independent of the conditioning variables z_{t-1} and x_t, so the conditional expectation of the target is zero and the optimal noise networks are the zero functions. This makes L_epsilon a null objective, so Alternator++'s claimed improvement mechanism has no support in the equations. The experimental results cannot rescue the central claim because the reported improvements, if reproducible, would be attributable to reparameterized scaling and hyperparameter choices rather than to learned noise models. No code or formal verification is provided, and the manuscript's own Limitations section does not acknowledge this degeneracy. I therefore recommend no change to the reader's REJECT verdict.","tokens_in":13892,"tokens_out":6613,"duration_ms":73164,"concrete_test":"Run a controlled ablation on Solar with the paper's density-estimation setup: train the full Alternator++ as described in Algorithm 1, and train the same model with epsilon_psi and epsilon_nu frozen at zero for all t, optimizing only f and g via L_alternator. Compare MMD over at least 5 seeds. If the two variants are statistically indistinguishable, the noise-matching loss contributes no learnable signal, confirming that the targets are independent of the network inputs; if the full model is better, the authors should identify and release the source of the nonzero gradient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the noise-matching loss in Eq. 10 trains epsilon_psi and epsilon_nu to reproduce the noises that generated the observed and latent trajectories. The manuscript never defines a forward process that maps data to these noises; instead, the text defines epsilon_x and epsilon_z as fresh N(0,I) samples per time step and batch element. For the term E[||epsilon_x - epsilon_psi(z_{t-1})||^2], epsilon_x is independent of z_{t-1}, so the pointwise minimizer is epsilon_psi(z) = E[epsilon_x | z] = 0. Similarly, epsilon_z is independent of (z_{t-1}, x_t) by construction, so the minimizer of the second term is epsilon_nu = 0. Algorithm 1 does not repair this: if x_t is generated as mu_xt + sigma_x epsilon_muxt, the observation noise is still independent of z_{t-1}, and if x_t is the observed data, it contains no information about the freshly sampled epsilon_z. Thus the L_epsilon term provides no learnable gradient signal; the Bayes-optimal noise networks are identically zero. With zero noise networks, Alternator++ reduces to an Alternator with rescaled f and g means, and the reported gains cannot be attributed to the proposed noise-modeling mechanism. The Limitations section acknowledges only schedule tuning, not this degeneracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Alternator++, an extension of the Alternator framework for time series modeling, which adds two trainable noise-prediction networks (one for observation noise, one for latent noise) and a noise-matching loss to the standard Alternator objective. The claimed mechanism is that this loss forces the noise networks to reproduce the Gaussian noise trajectories that generate the observed and latent sequences, thus enabling a low-dimensional latent Alternator to capture complex time-varying stochasticity while remaining computationally efficient. The authors report experiments on density estimation (MMD on Solar, Covid, Fred, NN5), imputation (MAE/MSE/CC at missing rates 10-90%), and sea-surface-temperature forecasting (CRPS/MSE), claiming improvements over Mamba, ScoreGrad, Dyffusion, and the original Alternator.","tokens_in":14160,"tokens_out":8769,"duration_ms":77591,"significance":"If the proposed noise-matching mechanism were valid, Alternator++ would be an interesting and useful contribution: it combines low-dimensional latent dynamics with learned noise models and shows promising computational advantages over diffusion-based baselines, and the empirical evaluation spans multiple tasks and datasets. However, the central training objective is degenerate as written: the targets of the noise-matching loss are independent standard Gaussian draws, so the loss cannot train the noise networks to approximate any data-dependent noise trajectories. This is a load-bearing flaw that invalidates the paper's core methodological claim, despite the plausibility of the overall research direction.","major_comments":[{"comment":"The noise-matching loss L_epsilon minimizes ||epsilon_z - eps_nu(z_{t-1}, x_t)||^2 + gamma_t ||epsilon_x - eps_psi(z_{t-1})||^2, where the text defines epsilon_x and epsilon_z as standard Gaussian variables sampled independently for each time step and batch element. Since these targets are independent of the networks' inputs, E[epsilon_x | z_{t-1}] = 0 and E[epsilon_z | z_{t-1}, x_t] = 0; the pointwise minimizers of the loss are therefore the zero functions. The loss cannot train the networks to approximate 'the noise trajectories that produce the observed and latent trajectories' because the paper never defines a forward process that determines those trajectories; no such well-defined targets exist. This degeneracy also holds under Algorithm 1, where x_t is a model sample independent of the freshly drawn epsilon_x. The central claimed mechanism of Alternator++ is thus unsupported.","section":"Section 3.2, Eq. (10)"},{"comment":"Algorithm 1 samples the observation x_t from the generative process (x_t = mu_x_t + sigma_x eps_mu x_t), whereas Section 3.2 states that x_t is the t-th observation from the training data. Under the algorithm, the reconstruction term ||x_t - mu_x_t||^2 equals sigma_x^2 ||eps_mu x_t||^2, which is a random constant independent of the parameters; the Alternator loss would then provide no gradient signal for mu_x_t or mu_z_t. Under the text version, the reconstruction term is meaningful, but the noise-matching targets remain independent of the networks' inputs, so the noise-matching loss is still degenerate. The two descriptions of the training procedure are not equivalent, and the paper must specify which one is actually used.","section":"Algorithm 1 vs. Section 3.2"},{"comment":"The hyperparameter description for the SST forecasting experiment states that 'we train two Adversarial Diffusion Models (ADM)' with a U-Net backbone, base channels of 128, ResNet blocks, and 55.39M parameters; this does not correspond to the Alternator++ architecture described in Section 3.1, which uses the networks f_theta, g_phi, eps_psi, and eps_nu. This makes the reported SST forecasting results unreproducible from the methods as described and suggests that the experimental appendix does not accurately document the model used for those results.","section":"Appendix B, Sea Surface Temperature Forecasting"}],"minor_comments":[{"comment":"The caption states that Alternator++ 'may underperform Mamba and Dyffusion in terms of CRPS,' but the reported CRPS values (0.212 for Alternator++ vs. 0.219 for Mamba and 0.224 for Dyffusion) indicate that Alternator++ has the lowest CRPS; this inconsistency should be clarified.","section":"Table 2 caption"},{"comment":"The symbols eps_x, eps_z, eps_mu x_t, eps_mu z_t, eps_psi, and eps_nu are used with overlapping notations; a table of notation would improve readability.","section":"Section 3.2, notation"},{"comment":"The claim that Alternator++ models state transitions in a 'non-Markovian way' is not supported, since the transition distribution depends on (z_{t-1}, x_t), which is a Markovian state in the joint state space.","section":"Section 5, related work"},{"comment":"The MMD computation (kernel choice, bandwidth, number of sampled trajectories) is not described, which limits the reproducibility of the density estimation results.","section":"Section 4.1, MMD evaluation"},{"comment":"Several inline equations and formulas (e.g., Eq. (7)) contain garbled symbols or missing operators in the typeset version; the authors should carefully proofread the final manuscript.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The paper's central contribution is a learning objective that, as written, is degenerate; this is not a matter of tuning or presentation. The inconsistency between Algorithm 1 and the main text further complicates any fix within the scope of a revision, and the appendix mismatch for the SST experiment raises reproducibility concerns. I recommend rejection, though a fundamentally revised manuscript with a properly defined noise-matching target (e.g., via an explicit forward process) could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on arXiv:2505.12544 (Alternator++). The core idea — adding trainable noise-prediction networks to the Alternator generative process — is a reasonable extension, and the paper is clearly written. But the training objective as stated is internally inconsistent, and the noise-matching loss is not well-defined. That breaks the central claim.\n\nThe problem is in Section 3.2. The noise-matching loss L_ε targets ε_x and ε_z, which the text defines as fresh Gaussian draws. It also says x_t in the loss is the observed data. If ε_x is fresh and independent of z_{t-1}, the minimizer of ||ε_x - ε_ψ(z_{t-1})||^2 is ε_ψ ≡ 0; the same holds for ε_ν. The noise networks then cannot learn anything. Algorithm 1 doesn't rescue it: it samples x_t from the model, and then the Alternator term ||x_t - μ_x||^2 is just σ_x^2 ||ε_μxt||^2, which is constant in the parameters. So the two parts of the objective conflict: either the reconstruction term has no gradient, or the noise-matching term regresses onto independent noise. In both cases the proposed mechanism provides no learning signal.\n\nThe paper never defines a forward process that maps data to ε_x and ε_z, unlike diffusion models, where the noise is added to the data. Without that, the 'noise trajectories that produce the observed trajectories' are not external quantities. The Limitations section only mentions schedule tuning, not this degeneracy.\n\nWhat's still valuable: the generative process and sampling are simple and fast, and the paper compares against a decent set of baselines (Mamba, ScoreGrad, Dyffusion). The MMD gains on three of four datasets are large, and the SST results are competitive, though with high variance. But since the mechanism is unsupported, those wins could just come from the rescaled f/g means and standard Alternator training, not from the noise models. No code is provided, so the authors can't point to an implementation that fixes the contradiction.\n\nI agree with the reject verdict. The flaw is load-bearing. Still, I'd send this to a serious referee: a good reviewer would catch the issue quickly, and the idea might be salvageable if the authors define a proper noise-matching target (e.g., use the actual noise drawn in the generative process) and align the pseudocode with the text. As written, it's not sound.","headline":"The noise-matching loss in Alternator++ has no well-defined target, so the claimed mechanism is unsupported and the empirical wins are likely spurious.","tokens_in":14676,"tokens_out":5518,"would_cite":false,"duration_ms":48539,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Alternator++ claims that learning the noise in observed and latent trajectories improves time-series density estimation, imputation, and forecasting beyond Mamba, ScoreGrad, and Dyffusion.","keywords":["Alternators","noise models","time series","latent dynamics","density estimation","imputation","forecasting","diffusion models"],"falsifier":"Calculate the minimizer of the noise-matching loss as written with $x_t$ fixed from the training data and $\\varepsilon_x \\sim \\mathcal{N}(0,I)$: the conditional expectation of $\\|\\varepsilon_x - \\varepsilon^t_\\psi(z_{t-1})\\|^2$ is minimized by the zero function, so a converged run should drive the observation noise network to zero unless the implemented objective differs from Eq. (10); reading the training code or checking the learned network outputs settles which case holds.","tokens_in":13652,"feed_emoji":"📈","tokens_out":16109,"duration_ms":139748,"temperature":0.7,"pith_summary":"The paper tries to establish that a low-dimensional latent sequence model can capture complex, time-varying stochasticity by learning the noise that drives each observation and latent update. It adds two neural networks, one predicting observation noise and one predicting latent noise, and trains them with a noise-matching loss added to the Alternator objective. The reported result is that this model, Alternator++, gives lower MMD than Mamba, ScoreGrad, and the original Alternator on three of four density-estimation benchmarks, better imputation on most datasets, and lower MSE on sea-surface-temperature forecasting, while sampling far faster than diffusion alternatives. The practical payoff, if the results hold, is diffusion-level flexibility without a high-dimensional state or iterative sampling.","feed_headline":"Alternator++ beats Mamba, ScoreGrad, Dyffusion on time-series tasks","feed_subtitle":"A small latent model learns its own noise and beats bigger baselines on time-series fit and speed.","key_machinery":"The central machinery is the pair of trainable noise-prediction networks $\\varepsilon^t_\\psi$ and $\\varepsilon^t_\\nu$ added to the Alternator equations. An Alternator pairs an observed sequence $x_{1:T}$ with a low-dimensional latent trajectory $z_{0:T}$ and alternates between generating $x_t$ from $z_{t-1}$ and updating $z_t$ from $z_{t-1}$ and $x_t$; Alternator++ lets the Gaussian noise in both steps have learned means given by these networks. The noise-matching loss $L_\\varepsilon$, balanced by $\\gamma_t$, trains the networks to predict the Gaussian draws used in the sampling equations, which is the mechanism the paper credits for capturing time-varying stochasticity.","core_discovery":"At its core, the paper claims that the noise in an Alternator need not be a fixed zero-mean Gaussian: it can be learned by two small networks. The observation and latent updates are written as $$x_t = \\sqrt{\\beta_t} f_\\$\\theta$(z_{t-1}) + \\sqrt{1-\\beta_t-\\$sigma_x^{2}$}\\,\\varepsilon^t_\\psi(z_{t-1}) + \\sigma_x \\$varepsilon^{{\\mu_x}}$_t$$ and $$z_t = \\sqrt{\\alpha_t} g_\\varphi(x_t) + \\sqrt{1-\\alpha_t-\\$sigma_z^{2}$}\\,\\varepsilon^t_\\nu(z_{t-1}, x_t) + \\sigma_z \\$varepsilon^{{\\mu_z}}$_t,$$ where $\\varepsilon^t_\\psi$ and $\\varepsilon^t_\\nu$ are the noise models. The added noise-matching loss trains these networks to reproduce the Gaussian draws used in those equations, and the paper argues this lets a low-dimensional latent Alternator capture complex stochasticity. Empirically, the paper reports lower MMD than Mamba, ScoreGrad, and the original Alternator on three of four density-estimation benchmarks, improved imputation on most datasets, and lower MSE on sea-surface-temperature forecasting, while avoiding the iterative sampling loop of diffusion models.","pith_inferences":["One extension beyond the paper is to define an explicit forward noising process so that $\\varepsilon_x$ and $\\varepsilon_z$ are genuinely the noises that produced the data; then the noise-matching loss becomes a denoising objective and the learned networks could be compared with diffusion score functions.","The recipe is portable: any latent dynamics model with Gaussian transition or observation noise could add a learned noise-prediction network and a matching loss as a cheap way to enrich its stochasticity.","Since the observation noise network conditions on $z_{t-1}$ only, a natural variation is to feed it the current observation $x_t$ as well, mirroring the latent noise network, and test whether imputation at high missing rates improves.","The paper notes the schedules $\\beta_t$ and $\\alpha_t$ need per-dataset tuning; learning them end-to-end rather than searching by hand would be the most direct next step."],"forward_implications":["If the reported gains hold, low-dimensional latent states are enough for expressive stochastic time-series generation, reducing the need for high-dimensional hidden states in state-space models.","The noise-matching loss supplies a mechanism for modeling nonzero, state-dependent noise means, something the original Alternator's fixed zero-mean noise cannot represent.","At missing rates from 10% to 90%, the model would give imputation with lower mean squared error and higher correlation than Mamba and Alternator on most datasets tested.","For forecasting, the model would provide probabilistic forecasts with one-step alternating sampling, far cheaper than diffusion-based samplers: about 3x faster than Dyffusion and 50x faster than MCVD on the SST task.","The same generative process serves as an encoder for new sequences, so no separate inference network is needed for representation learning."],"supporting_citations":[{"why":"Introduces the Alternator objective and generative process that Alternator++ extends.","marker":"(Rezaei and Dieng, 2024)"},{"why":"Supplies the denoising diffusion framework and the idea of predicting added Gaussian noise.","marker":"(Ho et al., 2020)"},{"why":"Shows that learned noise models improve diffusion sample quality, motivating the same idea in Alternators.","marker":"(Dhariwal and Nichol, 2021)"},{"why":"Mamba is the state-space baseline compared in density estimation and imputation.","marker":"(Gu and Dao, 2023)"},{"why":"ScoreGrad is the score-based diffusion baseline for density estimation and forecasting.","marker":"(Yan et al., 2021)"},{"why":"Dyffusion is the dynamics-informed diffusion baseline, and the SST forecasting setup comes from this line of work.","marker":"(Cachay et al., 2023)"},{"why":"The alpha-Alternator extension that adapts to noise levels, used as context and comparison point.","marker":"(Rezaei and Dieng, 2025)"},{"why":"Supplies the motivation that Mamba struggles with noise propagation in long sequences.","marker":"(Wang et al., 2025)"}],"fun_headline_variants":["Alternator++ learns its own noise, beats Mamba on time series","Noise-aware Alternator beats Mamba, ScoreGrad, Dyffusion","Learned noise boosts Alternator on time-series tasks","Noise-matching loss gives Alternator an edge on forecasting","Small latent model learns noise, beats bigger baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Gaussian noise samples drawn during training are exactly the noise that generated the observed and latent trajectories; the paper does not define how data would be turned into those specific noise samples, and the algorithm sometimes generates the observed values from the model rather than using the data.","fun_headline_variants_meta":{"raw":{"variants":["Alternator++ learns its own noise, beats Mamba on time series","Noise-aware Alternator beats Mamba, ScoreGrad, Dyffusion","Learned noise boosts Alternator on time-series tasks","Noise-matching loss gives Alternator an edge on forecasting","Small latent model learns noise, beats bigger baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000975,"raw_usage":{"total_tokens":4146,"prompt_tokens":954,"completion_tokens":3192,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":3104}},"tokens_in":570,"tokens_out":3192,"duration_ms":23115,"temperature":1.0,"reasoning_tokens":3104,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:31:56.586780+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Calculate the minimizer of the noise-matching loss as written with $x_t$ fixed from the training data and $\\varepsilon_x \\sim \\mathcal{N}(0,I)$: the conditional expectation of $\\|\\varepsilon_x - \\varepsilon^t_\\psi(z_{t-1})\\|^2$ is minimized by the zero function, so a converged run should drive the observation noise network to zero unless the implemented objective differs from Eq. (10); reading the training code or checking the learned network outputs settles which case holds.","supporting_citations":[{"cited_title":"The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance","cited_arxiv_id":"2502.04593","evidence_quote":"The alpha-Alternator extension that adapts to noise levels, used as context and comparison point."}],"review_version":1}