{"id":"50cdd8b7-136a-46ab-ae05-f3f45c065331","arxiv_id":"2506.07920","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"W4S4 initializes S4 state space models with WaLRUS wavelet frames and reports better delay reconstruction and classification accuracy than HiPPO-based S4, with frozen (A,B).","lead":"This paper proposes W4S4, a new way to initialize state space models using wavelet-based memory instead of standard polynomial projections. It reports that this initialization improves memory retention and classification accuracy, even when the core state matrices are frozen and fewer parameters are used.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing Theorem 1, which justifies truncating WaLRUS to Neff states, is stated twice but never proved, and Neff is not defined, so the efficient W4S4 implementation is not established.","rationale":"The reader's verdict was CONDITIONAL, primarily because the proof of Theorem 1 is promised in the main text but absent from Appendix 7.1. My stress-test read confirms that this is the single most load-bearing concern: the paper's efficiency argument, the parameter counts in Tables 2 and 3, and even the identity of the evaluated model all depend on the truncation theorem. The theorem is restated rather than proved, and Neff is not defined. This is an internally identified incompleteness, not a disagreement with external consensus. The empirical results, while suggestive, cannot rescue an unproved structural claim that determines what model was actually run. I therefore agree with the reader's weakest_assumption and see no reason to change the CONDITIONAL verdict. The proposed concrete test—independent diagonalization and truncated-kernel comparison—would settle whether the theorem is true as stated. If it passes, the main technical objection is removed; if it fails, the paper would need either a corrected theorem and truncation procedure or a revised claim about what W4S4 computes.","tokens_in":10422,"tokens_out":3418,"duration_ms":45029,"concrete_test":"Reconstruct the WaLRUS state matrix A from the companion WaLRUS/SaFARi construction (refs [12,13]), diagonalize it at several sizes (e.g., N=64,256,1024), and check: (1) whether all eigenvalues beyond Neff are exactly 1, up to machine precision, and (2) whether the kernel computed from the truncated system matches the full-system kernel with relative error below a stated tolerance (e.g., 1e-6) on random inputs. If the tail eigenvalues deviate from 1 or the truncated kernel differs beyond tolerance, Theorem 1 is false as stated and the efficient W4S4 implementation is unjustified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency and correctness claims for W4S4 rest on Theorem 1 in Section 3.2: after diagonalization, only the first Neff eigenvalues of the WaLRUS state matrix exceed 1, the remaining eigenvalues are exactly 1, and only the first Neff rows of the transformed kernel matter. The theorem is restated in Appendix 7.1, but that appendix contains no proof, only the same statement. This is not a cosmetic gap. The implemented model uses the theorem to truncate the state space: the kernel is computed from the top Neff rows, and V[0:Neff,:] is absorbed into C. If the tail eigenvalues are only approximately 1, or if the omitted eigen-components contribute measurably to the kernel, then the claim of computing the kernel 'without requiring low-rank approximations' fails, and the model actually evaluated is not the model described. Additionally, Neff is never defined in the manuscript, so the truncation rule cannot be independently checked. The paper's empirical results cannot compensate for this gap because the reported parameter counts (e.g., 215K in Table 2) make sense only if truncation was applied; without a valid theorem, the identity of the evaluated model is in question.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes W4S4, an SSM initialization built from WaLRUS wavelet-frame matrices within the SaFARi framework, intended as a replacement for HiPPO initialization in deep SSM architectures such as S4. The central technical claim is that the diagonalized WaLRUS state matrix has only Neff significant eigenvalues, allowing the state space to be truncated to Neff dimensions without loss of reconstruction fidelity (Theorem 1). The paper reports controlled delay-reconstruction ablations and text/speech classification experiments, claiming that W4S4 outperforms HiPPO-initialized S4 models even when (A, B) are frozen, and it advertises parameter efficiency and better memory retention.","tokens_in":10667,"tokens_out":5114,"duration_ms":61877,"significance":"If fully supported, the contribution would be notable: a wavelet-frame-based initialization with stable diagonalization, exact kernel computation, and state truncation could improve both the accuracy and parameter efficiency of SSM-based sequence models. The paper's decision to freeze (A, B) in classification experiments is a clean and commendable way to isolate the effect of initialization, and the delay-reconstruction ablations provide a useful controlled setup. However, the current manuscript does not yet support these claims: the load-bearing Theorem 1 is asserted without proof, Neff is never defined, several datasets promised in Section 5.1 are never reported on, and the parameter-count comparisons are inconsistent across tables. The authors themselves state in the Future Work section that 'more empirical evidence should be gradually added to this manuscript,' which is a candid acknowledgment that the evidence base is incomplete.","major_comments":[{"comment":"Theorem 1 is load-bearing but is never proved. It is stated in Section 3.2 with 'See Appendix for proof,' and restated in Appendix 7.1, but the appendix contains only the statement and the implementation steps, not a proof. Furthermore, Neff is never defined in terms of the wavelet frame construction, and the WaLRUS matrix itself is not constructed in the paper but only cited to the authors' preprints [12, 13]. Since the implemented model uses the theorem to truncate the state space, the evaluated model is not precisely specified without this proof. Please provide a complete proof, a precise definition of Neff, and an explicit statement of whether the tail eigenvalues are exactly 1 or only approximately 1; if the latter, give quantitative error bounds on the induced kernel approximation.","section":"3.2/7.1"},{"comment":"Section 5.1 promises evaluation on Long-Range Arena, Speech Commands, sequential CIFAR, BIDMC Vital Signs, and four multivariate forecasting datasets, but the experimental sections report results only for a small subset: delay-task log-MSE in Table 1 and text/speech classification in Tables 2 and 3. No results are reported for LRA tasks (ListOps, Retrieval, Images, Pathfinder, PathX) or for Electricity, Traffic, Weather, and Solar-Energy forecasting. The abstract and conclusion claim consistent improvements across long-range sequence modeling, which is not supported by the presented evidence. Please add the missing experiments or explicitly restrict the claims to the datasets actually evaluated.","section":"5.1/5.3"},{"comment":"Table 1 reports single log-MSE values without error bars, number of seeds, or statistical significance information. The 'Relative' column is undefined, and for entries like '× 2.45' it is unclear whether the ratio is computed on MSE or log-MSE. Several margins are small (Solar ×1.15, BIDMC ×1.05), so without variance information the claim of consistent improvement is not robust. Please report mean ± standard deviation over multiple runs and define exactly how the relative improvement is computed.","section":"Table 1"},{"comment":"The parameter-efficiency claim is not consistently supported by the data. In Table 2, W4S4 has 215K parameters versus 800K for the baselines, but in Table 3, W4S4 has 260K parameters versus 200K for the baselines, so the claim that W4S4 uses 'fewer parameters' is false on the speech benchmark. In addition, the comparisons across rows differ in architectural details that are not specified for W4S4, such as number of layers, hidden size, and number of SSM cores, and the baselines have learnable (A, B) while W4S4 does not. To support the conclusion in Section 6 that W4S4 achieves higher accuracy with fewer parameters and fixed (A, B), please provide a controlled comparison with matched architectural capacity and give the full configuration of W4S4.","section":"Tables 2/3 and Section 6"}],"minor_comments":[{"comment":"The reference for 'Attention Is All You Need' appears as a placeholder '[? ]'; please replace it with the full citation.","section":"2.3"},{"comment":"The text says WaLRUS achieves '0.725 smaller log-MSE' and '×5.31', while also giving MSE values of 7.5e-4 and 4e-3; the relationship between these numbers and the figures should be clarified.","section":"4"},{"comment":"Section 5.2 states that B is fixed as an all-ones vector following [16], while Section 5.3 says that '(A, B) frozen'; please make the training protocol consistent and specify exactly which parameters are frozen in each experiment.","section":"5.2/5.3"},{"comment":"Appendix 7.2 contains the typo 'auroregressive' instead of 'autoregressive', and 'SC-ar' is used in Table 3 without being defined at first use.","section":"7.2"},{"comment":"The heading 'Learnable (A, B)' uses checkmarks and crosses; please clarify that a checkmark means the (A, B) parameters are trained and a cross means they are frozen.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript depends heavily on two preprint references by the same group [12, 13] for the WaLRUS construction and its properties. If those preprints are not independently peer-reviewed, the proof of Theorem 1 in this manuscript is essential for the validity of the proposed method. I also note that the authors' own Future Work statement says 'more empirical evidence should be gradually added to this manuscript,' which suggests the submission is incomplete as is."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you the short version. W4S4 is a sensible, incremental idea: take the wavelet-frame WaLRUS construction from the authors' own preprints and use it to initialize S4, freezing A and B. The controlled delay-reconstruction experiments are the strongest part. In a minimal linear setting, WaLRUS beats HiPPO by about 5x MSE with optimized C, D, and the advantage persists across delays and when embedded in a single-layer S4. That is a clean result, and the classification tables, though incomplete, point in the same direction: W4S4 with frozen (A, B) matches or beats trained HiPPO baselines with fewer parameters. That is worth taking seriously.\n\nThe soft spots are real and in proportion. The biggest is Theorem 1, which justifies truncating the state to Neff dimensions. It is stated in Section 3.2 and restated in Appendix 7.1, but Appendix 7.1 contains no proof. Worse, Neff is never defined. This is not cosmetic: the implemented W4S4 model uses the truncation to compute the kernel and to justify the parameter counts. If the tail eigenvalues are only close to 1 rather than exactly 1, or if the omitted components contribute to the kernel, then the model evaluated is not the model described. The paper's own complexity claims rest on this theorem.\n\nSecond, the experiments are partial. Five datasets are listed in Section 5.1 but Table 1 only shows six rows and there is no LRA suite, no Electricity, Traffic, Weather results, no bidirectional Speech Commands, no error bars in Table 1, and the parameter counts in Tables 2 and 3 mix models of different sizes (the S4 baselines have 800K and 200K, W4S4 has 215K and 260K). The authors themselves write that 'more empirical evidence should be gradually added,' which is honest but not a substitute.\n\nOn novelty: the core WaLRUS construction and the memory-retention claim come from the authors' own SaFARi and WaLRUS preprints. The new piece here is the reparameterization into the time-invariant S4 form and the empirical evaluation in S4. That is a modest increment, but not zero.\n\nMy bottom line: the empirical direction looks plausible, but the paper is not yet complete. It needs a real proof or a clear statement of what the truncation actually is, a defined Neff, and a fuller benchmark table with standard error bars and matched model sizes. I would send it to referees only after those are addressed; a serious referee would likely bounce it back for major revision, but the delay-task evidence makes it worth engaging. If the theorem gets fixed, I'd probably cite it. As it stands, I wouldn't rely on it for my own work.","headline":"Promising WaLRUS initialization for S4 with consistent wins on delay and classification tasks, but the load-bearing truncation theorem is unproved and Neff undefined, so the efficient implementation isn't yet established.","tokens_in":11191,"tokens_out":2551,"would_cite":false,"duration_ms":28267,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that replacing HiPPO initialization with WaLRUS wavelet-frame initialization in S4 improves long-range memory and classification accuracy, even when the state matrices are frozen.","keywords":["state space models","WaLRUS","wavelet frames","HiPPO initialization","S4","long-range sequence modeling","sequence initialization","multiresolution analysis"],"falsifier":"Compute the full eigenvalue spectrum of the diagonalized WaLRUS matrix for a redundant wavelet frame of size N and test whether every eigenvalue beyond the first Neff equals 1 to machine precision; if more than Neff eigenvalues deviate from 1 by more than roundoff, truncation changes the kernel and the efficiency justification collapses.","tokens_in":10227,"feed_emoji":"📈","tokens_out":9124,"duration_ms":91008,"temperature":0.7,"pith_summary":"State space models (SSMs) remember input history through a linear recurrence, and their usefulness hinges on how the state matrix is initialized. This paper argues that initializing an S4 model with WaLRUS—a state matrix built from redundant wavelet frames—gives the model better long-range memory than the standard HiPPO initialization. In delay-reconstruction tests the wavelet core achieves about 5.3 times lower error, and in full S4 models with (A, B) frozen, W4S4 beats HiPPO-based S4 models whose (A, B) are trainable, reaching the highest reported accuracy on the text and speech benchmarks with fewer parameters. The paper's central claim is that a high-quality structured initialization can matter more than training the state dynamics from a worse starting point.","feed_headline":"Wavelet initialization beats trained HiPPO in S4","feed_subtitle":"A wavelet-based state initialization gives SSMs better long-range memory and higher accuracy with fewer parameters.","key_machinery":"The central object is the WaLRUS state matrix, a SaFARi-built SSM dynamics derived from redundant wavelet frames—overcomplete collections of time-frequency atoms that represent a signal at multiple resolutions. Its load-bearing property is that it admits a stable, exact diagonalization, and its eigenvalues plateau: only the first Neff exceed 1, and the rest equal 1. Theorem 1 uses this plateau to justify truncating the state to Neff dimensions before kernel computation, allowing W4S4 to avoid Diagonal-Plus-Low-Rank approximations while the complex-conjugate pairing of the real-valued spectrum keeps the memory footprint unchanged.","core_discovery":"On the paper's own terms, the discovery is that WaLRUS, constructed through the SaFARi framework from redundant wavelet frames, supplies a better prior for SSM state dynamics than HiPPO's polynomial projections. Because WaLRUS has a stable, exact diagonalization, the diagonal-plus-low-rank approximation that HiPPO-based S4 requires is unnecessary, and the state can be truncated to Neff dimensions using the spectral property stated in Theorem 1: only the first Neff eigenvalues exceed 1 and the remaining eigenvalues equal 1. When this initialization is embedded in S4 with (A, B) frozen, W4S4 outperforms HiPPO-initialized S4 variants on every benchmark tested, including 88.55% versus 86.82% on text classification with 215K parameters instead of 800K, and 94.37% versus 93.60% on autoregressive speech-command recognition.","pith_inferences":["Our inference: the spectral-plateau truncation argument should generalize to other SaFARi species built from redundant frames, since the plateau likely follows from frame redundancy rather than from wavelets specifically.","Our inference: letting A and B be trainable after WaLRUS initialization, which the paper leaves as future work, may combine the strong prior with learned adaptation and improve accuracy further.","Our inference: because the WaLRUS core wins even in a single-layer delay task, W4S4 is a natural fit for streaming or online settings where fast recurrent inference and long memory are both required.","Our inference: the results suggest multiresolution time-frequency localization is the property behind long-range memory; a direct test would compare W4S4 against another frame-based initialization with similar redundancy but different localization."],"forward_implications":["W4S4 with frozen (A, B) outperforms HiPPO-initialized S4 models with trainable (A, B) on text classification (88.55% vs 86.82%) and on autoregressive speech-command recognition (94.37% vs 93.60%), so initialization alone can carry much of the performance.","The WaLRUS core retains delayed input information about 5.3 times better than HiPPO-LegS in the minimal delay-reconstruction setting, showing the advantage exists before any deep network is added.","Because WaLRUS diagonalizes without Diagonal-Plus-Low-Rank approximation, W4S4 kernel computation is cheaper, which can translate into faster or lower-memory training for long sequences.","These results support the view that SSM initialization acts as a functional prior: the wavelet state supplies a useful set of temporal filters at initialization, and gradient descent has less work to do."],"supporting_citations":[{"why":"defines the HiPPO polynomial-projection memory that WaLRUS is compared against and that S4 initializes from","marker":"[10]"},{"why":"provides the S4 architecture and benchmark tasks into which W4S4 plugs the WaLRUS initialization","marker":"[11]"},{"why":"supplies the SaFARi frame-based construction that WaLRUS instantiates","marker":"[12]"},{"why":"introduces the WaLRUS wavelet-frame SSM whose diagonalization and spectral truncation W4S4 relies on","marker":"[13]"},{"why":"supplies the diagonal-SSM initialization and the practice of fixing B that the frozen-(A,B) experiments follow","marker":"[16]"},{"why":"provides the time-invariant translated SSM formulation (Eq. 3) that W4S4 adopts for kernel precomputation","marker":"[17]"},{"why":"defines the Long Range Arena benchmark that motivates and hosts several of the long-sequence evaluations","marker":"[18]"},{"why":"provides the Speech Commands dataset used for the autoregressive speech classification experiment","marker":"[19]"}],"fun_headline_variants":["Wavelet frames make S4 remember longer with less","Stable diagonalization gives S4 a wavelet edge over HiPPO","Wavelet initialization improves S4 without low-rank tricks","S4's new wavelet state matrix retains more in fewer parameters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is Theorem 1's claim that only the first Neff eigenvalues of the diagonalized WaLRUS matrix exceed 1 and all remaining eigenvalues are exactly 1, so the state can be truncated to Neff dimensions without losing reconstruction fidelity; the theorem is stated in Section 3.2 and restated in Appendix 7.1, but no proof appears there.","fun_headline_variants_meta":{"raw":{"variants":["Wavelet frames make S4 remember longer with less","Stable diagonalization gives S4 a wavelet edge over HiPPO","Wavelet initialization improves S4 without low-rank tricks","S4's new wavelet state matrix retains more in fewer parameters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000595,"raw_usage":{"total_tokens":2779,"prompt_tokens":931,"completion_tokens":1848,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":1779}},"tokens_in":547,"tokens_out":1848,"duration_ms":16645,"temperature":1.0,"reasoning_tokens":1779,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:23:54.666737+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the full eigenvalue spectrum of the diagonalized WaLRUS matrix for a redundant wavelet frame of size N and test whether every eigenvalue beyond the first Neff equals 1 to machine precision; if more than Neff eigenvalues deviate from 1 by more than roundoff, truncation changes the kernel and the efficiency justification collapses.","supporting_citations":[{"cited_title":"HiPPO: Recurrent memory with optimal polynomial projections","cited_arxiv_id":null,"evidence_quote":"defines the HiPPO polynomial-projection memory that WaLRUS is compared against and that S4 initializes from"},{"cited_title":"Efficiently modeling long sequences with structured state spaces","cited_arxiv_id":null,"evidence_quote":"provides the S4 architecture and benchmark tasks into which W4S4 plugs the WaLRUS initialization"},{"cited_title":"SaFARi: State-Space Models for Frame-Agnostic Representation","cited_arxiv_id":"2505.08977","evidence_quote":"supplies the SaFARi frame-based construction that WaLRUS instantiates"},{"cited_title":"WaLRUS: Wavelets for Long-range Representation Using SSMs","cited_arxiv_id":"2505.12161","evidence_quote":"introduces the WaLRUS wavelet-frame SSM whose diagonalization and spectral truncation W4S4 relies on"},{"cited_title":"On the parameterization and initialization of diagonal state space models","cited_arxiv_id":null,"evidence_quote":"supplies the diagonal-SSM initialization and the practice of fixing B that the frozen-(A,B) experiments follow"}],"review_version":1}