{"id":"7754a1a3-6268-4144-9b22-407fe6aa4aac","arxiv_id":"2501.11409","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Input reconstruction in echo state networks can be done without supervised targets by algebraically substituting known invertible parameters for the input signal.","lead":"This paper shows that an echo state network can learn to reconstruct its own input without seeing the input as a training target, provided the network's activation function is invertible and its input weights have full column rank. The key is a mathematical identity that replaces target inputs with known network parameters and observed reservoir states.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified: the central unsupervised IR theorem is algebraically sound; the reader's Eq. 3.2 concern does not land because hat B - B lies in range(A), so A A^+ = I is not required.","rationale":"The paper's central claim is a mathematical equivalence, not an empirical breakthrough: if sigma is invertible and A has full column rank, the input is an exact function of consecutive reservoir states and the known matrices, so the least-squares readout has an input-free expression. I re-derived Theorem 1 and Theorem 2 and found them internally consistent. The reader's stated reason for CONDITIONAL is an algebraic misconception: Eq. 3.2 does not require A A^+ = I. Full row rank of R_{1,T} implies hat B = A W_D + B, hence hat B - B = A W_D lies in range(A), and A A^+ acts as the identity on hat B - B. This holds precisely in the paper's experiments, where n_r > n_in. The only real caveat is condition (iii): without full row rank, the simplified Eq. 2.5 and Eq. 3.2 can deviate from the supervised solution, and the paper already flags this and even reports a numerical advantage in that regime. That caveat does not touch Theorem 1. I also checked the RLS algorithm: with hat B_1 = B, every update v g^T has v in range(A) while hat B - B stays in range(A), so no hidden A A^+ = I assumption appears. The numerical experiments are illustrative and no code is released, but these are not threats to the correctness of the main derivation. I therefore see no load-bearing concern requiring a verdict change; a one-line clarification that Eq. 3.2 assumes condition (iii) would be sufficient if the authors want to preempt misreading, but it is not a condition for acceptance.","tokens_in":21391,"tokens_out":18695,"duration_ms":206758,"concrete_test":"Run a deterministic check with n_r = 50, n_in = 1 (or n_r = 500, n_in = 3) using exact arithmetic or high precision: build a random full-row-rank R_{1,T}, set Y = sigma^-1(R_{2,T+1}) = A D + B R_{1,T}, compute hat B = Y R^+_{1,T}, then evaluate ||A A^+ (hat B - B) - (hat B - B)||_F and ||A W_R + B - hat B||_F. If both are at machine-precision level, Eq. 3.2 is confirmed and the reader's A A^+ = I objection is refuted. As a complementary edge-case check, repeat with a deliberately rank-deficient R_{1,T} to verify that Eq. 3.2 then fails, confirming that condition (iii) is doing the required work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"I checked the central derivation and do not find a load-bearing flaw. Theorem 1 / Eq. 2.4 follows exactly from Lemma 1: with invertible sigma and rank(A) = n_in, D_{1,T} = A^+ [sigma^-1(R_{2,T+1}) - B R_{1,T}], so substituting into D_{1,T} R^+_{1,T} yields W_R = W_D without ever needing the original input. The reader's weakest-assumption concern about Eq. 3.2 is not valid as stated. If condition (iii) holds (rank(R_{1,T}) = n_r), the least-squares minimizer of L_{sigma,T} is hat B = sigma^-1(R_{2,T+1}) R^+_{1,T} = A W_D + B, so hat B - B = A W_D lies in range(A). Consequently A A^+ (hat B - B) = hat B - B, and A W_R + B = hat B even though A A^+ != I in all experiments (n_r > n_in). The numerical condition is satisfied: Table 1 reports ||R R^+ - I||_F around 5e-11. The one genuine caveat is that Eq. 3.2 presupposes condition (iii); if R_{1,T} is rank-deficient, hat B - B need not stay in range(A) and the simplified replication formula is not covered. The paper explicitly lists condition (iii), so this is a stated regularity assumption, not an algebraic overreach. No code is provided, but the central claim does not depend on the numerical demonstrations.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Echo state networks (ESNs) conventionally train the readout layer to reconstruct the input using supervised targets. This paper shows that, when the activation function is invertible and the input matrix has full column rank, the least-squares readout for input reconstruction can be written in closed form using only the ESN parameters and observed reservoir states (Theorem 1, Eq. 2.4; Theorem 2, Eq. 2.5). The authors give an RLS algorithm for online readout computation and use the identity to reformulate dynamical system replication (Eq. 3.2) and a Kalman-filter-based noise filtering scheme (Algorithms 2 and 3) as tasks that do not require the original input sequence. Numerical experiments on a piecewise oscillatory signal, the Lorenz-63 system, and noisy reconstruction support the claims.","tokens_in":21724,"tokens_out":20711,"duration_ms":181158,"significance":"The central algebraic derivation is sound and I find no load-bearing error. Theorems 1 and 2 follow from Lemma 1 by direct substitution, and the numerical checks (Table 1) are consistent with the stated regularity conditions. I specifically examined the potential issue with Eq. (3.2): the equality A W_R + B = \\hat B does not require A A^+ = I. Under condition (iii), \\hat B - B = A D_{1,T} R^+_{1,T} lies in the range of A, so A A^+(\\hat B - B) = \\hat B - B. The paper states condition (iii) in Section 2.3, so this is a stated regularity assumption rather than an algebraic overreach. The contribution is conceptual: it identifies a new principle for reservoir computing and reinterprets two downstream tasks within the same framework. The paper is transparent about its limitations (invertibility of sigma, full parameter access, no benchmark comparisons), which are appropriate for a theory-oriented paper.","major_comments":[],"minor_comments":[{"comment":"The equality A W_R + B = \\hat B in Eq. (3.2) is valid under the full-row-rank condition (iii) of Theorem 2 and does not require A A^+ = I, because \\hat B - B lies in the range of A under that condition. A one-sentence cross-reference to condition (iii) at the point of use would prevent a reader from inferring an unintended square-matrix assumption.","section":"§3.1.2, Eq. (3.2)"},{"comment":"Algorithm 1 initializes \\hat B_1 = B, which appears to conflict with the later claim that \\hat B can be computed without access to B. Please clarify that this initialization is a warm start and that the RLS recursion converges to the same minimizer for B-independent initializations, or modify the algorithm to use a B-independent initialization when the goal is to avoid using B.","section":"Algorithm 1"},{"comment":"The phrase 'strictly simpler task' should be sharpened: it is the construction of the replicated system \\hat f_R on the reservoir state space that avoids using A and B; verifying its equivalence to the true dynamical system still relies on the structural conditions from Section 2.","section":"§3.1.2"},{"comment":"The sentence 'Noise filtering without adaptation of covariance matrix \\hat R_t also works when the input noise intensity during testing \\sqrt{tr Σ^(2)} < 1 is 100 to 101' is garbled; the intended noise-intensity range should be restated clearly.","section":"§3.2.3"},{"comment":"There is a typo: 'full column rank dondition' should read 'full column rank condition'.","section":"§4.4.1"}],"recommendation":"minor_revision","confidential_remarks":"The paper is mathematically sound and the central claim is defensible. The contribution is more conceptual than algorithmic, which may affect fit for journals expecting state-of-the-art empirical results; the authors are explicit about this intention. I recommend minor revision to address the clarity issues listed for the authors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The central identity is right: with invertible sigma and full column rank A, the input sequence is recoverable from reservoir states alone, so the readout for input reconstruction can be trained without seeing the input. The reader's flagged worry about Eq. 3.2 does not land—hat B - B sits in range(A), so AA^+ = I is not needed; condition (iii) covers the rest. The paper states its regularity assumptions explicitly, and the algebra checks out.\n\nWhat is genuinely new is the reformulation. It turns input reconstruction, dynamical system replication, and noise filtering into unsupervised tasks when the ESN parameters are known. That is a conceptually useful shift, not a breakthrough. Theorems 1 and 2 are elementary but clean, and the RLS algorithm is a sensible way to make the identity online. The numerical experiments are illustrative and honest; the authors admit they are not benchmarking against existing methods, and no code is provided. The limitations section is refreshingly direct about the conditions and about the fact that real-world performance is untested.\n\nSoft spots: The noise-filtering part is heuristic. The connection to Kalman filtering is plausible but the covariance adaptation is ad hoc, and the experiments only show that filtering helps, not that it beats anything. The 'strictly simpler' claim for dynamical system replication is defensible—computing hat B does not require A or B—but it's a slightly weaker sense of 'simpler' than the wording suggests. The neuroscience discussion is speculative, though clearly labeled as such.\n\nWho is this for? People who work on reservoir computing theory or on unsupervised sequence models. It is not a game-changer, but it is a correct, clearly written paper with a useful perspective. It deserves a serious referee.","headline":"A correct, modest algebraic reformulation that turns input reconstruction in ESNs into an unsupervised task; the math holds up and the paper deserves peer review.","tokens_in":22254,"tokens_out":2126,"would_cite":true,"duration_ms":21480,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Input reconstruction in echo state networks is an unsupervised task, not a supervised one, whenever the network's parameters are known and its activation function is invertible.","keywords":["echo state networks","reservoir computing","input reconstruction","unsupervised learning","signal reconstruction","theoretical neuroscience","dynamical system replication","noise filtering"],"falsifier":"A reader can verify the main theorem in a few lines of computation: with known $(A, B, \\sigma)$ satisfying the conditions, generate an input, record the reservoir states, and compare the state-only readout $W_R = A^+[\\sigma^{-1}(R_{2,T+1}) - B R_{1,T}]R^+_{1,T}$ with the supervised readout $W_D = D_{1,T}R^+_{1,T}$; in exact arithmetic they must coincide, so any discrepancy beyond floating-point error refutes Theorem 1. For the replication claim, on the paper's own Lorenz setup, evaluate the residual $(I - A A^+)(\\hat B - B)$ and check whether the divergence between the unsupervised system $\\hat f_R$ and the supervised system $\\hat f_D$ tracks its size as the reservoir dimension is varied; if it does, the simplification in the replication step is approximate rather than exact.","tokens_in":21195,"feed_emoji":"🧠","tokens_out":17488,"duration_ms":149217,"temperature":0.7,"pith_summary":"This paper claims that input reconstruction in an echo state network — a recurrent network whose fixed reservoir projects inputs into high-dimensional states while only the output layer is trained — can be done without ever using the input as a supervised target. The key observation is that under the state update $r_{t+1} = \\sigma(A d_t + B r_t)$, an invertible activation $\\sigma$ and a full-column-rank input matrix $A$ let the input be recovered from two consecutive reservoir states alone: $d_t = A^+[\\sigma^{-1}(r_{t+1}) - B r_t]$. Substituting this identity into the standard least-squares readout makes the readout for input reconstruction a closed-form function of reservoir states and the known parameters $A$, $B$, and $\\sigma$, so the task becomes unsupervised. The same identity is then applied to reformulate two downstream tasks — replicating the dynamical system that generated the input, and filtering noise out of contaminated inputs — as unsupervised procedures, with numerical experiments on synthetic inputs, the Lorenz attractor, and noisy observations. If correct, the work establishes a working principle for reservoir computing: fixed parameters are not just constraints to respect but specific values worth exploiting, and it draws a formal link between echo state networks and predictive-coding models of perception.","feed_headline":"Echo-state nets can learn to rebuild their own input unsupervised","feed_subtitle":"Inverting the reservoir update turns readout training into a function of states and known weights alone.","key_machinery":"The carrying object is an inversion identity for the ESN update (Lemma 1): $d_t = A^+[\\sigma^{-1}(r_{t+1}) - B r_t]$, which expresses the unobserved input at time $t$ as an explicit function of two consecutive reservoir states and the known matrices $A$, $B$, and $\\sigma$. Substituting this identity into the least-squares readout formula converts the input-reconstruction readout from a function of labeled target samples into a function of reservoir states alone, and this substitution is the algebraic heart of the paper. The second piece of machinery is the auxiliary least-squares problem $L_{\\sigma,T}(\\hat B) = \\sum_{t=1}^{T-1} \\|\\sigma^{-1}(r_{t+1}) - \\hat B r_t\\|^2$, whose minimizer $\\hat B$ appears in the simplified readout $W_R = A^+(\\hat B - B)$; because this loss depends only on $\\sigma$ and on the states, it supports the sequential recursive-least-squares algorithm, and because it does not involve $A$ or $B$, it is the basis for the paper's claim that dynamical-system replication is a strictly simpler task than input reconstruction in terms of internal knowledge requirements.","core_discovery":"The paper's central claim is that input reconstruction in an echo state network is an unsupervised task. For the state update $r_{t+1} = \\sigma(A d_t + B r_t)$, the authors prove (Theorem 1) that whenever the activation $\\sigma$ is invertible and the input matrix $A$ has full column rank, the input is recoverable from consecutive reservoir states by $d_t = A^+[\\sigma^{-1}(r_{t+1}) - B r_t]$, so the least-squares readout $W_D = D_{1,T}R^+_{1,T}$, which conventionally needs the original input as a target, equals $W_R = A^+[\\sigma^{-1}(R_{2,T+1}) - B R_{1,T}]R^+_{1,T}$, which uses only reservoir states and the known parameters. When the reservoir state matrix has full row rank (Theorem 2), this simplifies to $W_R = A^+(\\hat B - B)$, where $\\hat B$ minimizes the state-only loss $L_{\\sigma,T}(\\hat B) = \\sum_{t=1}^{T-1} \\|\\sigma^{-1}(r_{t+1}) - \\hat B r_t\\|^2$, which in turn admits an online recursive-least-squares update (Algorithm 1). The authors then show that the same machinery reformulates two applications: replicating the dynamical system that generated the input reduces to the autonomous map $\\hat f_R(r) = \\sigma[\\hat B r]$, whose training needs only the activation function $\\sigma$; and noise filtering proceeds by using this replicated map as the evolution law in an ensemble Kalman filter with adaptively estimated observation covariance. Numerical experiments demonstrate reconstruction of a synthetic input, replication of the Lorenz-63 attractor whose projected orbits match the true attractor's shape, and a reduction of reconstruction error from an RRMSE of about 1.03 to 0.50 under strong test-time noise. The authors draw the general principle: reservoir computing should exploit the specific values of fixed parameters, not merely keep them fixed.","pith_inferences":["Because all numerical experiments use reservoirs far larger than their inputs ($n_r > n_{in}$), the residual term $(I - A A^+)(\\hat B - B)$ is nonzero in every trial, so the unsupervised replicated system matches the supervised one only approximately; re-running the Lorenz experiment at several reservoir sizes would show whether the divergence between the two systems scales with that residual.","A practical recipe follows for physical or analog reservoirs: whenever a trustworthy model of the reservoir's own dynamics is available, the same inversion identity could recover external driving signals from internally measured states alone, which is the system-identification direction the paper leaves open.","The surrogate-inverse experiments with the non-invertible ReLU activation show reconstruction degrades gracefully rather than failing outright, suggesting that invertibility is sufficient but not strictly necessary; quantifying how the error grows as the activation departs from invertibility would sharpen the theory.","Algorithm 1's update is a local, covariance-tracking rule, which makes it a candidate plasticity mechanism for neural circuits; testing whether a neuron-like implementation can carry the precision-matrix recursion would extend the paper's brain-modeling claim."],"forward_implications":["Input reconstruction becomes a closed-form function of reservoir states and known parameters, so an ESN with known $A$, $B$, and $\\sigma$ can train its readout online without any label channel, which suits autonomous or embedded processing.","Any task that reduces to input reconstruction inherits the unsupervised formulation: the paper demonstrates replication of the Lorenz-63 attractor from reservoir states alone, with the projected orbits of the replicated system matching the true attractor's shape.","Noise filtering becomes a two-stage unsupervised procedure — train the readout and the replicated dynamics at one noise level, then run an ensemble Kalman filter with adaptive observation-covariance estimation on noisier inputs — and the experiments show filtered reconstruction beats the unfiltered readout whenever test-time noise exceeds training-time noise.","Because the reconstruction readout depends only on the network and not on which input drove it, the upper limit of an ESN's information-processing capacity is independent of the input type, which separates network properties that help input reconstruction from input properties that hurt it.","The state-only loss $L_{\\sigma,T}$ coincides with a predictive-coding objective that compares a stimulus-evoked state with an internally predicted one, giving a deductive rather than heuristic bridge from reservoir computing to models of perception in the brain."],"supporting_citations":[{"why":"defines the echo state network and its least-squares readout training, the supervised baseline the paper replaces.","marker":"(Jaeger, 2001)"},{"why":"introduces the supervised reservoir algorithm for replicating chaotic attractors that the paper reformulates as unsupervised.","marker":"(Pathak et al., 2017)"},{"why":"shows attractor reconstruction with the input time series as the target, the input-reconstruction framing the paper builds on.","marker":"(Lu et al., 2018)"},{"why":"supplies the recursive least squares algorithm used by the sequential unsupervised readout (Algorithm 1).","marker":"(Haykin, 2002)"},{"why":"originates Kalman filtering, the estimation framework the noise-filtering application relies on.","marker":"(Kalman, 1960)"},{"why":"provides the ensemble Kalman filter used in the noise-filtering algorithm (Algorithm 3).","marker":"(Evensen, 2003)"},{"why":"supplies the adaptive covariance-estimation technique used to update the observation-noise matrix online.","marker":"(Mehra, 1972)"}],"fun_headline_variants":["Unsupervised input reconstruction in echo state networks","No labels needed: ESNs reconstruct their own input","Inverting reservoir states teaches ESNs input reconstruction","ESNs reconstruct inputs from reservoir states alone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the step $A W_R + B = \\hat B$ in the dynamical-system application, which is exactly true only when the input dimension equals the reservoir dimension so that $A A^+ = I$; none of the paper's experiments satisfies this, since every reservoir is far larger than its input.","fun_headline_variants_meta":{"raw":{"variants":["Unsupervised input reconstruction in echo state networks","No labels needed: ESNs reconstruct their own input","Inverting reservoir states teaches ESNs input reconstruction","ESNs reconstruct inputs from reservoir states alone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000567,"raw_usage":{"total_tokens":2817,"prompt_tokens":1206,"completion_tokens":1611,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":822,"completion_tokens_details":{"reasoning_tokens":1559}},"tokens_in":822,"tokens_out":1611,"duration_ms":12226,"temperature":1.0,"reasoning_tokens":1559,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:18:13.114357+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader can verify the main theorem in a few lines of computation: with known $(A, B, \\sigma)$ satisfying the conditions, generate an input, record the reservoir states, and compare the state-only readout $W_R = A^+[\\sigma^{-1}(R_{2,T+1}) - B R_{1,T}]R^+_{1,T}$ with the supervised readout $W_D = D_{1,T}R^+_{1,T}$; in exact arithmetic they must coincide, so any discrepancy beyond floating-point error refutes Theorem 1. For the replication claim, on the paper's own Lorenz setup, evaluate the residual $(I - A A^+)(\\hat B - B)$ and check whether the divergence between the unsupervised system $\\hat f_R$ and the supervised system $\\hat f_D$ tracks its size as the reservoir dimension is varied; if it does, the simplification in the replication step is approximate rather than exact.","supporting_citations":[{"cited_title":"echo state","cited_arxiv_id":null,"evidence_quote":"defines the echo state network and its least-squares readout training, the supervised baseline the paper replaces."},{"cited_title":"R., Girvan, M., and Ott, E","cited_arxiv_id":null,"evidence_quote":"introduces the supervised reservoir algorithm for replicating chaotic attractors that the paper reformulates as unsupervised."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the recursive least squares algorithm used by the sequential unsupervised readout (Algorithm 1)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the ensemble Kalman filter used in the noise-filtering algorithm (Algorithm 3)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the adaptive covariance-estimation technique used to update the observation-noise matrix online."}],"review_version":1}