{"id":"ba9fae3d-3090-4cce-b331-7663d1696036","arxiv_id":"2504.12561","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Kernel Ridge Regression learns Hopfield network weights in closed form, matching the recall of Kernel Logistic Regression up to a storage load of 1.5 while training up to 20 times faster.","lead":"This paper applies Kernel Ridge Regression to train Hopfield associative memories, reporting recall performance similar to Kernel Logistic Regression but with a closed-form solution that trains in a fraction of the time. The practical message is that a standard regression-based kernel method can serve as a fast alternative for building high-capacity memory networks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The β=1.5 storage-capacity claim rests entirely on clean-state recall, which KRR's regression objective makes nearly trivial at λ=0.01; noise robustness is only demonstrated at β=0.2, so high-load associative retrieval is unproven.","rationale":"I agree with the reader's verdict and weakest assumption; the clean-state capacity test is the load-bearing gap, and the reader identified it correctly. The paper has real strengths that I would credit: the KRR closed-form solution (Eq. 1) is correctly derived; the timing comparison (20× faster than this KLR implementation at β=1.0) is plausible given O(P^3) direct solve versus 200 iterations of O(P^2N) kernel work; and Fig. 2 provides genuine evidence that kernel regression produces working basins of attraction at β=0.2, down to m(0)≈0.2. These support the speed and low-load claims. The problem is the abstract/conclusion, which inflate the clean-state fixed-point result into 'state-of-the-art storage capacity... successfully operating even for P>N' and attach the 'noise robustness' label measured only at β=0.2. The single experiment that would settle this is the Fig. 2 protocol at β=1.0 and β=1.5; if KRR still recalls from m(0)=0.9 inputs at those loads, the capacity claim stands as a genuine associative-memory result; if not, the paper's central claim reduces to fixed-point stability. I do not see an internal inconsistency or a demonstrated error in the KRR construction; the concern is missing evidence for the strongest claim, which is precisely the CONDITIONAL verdict. The missing pseudo-inverse baseline and the KLR iteration-count choice affect comparison framing, not the correctness of the KRR method, so they are secondary. Verdict remains CONDITIONAL (UNCHANGED), with acceptance contingent on the high-load basin test.","tokens_in":7932,"tokens_out":17115,"duration_ms":168523,"concrete_test":"Replicate Fig. 2's basin-of-attraction protocol at β=1.0 and β=1.5 (N=500, P=500 and P=750): for each stored pattern, generate 10 corrupted initial states with m(0) ∈ {0.5, 0.7, 0.8, 0.9, 0.95}, run T=25 synchronous updates with the paper's settings (γ=1/N, λ=0.01), and report success rate (m(25)>0.95). Use β=0.2 as a control to confirm the protocol reproduces Fig. 2. If KRR recalls m(0)=0.9 states with ≥95% success at β=1.5, the capacity claim is validated as associative memory; if success collapses, the clean-state capacity curve reflects fixed-point stability only and the 'operating even for P>N' claim must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim that KRR 'successfully operates even for P > N' rests only on the clean-state recall test of Section III.B, where s(0) = ξ^μ. Because KRR regresses each pattern onto itself (Eq. 1, α = (K+λI)^{-1}X), the activation at ξ^ν is ξ^ν − λ((K+λI)^{-1}X)_ν. With λ=0.01 and RBF kernel entries ~0.135 off-diagonal for random patterns, the shrinkage term has RMS ≈ 0.011, so sign(f(ξ^ν)) = ξ^ν holds deterministically; the test measures near-interpolation stability, not associative retrieval, exactly as Section III.B itself frames it ('stable fixed points'). The only corrupted-input experiment (Fig. 2) runs at β=0.2, yet the abstract pairs the β=1.5 capacity with 'noise robustness.' No evidence shows that basins of attraction survive at β ≥ 1.0. The risk is concrete: the summed kernel contributions from the P−1 non-target patterns grow linearly with P (≈101 at β=1.5), so the learned weights must cancel interference that grows with load; at a corrupted state the target's kernel weight shrinks (0.82 at m(0)=0.9), disrupting the cancellation. If recall from even mildly corrupted states fails at high load, KRR at P>N is an interpolating lookup with stable fixed points and no basins, and the central 'high-capacity associative memory' claim is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using kernel ridge regression (KRR) to train Hopfield-type associative memories. Because the dual variables solve a linear system (K+λI)α=X, training is non-iterative. Experiments with N=500 random bipolar patterns compare KRR with Hebbian learning, LLR, and KLR. The paper reports 100% clean-state recall up to storage load β=1.5, noise robustness at β=0.2 matching KLR, and substantially shorter training times. The central claim is that KRR matches KLR in capacity and robustness while being much faster to train.","tokens_in":8172,"tokens_out":5861,"duration_ms":63092,"significance":"If the capacity and robustness claims survive scrutiny, the closed-form training is a useful practical contribution: KRR retains the kernel advantage of KLR while replacing iterative logistic-loss optimization with a single linear solve, and the reported speedup over KLR at β=1.0 is concrete and reproducible. The paper also provides the first direct empirical KRR/KLR comparison for Hopfield training, and the timing methodology is transparent. However, the storage-capacity result rests on a clean-state fixed-point test that is nearly a restatement of the regression objective; the actual associative-memory property—retrieval from corrupted states at high load—is not demonstrated. The noise-robustness experiment covers only β=0.2, leaving the high-load regime untested. The significance therefore depends on a missing experiment rather than on the closed-form speed claim, which is solid.","major_comments":[{"comment":"The storage-capacity evaluation initializes recall from the clean stored pattern s(0)=ξ^μ and therefore measures only whether each pattern is a stable fixed point. For KRR, the prediction on the training patterns is H=K(K+λI)^{-1}X = X - λ(K+λI)^{-1}X, so with λ=0.01 the update from ξ^μ is essentially the identity plus a small shrinkage term. Thus the reported 100% success up to β=1.5 is an interpolation/fixed-point test, not a test of associative retrieval from corrupted inputs. The paper itself describes Section III.B as testing 'stable fixed points'; the abstract and conclusion, however, present this as 'storage capacity' and 'successfully operating even for P>N'. This conflates interpolation with content-addressable memory and is load-bearing for the main claim.","section":"§III.B, Eq. (1)"},{"comment":"Noise robustness is evaluated only at β=0.2 (P=100). At that low load the basin-of-attraction result is plausible and matches KLR, but it provides no evidence for the high-load regime where the paper claims capacity. In particular, at β=1.5 the summed kernel contributions of the P-1 non-target patterns grow with P, and the effect of a corrupted input is to reduce the target pattern's kernel weight; nothing in the paper shows that basins of attraction survive at β≥1.0. The claim that KRR achieves high capacity 'and noise robustness' at the same operating point is therefore unsupported. An experiment reporting final overlap m(T) versus initial overlap m(0) at β=1.0 or 1.5 would resolve this.","section":"§III.B, Fig. 2"},{"comment":"The learning-time advantage is reported only for β≤1.0, while the capacity claim extends to β=1.5. Since KRR's cost is dominated by forming and solving a P×P system, extrapolating 'especially at higher storage loads' to β=1.5 is an extrapolation; the timing table should include at least the largest β used in the capacity experiment. This is secondary to the capacity issue, but it directly affects the 'orders of magnitude faster at high capacity' summary.","section":"§III.C, Table 1"}],"minor_comments":[{"comment":"The sentence 'aiming to directly predict the target bipolar state ξ^μ_i ∈ {0,1}' should read {-1,+1}; the targets are bipolar, not binary.","section":"§II.B.2"},{"comment":"The predictor formula has an index inconsistency: f_i should be written as a sum over the stored patterns ν of K(x,ξ^ν) α_{ν i}, with the kernel argument and the dual-variable index aligned. As printed, the subscripts are mismatched.","section":"§II.B.2"},{"comment":"The recall success criterion m(T)>0.95 is reasonable, but reporting a single 100% curve without error bars or multiple independent pattern draws makes it hard to assess variability, especially at the phase-transition boundaries of the baselines.","section":"§III.A"},{"comment":"The recall update is stated as 's(t+1)=sign(h(s(t)))', but for Hebbian and LLR the update excludes self-coupling (j≠i). The kernel update does not have an analogous exclusion; it would be helpful to state explicitly whether the kernel methods include self-interaction and why this does not bias the recall results.","section":"§II.C"}],"recommendation":"major_revision","confidential_remarks":"The learning-time contribution is solid and likely publishable, but the headline capacity claim currently rests on a clean-state fixed-point test that is close to a restatement of the regression objective. The authors should be asked to add corrupted-input recall experiments across the full β range (especially β≥1) before the capacity/noise-robustness claims can be accepted. If those experiments fail, the paper should be reframed as a training-efficiency study with stable fixed points rather than high-capacity associative memory."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2504.12561. First, the learning-time result is solid: KRR's closed-form solution is 20x faster than the author's KLR implementation at beta=1.0 and orders of magnitude faster than LLR, and that comparison is honestly measured. Second, the headline storage-capacity claim—KRR operating at beta=1.5—does not hold up as associative memory. The capacity test initializes from the clean pattern itself, and because KRR regresses each pattern onto itself, the activation at a stored pattern is the pattern minus a lambda-weighted shrinkage term. At lambda=0.01 the sign is preserved essentially by construction. So the test shows the regressor interpolates the training data, not that the network has real basins of attraction at high load.\n\nThe paper does some things well. It frames the clean-state test as 'stable fixed points' rather than hiding it, and it is upfront that recall complexity is O(PN). The noise-robustness comparison at beta=0.2 gives a meaningful basin-size measurement, and the finding that KRR matches KLR there is new—this is the first direct KRR-vs-KLR comparison I know of for Hopfield learning. The speed advantage is also genuine and worth knowing: a non-iterative kernel method with closed-form dual variables is an obvious but useful observation, and the measurements are clearly reported.\n\nThe soft spots are concentrated in the evaluation design. The most important gap is that corrupted-input tests are only run at beta=0.2. The abstract and conclusion pair 'P>N' with 'noise robustness,' but no experiment shows basins surviving at beta>=1.0. The stress-test concern is concrete: at high load the summed kernel contributions from non-target patterns grow linearly with P, and the learned dual variables must cancel that interference; whether that cancellation generalizes to corrupted states is untested. A second gap is the absence of a pseudo-inverse baseline, which is also closed-form and is the standard comparison for linear associative memories. Without it, the 'efficient high-capacity' claim is under-constrained. Third, no code is provided, and lambda and gamma are fixed without sensitivity analysis. These are not fatal—the method is simple and the timing data is there—but they mean the paper is a methods note with a speed claim, not a demonstration of a high-capacity associative memory.\n\nWho gets value from this? Someone building Hopfield-style modules who wants a quick closed-form trainer and cares more about speed than about large basins. A serious referee should engage with it, because the speed comparison is useful and reproducible enough to be worth checking, and the capacity question is clearly scoped. I would recommend conditional acceptance with a required revision: either add corrupted-input sweeps across beta or rewrite the capacity claims to say only that KRR maintains stable fixed points at high load.","headline":"Closed-form KRR gives a real speedup over KLR, but the storage-capacity claim rests on a clean-state test that KRR passes almost automatically; the standalone value is the timing comparison, not the P>N memory claim.","tokens_in":8747,"tokens_out":2719,"would_cite":false,"duration_ms":26945,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes Kernel Ridge Regression as a non-iterative kernel learning method for Hopfield networks, claiming it matches KLR's capacity and robustness while training about 20 times faster at high load.","keywords":["kernel ridge regression","Hopfield networks","associative memory","storage capacity","kernel logistic regression","closed-form learning","noise robustness","RBF kernel"],"falsifier":"Run the paper's noise-robustness protocol at storage loads $\\beta=0.5$, $1.0$, and $1.5$; if KRR's mean final overlap drops below 0.95 for initial overlaps clearly above 0.2, the claimed high-load capacity does not extend to pattern completion.","tokens_in":7653,"feed_emoji":"🧠","tokens_out":9361,"duration_ms":86918,"temperature":0.7,"pith_summary":"The paper asks whether an associative memory can be made both high-capacity and cheap to train. It proposes replacing the iterative optimization of Kernel Logistic Regression with Kernel Ridge Regression, whose parameters come from a single linear solve. In simulations on 500-neuron networks of random bipolar patterns, KRR achieves 100 percent clean-state recall up to a storage load of $\\beta=1.5$ ($P>N$), matching KLR, and has nearly identical noise-robustness basins at $\\beta=0.2$. Training time stays under 0.1 seconds up to $\\beta=1.0$, about 20 times faster than KLR and hundreds of times faster than linear logistic regression. If correct, the result makes strong associative memories practical in settings where retraining must be fast.","feed_headline":"Kernel ridge regression pushes Hopfield storage to a load of 1.5","feed_subtitle":"One closed-form solve matches iterative kernel logistic regression on recall, about 20 times faster to train.","key_machinery":"The mechanism is the closed-form dual-variable equation of kernel ridge regression, $\\alpha = (K+\\lambda I)^{-1}X$, where $K$ is the $P\\times P$ RBF kernel matrix among stored patterns. Because this solution requires no iteration, training cost is dominated by one $O(P^2N)$ kernel-matrix construction and one $O(P^3)$ linear solve; recall then evaluates the kernel between the current state and every stored pattern and multiplies the resulting $1\\times P$ vector by the $P\\times N$ dual matrix. The kernel trick in an RBF feature space is what separates non-linearly packed patterns enough to create stable fixed points and large basins of attraction.","core_discovery":"The central claim is that treating pattern storage as a kernel ridge regression problem yields Hopfield-network memories with the same recall performance as KLR at a fraction of the learning cost. For each neuron $i$, KRR predicts the bipolar state by $f_i(\\xi) = \\sum_\\mu K(\\xi,\\xi^\\mu) \\alpha_{\\mu i}$, with dual variables $\\alpha_i = (K+\\lambda I)^{-1} y_i$ in closed form. Recall updates every neuron as $s_i(t+1) = \\mathrm{sign}(\\sum_\\mu K(s(t),\\xi^\\mu)\\alpha_{\\mu i})$. Under the RBF kernel with $\\gamma = 1/N$ and $\\lambda = 0.01$, the learned network keeps every stored pattern as a stable fixed point at loads up to $\\beta=1.5$, and from corrupted inputs it converges to the correct pattern when the initial overlap is about $0.2$, the same behavior as KLR. The paper also finds that for bipolar sign-threshold recall, logistic and squared-error losses produce virtually identical attractor landscapes, and presents this as the first direct empirical comparison of KRR and KLR for Hopfield-network learning.","pith_inferences":["One implicit boundary: the capacity test starts every recall from the stored pattern itself, so the $\\beta=1.5$ result directly certifies fixed-point stability, while the size of the basins of attraction at high load is not measured; corrupted-start recall was only tested at $\\beta=0.2$.","A natural next experiment is the same noise-robustness sweep at $\\beta=0.5$, $1.0$, and $1.5$; if KRR's required initial overlap rises sharply with load, practical pattern-completion capacity will be lower than clean-state capacity.","The near-identical KRR/KLR behavior suggests that any loss with a strong penalty for wrong-sign predictions near zero might yield similar memories, opening a family of closed-form or cheaply iterative losses.","Kernel approximations such as Nyström or random features, which the paper names as future work, directly address the $O(PN)$ recall cost and could scale this approach to much larger pattern sets."],"forward_implications":["At $\\beta=1.0$, KRR learns about 20 times faster than the KLR implementation with 200 updates and over 400 times faster than LLR with 100 updates, so high-capacity memories can be retrained very quickly at $N=500$.","Because KRR and KLR are virtually identical in capacity and noise robustness, squared-error loss can replace logistic loss in sign-based recall without sacrificing performance.","The clean-state recall results extend past $P>N$, so kernel-based associative memories can operate in regimes where classical Hebbian and linear methods fail completely.","The learning-time advantage grows with load, but recall complexity remains $O(PN)$ for both kernel methods, so the gain is in training rather than retrieval."],"supporting_citations":[{"why":"Defines the Hopfield network and the associative-memory recall task that every compared method is designed to solve.","marker":"[1]"},{"why":"Establishes the classical Hebbian storage limit around $\\beta=0.14$, the baseline that kernel methods are shown to exceed.","marker":"[2]"},{"why":"Supplies linear logistic regression, the linear supervised baseline that KRR must outperform in capacity and speed.","marker":"[4]"},{"why":"Introduces Kernel Logistic Regression for Hopfield networks and provides the prior results and implementation KRR is compared against.","marker":"[6]"},{"why":"Provides the Kernel Ridge Regression formulation and its closed-form dual solution that the proposed learning rule uses.","marker":"[7]"},{"why":"Supplies the kernel trick that implicitly maps patterns into a high-dimensional feature space, the mechanism behind the capacity gain.","marker":"[8]"}],"fun_headline_variants":["Closed-form Hopfield learning hits load 1.5, matches kernel logistic","KRR: non-iterative Hopfield memory, load 1.5, 20x faster training","Hopfield recall without iteration: KRR matches KLR at load 1.5","One closed-form solve trains Hopfield net to storage load 1.5"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that perfect recall when starting from the stored pattern measures associative-memory capacity; since KRR regresses each pattern onto itself, clean patterns are fixed points by construction, leaving corrupted-input retrieval at high loads untested.","fun_headline_variants_meta":{"raw":{"variants":["Closed-form Hopfield learning hits load 1.5, matches kernel logistic","KRR: non-iterative Hopfield memory, load 1.5, 20x faster training","Hopfield recall without iteration: KRR matches KLR at load 1.5","One closed-form solve trains Hopfield net to storage load 1.5"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000611,"raw_usage":{"total_tokens":2875,"prompt_tokens":1010,"completion_tokens":1865,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":1773}},"tokens_in":626,"tokens_out":1865,"duration_ms":13178,"temperature":1.0,"reasoning_tokens":1773,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:28:08.346648+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's noise-robustness protocol at storage loads $\\beta=0.5$, $1.0$, and $1.5$; if KRR's mean final overlap drops below 0.95 for initial overlaps clearly above 0.2, the claimed high-load capacity does not extend to pattern completion.","supporting_citations":[{"cited_title":"*WaŮ ^ \"z-W,޻N= ,d&M6