{"id":"949c7a21-15cf-4a6e-9fd1-fec6b3dd4ffb","arxiv_id":"2507.08766","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A CNN plus k-means plus multi-well Hopfield hybrid reaches roughly 99% on MNIST, but the reported best accuracy is inconsistent and the test set appears to have been used for tuning.","lead":"This paper combines CNN feature extraction, k-means clustering, and a multi-well Hopfield-style energy function to classify MNIST digits, reporting 99.2% to 99.44% accuracy. The result is plausible for MNIST, but the paper has inconsistent accuracy figures and tunes hyperparameters directly on the test set.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hopfield component is not shown to be load-bearing: the update equations are internally inconsistent, and the argmax decision rule may make the sy dynamics redundant; no nearest-centroid ablation is reported.","rationale":"I agree with the reader that test-set tuning is a serious validity threat: Table 2 and §3.3 report choosing CNN depth, K, beta, and lambda using the 10,000 test images, so the 99.2/99.26/99.44 numbers are not unbiased generalization estimates. That alone justifies a conditional verdict. My read differs in emphasis: the more load-bearing threat is the role of the Hopfield component. The paper's own math is inconsistent (Eq. 5 differs from the derivative of Eq. 4), and the decision rule is a prototype-argmax, so the multi-well dynamics may be redundant. If the proposed ablation shows no accuracy difference, the core contribution reduces to a known pipeline (pretrained CNN with k-means prototypes) and the abstract's mechanism claim fails. This is an internal-support problem rather than a disagreement with consensus. Because the flaws are fixable with a corrected derivation, a proper validation split, and a baseline, I keep the reader's CONDITIONAL verdict rather than escalating to REJECT; the check above would determine whether the mechanism concern actually lands.","tokens_in":9890,"tokens_out":9832,"duration_ms":131528,"concrete_test":"Run one experiment on the four-layer CNN features and K=12 wells from Table 2: (a) reproduce the paper's update (Eq. 5+6), (b) correct the gradient sign in Eq. 5 and run true gradient descent, and (c) classify each test image directly by argmax_m exp(-β||sx−μm,x||²) with sy=0 and no update; also compute a plain k-means nearest-centroid baseline. If (c) matches (a) within about 0.1 percentage points on the 10,000 test images, or if (b) does not recover Table 2's 99.44%, the Hopfield dynamics are not shown to be the mechanism and the central claim should be restated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's contribution rests on the claim that the multi-well energy function is minimized by Hopfield dynamics and that this dynamics drives classification. Two conditions are needed: the printed dynamics must actually descend E(sx,sy), and the dynamics must change the predictions relative to a static distance rule. Condition 1 is shaky: differentiating Eq. 4 gives -2β exp(-β||s−μm||²)(ym−sy) + 2λsy, not the expression in Eq. 5, so the derivation is inconsistent even if the sign pair in Eqs. 5–6 accidentally implements a descent update. Condition 2 is the load-bearing gap: the decision rule in §2.4/§3.2 is argmax_m exp(-β||s−μm||²), i.e., nearest-prototype choice. The sy update takes place in R^10 and is not demonstrated to change that argmax; the paper reports no ablation such as k-means nearest-centroid on the same CNN features with the same wells. The reported 99.2–99.44% accuracy could therefore be entirely attributable to CNN feature extraction plus prototype lookup, with the Hopfield machinery making no causal contribution. That would undercut the abstract's mechanism claim even if the accuracy numbers are reproducible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid model for MNIST classification that combines a pre-trained CNN feature extractor, k-means clustering to construct class-specific prototype wells, and a multi-well Hopfield-style energy function whose minimization over a 10-dimensional class state yields a prediction. The central empirical claim is a test accuracy of about 99% on the 10,000-image MNIST test set, with the best reported values ranging from 99.2% (abstract) to 99.44% (Table 2). The paper frames the contribution as an interpretable energy-based classification mechanism in which Hopfield dynamics, rather than a static distance rule, drive the final decision.","tokens_in":10112,"tokens_out":6146,"duration_ms":73497,"significance":"If the central claim were fully supported, an interpretable energy-based classifier reaching roughly 99% on MNIST would be a useful demonstration of hybrid Hopfield-CNN methods, and the paper's emphasis on prototype coverage and feature quality would be a reasonable practical insight. The paper also has strengths: it clearly separates feature extraction from the energy model, reports detailed hyperparameter scans, and states the decision rule explicitly. However, the evaluation currently does not establish the core mechanism: the Hopfield dynamics are not shown to be necessary or correctly implemented, and the tuning procedure uses the test set directly. The paper therefore currently falls short of the standard needed to support its headline claims.","major_comments":[{"comment":"The printed gradient is not the gradient of the energy in Eq. (4). The derivative of -exp(-β||s-µm||²) with respect to sy is 2β exp(-β||s-µm||²)(sy - ym), not 2β exp(-β||s-µm||²)(ym - sy) as written in Eq. (5). Consequently, the update sy^{t+1} = sy^t + η ∂E/∂sy in Eq. (6) is not gradient descent on E as claimed. The authors must correct the sign and either rerun the experiments with the stated update or revise the description to match the actual implementation.","section":"§2.4 and §3.2, Eqs. (5)–(6)"},{"comment":"The hyperparameter tuning is performed directly on the 10,000-image test set: the text says the process \"leverag[es] the MNIST dataset's 60,000 training and 10,000 test images\" and Table 2 reports test accuracies for many configurations. Selecting the best configuration by test accuracy introduces optimistic bias, so the reported 99.2–99.44% figures are not clean estimates of generalization. A validation split or nested procedure is needed, and the final model's test accuracy should be reported exactly once after model selection.","section":"§3.3 and Table 2"},{"comment":"The predicted class is the class of the well with the highest weight w_m = exp(-β||s-µm||²). At initialization sy=0, every well has the same ||ym||² = 1, so this rule reduces to nearest-centroid classification in the CNN feature space. The paper never reports an ablation that removes the Hopfield dynamics, such as classifying directly by the nearest k-means centroid on the same features, nor does it measure how often the sy update changes the argmax. Without such evidence, the claim that the multi-well Hopfield network is the load-bearing classification mechanism is unsupported; the accuracy could be entirely attributable to the CNN features and prototype lookup.","section":"§2.4 and §3.2, decision rule"},{"comment":"The headline accuracy is stated inconsistently: 99.2% in the abstract, 99.44% in Table 2 for the four-layer CNN with 120 wells and β=0.003, 99.26% in the conclusion and §5, and 91.44% plus 97.52% are also described as current results in §1.1. These numbers must be reconciled, and the exact configuration corresponding to the final reported accuracy must be specified.","section":"Abstract, §1.1, Table 2, §4"},{"comment":"The four-layer architecture reported as \"32→64→128→256\" is not fully specified: the text does not state where the pooling layers are inserted, whether BatchNorm and dropout are used in the final configuration, or how the 2304-dimensional feature vector is obtained. Without this information the best result in Table 2 cannot be reproduced. Please provide the exact architecture, training epochs, and all hyperparameters for the headline configuration.","section":"§2.1 and §3.1, Table 2"}],"minor_comments":[{"comment":"There are several typos and formatting errors: \"CIF AR-10\" should be \"CIFAR-10\", \"Demercigil\" should be \"Demircigil\", \"Manoucheri at al.\" should be \"Manouchehri et al.\", and the table heading \"T est Accuracy\" has an extra space.","section":"Throughout"},{"comment":"The sentence \"achieving 91.44% accuracy on MNIST—competitive with CNNs\" is confusing because the abstract and conclusions report roughly 99%; if 91.44% is an earlier single-layer result, this should be stated clearly.","section":"§1.1"},{"comment":"The statement that well separation has \"mean distance 1.11, minimum 0.49\" is not tied to a specific configuration and conflicts with the much smaller minimum distances (e.g., 0.25) reported in Table 2; please clarify which experiment this refers to.","section":"§2.2"},{"comment":"The text says \"gradient descent only happens in R10,\" but Eq. (6) and the energy in Eq. (4) still depend on sx through the well distances; the statement would be clearer if it said that sx is held fixed during the sy update.","section":"§2.4"},{"comment":"Some references are incomplete or inconsistently formatted, e.g., [25] uses \"Cote & Tatnall\" with initials in one place and full names in the bibliography; please check all entries against a consistent style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper would benefit from a clearer positioning relative to the existing hybrid CNN–Hopfield work of Keddous et al. [12], which the manuscript cites but does not compare against. The absence of code and the test-set tuning are the main reasons I cannot currently treat the reported accuracy as reliable. The requested ablations and the corrected energy update are, in my view, essential before the mechanism claim can be evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a modest but real prototype-classifier idea, and the reported accuracy is plausible; the evaluation and the central mechanism claim, however, have serious holes. The paper needs a clean rerun with a proper validation split and a nearest-centroid baseline before the headline can be trusted.\n\nWhat is actually here: CNN features, k-means prototypes per class, and a Gaussian-well energy from Krotov-Hopfield, combined in a way I have not seen exactly this way. That is a legitimate extension, and the empirical pattern in Table 2 (deeper CNN and more wells help up to a point, then saturate) is coherent. The tuning table is transparent about what was tried. Credit where due: someone built and ran this, and the numbers are not absurd.\n\nThe soft spots are real. The headline accuracy is inconsistent: 99.2% in the abstract, 99.26% in the conclusion, 99.44% in Table 2, and Section 1.1 says 91.44% for the three-layer configuration that Table 2 lists at 97.52%. That cannot stand. More importantly, the tuning in Section 3.3 was done directly on the 10,000-image test set, with no validation split, so the best reported numbers carry optimistic bias. There is also no code and no baseline that isolates the Hopfield component. The decision rule is argmax over well weights; a plain nearest-centroid lookup on the same CNN features would likely get most of the accuracy. Without an ablation, the central claim that Hopfield dynamics drive classification is unsupported. Finally, Eq. 5 is not the gradient of Eq. 4; the sign is flipped. The update in Eq. 6, with the printed gradient, happens to implement gradient descent on the true energy, but the derivation as written is wrong and must be fixed.\n\nNone of this kills the underlying idea. It is a plausible prototype-based classifier that deserves a careful rerun. But as submitted, the claims outrun the evidence.\n\nThis is a paper for readers interested in energy-based or prototype classifiers, not for anyone needing a rigorous MNIST benchmark. I would send it to review with a strong request to redo the evaluation with validation-based selection, add a nearest-centroid baseline, correct the equations, and release code. That is more useful than a desk reject, because the idea is testable and the reported gains are not implausible.","headline":"A plausible prototype-classifier idea undermined by test-set tuning, inconsistent numbers, and no ablation showing the Hopfield part does anything.","tokens_in":10693,"tokens_out":4391,"would_cite":false,"duration_ms":51784,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CNN plus k-means-built energy wells classifies MNIST at 99.2–99.4% accuracy while keeping decisions readable as energy minimization.","keywords":["Hopfield network","multi-well energy function","MNIST classification","k-means prototypes","convolutional feature extraction","energy-based classification","associative memory","hyperparameter tuning"],"falsifier":"A reader could settle the matter by rerunning the pipeline with a strict train/validation/test split, tuning every hyperparameter on a held-out validation set and touching the 10,000-image test set only once; if the resulting accuracy falls materially below the reported 99.2-99.44%, the headline numbers reflect test-set tuning rather than generalization.","tokens_in":9620,"feed_emoji":"🔢","tokens_out":4789,"duration_ms":55149,"temperature":0.7,"pith_summary":"The paper tries to show that a classical Hopfield-style associative memory can be upgraded into a competitive image classifier if it is fed good features and given many attractors per class. Concretely, a convolutional network maps each digit image to a normalized feature vector; k-means builds several prototype \"wells\" per digit class; and classification is done by gradient descent on a ten-dimensional class state until it settles into the lowest-energy well. The reported result is test accuracy near 99.2%, with one configuration at 99.44%, on the 10,000-image MNIST test set. A sympathetic reader should care because the claim is that interpretable, energy-based classification can reach the accuracy of end-to-end CNNs rather than lagging far behind.","feed_headline":"Hybrid Hopfield-CNN hits 99.4% on MNIST digits","feed_subtitle":"CNN features plus k-means-built wells make an energy landscape that classifies handwritten digits like a deep network.","key_machinery":"The central object is the multi-well energy function $E(s_x,s_y) = -\\sum_{m=1}^{M} \\exp(-\\beta\\|s-\\mu_m\\|^2) + \\lambda\\|s\\|^2$ over the concatenated feature-and-class state $s=[s_x,s_y]$, with wells $\\mu_m=[\\mu_{m,x}, y_m]$ formed from normalized k-means centroids and one-hot class vectors. The exponential term makes nearby wells dominant while $\\beta$ controls well sharpness, and $\\lambda$ keeps the class state bounded. Classification is gradient descent on $s_y$ in $\\mathbb{R}^{10}$ with the image features fixed, so the predicted class is the one whose wells dominate the energy landscape.","core_discovery":"The central claim is that a multi-well energy function over k-means prototypes, applied to convolutional features, is itself a high-accuracy classifier for MNIST. The paper shows that accuracy rises with CNN depth and with the number of wells per class, reaching 99.26% and 99.44% in the strongest configurations, and that the energy landscape provides a built-in explanation for each prediction: the winning class is the one whose wells carry the largest exponential weight. The authors present this as an interpretable alternative to end-to-end CNNs, with the energy function as the decision rule rather than a post-hoc explanation.","pith_inferences":["A natural stress test would degrade test images with noise or occlusions and check whether accuracy decays smoothly; if it does, the energy landscape is acting as a genuine associative memory rather than a lookup table.","The same CNN-plus-k-means-plus-energy pipeline could be applied to Fashion-MNIST or CIFAR-10, where the well-separation diagnostic would show whether the approach transfers beyond digits.","The reported numbers were selected after tuning on the test set, so an honest generalization estimate would require a validation split; reading the table maximum as a prediction for brand-new data likely overstates the model."],"forward_implications":["Accuracy climbs with CNN depth and with the number of wells up to about $K=12$ per class, then plateaus or drops at $K=15$, so the paper's own tuning rule is \"deep features first, wells second.\"","Well separation, measured by minimum and mean Euclidean distances between normalized centroids, tracks accuracy and can serve as a pre-test diagnostic for whether the wells are too overlapping.","Because the Hopfield layer only sees normalized features, a different CNN or even a non-convolutional feature extractor could be swapped in without changing the energy-minimization machinery.","The energy landscape is the decision rule, so every prediction comes with a distance-to-prototype justification that a fully connected softmax layer does not naturally provide."],"supporting_citations":[{"why":"Supplies the exponential energy term $E=-\\sum \\exp(-\\beta\\|s-\\mu\\|^2)$ that the paper adapts into its multi-well classifier.","marker":"[14]"},{"why":"Defines the MNIST benchmark and the convolutional baseline whose accuracy this hybrid aims to approach.","marker":"[16]"},{"why":"Provides evidence that deeper CNN architectures push MNIST accuracy above 99%, motivating the depth scaling in the tuning table.","marker":"[2]"},{"why":"Shows a prior hybrid CNN-Hopfield design, the comparison point for combining feature extraction with associative memory.","marker":"[12]"},{"why":"Gives a continuous-state Hopfield model with Gaussian wells that the paper contrasts with its multi-well construction.","marker":"[4]"}],"fun_headline_variants":["Energy landscape with k-means wells classifies MNIST at 99.4%","MNIST at 99.4% via Hopfield wells over CNN features","Multi-well Hopfield net with k-means prototypes hits 99.4% on MNIST","Interpretable energy-based classifier: 99.4% on MNIST","99.4% MNIST accuracy from CNN features and Hopfield energy wells"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported accuracies are honest estimates of generalization, but the model's hyperparameters were chosen using the 10,000-image test set itself, so the reported 99.2-99.44% may be optimistically biased.","fun_headline_variants_meta":{"raw":{"variants":["Energy landscape with k-means wells classifies MNIST at 99.4%","MNIST at 99.4% via Hopfield wells over CNN features","Multi-well Hopfield net with k-means prototypes hits 99.4% on MNIST","Interpretable energy-based classifier: 99.4% on MNIST","99.4% MNIST accuracy from CNN features and Hopfield energy wells"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000586,"raw_usage":{"total_tokens":2702,"prompt_tokens":842,"completion_tokens":1860,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":1754}},"tokens_in":458,"tokens_out":1860,"duration_ms":15349,"temperature":1.0,"reasoning_tokens":1754,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:09:58.232783+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle the matter by rerunning the pipeline with a strict train/validation/test split, tuning every hyperparameter on a held-out validation set and touching the 10,000-image test set only once; if the resulting accuracy falls materially below the reported 99.2-99.44%, the headline numbers reflect test-set tuning rather than generalization.","supporting_citations":[{"cited_title":"41, 11887–11892","cited_arxiv_id":null,"evidence_quote":"Supplies the exponential energy term $E=-\\sum \\exp(-\\beta\\|s-\\mu\\|^2)$ that the paper adapts into its multi-well classifier."},{"cited_title":"11, 2278–2324","cited_arxiv_id":null,"evidence_quote":"Defines the MNIST benchmark and the convolutional baseline whose accuracy this hybrid aims to approach."},{"cited_title":"Cire¸ san, Ueli Meier, Luca M","cited_arxiv_id":null,"evidence_quote":"Provides evidence that deeper CNN architectures push MNIST accuracy above 99%, motivating the depth scaling in the tuning table."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows a prior hybrid CNN-Hopfield design, the comparison point for combining feature extraction with associative memory."},{"cited_title":"2, 288–299","cited_arxiv_id":null,"evidence_quote":"Gives a continuous-state Hopfield model with Gaussian wells that the paper contrasts with its multi-well construction."}],"review_version":1}