{"id":"c599f703-708a-4364-a31a-9530ea42e1a9","arxiv_id":"2412.20930","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A quantum annealing based hard attention network selects image blocks by solving a QUBO, and claims faster, smoother, and more noise-robust training than two classical hard attention baselines.","lead":"This paper proposes a hard attention layer for image classifiers that uses a quantum annealer to pick which image blocks to keep, instead of learning the mask with gradients. The authors report faster convergence and better noise robustness than two classical hard attention baselines on MNIST and CIFAR-10.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (7) minimizes a nonnegative Gram form; after ReLU it selects low-energy image blocks rather than salient ones, so the annealer's global optimum is not an attention mask.","rationale":"The reader's rejection is well-founded, but its weakest-assumption phrasing leaves room for the reply that saliency is an empirical matter. My stress-test closes that loophole: for the specific architecture in Algorithm 1, the objective is not merely arbitrary, it is directionally wrong. Because the Gram matrix is entrywise nonnegative after ReLU, minimizing x^T Q x selects low-energy blocks; maximizing it, or adding a label-dependent term, would be required for a saliency mask. This is an internal mathematical tension with the paper's own placement of QAHAM after a ReLU, not a disagreement with any external consensus. It also makes the reported smooth curves and fast convergence unsurprising: the mask suppresses the strongest features, giving the downstream classifier a low-variation, low-information input that can look stable but does not implement attention. The paper provides no code, no error bars, and no independent verification, so the empirical claims cannot be checked, but the conceptual problem is already sufficient. The proposed brute-force and sign-flip tests would settle empirically whether the selected blocks are actually salient; absent such evidence, the central claim of a saliency-preserving hard attention mechanism is unsupported.","tokens_in":8995,"tokens_out":5987,"duration_ms":64663,"concrete_test":"Take one trained QAHAN model and a held-out batch. Freeze the network, extract the post-ReLU feature tensor after Conv2, and form the block vectors B_a. For a small subproblem (e.g., n=25 blocks, k=5), solve Eq. (7) by brute-force enumeration of all C(25,5)=53130 masks. Then compare selected versus unselected blocks on (i) mean L2 norm and (ii) linear-probe classification accuracy. If selected blocks have lower mean norm or no better class separability than unselected blocks, the QUBO is selecting low-information content. A decisive control is to repeat the full MNIST/CIFAR training with the sign of x^TQx flipped in Eq. (7) (maximize instead of minimize); if the flipped model matches or exceeds the reported QAHAN accuracy, the minimization direction is the central error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the D-Wave minimizer of Eq. (7) returns an attention mask xopt that highlights important content. This fails at the level of the objective. In Section III-A, Q is the Gram matrix with entries B_a · B_b (Eq. 9), so x^T Q x = ||Σ_a x_a B_a||² (Eq. 10). Algorithm 1 places QAHAM after the second ReLU, so every block vector B_a is entrywise nonnegative. Consequently every Gram entry is nonnegative and the quadratic form is a sum of nonnegative terms. For a fixed number k of selected blocks, the minimum is biased toward low-energy blocks: a zero-activation block contributes nothing and is always preferred, while a strongly activated block increases the objective. The sparsity term λ1(Σ x_a − k)² and the adhesion penalty λ2 Σ x_a x_{a+1} only constrain cardinality and adjacency; they add no label or class-saliency information. Thus the true global minimum of Eq. (7) is a mask biased toward inactive or weakly activated feature-map locations, not a mask of discriminative image regions. Quantum tunneling cannot repair this, because the minimizer itself is the wrong object.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes QAHAM, a hard attention mechanism whose binary mask is obtained by minimizing a QUBO objective on a D-Wave quantum annealer. The input feature map is partitioned into blocks, the Gram matrix Q of the block vectors is formed, and the annealer minimizes Eq. (7): x^T Q x plus a cardinality penalty and an adjacency penalty. The resulting mask is applied to the features to form QAHAN, a convolutional network with two conv layers, a QAHAM layer, and a fully connected layer. Experiments on MNIST and CIFAR-10 compare QAHAN with two hard-attention baselines, claiming faster convergence, smoother accuracy/loss curves, and better noise robustness. The paper also visualizes the attention masks and reports results under Gaussian noise.","tokens_in":9257,"tokens_out":3446,"duration_ms":35345,"significance":"If the proposed mechanism worked as claimed, it would be a useful demonstration of quantum annealing for a discrete attention selection problem, and the use of real D-Wave hardware together with PyTorch is a strength. However, the central objective is not derived from any task-relevant saliency criterion, and the experimental evidence consists of single-run curves with no statistical support. As presented, the paper does not establish that the annealed mask selects class-relevant image regions, and the strong empirical claims are not substantiated.","major_comments":[{"comment":"The objective being minimized on the annealer is not an attention objective. Since QAHAM is applied after the second ReLU (Algorithm 1, step 2), each block vector B_a is entrywise nonnegative, so Q has nonnegative entries and x^T Q x = ||Σ_a x_a B_a||^2 is a sum of nonnegative terms. Minimizing Eq. (7) therefore favors selecting blocks with small total activation, including all-zero blocks, rather than blocks that are informative for classification. The penalties in Eqs. (12) and (13) only constrain cardinality and adjacency and contain no label or class-saliency information. Consequently, even a perfect global minimization of Eq. (7) returns a mask biased toward low-energy regions, not a hard attention mask. This is a load-bearing flaw: the paper's central claim that x_opt is a set of attention scores is not supported by the mathematical construction.","section":"Section III-A, Eq. (10)"},{"comment":"All experimental comparisons are based on single training runs with no error bars, no multiple seeds, and no statistical tests. Claims such as 'converges faster,' 'smoother accuracy and loss curves,' and 'superior noise robustness' are inferred from visual inspection of individual curves, which is not sufficient to distinguish the proposed method from random initialization or training noise. The implementations of the two baseline hard attention mechanisms (Mnih's and Elsayed's) are not described in terms of architecture, training procedure, or hyperparameter tuning, so the comparison is not reproducible.","section":"Section IV-A and Figs. 2-7"},{"comment":"The reported dimensions are internally inconsistent. Two 5x5 stride-2 padding-2 convolutions reduce a 28x28 MNIST input to 14x14 and then to 7x7, and a 32x32 CIFAR-10 input to 16x16 and then to 8x8. The text states l = m = 56 for MNIST and l = m = 64 for CIFAR, and the FC input sizes are given as 3136 and 4196 in the text but 4096 in Table I. These numbers do not match the feature-map geometry described in Section III-A. The relation between the number of blocks l×m, the flattened feature dimension, and the Q matrix dimension needs to be clarified or corrected.","section":"Section IV-A, Table I"},{"comment":"The procedure for obtaining the QUBO each forward pass is not fully specified. It is unclear whether Q is recomputed for every mini-batch or every epoch, how many D-Wave reads are used, what annealing time is set, how embedding and chain strength are chosen, and what the wall-clock overhead of the annealer calls is. The paper claims faster convergence, but no timing data are reported; if 'faster' refers to epochs rather than wall-clock time, the claim is not informative about practical speed. These details are necessary to reproduce the experiments and to assess the practical contribution.","section":"Algorithm 1 and Section IV-B"}],"minor_comments":[{"comment":"Reference [2] is rendered as '[2?]' in the sentence about HAM; the citation needs to be fixed.","section":"Section II-A"},{"comment":"References [20] and [27] are the same paper by Willsch et al.; the duplicate listing should be removed or merged.","section":"References"},{"comment":"The text contains a typo: 'CIRAR-10' should be 'CIFAR-10.'","section":"Section IV"},{"comment":"The notation n is used interchangeably with l×m in Eqs. (7) and (9)-(11); a consistent notation should be adopted. Also, QAHASM defined in Eq. (11) as xx^T is not used in the rest of the paper; if it is intended to represent the outer product of the mask, its role should be explained.","section":"Section III-A and Eq. (11)"}],"recommendation":"reject","confidential_remarks":"The central objection is not about disagreement with consensus; it is an internal inconsistency between the objective being minimized and the claimed attention behavior. Because the flaw is in the core formulation, a revision would require redesigning the QUBO and re-running all experiments, which is beyond a standard major revision. I therefore recommend rejection, despite the paper's positive use of real quantum hardware."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the thing you need to know about this one is that the central mechanism is wrong in an instructive way. The QAHAM layer defines x^T Q x in Eq. (7) as the squared norm of the sum of selected image blocks (Eq. 10). Since the layer sits after the second ReLU, every block vector is entrywise nonnegative, so every Gram entry is nonnegative and the quadratic form is a sum of nonnegative terms. For a fixed number of selected blocks k, the minimizer is the set of blocks with the smallest L2 norms. The sparse and adhesion penalties only control cardinality and adjacency; they contain no label or saliency information. So the D-Wave annealer returns a mask biased toward inactive feature-map locations, not a mask of discriminative regions. Quantum tunneling can't fix that because the minimizer itself is the wrong object.\n\nWhat is actually new: the specific QUBO with the sparsity and adhesion penalties applied to block selection doesn't appear in the cited prior work, and the author did build the full pipeline and ran it on D-Wave hardware with PyTorch. The paper is clearly written in outline, and the literature review is reasonable. I'll credit that.\n\nThe soft spots: first, the objective. As above, minimizing the norm of a sum of nonnegative vectors is a low-energy selection. The author never justifies why this captures important content, and there is no label-dependent term anywhere. Second, the experiments are single curves with no error bars, no seeds, no ablations, no code release. The 'global optimum' claim is asserted rather than demonstrated; the D-Wave details (number of reads, chain strength, embedding, success probability) are missing. Third, the hyperparameters k, lambda1, lambda2 are hand-picked with no sensitivity analysis. These are more than presentation issues; they mask whether the method actually does anything.\n\nThe conclusion in the paper claims faster convergence, higher accuracy, and noise robustness, but given the objective direction, it's not even clear what the QAHAM layer is selecting. The comparison to Mnih's and Elsayed's HAMs is thus hard to interpret.\n\nWho is this for? Someone tracking ways to turn attention into QUBOs might skim it, but the current form is not usable. I would neither cite it nor send it to a referee. It's a clear write-up of a flawed idea; a desk reject is appropriate. If you're feeling generous, you could use it as a case study of why post-ReLU Gram matrices are the wrong building block for saliency.","headline":"The QUBO objective in Eq. (7) minimizes the squared norm of a sum of post-ReLU blocks, so the annealer's global optimum is a low-energy mask, not a salient one; the paper is clearly written but the central idea doesn't work.","tokens_in":9763,"tokens_out":3909,"would_cite":false,"duration_ms":37872,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["81P68","68T07"],"pacs":["03.67.-a"],"model":"deepseek-v4-flash","headline":"The paper claims that solving a QUBO formulation of hard attention on a D-Wave annealer produces an optimal binary mask, and that the resulting network converges faster, achieves higher accuracy, and is more noise-robust than two…","keywords":["quantum annealing","hard attention","QUBO","D-Wave","image classification","MNIST","CIFAR-10","attention mechanism"],"falsifier":"Train the same network with the QUBO term changed to maximize $||\\sum_a x_a B_a||^2$ instead of minimizing it, or with a random mask of the same sparsity $k$, and compare test accuracy on MNIST and CIFAR-10; if the minimization objective is doing the work, these variants should perform clearly worse.","tokens_in":8778,"feed_emoji":"⚛️","tokens_out":3723,"duration_ms":34841,"temperature":0.7,"pith_summary":"The paper proposes a hard attention mechanism whose binary selection mask is found by a quantum annealer instead of by gradient descent. It claims that solving the QUBO objective on D-Wave produces an optimal mask, and that a convolutional network using this mask trains faster, reaches higher accuracy, and tolerates noise better than two conventional hard attention networks on MNIST and CIFAR-10. The point of the claim is that quantum annealing can replace the non-differentiable selection step in hard attention, avoiding local optima and unstable gradients.","feed_headline":"Quantum annealer chooses image regions that speed up training","feed_subtitle":"A hard-attention layer solved as QUBO on D-Wave beats gradient-trained attention on MNIST and CIFAR-10.","key_machinery":"The load-bearing object is the QAHAM QUBO, written as $x^T Q x + \\lambda_1(\\sum_a x_a - k)^2 + \\lambda_2 \\sum_{a=1}^{n-1} x_a x_{a+1}$, where $Q$ is the Gram matrix of image block vectors. The first term is the squared norm of the sum of selected blocks, the second enforces that exactly $k$ blocks are selected, and the third penalizes selecting neighboring blocks together. A quantum annealer searches the ground state of the corresponding Ising Hamiltonian, producing the optimal binary mask $x_{\\text{opt}}$.","core_discovery":"On its own terms, the paper's central claim is that the hard attention selection problem can be cast as a quadratic unconstrained binary optimization (QUBO) and solved on a quantum annealer, yielding a binary mask $x_{\\text{opt}}$ that minimizes the QAHAM objective. That mask is then applied to convolutional features to produce hard-attention processed features. The paper reports that this QAHAN keeps training accuracy between 0.996 and 0.999 and test accuracy near 1.0 on MNIST, and that it clearly outperforms the Mnih and Elsayed hard attention baselines on CIFAR-10, with smoother loss curves and stable accuracy under noise.","pith_inferences":["Because the QUBO has no label-dependent term, the mask selection is class-agnostic; a supervised variant that adds a discriminative term might improve accuracy further.","A minimal control experiment comparing QAHAN against a network with a random binary mask of the same sparsity would isolate how much of the gain comes from the annealing optimization rather than from the convolutional backbone.","The same block-selection QUBO could be applied to non-image data such as text or speech tokens, since it operates on flattened block vectors.","If the squared-norm objective genuinely selects salient blocks, then maximization of the same term should degrade performance, providing a direct test of the paper's core assumption."],"forward_implications":["Training the attention mask requires no gradient through a discrete sampler, so the non-differentiability problem of hard attention mechanisms is bypassed.","If the annealer returns true optima, the attention layer avoids the policy-gradient failure mode of getting stuck in local optima.","Smoother loss and accuracy curves imply the mask is stable across epochs, which could transfer to other architectures needing discrete selection.","Noise robustness suggests the selected blocks are not overly sensitive to small pixel perturbations.","The same QUBO formulation could be applied to other block-structured inputs, not just images."],"supporting_citations":[{"why":"Defines the recurrent hard attention baseline that QAHAN is compared against.","marker":"[1]"},{"why":"Defines the Saccader hard attention baseline used as the second comparison model.","marker":"[4]"},{"why":"Introduces quantum annealing as the optimization method the paper exploits.","marker":"[18]"},{"why":"Provides the QUBO-to-Hamiltonian formulation the QAHAM objective is mapped onto.","marker":"[26]"},{"why":"Supplies the D-Wave annealer platform on which the QUBO is solved.","marker":"[27]"},{"why":"Provides PyTorch, the deep learning framework for the QAHAN implementation.","marker":"[28]"},{"why":"Supplies the MNIST dataset used in the classification experiments.","marker":"[29]"},{"why":"Supplies the CIFAR-10 dataset used in the classification experiments.","marker":"[30]"}],"fun_headline_variants":["Quantum annealer solves hard attention as QUBO for faster ML","Hard attention gets quantum boost from D-Wave annealer","Quantum annealing attention outperforms gradient-based on CIFAR-10","QAHAN: Annealed attention converges faster without gradients"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the binary mask minimizing the squared norm of the sum of selected image blocks is the mask that keeps the blocks most useful for classification, even though the objective never looks at the labels.","fun_headline_variants_meta":{"raw":{"variants":["Quantum annealer solves hard attention as QUBO for faster ML","Hard attention gets quantum boost from D-Wave annealer","Quantum annealing attention outperforms gradient-based on CIFAR-10","QAHAN: Annealed attention converges faster without gradients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1460,"prompt_tokens":851,"completion_tokens":609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":467,"completion_tokens_details":{"reasoning_tokens":538}},"tokens_in":467,"tokens_out":609,"duration_ms":5741,"temperature":1.0,"reasoning_tokens":538,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:06:16.133800+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same network with the QUBO term changed to maximize $||\\sum_a x_a B_a||^2$ instead of minimizing it, or with a random mask of the same sparsity $k$, and compare test accuracy on MNIST and CIFAR-10; if the minimization objective is doing the work, these variants should perform clearly worse.","supporting_citations":[{"cited_title":"Recurrent models of visual attention,","cited_arxiv_id":null,"evidence_quote":"Defines the recurrent hard attention baseline that QAHAN is compared against."},{"cited_title":"Saccader: Improving accuracy of hard attention models for vision,","cited_arxiv_id":null,"evidence_quote":"Defines the Saccader hard attention baseline used as the second comparison model."},{"cited_title":"Quantum annealing in the transvers e Ising model,","cited_arxiv_id":null,"evidence_quote":"Introduces quantum annealing as the optimization method the paper exploits."},{"cited_title":"Short-depth QAOA circuits and quantum annealing on higher-order ising models,","cited_arxiv_id":null,"evidence_quote":"Provides the QUBO-to-Hamiltonian formulation the QAHAM objective is mapped onto."},{"cited_title":"Benchmarking advantage and D-Wave 2000Q quantum annealers with exact cover problems,","cited_arxiv_id":null,"evidence_quote":"Supplies the D-Wave annealer platform on which the QUBO is solved."},{"cited_title":"Pytorch: An imperative style, high- performance deep learning library,","cited_arxiv_id":null,"evidence_quote":"Provides PyTorch, the deep learning framework for the QAHAN implementation."},{"cited_title":"Gradient-based learning applied to document recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the MNIST dataset used in the classification experiments."}],"review_version":1}