{"id":"69461310-e134-4479-a112-007638ab59d4","arxiv_id":"2507.15063","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Quantum annealing matched simulated annealing on small QUBO-based machine learning tasks, with a hybrid classical clustering plus quantum medoid refinement pipeline giving the best submitted retrieval score in the clustering task.","lead":"This competition paper applies quantum annealing and simulated annealing to three machine learning tasks, feature selection, instance selection, and clustering, by encoding them as QUBO problems and running them on a D-Wave annealer.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Efficiency claim rests on an unvalidated timing comparison; QA advantage is not established as reported.","rationale":"The paper is an honest working-notes submission: it reports code availability, acknowledges that Task 2 differences are not statistically significant, flags the invalid clustering submissions (Experiments 2 and 12), and notes in Section 5.3 that the most promising quantum experiments were not submitted. Those admissions lower the stakes and support a conditional reading rather than outright rejection. The load-bearing gap is narrower: the single quantitative efficiency result is Figure 2(b), and no text describes how the two times were measured. D-Wave's 'annealing time' is a hardware parameter (time per anneal), not the cost of obtaining a solution; the same issue affects the 'total inference time' concept used in Task 2. Without a breakdown, the 10x claim is not verifiable from the paper. Table 2's two QA runs are a natural experiment in variance: identical configuration produced k=13 and k=15, so the better score is within sampling noise. A concrete reproducibility test—end-to-end wall-time comparison plus repeated runs—would settle whether the claim lands. If it fails, the abstract should be softened to 'QA matches SA on small instances with comparable solution quality, but wall-clock efficiency is not established.' I agree with the reader's weakest-assumption statement and see no need to change the conditional verdict, hence UNCHANGED.","tokens_in":14791,"tokens_out":6411,"duration_ms":65352,"concrete_test":"Using the released feature-selection repository, rerun the Figure 2 comparison for the MQ2007 QUBO with the same 100 reads per solver and identical QUBO matrix: measure QA end-to-end wall time from client submission to completed samples (including queue, embedding, programming, anneal, and readout, e.g., from the D-Wave timing info) and SA wall time from the same driver script. Repeat 20 times and record k and nDCG distributions. If end-to-end QA time is not consistently ~10x below SA, or if the 13-feature result is not a stable mode, the abstract's efficiency/competitiveness claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim that QA is 'computationally more efficient' and 'competitive' stands or falls on Section 4.1 / Figure 2(b), which reports QA at roughly one-tenth the time of SA. No timing methodology is given: it is unclear whether the QA number is QPU anneal time, access time, or submission-to-result wall time, and whether the SA number is wall-clock or CPU time for the same number of reads. On D-Wave, anneal time per read is on the order of microseconds, while queue, embedding, programming, and readout are excluded; comparing that component to full SA runtime is not apples-to-apples and can produce a large speedup that has nothing to do with algorithmic superiority. The two QA leaderboard runs in Table 2—same code and QUBO, returning k=13 and k=15—show high run-to-run variance, so the better nDCG of 0.4552 may be a favorable draw rather than a systematic QA advantage. The paper itself states no significance or variance analysis for this comparison. Because the only quantitative support for 'more efficient' is this unvalidated 10x figure, the headline claim is not yet supported as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a QuantumCLEF 2025 competition entry in which three machine-learning tasks—feature selection, instance selection, and clustering—are formulated as QUBO problems and solved with both D-Wave quantum annealing (QA) and simulated annealing (SA). For feature selection, the authors construct QUBO matrices from mutual-information- and permutation-importance-based terms, report leaderboard nDCG scores for two QA runs (0.4552 with 13 features and 0.4436 with 15 features), and claim that QA is about ten times faster than SA. For instance selection, they extend the bcos QUBO approach with SVC-margin and instance-deletion diagonal heuristics, reporting F1 scores at 25% reduction. For clustering, they build a classical-to-quantum pipeline that pre-clusters documents and then uses a constrained QUBO medoid-refinement stage; their k-Medoids submission achieved a leaderboard nDCG of 0.58. The overall conclusion is that QA can be a competitive and efficient tool for discrete ML optimization on current hardware.","tokens_in":14997,"tokens_out":4312,"duration_ms":47799,"significance":"If the efficiency and competitiveness claims were fully substantiated, the paper would provide useful evidence that quantum annealing can serve as a practical heuristic for small-scale discrete optimization problems and as a component in hybrid classical-quantum ML pipelines. The paper has notable strengths: all code is made publicly available through three GitHub repositories; the QUBO formulations are grounded in prior work rather than invented ad hoc; and the authors are unusually candid about limitations, including the lack of statistical significance in Task 2, the all-teams-below-baseline result on the Vader dataset, and the invalid clustering submissions caused by a dimensionality mismatch. These strengths make the manuscript a useful working-notes record. However, the headline claims go beyond what the reported evidence supports, mainly because the timing comparison in Task 1 lacks methodology, the QA feature-selection advantage rests on two unreplicated runs, and the clustering improvements are not shown to come from the quantum refinement stage.","major_comments":[{"comment":"The claim that QA is 'computationally more efficient' rests on an unvalidated timing comparison. Figure 2(b) reports annealing time in milliseconds, but the paper never states whether the QA figure is QPU anneal time, access time, or submission-to-result wall time, nor whether the SA figure is wall-clock or CPU time for the same QUBO instance and the same number of reads. On D-Wave hardware, anneal time per read is on the order of microseconds, while queue, embedding, programming, and readout overhead are typically excluded; comparing only that component to a full SA runtime can produce a large speedup that has nothing to do with algorithmic advantage. The authors should specify exactly what was measured for both solvers, confirm that the comparison used the same QUBO and read count, and report variance over repeated runs. Without this, the central efficiency claim is not supported as reported.","section":"Section 4.1, Figure 2(b) and Table 2"},{"comment":"The two QA feature-selection submissions had identical code and QUBO configuration but returned different feature counts, k=13 and k=15, with leaderboard nDCG scores of 0.4552 and 0.4436. The paper attributes the divergence to 'inherent randomness and probabilistic nature' but provides no repeated trials, no mean/standard deviation, and no significance analysis. The better score may therefore be a favorable draw from a high-variance process rather than a systematic QA advantage. At minimum, the authors should report multiple QA runs for the same configuration and compare the distribution of solution quality and runtime against SA; otherwise the statement that QA achieved 'comparable effectiveness' with fewer features is not established.","section":"Section 4.1, Table 2"},{"comment":"The Task 2 results do not support any claim of QA or SA advantage. The paper's own text states that differences are not statistically significant for the Yelp dataset and that all teams perform worse than the baseline on the Vader dataset. The submitted QA run on Vader, for example, has F1 62.6(7.5) versus the baseline 88.9(0.8). The conclusion that 'QA often matched or exceeded the performance of SA in less time' therefore cannot be justified by the instance-selection experiments. The authors should either restrict their claims to feature selection and clustering or provide additional evidence, such as significance testing or a more difficult benchmark, before generalizing beyond Task 1.","section":"Section 4.2, Table 3"},{"comment":"The abstract claims 'consistent improvements in cluster compactness and retrieval metrics' from the classical-to-quantum medoid-refinement pipeline, but Table 4 does not demonstrate consistency. For example, at k=10 the internal nDCG values range from 0.32 to 0.60 across configurations, with several runs (e.g., Runs 5, 6, 14, 15) far below the corresponding baselines. Moreover, the two submitted UMAP-based clustering runs (Experiments 2 and 12) were invalid because dimensionality-reduced centroids were evaluated against full-dimensional queries, so only one of three official clustering submissions was valid. The table also lacks an ablation that compares the same classical clustering method with and without the quantum refinement stage, so any DBI improvement cannot be attributed to the QUBO refinement rather than to the choice of classical clustering algorithm. The authors should provide a direct before/after comparison and clarify which configurations actually support the consistency claim.","section":"Section 4.3, Table 4"}],"minor_comments":[{"comment":"The equation numbering in Section 3.4 is inconsistent: the text refers to 'Eq. (2)', 'Eq. (3)', and 'Eq. (5)' when the displayed equations are numbered (4) and (5), and the earlier general QUBO is Eq. (1). Please renumber or re-reference the equations so that the medoid objective, the Welsch similarity, and the parameter choices (alpha, beta, gamma) are clearly tied to the correct displayed formulas.","section":"Section 3.4, Equations (2)-(5)"},{"comment":"The tables would be easier to interpret if the number of runs and variance were reported. In particular, Table 2 lists only two QA runs with no repeated trials, and Table 1 mixes local validation scores with leaderboard scores in the same cells using parentheses; a clearer separation (e.g., two columns) would avoid ambiguity.","section":"Table 1 and Table 2"},{"comment":"There are several typos and awkward phrasings, including 'Another argument that supports this insight can is illustrated' and 'statisticaly' in the conclusion. These should be corrected in a final revision.","section":"Section 4.2, text after Figure 3"},{"comment":"Some reference markers are inconsistently formatted, such as '[ch. 31][15]' in Section 3.3, and the citation style for the D-Wave Ocean SDK and dimod constraint generator could be made more uniform. Also, the term 'centroid embeddings' in the abstract is used interchangeably with 'medoids' in the body; please define the relationship clearly.","section":"References and notation"}],"recommendation":"major_revision","confidential_remarks":"This is a competition working note rather than a full research article, and the contributions are incremental. The manuscript's main value is its candid documentation of practical QA experiments and the public release of code. The abstract and conclusion overstate the evidence, particularly the efficiency claim in Task 1 and the 'consistent improvements' claim in Task 3. The paper is within the scope of a venue that accepts such working notes, but the load-bearing claims need to be tightened before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this is an honest working-notes paper from QuantumCLEF 2025, and its best parts are the two new instance-selection diagonal heuristics and the careful comparison of classical pre-clustering methods before QUBO medoid refinement. The headline claim that QA is 'computationally more efficient' is not supported by the evidence as reported.\n\nThe genuinely new pieces are modest but real. For Task 2, the SVC-margin distance and instance-deletion influence heuristics are reasonable extensions of the bcos QUBO backbone, and the authors correctly note the batching issue. For Task 3, the systematic comparison of k-Medoids, HDBSCAN, GMM, and hybrid as upstream candidate generators, with the fixed-k constraint enforced via dimod.generators.combinations, is a useful practical datapoint. The paper is also commendably candid: it states that Task 2 differences are not statistically significant, that all teams lost to baseline on Vader, and that two of three clustering submissions were invalid due to a dimensionality mismatch. That kind of transparency is worth something.\n\nThe soft spots are exactly where the abstract overreaches. The 'one-tenth the time' claim rests on Figure 2(b) with no timing methodology. It is not clear whether QA time is anneal time, access time, or wall time, and whether SA time includes the same overheads. Given that the two QA feature-selection runs with identical code returned different k (13 vs 15), the better nDCG may be a favorable draw. That efficiency claim needs a serious apples-to-apples comparison before it can stand. On clustering, the claim of 'consistent improvements' is not backed by the table: internal nDCG is mixed relative to baselines, and there is no ablation that isolates the quantum refinement step from the classical pre-clustering. The one strong submitted run (Exp 1) is suggestive but not systematic.\n\nThe math and QUBO formulations are mostly imported from cited work, and the derivations are fine. Citations are appropriate; self-citation is not an issue. The data is thin but honestly presented. This is a paper for people working on hybrid quantum-classical pipelines who want a practical datapoint, not for someone seeking a settled verdict on QA vs SA.\n\nI would send it to peer review, expecting heavy revision on the timing comparison and a more careful abstract. It deserves serious referee time.","headline":"Honest working-notes paper with two genuinely new heuristics; the QA-efficiency claim rests on an unvalidated timing comparison and the abstract overstates the clustering results.","tokens_in":15600,"tokens_out":3003,"would_cite":false,"duration_ms":30148,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper aims to show that quantum annealing can solve discrete machine-learning optimization problems—feature selection, instance selection, and clustering—competitively with classical simulated annealing, at about a tenth of the…","keywords":["quantum annealing","QUBO","simulated annealing","feature selection","instance selection","clustering","medoid refinement","hybrid quantum-classical pipeline"],"falsifier":"Run the same MQ2007 feature-selection QUBO many times on QA and SA with identical reads, and time the full pipeline from submission to returned samples; if QA's median time is not roughly one-tenth of SA's, or if the 13-feature outcome falls inside the run-to-run variance of the 15-feature outcome, the efficiency and competitiveness claim loses its support.","tokens_in":14533,"feed_emoji":"⚛️","tokens_out":9661,"duration_ms":90904,"temperature":0.7,"pith_summary":"This paper tries to establish that quantum annealing (QA) is already a competitive and practical solver for small discrete machine-learning optimization problems, not just a theoretical prospect. It casts feature selection, instance selection, and clustering as quadratic unconstrained binary optimization (QUBO) problems and runs the same formulations on a quantum annealer and on classical simulated annealing. The headline result is that QA matches simulated annealing's solution quality on feature selection while taking about one-tenth of the time, and that a classical-then-quantum pipeline for clustering consistently tightens clusters and improves retrieval. If true, it would mean current quantum hardware can be slotted into real ML workflows as a faster heuristic for combinatorial subproblems, with the caveat that the comparison depends on a fair timing measurement.","feed_headline":"Quantum annealing cuts feature-selection runtime tenfold","feed_subtitle":"A hybrid classical-quantum pipeline also tightens clustering, pointing to a practical near-term role for current annealers.","key_machinery":"The load-bearing object is the QUBO objective $f(x) = x^T Q x$, a quadratic function over binary variables that quantum annealers accept natively; every task in the paper is reduced to choosing the matrix $Q$ so that the minimum of $f$ encodes the desired subset or medoid selection. Feature selection puts importance scores on the diagonal and redundancy scores off the diagonal, with a penalty enforcing the number of selected features. Instance selection uses cosine similarities between embeddings off the diagonal and the paper's new SVC-distance or deletion-influence scores on the diagonal. Clustering uses the k-medoids QUBO objective, with pairwise distances converted by the Welsch M-estimator (a correntropy loss) and with the exact-k constraint generated through a dedicated quadratic constraint generator rather than a plain penalty; the quantum stage only refines medoids proposed by a classical clustering stage.","core_discovery":"On its own terms, the paper's central discovery is that a QUBO formulation of each task lets quantum annealing reach essentially the same solution quality as simulated annealing with far less compute. For feature selection on the MQ2007 collection, QA achieved a leaderboard nDCG@10 of 0.4552 with only 13 selected features, close to the top score of 0.4580 at 21 features, and did so in roughly one-tenth of the annealing time of SA. In instance selection, the paper extends the bcos QUBO with two new diagonal importance heuristics—distance to an SVM margin and an instance-deletion influence score—but finds no statistically significant difference among methods at 25% reduction on the provided datasets, which it argues are too easy. In clustering, the pipeline runs classical k-Medoids, HDBSCAN, or GMM to produce candidate medoids, then applies a fixed-k QUBO medoid refinement; the best submitted run outperformed the reported baselines, and the authors report consistent improvements in compactness and retrieval metrics.","pith_inferences":["The reported 10x speedup is for annealing time, not end-to-end pipeline time; a full workflow benchmark that includes QUBO construction, embedding, and queue time would likely soften the advantage, so the competitive-efficiency claim should be read as solver-level, not system-level.","The two identical QA feature-selection submissions returning 13 and 15 features suggest high run-to-run variance; leaderboard comparisons built on a single QA draw may overstate the method's typical performance, and repeated sampling would give a fairer estimate.","The clustering pipeline's benefit probably depends on the upstream classical clustering quality: quantum refinement adds the most when classical candidates are noisy or overlapping, and could be neutral when classical clusters are already clean; this dependency is testable by ablating the refinement step.","The instance-selection diagonal heuristics are computed per 80-document batch, so a natural next test is computing the importance scores globally before batching; the paper's own batching discussion implies this could remove a known inconsistency."],"forward_implications":["Feature-selection QUBOs of the size tested here (tens of features) can be solved by QA with quality on par with SA at about a tenth of the annealing time, so time-sensitive ML pipelines could offload this step to an annealer.","Very small feature subsets can carry most of the ranking signal: the 13-feature QA selection nearly matched a 21-feature leaderboard leader, so learned feature budgets may be tighter than commonly assumed.","A hybrid classical-quantum clustering pipeline—classical candidates, quantum-constrained medoid refinement—can enforce exact k while improving compactness and retrieval metrics, pointing to refinement rather than full-problem solving as the practical role of current annealers.","In instance selection, QUBO-based heuristics and random sampling are statistically indistinguishable on the provided sentiment datasets at 25% reduction, implying that harder benchmarks are needed before QA-based instance selection can show a measurable benefit.","Fixed-k constraints encoded as quadratic penalty terms through a dedicated constraint generator are more stable on quantum hardware than simple linear or diagonal penalties, which matters for any QUBO task requiring exact cardinality."],"supporting_citations":[{"why":"Supplies the importance-plus-redundancy QUBO construction that Task 1 adapts for feature selection.","marker":"[8]"},{"why":"Provides the bcos QUBO instance-selection backbone and the per-batch processing logic that Task 2 extends with new diagonal heuristics.","marker":"[12]"},{"why":"Supplies the k-medoids QUBO objective used in the clustering refinement stage.","marker":"[17]"},{"why":"Provides the fixed-k quadratic constraint generator that makes exact medoid counts reliable on quantum hardware.","marker":"[28]"},{"why":"Is the SDK used to embed QUBOs and submit them to the annealer, so it carries the implementation of every QA run.","marker":"[22]"},{"why":"Gives the general QUBO and Ising mapping that justifies expressing ML tasks as $x^T Q x$ optimization.","marker":"[6]"},{"why":"Supplies the Welsch M-estimator and correntropy loss used to convert pairwise distances into robust similarity scores in clustering.","marker":"[27]"},{"why":"Is the prior hybrid hierarchical quantum-clustering approach whose classical-preprocessing-plus-quantum-refinement structure the pipeline adopts.","marker":"[18]"},{"why":"Provides the mutual-information and conditional-mutual-information definitions used to populate feature importance and redundancy entries.","marker":"[24]"}],"fun_headline_variants":["Quantum annealing speeds feature selection 10x with matching quality","Hybrid quantum-classical clustering tightens metrics","Quantum annealing matches simulated annealing, 10x faster","QUBO-based ML tasks: QA competitive with SA at 10% time","Hybrid pipeline improves clustering via quantum medoid refinement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claims assume the timing comparison is apples-to-apples—same QUBO instance, same number of reads, and QA time reflecting solver time rather than queue or embedding overhead—and that the 13-feature QA result is a representative outcome rather than a lucky draw from a high-variance process.","fun_headline_variants_meta":{"raw":{"variants":["Quantum annealing speeds feature selection 10x with matching quality","Hybrid quantum-classical clustering tightens metrics","Quantum annealing matches simulated annealing, 10x faster","QUBO-based ML tasks: QA competitive with SA at 10% time","Hybrid pipeline improves clustering via quantum medoid refinement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00172,"raw_usage":{"total_tokens":6795,"prompt_tokens":925,"completion_tokens":5870,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":5788}},"tokens_in":541,"tokens_out":5870,"duration_ms":42120,"temperature":1.0,"reasoning_tokens":5788,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:41:59.991170+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same MQ2007 feature-selection QUBO many times on QA and SA with identical reads, and time the full pipeline from submission to returned samples; if QA's median time is not roughly one-tenth of SA's, or if the 13-feature outcome falls inside the run-to-run variance of the 15-feature outcome, the efficiency and competitiveness claim loses its support.","supporting_citations":[{"cited_title":"Pasin, W","cited_arxiv_id":null,"evidence_quote":"Provides the bcos QUBO instance-selection backbone and the per-batch processing logic that Task 2 extends with new diagonal heuristics."},{"cited_title":"Bauckhage, N","cited_arxiv_id":null,"evidence_quote":"Supplies the k-medoids QUBO objective used in the clustering refinement stage."},{"cited_title":"Pasvolsky, D.-W","cited_arxiv_id":null,"evidence_quote":"Provides the fixed-k quadratic constraint generator that makes exact medoid counts reliable on quantum hardware."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the SDK used to embed QUBOs and submit them to the annealer, so it carries the implementation of every QA run."},{"cited_title":"Lucas, Ising formulations of many np problems, Frontiers in Physics 2 (2014) 5","cited_arxiv_id":null,"evidence_quote":"Gives the general QUBO and Ising mapping that justifies expressing ML tasks as $x^T Q x$ optimization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Welsch M-estimator and correntropy loss used to convert pairwise distances into robust similarity scores in clustering."},{"cited_title":"Alvarez-Giron, J","cited_arxiv_id":null,"evidence_quote":"Is the prior hybrid hierarchical quantum-clustering approach whose classical-preprocessing-plus-quantum-refinement structure the pipeline adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the mutual-information and conditional-mutual-information definitions used to populate feature importance and redundancy entries."}],"review_version":1}