{"id":"bd72088b-279b-45df-9bff-a04e091bc849","arxiv_id":"2411.15425","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Using simulated annealing on a QUBO feature selection problem cuts random forest training time for Bitcoin mixer detection by about 30% with F1 falling from 0.92 to 0.91.","lead":"This paper applies quantum-inspired feature selection, mainly simulated annealing on a QUBO model, to pick a subset of transaction features for classifying Bitcoin mixer addresses. The authors report a 30.3% reduction in random forest training time while keeping mixer F1 around 91%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Feature selection in §3.4 runs on the full dataset before the 10-fold CV in §3.5, so the reported 0.91 F1 is likely optimistically biased; a nested-CV re-run is needed to support the central claim.","rationale":"The central claim is conditional on an unbiased performance estimate. The paper's own text (§3.4 vs §3.5) indicates that feature selection occurs before cross-validation, and the QUBO objective explicitly uses the class-correlation term |rho_oj|. This is the weakest link because every downstream comparison inherits it. A large dataset does not automatically remove selection bias; label-based feature selection can overfit even with many samples, especially when the selected set is small. The proposed nested-CV test is decisive: it either reproduces the 0.91 F1 and the time reduction, or it shows the reported number was an artifact. I agree with the reader's diagnosis, and no separate concern is needed. The paper's additional limitations—data imbalance and a planned but absent ablation study—are noted in §5.1, but they are secondary; the leakage issue alone is enough to withhold acceptance. Therefore the reader's REJECT verdict should stand, pending the test.","tokens_in":12754,"tokens_out":4031,"duration_ms":36860,"concrete_test":"Re-run the SA-QUBO feature selection inside each fold of the 10-fold CV: for each training fold, compute Spearman correlations and solve the QUBO on that fold only, then evaluate the held-out fold with the resulting feature subset. Compare held-out F1, AUC, and training time to Table 6, and also compare against random 23-feature subsets and a standard filter (e.g., chi-square or mRMR). If nested SA-RF F1 falls materially below 0.91 or is no better than random-subset RF, the central claim is unsupported. The public code repository makes this check feasible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result depends on the SA-selected feature set being chosen without using held-out labels. Section 3.4 states that, before the QISF process, classes are binarized and Spearman correlations are computed between each feature and the mixer class; the QUBO objective in §2.1 then uses these |rho_oj| values to select 23 features. Section 3.5 applies 10-fold cross-validation only to classifier training and hyperparameter optimization. On the natural reading, the feature subset is fixed on the full dataset, so the test-fold labels influence which features are used. That is a selection-leakage setup, and it biases the F1/AUC numbers in Tables 6 and 7 upward relative to an honest nested procedure. The abstract's '91% F1 while maintaining' claim therefore cannot be taken at face value. The absence of any baseline feature selector (e.g., mRMR, chi-square, or random 23-feature subsets) also leaves open the possibility that the main benefit is just the mechanical effect of using fewer features. The 30.3% training-time reduction is expected when moving from 69 to 23 features, independent of whether the selection method is good.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a quantum-inspired feature selection (QIFS) method based on a QUBO objective optimized by simulated annealing (SA), D-Wave quantum annealing (QA), and a hybrid BQM solver, applied to Bitcoin address classification with a focus on detecting mixer addresses. Using 69 transaction-history features and seven classifiers, the authors report that SA-selected features (23 features) reduce random forest training time by 30.3% while maintaining an F1-score of 0.91 for the mixer class, compared with 0.92 using all features. The paper also reports performance for QA and QA BQM feature subsets and analyzes feature importance and class-wise metrics.","tokens_in":13025,"tokens_out":5269,"duration_ms":45558,"significance":"If the central claim were established, the contribution would be a useful empirical demonstration that QUBO-based feature selection can lower computational cost in blockchain analytics with minimal loss in detection quality. The paper also has strengths: it reports a substantial original data-collection effort, makes code available, and the arithmetic behind the 30.3% training-time reduction is consistent with the timing entries in Table 6. However, the evaluation protocol suffers from a selection-leakage problem that directly affects the headline F1/AUC numbers, and the absence of any baseline feature selector means the reported improvement cannot currently be attributed to the proposed method. The significance is therefore conditional on a corrected re-analysis.","major_comments":[{"comment":"Feature selection is performed on the full dataset before the 10-fold cross-validation. Section 3.4 describes binarizing the classes and computing Spearman correlations between each feature and the mixer label over the whole dataset, then solving the QUBO to select features. Section 3.5 then applies 10-fold cross-validation only to classifier training and hyperparameter optimization. On this description, the held-out test-fold labels influence which features are selected, so the F1 and AUC values in Tables 6 and 7 are optimistically biased. The paper never states that feature selection is nested inside each training fold or performed on a separate selection set. Please re-run the evaluation with nested cross-validation (or a strict train/validation/test split) and report whether the 0.91 F1 and 0.99 AUC for SA Random Forest survive.","section":"Sections 3.4 and 3.5"},{"comment":"There is no comparison against simple feature-selection baselines. The central claim is that QIFS with SA is valuable, but the only comparison is between 23 SA-selected features and all 69 features. A 30.3% training-time reduction is largely the mechanical consequence of using fewer features; whether the selected features are good ones cannot be assessed without baselines such as random 23-feature subsets, the top-23 features by absolute Spearman correlation, or classical selectors like mRMR or chi-square. Without such baselines, the paper does not show that the QUBO/SA procedure, rather than merely reducing dimensionality, drives the reported result.","section":"Tables 6 and 7"},{"comment":"Several QA and QA BQM rows are numerically identical despite the two methods selecting different feature sets. In Table 7, QA Random Forest and QA BQM Random Forest have the same AUC vector (0.79, 0.80, 0.78, 0.88, 0.93, 0.85); the same holds for XGBoost, LightGBM, SVM *10%, and Neural Network *10%. In Table 6, QA XGBoost and QA BQM XGBoost are identical in precision, recall, F1, accuracy, AUC, and training time, as are QA LightGBM and QA BQM LightGBM. If these entries are not typographical errors, the results contradict the different feature sets; if they are errors, the tables need correction. This issue must be resolved before the comparative claims about QA versus QA BQM can be evaluated.","section":"Tables 6 and 7"},{"comment":"The paper reports F1 0.91 for SA Random Forest versus 0.92 for full-feature Random Forest but gives no variance, confidence intervals, or per-fold results. Given the class distribution in Table 1 and the likely correlation of metrics across folds, a 0.01 F1 difference may be within noise. The authors should report fold-level statistics or at least standard deviations for the key comparisons, especially because the headline claim is that accuracy is 'maintained' rather than degraded.","section":"Section 4 and Table 6"}],"minor_comments":[{"comment":"The abstract calls both SA and QA 'quantum-inspired,' but quantum annealing on D-Wave hardware is actual quantum computing, not quantum-inspired. Please use consistent terminology.","section":"Abstract and Section 2.2"},{"comment":"The text says the features are categorized into five groups, but Table 2 lists four groups: Basic Statistics, Extra Statistics, Moments, and Transaction Patterns. Please correct the count or add the missing group.","section":"Section 3.3"},{"comment":"The GitHub URL 'https://github.com/Siemingfong/Quantom Annealing' contains a space and a typo ('Quantom'); it should be a valid URL, e.g., with '%20' or a corrected repository name.","section":"Section 3.4"},{"comment":"The sentence 'We training of seven common classification algorithms' is ungrammatical; please revise.","section":"Section 3.5"},{"comment":"The text says 'SA Random Forest exhibits a faster 30.3% training time 212 minutes and 23.2 seconds compared to Quantum Random Forest 305 minutes and 5.7 seconds.' The phrase 'Quantum Random Forest' should presumably be 'Random Forest,' and the sentence should be rewritten for clarity.","section":"Section 3.5 and Table 6"},{"comment":"The table title 'Evaluation of All Features and Full Features and SA QUBO Feature Selection' is redundant ('All Features' and 'Full Features' are the same) and does not mention QA/QA BQM rows. Please revise.","section":"Table 7"},{"comment":"Precision and recall values are reported as percentages with a percent sign in some places and as decimals in others (e.g., '0.91%' versus '0.91'); please standardize the notation.","section":"Section 4 and Figures 6-9"},{"comment":"The future-work section states that data imbalance is a limitation and proposes SMOTE, but Table 1 shows that the Mixer class is the largest class (289,006 addresses). If the focus is mixer detection, the class is not a minority; please clarify which classes are imbalanced and how this affects the mixer-specific results.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The duplicated entries between QA and QA BQM in Tables 6 and 7 are a data-integrity concern that should be verified before the paper is reconsidered. The selection-leakage issue is fixable by re-running the experiments with nested cross-validation, but it is fundamental to the headline claim; the paper should not be accepted in its current form. The authors should also be asked to make the evaluation code and exact fold splits available to support the corrected results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is an applied feature-selection paper, not a quantum computing paper. The claimed 30.3% training-time reduction is what you expect from going from 69 to 23 features; the real question is whether the selected features actually preserve F1 on held-out data, and the paper never answers that because the feature selection happens on the full dataset before the 10-fold CV. That is the thing to remember.\n\nWhat is genuinely here: a large labeled Bitcoin address dataset (694k addresses, ~3.4M transactions), a set of 69 transaction-history summarization features including some new transaction-pattern features, and an honest comparison of seven classifiers with full vs selected features. Code is on GitHub. And to their credit, the authors report that quantum annealing does worse than classical simulated annealing; the only competitive result is SA, which is a classical heuristic. So 'quantum-inspired' is doing little work in the headline.\n\nThe soft spots are real. Section 3.4 computes Spearman correlations between every feature and the mixer label on the full dataset before the QISF selection, and Section 3.5 applies 10-fold CV only to classifier training. The natural reading is that the selected feature subset is fixed using the test-fold labels. That makes the 0.91 F1 and the SA Random Forest result in Table 6 optimistically biased. No nested CV, no random-subset baseline, no simple filter baseline. Without those, you cannot separate the value of the QUBO selection from the mechanical effect of using fewer features. The QUBO objective maximizes correlation with the label by construction, so the 'key features' are important by definition; the paper never shows the selection method beats just taking the top-K correlated features. There is also no ablation separating transaction-pattern features from the older features, which makes it hard to know what the actual contribution is. None of these are fatal to the underlying idea; they are fixable in revision.\n\nWho is this for? People working on blockchain address classification or on feature selection with QUBO. It is a decent empirical study with a methodological hole. I would send it to reviewers — the data work is real and the flaw is the kind a careful referee can catch and the authors can fix. I would not cite the headline result until the leakage is addressed.\n\nRecommendation: send to peer review as a major-revision candidate, not a desk reject.","headline":"The 30% training-time saving is mechanical, but the 'maintaining 91% F1' claim is compromised by feature selection before cross-validation; worth a serious revision, not a desk reject.","tokens_in":13540,"tokens_out":3273,"would_cite":false,"duration_ms":29913,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"23 quantum-selected features cut Bitcoin mixer training time 30%","keywords":["Bitcoin","mixer addresses","feature selection","QUBO","simulated annealing","quantum annealing","random forest","blockchain analytics"],"falsifier":"Repeat the experiment with the feature-selection step redone inside each of the ten training folds, then compare the mixer F1 score to 0.91. If the nested F1 falls well below 0.91, the reported gain partly comes from seeing test labels during selection.","tokens_in":12574,"feed_emoji":"🪙","tokens_out":8517,"duration_ms":69458,"temperature":0.7,"pith_summary":"The paper tries to establish that a quantum-inspired feature-selection step can shrink a Bitcoin transaction dataset to the features that matter for identifying mixer addresses. The method casts feature selection as a binary optimization problem that rewards features correlated with the mixer label and penalizes redundant features, and it solves that problem with simulated annealing and quantum annealing. On 694,676 labeled addresses, simulated annealing picked 23 of 69 features; the resulting random forest trained about 30% faster and kept a 91% F1-score for mixers, versus 92% with all features. If correct, the practical payoff is cheaper, faster screening of high-risk Bitcoin addresses.","feed_headline":"23 quantum-selected features cut Bitcoin mixer training time 30%","feed_subtitle":"Simulated annealing trims 69 transaction features to 23; mixer F1 stays at 0.91 with random forest.","key_machinery":"The central object is a QUBO cost function with binary variables $x_j$ marking whether feature $j$ is selected: $$f(x) = -\\$\\alpha$ \\sum_{j=1}^n x_j |\\rho_{oj}| + (1-\\$\\alpha$)\\sum_{j=1}^n\\sum_{k\\neq j} x_j x_k |\\rho_{jk}|,$$ where $|\\rho_{oj}|$ is the Spearman rank correlation between feature $j$ and the binarized mixer label and $|\\rho_{jk}|$ is the Spearman correlation between features $j$ and $k$. The first term rewards predictive influence and the second penalizes redundancy, with $\\alpha$ balancing the two. The paper minimizes this objective with simulated annealing, quantum annealing, and a hybrid BQM solver, then uses the selected variables as the feature set for supervised classifiers, chiefly random forest.","core_discovery":"The paper's central claim is that a quadratic unconstrained binary optimization (QUBO) feature selector, solved with simulated annealing, can identify the transaction-history features most predictive of Bitcoin mixer use and thereby make a random forest classifier substantially cheaper to train without losing meaningful accuracy. On a dataset of 694,676 labeled addresses with 69 summarized features each, the simulated-annealing selector chose 23 features; the resulting SA Random Forest reached a mixer F1-score of 0.91 and AUC of 0.99 while training in 212 minutes, compared with F1 0.92, AUC 0.99, and 305 minutes for the random forest using all features. Quantum annealing and a hybrid binary-quadratic-model solver selected only 9 and 7 features and produced weaker classifiers (F1 0.60 and 0.70), so the reported practical win is specifically the simulated-annealing route. The authors interpret this as evidence that quantum-inspired combinatorial feature selection can accelerate blockchain analytics on high-risk address detection.","pith_inferences":["A test the paper does not run: replacing the QUBO solver with a simple top-$k$ Spearman ranking of features would reveal how much of the 30% training-time saving comes from the redundancy term rather than from dimensionality reduction alone.","Because the selection appears to use the full dataset before cross-validation, the 0.91 F1 is likely to drop under a nested selection protocol; the size of the drop would quantify the leakage.","The same QUBO-plus-simulated-annealing recipe could be applied to other rare-address classes such as gambling or faucets, where the paper reports weak F1, to see whether feature selection helps or hurts minority classes.","For larger feature sets, the 0.0055-second SA timing may not scale linearly; the practical speedup claim should be re-measured as the number of candidate features grows."],"forward_implications":["Analysts can train a random forest on 23 rather than 69 transaction-history features and still detect mixer addresses with F1 around 0.91, recall 0.97, and AUC 0.99.","Using all features remains slightly more accurate (F1 0.92), so the practical trade-off is about 1 point of F1 for a 30% reduction in training time.","Simulated annealing is the effective optimizer in this setup: quantum annealing selected fewer features but classifiers trained on them reached only F1 0.60 to 0.75.","The selected features center on transaction counts, balances, lifetime, and higher-order moments, with lifetime emerging in the SA-selected model.","The authors propose the method as transferable to other high-dimensional data domains, including cybersecurity, healthcare, and predictive maintenance."],"supporting_citations":[{"why":"Supplies the QUBO feature-selection objective balancing feature influence and dependence that the paper adapts.","marker":"[14]"},{"why":"The simulated annealing procedure used to optimize the QUBO and select 23 features.","marker":"[16]"},{"why":"The random forest algorithm whose training-time reduction is the paper's headline result.","marker":"[11]"},{"why":"Source of the basic transaction-history summary features on which the 69-feature set builds.","marker":"[25]"},{"why":"Source of the extended statistics and moment features, plus the Bitcoin address classification setup extended here.","marker":"[26]"},{"why":"Supplies the Python simulated-annealing sampler that produced the 23-feature selection.","marker":"[17]"},{"why":"Provides the random forest and cross-validation implementation used to measure F1 and training time.","marker":"[23]"}],"fun_headline_variants":["Simulated annealing picks 23 features, cuts training time 30%","Quantum-inspired feature selection cuts training time for Bitcoin mixers","23 features, 30% faster training: SA beats QUBO for Bitcoin","How simulated annealing trims Bitcoin features without losing F1","Feature selection via SA cuts Bitcoin classifier training by 30%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline's advertised F1 depends on the feature selector never seeing the evaluation labels: the paper computes Spearman correlations and solves the QUBO before the 10-fold classifier split, so the honest comparison requires the selection to be repeated inside each fold.","fun_headline_variants_meta":{"raw":{"variants":["Simulated annealing picks 23 features, cuts training time 30%","Quantum-inspired feature selection cuts training time for Bitcoin mixers","23 features, 30% faster training: SA beats QUBO for Bitcoin","How simulated annealing trims Bitcoin features without losing F1","Feature selection via SA cuts Bitcoin classifier training by 30%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000865,"raw_usage":{"total_tokens":3722,"prompt_tokens":891,"completion_tokens":2831,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":2741}},"tokens_in":507,"tokens_out":2831,"duration_ms":16928,"temperature":1.0,"reasoning_tokens":2741,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:18:57.351961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the experiment with the feature-selection step redone inside each of the ten training folds, then compare the mixer F1 score to 0.91. If the nested F1 falls well below 0.91, the reported gain partly comes from seeing test labels during selection.","supporting_citations":[{"cited_title":"https://1qbit.com","cited_arxiv_id":null,"evidence_quote":"Supplies the QUBO feature-selection objective balancing feature influence and dependence that the paper adapts."},{"cited_title":"Science 220(4598), 671–680 (1983)","cited_arxiv_id":null,"evidence_quote":"The simulated annealing procedure used to optimize the QUBO and select 23 features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the basic transaction-history summary features on which the 69-feature set builds."},{"cited_title":"In: 2019 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), pp","cited_arxiv_id":null,"evidence_quote":"Source of the extended statistics and moment features, plus the Bitcoin address classification setup extended here."},{"cited_title":"https://pypi.org/project/dwave-neal/","cited_arxiv_id":null,"evidence_quote":"Supplies the Python simulated-annealing sampler that produced the 23-feature selection."}],"review_version":1}