{"id":"89e1c704-e1e9-4f5a-b220-762ecf281280","arxiv_id":"2602.17842","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Tree ensembles reach Macro-F1 0.978 versus GraphSAGE 0.805 on a new 16,433-wallet stablecoin AML dataset, but label-derived features and a non-temporal split inflate the apparent accuracy.","lead":"StableAML builds a new labeled dataset of stablecoin wallets on Ethereum and reports that tree ensembles beat graph neural networks at detecting laundering and sanctions-linked addresses. The reported near-perfect scores are inflated by features that encode the labels themselves and by a split that lets post-label behavior leak into evaluation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Label-derived adjacency features likely leak the target: 'receivedFromFlagged', 'sentToFlagged', '2ndWithFlagged', '3rdWithFlagged', and 'clusterScore' are built from the same flagged-address lists that define the labels, so the near-perfect F1 may be label propagation, not behavioral detection.","rationale":"The reader's weakest_assumption identifies exactly the same mechanism: label-derived adjacency features and full-window aggregation. The paper itself (Appendix A, Section 3.2.2, Section 3.3) confirms that flagged-list features are engineered and that features aggregate over the full observation window. No ablation or temporal split is reported. This is the single most load-bearing concern because it directly undermines both components of the central claim: the performance gap over GNNs and the typology differentiation. The reader's CONDITIONAL verdict is appropriate: the dataset could become useful after removing leaked features and enforcing temporal splits, but the current evidence does not establish the claim. My read does not move the verdict.","tokens_in":17064,"tokens_out":2017,"duration_ms":19306,"concrete_test":"Retrain CatBoost and GraphSAGE after (a) removing every feature containing 'Flagged', 'Cluster', 'isPartOfCluster', and 'clusterScore' from the 68-feature set, and (b) replacing the random split with a temporal split: train on wallets labeled before Jan 1, 2023, test on wallets labeled after that date, with all features computed only from transactions before each wallet's label date. If CatBoost's Macro-F1 drops from 0.9775 to near the GNN's 0.8048 (or below), the central claim is an artifact of label leakage. Also re-run the class-specific feature importance and typology analysis on the reduced feature set; if the Cybercrime vs. Blocklisted separation disappears, the typology claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — tree ensembles (Macro-F1 0.9775) beat GNNs and separate laundering typologies — depends on the engineered features being legitimate behavioral signals. But Appendix A (Table 8) lists features such as receivedFromFlagged, sentToFlagged, 2ndWithFlagged, 3rdWithFlagged, clusterScore ('Number of flagged direct connections'), and isPartOfClusterFrom/To. These are constructed directly from the same flagged-address lists (Etherscan reports, SlowMist/PeckShield, OFAC SDN) that generated the labels in Section 3.1. A wallet is labeled Cybercrime or Blocklisted precisely because it appears on those lists, so any transaction with a 'flagged' address is a near-deterministic indicator of the label. This is label propagation through feature engineering, not independent behavioral detection. Additionally, features are aggregated over the full 2017–2025 window (Section 3.3), including activity after a wallet's label was assigned (e.g., post-freeze reactions), enabling temporal leakage: the model sees future behavior that would not be available when predicting a label at a given time. The random wallet-level split does not prevent this. If these features are removed and a temporal split is enforced, the reported CatBoost advantage and typology decomposition may collapse — the paper does not report any such ablation. Without that, the headline result is consistent with a circularity artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces StableAML, a labeled Ethereum stablecoin dataset of 16,433 wallets (Normal, Cybercrime, Blocklisted) built from USDT/USDC transfer events, and compares logistic regression, four tree ensembles, a DNN, and GraphSAGE on 68 engineered features. The headline result is that tree ensembles, especially CatBoost, achieve Macro-F1 ≈ 0.9775, far outperforming GraphSAGE (≈ 0.8048), and that feature-importance analysis reveals distinct behavioral signatures for cybercrime versus blocklisted wallets. The authors argue this demonstrates the superiority of domain-informed feature engineering over graph neural networks for stablecoin AML.","tokens_in":17442,"tokens_out":2259,"duration_ms":26764,"significance":"If the results are valid, the paper would provide a useful benchmark dataset and an important practical message for AML compliance: tabular behavioral features can outperform GNNs on fragmented stablecoin transaction graphs. The dataset is made publicly available, and hyperparameter details are included, which are strengths for reproducibility. However, the central comparison and the typology interpretation rest on a set of features that are constructed from the same flagged-address lists used to define the target labels. This creates a direct label-leakage pathway that can explain the near-perfect F1 scores without any genuine behavioral detection. The paper currently does not provide the ablation needed to show that the reported results survive removal of these leaked features or a temporally honest split. The significance of the contribution is therefore conditional on this missing evidence.","major_comments":[{"comment":"The features receivedFromFlagged, sentToFlagged, 2ndWithFlagged, 3rdWithFlagged, clusterScore ('Number of flagged direct connections'), and isPartOfClusterFrom/To are constructed directly from the same flagged-address lists (Etherscan reports, SlowMist/PeckShield, OFAC) that define the Cybercrime and Blocklisted labels in Section 3.1. A wallet that has any transaction with a flagged address is almost deterministically labeled as illicit. Including these features as inputs means the model is performing label propagation, not learning behavioral detection. This is the most load-bearing issue: the headline Macro-F1 values (Table 5) and the typology separation (Section 5.4) are plausible in-sample artifacts. The authors must report an ablation that removes all features whose construction references flagged-address lists, and a version where second/third-degree flagged features are excluded o","section":"Section 3.1, Appendix A (Table 8)"},{"comment":"As stated in Section 3.3, both the graph structure and node features are 'cumulative aggregates over the full observation window' (2017-2025). The labels are assigned at specific points in time (e.g., when a wallet is frozen or sanctioned), so the features include activity that occurs after the label is assigned. The random wallet-level split does not address this temporal leakage: a model evaluated on the test set has access to post-label behavior that would not be available when the label is predicted in practice. A temporal split — where features for each wallet are computed using only transactions before the label date, and training/testing are separated in time — is needed to support the paper's practical claims about detection for compliance.","section":"Sections 3.3 and 4"},{"comment":"The claimed advantage of tree ensembles over GraphSAGE (Macro-F1 0.9775 vs 0.8048) cannot be interpreted without the leak-free ablation. Since the GNN also uses the same 68 leaky node features, it is plausible that tree ensembles exploit the leaked flagged-connection features more directly, while GNN message passing partially dilutes them. The paper does not report results with the leaky features removed, nor does it compare models on a temporally split dataset. Without these experiments, the central claim that 'domain-informed tree ensembles significantly outperform graph neural networks' is not established.","section":"Section 5.2 and Table 5"}],"minor_comments":[{"comment":"The comparison with Elliptic dataset results is not apples-to-apples: Elliptic uses a different asset class, different label definitions, and a different task (transaction classification vs. wallet classification). This should be caveated clearly, or the comparison should be removed.","section":"Table 7, Section 5.5"},{"comment":"The feature name 'circleDetected' is described as 'Reciprocal transfers within 24 hours' but the name is not self-explanatory. A short explanation of why this indicates circular activity would improve clarity.","section":"Appendix A, Table 8"},{"comment":"Panels are described as 'Per-class OvR performance curves' but it is not immediately clear from the caption whether the curves are ROC, precision-recall, or something else. Please label axes in the figure.","section":"Figure 4"},{"comment":"The softmax equation uses z_k(x) but the definition of z_k is not explicit; clarify that z is the final linear projection of the hidden representation.","section":"Section 4.3, Eq. (1)"}],"recommendation":"major_revision","confidential_remarks":"This is a case where the central claimed result is currently supported by a circular feature set. The paper needs a clear ablation study, and if the ablation shows the results collapse, the contribution will need to be reframed substantially. The authors should be asked to release the exact labels and feature construction code to allow verification. No concerns about misconduct; the issue is methodological transparency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plain view: this paper is a useful dataset contribution wrapped around a result that doesn't hold as stated. The central claim — tree ensembles (CatBoost Macro-F1 0.9775) outperform GNNs (0.8048) and separate laundering typologies — is contaminated by label leakage. Several of the most predictive features (receivedFromFlagged, sentToFlagged, 2ndWithFlagged, 3rdWithFlagged, clusterScore) are built from the same flagged-address lists that generated the labels. CatBoost is essentially reading the answer key. The GNN underperformance is not evidence about graph vs. tabular methods once the model can see 'did this wallet interact with a flagged address?' and the label is 'this wallet is flagged.' That's not a subtle confound; it's a direct one.\n\nWhat is genuinely new: the stablecoin-only labeled dataset (16,433 wallets, three classes, 68 features) appears to be the first of its kind, and it's a reasonable starting point for the AML community. The feature engineering is thoughtful — the multi-hop and temporal features are sensible if they were computed from activity only. The typology discussion (cybercrime as high-velocity multi-hop, blocklisted as constrained and static) is plausible but post-hoc, built on the same leaky features, so I'd treat it as speculation until confirmed on a clean split.\n\nThe other soft spots are real but secondary: features are aggregated over the full 2017–2025 window including post-label activity, and the random wallet-level split doesn't fix temporal leakage. No error bars, no ablation without flag-derived features, and no simple rule-based baseline like 'count of flagged neighbors.' Given the binary F1 of 0.999, a negative control was needed. The paper doesn't provide one.\n\nAre these flaws load-bearing? Yes. The published numbers and the tree-vs-GNN story would likely collapse without the leaked features. But the dataset and benchmark are worth saving. If the authors drop flag-derived features, enforce a temporal split, add a rule-based baseline, and re-run, the paper could be a solid reference for stablecoin AML and compliance tooling.\n\nFor you: bring it to reading group as a cautionary example of leakage in ML for blockchain. It's a good teaching case. I would not cite the current results. For peer review: yes, a serious editor should send it out — but the reviewer's first request should be the ablation. The paper is not beyond repair, but it needs major revision and a clear demonstration that the findings survive decontamination.","headline":"The dataset is a real contribution, but the headline result — tree ensembles beating GNNs on stablecoin AML — is likely an artifact of label leakage; don't trust the F1 numbers until the authors remove flag-derived features and re-run.","tokens_in":17917,"tokens_out":2002,"would_cite":false,"duration_ms":22029,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Domain-informed tree ensembles, not graph neural networks, best detect money-laundering wallets in Ethereum stablecoin flows, and the model also separates cybercrime syndicates from sanctioned entities.","keywords":["anti-money laundering","stablecoins","Ethereum","behavioral wallet detection","tree ensembles","graph neural networks","feature engineering","blockchain analytics"],"falsifier":"Take the best CatBoost configuration, drop all features derived from flagged-address lists (receivedFromFlagged, sentToFlagged, 2ndWithFlagged, clusterScore, etc.), and replace the random 80/20 split with a temporal split (train on wallets whose first transfer precedes a cutoff, test on wallets first active after it). If Macro-F1 falls well below the reported 0.98, or if the cybercrime/blocklisted separation blurs, the headline result is largely an artifact of label leakage and full-window feature computation rather than a portable detection signal.","tokens_in":16954,"feed_emoji":"🪙","tokens_out":5880,"duration_ms":54226,"temperature":0.7,"pith_summary":"This paper claims that, for detecting money laundering in USDT/USDC transfers on Ethereum, 68 hand-engineered behavioral features fed into tree ensembles (best: CatBoost) outperform graph neural networks, which stumble on the fragmented stablecoin transaction graph. On a new dataset of 16,433 wallets labeled Normal, Cybercrime, or Blocklisted, CatBoost reaches a Macro-F1 of 0.9775, versus 0.8048 for GraphSAGE. The model further claims to distinguish two laundering typologies: the complex, high-velocity layering of cybercrime syndicates versus the constrained, static footprints of sanctioned or frozen wallets. These results are positioned as an empirical baseline for AML screening under emerging stablecoin regulations, where the persistent transparency of centralized stablecoins serves as a choke point.","feed_headline":"Tree ensembles beat graph nets at stablecoin AML detection","feed_subtitle":"CatBoost hits 0.98 Macro-F1 on a new Ethereum stablecoin dataset and separates cybercrime from sanctioned wallets.","key_machinery":"The central object is a 68-feature engineered feature set, grouped into interaction, derived network, transfer, and temporal/direct categories. The load-bearing elements are the second-degree network features (e.g., 2ndWithFlagged, 2ndWithOver10k) and interaction flags (sentToFlagged, receivedFromFlagged, sentToSwap), which are aggregated over the full 2017–2025 observation window. These are fed to gradient-boosted trees, whose piecewise-constant decision boundaries partition the discrete behavioral signals; CatBoost's ordered boosting and native categorical handling are the specific variant used. The graph baseline uses GraphSAGE with mean aggregation over a stablecoin-only weighted graph,","core_discovery":"On a purpose-built dataset of 16,433 Ethereum wallets labeled Normal, Cybercrime, or Blocklisted, restricted to USDT and USDC transfer events, gradient-boosted tree ensembles—especially CatBoost—achieve near-perfect binary separation and strong three-class Macro-F1, clearly outperforming a GraphSAGE graph neural network. The paper attributes the GNN's weakness to the fragmentation of the stablecoin-only transaction graph, which breaks the message-passing paths needed for neighborhood aggregation. Feature-importance analysis shows that second-degree exposure features (e.g., 2ndWithMultipleSameValue, 2ndWithOver10k) and direct contract/CEX interactions drive the predictions, and these map onto","pith_inferences":["Because several engineered features are computed from the same flagged-address lists that generated the labels, and because features use the full 2017–2025 window (including post-label activity), the reported Macro-F1 likely overestimates out-of-sample performance; a leakage-controlled evaluation would probably lower the numbers.","The GNN comparison may understate graph methods: the graph was restricted to USDT/USDC edges only, so cross-asset swaps that break the chain are excluded by design; a multi-asset graph could restore message-passing paths and change the ranking.","The claimed typology separation could be tested forward: after the 2025 cutoff, track wallets the model labels as cybercrime and see if their subsequent transaction velocity and hop counts exceed those of wallets labeled blocklisted.","If the result holds under leakage control, it suggests that the persistent transparency of centralized stablecoins is enough to screen illicit behavior even as privacy layers obscure native-asset graphs."],"forward_implications":["AML screening for stablecoins can rely on tabular behavioral features computed from token transfer events, without needing a connected transaction graph.","The three-class model gives compliance teams a way to separate active cybercrime laundering from sanctioned/frozen wallets, enabling different response actions (e.g., freeze vs. monitor).","The interpretability analysis maps top features to placement, layering, and integration stages, offering an audit trail for regulators under frameworks like MiCA and the GENIUS Act.","The dataset and methodology establish a stablecoin-specific baseline for future AML research and for deterministic, auditable wallet screening."],"fun_headline_variants":["CatBoost beats GNNs on stablecoin AML detection","Tree ensembles outperform graph nets for stablecoin AML","Stablecoin AML: tree models beat GNNs on Ethereum","Behavioral stablecoin AML: tree ensembles surpass graph nets","Tree ensembles expose cybercrime vs sanctioned in stablecoin AML"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The model assumes each wallet's entire 2017–2025 transaction history is available at scoring time, and that features such as receivedFromFlagged and clusterScore—which are built from the same flagged-address lists that define the labels—do not leak the target during training.","fun_headline_variants_meta":{"raw":{"variants":["CatBoost beats GNNs on stablecoin AML detection","Tree ensembles outperform graph nets for stablecoin AML","Stablecoin AML: tree models beat GNNs on Ethereum","Behavioral stablecoin AML: tree ensembles surpass graph nets","Tree ensembles expose cybercrime vs sanctioned in stablecoin AML"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001048,"raw_usage":{"total_tokens":4242,"prompt_tokens":746,"completion_tokens":3496,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":3415}},"tokens_in":490,"tokens_out":3496,"duration_ms":25570,"temperature":1.0,"reasoning_tokens":3415,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:05:55.881286+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the best CatBoost configuration, drop all features derived from flagged-address lists (receivedFromFlagged, sentToFlagged, 2ndWithFlagged, clusterScore, etc.), and replace the random 80/20 split with a temporal split (train on wallets whose first transfer precedes a cutoff, test on wallets first active after it). If Macro-F1 falls well below the reported 0.98, or if the cybercrime/blocklisted separation blurs, the headline result is largely an artifact of label leakage and full-window feature computation rather than a portable detection signal.","supporting_citations":[],"review_version":1}