{"id":"7394edb9-426e-4c72-a778-855ab477ade9","arxiv_id":"2501.15553","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Real-CATS provides 103,203 criminal and 106,196 benign cryptocurrency addresses on Bitcoin and Ethereum, with transaction profiles and a temporal test set for cybercrime detection research.","lead":"Real-CATS is a new labeled dataset of over 200,000 cryptocurrency addresses, split into criminal and benign classes, built from victim reports and exchange records on Bitcoin and Ethereum. It gives non-blockchain researchers a ready-made benchmark for training and testing cybercrime detection models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benign labels are contaminated by exchange-owned change addresses because the selection method in §3.1 labels every non-sender output of a hot-wallet transaction as a customer, with no change-address detection.","rationale":"The central claim is that Real-CATS is a practical training ground with reliable benign labels. The most vulnerable step is the benign-address selection in Section 3.1. The reader identified the untested KYC assumption as the weakest point; I agree that label reliability is the key issue, but the more specific and testable flaw is that the selection rule cannot distinguish a customer output from a change output in Bitcoin. Because Bitcoin transactions often include a change output to a sender-controlled address, and because the paper explicitly avoids incoming transactions due to deposit-address complexity but does not apply the same caution to outgoing change outputs, the 'benign' set is likely contaminated with exchange-owned addresses. This is a mechanical property of the UTXO model, independent of whether exchanges enforce KYC. If the contamination is substantial, the paper's claims of Comprehensiveness, Classifiability, and Real-world Transferability are not supported, and the high accuracies in Table 5 may be an artifact of distinguishing criminals from exchange infrastructure. The confusion matrices in Table 6 also do not reconcile with the stated Sup-CATS totals (11,058 benign, 3,147 criminal), adding further uncertainty to the transferability claim. Nonetheless, the criminal labels from Chainabuse and the transaction records are potentially valuable, so the dataset is salvageable if the authors quantify and remove exchange-owned addresses and correct the evaluation tables. The reader's CONDITIONAL verdict remains appropriate; no change to the verdict is needed, but the condition should explicitly require a change-address analysis and re-evaluation.","tokens_in":16280,"tokens_out":8474,"duration_ms":73157,"concrete_test":"Download the released BB address file and the seed exchange hot-wallet addresses used in Section 3.1. Apply the common-input heuristic: if a BB address and a seed hot wallet appear as co-inputs in any Bitcoin transaction, they are likely controlled by the same entity. Also detect change addresses by scanning each BB address's first transaction for the pattern of one input from the seed wallet and two outputs, with one output being the BB address. Compute the fraction of BB addresses that are exchange-owned by either test. If the fraction is above 5%, the benign-label definition fails and the authors must remove or reclassify these addresses and rerun Table 5 and Table 6. An independent manual check of 100 random BB addresses on a block explorer should confirm the automated result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.1, benign addresses are collected by taking 'next-hop addresses that are different from the sender address' from outgoing transactions of exchange hot wallets. In Bitcoin, a standard outgoing payment from a hot wallet creates two outputs: the customer payment and the change output, which goes to an address controlled by the exchange. The paper does not apply any change-address heuristic (e.g., one-input/two-output with the extra output clustering with the sender), so exchange-owned change addresses are labeled benign. The subsequent deletion of addresses that also appear in the criminal list does not remove these addresses unless they were independently reported. Consequently, the 'benign' set is not composed of KYC-verified exchange customers as claimed; it is likely to contain a substantial number of exchange-internal addresses. This contaminates both the classifiability evidence in Table 5 and the real-world transferability evidence in Section 4.3, because a model may be learning to separate reported criminals from exchange infrastructure rather than from ordinary users. The paper itself reports that 76.56% of Bitcoin benign addresses have zero balance (Section 4.1), which is consistent with a large fraction of swept change addresses, though not conclusive. No quantification of this contamination is provided.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Real-CATS, a dual-chain labeled address dataset for cryptocurrency cybercrime detection, containing 103,203 criminal and 106,196 benign addresses on Bitcoin and Ethereum, together with statistical transaction profiles (32 features for Bitcoin, 52 for Ethereum), detailed transaction records, and Ethereum token-interaction files. It also introduces Sup-CATS, a one-day Ethereum test set intended to simulate real-world deployment. The authors claim four C3R properties, demonstrate classifiability with five machine-learning models, show customizability by extending a small subset into a transaction graph for Trans2Vec, and evaluate real-world transferability by training on Real-CATS and testing on Sup-CATS.","tokens_in":16452,"tokens_out":9057,"duration_ms":82223,"significance":"If validated, Real-CATS would be a genuinely useful community resource: it is substantially larger than Elliptic, it covers both Bitcoin and Ethereum, it provides both aggregate profiles and raw transaction records, and its Sup-CATS temporal split goes beyond the random splits commonly used in this literature. The authors release the data publicly, which supports reproducibility, and the Chainabuse-based labeling and manual category aggregation are described in enough detail to be audited. However, the current evidence for the dataset's central claims is weakened by label-construction issues in the benign set and the Sup-CATS test set, and by numerical inconsistencies in the reported evaluation tables.","major_comments":[{"comment":"The benign-address collection method labels every non-sender output of an outgoing exchange-hot-wallet transaction as a benign customer address. In Bitcoin, a standard outgoing payment normally creates two outputs: the recipient's payment and a change output controlled by the exchange itself. Because no change-address detection is applied, these exchange-controlled change addresses enter the benign set, and the subsequent deletion of addresses that also appear in the criminal list does not remove them. The 76.56% zero-balance fraction among Bitcoin benign addresses reported in Section 4.1 is consistent with a large population of swept change addresses, though not conclusive. Since the benign class is load-bearing for the classifiability results in Table 5 and the transfer results in Section 4.3, the authors should either apply and report change-address heuristics (for example, one-input/two-output clustering with the sender), quantify the fraction of suspected exchange-internal change addresses, or otherwise demonstrate that the benign set consists of KYC-verified exchange customers rather than exchange-owned infrastructure.","section":"§3.1"},{"comment":"The construction of Sup-CATS explicitly treats all unlabeled addresses sampled on August 12, 2024, as benign: 'We scan and sample the unlabeled addresses as benign with a 1% probability.' Unlabeled addresses on a blockchain are not necessarily benign; they can include criminal addresses not yet reported or attributed. This untested assumption contaminates the confusion matrices in Table 6 and therefore the claimed real-world transferability. In addition, the row and column labels of Table 6 are ambiguous, and the counts do not reconcile with the stated Sup-CATS totals: under the conventional reading, the Random Forest matrix implies 5,264 actual criminal and 8,938 actual benign addresses, while the LGBM matrix implies 2,989 actual criminal and 11,213 actual benign addresses, neither matching the reported 3,147 and 11,058. Please clarify the matrix orientation, report how many addresses were excluded by preprocessing and why, and provide sensitivity analysis for the 1% sampling rate.","section":"§4.3"},{"comment":"The Ethereum Random Forest row reports Precision=0.9085, Recall=0.8237, F1=0.8640, and Accuracy=0.7116. Under the standard definitions, these precision and recall values imply an accuracy of at least about 0.74 for any positive-class prevalence, so the reported accuracy of 0.7116 is numerically impossible. This suggests an error in the computation or reporting of at least one metric. Because Table 5 is the primary evidence for the Classifiability characteristic, these numbers must be corrected and regenerated before the central claim can be assessed.","section":"§4.1, Table 5"},{"comment":"The customizability experiment uses a single random subset of 2,500 addresses with a 1:1 criminal-to-benign ratio and reports Trans2Vec results without a random seed, repeated trials, or confidence intervals. As this experiment is used to argue that Real-CATS supports node-embedding methods, the authors should report the variance over multiple subsamples or explain why a single draw is sufficient, especially since the subsample and the extended graph are not otherwise characterized.","section":"§4.2"}],"minor_comments":[{"comment":"There are several typos and grammatical errors, including 'satifies' and 'fundemental' in the abstract, 'we presents Real-CATS' and 'comprises include' in Section 1, and 'Li f etime' in the captions of Figures 2-4.","section":"Abstract and Section 1"},{"comment":"The sentence 'We use the implication in sklearn' should read 'We use the implementation in sklearn'; also, the authors should state explicitly which features were used for training, since Table 1 includes transaction-hash fields that are non-numeric and would be inappropriate for the sklearn models.","section":"§4.1"},{"comment":"The sentence 'we delete all addresses labeled as benign and criminal simultaneously' is ambiguous: it should specify whether an address appearing in both lists is removed from one set, both sets, or excluded from the analysis entirely, and how many addresses were affected.","section":"§3.1"},{"comment":"The comparison with existing datasets in Figure 3 is based on a merged set of Elliptic and the illicit-address dataset, but the paper does not describe the merging process, the overlap removal, or the feature-extraction pipeline used for those addresses; this should be documented for reproducibility.","section":"§4.1"},{"comment":"The claim that 'the high false positive rate severely affects the detection performance' is not directly supported by the reported Random Forest confusion matrix, which shows about 790 false positives; the sentence should be tied to the specific metric and model being discussed.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The dataset could be a valuable community resource if the label-construction issues are addressed. The benign-set contamination from Bitcoin change addresses and the unverified 'unlabeled equals benign' assumption in Sup-CATS are both load-bearing for the paper's main claims, and the Table 5 inconsistency suggests that the experimental results need to be carefully rechecked before the paper is suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Real-CATS is a genuinely useful public resource: a multi-chain labeled address dataset with transaction profiles, raw records, and a temporal test set. The field has needed something like this. The criminal labels come from Chainabuse victim reports, the profiles include token interactions that other datasets skip, and the Sup-CATS transfer experiment is an honest attempt to show how hard single-day deployment is. That experiment reveals a real problem—LightGBM collapses to 9.9% precision—which is exactly the kind of information a benchmark paper should surface. Credit where it is due: the data is open, the collection pipeline is described concretely, and the C3R framing is a reasonable organizing device.\n\nThe soft spot is the benign set. In Section 3.1, benign addresses are taken as the next-hop outputs of exchange hot wallet transactions, filtered only to exclude the sender address. No change-address heuristic is applied. In Bitcoin, a standard outgoing payment creates a change output that belongs to the exchange itself, so a large fraction of the \"exchange customers\" is likely exchange-owned change addresses. The paper's own observation that 76.56% of Bitcoin benign addresses have zero balance is consistent with swept change addresses, not with typical KYC-verified customers. This contaminates both the classifiability results in Table 5 and the transferability claims in Section 4.3, because a classifier may be learning to separate reported criminals from exchange infrastructure rather than from ordinary users. The authors do not quantify the contamination, and deleting addresses that also appear in the criminal list does not help.\n\nThere is a second, smaller issue: Sup-CATS treats all unlabeled addresses sampled from a day's blocks as benign, which is a known shortcut but is not acknowledged as an assumption. And the numbers in Table 5 are internally inconsistent—Random Forest on Ethereum is reported with precision 0.9085, recall 0.8237, but accuracy 0.7116, which cannot coexist under any plausible class prior. That looks like a transcription error, and it makes the reader wonder about the other reported values.\n\nThe central flaw is the benign label construction, and it is load-bearing. The dataset is still worth having, but not as ground truth for benign behavior until the authors apply change-address detection or otherwise justify the benign set. This is a serious referee situation, not a desk reject: the resource is valuable enough to merit repair, and the authors have shown they can do real evaluation work. I would not cite the benign labels as clean until the contamination is addressed.","headline":"A valuable multi-chain labeled dataset, but the benign labels are contaminated by exchange change addresses and the evaluation tables have inconsistencies; worth repairing and then publishing.","tokens_in":17008,"tokens_out":2449,"would_cite":false,"duration_ms":25297,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Real-CATS puts 200K labeled crypto addresses in researchers' hands.","keywords":["Real-CATS","cryptocurrency cybercrime detection","labeled blockchain addresses","transaction profiles","Bitcoin and Ethereum","anti-money laundering benchmark","real-world transferability","machine learning dataset"],"falsifier":"Take the released benign address list and intersect it with new victim-submitted abuse reports appearing after the collection cutoff and with known laundering-cluster heuristics; if a non-negligible fraction, say 1-2% or more, of supposed benign addresses are flagged, the benign-selection premise fails and the classifiability and transferability numbers become suspect.","tokens_in":16031,"feed_emoji":"🪙","tokens_out":7538,"duration_ms":63005,"temperature":0.7,"pith_summary":"Cryptocurrency cybercrime detection has been blocked by the absence of a large, publicly available, real-world labeled address dataset; most studies use private or partially disclosed data, making results hard to compare. This paper presents Real-CATS, a dataset of 103,203 criminal addresses taken from victim-submitted abuse reports and 106,196 benign addresses drawn from exchange hot-wallet outputs on Bitcoin and Ethereum. Each address comes with a statistical transaction profile (32 features for Bitcoin, 52 for Ethereum) and the underlying transaction records, so researchers can either run classifiers directly or build custom graphs. The paper argues the set satisfies four properties—Comprehensiveness, Classifiability, Customizability, and Real-world Transferability—and demonstrates classification with several standard algorithms, plus a supplementary single-day Ethereum test set that simulates real deployment. If the dataset holds up, it gives statistics and AI researchers without blockchain expertise a shared benchmark to develop and compare detection methods.","feed_headline":"Real-CATS brings 200K labeled crypto addresses to detection research","feed_subtitle":"The Real-CATS benchmark gives researchers 200K labeled addresses and a real-world transfer test.","key_machinery":"The central object is the dataset construction pipeline: criminal addresses and labels are aggregated from victim-submitted abuse reports, benign addresses are selected from the outputs of exchange hot wallets under the assumption that exchanges enforce strict Know-Your-Customer checks, and every address is turned into a transaction profile (32 features for Bitcoin, 52 for Ethereum, including token interactions on Ethereum) plus released transaction records. The load-bearing mechanism is the C3R claim—Comprehensiveness, Classifiability, Customizability, and Real-world Transferability—because those four properties are what make the dataset a usable benchmark rather than a mere list of addresses. Sup-CATS, a supplementary single-day Ethereum sample with newly labeled criminal addresses, is the device that converts the transferability claim into a concrete test.","core_discovery":"The paper's central claim is that Real-CATS is the first practical training ground for cryptocurrency cybercrime detection research: a dual-chain dataset whose criminal labels come from real-world victim reports rather than heuristic inference, whose benign labels come from customers of centralized exchanges under the Know-Your-Customer assumption, and whose released data include both hand-crafted transaction profiles and raw transaction records. It further claims that this design satisfies the C3R characteristics, and that the supplementary Sup-CATS set provides a temporal deployment test in which a model trained on older Real-CATS data is evaluated on all addresses active on a single day. On that test, a random-forest classifier trained on the profiles detects roughly three-quarters of the day's newly labeled criminal addresses, while a gradient-boosted model that looked strong on random splits degrades sharply, which the paper reads as evidence that Real-CATS exposes real-world transferability gaps that simpler evaluation schemes miss.","pith_inferences":["A natural next test is to measure label noise in the benign set by checking whether released benign addresses later appear in abuse reports or in known laundering clusters; the paper does not quantify this.","The Sup-CATS scenario covers only Ethereum, so the paper's real-world transferability claim is strongest for Ethereum; extending the same single-day sampling to Bitcoin would close that gap.","Because raw transaction records are released, the dataset invites a broader comparison: profile-based detectors versus graph neural networks and random-walk embeddings on exactly the same address set, which the paper only begins with a 2,500-address subset.","The C3R recipe could generalize to other chains, such as Polygon or Solana, if abuse-report coverage and exchange hot-wallet identification exist there; this is the authors' stated hope, not yet demonstrated."],"forward_implications":["Researchers can for the first time compare feature-engineering and graph-embedding detectors on a shared 200K-address, two-chain benchmark with consistent labels and a published benign set.","The temporal Sup-CATS split gives a concrete way to test whether a model that performs well on random splits will survive deployment on a future day's active addresses.","Because transaction records are included, address profiles can be extended or replaced by user-defined features, attributed graphs, or token-interaction representations without recollecting blockchain data.","Any detector trained on Real-CATS inherits the dataset's real-world noise, so good results are more likely to transfer to live settings than results on heuristically labeled or 'cleaner' datasets.","The dataset also supports measurement studies beyond detection, such as estimating cybercrime revenue, tracking money flow, and studying how criminals move funds."],"supporting_citations":[{"why":"Establishes the existing Bitcoin anti-money-laundering benchmark whose heuristic labeling and transferability limits Real-CATS is positioned against.","marker":"[29]"},{"why":"Supplies the network-embedding script that the customizability demonstration uses to turn Real-CATS transaction records into a graph.","marker":"[23]"},{"why":"Represents the Ethereum phishing-detection baseline that selects benign addresses randomly, the practice Real-CATS seeks to replace with exchange-derived benign addresses.","marker":"[12]"},{"why":"Defines the feature-engineering approach for detecting Bitcoin scams that motivates Real-CATS's transaction-profile features.","marker":"[20]"},{"why":"Provides the observed zero-balance pattern for reported scam addresses that Real-CATS uses to validate its criminal-address behavior.","marker":"[41]"},{"why":"Supplies the public illicit-address set merged with the Bitcoin benchmark for the pattern comparison that shows existing data are too clean.","marker":"[45]"},{"why":"Contributes the exchange-address detection features and back-and-forth exploration approach from which several Real-CATS profile features are drawn.","marker":"[30]"},{"why":"Establishes the short-lifetime pattern of phishing addresses used to validate Real-CATS criminal and benign lifespan distributions.","marker":"[9]"}],"fun_headline_variants":["Real-CATS: 200K labeled crypto addresses from real victim reports","Real-CATS benchmark: 200K real-world crypto addresses for detection","New dataset Real-CATS offers 200K labeled crypto addresses","Real-CATS: 200K crypto addresses labeled from real victim reports","Real-CATS exposes real-world gaps in crypto cybercrime detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole benign class rests on the assumption that centralized exchanges enforce strict Know-Your-Customer checks, so addresses that receive funds from exchange hot wallets are clean; the paper never quantifies how much contamination remains after removing only explicitly criminal-labeled addresses.","fun_headline_variants_meta":{"raw":{"variants":["Real-CATS: 200K labeled crypto addresses from real victim reports","Real-CATS benchmark: 200K real-world crypto addresses for detection","New dataset Real-CATS offers 200K labeled crypto addresses","Real-CATS: 200K crypto addresses labeled from real victim reports","Real-CATS exposes real-world gaps in crypto cybercrime detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00051,"raw_usage":{"total_tokens":2505,"prompt_tokens":990,"completion_tokens":1515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":606,"completion_tokens_details":{"reasoning_tokens":1423}},"tokens_in":606,"tokens_out":1515,"duration_ms":9772,"temperature":1.0,"reasoning_tokens":1423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:11:08.510542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the released benign address list and intersect it with new victim-submitted abuse reports appearing after the collection cutoff and with known laundering-cluster heuristics; if a non-negligible fraction, say 1-2% or more, of supposed benign addresses are flagged, the benign-selection premise fails and the classifiability and transferability numbers become suspect.","supporting_citations":[{"cited_title":"Who are the phishers? phishing scam detection on ethereum via network embedding","cited_arxiv_id":null,"evidence_quote":"Supplies the network-embedding script that the customizability demonstration uses to turn Real-CATS transaction records into a graph."},{"cited_title":"Phishing scam detection on ethereum: towards financial security for blockchain ecosystem","cited_arxiv_id":null,"evidence_quote":"Represents the Ethereum phishing-detection baseline that selects benign addresses randomly, the practice Real-CATS seeks to replace with exchange-derived benign addresses."},{"cited_title":"Data mining for detecting bitcoin ponzi schemes","cited_arxiv_id":null,"evidence_quote":"Defines the feature-engineering approach for detecting Bitcoin scams that motivates Real-CATS's transaction-profile features."},{"cited_title":"On the dark side of the coin: Characterizing bitcoin use for illicit activities","cited_arxiv_id":null,"evidence_quote":"Provides the observed zero-balance pattern for reported scam addresses that Real-CATS uses to validate its criminal-address behavior."},{"cited_title":"Identifying illicit addresses in bitcoin network","cited_arxiv_id":null,"evidence_quote":"Supplies the public illicit-address set merged with the Bitcoin benchmark for the pattern comparison that shows existing data are too clean."},{"cited_title":"Watch your back: Identifying cybercrime financial relationships in bitcoin through back-and-forth exploration","cited_arxiv_id":null,"evidence_quote":"Contributes the exchange-address detection features and back-and-forth exploration approach from which several Real-CATS profile features are drawn."},{"cited_title":"Txphishscope: Towards detecting and un- derstanding transaction-based phishing on ethereum","cited_arxiv_id":null,"evidence_quote":"Establishes the short-lifetime pattern of phishing addresses used to validate Real-CATS criminal and benign lifespan distributions."}],"review_version":1}