{"id":"a105bfb4-a8ec-418e-8daf-7a61f31217ce","arxiv_id":"2412.18129","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A cross-chain transaction classifier using motif statistics and event-log embeddings reaches 99.7% accuracy on known bridges and 94.8% accuracy on held-out bridges.","lead":"XSema is a machine-learning framework that labels blockchain transactions as cross-chain deposits, cross-chain withdrawals, or ordinary transactions by combining asset-transfer graph motifs with event-log text embeddings. The authors built a 10-bridge dataset and report over 94% accuracy on unseen bridges, which could help regulators monitor cross-chain money movement.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ground-truth labels rest on bridge-browser search scraping (Sec. IV-A); any missed or misindexed cross-chain transaction contaminates the negative class and all reported accuracy margins.","rationale":"The paper's contribution is empirical, and its headline numbers depend entirely on the label set. Read charitably, the framework is plausible, the held-out-bridge generalizability split is appropriate, and the reported margins are large enough to be meaningful if the labels are trustworthy. The load-bearing weakness is that Sec. IV-A describes label construction only as scraping bridge-browser search results, with no evidence of accuracy or completeness and no artifact for independent verification. This is not an attack on the authors; it is a mechanical consequence of the fact that every table in the paper is a comparison against those labels. The identical precision/recall/accuracy columns across all rows reinforce the need for raw confusion matrices and a precise metric definition, but fixing the metric computation alone would not resolve the label-risk concern. Because the issue is addressable by releasing transaction hashes, labels, and a verified sample, and because the method itself is not internally inconsistent, the conditional verdict is appropriate. I see no basis to reject the paper outright, but the empirical claims should not be taken at face value until the label-generation process is independently checked.","tokens_in":11107,"tokens_out":4794,"duration_ms":50121,"concrete_test":"Independently re-derive ground truth for a stratified random sample (e.g., 100 DT, 100 WT, 200 NT) using raw on-chain data: parse event logs via Etherscan/BscScan/PolygonScan RPCs, match deposit/withdrawal event names and transfer IDs to the corresponding bridge contracts, and confirm that each sampled DT/WT has a paired counterpart on the other chain. Also scan the sampled NT transactions for any that emit known bridge deposit/withdrawal events, counting those as label errors. If the measured label error rate exceeds roughly 1%, recompute Tables IV and V with corrected labels and check whether the reported 9-10 point margins over MoTS survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that XSema beats MoTS by over 9% on generality and over 10% on generalization. Every one of those numbers is measured against labels built by 'access[ing] the cross-chain bridge browsers and utiliz[ing] their search functions to filter potential cross-chain transactions, subsequently scraping the search results to construct accurate labels' (Sec. IV-A). This assumes the explorer search is both accurate and complete: a deposit or withdrawal that the browser fails to index is silently labeled as non-cross-chain, and a misindexed transaction creates a false DT/WT label. The paper gives no error-rate estimate, no independent verification protocol, and no code or data release, so the 33,941 labels cannot be audited. The risk is not merely random noise: the model's features are exactly the on-chain event names and transfer graphs that explorers use to categorize transactions, so label errors are correlated with the input signal and can inflate apparent accuracy. With the top generality result at 99.72%, the test set is nearly saturated, leaving little room for label noise before the reported margins degrade. A secondary internal red flag is that precision, recall, and accuracy are identical in every row of Tables IV and V, which is inconsistent with the macro metrics defined in Sec. IV-B and suggests the metric computation is not as reported; however, even correcting that would not resolve the label-risk problem.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes XSema, a supervised framework for cross-chain transaction semantic extraction. Given a transaction's metadata, XSema builds an asset-transfer graph and a message-passing text sequence from event logs, then combines a 16-dimensional motif feature vector with a pre-trained-code-model embedding and an MLP, and feeds the concatenation into a downstream classifier. The task is three-way classification of source-chain deposit, destination-chain withdrawal, and non-cross-chain transactions. The authors construct a dataset of 11,879 cross-chain transaction pairs and 10,183 non-cross-chain transactions across 10 bridges and three chains, and report generality experiments (random 80:20 split over all bridges) reaching 99.72% accuracy with DT+UniXcoder, and generalizability experiments (train on four bridges, test on six unseen bridges) reaching 94.81% accuracy with SVM+GraphCodeBERT. They also analyze motif distributions and event-log word clouds for semantic differences between transaction types.","tokens_in":11317,"tokens_out":2237,"duration_ms":22850,"significance":"If the empirical claims held up, the paper would provide a useful new dataset and a practical baseline for cross-chain transaction classification, a topic of growing importance for blockchain security and regulation. The strongest design element is the held-out-bridge generalization experiment (Section IV-E), which directly targets the challenge of unseen bridge mechanisms and goes beyond a purely random-split evaluation. The framework is clearly specified, and the use of a public motif-based feature extractor plus pre-trained code models is reproducible in principle. However, the current evidence is not yet sufficient: the ground-truth labels rest on unverified bridge-browser scraping, the reported metrics are internally inconsistent, no code or data are released, and the near-saturated accuracy numbers leave essentially no room for label noise or implementation error. The semantic analysis in Section IV-F is suggestive but informal.","major_comments":[{"comment":"The load-bearing assumption of the entire evaluation is that the bridge-browser search results used to label transactions are accurate and complete. Section IV-A states that the authors 'accessed the cross-chain bridge browsers and utilized their search functions to filter potential cross-chain transactions, subsequently scraping the search results to construct accurate labels.' If an explorer misses, delays, or misindexes a deposit or withdrawal, that transaction is mislabeled as non-cross-chain (or given the wrong class), and every accuracy number in Tables IV and V is measured against corrupted labels. The paper provides no error-rate estimate, no manual verification protocol, and no code or data release that would allow the 33,941 labels to be audited. Because the model's features (event names and transfer graphs) are exactly the signals explorers use to categorize transactions, label errors are plausibly correlated with the input features and can inflate apparent accuracy. The authors should report an independent verification study (e.g., manual review of a stratified sample, or cross-checking against a second explorer or against the bridge contracts' emitted events), quantify label noise, and make the data and labeling scripts available.","section":"IV-A (Dataset)"},{"comment":"The metric reporting is internally inconsistent: in every row of Tables IV and V, precision, recall, and accuracy are identical to three decimal places, and the F1-macro differs from them. For a three-class problem with unequal class sizes (10,183 non-cross-chain vs. 11,879 deposits vs. 11,879 withdrawals), identical macro-averaged precision and recall are essentially impossible to obtain by chance in all 24 rows, and identical precision and accuracy are also not expected. This pattern suggests that the reported precision and recall are not computed as defined in Eqs. (3)-(5), or that the metrics are micro-averages over a single collapsed binary problem, or that only the accuracy is actually computed and then copied. The authors must provide per-class precision, recall, and F1 for each setting, clarify the averaging scheme, and correct the tables. This issue does not by itself refute the central claim, but it undermines the trustworthiness of the reported margins over MoTS.","section":"IV-B and Tables IV-V"},{"comment":"The performance comparisons lack any measure of uncertainty. The central claims are margins of 'over 9%' and 'over 10%' over MoTS, but no confidence intervals, standard deviations across repeated runs, or significance tests are reported. Given that the test sets are of moderate size (the generalizability experiment tests on six bridges with on the order of 1,000 to 2,000 samples in some classes, e.g., Connext with 90 pairs and Wormhole with 53 pairs), a margin of a few percentage points could be within noise. The authors should report confidence intervals or repeated-trial statistics, and ideally per-bridge breakdowns for the held-out bridges, which would also reveal which bridge types drive the reported gains.","section":"IV-D and IV-E"},{"comment":"The generality experiment uses an 80:20 random split over all bridges, which is appropriate for measuring in-distribution performance but not for the claim of 'generality' across bridges, since the same bridges appear in training and test. The paper should clarify whether the split is at the transaction level or at the bridge level, and should report the class distribution and per-bridge accuracy in the test set. The near-saturated top result of 99.72% accuracy also warrants scrutiny: with such a high accuracy, the reported 9-10% improvements over MoTS could be dominated by a small number of misclassified samples, so the robustness of the margins should be examined, for example by ablating each feature module and by perturbing the label set.","section":"IV-D (Generality experiment)"}],"minor_comments":[{"comment":"The abstract and introduction claim 'surpassing existing methods by over 9% for the generality metric and over 10% for the generalization metric,' but the exact comparison setting (which classifier and which pre-trained model) is not stated until later; please specify the baseline configuration in the abstract or at the first mention of the margin.","section":"Abstract and Introduction"},{"comment":"The choice of max length = 256 for the event-log text sequence is stated without justification or sensitivity analysis; please report how performance varies with this hyperparameter.","section":"III-D"},{"comment":"Equation (2) shows concatenation of the 16-dimensional motif vector and the 16-dimensional text representation, but the text representation dimension is not derived explicitly from the pre-trained model and MLP; please clarify the output dimension of the MLP and how it is set to 16.","section":"III-E and Fig. 1"},{"comment":"The dataset description does not explain how the 'non-cross-chain transactions' were sampled to avoid overlap with the source-chain deposit transactions of the labeled cross-chain pairs; please clarify the exclusion criteria.","section":"IV-A"},{"comment":"The word-cloud and heatmap analyses are informal; please provide quantitative statistics (e.g., effect sizes or statistical tests) to support the claims that motif distributions are 'more concentrated' and that terms like 'toChainId' are 'particularly prevalent.'","section":"IV-F"},{"comment":"The paper cites the Chainspot platform but does not provide a reference or URL; please add a citation for the platform and for the bridge explorer tools used in data labeling.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward empirical study with a useful dataset and a sensible held-out-bridge design, but the current version cannot be accepted because the label-construction process is not verifiable and the metric tables are internally inconsistent. The fact that the motif feature extractor is inherited from reference [21], which shares authors with this submission, is not by itself a problem because the held-out-bridge experiment provides a non-circular test of the framework's generality; however, the absence of code/data and the lack of any manual label verification are more serious. I would recommend that the editor request a major revision focused on data transparency, metric correction, and uncertainty quantification. The fit with the journal's scope is adequate for a systems/security venue, though the paper would benefit from a sharper statement of the practical deployment scenario."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: XSema is a sensible, clearly written empirical paper that builds a new cross-chain transaction dataset across 10 bridges and combines MoTS-style motif features with pre-trained code-model embeddings of event logs to classify deposits, withdrawals, and normal transactions. The held-out-bridge generalization experiment is the right design, and the semantic analysis (motif heatmap, term clouds) gives some real texture. If the accuracy numbers hold after independent reproduction, this is a useful tool for regulators and analytics teams.\n\nThe soft spots are mostly about verification. The labels come from scraping bridge browsers' search results (Sec. IV-A). No error rate, no manual check, no release of code or data. If an explorer misses a cross-chain tx, that tx lands in the negative class; if it misindexes, you get a false label. And because the model's features are exactly the event names and transfer graphs the explorers use to categorize, label noise is correlated with the input signal. That can inflate accuracy precisely at the saturating 99% level they report. This is a load-bearing concern, not a nit.\n\nThere is also a reporting red flag: precision, recall, and accuracy are identical in every row of Tables IV and V. That is not what the macro metrics defined in Sec. IV-B should produce on these class distributions. Likely they printed overall accuracy in all three columns. That needs fixing before anyone trusts the numbers.\n\nOn credit where due: the framework is straightforwardly specified, the held-out split is a fair test of C2, and the paper doesn't overclaim—it says 'effectively distinguishes' and then reports numbers. The reliance on MoTS's motif extractor is fine; the held-out test provides external grounding. The citation pattern to the group's own prior work is appropriate, not self-promotional.\n\nBottom line: the central design likely survives, but the evidence is currently not independently auditable. This paper deserves a serious referee, but the referee should insist on code/data release, error bars, and a corrected metric table. I would not cite the accuracy figures until I can reproduce them. For a reading group, it's worth a maybe: the label-construction risk is a good discussion topic.","headline":"A plausible framework and useful new dataset for cross-chain transaction semantics, but the headline accuracy numbers rest on unverifiable explorer-derived labels and a sloppy metric table; fix those before trusting any number.","tokens_in":11875,"tokens_out":2126,"would_cite":false,"duration_ms":20082,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"XSema distinguishes cross-chain deposits, withdrawals, and ordinary transactions by combining asset-transfer graph motifs with event-log text, reaching 99.72% accuracy on known bridges and 94.81% on unseen bridges.","keywords":["cross-chain transactions","semantic extraction","asset transfer graph","network motifs","message-passing event logs","pre-trained code models","transaction classification","blockchain bridges"],"falsifier":"Take a random sample of transactions labeled as deposits and withdrawals, and independently verify each one by matching the source-chain deposit event's transfer ID (or equivalent identifier) to the corresponding destination-chain withdrawal event in the bridge's contracts; if a nontrivial fraction of labels fail to match, recompute XSema's accuracy on the corrected labels and compare it with the reported 99.72% and 94.81% figures.","tokens_in":10861,"feed_emoji":"⛓️","tokens_out":4811,"duration_ms":40669,"temperature":0.7,"pith_summary":"The paper argues that cross-chain transactions leave two detectable traces: the shape of the asset-transfer graph on the source and destination chains, and the text of the event logs emitted during execution. XSema encodes both traces—16 directed network motifs for the graph, and code-pretrained embeddings of concatenated event names for the text—and feeds the concatenated vector to a standard classifier. On a new dataset of 11,879 cross-chain transaction pairs from ten bridges plus 10,183 ordinary Ethereum transactions, the framework classifies deposits, withdrawals, and non-cross-chain transactions with up to 99.72% accuracy when tested on the same bridges and 94.81% accuracy when tested on six bridges excluded from training. The authors' point is that transaction semantics for cross-chain regulation can be extracted without hand-crafting rules per bridge, because the underlying asset-transfer and message-passing patterns generalize.","feed_headline":"Cross-chain transactions classified with 99.7% accuracy","feed_subtitle":"XSema blends asset-transfer graph shapes with event-log text to tell deposits, withdrawals, and ordinary transactions apart.","key_machinery":"The central objects are (1) the asset transfer graph, a directed graph whose edges are external, internal, ERC-20, and ERC-721 transfers involving the transaction, summarized as a 16-dimensional vector of directed motif frequencies (M1–M16); and (2) the message-passing text, a sequence of concatenated event-log names from the transaction, encoded by a code pre-trained model and an MLP into another 16-dimensional vector. The two vectors are concatenated into the transaction's semantic representation, which a standard classifier maps to deposit, withdrawal, or non-cross-chain. The motif statistics capture the structural signature of asset flow, while the event-log encoding captures bridge-specific but transferable message-passing vocabulary.","core_discovery":"XSema treats cross-chain semantic extraction as a three-way classification problem—deposit transaction on the source chain, withdrawal transaction on the destination chain, and non-cross-chain transaction—and claims that a representation combining asset-transfer structure with message-passing text is sufficient to separate these classes. The framework analyzes the transaction's asset transfer graph using 16 directed network motifs, constructs a text sequence from the event log names, encodes that text with a code pre-trained model (CodeBERT, GraphCodeBERT, or UniXcoder) followed by an MLP, and concatenates both 16-dimensional representations before classification. The reported results show XSema beating the prior generalized transaction-semantic method by over 9% in accuracy on the same set of bridges and by over 10% on bridges not seen in training, with the best configurations reaching 99.72% and 94.81% accuracy respectively.","pith_inferences":["If the label-scraping assumption holds, the same two-view representation could be applied to other cross-chain mechanisms (e.g., layer-2 message passing or atomic swaps) by redefining the asset-transfer edges and event text appropriately, though the 16-motif vocabulary may need extension for larger graphs.","The sharp drop from 99.72% on seen bridges to 94.81% on unseen bridges hints that a share of the signal is bridge-specific vocabulary in event names; a domain-adaptation step on event text might close that gap.","Because the framework returns interpretable motif and event-word statistics, regulators could use the same features to triage large transaction volumes for manual review rather than relying on black-box scores."],"forward_implications":["Cross-chain transactions on known bridges can be separated from ordinary transactions at better than 99% accuracy using only transaction metadata available through standard RPC APIs.","Training on four high-volume bridges transfers to six unseen bridges with over 94% accuracy, indicating that bridge-specific event names and contract layouts are not needed for classification.","The method needs no per-bridge hand-crafted rules: the same motif statistics and event-log encoding work across ten different bridge implementations.","The concentration of cross-chain transactions on a few high-order motifs (motifs 10, 11, and 13 for deposits; motif 11 for withdrawals) suggests a compact structural signature usable for monitoring."],"supporting_citations":[{"why":"Supplies the generalized transaction-semantic baseline (motif-based) that XSema must outperform, and the motif computation method.","marker":"[21]"},{"why":"Provides the higher-order network motif concept and its structural significance, underpinning the 16-dimensional asset-transfer feature vector.","marker":"[24]"},{"why":"One of the code pre-trained encoders used to embed event-log text in the message-passing semantic module.","marker":"[25]"},{"why":"Another code pre-trained encoder variant tested for event-log text embedding in XSema.","marker":"[26]"},{"why":"A third code pre-trained encoder variant tested in XSema, yielding the best generality result.","marker":"[27]"},{"why":"Provides the machine-learning classifiers (Decision Tree, SVM, Random Forest, AdaBoost) used in the XSema pipeline.","marker":"[28]"}],"fun_headline_variants":["XSema classifies cross-chain transactions with 99.7% accuracy","99.7% accuracy: XSema decodes cross-chain semantics","XSema separates transaction types on cross-chain with 99.7%","Cross-chain semantic extraction reaches 99.7% with XSema"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training and test labels come from searching each bridge's own transaction browser and scraping the results, so the entire evaluation assumes those search tools identify every cross-chain deposit and withdrawal correctly and completely.","fun_headline_variants_meta":{"raw":{"variants":["XSema classifies cross-chain transactions with 99.7% accuracy","99.7% accuracy: XSema decodes cross-chain semantics","XSema separates transaction types on cross-chain with 99.7%","Cross-chain semantic extraction reaches 99.7% with XSema"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000626,"raw_usage":{"total_tokens":2891,"prompt_tokens":932,"completion_tokens":1959,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":1880}},"tokens_in":548,"tokens_out":1959,"duration_ms":17498,"temperature":1.0,"reasoning_tokens":1880,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:00:09.499886+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of transactions labeled as deposits and withdrawals, and independently verify each one by matching the source-chain deposit event's transfer ID (or equivalent identifier) to the corresponding destination-chain withdrawal event in the bridge's contracts; if a nontrivial fraction of labels fail to match, recompute XSema's accuracy on the corrected labels and compare it with the reported 99.72% and 94.81% figures.","supporting_citations":[{"cited_title":"Know your transactions: Real-time and generic transaction semantic representation on blockchain & web3 ecosystem,","cited_arxiv_id":null,"evidence_quote":"Supplies the generalized transaction-semantic baseline (motif-based) that XSema must outperform, and the motif computation method."},{"cited_title":"Higher-order organiza- tion of complex networks,","cited_arxiv_id":null,"evidence_quote":"Provides the higher-order network motif concept and its structural significance, underpinning the 16-dimensional asset-transfer feature vector."},{"cited_title":"Scikit-learn: Machine learning in python,","cited_arxiv_id":null,"evidence_quote":"Provides the machine-learning classifiers (Decision Tree, SVM, Random Forest, AdaBoost) used in the XSema pipeline."}],"review_version":1}