Pith. sign in

REVIEW 5 major objections 5 minor 81 references

LLM pipeline names attacker and victim in 85% of 27 Ethereum hacks

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

TraceLLM automatically generates human-readable security reports for Ethereum hacks by feeding LLMs a mix of execution traces, decompiled code, and balance changes.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Useful integration and a genuine 148-incident generalization test, but the evaluation has an unaddressed train/test overlap that could inflate the headline numbers and must be fixed before I'd trust it. the 5 major comments →

arxiv 2509.03037 v1 pith:KHZIXZEP submitted 2025-09-03 cs.CR cs.ETcs.SE

TraceLLM: Security Diagnosis Through Traces and Smart Contracts in Ethereum

Shuzheng Wang , Yue Huang , Zhuoer Xu , Yuming Huang , Jing Tang This is my paper
classification cs.CR cs.ETcs.SE
keywords smart contract securityEthereum forensicsexecution tracesanomaly path detectionLLM security analysisdecompilationincident reportsblockchain benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TraceLLM is a proposed end-to-end pipeline that answers a forensic question: who attacked whom, how, and through which vulnerable function. It does this by feeding an LLM a small, targeted slice of on-chain evidence: a reconstructed execution tree, the anomalous paths inside it, decompiled code of unverified contracts, and net balance changes. The paper's central claim is that this joint trace-and-code view lets the model recover attacker and victim addresses with 85.19% precision and factually accurate attack mechanisms in 70.37% of 27 incidents whose ground truth comes from expert reports, compared with 44.44% for the strongest baseline that sends raw traces alongside code-analysis results. If true, much of the manual trace-to-code alignment that dominates post-incident forensics could be automated, and the method introduces the first labeled dataset of 11,228 execution paths as a benchmark for anomaly-path detection in blockchain traces. The paper also argues the approach generalizes across 148 real Ethereum incidents with 66.22% expert-verified report accuracy.

Core claim

TraceLLM's load-bearing result is that transaction anomaly detection and code vulnerability detection can be fused at the level of a single execution. The framework reconstructs flat EVM traces into a call tree, scores each root-to-leaf path with structural, semantic, and lexical features, flags the most anomalous paths, expands them into compact k-hop enclosing subgraphs, and hands those subgraphs together with decompiled contract code and balance changes to an LLM for report generation. On 27 incidents with expert-written ground-truth reports, this joint view identifies attacker and victim addresses with 85.19% precision and correctly describes the attack mechanism and vulnerable functions

What carries the argument

The central object is the reconstructed execution call tree: flat EVM trace entries are re-parented by stack rules into a tree whose edges carry method, value, call type, and result. Each root-to-leaf path receives five features (fanout, depth, pattern frequency, density of known-dangerous methods, and TF-IDF over method signatures), and a logistic-regression path scorer ranks the anomalous sub-paths. An LLM-refined decompiler recovers readable code for unverified contracts, and a k-hop enclosing subgraph (the predecessors and successors around path nodes, tuned to k=1) supplies contextual structure to the LLM, which fuses subgraph, code, and balance changes into a human-readable incident re

Load-bearing premise

The evaluation rests on expert-written reports being the correct and complete ground truth, and on the 15 incidents used to train the anomaly-path classifier being disjoint from the 27 incidents used to measure report accuracy; a disjointness the paper never states.

What would settle it

Inspect the released incident lists and code: if any of the 15 training incidents for the anomaly-path classifier also appears among the 27 used to evaluate report generation, the central accuracy claims are inflated. Re-running the pipeline with the missing report-generation prompt and a strictly held-out split would settle whether the reported 85.19% and 70.37% figures hold.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Post-incident forensics could shift from manual trace alignment to reviewing machine-generated drafts: analysts verify and refine rather than reconstruct the attack path from scratch.
  • The 11,228-path anomaly dataset, with 1,530 labeled attack paths, becomes a reusable benchmark for trace-level detection rather than transaction-level detection.
  • Adding decompiled-code context alongside raw traces yields a roughly 26-point report-accuracy gain over the strongest code-analysis-plus-raw-trace baseline in the 27-case test.
  • Because the pipeline relies on standard EVM execution semantics, the same modules transfer to other EVM-compatible chains with minimal changes, as the paper argues in its extensibility discussion.
  • The k=1 enclosing subgraph captures most of the address-identification benefit at a fraction of the token cost of deeper expansions, indicating that compact context beats broad context for this task.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural test the paper does not run is a strict external validation where the 15 incidents used to train the anomaly-path classifier are confirmed disjoint from the 27 incidents used to evaluate report generation; the published accuracy numbers should be read with that caveat until the overlap question is settled.
  • The 66.22% expert-verified accuracy on the 148-incident set leaves room for a residual-failure analysis that isolates whether the bottleneck is decompiler fidelity, path ranking, or LLM reasoning; an ablation the modular design enables but the paper does not perform.
  • The report-generation prompt is referenced as '??' and absent from Appendix A, so independent reproduction is currently impossible; publishing that prompt is the minimal condition for the claimed benchmark to function as one.
  • If the approach transfers to routine transactions, transaction-explanation tools could show users a plain-language narrative of what a contract interaction will do before signing; an application the paper mentions in discussion but does not evaluate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes TraceLLM, a modular LLM-based pipeline for Ethereum post-incident forensics. It takes incident descriptions, resolves addresses and time ranges, resolves proxy implementations and contract creators, retrieves or decompiles contract code, reconstructs call trees from flat EVM traces, extracts structural/semantic path features, trains a logistic-regression classifier to rank anomalous execution paths, extracts k-hop enclosing subgraphs, and feeds these to an LLM to generate human-readable security reports. The evaluation reports 85.19% precision on attacker/victim identification and 70.37% factual precision on 27 expert-report cases, outperforming proxy baselines built from Mythril, Slither, and GPTScan; a larger run on 148 incidents reports 82.43% attacker/victim and 66.22% overall expert-verified accuracy. Module-level experiments report 83.92% recall for anomaly path detection and 78.77% decompilation accuracy.

Significance. If the reported numbers hold, TraceLLM would be a meaningful step toward automating the trace-to-code alignment that currently dominates post-incident smart-contract forensics. The paper introduces a new anomaly-path dataset and benchmark, evaluates against multiple baselines, and tests on real incidents, which are strengths. However, several load-bearing evaluation details are missing or inconsistent in the submitted manuscript, so the central quantitative claims cannot yet be verified. The direction is significant and the issues appear addressable.

major comments (5)
  1. [§6.2.1, §6.2.3] The paper never states whether the 15 incidents used to train and evaluate the anomaly-path classifier and to select k=1 are disjoint from the 27 incidents used for end-to-end report generation. Both sets are drawn from the same pool described in §6.2. If the 15 are a subset of the 27, then for those incidents the logistic-regression classifier (Eq. 9) has been trained on labels derived from the same expert reports that serve as RQ1 ground truth, and the k=1 hyperparameter (Table 3) is selected on evaluation data. Because the LLM receives the k-hop enclosing subgraph S_k(P) of top-ranked paths (Eq. 13), this could directly inflate the 85.19% attacker/victim and 70.37% factual-precision figures. Please state explicitly how the 15 and 27 incident sets relate, and report a sensitivity analysis on disjoint versus overlapping subsets.
  2. [§5.2.3 / Appendix A] The report-generation prompt is missing. The text says 'The prompt is shown in ?? in Appendix' and Appendix A contains '?? provides the prompt for report generation.' This prompt is the core instrument of both TraceLLM and the proxy baselines (§6.2.1), so its absence prevents replication and makes the reported comparison unverifiable. The prompt and any baseline prompt variants must be included.
  3. [Table 2] The checkmarks in Table 2 are inconsistent with the percentages reported in the text. Counting the table as printed: GPTScan has 11 successes (40.7%), not 12 (44.44%); Slither has 10 (37.0%), not 11 (40.74%); Mythril has 5 (18.5%), not 6 (22.22%). The reported comparative margins depend on these counts. Please correct the table or the reported percentages, and verify all derived claims.
  4. [§5.2.1, Algorithm 1] The call-tree reconstruction algorithm does not correctly handle non-consecutive sibling calls. The sibling rule in Definition 1 and the inner while loop only group consecutive calls from the same caller. In an EVM trace, a parent that makes a second call after a nested call returns produces a non-consecutive sibling: e.g., A→B, B→C, B→D. At i=2, call_i.from = B ≠ C = call_{i-1}.to, so line 4 starts a new root tree for B→D instead of attaching it under A→B. The resulting call trees can be fragmented, and the path features in Eqs. (1)–(8) and the enclosing subgraphs S_k(P) are computed over this potentially incorrect structure. Please validate reconstruction against a tracer with built-in call depth or revise the stack logic.
  5. [§6.2.2] The generalizability claim (82.43% attacker/victim and 66.22% overall across 148 incidents) relies on 'expert analysis,' but no protocol is given: number of experts, independence, blinding to TraceLLM's outputs, or inter-rater agreement. Since this is a central quantitative claim, the expert verification process must be described in sufficient detail for the result to be assessed.
minor comments (5)
  1. [§6.2.3 / Data availability] The paper says the anomaly-path dataset will be publicly released and is 'the first publicly available dataset' of its kind, but no repository link or availability statement is provided. Add a URL or explicit availability note.
  2. [§6.2.1] The metric 'precision' is used for attacker/victim identification and for factual report accuracy, but the exact computation (per-event all-or-nothing, whether both attacker and victim must be correct, how partial credit is handled) is never formally defined. Define the metric precisely.
  3. [§6.2.3 / Extractor] The decompilation equivalence evaluation uses three LLM judges and requires all three to agree, but no agreement statistics or validation against human judges are reported. At minimum, report the distribution of judgments and agreement rates.
  4. [Global] Typos and formatting issues: 'decomiled' in Section 3, 'drawn as' where 'written as' is intended near Eq. (1), and the variable M in Eq. (5) is introduced only as a referenced set in Table 1 rather than being formally defined.
  5. [§6.2.1 / Baselines] The baseline construction feeds the source code of victim contracts identified in expert reports into the baseline tools. This should be stated as a design choice and its effect on the comparison discussed, since it gives the baselines ground-truth information about where to look.

Circularity Check

1 steps flagged

Unstated overlap between the 15 incidents used to train the anomaly-path classifier and the 27/148 incident evaluation sets leaves the headline precisions potentially trained-on-test.

specific steps
  1. fitted input called prediction [§6.2 Methodology; §6.2.1 RQ1; §6.2.3 RQ3 Analyzer (Eq. 9, Eq. 13, Table 3)]
    "Among these, 27 events with credible expert reports are selected to validate RQ1. Expert reports are treated as the ground truth. ... we evaluated our method on 15 real-world blockchain security incidents with human-written expert reports to measure the accuracy of anomaly execution path identification. ... labeled them by matching victim contracts, vulnerable functions, and attacker addresses from the reports."

    The anomaly-path classifier (Eq. 9) and the k=1 hyperparameter (Table 3) are fit on 15 incidents whose path labels are copied from expert reports. The end-to-end RQ1 evaluation is run on 27 incidents whose ground truth is expert reports from the same SlowMist corpus, and RQ2 runs on all 148 incidents. The paper never states that the 15 training incidents are excluded from the 27 or the 148. If they are not excluded, the top-k subgraph (Eq. 13) supplied to the LLM is selected by a model trained on the very ground-truth labels used to score the generated reports, so the reported 85.19%/70.37%/66.22% figures are partly a measure of training-data memorization rather than out-of-sample prediction. The absence of any holdout sentence makes the central quantitative claim non-verifiable as a predi

full rationale

The end-to-end evaluation is externally grounded in the sense that expert reports are used as ground truth and the LLM is a third-party model; the 121-event generalizability run is at least partly out-of-sample relative to the 27-event set. The pipeline itself (call-tree reconstruction, TF-IDF, logistic regression, k-hop subgraph, LLM) is not definitionally equivalent to the expert reports. The main circularity risk is the unstated relation between the 15 incidents used to fit Eq. 9 and select k=1 (Table 3) and the 27/148 incidents used to score report generation. Both sets are drawn from the same SlowMist expert-report corpus, and no sentence excludes the 15 from the evaluation sets, so the paper does not demonstrate that the headline numbers are out-of-sample. To the extent overlap exists, the enclosing subgraph (Eq. 13) is produced by a classifier trained on the same ground-truth labels, making the 'prediction' partly a fit. I set the score to 4 rather than 6 because the overlap is not positively established and the LLM report-generation step still has independent content. The missing report-generation prompt ('??' in §5.2.3 and Appendix A) is an omitted component that blocks reproduction; it is a correctness/reproducibility issue, not an additional circular step. Self-citations ([14], [49], [50]) are not load-bearing.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claims are empirical, not derivations from first principles. The system's behavior is governed by a trained classifier, hand-set thresholds (top-20, k=1), a manually curated signature table, and LLM prompts (one of which is missing from the paper). The only axioms beyond standard math are domain assumptions about EVM trace structure, expert-report ground truth, and LLM-based judging, all of which are plausible but unverified within the paper.

free parameters (5)
  • Logistic regression weights w and bias b = not reported
    Parameters of the binary path classifier (Eq. 9) fit to the 15-incident training set; values are not reported anywhere.
  • Top-k path cutoff = 20
    The number of highest-scoring paths sent to the LLM is fixed at 20 by design (§6.2.3), a hand-chosen threshold affecting report generation input.
  • k-hop closure depth = 1
    Chosen by evaluating k=0..5 on the 15-incident evaluation set (Table 3); this is hyperparameter tuning on the test data.
  • Suspicious method signature set M = 13 signatures (Table 1)
    Hand-curated list of method signatures mapped to SWC vulnerability classes; the semantic anomaly score Eq. 5 depends entirely on this set.
  • LLM sampling hyperparameters = temperature 0.7, top_p 1, max tokens 2000
    Decoding settings for Gemini 2.0 Flash (§6.1); these affect output variability and hence report precision.
axioms (5)
  • domain assumption EVM call traces preserve strict nesting order so that parent-child relations can be recovered by Algorithm 1's rule (child iff call_i.from == call_{i-1}.to)
    Invoked in §5.2.1 Definition 1 and Algorithm 1; if trace flattening or failed-call logs break this ordering, the call tree and all downstream path features are wrong.
  • domain assumption Expert reports are accurate ground truth for attacker/victim addresses and attack mechanisms
    Used throughout §6.2 to label paths and score reports; no second-annotator measure is given.
  • ad hoc to paper The three LLM judges (OpenAI-o1, Claude 3.5, DeepSeek-R1) agreeing on consistency is a valid measure of decompilation equivalence
    Introduced in §6.2.3 Extractor evaluation; no human validation of the LLM judgments is reported.
  • domain assumption The 13 method signatures in Table 1 cover the exploit-relevant functions for the attack types in the dataset
    The semantic anomaly score S(P) in Eq. 5 and the TF-IDF features rely on this coverage; novel attack types using unlisted functions will be scored as benign.
  • standard math Standard statistical machinery: logistic regression, TF-IDF, precision/recall
    Used in §5.2.2 and §6.2; standard methods with standard assumptions.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of TraceLLM: Security Diagnosis Through Traces and Smart Contracts in Ethereum." pith.science (2026). https://pith.science/paper/KHZIXZEP

@misc{pith2026250903037,
  author       = {Pith},
  title        = {Pith review of: TraceLLM: Security Diagnosis Through Traces and Smart Contracts in Ethereum},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KHZIXZEP}},
  note         = {Machine review of arXiv:2509.03037}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Ethereum smart contracts hold tens of billions of USD in DeFi and NFTs, yet comprehensive security analysis remains difficult due to unverified code, proxy-based architectures, and the reliance on manual inspection of complex execution traces. Existing approaches fall into two main categories: anomaly transaction detection, which flags suspicious transactions but offers limited insight into specific attack strategies hidden in execution traces inside transactions, and code vulnerability detection, which cannot analyze unverified contracts and struggles to show how identified flaws are exploited in real incidents. As a result, analysts must still manually align transaction traces with contract code to reconstruct attack scenarios and conduct forensics. To address this gap, TraceLLM is proposed as a framework that leverages LLMs to integrate execution trace-level detection with decompiled contract code. We introduce a new anomaly execution path identification algorithm and an LLM-refined decompile tool to identify vulnerable functions and provide explicit attack paths to LLM. TraceLLM establishes the first benchmark for joint trace and contract code-driven security analysis. For comparison, proxy baselines are created by jointly transmitting the results of three representative code analysis along with raw traces to LLM. TraceLLM identifies attacker and victim addresses with 85.19\% precision and produces automated reports with 70.37\% factual precision across 27 cases with ground truth expert reports, achieving 25.93\% higher accuracy than the best baseline. Moreover, across 148 real-world Ethereum incidents, TraceLLM automatically generates reports with 66.22\% expert-verified accuracy, demonstrating strong generalizability.

Figures

Figures reproduced from arXiv: 2509.03037 by Jing Tang, Shuzheng Wang, Yue Huang, Yuming Huang, Zhuoer Xu.

Figure 1
Figure 1. Figure 1: A high-level workflow of TraceLLM. 3 TraceLLM Overview In this section, we present the overall design of TraceLLM, a modu￾lar framework for security analysis of Ethereum smart contracts. At a high level, TraceLLM accepts either natural language queries from end users or monitoring signals from key opinion leaders (KoLs), such as security researchers and watchdog accounts. These inputs are converted into st… view at source ↗
Figure 2
Figure 2. Figure 2: LLM-refined code extractor workflow. 5.1 Extractor Meaningful semantic interpretation of contract behavior neces￾sitates understanding its underlying code logic. Accordingly, the Extractor module retrieves contract metadata for all addresses iden￾tified by the Detector. Within the Ethereum ecosystem, every de￾ployed smart contract stores its compiled bytecode on-chain for execution by the EVM. Although byt… view at source ↗
Figure 4
Figure 4. Figure 4: The exploitation mechanism of the PlayDapp hack. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: shows the average recall across 15 LOGO folds. Our solution achieves the highest recall (0.8392), outperforming all base￾lines, including Random Forest (0.8124) and Priority Score (0.7889). GIN (0.7340) and SAGE (0.7033) lag behind, suggesting that generic GNN architectures struggle to capture the semantic and hierarchi￾cal features of EVM call trees in this anomaly detection setting. The semantic-based un… view at source ↗
Figure 7
Figure 7. Figure 7: presents the accuracy comparison across different con￾tract size ranges. The red bars represent our framework, and the blue bars correspond to Panoramix. Overall, Panoramix achieves 70.25% average accuracy, while our framework improves upon this by 8.52%. Notably, in the 0–15 KB and >60 KB contract size ranges, both approaches perform less effectively. This can be attributed to the fact that very small con… view at source ↗
Figure 8
Figure 8. Figure 8: Generation prompt for query normalization. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Prompt for code refining. A Prompts This section provides prompt used in TraceLLM [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Prompt for consistent judging [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

81 extracted references · 67 canonical work pages · 2 internal anchors

  1. [1]

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954 (2024)

    Pith/arXiv arXiv 2024

  2. [2]

    Lei Cai, Zhengzhang Chen, Chen Luo, Jiaping Gui, Jingchao Ni, Ding Li, and Haifeng Chen. 2021. Structural temporal graph neural networks for anomaly de- tection in dynamic graphs. InProceedings of the 30th ACM international conference on Information & Knowledge Management . 3747–3756

    2021

  3. [3]

    Chong Chen, Jianzhong Su, Jiachi Chen, Yanlin Wang, Tingting Bi, Jianxing Yu, Yanli Wang, Xingwei Lin, Ting Chen, and Zibin Zheng. 2023. When chatgpt meets smart contract vulnerability detection: How far are we? ACM Transactions on Software Engineering and Methodology (2023)

    work page 2023

  4. [4]

    ConsenSysDiligence. 2024. Mythril. https://github.com/ConsenSysDiligence/ mythril?tab=readme-ov-file

    work page 2024

  5. [5]

    Isaac David, Liyi Zhou, Kaihua Qin, Dawn Song, Lorenzo Cavallaro, and Arthur Gervais. 2023. Do you still need a manual smart contract audit? arXiv preprint arXiv:2306.12338 (2023)

    Pith/arXiv arXiv 2023

  6. [6]

    Defillama. 2025. Total Value Locked in DeFi on Ethereum . https://defillama.com/ chain/Ethereum

    work page 2025

  7. [7]

    Etherscan. 2025. Ethereum Charts & Statistics . https://etherscan.io/charts

    work page 2025

  8. [8]

    Etherscan. 2025. Etherscan API Document. https://docs.etherscan.io/etherscan- v2/api-endpoints/contracts#get-contract-creator-and-creation-tx-hash

    work page 2025

  9. [9]

    Josselin Feist, Gustavo Grieco, and Alex Groce. 2019. Slither: a static analysis framework for smart contracts. In 2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB) . IEEE, 8–15

    work page 2019

  10. [10]

    Ethereum Foundation. 2025. Ethereum Virtual Machine (EVM) . https://ethereum. org/en/developers/docs/evm/

    work page 2025

  11. [11]

    Ethereum Foundation. 2025. Introduction to smart contracts . https://ethereum. org/en/developers/docs/smart-contracts

    work page 2025

  12. [12]

    Yu Gai, Liyi Zhou, Kaihua Qin, Dawn Song, and Arthur Gervais. 2023. Blockchain large language models. arXiv preprint arXiv:2304.12749 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  13. [13]

    Sihao Hu, Tiansheng Huang, Ka-Ho Chow, Wenqi Wei, Yanzhao Wu, and Ling Liu

  14. [14]

    Yue Huang, Shuzheng Wang, Yuming Huang, and Jing Tang. 2024. Two sides of the same coin: Large-scale measurements of builder and rollup after eip-4844. arXiv preprint arXiv:2411.03892 (2024)

    work page arXiv 2024

  15. [15]

    William E Bodell III, Sajad Meisami, and Yue Duan. 2023. Proxy Hunting: Un- derstanding and Characterizing Proxy-based Upgradeable Smart Contracts in Blockchains. In 32nd USENIX Security Symposium (USENIX Security 23) . 1829– 1846

    work page 2023

  16. [16]

    Jinan Jiang, Zihao Li, Haoran Qin, Muhui Jiang, Xiapu Luo, Xiaoming Wu, Haoyu Wang, Yutian Tang, Chenxiong Qian, and Ting Chen. 2024. Unearthing Gas- Wasting Code Smells in Smart Contracts with Large Language Models. IEEE Transactions on Software Engineering (2024)

    work page 2024

  17. [17]

    Zongwei Li, Xiaoqi Li, Wenkai Li, and Xin Wang. 2025. Scalm: Detecting bad practices in smart contracts through llms. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 470–477

    work page 2025

  18. [18]

    Zhong Li, Yuxuan Zhu, and Matthijs Van Leeuwen. 2023. A survey on explainable anomaly detection. ACM Transactions on Knowledge Discovery from Data 18, 1 (2023), 1–54

    work page 2023

  19. [19]

    Ye Liu, Yue Xue, Daoyuan Wu, Yuqiang Sun, Yi Li, Miaolei Shi, and Yang Liu

  20. [20]

    Yushan Liu, Mu Zhang, Ding Li, Kangkook Jee, Zhichun Li, Zhenyu Wu, Jungh- wan Rhee, and Prateek Mittal. 2018. Towards a timely causality analysis for enterprise security.. In NDSS, Vol. 24. 141

    work page 2018

  21. [21]

    arXiv preprint arXiv:2405.02580 (2024)

    Propertygpt: Llm-driven formal verification of smart contracts through retrieval-augmented property generation. arXiv preprint arXiv:2405.02580 (2024)

    Pith/arXiv arXiv 2024

  22. [22]

    Jie Ma, Ningyu He, Jinwen Xi, Mingzhe Xing, Haoyu Wang, Ying Gao, and Yinliang Yue. 2025. OpDiffer: LLM-Assisted Opcode-Level Differential Testing of Ethereum Virtual Machine. Proceedings of the ACM on Software Engineering 2, ISSTA (2025), 1559–1582

    work page 2025

  23. [23]

    Paul Louis, Shweta Ann Jacob, and Amirali Salehi-Abari. 2022. Sampling enclos- ing subgraphs for link prediction. In Proceedings of the 31st ACM international conference on information & knowledge management . 4269–4273

    work page 2022

  24. [24]

    Robert McLaughlin, Christopher Kruegel, and Giovanni Vigna. 2023. A large scale study of the ethereum arbitrage ecosystem. In 32nd USENIX Security Symposium (USENIX Security 23). 3295–3312

    work page 2023

  25. [25]

    Wei Ma, Daoyuan Wu, Yuqiang Sun, Tianwen Wang, Shangqing Liu, Jian Zhang, Yue Xue, and Yang Liu. 2025. Combining Fine-Tuning and LLM-Based Agents for Intuitive Smart Contract Auditing with Justifications. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) . IEEE, 1742–1754

    work page 2025

  26. [26]

    MythX. 2020. Smart Contract Weakness Classification. https://swcregistry.io/

    work page 2020

  27. [27]

    Fieke Miedema, Kelvin Lubbertsen, Verena Schrama, and Rolf Van Wegberg. 2023. Mixed Signals: Analyzing{Ground-Truth} Data on the Users and Economics of a Bitcoin Mixing Service. In 32nd USENIX Security Symposium (USENIX Security 23). 751–768

    work page 2023

  28. [28]

    Zhiyuan Peng, Xin Yin, Rui Qian, Peiqin Lin, Yongkang Liu, Chenhao Ying, and Yuan Luo. 2025. SolEval: Benchmarking Large Language Models for Repository- level Solidity Code Generation. arXiv preprint arXiv:2502.18793 (2025)

    Pith/arXiv arXiv 2025

  29. [29]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730–27744

    2022

  30. [30]

    Kaihua Qin, Stefanos Chaliasos, Liyi Zhou, Benjamin Livshits, Dawn Song, and Arthur Gervais. 2023. The blockchain imitation game. In 32nd USENIX Security Symposium (USENIX Security 23) . 3961–3978

    work page 2023

  31. [31]

    Rifkie Primartha and Bayu Adhi Tama. 2017. Anomaly detection using random forest: A performance revisited. In 2017 International conference on data and software engineering (ICoDSE). IEEE, 1–6

    work page 2017

  32. [32]

    Chaofan Shou, Jing Liu, Doudou Lu, and Koushik Sen. 2024. Llm4fuzz: Guided fuzzing of smart contracts with large language models. arXiv preprint arXiv:2401.11108 (2024)

    Pith/arXiv arXiv 2024

  33. [33]

    2023.{GAP}: Differentially private graph neural networks with aggre- gation perturbation

    Sina Sajadmanesh, Ali Shahin Shamsabadi, Aurélien Bellet, and Daniel Gatica- Perez. 2023.{GAP}: Differentially private graph neural networks with aggre- gation perturbation. In 32nd USENIX Security Symposium (USENIX Security 23) . 3223–3240

    work page 2023

  34. [34]

    Slowmist. 2025. SlowMist. https://slowmist.medium.com/

    work page 2025

  35. [35]

    Slowmist. 2024. Hacked Target: PlayDapp. https://hacked.slowmist.io/zh/search/

    work page 2024

  36. [36]

    Liya Su, Xinyue Shen, Xiangyu Du, Xiaojing Liao, XiaoFeng Wang, Luyi Xing, and Baoxu Liu. 2021. Evil under the sun: Understanding and discovering attacks on ethereum decentralized applications. In 30th USENIX Security Symposium (USENIX Security 21). 1307–1324

    work page 2021

  37. [37]

    Sunbeom So, Seongjoon Hong, and Hakjoo Oh. 2021. {SmarTest}: Effec- tively hunting vulnerable transaction sequences in smart contracts through language{Model-Guided} symbolic execution. In 30th USENIX Security Sympo- sium (USENIX Security 21) . 1361–1378

    work page 2021

  38. [38]

    Tianle Sun, Ningyu He, Jiang Xiao, Yinliang Yue, Xiapu Luo, and Haoyu Wang

    work page

  39. [39]

    Jiaze Sun, Zhiqiang Yin, Hengshan Zhang, Xiang Chen, and Wei Zheng. 2025. Adversarial generation method for smart contract fuzz testing seeds guided by chain-based LLM. Automated Software Engineering 32, 1 (2025), 12

    work page 2025

  40. [40]

    Zhiyuan Sun, Xiapu Luo, and Yinqian Zhang. 2023. Panda: Security analysis of algorand smart contracts. In 32nd USENIX Security Symposium (USENIX Security 23). 1811–1828

    work page 2023

  41. [41]

    In 33rd USENIX Security Symposium (USENIX Security 24)

    All your tokens are belong to us: Demystifying address verification vulner- abilities in solidity smart contracts. In 33rd USENIX Security Symposium (USENIX Security 24). 3567–3584

    work page

  42. [42]

    Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Haijun Wang, Zhengzi Xu, Xiaofei Xie, and Yang Liu. 2024. Gptscan: Detecting logic vulnerabilities in smart contracts by combining gpt with program analysis. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering . 1–13

    work page 2024

  43. [43]

    Massimiliano Taverna and Kenneth G Paterson. 2023. Snapping snap sync: practical attacks on go Ethereum synchronising nodes. In 32nd USENIX Security Symposium (USENIX Security 23) . 3331–3348

    work page 2023

  44. [44]

    SunWeb3Sec. 2025. DeFi Hacks Reproduce . https://github.com/SunWeb3Sec/ DeFiHackLabs/tree/main

    work page 2025

  45. [45]

    Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. 2024. LLM4Decompile: De- compiling Binary Code with Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 3473–3487

    work page 2024

  46. [46]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    Pith/arXiv arXiv 2023

  47. [47]

    Christof Ferreira Torres, Ramiro Camino, et al. 2021. Frontrunner jones and the raiders of the dark forest: An empirical study of frontrunning on the ethereum blockchain. In 30th USENIX Security Symposium (USENIX Security 21). 1343–1359

    work page 2021

  48. [48]

    Christof Ferreira Torres, Mathis Steichen, et al. 2019. The art of the scam: De- mystifying honeypots in ethereum smart contracts. In 28th USENIX Security Symposium (USENIX Security 19) . 1591–1607

    work page 2019

  49. [49]

    Fan Wang, Juyong Jiang, Chansung Park, Sunghun Kim, and Jing Tang. [n. d.]. KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models. In The Thirteenth International Conference on Learning Representations

    work page

  50. [50]

    Trustwallet. 2025. Trust Wallet Token Repository. https://github.com/trustwallet/ asset

    work page 2025

  51. [51]

    Che Wang, Jiashuo Zhang, Jianbo Gao, Libin Xia, Zhi Guan, and Zhong Chen

    work page

  52. [52]

    In Proceedings of the 39th IEEE/ACM International Conference on Auto- mated Software Engineering

    Contracttinker: Llm-empowered vulnerability repair for real-world smart contracts. In Proceedings of the 39th IEEE/ACM International Conference on Auto- mated Software Engineering. 2350–2353

    work page

  53. [53]

    Yibo Wang, Yuzhe Tang, Kai Li, Wanning Ding, and Zhihua Yang. 2024. Under- standing Ethereum Mempool Security under Asymmetric{DoS} by Symbolized Stateful Fuzzing. In 33rd USENIX Security Symposium (USENIX Security 24) . 4747– 4764

    work page 2024

  54. [54]

    Shuzheng Wang, Yue Huang, Wenqin Zhang, Yuming Huang, Xuechao Wang, and Jing Tang. 2025. Private Order Flows and Builder Bidding Dynamics: The Road to Monopoly in Ethereum’s Block Building Market. In Proceedings of the ACM on Web Conference 2025. 2144–2157

    work page 2025

  55. [55]

    Sally Junsong Wang, Kexin Pei, and Junfeng Yang. 2024. Smartinv: Multimodal learning for smart contract invariant inference. In 2024 IEEE Symposium on Security and Privacy (SP) . IEEE, 2217–2235. xxx, April 28– 02, 2025, xxx Shuzheng Wang, Yue Huang, Zhuoer Xu, Yuming Huang and Jing Tang

    work page 2024

  56. [56]

    Wansen Wang, Wenchao Huang, Zhaoyi Meng, Yan Xiong, Fuyou Miao, Xianjin Fang, Caichang Tu, and Renjie Ji. 2023. Automated inference on financial security of Ethereum smart contracts. In 32nd USENIX Security Symposium (USENIX Security 23). 3367–3383

    work page 2023

  57. [57]

    Gavin Wood et al. 2014. Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151, 2014 (2014), 1–32

    2014

  58. [58]

    Yanling Wang, Jing Zhang, Shasha Guo, Hongzhi Yin, Cuiping Li, and Hong Chen. 2021. Decoupling representation learning and classification for gnn-based anomaly detection. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval . 1239–1248

    work page 2021

  59. [59]

    Zhiyuan Wei, Jing Sun, Zijiang Zhang, Xianhao Zhang, Meng Li, and Zhe Hou

    work page

  60. [60]

    arXiv preprint arXiv:2410.09381 (2024)

    LLM-SmartAudit: Advanced Smart Contract Vulnerability Detection. arXiv preprint arXiv:2410.09381 (2024)

    Pith/arXiv arXiv 2024

  61. [61]

    Sam Werner, Daniel Perez, Lewis Gudgeon, Ariah Klages-Mundt, Dominik Harz, and William Knottenbelt. 2022. Sok: Decentralized finance (defi). In Proceedings of the 4th ACM Conference on Advances in Financial Technologies . 30–46

    work page 2022

  62. [62]

    Shuo Yang, Xingwei Lin, Jiachi Chen, Qingyuan Zhong, Lei Xiao, Renke Huang, Yanlin Wang, and Zibin Zheng. 2024. Hyperion: Unveiling DApp Inconsis- tencies using LLM and Dataflow-Guided Symbolic Execution. arXiv preprint arXiv:2408.06037 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  63. [63]

    Siwei Wu, Zhou Yu, Dabao Wang, Yajin Zhou, Lei Wu, Haoyu Wang, and Xingliang Yuan. 2023. Defiranger: Detecting defi price manipulation attacks. IEEE Transactions on Dependable and Secure Computing 21, 4 (2023), 4147–4161

    work page 2023

  64. [64]

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018)

    Pith/arXiv arXiv 2018

  65. [65]

    Aviv Yaish, Kaihua Qin, Liyi Zhou, Aviv Zohar, and Arthur Gervais. 2024. Specula- tive{Denial-of-Service} Attacks In Ethereum. In33rd USENIX security symposium (USENIX Security 24). 3531–3548

    work page 2024

  66. [66]

    Shuo Yang, Jiachi Chen, and Zibin Zheng. 2023. Definition and detection of defects in nft smart contracts. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis . 373–384

    work page 2023

  67. [67]

    Zhuo Zhang, Zhiqiang Lin, Marcelo Morales, Xiangyu Zhang, and Kaiyuan Zhang

  68. [68]

    Haaroon Yousaf, George Kappos, and Sarah Meiklejohn. 2019. Tracing transac- tions across cryptocurrency ledgers. In28th USENIX Security Symposium (USENIX Security 19). 837–850

    work page 2019

  69. [69]

    Lei Yu, Zhirong Huang, Hang Yuan, Shiqi Cheng, Li Yang, Fengjun Zhang, Chenjie Shen, Jiajia Ma, Jingyuan Zhang, Junyi Lu, et al . 2025. Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection. Proceedings of the ACM on Software Engineering 2, ISSTA (2025), 182–205

    work page 2025

  70. [70]

    Qi Yuan, Baoying Huang, Jie Zhang, Jiajing Wu, Haonan Zhang, and Xi Zhang

    work page

  71. [72]

    Mengya Zhang, Xiaokuan Zhang, Yinqian Zhang, and Zhiqiang Lin. 2020. {TXSPECTOR}: Uncovering attacks in ethereum from transactions. In 29th USENIX Security Symposium (USENIX Security 20) . 2775–2792

    work page 2020

  72. [75]

    Shunfan Zhou, Zhemin Yang, Jie Xiang, Yinzhi Cao, and Yuan Zhang. 2020. An ever-evolving game: Evaluation of real-world attacks and defenses in ethereum ecosystem. In 29th USENIX Security Symposium (USENIX Security 20) . 2793–2810. Based on the user input ([user query]) and the retrieved context ([retrieved context]), generate a normalized scope for block...

    work page 2020

  73. [76]

    Focus on structural and semantic essence rather than copying any specific names from the input or context

    Using the syntax style demonstrated in the provided example, generate normalized JSON. Focus on structural and semantic essence rather than copying any specific names from the input or context

    work page

  74. [77]

    '$' denotes a symbolic variable, such as $address or $block for placeholders

    work page

  75. [78]

    MUST NOT replicate irrelevant details from the [retrieved context]

    work page

  76. [79]

    MUST ensure that only Ethereum-compatible contract addresses are returned

    work page

  77. [80]

    MUST normalize temporal expressions into block ranges using block height and block interval assumptions

    work page

  78. [81]

    contracts

    The output MUST NOT contain any elements not defined in the JSON schema. [User query]: {user_input} [Retrieved context]: {retrieved_context_topk} The output MUST be in the form of a JSON object: { "contracts":{"address": "0x..."}, "time": { "start_block": <integer or null>, "end_block": <integer or null> } } REMEMBER, the output must strictly conform to t...

    work page 2025

  79. [2020]

    In 2020 IEEE international symposium on circuits and systems (ISCAS)

    Detecting phishing scams on ethereum based on transaction records. In 2020 IEEE international symposium on circuits and systems (ISCAS) . IEEE, 1–5

    work page 2020

  80. [2023]

    In 32nd USENIX Security Symposium (USENIX Security 23)

    Your exploit is mine: Instantly synthesizing counterattack smart contract. In 32nd USENIX Security Symposium (USENIX Security 23) . 1757–1774

    work page

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.