Pith. sign in

REVIEW 5 major objections 5 minor 18 references

Decompiling Smart Contracts with a Large Language Model

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read A two-stage pipeline turns EVM bytecode into readable, semantically faithful Solidity.

desk verdict A useful hybrid decompiler with a large dataset, but the headline 0.82 similarity rests on a metric the paper never defines; the idea is sound and worth refereeing. read the letter →

arxiv 2506.19624 v1 pith:4M7HFI56 submitted 2025-06-24 cs.CR

classification cs.CR
keywords smartcontractdecompilationEVMbytecodethree-addresslargelanguagemodelsSoliditysemanticsimilaritysecurityauditinglow-rankfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ethereum has more than 78 million deployed smart contracts, yet under one percent are source-verified, so security review routinely has to start from raw bytecode. This paper argues that decompilation becomes tractable when split into two stages: static analysis that lifts EVM bytecode into three-address code, followed by a three-billion-parameter language model, fine-tuned on 238,446 function pairs, that rewrites the three-address code as natural Solidity. On a held-out set of 9,731 functions the pipeline achieves an average semantic similarity of 0.82 with the original source, with 78.3% of outputs above 0.8 similarity, and it preserves readability markers that conventional decompilers miss. If the claim holds, auditors, incident responders, and security researchers could read unverified contracts almost as fluently as they read open-source ones, closing the transparency gap on the largest blockchain.

What carries the argument

The load-bearing object is three-address code (TAC), an intermediate representation in which each instruction has at most three operands and data flow is explicit, for example temp1 = a + b; temp2 = c - d; result = temp1 * temp2. The paper treats TAC as the semantic bridge between bytecode and Solidity: it has intermediate entropy, preserves operand relationships and structured control flow, and gives the language model a translation task rather than a reconstruction task. The second mechanism is low-rank fine-tuning of a three-billion-parameter model; the ablation shows this stage is essential, because the base model's average semantic similarity drops by 45 percent and its output reverts to goto-heavy, generic code.

What would settle it

Compile a set of verified DeFi functions that use $10^{18}$ fixed-point scaling, nested storage mappings, and time-weighted reward calculations, decompile them with this pipeline, compile the generated Solidity, and run differential tests against the original bytecode on random and adversarial inputs; if behavioral mismatches recur and semantic similarity on that set stays near the 0.52 level the paper reports for its own staking case, the general claim of semantic fidelity fails for a common contract class.

Watch

Extended reading notes

Core claim

The paper's central claim is that combining a structured intermediate representation with a small specialized language model beats both rule-based decompilers and direct neural translation. The bytecode-to-TAC stage does the hard structural work of resolving stack operations, control flow, and function boundaries; the fine-tuned model then supplies what static analysis cannot, namely idiomatic variable names, signatures, and high-level constructs. The reported result is an average semantic similarity of 0.82 to original source and an edit-distance distribution concentrated below 0.4, with security-critical tokens such as require and msg.sender preserved within two percent of their original frequency. Two case studies show the practical effect: one recovers a token-enumeration function nearly unchanged, and another exposes a logic flaw in an unverified token-locking contract that allowed repeated withdrawals before the lock expiry.

Load-bearing premise

The conversion from EVM bytecode to three-address code preserves the semantic content of the original bytecode; anything the static-analysis stage drops, such as type information, storage structure, or arithmetic precision, cannot be recovered by the language model.

Editorial extensions

If this is right

  • Unverified contracts become practical audit targets: the decompiled output exposes require guards, access-control checks, and function signatures that auditors need to spot vulnerabilities.
  • The two-stage design decouples structural recovery from language generation, so improvements in static analysis should directly raise the quality ceiling of the neural stage.
  • Small, carefully fine-tuned models can perform specialized technical translation as well as or better than much larger generic models, reducing the compute needed for decompilation services.
  • Preserving security-critical tokens at near-original frequencies means automated scanners could run on decompiled code as a first-pass filter for dangerous patterns.
  • The approach suggests a reusable recipe: lift a low-level bytecode to a structured, low-entropy intermediate form and let a domain-tuned language model render it in the target language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's own staking-reward example scores only 0.52 and loses $10^{18}$ scaling and time-weighted reward logic, I infer that the pipeline is weakest precisely on high-value DeFi math; building a training subset that oversamples fixed-point arithmetic is a direct test and likely remedy.
  • The entropy argument implies the same TAC-plus-LLM recipe should transfer to other virtual machines with a readable high-level target, such as other EVM-compatible chains or WebAssembly; the paper leaves that as a natural next step rather than a demonstrated result.
  • A behavioral check, compiling the generated Solidity and differentially testing it against the original bytecode, would convert similarity scores into a guarantee of equivalent execution, which is the property auditors actually need.
  • The near-perfect frequency match of keywords like require may partly reflect token-level mimicry rather than verified semantics, so numerical constants and storage layouts still deserve manual review.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a hybrid smart-contract decompiler that first converts EVM bytecode into three-address code (TAC) via static analysis and then uses a LoRA-fine-tuned Llama-3.2-3B model to generate Solidity source. The authors construct a dataset of 238,446 TAC/Solidity function pairs, evaluate on 9,731 held-out functions, and report an average 'semantic similarity' of 0.82 with the original source, together with improved readability over traditional decompilers. They also present two case studies (one successful, one showing substantial semantic loss), an ablation showing the value of fine-tuning, entropy measurements over representations, and two security-audit examples. The main quantitative claim, however, is built on an undefined similarity metric, and the baseline comparison is not described.

Significance. If the quantitative claims survive a properly specified evaluation, the work is a useful contribution: it demonstrates that a small specialized LLM operating on a structured IR can produce readable Solidity from bytecode, and it provides a public dataset, a public system, and an honest failure analysis. The held-out evaluation, the ablation, and the real-world audit examples are strengths. The current bottleneck is evaluation rigor: until the semantic-similarity metric is defined, baseline decompilers are run on the same test set, and behavioral equivalence is checked, the headline 0.82 cannot be taken as evidence of semantic preservation. The approach is nonetheless plausible, and the artifacts make the requested fixes feasible.

major comments (5)
  1. [§4.1–4.2, Figs. 3–5] The paper's headline result ('average semantic similarity of 0.82') rests on a metric that is never defined. Sections 4.1–4.2 state that 'multiple complementary metrics' are used, and Figures 3–5 report similarity distributions, thresholds, and comparisons, but no formula, implementation, or reference for 'semantic similarity' (or for the 'normalized edit distance') is given. The number cannot be reproduced or interpreted: it is unknown whether it is token overlap, an embedding-based score, an AST match, or an execution-equivalence oracle. I request a precise definition, the computation code, and a worked example; without this, the central quantitative claim is unfalsifiable.
  2. [§4.2, Fig. 4] The claimed advantage over traditional decompilers is not verifiable. The text says traditional decompilers 'typically achieve such scores for only 40% to 50% of functions' (Fig. 4 caption) and exhibit edit distances 'concentrated in the 0.6–0.8 range' (§4.2), but no decompiler names, versions, configuration, or per-tool scores are provided, and no baseline results are reported on the same 9,731-function test set with the same metric. Without this, the comparison is anecdotal. Please run at least one standard smart-contract decompiler on the same held-out functions and report a table of scores under the same metric.
  3. [§5.2, Fig. 9] The staking-rewards case study undermines the validity of the semantic-similarity metric: the decompiled function returns 1 or 0 instead of computing the time-weighted reward, yet it receives a similarity score of 0.52. A metric that scores a behaviorally wrong function this high is not measuring semantic preservation in the security-relevant sense. This does not disprove the approach, but it shows the headline 0.82 must be complemented by an execution-equivalence or behavioral test on representative functions before the paper can claim semantic fidelity.
  4. [§3.1, §2.3] The pipeline's upper bound is set by the bytecode-to-TAC stage, which the paper asserts 'preserves the semantic content of the original bytecode' (§3.1) without proof or evaluation. The paper itself documents a failure mode in §5.2 where the TAC-derived output lost 1e18 scaling and time-weighted logic, and §2.3 lists type recovery, storage patterns, and arithmetic precision as known losses during compilation. I ask for an explicit evaluation of the TAC stage (e.g., round-trip execution equivalence between bytecode and TAC, or a manual audit of a random sample) so readers can separate TAC losses from LLM generation errors.
  5. [§4.1] The evaluation reports point estimates over 9,731 functions without confidence intervals, and it is not clear whether the holdout was performed at the contract level or the function level. If two functions from the same contract appear in both training and test splits, the model can memorize contract-specific idioms, inflating the reported similarity. Please report bootstrap intervals for the averages and clarify (or implement) a contract-disjoint split.
minor comments (5)
  1. [Figs. 3 and 4] Figure 3 (left) states that 82.5% of functions have normalized edit distance below 0.4, while Figure 4 (left) says 60% of functions achieve distances under this threshold; these statements are mutually inconsistent and should be reconciled.
  2. [§8] The entropy analysis in Section 8 is not load-bearing but is presented without methodological details (tokenizer, corpus subset, estimation method); please add a short description or clearly label it as preliminary and speculative.
  3. [§4.1] The paper claims the evaluation dataset and metrics calculation code are publicly available, but no URL or repository identifier is given; please provide a permanent link.
  4. [Figs. 10 and 11] The code listings in Figures 10 and 11 contain line-break and arrow artifacts (e.g., '→' and stray commas) that make the examples hard to read; please replace them with clean listings.
  5. [Abstract and §1] The phrases 'for the first time' and 'pioneering' should be qualified, since the related-work section itself lists LLM-based decompilers such as LLM4Decompile and DeGPT; a systematic comparison with these systems is needed before claiming priority.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: the central claim is an empirical held-out evaluation of a fine-tuned model, and no load-bearing step reduces by construction to its own inputs.

full rationale

The paper's pipeline is bytecode -> three-address code (TAC) -> generated Solidity, with a Llama-3.2-3B model fine-tuned on 238,446 TAC-to-Solidity pairs and evaluated on 9,731 held-out functions. The headline result (average semantic similarity 0.82) is a measured test-set statistic, not a quantity defined in terms of the model's own training fit or a fitted parameter renamed as a prediction. No equation in the paper defines the evaluated quantity as an identity with an input, and no free parameter is tuned to force the reported number. The TAC conversion is a design premise ('This transformation preserves the semantic content of the original bytecode'), and the staking case study openly reports a 0.52 score where functionality was lost; this is evidence of a limitation or an evaluation-metric concern, not circularity. The entropy analysis in Section 8 is speculative and does not feed back into the evaluation or the derivation of the reported results. The only substantively serious issue is that the paper nowhere defines the semantic-similarity metric, making the quantitative claim hard to reproduce; however, under the stipulated rules, a missing definition is not a circular step because there is no exhibited reduction of the claimed result to the paper's inputs. The paper also does not rely on a load-bearing self-citation chain: the related-work citations of other decompilers are contextual and external, and no uniqueness theorem or prior author-derived ansatz is invoked to force the chosen design. Overall, the derivation chain is self-contained as an empirical systems evaluation, and no specific circularity step can be quoted.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central results rest on two premises: the fidelity of the bytecode-to-TAC stage and the quality or distribution match of the paired training data. The listed free parameters are model hyperparameters chosen without systematic search or ablation. No new entities are introduced.

free parameters (4)
  • LoRA rank = 16
    Chosen for fine-tuning Llama-3.2-3B; the exact value affects model capacity and output quality.
  • LoRA target modules = query, key, value, projection layers
    Choice of which attention layers to adapt; affects learned representation.
  • Maximum sequence length = 20,000 tokens
    Truncation threshold that can drop code content for very long functions; stated in Section 3.3.
  • Learning rate schedule
    AdamW with warmup and linear decay is mentioned, but exact values are absent, impacting reproducibility.
assumptions (4)
  • domain assumption EVM bytecode-to-TAC conversion via static analysis preserves all semantic content of the original bytecode.
    Section 3.1 states the transformation 'preserves the semantic content of the original bytecode'; if false, decompilation cannot be semantically faithful.
  • domain assumption The training pairs (TAC, Solidity) from verified contracts are correctly aligned and representative of real-world contracts.
    Section 3.2 describes the dataset; misalignment or distribution shift would invalidate the evaluation.
  • domain assumption The semantic similarity metric reflects true semantic equivalence.
    Section 4.1 mentions metrics but never defines them; the headline 0.82 depends on this assumption.
  • domain assumption Base Llama-3.2-3B serves as a valid starting point whose pretraining provides useful code understanding.
    The paper relies on the base model's pretrained capabilities without auditing them; this is a background assumption for the fine-tuning approach.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decompiling Smart Contracts with a Large Language Model." pith.science (2026). https://pith.science/paper/4M7HFI56

@misc{pith2026250619624,
  author       = {Pith},
  title        = {Pith review of: Decompiling Smart Contracts with a Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4M7HFI56}},
  note         = {Machine review of arXiv:2506.19624}
}
read the original abstract

The widespread lack of broad source code verification on blockchain explorers such as Etherscan, where despite 78,047,845 smart contracts deployed on Ethereum (as of May 26, 2025), a mere 767,520 (< 1%) are open source, presents a severe impediment to blockchain security. This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode, a fundamental research challenge with direct implications for identifying vulnerabilities and understanding malicious behavior. Prevailing decompilers struggle to reverse bytecode in a readable manner, often yielding convoluted code that critically hampers vulnerability analysis and thwarts efforts to dissect contract functionalities for security auditing. This paper addresses this challenge by introducing a pioneering decompilation pipeline that, for the first time, successfully leverages Large Language Models (LLMs) to transform Ethereum Virtual Machine (EVM) bytecode into human-readable and semantically faithful Solidity code. Our novel methodology first employs rigorous static program analysis to convert bytecode into a structured three-address code (TAC) representation. This intermediate representation then guides a Llama-3.2-3B model, specifically fine-tuned on a comprehensive dataset of 238,446 TAC-to-Solidity function pairs, to generate high-quality Solidity. This approach uniquely recovers meaningful variable names, intricate control flow, and precise function signatures. Our extensive empirical evaluation demonstrates a significant leap beyond traditional decompilers, achieving an average semantic similarity of 0.82 with original source and markedly superior readability. The practical viability and effectiveness of our research are demonstrated through its implementation in a publicly accessible system, available at https://evmdecompiler.com.

Figures

Figures reproduced from arXiv: 2506.19624 by the authors.

Figure 1
Figure 1. High-level visual overview of our smart contract decompilation pipeline. We first convert EVM bytecode into a structured three-address code [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Distribution of code length differences between original and de [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Quality metrics distributions demonstrating our system’s decompilation accuracy. Left: Normalized edit distance distribution showing strong [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Cumulative distribution functions (CDFs) revealing the reliability and consistency of our decompilation approach. Left: Edit distance CDF showing [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Detailed distribution analysis through complementary visualization techniques. Left: Edit distance box plot revealing a median of 0.3 with tight [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Token frequency analysis comparing language construct usage between original and decompiled code. Left: Ground truth token distribution [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Relationship between semantic similarity and edit distance across [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Comparison of original and decompiled NFT enumeration function implementations. Note the preservation of structure and variable naming, [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Comparison of original and decompiled staking reward calculation implementations. The decompiled version shows significant simplification, [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 11
Figure 11. Figure 11: Key vulnerable functions decompiled from the MEV bot con [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 10
Figure 10. Figure 10: Decompiled unlockToken function from the unverified Dx Protocol contract. Our decompiler successfully reconstructed the code, exposing a critical vulnerability that allowed repeated withdrawals before the official unlock time. rity. By transforming opaque bytecode int…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 16 canonical work pages

  1. [1]

    Dx Protocol Vulnerability Disclosure: Mitigation of a $5,200,000 Smart Contract Exploit,

    Lockigod, “Dx Protocol Vulnerability Disclosure: Mitigation of a $5,200,000 Smart Contract Exploit,” https://blog.decurity.io/ dx-protocol-vulnerability-disclosure-bddff88aeb1d, July 2023

  2. [2]

    MEV Bot Exploit Transaction Anal- ysis,

    BlockSec, “MEV Bot Exploit Transaction Anal- ysis,” https://app.blocksec.com/explorer/tx/bsc/ 0xec96f2afe1fe800fbaaf1eeac57565f3699f0465cec78c683fdad2a1f0aa0641, April 2025

  3. [3]

    Decompilation of binary programs,

    C. Cifuentes and K. J. Gough, “Decompilation of binary programs,” Software: Practice and Experience , vol. 25, no. 7, pp. 811–829, 1995

  4. [4]

    Decompiling java using staged encapsulation,

    J. Miecznikowski and L. Hendren, “Decompiling java using staged encapsulation,” in Proceedings Eighth Working Conference on Re- verse Engineering . IEEE, 2001, pp. 368–374

  5. [5]

    Native x86 decom- pilation using{Semantics-Preserving} structural analysis and iterative {Control-Flow} structuring,

    D. Brumley, J. Lee, E. J. Schwartz, and M. Woo, “Native x86 decom- pilation using{Semantics-Preserving} structural analysis and iterative {Control-Flow} structuring,” in 22nd USENIX Security Symposium (USENIX Security 13) , 2013, pp. 353–368

  6. [6]

    Helping johnny to analyze malware: A usability-optimized decompiler and malware analysis user study,

    K. Yakdan, S. Dechand, E. Gerhards-Padilla, and M. Smith, “Helping johnny to analyze malware: A usability-optimized decompiler and malware analysis user study,” in 2016 IEEE Symposium on Security and Privacy (SP) . IEEE, 2016, pp. 158–177

  7. [7]

    No more gotos: Decompilation using pattern-independent control-flow structuring and semantic-preserving transformations

    K. Yakdan, S. Eschweiler, E. Gerhards-Padilla, and M. Smith, “No more gotos: Decompilation using pattern-independent control-flow structuring and semantic-preserving transformations.” in NDSS. Cite- seer, 2015

  8. [8]

    Erays: reverse engineering ethereum’s opaque smart contracts,

    Y . Zhou, D. Kumar, S. Bakshi, J. Mason, A. Miller, and M. Bailey, “Erays: reverse engineering ethereum’s opaque smart contracts,” in 27th USENIX security symposium (USENIX Security 18) , 2018, pp. 1371–1385

Show all 18 references
  1. [9]

    Gigahorse: thorough, declarative decompilation of smart contracts,

    N. Grech, L. Brent, B. Scholz, and Y . Smaragdakis, “Gigahorse: thorough, declarative decompilation of smart contracts,” in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 2019, pp. 1176–1186

  2. [10]

    Elipmoc: Advanced decompilation of ethereum smart contracts,

    N. Grech, S. Lagouvardos, I. Tsatiris, and Y . Smaragdakis, “Elipmoc: Advanced decompilation of ethereum smart contracts,” Proceedings of the ACM on Programming Languages , vol. 6, no. OOPSLA1, pp. 1–27, 2022

  3. [11]

    An empirical study of smart contract decompilers,

    X. Liu, B. Hua, Y . Wang, and Z. Pan, “An empirical study of smart contract decompilers,” in 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 2023, pp. 1–12

  4. [12]

    Using recurrent neural net- works for decompilation,

    D. S. Katz, J. Ruchti, and E. Schulte, “Using recurrent neural net- works for decompilation,” in 2018 IEEE 25th International Confer- ence on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 2018, pp. 346–356

  5. [13]

    Coda: An end-to-end neural program decompiler,

    C. Fu, H. Chen, H. Liu, X. Chen, Y . Tian, F. Koushanfar, and J. Zhao, “Coda: An end-to-end neural program decompiler,”Advances in Neural Information Processing Systems , vol. 32, 2019

  6. [14]

    Boosting neural networks to decompile optimized binaries,

    Y . Cao, R. Liang, K. Chen, and P. Hu, “Boosting neural networks to decompile optimized binaries,” in Proceedings of the 38th Annual Computer Security Applications Conference , 2022, pp. 508–518

  7. [15]

    {DnD}: A{Cross- Architecture} deep neural network decompiler,

    R. Wu, T. Kim, D. J. Tian, A. Bianchi, and D. Xu, “{DnD}: A{Cross- Architecture} deep neural network decompiler,” in 31st USENIX Security Symposium (USENIX Security 22) , 2022, pp. 2135–2152

  8. [16]

    Decompiling x86 deep neural network executables,

    Z. Liu, Y . Yuan, S. Wang, X. Xie, and L. Ma, “Decompiling x86 deep neural network executables,” in 32nd USENIX Security Symposium (USENIX Security 23) , 2023, pp. 7357–7374

  9. [17]

    Degpt: Optimizing decompiler output with llm,

    P. Hu, R. Liang, and K. Chen, “Degpt: Optimizing decompiler output with llm,” in Proceedings 2024 Network and Distributed System Se- curity Symposium (2024). https://api. semanticscholar . org/CorpusID, vol. 267622140, 2024

  10. [18]

    Llm4decompile: Decom- piling binary code with large language models,

    H. Tan, Q. Luo, J. Li, and Y . Zhang, “Llm4decompile: Decom- piling binary code with large language models,” arXiv preprint arXiv:2403.05286, 2024. 14

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.