REVIEW 5 major objections 5 minor 18 references
Decompiling Smart Contracts with a Large Language Model
T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A two-stage pipeline turns EVM bytecode into readable, semantically faithful Solidity.
desk verdict A useful hybrid decompiler with a large dataset, but the headline 0.82 similarity rests on a metric the paper never defines; the idea is sound and worth refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is three-address code (TAC), an intermediate representation in which each instruction has at most three operands and data flow is explicit, for example temp1 = a + b; temp2 = c - d; result = temp1 * temp2. The paper treats TAC as the semantic bridge between bytecode and Solidity: it has intermediate entropy, preserves operand relationships and structured control flow, and gives the language model a translation task rather than a reconstruction task. The second mechanism is low-rank fine-tuning of a three-billion-parameter model; the ablation shows this stage is essential, because the base model's average semantic similarity drops by 45 percent and its output reverts to goto-heavy, generic code.
What would settle it
Compile a set of verified DeFi functions that use $10^{18}$ fixed-point scaling, nested storage mappings, and time-weighted reward calculations, decompile them with this pipeline, compile the generated Solidity, and run differential tests against the original bytecode on random and adversarial inputs; if behavioral mismatches recur and semantic similarity on that set stays near the 0.52 level the paper reports for its own staking case, the general claim of semantic fidelity fails for a common contract class.
Extended reading notes
Core claim
The paper's central claim is that combining a structured intermediate representation with a small specialized language model beats both rule-based decompilers and direct neural translation. The bytecode-to-TAC stage does the hard structural work of resolving stack operations, control flow, and function boundaries; the fine-tuned model then supplies what static analysis cannot, namely idiomatic variable names, signatures, and high-level constructs. The reported result is an average semantic similarity of 0.82 to original source and an edit-distance distribution concentrated below 0.4, with security-critical tokens such as require and msg.sender preserved within two percent of their original frequency. Two case studies show the practical effect: one recovers a token-enumeration function nearly unchanged, and another exposes a logic flaw in an unverified token-locking contract that allowed repeated withdrawals before the lock expiry.
Load-bearing premise
The conversion from EVM bytecode to three-address code preserves the semantic content of the original bytecode; anything the static-analysis stage drops, such as type information, storage structure, or arithmetic precision, cannot be recovered by the language model.
Editorial extensions
If this is right
- Unverified contracts become practical audit targets: the decompiled output exposes require guards, access-control checks, and function signatures that auditors need to spot vulnerabilities.
- The two-stage design decouples structural recovery from language generation, so improvements in static analysis should directly raise the quality ceiling of the neural stage.
- Small, carefully fine-tuned models can perform specialized technical translation as well as or better than much larger generic models, reducing the compute needed for decompilation services.
- Preserving security-critical tokens at near-original frequencies means automated scanners could run on decompiled code as a first-pass filter for dangerous patterns.
- The approach suggests a reusable recipe: lift a low-level bytecode to a structured, low-entropy intermediate form and let a domain-tuned language model render it in the target language.
Reading between the lines
- Because the paper's own staking-reward example scores only 0.52 and loses $10^{18}$ scaling and time-weighted reward logic, I infer that the pipeline is weakest precisely on high-value DeFi math; building a training subset that oversamples fixed-point arithmetic is a direct test and likely remedy.
- The entropy argument implies the same TAC-plus-LLM recipe should transfer to other virtual machines with a readable high-level target, such as other EVM-compatible chains or WebAssembly; the paper leaves that as a natural next step rather than a demonstrated result.
- A behavioral check, compiling the generated Solidity and differentially testing it against the original bytecode, would convert similarity scores into a guarantee of equivalent execution, which is the property auditors actually need.
- The near-perfect frequency match of keywords like require may partly reflect token-level mimicry rather than verified semantics, so numerical constants and storage layouts still deserve manual review.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a hybrid smart-contract decompiler that first converts EVM bytecode into three-address code (TAC) via static analysis and then uses a LoRA-fine-tuned Llama-3.2-3B model to generate Solidity source. The authors construct a dataset of 238,446 TAC/Solidity function pairs, evaluate on 9,731 held-out functions, and report an average 'semantic similarity' of 0.82 with the original source, together with improved readability over traditional decompilers. They also present two case studies (one successful, one showing substantial semantic loss), an ablation showing the value of fine-tuning, entropy measurements over representations, and two security-audit examples. The main quantitative claim, however, is built on an undefined similarity metric, and the baseline comparison is not described.
Significance. If the quantitative claims survive a properly specified evaluation, the work is a useful contribution: it demonstrates that a small specialized LLM operating on a structured IR can produce readable Solidity from bytecode, and it provides a public dataset, a public system, and an honest failure analysis. The held-out evaluation, the ablation, and the real-world audit examples are strengths. The current bottleneck is evaluation rigor: until the semantic-similarity metric is defined, baseline decompilers are run on the same test set, and behavioral equivalence is checked, the headline 0.82 cannot be taken as evidence of semantic preservation. The approach is nonetheless plausible, and the artifacts make the requested fixes feasible.
major comments (5)
- [§4.1–4.2, Figs. 3–5] The paper's headline result ('average semantic similarity of 0.82') rests on a metric that is never defined. Sections 4.1–4.2 state that 'multiple complementary metrics' are used, and Figures 3–5 report similarity distributions, thresholds, and comparisons, but no formula, implementation, or reference for 'semantic similarity' (or for the 'normalized edit distance') is given. The number cannot be reproduced or interpreted: it is unknown whether it is token overlap, an embedding-based score, an AST match, or an execution-equivalence oracle. I request a precise definition, the computation code, and a worked example; without this, the central quantitative claim is unfalsifiable.
- [§4.2, Fig. 4] The claimed advantage over traditional decompilers is not verifiable. The text says traditional decompilers 'typically achieve such scores for only 40% to 50% of functions' (Fig. 4 caption) and exhibit edit distances 'concentrated in the 0.6–0.8 range' (§4.2), but no decompiler names, versions, configuration, or per-tool scores are provided, and no baseline results are reported on the same 9,731-function test set with the same metric. Without this, the comparison is anecdotal. Please run at least one standard smart-contract decompiler on the same held-out functions and report a table of scores under the same metric.
- [§5.2, Fig. 9] The staking-rewards case study undermines the validity of the semantic-similarity metric: the decompiled function returns 1 or 0 instead of computing the time-weighted reward, yet it receives a similarity score of 0.52. A metric that scores a behaviorally wrong function this high is not measuring semantic preservation in the security-relevant sense. This does not disprove the approach, but it shows the headline 0.82 must be complemented by an execution-equivalence or behavioral test on representative functions before the paper can claim semantic fidelity.
- [§3.1, §2.3] The pipeline's upper bound is set by the bytecode-to-TAC stage, which the paper asserts 'preserves the semantic content of the original bytecode' (§3.1) without proof or evaluation. The paper itself documents a failure mode in §5.2 where the TAC-derived output lost 1e18 scaling and time-weighted logic, and §2.3 lists type recovery, storage patterns, and arithmetic precision as known losses during compilation. I ask for an explicit evaluation of the TAC stage (e.g., round-trip execution equivalence between bytecode and TAC, or a manual audit of a random sample) so readers can separate TAC losses from LLM generation errors.
- [§4.1] The evaluation reports point estimates over 9,731 functions without confidence intervals, and it is not clear whether the holdout was performed at the contract level or the function level. If two functions from the same contract appear in both training and test splits, the model can memorize contract-specific idioms, inflating the reported similarity. Please report bootstrap intervals for the averages and clarify (or implement) a contract-disjoint split.
minor comments (5)
- [Figs. 3 and 4] Figure 3 (left) states that 82.5% of functions have normalized edit distance below 0.4, while Figure 4 (left) says 60% of functions achieve distances under this threshold; these statements are mutually inconsistent and should be reconciled.
- [§8] The entropy analysis in Section 8 is not load-bearing but is presented without methodological details (tokenizer, corpus subset, estimation method); please add a short description or clearly label it as preliminary and speculative.
- [§4.1] The paper claims the evaluation dataset and metrics calculation code are publicly available, but no URL or repository identifier is given; please provide a permanent link.
- [Figs. 10 and 11] The code listings in Figures 10 and 11 contain line-break and arrow artifacts (e.g., '→' and stray commas) that make the examples hard to read; please replace them with clean listings.
- [Abstract and §1] The phrases 'for the first time' and 'pioneering' should be qualified, since the related-work section itself lists LLM-based decompilers such as LLM4Decompile and DeGPT; a systematic comparison with these systems is needed before claiming priority.
Circularity Check
No significant circularity found: the central claim is an empirical held-out evaluation of a fine-tuned model, and no load-bearing step reduces by construction to its own inputs.
full rationale
The paper's pipeline is bytecode -> three-address code (TAC) -> generated Solidity, with a Llama-3.2-3B model fine-tuned on 238,446 TAC-to-Solidity pairs and evaluated on 9,731 held-out functions. The headline result (average semantic similarity 0.82) is a measured test-set statistic, not a quantity defined in terms of the model's own training fit or a fitted parameter renamed as a prediction. No equation in the paper defines the evaluated quantity as an identity with an input, and no free parameter is tuned to force the reported number. The TAC conversion is a design premise ('This transformation preserves the semantic content of the original bytecode'), and the staking case study openly reports a 0.52 score where functionality was lost; this is evidence of a limitation or an evaluation-metric concern, not circularity. The entropy analysis in Section 8 is speculative and does not feed back into the evaluation or the derivation of the reported results. The only substantively serious issue is that the paper nowhere defines the semantic-similarity metric, making the quantitative claim hard to reproduce; however, under the stipulated rules, a missing definition is not a circular step because there is no exhibited reduction of the claimed result to the paper's inputs. The paper also does not rely on a load-bearing self-citation chain: the related-work citations of other decompilers are contextual and external, and no uniqueness theorem or prior author-derived ansatz is invoked to force the chosen design. Overall, the derivation chain is self-contained as an empirical systems evaluation, and no specific circularity step can be quoted.
Assumptions & free parameters
free parameters (4)
- LoRA rank =
16
- LoRA target modules =
query, key, value, projection layers
- Maximum sequence length =
20,000 tokens
- Learning rate schedule
assumptions (4)
- domain assumption EVM bytecode-to-TAC conversion via static analysis preserves all semantic content of the original bytecode.
- domain assumption The training pairs (TAC, Solidity) from verified contracts are correctly aligned and representative of real-world contracts.
- domain assumption The semantic similarity metric reflects true semantic equivalence.
- domain assumption Base Llama-3.2-3B serves as a valid starting point whose pretraining provides useful code understanding.
Cite this review
Pith. "Pith review of Decompiling Smart Contracts with a Large Language Model." pith.science (2026). https://pith.science/paper/4M7HFI56
@misc{pith2026250619624,
author = {Pith},
title = {Pith review of: Decompiling Smart Contracts with a Large Language Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/4M7HFI56}},
note = {Machine review of arXiv:2506.19624}
}
read the original abstract
The widespread lack of broad source code verification on blockchain explorers such as Etherscan, where despite 78,047,845 smart contracts deployed on Ethereum (as of May 26, 2025), a mere 767,520 (< 1%) are open source, presents a severe impediment to blockchain security. This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode, a fundamental research challenge with direct implications for identifying vulnerabilities and understanding malicious behavior. Prevailing decompilers struggle to reverse bytecode in a readable manner, often yielding convoluted code that critically hampers vulnerability analysis and thwarts efforts to dissect contract functionalities for security auditing. This paper addresses this challenge by introducing a pioneering decompilation pipeline that, for the first time, successfully leverages Large Language Models (LLMs) to transform Ethereum Virtual Machine (EVM) bytecode into human-readable and semantically faithful Solidity code. Our novel methodology first employs rigorous static program analysis to convert bytecode into a structured three-address code (TAC) representation. This intermediate representation then guides a Llama-3.2-3B model, specifically fine-tuned on a comprehensive dataset of 238,446 TAC-to-Solidity function pairs, to generate high-quality Solidity. This approach uniquely recovers meaningful variable names, intricate control flow, and precise function signatures. Our extensive empirical evaluation demonstrates a significant leap beyond traditional decompilers, achieving an average semantic similarity of 0.82 with original source and markedly superior readability. The practical viability and effectiveness of our research are demonstrated through its implementation in a publicly accessible system, available at https://evmdecompiler.com.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Dx Protocol Vulnerability Disclosure: Mitigation of a $5,200,000 Smart Contract Exploit,
Lockigod, “Dx Protocol Vulnerability Disclosure: Mitigation of a $5,200,000 Smart Contract Exploit,” https://blog.decurity.io/ dx-protocol-vulnerability-disclosure-bddff88aeb1d, July 2023
work page 2023
-
[2]
MEV Bot Exploit Transaction Anal- ysis,
BlockSec, “MEV Bot Exploit Transaction Anal- ysis,” https://app.blocksec.com/explorer/tx/bsc/ 0xec96f2afe1fe800fbaaf1eeac57565f3699f0465cec78c683fdad2a1f0aa0641, April 2025
work page 2025
-
[3]
Decompilation of binary programs,
C. Cifuentes and K. J. Gough, “Decompilation of binary programs,” Software: Practice and Experience , vol. 25, no. 7, pp. 811–829, 1995
work page 1995
-
[4]
Decompiling java using staged encapsulation,
J. Miecznikowski and L. Hendren, “Decompiling java using staged encapsulation,” in Proceedings Eighth Working Conference on Re- verse Engineering . IEEE, 2001, pp. 368–374
work page 2001
-
[5]
D. Brumley, J. Lee, E. J. Schwartz, and M. Woo, “Native x86 decom- pilation using{Semantics-Preserving} structural analysis and iterative {Control-Flow} structuring,” in 22nd USENIX Security Symposium (USENIX Security 13) , 2013, pp. 353–368
work page 2013
-
[6]
Helping johnny to analyze malware: A usability-optimized decompiler and malware analysis user study,
K. Yakdan, S. Dechand, E. Gerhards-Padilla, and M. Smith, “Helping johnny to analyze malware: A usability-optimized decompiler and malware analysis user study,” in 2016 IEEE Symposium on Security and Privacy (SP) . IEEE, 2016, pp. 158–177
work page 2016
-
[7]
K. Yakdan, S. Eschweiler, E. Gerhards-Padilla, and M. Smith, “No more gotos: Decompilation using pattern-independent control-flow structuring and semantic-preserving transformations.” in NDSS. Cite- seer, 2015
work page 2015
-
[8]
Erays: reverse engineering ethereum’s opaque smart contracts,
Y . Zhou, D. Kumar, S. Bakshi, J. Mason, A. Miller, and M. Bailey, “Erays: reverse engineering ethereum’s opaque smart contracts,” in 27th USENIX security symposium (USENIX Security 18) , 2018, pp. 1371–1385
work page 2018
Show all 18 references
-
[9]
Gigahorse: thorough, declarative decompilation of smart contracts,
N. Grech, L. Brent, B. Scholz, and Y . Smaragdakis, “Gigahorse: thorough, declarative decompilation of smart contracts,” in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 2019, pp. 1176–1186
2019
-
[10]
Elipmoc: Advanced decompilation of ethereum smart contracts,
N. Grech, S. Lagouvardos, I. Tsatiris, and Y . Smaragdakis, “Elipmoc: Advanced decompilation of ethereum smart contracts,” Proceedings of the ACM on Programming Languages , vol. 6, no. OOPSLA1, pp. 1–27, 2022
2022
-
[11]
An empirical study of smart contract decompilers,
X. Liu, B. Hua, Y . Wang, and Z. Pan, “An empirical study of smart contract decompilers,” in 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 2023, pp. 1–12
2023
-
[12]
Using recurrent neural net- works for decompilation,
D. S. Katz, J. Ruchti, and E. Schulte, “Using recurrent neural net- works for decompilation,” in 2018 IEEE 25th International Confer- ence on Software Analysis, Evolution and Reengineering (SANER) . IEEE, 2018, pp. 346–356
2018
-
[13]
Coda: An end-to-end neural program decompiler,
C. Fu, H. Chen, H. Liu, X. Chen, Y . Tian, F. Koushanfar, and J. Zhao, “Coda: An end-to-end neural program decompiler,”Advances in Neural Information Processing Systems , vol. 32, 2019
2019
-
[14]
Boosting neural networks to decompile optimized binaries,
Y . Cao, R. Liang, K. Chen, and P. Hu, “Boosting neural networks to decompile optimized binaries,” in Proceedings of the 38th Annual Computer Security Applications Conference , 2022, pp. 508–518
2022
-
[15]
{DnD}: A{Cross- Architecture} deep neural network decompiler,
R. Wu, T. Kim, D. J. Tian, A. Bianchi, and D. Xu, “{DnD}: A{Cross- Architecture} deep neural network decompiler,” in 31st USENIX Security Symposium (USENIX Security 22) , 2022, pp. 2135–2152
2022
-
[16]
Decompiling x86 deep neural network executables,
Z. Liu, Y . Yuan, S. Wang, X. Xie, and L. Ma, “Decompiling x86 deep neural network executables,” in 32nd USENIX Security Symposium (USENIX Security 23) , 2023, pp. 7357–7374
2023
-
[17]
Degpt: Optimizing decompiler output with llm,
P. Hu, R. Liang, and K. Chen, “Degpt: Optimizing decompiler output with llm,” in Proceedings 2024 Network and Distributed System Se- curity Symposium (2024). https://api. semanticscholar . org/CorpusID, vol. 267622140, 2024
2024
-
[18]
Llm4decompile: Decom- piling binary code with large language models,
H. Tan, Q. Luo, J. Li, and Y . Zhang, “Llm4decompile: Decom- piling binary code with large language models,” arXiv preprint arXiv:2403.05286, 2024. 14
2024 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.