SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework

Hanan EL Bakkali; Oualid Zaazaa

REVIEW 4 major objections 4 minor 67 references

SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A tuned ChatGPT can write smart-contract detection rules with 91.1% exact match.

desk verdict Useful prompt-design data, but the headline claim of improved detection accuracy is unmeasured; treat the 92% as rule imitation, not detection performance. read the letter →

arxiv 2411.19234 v1 pith:C3AESY3J submitted 2024-11-28 cs.CR cs.AI

Oualid Zaazaa , Hanan El Bakkali This is my paper · ORCID

classification cs.CRcs.AI

keywords smartcontractvulnerabilitydetectionlargelanguagemodelsChatGPTin-contextlearningstaticanalysisSolidityexactmatch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the bottleneck in rule-based smart-contract analyzers—experts hand-writing detection conditions for each vulnerability—can be removed by letting a large language model generate those conditions. It presents SmartLLMSentry, a modular static analyzer whose new detectors are produced by ChatGPT rather than written by hand. The authors report that with enough training examples and a carefully structured prompt, the model reproduces reference detector conditions with 91.1% exact-match accuracy on a 38-snippet test set, and that GPT-3.5 outperforms GPT-4. If true, this would let a security tool absorb newly discovered vulnerability classes from a small set of vulnerable code instances, reducing the expert effort that currently slows rule integration.

What carries the argument

The generative detector condition is the central object: a single `if` statement written against Solidity-ast and TypeScript that the analyzer runs to flag a vulnerability. The best-performing prompt, Prcbi, combines a security-auditor role, the Solidity-ast and TypeScript context, a requirement to output only the condition, and a warning that AST fields such as nodeType vary across expressions. The evaluation metric is Exact Match, defined as generated code having the same logic or the same syntax as the reference snippet. The framework surrounds this with a validator that drops detectors scoring below 80% and an integrator that labels accepted detectors as generated.

What would settle it

Take the best generated detector conditions (Prcbi prompt, 112 training examples) and run them against a held-out corpus of real deployed smart contracts with known vulnerabilities; if exact-match accuracy stays high while precision and recall on those contracts are no better than the hand-written baseline, the central claim that this improves vulnerability detection is falsified.

Watch

Extended reading notes

Core claim

The central claim is that ChatGPT can generate valid detector conditions for smart-contract vulnerabilities, and that these conditions can be integrated into the SmartLLMSentry static analyzer without continual expert rule authoring. In the paper's setup, a ChatGPT model is trained on JSONL examples of five vulnerability types and prompted to emit only an `if` condition against the Solidity AST and TypeScript APIs. With 112 training examples and the most detailed prompt, the paper reports an exact-match accuracy of 91.1% (the results table shows 92.1% for the best configuration). The same experiments show GPT-3.5 consistently beating GPT-4, and more training data helping GPT-3.5 while hurting GPT-4. The paper also claims the framework's validator and integrator loop can reject weak detectors and insert accepted ones into the analyzer.

Load-bearing premise

The whole evaluation rests on treating exact match to a reference code snippet as the measure of detector quality, without showing that a high exact-match score improves precision or recall on real vulnerable contracts.

Editorial extensions

If this is right

A newly discovered vulnerability class could be added to SmartLLMSentry by collecting a few dozen vulnerable instances and letting ChatGPT propose the detector condition, instead of asking an expert to write it.
Prompt content matters as much as model choice: adding role, AST context, output constraints, and AST-variation warnings raised GPT-3's exact match from 36.8% with the basic prompt to 89.5% at 100 training examples.
More training data helps the GPT-3 model (from 89.5% to 92.1% exact match for Prcbi) but not GPT-4, so the choice of model and the amount of data interact.
Detector generation is a continuous loop rather than a one-shot output: generated conditions pass through a validator with an 80% accuracy threshold before integration, and integrated rules are labeled for debugging.
The five studied classes—array length manipulation, hardcoded gas, transaction order dependence, locked money, and improper exception handling—expand the analyzer's coverage to types the authors say were previously underexplored.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The exact-match score is a proxy for detector usefulness; a condition that matches a reference snippet could still miss real-world variants or flag safe code, so the practical gain in precision and recall on deployed contracts is not established by this paper.
A direct test of the framework would run the best generated detectors against a labeled corpus of real contracts and compare true-positive and false-positive rates with the hand-written baseline; that comparison is the natural next validation step.
The five vulnerability classes are relatively rare in the SWE taxonomy; applying the same prompt pipeline to high-prevalence issues such as reentrancy would indicate whether the result transfers beyond the selected classes.
Because the GPT-4 decline with more data rests on a single 38-example test set and one fine-tuning run per configuration, the model ranking could change with more seeds and a larger evaluation set.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague — this is a small empirical study worth knowing about, but the headline claim outruns the evidence. The genuinely useful part is the prompt-design comparison: four prompts, two fine-tuned models, two training-set sizes, on five vulnerability classes they assembled (SWE-161, SWE-134, SWE-114, SWE-138, SWE-140). The best result is 92.1% exact-match for gpt-3.5-turbo with the most detailed prompt (Prcbi) and 112 training examples. The authors report seeds and hyperparameters, which is more than many such papers do, and the curious finding that gpt-4o-mini got worse with more data is reported honestly rather than massaged. That is a legitimate empirical data point for anyone building LLM-assisted static analysis.

The soft spots are substantial, though. The central claim — that this significantly enhances the speed and accuracy of vulnerability detection — is not measured. Exact match against the authors' own reference snippets is a proxy for rule quality, not a measure of detection precision or recall on real contracts. Nothing in the paper integrates the generated rules into the analyzer and tests them on a labeled corpus, and there is no baseline comparison against existing analyzers or even against a non-fine-tuned prompt. The test set is 38 examples, each configuration run once, with no error bars despite the seeds being listed. The abstract's 91.1% does not match the body's 92.1%. On top of that, Section II.C argues for in-context learning over fine-tuning, while the experiments actually fine-tune the models — a real internal contradiction. The claimed "three new root causes" are not identified as new relative to any specific literature, so that contribution is unsubstantiated.

None of this makes the paper worthless. It is a reproducible first step: the dataset description and prompt designs are concrete, the failure of more data to help GPT-4 is an interesting observation, and the threats-to-validity section shows the authors know some of these limits. But a reader should treat the 92% as "the model can imitate the reference detector conditions" and nothing more.

I would send this to a serious referee, because the question is timely and the experimental material is usable, but I would expect a major revision that either adds an end-to-end precision and recall evaluation or explicitly narrows the claims to rule imitation. For my own work I would not cite it yet. Worth a reading-group slot if you want to discuss proxy metrics in LLM-for-security evaluation.

Referee Report

4 major / 4 minor

Summary. The paper proposes SmartLLMSentry, a framework that uses large language models (ChatGPT) with in-context learning and fine-tuning to generate static analysis detection rules for smart contract vulnerabilities. The authors construct a dataset of five vulnerability types, design four increasingly detailed prompts, fine-tune two OpenAI models (gpt-3.5-turbo-1106 and gpt-4o-mini-2024-07-18), and evaluate the generated if-conditions by an exact-match (EM) metric against reference snippets on a 38-example test set. They report a best EM score of 92.1% for the GPT-3 model with the most detailed prompt and the larger training set, and conclude that the approach significantly enhances the speed and accuracy of vulnerability detection.

Significance. If the central claim were established, the framework would offer a plausible way to reduce expert effort in writing static analysis rules, which is a real bottleneck in smart contract security tooling. The paper has some strengths: it reports the exact seeds used for fine-tuning, provides a clear ablation of prompt components, and studies the effect of training-set size. However, the significance is currently limited because the only quantitative evidence is an exact-match score against the authors' own reference snippets; there is no demonstration that the generated rules improve detection precision or recall on real contracts, so the headline claim of enhanced vulnerability detection rests on an unvalidated proxy.

major comments (4)

[Section V-A3 (Evaluation Metrics)] The exact-match metric is defined as generated code having 'the exact same logic or the exact same syntax' as a reference snippet, and it is used for every reported result. This metric measures the model's ability to imitate the authors' expected outputs, not its ability to detect vulnerabilities in unseen contracts. A rule can match a reference snippet yet produce false positives or false negatives on real code, and a semantically correct but syntactically different rule would be counted as a mismatch. The manuscript never integrates the generated conditions into the SmartLLMSentry analyzer and measures detection on a labeled corpus, and it provides no comparison with existing tools such as Slither or MadMax or with a non-LLM baseline. Consequently, the abstract and conclusion claims that SmartLLMSentry 'significantly enhances the speed and accuracy of vulnerability detection' are not supported by the evidence presented. Please add an end-to-end evaluation that reports precision, recall, and F1 on a held-out set of vulnerable and benign contracts, and reconcile the metric with downstream detection quality.
[Section III-A and Section VII (Conclusion)] The framework design in Section III-A describes a 'continuous and automated enhancement' pipeline with a generator, validator, and integrator, and the contributions list claims that the system 'eliminates the need for continual expert intervention.' However, the concluding section states that 'the current model's generated rules are not fully automated in the framework and still require some expert intervention.' This is a direct contradiction on a load-bearing point: the automation of rule creation is one of the paper's main claimed contributions. Please clarify precisely which steps are automated and which require expert involvement, and provide evidence that the described validator/integrator components have actually been implemented and exercised, rather than only proposed.
[Section V-A and Abstract] The abstract reports an exact-match accuracy of 91.1%, while Section V-A reports 92.1% for the same best configuration (GPT-3, prompt Prcbi, 112 training instances). Since the test set has only 38 examples, the difference between 91.1% and 92.1% corresponds to a difference of less than one example, and the paper does not report how many of the 38 examples were exact matches. Each configuration is evaluated once, with no repeated runs or confidence intervals, so the reported rankings between models and prompt types may reflect sampling noise. Please reconcile the abstract number with the body, state raw match counts (e.g., 35/38), and report variance across multiple runs or an appropriate statistical analysis.
[Section II-D (Smart contract vulnerabilities)] The contribution list asserts that the authors 'identified and analyzed three new common root causes for vulnerabilities which, to the best of our knowledge, have not previously been explored in the scientific literature,' but the manuscript provides no comparison with prior vulnerability taxonomies or evidence that these root causes are genuinely new. The descriptions in Section II-D cite standard sources for known vulnerability classes (e.g., SWE-134, SWE-114), and the references include two self-citations to an unpublished manuscript. This novelty claim is unsubstantiated and should be either supported with a concrete literature comparison or removed from the contributions.

minor comments (4)

[General] The manuscript has multiple typos and presentation errors, including 'Finetunning' in the section heading, 'Descussion' in the title of Section V, 'inorder' in the introduction to Section V, and duplicated figure and table numbering (Figure 1 is used for both the framework design and the results chart, and Table I appears twice). These should be corrected.
[Section V-A2 (Used GPT versions)] The paper refers to the models as 'GPT-3' and 'GPT-4' throughout, but the actual models are gpt-3.5-turbo-1106 and gpt-4o-mini-2024-07-18. gpt-4o-mini is not the full GPT-4 model, so the terminology may mislead readers; please use the exact model names whenever results are reported.
[Section V-A3 (Evaluation Metrics)] The exact-match formula is written with an equals sign rather than a mathematical definition, and the paper does not give an example of what counts as 'same logic' versus 'same syntax.' A worked example and a precise matching procedure would make the metric reproducible.
[Section IV-B (Data Pre-processing)] The dataset is said to be 'available upon request from the authors,' but no link or repository is provided. Given that the paper's contribution list includes building a dataset, please make the dataset publicly accessible or provide a clear justification for why it cannot be shared.

Circularity Check

1 steps flagged · score 2.0 of 10

Only minor circularity: the framework's internal validator scores detectors on the same instances used to generate them, while the headline 91.1% exact-match result is a separate held-out empirical measurement.

fitted input called prediction [Section III-A, Framework design (validator component)]
"Following the generation of detectors, the validator assesses their performance by evaluating their accuracy using the same instances provided to the generator Figure 1. The validator is critical for ensuring the reliability of the detectors. If a detector achieves an accuracy rate below 80 %, it is deemed inadequate and rejected."

The validator's accuracy is computed on the exact vulnerability instances that were given to the generator, so a high score only indicates that the generated detector reproduces the examples it already saw; it is a fit-to-input score, not an estimate of detection performance on unseen contracts. This is a train-on-test leak in the framework's acceptance gate. The paper's reported 91.1% exact-match result, however, is measured on a separate 38-example held-out test set, so the central quantitative claim does not reduce to this step.

full rationale

The central result is empirical and self-contained: the authors fine-tuned GPT-3/GPT-4 on 112 instruction-to-condition examples and measured exact-match accuracy on 38 held-out test examples, so the 91.1% figure is a genuine out-of-sample measurement rather than a fitted parameter re-labeled as a prediction. The main circularity concern is confined to the framework's internal validator, which evaluates generated detectors on the same instances used to generate them; as described, that acceptance gate measures training-set fit, not generalization. The self-citations to the authors' prior vulnerability survey are not load-bearing for the quantitative evaluation, and there is no imported uniqueness theorem or ansatz-by-citation. The larger validity problem is that exact match against the authors' reference snippets is a proxy for detector quality, and the paper does not measure precision/recall on real vulnerable contracts; that is a correctness/validity issue rather than a circular derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

This is an empirical ML paper, so the ledger records design choices and assumptions rather than derived parameters. The main contributors are the chosen threshold, hyperparameters, dataset sizes, and the unvalidated EM proxy.

free parameters (3)

Validator accuracy threshold = 80%
Detectors below 80% accuracy are rejected; the threshold is a design choice without sensitivity analysis.
Fine-tuning hyperparameters (epochs, batch size, learning rate multiplier) = 3, 1, 2
Reported as the optimal configuration from initial experiments, but no search or robustness check is shown.
Training set sizes = 100 and 112 instances
Sizes were selected to follow OpenAI guidance and to test volume effects; the 12-instance increment is arbitrary and the split is not statistically justified.

assumptions (4)

domain assumption Exact match between generated and reference code is a sufficient measure of detector quality
The entire evaluation uses EM as the only metric, without linking it to detection precision/recall on real contracts.
domain assumption The five selected vulnerability classes are representative and their ground truth labels are correct
The selection is described only as random, and the ground truth snippets are not shown or independently audited.
domain assumption OpenAI fine-tuning API runs are stable enough that a single run per configuration is meaningful
One seed per configuration is used; no repeated runs or variance estimates are reported.
domain assumption The authors' taxonomy of vulnerabilities (ref [3], self-cited) provides the correct definitions for the SWE-XXX identifiers
The paper relies on its own prior taxonomy for vulnerability descriptions, and the identifiers are not cross-referenced to a standard registry.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework." pith.science (2026). https://pith.science/paper/C3AESY3J

@misc{pith2026241119234,
  author       = {Pith},
  title        = {Pith review of: SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C3AESY3J}},
  note         = {Machine review of arXiv:2411.19234}
}

read the original abstract

Smart contracts are essential for managing digital assets in blockchain networks, highlighting the need for effective security measures. This paper introduces SmartLLMSentry, a novel framework that leverages large language models (LLMs), specifically ChatGPT with in-context training, to advance smart contract vulnerability detection. Traditional rule-based frameworks have limitations in integrating new detection rules efficiently. In contrast, SmartLLMSentry utilizes LLMs to streamline this process. We created a specialized dataset of five randomly selected vulnerabilities for model training and evaluation. Our results show an exact match accuracy of 91.1% with sufficient data, although GPT-4 demonstrated reduced performance compared to GPT-3 in rule generation. This study illustrates that SmartLLMSentry significantly enhances the speed and accuracy of vulnerability detection through LLMdriven rule integration, offering a new approach to improving Blockchain security and addressing previously underexplored vulnerabilities in smart contracts.

Figures

Figures reproduced from arXiv: 2411.19234 by the authors.

**Figure 1.** Once a new vulnerability is detected in the wild, a [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 31 canonical work pages

[1]

, & Khabbazian, M

Shabani Baghani, A., Rahimpour, S. , & Khabbazian, M. (2022). The DAO Induction Attack: Analysis and Countermeasure. IEEE Internet of Things Journal , 9(7), 4875 –4887. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2021.3108154

arXiv 2022
[2]

Fatima Samreen, N., & Alalfi, M. H. (2020). Reent rancy Vulnerability Identification in Ethereum Smart Contracts. 2020 IEEE International Workshop on Blockchain Oriented Software Engineering (IWBOSE), 22–

work page 2020
[3]

Zaazaa, O., & Bakkali, H. E. (n.d.). Unveiling th e Landscape of Smart Contract Vulnerabilities: A Detailed Examination and Codification of Vulnerabilities in Prominent Blockchains

work page
[4]

Matulevicius, N., & Cordeiro, L. C. (2021). Verifying Security Vulnerabilities for Blockchain-based Smart Contracts. 2021 XI Brazilian Symposium on Computing Systems Engineering (SBESC) , 1 –8. https://doi.org/10.1109/SBESC53686.2021.9628229

work page arXiv 2021
[5]

etherscan.io. (n.d.). Ethereum Daily Deployed Contracts Chart | Etherscan. E thereum (ETH) Blockchain Explorer. Retrieved July 22, 2024, from https://etherscan.io/chart/deployed -contracts

work page 2024
[6]

Singh, N., Meherhomji, V., & Chandavarkar, B. R. (2020). Automated versus Manual Ap proach of Web Application Penetration Testing. 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT) , 1–6

work page 2020
[7]

L., Almeida, D., Altensch midt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altensch midt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B. (2023). GPT-4 Technical Report (arXiv:2303.08774). arXiv

arXiv 2023
[8]

Cao, J., Li, M., Wen, M., & Cheung, S. (2023). A stu dy on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair. Association for Computing Machinery , 1(1)

work page 2023

Show all 67 references

[9]

Sobania, D., Briesch, M., Hanna, C., & Petke, J. (2023). An Analysis of the Automatic Bug Fixing Performance of ChatGPT. 2023 IEEE/ACM International Workshop on Automated Program Repair (APR) , 23 –30. https://doi.org/10.1109/APR59189.2023.00012

2023
[10]

OpenAI Platform . (n.d.). Retrieved June 11, 2024, from https://platform.openai.com

2024
[11]

Austin, A., Holmgreen, C., & Williams, L. (2013). A comparison of the efﬁciency and effectiveness of vulnerability discovery techniques. Information and Software Technology , 55(7), 1279 –1288. https://doi.org/10.1016/j.infsof.2012.11.007

2013 doi
[12]

Schneidewind, C., Grishchenko, I., Scherer, M., & Maffei, M. (2020). eThor: Practical and Provably Sound Static Analysis of Ethereum Smart Contracts. Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security , 621 –640. https://doi.org/10.1145/33722...

2020
[13]

D., Pham, L

Nguyen, T. D., Pham, L. H., & Sun, J. (2021). SGUARD: Towards Fixing Vuln erable Smart Contracts Automatically. 2021 IEEE Symposium on Security and Privacy (SP) , 1215 –1229. https://doi.org/10.1109/SP40001.2021.00057

2021
[14]

Wang, D., Jiang, B., & Chan, W. K. (2020). WANA: Symbolic Execution of Wasm Bytecode for Cross -Platform Smart Contract Vulnerability Detection*#. CoRR, abs/2007.15510, 12. https://doi.org/10.48550/arXiv.2007.15510

2020 doi arXiv
[15]

Hao, X., Ren, W., Zheng, W., & Zhu, T. (2020). SCScan: A SVM-Based Scanning System for Vulnerabilities in Blockchain Smart Contracts. 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) , 1598 –1605. https://doi.org...

2020
[16]

Ye, J., Ma, M., Lin, Y., Sui, Y., & Xue, Y. (2020). Clairvoyance: Cross- contract static analysis for detecting practical reentrancy vulnerabilities in smart contracts. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings , 274 –

2020
[17]

Feist, J., Grieco, G., & Groce, A. (2019). Slither: A Static Analysis Framework for Smart Contracts. 2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), 8–15. https://doi.org/10.1109/WETSEB.2019.0 0008

2019 doi
[18]

Tang, Y., Li, Z., & Bai, Y. (2021). Rethinking of Reentrancy on the Ethereum. 2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf o n Cyber Science and Techno...

2021
[19]

Grech, N., Kong, M., Jurisevic, A., Brent, L., Scholz, B., & Smaragdakis, Y. (2018). MadMax: Surviving out -of-gas conditions in Ethereum smart contracts. Proceedings of the ACM on Programming Languages , 2(OOPSLA), 1–27. https://doi.org/10.1145/3276486

2018 doi
[20]

D., Pham, L

Nguyen, T. D., Pham, L. H., Sun, J., Lin, Y., & Minh, Q. T. (2020). sFuzz: An efficient adaptive fuzzer for solidity smart contracts. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering , 778 –788. https://doi.org/10.1145/3377811.3380334

2020
[21]

Ren, M., Ma, F., Yin, Z., Li, H., Fu, Y., Chen, T., & Jiang, Y. (2021). SCStudio: A secure and efficient integrated development environment for smart contracts. Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis , 666 –669. https://doi...

2021
[22]

Gao, Z., Jayasundara, V., Jiang, L., Xia, X., Lo, D., & Grundy, J. (2019). SmartEmbed: A Tool for Clone and Bug Detection in Smart Contracts through Structural Code Embedding. 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), 394–397. https://do...

2019
[23]

Yu, X., Zhao, H., Hou, B., Ying, Z., & Wu, B. (2021). DeeSCVHunter: A Deep Learning -Based Framework for Smart Contract Vulnerability Detection. 2021 International Joint Conference on Neural Networks (IJCNN), 1–8. https://doi.org/10.1109/IJCNN52387.2021.9534324

2021
[24]

Wu, H., Zhang, Z., Wang, S., Lei, Y., Lin, B., Qin, Y., Zhang, H., & Mao, X. (2021). Peculiar: Smart Contract Vulnerability Detection Bas ed on Crucial Data Flow Graph and Pre -training Techniques. 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISS...

2021
[25]

P., & Okamura, S

Ashizawa, N., Yanai, N., Cruz, J. P., & Okamura, S. (2021). Eth2Vec: Learning Contract -Wide Code Representations for Vulnerability Detection on Ethereum Smart Contracts. Proceedings of the 3rd ACM International Symposium on Blockchain and Secure Critic al Infrastructure, 47–5...

2021
[26]

A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., … Liang, P. (2021). On the ...

2021 arXiv
[27]

Brown, T. B., Mann, B., Ryder, N., Subbiah, M ., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Mod...

2020
[28]

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga -Alonso, A., Kluska, A., Lewkow ycz, A., Agarwal, A., Power, A., Ray, A., Warstadt, A., Kocurek, A. W., Safaya, A., Tazarv, A., … Wu, Z. (2022). Beyond the...

2022 arXiv
[29]

https://doi.org/10.1109/IWBOSE50093.2020.9050260

2020
[30]

GPT-4 | OpenAI . (n.d.). Retrieved June 12, 2024, from https://openai.com/index/gpt-4/

2024
[31]

Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., … W en, J. -R. (2023). A Survey of Large Language Models. ArXiv Preprint ArXiv:2303.18223

2023 arXiv
[32]

Kevian, D., Syed, U., Guo, X., Havens, A., Dullerud, G., Seiler, P., Qin, L., & Hu, B. (2024). Capabilities of Large Language Models in Control Engineering: A Benchmark Study on GPT-4, Claude 3 Opus, and Gemini 1.0 Ultra (arXiv:2404.03647). arXiv

2024 arXiv
[33]

Gemini Team, Reid, M., Savinov, N., Teplyashin, D., Dmitry, Lepikhin, Lillicrap, T., Alayrac, J., Soricut, R., Lazaridou, A., Firat, O., Schrittwieser, J., Antonoglou, I., Anil, R., Borgeaud, S., Dai, A., Millican, K., Dyer, E., Glaese, M., … Vinyals, O. (2024). Gemini 1.5: Un...

2024 arXiv
[34]

F., & Liu, L

Hu, S., Huang, T., İlhan, F., Tekin, S. F., & Liu, L. (2023). Large Language Model-Powered Smart Contract Vulnerability Detection: New Perspectives. 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applicatio ns (TPS -ISA), 297 –

2023
[35]

Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., & Jiang, D. (2023). WizardCoder: Empowering Code Large Language Models with Evol-Instruct. ArXiv Preprint ArXiv:2306.08568

2023 arXiv
[37]

Zhang, Y., Feng, S., & Tan, C. (2022). Active Example Selection for In - Context Learning. ArXiv Preprint ArXiv:2211.04486 . https://doi.org/10.48550/ar Xiv.2211.04486

2022 doi arXiv
[38]

Sun, Y., Wu, D., Xue, Y., Liu, H., Ma, W., Zhang, L., Shi, M., & Liu, Y. (2024). LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs’ Vulnerability Reasoning. ArXiv

2024
[39]

Wang, Y.-X., Ramanan, D., & Hebert, M. (2017). Growing a Brain: Fine- Tuning by Increasing Model Capacity. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 3029–3038

2017
[40]

W., Chen, Z., & Ma, Y

Church, K. W., Chen, Z., & Ma, Y. (2021). Emerging trends: A gentle introduction to ﬁne -tuning. Natural Language Engineering , 27(6), 763–

2021
[41]

Pan, J., Gao, T., Chen, H., & Chen, D. (2023). What In-Context Learning “Learns” In -Context: Disentangling Task Recognition and Task Learning. Princeton University

2023
[42]

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Computing Surveys , 55(9), 1–35. https://doi.org/10.1145/3560815

2023 doi
[43]

R., Wang, A., & Ba, J

Pitis, S., Zhang, M. R., Wang, A., & Ba, J. (2023). Boosted Prompt Ensembles for Large Language Models. ArXiv Preprint ArXiv:2304.05970

2023 arXiv
[44]

Zhao, Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate Before Use: Improving Few -shot Performance of Language Models. Proceedings of the 38th International Conference on Machine Learning, 12697–12706. https://proceedings.mlr.press/v139/zhao21c.html

2021
[45]

Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., Zheng, K., Lian, D., & Chen, E. (2024). When large language models meet personalization: Perspectives of challenges and opportunities. World Wide Web , 27(4), 42. https://doi.org/10....

2024 doi
[46]

F., Araki, J., & Neubig, G

Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020). How Can We Know What Language Models Know? Transactions of the Association for Computational Linguistics , 8, 423 –438. https://doi.org/10.1162/tacl_a_00324

2020 doi
[47]

Liu, J., Shen, D., Zhang, Y., Dola n, B., Carin, L., & Chen, W. (2022). What Makes Good In -Context Examples for GPT -3? Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures , 100–114. h...

2022 doi
[48]

Gu, Y., Han, X., Liu, Z., & Huang, M. (2022). PPT: Pre -trained Prompt Tuning for Few-shot Learning. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8410--8423. https://doi.org/10.18653/v1/2022.acl -long.576

2022 doi
[50]

L., & Liang, P

Li, X. L., & Liang, P. (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation. In C. Zong, F. Xia, W. Li, & R. Navigli (Eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Na...

2021 doi
[51]

Liu, V., & Chilton, L. B. (2022). Design Guidelines for Prompt Engineering Text -to-Image Generative Models. CHI Conference on Human Factors in Computing Systems , 1 –23. https://doi.org/10.1145/3491102.3501825

2022
[52]

Maddigan, P., & Susnjak, T. (2023). Chat2VIS: Generating Data Visualizations via Natural Language Using ChatGPT, Codex and GPT -3 Large Language Models. IEEE Access, 11, 45181–45193. IEEE Access. https://doi.org/10.1109/ACCESS.2023.3274199 Journal of Metaverse Zaazaa & El Bakk...

2023
[53]

Lester, B., Al -Rfou, R., & Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 3045 --3059. https://doi.org/10.18653/v1/2021.emnlp -main.243

2021 doi
[54]

White, J., Hays, S., Fu, Q., Spencer -Smith, J., & Schmidt, D. C. (2023). ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design. Generative AI for Effective Software Development, 71--108. https://doi.org/10.1007/978 -3- ...

2023 doi
[55]

Demir, M., Alalfi, M., Turetken, O., & Ferworn, A. (2019). Security Smells in Smart Contracts. 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS -C), 442 –

2019
[56]

EIP-1884: Repricing for trie -size-dependent opcodes . (n.d.). Ethereum Improvement Proposals. Retrieved August 12, 2024, from https://eips.ethereum.org/EIPS/eip-1884

2024
[57]

Liu, C., Bao, X., Zhang, H., Zhang, N., Hu, H., Zhang, X., & Yan, M. (2023). Imp roving ChatGPT Prompt for Code Generation. ArXiv Preprint ArXiv:2305.08360

2023 arXiv
[58]

Staderini, M., Palli, C., & Bondavalli, A. (2020). Classification of Ethereum Vulnerabilities and their Propagations. 2020 Second International Conference on Blockchain Computing and Applications (BCCA), 44–51. https://doi.org/10.1109/BCCA50787.2020.9274458

2020
[59]

(2024, August 12)

ERC 721 —OpenZeppelin Docs . (2024, August 12). https://docs.openzeppelin.com/contracts/2.x/api/token/ERC721

2024
[60]

Publications/reviews/2023-07-arcade-securityreview.pdf at master · trailofbits/publications. (n.d.). GitHub. Retrieved August 12, 2024, from https://github.com/trailofbits/publications/blob/master /reviews/2023-07- arcade-securityreview.pdf

2023
[61]

Sending Ether (transfer, send, call) | Solidity by Example | 0.8.24 . (n.d.). Retrieved August 12, 2024, from https://solidity-by- example.org/sending-ether/

2024
[62]

(2021, July 14)

How Ethereum’s Istanbul Network Upgrade Affects DeFi . (2021, July 14). Defi Pulse Blog. https://defipulse.com/blog/how -ethereums- istanbul-network-upgrade-affects-defi/

2021
[63]

PublicReports/Solidity Smart Contract Audits/Cere_Bridge_Smart_Contract_Security_Audit_Solidity_Report_ Halborn_Final.pdf at master · HalbornSecurity/PublicReports . (n.d.). GitHub. Retrieved August 12, 2024, from https://github.com/HalbornSecurity/PublicReports/blob/master/So...

2024
[64]

Hou, W., & Ji, Z. (2024). A systematic ev aluation of large language models for generating programming code. ArXiv Preprint ArXiv:2403.00894

2024 arXiv
[67]

PublicReports/Solidity Smart Contract Audits/Persistence_StkBNB_Smart_Contract_Security_Audit_Report_H alborn_Final.pdf at master · HalbornSecurity/PublicReports . (n.d.). GitHub. Retrieved August 12, 2024, from https://github.com/HalbornSecurity/PublicReports/blob/master/Soli...

2024
[275]

https://doi.org/10.1145/3377812.3390908
[306]

https://doi.org/10.1109/TPS-ISA58951.2023.00044

2023
[449]

https://doi.org/10.1109/QRS-C.2019.00086

2019
[778]

https://doi.org/10.1017/S1351324921000322

doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

, & Khabbazian, M

Shabani Baghani, A., Rahimpour, S. , & Khabbazian, M. (2022). The DAO Induction Attack: Analysis and Countermeasure. IEEE Internet of Things Journal , 9(7), 4875 –4887. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2021.3108154

arXiv 2022

[2] [2]

Fatima Samreen, N., & Alalfi, M. H. (2020). Reent rancy Vulnerability Identification in Ethereum Smart Contracts. 2020 IEEE International Workshop on Blockchain Oriented Software Engineering (IWBOSE), 22–

work page 2020

[3] [3]

Zaazaa, O., & Bakkali, H. E. (n.d.). Unveiling th e Landscape of Smart Contract Vulnerabilities: A Detailed Examination and Codification of Vulnerabilities in Prominent Blockchains

work page

[4] [4]

Matulevicius, N., & Cordeiro, L. C. (2021). Verifying Security Vulnerabilities for Blockchain-based Smart Contracts. 2021 XI Brazilian Symposium on Computing Systems Engineering (SBESC) , 1 –8. https://doi.org/10.1109/SBESC53686.2021.9628229

work page arXiv 2021

[5] [5]

etherscan.io. (n.d.). Ethereum Daily Deployed Contracts Chart | Etherscan. E thereum (ETH) Blockchain Explorer. Retrieved July 22, 2024, from https://etherscan.io/chart/deployed -contracts

work page 2024

[6] [6]

Singh, N., Meherhomji, V., & Chandavarkar, B. R. (2020). Automated versus Manual Ap proach of Web Application Penetration Testing. 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT) , 1–6

work page 2020

[7] [7]

L., Almeida, D., Altensch midt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B

OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altensch midt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B. (2023). GPT-4 Technical Report (arXiv:2303.08774). arXiv

arXiv 2023

[8] [8]

Cao, J., Li, M., Wen, M., & Cheung, S. (2023). A stu dy on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair. Association for Computing Machinery , 1(1)

work page 2023