Pith. sign in

REVIEW 4 major objections 5 minor 61 references

Human Side of Smart Contract Fuzzing: An Empirical Study

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Smart contract fuzzing tools fail practitioners on usability, not just vulnerability detection, and the barriers differ by background.

desk verdict A useful first taxonomy of smart contract fuzzer usability challenges, but the 'domain-specific' framing overreaches the evidence and the small user study only supports exploratory claims. read the letter →

arxiv 2506.07389 v1 pith:JILP4LB6 submitted 2025-06-09 cs.HC cs.SE

Guanming Qiao , Partha Protim Paul This is my paper
classification cs.HCcs.SE
keywords smartcontractfuzzingusabilityease-of-usetaxonomyGitHubissueanalysisuserstudyblockchainhumanfactors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks why smart contract fuzzing tools, despite strong technical capabilities, are hard for practitioners to adopt. It builds a taxonomy of ease-of-use and usefulness challenges from 381 GitHub issues filed against two widely used smart contract fuzzers, Echidna and Foundry, and then runs a 90-to-120-minute task-oriented user study with six practitioners from two backgrounds: smart contract developers and traditional software security researchers. The core claim is that smart contract fuzzing has domain-specific human barriers, including blockchain emulation problems, gas-constraint false results, manual and inflexible setup, and documentation that assumes fuzzing knowledge, and that these barriers hit the two practitioner groups differently. A sympathetic reader would care because these barriers sit between a powerful vulnerability-finding technique and widespread adoption in a domain where financial loss from bugs is common.

What carries the argument

The central mechanism is a two-dimensional challenge taxonomy. Dimension A classifies each challenge by nature, distinguishing usefulness from ease-of-use and further subcategorizing into reliability, consistency, performance, essential features, error prevention, interpretability, and learnability, while Dimension B locates the challenge within the smart contract fuzzing workflow of setup, the fuzzing loop, and integration or deployment. The taxonomy is built through inductive content analysis: GitHub issues are summarized by three large language models, manually validated, and sorted into categories by two researchers using card sorting. The second mechanism is the task-oriented user study, where all participants fuzz the same adapted DAO-hack contract with the same tool across seven subtasks, with performance scored by a rubric while think-aloud transcripts and AI-tool interactions are recorded. The combination lets the paper triangulate what users report in issues with what users actually do in a controlled session.

What would settle it

Run the same issue-coding on a third, comparably popular smart contract fuzzer such as Smartian or ityFuzz, and repeat the user study with at least twenty practitioners per group; if the five domain-specific challenge categories do not recur or the smart contract developers' advantage in invariant writing disappears, then the paper's generalizations fail.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the challenges of smart contract fuzzing are not only technical but human, and that they differ qualitatively from traditional program fuzzing. The five domain-specific challenge categories are cross-chain compatibility issues, gas constraints that produce false positives and negatives, blockchain state management and on-chain fuzzing reliability, a manual yet inflexible setup process, and a lack of smart-contract-specific documentation. In the user study, only two of six participants found more than the trivial vulnerability within the session, and five of six reported no intention to adopt the tool. The paper's differential findings are that smart contract developers struggle more with the fuzzing mental model and documentation, while fuzzing practitioners are better at harness writing but more frustrated by missing automation; developers wrote better invariants thanks to DeFi business knowledge; no participant used the recommended multi-actor pattern; and LLM assistants helped with getting started but were misled by misleading fuzzer output, such as the undocumented 'fail_on_revert' behavior reporting success when all calls revert.

Load-bearing premise

The whole taxonomy and all group-difference findings rest on the assumption that 381 GitHub issues from two repositories and six recruited participants represent the broader population of smart contract fuzzing practitioners.

Editorial extensions

If this is right

  • Tool developers who address the five identified categories, especially automating harness generation and documenting the fuzzer's underlying mental model, should see higher adoption and fewer one-on-one onboarding interactions.
  • Undocumented behaviors like 'fail_on_revert' should be treated as usability bugs; fixing them would prevent a class of false-confidence results where users believe all tests passed.
  • Decoupling invariant writing from fuzz harness writing could exploit smart contract developers' comparative strength in reasoning about expected contract behavior and lower the barrier for that group.
  • Documentation with real-world, complete code examples and explicit fuzzing concepts would likely reduce the conceptual gap that smart contract developers experienced in the study.
  • The taxonomy gives researchers and tool designers a checklist for evaluating future smart contract fuzzers on human factors rather than on coverage and bug counts alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the taxonomy generalizes, the same two-dimensional coding of nature by workflow stage could be applied to fuzzers in other stateful domains, such as protocol or DeFi-specific tools, and likely reveal analogous documentation and automation gaps.
  • The finding that LLM assistants inherit the fuzzer's misleading output suggests a testable extension: feeding raw fuzzer logs to LLMs with explicit instructions to detect patterns like 'all calls reverted' could reduce circular reasoning errors.
  • The six-participant sample is small, but the group-difference pattern suggests a falsifiable hypothesis: role-based onboarding, a fuzzing primer for smart contract developers and automation-focused scaffolding for security engineers, should measurably improve task completion rates.
  • The paper's own proposed prototype is a natural next step; a follow-up user study measuring adoption intention before and after using such a prototype would test whether the taxonomy's categories are indeed the binding constraints.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript reports an empirical study of human-side challenges in smart contract fuzzing. The authors analyze 381 GitHub issues from two smart contract fuzzers, Echidna and Foundry, using LLM-assisted summarization followed by human validation, and inductively develop a taxonomy of ease-of-use and usefulness challenges organized by challenge nature and fuzzing-workflow phase. They complement this with a task-oriented user study in which three smart contract developers and three traditional fuzzing practitioners attempt to find vulnerabilities in a DAO-inspired contract using Foundry. The paper claims that the resulting taxonomy reveals domain-specific challenges, including blockchain emulation issues, documentation gaps, and setup automation problems, and that these challenges affect the two practitioner groups differently. The manuscript also reports strategies, including the limited and sometimes misleading role of AI assistants, and proposes design recommendations for smart contract fuzzers.

Significance. If the findings were fully supported, this would be a valuable contribution to the human factors literature on fuzzing and to smart contract tool design. The paper identifies a gap in prior work, which has focused on technical fuzzer performance rather than practitioner experience, and it triangulates two complementary evidence sources: repository issue analysis and a controlled user study. The two-author card-sorting procedure grounded in a real-world issue corpus, the use of LLM summarization with explicit human validation, and the concrete recommendations for tool designers are useful elements. However, the central claim of domain-specificity is not currently supported by the research design, and the transparency of the taxonomy and user-study evidence is insufficient. The contribution is therefore better characterized as an exploratory taxonomy of challenges reported by users of two smart contract fuzzers, with qualitative observations about practitioner differences, rather than as a definitive account of what is unique to smart contract fuzzing.

major comments (4)
  1. [§3, §5.1, Table 4] The central claim that the findings reveal challenges 'not present in traditional software fuzzing' is not supported by the study design. Sub-RQ1 is explicitly framed as identifying challenges 'not present in traditional software fuzzing,' but the content analysis in §4.1 samples only issues from two smart contract fuzzers (Echidna and Foundry) and includes no comparison corpus from traditional fuzzers such as AFL++, libFuzzer, or OSS-Fuzz. Without such a baseline, the study cannot distinguish challenges unique to smart contract fuzzing from challenges common to all fuzzing tools. Indeed, the taxonomy in Table 4 includes categories such as 'Manual Yet Inflexible Fuzzer Setup Process,' 'Lack of SC-Specific Documentation,' 'Automatic Generation of Test Suite,' and 'Fine-Grained Configuration Options,' which prior work on traditional fuzzing (Nourry et al., TOSEM 2023; Plöger et al., CHI 2023, both cited by the paper) already documents. I recommend substantially reframing the findings as 'challenges observed in smart contract fuzzing' and, if the uniqueness claim is retained, collecting or systematically comparing against a traditional-fuzzer issue corpus.
  2. [§4.1.3, Appendix §8.5] The taxonomy development lacks essential methodological transparency and reproducibility. The paper reports that two researchers 'collaboratively analyzed' the issues using card sorting, but it provides no inter-rater reliability metric, no codebook, no per-category issue counts, and no published dataset. Without per-category counts, the reader cannot assess which challenges are most prevalent or whether the categories are well supported by the data; without a reliability measure, the card-sorting results cannot be distinguished from idiosyncratic categorization by two co-authors. I request that the authors publish the de-identified issue dataset and category assignments, report the number of issues in each Level 3 category, and provide an inter-rater agreement statistic for at least a subset of the coding, or explicitly justify why these are omitted.
  3. [§4.2.2, §5.2, Table 1] The user study has only six participants, with three in each group, uses a single contract, and all participants elected to use Foundry. Despite this, the paper states group-difference findings such as Finding #1 and Finding #2 as though they are established empirical results. The quantitative completion rates in Table 1 are reported as percentages without any statistical test, confidence interval, or effect-size measure, and the differences between group averages (e.g., 33% vs. 55% for triggering bugs) are derived from three observations per group. The qualitative observations are useful, but the claims about differential performance should be softened to exploratory observations, or the analysis should include appropriate statistical or qualitative rigor (e.g., a clear analysis protocol, more participants, or a formal qualitative analysis method such as thematic analysis).
  4. [Table 1, §4.2.1, Appendix §8.2.4] There is an apparent internal inconsistency regarding peer contracts in the user study. Table 1 reports that no participant deployed a peer contract (the row 'Deployed peer contracts' is 0 for all six participants), yet the scoring rubric in §8.2.2 includes 'Wrote and deployed a peer contract (normal or malicious)' and Table 2 describes the Level 3 reentrancy vulnerability as causing 'severe loss during transactions with malicious peer contracts.' The paper does not explain how participants such as P4, who triggered all three bugs, could have triggered the reentrancy bug without deploying a malicious peer contract. Either the scoring criteria were applied differently than stated, the description of the reentrancy vulnerability is incomplete, or the deployed-peer-contract row measures something other than what the subtask describes. Please clarify this discrepancy.
minor comments (5)
  1. [§4.1.1] The issue selection strategy relied on the keywords 'fuzz' and 'invariant' in two repositories; this may exclude relevant issues that use other terminology, and the paper should acknowledge this as a limitation of the sampling frame.
  2. [§5.1, Finding #3] There are several typos and inconsistent terms: 'Echinda' should be 'Echidna,' 'ECR-20' should be 'ERC-20,' 'Ityfuzz' should be 'ItyFuzz,' and the Appendix title 'LLM GitHub Isuse Summarization Pipeline' contains a misspelling of 'Issue.'
  3. [§4.2.2] The user study involved human participants, but the paper does not report ethics approval or institutional review board information beyond stating that written consent was obtained; this should be reported explicitly.
  4. [§6.0.1] The sentence 'Both SC-specific ease-of-use challenges (4 and 5) identified in our GitHub Issue analysis in Sub-RQ1 are corroborated...' refers to numbered items in §5.1, but the taxonomy table in Appendix §8.5 does not use those numbers, which makes the cross-reference unclear; please align the numbering.
  5. [§5.2] The paper states that all participants chose to work with Foundry, which means the user-study findings are limited to that tool; this limitation should be stated earlier and more prominently, since the GitHub-issue analysis covers both Echidna and Foundry.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the taxonomy is inductively derived from GitHub issues and the user study is an independent observation; the unsupported 'domain-specific' label is a validity gap, not a circular reduction.

full rationale

The paper makes no formal derivation claim; its contributions are an inductively built taxonomy and an observational user study. Section 4.1.3 states that the taxonomy was developed by card sorting from 381 GitHub issues, and Section 4.2 describes a user study in which six participants performed a fixed fuzzing task. Neither stage presupposes the other's conclusions. The only step that might look circular is using the same workflow concepts to structure both the taxonomy (Dimension B) and the user-study subtasks in Section 8.2.1, but those subtasks are generic fuzzing steps (analyze the contract, write a harness, deploy contracts, initialize state, write assertions, run the fuzzer, interpret output), and the participants' behaviors, quotes, and scores are not derived from the taxonomy. The paper's 'domain-specific' wording is a construct-validity concern: Sub-RQ1 asks for challenges 'not present in traditional software fuzzing,' yet only issues from Echidna and Foundry were collected, and some taxonomy categories such as 'Automatic Generation of Test Suite' and 'Hard-to-Configure Fuzzing Parameters' are generic usability categories already documented for traditional fuzzers in the cited prior work. That is an overreach of the evidence, not a circular reduction: no equation, fitted parameter, self-citation chain, or definitional identity makes the findings equal to their inputs. The paper also explicitly limits the user-study portion to Foundry ('Since all participants chose to work with Foundry, our findings are limited to the challenges manifested in the Foundry tool.'), which further shows that the authors are not claiming the results are forced. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities. The paper rests on sampling and task-design assumptions common to qualitative HCI studies; the most fragile are the representativeness of the two tools and the six participants, plus the validity of the modified DAO contract as a balanced task.

assumptions (3)
  • domain assumption The 381 GitHub issues obtained from the Echidna and Foundry repositories with the keywords 'fuzz' and 'invariant' are a representative source of practitioner challenges in smart contract fuzzing.
    Invoked in §4.1.1; the entire taxonomy is induced from this corpus. If the keyword filter or repository choice systematically misses certain user populations, the taxonomy would be incomplete.
  • domain assumption The six user-study participants (three smart contract developers and three fuzzing researchers) are representative of their respective practitioner groups.
    Invoked in §4.2.2 and used in §5.2 to make group-difference claims. With n=3 per group, sampling variability is a serious concern.
  • domain assumption The modified DAO hack contract with two added synthetic vulnerabilities is a valid, unbiased experimental task for comparing the two groups.
    Invoked in §4.2.1. The task may favor one group's prior knowledge; no pilot validation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Human Side of Smart Contract Fuzzing: An Empirical Study." pith.science (2026). https://pith.science/paper/JILP4LB6

@misc{pith2026250607389,
  author       = {Pith},
  title        = {Pith review of: Human Side of Smart Contract Fuzzing: An Empirical Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JILP4LB6}},
  note         = {Machine review of arXiv:2506.07389}
}
read the original abstract

Smart contract (SC) fuzzing is a critical technique for detecting vulnerabilities in blockchain applications. However, its adoption remains challenging for practitioners due to fundamental differences between SCs and traditional software systems. In this study, we investigate the challenges practitioners face when adopting SC fuzzing tools by conducting an inductive content analysis of 381 GitHub issues from two widely used SC fuzzers: Echidna and Foundry. Furthermore, we conducted a user study to examine how these challenges affect different practitioner groups, SC developers, and traditional software security professionals, and identify strategies practitioners use to overcome them. We systematically categorize these challenges into a taxonomy based on their nature and occurrence within the SC fuzzing workflow. Our findings reveal domain-specific ease-of-use and usefulness challenges, including technical issues with blockchain emulation, and human issues with a lack of accessible documentation and process automation. Our results provide actionable insights for tool developers and researchers, guiding future improvements in SC fuzzer tool design.

Figures

Figures reproduced from arXiv: 2506.07389 by the authors.

Figure 1
Figure 1. Pipeline for Summarizing GitHub Issues Using LLMs [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. SC Fuzzing Workflow Overview Fuzzing Loop (central section): This is the iterative core of the fuzzing process. It involves: • Corpus (A set of inputs and Functions): A collection of input data and function calls is maintained. • Input: An input is selected from the corpus. • Mutation: The selected input is mutated or modified to generate new test inputs. • Execute: The mutated input is used to execute the smart con… view at source ↗
Figure 3
Figure 3. Distribution of Github issues based on fuzzing workflow [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 57 canonical work pages

  1. [1]

    The DAO Attacked: Code Issue Leads to $60 Million Ether Theft

    CoinDesk 2016. The DAO Attacked: Code Issue Leads to $60 Million Ether Theft . CoinDesk. https://www.coindesk.com/markets/2016/06/17/the-dao- attacked-code-issue-leads-to-60-million-ether-theft/ Accessed: 2022

    work page 2016

  2. [2]

    Ethereum Market Cap

    2023. Ethereum Market Cap. https://ycharts.com/indicators/ethereum_market_cap. Accessed: February 2023

    work page 2023

  3. [3]

    0xalpharush. [n. d.]. Confusing users by creating a dichotomy between fuzz testing and invariant testing. https://github.com/foundry-rs/foundry/ issues/4162. Accessed: 2025-02-15

    work page 2025

  4. [4]

    0xmikko. [n. d.]. Race condition in tests with false negative results. https://github.com/foundry-rs/foundry/issues/6450. Accessed: 2025-02-15

    work page 2025

  5. [5]

    77abe77. [n. d.]. State inconsistencies between handler and invariant function contexts. https://github.com/foundry-rs/foundry/issues/5799. Accessed: 2025-02-15

    work page 2025

  6. [6]

    Deepseek AI. [n. d.]. Deepseek R1. https://huggingface.co/deepseek-ai/DeepSeek-R1. Accessed: 2025-02-15

    work page 2025

  7. [7]

    Alchemy. n.d.. ABI Parser: Decode and Encode Smart Contract Data. Alchemy Website. https://www.alchemy.com/dapps/abi-parser Accessed: 2025-04-09

    work page 2025

  8. [8]

    apbendi. [n. d.]. Leak into fuzz tests. https://github.com/foundry-rs/foundry/issues/7462. Accessed: 2025-02-15

    work page 2025

Show all 61 references
  1. [9]

    Azvect. [n. d.]. Improved gas control. https://github.com/foundry-rs/foundry/issues/7892. Accessed: 2025-02-15

    2025

  2. [10]

    Jinsheng Ba, Marcel Böhme, Zahra Mirzamomen, and Abhik Roychoudhury. 2022. Stateful Greybox Fuzzing. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, MA, 3255–3272. https://www.usenix.org/conference/usenixsecurity22/presentation/ba

    2022

  3. [11]

    Xiaomin Bai, Zijing Cheng, Zhangbo Duan, and Kai Hu. 2018. Formal Modeling and Verification of Smart Contracts. In Proceedings of the 2018 7th International Conference on Software and Computer Applications (Kuantan, Malaysia) (ICSCA ’18). ACM, 322–326. doi:10.1145/3185089.3185138

    2018

  4. [12]

    Sofia Bekrar, Chaouki Bekrar, Roland Groz, and Laurent Mounier. 2011. Finding software vulnerabilities by smart fuzzing. In 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation

    2011

  5. [13]

    bluele. [n. d.]. bug(cheatcodes): calling eth_getTransactionByHash with rpc cheatcode failed. https://github.com/foundry-rs/foundry/issues/7858. Accessed: 2025-02-15

    2025

  6. [14]

    Lexi Brent, Neville Grech, Sifis Lagouvardos, Bernhard Scholz, and Yannis Smaragdakis. 2020. Ethainter: a smart contract security analyzer for composite vulnerabilities. PLDI (2020)

    2020

  7. [15]

    Bybit. n.d.. Incident Update - ETH Cold Wallet Incident . Bybit Announcements. https://announcements.bybit.com/article/incident-update---eth- cold-wallet-incident-blt292c0454d26e9140/

  8. [16]

    Marcel Böhme and Brandon Falk. 2020. Fuzzing: On the exponential cost of vulnerability discovery. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE’20) . 713–724

    2020

  9. [17]

    Ting Chen, Zihao Li, Yufei Zhang, Xiapu Luo, Ting Wang, Teng Hu, Xiuzhuo Xiao, Dong Wang, Jin Huang, and Xiaosong Zhang. 2019. A large-scale empirical study on control flow identification of smart contracts. ESEM (2019)

    2019

  10. [18]

    Jaeseung Choi, Doyeon Kim, Soomin Kim, Gustavo Grieco, Alex Groce, and Sang Kil Cha. 2021. Smartian: Enhancing smart contract fuzzing with static and dynamic data-flow analyses. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 227–239

    2021

  11. [19]

    Crytic. 2023. Invariant Tests Can Be Skipped if Failed During Inputs Generation. https://github.com/crytic/echidna/issues/980. Issue #980, Accessed: April 5, 2025

    2023

  12. [20]

    Zhen Yu Ding and Claire Le Goues. 2021. An Empirical Study of OSS-Fuzz Bugs. arXiv:2103.11518 [cs.SE]

    2021 arXiv

  13. [21]

    Thomas Durieux, João F Ferreira, Rui Abreu, and Pedro Cruz. 2020. Empirical Review of Automated Analysis Tools on 47,587 Ethereum Smart Contracts. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea). ACM, 530–541. doi:10.11...

    2020

  14. [22]

    Foundry. n.d.. Invariant Testing - Actor Management. Foundry Book. https://book.getfoundry.sh/forge/invariant-testing#actor-management Accessed: 2025-04-09

    2025

  15. [23]

    Foundry-rs. 2022. Fuzzer Should Respect cheatcode vm.assume. https://github.com/foundry-rs/foundry/issues/990. Issue #990, Accessed: April 5, 2025

    2022

  16. [24]

    Foundry-rs. 2023. Forge Static Analysis Tool to Detect Fund Vulnerable Functions. https://github.com/foundry-rs/foundry/issues/7313. Issue #7313, Accessed: April 5, 2025

    2023

  17. [25]

    Foundry-rs. 2023. Invariant Tests Stopping When Invariant Fails. https://github.com/foundry-rs/foundry/issues/5466. Issue #5466, Accessed: April 5, 2025

    2023

  18. [26]

    Foundry-rs. 2024. Fuzz Error Assertion with Message Fails Test But `forge_snapshot`Shows Successful Test. https://github.com/foundry-rs/foundry/ issues/9393. Issue #9393, Accessed: April 5, 2025

    2024

  19. [27]

    Foundry-rs. 2024. Fuzzer Stopping the Fuzzing After the invariant is broken. https://github.com/foundry-rs/foundry/issues/8934. Issue #8934, Accessed: April 5, 2025

    2024

  20. [28]

    Foundry-rs. 2024. Fuzzer: Store Stats per Invariant so Failed Invariants Don’t Terminate Fuzzing. https://github.com/foundry-rs/foundry/pull/10190. Pull Request #10190, Accessed: April 5, 2025

    2024

  21. [29]

    Foundry-rs. 2024. Fuzzer Unable to Detect a Bug Due to Invariant With Trivial Assertion. https://github.com/foundry-rs/foundry/issues/8789. Issue #8789, Accessed: April 5, 2025

    2024

  22. [30]

    Foundry-rs. n.d.. Foundry. GitHub. https://github.com/foundry-rs/foundry 10 Guanming Qiao and Partha Protim Paul

  23. [31]

    Fuzzland. n.d.. Invgen: Automated Invariant Generation for Smart Contract Fuzzing. GitHub Repository. https://github.com/fuzzland/invgen Accessed: 2025-04-09

    2025

  24. [32]

    Gustavo Grieco, Will Song, Artur Cygan, Josselin Feist, and Alex Groce. 2020. Echidna: effective, usable, and fast fuzzing for smart contracts. Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (2020)

    2020

  25. [33]

    Nielsen Norman Group. n.d.. Usability 101: Introduction to Usability . Nielsen Norman Group. https://www.nngroup.com/articles/usability-101- introduction-to-usability/

  26. [34]

    Ningyu He, Ruiyi Zhang, Haoyu Wang, Lei Wu, Xiapu Luo, Yao Guo, Ting Yu, and Xuxian Jiang. 2021. EOSAFE: Security Analysis of EOSIO Smart Contracts. In 30th USENIX Security Symposium (USENIX Security 21) . USENIX Association, 1271–1288. https://www.usenix.org/conference/ useni...

    2021

  27. [35]

    Matthew Kelly, Christoph Treude, and Alex Murray. 2019. A case study on automated fuzz target generation for large codebases. In Proceedings of the 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’19) . 1–6

    2019

  28. [36]

    Klooster

    et al. Klooster. 2022. Effectiveness and Scalability of Fuzzing Techniques in CI/CD Pipelines. arXiv:2205.14964 [cs.SE]

    2022 arXiv

  29. [37]

    Jun Li, Bodong Zhao, and Chao Zhang. 2018. Fuzzing: a survey. Cybersecurity 1, 1 (2018), 1–13

    2018

  30. [38]

    Yuwei Li, Shouling Ji, Yuan Chen, Sizhuang Liang, Wei-Han Lee, Yueyao Chen, Chenyang Lyu, Chunming Wu, Raheem Beyah, Peng Cheng, Kangjie Lu, and Ting Wang. 2021. UNIFUZZ: A Holistic and Pragmatic Metrics-Driven Platform for Evaluating Fuzzers. In 30th USENIX Security Symposium...

    2021

  31. [39]

    Hongliang Liang, Xiaoxiao Pei, Xiaodong Jia, Wuwei Shen, and Jian Zhang. 2018. Fuzzing: State of the art. IEEE Transactions on Reliability 67, 3 (2018), 1199–1218

    2018

  32. [40]

    Jie Liang, Mingzhe Wang, Yuanliang Chen, Yu Jiang, and Renwei Zhang. 2018. Fuzz testing in practice: Obstacles and solutions. In Proceedings of the 25th International Conference on Software Analysis, Evolution and Reengineering (SANER’18) . 562–566

    2018

  33. [41]

    Fuchen Ma, Zhenyang Xu, Meng Ren, Zijing Yin, Yuanliang Chen, Lei Qiao, Bin Gu, Huizhong Li, Yu Jiang, and Jiaguang Sun. 2021. Pluto: Exposing vulnerabilities in inter-contract scenarios. IEEE Transactions on Software Engineering 48, 11 (2021), 4380–4396

    2021

  34. [42]

    Melvillian. [n. d.]. Invariant test fuzzed functions always use msg.value == 0. https://github.com/foundry-rs/foundry/issues/8449. Accessed: 2025-02-15

    2025

  35. [43]

    Meta. [n. d.]. LLama 3.3 70B. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct/. Accessed: 2025-02-15

    2025

  36. [44]

    mistralai. [n. d.]. Mixtral-8x7B-Instruct-v0.1. https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1. Accessed: 2025-02-15

    2025

  37. [45]

    Roberto Natella. 2022. StateAFL: Greybox fuzzing for stateful network servers.Empirical Software Engineering 27, 7 (4 10 2022), 191. doi:10.1007/s10664- 022-10233-3

    2022 doi

  38. [46]

    Olivier Nourry, Yutaro Kashiwa, Bin Lin, Gabriele Bavota, Michele Lanza, and Yasutaka Kamei. 2023. The Human Side of Fuzzing: Challenges Faced by Developers during Fuzzing Activities. ACM Transactions of Software Engineering and Methodology (2023)

    2023

  39. [47]

    Stephan Plöger, Mischa Meier, and Matthew Smith. 2021. A Qualitative Usability Evaluation of the Clang Static Analyzer and libFuzzer with CS Students and CTF Players. In Seventeenth Symposium on Usable Privacy and Security (SOUPS’21)

    2021

  40. [48]

    Stephan Plöger, Mischa Meier, and Matthew Smith. 2023. A Usability Evaluation of AFL and libFuzzer with CS Students. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23)

    2023

  41. [49]

    Ilham Qasse, Mohammad Hamdaqa, and Björn Þór Jónsson. 2023. Smart Contract Upgradeability on the Ethereum Blockchain Platform: An Exploratory Study. arXiv preprint arXiv:2304.06568 (2023)

    2023 arXiv

  42. [50]

    Meng Ren et al. 2021. Empirical evaluation of smart contract testing: what is the best choice?. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2021)

    2021

  43. [51]

    Sarwar Sayeed, Hector Marco-Gisbert, and Tom Caira. 2020. Smart Contract: Attacks and Protections. IEEE Access 8 (2020), 24416–24427

    2020

  44. [52]

    Chaofan Shou, Shangyin Tan, and Koushik Sen. 2023. ItyFuzz: Snapshot-Based Fuzzer for Smart Contract. Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (2023), 322–333

    2023

  45. [53]

    Chaofan Shou, Shangyin Tan, and Koushik Sen. 2023. ItyFuzz: Snapshot-Based Fuzzer for Smart Contract. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (Seattle, WA, USA) (ISSTA 2023). Association for Computing Machinery, New York,...

    2023

  46. [54]

    Spencer and J.J

    D. Spencer and J.J. Garrett. 2009. Card Sorting: Designing Usable Categories . Rosenfeld Media. https://books.google.ca/books?id=_h4D9gqi5tsC

    2009

  47. [55]

    unboxedtype. [n. d.]. Questions regarding functional capabilities of the tool. https://github.com/crytic/echidna/issues/255. Accessed: 2025-02-15

    2025

  48. [56]

    wadealexc. [n. d.]. Make it easier to re-run failed fuzz tests with verbose logging. https://github.com/foundry-rs/foundry/issues/7206. Accessed: 2025-02-15

    2025

  49. [57]

    et al. Wu. 2024. Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE ’24)

    2024

  50. [58]

    Rui Xi, Zehua Wang, and Karthik Pattabiraman. 2024. POMABuster: Detecting Price Oracle Manipulation Attacks in Decentralized Finance . In 2024 IEEE Symposium on Security and Privacy (SP) . IEEE Computer Society, Los Alamitos, CA, USA, 3923–3942. doi:10.1109/SP54263.2024.00257

    2024

  51. [59]

    J. Yang, X. Zhao, H. Zhang, et al. 2025. CSAFuzzer: Fuzzing smart contracts combining with static analysis. Empirical Software Engineering 30, 1 (2025), 62. doi:10.1007/s10664-025-10623-3

    2025 doi

  52. [60]

    Ynyesto. [n. d.]. Invariant BSC fork test fails. https://github.com/foundry-rs/foundry/issues/748. Accessed: 2025-02-15

    2025

  53. [61]

    target program

    Joobeom Yun, Fayozbek Rustamov, Juhwan Kim, and Youngjoo Shin. 2022. Fuzzing of embedded systems: A survey. Comput. Surveys (2022). Human Side of Smart Contract Fuzzing: An Empirical Study 11 8 Appendix 8.1 LLM GitHub Isuse Summarization Pipeline Fig. 1. Pipeline for Summarizi...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.