Pith. sign in

REVIEW 4 major objections 5 minor 20 references

ContractTrace: Retracing Smart Contract Versions for Security Analyses

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Smart-contract version histories can be traced from proxy delegatecall records, enabling vulnerability lifecycle analysis.

desk verdict A genuinely useful open dataset for smart contract lineages, but the accuracy claim needs independent validation before lineageSet can be treated as ground truth. read the letter →

arxiv 2412.20866 v2 pith:UFYETCYS submitted 2024-12-30 cs.SE

Fatou Ndiaye Mbodji , Vinny Adjibi , Moustapha Awwalou Diouf , Gervais Mendy , Kui Liu , Jacques Klein , Tegawende Bissyande This is my paper · ORCID
classification cs.SE
keywords smartcontractsEthereumproxypatterndelegatecallcontractlineagevulnerabilitylifecyclesimilarity-baseddetectionopendataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ContractTrace proposes that smart-contract version histories can be reconstructed from the blockchain itself: a proxy contract's delegatecall records show which implementation address it forwarded to, and the ordered list of those callees is the contract's lineage. The paper builds this into an automated pipeline that yields lineageSet, an open dataset of 1,055 deployed Ethereum contracts grouped into 347 lineages with 706 predecessor-successor pairs. A security case study uses these lineages to show that roughly half of detected vulnerabilities are newly introduced in successor versions, about one in five disappear, and unresolved vulnerabilities persist on average for 283 days. The paper also evaluates Etherscan's Locality-Sensitive Hashing similarity search against lineageSet and finds that LSH recovers at most 15.8% of true lineage relationships, supporting the claim that similarity-based lineage construction is unreliable.

What carries the argument

The central mechanism is the proxy pattern and its delegatecall instruction: a proxy holds a configurable address and forwards calls to that address while executing them in the proxy's own storage context, so every change of the callee address is a version change of the same logical contract. ContractTrace treats the sequence of callee addresses $\Sigma = S_1 S_2 \cdots S_n$ for a unique proxy as a lineage, and defines the predecessor/successor pair as adjacent elements $S_{i-1}$ and $S_i$. The pipeline finds proxies by the keccak-256 signature of the upgrade function, extracts delegatecall timestamps from on-chain data, and enforces chronological, non-overlapping activity periods (Rule 3) so that each lineage is a linear chain rather than a tree. This design is what converts ordinary transaction history into a version lineage suitable for vulnerability-lifecycle analysis.

What would settle it

Check a sample of lineageSet against on-chain admin-upgrade events and proxy storage slots: if a delegatecall target appears in the lineage but was never announced as an upgrade by the proxy's own events, or if two lineage members have overlapping delegatecall activity while user transactions continue, then the linear-version assumption fails and the lineage contains a false or missing version. Concretely, replay the delegatecall logs for one proxy and compare each callee's activity interval to the timestamps in the proxy's upgrade events; any mismatch would show that delegatecall history alone is not a faithful version history.

Watch

Extended reading notes

Core claim

The paper's central claim is that proxy-based upgrade records, not code similarity, are the trustworthy signal for linking smart-contract versions. Because a proxy executes its logic in the context of a configurable callee address, the on-chain history of delegatecall targets from that proxy identifies the successive implementations of the same logical contract. ContractTrace automates this by locating proxies through the keccak-256 signature of the upgrade function in BigQuery, querying Etherscan for callee source code, and applying three rules: every lineage member must be a callee of the same proxy, a lineage has at least two versions, and versions' delegatecall activity periods are chronological and non-overlapping. Running this pipeline produced lineageSet, and testing the same lineages against the LSH-based similarity engine shows that similarity-based methods achieve high precision only at thresholds so strict that recall falls below 7%, so the conservative proxy-based method is offered as a ground-truth benchmark for future lineage construction.

Load-bearing premise

The load-bearing assumption is that every contract address a proxy forwards to is a genuine new version of the same logical contract and that those versions' active periods never overlap; a proxy that forwards to a library, router, or temporary helper would be silently recorded as a version, and simultaneously active valid versions would be dropped.

Editorial extensions

If this is right

  • Security researchers can use lineageSet instead of Etherscan's flat contract list to compute how long a vulnerability survives: the paper measures an average of 283 days before a vulnerability disappears without a new one appearing.
  • Automated program repair for Ethereum can target the versions where vulnerabilities persist, and the dataset can be rerun periodically to capture newly deployed versions.
  • Similarity-based lineage construction, as implemented by LSH, should be revalidated against lineageSet before being used to answer evolution questions, since even at the low-similarity threshold recall is under 16%.
  • The vulnerability case study's method can be extended to other properties, such as defects, gas consumption, and access-control changes, because the lineage file and function pairing supports comparison at contract, file, and function granularity.
  • Because lineageSet is openly available, future lineage-construction methods can be benchmarked against a common ground truth instead of non-shared private datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the delegatecall assumption is relaxed, the same infrastructure could be extended to other proxy standards, such as EIP-1967 storage slots or transparent proxies, by reading their upgrade events, potentially more than tripling the observed 347-lineage coverage.
  • The 15.8% maximum recall of LSH should not be read as the ceiling of similarity-based methods; a fair benchmark would restrict both methods to the same proxy-upgraded population, since lineageSet contains only proxy-backed contracts by construction.
  • A direct falsification test would compare each lineage's delegatecall timeline against the proxy's own emitted upgrade events, such as Upgraded(address); any timeline that includes a callee never announced as an upgrade is a false version.
  • The reported newly introduced vulnerability rate of 49.19% is a union of three detectors, so it likely overstates true new defects; the dataset enables a sensitivity analysis using intersection or majority voting without re-collecting lineages.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ContractTrace, an automated infrastructure that constructs smart-contract lineages from Ethereum proxy contracts by collecting delegatecall callees and ordering them chronologically (Rules 1–3, Section V-A). The resulting dataset, lineageSet, contains 1,055 contracts grouped into 347 lineages. The authors demonstrate the dataset on a vulnerability-lifecycle case study (Section VI) and evaluate a similarity-based lineage-construction approach (LSH) against lineageSet as ground truth (Section VII), reporting low precision and recall for LSH and concluding that the proxy-based method is superior. The paper frames its contribution as an open, reproducible dataset and infrastructure for studying smart-contract evolution and security.

Significance. If the lineage construction is accurate, lineageSet would be a valuable community resource: it is open, automatically built, and larger than prior lineage datasets, which has been a recognized gap in empirical smart-contract research. The paper also ships a case study that connects lineage information to vulnerability lifecycles, a security-relevant analysis that is otherwise difficult to perform. The LSH evaluation, while problematic as currently designed, addresses a real question about the reliability of similarity-based lineage construction. The authors should be credited for making the dataset and code publicly available and for being transparent about some design choices in Section IX. However, the central claim of accuracy is not yet established: the proxy-callee assumption, the linear-ordering rule, and the ground-truth validation all need stronger support before the dataset can be used as a benchmark.

major comments (4)
  1. [Section V-A, Step 2 (Rule 3)] Rule 1 equates every delegatecall callee of a proxy with a successive version of the same logical contract. Delegatecall is a general EVM primitive: proxies may forward to libraries, routers, or temporarily used addresses for reasons unrelated to version upgrades. The paper provides no independent verification (e.g., manual inspection, project release histories, admin logs, or cross-referencing with verified source-code naming conventions) that the 1,055 collected callees are actual implementations of the same logical contract. Without such verification, the precision of lineageSet is unproven, and the ground-truth status claimed in Section VII is not established. Please provide a quantitative validation on a random sample of lineages, with a concrete protocol for distinguishing genuine versions from unrelated callees.
  2. [Section VII and Table II] Rule 3 forces a linear, non-overlapping ordering of versions within a lineage. As acknowledged in Section IX, this may exclude valid overlapping or branched deployments, but the paper does not quantify how often this occurs or how it affects the downstream vulnerability-lifecycle results in Section VI. Please report the number of callees or proxy-to-callee relationships that were discarded because of overlap or branching, and provide a sensitivity analysis showing how the size and composition of lineageSet change when Rule 3 is relaxed (e.g., allowing overlapping activity with ordering by first delegatecall).
  3. [Section VII, Methodology step (c)] The Key Insight box states that the conservative proxy-based approach has 'significantly higher precision and recall' than LSH, but Table II reports precision and recall only for the LSH-based method. No precision or recall is computed for ContractTrace itself. To support the comparative claim, the authors must compute analogous precision and recall for ContractTrace—ideally against an externally validated ground truth or at least against a manually verified subset—and report the comparison in the same table.
  4. [Section V-A, Step 1 (data collection)] The evaluation uses lineageSet as the ground truth for measuring LSH precision and recall, but lineageSet is produced by the very method being promoted. This is circular: any systematic bias in ContractTrace (e.g., false inclusions from Rule 1 or false exclusions from Rule 3) is inherited by the ground truth and propagates into the LSH evaluation. Please add an independent ground truth for at least a stratified random sample of lineages, and report the agreement between ContractTrace and that independent ground truth (e.g., Cohen's kappa or a confusion matrix at the pair or lineage level).
minor comments (5)
  1. [Table I] Table I reports 706 predecessor/successor pairs, but the arithmetic from the stated lineage counts (1,055 contracts minus 347 lineages, each lineage of size n contributing n-1 pairs) yields 708 pairs. Please reconcile this discrepancy or correct the table.
  2. [Section VI] The filename similarity threshold of 'two-character difference' is an ad-hoc free parameter. Please state explicitly how this threshold was chosen and provide the sensitivity of the resulting file-pair statistics (e.g., the 98% average similarity) to this parameter.
  3. [Section VI] The vulnerability-lifecycle case study reports percentages (e.g., 49.19% newly introduced, 21.53% disappeared) but does not state the number of vulnerabilities these correspond to or how the union/intersection of tools was exactly combined. Please clarify the denominator and the tool-combination procedure to make the numbers reproducible.
  4. [Section V-B] The description of LSH as implemented in Etherscan is brief and does not specify what exactly is fingerprinted (bytecode, source code, or metadata) or how the low/medium/high thresholds are defined. Without this, the interpretability of Table II is limited. Please provide a reference to the LSH implementation or a precise specification.
  5. [Section V-B] The caption of Figure 2 is incomplete ('Sizes of lineages in lineageSet' is followed by a fragment). Please revise the caption and ensure all figures are referenced in the text with the correct number (the text appears to reference Figure 1 from another work in Section II-B but does not clearly label it).

Circularity Check

2 steps flagged · score 4.0 of 10

Evaluation is self-referential: lineageSet is both ContractTrace's output and the ground truth used to confirm its accuracy, so the 'higher precision and recall' claim for the proxy-based approach is true by construction.

  1. self definitional [Section VII (Evaluation methodology), citing lineages produced in Section V-B]
    "Ground Truth Lineages Data : lineageSet is used as the benchmark for evaluating the performance of the LSH-based method. ... we leverage lineageSet, built conservatively, as a ground truth dataset."

    lineageSet is produced by ContractTrace in Section V-B ('Following the implementation and execution of the two-step process outlined above, we obtain lineageSet'). Using that same artifact as ground truth to 'validated our lineage detection methodology' (abstract) means the evaluation compares LSH against ContractTrace's own output, not against an independent oracle. Because Rule 1 defines every delegatecall callee of a proxy as a lineage member and Rule 3 forces a linear chronology, lineageSet cannot falsify the core assumption that delegatecall callees are true contract versions. The claimed confirmation of accuracy is therefore circular with respect to the tool's construction rules.

  2. other [Section VII, 'Key Insight' box]
    "Low Recall Across All Scenarios: Similarity-based approach to building lineages for smart contracts leads to low recall. When considering all contracts in Ethereum, the conservative proxy-based approach used to build lineageSet has a significantly higher precision and recall."

    No precision or recall is computed for the proxy-based approach in Table II; only LSH precision and recall are reported. Since lineageSet serves as the ground-truth benchmark and is itself built by the proxy-based approach, the proxy-based approach's precision and recall against that benchmark are 100% by construction. The 'Key Insight' therefore does not empirically establish that ContractTrace outperforms LSH in accuracy; it restates that the benchmark was generated by the method being promoted. The abstract's claim of 'confirming the security relevance and accuracy of our technique' is not supported by an independent measurement.

full rationale

The construction of lineageSet itself is not definitionally circular in a harmful way: it is grounded in observable on-chain proxy delegatecall events, which is an external data source, and the paper makes its classification rules explicit. The circularity arises in the evaluation layer. lineageSet is both the output of ContractTrace and the ground truth against which the alternative LSH approach is measured, so the evaluation can only measure how well LSH agrees with ContractTrace's own labeling decisions. The paper does not independently verify that every delegatecall callee is a genuine version (libraries, routers, and temporary callees could be mislabeled under Rule 1), and the authors acknowledge in Section IX that Rule 3 may exclude legitimate overlapping versions. These threats mean the reported LSH precision/recall numbers are relative to an unvalidated benchmark, and the 'Key Insight' that the proxy-based approach has higher precision and recall is true by construction rather than by measurement. This warrants a moderate circularity score, but not a high one, because the dataset is openly available, the pipeline is transparent, and the limitation is at least partially acknowledged.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or theoretical entities are proposed. ContractTrace and lineageSet are implemented software artifacts with open code, so they are not invented entities in the sense of unsupported postulates. The main unresolved premises are the proxy-to-lineage equivalence and the linear ordering rule.

free parameters (2)
  • filename matching threshold = two-character difference
    In Section V-A Step 2, predecessor/successor file pairs are matched when filenames differ by at most two characters; this threshold is chosen by hand and affects the file-level dataset, but is not a fitted constant in a model.
  • minimum lineage size = 2 versions
    Rule 2 defines a lineage only when len(Σ) ≥ 2; this design choice determines the composition of lineageSet and is not derived from data.
assumptions (4)
  • domain assumption Etherscan source verification guarantees that submitted source code matches deployed bytecode at the given address.
    Section V-A Step 1 relies on Etherscan verified contracts; if verification can be gamed, lineages built from source code may not reflect deployed code.
  • ad hoc to paper Every delegatecall target of a proxy is a successive version of the same logical contract, with no unrelated callees.
    Rule 1 in Section V-A Step 2 treats all callee addresses reached by proxy delegatecalls as lineage members; proxies may also call unrelated contracts such as libraries, making false-positive lineages possible.
  • ad hoc to paper Contract versions in a lineage are linearly ordered with no overlapping activity periods.
    Rule 3 in Section V-A Step 2 imposes lastDelegateCall(P, Si) < firstDelegateCall(P, Si+1); this excludes valid lineage branches and can drop versions, as acknowledged in Section IX.
  • domain assumption BigQuery and Etherscan provide complete and current enumeration of proxy and upgrade transactions.
    Section V-A Step 1 uses Google BigQuery to identify contracts calling upgradeProxy; missed transactions or incomplete datasets would omit lineage members.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ContractTrace: Retracing Smart Contract Versions for Security Analyses." pith.science (2026). https://pith.science/paper/UFYETCYS

@misc{pith2026241220866,
  author       = {Pith},
  title        = {Pith review of: ContractTrace: Retracing Smart Contract Versions for Security Analyses},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UFYETCYS}},
  note         = {Machine review of arXiv:2412.20866}
}
read the original abstract

Due to the inherent immutability of blockchain technology, smart contract updates require their deployment at new addresses rather than modifying existing ones, thus fragmenting version histories and creating critical blind spots for analyses. Indeed, for example, this fragmentation severely hinders security researchers ability to track vulnerability lifecycles across contract versions. While platforms like Etherscan provide detailed information about Ethereum smart contracts, they lack crucial functionality to trace predecessor-successor relationships within smart contract lineages, preventing systematic analysis of how vulnerabilities emerge, propagate, and potentially remain unresolved across versions.To address the challenge of tracing smart contract lineages, we adopt a Design Science Research (DSR) approach and introduce ContractTrace, an automated infrastructure that accurately identifies and links versions of smart contracts into coherent lineages. This tool enables the construction of lineageSet, an up-to-date, open-source dataset specifically designed to support security research on vulnerability, defect or any other property evolution patterns in smart contracts. Through a security-focused case study we demonstrate how ContractTrace reveals previously obscured vulnerability life-cycles within smart contract lineages, tracking whether critical security flaws persist or get resolved across versions. This capability is essential for understanding vulnerability propagation patterns and evaluating the effectiveness of security patches in blockchain environments. In the evaluation phase of our DSR approach, we validated our lineage detection methodology against an alternative approach using Locality-Sensitive Hashing (LSH) to cluster contract versions, confirming the security relevance and accuracy of our technique.

Figures

Figures reproduced from arXiv: 2412.20866 by the authors.

Figure 1
Figure 1. An Example of the Upgradable Contract from Chen et al. [8]. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Sizes of lineages in lineageSet In figure 2, we classify lineages according to their size in terms of their number of contract versions. The lineage with the most number of contract versions encompasses 14 versions of that contract. one hundred seventy (170) smart contract lineages have only two (2) versions collected. It takes an average of 23 days to deploy a new version of a smart contract. We have restructured t… view at source ↗
Figure 3
Figure 3. Distribution of Similarity Rates in Predecessor/Successor Pairs of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Q1.1: Vulnerabilities and Vulnerable Files by Tool [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

  1. [1]

    Why do smart contracts self-destruct? investigating the selfdestruct function on ethereum,

    J. Chen, X. Xia, D. Lo, and J. Grundy, “Why do smart contracts self-destruct? investigating the selfdestruct function on ethereum,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 31, no. 2, pp. 1–37, 2021

    work page 2021

  2. [2]

    Recom- mending differentiated code to support smart contract update,

    Y . Huang, Q. Kong, N. Jia, X. Chen, and Z. Zheng, “Recom- mending differentiated code to support smart contract update,” in 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC). IEEE, 2019, pp. 260–270

    work page 2019

  3. [3]

    Characterizing code clones in the ethereum smart contract ecosystem,

    N. He, L. Wu, H. Wang, Y . Guo, and X. Jiang, “Characterizing code clones in the ethereum smart contract ecosystem,” in Financial Cryptography and Data Security: 24th International Conference, FC 2020, Kota Kinabalu, Malaysia, February 10– 14, 2020 Revised Selected Papers 24 . Springer, 2020, pp. 654– 675

    work page 2020

  4. [4]

    An organized repos- itory of ethereum smart contracts’ source codes and metrics,

    G. A. Pierro, R. Tonelli, and M. Marchesi, “An organized repos- itory of ethereum smart contracts’ source codes and metrics,” Future internet, vol. 12, no. 11, p. 197, 2020

    work page 2020

  5. [5]

    Formalizing and securing relationships on public networks,

    N. Szabo, “Formalizing and securing relationships on public networks,” First monday, 1997

    work page 1997

  6. [6]

    Bitcoin: A peer-to-peer electronic cash system,

    S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” Decentralized Business Review , p. 21260, 2008

    work page 2008

  7. [7]

    A next-generation smart contract and decen- tralized application platform,

    V . Buterin et al., “A next-generation smart contract and decen- tralized application platform,” white paper , vol. 3, no. 37, pp. 2–1, 2014

    work page 2014

  8. [8]

    Maintenance- related concerns for post-deployed ethereum smart contract de- velopment: issues, techniques, and future challenges,

    J. Chen, X. Xia, D. Lo, J. Grundy, and X. Yang, “Maintenance- related concerns for post-deployed ethereum smart contract de- velopment: issues, techniques, and future challenges,” Empirical Software Engineering, vol. 26, no. 6, p. 117, 2021

    work page 2021

Show all 20 references
  1. [9]

    Smartembed: A tool for clone and bug detection in smart contracts through structural code embedding,

    Z. Gao, V . Jayasundara, L. Jiang, X. Xia, D. Lo, and J. Grundy, “Smartembed: A tool for clone and bug detection in smart contracts through structural code embedding,” in 2019 IEEE In- ternational Conference on Software Maintenance and Evolution (ICSME). IEEE, 2019, pp. 394–397

    2019

  2. [10]

    Deckard: Scalable and accurate tree-based detection of code clones,

    L. Jiang, G. Misherghi, Z. Su, and S. Glondu, “Deckard: Scalable and accurate tree-based detection of code clones,” in 29th International Conference on Software Engineering (ICSE’07) . IEEE, 2007, pp. 96–105

    2007

  3. [11]

    Nicad: Accurate detection of near- miss intentional clones using flexible pretty-printing and code normalization,

    C. K. Roy and J. R. Cordy, “Nicad: Accurate detection of near- miss intentional clones using flexible pretty-printing and code normalization,” in 2008 16th iEEE international conference on program comprehension. IEEE, 2008, pp. 172–181

    2008

  4. [12]

    Similarity search in high dimensions via hashing,

    A. Gionis, P. Indyk, R. Motwani et al. , “Similarity search in high dimensions via hashing,” in Vldb, vol. 99, no. 6, 1999, pp. 518–529

    1999

  5. [13]

    An empirical study on real bug fixes from solidity smart contract projects,

    Y . Wang, X. Chen, Y . Huang, H.-N. Zhu, J. Bian, and Z. Zheng, “An empirical study on real bug fixes from solidity smart contract projects,” Journal of Systems and Software , vol. 204, p. 111787, 2023

    2023

  6. [14]

    Slither: a static analysis framework for smart contracts,

    J. Feist, G. Grieco, and A. Groce, “Slither: a static analysis framework for smart contracts,” in 2019 IEEE/ACM 2nd Inter- national Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB) . IEEE, 2019, pp. 8–15

    2019

  7. [15]

    https://github.com/ConsenSys/mythril

  8. [16]

    https://github.com/nveloso/conkas

  9. [17]

    Empirical review of automated analysis tools on 47,587 ethereum smart contracts,

    T. Durieux, J. F. Ferreira, R. Abreu, and P. Cruz, “Empirical review of automated analysis tools on 47,587 ethereum smart contracts,” in Proceedings of the ACM/IEEE 42nd International conference on software engineering , 2020, pp. 530–541

    2020

  10. [18]

    Megadiff: A dataset of 600k java source code changes categorized by diff size,

    M. Monperrus, M. Martinez, H. Ye, F. Madeiral, T. Durieux, and Z. Yu, “Megadiff: A dataset of 600k java source code changes categorized by diff size,” arXiv preprint arXiv:2108.04631 , 2021

    2021 arXiv

  11. [19]

    Improving the precision of automatic program repair with machine learning,

    H. Ye, “Improving the precision of automatic program repair with machine learning,” Ph.D. dissertation, KTH Royal Institute of Technology, 2023

    2023

  12. [20]

    Ac/c++ code vulnerability dataset with code changes and cve summaries,

    J. Fan, Y . Li, S. Wang, and T. N. Nguyen, “Ac/c++ code vulnerability dataset with code changes and cve summaries,” in Proceedings of the 17th International Conference on Mining Software Repositories, 2020, pp. 508–512

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.