{"id":"207f3a15-3ca9-49e6-9cab-0c0d67ad56bf","arxiv_id":"2412.20866","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ContractTrace builds smart contract lineages from Ethereum proxy delegatecall records, yielding an open dataset and showing that similarity-based lineage methods achieve low recall.","lead":"This paper introduces ContractTrace, a tool that uses proxy contracts on Ethereum to automatically link successive versions of smart contracts into lineages, and an open dataset of 1,055 contracts in 347 lineages. The resource is meant to help researchers track how vulnerabilities appear, persist, and get fixed across deployed contract versions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proxy-callee assumption in Rule 1 conflates delegatecalls with version upgrades; without independent verification, lineageSet's ground-truth status is unproven and the LSH evaluation may inherit label noise.","rationale":"The reader's weakest-assumption identification is correct and matches my independent reading of the paper. The central argument requires that proxy delegatecall history uniquely and completely traces the sequence of real contract versions. That condition is most insecure in Rule 1 (all callees are lineage members) and Rule 3 (strict linear non-overlap), exactly as the reader states. The paper's own Section IX acknowledges the overlap/coverage limitation but provides no independent validation of the proxy-to-upgrade equivalence. Because Section VII uses lineageSet as ground truth for comparing LSH-based lineage construction, label noise in lineageSet directly biases the reported precision and recall, so the accuracy question is not cosmetic; it is load-bearing for the paper's main contributions. I therefore agree with the reader's conditional verdict. I do not recommend a full rejection: the artifact is openly available, the dataset is reproducible from on-chain data, and the case study is a sensible demonstration. The appropriate remedy is a targeted validation step, which is exactly what a conditional acceptance should require. I also note that the paper contains explicit limitation statements in Section IX that strengthen its credibility, but those statements do not substitute for quantitative validation of the lineage labels. The proposed concrete test—checking implementation-slot updates and full delegatecall traces on a random sample—would settle whether the concern actually lands because it directly measures the false-discovery rate of the lineage construction rules.","tokens_in":12178,"tokens_out":2824,"duration_ms":30000,"concrete_test":"Sample, say, 50 lineages from lineageSet uniformly at random. For each callee address in each sampled lineage, extract the proxy's implementation-slot history from chain data: look for EIP-1967 storage slot 0x360894a13ba1a3210667c828492db98dca3e2076cc3735a920a3ca505d382bb changes or TransparentUpgradeableProxy/ProxyAdmin 'Upgraded' events, and check that every lineage callee appears as the implementation slot value after an authorized upgrade transaction. Independently verify that no callee address is ever delegatecalled only as a library or as a one-off router, and scan the proxy's full trace for delegatecalls to addresses absent from the lineage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that ContractTrace 'accurately identifies and links versions of smart contracts into coherent lineages' and that lineageSet can serve as ground truth. The load-bearing condition for this claim is that every delegatecall from a proxy is an upgrade to the same logical contract, and never a library call, a router call, or a temporary use. Section V-A Step 1 collects callees via the 'upgradeProxy' method signature, but Step 2's Rule 1 then promotes every callee ever delegatecalled by the proxy into the lineage, while Rule 3 forces a linear, non-overlapping ordering. The paper reports 1,055 contracts across 347 lineages, but no independent check establishes that these addresses are actually implemented versions of the same project. Delegatecall is a general EVM primitive: proxies and non-proxy contracts routinely delegatecall libraries (e.g., OpenZeppelin's Address or SafeERC20) and router-style contracts, and a proxy can be reconfigured to a temporary address before reverting. If any such callee enters Rule 1, it silently becomes a false lineage member. Conversely, Rule 3 drops valid simultaneous versions, a limitation the authors explicitly acknowledge in Section IX, but they do not quantify it. The consequence is not only a possible overestimate of lineageSet's precision; Section VII uses lineageSet as the ground truth for the LSH evaluation, so any label error propagates directly into the reported precision/recall numbers and weakens the paper's key comparison result. No independent source, such as EIP-1967 implementation-slot updates, proxy admin event logs, project release histories, or manual lineage inspection, is used to validate the proxy-to-lineage equivalence, despite the paper's stated goal of a high-confidence dataset.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ContractTrace, an automated infrastructure that constructs smart-contract lineages from Ethereum proxy contracts by collecting delegatecall callees and ordering them chronologically (Rules 1–3, Section V-A). The resulting dataset, lineageSet, contains 1,055 contracts grouped into 347 lineages. The authors demonstrate the dataset on a vulnerability-lifecycle case study (Section VI) and evaluate a similarity-based lineage-construction approach (LSH) against lineageSet as ground truth (Section VII), reporting low precision and recall for LSH and concluding that the proxy-based method is superior. The paper frames its contribution as an open, reproducible dataset and infrastructure for studying smart-contract evolution and security.","tokens_in":12585,"tokens_out":2387,"duration_ms":25647,"significance":"If the lineage construction is accurate, lineageSet would be a valuable community resource: it is open, automatically built, and larger than prior lineage datasets, which has been a recognized gap in empirical smart-contract research. The paper also ships a case study that connects lineage information to vulnerability lifecycles, a security-relevant analysis that is otherwise difficult to perform. The LSH evaluation, while problematic as currently designed, addresses a real question about the reliability of similarity-based lineage construction. The authors should be credited for making the dataset and code publicly available and for being transparent about some design choices in Section IX. However, the central claim of accuracy is not yet established: the proxy-callee assumption, the linear-ordering rule, and the ground-truth validation all need stronger support before the dataset can be used as a benchmark.","major_comments":[{"comment":"Rule 1 equates every delegatecall callee of a proxy with a successive version of the same logical contract. Delegatecall is a general EVM primitive: proxies may forward to libraries, routers, or temporarily used addresses for reasons unrelated to version upgrades. The paper provides no independent verification (e.g., manual inspection, project release histories, admin logs, or cross-referencing with verified source-code naming conventions) that the 1,055 collected callees are actual implementations of the same logical contract. Without such verification, the precision of lineageSet is unproven, and the ground-truth status claimed in Section VII is not established. Please provide a quantitative validation on a random sample of lineages, with a concrete protocol for distinguishing genuine versions from unrelated callees.","section":"Section V-A, Step 2 (Rule 3)"},{"comment":"Rule 3 forces a linear, non-overlapping ordering of versions within a lineage. As acknowledged in Section IX, this may exclude valid overlapping or branched deployments, but the paper does not quantify how often this occurs or how it affects the downstream vulnerability-lifecycle results in Section VI. Please report the number of callees or proxy-to-callee relationships that were discarded because of overlap or branching, and provide a sensitivity analysis showing how the size and composition of lineageSet change when Rule 3 is relaxed (e.g., allowing overlapping activity with ordering by first delegatecall).","section":"Section VII and Table II"},{"comment":"The Key Insight box states that the conservative proxy-based approach has 'significantly higher precision and recall' than LSH, but Table II reports precision and recall only for the LSH-based method. No precision or recall is computed for ContractTrace itself. To support the comparative claim, the authors must compute analogous precision and recall for ContractTrace—ideally against an externally validated ground truth or at least against a manually verified subset—and report the comparison in the same table.","section":"Section VII, Methodology step (c)"},{"comment":"The evaluation uses lineageSet as the ground truth for measuring LSH precision and recall, but lineageSet is produced by the very method being promoted. This is circular: any systematic bias in ContractTrace (e.g., false inclusions from Rule 1 or false exclusions from Rule 3) is inherited by the ground truth and propagates into the LSH evaluation. Please add an independent ground truth for at least a stratified random sample of lineages, and report the agreement between ContractTrace and that independent ground truth (e.g., Cohen's kappa or a confusion matrix at the pair or lineage level).","section":"Section V-A, Step 1 (data collection)"}],"minor_comments":[{"comment":"Table I reports 706 predecessor/successor pairs, but the arithmetic from the stated lineage counts (1,055 contracts minus 347 lineages, each lineage of size n contributing n-1 pairs) yields 708 pairs. Please reconcile this discrepancy or correct the table.","section":"Table I"},{"comment":"The filename similarity threshold of 'two-character difference' is an ad-hoc free parameter. Please state explicitly how this threshold was chosen and provide the sensitivity of the resulting file-pair statistics (e.g., the 98% average similarity) to this parameter.","section":"Section VI"},{"comment":"The vulnerability-lifecycle case study reports percentages (e.g., 49.19% newly introduced, 21.53% disappeared) but does not state the number of vulnerabilities these correspond to or how the union/intersection of tools was exactly combined. Please clarify the denominator and the tool-combination procedure to make the numbers reproducible.","section":"Section VI"},{"comment":"The description of LSH as implemented in Etherscan is brief and does not specify what exactly is fingerprinted (bytecode, source code, or metadata) or how the low/medium/high thresholds are defined. Without this, the interpretability of Table II is limited. Please provide a reference to the LSH implementation or a precise specification.","section":"Section V-B"},{"comment":"The caption of Figure 2 is incomplete ('Sizes of lineages in lineageSet' is followed by a fragment). Please revise the caption and ensure all figures are referenced in the text with the correct number (the text appears to reference Figure 1 from another work in Section II-B but does not clearly label it).","section":"Section V-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a real gap and the dataset has clear potential value, but the validation currently rests on assumptions that are not independently checked. I would like to see the authors add at least a manual or semi-automated validation of a representative sample of lineages, quantify the effect of Rule 3, and compute actual precision/recall for ContractTrace rather than only for LSH. If those additions substantially support the current claims, the paper could become acceptable; in its present form, the ground-truth assertion is premature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"ContractTrace builds a lineage dataset for Ethereum smart contracts from proxy delegatecall records and makes it open. That is the real contribution: a shared, automatically built corpus of 1,055 contracts in 347 lineages, where prior work kept such data private. The vulnerability lifecycle case study shows the dataset can be used, and the LSH comparison usefully demonstrates that similarity-based clustering recovers only a small fraction of proxy-derived lineages.\n\nThe main soft spot is the shift from \"proxy calls this address\" to \"this address is a version of the same logical contract.\" Delegatecall is a general primitive; proxies can point at libraries, routers, or temporary addresses, and Rule 3's non-overlap constraint drops versions that were active simultaneously. The authors acknowledge the latter in Section IX, but they do not quantify it, and they never independently check the proxy-to-lineage equivalence against release histories, EIP-1967 slots, or admin logs. That matters because lineageSet is then used as ground truth for the LSH evaluation, so label noise from Rules 1 and 3 propagates directly into the precision/recall figures. The key insight that ContractTrace has \"significantly higher precision and recall\" than LSH is not actually measured; only LSH's performance against lineageSet is reported.\n\nNone of this kills the artifact. The dataset is a useful starting point for empirical work on smart contract evolution, and the authors are upfront about the main limitations in the threats-to-validity section. Researchers studying vulnerability lifecycles, upgrade patterns, or empirical software engineering on Ethereum will get value from it, even if they should treat lineageSet as a candidate set rather than confirmed truth. The paper deserves a serious referee. What it should not do is present lineageSet as validated ground truth without at least a manual audit or a comparison against a second source.","headline":"A genuinely useful open dataset for smart contract lineages, but the accuracy claim needs independent validation before lineageSet can be treated as ground truth.","tokens_in":13039,"tokens_out":3055,"would_cite":true,"duration_ms":30476,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Smart-contract version histories can be traced from proxy delegatecall records, enabling vulnerability lifecycle analysis.","keywords":["smart contracts","Ethereum","proxy pattern","delegatecall","contract lineage","vulnerability lifecycle","similarity-based lineage detection","open dataset"],"falsifier":"Check a sample of lineageSet against on-chain admin-upgrade events and proxy storage slots: if a delegatecall target appears in the lineage but was never announced as an upgrade by the proxy's own events, or if two lineage members have overlapping delegatecall activity while user transactions continue, then the linear-version assumption fails and the lineage contains a false or missing version. Concretely, replay the delegatecall logs for one proxy and compare each callee's activity interval to the timestamps in the proxy's upgrade events; any mismatch would show that delegatecall history alone is not a faithful version history.","tokens_in":11974,"feed_emoji":"🔗","tokens_out":7118,"duration_ms":66732,"temperature":0.7,"pith_summary":"ContractTrace proposes that smart-contract version histories can be reconstructed from the blockchain itself: a proxy contract's delegatecall records show which implementation address it forwarded to, and the ordered list of those callees is the contract's lineage. The paper builds this into an automated pipeline that yields lineageSet, an open dataset of 1,055 deployed Ethereum contracts grouped into 347 lineages with 706 predecessor-successor pairs. A security case study uses these lineages to show that roughly half of detected vulnerabilities are newly introduced in successor versions, about one in five disappear, and unresolved vulnerabilities persist on average for 283 days. The paper also evaluates Etherscan's Locality-Sensitive Hashing similarity search against lineageSet and finds that LSH recovers at most 15.8% of true lineage relationships, supporting the claim that similarity-based lineage construction is unreliable.","feed_headline":"Proxy logs link 1,055 Ethereum contracts into 347 lineages","feed_subtitle":"Open dataset lets researchers track when smart-contract vulnerabilities appear, persist, and get fixed.","key_machinery":"The central mechanism is the proxy pattern and its delegatecall instruction: a proxy holds a configurable address and forwards calls to that address while executing them in the proxy's own storage context, so every change of the callee address is a version change of the same logical contract. ContractTrace treats the sequence of callee addresses $\\Sigma = S_1 S_2 \\cdots S_n$ for a unique proxy as a lineage, and defines the predecessor/successor pair as adjacent elements $S_{i-1}$ and $S_i$. The pipeline finds proxies by the keccak-256 signature of the upgrade function, extracts delegatecall timestamps from on-chain data, and enforces chronological, non-overlapping activity periods (Rule 3) so that each lineage is a linear chain rather than a tree. This design is what converts ordinary transaction history into a version lineage suitable for vulnerability-lifecycle analysis.","core_discovery":"The paper's central claim is that proxy-based upgrade records, not code similarity, are the trustworthy signal for linking smart-contract versions. Because a proxy executes its logic in the context of a configurable callee address, the on-chain history of delegatecall targets from that proxy identifies the successive implementations of the same logical contract. ContractTrace automates this by locating proxies through the keccak-256 signature of the upgrade function in BigQuery, querying Etherscan for callee source code, and applying three rules: every lineage member must be a callee of the same proxy, a lineage has at least two versions, and versions' delegatecall activity periods are chronological and non-overlapping. Running this pipeline produced lineageSet, and testing the same lineages against the LSH-based similarity engine shows that similarity-based methods achieve high precision only at thresholds so strict that recall falls below 7%, so the conservative proxy-based method is offered as a ground-truth benchmark for future lineage construction.","pith_inferences":["If the delegatecall assumption is relaxed, the same infrastructure could be extended to other proxy standards, such as EIP-1967 storage slots or transparent proxies, by reading their upgrade events, potentially more than tripling the observed 347-lineage coverage.","The 15.8% maximum recall of LSH should not be read as the ceiling of similarity-based methods; a fair benchmark would restrict both methods to the same proxy-upgraded population, since lineageSet contains only proxy-backed contracts by construction.","A direct falsification test would compare each lineage's delegatecall timeline against the proxy's own emitted upgrade events, such as Upgraded(address); any timeline that includes a callee never announced as an upgrade is a false version.","The reported newly introduced vulnerability rate of 49.19% is a union of three detectors, so it likely overstates true new defects; the dataset enables a sensitivity analysis using intersection or majority voting without re-collecting lineages."],"forward_implications":["Security researchers can use lineageSet instead of Etherscan's flat contract list to compute how long a vulnerability survives: the paper measures an average of 283 days before a vulnerability disappears without a new one appearing.","Automated program repair for Ethereum can target the versions where vulnerabilities persist, and the dataset can be rerun periodically to capture newly deployed versions.","Similarity-based lineage construction, as implemented by LSH, should be revalidated against lineageSet before being used to answer evolution questions, since even at the low-similarity threshold recall is under 16%.","The vulnerability case study's method can be extended to other properties, such as defects, gas consumption, and access-control changes, because the lineage file and function pairing supports comparison at contract, file, and function granularity.","Because lineageSet is openly available, future lineage-construction methods can be benchmarked against a common ground truth instead of non-shared private datasets."],"supporting_citations":[{"why":"Defines the predecessor/successor notion and a similarity-plus-manual-check method that this paper contrasts with its proxy-based approach.","marker":"[1]"},{"why":"Builds lineages by same creator and similarity threshold, the method whose reliability is questioned.","marker":"[2]"},{"why":"Documents heavy code reuse in Ethereum, motivating the claim that similarity-based lineage construction is unreliable.","marker":"[3]"},{"why":"Provides the organized smart-contract repository that lineageSet extends with lineage information.","marker":"[4]"},{"why":"Reports developer survey data on upgrade practices, supporting the proxy-only coverage decision.","marker":"[8]"},{"why":"Supplies the SmartEmbed similarity embedding used in the prior predecessor/successor approach.","marker":"[9]"},{"why":"Describes LSH similarity search, the mechanism behind the Etherscan engine used in evaluation.","marker":"[12]"},{"why":"Slither is one of the three vulnerability detectors applied to lineage versions in the case study.","marker":"[14]"},{"why":"Empirical review of vulnerability detection tools that motivates the tool selection and the union/intersection interpretation.","marker":"[17]"}],"fun_headline_variants":["Proxy logs link 1,055 Ethereum contracts into 347 lineages","Track smart-contract vulnerabilities across 347 lineages from proxy history","Proxy-based lineage detection beats code similarity for smart contracts","New dataset links 1,055 smart contracts into 347 version lineages","Smart-contract lineage tracking via proxy logs exposes vulnerability trends"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that every contract address a proxy forwards to is a genuine new version of the same logical contract and that those versions' active periods never overlap; a proxy that forwards to a library, router, or temporary helper would be silently recorded as a version, and simultaneously active valid versions would be dropped.","fun_headline_variants_meta":{"raw":{"variants":["Proxy logs link 1,055 Ethereum contracts into 347 lineages","Track smart-contract vulnerabilities across 347 lineages from proxy history","Proxy-based lineage detection beats code similarity for smart contracts","New dataset links 1,055 smart contracts into 347 version lineages","Smart-contract lineage tracking via proxy logs exposes vulnerability trends"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000649,"raw_usage":{"total_tokens":3000,"prompt_tokens":989,"completion_tokens":2011,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":1925}},"tokens_in":605,"tokens_out":2011,"duration_ms":16568,"temperature":1.0,"reasoning_tokens":1925,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:07:36.415068+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check a sample of lineageSet against on-chain admin-upgrade events and proxy storage slots: if a delegatecall target appears in the lineage but was never announced as an upgrade by the proxy's own events, or if two lineage members have overlapping delegatecall activity while user transactions continue, then the linear-version assumption fails and the lineage contains a false or missing version. Concretely, replay the delegatecall logs for one proxy and compare each callee's activity interval to the timestamps in the proxy's upgrade events; any mismatch would show that delegatecall history alone is not a faithful version history.","supporting_citations":[{"cited_title":"Why do smart contracts self-destruct? investigating the selfdestruct function on ethereum,","cited_arxiv_id":null,"evidence_quote":"Defines the predecessor/successor notion and a similarity-plus-manual-check method that this paper contrasts with its proxy-based approach."},{"cited_title":"Recom- mending differentiated code to support smart contract update,","cited_arxiv_id":null,"evidence_quote":"Builds lineages by same creator and similarity threshold, the method whose reliability is questioned."},{"cited_title":"Characterizing code clones in the ethereum smart contract ecosystem,","cited_arxiv_id":null,"evidence_quote":"Documents heavy code reuse in Ethereum, motivating the claim that similarity-based lineage construction is unreliable."},{"cited_title":"An organized repos- itory of ethereum smart contracts’ source codes and metrics,","cited_arxiv_id":null,"evidence_quote":"Provides the organized smart-contract repository that lineageSet extends with lineage information."},{"cited_title":"Maintenance- related concerns for post-deployed ethereum smart contract de- velopment: issues, techniques, and future challenges,","cited_arxiv_id":null,"evidence_quote":"Reports developer survey data on upgrade practices, supporting the proxy-only coverage decision."},{"cited_title":"Smartembed: A tool for clone and bug detection in smart contracts through structural code embedding,","cited_arxiv_id":null,"evidence_quote":"Supplies the SmartEmbed similarity embedding used in the prior predecessor/successor approach."},{"cited_title":"Similarity search in high dimensions via hashing,","cited_arxiv_id":null,"evidence_quote":"Describes LSH similarity search, the mechanism behind the Etherscan engine used in evaluation."},{"cited_title":"Slither: a static analysis framework for smart contracts,","cited_arxiv_id":null,"evidence_quote":"Slither is one of the three vulnerability detectors applied to lineage versions in the case study."},{"cited_title":"Empirical review of automated analysis tools on 47,587 ethereum smart contracts,","cited_arxiv_id":null,"evidence_quote":"Empirical review of vulnerability detection tools that motivates the tool selection and the union/intersection interpretation."}],"review_version":1}