{"id":"e41e1f6a-cb69-43f5-988d-bc504b3de876","arxiv_id":"2508.02397","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"JC-Finder detects Java third-party library reuse via class-level code clone matching, achieving F1 of 0.818 and finding 26.20% more libraries than package-manager-based scans alone.","lead":"This paper presents JC-Finder, a tool that detects when Java projects copy code from third-party libraries instead of declaring them as dependencies. It finds such reuse faster and more accurately than a function-level baseline, which matters for security audits and license compliance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PageRank cutoff is fitted on the same ground truth used for the headline F1, so recall 0.986 may be overfit; a held-out calibration check is needed.","rationale":"The reader identified the PageRank calibration as the primary fragile premise, and my analysis reaches the same conclusion. The central claim of JC-Finder is that class-level features with linked-function ASTs and supporting-class filtering outperform function-level clone-based SCA. For that claim to hold, the feature-refinement pipeline must work on unseen projects, not just on the 1,000 projects whose labels were used to set the centrality threshold. The paper's own discussion acknowledges missing-source-JAR misattribution and web-source false positives, but those are documented limitations rather than hidden fitting. The PageRank cutoff, in contrast, is a silent selection on the evaluation set: Section 4.2.2 estimates the 50% threshold from 385 ground-truth clone pairs, and Section 5.2 then reports accuracy on the same ground truth. This makes the reported recall of 0.986 not an independent estimate of generalization performance. Because the filter removes 50% of class features, the risk is concrete: if real reuse often involves helper or low-centrality classes underrepresented in the small 385-pair sample, the tool will miss those TPLs in practice. The proposed held-out calibration is a direct, low-cost check that would settle whether the headline numbers are robust. My verdict remains CONDITIONAL, matching the reader's assessment: the approach is promising and the empirical design is mostly sound, but the headline effectiveness numbers need independent validation before they can be accepted as stated.","tokens_in":19455,"tokens_out":2965,"duration_ms":36612,"concrete_test":"Split the 1,000 ground-truth projects into two disjoint halves (or use repeated 2-fold cross-validation). Fit the PageRank percentile cutoff using only the clone pairs from the first half, then evaluate precision and recall on the second half using that fixed cutoff; repeat with the halves swapped. Report the cross-validated recall and F1 against the reported 0.986/0.818. If the held-out recall drops by more than 0.05 or the F1 drops materially, the headline numbers are overfit to the calibration set. As a secondary check, rerun the evaluation with no centrality filter to quantify how much of the reported recall depends on the fitted cutoff.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is in Section 4.2.2: JC-Finder removes all classes in the bottom 50% of within-library PageRank percentiles, and this cutoff is justified by observing that 95.63% of 385 manually confirmed clone pairs fall in the top 50%. Those 385 pairs are drawn from the same ground-truth dataset (Section 5.1.2) that is later used to compute Table 1's precision, recall, and F1 for RQ1. The filter therefore is calibrated on the evaluation labels, so the reported recall of 0.986 and F1 of 0.818 are in part the result of a threshold chosen to fit that very dataset. Since the filter discards half of all class features, any real reuse that flows through peripheral or utility classes not well represented in the 385-pair sample would be systematically filtered out before matching, lowering recall in new projects. The paper gives no evidence that the 95.63% statistic generalizes beyond the 1,000-project ground truth, and it does not report confidence intervals or cross-validated performance. The timestamp rule in Section 4.2.3 is a separate, acknowledged limitation; the PageRank calibration is more load-bearing because it silently uses test labels to set a wholesale feature-pruning threshold.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents JC-Finder, a clone-based Software Composition Analysis tool for Java that operates at class granularity. Classes are represented as 'Class ASTs' built from normalized function ASTs, with intra-class call relations expanded and external calls replaced by placeholders. The reference index is constructed from 9,965 Maven libraries (543,286 versions) and refined by removing supporting classes, low-PageRank classes, and duplicate classes resolved by earliest release timestamp. TPL reuse is decided by matching test-project class features against the refined index. The evaluation on 1,000 GitHub projects reports precision 0.698, recall 0.986, F1 0.818, and an average resolution time of 14.2 seconds per project, compared with a Java-adapted CENTRIS baseline (CENTRIS4J). A further study of 7,947 projects reports that 789 projects reuse TPLs by cloning and that JC-Finder finds 26.20% more TPLs than package-manager declarations.","tokens_in":19688,"tokens_out":6575,"duration_ms":70814,"significance":"If the reported results were obtained under a sound evaluation protocol, the paper would make a useful contribution: it is the first Java-specific clone-based SCA tool at class granularity, it provides a manually labeled ground-truth set, and the tool and data are released. The efficiency gain (52 h vs 220 h for feature generation; 14.2 s vs 126.6 s per project) is substantial, and the 26.20% additional TPL finding gives concrete evidence that clone-based SCA complements package-manager detection. However, the headline accuracy is currently undermined because key thresholds are selected on the same ground truth later used for the final evaluation, so the reported precision, recall, and F1 are partly fitted to the test labels rather than being independent estimates.","major_comments":[{"comment":"The 50% PageRank cutoff is chosen after inspecting 385 clone pairs that were confirmed using the same ground-truth labeling procedure and the same 1,000-project dataset later used to compute Table 1. Because this cutoff discards all class features in the bottom half of the centrality distribution, the reported recall of 0.986 and F1 of 0.818 are partly fitted to the evaluation labels. The fact that 95.63% of the 385 pairs fall in the top 50% does not establish generalization beyond that sample. Please re-run the calibration on a held-out portion of the ground truth, or use nested cross-validation, and report metrics with and without the PageRank filter on the held-out set.","section":"§4.2.2, §5.1.2, §5.2"},{"comment":"The trivial-function complexity threshold of 60 is a free parameter set by inspecting 400 functions and by the authors' agreement, which the manuscript itself flags as expertise-dependent in Section 6.1. Since this threshold determines which functions survive into class features, it affects all downstream matching and thus the central claim. Please provide a sensitivity analysis over the complexity threshold and validate the chosen value on data not used to set it.","section":"§4.2.1, §6.1"},{"comment":"The timestamp rule for deduplicating cloned classes assumes the earliest released source JAR is the origin of a class, but 158,316 of the 543,286 collected versions lack source JARs, and Figure 6 shows that missing source JARs can flip the attribution and create both false positives and false negatives. The manuscript acknowledges this limitation but does not quantify its effect on the reported precision and recall. Please report how many duplicated features are affected by missing-source-JAR ambiguity and run a sensitivity analysis in which ambiguous features are excluded or handled conservatively.","section":"§4.2.3, Figure 6, §5.2"},{"comment":"The comparative evaluation is not conducted under an independent calibration protocol: JC-Finder's PageRank cutoff is tuned on the same 1,000-project ground truth, and CENTRIS4J's 15% threshold is also described as offering optimal F1 on that same ground truth. Even if the baseline tuning is favorable to CENTRIS4J, the magnitude of the reported advantage is uncertain. Please separate calibration from test, for example by splitting the labeled projects into calibration and test sets or by evaluating on an independent labeled sample, and report the comparison on the test portion only.","section":"§5.2, Table 1"},{"comment":"The ground truth is deliberately described as 'sound but not complete', and only 68 of the 1,000 projects are confirmed positive. Under an incomplete ground truth, unlabeled true reuses can be counted as false positives, which would understate precision, while recall is computed only against the known positives. Please report how ambiguous or unlabeled cases were handled in the metric computation, the number of such cases, and a sensitivity analysis on the completeness assumption.","section":"§5.1.2, §6.1"}],"minor_comments":[{"comment":"The reported differences over CENTRIS4J are inconsistent: Table 1 implies precision, recall, and F1 differences of 0.428, 0.280, and 0.427, whereas the Introduction reports 0.462 and 0.353 and the RQ1 answer reports 0.462, 0.353, and 0.475. Please reconcile these numbers.","section":"Section 1 and Section 5.2 (Answering RQ1)"},{"comment":"Section 5.1.1 reports 7,974 remaining projects, while RQ3 and the abstract use 7,947. Please correct the typo and ensure all project counts are consistent.","section":"§5.1.1, §5.4, Abstract"},{"comment":"The deduplication rule is described as first merging features within groups and then removing duplicates between groups, but the exact decision rule for cases where the earliest timestamps are equal or where the same feature appears in many groups is not stated. Please give a formal rule or a small worked example.","section":"§4.2.3"},{"comment":"In the conjugate clone percentage formula, the symbol C_{A,B} is used for both the set of clone pairs and its cardinality, and the set-builder notation in the displayed equation is missing the existential condition. Please disambiguate the notation.","section":"§3.3.1"},{"comment":"The axes of Figure 7 are not labeled with units, so it is unclear what 'project size' means and how scan time is measured. Please add axis labels and units.","section":"Figure 7"},{"comment":"The threshold grid for CENTRIS4J is described only as 'from 5% to 25%'; please state the step size and the exact F1 values at each threshold for transparency.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The central idea of class-level clone-based SCA for Java is plausible and the engineering contribution is real, including the released tool, data, and ground-truth labels. The main validity threat is the calibration of the PageRank cutoff and the complexity threshold on the same ground truth used for the headline metrics; this is fixable with a held-out split or external validation. I would not reject the paper, but I would require the authors to demonstrate that the reported accuracy holds on data not used to set the filters before accepting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"JC-Finder is a real contribution to Java SCA: class-level clone detection with linked function ASTs, plus a manually vetted ground truth. The headline recall of 0.986 and F1 of 0.818 should not be taken at face value, because the PageRank cutoff that discards half the classes was chosen by inspecting the same ground truth that later computes those numbers.\n\nWhat is genuinely new: it is the first clone-based SCA aimed specifically at Java, using class granularity instead of function-level (CENTRIS) or file-level (Snyk). The labeled dataset of 1,000 projects, built by two experts over two months, is a tangible asset. The finding that clone-based reuse adds about 26% to what PM-based SCA sees is useful even if the exact figure depends on a debatable convention. The efficiency gain (14.2s vs 126.6s per project) is plausible and the filtering rationale is sensible.\n\nThe soft spots are real but fixable. The stress-test is on target: Section 4.2.2 sets the bottom-50% PageRank filter after observing that 95.63% of 385 confirmed clone pairs sit in the top half, and those pairs are drawn from the ground truth later used for Table 1. That is calibration on the test labels. The trivial-function threshold (complexity<60) comes from a 400-function sample with no held-out check. With only 68 positive projects and 167 TPLs, the reported recall has a wide confidence interval. The Type-3 clone claim overreaches: the method is exact AST-skeleton hashing with name typing and order-insensitive merging, which covers Type-1/2 and some reordering, but not statement insertion/deletion. The IR=1 convention for projects with zero PM dependencies can inflate the 26.20% average. There are also internal arithmetic inconsistencies in the CENTRIS4J comparison: the intro reports +0.280/+0.428, Section 5.2 reports +0.462/+0.353, and neither matches the deltas in Table 1.\n\nNone of this sinks the paper. The core idea is sound, the dataset is reusable, and the central observation—that clone-based reuse is a substantial blind spot for Java SCA—stands. It deserves a serious referee. I would ask the authors to calibrate on a held-out set, report confidence intervals, correct the clone-type claim, and clean up the IR metric. Then this should be a solid venue paper.\n\nI'd bring it to our reading group, mostly to talk about the evaluation pitfalls.","headline":"A genuinely new Java SCA tool with a valuable dataset, but the headline F1 is partly calibrated on the evaluation ground truth, so treat the numbers with caution.","tokens_in":20281,"tokens_out":4827,"would_cite":true,"duration_ms":54001,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"JC-Finder claims that clone-based third-party library reuse in Java is best detected at class level, representing each class as an order-insensitive tree of interlinked function ASTs, and that this representation plus aggressive feature…","keywords":["code clone detection","software composition analysis","third-party library reuse","Java","class-level granularity","abstract syntax tree","PageRank centrality","Maven"],"falsifier":"Build an independent ground truth of clone-based TPL reuse for a fresh set of Java projects, using license headers, 'forked from' comments, and change logs, then check how many confirmed reused classes fall into the bottom 50% of their library's PageRank distribution. If that fraction is close to the paper's observed 4.37%, the reported recall of 0.986 should reproduce; if it is substantially larger, the centrality filter is discarding real evidence and the reported recall will not hold up.","tokens_in":19203,"feed_emoji":"☕","tokens_out":5318,"duration_ms":62008,"temperature":0.7,"pith_summary":"The paper tries to establish that third-party library (TPL) reuse introduced by copy-and-paste in Java is better detected at the granularity of classes than of files or individual functions. It argues that Java's object-oriented structure makes the class the natural unit of reuse: functions inside a class tend to be copied together, so preserving their caller-callee and co-existence relations matters. To use this insight, JC-Finder builds a 'Class AST' from linked function ASTs and filters away supporting, trivial, low-centrality, and duplicated classes before matching against a reference set of 9,965 Maven libraries. The authors report that this approach reaches recall 0.986 and precision 0.698, roughly nine times faster than a Java-adapted version of the function-level tool CENTRIS, and that it identifies 26.20% more TPLs than package-manager-based scanning alone. If correct, the result means Java can have clone-aware software composition analysis at scale, and package-manager-only SBOMs systematically undercount real reuse.","feed_headline":"Class-level clones expose 26% more Java library reuse","feed_subtitle":"A Java SCA tool builds linked class ASTs, filtering noise, to beat function-level scans on speed and accuracy.","key_machinery":"The Class AST with Linked Function ASTs is the central object: each function is parsed into an AST whose identifier names and values are replaced by type placeholders, then internal function invocations are expanded inline, external calls become Dummy External Nodes, recursion becomes a Dummy Recursive Node, and the resulting function trees are attached under a class root with an order-insensitive hash so rearranged code still matches. This carries the argument because it treats the class as the unit of reuse, preserves intra-class relationships, and tolerates Type 1, 2, and 3 clones. The supporting machinery is the three-stage feature refiner: supporting-class rules (C1 through C4), a PageRank centrality cutoff at the 50th percentile, and timestamp-based deduplication of classes shared across libraries.","core_discovery":"The central discovery is that class-level granularity with inter-function relationships solves the accuracy problem of clone-based SCA for Java while also saving computation. The paper shows empirically from 1,000 GitHub projects that functions in the same class are usually cloned together: 76% of caller functions have all their callees cloned alongside them, and 48% of classes are fully cloned in conjugate pairs. This motivates representing a class as a Class AST whose function subtrees are linked along internal invocation edges, with external calls replaced by dummy nodes and recursion broken deterministically. Feature refinement then removes interface and empty classes, trivial functions below a complexity score of 60, structural design-pattern classes, test classes, classes in the bottom 50% of the library's PageRank centrality distribution, and duplicate classes resolved by earliest Maven release timestamp. Matching a project class against this refined library feature set yields the reported 0.986 recall, 0.698 precision, and 0.818 F1-score on a manually labelled ground truth of 1,000 projects.","pith_inferences":["A direct testable extension is to vary the PageRank cutoff and re-measure precision and recall; if precision is insensitive across a range of cutoffs, the 50% threshold is robust, otherwise it is overfit to the ground truth.","The timestamp-based origin rule implies that reuse whose original release predates Maven source JAR availability will be systematically misattributed; adding provenance signals such as commit history or package naming could reduce that error.","If JC-Finder were integrated into vulnerability scanning, the 26.20% extra TPLs implies current SBOM-driven vulnerability databases may miss a comparable fraction of clone-based exposures.","The class-level linked-AST representation could generalize to other object-oriented languages, but the paper only evaluates Java, so any cross-language claim remains an extension rather than a result."],"forward_implications":["If JC-Finder's reported accuracy holds, Java SCA tools can safely include clone-introduced TPLs in their SBOMs, catching reuse that package-manager scans miss.","The 26.20% improvement rate suggests that roughly one in four clone-detected TPLs in the studied 7,947 projects would be invisible to PM-based scans.","Because the method replaces threshold-based matching with exact class-feature matching, it removes the need to tune a similarity threshold per project or per library.","Class-level features generated once for 543,286 library versions can be reused for scanning arbitrary future projects, making the approach practical for continuous integration."],"supporting_citations":[{"why":"Supplies the state-of-the-art function-level clone-based SCA tool CENTRIS that JC-Finder adapts to Java as CENTRIS4J and compares against.","marker":"[79]"},{"why":"Provides the AST-based tree similarity approach for detecting similar Java classes that inspires JC-Finder's feature extraction.","marker":"[70]"},{"why":"The clone detector TACC used in the preliminary study to establish associated and conjugate clone percentages.","marker":"[77]"},{"why":"PageRank is the centrality measure used to filter out classes judged non-representative before matching.","marker":"[63]"},{"why":"Supplies the order-insensitive hash calculation that makes Class AST hashes invariant to code reordering.","marker":"[59]"},{"why":"JavaParser is the parser used to partition files into classes and generate function ASTs.","marker":"[17]"},{"why":"Supplies evidence that code clones are common in Java projects, motivating the need for a clone-based SCA tool.","marker":"[1]"}],"fun_headline_variants":["Class-level trees expose 26% more Java library reuse","JC-Finder: 9x faster Java clone library detection","Class-level ASTs beat function-level in Java SCA","Undeclared Java libs spotted via class tree analysis","Java clone SCA: class-level trees lift F1 to 0.818"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The most fragile premise is that classes representing true library reuse almost always fall in the top half of their library's PageRank centrality distribution, so discarding the bottom half loses at most a few percent of recall; this cutoff was chosen from 385 manually confirmed clone pairs in the same ground truth later used to measure accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Class-level trees expose 26% more Java library reuse","JC-Finder: 9x faster Java clone library detection","Class-level ASTs beat function-level in Java SCA","Undeclared Java libs spotted via class tree analysis","Java clone SCA: class-level trees lift F1 to 0.818"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000379,"raw_usage":{"total_tokens":2095,"prompt_tokens":1108,"completion_tokens":987,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":724,"completion_tokens_details":{"reasoning_tokens":899}},"tokens_in":724,"tokens_out":987,"duration_ms":11399,"temperature":1.0,"reasoning_tokens":899,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T04:57:58.162990+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build an independent ground truth of clone-based TPL reuse for a fresh set of Java projects, using license headers, 'forked from' comments, and change logs, then check how many confirmed reused classes fall into the bottom 50% of their library's PageRank distribution. If that fraction is close to the paper's observed 4.37%, the reported recall of 0.986 should reproduce; if it is substantially larger, the centrality filter is discarding real evidence and the reported recall will not hold up.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the state-of-the-art function-level clone-based SCA tool CENTRIS that JC-Finder adapts to Java as CENTRIS4J and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the AST-based tree similarity approach for detecting similar Java classes that inspires JC-Finder's feature extraction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The clone detector TACC used in the preliminary study to establish associated and conjugate clone percentages."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PageRank is the centrality measure used to filter out classes judged non-representative before matching."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the order-insensitive hash calculation that makes Class AST hashes invariant to code reordering."},{"cited_title":"JavaParser","cited_arxiv_id":null,"evidence_quote":"JavaParser is the parser used to partition files into classes and generate function ASTs."}],"review_version":1}