{"id":"789b80bf-d3d2-46e6-acb7-a650ae6ff5e0","arxiv_id":"2504.12261","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A 635,003-artifact Maven study reports that zero-dependency artifacts have comparable popularity and fewer CVEs, but the CVE comparison ignores 12x group-size differences and the zero-propagation result is true by definition.","lead":"This study compared Maven Central artifacts with no dependencies against artifacts with dependencies, across popularity, security, and maintenance metrics. The paper claims independence brings security benefits, but the headline vulnerability comparison uses unnormalized raw counts and a definitional zero-propagation result.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Raw CVE counts are compared across groups differing 12x in size; per-artifact vulnerability rates favor dependent artifacts, so the 'fewer vulnerabilities' claim and the safety conclusion are unsupported.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: raw CVE and CWE counts are compared across groups with a roughly 12x size difference. This is not a stylistic issue; the central advertised advantage of independent artifacts—'fewer vulnerabilities'—depends on this comparison. The per-artifact rates in Table I and Table II show that the conclusion likely inverts after normalization. Because the safety recommendation is the paper's main contribution, and because the supporting comparison is invalid as presented, the REJECT verdict is appropriate. I see no basis to change the reader's verdict, though the underlying question could be revisited with normalized rates, clear definitions of 'propagated vulnerability,' and statistical tests.","tokens_in":6540,"tokens_out":3792,"duration_ms":37534,"concrete_test":"Recompute Table II's vulnerability metrics as rates with artifact counts as exposure: for NumCVE(Last Release), test the rate ratio (60/22081)/(179/267640) with a Poisson confidence interval; for NumArtifactsWithCVE, test (69/22081)/(793/267640). If either rate ratio is not significantly below 1, or if the independent-artifact rate is significantly above 1, the 'fewer vulnerabilities' claim in the abstract is false and the safety recommendation should be withdrawn. Optionally repeat with per-release or per-artifact-year exposure to test robustness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central safety claim rests on 'fewer vulnerabilities (60 CVEs vs. 179 CVEs)' in the abstract and on the NumCVE(Last Release) and NumArtifactsWithCVE rows of Table II. These are raw counts, but the comparison groups are not equal: Table I reports 22,081 active independent artifacts versus 267,640 active dependent artifacts. Normalizing by group size gives independent artifacts a CVE rate of 60/22081 ≈ 2.72 per 1000 artifacts, versus 179/267640 ≈ 0.67 per 1000 for dependent artifacts—about four times higher. The artifacts-with-CVE row shows a similar pattern: 69/22081 ≈ 3.1 per 1000 for independent versus 793/267640 ≈ 3.0 per 1000 for dependent artifacts. Thus the 'fewer vulnerabilities' advantage disappears and may invert once group size is accounted for. Since the recommendation that independent artifacts are a safe, self-contained alternative depends directly on this comparison, the headline conclusion is not supported as presented. The 'zero propagated vulnerabilities' result is also definitional under the paper's independence definition rather than an empirical finding, and it does not offset the unnormalized vulnerability counts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes 658,078 artifacts from the Maven Central ecosystem, classifying artifacts with no dependencies as 'independent' (in-degree = 0) and the rest as 'dependent.' It reports prevalence, ecosystem influence via PageRank and out-degree centrality, and compares 18 usability and maintenance metrics between the two groups. The abstract claims independent artifacts have comparable popularity, fewer vulnerabilities (60 vs. 179 CVEs), and zero propagated vulnerabilities, concluding they provide a 'safe, self-contained alternative' to traditional dependencies. Data are drawn from the Goblin dataset, and a replication package is provided.","tokens_in":6703,"tokens_out":9337,"duration_ms":90259,"significance":"If the findings were sound, the paper would address a genuinely under-studied population of Maven artifacts and could inform dependency-selection and maintenance decisions. The replication package and the use of a large, recent dataset are strengths. However, the central quantitative claims are undermined by a failure to normalize vulnerability counts across groups of very different sizes, by a definitional tautology in the 'zero propagated vulnerabilities' claim, and by a misreading of the centrality data. As presented, the paper's headline safety conclusion is not supported, so its significance for practitioners and researchers is currently not established.","major_comments":[{"comment":"The claim 'fewer vulnerabilities (60 CVEs vs. 179 CVEs)' compares raw counts without accounting for group sizes: Table I reports 22,081 active independent artifacts versus 267,640 active dependent artifacts, a 12x difference. Per-artifact CVE rates are 60/22,081 = 2.72 per 1,000 for independent artifacts versus 179/267,640 = 0.67 per 1,000 for dependent artifacts, which reverses the claimed advantage. The NumArtifactsWithCVE row shows near-identical rates (69/22,081 = 3.1 per 1,000 versus 793/267,640 = 3.0 per 1,000). Because the paper's central recommendation depends on this comparison, the headline conclusion is unsupported as presented.","section":"Abstract; Section III-C; Table II"},{"comment":"The text states that the median out-degree centrality for dependent artifacts (5.06E-6) is 'notably larger' than for independent artifacts (6.3E-7). However, Table II reports the independent-artifact median as 5.04E-6; the value 6.3E-7 is the minimum, not the median. The medians are nearly identical (5.06E-6 versus 5.04E-6), so the 'notably larger' claim is not supported by the data presented. The same paragraph also asserts statistical significance without reporting any test.","section":"Section III-B; Table II"},{"comment":"The 'zero propagated vulnerabilities' claim for independent artifacts is a direct consequence of the paper's definition: independent artifacts are defined as having no dependencies, and propagated vulnerabilities are defined as inherited from dependencies. This is a definitional equivalence, not an empirical discovery. Presenting it as a research finding overstates the contribution and should be reframed as an inherent property of the classification.","section":"Abstract; Section III-C; Section IV"},{"comment":"The abstract describes 'comparable popularity (25.58 vs. 7.30)' for independent versus dependent artifacts. These numbers are the means from Table II and show independent artifacts have more than three times the popularity of dependent artifacts, so describing them as 'comparable' is misleading. Either the interpretation is wrong or the wrong metric is being reported; in either case, the statement does not match the data.","section":"Abstract; Section III-C"},{"comment":"The paper repeatedly claims that differences are 'statistically significant' (e.g., in the PageRank and centrality discussion) but reports no significance tests, effect sizes, or confidence intervals for any of the 18 metrics. With sample sizes in the tens of thousands, trivial differences can achieve statistical significance, so the reader cannot interpret the practical importance of the reported differences without additional statistics.","section":"Section III-A; Section III-B; Section III-C"}],"minor_comments":[{"comment":"The phrase 'number of incoming dependencies' is ambiguous: in standard graph terminology, an incoming edge to an artifact represents a dependent, whereas the paper uses it to mean the artifact's own dependencies on upstream artifacts. Please clarify the definition or use a less confusing term.","section":"Section II"},{"comment":"The row label 'NumAtrtifactsWithCVE' contains a typo; it should be 'NumArtifactsWithCVE'.","section":"Table II"},{"comment":"It is unclear whether NumCVE(Last Release) and NumArtifactsWithCVE are totals, means, or per-release counts. The table header says 'Mean/Median/Min/Max' for other rows, but these two rows only show a single number, so their meaning should be stated explicitly.","section":"Table II; Section III-C"},{"comment":"The sentence about release frequency states that the median release frequency for independent artifacts implies an average of 166 days between releases, but the units of ReleaseFrequency are not defined in the table. Please specify whether the metric is releases per day, per week, or another unit.","section":"Section III-C"},{"comment":"The accessed date reads '2023, Jsnuary 28'; this appears to be a typo for 'January'.","section":"Reference [16]"}],"recommendation":"reject","confidential_remarks":"The paper investigates a potentially interesting and under-studied population of Maven artifacts and provides a replication package, which is commendable. However, the main vulnerability comparison is invalid because raw counts are compared across groups that differ by an order of magnitude in size, and the per-artifact rates actually favor dependent artifacts. The 'zero propagated vulnerabilities' claim is tautological under the paper's own definitions. These issues undermine the paper's central thesis, and the errors are not confined to presentation; a revision would need to redo the analysis and likely change the headline conclusions substantially. I therefore recommend rejection, though I would encourage the authors to resubmit a corrected study after re-analyzing the data with appropriate normalization and statistical testing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: the paper's headline vulnerability claim is based on raw counts across groups that differ by 12x, and the per-artifact rates actually favor dependent artifacts. On top of that, the text and Table II disagree on the degree-centrality numbers. The research question is real, but the evidence as presented doesn't support the conclusion.\n\nWhat is new: this is the first empirical characterization I know of zero-dependency Maven artifacts as a class. The authors use the Goblin dataset, provide a replication package, and their maintainability findings—lower release frequency, fewer contributors, more missing licenses, less README coverage—are consistent and plausible. Those parts are worth keeping.\n\nSoft spots, in order of severity. The 'fewer vulnerabilities (60 vs 179)' in the abstract compares absolute counts for 22,081 independent artifacts and 267,640 dependent artifacts. Normalized, independent artifacts have roughly 2.7 CVEs per thousand versus 0.67 per thousand for dependent artifacts; the artifacts-with-CVE row is about 3.1 vs 3.0 per thousand. So the 'safe, self-contained alternative' claim flips. Second, the paper is internally inconsistent: Section III says dependent artifacts have 793 CVEs versus 69, while the abstract and Table II use 179 versus 60. It also says the median out-degree centrality for independent artifacts is 6.3E-7 and 'notably larger' for dependent artifacts, but Table II lists the independent median at 5.04E-6—essentially equal. Those aren't cosmetic. Third, 'zero propagated vulnerabilities' is true by definition under their independence definition, not an empirical result. It also doesn't offset the higher own-vulnerability rate. Fourth, 'statistically significant' appears without any test being reported.\n\nI agree with the reader's reject verdict. The paper needs normalized metrics, corrected tables, and real significance tests. That said, I would still send it to peer review rather than desk-reject: the question is legitimate, the dataset work is reproducible, and the maintainability observations are salvageable. With a major revision this could become a solid empirical study.","headline":"The paper's central safety claim rests on unnormalized CVE counts that invert per artifact, and the text contradicts its own table; the underlying zero-dependency Maven question is still worth a proper study.","tokens_in":7236,"tokens_out":4954,"would_cite":false,"duration_ms":47671,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Independent Maven artifacts—those no other artifact depends on—match dependent artifacts in popularity while showing far fewer total CVEs, and the paper argues they deserve a place in dependency choices.","keywords":["Maven Central","dependency graph","independent artifacts","software vulnerabilities","CVE","PageRank","software ecosystem","transitive dependencies"],"falsifier":"Take the paper's own Table II counts and divide by group size: $60/22081$ for independent artifacts versus $179/267640$ for dependent artifacts. Because the per-artifact rate is higher for the independent group, the 'fewer vulnerabilities' claim would be falsified under rate normalization; it would survive only if a severity-weighted or release-adjusted comparison still favored independent artifacts.","tokens_in":6285,"feed_emoji":"📦","tokens_out":9112,"duration_ms":86152,"temperature":0.7,"pith_summary":"The paper studies the 22,081 active Maven Central artifacts that no other artifact depends on—a population previous work mostly ignored—and compares them with 267,640 dependent artifacts across 18 metrics. Its central claim is that these independent artifacts are not minor: they attract comparable mean download popularity (25.58 vs. 7.30 over the past year), account for fewer total CVE records (60 vs. 179), and have zero propagated vulnerabilities, because nothing downstream can inherit a vulnerability through them. The authors conclude that independent artifacts give developers a safe, self-contained dependency alternative, provided maintainers address their slower release cadence and weaker documentation and licensing. The significance is practical: if the claim holds, dependency-minimizing projects can choose such artifacts without sacrificing popularity or inheriting transitive risk.","feed_headline":"Independent Maven artifacts rival dependent ones in popularity","feed_subtitle":"Maven artifacts nobody else depends on match popularity, with fewer CVEs but slower fixes.","key_machinery":"The central object is the Maven Central dependency graph, where nodes are artifacts and edges record dependency relationships, and the partition of that graph by in-degree: artifacts with zero incoming edges are called independent, all others dependent. Two measures carry the influence argument: PageRank, a link-analysis score of how connected a node is in the dependency network, and out-degree centrality, the fraction of other artifacts a node points to. The security argument runs through CVE and CWE records attached to releases, with the propagation mechanism that an artifact with no incoming edges cannot pass a vulnerability downstream, while a dependent artifact can propagate transitively. The same graph also yields the maintenance metrics, including release counts, release frequency, dependents, and repository health, that form the comparison.","core_discovery":"Defining independent artifacts as Maven Central artifacts with zero incoming dependencies, the paper reports that they are a meaningful slice of the ecosystem (15.4% of studied artifacts, 22,081 active) and remain influential: their median PageRank (0.40) is close to that of dependent artifacts (0.44), even though their out-degree centrality is lower. On the headline metrics, the paper finds comparable popularity (25.58 vs. 7.30), fewer CVEs (60 vs. 179), and zero propagated vulnerabilities, while also finding less frequent releases (median 0.006 vs. 0.02) and weaker maintenance markers such as fewer contributors and more missing licenses. The authors conclude that independent artifacts can serve as safe, self-contained alternatives to traditional dependencies and recommend that maintainers prioritize them for updates, issue responses, and documentation.","pith_inferences":["In my reading of the paper's own Table II, rate-normalizing changes the security picture: the independent group has 60 CVEs across 22,081 artifacts (about 2.7 per 1,000) while the dependent group has 179 across 267,640 (about 0.7 per 1,000), so 'fewer vulnerabilities' is sensitive to whether one compares absolute or per-artifact counts.","The definition used in the paper—no incoming dependencies—does not by itself rule out outgoing dependencies, so the 'self-contained' reading should be tested against the dependency graph before treating these artifacts as dependency-free.","A comparable phenomenon may exist in other package ecosystems: if dependency-free components elsewhere also show high download counts but slow maintenance, tooling that nudges maintainers toward regular releases would address a cross-ecosystem gap."],"forward_implications":["Projects that want to minimize transitive risk can choose independent artifacts without conceding user adoption, since their download popularity is comparable to dependent artifacts.","Dependency chains that include an independent artifact terminate there for downstream vulnerability propagation, so a vulnerable upstream library cannot reach downstream consumers through that artifact.","Consumers of independent artifacts inherit a slower fix cadence: with a lower median release frequency, security patches take longer to arrive, so choosing them trades transitive safety for response time.","Maintainers of independent artifacts hold the key to realizing the safety benefit: regular updates, license completion, and better documentation would remove the main disadvantages the study identifies."],"supporting_citations":[{"why":"Supplies the Maven Central dependency graph, release metadata, and vulnerability links used for every count in the study.","marker":"[13]"},{"why":"Provides the PageRank algorithm used to compare ecosystem influence of the two artifact groups.","marker":"[10]"},{"why":"Defines the out-degree centrality measure used to compare how connected artifacts are in the dependency network.","marker":"[11]"},{"why":"Establishes that transitive and granularity effects can spread vulnerabilities through Maven, motivating the focus on artifacts that cannot propagate.","marker":"[7]"},{"why":"Shows that apparently trivial npm packages are actually important, supplying the analogy that independent Maven artifacts may be similarly underestimated.","marker":"[8]"},{"why":"Shows upstream libraries that delay fixes keep vulnerabilities open in dependent applications, providing the transitive-risk rationale.","marker":"[5]"}],"fun_headline_variants":["Zero inbound deps: Maven artifacts stay popular, avoid CVEs","Independent Maven artifacts: fewer CVEs, same popularity","Maven's independent artifacts: safe, used, but under-maintained","15% of Maven artifacts have no dependents yet remain influential"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The security comparison treats raw CVE totals as directly comparable even though the independent group has roughly one-twelfth as many active artifacts as the dependent group, so a per-artifact rate would not support the 'fewer vulnerabilities' claim.","fun_headline_variants_meta":{"raw":{"variants":["Zero inbound deps: Maven artifacts stay popular, avoid CVEs","Independent Maven artifacts: fewer CVEs, same popularity","Maven's independent artifacts: safe, used, but under-maintained","15% of Maven artifacts have no dependents yet remain influential"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000359,"raw_usage":{"total_tokens":1958,"prompt_tokens":976,"completion_tokens":982,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":907}},"tokens_in":592,"tokens_out":982,"duration_ms":9385,"temperature":1.0,"reasoning_tokens":907,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:33:30.591657+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the paper's own Table II counts and divide by group size: $60/22081$ for independent artifacts versus $179/267640$ for dependent artifacts. Because the per-artifact rate is higher for the independent group, the 'fewer vulnerabilities' claim would be falsified under rate normalization; it would survive only if a severity-weighted or release-adjusted comparison still favored independent artifacts.","supporting_citations":[{"cited_title":"Navigating and exploring software dependency graphs using goblin,","cited_arxiv_id":null,"evidence_quote":"Supplies the Maven Central dependency graph, release metadata, and vulnerability links used for every count in the study."},{"cited_title":"Pagerank","cited_arxiv_id":null,"evidence_quote":"Provides the PageRank algorithm used to compare ecosystem influence of the two artifact groups."},{"cited_title":"On the effect of transitivity and granularity on vulnerability propagation in the maven ecosystem,","cited_arxiv_id":null,"evidence_quote":"Establishes that transitive and granularity effects can spread vulnerabilities through Maven, motivating the focus on artifacts that cannot propagate."},{"cited_title":"On the untriviality of trivial packages: An empirical study of npm javascript packages,","cited_arxiv_id":null,"evidence_quote":"Shows that apparently trivial npm packages are actually important, supplying the analogy that independent Maven artifacts may be similarly underestimated."},{"cited_title":"Mitigating persistence of open-source vulnerabilities in maven ecosystem,","cited_arxiv_id":null,"evidence_quote":"Shows upstream libraries that delay fixes keep vulnerabilities open in dependent applications, providing the transitive-risk rationale."}],"review_version":1}