{"id":"ef1997a8-7113-43cf-a10f-76a5212a5fd7","arxiv_id":"2502.04621","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Maven maintainers patch critical vulnerabilities about 48% faster than low-severity ones after public disclosure, and dependent packages take a median of 151 days to adopt available fixes.","lead":"This study tracks 3,362 known security flaws in the Maven Java package repository to see how quickly they get patched and how quickly downstream packages adopt the fix. It finds that severe flaws tend to be patched faster than minor ones after disclosure, but many dependent packages still wait months to update.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's headline 48.3% response-time reduction is undermined by an internal inconsistency in how response time is defined: Section II-B says it is the time from CVE Introduction to Patch, but Table II values are negative for Patch-Before-Publish, which is only possible if the reference date is…","rationale":"The reader's weakest assumption concerned completeness and alignment of OSV.dev and Goblin data, and the reader's rationale also noted the absence of significance tests. My concern is more specific and more directly tied to the headline: the paper contains an internal contradiction between the stated lifecycle metric (introduction-to-patch) and the actual values in Table II, which can only be publication-to-patch. This is the single most load-bearing issue because if the metric is not what the abstract claims, the 48.3% reduction is not even well-defined, let alone trustworthy. The negative Patch-Before-Publish values are a smoking gun: no convention for 'introduction date' makes a time-to-mitigate negative if it is measured from the first vulnerable release to the patch. Therefore, either the methodology text is wrong and the analysis actually uses publication dates (which would make the headline substantively correct but require correcting Section II-B and adding statistical support), or the analysis uses the stated definition and Table II is internally impossible (indicating a serious bug). The proposed test settles this dichotomy by checking the replication package code and recomputing both possible metrics. I keep the verdict at CONDITIONAL (UNCHANGED) because the paper may be salvageable with a correction and additional statistical evidence, but the current text does not support the headline claim as written. I partially agree with the reader because we both flag the response-time calculation as fragile, but I identify a different mechanism than data completeness: an internal measurement inconsistency independent of OSV/Goblin alignment.","tokens_in":7795,"tokens_out":6775,"duration_ms":65878,"concrete_test":"Inspect the replication package's analysis code to identify the exact date subtraction that produced Table II. Then independently recompute the response time for all Publish-Before-Patch CVEs using (a) CVE Publication Date minus CVE Patch Date and (b) CVE Introduction Date minus CVE Patch Date. If the reported values match (a), Section II-B's definition is wrong and the headline should be reworded and supplemented with per-cell sample sizes and 95% confidence intervals; if they match (b), the negative Patch-Before-Publish values are impossible and the analysis has a date-handling bug. Report the number of CVEs per severity cell to assess whether the 48.3% reduction is statistically credible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim ('reduce response time by 48.3% from low (151 days) to critical severity (78 days)') rests on the Publish-Before-Patch row of Table II. Section II-B explicitly states: 'The lifecycle is calculated as the time delta, in days, between the CVE Introduction Date and the CVE Patch Date.' Yet Table II reports negative mean times for Patch-Before-Publish (-867, -1128, -897, -790). These negative values are only possible if the reference date is the CVE Publication Date, because the CVE Introduction Date necessarily precedes the patch date, making patch minus introduction always positive. The methodology text and the reported numbers therefore contradict each other. The Abstract presents these values as 'response time after public disclosure,' but the paper never unambiguously defines this metric; it defines a different quantity (full lifecycle from introduction to patch). If the code actually computes publication-to-patch, then Section II-B is wrong and the authors must correct it, report per-cell sample sizes, and provide confidence intervals or significance tests. If the code computes introduction-to-patch, then the negative Patch-Before-Publish values are impossible, indicating a bug, and the 'after public disclosure' claim is unsupported. Either way, the headline statistic currently lacks a verified, clearly defined measurement basis. A related concern is that the underlying cells are small: Low severity accounts for 3.12% of all CVEs and Publish-Before-Patch for 5.6%, so the 48.3% reduction may reflect a few outliers rather than a robust trend, yet no statistical testing is reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies 3,362 CVEs affecting Maven artifacts by combining OSV.dev vulnerability data with the Goblin dependency graph and OpenDigger GitHub metrics. It formulates three research questions: (RQ1) the lifecycle of vulnerabilities, (RQ2) correlations between repository characteristics and vulnerability presence, and (RQ3) how long directly dependent packages take to adopt patches. The headline finding is that in Publish-Before-Patch cases, maintainers patch critical vulnerabilities in about 78 days compared with 151 days for low-severity vulnerabilities, a 48.3% reduction. The paper also reports that 81.6% of vulnerabilities are patched before public disclosure, that issue-activity and contributor metrics correlate with vulnerability presence, and that Available Patch Adoption is the most common dependent behavior with a median response time of 151 days.","tokens_in":8148,"tokens_out":3157,"duration_ms":32162,"significance":"If the results hold, the paper provides one of the larger automated studies of Maven vulnerability lifecycles, scaling from the 312 manually analyzed CVEs of prior work to 3,362 CVEs. The use of established data sources (OSV.dev, Goblin, OpenDigger), the availability of a replication package, and the explicit acknowledgment of ethical considerations are strengths. The RQ3 dependency-adoption taxonomy (Reactive, Available, Proactive) is a useful framing for patch-propagation behavior. However, the central quantitative claim about severity-dependent response times rests on a metric definition that is internally inconsistent with the reported numbers, and several supporting analyses lack the statistical detail needed to assess their reliability.","major_comments":[{"comment":"The methodology text and the reported numbers contradict each other. Section II-B states that 'the lifecycle is calculated as the time delta, in days, between the CVE Introduction Date and the CVE Patch Date,' which is necessarily non-negative because a patch cannot be released before the vulnerability is introduced. Table II nevertheless reports negative mean times for Patch-Before-Publish (-867, -1128, -897, -790). These values are only possible if the reference point is the CVE Publication Date rather than the CVE Introduction Date. The abstract's phrasing 'after public disclosure' suggests the authors intended publication-to-patch intervals, but the paper never unambiguously defines this metric. The authors must correct this inconsistency: if the intended metric is publication-to-patch, rewrite Section II-B accordingly; if the computed metric is introduction-to-patch, the negative values indicate a computation error. Either way, the headline 48.3% reduction currently lacks a verified and clearly defined measurement basis.","section":"Section II-B and Table II"},{"comment":"The severity trend in Table II is presented as a key result, but no per-cell sample sizes, confidence intervals, or significance tests are reported. The underlying cells are small: low-severity CVEs are 3.12% of the data and Publish-Before-Patch cases are 12.8% overall, so the 151-day versus 78-day comparison may rest on a handful of observations. A point estimate without uncertainty is not sufficient to support the 48.3% reduction claim. Please report the number of CVEs in each Patch-Type by Severity cell, along with confidence intervals or a significance test (e.g., a non-parametric comparison of response-time distributions across severity levels).","section":"Section III-A, Table II"},{"comment":"The correlation analysis acknowledges a 'severe class imbalance between historical repositories with CVEs (456) and those without (10,675)' but does not correct for it. With such imbalance, rank-biserial correlations and Mann-Whitney p-values can be sensitive to base rates and the very large control group can make small differences appear statistically significant. The paper reports effect sizes but no p-values or confidence intervals in Table III, despite claiming 'p < 0.05' for all metrics. Please report the actual p-values or confidence intervals, clarify the unit of analysis (456 CVEs versus 175 repositories with CVEs versus 10,675 non-vulnerable repositories), and consider a sensitivity analysis such as downsampling the majority class or using a prevalence-adjusted effect size.","section":"Section II-C and Table III"},{"comment":"The threats-to-validity section states that the absence of CVE discovery dates means the calculated time to mitigation 'may not reflect the actual duration maintainers were aware of the vulnerability' and 'providing a lower bound on mitigation delays.' This is directly relevant to the headline interpretation. If the corrected metric is publication-to-patch, the paper should explicitly separate the disclosure-to-patch interval from the introduction-to-patch lifecycle and explain which quantity Table II reports. The current presentation conflates these, and the 'after public disclosure' language in the abstract is not supported by the methodology as written.","section":"Section IV"}],"minor_comments":[{"comment":"There is a typo in the first sentence: 'Patch-Before-Patch' should be 'Patch-Before-Publish'.","section":"Section III-A"},{"comment":"Table II is labeled 'Mean time to mitigate,' while RQ3 results are reported as medians. Please make the choice of central tendency explicit and consistent, or explain why means are used for RQ1 and medians for RQ3.","section":"Table II"},{"comment":"The discussion says 'all metrics (p < 0.05)' but the table does not show the p-values. Please add them or state the exact p-value range, since readers cannot verify the significance claim from Table III.","section":"Section III-B"},{"comment":"The boxplot caption does not indicate what the axes represent or whether the plotted values are raw days or log-transformed. Please clarify, especially since response-time distributions are typically heavy-tailed.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of an empirical software engineering venue and the replication package is a positive feature. The main issue is the unresolved contradiction between the stated response-time definition and the negative values in Table II; this is fixable but it is load-bearing because it concerns the headline 48.3% claim. The authors should also be asked to add uncertainty quantification to Table II and to clarify the unit of analysis in the RQ2 correlation study. I do not see grounds for rejection, but the revision must address the definitional inconsistency and the missing statistical detail before the results can be considered reliable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this is a genuinely useful empirical study, but the headline \"48.3% reduction\" is not yet trustworthy. The paper defines the CVE lifecycle as the time from CVE Introduction to Patch (Section II-B), yet Table II reports negative mean times for Patch-Before-Publish. Since introduction necessarily precedes the patch, negative values are impossible under that definition. The abstract's \"after public disclosure\" phrasing suggests they actually computed publication-to-patch, but the paper never says so. That is a load-bearing inconsistency.\n\nWhat is actually new: 3,362 CVEs in Maven with lifecycle classification, scaling prior work that manually analyzed 312 CVEs and adding a new angle on how direct dependents adopt patches. The RQ3 adoption categories (Reactive/Available/Proactive) are a sensible operationalization. The data comes from external sources (OSV.dev, Goblin, OpenDigger), there are no fitted parameters, and the replication package is promised. The correlation analysis is straightforward and honestly labeled as observational.\n\nSoft spots: the severity trend is a point estimate with no confidence intervals or significance tests, and cell sizes are small (Low severity is 3.12% of CVEs; Publish-Before-Patch is 5.6%). The RQ2 correlations are run on a severely imbalanced sample (456 vs 10,675), acknowledged but not corrected. The replication package has no commit hash, which matters for a dataset paper. The threats to validity section is actually good—it flags missing lower bounds and discovery dates—but those same gaps undermine the 81.6% Patch-Before-Publish share.\n\nWho this is for: people building SCA tools or studying ecosystem security; it is a useful reference dataset. It deserves a serious referee, but only after the metric definition is fixed and the headline claim gets uncertainty quantification. I would send it to review with a request for major revision.","headline":"Useful large-scale Maven vulnerability lifecycle data, but the headline response-time claim rests on a metric that is defined inconsistently and reported without uncertainty estimates.","tokens_in":8634,"tokens_out":1729,"would_cite":false,"duration_ms":16890,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Critical Maven CVEs are patched in 78 days vs 151 for low severity","keywords":["Maven","CVE lifecycle","vulnerability patching","dependency networks","patch adoption","software supply chain","repository metrics","empirical software engineering"],"falsifier":"Recompute the headline numbers using only CVEs whose records give an explicit lower bound for affected versions, and compare patch release dates against the package registry's actual release history for a sample of critical CVEs; if the 151-to-78-day severity gradient or the 81.6% patch-before-publish share collapses, the result is an artifact of incomplete metadata.","tokens_in":7603,"feed_emoji":"🐛","tokens_out":8257,"duration_ms":73877,"temperature":0.7,"pith_summary":"This paper traces the full lifecycle of 3,362 Maven vulnerabilities, from introduction through disclosure, patching, and adoption by dependent packages. It aims to show that maintainers respond to disclosed vulnerabilities in proportion to severity: critical flaws are patched in about 78 days on average, versus 151 days for low-severity flaws, a 48.3% reduction. It also claims that 81.6% of vulnerabilities are patched before public disclosure, while only 5.6% remain unresolved, and that project characteristics such as issue activity and concentration of contributions among few people correlate with a package having a CVE. Establishing these patterns matters because Maven underpins a large share of Java software, so the speed and behavior of patching in this ecosystem directly affect downstream supply-chain risk.","feed_headline":"Critical Maven CVEs are patched in 78 days vs 151 for low severity","feed_subtitle":"Analysis of 3,362 CVEs finds severity drives patch speed, but most dependent packages still wait months to adopt fixes.","key_machinery":"The analytical engine is a CVE lifecycle classification built from three dated events: the public disclosure date, the release date of the first patched version, and the first release date of a dependent package that adopts the patch. Comparing the order of these dates partitions vulnerabilities into No-Patch, Patch-Before-Publish, and Publish-Before-Patch categories, and partitions dependent-package behavior into Reactive, Available, and Proactive Adoption. Response time is then measured as the difference between the patch release date and the last vulnerable release date. For the correlation analysis, the mechanism is the rank-biserial correlation coefficient, a non-parametric measure of association between a binary variable and a continuous metric, paired with a non-parametric significance test applied to seventeen repository activity metrics. This machinery is what turns raw vulnerability records and release timestamps into the paper's percentages and day counts.","core_discovery":"The central discovery is a severity-driven patching pattern in Maven. In the 12.8% of cases where public disclosure happens before a patch exists, average time-to-patch falls monotonically with severity: 151 days for low, 123 for moderate, 84 for high, and 78 for critical. The other headline result is that the large majority of vulnerabilities, 81.6%, are patched before public disclosure, which the paper interprets as proactive vendor behavior, and only 5.6% have no patch at all. For downstream packages, the most common behavior is adopting an already available patch, accounting for 62.9% of dependency cases, yet adopters still take a median 151 days to do so, while the rarer cases where adoption waits for a post-disclosure patch take 249 days. The paper also reports that repository metrics like issue comment volume, issue closure counts, and contributor concentration correlate with vulnerability presence, though it treats these associations as observational rather than causal.","pith_inferences":["This synthesis suggests a testable extension: recompute the severity-response gradient after controlling for project size; the paper itself notes issue activity may be a byproduct of scale, and the 48.3% reduction could partly reflect that larger projects have more resources to patch fast.","Because vulnerability records often lack lower bounds for affected versions, the paper's negative response times for Patch-Before-Publish cases should be read as lower bounds, meaning true pre-disclosure lead times may be even longer than reported.","If the same lifecycle classification were applied to other package ecosystems, the 151-to-78-day severity gradient would become a directly comparable metric for cross-ecosystem patching culture.","The correlation results rest on only a small subset of Maven packages that link to publicly accessible development histories, so the CVE-presence correlations may not generalize to the full Maven population."],"forward_implications":["If the severity trend holds, security teams can use severity scores to predict which disclosed vulnerabilities will be patched fastest and to flag low-severity backlogs for extra attention.","If 81.6% of Maven vulnerabilities are truly patched before disclosure, then public CVE data understates how much proactive patching already happens in the ecosystem.","If dependent packages wait a median 151 days even when a fix is available, then ecosystem risk is dominated by adoption lag rather than by patch availability.","If activity metrics like issue volume and contributor concentration correlate with CVE presence, then popular, heavily developed libraries deserve extra vulnerability monitoring.","If Reactive Adoption takes a median 249 days, then vulnerabilities disclosed before a patch is ready are the highest-leverage target for early-warning tooling."],"supporting_citations":[{"why":"Supplies the Maven dependency graph with artifact, release, and dependency timestamps used to define patch and adoption dates.","marker":"[13]"},{"why":"Provides the CVE publication dates, severity ratings, and affected version ranges that drive the lifecycle classification.","marker":"[5]"},{"why":"Prior manual analysis of Maven CVEs that this study scales up and compares against for lifecycle patterns.","marker":"[12]"},{"why":"Source of the repository activity metrics used for the correlation with CVE presence.","marker":"[17]"},{"why":"Non-parametric significance test used to assess whether the observed correlations are statistically meaningful.","marker":"[21]"},{"why":"Supplies the lifecycle analysis methodology for another ecosystem that this paper adapts to Maven.","marker":"[8]"},{"why":"Highlights incomplete CVE lower bounds, the paper's stated main threat to the validity of its patching-behavior claims.","marker":"[28]"},{"why":"Defines the contributor concentration metric that appears among the strongly correlating repository characteristics.","marker":"[23]"}],"fun_headline_variants":["Maven critical CVEs patched 48% faster than low severity","Critical Maven CVEs fixed in 78 days; low severity takes 151","Severity predicts Maven patch speed: 78d critical vs 151d low","Maven's publish-before-patch: severe flaws patched faster","Maven study: 81.6% of CVEs patched before public disclosure"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every headline number depends on the vulnerability database's publication dates and affected-version ranges being complete and correctly aligned with the dependency graph's release timestamps; if those dates are wrong or missing lower bounds, the 81.6% patch-before-publish share and the 48.3% severity response-time gap can shift or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Maven critical CVEs patched 48% faster than low severity","Critical Maven CVEs fixed in 78 days; low severity takes 151","Severity predicts Maven patch speed: 78d critical vs 151d low","Maven's publish-before-patch: severe flaws patched faster","Maven study: 81.6% of CVEs patched before public disclosure"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001155,"raw_usage":{"total_tokens":4811,"prompt_tokens":998,"completion_tokens":3813,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":3709}},"tokens_in":614,"tokens_out":3813,"duration_ms":28609,"temperature":1.0,"reasoning_tokens":3709,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T22:04:44.482283+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the headline numbers using only CVEs whose records give an explicit lower bound for affected versions, and compare patch release dates against the package registry's actual release history for a sample of critical CVEs; if the 151-to-78-day severity gradient or the 81.6% patch-before-publish share collapses, the result is an artifact of incomplete metadata.","supporting_citations":[{"cited_title":"Navigating and exploring software dependency graphs using goblin,","cited_arxiv_id":null,"evidence_quote":"Supplies the Maven dependency graph with artifact, release, and dependency timestamps used to define patch and adoption dates."},{"cited_title":"Osv: Open source vulnerabilities,","cited_arxiv_id":null,"evidence_quote":"Provides the CVE publication dates, severity ratings, and affected version ranges that drive the lifecycle classification."},{"cited_title":"Discovery of Timeline and Crowd Reaction of Software Vulnerability Disclosures","cited_arxiv_id":"2411.07480","evidence_quote":"Prior manual analysis of Maven CVEs that this study scales up and compares against for lifecycle patterns."},{"cited_title":"OpenDigger,","cited_arxiv_id":null,"evidence_quote":"Source of the repository activity metrics used for the correlation with CVE presence."},{"cited_title":"On a test of whether one of two random variables is stochastically larger than the other,","cited_arxiv_id":null,"evidence_quote":"Non-parametric significance test used to assess whether the observed correlations are statistically meaningful."},{"cited_title":"Empirical analysis of security vulnerabilities in python packages,","cited_arxiv_id":null,"evidence_quote":"Supplies the lifecycle analysis methodology for another ecosystem that this paper adapts to Maven."},{"cited_title":"Mitigating persistence of open-source vulnerabilities in maven ecosystem,","cited_arxiv_id":null,"evidence_quote":"Highlights incomplete CVE lower bounds, the paper's stated main threat to the validity of its patching-behavior claims."},{"cited_title":"Contributor absence factor,","cited_arxiv_id":null,"evidence_quote":"Defines the contributor concentration metric that appears among the strongly correlating repository characteristics."}],"review_version":1}