{"id":"69c6cd5b-3c64-401a-a74a-8417c707573e","arxiv_id":"2411.11202","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A time dependency tree model combines dependency graphs, source-code version history, and fitted CVE disclosure curves to estimate the probability that a project faces a CVE within a future window.","lead":"This paper introduces a formal model that estimates the probability a software project will face a new security vulnerability in a future time window, using its dependency tree and the evolution of its libraries over time. A generalist should read it because it promises practical, white-box risk forecasts for deciding when to update software dependencies.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The validation is in-sample: cluster CDFs in Sec. 6.2 are fitted on all 394 evidence instances, including CVE-2021-39139, the same CVE Sec. 6.3 claims to foretell, so the reported probability rise is not an out-of-sample forecast.","rationale":"I read the paper's central claim as a forecasting claim: given a project and a future horizon, the model outputs a probability that a CVE will be disclosed for the project or one of its dependencies. For that claim to be true, the probability curves must be estimated from data available before the forecast origin. The paper explicitly acknowledges this in the RQ section, where label creep is defined as a necessary condition, and again in Section 8.2, where using probability estimates as part of the clustering data is called label creep. Yet the empirical demonstration in Section 6.3 fits the cluster CDFs on the full 394-instance corpus in Section 6.2 and then uses those curves to 'foretell' CVE-2021-39139, which is part of that corpus. This is not a subtle issue: the difference between p_hat_2 and p_hat_3 is driven by the KDE mass in the window after xstream:1.4.17's release, and the target CVE's grace period lies in that window. The reported 0.096 to 0.158 rise is thus a property of in-sample fitting, not demonstrated predictive skill. The paper also provides no calibration or baseline comparison, and this single retrospective anecdote is the only empirical support for the central claim. I agree with the reader's rejection; my stress test did not find a reason to soften it. The proposed temporal holdout is feasible with the released artifact and would settle the point. If the holdout shows the rise survives and the probabilities calibrate, the paper could be reconsidered.","tokens_in":32030,"tokens_out":5909,"duration_ms":57917,"concrete_test":"Rerun the Section 6.2/6.3 demonstration with a strict temporal split: for each forecast origin t_k (e.g., Jun 25 and Jul 25, 2021), fit the four cluster KDEs using only evidence instances whose CVE publication date is before t_k, excluding CVE-2021-39139 entirely, then recompute the leaf probabilities and the total p_hat_2 and p_hat_3. If the reported rise disappears, or if no cluster KDE can be fitted from the pre-t_k data, the reported validation is contaminated and the central forecasting claim needs substantially different support; a full walk-forward calibration over many historical CVEs, with a reliability diagram or Brier score, would settle the question.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a forecasting claim: the model estimates the probability that a software project faces a CVE disclosure in a future time window. For that claim to hold, the curves used for prediction must be fitted without information about the CVE being predicted. The paper itself states this requirement: Section 1 says any metric answering RQ1 must avoid label creep, and data used to foretell an unseen vulnerability kappa in a future time window T must exclude information on the occurrence of kappa during T. The empirical demonstration violates this requirement. Section 6.2 fits each cluster KDE (Fig. 12) on grace periods from all 394 evidence library instances, with no temporal split. CVE-2021-39139 is among those evidence instances: Section 6.1 selects high/critical Java CVEs from Snyk, and xstream:1.4.17 is the evidence instance for that CVE, the very leaf l_z analyzed in Section 6.3. Section 6.3 then uses the fitted CDFs to compute p_hat_2 ~ 0.096 and p_hat_3 ~ 0.158 and claims a rise that could have been produced before Aug 2021. But the rise is driven by KDE mass in the time window [42, 117] days after xstream:1.4.17's release, and the target CVE's grace period lies inside that window. The demonstration is therefore in-sample. No out-of-sample evaluation, calibration analysis, or baseline comparison is reported, and Section 8.2's own warning against label creep is not applied to the validation. Without a temporal holdout, the abstract's claim that the model can estimate the probability of a future CVE disclosure is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a white-box model for forecasting the probability that a software project faces a CVE disclosure in a future time window. The model formalizes library dependencies and source-code evolution through 'time dependency trees' (TDTs), clusters libraries by own-code size and web-orientation, fits kernel density estimates (KDEs) to historical grace periods (release-to-CVE times), and propagates per-dependency probabilities through an attack-tree structure. The empirical demonstration uses 1255 CVEs and 768 Java libraries from Maven, focusing on a motivating example where jira-core depends on xstream:1.4.17 and the model allegedly forecasts CVE-2021-39139. The paper claims that the resulting probabilities can guide dependency-update decisions.","tokens_in":32342,"tokens_out":7530,"duration_ms":65833,"significance":"If the forecasting claim were validated, the TDT formalism would be a useful, lightweight, interpretable contribution to vulnerability risk assessment, addressing a gap in the literature that mostly focuses on detection of existing vulnerabilities. The paper's emphasis on white-box modeling, on forecasting rather than detection, and on releasing an open-source artifact are clear strengths. However, the significance is contingent on the empirical demonstration, and that demonstration is invalidated by label creep: the KDEs used for prediction are fitted on data that include the very CVE the paper claims to forecast. Without a valid out-of-sample evaluation, the central claim that the model can forecast future CVE disclosures is unsupported.","major_comments":[{"comment":"Section 6.2 fits the KDEs on all 394 evidence instances, which include the grace period of xstream:1.4.17 for CVE-2021-39139; Section 6.3 then uses these KDEs to compute p̂2≈0.096 and p̂3≈0.158 and claims the rise 'could have been produced before Aug 2021'. This violates the paper's own definition of label creep in Section 1 ('data used to foretell an unseen vulnerability κ in a future time window T must exclude information on the occurrence of κ during T') and the warning in Section 8.2. The probability rise is necessarily influenced by the very CVE it claims to forecast, so the validation is in-sample and the central forecasting claim is unsupported as demonstrated.","section":"Sec. 6.2 / 6.3"},{"comment":"Section 6 contains no out-of-sample evaluation: there is no temporal holdout, no cross-validation, no calibration analysis, and no comparison with a baseline model (e.g., a constant-rate or release-age-only model). The only quantitative validation is the retrospective xstream example, which is contaminated by the fitting data as noted above. Consequently, the manuscript provides no evidence that the model generalizes to CVEs not present in the fitting set.","section":"Sec. 6"},{"comment":"Section 5.2 fits F_ι to the grace periods of the 394 evidence instances, i.e., library instances already known to be affected by at least one high/critical CVE; Section 5.4 then uses F_ι(Δ+n)-F_ι(Δ) as the absolute probability that a dependency faces a CVE in the next n days. Because F_ι is a conditional distribution (given that a CVE occurs), it overestimates the risk for library instances that may never receive a CVE. The model lacks a base-rate or zero-inflation component, and this limitation is not acknowledged in the paper.","section":"Sec. 5.2 / 5.4"}],"minor_comments":[{"comment":"The caption spells 'com.attlassian.jira' with a double 't'; the correct Maven coordinate is 'com.atlassian.jira'.","section":"Fig. 1 caption"},{"comment":"The inclusion-exclusion formula for the CDF of the fastest CVE is written as F_{A1+...+AN}(t)=∑_{J⊆{1,...,N}}(-1)^{|J|+1}|F_{Σ{A_i|i∈J}}(t)|; this notation is undefined (F of a set? absolute value of a CDF?) and, under the independence assumption used elsewhere in the paper, the correct expression should be 1-∏_{i=1}^N (1-F_{A_i}(t)).","section":"Sec. 7, Eq. (1)"},{"comment":"The KDE fits in Fig. 12 are presented without confidence bands or any measure of fit uncertainty; given the small cluster sizes (40-188 points), a bootstrap or similar would help assess the reliability of the tail behavior that drives the main example.","section":"Fig. 12"},{"comment":"The condition |C_ι|>30 is stated as necessary for fitting a pdf, but no justification or reference is given for this threshold.","section":"Sec. 5.1"}],"recommendation":"reject","confidential_remarks":"The label-creep issue is not a presentation problem: the only quantitative validation in the paper is circular, and the paper itself identifies label creep as a requirement for any forecasting metric. Fixing this would require re-running the entire empirical analysis with a proper temporal holdout and providing calibration evidence, which is beyond a routine revision. I recommend rejection of the current version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the time dependency tree formalism—c-chains, d-matrices, indexing dependency trees over time—is genuinely new and well worked out. It gives researchers a clean way to talk about library evolution and dependency structure in one graph, and the mapping to attack trees for probability propagation is sensible. That part deserves a careful read. Second, the empirical demonstration in Section 6 does not support the abstract's forecasting claim. The KDE curves in Section 6.2 are fitted on the full set of 394 evidence instances, which includes CVE-2021-39139; Section 6.3 then uses those same curves to 'foretell' that CVE. That is label creep, and the paper itself warns against it in Section 1 and Section 8.2. The stress-test note is right.\n\nThe authors do several things well: they define evidence instances carefully, they release an artifact, they are explicit about assumptions (clustering, independence), and they flag overfitting risks in the discussion. The writing is clear and the formal sections are precise.\n\nThe load-bearing flaw is the validation. There is no temporal holdout, no calibration analysis, no error bars, no baseline comparison. The reported rise in probability from 0.096 to 0.158 is just the fitted CDF increment evaluated on the target's own grace period. As a forecasting claim, that is circular. Two smaller issues: the leaf probability is an unconditional CDF increment, not a conditional hazard given no CVE so far, so it will underestimate risk for old dependencies; and the OR propagation assumes independence across leaves, which the paper acknowledges but does not test. Those are modeling choices, not fatal, but they should be discussed.\n\nThis paper is for the vulnerability forecasting and software supply-chain risk community. The formal model is worth engaging with, but the empirical section needs to be redone with a proper temporal split. The authors know what label creep is; they just did not apply their own rule to the validation.\n\nSend it to peer review. A serious referee can push for the holdout experiment and calibration, and the formal TDT part deserves to be in the literature. I would not accept it as is.","headline":"The TDT framework is a genuinely new formalization, but the paper's only empirical demonstration is in-sample: the CVE it claims to 'foretell' was used to fit the very curves that produce the forecast.","tokens_in":32948,"tokens_out":2277,"would_cite":false,"duration_ms":22423,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the probability of a CVE disclosure in a future window can be estimated at library level from dependencies and source-code evolution, via a white-box time dependency tree model.","keywords":["vulnerability forecasting","CVE disclosure prediction","time dependency trees","attack tree analysis","software supply chain risk","kernel density estimation","library dependencies","Maven ecosystem"],"falsifier":"Re-run the jira-core demonstration with a temporal holdout: fit the cluster CDFs using only CVEs disclosed before, say, 1 June 2021, recompute the 45-day probability for jira-core:8.19.1 at the end of June and July 2021, and check whether the rising signal toward xstream:1.4.17 still appears before CVE-2021-39139's disclosure on 30 August 2021; if the probability no longer rises, or rises only because CVE-2021-39139 is in the fitting data, the forecasting claim is refuted, whereas a positive result would show the signal is not an artifact of in-sample fitting.","tokens_in":2065,"feed_emoji":"📈","tokens_out":4986,"duration_ms":136637,"temperature":0.7,"pith_summary":"The paper tries to establish that future security vulnerabilities are forecastable at the granularity of individual libraries, not just as aggregate counts for a whole ecosystem. It claims that by combining the dependency tree of a project with the release history of each library, one can compute the probability that the project faces a CVE disclosure within a chosen future window, and can rank which dependency contributes most to that risk. A developer weighing whether to update a library would get a concrete number to compare. The authors demonstrate the model on 1255 CVEs and 768 Java libraries, including a reconstruction of the 2021 jira-core/xstream case where the rising probability signal would have advised the update that hindsight proved correct.","feed_headline":"Forecasting the odds of a CVE in your dependency tree","feed_subtitle":"Developers get a white-box probability that their project or a dependency faces a CVE in the coming window.","key_machinery":"The load-bearing object is the time dependency tree (TDT): a dependency graph extended with a time dimension. A c-chain orders the versions of one library by release date, and a d-matrix tracks which versions of a dependency appear across successive dependency trees of the main library, allowing version stuttering, adoption, and dropping of dependencies. Indexing the TDT at a time point $t_k$ recovers the ordinary dependency tree, which is converted into an attack tree via the bijection of Section 3.2: each library instance becomes a basic attack step and each non-leaf node becomes an OR gate. Each leaf, a dependency $x$ released $\\Delta_x^k$ days before $t_k$, is labelled with $\\hat{p}_x^k$ as above, where $F_\\iota$ is the CDF of the cluster containing $x$; the cluster CDF is a kernel-density estimate fitted once per ecosystem on historical grace periods, stratified by own-code size (Small/Medium versus Large, threshold 100k LoC) and web-orientation (Remote network versus Local). OR-propagation through standard quantitative attack-tree algorithms yields the project-level estimate $\\hat{p}_k$. For the few-dependency case, an analytical alternative uses $F_{A+B}(t) = 1 - S_A(t)S_B(t)$ and inclusion-exclusion.","core_discovery":"The paper's central claim is that CVE disclosures are forecastable at library level: the probability that a project faces a CVE in a future window of $n$ days can be computed from the dependency structure and the release history of each library instance. Each dependency contributes a displaced integral of a cluster CDF, $\\hat{p}_x^k = F_\\iota(\\Delta_x^k + n) - F_\\iota(\\Delta_x^k)$, and these leaf probabilities are combined through an attack-tree aggregation to give the project-level probability. The authors argue the model is white-box and lightweight, and demonstrate it on jira-core in 2021, where the 45-day probability rises from about 0.096 on 25 June to 0.158 on 25 July, driven mainly by mxparser:1.2.1 and xstream:1.4.17 — a signal that would have supported updating xstream before CVE-2021-39139 was disclosed.","pith_inferences":["A temporal-holdout re-run would settle the forecasting claim: fitting the cluster CDFs only on CVEs disclosed before the forecast date, then recomputing the jira-core probabilities, would show whether the pre-disclosure signal survives when the predicted CVE is not inside the fitting data—a test the paper's own label-creep discussion calls for but its demonstration does not perform.","The same pipeline could be deployed as a release-time check: compute the project-level probability before each release, and treat a rising $n$-day probability concentrated in one dependency as a trigger to update or replace that library; the paper shows the ingredients, such as the pdf reset after an update, but leaves the decision rule implicit.","Transfer to other package ecosystems is the natural next experiment, but the cluster features are calibrated on Java/Maven, so a replication on npm or PyPI would test whether the four-cluster stratification generalizes or needs re-calibration.","The OR-gate aggregation treats CVE disclosures in different dependencies as independent events, yet the paper itself notes that parallel release chains of one library share vulnerabilities; correlated disclosures would shift the joint probability, and quantifying that correlation is an open direction the TDT machinery could support."],"forward_implications":["A developer can compute, for any date and any horizon $n$, the probability that their project faces a CVE, and can compare update policies—keeping a dependency versus moving to a newer instance—by the resulting change in that probability.","The attack-tree decomposition exposes which library instances drive the project-level risk, so security-sensitive points can be identified and prioritized for updates.","Because cluster CDFs are fitted once per ecosystem and reused, per-project estimation scales linearly in the number of dependencies and takes seconds to minutes, making routine forecasting feasible for projects with dozens of dependencies.","For small dependency sets, the analytical survival-function convolution provides the full probability density of the fastest expected CVE, revealing local minima and maxima that point estimates at discrete time points miss.","Beyond forecasting, the TDT representation supports detecting pervasive dependencies and single-points-of-failure that threaten several versions of a project at once, and measuring ecosystem health from the fraction of popular c-chains affected by disclosed CVEs."],"supporting_citations":[{"why":"Supplies the 100k-LoC own-size threshold and the technical-leverage measure used to split libraries into Small/Medium versus Large clusters.","marker":"[44]"},{"why":"Supplies the structured web-orientation classification (Remote network versus Local) that defines the second clustering dimension.","marker":"[49]"},{"why":"One of the two kernel-density-estimation methods the paper cites as the basis for fitting cluster PDFs from grace-period data.","marker":"[50]"},{"why":"The second kernel-density-estimation reference grounding the empirical CDF fitting.","marker":"[56]"},{"why":"Provides the quantitative attack-tree propagation algorithms used to aggregate leaf probabilities into the project-level estimate.","marker":"[40]"},{"why":"Grounds the dependency terminology and the counting of actually vulnerable evidence library instances used to build the grace-period dataset.","marker":"[52]"},{"why":"The vulnerability database that supplied the 1255 high- and critical-severity CVEs and 768 libraries of the empirical demonstration.","marker":"[61]"},{"why":"The exploit-prediction approach that the paper contrasts with forward-looking forecasting to motivate the research question.","marker":"[27]"}],"fun_headline_variants":["White-box forecast: your next CVE from these libraries","Library-level CVE odds from dependency and code history","Model scores dependency risk for future CVE disclosures","Know your CVE risk: a white-box, lightweight forecaster","Forecasting CVE disclosures from dependencies and code change"],"cache_read_input_tokens":34944,"weakest_assumption_plain":"The entire method rests on the assumption that the disclosure curves fitted on the full historical CVE record, which includes the very CVEs the demonstration claims to foretell, still hold for future time windows, and the validation never fits on a temporal cutoff to test that assumption.","fun_headline_variants_meta":{"raw":{"variants":["White-box forecast: your next CVE from these libraries","Library-level CVE odds from dependency and code history","Model scores dependency risk for future CVE disclosures","Know your CVE risk: a white-box, lightweight forecaster","Forecasting CVE disclosures from dependencies and code change"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000145,"raw_usage":{"total_tokens":1147,"prompt_tokens":883,"completion_tokens":264,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":185}},"tokens_in":499,"tokens_out":264,"duration_ms":3508,"temperature":1.0,"reasoning_tokens":185,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:48:21.549450+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the jira-core demonstration with a temporal holdout: fit the cluster CDFs using only CVEs disclosed before, say, 1 June 2021, recompute the 45-day probability for jira-core:8.19.1 at the end of June and July 2021, and check whether the rising signal toward xstream:1.4.17 still appears before CVE-2021-39139's disclosure on 30 August 2021; if the probability no longer rises, or rises only because CVE-2021-39139 is in the fitting data, the forecasting claim is refuted, whereas a positive result would show the signal is not an artifact of in-sample fitting.","supporting_citations":[{"cited_title":"Cross-ecosystem categorization: A manual-curation protocol for the categorization of Java Maven libraries along Python PyPI Topics","cited_arxiv_id":"2403.06300","evidence_quote":"Supplies the structured web-orientation classification (Remote network versus Local) that defines the second clustering dimension."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the dependency terminology and the counting of actually vulnerable evidence library instances used to build the grace-period dataset."},{"cited_title":"Snyk vulnerability DB","cited_arxiv_id":null,"evidence_quote":"The vulnerability database that supplied the 1255 high- and critical-severity CVEs and 768 libraries of the empirical demonstration."},{"cited_title":"Enhancing Vulnerability Prioritization: Data-Driven Exploit Predictions with Community-Driven Insights","cited_arxiv_id":"2302.14172","evidence_quote":"The exploit-prediction approach that the paper contrasts with forward-looking forecasting to motivate the research question."}],"review_version":1}