{"id":"3e86f9b1-1dfa-4abf-9723-4d2466898886","arxiv_id":"2608.08162","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An SOAR+SDN framework that combines passive asset discovery, adaptive two-stage scanning, context-aware risk banding, and automatic mitigation, with testbed results claiming large scan-time and prioritization-workload reductions.","lead":"The paper builds an automated security pipeline that discovers devices, scans them in two stages, ranks vulnerabilities by risk, and blocks or quarantines risky devices through software-defined networking. It reports large reductions in scan time and in the number of vulnerabilities needing immediate action, though part of that evaluation is circular.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 75% workload-reduction claim rests on a circular evaluation in Section V-F: KEV is both an input feature and the ground-truth label, so 'zero missed exploited vulnerabilities' is guaranteed by construction rather than demonstrated.","rationale":"The reader's verdict is CONDITIONAL, and my stress-test supports that verdict, so no change is needed. The reader's stated weakest_assumption was the device-classification accuracy concern from Section VI-C, but the reader's rationale also identified the circular risk-model validation as the reason for CONDITIONAL. I agree with the rationale; the circularity is the more load-bearing concern because it directly affects the paper's strongest claim: the approximate 75% reduction in immediate-mitigation workload without missing exploited vulnerabilities. The reader's stated weakest_assumption addresses a different promise (minimizing disruption to fragile devices) and is explicitly acknowledged as a limitation in Section VI-C, making it a known boundary rather than an unvalidated central assertion. The circularity, by contrast, is presented as a validated statistical result. The paper's own Section V-F admits that using confirmed exploitation alone would drive false positives to zero but 'would also remove any predictive capability' - yet no predictive capability is actually tested. The evaluation is in-sample: KEV is both feature and label. The 75% figure is a policy outcome under a chosen EPSS cutoff, not evidence that the model anticipates exploitation. This does not invalidate the engineering contributions or the measured scan-time and latency results, which are credible small-testbed measurements. It does mean the headline risk-prioritization claim should be reframed as a threshold-policy demonstration, with a prospective or temporal out-of-sample evaluation needed to support predictive claims. The proposed concrete test (temporal holdout with KEV removed from inputs) would settle whether the concern lands. If recall remains high on future KEV entries, the model has genuine predictive value; if not, the 'without missing' claim is an artifact of the evaluation design.","tokens_in":30893,"tokens_out":2592,"duration_ms":25954,"concrete_test":"Re-run the Section V-F evaluation with a temporal holdout. Use EPSS scores and KEV entries available as of a fixed date T (e.g., 2025-01-01) to score vulnerabilities published before T, then test against KEV entries added after T. Remove KEV from the model's input features entirely, using only EPSS (and CVSS) to assign risk bands. Compute recall on the future KEV positives and the reduction in the immediate-mitigation set relative to a CVSS-only baseline. If recall on future KEV entries falls materially below 100%, the 'without missing any vulnerability with verified exploitation' claim is not supported as a predictive statement. As a secondary check, report how the 75% reduction and the 19 false positives vary across EPSS cutoff values (e.g., 0.05, 0.1, 0.2, 0.3).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline quantitative claim is that the context-aware risk model reduces vulnerabilities requiring immediate mitigation by ~75% (420 to 103) while preserving complete coverage of verified exploited vulnerabilities (Section V-F, Figure 4). This claim is not supported as a predictive result because the evaluation is circular. In Algorithm 2, exploitability is set to high if a CVE is in CISA KEV or if EPSS >= cutoff. In the Section V-F evaluation, the positive ground-truth class is defined as KEV membership, and the model uses KEV membership as an input feature. Consequently, every positive case is prioritized by construction; the observed 100% recall of KEV entries is tautological. The 75% reduction is computed on a case-control sample of 2025 CVEs with CVSS >= 9.0, with 1:4 matching, comparing the model's immediate-mitigation set against a CVSS-only baseline. This measures the filtering effect of the EPSS cutoff, not the model's ability to identify exploited vulnerabilities. The paper explicitly acknowledges that the 19 remaining false positives are 'the cost of anticipating exploitation,' but it provides no temporal or out-of-sample test showing that high-EPSS non-KEV vulnerabilities are actually more likely to be exploited later. Because the central contribution is framed in the abstract and conclusions as 'without missing any vulnerability with verified exploitation,' the lack of a prospective evaluation leaves the key performance claim unvalidated. The device-classification concern identified by the reader is real but secondary; it affects the disruption-safety promise, whereas the circular validation directly undermines the primary workload-reduction result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a SOAR-orchestrated vulnerability management framework for heterogeneous networks, combining passive asset discovery, adaptive two-stage scanning, a CVSS/EPSS/KEV/context risk-band model, and SDN/OpenFlow/IDS-based mitigation. The main reported results are: up to a 91% reduction in adaptive scan time while preserving 71% vulnerability coverage after the fast scan; approximately a 75% reduction in vulnerabilities requiring immediate mitigation with complete coverage of verified exploited vulnerabilities; millisecond-scale mitigation enforcement; and scalable concurrent assessment of up to 32 hosts. The evaluation is performed on virtualized and hybrid testbeds, and the paper explicitly acknowledges several limitations, including the NVT filter providing no time savings, quarantine completion depending on DHCP renewal, and the device classifier not being evaluated.","tokens_in":31104,"tokens_out":7692,"duration_ms":76882,"significance":"If the results hold, the framework would be a practically useful integrated system: the adaptive scanning and SDN enforcement measurements are direct and plausible, and the authors are transparent about several limitations, which is good scientific practice. The central quantitative claim about the context-aware risk model reducing immediate-mitigation workload is, however, not currently supported as a predictive result. The evaluation in Section V-F is circular because KEV membership is both a model input and the ground-truth label, and the contextual component named in the headline is explicitly not exercised in that experiment. With a reframed or supplemented validation, the contribution would still be meaningful for automated vulnerability management, but the abstract and conclusions as written overstate what has been demonstrated.","major_comments":[{"comment":"The claim that the model reduces immediate-mitigation workload by approximately 75% while missing no exploited vulnerability is not supported as a predictive result. In Algorithm 2, exploitability is set to high when a CVE is listed in the CISA KEV catalog, and in Section V-F the positive ground-truth class is defined as KEV membership. Every positive case is therefore prioritized by construction, so the reported 84/84 true positives and zero false negatives are tautological. The reduction from 420 to 103 vulnerabilities is the filtering effect of the EPSS cutoff on the negative controls, not a measured ability to anticipate exploitation. The paper acknowledges that the 19 remaining positives are 'the cost of anticipating exploitation,' but no temporal or out-of-sample test shows that high-EPSS non-KEV vulnerabilities are actually exploited later. Moreover, the false-positive count and hence the 75% figure depend on the EPSS cutoff, which Section V-E states was not calibrated against a labeled exploitation dataset, and no sensitivity analysis is reported. Please either reframe the claim as workload reduction conditional on KEV/EPSS inputs, or add a temporal holdout evaluation (e.g., use KEV entries known before a cutoff date to predict entries added afterward). The abstract and conclusions should not state 'without missing any vulnerability with verified exploitation' as an empirical finding.","section":"Section V-F and Algorithm 2"},{"comment":"The paper's headline attributes the 75% workload reduction to the 'context-aware risk model,' but Section V-F explicitly states: 'The contextual component is not exercised in this experiment, since the evaluation operates on published vulnerability data rather than on deployed hosts.' The quantitative validation therefore covers only the EPSS/KEV exploitation axis; the context promotion mechanism from Section III-C and Table IV (sensitive zone, constrained device) is illustrated only qualitatively in Section V-E using a single host (Table XVI). This is a load-bearing mismatch between the claim and the evidence: the abstract and conclusions should either be restricted to the exploitation-aware component, or a quantitative experiment exercising the contextual component on deployed hosts should be added.","section":"Section V-F"},{"comment":"The device classification step is load-bearing for the safety promise of the framework. The claim that resource-constrained devices are not disrupted by active probing depends on fragile devices never being classified as traditional IT hosts. Section IV-B states that 'The classifier was not evaluated here,' and Section VI-C admits that 'The current architecture does not account for edge-case misclassifications, such as identifying a resource-constrained IoT device as a traditional IT host. This could inadvertently subject fragile assets to disruptive active probing.' This is the exact error direction that would break the 'minimizing disruption' objective of RQ1. Because the default-to-constrained rule does not address the fragile-as-IT case, the paper should either report an evaluation of classification accuracy with a misclassification-cost analysis, or explicitly limit the safety and non-disruption claims to the case where the classifier is correct.","section":"Section IV-B and Section VI-C"}],"minor_comments":[{"comment":"Two consecutive, nearly identical paragraphs begin with 'Table XIII reports the per-stage latency...' and 'Excluding scan execution...'; one copy should be removed. The same section also contains the typo 'is is approximately 2.0%.'","section":"Section V-D"},{"comment":"The column header 'Red.%' is ambiguous; it should read 'Reduction (%)' and the units of 'Time (m)' should be stated explicitly as minutes.","section":"Table IX"},{"comment":"The phrase 'Centralizing this scan logic in to the security management component' should be corrected to 'into the security management component,' and 'This work main novelty' in the introduction should be 'This work's main novelty.'","section":"Section IV-B"},{"comment":"The 'Aware Deep (complement)' row reports 42.9% coverage, but the text explains that deep-scan findings partially overlap with fast-scan findings because both identify host-level issues; presenting the unique additional coverage contributed by the deep scan would make the two-stage benefit clearer.","section":"Table XII and Section V-C"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing here is the closed loop. The paper actually builds a SOAR-orchestrated pipeline that combines passive DHCP-based classification, a DShield-driven two-stage fast/deep scan, a CVSS/EPSS/KEV/context risk matrix, and SDN/IDS enforcement, and runs it on a real testbed with physical hosts. The scan-time and mitigation-latency measurements are plausible and internally consistent, and the authors are unusually candid: they report that the NVT risk filter saves no time, that constrained-device scans miss a critical backdoor that only active probing would confirm, and that quarantine only completes after DHCP lease renewal. That transparency is real credit. The citation pattern is also fine; the heavy reliance on their own prior work is legitimate here because they are extending it.\n\nThe soft spot is the headline claim. The abstract and conclusions say the risk model cuts immediate-mitigation workload by ~75% without missing any verified exploited vulnerability. Section V-F validates this on a 2025 case-control sample where KEV membership is both the input feature that sets exploitability high in Algorithm 2 and the ground-truth label for the positive class. So perfect recall of the 84 KEV entries is guaranteed by the model definition. The 94% false-positive reduction then mostly measures the EPSS cutoff, not the model's ability to predict exploitation. The paper's own Section V-E admits no labeled ground-truth dataset exists, which makes the subsequent V-F framing all the more problematic. The stress-test note holds up. What is needed is a temporal or out-of-sample test: use EPSS at publication time to predict later KEV entry, and report sensitivity to the EPSS cutoff and the port-list size.\n\nThe device-classification concern is real but secondary, and the authors flag it themselves in Section VI-C. The paper's other claims—scan coverage trade-offs, millisecond enforcement, scalability—hold up as small-testbed results.\n\nThis is a solid engineering system with one overstated evaluation. It deserves a serious referee, but the authors should be pushed to redo the risk-model validation prospectively and tone down the abstract's 'without missing' claim.","headline":"Genuinely integrated SOAR/SDN vulnerability management with credible testbed measurements, but the headline 75% workload-reduction claim is circular because KEV serves as both input feature and ground-truth label.","tokens_in":31767,"tokens_out":1841,"would_cite":false,"duration_ms":19655,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that a closed-loop, context-aware vulnerability management system can cut the urgent mitigation workload by about 75% while keeping every known-exploited vulnerability in the priority set.","keywords":["security orchestration automation and response","vulnerability management","adaptive two-stage scanning","context-aware risk prioritization","exploit prediction scoring system","software-defined networking","internet of things","automated mitigation"],"falsifier":"Run the risk model on vulnerabilities whose exploitation is independently confirmed but that were absent from the known-exploited catalog at the time of scoring; if any such exploited vulnerability falls outside the immediate-mitigation band, the central claim of no missed exploited vulnerabilities fails.","tokens_in":30607,"feed_emoji":"🛡️","tokens_out":8196,"duration_ms":79790,"temperature":0.7,"pith_summary":"This paper argues that vulnerability management for heterogeneous networks can be made faster, safer, and less labor-intensive by replacing uniform scanning and severity-only prioritization with a closed, automated loop that adapts to each device. The proposed framework passively classifies assets from DHCP metadata, runs a quick threat-informed scan and then a deeper scan only when warranted, ranks every finding into four risk bands by combining CVSS severity, EPSS exploitation probability, known-exploited status, and device context, and enforces proportional SDN/IDS responses from logging to service blocking to full quarantine. In the authors' experiments the two-stage scan reduced assessment time by up to 91% while the fast stage alone kept 71% of baseline vulnerability coverage, and the context-aware risk model cut the number of vulnerabilities flagged for immediate mitigation by roughly 75% without dropping any vulnerability with verified exploitation. If these results hold, security teams can focus remediation on a much smaller genuinely urgent set, and fragile IoT devices are spared the disruptive active probing that uniform scanners would send their way.","feed_headline":"Risk model cuts urgent vulnerability fixes by 75% with zero misses","feed_subtitle":"Adding exploitation likelihood and device context keeps every known-exploited flaw in the immediate response set.","key_machinery":"The load-bearing mechanism is the four-band risk model supported by an adaptive scan front end. The risk model is a $3 \\times 2$ matrix: CVSS severity tiers (high, moderate, low) crossed with exploitation likelihood (high or low, derived from EPSS probability and known-exploited catalog membership, with missing evidence treated as high by default); an exploited moderate-severity finding outranks an unexploited severe one, and context (sensitive zone or constrained device) promotes the band by exactly one step. This matrix is what produces the 75% reduction in urgent findings while retaining every known-exploited vulnerability. The scan front end makes the reduction operationally useful by pairing a threat-informed fast port list—the union of weekly, monthly, and yearly most-attacked ports—with a device-class-specific test profile, so the fast stage returns 71% of baseline findings in about 14 minutes on a constrained device instead of an hour or more.","core_discovery":"The paper's central claim is that an orchestrated closed loop—passive asset discovery, adaptive two-stage assessment, context-aware risk scoring, and automated mitigation—manages the full vulnerability lifecycle better than the conventional sequential pipeline. Device class decides scan depth: resource-constrained assets receive only reconnaissance-oriented tests and remain under passive monitoring, while traditional IT hosts get the full active test set. The fast scan uses a dynamically built list of the most attacked ports so an initial risk posture and mitigation decision can be made in minutes; the deep scan covers the complementary ports afterwards. Risk is assigned per finding through a severity-by-exploitability matrix whose output band (CRITICAL, HIGH, MEDIUM, LOW) can be promoted one step for sensitive-zone or constrained devices, and each band maps to a distinct SDN-enforced action executed within milliseconds. The statistical validation uses a case-control design on 2025 published vulnerabilities with CVSS 9.0 or higher: all 84 known-exploited cases are retained by the model, while the number of non-exploited vulnerabilities escalated to immediate mitigation falls from 336 to 19, a 75% reduction in the urgent set.","pith_inferences":["Beyond the paper: the 'zero missed exploited vulnerabilities' result is partly definitional, because the known-exploited catalog is both a model input and the evaluation ground truth; a stronger test would use independently confirmed exploitation events with timestamps to see whether the model flags them before or at first exploitation.","Beyond the paper: the worst-case default for missing exploitability evidence means freshly disclosed vulnerabilities with no EPSS or catalog entry are still escalated, so the 75% reduction may be smaller in a zero-day-heavy workload than in this retrospective evaluation.","Beyond the paper: the per-finding risk bands and per-service enforcement could be reused as inputs for zero-trust access decisions, turning each host's vulnerability posture into a continuous, context-aware trust signal."],"forward_implications":["The urgent-remediation queue shrinks by about three quarters, so analysts and automated response systems concentrate on a small set of findings that are either being exploited or very likely to be.","Because the fast scan returns an actionable posture in minutes, mitigation for high-risk findings can start before the deep scan finishes, shortening the exposure window.","Resource-constrained IoT devices receive only reconnaissance-oriented tests and continuous passive monitoring, reducing the risk that scanning itself disrupts fragile services.","Mitigation rules land on the data plane in milliseconds, so a host admitted with a critical finding can be contained before normal operation proceeds.","Scan execution, not orchestration, is the scalability bottleneck; adding CPU or distributing scan engines is the direct path to larger deployments."],"supporting_citations":[{"why":"supplies the EPSS exploitation-probability scores that form the exploitability axis of the risk model.","marker":"[6]"},{"why":"provides the vulnerability-management chaining approach with an EPSS threshold that the proposed model extends with context.","marker":"[7]"},{"why":"is the baseline scanning framework used for the 32-host scalability comparison.","marker":"[9]"},{"why":"supplies the case-control methodology used to statistically validate exploitation-aware prioritization.","marker":"[12]"},{"why":"defines the CVSS severity scoring that anchors the severity tiers of the risk matrix.","marker":"[13]"},{"why":"motivates device-type identification and SDN-based flow confinement for vulnerable IoT devices.","marker":"[15]"},{"why":"shows how DHCP metadata fingerprints devices, the basis for the passive classification stage.","marker":"[20]"}],"fun_headline_variants":["Adaptive assessment cuts scan time 91%, slashes urgent fixes 75%","Zero missed exploited vulnerabilities as urgent fix list shrinks 75%","Context-aware risk model trims urgent mitigations 75% with zero misses","SOAR-driven loop: 91% faster scans, 75% fewer urgent patches","Risk model keeps all exploited flaws, cuts urgent fixes 75%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes the passive device classifier is accurate enough that a fragile IoT device is never misclassified as a traditional IT host, because a misclassification would subject that device to disruptive active probing that could degrade or crash it.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive assessment cuts scan time 91%, slashes urgent fixes 75%","Zero missed exploited vulnerabilities as urgent fix list shrinks 75%","Context-aware risk model trims urgent mitigations 75% with zero misses","SOAR-driven loop: 91% faster scans, 75% fewer urgent patches","Risk model keeps all exploited flaws, cuts urgent fixes 75%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000635,"raw_usage":{"total_tokens":2976,"prompt_tokens":1038,"completion_tokens":1938,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":1838}},"tokens_in":654,"tokens_out":1938,"duration_ms":13181,"temperature":1.0,"reasoning_tokens":1838,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:19:57.419394+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the risk model on vulnerabilities whose exploitation is independently confirmed but that were absent from the known-exploited catalog at the time of scoring; if any such exploited vulnerability falls outside the immediate-mitigation band, the central claim of no missed exploited vulnerabilities fails.","supporting_citations":[{"cited_title":"Toward automatic detection and mitigation of high-risk cybersecurity vulnerabilities at networked systems,","cited_arxiv_id":null,"evidence_quote":"is the baseline scanning framework used for the 32-host scalability comparison."},{"cited_title":"Common vulnerability scoring system,","cited_arxiv_id":null,"evidence_quote":"defines the CVSS severity scoring that anchors the severity tiers of the risk matrix."},{"cited_title":"Iot sentinel: Automated device-type identification for se- curity enforcement in iot,","cited_arxiv_id":null,"evidence_quote":"motivates device-type identification and SDN-based flow confinement for vulnerable IoT devices."},{"cited_title":"All things considered: An analysis of {IoT}devices on home networks,","cited_arxiv_id":null,"evidence_quote":"shows how DHCP metadata fingerprints devices, the basis for the passive classification stage."}],"review_version":1}