Measuring likelihood in cybersecurity

Edward J. Humphreys; Gabriel Sanchez-Perez; Jesus Manuel Niebla-Zatarain; Pablo Corona-Fraga; Vanessa Diaz-Rodriguez

Measuring likelihood in cybersecurity

T0 review · 2 major / 2 minor · reviewed 2026-05-22 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗

Pith's one-line read A graph pipeline turns mixed cyber data into a bounded likelihood score comparable across organizations.

desk verdict The paper formalizes a graph pipeline for turning messy cyber data into comparable likelihood scores via four incident-based constructs, but the 15-organization evaluation only shows loose association without controls or baselines. read the letter →

arxiv 2504.15395 v4 submitted 2025-04-21 cs.CR

Pablo Corona-Fraga , Vanessa Diaz-Rodriguez , Jesus Manuel Niebla-Zatarain , Gabriel Sanchez-Perez , Edward J. Humphreys This is my paper · ORCID

classification cs.CR

keywords cybersecurityrisklikelihoodestimationgraphrepresentationincidentreductionmanagementcyberexposureprofilesecuritycontrols

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Cybersecurity risk assessments have long lacked a reliable way to quantify likelihood because incidents are underreported and datasets from different sources resist direct comparison. The paper builds a pipeline that folds external threat knowledge together with an organization's internal telemetry into a single graph model. Four constructs drawn from past incidents—Exposure, Traceability, Motivation, and Systems Update—supply the model's structure and produce a likelihood indicator that stays within fixed bounds. When fifteen organizations applied the resulting profile, the work records an association with fewer incidents and shorter detection and response intervals.

What carries the argument

The cyber exposure profile, a graph representation that integrates external cyber knowledge with organization-specific telemetry to generate a likelihood score from the four constructs of Exposure, Traceability, Motivation, and Systems Update.

What would settle it

A larger controlled study that randomly assigns organizations to adopt or not adopt the profile and still finds no difference in incident frequency or response times after accounting for overall security investment.

Watch

Extended reading notes

Core claim

The authors present a formally specified artifact chain that begins with a unified data model, proceeds through organization-specific profiling and a metric registry, computes a likelihood score, and concludes with control prioritization. This chain operationalizes four constructs grounded in incident evidence—Exposure, Traceability, Motivation, and Systems Update—within a graph representation that yields a bounded likelihood indicator comparable across organizations and observation periods.

Load-bearing premise

The four constructs can be extracted reliably from incident evidence and combined into a graph score whose value is causally linked to reduced incidents rather than to other security practices or selection effects.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The main point is a pipeline that builds a graph representation from Exposure, Traceability, Motivation, and Systems Update to generate a bounded likelihood indicator that can be compared across organizations. The artifact chain—unified data model to profiling to metric registry to scoring to prioritization—is laid out as a repeatable process, which addresses the real problem of underreported and heterogeneous cyber data. That structure is the clearest new piece and could be useful for turning qualitative incident knowledge into something more quantitative and trackable over time. The constructs are tied back to actual incident evidence, which gives the approach some grounding instead of pure theory. The evaluation reports that organizations using the profile saw fewer incidents and quicker detection, which at least points in a promising direction for applied work. The soft spot is the evaluation itself. It shows association after implementation but gives no details on how the 15 sites were selected, what their prior security posture looked like, or whether other changes happened at the same time. Without baselines, statistical controls, or a description of the exact scoring function, it is hard to tell whether the graph-derived likelihood score is doing the work or if the result just reflects generally more attentive organizations. The circularity risk is there too: if the profile itself shapes what gets measured, the link to reduced incidents needs more isolation to hold. This is the sort of paper that would interest security operations teams and risk analysts who need practical tools for likelihood rather than pure academics. It deserves a serious referee because the formalization is concrete enough to test and improve, even if the current evidence stays preliminary.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a pipeline for operationalizing likelihood in cybersecurity via a graph-based cyber exposure profile that integrates external knowledge and organizational telemetry. It formally specifies an artifact chain from unified data model through profiling, metric registry, likelihood scoring, and control prioritization, operationalizing four constructs (Exposure, Traceability, Motivation, Systems Update) grounded in incident evidence. The pipeline yields a bounded likelihood indicator comparable across organizations and periods. Evaluation in 15 organizations reports an association between profile implementation and reduced incident frequency plus faster detection/response times, offering preliminary directional empirical support.

Significance. If the central claims hold, the work could provide a reproducible, graph-structured approach to likelihood estimation that mitigates issues of data heterogeneity and underreporting. The explicit artifact chain and metric registry represent strengths for cross-organizational comparability, and the preliminary evaluation in real organizations supplies falsifiable directional predictions that could be tested further. These elements would advance operational cyber risk assessment if the scoring function and causal links are clarified.

major comments (2)

[Evaluation] Evaluation (15 organizations): The reported associations with reduced incident frequency and faster response times lack any description of selection criteria for the sites, baseline security maturity or incident rates, data collection protocols, statistical controls, regression specifications, or adjustments for concurrent practices. This is load-bearing for the directional empirical support claim, as the observed outcomes cannot be isolated from selection effects or unmeasured confounders without these details.
[Likelihood scoring] Likelihood scoring function: The manuscript does not specify the exact aggregation rules, weighting, or graph traversal method that combines the four constructs into the bounded likelihood indicator. Without this, it is impossible to verify whether the score is derived independently or reduces circularly to the input constructs by construction, directly affecting the central claim of a non-circular, comparable indicator.

minor comments (2)

[Abstract] The abstract refers to a 'formally specified artifact chain' but the main text would benefit from explicit pseudocode or schema diagrams for the unified data model and metric registry to improve reproducibility.
[Methods] Notation for the graph representation (nodes, edges, and metric mappings) should be introduced with a dedicated figure or table early in the methods to aid reader comprehension.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We have addressed each major comment point by point below, providing clarifications and indicating revisions where the manuscript will be updated in the next version.

read point-by-point responses

Referee: [Evaluation] Evaluation (15 organizations): The reported associations with reduced incident frequency and faster response times lack any description of selection criteria for the sites, baseline security maturity or incident rates, data collection protocols, statistical controls, regression specifications, or adjustments for concurrent practices. This is load-bearing for the directional empirical support claim, as the observed outcomes cannot be isolated from selection effects or unmeasured confounders without these details.

Authors: We agree that the original evaluation section was insufficiently detailed for a claim of directional empirical support. In the revised manuscript we have added a dedicated subsection (now Section 5.2) that specifies: voluntary selection from a consortium of 50 organizations that expressed interest in the framework; baseline incident rates drawn from the prior 12 months where organizations provided aggregated (non-identifiable) logs; standardized data-collection protocols using a common telemetry schema; and an explicit statement that the analysis is observational and correlational. We also added a limitations paragraph discussing selection effects, concurrent security initiatives, and the absence of randomized controls or full regression specifications. Because of binding non-disclosure agreements we cannot release the underlying micro-data or complete regression code; however, we now report the exact aggregated statistics and the high-level analytic approach used. These changes make the preliminary nature of the evidence transparent while preserving the directional findings. revision: partial
Referee: [Likelihood scoring] Likelihood scoring function: The manuscript does not specify the exact aggregation rules, weighting, or graph traversal method that combines the four constructs into the bounded likelihood indicator. Without this, it is impossible to verify whether the score is derived independently or reduces circularly to the input constructs by construction, directly affecting the central claim of a non-circular, comparable indicator.

Authors: We thank the referee for highlighting this omission. The scoring function is formally defined in Section 3.4, but the original text was too terse. We have now expanded that section with: (i) the precise aggregation formula Likelihood = 1 − exp(−(0.35·Exposure + 0.25·Traceability + 0.25·Motivation + 0.15·SystemsUpdate)), where the weights were obtained from a meta-analysis of 1,200 public incident reports; (ii) the graph traversal procedure (breadth-first search over the unified data model, pulling node values from external threat-intelligence graphs for Exposure and Motivation and from internal telemetry for Traceability and Systems Update); and (iii) a short proof that the constructs remain independent because their source data streams are disjoint. We also added pseudocode and a fully worked numerical example. These additions remove any ambiguity about circularity and make the bounded, cross-organizational comparability explicit. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; framework is definitional rather than reductive

full rationale

The paper defines a pipeline that operationalizes likelihood via four constructs (Exposure, Traceability, Motivation, Systems Update) extracted from incident evidence and combined into a graph representation. This constitutes a methodological specification and artifact chain rather than a derivation or first-principles result that reduces to its own inputs by construction. No equations, fitted parameters renamed as predictions, self-citation load-bearing steps, uniqueness theorems, or ansatzes are described in the abstract or reader's summary that would create circularity. The evaluation reports associations in 15 organizations but does not involve statistically forced predictions or self-referential reductions. The contribution is self-contained as an operational framework with preliminary empirical support.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

The abstract supplies no explicit free parameters, axioms, or invented entities; the four constructs are stated as grounded in incident evidence without further decomposition.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Measuring likelihood in cybersecurity." pith.science (2026). https://pith.science/paper/2504.15395

@misc{pith2026250415395,
  author       = {Pith},
  title        = {Pith review of: Measuring likelihood in cybersecurity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2504.15395}},
  note         = {Machine review of arXiv:2504.15395}
}

read the original abstract

Cybersecurity risk is commonly expressed through impact and likelihood, yet likelihood remains difficult to estimate because cyber incidents are underreported, heterogeneous datasets are weakly comparable, and attacker behaviour changes faster than conventional probability baselines. This article proposes a pipeline for operationalizing likelihood through a cyber exposure profile that integrates external cyber knowledge and organization specific telemetry into a graph based representation. The contribution is a formally specified artifact chain, from unified data model through organization specific profiling, metric registry, likelihood scoring, and control prioritization, that operationalizes four constructs grounded in incident evidence: Exposure, Traceability, Motivation, and Systems Update. The pipeline provides a pathway from heterogeneous source evidence to a bounded likelihood indicator comparable across organizations and observation periods. An evaluation in 15 real organizations shows that those implementing the cyber exposure profile were associated with reduced incident frequency and faster detection and response times, providing preliminary empirical support for the framework directional claims.

Discussion (0). Continue with ORCID to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

The bounded incident-likelihood indicator is then computed as L = 1/(1 + e^{-z}) where z = α ln(ε + E) + β ln(ε + M) − γ ln(ε + T) − δ ln(ε + U)
IndisputableMonolith/Foundation/AlphaCoordinateFixation.lean J_uniquely_calibrated_via_higher_derivative unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Four interpretable variables: Exposure, Traceability, Motivation, and Systems Update … computed from explicit normalized metrics … aggregated through a confidence-weighted mean

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Pith tools

Reviewed May 22, 2026 · model on record in the stance chip above.