{"id":"55b46040-4fbe-4061-9225-7bcd668e541b","arxiv_id":"2501.16487","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"NRE estimates entity-level cyber risks by propagating measured or assumed risks through a network graph learned from flow data, then uses the estimates for network-state inference.","lead":"This paper proposes Network Risk Estimation (NRE), a method that estimates cyber-risk scores for all devices in a network from connection logs and sparse risk measurements. It models how risk spreads between devices using a learned functional connectivity graph and a Kalman filter, then evaluates the approach on the public CIC-IDS-2017 intrusion detection dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No-measurement evaluation gap: Section 5.3 explicitly runs NRE with no risk measurements, so the central claim that estimates are refined by sparse measurements is never tested; the FBNSI comparison is also uncontrolled.","rationale":"I read the paper as a method proposal with a proof-of-concept, not a formal guarantee. For the central claim to hold, two things must be true: (i) the risk propagation model captures how risks move through the network, and (ii) the Kalman update improves estimates when measurements arrive. The reader's weakest assumption targets (i), pointing to the linear, correlation-based propagation model. That concern is real, but the paper explicitly frames NRE as compatible with other functional-connectivity inference methods, so (i) is presented as an adjustable design choice. By contrast, (ii) is the advertised differentiator and it is never exercised in the main experiment: Section 5.3.1 explicitly runs NRE without any measurements. The Section 5.1 illustration uses a single synthetic measurement but provides no quantitative test of whether the update helps. The comparison against FBNSI also confounds the method with entity count and with selection of the best classifier, so the headline claim of 'robust description' rests on an uncontrolled benchmark. These issues are fixable with a straightforward experiment, which is why I would keep the reader's conditional verdict rather than move to accept or reject. My concrete test directly probes whether the measurement-refinement step delivers the claimed benefit when measurements are actually provided.","tokens_in":18939,"tokens_out":5953,"duration_ms":61115,"concrete_test":"Re-run the Section 5.3 protocol with actual sparse measurements: at each graph window, supply noisy risk measurements for a random subset of entities (e.g., 10% of the 141 entities, with risk values derived from known attack labels or from synthetic ground truth), run the full Kalman update from Section 4.2.1, and compare network-state AUC/balanced accuracy and held-out entity-level error against the paper's no-measurement variant on the same test set. If the measurement-update version does not outperform the propagation-only version, the central refinement claim fails. In the same experiment, match entity counts for FBNSI (either restrict NRE to the 13 insider entities or provide FBNSI with the 141-entity feature set) to determine whether the reported NRE advantage survives the comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's abstract promises that NRE 'refines the estimates whenever risk measurements are provided,' and the Kalman update in Section 4.2.1 is the component that supposedly delivers this. Yet the main quantitative experiment in Section 5.3.1 states that risk estimation is performed 'without any measurements provided.' The resulting comparison with FBNSI, reported in Figure 12, therefore evaluates only the open-loop propagation xt+τ = F(t)xt, not the measurement-refinement loop that distinguishes NRE from a generic graph-diffusion anomaly score. Section 5.1 includes one synthetic measurement, but it is an illustrative snapshot with no quantitative accuracy metric against any entity-level ground truth. Consequently, the central claim that sparse measurements improve or refine NRE's estimates is unsupported by the experimental record. The comparison with FBNSI is additionally not controlled for entity count: NRE uses 141 entities while FBNSI is restricted to the 13 insider entities, and reported numbers are the best balanced accuracy among three classifiers selected on the validation set. This is an evidence gap rather than an internal contradiction, but it is load-bearing because the measurement-update step is exactly the advertised contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Network Risk Estimation (NRE), a probabilistic framework that estimates entity-level risks in a cyber network from connection data and sparse risk measurements. The method builds a functional connectivity graph from Pearson correlations of aggregated flow signals, assumes linear risk propagation via the update xt+τ = F(t)xt, and applies a Kalman filter to refine estimates when measurements arrive. The authors also introduce a relief factor to control risk scaling, use spectral partitioning for scalability, and demonstrate an application to safe routing. The main experimental evaluation is a binary classification task on the CIC-IDS-2017 dataset in which the network state (ATTACK vs. BENIGN) is inferred from the risk estimates, compared against a flow-based baseline (FBNSI), together with running-time measurements.","tokens_in":19219,"tokens_out":2833,"duration_ms":28289,"significance":"If validated, NRE would be a useful scalable tool for quantitative cyber-risk assessment that goes beyond direct measurement of a small set of entities. The paper is clearly written, the Kalman-filter derivation in Appendix A is standard and correct, and the implementation is open-source. The authors correctly identify a real practical problem: risk measurements are sparse in large, dynamic networks. The proposed architecture—graph inference, propagation, and Bayesian update—is plausible and the running-time analysis indicates that real-time operation is feasible on the tested scale. However, the central claim that sparse measurements refine and improve the risk estimates is not supported by the experiments as reported, and the comparison with FBNSI is not controlled. These gaps are load-bearing because the measurement-update step is the advertised contribution over a generic graph-diffusion score.","major_comments":[{"comment":"The main quantitative experiment explicitly states that risk estimation is performed \"without any measurements provided.\" This means the Kalman update in Eq. (6)—the component that the abstract and Section 4.2.1 claim refines estimates whenever risk measurements are provided—is never exercised in the evaluation. Section 5.1 provides only one synthetic measurement on entity 1 and reports no quantitative accuracy against any entity-level ground truth. The central contribution of measurement-based refinement is therefore untested. I would like to see an experiment where a varying number of entities are measured, with the update step enabled, and the resulting entity-level risk estimates are compared against a ground-truth risk score (e.g., attack labels or simulated compromise states) and against the open-loop propagation xt+τ = F(t)xt. Without such an experiment, the paper does not substantiate its main advertised capability.","section":"Section 5.3.1"},{"comment":"The comparison with FBNSI is not controlled on several axes. NRE uses 141 discovered entities, while FBNSI is restricted to the 13 entities of the insider network; NRE operates in a 141-dimensional feature space, whereas FBNSI uses one or two flow-derived features; the reported test metric is the peak balanced accuracy among three classifiers chosen on the validation set; and the test window size τ = 180 s is optimal for FBNSI but suboptimal for NRE, whose tuned value was τ = 90 s. Because these factors are confounded, the conclusion that \"NRE is descriptive of network state regardless of the modeled aspect of flows\" is not established. A fair comparison would use the same entity set, report results for all classifiers (or a fixed classifier), and evaluate NRE at both its tuned and the baseline-compatible window sizes, with sensitivity analysis.","section":"Section 5.3.1 and Figure 12"},{"comment":"The risk propagation model xt+τ = F(t)xt assumes that future risk is a weighted linear combination of current risks, with weights equal to the absolute Pearson correlation of aggregated flow signals. This is an assumption, not a derived consequence of attack behavior, and the paper acknowledges that the graph is only an association network (Section 4.1.2). The risk scores may therefore reflect correlation structure rather than true risk. This is a correctness-risk concern rather than an internal inconsistency, but it should be addressed with a concrete test: for example, a synthetic network with known directed/nonlinear propagation, or an evaluation where entity-level risk estimates are compared to ground-truth compromise states during an attack. Without such a check, the claim that the estimates provide \"a robust description of the network state in the presence of threats\" remains unsupported.","section":"Section 4.2, Eq. (3)"}],"minor_comments":[{"comment":"The narrative refers to the measurement target as \"entity 1\" but then identifies the highest-risk entity as \"entity 0\" with IP 192.168.10.3; the indexing is confusing and should be reconciled.","section":"Section 5.1"},{"comment":"In Table 1, the reference to the flow-attribute source appears as the literal LaTeX command \"citecicflowmeter\"; this should be replaced with a proper citation.","section":"Table 1"},{"comment":"The statement that the Kalman filter \"guarantees that the estimation error will converge to zero\" is imprecise for time-varying F(t) and when the model is misspecified; the authors should state the standard conditions (fixed, known parameters and correct model) or qualify the claim.","section":"Section 4.2.1"},{"comment":"The relief factor ρr is described as a design parameter and a rule of thumb ρr* = 1 − 1/λmax is given, but the paper does not state how ρr was set in the Section 5.1 experiment beyond \"slightly below the rule of thumb\"; a concrete value or a sensitivity check would aid reproducibility.","section":"Section 4.2.2"},{"comment":"The feature-engineering step for NRE is not fully specified: the text says the risk estimate vector x̂t|t ∈ R^141 is used as a sample, but the exact way the risk at a single time step is derived from a chunk of flows (e.g., which time index within the chunk) should be stated precisely.","section":"Section 5.3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper has a sound methodological core and the Kalman derivation is correct, but the evaluation does not test the measurement-update loop that distinguishes NRE from a generic graph-diffusion method. The FBNSI comparison also conflates entity-set size, feature dimension, classifier selection, and window-size choice. These are fixable with additional experiments, so major revision is appropriate rather than rejection. I would also encourage the authors to temper the claim that this is the first work to model risk propagation based on entity relationships; the related-work discussion does not fully establish novelty against the broader graph-based intrusion-detection literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick take: Bayer et al. apply functional connectivity graphs and a Kalman filter to entity-level risk estimation in cyber networks. The combination is new in this domain, the method is described cleanly, and the code is public. But the central claim—that sparse risk measurements refine the estimates—is never actually tested in the evaluation. The main proof-of-concept in Section 5.3 runs 'without any measurements provided,' so the comparison against FBNSI is really testing the open-loop diffusion features, not the update loop that distinguishes NRE from a generic anomaly score.\n\nWhat the paper does well: the framework is modular (any graph-inference method can be plugged in), the Kalman derivation in Appendix A is correct and standard, and the running-time analysis gives a credible picture of real-time feasibility. The safe-routing example is a nice illustration of how quantitative risk estimates could be used.\n\nThe soft spots are real and mostly in the evaluation. The measurement-update step is the advertised contribution, and it gets only an illustrative synthetic snapshot in Section 5.1 with no accuracy metric against ground truth. The FBNSI comparison is not controlled: NRE operates on 141 entities, FBNSI on 13, and the reported number is the best balanced accuracy among three classifiers chosen on validation. That's a generous-selection bias. Also, the risk propagation model itself is a linear diffusion on absolute Pearson correlations; it assumes risk spreads like a heat equation, which is plausible but unvalidated. The word 'risk' might just be encoding correlation-based centrality. The authors do acknowledge the inference limits and the linearity assumption, so this is an evidence gap, not an internal contradiction.\n\nIf I were refereeing, I'd ask for an experiment where a subset of entities have ground-truth risk labels, the Kalman update is applied, and the estimate accuracy is compared with and without measurements. That would directly test the core premise. The current form is a well-written proposal with a mismatched evaluation.\n\nVerdict: worth sending to peer review, because the idea is relevant and the fix is feasible. I wouldn't cite it as evidence yet, but I'd watch for the revision.","headline":"A clearly written graph-based risk propagation proposal for cyber networks, but the experiments never test the advertised measurement-refinement step.","tokens_in":19696,"tokens_out":2173,"would_cite":false,"duration_ms":20152,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Entity-level risks in a cyber network can be estimated from connection logs and sparse measurements by learning a functional connectivity graph and propagating risk through it with a Kalman filter.","keywords":["cyber network security","quantitative risk assessment","functional connectivity graph","Kalman filter","risk propagation","network state inference","safe routing","sparse measurements"],"falsifier":"Run NRE on a network with known ground-truth infection times (for example, a red-team or worm-spread experiment): if entities NRE ranks as highest-risk are not infected before its low-ranked entities, the propagation model is wrong. A cleaner check is to replace the absolute Pearson correlation weights with a directed measure on the same flows and see whether the top-risk ordering changes materially.","tokens_in":18754,"feed_emoji":"🛡️","tokens_out":9381,"duration_ms":86908,"temperature":0.7,"pith_summary":"Network Risk Estimation (NRE) is proposed as a data-driven replacement for risk measurement when only some endpoints can be measured. It tries to establish that entity risks propagate along learned relationships in the connection data, so that sparse observations can be extended into a full probabilistic risk picture. If this works, security teams can score every endpoint, route packets away from risky zones, and detect abnormal network states in real time. The paper demonstrates the scheme on a public intrusion dataset, where estimated entity risks support network-state classification even when raw measurements alone are not descriptive.","feed_headline":"A few risk sensors can score the whole cyber network","feed_subtitle":"Learned connection patterns plus a Kalman filter spread measured risk to every endpoint.","key_machinery":"The carrying object is the functional connectivity graph, a weighted matrix $F^{(t)}$ whose entry $F^{(t)}_{ij}$ is the magnitude of the Pearson correlation between entity $i$'s and entity $j$'s aggregated flow signals in a time window. This matrix is the linear system model: $x_{t+\\tau}=F^{(t)}x_t$ says each entity's risk carries over and gains a weighted sum of its neighbors' risks. The Kalman predict/update recursions convert sparse measurements $z_t=H_t x_t+v_t$ into MMSE estimates, while a \"relief factor\" rescales posteriors to keep risks bounded, and spectral partitioning splits the network into independently estimated groups for scaling.","core_discovery":"The paper's central claim is that risk behaves like a diffusing quantity on a time-varying weighted graph of entity influence. From flow records, the method builds a functional connectivity graph whose edge weights are the absolute Pearson correlations of synchronized per-entity activity signals. It then assumes the next risk vector is the current one multiplied by this graph matrix, $x_{t+\\tau}=F^{(t)}x_t$, and uses a Kalman filter to fuse this propagation model with whichever entities happen to be measured. The output is a mean risk estimate $\\hat{x}_{t|t}$ and an error covariance $P_{t|t}$ for every entity, which the paper argues provides a complete description of network state and supports quantitative management decisions such as safe routing.","pith_inferences":["A natural extension, left implicit, is that the same functional-connectivity-plus-Kalman recipe applies to other dynamic networks where a latent state diffuses over measured activity, such as congestion, contamination, or financial stress.","The paper does not study feedback: safe routing changes flow patterns, which changes the functional connectivity graph, and the estimates would need to account for that loop in deployment.","The experiments validate against network-level attack labels, not ground-truth entity compromises; a direct test of whether top-ranked entities are compromised before low-ranked ones is still needed.","The paper notes other influence measures are possible; using directed information would test how much the undirected Pearson simplification matters for identifying directions of risk flow."],"forward_implications":["Every entity in the network gets a quantitative risk score and a variance, including entities that are never directly measured.","Routing and other network management can be made security-aware by minimizing the maximum mean risk along a path, as illustrated by the simple safe-routing example.","When measurements are sparse or the measured connection parameter is weakly informative, NRE's propagated entity risks support attack/benign classification more consistently than a flow-only measurement model in the paper's experiments.","The method's running time scales as $O(n^{1.81})$ in entity count and the paper measures a 255-to-1 simulation-to-data time ratio, indicating real-time deployment is feasible with partitioned sub-networks.","The same probabilistic estimator applies whenever risk measurements arrive irregularly: no measurement just means running the predict step."],"supporting_citations":[{"why":"Supplies the public intrusion-detection dataset, insider network topology, and attack/benign labels used in all experiments.","marker":"[29]"},{"why":"Provides the discrete Kalman filter predict/update recursions that form the risk estimation core.","marker":"[30]"},{"why":"Establishes that the conditional-expectation estimator is MMSE, justifying the optimal linear estimator.","marker":"[5]"},{"why":"Gives association network inference, the basis for the functional connectivity graph construction.","marker":"[14]"},{"why":"Defines the communication-graph approach to entity relationships that NRE generalizes.","marker":"[20]"},{"why":"Provides the spectral ratio-cut partitioning used to split the network into independent risk subproblems.","marker":"[12]"},{"why":"Serves as the cloud risk-assessment framework that NRE contrasts for its reliance on expert input rather than real-time data.","marker":"[26]"},{"why":"One of the flow-based network-state inference methods used as the FBNSI comparison baseline.","marker":"[18]"},{"why":"Supplies the flow-aggregation and classification technique used in the FBNSI baseline.","marker":"[33]"}],"fun_headline_variants":["Risk measured on few nodes, estimated on all","Kalman filter turns sparse risk data into full network view","Propagate risk across cyber networks with learned connections","New method scores every endpoint from a handful of risk sensors","Risk diffuses: estimate network state from sparse measurements"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each entity's future risk is a weighted sum of its neighbors' current risks, with weights equal to the absolute correlation of their past activity signals; if attacks spread in a directed, nonlinear, or correlation-invisible way, the estimated scores could reflect statistical association rather than real compromise risk.","fun_headline_variants_meta":{"raw":{"variants":["Risk measured on few nodes, estimated on all","Kalman filter turns sparse risk data into full network view","Propagate risk across cyber networks with learned connections","New method scores every endpoint from a handful of risk sensors","Risk diffuses: estimate network state from sparse measurements"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000346,"raw_usage":{"total_tokens":1885,"prompt_tokens":920,"completion_tokens":965,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":888}},"tokens_in":536,"tokens_out":965,"duration_ms":7637,"temperature":1.0,"reasoning_tokens":888,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T12:54:38.532371+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run NRE on a network with known ground-truth infection times (for example, a red-team or worm-spread experiment): if entities NRE ranks as highest-risk are not infected before its low-ranked entities, the propagation model is wrong. A cleaner check is to replace the absolute Pearson correlation weights with a directed measure on the same flows and see whether the top-risk ordering changes materially.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the discrete Kalman filter predict/update recursions that form the risk estimation core."},{"cited_title":"Brown and Patrickyc Hwang","cited_arxiv_id":null,"evidence_quote":"Establishes that the conditional-expectation estimator is MMSE, justifying the optimal linear estimator."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives association network inference, the basis for the functional connectivity graph construction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the communication-graph approach to entity relationships that NRE generalizes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the spectral ratio-cut partitioning used to split the network into independent risk subproblems."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as the cloud risk-assessment framework that NRE contrasts for its reliance on expert input rather than real-time data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"One of the flow-based network-state inference methods used as the FBNSI comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the flow-aggregation and classification technique used in the FBNSI baseline."}],"review_version":1}