{"id":"c27a103d-88dd-4fbf-ab72-e5e485ad1d71","arxiv_id":"2511.22078","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ARES combines GraphSAGE embeddings with half-space trees to detect anomalous edges in streaming graphs, outperforming three state-of-the-art baselines on seven cyber-attack datasets.","lead":"A new streaming anomaly detector, ARES, pairs a graph neural network with half-space trees to score edges in temporal graphs in near-constant time. On seven cyber-attack datasets it reports higher ROC-AUC and average precision than MIDAS, AnoGraph, and SLADE-H, with a validation-based threshold rule.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) in §3.2 is inverted: it scores dense leaves as more anomalous, contradicting the stated sparse-leaf rule; if the code reproduces the formula, Table 1's near-perfect AUCs cannot be trusted.","rationale":"The reader's weakest assumption pinpoints the same issue: the internal validity of Eq. (3). This is the most load-bearing concern because Eq. (3) is the anomaly scorer itself; Eq. (4) builds the final edge score from it. An error in the direction of the score would invalidate every AUC/AP number in Table 1. The code link and ablation studies are real evidence, but they cannot adjudicate a contradiction between the printed formula and the intended semantics. My concrete check would settle it: if the code uses the inverse formula, the paper needs only a correction; if it uses the printed formula, the central claim collapses. I also flag Section 3.3's assertion that freezing the GNN encoder preserves informative embeddings on the evolving graph; it is supported only by a citation to [Coppolillo et al. 2025] and is not demonstrated, but it is secondary to the Eq. (3) inconsistency. The paper's own Section 4 and Appendix D acknowledge limitations about rare-anomaly assumptions and threshold instability, further reinforcing the need for a precise scoring formula. The reader's CONDITIONAL verdict remains appropriate; no verdict change is needed.","tokens_in":22614,"tokens_out":15936,"duration_ms":149275,"concrete_test":"Run the released ARES code (anonymous.4open.science/r/ARES-4573) with the HST scorer isolated from the GNN on a synthetic stream: 1,000 dense Gaussian points as normal and 10 distant uniform outliers. Compute the ROC-AUC of the raw HST scores. If the implementation matches Eq. (3) literally, AUC should be below 0.5; if it uses an inverse-mass formula (as the text implies), AUC should be near 1. Additionally inspect the scoring function to confirm whether it computes (1/Node.r)×2^{Node.h} or Node.r×2^{Node.h}. This single test distinguishes a harmless typo from an inverted core algorithm.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines the HST anomaly score as HST(X)=1/Z Node.r×2^{Node.h} (Eq. 3), with Node.r described as the counter of instances in the leaf and Node.h the leaf depth. The same paragraph states that data points in sparsely populated subregions are considered anomalous. These statements are incompatible: a sparse leaf has a small Node.r, so Eq. 3 assigns a lower score, while a dense leaf receives a higher score. Taken literally, the anomaly ranking is inverted and the central mechanism of ARES cannot produce the reported ROC-AUC/AP values. The final score in Eq. 4 is a weighted average of these inverted scores, so the problem propagates to every reported result. The only coherent resolutions are that the displayed formula is missing a reciprocal and the implementation uses 1/Node.r, or that Node.r is not the raw count, or that the implementation is also inverted and the empirical results are not reproducible. Section 4's admission that HST assumes rare anomalies yet works surprisingly well on DARPA (59.9% anomalies) makes the issue sharper: with an inverted score, the method would systematically flag normal edges. Appendix D's acknowledgment of thresholding instability is secondary; the scoring formula itself is the load-bearing element.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"ARES is an unsupervised streaming edge-anomaly detection framework that combines a GraphSAGE autoencoder (trained once on an initial snapshot) with Half-Space Tree (HST) scoring over node/edge embeddings. A weighted score (Eq. 4) merges source-node, destination-node, and edge HST scores; a Gini-based supervised thresholding rule (Eq. 5) is added for F1/B-Accuracy evaluation. The paper reports ROC-AUC/AP results on seven cyber-attack datasets and two trust networks, an ablation study, timing experiments, and a complexity argument that per-edge cost is constant when hyperparameters are bounded.","tokens_in":23040,"tokens_out":1935,"duration_ms":20097,"significance":"If the method is sound, the contribution is practically relevant: it offers a largely unsupervised, streaming-capable edge anomaly detector that reports strong AUC/AP across heterogeneous benchmarks, with publicly available code, multi-seed experiments, ablations, and a stated complexity analysis. The architectural combination of a lightweight GNN encoder with HST is plausible and worth investigating. However, the central scoring formula in Eq. (3) is internally inconsistent with the described sparse-leaf anomaly rule, and this issue is load-bearing for every reported result.","major_comments":[{"comment":"Eq. (3) defines HST(X) = (1/Z) Node.r × 2^{Node.h}, and the text says Node.r is the counter of instances in the leaf. The same paragraph states that data points in sparsely populated subregions are anomalous. These statements are contradictory: a sparse leaf has small Node.r, so Eq. (3) assigns it a lower score, while dense leaves receive higher scores. Taken literally, the anomaly ranking is inverted and the mechanism cannot yield the reported near-perfect AUCs; the problem propagates through Eq. (4). The authors must clarify whether the formula is missing a reciprocal, whether Node.r is not a raw count, or whether the implementation differs from the paper. This is not a presentation issue: Table 1 and all downstream claims depend on the correct sign of the scoring function.","section":"§3.2, Eq. (3)"},{"comment":"The paper assumes that a GNN trained only on the initial snapshot remains representative for the entire evolving stream. This is asserted with one citation (Coppolillo et al.) and a qualitative argument that G_t is continuously updated, but no experiment demonstrates that the frozen encoder continues to produce embeddings that separate anomalies over time. Since HST scores are computed in this fixed embedding space, drift in the embedding distribution directly affects detection. A controlled experiment (e.g., retraining periodically versus freezing, or measuring embedding drift against anomaly performance) would substantiate the claim.","section":"§3.3 and Section 4"},{"comment":"The thresholding mechanism is acknowledged in Appendix D to be unstable under distribution shift between validation and test: for ISCX2012 the validation set has 99.962% normal edges while the test set has 2.4% anomalous edges, and the paper states this can result in a poor threshold. This materially weakens the RQ2 claims and the F1/B-Accuracy comparisons in Table 2. The authors should quantify how often the Gini threshold degrades performance versus an oracle threshold, and should temper the claim that the thresholding method is 'simple yet effective' in the presence of concept drift.","section":"Appendix D and §5.1"}],"minor_comments":[{"comment":"The phrase 'score(e_t) encodes the probability P(l_e_t=1 | e_t, G_t)' is unsupported: HST scores are not calibrated probabilities, and no calibration analysis is provided. Please rephrase as an anomaly score or add calibration evidence.","section":"§3.2"},{"comment":"The notation for weights is inconsistent: the text mentions w_1, w_2, w_3 while Eq. (4) uses w_s, w_d, w_e. Also, the relation to the normalized weights is not defined precisely.","section":"§3.2 and Eq. (4)"},{"comment":"Timing results are reported in seconds but without stating hardware/software configuration in the table caption or the number of repeated runs beyond the main seed setup. Please clarify.","section":"§5.1 and Table 4"},{"comment":"Typos include 'extention' (Section 3.2), 'GraphSage' in Table 15 vs 'GraphSAGE' elsewhere, and 'the author propose' in Section 2. Please proofread.","section":"General"},{"comment":"The hyperparameter list includes 'Weights (1.0, 0.0, 0.0), (0.33, 0.33, 0.33)'; the first tuple appears to place all weight on the edge term, which is inconsistent with Eq. (4) if w_s and w_d correspond to source/destination. Please clarify the intended mapping.","section":"Appendix A, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The Eq. (3) issue is the decisive point. If the implementation actually uses a reciprocal or otherwise matches the sparse-leaf description, then a corrected formula and a note in the code would make the paper publishable after the other requested revisions. If the implementation matches the printed formula, the empirical results would need to be re-examined, because the reported AUCs could not arise from the described mechanism. I would encourage the editor to ask the authors to release or inspect the exact HST scoring code before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a legitimate new pipeline—GraphSAGE autoencoder embeddings fed into Half-Space Trees for streaming edge anomaly detection, plus a Gini-index threshold rule—and the evaluation is broad. But there's a load-bearing inconsistency in the central scoring equation that needs to be resolved before the numbers can be taken at face value.\n\nThe novelty is modest but real: no one has combined these exact pieces, and the validation-based thresholding on score dispersion is a simple, sensible trick. The paper covers seven real-world datasets, runs nine seeds, includes ablations against RRCF and other GNN encoders, reports time/space complexity, and links to code. That is solid workmanship.\n\nThe soft spot is the formula in Eq. (3). The text says a sparse leaf should be more anomalous, but the formula HST(X) = (1/Z) * Node.r * 2^Node.h assigns higher scores to leaves with larger counters. If Node.r is truly the instance count, then a dense leaf gets a high score and a sparse leaf gets a low score—exactly the opposite of what the paragraph claims. This is not a cosmetic issue: the final score is a weighted average of these values, so every reported AUC and AP depends on it. It is probably a typo—the intended score likely uses 1/Node.r or some inverse-mass measure—but the paper as written is internally inconsistent. If the implementation matches the printed formula, the results are not credible. Since the code is available, this is checkable, and the authors should fix it or clarify the notation before anyone builds on it.\n\nSecondary issues are milder. The GNN is trained only on the initial snapshot; the paper asserts this is fine by citing prior work, but doesn't demonstrate that embedding quality stays informative over long streams. The claim that score(e_t) encodes P(ℓ=1|e_t, G_t) is uncalibrated—it's a score, not a probability. And the authors themselves acknowledge in Appendix D that the threshold is unstable under distribution shift between validation and test, which is honest but limits the practical value of the supervised thresholding.\n\nBottom line: if Eq. (3) is a typo, this is a decent engineering contribution to the edge-stream anomaly detection subfield. As written, it needs a correction or a clear explanation before the main claims can be trusted. I'd still send it to a serious referee—the problem is well-placed and the experimental setup is thorough—but the referee should demand that fix and a note on why the results are insensitive to the scoring direction.\n\nFor a reading group, it's worth a session just to untangle Eq. (3) and discuss whether the empirical wins are real or an artifact of a flipped formula. I wouldn't cite it in my own work until the equation is resolved.","headline":"Plausible engineering combination with a load-bearing formula problem: Eq. (3) looks inverted, and the empirical results hinge on that scoring rule.","tokens_in":23451,"tokens_out":2246,"would_cite":false,"duration_ms":23054,"reading_group":"yes","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ARES combines a frozen graph embedding with half-space tree scoring to catch anomalous edges in live streams.","keywords":["anomaly detection","edge streams","temporal graphs","graph neural networks","half-space trees","unsupervised learning","cyber-attack detection","streaming"],"falsifier":"Feed a fixed embedding space to the HST so that one leaf receives thousands of points and a neighboring leaf receives one, then compare the anomaly scores assigned to points in each leaf: the claim predicts the sparse leaf scores higher, while a literal reading of Eq. (3) predicts the crowded leaf does. A streaming experiment that introduces a new class of nodes after training would similarly test whether frozen embeddings still separate anomalies.","tokens_in":22536,"feed_emoji":"🛡️","tokens_out":5890,"duration_ms":54193,"temperature":0.7,"pith_summary":"The paper claims that edge anomalies in streaming temporal graphs can be recognized without retraining and without labels by pairing a Graph Neural Network encoder, trained once on an initial snapshot, with Half-Space Trees that partition the encoder's latent space. The resulting model, ARES, is designed to score each incoming edge in bounded (effectively constant) time, and its two variants are evaluated on seven cyber-attack datasets. The claim is that this combination outperforms prior stream-anomaly detectors on almost all of those benchmarks while also giving a simple supervised Gini-index thresholding rule that turns raw scores into decisions. A sympathetic reader would care because cyber-defenses need per-edge decisions on high-volume streams, with neither fresh labels nor expensive retraining.","feed_headline":"Graph AI plus tree scorer flags edge anomalies in constant time","feed_subtitle":"Unsupervised ARES beats prior stream detectors on six of seven cyber-attack sets while scoring each edge in bounded time.","key_machinery":"The machinery is a two-stage scoring loop. A graph autoencoder built with GraphSAGE, trained once on the initial snapshot, projects nodes into a latent space; edge embeddings are either the average or the difference of the endpoint embeddings. Two Half-Space Trees then randomly partition that latent space and maintain per-leaf counters with a sliding-window update, and the stated anomaly score combines leaf depth and counter mass (Eq. 3), with the final score in Eq. 4 a weighted blend of source, destination, and edge tree responses. This loop is what lets rare latent regions define 'anomalous' without any retraining or labels at scoring time.","core_discovery":"The central claim is that a GraphSAGE autoencoder trained once on the starting graph produces node and edge embeddings that remain useful as the graph evolves, and that two Half-Space Trees—one over node embeddings, one over edge embeddings—can turn those embeddings into reliable anomaly scores for both spike and burst edge anomalies. The final score is a weighted combination of source-node, destination-node, and edge tree scores, and the paper reports that ARES-Static and ARES-Dynamic rank first or second in ROC-AUC and AP on six of seven datasets, with the largest gains on CTU-13 scenarios. The thresholding mechanism selects a cut by minimizing Gini impurity on a small labeled validation s","pith_inferences":["If Eq. (3) is read literally, a leaf with a larger counter receives a higher score, contradicting the stated principle that sparse leaves are anomalous; the reported rankings therefore depend on the intended or implemented definition of that formula rather than the printed one.","The claim that frozen embeddings stay informative under concept drift is plausible but rests on a citation, not on a controlled experiment; testing on streams that introduce entirely new node populations or shifted feature distributions would settle it.","The thresholding method is adaptive only within the validation window; the paper's own ISCX2012 example shows the validation and test anomaly rates can differ by orders of magnitude, so the threshold is exposed to exactly the drift the detector is meant to survive.","A natural extension, left implicit in the paper, is to make the threshold unsupervised—using the score distribution's dispersion rather than labels—which would bring the whole pipeline back to the no-labels setting the title promises."],"forward_implications":["Each edge can be scored in bounded time once tree depth, tree count, window size, and GNN hyperparameters are fixed, making the scheme viable for high-volume streaming logs.","An encoder trained only on an initial snapshot can, in principle, drive drift-adaptive anomaly scores as long as the graph itself keeps updating the context around each edge.","A single architecture covers both spike anomalies (rare individual connections) and burst anomalies (coordinated floods), where earlier methods specialized in one or the other.","The Gini-based thresholding rule gives a concrete, small-label way to separate normal from anomalous edges, improving F1 and balanced accuracy relative to tuned competitor thresholds.","A static variant that never updates tree counters remains competitive even on DARPA, where nearly sixty percent of edges are anomalous—a regime where simple sparsity-based scoring should struggle."],"fun_headline_variants":["GNN + tree scoring catches edge anomalies on the fly","Unsupervised ARES uses GNN and trees to flag edge attacks","Real-time edge anomaly detection: ARES beats prior stream methods","Combining GNN and tree scoring to catch unusual edges fast"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The results stand only if (a) an HST leaf's score is higher when its region is sparser and (b) embeddings produced by a GNN trained on the initial snapshot remain informative as the stream evolves; the first is contradicted by the formula as printed, and the second is asserted without direct evidence.","fun_headline_variants_meta":{"raw":{"variants":["GNN + tree scoring catches edge anomalies on the fly","Unsupervised ARES uses GNN and trees to flag edge attacks","Real-time edge anomaly detection: ARES beats prior stream methods","Combining GNN and tree scoring to catch unusual edges fast"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000692,"raw_usage":{"total_tokens":2978,"prompt_tokens":759,"completion_tokens":2219,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":2147}},"tokens_in":503,"tokens_out":2219,"duration_ms":13959,"temperature":1.0,"reasoning_tokens":2147,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T19:50:23.518949+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed a fixed embedding space to the HST so that one leaf receives thousands of points and a neighboring leaf receives one, then compare the anomaly scores assigned to points in each leaf: the claim predicts the sparse leaf scores higher, while a literal reading of Eq. (3) predicts the crowded leaf does. A streaming experiment that introduces a new class of nodes after training would similarly test whether frozen embeddings still separate anomalies.","supporting_citations":[],"review_version":1}