{"id":"9b4d1b64-a5d7-417a-aeb8-140cd93085a0","arxiv_id":"2502.03378","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A machine-learning system, LOV, classifies RPKI-invalid BGP routes as benign conflicts or hijacks and builds a whitelist of 52,846 benign routes from six months of live Internet data.","lead":"This paper introduces LOV, a system that uses machine learning to tell harmless routing mistakes from real hijacks in the RPKI security framework. It reports a six-month test that whitelisted over 52,000 such mistakes, so networks filtering invalid routes do not lose legitimate traffic.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benign-conflict ground truth is defined by one-month persistence, and the same labeling rule is reused in training, holdout evaluation, and quarantine, so the 52,846-route whitelist claim rests on an unvalidated assumption that persistent RPKI-invalid announcements cannot be hijacks.","rationale":"The paper presents a substantial empirical system, a clearly described feature set, and an honest discussion of limitations, so the reader's CONDITIONAL verdict is appropriate. The most load-bearing concern is precisely the ground-truth labeling rule in Section 5.2.1: benignity is inferred from one-month persistence, and that same heuristic flows into the training set, the 'new' holdout, the quarantine duration, and the final whitelist. This creates a self-referential evaluation loop: the classifier is tested against the same labeling criterion that produced its training data. No independent, operator-confirmed or otherwise externally validated ground truth is used to measure how often a persistent RPKI-invalid route is actually a hijack, nor how many short-lived legitimate conflicts are missed. The authors acknowledge the short-lived-conflict limitation but do not quantify its effect on the headline 52,846-route whitelist or the 79% estimate. The proposed concrete test would settle the concern: if a temporally disjoint, persistence-independent label set shows high precision and recall for the whitelist criteria, the central claim is supported; if not, the whitelist could contain hijacks or exclude most genuinely benign conflicts. Since the reader already conditioned the verdict on this unresolved assumption, the stress-test pass does not change the verdict.","tokens_in":26637,"tokens_out":3279,"duration_ms":36230,"concrete_test":"Build an independent temporal holdout: collect RPKI-invalid routes from a later period (e.g., 2023-2024), label a sample of 500-1000 routes through operator surveys, IRR/registry records, known hijack reports, and confirmed benign cases, without using route persistence as a label. Evaluate the trained RF classifier, the tightness threshold T>0.3, and the 14-day quarantine/behavior-monitoring criteria on this sample; report precision and recall for benign conflicts vs. hijacks, and the fraction of whitelisted routes that are confirmed hijacks. Additionally, check whether any whitelisted (origin AS, prefix) pairs overlap known hijack events that lasted more than 14 days in public incident databases.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition for the central claim is that the classifier's benign labels are correct. In Section 5.2.1, benign ground truth is defined as RPKI-invalid routes that 'remained stable and persistent throughout the period' May 30-June 30, 2022. This is not an independent ground truth: it is a duration heuristic applied to the same data used for training. The 'new' benign data in Section 5.2.3 is simply the 7,223 leftover instances from that same one-month pool, so the roughly 95% accuracy on 'new' benign instances is not evidence of generalization to independently labeled conflicts. The quarantine period in Section 7 is set to 14 days based on the observed recurrence of these same benign conflicts (Section 8.1.1: 80% of recurring conflicts have average frequency below 14 days), so the whitelist criterion inherits the duration heuristic. If a genuine hijack persists for a month, it is labeled benign by construction; if many legitimate conflicts last only hours or days, they are absent from training and the classifier learns the wrong boundary. The abstract's '79% of RPKI-invalid routes are benign' and the 52,846 whitelisted routes both depend on this classifier output. The authors honestly acknowledge in Section 5.2.1 and Appendix G.1 that short-lived benign conflicts are missing, and Section 9 states that all code, data, and models are confidential, so the whitelist cannot be independently audited. The central claim therefore remains conditional on the persistence-as-benignity assumption, which is exactly the soft spot in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LOV, a mechanism that uses a machine-learned classifier, a post-analyzer based on AS hegemony anomalies, and a quarantine/review process to whitelist RPKI-invalid BGP routes that are believed to be benign origin conflicts rather than hijacks. The authors train several classifiers on a ground-truth dataset of 2,000 benign conflicts and 2,000 hijacks, select a Random Forest model on cross-validation and on a held-out set, and then apply LOV to six months of live BGP data, reporting that about 79% of daily RPKI-invalid routes are benign and that 52,846 routes were whitelisted. The paper also categorizes root causes of benign conflicts and discusses deployment questions and limitations.","tokens_in":26915,"tokens_out":1716,"duration_ms":17634,"significance":"If the central claim is correct, LOV addresses a practical obstacle to ROV adoption by reducing collateral filtering of legitimate traffic while preserving hijack protection, and the six-month measurement is a useful data point on the prevalence and persistence of RPKI-invalid but benign routes. The paper's feature set is interpretable and largely grounded in public data sources, and the authors are explicit about several limitations, including the confidential nature of the code, data, and models and the acknowledged absence of short-lived benign conflicts from the ground truth. However, the significance of the headline numbers depends on an unvalidated labeling assumption, so the contribution is best assessed as a promising framework whose quantitative claims need stronger independent validation.","major_comments":[{"comment":"The benign ground truth is defined as RPKI-invalid routes that 'remained stable and persistent throughout the period' of May 30 to June 30, 2022, and the same one-month persistence assumption is reused in the holdout evaluation: the 7,223 'new' benign instances in Section 5.2.3 are simply the leftover instances from that same pool. The reported 94.6% accuracy on new benign instances therefore does not demonstrate generalization to independently labeled benign conflicts, and the classifier may simply be learning the persistent-route heuristic. The paper should either obtain benign labels from an independent source (e.g., operator confirmations, known ROA corrections, or IRR/registry cross-checks) or explicitly test how accuracy changes when short-lived RPKI-invalid routes are included in the evaluation.","section":"Section 5.2.1 and Section 5.2.3"},{"comment":"The quarantine whitelisting criteria inherit the same duration heuristic used to build the training labels. The 14-day quarantine period is justified by the observed recurrence of classifier-labeled benign conflicts (Section 8.1.1 reports that 80% of recurring conflicts have average frequency below 14 days), and the tightness threshold T_thr=0.3 is chosen from the CDF of classifier-labeled benign conflicts to whitelist 80% of them. Consequently, the whitelist of 52,846 routes is not independently validated against a ground truth that distinguishes benign origin errors from hijacks that persist for weeks. The paper should measure the false-positive rate of the whitelist, for example by checking whitelisted (AS, prefix) pairs against operator confirmations, known hijack databases, or subsequent ROA corrections, and should report how many whitelisted routes later became RPKI-valid or were confirmed as hijacks.","section":"Section 7 and Section 8.1.4"},{"comment":"The paper acknowledges in Section 5.2.1 and Appendix G.1 that short-lived benign conflicts are missing from the training data, and Section 9 states that 'all codes, data, and models are kept confidential and are not publicly available.' This combination is load-bearing for the central claim because the published whitelist cannot be audited and the classifier is not shown to generalize to the very class of transient conflicts that the one-month labeling rule excludes. At minimum, the authors should publish the feature values (which are derived from public data) and the full list of the 52,846 whitelisted (AS, prefix) pairs, or provide an independent audit mechanism, so that the whitelist's false-positive rate can be assessed by others.","section":"Section 5.2.1 and Section 9"},{"comment":"The measurement results are dominated by AS212483, which the authors suspect of feeding incorrect data to RouteViews collectors, and the post-analyzer is evaluated on only 8 hijacking events, for which it failed to verify 3 of 8 events due to missing hegemony data or threshold shifts (Section 6.3). The paper should quantify the sensitivity of the headline numbers (79% benign, 52,846 whitelisted) to the exclusion or inclusion of data from suspected faulty collectors, and should report the post-analyzer's false-positive rate on known benign routes, since the post-analyzer is tasked with reducing classifier errors but is only tested on incidents.","section":"Section 8.1.2 and Section 8.1.3"}],"minor_comments":[{"comment":"The abstract states that LOV 'whitelists 52,846 routes with benign origin errors,' but the whitelist is the output of the full pipeline including the quarantine stage; the phrasing could be clarified to distinguish classifier-identified benign conflicts from whitelisted ones.","section":"Abstract and Introduction"},{"comment":"The Depen feature is described as a 'new relationship' but is computed from the existing IHR local hegemony metric; the novelty of the feature definition should be stated more precisely, and the paper should report the coverage of local hegemony values in the training and measurement data, since Section 5.2.4 notes that about 35% of ground-truth routes lack this data.","section":"Section 5.1, Table 1"},{"comment":"The holdout hijack set is collected from only 8 radar incidents, and the table reports 'Acc' for the RF classifier per event; the paper should clarify whether these accuracy values are computed on deduplicated routes and whether the same incidents overlap with the BGPmon training period.","section":"Section 5.2.3, Table 3"},{"comment":"The Z-test uses 50 historical hegemony values and a significance level of 0.05, but the paper does not describe how the historical window is selected when a route is first observed or when an AS has sparse hegemony data; this should be clarified, since the post-analyzer's failure on 3 of 8 events is attributed to missing data.","section":"Section 6.2"},{"comment":"The tightness formula T is defined as a weighted sum of the seven features, but the weights are only described as the feature importance scores from Section 5.2.4; providing the exact weight values and the chosen T_thr threshold would improve reproducibility.","section":"Section 7"},{"comment":"Several references, such as [7] for RPKI-invalid announcement features and [36] for ROV deployment rates, are cited without page numbers or section numbers; adding those details would help readers verify the claims about prior work.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable systems contribution with an interesting feature set and an honest discussion of limitations, but its central quantitative claims rest on a self-referential labeling loop: persistent RPKI-invalid routes are labeled benign, the classifier is trained and evaluated on those labels, and the quarantine period is then tuned on the same class of routes. I would encourage the editor to send the paper back with a request for independent validation of the benign labels and public release of the whitelist or feature data; without that, the 79% and 52,846 claims cannot be audited. The authors' explicit admission that code, data, and models are confidential (Section 9) is a significant reproducibility concern for a measurement paper, but it is not by itself grounds for rejection if the whitelist itself is made available."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a serious referee. LOV is a well-built answer to a real problem: ROV drops legitimate routes when ROAs conflict with benign announcements, and that collateral damage is a known drag on RPKI deployment. The paper's contribution is a supervised ML classifier over seven relational features, a post-analyzer using AS hegemony, a quarantine review process, and a six-month live measurement. The root-cause analysis that goes beyond human error is genuinely useful. The empirical work is substantial and the feature set is described clearly.\n\nCredit where due: the authors explicitly flag the missing short-lived benign conflicts (Section 5.2.1, Appendix G.1) and the confidentiality of code, data, and models (Section 9). That openness about limits is a point in their favor.\n\nSoft spot: the ground truth. Benign labels come from a one-month persistence heuristic (Section 5.2.1). The same assumption leaks into the \"new data\" evaluation — the holdout is the leftover from that same pool — and into the quarantine criterion (14 days, chosen because most of these same conflicts recur within 14 days). So the headline numbers, roughly 79% of invalid routes benign and 52,846 whitelisted, rest on the assumption that persistent RPKI-invalid announcements are not hijacks. A month-long hijack, or a hijack that recurs, would be labeled benign by construction. The paper also never measures the whitelist's false-positive rate, and because the artifacts are confidential, the audit trail stops at the pipeline description.\n\nIs this fatal? Not necessarily, but it is load-bearing. The quarantine and post-analyzer reduce the risk of hijacks slipping through, and the email surveys provide some independent signal (10 confirmations, though from a low response rate). But the central claim is conditional until the persistence-as-benignity assumption is validated against independently labeled incidents or the data is released. The authors' honest acknowledgment keeps this in conditional-accept territory rather than reject.\n\nWho it's for: network security researchers and operators working on RPKI/ROV. It is a solid design-space exploration and a good basis for discussion. My recommendation: send to peer review. The contribution is substantial and the main weakness is identifiable and fixable — independent ground truth, or a carefully scoped claim that does not overstate the whitelist's reliability.","headline":"Substantial, honest RPKI whitelisting work whose central claim rests on a self-referential ground truth; worth refereeing, but the headline numbers stay conditional.","tokens_in":27539,"tokens_out":2409,"would_cite":true,"duration_ms":24046,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a machine-learning system called LOV can tell benign RPKI routing conflicts from hijacks well enough to publish a whitelist of 52,846 safe routes.","keywords":["RPKI","ROV","BGP hijacking","benign conflicts","whitelist","machine learning","route origin validation","Internet routing security"],"falsifier":"Re-run LOV's pipeline on the same six-month BGP data but replace the duration-based ground truth with operator-confirmed outcomes: any (AS, prefix) pair in the published whitelist that an independent hijack report confirms as malicious would refute the claim. A complementary test would feed LOV a set of known short-lived legitimate announcements and count how many never enter the whitelist because they fail the 14-day persistence and activity criteria.","tokens_in":26346,"feed_emoji":"🛡️","tokens_out":8645,"duration_ms":73621,"temperature":0.7,"pith_summary":"The paper introduces Learning Origin Validation (LOV), a system for telling apart two kinds of routes that RPKI's Route Origin Validation (ROV) currently treats the same: genuine hijacks and 'benign conflicts' caused by misconfiguration, multi-homing, or routing policy. LOV learns a 'tightness' score between the AS announcing a prefix and the AS authorized by the route-origin certificate (ROA), and uses that score, plus post-hoc verification and a quarantine period, to build a daily whitelist of (AS, prefix) pairs. Over six months of live BGP data, LOV whitelisted 52,846 routes, and the paper estimates that roughly 79% of RPKI-invalid routes per day are benign conflicts rather than hijacks. If true, ROV-enforcing networks can install the whitelist and stop dropping legitimate traffic, removing one of the main economic disincentives to RPKI deployment while still blocking hijacks.","feed_headline":"Machine learning whitelists 52,846 safe BGP routes in six months","feed_subtitle":"LOV learns to separate benign misconfigurations from hijacks, removing a key barrier to ROV adoption.","key_machinery":"The load-bearing machinery is a learned tightness relation between the two conflicting origins. For each RPKI-invalid route, LOV compares the origin AS in the BGP announcement with the AS authorized in the covering ROA and computes seven features capturing organizational, business, dependency, registry, and geographic connections; feature values default to 'no relation' so that an uncertain route leans toward being classified as a hijack. The Random Forest combines these into a classification, and the same features, weighted by their importance scores, form the tightness threshold that admits routes to the whitelist. This design lets LOV whitelist long-lived benign routes without weakening ROV's rejection of genuinely unauthorized origins.","core_discovery":"The central claim is that benign conflicts and hijacks are separable by the relationship between the two origins involved, not by their behavior alone. The paper defines seven features—OriginMatch, PC, MOAS, Parent, Depen, AltSources, and ASdist—that capture whether the announced origin and the ROA-authorized origin are the same organization, provider and customer, parent and child, mutually dependent, cross-validated by Internet routing registries, or geographically close. A Random Forest trained on these features achieves about 95% accuracy on new benign conflicts and near-100% accuracy on new hijacks. The post-analyzer then checks suspected hijacks for anomalous surges in global AS visibility, and the quarantine stage admits only routes with a tightness score above 0.3, or routes that survive 14 days of behavior monitoring and activity checks. The paper asserts that this pipeline whitelisted 52,846 benign conflicts in a six-month live measurement, with about 79% of daily RPKI-invalid routes attributed to benign causes.","pith_inferences":["The paper leaves untested whether its duration-based ground truth undercounts short-lived benign conflicts, which would mean the 79% share is an upper bound on the traffic LOV can actually save.","A simple policy change suggested by the paper's own data—having ROV accept routes whose origin matches the ROA even when the prefix length exceeds MaxLength—might recover most of LOV's traffic without any machine learning; this is testable by replaying the measurement data through that rule.","Because the quarantine requires routes to appear on at least two days in a week and remain active for 14 days, LOV is biased toward stable, frequently-announced prefixes; rarely-used backup prefixes with benign conflicts would continue to be filtered.","The paper's confirmed hijack cases include perpetrators in the middle of the AS path, which origin-based validation cannot see; coupling LOV's whitelist with path-validation mechanisms would close a gap that the current design inherits from ROV."],"forward_implications":["Networks that enforce ROV can install LOV's daily whitelist and stop dropping the roughly 79% of invalid routes that are benign, recovering legitimate traffic and the associated revenue.","Because the whitelist targets long-lived benign conflicts rather than short-lived hijacks, it avoids the synchronization and update problems that plague blacklist-based defenses.","The measurement identifies four non-human-error causes of benign conflicts—prefix deaggregation, AS-path dependencies, multi-origin ownership, and delayed ROA updates—giving operators concrete targets for fixing their ROA and routing configurations.","The finding that about 72% of benign conflicts are prefix-length mismatches with a matching origin raises a testable question about whether ROV should filter all invalid routes or only origin mismatches.","Broader ROV adoption becomes more attractive, since operators' main hesitation—loss of legitimate traffic—is addressed by a drop-in whitelist with negligible per-route overhead."],"supporting_citations":[{"why":"Defines the ROV validation procedure that LOV augments with a whitelist.","marker":"[24]"},{"why":"Longitudinal study of RPKI deployment and invalid route origins that motivates the feature design and the benign-conflict problem.","marker":"[7]"},{"why":"Prior work on mitigating human errors in ROA issuance that LOV extends beyond ROA errors.","marker":"[18]"},{"why":"Prior heuristic for identifying benign conflicts that LOV contrasts as slower and prone to false negatives.","marker":"[23]"},{"why":"Supplies ground-truth BGP hijack instances used to train and evaluate the classifier.","marker":"[8]"},{"why":"Supplies independent hijack events used as holdout data for evaluating generalization.","marker":"[31]"},{"why":"Provides the live BGP update stream that LOV validates and classifies.","marker":"[45]"},{"why":"The RPKI validator used to mark BGP announcements as invalid.","marker":"[44]"},{"why":"Provides AS hegemony values used in the dependency feature and in the post-analyzer's visibility anomaly test.","marker":"[26]"},{"why":"Identifies MaxLength mismatches as a class of ROA conflicts, grounding the OriginMatch feature.","marker":"[17]"}],"fun_headline_variants":["LOV whitelists 52K benign BGP conflicts in half a year","AI separates benign BGP conflicts from hijacks, whitelisting 52K","LOV machine learning saves 52,846 legitimate BGP routes","RPKI invalid isn't always bad: LOV whitelists benign conflicts","Six months of RPKI conflicts: 52,846 benign routes whitelisted"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that any route that stayed unchanged and visible for the whole month of June 2022 is benign, and any route in a hijack-monitoring feed is malicious; if a hijack can persist for a month, or a legitimate conflict can last only hours, the training boundary is wrong.","fun_headline_variants_meta":{"raw":{"variants":["LOV whitelists 52K benign BGP conflicts in half a year","AI separates benign BGP conflicts from hijacks, whitelisting 52K","LOV machine learning saves 52,846 legitimate BGP routes","RPKI invalid isn't always bad: LOV whitelists benign conflicts","Six months of RPKI conflicts: 52,846 benign routes whitelisted"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000637,"raw_usage":{"total_tokens":2937,"prompt_tokens":945,"completion_tokens":1992,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":1889}},"tokens_in":561,"tokens_out":1992,"duration_ms":13939,"temperature":1.0,"reasoning_tokens":1889,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T04:56:12.713739+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run LOV's pipeline on the same six-month BGP data but replace the duration-based ground truth with operator-confirmed outcomes: any (AS, prefix) pair in the published whitelist that an independent hijack report confirms as malicious would refute the claim. A complementary test would feed LOV a set of known short-lived legitimate announcements and count how many never enter the whitelist because they fail the 14-day persistence and activity criteria.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the ROV validation procedure that LOV augments with a whitelist."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Longitudinal study of RPKI deployment and invalid route origins that motivates the feature design and the benign-conflict problem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior work on mitigating human errors in ROA issuance that LOV extends beyond ROA errors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior heuristic for identifying benign conflicts that LOV contrasts as slower and prone to false negatives."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies ground-truth BGP hijack instances used to train and evaluate the classifier."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies independent hijack events used as holdout data for evaluating generalization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the live BGP update stream that LOV validates and classifies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The RPKI validator used to mark BGP announcements as invalid."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides AS hegemony values used in the dependency feature and in the post-analyzer's visibility anomaly test."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Identifies MaxLength mismatches as a class of ROA conflicts, grounding the OriginMatch feature."}],"review_version":1}