{"id":"86dc8cf8-7adc-4a10-ad8b-927929ba2e33","arxiv_id":"2411.17987","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"P4-NIDS combines a NetFlow v9 exporter and a decision-tree intrusion detector in P4, showing near line-rate throughput on a 40 Gbps Netronome SmartNIC.","lead":"This paper implements a network monitoring and intrusion detection system directly in P4 on a programmable SmartNIC, claiming negligible throughput impact up to about 8 million packets per second. It matters because in-band security monitoring could avoid the latency and cost of sending traffic to out-of-band analyzers in high-speed data centers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'high-accuracy detection' claim is unsupported: Table 1 F1 scores are inherited from the offline model in [13], not measured for the P4 data-plane implementation on either BMv2 or Netronome hardware.","rationale":"The reader's weakest_assumption identifies exactly this issue, and I agree. The paper's central contribution is an in-band P4 NIDS with both throughput and accuracy advantages. The throughput evaluation has real value: Figure 7 and Table 3 show measurements on a Netronome SmartNIC with Pktgen-DPDK, and Table 2 documents memory utilization. However, the throughput result is also somewhat limited by the packet generator's maximum of roughly 7.7-7.8 Mpps at 64-byte packets (Table 3), so the '8 mpps' headline is near the measurement ceiling, and no Basic Port Forwarding throughput line appears in Figure 7. A direct baseline comparison would strengthen the 'negligible impact' phrasing. But the more fundamental gap is accuracy: no classification result from the P4 data plane is presented. Since the P4 code is not released in full, reproducibility is limited, and the only accuracy evidence is the offline model's F1. A deterministic tree can in principle preserve accuracy, but feature extraction in P4 is exactly where subtle mismatches arise (stateful counters, flow timeouts, packet-size binning, bit widths). Therefore the high-accuracy claim is conditionally acceptable at best. The recommended action is to keep the CONDITIONAL verdict and require the accuracy-measurement check before the claim can be accepted.","tokens_in":11922,"tokens_out":6163,"duration_ms":54888,"concrete_test":"Run a recorded trace from CIC-IDS-2018 and UNSW-NB15 through the P4 program on both BMv2 and the Netronome SmartNIC, reading the 'malicious' flag register per flow and comparing predictions to ground-truth labels. Also dump the P4 NetFlow register values for each flow and compare them pairwise to the offline feature extractor's output for the same packets. If the per-flow F1 matches Table 1 within a small margin and all tree-relevant features match, the accuracy transfer is confirmed; otherwise the accuracy claim must be re-scoped to the offline model only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract promises 'high-accuracy detection' alongside negligible throughput impact, and Section 5.1's Table 1 reports F1 = 99.76 (UNSW-NB15) and 98.09 (CIC-IDS-2018) under the row 'Our solution (BMv2)'. However, the paper does not state that these numbers were obtained from the P4 implementation; the surrounding text says the DT classifier was 'proposed and evaluated in [13]' and that the P4 work 'used a pre-trained DT model, as proposed and evaluated in [13]'. Thus Table 1 appears to restate the offline model's scores. The load-bearing assumption is that a deterministic P4 re-implementation of the tree, driven by registers filled by the P4 NetFlow generator, reproduces the offline model's accuracy. This requires the P4-computed features to be semantically identical to the dataset features used in training (e.g., TCP_WIN_MAX_OUT, NUM_PKTS_1024_TO_1514_BYTES, MIN_TTL, packet/byte counters over the same flow timeouts). Listing 1.1 shows thresholds like 'TCP_WIN_MAX_OUT <= 26865', but no measurement is reported that validates these register values against the offline feature extractor, nor is any classification accuracy reported on the Netronome SmartNIC. If a counter, byte-order, or feature-definition mismatch exists, F1 could degrade substantially while the paper's headline accuracy remains the offline model's value. This is the weakest pillar of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes P4-NIDS, a P4 implementation of a NetFlow v9 exporter and a decision-tree-based intrusion detection system deployed on both BMv2 and a Netronome CX 2x40GbE SmartNIC. The monitoring component extracts extended NetFlow fields at line rate, and the NIDS component implements a pre-trained decision tree from [13] using P4 registers. The evaluation reports low memory overhead, CPU utilization of 0.3% in emulation, SmartNIC throughput that tracks the offered load up to ~25.9 Gbps / ~7.8 Mpps, and F1 scores of 99.76 (UNSW-NB15) and 98.09 (CIC-IDS-2018). The paper claims these results show high-accuracy detection with negligible throughput impact, surpassing state-of-the-art in-band monitoring and IDS solutions.","tokens_in":12188,"tokens_out":11423,"duration_ms":84882,"significance":"The paper makes a useful contribution by demonstrating that a flow-based decision-tree NIDS can be implemented in P4 and run on a commercial SmartNIC with modest resource overhead, and by providing a comparison with FlowStalker in emulation. The hardware testbed and throughput measurements are a strength that distinguishes the work from many prior P4 proposals that stop at BMv2. However, the headline accuracy claim is inherited from an offline model [13] rather than measured on the P4 implementation, and the throughput evaluation lacks a no-NIDS hardware baseline and is limited by the traffic generator's 25.9 Gbps cap. The significance of the paper would increase substantially if the authors added an accuracy validation of the P4 data plane or explicitly scoped the claim to inherited offline-model accuracy.","major_comments":[{"comment":"The F1 scores reported in Table 1 under 'Our solution (BMv2)' are not measurements of the P4 implementation. Section 4.2 states that the DT classifier was 'proposed and evaluated in [13]' and that the P4 program deploys this 'pre-trained DT model'; no experiment is described in which the BMv2 or Netronome implementation actually classifies traffic from UNSW-NB15 or CIC-IDS-2018 and computes an F1 score. The abstract's claim of 'high-accuracy detection' therefore rests on the unverified assumption that the P4 register-based re-implementation preserves the offline model's accuracy. The row label is misleading and should be changed (e.g., to 'Model [13] (offline)') or replaced with data-plane accuracy measurements.","section":"5.1, Table 1"},{"comment":"The paper does not validate that the NetFlow features computed by the P4 program are semantically identical to the features used to train the offline model in [13]. The P4 code reads register values such as TCP_WIN_MAX_OUT, MIN_TTL, NUM_PKTS_1024_TO_1514_BYTES, and packet/byte counters keyed by a hash of the 5-tuple, but no experiment compares these register values with the offline feature extractor's definitions (e.g., flow timeout, directionality, counter widths, TCP window sampling). A feature mismatch would degrade classification accuracy without affecting the throughput measurements. A concrete validation—for instance, exporting P4-computed features for a replayed dataset trace and comparing them with the dataset's feature values—should be added or the accuracy claim should be explicitly limited to the offline model.","section":"4.2, Listing 1.1"},{"comment":"The claim of 'negligible impact on throughput' is not directly measured against a hardware baseline. Figure 7 shows SmartNIC throughput for five NetFlow-field configurations and the DT-enabled configuration, but does not include a 'Basic Port Forwarding' (no monitoring/IDS) curve, even though Section 5.2 states that the same three scenarios as in emulation were evaluated. In addition, Pktgen-DPDK caps the offered load at about 25.9 Gbps (Table 3), so the SmartNIC is never tested at its 40 Gbps line rate, and the maximum measured packet rate is about 7.8 Mpps rather than the '8 million packets per second' stated in the abstract. The conclusion of negligible impact is therefore inferred from the throughput tracking the offered load, not from a baseline comparison, and the stated traffic-rate ceiling is not reached.","section":"5.2, Figure 7 and Table 3"}],"minor_comments":[{"comment":"The text says 'the system supports 12 NetFlow fields', while Section 5.2 and Figure 7 describe 'DT-based NIDS using 22 NetFlow features'. Please clarify the field count and how the extra features were added.","section":"4.1"},{"comment":"The sentence 'the minimal memory increase, which remains within the kilobyte range' is contradicted by the table, where External Memory grows from 965.37 to 969.89 MB (about 4.5 MB). Please correct the wording or report the actual deltas.","section":"5.2, Table 2"},{"comment":"The table column 'Classifier Memory' does not match the component list in the text, which defines 'CLS (CLS Memory)'. Please align the terminology.","section":"5.2, Table 2"},{"comment":"The code excerpt has unbalanced braces: the inner if-else chain starting at line 14 is not closed before the outer '} else {' at line 19. Please provide a complete, compilable snippet or note where clauses are omitted.","section":"Listing 1.1"},{"comment":"The y-axis label 'Drop of Throughput %' uses decimal tick values (0.000, 0.025, ..., 0.125), which are fractions, not percentages. Please fix the axis or the text, and reconcile with the statement that the solution shows '25% to 50% less throughput drop'.","section":"5.1, Figure 5"},{"comment":"The claim that CPU utilisation 'consistently remained at 0.3%' across all three scenarios lacks information about the measurement method, the traffic load, and whether this is a meaningful comparison for BMv2. Please add the measurement details.","section":"5.1"},{"comment":"The abstract states 'traffic rates up to 8 million packets per second', but Table 3's maximum is approximately 7.83 Mpps. Please adjust the wording to match the measured range or present data that reaches 8 Mpps.","section":"Abstract"},{"comment":"The column header 'CIC-2018/2017' combines two datasets; since the proposed method is evaluated on CIC-IDS-2018 and Leo on CIC-IDS-2017, please split the columns or state the mismatch explicitly in the table caption.","section":"Table 1"},{"comment":"Reference [13] lacks publication venue and year format details; please provide the full bibliographic information for the 'Light Bulb' paper.","section":"Reference [13]"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a plausible engineering effort, but the most prominent claim—high-accuracy detection—is not actually measured for the P4 data plane, and the accuracy numbers come from a previous paper by three of the same authors (self-citation). The title and abstract overstate what the evaluation demonstrates. I would recommend asking the authors to either provide real data-plane accuracy measurements or substantially re-scope the claims, and to add a hardware baseline for the throughput comparison. The paper's scope (cs.NI) is appropriate; the contribution is a feasible P4 implementation with a real SmartNIC deployment, which is worth publishing if the claims are corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, worth a look if you care about in-band monitoring on P4 hardware. The genuinely new bit is the NetFlow v9 exporter with an extended set of fields (up to 22) implemented in P4 and evaluated on a Netronome SmartNIC. The hardware evaluation is real: they measure memory usage and show throughput tracks offered load up to the generator's ~25.9 Gbps cap, for 9–22 fields and with the decision tree active. Their exporter also looks about 4x faster than FlowStalker in BMv2. That is a useful engineering data point.\n\nThe soft spot is the accuracy story. Table 1 lists F1 scores of 99.76 and 98.09 under the label \"Our solution (BMv2)\", but those numbers are inherited from the offline decision-tree model in [13] (three of the same authors). They are not measured on the P4 implementation, either in BMv2 or on the Netronome. The paper does say earlier that they used a pre-trained model from [13], but the table and abstract present those offline scores as this paper's detection performance. That is a load-bearing mismatch: the \"high-accuracy detection\" half of the central claim is unverified for the actual data plane program. Feature extraction in P4 (counters, timeouts, byte order) could easily diverge from the offline features, and there is no measurement showing it doesn't. The fix is straightforward: either run the P4 classifier against the datasets (BMv2 can do that slowly), or clearly relabel the numbers as offline reference scores and report only the hardware throughput as this paper's measured contribution.\n\nMinor issues: the hardware throughput has no basic-forwarding baseline in Figure 7, so the \"negligible impact\" claim is inferred from matching offered load rather than directly compared. The 8 mpps claim is only for 64-byte packets at 5.3 Gbps; at larger packet sizes the generator caps around 3.2 mpps / 25.9 Gbps. The P4 listing in Listing 1.1 has some obvious brace errors, but that is cosmetic.\n\nOverall, the engineering contribution is real and the hardware results are novel. The accuracy claim needs verification or qualification, and the code and data should be released. I would send this to a serious referee, but with the expectation of major revision. The NetFlow exporter part is solid enough to cite; the NIDS accuracy part should not be cited as measured.","headline":"Real hardware NetFlow exporter work with a load-bearing accuracy claim that is inherited from an offline model, not measured on the P4 implementation.","tokens_in":12763,"tokens_out":3154,"would_cite":true,"duration_ms":27811,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a single P4 data-plane program can export extended NetFlow v9 records and run a decision-tree intrusion detector at wire speed, with negligible throughput impact up to 8 million packets per second.","keywords":["P4","Network monitoring","NetFlow v9","Intrusion detection","Decision tree","Programmable data plane","SmartNIC","Throughput evaluation"],"falsifier":"Replay a labeled UNSW-NB15 or CIC-IDS-2018 traffic trace through the Netronome SmartNIC running the P4-NIDS program, read each flow's 'malicious flag' register, and compare against ground truth; if the F1 score is substantially below the reported 99.76 or 98.09, the high-accuracy claim would be refuted.","tokens_in":11694,"feed_emoji":"🛡️","tokens_out":4704,"duration_ms":38829,"temperature":0.7,"pith_summary":"This paper tries to show that network monitoring and intrusion detection can run entirely inside a P4-programmable data plane, without offloading to a controller or external middlebox. It claims that a P4 NetFlow exporter can generate extended NetFlow v9 records at wire speed, and that the same pipeline can host a decision-tree intrusion detector that classifies flows while forwarding traffic. On a 40 Gbps Netronome SmartNIC, the paper reports negligible throughput impact up to about 8 million packets per second, with memory overhead in the kilobyte range. The motivation is that traditional out-of-band monitoring and detection cannot keep up with Terabit-scale cloud networks.","feed_headline":"In-band P4 IDS keeps wire speed up to 8 million packets/sec","feed_subtitle":"NetFlow export plus decision-tree detection run in the switch with negligible throughput loss on 40 Gbps hardware.","key_machinery":"The load-bearing mechanism is a register-based NetFlow table: a hash of the 5-tuple selects a flow entry, and each NetFlow field lives in its own P4 register array, so the decision tree can read all features directly in the data plane while the control plane periodically exports them. The decision tree itself is encoded as a chain of if-else statements, which is how P4's limited control flow is made to execute a pre-trained classifier. The hash-indexed register array is what lets the system keep cross-packet state without controller involvement.","core_discovery":"The central claim is that a single P4 program can combine a NetFlow v9 generator and a machine-learning-based intrusion detection system in the data plane, preserving forwarding performance. The NetFlow component hashes each packet's 5-tuple into a Flow ID that indexes a set of register arrays, one per NetFlow field, supporting 12 fields and extended to 22 in the decision-tree scenario. The IDS component is a pre-trained decision tree, taken from an earlier work, unfolded into nested if-else statements that read the register fields and write a 'malicious flag'. The paper reports F1 scores of 99.76 on UNSW-NB15 and 98.09 on CIC-IDS-2018 for this classifier, and shows hardware throughput that closely matches offered load across the tested range regardless of the number of NetFlow fields or the active decision-tree classifier.","pith_inferences":["I would read the F1 scores as a property of the offline decision-tree model, not of the P4 execution, because the paper does not report measuring the hardware pipeline's classifications against labeled traffic; a direct on-hardware accuracy test would settle whether the P4 version inherits that accuracy.","The same register-array design could likely host other simple classifiers such as small random forests or threshold-based anomaly rules, provided their feature sets fit in P4 registers and their control flow can be encoded as nested if-else statements.","Hash collisions in the flow table could merge distinct flows and distort both NetFlow statistics and classification; an evaluation varying the hash width and table size would quantify this risk.","The hardware test's offered-load ceiling is set by the generator NIC, so the 8 mpps result uses 64-byte packets at about 5.3 Gbps and 1024-byte packets at about 25.8 Gbps; a true 40 Gbps full-line-rate test would need a stronger traffic generator."],"forward_implications":["If the claims hold, operators can deploy monitoring and basic intrusion detection on the switch itself, removing the collector and controller from the latency path.","NetFlow export with 9 to 22 fields can run at wire speed with only kilobyte-scale memory increases, making Terabit-scale monitoring practical on programmable NICs.","The decision-tree classifier adds no measurable throughput drop in the hardware experiments, so security classification can be 'always on' rather than sampled or offloaded.","The roughly 4x throughput advantage over the evaluated flow-monitoring baseline suggests in-band exporters can be substantially cheaper than earlier P4 designs."],"supporting_citations":[{"why":"Supplies the pre-trained decision-tree classifier whose offline F1 scores are reported as the solution's accuracy.","marker":"[13]"},{"why":"FlowStalker, the baseline for NetFlow extraction throughput and throughput-drop comparisons.","marker":"[19]"},{"why":"Defines P4, the language the entire solution is written in.","marker":"[7]"},{"why":"NetFlow survey used to motivate and specify the extended v9 feature set.","marker":"[12]"},{"why":"Defines the 12 NetFlow features the monitoring component generates.","marker":"[26]"},{"why":"BMv2, the software switch used in emulated evaluation and the platform named in the accuracy table.","marker":"[20]"},{"why":"CIC-IDS-2018, one of the two benchmark datasets used for the F1 comparison.","marker":"[32]"},{"why":"UNSW-NB15, the other benchmark dataset used for the F1 comparison.","marker":"[33]"},{"why":"Pktgen-DPDK, the traffic generator used to produce offered loads in the hardware throughput tests.","marker":"[34]"}],"fun_headline_variants":["P4 in-band IDS keeps 8Mpps, adds NetFlow v9","8Mpps P4 IDS: NetFlow + ML detection at line rate","Single P4 program does NetFlow and ML IDS at 8Mpps","Wire-speed P4 IDS: 8Mpps with no throughput loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's high-accuracy claim rests on the assumption that the P4 register-based realization classifies traffic exactly as accurately as the offline decision-tree model, since the F1 scores come from the offline model and the hardware test measures throughput only.","fun_headline_variants_meta":{"raw":{"variants":["P4 in-band IDS keeps 8Mpps, adds NetFlow v9","8Mpps P4 IDS: NetFlow + ML detection at line rate","Single P4 program does NetFlow and ML IDS at 8Mpps","Wire-speed P4 IDS: 8Mpps with no throughput loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00047,"raw_usage":{"total_tokens":2323,"prompt_tokens":916,"completion_tokens":1407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":1320}},"tokens_in":532,"tokens_out":1407,"duration_ms":10563,"temperature":1.0,"reasoning_tokens":1320,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:37:16.839427+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replay a labeled UNSW-NB15 or CIC-IDS-2018 traffic trace through the Netronome SmartNIC running the P4-NIDS program, read each flow's 'malicious flag' register, and compare against ground truth; if the F1 score is substantially below the reported 99.76 or 98.09, the high-accuracy claim would be refuted.","supporting_citations":[{"cited_title":"Ghorbani","cited_arxiv_id":null,"evidence_quote":"CIC-IDS-2018, one of the two benchmark datasets used for the F1 comparison."},{"cited_title":"UNSW-NB15: A Comprehensive Data set for Net- work Intrusion Detection systems (UNSW-NB15 Network Data Set)","cited_arxiv_id":null,"evidence_quote":"UNSW-NB15, the other benchmark dataset used for the F1 comparison."},{"cited_title":"Pktgen-DPDK: DPDK Based Packet Generator, 2024","cited_arxiv_id":null,"evidence_quote":"Pktgen-DPDK, the traffic generator used to produce offered loads in the hardware throughput tests."},{"cited_title":"Network Intrusion Detection System in a Light Bulb","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained decision-tree classifier whose offline F1 scores are reported as the solution's accuracy."},{"cited_title":"Schaeffer-Filho","cited_arxiv_id":null,"evidence_quote":"FlowStalker, the baseline for NetFlow extraction throughput and throughput-drop comparisons."},{"cited_title":"P4: Programming Protocol-independent Packet Processors","cited_arxiv_id":null,"evidence_quote":"Defines P4, the language the entire solution is written in."},{"cited_title":"Evaluating standard feature sets towards increased generalisability and explainability of ml-based net- work intrusion detection","cited_arxiv_id":null,"evidence_quote":"Defines the 12 NetFlow features the monitoring component generates."},{"cited_title":"Implementing Your Switch Target with BMv2, 1 2017","cited_arxiv_id":null,"evidence_quote":"BMv2, the software switch used in emulated evaluation and the platform named in the accuracy table."}],"review_version":1}