Pith. sign in

REVIEW 3 major objections 6 minor 23 references

A Public Network Trace of a Control and Automation System

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper releases the first publicly available network trace from a real building management system: a one-week, benign, header-only capture from the HVAC network of a university campus, aimed at flow-based intrusion detection research.

desk verdict A genuinely useful real-world BMS trace with a completeness claim that needs to be validated before the dataset is used as a ground truth for flow-based IDS research. read the letter →

arxiv 1908.02118 v1 pith:L7W3NLJW submitted 2019-08-06 cs.CR cs.NIcs.SYeess.SY

Gorby Kabasele Ndonda , Ramin Sadre This is my paper · ORCID
classification cs.CRcs.NIcs.SYeess.SY
keywords buildingmanagementsystemHVACSCADAnetworktracepacketcaptureflow-basedintrusiondetectionanonymizationtrafficcharacterization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents what it claims is the first public network trace collected from a real building management system (BMS), specifically the heating, ventilation, and air-conditioning (HVAC) network of a university campus. It argues that researchers studying network-based intrusion detection for automation systems have so far been limited to synthetic traces or small testbeds, because real installations are not normally opened for experimentation. To close that gap, the authors captured seven days of packet headers from the production system, processed them with clock-correction and anonymisation, and published them. If the dataset is genuine and representative, it gives flow-based intrusion detection researchers a reproducible way to test their solutions against realistic, benign automation traffic. The paper characterises the trace, showing long-lived connections, small packet sizes, diurnal patterns, and some deliberate irregularities such as software-update traffic.

What carries the argument

The object that carries the contribution is the published trace itself, together with the capture and post-processing procedure that makes it credible as a record of real BMS traffic. The capture uses two routers with port mirroring because the paths between the control server and gateways are asymmetric; the two devices record different views of the same packets. To merge the views into a correctly ordered stream, the clock offset between the two capture devices is estimated every second with ICMP echo exchanges, using the formula $s = t_B - t_A - RTT/2$ and assuming the network delay is symmetric in both directions. This correction reduces out-of-order packets from about 8500 to 85 per hour. The merged trace is then filtered to hosts belonging to the HVAC system, certain management protocols are removed, and the remaining packets are anonymised with a prefix-preserving cryptographic anonymisation scheme, with payloads truncated to headers and MAC OUIs zeroed before publication.

What would settle it

Check the published trace for internal timing consistency: if TCP replies appear timestamped earlier than their requests by more than the clock error, or if the per-gateway packet counts diverge from the BMS's own recorded activity for the same week, then the capture or the correction procedure is flawed and the claim of a faithful public trace fails.

Watch

Extended reading notes

Core claim

The central claim is that a trustworthy public trace of a real automation system now exists and is described in sufficient detail to be used by other researchers. The dataset consists of one week of header-only traffic from an HVAC management system on a university campus, spanning seven HMI workstations, one control server, and eight gateways that connect 58 non-IP field devices. The authors argue that the traffic exhibits the expected properties of automation networks—permanent TCP connections that exchange small control packets, gateway-specific behaviour, and diurnal and weekly activity patterns—while also containing realistic irregularities. Because the trace is intentionally benign, it is not an attack-labelled dataset, but it is offered as a foundation for flow-level characterisation, for parametrising synthetic traffic generators, and for replay experiments in which attack traffic is mixed into the background.

Load-bearing premise

The dataset faithfully represents the traffic of the real HVAC system: the two capture points recorded every relevant packet, the clock-offset correction based on symmetric network delay produces correct ordering, and the filtering removed only non-HVAC traffic.

Editorial extensions

If this is right

  • Flow-based intrusion detection researchers can evaluate false-positive rates against benign traffic from a production automation system rather than synthetic or testbed traffic.
  • The trace can be replayed and combined with synthetic attack traffic, providing reproducible attack-injection experiments without placing any real installation at risk.
  • The reported flow statistics (long-lived connections, small packet sizes, gateway-specific volumes) can be used to parametrise traffic models for synthetic trace generation.
  • The declared 'anomalies', such as software updates and changing DCE/RPC ports, give anomaly-based detectors a realistic mix of regular and irregular activity to process.
  • Other researchers can use the dataset as a reference point for comparing future BMS or SCADA traces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ICMP-based clock-correction method, which assumes symmetric delay, is reusable in any multi-vantage-point capture; the residual 85 out-of-order packets per hour sets a floor on timing accuracy that may matter for microsecond-level analyses.
  • The observation that DCE/RPC traffic uses changing ports suggests a concrete robustness test for anomaly detectors: a protocol shifting across ports could easily be misclassified as an attack, and the trace provides a natural case study.
  • Because the trace contains headers only, researchers who need payload-level protocol details (for example, function codes in industrial protocols) cannot use it directly; a payload-inclusive companion capture would be a natural extension if privacy constraints can be resolved.
  • Comparing this trace against previously reported SCADA traffic characteristics would quantify how much BMS behaviour differs across automation sectors, a comparison the paper does not fully develop.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents a public dataset of packet headers collected from the HVAC management system of a university campus over seven days. Traffic was captured by port mirroring at two routers, merged and clock-corrected, filtered to HVAC-related hosts, anonymized with CryptoPan, and published as hourly pcap files. The paper reports basic statistics (47 IPs, ~23k flows, 11.2 GB) and characterizes flow sizes/durations, application protocols (DCE/RPC, NetBIOS, proprietary port 2499, S7), diurnal/weekend patterns, packet size distributions for two gateways, and flow stability. It argues the dataset is the first public trace from a real BMS and can support flow-based IDS evaluation and synthetic trace generation.

Significance. If the data are authentic, complete, and accessible, the paper fills a real gap: most public SCADA/BMS datasets come from testbeds or synthetic environments, while this is a real production system trace of 11.2 GB. The descriptive statistics are consistent with known automation-system behavior (long-lived TCP connections, small payloads, periodic polling), and the authors are transparent that the trace is benign-only and payload-free. These features make the dataset a plausible benchmark for flow-based IDS false-positive evaluation and traffic modeling. However, the significance is conditional on two things the manuscript does not yet fully establish: that the two mirror points capture all relevant traffic, and that the filtering/anonymization steps are precisely documented. Those conditions are testable from the published pcaps, so the paper can be strengthened without changing its scope.

major comments (3)
  1. [III.A] The paper states that capturing at two routers is necessary because paths are asymmetric, but it does not provide evidence that the union of those two mirror points covers every path between the control server, HMI stations, and gateways. If a path traverses a third router or a direct switch link that is not mirrored, entire conversations or one direction of a conversation would be missing, which would change the flow durations and per-gateway statistics in Section IV and the validity of Table II. Please add a topology diagram showing the mirror points and their coverage, or report a per-connection integrity check (e.g., TCP sequence/acknowledgment continuity) that demonstrates no packets are missing from the published pcaps; otherwise, the claims should be restricted to 'traffic observed at the two mirror points.'
  2. [III.B] The filtering and anonymization step is not specified precisely enough to reproduce the dataset. The rule 'we only keep packets that either originate from or are destined to an host being part of the HVAC system' requires the exact list of IP addresses or subnets and how the boundary of the HVAC system was determined; the removal of ARP, STP, VRRP, LLMR, and SNMP should state the exact criteria (EtherType, protocol, ports) and what fraction of captured traffic was discarded. Because the dataset is the main contribution, vague filtering rules make it impossible for downstream users to know what portion of the original network activity the pcap files represent.
  3. [IV.A / Table II] The paper does not provide file-level metadata for the published dataset, such as a file manifest, checksums, and a persistent identifier, nor does it state whether the pcap files contain raw mirror captures or the filtered/merged output. As the central claim is that the dataset is publicly available and reusable, a stable archival link and integrity documentation should be provided; otherwise the reproducibility of the paper's statistics and the long-term availability of the data cannot be verified.
minor comments (6)
  1. [III.B] The abbreviation 'LLMR' should be 'LLMNR' (Link-Local Multicast Name Resolution).
  2. [Abstract / II] The text contains 'HV AC' instead of 'HVAC' in several places, apparently a LaTeX line-break artifact; this should be fixed in the final version.
  3. [IV.E] The phrase 'apart form' should be 'apart from'.
  4. [III.A] Please state explicitly that the clock-offset formula assumes equal forward and reverse delays, and comment on the residual 85 out-of-order packets per hour; if the residual is negligible for flow-level statistics, say so in the text.
  5. [Figures 2 and 5] Axis labels should be self-contained, with units and time windows clearly specified, so that a reader does not have to infer them from the body text.
  6. [II.B / IV.B] The related-work comparison would be strengthened by a small table listing existing public SCADA/BMS datasets and their provenance, which would also make the 'first public trace' claim easier to evaluate.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a self-contained measurement and dataset description; self-citations are contextual and not load-bearing.

full rationale

This paper does not contain a derivation chain in which a result is obtained from an input that already contains that result. Its contribution is a publicly released packet-header trace from a real HVAC/BMS installation, together with descriptive statistics of that trace. The clock-offset correction in Section III.A assumes symmetric network delay, but that is an explicitly stated measurement assumption, not an input that pre-encodes the paper's traffic characterizations. The flow statistics in Section IV are summaries of the collected packets, not predictions from a fitted model. The paper's self-citations (e.g., Barbosa, Sadre, and Pras in References [2], [3], and [9]) are used only for comparison and context in the discussion of SCADA traffic; they do not establish the content or novelty of the new dataset. The claim that this is the first public trace from a real BMS is a knowledge claim that is externally checkable through the published repository and existing literature; it is not forced by any equation or fitted parameter in this paper. Potential completeness limitations of the two-vantage-point capture are data-integrity concerns, not circularity. Consequently, no circular step can be identified from the text.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The dataset paper relies on the authenticity and completeness of the captured traffic, which is not independently verifiable from the paper. The methodological choices (clock synchronization, filtering, anonymization) are assumptions that could affect the dataset's validity.

assumptions (4)
  • domain assumption The collected traffic is benign and contains no attacks.
    The authors state the dataset contains only benign traffic, but there is no ground truth to verify this. This assumption underlies the dataset's intended use for false-positive testing. Section I.
  • domain assumption The network delay between the two capture devices is symmetric.
    Used in the time offset correction (Section III-A). If delays are asymmetric, the packet ordering in the merged trace is inaccurate.
  • domain assumption The filtering step correctly identifies and removes all non-HVAC and management traffic.
    The authors remove ARP, STP, VRRP, LLMR, and SNMP. Any remaining non-HVAC traffic could distort the trace characterization. Section III-B.
  • domain assumption The one-week capture period is representative of the system's normal operation.
    The dataset is meant to characterize typical behavior, but only seven days are captured. Section IV-A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Public Network Trace of a Control and Automation System." pith.science (2026). https://pith.science/paper/L7W3NLJW

@misc{pith2026190802118,
  author       = {Pith},
  title        = {Pith review of: A Public Network Trace of a Control and Automation System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L7W3NLJW}},
  note         = {Machine review of arXiv:1908.02118}
}
read the original abstract

The increasing number of attacks against automation systems such as SCADA and their network infrastructure have demonstrated that there is a need to secure those systems. Unfortunately, directly applying existing ICT security mechanisms to automation systems is hard due to constraints of the latter, such as availability requirements or limitations of the hardware. Thus, the solution privileged by researchers is the use of network-based intrusion detection systems (N-IDS). One of the issue that many researchers encounter is how to validate and evaluate their N-IDS. Having access to a real and large automation systems for experimentation is almost impossible as companies are not inclined to give access to their systems due to obvious concerns. The few public traffic datasets that could be used for off-line experiments are either synthetic or collected at small testbeds. In this paper, we will describe and characterize a public traffic dataset collected at the HVAC management system of a university campus. Although the dataset contains only packet headers, we believe that it can help researchers, in particular designers of flow-based IDS, to validate their solutions under more realistic conditions. The traces can be found on https://github.com/gkabasele/HVAC_Traces.

Figures

Figures reproduced from arXiv: 1908.02118 by the authors.

Figure 1
Figure 1. Network overview We use tcpdump to capture the packets. For each packet, tcpdump records a timestamp of when the packet was captured based on the clock of the capturing device. Since we use two devices, we synchronize their clocks with NTP so that packets appear later in the correct order when merging the individ￾ual packet traces based on the timestamps. Unfortunately, it showed in our first test runs that this clo… view at source ↗
Figure 2
Figure 2. Flow characteristics (a) Packets per hour (b) KBytes per hour (c) Packet size distribution [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Traffic observed at gateway X (a) Packets per hour (b) KBytes per hour (c) Packet size distribution [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Traffic observed at gateway Y publications. The SCADA network traffic of different utility companies is studied in [2], [3]. In [8] and [22], the authors analyze the network traffic of BACnet-based BMS. To our knowledge, none of the datasets used in those and similar p…
Figure 5
Figure 5. Figure 5: Flows seen per hour can help to improve the accuracy of simulated environments. There is some work on how to generate synthetic traces from another one [20], [21]. It relies on statistical models and machine learning techniques to generate traces that have similar prop…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 23 canonical work pages

  1. [1]

    https://www.netresec.com/ index.ashx?page=PCAP4SICS

    Capture files from 4sics geek lounge 2015. https://www.netresec.com/ index.ashx?page=PCAP4SICS. Accessed: October 13, 2018

    work page 2015

  2. [2]

    R. R. R. Barbosa, R. Sadre, and A. Pras. Difficulties in modeling scada traffic: A comparative analysis. In Passive and Active Measurement , pages 126–135, 2012

    work page 2012

  3. [3]

    R. R. R. Barbosa, R. Sadre, and A. Pras. A first look into scada network traffic. In 2012 IEEE Network Operations and Management Symposium, pages 518–521, 2012

    work page 2012

  4. [4]

    Edward J. M. Colbert and Steve Hutchinson. Intrusion Detection in Industrial Control Systems , pages 209–237. Springer International Publishing, Cham, 2016

    work page 2016

  5. [5]

    Dittrich, M

    D. Dittrich, M. Bailey, and E. Kenneally. Applying Ethical Principles to Information and Communication Technology Research: A Companion to the Menlo Report. Technical report, U.S. Department of Homeland Security, Oct 2013

    work page 2013

  6. [6]

    Cheung S. et al. Using model-based intrusion detection for scada networks. In Proceedings of the SCADA Security Scientific Symposium , Miami Beach, Florida, January 2007

    work page 2007

  7. [7]

    Claise B. et al. Specification of the ip flow information export (ipfix) protocol for the exchange of flow information. STD 77, RFC Editor, September 2013. http://www.rfc-editor.org/rfc/rfc7011.txt

    work page 2013

  8. [8]

    Krej ˇcí’ R. et al. Traffic measurement and analysis of building automation and control networks. In Dependable Networks and Services , pages 62–

    work page

Show all 23 references
  1. [9]

    Rafael R. R. Barbosa et al. Flow whitelisting in scada networks. International Journal of Critical Infrastructure Protection , 6(3):150– 158, 2013

    2013

  2. [10]

    Ring M. et al. Flow-based benchmark data sets for intrusion detection. In European Conference on Cyber Warfare and Security, pages 361–369, 2017

    2017

  3. [11]

    Sperotto A. et al. An overview of ip flow-based intrusion detection. IEEE Communications Surveys Tutorials , 12(3):343–356, 2010

    2010

  4. [12]

    Stouffer Keith A. et al. Sp 800-82. guide to industrial control systems (ics) security: Supervisory control and data acquisition (scada) systems, distributed control systems (dcs), and other control system configura- tions such as programmable logic controllers (plc). Technical...

    2011

  5. [13]

    Ammar, and Sue B

    Jinliang Fan, Jun Xu, Mostafa H. Ammar, and Sue B. Moon. Prefix- preserving ip address anonymization: measurement-based security eval- uation and a new cryptography-based scheme. Computer Networks , 46(2):253 – 272, 2004

    2004

  6. [14]

    García-Teodoro, J

    P. García-Teodoro, J. Díaz-Verdejo, G. Maciá-Fernández, and E. Vázquez. Anomaly-based network intrusion detection: Techniques, systems and challenges. Computers & Security , 28(1):18 – 28, 2009

    2009

  7. [15]

    A. M. Khandker, P. Honeyman, and T. J. Teorey. Performance of dce rpc. In Second International Workshop on Services in Distributed and Networked Environments, pages 2–10, June 1995

    1995

  8. [16]

    Tcpreplay - pcap editing and replaying utilities, 2018

    AppNeta Klassen, Fred. Tcpreplay - pcap editing and replaying utilities, 2018

    2018

  9. [17]

    Lemay and J.M

    A. Lemay and J.M. Fernandez. Providing scada network data sets for intrusion detection research. In 9th USENIX Workshop on Cybersecurity Experimentation and Test (CSET’16), 2016

    2016

  10. [18]

    Hui Lin, Adam Slagell, Catello Di Martino, Zbigniew Kalbarczyk, and Ravishankar K. Iyer. Adapting bro into scada: Building a specification- based intrusion detection system for the dnp3 protocol. In Proceedings of the Eighth Annual Cyber Security and Information Intelligence Re...

    2013

  11. [19]

    MODBUS MESSAGING ON TCP/IP IMPLEMENTATION GUIDE V1.0b , 2006

    Modbus. MODBUS MESSAGING ON TCP/IP IMPLEMENTATION GUIDE V1.0b , 2006

    2006

  12. [20]

    Flow- based network traffic generation using generative adversarial networks

    Markus Ring, Daniel Schlör, Dieter Landes, and Andreas Hotho. Flow- based network traffic generation using generative adversarial networks. CoRR, abs/1810.07795, 2018

    2018 arXiv

  13. [21]

    Varet and N

    A. Varet and N. Larrieu. How to generate realistic network traffic? In 2014 IEEE 38th Annual Computer Software and Applications Confer- ence, pages 299–304, July 2014

    2014

  14. [22]

    Zheng and A

    Z. Zheng and A. L. N. Reddy. Safeguarding building automation networks: The-driven anomaly detector based on traffic analysis. In 2017 26th International Conference on Computer Communication and Networks (ICCCN), pages 1–11, 2017

    2017

  15. [73]

    Springer Berlin Heidelberg, 2012

    2012

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.