Implementation of IOAM for IPv6 in the Linux Kernel

Benoit Donnet; Frank Brockners; Justin Iurman

REVIEW 3 major objections 5 minor 14 references

Implementation of IOAM for IPv6 in the Linux Kernel

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports the first Linux kernel implementation of IPv6 In-situ OAM and measures its worst-case bandwidth cost at roughly 7 percent.

desk verdict First open-source Linux kernel IOAM for IPv6 is real and worth knowing about; the performance numbers are too fragile to carry the 'reasonable overhead' claim as stated. read the letter →

arxiv 1908.07240 v1 pith:UD2VL27E submitted 2019-08-20 cs.NI

Justin Iurman , Benoit Donnet , Frank Brockners This is my paper · ORCID

classification cs.NI

keywords In-situOAMIOAMLinuxkernelIPv6Hop-by-Hopoptionsnetworktelemetrypacketprocessingopen-sourceimplementation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents the first implementation of In-situ Operations, Administration, and Maintenance (IOAM) for IPv6 inside the Linux kernel, released as open source. IOAM is a technique for collecting telemetry along a packet's path by embedding data in the packet itself, so operators can observe network behavior without injecting separate measurement traffic. The authors argue that the implementation is practical: in a controlled five-node testbed, inserting IOAM headers and data reduces achievable bandwidth by at most about 7 percent, and the added per-node delay is only a few microseconds. This matters because a working, efficient kernel implementation lets operators run in-band telemetry on standard Linux routers and gives the standardization effort a concrete reference for how IOAM behaves on real forwarding paths.

What carries the argument

The load-bearing mechanism is the kernel module's pre-computed IOAM buffer and the ioam_parsed_eh parse record. At registration time the module builds a ready-to-insert extension-header buffer with aligned IOAM options and padding for each encapsulating interface, and stores known IOAM namespaces in a hash table keyed by namespace identifier. During packet processing the IPv6 code keeps a pointer to a fixed-size ioam_parsed_eh structure in the kernel packet control buffer, so insertion and deletion can locate the exact offset and size of the IOAM blocks without any per-packet allocation. This combination of a prebuilt buffer, hash lookup, and per-packet parse metadata is what keeps IOAM handling allocation-free and fast enough to produce the reported 7 percent worst-case bandwidth cost.

What would settle it

Run the same patched 4.12 kernel on two physical hosts with standard NICs, send 1200-byte UDP packets at line rate, and measure goodput when 100 percent of packets receive an IOAM trace option; a bandwidth reduction substantially larger than the reported 7 percent would show the virtual testbed masked the true insertion cost.

Watch

Extended reading notes

Core claim

The paper's central claim is that it provides the very first IPv6 IOAM implementation in the Linux kernel, built as a kernel module on kernel 4.12, and that this implementation is efficient enough for real use. In the evaluation, when every packet in a test domain carries IOAM data, the achievable bandwidth drops by roughly 7 percent relative to the same kernel without IOAM registration, and the added per-node processing delay is only a few microseconds. The implementation follows the IOAM IPv6 options and data-fields drafts, supports the trace and edge-to-edge option types, and is designed to avoid per-packet memory allocation by pre-building an extension-header buffer at registration time and storing parse results in the packet control buffer. The authors also show that inserting IOAM into a small fraction of traffic costs very little: a 1 percent insertion rate costs about 2 percent bandwidth.

Load-bearing premise

The entire performance story depends on the assumption that packet processing measured on virtual Linux containers linked by veth pairs and driven by pktgen predicts what real routers and physical network cards would experience.

Editorial extensions

If this is right

Operators on Linux-based routers can enable in-band telemetry for a small, predictable fraction of traffic without adding separate measurement packets; at a 1 percent insertion rate the measured bandwidth cost is about 2 percent.
The kernel code gives the standardization effort a concrete reference point, since the insertion and deletion logic embodies the alignment and padding rules from the IPv6 and IOAM option specifications.
At the egress node, removing the entire Hop-by-Hop extension header costs the same regardless of how many IOAM namespaces it contains, so egress processing stays simple as telemetry richness grows.
Combining this network-level tracing with Layer 5-7 tracing tools would create a continuous Layer 2-7 view of a request's path, which the paper identifies as a natural next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the per-packet cost appears to be dominated by whether the existing packet buffer has free room, a testable extension is to instrument the kernel to count how often IOAM insertion triggers re-allocation; the bandwidth drops in the paper predict that re-allocation, not option count, is the main cost driver.
The virtual testbed uses veth pairs that bypass physical NIC overhead and driver costs, so the absolute throughput figures likely do not transfer directly to hardware routers; the portable result is the design lesson that prebuilt buffers avoid per-packet allocation.
If this implementation were ported to a forwarding ASIC, the same pre-computed extension-header buffer technique suggests IOAM insertion could be done with fixed memory per interface and no per-packet allocation, making in-band telemetry practical at line rate.
The delay step pattern at the encapsulating node, with groups of 1-2, 3-5, and 6-7 namespaces, implies a byte-count threshold effect; a future implementation could test whether the same groups appear when inserting equivalent byte counts in a single namespace.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

You should know this paper for the implementation, not for the measurements. It delivers the first open-source IPv6 IOAM data plane in the Linux kernel, with code on GitHub and a careful description of the design choices: pre-allocated buffers, a pointer in the skb control block to avoid allocation during packet processing, and the parsing/insertion/deletion logic. That is a concrete, verifiable engineering contribution, and it genuinely fills a gap between the IETF drafts and something operators can test. The paper is also honest about its limits—it calls the evaluation preliminary and notes where kernel data (queue depth, buffer occupancy) is not available.

The soft spot is exactly where the stress-test note lands. The quantitative claims rest on a single testbed: LXC containers on one machine, veth pairs, pktgen, and one packet size (1200 bytes). Bandwidth is reported as means over 30 runs with no error bars or distributions. Delay in Fig. 12 is minimum delay, which is the most favorable statistic for a 'reasonable delay' argument, and the non-monotonic pattern at Porthos is dismissed as 'noise' without quantifying it. The 7% bandwidth drop at 100% IOAM insertion is for 1200-byte packets; since pktgen is per-packet CPU bound, the percentage cost at smaller packet sizes could be substantially larger. Fig. 10 already shows a sharp cliff once insertion exceeds the packet buffer headroom, a threshold that depends on packet size. So the conclusion that IOAM overhead is 'reasonable' is not yet supported beyond this specific configuration. That doesn't undermine the implementation claim—the code exists and does what the paper says—but it does mean the performance section should be read as a pilot study, not as authoritative numbers.

Citation pattern is fine; the relevant drafts and standards are there. Nothing circular.

Who benefits? Anyone working on in-band telemetry, IOAM standardization, or Linux kernel packet processing. For that audience the implementation description is valuable even if the evaluation is preliminary. I'd send it to a serious referee, with the expectation that the performance section needs either more experiments or a heavily qualified rewrite. The engineering core does not need to be re-done.

Recommendation: engage with it, but treat the performance claims as provisional.

Referee Report

3 major / 5 minor

Summary. The paper reports an open-source implementation of IPv6 In-situ OAM (IOAM) in the Linux kernel, using kernel 4.12 and a loadable kernel module. It describes a user-space API for node registration, the kernel structures used for efficient extension-header parsing, and the algorithms for inserting and deleting IOAM headers without per-packet allocation during the common-case data path. The implementation is evaluated on a five-container testbed on a single physical machine, with pktgen-generated UDP traffic of 1200 bytes across three nodes acting as an IOAM domain. The reported results show roughly a 7% bandwidth drop when every packet carries one IOAM trace option, and sub-microsecond additional delays at the encapsulating, transit, and decapsulating nodes. The paper's central claims are that this is the first Linux-kernel IPv6 IOAM implementation and that the overhead is 'reasonable'.

Significance. If the implementation and measurements are sound, this is a useful reference artifact: a complete, publicly available Linux-kernel IOAM implementation can inform IETF standardization and give operators a concrete baseline for in-band telemetry. The design choices described in Section 2, in particular the pre-built extension-header buffers, allocation-free insertion/deletion, and use of the sk_buff control buffer for per-packet parse state, are sensible and are the strongest part of the paper. The open-source code is a concrete strength that makes the implementation claims verifiable. However, the quantitative performance claim, which is the main advertised contribution after the implementation itself, is only weakly supported by the evaluation as currently presented, because of the limited parameter space and the absence of variance reporting.

major comments (3)

[Section 3.1, Fig. 9] The headline figure of a roughly 7% bandwidth reduction is measured with a single packet size of 1200 bytes and with pktgen as the traffic generator. Because pktgen is per-packet CPU-bound, the cost of IOAM header insertion is largely a per-packet cost, so the percentage bandwidth loss will depend strongly on packet size: at smaller packet sizes the relative loss would be expected to grow, while at larger sizes (up to the MTU) it would shrink. The conclusion in Section 1 that IOAM 'does not strongly reduce the bandwidth capabilities' is therefore not established beyond this one operating point. The paper should either add a packet-size sweep or explicitly restrict the claim to the tested configuration.
[Section 3.2, Figs. 8-12] Although each data point is said to be the mean of 30 runs, no confidence intervals, standard deviations, or statistical tests are reported anywhere in Section 3.2. This makes it impossible to judge whether the differences emphasized in the text (for example, the 7% drop versus the smaller differences in Figs. 10 and 11, or the groups of delays in Fig. 12) are real effects or run-to-run variation. The problem is most acute in Fig. 12, which reports only minimum delays and then attributes the non-monotonic behavior of the intermediate node to 'noise' without showing any measure of that noise. A minimum is the most favorable statistic for a claim of 'reasonable' delay, and the unquantified variance directly weakens that claim. Please provide full distributions, error bars, and, where relevant, a statistical comparison of the conditions.
[Section 3.1, testbed description] The testbed consists of five LXC containers on a single physical machine connected by veth pairs, with one traffic generator and one receiver. This setup isolates the kernel data path well, but it does not include a real NIC, interrupt handling, or multi-hop physical links, any of which could materially change the cost of extension-header processing relative to the packet send/receive path. Since the conclusion of 'reasonable' delay and bandwidth overhead is stated in general terms (Section 1 and Section 4), the paper should either present additional measurements on hardware or a multi-host testbed, or explicitly limit the performance claim to a kernel-only container environment.

minor comments (5)

[Section 2.2] The phrase 'the trade-off is to find a balanced between potential collisions and memory allocation' should be 'a balance'.
[Section 3.1] The justification for choosing 1200-byte packets is that some tests require inserting about 200 bytes while respecting the default 1500-byte MTU; this is reasonable, but the paper should clarify why the same MTU-based reasoning did not lead to testing at, say, 600 or 900 bytes as a sensitivity check.
[Fig. 12] The y-axis labels show '1ns' through '7ns' while the axis title reads 'Delay (µsec)'; this unit inconsistency should be corrected. If the delays are in nanoseconds, the axis and text should agree.
[Throughout] The paper mixes 'IOAM' and 'Ioam' (e.g., the abstract uses both, and Section 1 switches between them). One consistent spelling should be chosen.
[References [4], [5], [11]] Several cited IETF drafts are work-in-progress versions from 2017-2019; since the paper is being published later, please check whether updated versions or RFCs exist and cite the most current documents, especially for the IOAM data fields and IPv6 options.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports a direct Linux kernel implementation and direct measurements, with no fitted quantity renamed as a prediction.

full rationale

The paper's central claims are (1) a first Linux-kernel implementation of IPv6 IOAM and (2) a preliminary performance evaluation. Neither claim is derived by fitting a parameter or by invoking a self-referential uniqueness theorem. The implementation is grounded in IETF Internet-Drafts for IOAM data fields and IPv6 options; citing those drafts is citing the protocol specification being implemented, not importing a conclusion of the present paper. The bandwidth and delay results are measured directly on a containerized testbed with pktgen, with the paper stating the testbed parameters, packet size, run count, and aggregation: 'Each experiment lasts 60 seconds and is run 30 times. Each data point in subsequent plots represents the mean value over those 30 runs.' The reported 'drop of 7%' is read directly from the bandwidth measurements in Fig. 8, and the additional-delay values are minimum measured delays in Fig. 12, so no quantity reduces by construction to an input. The 'very first implementation' claim is a firstness claim supported by the public source repository; the paper does not attempt to prove uniqueness through a citation to the same authors, and no equation or fitted model is used to generate the performance claims. The evaluation may have external-validity limitations, such as a single packet size, a veth/LXC testbed, and minimum-delay reporting, but those are correctness and robustness concerns, not circularity. Consequently there are no self-definitional steps, no fitted inputs renamed as predictions, and no load-bearing self-citation chain.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The implementation depends on the IETF IOAM drafts for data formats and on standard IPv6 extension header rules. It also depends on the testbed assumption that containers and veth pairs represent network nodes accurately for performance measurement.

assumptions (3)

domain assumption The IOAM behavior and data fields are as defined in IETF drafts [5] and [11] (draft-ietf-ippm-ioam-data-05 and draft-ioametal-ippm-6man-ioam-ipv6-options-02).
The implementation is validated only against these drafts; if the drafts contain errors or change, the implementation may not interoperate.
domain assumption LXC containers connected by veth pairs behave like network devices for packet processing and performance measurement.
The testbed in Section 3.1 is a single host with containers; isolation and driver behavior may not match dedicated router hardware.
domain assumption pktgen running in the kernel generates traffic at the full line rate for a 1200-byte UDP packet stream.
The bandwidth results assume the traffic generator is not the bottleneck; otherwise, the measured drop could reflect pktgen limitations rather than IOAM overhead.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Implementation of IOAM for IPv6 in the Linux Kernel." pith.science (2026). https://pith.science/paper/UD2VL27E

@misc{pith2026190807240,
  author       = {Pith},
  title        = {Pith review of: Implementation of IOAM for IPv6 in the Linux Kernel},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UD2VL27E}},
  note         = {Machine review of arXiv:1908.07240}
}

read the original abstract

In-situ Operations, Administration, and Maintenance (IOAM) is currently under standardization at the IETF. It allows for collecting telemetry and operational information along a path, within the data packet, as part of an existing (possibly additional) header. This paper discusses the very first implementation of IOAM for the Linux kernel with IPv6 as encapsulation protocol. We also propose a first preliminary evaluation of our implementation under a controlled environment. Our IOAM implementation is available as open source.

Figures

Figures reproduced from arXiv: 1908.07240 by the authors.

**Figure 1.** Ioam node registration structure. 2 Linux-Kernel Implementation In this section, we carefully describe how we have implemented Ioam within the Linux kernel 4.12.1 We first cover the Ioam node registration as well as any Ioam resources allocation (Sec. 2.1 and 2.2). We next explain how packet parsing can be done efficiently for IPv6 Extension Headers (Sec. 2.3). We finally focus on how Ioam headers are inserted and d… view at source ↗

**Figure 2.** Ioam buffer representing an IPv6 Extension Header. struct ioam_parsed_eh{ struct ioam_block eh; //Extension Header offset and size struct ioam_block last_pad; //last padding offset and size (if any) u8 pad_size; //total padding size u16 decap_size; //total size of Ioam data to remove u8 free_idx; struct ioam_block decaps[IOAM_MAX_NS_NB]; }; [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Structure with additional data for an Ex [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Simplified example of IPv6 input for a Hop [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 6.** Figure 6: Ioam deletion and alignment. Just like the header insertion algorithm, there is no allocation involved here. Of course, to stay efficient, options are not re-ordered so the arrangement could not be optimal. Still, it seems that alignment is still respected after the d…

**Figure 7.** Figure 7: Topology used for measurements. We first present our testbed (Sec. 3.1) and, next (Sec. 3.2), discuss our results. 3.1 Testbed For performing the performance measurements, we setup a testbed on a single physical machine based on Intel Xeon CPU E5-2683 v4 at 2.10GHz, 16…

**Figure 9.** Figure 9: Impact of Ioam data insertion on the overall bandwidth. 1 2 3 4 5 6 7 8 9 10 11 # IOAM trace options 2.6 2.7 2.8 2.9 3.0 3.1 3.2 Bandwidth (Gbps) Baseline IOAM [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

**Figure 10.** Figure 10: Impact of the number of Ioam Trace options on the overall bandwidth. Ioam domain are involved in Ioam data injection), the bandwidth is dropped by roughly 7%. However, this extreme case is unrealistic as we do not expect any operator inserting telemetry data in every…

**Figure 12.** Figure 12: Additional delay due to Ioam operation in each node within the Ioam domain. til five Ioam namespaces, where we observe another drop (attributed to the fact that there are more and more bytes to insert). Finally, we estimate the additional delay due to Ioam processing …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages

[1]

An overview of operations, admin- istration, and maintenance (OAM) tools,

T. Mizrahi, N. Sprecher, E. Bellagamba, and Y. Weingarten, “An overview of operations, admin- istration, and maintenance (OAM) tools,” Internet Engineering Task Force, RFC 7276, June 2014

work page 2014
[2]

traceroute,

V. Jacobson et al., “traceroute,” UNIX, man page, 1989, see source code: ftp://ftp.ee.lbl.gov/ traceroute.tar.gz

work page 1989
[3]

Bidirectional forwarding de- tection (BFD),

D. Katz and D. Ward, “Bidirectional forwarding de- tection (BFD),” Internet Engineering Task Force, RFC 5880, June 2010

work page 2010
[4]

Require- ments for in-situ OAM,

F. Brockners, S. Bhandari, S. Dara, C. Pig- nataro, H. Gredler, J. Leddy, S. Youell, D. Mozes, T. Mizrahi, P. Lapukhov, and R. Chang, “Require- ments for in-situ OAM,” Internet Engineering Task Force, Internet Draft (Work in Progress) draft- brockners-inband-oam-requirements-03, March 2017

work page 2017
[5]

Data ﬁelds for in-situ OAM,

F. Brockners, S. Bhandari, C. Pignataro, H. Gredler, J. Leddy, S. Youell, T. Mizrahi, D. Mozes, P. La- pukhov, R. Chang, and D. L. J. Bernier, “Data ﬁelds for in-situ OAM,” Internet Engineering Task Force, Internet Draft (Work in Progress) draft-ietf-ippm- ioam-data-05, March 2019

work page 2019
[6]

Internet protocol,

J. Postel, “Internet protocol,” Internet Engineering Task Force, RFC 791, September 1981. 8

work page 1981
[7]

Segment routing ar- chitecture,

C. Filsﬁls, S. Previdi, L. Ginsberg, B. Decraene, S. Litkowski, and R. Shakir, “Segment routing ar- chitecture,” Internet Engineering Task Force, RFC 8402, July 2018

work page 2018
[8]

IPv6 hop-by-hop options extension header,

F. Baker and R. Bonica, “IPv6 hop-by-hop options extension header,” Internet Engineering Task Force, Internet Draft (Work in Progress) draft-ietf-6man- hbh-header-handling-03, March 2016

work page 2016

Show all 14 references

[9]

Vendor-neutral APISs and instru- mentation for distributed tracing,

OpenTracing, “Vendor-neutral APISs and instru- mentation for distributed tracing,” 2019, https:// opentracing.io

2019
[10]

Easily collect telemetry like metrics and distributed traces from your services,

OpenCensus, “Easily collect telemetry like metrics and distributed traces from your services,” 2019, https://opencensus.io

2019
[11]

In- situ OAM ipv6 options,

S. Bhandari, F. Brockners, C. Pignataro, H. Gredler, J. Leddy, S. Youell, T. Mizrahi, K. A., G. B., P. Lapukhov, S. M., K. S., and A. R., “In- situ OAM ipv6 options,” Internet Draft (Work in Progress) draft-ioametal-ippm-6man-ioam-ipv6- options-02, March 2019

2019
[12]

Internet protocol, version 6 (ipv6) speciﬁcation,

S. Deering and R. Hinden, “Internet protocol, version 6 (ipv6) speciﬁcation,” Internet Engineering Task Force, RFC 8200, July 2017

2017
[13]

Linux containers,

Linux, “Linux containers,” 2019, see https:// linuxcontainers.org

2019
[14]

Pktgen, the Linux packet generator,

R. Olsson, “Pktgen, the Linux packet generator,” in Proc. Linux Symposium (OLS) , July 2005. 9

2005

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

An overview of operations, admin- istration, and maintenance (OAM) tools,

T. Mizrahi, N. Sprecher, E. Bellagamba, and Y. Weingarten, “An overview of operations, admin- istration, and maintenance (OAM) tools,” Internet Engineering Task Force, RFC 7276, June 2014

work page 2014

[2] [2]

traceroute,

V. Jacobson et al., “traceroute,” UNIX, man page, 1989, see source code: ftp://ftp.ee.lbl.gov/ traceroute.tar.gz

work page 1989

[3] [3]

Bidirectional forwarding de- tection (BFD),

D. Katz and D. Ward, “Bidirectional forwarding de- tection (BFD),” Internet Engineering Task Force, RFC 5880, June 2010

work page 2010

[4] [4]

Require- ments for in-situ OAM,

F. Brockners, S. Bhandari, S. Dara, C. Pig- nataro, H. Gredler, J. Leddy, S. Youell, D. Mozes, T. Mizrahi, P. Lapukhov, and R. Chang, “Require- ments for in-situ OAM,” Internet Engineering Task Force, Internet Draft (Work in Progress) draft- brockners-inband-oam-requirements-03, March 2017

work page 2017

[5] [5]

Data ﬁelds for in-situ OAM,

F. Brockners, S. Bhandari, C. Pignataro, H. Gredler, J. Leddy, S. Youell, T. Mizrahi, D. Mozes, P. La- pukhov, R. Chang, and D. L. J. Bernier, “Data ﬁelds for in-situ OAM,” Internet Engineering Task Force, Internet Draft (Work in Progress) draft-ietf-ippm- ioam-data-05, March 2019

work page 2019

[6] [6]

Internet protocol,

J. Postel, “Internet protocol,” Internet Engineering Task Force, RFC 791, September 1981. 8

work page 1981

[7] [7]

Segment routing ar- chitecture,

C. Filsﬁls, S. Previdi, L. Ginsberg, B. Decraene, S. Litkowski, and R. Shakir, “Segment routing ar- chitecture,” Internet Engineering Task Force, RFC 8402, July 2018

work page 2018

[8] [8]

IPv6 hop-by-hop options extension header,

F. Baker and R. Bonica, “IPv6 hop-by-hop options extension header,” Internet Engineering Task Force, Internet Draft (Work in Progress) draft-ietf-6man- hbh-header-handling-03, March 2016

work page 2016