Pith. sign in

REVIEW 5 major objections 4 minor 17 references

eHashPipe: Lightweight Top-K and Per-PID Resource Monitoring with eBPF

T0 review · 5 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read eHashPipe moves a network sketch into the kernel via eBPF and uses it to name the top CPU and memory consumers faster and more precisely than top.

desk verdict A reasonable engineering idea — HashPipe in eBPF for per-PID CPU/memory top-k — undermined by a contradictory accuracy claim and a ground-truth built on the very tool it claims to beat. read the letter →

arxiv 2509.09879 v1 pith:REHPXSFP submitted 2025-09-11 cs.PF

Yuanjun Dai (1) , Qingzhe Guo (1) , Xiangren Wang (2) ((1) Case Western Reserve University , Cleveland , USA , (2) University of Florida , Gainesville , USA) This is my paper · ORCID
classification cs.PF
keywords eBPFtop-kmonitoringHashPipesketchCPUutilizationtrackingmemoryprofilingkernelobservabilityper-processreal-timetelemetry
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

eHashPipe claims that a sketch algorithm for network heavy hitters can move into the Linux kernel via eBPF and track which processes consume the most CPU and memory in real time. The paper implements two in-kernel pipelines, one fed by allocation/deallocation probes and one by the scheduler's context-switch tracepoint, and reports top-k precision of 100% at k = 1, 5, and 10 and 83–95% at k = 20–30 when compared against top-derived ground truth. The system also tracks user-specified PIDs exactly in separate BPF maps, avoiding sketch evictions for critical processes. Its claimed advantage is responsiveness: updates every ~0.01 s, roughly 14 times finer than top's effective sampling resolution, at a memory cost of about 1.4 MB total and moderate CPU overhead. This matters because existing eBPF monitors summarize single resources but do not prioritize across processes, and userspace tools smooth away bursts that matter for diagnosing training or cloud workloads.

What carries the argument

The carrying object is the HashPipe sketch: d stages, each with a hash table of fixed-size slots holding (PID, usage count); a memory event starts at stage 0, where the slot is forced to accept the new entry and the displaced entry moves to stage 1, which swaps only if the newcomer's count exceeds the resident's. The same cascade runs for CPU deltas, which are captured by timestamping sched_switch events per thread and attributing each on-CPU interval to the owning PID. eBPF probes on 12 allocation functions plus free/munmap feed the memory pipeline; the always-kick-out rule guarantees every process enters the structure, while later-stage comparison keeps the heaviest PIDs in the chain. Dedi

What would settle it

Run a workload with a known mix of short-lived and long-lived processes, and compare eHashPipe's top-k list against deltas of /proc/<pid>/stat utime+stime (or cgroup v2 cpu.stat) sampled at 1 ms; if the overlap falls materially below the paper's reported 83–100% for k ≤ 30, or if eHashPipe misses bursts that the /proc deltas show, the central claim is refuted. Equivalently, construct a synthetic allocation workload whose true top-k is known from the allocation sizes, and check the sketch's output directly.

Watch

Extended reading notes

Core claim

The paper advances the claim that HashPipe, a multi-stage sketch that keeps a small table of counters per stage and evicts the smallest entry on collision, works for process-level resource streams as well as network flow streams. Each memory event or scheduling delta is hashed into a pipeline; stage 0 always kicks out the incumbent, later stages replace only when the incoming count is larger, and the evicted entry cascades down. The result is a compact approximate top-k structure updated in constant time per event. The paper pairs this with per-PID lossless maps so a chosen process is never evicted. The evaluation claims 95–100% overlap with top's top-k set for k ≤ 20 on CPU, 90–100% on memo

Load-bearing premise

The accuracy claims rest on ground truth collected from the top command's interval-based snapshots; if top misranks or misses short-lived processes, the comparison does not measure true top-k fidelity.

Editorial extensions

If this is right

  • Top-k monitoring can run in the kernel at ~0.01 s reporting rates, making transient CPU and memory bursts visible to schedulers, orchestrators, and operators without userspace polling.
  • Per-PID exact tracking can coexist with approximate global top-k, so critical workloads are never approximated away by eviction.
  • The same sketch pipeline can be extended to other additive or monotonic system counters, such as I/O bytes or network packets, as the paper's modular design already separates pipeline logic from event source.
  • The measured footprint (~1.2 MB for memory tracking, ~183 KB for CPU tracking) makes the approach viable on resource-constrained cloud and edge nodes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported accuracy is measured against top-derived snapshots, which sample /proc on a fixed interval; a more direct comparison using per-process accounting from /proc/<pid>/stat or cgroup cpu.stat could change the precision numbers, especially for short-lived processes that top's interval may alias.
  • The 14x temporal-resolution claim compares reporting intervals rather than event-detection latency; a controlled experiment that injects a known CPU burst and measures time-to-detection would isolate the system's true reactivity.
  • HashPipe's accuracy depends on the skewness of the resource distribution; workloads where many processes have similar usage are the natural stress test, since the compare-and-swap rule favors heavy hitters and may lose middle-of-pack processes.
  • The same instrumentation could be pointed at GPU kernel events or network softIRQ events, as the paper's own future work suggests, turning eHashPipe into a multi-resource observability plane rather than a CPU/memory monitor.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes eHashPipe, an eBPF-based system that adapts the HashPipe sketch algorithm to track top-k CPU and memory-consuming processes in the Linux kernel, alongside exact tracking for user-specified PIDs. It describes two in-kernel pipelines: one for memory events (allocations and deallocations) and one for on-CPU time derived from sched_switch tracepoints. The evaluation claims high top-k accuracy relative to top, roughly 14x finer temporal resolution, and low overhead. The authors argue this provides lightweight, responsive observability suitable for cloud and edge environments.

Significance. If the results held, eHashPipe would be a practically useful contribution to kernel-level observability, combining the compact HashPipe sketch with eBPF for top-k resource monitoring. The implementation includes useful engineering details, such as physically unrolled loops to satisfy eBPF verifier constraints, configurable pipeline depth/slots, and atomic updates to handle race conditions. However, the current evidence is not sufficient: the headline accuracy claim is internally inconsistent, the ground truth is derived from the very tool the paper claims to supersede, no error bars or repeated trials are reported, and the memory pipeline's handling of negative deallocation events raises a correctness concern. The idea remains potentially viable, but the empirical and algorithmic basis needs substantial rework.

major comments (5)
  1. [Abstract; Table 1] The abstract states '100 percent Top-k precision for CPU and memory at k = 1, 5, and 10,' but Table 1 reports Memory Accuracy of only 90.0% at k=10. This is a direct internal contradiction in the paper's headline result. The abstract must be corrected and the experiment repeated to determine which number is actually supported.
  2. [§IV-C] Ground truth is constructed from snapshots of the top command, which aggregates /proc values. This is not an independent baseline: top is the tool eHashPipe claims to outperform, and its interval-based sampling and smoothing can misrank or miss short-lived processes. No repeated trials or error bars are reported for Table 1, so the accuracy numbers cannot be considered statistically reliable.
  3. [Algorithm 1; §II-A.3] Deallocation events are fed into the pipeline as entries with negative size. In Stage 0, the always-kick rule swaps the new entry into the slot regardless of value, so a negative deallocation can evict a large positive heavy-hitter and corrupt the memory top-k result. The evaluation does not include a workload with high allocation/deallocation churn, so this issue is not exposed. The deallocation path should be revised (e.g., only updating matching entries or bypassing stage 0 for negative entries) and a churn-focused experiment added.
  4. [§IV-E] The claimed 'about 14x finer temporal resolution' is not substantiated. No measurement methodology, raw trace data, or statistical support is given. Moreover, §IV-D says detection frequency is configured once every 2 seconds to align with top, while §IV-E describes a 0.01-second target sampling interval. These contradictory statements need to be reconciled and the resolution claim demonstrated with concrete measurements.
  5. [§IV-F] The CPU overhead of 'about 20%' is reported without a baseline or methodology. 20% is not obviously 'very low' and needs context, such as comparison against top or an unmonitored baseline, and a description of how the overhead was measured.
minor comments (4)
  1. [Throughout] Inconsistent capitalization: eHashPipe in some places, eHashpipe in others. The abstract also contains 'T op-k' formatting errors. Please unify.
  2. [§II-A.1] The text mentions monitoring '12 key allocation functions' but does not list them. Provide the actual probe list.
  3. [Figures 1-6] Figure captions are generic ('Fig. 1: Always Kick Out'); the figures need axes, units, and clear descriptions. The text should reference specific aspects of each figure.
  4. [§IV-C] There is a typo: '/proc sfile' should be '/proc filesystem'. Also specify the exact top command, interval, and delay used for ground truth construction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical measurements, not derivations that reduce to their inputs.

full rationale

eHashPipe is an empirical systems paper; it does not present a derivation chain in which an output quantity is constructed from the same quantity it is claimed to predict. The top-k accuracy numbers are measured by comparing eHashPipe's sketch output to a ground-truth list produced by `top` (Section IV-C). That ground-truth choice raises a benchmark-validity concern, because the paper elsewhere argues `top` is coarse and smoothed, but it is not a circular derivation: eHashPipe's kernel-event updates and `top`'s /proc polling are independent measurement paths, and no eHashPipe parameter is fitted to `top`'s output. The only self-citation is reference [14], used as background for the observation that deep-learning training workloads produce cyclic CPU/memory patterns; this is not load-bearing for any of the paper's predictive claims. The 'lossless' tracking for user-specified PIDs is exact by construction (a dedicated BPF map), but this is an implementation property, not a presented derivation result. No equation, fitted parameter, or uniqueness theorem is reused as the claimed finding, so the central results are not circular. I additionally note an internal inconsistency (abstract says 100% memory accuracy at k=10 while Table 1 reports 90.0%), but that is a correctness/consistency issue, not a circularity issue.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper rests on the unvalidated assumption that top yields correct ground truth, on the completeness of 12 probed memory functions, and on HashPipe's known approximation properties. No free parameters are reported, so the exact configuration behind the accuracy numbers is unknown.

free parameters (3)
  • pipeline depth d = not reported (memory module uses 5 stages per overhead description)
    Number of HashPipe stages; chosen empirically to fit eBPF verifier limits.
  • slots per stage N = 2000 (inferred from map size description)
    Capacity of each stage; affects collision rate and accuracy.
  • hash seed coefficients a_i, b_i = not reported
    Random hash functions for slot selection; not stated, hindering reproducibility.
assumptions (4)
  • standard math HashPipe sketch approximates top-k heavy hitters with bounded error
    Relies on Sivaraman et al. [10] for the algorithm's properties; cited but not proved.
  • domain assumption sched_switch timestamps fully capture on-CPU time per thread
    Section II-B assumes every context switch generates a tracepoint and the delta between scheduling events is the on-CPU duration.
  • domain assumption The 12 probed allocation/deallocation functions cover all significant memory events
    Section II-A1 states probes attach to 12 key allocation functions; coverage completeness unvalidated.
  • domain assumption top and /proc provide a valid ground truth for top-k sets
    Section IV-C constructs ground truth from top; if top is inaccurate, the accuracy metric is invalid.

how reviews work

0 comments
Cite this review

Pith. "Pith review of eHashPipe: Lightweight Top-K and Per-PID Resource Monitoring with eBPF." pith.science (2026). https://pith.science/paper/REHPXSFP

@misc{pith2026250909879,
  author       = {Pith},
  title        = {Pith review of: eHashPipe: Lightweight Top-K and Per-PID Resource Monitoring with eBPF},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/REHPXSFP}},
  note         = {Machine review of arXiv:2509.09879}
}
read the original abstract

System-level resource monitoring with both precision and efficiency is a continuous challenge. We introduce eHashPipe, a lightweight, real-time resource observability system utilizing eBPF and the HashPipe sketching algorithm. eHashPipe supports two tracking modes: Top-k monitoring to identify the most resource-demanding processes and specific PID tracking to detail the behavior of selected processes. We implement two in-kernel eBPF pipelines for on-CPU time and memory usage. Unlike traditional userspace polling tools, eHashPipe operates in the kernel to reduce latency and context-switch overhead while keeping the runtime footprint small. During our experiments, eHashPipe attains 100 percent Top-k precision for CPU and memory at k = 1, 5, and 10, 95.0/90.0 percent at k = 20, and 93.3/83.3 percent at k = 30 compared to the ground truth. It exposes short-lived bursts with about 14 times finer temporal resolution than top while imposing very low overhead. These results show that eHashPipe delivers accurate, responsive insight with minimal impact, making it well suited for latency-sensitive cloud and edge environments.

Figures

Figures reproduced from arXiv: 2509.09879 by the authors.

Figure 2
Figure 2. Comparison Kick Out selected processes, we maintain a separate BPF map that records their memory usage with full accuracy and with￾out evictions. This lossless tracking mechanism ensures that critical or user-specified processes can be monitored precisely, regardless of their rank or competition in the main HashPipe pipeline. • Stage 0 (forced eviction): For every memory event, a hash is computed to locate the appro… view at source ↗
Figure 1
Figure 1. Always Kick Out 2) Pipeline Logic: • Direct PID Monitoring (Lossless Tracking): Before a memory event enters the pipeline, we first check whether its pid matches any user-specified target PIDs. For these [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 6
Figure 6. Memory usage recorded by eHashpipe. each, 40 bytes per entry) and auxiliary maps. The CPU HashPipe module uses about 183 KB. CPU Overhead. Measured with perf stat, the eBPF process incurs only about 20% CPU overhead under our evaluation setup. G. Summary of Findings Our evaluation demonstrates that eHashpipe provides a lightweight, high-fidelity, and responsive in-kernel monitoring solution that significantly outper… view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: CPU usage recorded by top under various intervals [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 4
Figure 4. Figure 4: Memory usage recorded by top under various intervals [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: CPU usage recorded by eHashpipe. F. Resource Overhead In our evaluation setup, we measured the memory footprint and CPU overhead of both HashPipe modules (memory and CPU). Memory usage. The eHashPipe memory module uses ap￾proximately 1.2 MB, including 5 inner maps (200…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references

  1. [1]

    Characterizing in-kernel ob- servability of latency-sensitive request-level metrics with ebpf,

    M. Rezvani, A. Jahanshahi, and D. Wong, “Characterizing in-kernel ob- servability of latency-sensitive request-level metrics with ebpf,” in2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 2024, pp. 24–35

    2024

  2. [2]

    A profiling- based benchmark suite for warehouse-scale computers,

    A. Abel, Y . Li, R. O’Grady, C. Kennelly, and D. Gove, “A profiling- based benchmark suite for warehouse-scale computers,” in2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 2024, pp. 325–327

    2024

  3. [3]

    Enabling performance observability for heterogeneous hpc workflows with soma,

    D. Yokelson, M. Titov, S. Ramesh, O. Kilic, M. Turilli, S. Jha, and A. Malony, “Enabling performance observability for heterogeneous hpc workflows with soma,” inProceedings of the 53rd International Conference on Parallel Processing, 2024, pp. 220–230

    2024

  4. [4]

    Flexflow: A flexible dataflow accelerator architecture for convolutional neural networks,

    W. Lu, G. Yan, J. Li, S. Gong, Y . Han, and X. Li, “Flexflow: A flexible dataflow accelerator architecture for convolutional neural networks,” in 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2017, pp. 553–564

    2017

  5. [5]

    Gregg,Systems performance: enterprise and the cloud

    B. Gregg,Systems performance: enterprise and the cloud. Pearson Education, 2014

    2014

  6. [6]

    Exploiting rdma opera- tions for providing efficient fine-grained resource monitoring in cluster- based servers,

    K. Vaidyanathan, H.-W. Jin, and D. K. Panda, “Exploiting rdma opera- tions for providing efficient fine-grained resource monitoring in cluster- based servers,” in2006 IEEE International Conference on Cluster Computing. IEEE, 2006, pp. 1–10

    2006

  7. [7]

    Survey and analysis of kernel and userspace tracers on linux: Design, implementation, and overhead,

    M. Gebai and M. R. Dagenais, “Survey and analysis of kernel and userspace tracers on linux: Design, implementation, and overhead,”ACM Computing Surveys, vol. 51, no. 2, pp. 26:1–26:33, 2018

    2018

  8. [8]

    Fast in-kernel traffic sketching in ebpf,

    S. Miano, X. Chen, R. B. Basat, and G. Antichi, “Fast in-kernel traffic sketching in ebpf,”ACM SIGCOMM Computer Communication Review, vol. 53, no. 1, pp. 3–13, 2023

    2023

Show all 17 references
  1. [9]

    ebpf-enhanced complete observability solu- tion for cloud-native microservices,

    B. Sharma and D. Nadig, “ebpf-enhanced complete observability solu- tion for cloud-native microservices,” inICC 2024-IEEE International Conference on Communications. IEEE, 2024, pp. 1980–1985

    2024

  2. [10]

    Heavy-hitter detection entirely in the data plane,

    V . Sivaraman, S. Narayana, O. Rottenstreich, S. Muthukrishnan, and J. Rexford, “Heavy-hitter detection entirely in the data plane,” in Proceedings of the Symposium on SDN Research, 2017, pp. 164–176

    2017

  3. [11]

    A. S. Tanenbaum and H. Bos,Modern Operating Systems, 4th ed. Pearson, 2015

    2015

  4. [12]

    Gregg,BPF Performance Tools: Linux System and Application Observability

    B. Gregg,BPF Performance Tools: Linux System and Application Observability. Addison-Wesley, 2020

    2020

  5. [13]

    Clang Loop Optimization Pragmas,

    LLVM Project, “Clang Loop Optimization Pragmas,” https://clang.llvm. org/docs/LanguageExtensions.html#id14, 2024, accessed: 2025-07-09

    2024

  6. [14]

    Dnn architecture attacks via network and power side channels,

    Y . Dai, Q. Guo, and A. Wang, “Dnn architecture attacks via network and power side channels,” inInternational Conference on Security and Privacy in Communication Systems. Springer, 2023, pp. 63–87

    2023

  7. [15]

    Beyond data and model parallelism for deep neural networks,

    Z. Jia, M. Zaharia, and A. Aiken, “Beyond data and model parallelism for deep neural networks,” inProceedings of the 2nd Conference on Machine Learning and Systems (MLSys), 2019

    2019

  8. [16]

    Efficient flow scheduling in distributed deep learning training with echelon formation,

    R. Pan, Y . Lei, J. Li, Z. Xie, B. Yuan, and Y . Xia, “Efficient flow scheduling in distributed deep learning training with echelon formation,” inProceedings of the 21st ACM Workshop on Hot Topics in Networks, 2022, pp. 93–100

    2022

  9. [17]

    Development and pilot testing of an online case-based approach to shared decision making skills training for clinicians,

    R. J. V olk, N. K. Shokar, V . B. Leal, R. J. Bulik, S. K. Linder, P. D. Mullen, R. M. Wexler, and G. S. Shokar, “Development and pilot testing of an online case-based approach to shared decision making skills training for clinicians,”BMC Medical Informatics and Decision Makin...

    2014

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.