Towards Real-Time PixOOD: Efficient Anomaly Segmentation for Autonomous Vehicles

Federico Aromolo; Federico Nesti; Giorgio Buttazzo; Luca de Martino

REVIEW 2 major objections 5 minor 20 references

Towards Real-Time PixOOD: Efficient Anomaly Segmentation for Autonomous Vehicles

T0 review · 2 major / 5 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reformulated PixOOD scorer and TensorRT build push anomaly segmentation to 182 FPS on desktop and 75 FPS on Jetson, meeting real-time onboard targets.

desk verdict Solid systems engineering: they killed the CPU Neyman–Pearson bottleneck and actually hit real-time on Orin, with honest full-frame caveats. read the letter →

arxiv 2607.28483 v1 pith:IGTORGFN submitted 2026-07-30 cs.CV

Luca de Martino , Federico Aromolo , Federico Nesti , Giorgio Buttazzo This is my paper

classification cs.CV

keywords real-timeanomalysegmentationembeddedsystemsrailwaysafetyautonomousdrivinghardwareaccelerationPixOODTensorRTNeyman-Pearsonscoring

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autonomous vehicles and trains need per-pixel flags for unexpected objects, yet leading anomaly segmenters are too slow for embedded hardware. This paper takes PixOOD, a prototype-based out-of-distribution detector, and removes its main bottleneck: a CPU Neyman–Pearson scoring stage that forced a GPU–host round-trip every frame. The authors recompute class densities in log-space on the GPU and replace the CPU CDF lookup with a precomputed grid sampled by a TensorRT-friendly operator, then compile the whole pipeline into two TensorRT engines. On a desktop RTX 4060 they reach up to 182 FPS; on a Jetson AGX Orin they reach 75 FPS at 640 px—roughly 20× and 18× the original baseline—while holding average-precision drift under 0.15 %. They also show that the usual road-only evaluation protocol hides many off-road false positives, so full-frame railway data become the more honest deployment test. The result is concrete evidence that a state-of-the-art anomaly segmenter can run onboard at safety-relevant rates.

What carries the argument

GPU-native Neyman–Pearson scoring: log-space batched multivariate-normal densities plus offline 200×200 per-class score maps looked up by bilinear grid_sample, eliminating the per-frame CPU CDF round-trip and enabling full TensorRT export.

What would settle it

Measure end-to-end AP and FPR95 of the TensorRT grid_sample pipeline against the original SciPy CDF scorer on the same full-frame LostAndFound and OSDaR-AR splits; a deviation well above 0.15 % AP, or failure to sustain ≥25 FPS at the stated power budgets, would falsify the deployment claim.

Watch

Extended reading notes

Core claim

After a GPU reformulation of PixOOD’s Neyman–Pearson scoring stage and platform-specific TensorRT compilation, the full pipeline reaches up to 182 FPS on an RTX 4060 and 75 FPS on a Jetson AGX Orin (DINOv2-RS FP16 at 640 px)—about 20× and 18× faster than the original baseline—with less than 0.15 % AP deviation from the original scorer, making real-time onboard anomaly segmentation practical for driving and railway systems.

Load-bearing premise

That the offline 200-by-200 score grids looked up with bilinear sampling are close enough to the original CPU CDF interpolation, and that the reported road-region and railway accuracies are enough evidence of safety usefulness even when full-frame road AP collapses.

Editorial extensions

If this is right

Onboard systems can run a leading anomaly segmenter above the 25 FPS safety floor on both desktop and Jetson-class hardware.
Full-frame evaluation becomes the default deployment metric, because road-only masks hide large off-road false-positive rates.
Railway perception stacks can adopt the same accelerated PixOOD pipeline without outlier-exposure training.
Energy-per-frame numbers (down to ~0.44 J on Orin at 640 px) give concrete budgets for power-constrained vehicles.
The same ONNX-to-TensorRT split can be reused for other prototype-based detectors that currently stall on CPU scoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same log-space plus grid_sample pattern should transfer to any density-ratio OOD head that today still ships a SciPy or NumPy post-process.
Once camera-only latency is no longer the bottleneck, the calibrated LiDAR clouds in OSDaR-AR become the natural next fusion target rather than a research curiosity.
If full-frame AP stays near zero on ordinary road benchmarks, safety cases may need explicit off-road ignore policies or domain-specific retraining rather than higher FPS alone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The load-bearing result is real and useful: after a GPU log-space rewrite of PixOOD’s Neyman–Pearson stage plus dual TensorRT engines, they get ~182 FPS on a 4060 and ~75 FPS on Jetson AGX Orin at 640 px (DINOv2 FP16), roughly 18–20× over the SciPy-CPU baseline, with <0.15% AP drift on the scorer. That is the paper. Energy numbers and a clear bottleneck diagnosis (host round-trip + float64 class-serial CDF) make the speed claim easy to trust on its own terms.

What is new is not OOD theory. It is the ONNX-exportable stand-in for the CDF (offline 200×200 score maps + grid_sample), the attention-export fix for DINOv2/v3 under TensorRT, precision choices per backbone, and a dual-domain latency/energy table that the field mostly skips. They also do the right thing on evaluation: road-region LaF looks strong with ViT-S, full-frame LaF AP collapses below 1%, and they say so. OSDaR-AR full-frame numbers are the more honest deployment signal.

Soft spots, in proportion. The bilinear maps are an approximation of SciPy interp1d; they report tiny AP deviation, which is enough for a systems paper but not a formal equivalence proof. Safety usefulness is not closed: no multi-method latency bake-off, no closed-loop, and full-frame driving false positives remain ugly. Novelty is incremental engineering on known parts (PixOOD, DINO, TensorRT). Mild self-citation to OSDaR-AR is fine given the domain gap. No code/weights in the text weakens reproducibility claims.

Who it is for: people shipping anomaly segmentation on Jetson-class hardware in driving or rail, and anyone tired of accuracy-only OOD tables. Math is elementary and sound; measurements look careful (2000-iter CUDA sync, named platforms, precision ablations).

I would send it to peer review. Expect systems/CV referees to push on full-frame FPs, map fidelity, and code release—not to desk-reject. Worth a skim if you care about onboard OOD; not a reading-group centerpiece unless the group is embedded perception.

Referee Report

2 major / 5 minor

Summary. The paper accelerates PixOOD for real-time anomaly segmentation by (i) reformulating the Neyman–Pearson scoring stage as a GPU log-space multivariate normal density plus bilinear lookup of offline 200×200 per-class score maps via grid_sample, and (ii) compiling the full pipeline into two TensorRT engines (backbone+MLP; KNN+scoring). On DINOv2/DINOv3 ViT-S variants trained on Cityscapes or RailSem19, the deployed system reaches up to 182 FPS on an RTX 4060 and 75 FPS on a Jetson AGX Orin at 640 px (about 18–20× over the original SciPy-CPU baseline), with stated AP deviation under 0.15%. Accuracy is reported on LostAndFound (road-region and full-frame) and OSDaR-AR (full-frame, multiple resolutions), together with latency and energy on both platforms.

Significance. If the speed and accuracy-preservation results hold, the work is a useful systems contribution: it shows that a leading prototype-based anomaly segmenter can meet a 25 FPS onboard target on embedded automotive/railway hardware without outlier exposure. Strengths include concrete bottleneck diagnosis (CPU CDF round-trip), named operators and precision choices (FP16/FP32, non-fused attention fix), CUDA-synchronized timing over 2000 iterations, dual-platform energy numbers, and an honest dual-protocol LaF evaluation that exposes full-frame false positives. Cross-domain coverage (driving + OSDaR-AR railway) and open acknowledgment that standard road-region metrics are optimistic add practical value beyond a pure accuracy paper.

major comments (2)

[Section 4.1; Tables 1–2] Section 4.1 states “less than 0.15% AP deviation from the original implementation” for the GPU scorer, and Tables 1–2 report detection metrics without a side-by-side original SciPy vs. torch.searchsorted vs. grid_sample (and vs. final TensorRT) comparison on the same checkpoints, datasets, and protocols. Because the central claim couples large speedups with negligible accuracy loss, a compact table of AP/FPR95 for those three scorers (at least on LaF road-region and OSDaR-AR at 640/896 px) is needed so readers can verify that the ONNX-exportable bilinear maps—not only the non-exportable searchsorted reference—preserve the published operating point under both protocols.
[Table 3; Section 5.3] Table 3 and Section 5.3 compare energy to a baseline that “does not include the CPU scoring cost and is therefore a lower bound,” while speedups use end-to-end baseline latency that does include that cost. The 6–20× faster claim is accordingly well-supported; the energy advantage is harder to interpret. Please either instrument host/CPU energy for the SciPy path or restrict energy claims to same-pipeline precision/resolution ablations (FP16 vs FP32, 640 vs 896) so the efficiency narrative does not mix incompatible baselines.

minor comments (5)

[Figure 2] Figure 2 is helpful but does not mark where the offline 200×200 score maps are injected as static weights; a small annotation would clarify the deployment graph.
[Section 5.1] Resolution labels (640/896/1792 px) are defined as the largest side after resize with patch-multiple rounding; stating the exact H×W tensors used for DINOv2 vs DINOv3 in a footnote or appendix would aid reproduction.
[Section 5.2; Table 1] Table 1 full-frame AP <1% is an important negative result; a brief qualitative note (or supplemental figure) on typical off-road false-positive modes would help readers judge residual risk beyond the OSDaR-AR numbers.
[Section 2] Related work correctly notes the lack of embedded latency/energy studies; a short pointer to any concurrent real-time OOD segmentation systems (even if not prototype-based) would situate the 75 FPS Orin result more clearly.
Minor typography: “Neyman–Pearson” vs “Neyman-Pearson,” and spacing in “20×” / “3.78×” should be made consistent; “float64”/“float32” notation is fine but could be unified with FP16/FP32 used later.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical systems speedups and accuracy are measured, not forced by definition or self-citation.

full rationale

This is a deployment/acceleration paper. The load-bearing claims are (i) a GPU reformulation of PixOOD’s Neyman–Pearson stage (log-space density + grid_sample CDF maps) and (ii) TensorRT compilation yielding measured FPS/energy on RTX 4060 and Jetson AGX Orin, with AP/FPR95 on LostAndFound and OSDaR-AR. None of these reduce by construction to fitted inputs or to a self-defined quantity: latency and energy are timed with CUDA events; accuracy is scored against external pixel labels; the <0.15% AP deviation is an empirical comparison to the original SciPy scorer, not an algebraic identity. Self-citations (OSDaR-AR dataset; prior TensorRT multitasking work) supply data or tooling context and do not underwrite a uniqueness theorem or force the speedup figures. No self-definitional loop, fitted-as-prediction step, or renamed known law appears in the derivation chain. Score 0 with empty steps is the appropriate finding.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper is an engineering acceleration of PixOOD. Load-bearing background includes optimality/usefulness of N-P likelihood-ratio scoring on 2D (logit, KNN-distance) features, frozen DINOv2/v3 features, and standard TensorRT/ONNX semantics. Free choices that affect reported FPS/AP include grid resolution, input resolution labels, precision per backbone, prototype count, and the 25 FPS target. No new physical entities are postulated.

free parameters (5)

CDF/score map grid resolution = 200×200 per class
Chosen as 200×200 nodes per class to balance accuracy vs memory/cache; directly defines the deployed grid_sample approximation quality.
Prototypes per semantic class K = K = 1000
Inherited/chosen clustering budget for in-distribution manifold; affects KNN cost and feature geometry.
Input resolution labels (640 / 896 / 1792 px) = 640 px primary embedded target
Hand-selected operating points for embedded vs accuracy trade-off; primary Orin claim uses 640 px.
Per-architecture floating-point precision = DINOv2 FP16; DINOv3 FP32
FP16 for DINOv2, FP32 for DINOv3 due to rotary-embedding / If-node instability; chosen for numerical stability not learned from data.
Real-time FPS target = 25 FPS
Design threshold motivating optimizations; cited as faster-than-human-driver motivation rather than derived.

assumptions (5)

domain assumption Generalized Neyman–Pearson likelihood ratio on per-class 2D Gaussians over (MLP logit, nearest-prototype L2) is an appropriate anomaly score for pixel OOD segmentation.
Taken from PixOOD (Section 3.2); the paper optimizes its evaluation, not re-derives optimality for deployment domains.
domain assumption Frozen DINOv2/DINOv3 ViT-S patch features plus Cityscapes/RailSem19-trained MLP/prototypes transfer sufficiently to LaF and OSDaR-AR anomalies.
Sections 3–5; accuracy claims rest on this transfer without outlier exposure.
ad hoc to paper float32 log-space Gaussian density plus bilinear sampling of precomputed score maps matches float64 SciPy CDF scoring within negligible AP error.
Core correctness premise of Section 4.1 deployment path; supported by reported <0.15% AP deviation but not by a formal error bound.
domain assumption ONNX/TensorRT compilation with explicit non-fused attention preserves task behavior aside from known FP16 rotary issues handled by FP32.
Section 4.2; required for equating TensorRT FPS to the intended model.
domain assumption Standard AP and FPR95 (road-region or full-frame) are meaningful proxies for onboard safety utility at the reported operating points.
Section 3.1 and 5; common in the field but strained by full-frame LaF AP collapse.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Real-Time PixOOD: Efficient Anomaly Segmentation for Autonomous Vehicles." pith.science (2026). https://pith.science/paper/IGTORGFN

@misc{pith2026260728483,
  author       = {Pith},
  title        = {Pith review of: Towards Real-Time PixOOD: Efficient Anomaly Segmentation for Autonomous Vehicles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IGTORGFN}},
  note         = {Machine review of arXiv:2607.28483}
}

read the original abstract

Real-time anomaly segmentation is essential for the safety of autonomous systems. Although recent approaches offer high accuracy, their computational cost limits their deployment on embedded hardware. This work presents an efficient and accelerated pipeline designed for both embedded and desktop platforms, targeting the autonomous driving and railway domains. The proposed approach reformulates the Neyman-Pearson scoring stage of PixOOD, a state-of-the-art out-of-distribution detection method, and deploys the full pipeline through hardware-optimized TensorRT compilation, reaching up to 182 FPS on a desktop NVIDIA RTX 4060 GPU and 75 FPS on the NVIDIA Jetson AGX Orin embedded platform, respectively 20x and 18x faster than the original baseline. The achieved results demonstrate that advanced anomaly segmentation can be efficiently deployed for onboard processing in autonomous driving and railway applications.

Figures

Figures reproduced from arXiv: 2607.28483 by the authors.

**Figure 1.** Two examples of anomaly segmentation. Red overlays mark the pixels flagged as anomalous. Left: an OSDaR-AR railway scene, where a cow placed on the tracks is correctly detected as an out-of-distribution obstacle. Right: a LostAndFound road scene, where a small obstacle on the road surface is flagged as an anomaly. 3.1 Datasets and evaluation protocols Standard anomaly segmentation benchmarks for autonomous driving, … view at source ↗

**Figure 2.** PixOOD processing pipeline. Top: the original PyTorch implementation [18], in which the ViT backbone, MLP decoder, and KNN prototype module run on the GPU (blue), while the Neyman–Pearson scoring stage runs on the CPU (orange). Bottom: the proposed accelerated implementation, organized into two TensorRT engines that run entirely on the GPU. module, both applied to each patch embedding z to produce its two-component … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

20 extracted references · 3 linked inside Pith

[1]

In: 2025 IEEE Real-Time Systems 12 L

Aromolo, F., Stevanato, A., Biondi, A., Buttazzo, G.: Real-time multitasking of deep neural networks with Nvidia TensorRT. In: 2025 IEEE Real-Time Systems 12 L. de Martino et al. Symposium (RTSS). pp. 364–377 (2025)

2025
[2]

International Journal of Computer Vision129(11), 3119–3135 (2021)

Blum, H., Sarlin, P.E., Nieto, J., Siegwart, R., Cadena, C.: The Fishyscapes benchmark: Measuring blind spots in semantic segmentation. International Journal of Computer Vision129(11), 3119–3135 (2021)

2021
[3]

In: IEEE/CVF CVPR

Bogdoll, D., Nitsche, M., Zöllner, J.M.: Anomaly detection in autonomous driving: A survey. In: IEEE/CVF CVPR. pp. 4488–4499 (2022)

2022
[4]

arXiv:2104.14812 (2021)

Chan, R., Lis, K., Uhlemeyer, S., Blum, H., Honari, S., Siegwart, R., Fua, P., Salzmann, M., Rottmann, M.: SegmentMeIfYouCan: A benchmark for anomaly segmentation. arXiv:2104.14812 (2021)

arXiv 2021
[5]

In: IEEE/CVF CVPR

Cordts, M., Omran, M., Ramos, S., Rehfeld, T., et al.: The Cityscapes dataset for semantic urban scene understanding. In: IEEE/CVF CVPR. pp. 3213–3223 (2016)

2016
[6]

In: IEEE/CVF ICCV

Jung, S., Lee, J., Gwak, D., Choi, S., Choo, J.: Standardized max logits: A simple yet effective approach for identifying unexpected road obstacles in urban-scene segmentation. In: IEEE/CVF ICCV. pp. 15425–15434 (2021)

2021
[7]

In: International Conference on Architectural Support for Programming Languages and Operating Systems

Lin, S.C., Zhang, Y., Hsu, C.H., Skach, M., Haque, M.E., Tang, L., Mars, J.: The architectural implications of autonomous driving: Constraints and acceleration. In: International Conference on Architectural Support for Programming Languages and Operating Systems. pp. 751–766 (2018)

2018
[8]

In: European Conference on Computer Vision (ECCV)

Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: European Conference on Computer Vision (ECCV). pp. 740–755. Springer (2014)

2014

Show all 20 references

[9]

In: IEEE/CVF ICCV

Lis, K., Nakka, K., Fua, P., Salzmann, M.: Detecting the unexpected via image resynthesis. In: IEEE/CVF ICCV. pp. 2152–2161 (2019)

2019
[10]

Advances in Neural Information Processing Systems33, 21464–21475 (2020)

Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based out-of-distribution detection. Advances in Neural Information Processing Systems33, 21464–21475 (2020)

2020
[11]

In: IEEE/CVF ICCV

Nayal, N., Yavuz, M., Henriques, J.F., Güney, F.: RbA: Segmenting unknown regions rejected by all. In: IEEE/CVF ICCV. pp. 711–722 (2023)

2023
[12]

arXiv:2602.22920 (2026)

Nesti, F., D’Amico, G., Marinoni, M., Buttazzo, G.: OSDaR-AR: Enhancing railway perception datasets via multi-modal augmented reality. arXiv:2602.22920 (2026)

2026
[13]

arXiv:2304.07193 (2023)

Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., et al.: DINOv2: Learning robust visual features without supervision. arXiv:2304.07193 (2023)

2023 arXiv
[14]

In: IEEE/RSJ IROS

Pinggera, P., Ramos, S., Gehrig, S., Franke, U., Rother, C., Mester, R.: Lost and found: detecting small road hazards for self-driving vehicles. In: IEEE/RSJ IROS. pp. 1099–1106. IEEE (2016)

2016
[15]

In: IEEE/CVF CVPR

Shoeb, Y., Nowzad, A., Gottschalk, H.: Out-of-distribution segmentation in au- tonomous driving: Problems and state of the art. In: IEEE/CVF CVPR. pp. 4310–4320 (2025)

2025
[16]

arXiv:2508.10104 (2025)

Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., et al.: DINOv3. arXiv:2508.10104 (2025)

2025 arXiv
[17]

arXiv:2603.11441 (2026)

Turkcan, M.K.: Detect anything in real time: From single-prompt segmentation to multi-class detection. arXiv:2603.11441 (2026)

2026
[18]

IEEE Transactions on Pattern Analysis and Machine Intelligence (2026)

Vojíř, T., Jan, Š., Matas, J.: PixOOD: Pixel-level out-of-distribution detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (2026)

2026
[19]

In: IEEE/CVF CVPR Workshops

Zendel, O., Murschitz, M., Zeilinger, M., Steininger, D., Abbasi, S., Beleznai, C.: RailSem19: A dataset for semantic rail scene understanding. In: IEEE/CVF CVPR Workshops. pp. 1221–1229 (2019)

2019
[20]

In: IEEE/CVF CVPR

Zhao, W., Li, J., Dong, X., Xiang, Y., Guo, Y.: Segment every out-of-distribution object. In: IEEE/CVF CVPR. pp. 3910–3920 (2024)

2024

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.

[1] [1]

In: 2025 IEEE Real-Time Systems 12 L

Aromolo, F., Stevanato, A., Biondi, A., Buttazzo, G.: Real-time multitasking of deep neural networks with Nvidia TensorRT. In: 2025 IEEE Real-Time Systems 12 L. de Martino et al. Symposium (RTSS). pp. 364–377 (2025)

2025

[2] [2]

International Journal of Computer Vision129(11), 3119–3135 (2021)

Blum, H., Sarlin, P.E., Nieto, J., Siegwart, R., Cadena, C.: The Fishyscapes benchmark: Measuring blind spots in semantic segmentation. International Journal of Computer Vision129(11), 3119–3135 (2021)

2021

[3] [3]

In: IEEE/CVF CVPR

Bogdoll, D., Nitsche, M., Zöllner, J.M.: Anomaly detection in autonomous driving: A survey. In: IEEE/CVF CVPR. pp. 4488–4499 (2022)

2022

[4] [4]

arXiv:2104.14812 (2021)

Chan, R., Lis, K., Uhlemeyer, S., Blum, H., Honari, S., Siegwart, R., Fua, P., Salzmann, M., Rottmann, M.: SegmentMeIfYouCan: A benchmark for anomaly segmentation. arXiv:2104.14812 (2021)

arXiv 2021

[5] [5]

In: IEEE/CVF CVPR

Cordts, M., Omran, M., Ramos, S., Rehfeld, T., et al.: The Cityscapes dataset for semantic urban scene understanding. In: IEEE/CVF CVPR. pp. 3213–3223 (2016)

2016

[6] [6]

In: IEEE/CVF ICCV

Jung, S., Lee, J., Gwak, D., Choi, S., Choo, J.: Standardized max logits: A simple yet effective approach for identifying unexpected road obstacles in urban-scene segmentation. In: IEEE/CVF ICCV. pp. 15425–15434 (2021)

2021

[7] [7]

In: International Conference on Architectural Support for Programming Languages and Operating Systems

Lin, S.C., Zhang, Y., Hsu, C.H., Skach, M., Haque, M.E., Tang, L., Mars, J.: The architectural implications of autonomous driving: Constraints and acceleration. In: International Conference on Architectural Support for Programming Languages and Operating Systems. pp. 751–766 (2018)

2018

[8] [8]

In: European Conference on Computer Vision (ECCV)

Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: European Conference on Computer Vision (ECCV). pp. 740–755. Springer (2014)

2014