pith. sign in

arxiv: 2301.01201 · v6 · submitted 2022-12-20 · 💻 cs.CV · cs.LG· eess.IV

Uncertainty in Real-Time Semantic Segmentation on Embedded Systems

Ethan Goan , Clinton Fookes This is my paper

Pith reviewed 2026-05-24 10:07 UTC · model grok-4.3

classification 💻 cs.CV cs.LGeess.IV
keywords semantic segmentationepistemic uncertaintyembedded systemsreal-time inferenceBayesian regressionmoment propagationcomputer visionuncertainty estimation
0
0 comments X

The pith

Combining pre-trained features with Bayesian regression and moment propagation yields meaningful epistemic uncertainty for real-time semantic segmentation on embedded hardware.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops a technique to equip real-time semantic segmentation models with the ability to estimate their own uncertainty when running on embedded devices. The approach uses features from pre-trained networks and applies Bayesian methods with moment propagation to generate uncertainty measures. It achieves this without compromising the speed or accuracy needed for applications like autonomous driving. Readers should care because knowing when a model is uncertain can improve reliability in high-stakes environments where mistakes have serious consequences.

Core claim

The paper demonstrates that deep feature extraction from pre-trained models combined with Bayesian regression and moment propagation can produce meaningful epistemic uncertainty estimates on embedded hardware in real time while preserving the predictive performance of the segmentation model.

What carries the argument

Bayesian regression with moment propagation applied to features extracted from pre-trained deep models.

If this is right

  • The approach maintains real-time performance on resource-constrained hardware.
  • It generates epistemic uncertainty estimates that are meaningful for decision-making.
  • Predictive accuracy for semantic segmentation remains unchanged.
  • Applications in autonomous vehicles and human-computer interaction benefit from uncertainty awareness.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This could enable better safety mechanisms by rejecting or flagging uncertain predictions in real-world deployments.
  • Similar techniques might apply to other vision tasks requiring both speed and reliability on edge devices.
  • Further work could explore calibration of these uncertainty estimates against actual model errors.

Load-bearing premise

Moment propagation combined with Bayesian regression on features from pre-trained models can be implemented to run in real time on embedded hardware while producing uncertainty estimates that are meaningful and do not degrade segmentation accuracy.

What would settle it

A test where the proposed method either fails to run in real time on embedded hardware or where its uncertainty estimates show no correlation with segmentation errors on validation data.

Figures

Figures reproduced from arXiv: 2301.01201 by Clinton Fookes, Ethan Goan.

Figure 1
Figure 1. Figure 1: Example of semantic segmentation results obtainable [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Summary of the proposed model. The pink box repre [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Examples of predictions from the proposed model for the CityScapes Dataset utilising the BiSeNetv1 backbone (top row) and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualisation of class-conditional uncertainty for a sam [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Examples of predictions from the proposed model for the ADE20k Dataset (top row) and the CoCoStuff dataset (bottom row) [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualisation of class-conditional uncertainty for a sam [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
read the original abstract

Application for semantic segmentation models in areas such as autonomous vehicles and human computer interaction require real-time predictive capabilities. The challenges of addressing real-time application is amplified by the need to operate on resource constrained hardware. Whilst development of real-time methods for these platforms has increased, these models are unable to sufficiently reason about uncertainty present when applied on embedded real-time systems. This paper addresses this by combining deep feature extraction from pre-trained models with Bayesian regression and moment propagation for uncertainty aware predictions. We demonstrate how the proposed method can yield meaningful epistemic uncertainty on embedded hardware in real-time whilst maintaining predictive performance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper proposes combining deep feature extraction from pre-trained models with Bayesian regression and moment propagation to enable epistemic uncertainty estimation in real-time semantic segmentation on embedded hardware, claiming this yields meaningful uncertainty without degrading predictive performance.

Significance. If the quantitative claims hold, the work would address a practical gap in deploying uncertainty-aware models for safety-critical real-time applications on resource-constrained devices. No machine-checked proofs, reproducible code, or parameter-free derivations are described.

major comments (1)
  1. [Abstract] Abstract: the central claim that the method 'can yield meaningful epistemic uncertainty on embedded hardware in real-time whilst maintaining predictive performance' is unsupported by any quantitative results, baselines, hardware benchmarks, or ablation studies, preventing evaluation of the demonstration.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and the opportunity to clarify the manuscript. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that the method 'can yield meaningful epistemic uncertainty on embedded hardware in real-time whilst maintaining predictive performance' is unsupported by any quantitative results, baselines, hardware benchmarks, or ablation studies, preventing evaluation of the demonstration.

    Authors: The full manuscript contains quantitative results, baseline comparisons, embedded hardware benchmarks (including latency and memory measurements), and ablation studies in Sections 4 and 5 that support the claim. These sections report mIoU preservation alongside uncertainty metrics on standard datasets and embedded platforms. We agree, however, that the abstract itself does not cite specific numbers and therefore does not allow immediate evaluation of the claim. We will revise the abstract to incorporate the key quantitative findings (e.g., real-time FPS on target hardware and mIoU deltas) while remaining within length limits. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper describes a method that combines standard pre-trained feature extractors with Bayesian regression and moment propagation to produce uncertainty estimates. No equations or claims reduce by construction to fitted inputs, self-definitions, or load-bearing self-citations; the central demonstration is an engineering claim about real-time feasibility on embedded hardware rather than a closed mathematical loop. The provided abstract and reader's assessment confirm the approach uses independent components without redefinition of target quantities.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that pre-trained feature extractors remain effective when paired with Bayesian regression and that moment propagation can be performed efficiently enough for real-time embedded use. No free parameters or invented entities are named in the abstract.

axioms (1)
  • domain assumption Features from pre-trained models can be used directly as input to Bayesian regression for uncertainty estimation without retraining the extractor
    The method description relies on this reuse to achieve real-time performance.

pith-pipeline@v0.9.0 · 5618 in / 1263 out tokens · 24009 ms · 2026-05-24T10:07:46.447841+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 43 canonical work pages · 2 internal anchors

  1. [1]

    Semantic object classes in video: A high- definition ground truth database,

    Gabriel J Brostow, Julien Fauqueur, and Roberto Cipolla, “Semantic object classes in video: A high- definition ground truth database,”Pattern Recognition Letters, vol. 30, no. 2, pp. 88–97, 2009. 1, 6

    work page 2009

  2. [2]

    Coco-stuff: Thing and stuff classes in context,

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari, “Coco-stuff: Thing and stuff classes in context,” in Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR) , June 2018. 1, 7

    work page 2018

  3. [3]

    The cityscapes dataset for semantic urban scene under- standing,

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele, “The cityscapes dataset for semantic urban scene under- standing,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 3213–3223. 1, 6

    work page 2016

  4. [4]

    Real-time semantic segmentation of crop and weed for precision agriculture robots leveraging background knowledge in cnns,

    Andres Milioto, Philipp Lottes, and Cyrill Stachniss, “Real-time semantic segmentation of crop and weed for precision agriculture robots leveraging background knowledge in cnns,” in 2018 IEEE international con- ference on robotics and automation (ICRA) . IEEE, 2018, pp. 2229–2235. 1

    work page 2018

  5. [5]

    Scene parsing through ade20k dataset,

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba, “Scene parsing through ade20k dataset,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, 2017. 1, 7

    work page 2017

  6. [6]

    Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

    Jishnu Mukhoti and Yarin Gal, “Evaluating bayesian deep learning methods for semantic segmentation,” arXiv preprint arXiv:1811.12709, 2018. 1, 2

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  7. [7]

    Bayesian u-net: Estimating uncertainty in semantic segmentation of earth observation images,

    Cl ´ement Dechesne, Pierre Lassalle, and S ´ebastien Lef`evre, “Bayesian u-net: Estimating uncertainty in semantic segmentation of earth observation images,” Remote Sensing, vol. 13, no. 19, 2021. 1

    work page 2021

  8. [8]

    Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural net- works,

    Michael Kampffmeyer, Arnt-Borre Salberg, and Robert Jenssen, “Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural net- works,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops , 2016, pp. 1–9. 1, 2

    work page 2016

  9. [9]

    Un- certainty reduction for model adaptation in seman- tic segmentation,

    Prabhu Teja Sivaprasad and Francois Fleuret, “Un- certainty reduction for model adaptation in seman- tic segmentation,” in 2021 Ieee/Cvf Conference On Computer Vision And Pattern Recognition, Cvpr 2021. IEEE, 2021, number CONF, pp. 9608–9618. 1

    work page 2021

  10. [10]

    Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation,

    Zhedong Zheng and Yi Yang, “Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation,” International Jour- nal of Computer Vision , vol. 129, no. 4, pp. 1106– 1120, 2021. 1

    work page 2021

  11. [11]

    Deep kernel learning,

    Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhut- dinov, and Eric P Xing, “Deep kernel learning,” in Artificial intelligence and statistics. PMLR, 2016, pp. 370–378. 1, 2

    work page 2016

  12. [12]

    Scalable bayesian optimization using deep neural networks,

    Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams, “Scalable bayesian optimization using deep neural networks,” in International conference on machine learning. PMLR, 2015, pp. 2171–2180. 1, 2

    work page 2015

  13. [13]

    Vision transformer adapter for dense predictions,

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao, “Vision trans- former adapter for dense predictions,” arXiv preprint arXiv:2205.08534, 2022. 2

    work page arXiv 2022

  14. [14]

    Segmentation transformer: Object- contextual representations for semantic segmenta- tion,

    Yuhui Yuan, Xiaokang Chen, Xilin Chen, and Jing- dong Wang, “Segmentation transformer: Object- contextual representations for semantic segmenta- tion,” 2020. 2

    work page 2020

  15. [15]

    Internim- age: Exploring large-scale vision foundation mod- els with deformable convolutions,

    Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, et al., “Internim- age: Exploring large-scale vision foundation mod- els with deformable convolutions,” arXiv preprint arXiv:2211.05778, 2022. 2

    work page arXiv 2022

  16. [16]

    Scene segmentation with dual relation- aware attention network,

    Jun Fu, Jing Liu, Jie Jiang, Yong Li, Yongjun Bao, and Hanqing Lu, “Scene segmentation with dual relation- aware attention network,” IEEE Transactions on Neu- ral Networks and Learning Systems, vol. 32, no. 6, pp. 2547–2560, 2020. 2

    work page 2020

  17. [17]

    Fully convolutional networks for semantic segmenta- tion,

    Jonathan Long, Evan Shelhamer, and Trevor Darrell, “Fully convolutional networks for semantic segmenta- tion,” in Proceedings of the IEEE conference on com- puter vision and pattern recognition, 2015, pp. 3431–

    work page 2015

  18. [18]

    ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation

    Adam Paszke, Abhishek Chaurasia, Sangpil Kim, and Eugenio Culurciello, “Enet: A deep neural net- work architecture for real-time semantic segmenta- tion,” arXiv preprint arXiv:1606.02147, 2016. 2, 6

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  19. [19]

    Deep residual learning for image recognition,

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. 2

    work page 2016

  20. [20]

    Swiftnet: Real-time video object segmentation,

    Haochen Wang, Xiaolong Jiang, Haibing Ren, Yao Hu, and Song Bai, “Swiftnet: Real-time video object segmentation,” in Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2021, pp. 1296–1305. 2

    work page 2021

  21. [21]

    Bisenet: Bilateral segmentation network for real-time semantic segmen- tation,

    Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang, “Bisenet: Bilateral segmentation network for real-time semantic segmen- tation,” CoRR, 2018. 2, 6

    work page 2018

  22. [22]

    Bisenet v2: Bi- lateral network with guided aggregation for real-time semantic segmentation,

    Changqian Yu, Changxin Gao, Jingbo Wang, Gang Yu, Chunhua Shen, and Nong Sang, “Bisenet v2: Bi- lateral network with guided aggregation for real-time semantic segmentation,” CoRR, 2020. 2, 6

    work page 2020

  23. [23]

    Pidnet: A real-time semantic segmen- tation network inspired from pid controller,

    Jiacong Xu, Zixiang Xiong, and Shankar P Bhat- tacharyya, “Pidnet: A real-time semantic segmen- tation network inspired from pid controller,” arXiv preprint arXiv:2206.02066, 2022. 2, 6

    work page arXiv 2022

  24. [24]

    Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding,

    Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla, “Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding,” in British Machine Vision Confer- ence 2017, BMVC 2017, London, UK, September 4-7,

    work page 2017

  25. [25]

    2017, BMV A Press. 2

    work page 2017

  26. [26]

    What uncertainties do we need in bayesian deep learning for computer vi- sion?,

    Alex Kendall and Yarin Gal, “What uncertainties do we need in bayesian deep learning for computer vi- sion?,” Advances in neural information processing systems, vol. 30, 2017. 2, 5

    work page 2017

  27. [27]

    Uncertainty estimation using a sin- gle deep deterministic neural network,

    Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal, “Uncertainty estimation using a sin- gle deep deterministic neural network,” in Interna- tional conference on machine learning. PMLR, 2020, pp. 9690–9700. 2

    work page 2020

  28. [28]

    Deep deterministic uncer- tainty for semantic segmentation,

    Jishnu Mukhoti, Joost van Amersfoort, Philip HS Torr, and Yarin Gal, “Deep deterministic uncer- tainty for semantic segmentation,” arXiv preprint arXiv:2111.00079, 2021. 2

    work page arXiv 2021

  29. [29]

    Efficient uncertainty estima- tion for semantic segmentation in videos,

    Po-Yu Huang, Wan-Ting Hsu, Chun-Yueh Chiu, Ting- Fan Wu, and Min Sun, “Efficient uncertainty estima- tion for semantic segmentation in videos,” inProceed- ings of the European Conference on Computer Vision (ECCV), 2018, pp. 520–535. 2

    work page 2018

  30. [30]

    Bayesian deep basis fitting for depth completion with uncertainty,

    Chao Qu, Wenxin Liu, and Camillo J Taylor, “Bayesian deep basis fitting for depth completion with uncertainty,” in Proceedings of the IEEE/CVF in- ternational conference on computer vision , 2021, pp. 16147–16157. 2

    work page 2021

  31. [31]

    Variational learning in nonlinear gaussian belief networks,

    Brendan J Frey and Geoffrey E Hinton, “Variational learning in nonlinear gaussian belief networks,” Neu- ral Computation, vol. 11, no. 1, pp. 193–213, 1999. 2

    work page 1999

  32. [32]

    Lightweight proba- bilistic deep networks,

    Jochen Gast and Stefan Roth, “Lightweight proba- bilistic deep networks,” CoRR, 2018. 2, 4

    work page 2018

  33. [33]

    kevin Murphey, Machine learning, a probabilistic perspective, MIT Press, Cambridge, MA, 2012. 3

    work page 2012

  34. [34]

    A simple baseline for bayesian uncertainty in deep learn- ing,

    Wesley J. Maddox, Pavel Izmailov, Timur Garipov, Dmitry P. Vetrov, and Andrew Gordon Wilson, “A simple baseline for bayesian uncertainty in deep learn- ing,” in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion Processing Systems 2019, NeurIPS 2019, Decem- ber 8-14, 2019, Vancouver, BC, Canada , Hanna M. Walla...

    work page 2019

  35. [35]

    On the existence of a normal approximation to the distribu- tion of the ratio of two independent normal random variables,

    Elo ´ısa D´ıaz-Franc´es and Francisco J Rubio, “On the existence of a normal approximation to the distribu- tion of the ratio of two independent normal random variables,” Statistical Papers, vol. 54, no. 2, pp. 309– 323, 2013. 4

    work page 2013

  36. [36]

    Understanding measures of uncertainty for adversarial example detection,

    Lewis Smith and Yarin Gal, “Understanding measures of uncertainty for adversarial example detection,” in Uncertainty in AI, 2018. 5

    work page 2018

  37. [37]

    Training region-based object detectors with online hard example mining,

    Abhinav Shrivastava, Abhinav Gupta, and Ross Gir- shick, “Training region-based object detectors with online hard example mining,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 761–769. 5

    work page 2016

  38. [38]

    Pp-liteseg: A superior real- time semantic segmentation model,

    Juncai Peng, Yi Liu, Shiyu Tang, Yuying Hao, Lutao Chu, Guowei Chen, Zewu Wu, Zeyu Chen, Zhiliang Yu, Yuning Du, et al., “Pp-liteseg: A superior real- time semantic segmentation model,” arXiv preprint arXiv:2204.02681, 2022. 6

    work page arXiv 2022

  39. [39]

    Uncertainty reduction for model adaptation in semantic segmentation,

    Francois Fleuret et al., “Uncertainty reduction for model adaptation in semantic segmentation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9613–9623. 7

    work page 2021

  40. [40]

    Uncertainty-aware pseudo label refinery for domain adaptive semantic segmentation,

    Yuxi Wang, Junran Peng, and ZhaoXiang Zhang, “Uncertainty-aware pseudo label refinery for domain adaptive semantic segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 9092–9101. 7

    work page 2021

  41. [41]

    Uncertainty-aware consistency regularization for cross-domain semantic segmentation,

    Qianyu Zhou, Zhengyang Feng, Qiqi Gu, Guangliang Cheng, Xuequan Lu, Jianping Shi, and Lizhuang Ma, “Uncertainty-aware consistency regularization for cross-domain semantic segmentation,” Com- puter Vision and Image Understanding , vol. 221, pp. 103448, 2022. 7

    work page 2022

  42. [42]

    Morris H DeGroot, Optimal statistical decisions, John Wiley & Sons, 2005. 8

    work page 2005

  43. [43]

    Mykel J Kochenderfer, Tim A Wheeler, and Kyle H Wray, Algorithms for decision making , MIT press,

    work page