REVIEW 4 major objections 5 minor 32 references
Deep Learning-Based Semantic Segmentation for Real-Time Kidney Imaging and Measurements with Augmented Reality-Assisted Ultrasound
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that pairing real-time deep-learning segmentation with an augmented-reality headset can automate kidney length, width, thickness, and volume measurements from ultrasound with errors of only a few millimeters.
desk verdict Reasonable AR-US integration with a solid segmentation comparison, but the headline measurement claim rests on an undefined 'capsule' mask and no volume validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The measurement machinery is a PCA-aligned bounding box computed from a binary 'capsule' kidney mask. Given the active pixels of the mask, principal component analysis gives the kidney's orientation; the mask is rotated to align with the axes, the minimum and maximum extents define the bounding rectangle, and rotating back yields the length, width, and thickness. These feed the ellipsoid volume formula $V = \frac{\pi}{6} L W T$. The same mask comes from two segmentation classes, cortex and central complex, which the evaluated models (nnU-Net, Segmenter, YOLO v11 with MedSAM or LiteMedSAM) predict per frame.
What would settle it
Run the automatic bounding-box measurement on physician-annotated kidney masks across diverse shapes—normal adult, pediatric, polycystic, and shrunken kidneys—and compare directly with the same clinicians' manual caliper measurements; if length or width errors from perfect masks exceed the few-millimeter target in a substantial fraction of cases, the geometric proxy rather than the segmentation would be the failing link.
Extended reading notes
Core claim
The central claim is that a binary capsule segmentation of the kidney—formed from the cortex and central complex classes produced by the segmentation model—can be turned into clinically usable dimensions by aligning a bounding box to the segmentation's principal axis via PCA, computing length, width, and thickness from that box, and estimating volume with the ellipsoid formula $V = \frac{\pi}{6} L W T$. Feeding this pipeline with segmentation from nnU-Net, Segmenter, or a YOLO-plus-MedSAM cascade yields length, width, and thickness errors of a few millimeters relative to physician measurements on an in-house dataset, with Segmenter fast enough (about 23 ms per frame) for real-time use. The paper further claims that projecting this overlay onto an AR headset and providing two streaming pipelines (wireless for API-connected scanners and wired video capture for any ultrasound machine) makes the automation practical in point-of-care settings. The strongest version of the claim is that automated measurement error is close to clinical interobserver variability even with limited training data and on an out-of-distribution test set.
Load-bearing premise
The load-bearing premise is that the length, width, and thickness a sonographer records with manual calipers can be reproduced by a PCA-aligned bounding box around a binary capsule mask; if clinical measurement conventions do not match what that bounding box measures, the automated numbers will be biased even when the segmentation is perfect.
Editorial extensions
If this is right
- Sonographers could keep their eyes on the patient while live kidney dimensions and volume appear as an overlay, reducing gaze shifts and cognitive load.
- Measurement error in the few-millimeter range would make automated calipers a viable assist for routine kidney volume assessment, with physicians correcting the overlay in seconds.
- Because Segmenter runs at around 23 ms per frame, near-real-time feedback is achievable on current hardware; nnU-Net remains the accuracy reference when speed is less critical.
- The wired video-capture pipeline extends the same AR assistance to any ultrasound machine with video output, not just scanners with a programming interface.
- A public dataset and open models make the pipeline reproducible and directly extendable to training tools for novice sonographers.
Reading between the lines
- If the capsule mask is obtained deterministically from the cortex and central-complex outputs, then measurement error is bottlenecked by segmentation quality; improving transverse-view segmentation should cut width and thickness errors more than length errors.
- The bounding-box proxy ties automated dimensions to the shape of the segmented mask, so kidneys with irregular outlines or poor acoustic windows may need an alternative measurement definition before the pipeline generalizes to pathological cases.
- Because even physician-annotated masks run through the algorithm show roughly 4 mm length error, part of the reported error is the geometric proxy itself rather than the deep-learning models; aligning the measurement protocol with clinical caliper conventions could shrink the gap without retraining.
- A direct user study comparing gaze behavior, task time, and diagnostic confidence with and without the AR overlay would test the paper's claim that the headset reduces cognitive load, which the current evaluation does not measure.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript integrates deep learning based semantic segmentation with augmented reality on HoloLens-2 for kidney ultrasound, proposing two streaming pipelines (wireless API-based and wired video grabber) and an automatic measurement workflow that computes length, width, thickness, and ellipsoid volume. Four segmentation models (nnU-Net, Segmenter, YOLO v11 with MedSAM, and YOLO v11 with LiteMedSAM) are compared on the public OpenUS Kidney Dataset with 5-fold cross-validation, and measurement errors are reported on a 20-image in-house dataset from five volunteers. The authors claim that Segmenter enables real-time inference, that the system automates kidney volumetric measurements, and that measurement errors are close to clinical interobserver variability, with the pipeline released as open source.
Significance. The paper has clear strengths: it uses a public dataset, open-source models, 5-fold cross-validation, and it releases an open-source pipeline, which supports reproducibility. If the measurement pipeline were properly validated, the work would be a useful practical contribution to point-of-care ultrasound and AR-assisted workflows. However, the central claims about automated volumetric measurements and real-time end-to-end operation are currently under-supported: the capsule segmentation used for measurements is never defined, volume accuracy is never evaluated, interobserver variability is not measured on the same protocol, and only model inference time is reported rather than end-to-end latency.
major comments (4)
- [General framework; DL-Based Kidney Segmentation] The 'capsule segmentation' is load-bearing but never defined. The models output two classes, renal cortex and central complex, and the General framework states only that 'The capsule segmentation is used to extract kidney dimensions.' No rule is given for deriving a capsule mask from these two classes, yet Eqs. (2)-(4) are applied to a binary matrix whose provenance is unspecified. Please specify the capsule construction, for example as a union of the two classes, a contour, or a morphologically closed mask, and justify that it agrees with the kidney boundary used by sonographers.
- [Table 2; Measurements evaluations] The Ground Truth row of Table 2 shows that the automatic algorithm applied to physician-annotated segmentations already has mean errors of 4.22 mm for length, 3.84 mm for width, and 4.16 mm for thickness relative to device calipers. The paper attributes this to misalignment between manual point-of-care measurements and post-processed segmentation, but this is precisely the comparison that must be validated, not assumed. Without an interobserver variability measurement on the same in-house protocol, or clearly matched literature values for the same measurement axes, the statement that errors are comparable to interobserver variability is not supported. The standard deviations in the model rows are also large, up to 17 mm, so reporting means alone overstates consistency.
- [Automatic Measurements; Eq. (1)] Volume is the headline output, but no volume error is reported anywhere. Equation (1) combines length, width, and thickness from two views, yet the Measurements evaluations section gives only linear dimension errors. To support the abstract claim of automated kidney volumetric measurements, the authors should report volume errors against a reference, such as manual ellipsoid volume from the same images or a volumetric standard, including bias and limits of agreement. The assumption that the PCA-aligned bounding rectangle in Eqs. (2)-(4) recovers the clinician-defined kidney axes is also asserted rather than tested; the Ground Truth row of Table 2 suggests a systematic offset that should be investigated.
- [Table 1; Data Streaming Between US Device and PC] The real-time claim is based on model inference time only. Table 1 shows that only Segmenter (23.4 ms) clearly meets a typical real-time threshold, while nnU-Net (338 ms) and YOLO with MedSAM (334.6 ms) do not, and YOLO with LiteMedSAM is borderline and variable (76.8 +/- 38.5 ms). More importantly, no end-to-end latency is reported for either pipeline, including acquisition, streaming, pre- and post-processing, and AR rendering. Without this measurement and without a stated real-time criterion, the manuscript's claim of real-time automated volumetric measurements is not established for the complete system.
minor comments (5)
- [Measurements evaluations] The sentence 'nnU-Net and Segmenter achieve higher accuracy for coronal-view images than for coronal images' should read 'than for transverse-view images', as written it is contradictory.
- [General framework] The phrase 'volume volume [5]' contains a duplicated word and should be corrected to 'volume [5]' or 'volume (Eq. 1)'.
- [Automatic Measurements] The sentence 'Points in P are rotated by -theta to align with the axes normal axes' is garbled; please rewrite as 'to align with the principal axes'.
- [Implementation details] The text states that all models were trained and validated on 512 x 512 resolution images, but the native resolution of the OpenUS Kidney Dataset images is not described; please specify the resampling or padding procedure used.
- [Table 2] The label 'Ground Truth' for the row that applies the automatic algorithm to physician-annotated segmentations is ambiguous; consider renaming it to 'Annotated masks + automatic algorithm' or 'Manual segmentations with automatic measurement' to distinguish it from the physician's device measurements.
Circularity Check
No significant circularity: the segmentation and measurement results are benchmarked against external physician annotations and public data; the only self-citations are background reviews and are not load-bearing.
full rationale
The paper's derivation chain is empirical rather than circular. Segmentation models are trained and evaluated on the external OpenUS Kidney Dataset, and measurement accuracy is obtained by comparing PCA-bounding-box dimensions computed from segmentation masks against physician caliper measurements taken directly on the US device (Table 2). No claimed prediction is defined in terms of the target value: the capsule mask, PCA orientation, and bounding-box equations (Eqs. 2-4) are standard geometric post-processing operations, not fitted to the reported errors. The ellipsoid volume formula (Eq. 1) is a standard clinical formula taken from an external reference. The only self-citations (Egger et al. [12], Gsaxner et al. [13]) are background reviews on deep learning and AR in medicine; they do not supply segmentation, measurement, or volume results, so they are not load-bearing. The paper even reports a 'Ground Truth' row where physician-annotated segmentations run through the automatic algorithm produce residual errors of about 4 mm, which is an honest external validation that the algorithm has systematic bias rather than a circular construction. The paper also explicitly acknowledges limitations, including the small in-house dataset, demographic differences from the training set, and the need for large-scale clinical validation, which is consistent with an empirical study. The undefined 'capsule segmentation' in the General framework and the absence of a directly validated volume error are completeness or correctness concerns, not circularity, because no equation or fitted parameter reduces the central claim to its own input.
Assumptions & free parameters
assumptions (5)
- domain assumption OpenUS Kidney Dataset annotations are accurate ground truth for kidney cortex and central complex.
- domain assumption The ellipsoid formula V = pi/6 * L * W * T approximates clinical kidney volume.
- ad hoc to paper A binary mask (referenced as 'capsule segmentation') can be derived from the two output classes (cortex and central complex) to define the kidney boundary for PCA-based measurements.
- ad hoc to paper PCA-aligned bounding rectangle of the binary mask recovers the clinician-defined kidney axes.
- domain assumption In-house physician measurements on the ultrasound device are reliable ground truth for evaluating measurement error.
Cite this review
Pith. "Pith review of Deep Learning-Based Semantic Segmentation for Real-Time Kidney Imaging and Measurements with Augmented Reality-Assisted Ultrasound." pith.science (2026). https://pith.science/paper/V6H74QMH
@misc{pith2026250623721,
author = {Pith},
title = {Pith review of: Deep Learning-Based Semantic Segmentation for Real-Time Kidney Imaging and Measurements with Augmented Reality-Assisted Ultrasound},
year = {2026},
howpublished = {\url{https://pith.science/paper/V6H74QMH}},
note = {Machine review of arXiv:2506.23721}
}
read the original abstract
Ultrasound (US) is widely accessible and radiation-free but has a steep learning curve due to its dynamic nature and non-standard imaging planes. Additionally, the constant need to shift focus between the US screen and the patient poses a challenge. To address these issues, we integrate deep learning (DL)-based semantic segmentation for real-time (RT) automated kidney volumetric measurements, which are essential for clinical assessment but are traditionally time-consuming and prone to fatigue. This automation allows clinicians to concentrate on image interpretation rather than manual measurements. Complementing DL, augmented reality (AR) enhances the usability of US by projecting the display directly into the clinician's field of view, improving ergonomics and reducing the cognitive load associated with screen-to-patient transitions. Two AR-DL-assisted US pipelines on HoloLens-2 are proposed: one streams directly via the application programming interface for a wireless setup, while the other supports any US device with video output for broader accessibility. We evaluate RT feasibility and accuracy using the Open Kidney Dataset and open-source segmentation models (nnU-Net, Segmenter, YOLO with MedSAM and LiteMedSAM). Our open-source GitHub pipeline includes model implementations, measurement algorithms, and a Wi-Fi-based streaming solution, enhancing US training and diagnostics, especially in point-of-care settings.
Figures
Reference graph
Works this paper leans on
-
[1]
Impact of point-of-care ultra- sound on quality of care in clinical practice,
S. Adhikari, R. Amini, L. A. Stolz, and M. Blaivas, “Impact of point-of-care ultra- sound on quality of care in clinical practice,”Reports in Medical Imaging, pp. 81–93, 2014
work page 2014
-
[2]
Cognitive load predicts point-of- care ultrasound simulator performance,
S. Aldekhyl, R. B. Cavalcanti, and L. M. Naismith, “Cognitive load predicts point-of- care ultrasound simulator performance,” Perspectives on medical education , vol. 7, pp. 23–32, 2018
work page 2018
-
[3]
Learning curves in emergency ultrasound education,
D. J. Blehar, B. Barton, and R. J. Gaspari, “Learning curves in emergency ultrasound education,” Academic emergency medicine, vol. 22, no. 5, pp. 574–582, 2015
work page 2015
-
[4]
Head-mounted displays in ultrasound scanning,
J. Havukumpu, J. Takatalo, G. Nyman, and J. H ¨akkinen, “Head-mounted displays in ultrasound scanning,” in Human Computer Interaction: New Developments . Inte- chOpen, 2008
work page 2008
-
[5]
Work-related musculoskeletal disorders in ultrasound: Can you reduce risk?
G. Harrison and A. Harris, “Work-related musculoskeletal disorders in ultrasound: Can you reduce risk?” Ultrasound, vol. 23, no. 4, pp. 224–230, 2015
work page 2015
-
[6]
Kidney ultrasound for nephrol- ogists: a review,
R. K. Singla, M. Kadatz, R. Rohling, and C. Nguan, “Kidney ultrasound for nephrol- ogists: a review,” Kidney Medicine, vol. 4, no. 6, p. 100464, 2022
work page 2022
-
[7]
Interobserver and intraobserver variations in sonographic renal length measurements in children
A. Schlesinger, R. Hernandez, J. Zerin, T. Marks, and R. Kelsch, “Interobserver and intraobserver variations in sonographic renal length measurements in children.” AJR. American journal of roentgenology, vol. 156, no. 5, pp. 1029–1032, 1991
work page 1991
-
[8]
Measuring the kidneys using sonography,
M. White, “Measuring the kidneys using sonography,” Radiologic Technology , vol. 96, no. 1, pp. 20–24, 2024
work page 2024
Show all 32 references
-
[9]
Intraobserver and interobserver variations in sonographic measurements of kidney size in adult volunteers: a comparison of linear measurements and volumetric estimates,
S. Emamian, M. Nielsen, and J. Pedersen, “Intraobserver and interobserver variations in sonographic measurements of kidney size in adult volunteers: a comparison of linear measurements and volumetric estimates,” Acta radiologica, vol. 36, no. 4, pp. 399–401, 1995. 12
1995
-
[10]
How reliable are ultrasound measurements of renal length in adults?
M. Ablett, A. Coulthard, R. Lee, D. Richardson, T. Bellas, J. Owen, M. Keir, and T. Butler, “How reliable are ultrasound measurements of renal length in adults?” The British journal of radiology, vol. 68, no. 814, pp. 1087–1089, 1995
1995
-
[11]
Deep learning in medical image analysis,
D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis,” Annual review of biomedical engineering, vol. 19, no. 1, pp. 221–248, 2017
2017
-
[12]
Medical deep learning—a systematic meta-review,
J. Egger, C. Gsaxner, A. Pepe, K. L. Pomykala, F. Jonske, M. Kurz, J. Li, and J. Kleesiek, “Medical deep learning—a systematic meta-review,” Computer methods and programs in biomedicine, vol. 221, p. 106874, 2022
2022
-
[13]
The hololens in medicine: A systematic review and taxonomy,
C. Gsaxner, J. Li, A. Pepe, Y . Jin, J. Kleesiek, D. Schmalstieg, and J. Egger, “The hololens in medicine: A systematic review and taxonomy,” Medical Image Analysis, vol. 85, p. 102757, 2023
2023
-
[14]
Microsoft hololens 2 in medical and healthcare context: state of the art and future prospects,
A. Palumbo, “Microsoft hololens 2 in medical and healthcare context: state of the art and future prospects,” Sensors, vol. 22, no. 20, p. 7709, 2022
2022
-
[15]
Artificial intelligence-aided ultrasound in renal dis- eases: a systematic review,
X. Liang, M. Du, and Z. Chen, “Artificial intelligence-aided ultrasound in renal dis- eases: a systematic review,” Quantitative Imaging in Medicine and Surgery , vol. 13, no. 6, p. 3988, 2023
2023
-
[16]
Deep learning in kid- ney ultrasound: overview, frontiers, and challenges,
H. J. De Jesus-Rodriguez, M. A. Morgan, and H. Sagreiya, “Deep learning in kid- ney ultrasound: overview, frontiers, and challenges,” Advances in Chronic Kidney Disease, vol. 28, no. 3, pp. 262–269, 2021
2021
-
[17]
A narrative review on the application of artificial intelligence in renal ultra- sound,
T. Xu, X.-Y . Zhang, N. Yang, F. Jiang, G.-Q. Chen, X.-F. Pan, Y .-X. Peng, and X.-W. Cui, “A narrative review on the application of artificial intelligence in renal ultra- sound,” Frontiers in Oncology, vol. 13, p. 1252630, 2024
2024
-
[18]
Analysis of kidney ultrasound images using deep learn- ing and machine learning techniques: A review,
M. George and H. Anita, “Analysis of kidney ultrasound images using deep learn- ing and machine learning techniques: A review,” Pervasive Computing and Social Networking: Proceedings of ICPCSN 2021 , pp. 183–199, 2022
2021
-
[19]
Application of visual transformer in renal image analysis,
Y . Yin, Z. Tang, and H. Weng, “Application of visual transformer in renal image analysis,” BioMedical Engineering OnLine, vol. 23, no. 1, p. 27, 2024
2024
-
[20]
The growing role for semantic segmentation in urology,
J. Rickman, G. Struyk, B. Simpson, B. C. Byun, and N. Papanikolopoulos, “The growing role for semantic segmentation in urology,”European Urology F ocus, vol. 7, no. 4, pp. 692–695, 2021
2021
-
[21]
The use of virtual reality and augmented reality in ultra- sound education, a narrative review of the literature,
T. Saliba and S. Pather, “The use of virtual reality and augmented reality in ultra- sound education, a narrative review of the literature,”Journal of Clinical Ultrasound, vol. 53, no. 2, pp. 315–324, 2025. 13
2025
-
[22]
Augmented reality visualiza- tion for ultrasound-guided interventions: a pilot randomized crossover trial to assess trainee performance and cognitive load,
S.-C. Liao, S.-C. Shao, S.-Y . Gao, and E. C.-C. Lai, “Augmented reality visualiza- tion for ultrasound-guided interventions: a pilot randomized crossover trial to assess trainee performance and cognitive load,” BMC Medical Education , vol. 24, no. 1, p. 1058, 2024
2024
-
[23]
The open kidney ultrasound data set,
R. Singla, C. Ringstrom, G. Hu, V . Lessoway, J. Reid, C. Nguan, and R. Rohling, “The open kidney ultrasound data set,” in International Workshop on Advances in Simplifying Medical Ultrasound. Springer, 2023, pp. 155–164
2023
-
[24]
nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,
F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,” Nature methods, vol. 18, no. 2, pp. 203–211, 2021
2021
-
[25]
Segmenter: Transformer for se- mantic segmentation,
R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Transformer for se- mantic segmentation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 7262–7272
2021
-
[26]
Ultralytics YOLO,
G. Jocher, J. Qiu, and A. Chaurasia, “Ultralytics YOLO,” Jan. 2023. [Online]. Available: https://github.com/ultralytics/ultralytics
2023
-
[27]
Segment anything in medical images,
J. Ma, Y . He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,” Nature Communications, vol. 15, no. 1, p. 654, 2024
2024
-
[28]
Rep-medsam: Towards real-time and universal medical image segmentation,
M. Wei, S. Chen, S. Wu, and D. Xu, “Rep-medsam: Towards real-time and universal medical image segmentation,” in CVPR 2024: Segment Anything In Medical Images On Laptop, 2024
2024
-
[29]
nnu-net revisited: A call for rigorous validation in 3d medical image segmen- tation,
F. Isensee, T. Wald, C. Ulrich, M. Baumgartner, S. Roy, K. Maier-Hein, and P. F. J¨ager, “nnu-net revisited: A call for rigorous validation in 3d medical image segmen- tation,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, M. G. Linguraru, Q. Dou...
2024
-
[30]
A simple method to estimate renal volume from computed tomography,
R. H. Breau, E. Clark, B. Bruner, P. Cervini, T. Atwell, G. Knoll, and B. C. Leibovich, “A simple method to estimate renal volume from computed tomography,” Can Urol Assoc J., pp. 7(5–6):189–192, 2013
2013
-
[31]
Measures of the amount of ecologic association between species,
L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945. [Online]. Available: http: //www.jstor.org/stable/1932409
1945
-
[32]
Measuring intra-and inter-observer agreement in identifying and localizing 14 structures in medical images,
M. P. Sampat, Z. Wang, M. K. Markey, G. J. Whitman, T. W. Stephens, and A. C. Bovik, “Measuring intra-and inter-observer agreement in identifying and localizing 14 structures in medical images,” in 2006 International Conference on Image Process- ing. IEEE, 2006, pp. 81–84. 15
2006
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.