VEELA: A Clinically-Constrained Benchmark for Liver Vessel Segmentation in Computed Tomography Angiography
Pith reviewed 2026-05-22 06:25 UTC · model grok-4.3
The pith
A new liver vessel dataset with visibility-only annotations shows that multiple complementary metrics are essential to assess clinically meaningful segmentation.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
VEELA establishes a clinically-constrained benchmark by curating 40 CTA scans with manual slice-by-slice annotations under a strict visibility-driven policy without anatomically inferred interpolation, and demonstrates through complementary metrics that different evaluation approaches capture distinct aspects of vascular integrity, underscoring the need for multi-perspective assessment in vessel segmentation.
What carries the argument
The VEELA dataset created with a strict visibility-driven annotation policy that delineates only visible structures without interpolation, paired with a multi-metric benchmarking framework using clDice, IoU, NSD, and area/length measures.
If this is right
- Segmentation algorithms must be tested on both topological continuity and boundary accuracy to ensure they produce complete and usable vascular maps.
- Benchmarking on VEELA allows direct comparison with prior CHAOS challenge results for hepatic and portal vessel tasks.
- Datasets using visibility-only rules will expose peripheral vessel ambiguities that affect downstream clinical applications like surgical planning.
Where Pith is reading between the lines
- Models trained under these annotation constraints may handle real-world scanner variability more reliably than those trained on interpolated labels.
- Extending visibility-driven curation to other vascular territories could improve consistency across medical imaging benchmarks.
- Clinical workflows might adopt multi-metric dashboards instead of single scores when reviewing automated vessel segmentations.
Load-bearing premise
A strict visibility-driven annotation policy without anatomically inferred interpolation produces labels that better reflect clinical reality and imaging uncertainty.
What would settle it
A study in which practicing radiologists or surgeons rate the practical utility of segmentations and show that performance on a single metric such as IoU predicts clinical value as well as or better than the full set of complementary metrics.
Figures
read the original abstract
Accurate segmentation of hepatic and portal vessels in contrast-enhanced computed tomography angiography (CTA) remains challenging due to complex vascular topology, peripheral visibility limitations, and acquisition-induced ambiguities. While existing public datasets offer valuable benchmarks, few include clinically realistic annotation constraints. We introduce VEELA (Vessel Extraction and Extrication for Liver Analysis), a rigorously curated liver vessel dataset derived from 40 CTA scans inherited from the CHAOS grand-challenge cohort. All vessels were manually delineated slice-by-slice under multi-expert consensus, using a strict visibility-driven annotation policy and avoiding anatomically inferred interpolation. This design explicitly captures anatomical variability and imaging-related uncertainty. As a continuation of the CHAOS challenge, VEELA enables reproducible cross-benchmark evaluation while extending the scope to fine-grained hepatic and portal vessel segmentation. We further establish a standardized benchmarking framework and analyze complementary evaluation metrics, including topology-aware (clDice), overlap-based (IoU), boundary-sensitive (NSD), and geometry-aware (area, length) measures. Our results demonstrate that different metrics capture distinct aspects of vascular integrity, underscoring the necessity of multi-perspective evaluation for clinically meaningful vessel segmentation. VEELA is publicly released to facilitate reproducible research and support the development of robust vascular segmentation methods. Researchers can access the evaluation metrics, dataset, and submission platform at https://www.synapse.org/Synapse:syn65471967.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces VEELA, a dataset of 40 CTA scans for liver vessel segmentation, annotated using a strict visibility-driven policy without anatomically inferred interpolation under multi-expert consensus. It extends the CHAOS challenge and provides a benchmarking framework analyzing multiple metrics including clDice, IoU, NSD, and geometry-aware measures, demonstrating their complementarity for clinically meaningful evaluation.
Significance. If validated, the dataset offers a clinically realistic benchmark that accounts for imaging uncertainties in vascular structures, potentially leading to more robust segmentation algorithms. The multi-metric analysis underscores the need for comprehensive evaluation beyond single overlap measures.
major comments (2)
- Abstract: The assertion that the strict visibility-driven annotation policy without interpolation better reflects clinical reality and uncertainty is central to the paper's contribution but lacks supporting evidence such as comparisons to interpolation-based annotations or radiologist validation studies.
- Abstract: Quantitative results, error analysis, or validation of the annotation consensus process are not provided, making it difficult to assess the reliability of the dataset and the claims about metric complementarity.
minor comments (2)
- Consider adding a dedicated section detailing the annotation protocol, including how multi-expert consensus was reached and any specific guidelines for visibility assessment.
- The manuscript could include more information on the dataset split for training/validation/testing to facilitate reproducible benchmarking.
Simulated Author's Rebuttal
We sincerely thank the referee for their detailed and constructive comments on our manuscript introducing the VEELA dataset. Their feedback highlights important aspects regarding the validation of our annotation policy and the presentation of results. We address each major comment below and outline the revisions we plan to make.
read point-by-point responses
-
Referee: Abstract: The assertion that the strict visibility-driven annotation policy without interpolation better reflects clinical reality and uncertainty is central to the paper's contribution but lacks supporting evidence such as comparisons to interpolation-based annotations or radiologist validation studies.
Authors: We acknowledge that the manuscript would benefit from additional justification for this claim. The visibility-driven policy was chosen based on consultations with clinical experts to mimic real-world annotation practices where uncertain or invisible vessel segments are not annotated to avoid introducing errors. While we do not present new comparative studies or dedicated radiologist validation experiments in the current work, we will expand the discussion section to include references to clinical literature supporting this approach and provide qualitative examples from the dataset illustrating the differences. We will also revise the abstract to more precisely state the motivation without overstating the evidence. revision: partial
-
Referee: Abstract: Quantitative results, error analysis, or validation of the annotation consensus process are not provided, making it difficult to assess the reliability of the dataset and the claims about metric complementarity.
Authors: We agree that including quantitative validation of the annotation process would enhance the manuscript's credibility. The full paper describes the multi-expert consensus procedure in detail in the Methods section. We will add an inter-annotator agreement analysis, such as average Dice coefficients between the three experts, and include error analysis examples in a new subsection. Additionally, we will provide more quantitative results supporting the complementarity of the metrics in the results section. These additions will be incorporated in the revised version. revision: yes
Circularity Check
No circularity: benchmark and dataset release with no derivations or fitted predictions
full rationale
The manuscript is a data release and benchmark definition paper. It introduces VEELA as a curated CTA dataset with a visibility-driven annotation policy and evaluates standard metrics (clDice, IoU, NSD, geometric measures) for complementarity. No equations, parameter fitting, predictions, or self-citation chains appear in the central claims; the annotation policy is presented as an explicit design choice rather than a derived result. The work is self-contained against external benchmarks and contains no load-bearing steps that reduce to inputs by construction.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Standard domain assumption that accurate manual vessel delineation under visibility constraints improves clinical relevance of segmentation benchmarks.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
strict visibility-driven annotation policy and avoiding anatomically inferred interpolation... captures anatomical variability and imaging-related uncertainty
-
IndisputableMonolith/Foundation/AlexanderDuality.leanalexander_duality_circle_linking unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
complementary evaluation metrics, including topology-aware (clDice), overlap-based (IoU), boundary-sensitive (NSD), and geometry-aware (area, length) measures
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Computers in Biology and Medicine , volume=
Selver, M Alper and Kocao. Computers in Biology and Medicine , volume=. 2008 , publisher=
work page 2008
-
[2]
Selle, Dirk and Preim, Bernhard and Schenk, Andrea and Peitgen, H-O , journal=. 2002 , publisher=
work page 2002
-
[3]
Fasel, JH and Selle, Dirk and Evertsz, CJ and Terrier, Francois and Peitgen, HO and Gailloud, Philippe , journal=. 1998 , doi=
work page 1998
-
[4]
Balogh, Julius and Victor III, David and Asham, Emad H and Burroughs, Sherilyn Gordon and Boktour, Maha and Saharia, Ashish and Li, Xian and Ghobrial, R Mark and Monsour Jr, Howard P , journal=. 2016 , publisher=
work page 2016
-
[5]
Reitinger, Bernhard and Bornik, Alexander and Beichel, Reinhard and Schmalstieg, Dieter , journal=. 2006 , publisher=
work page 2006
-
[6]
Abdel-Misih, Sherif RZ and Bloomston, Mark , journal=. 2010 , publisher=
work page 2010
-
[7]
Lehmann, Kai S and Ritz, Joerg-P and Valdeig, Steffi and Schenk, Andrea and Holmer, Christoph and Peitgen, Heinz-O and Buhr, Heinz-J and Frericks, Bernd B , journal=. 2008 , publisher=
work page 2008
-
[8]
Conversano, Francesco and Franchini, Roberto and Demitri, Christian and Massoptier, Laurent and Montagna, Francesco and Maffezzoli, Alfonso and Malvasi, Antonio and Casciaro, Sergio , journal=. 2011 , publisher=
work page 2011
-
[9]
Wong, Humberto and Desser, Terry S and Jeffrey, R Brooke , journal=. 2008 , publisher=
work page 2008
-
[10]
Murphy, DJ and Aghayev, A and Steigner, ML , journal=. 2018 , publisher=
work page 2018
-
[11]
Chen, WP and Chen, JH and Hwang, JI and Tsai, JW and Chen, JS and Hung, SW and Su, YG and Lee, SK , journal=. 1999 , publisher=
work page 1999
-
[12]
Fedorov, Andriy and Beichel, Reinhard and Kalpathy-Cramer, Jayashree and Finet, Julien and Fillion-Robin, Jean-Christophe and Pujol, Sonia and Bauer, Christian and Jennings, Dominique and Fennessy, Fiona and Sonka, Milan and Buatti, John and Aylward, Stephen and Miller, James V and Pieper, Steve and Kikinis, Ron , journal=. 2012 , publisher=
work page 2012
- [13]
-
[14]
IEEE Transactions on Medical Imaging , volume=
Geg. IEEE Transactions on Medical Imaging , volume=. 2011 , publisher=
work page 2011
- [15]
-
[16]
Frangi, Alejandro F and Niessen, Wiro J and Vincken, Koen L and Viergever, Max A , booktitle=. 1998 , organization=
work page 1998
-
[17]
IEEE Transactions on Medical Imaging , volume=
Jerman, Tim and Pernu. IEEE Transactions on Medical Imaging , volume=. 2016 , publisher=
work page 2016
- [18]
-
[19]
Dhanachandra, Nameirakpam and Manglem, Khumanthem and Chanu, Yambem Jina , journal=. 2015 , publisher=
work page 2015
-
[20]
Lesage, David and Angelini, Elsa D and Bloch, Isabelle and Funka-Lea, Gareth , journal=. 2009 , publisher=
work page 2009
-
[21]
Jiang, Huiyan and He, Baochun and Fang, Di and Ma, Zhiyuan and Yang, Benqiang and Zhang, Libo , journal=. 2013 , publisher=
work page 2013
-
[22]
Medical Image Analysis , volume=
Kavur, A Emre and Gezer, N Sinem and Bar. Medical Image Analysis , volume=. 2021 , publisher=
work page 2021
-
[23]
Eli Gibson and Wenqi Li and Carole Sudre and Lucas Fidon and Dzhoshkun I. Shakir and Guotai Wang and Zach Eaton-Rosen and Robert Gray and Tom Doel and Yipeng Hu and Tom Whyntie and Parashkev Nachev and Marc Modat and Dean C. Barratt and Sébastien Ourselin and M. Jorge Cardoso and Tom Vercauteren , journal=. 2018 , issn=
work page 2018
-
[24]
Xu, Minfeng and Wang, Yu and Chi, Ying and Hua, Xiansheng , booktitle=. 2020 , doi=
work page 2020
-
[25]
Kuang, Haopeng and Yang, Dingkang and Wang, Shunli and Wang, Xiaoying and Zhang, Lihua , booktitle=. 2023 , organization=
work page 2023
-
[26]
Soler, Luc and Hostettler, Alexandre and Agnus, Vincent and Charnoz, Arnaud and Fasquel, J and Moreau, Johan and Osswald, A and Bouhadjar, Mourad and Marescaux, Jacques , journal=
-
[27]
Zbinden, Lukas and Catucci, Damiano and Suter, Yannick and Berzigotti, Annalisa and Ebner, Lukas and Christe, Andreas and Obmann, Verena Carola and Sznitman, Raphael and Huber, Adrian Thomas , journal=. 2022 , publisher=
work page 2022
-
[28]
Yagis, Ekin and Aslani, Shahab and Jain, Yashvardhan Susheel and Zhou, Yang and Rahmani, Shahrokh and Brunet, Joseph and Bellier, Alexandre and Werlein, Christopher and Ackermann, Maximilian and Jonigk, Danny and Tafforeau, Paul and Lee, Peter D and Walsh, Claire L , journal=. 2024 , publisher=
work page 2024
-
[29]
Zeng, Ye Zhan and Zhao, Yu Qian and Liao, Miao and Zou, Bei Ji and Wang, Xiao Fang and Wang, Wei , journal=. 2016 , publisher=
work page 2016
-
[30]
Esneault, Simon and Lafon, Cyril and Dillenseger, Jean-Louis , journal=. 2009 , publisher=
work page 2009
-
[31]
Pak, Linda M and Chakraborty, Jayasree and Gonen, Mithat and Chapman, William C and Do, Richard KG and Koerkamp, Bas Groot and Verhoef, Kees and Lee, Ser Yee and Massani, Marco and van der Stok, Eric P and Marcaccia, Marco and Amisano, Marco and Simpson, Amber L and Jarnagin, William R , journal=. 2018 , publisher=
work page 2018
-
[32]
Wu, Mian and Qian, Yinling and Liao, Xiangyun and Wang, Qiong and Heng, Pheng-Ann , journal=. 2023 , publisher=
work page 2023
-
[33]
Chen, Wen and Zhao, Liang and Bian, Rongrong and Li, Qingzhou and Zhao, Xueting and Zhang, Ming , journal=. 2024 , publisher=
work page 2024
-
[34]
Sobotka, Daniel and Herold, Alexander and Perkonigg, Matthias and Beer, Lucian and Bastati, Nina and Sablatnig, Alina and Ba-Ssalamah, Ahmed and Langs, Georg , journal=. 2024 , publisher=
work page 2024
-
[35]
Liu, Zhe and Teng, Qiaoying and Song, Yuqing and Hao, Wen and Liu, Yi and Zhu, Yan and Li, Yuefeng , journal=. 2024 , publisher=
work page 2024
-
[36]
Zhou, Yinghong and Zheng, Yu and Tian, Yinfeng and Bai, Youfang and Cai, Nian and Wang, Ping , journal=. 2024 , publisher=
work page 2024
-
[37]
Li, Shengwei and Li, Xiao-Guang and Zhou, Fanyu and Zhang, Yumeng and Bie, Zhixin and Cheng, Lin and Peng, Jinzhao and Li, Bin , journal=. 2024 , publisher=
work page 2024
-
[38]
Garret, Guillaume and Vacavant, Antoine and Frindel, Carole , booktitle=. 2024 , organization=
work page 2024
-
[39]
Chen, Huai and Wang, Xiuying and Li, Hui and Wang, Lisheng , journal=. 2024 , publisher=
work page 2024
-
[40]
Hatamizadeh, Ali and Nath, Vishwesh and Tang, Yucheng and Yang, Dong and Roth, Holger R and Xu, Daguang , booktitle=. 2021 , organization=
work page 2021
-
[41]
Image and Vision Computing , volume=
Yaz. Image and Vision Computing , volume=. 2024 , publisher=
work page 2024
-
[42]
Liu, Ze and Lin, Yutong and Cao, Yue and Hu, Han and Wei, Yixuan and Zhang, Zheng and Lin, Stephen and Guo, Baining , booktitle=. 2021 , doi=
work page 2021
-
[43]
Isensee, Fabian and Jaeger, Paul F and Kohl, Simon AA and Petersen, Jens and Maier-Hein, Klaus H , journal=. 2021 , publisher=
work page 2021
-
[44]
International Challenge on Cross-Modality Domain Adaptation for Medical Image Segmentation , pages=
Yaz. International Challenge on Cross-Modality Domain Adaptation for Medical Image Segmentation , pages=. 2023 , publisher=
work page 2023
-
[45]
Gao, Zhan and Zong, Qiuhao and Wang, Yiqi and Yan, Yan and Wang, Yuqing and Zhu, Ning and Zhang, Jin and Wang, Yunfu and Zhao, Liang , journal=. 2023 , publisher=
work page 2023
-
[46]
and Litjens, Geert and Menze, Bjoern and Ronneberger, Olaf and Summers, Ronald M
Antonelli, Michela and Reinke, Annika and Bakas, Spyridon and Farahani, Keyvan caps and Kopp-Schneider, Annette and Landman, Bennett A. and Litjens, Geert and Menze, Bjoern and Ronneberger, Olaf and Summers, Ronald M. and van Ginneken, Bram and Bilello, Michel and Bilic, Patrick and Christ, Patrick F. and Do, Richard K. G. and Gollub, Marc J. and Heckers,...
work page 2022
-
[47]
Maier-Hein, Lena and Reinke, Annika and Godau, Patrick and Tizabi, Minu D. and B. Nature Methods , volume=. 2024 , publisher=
work page 2024
-
[48]
Xu, Jichen and Dong, Anqi and Yang, Yang and Jin, Shuo and Zeng, Jianping and Xu, Zhengqing and Jiang, Wei and Zhang, Liang and Dong, Jiahong and Wang, Bo , journal=. 2025 , publisher=
work page 2025
-
[49]
International Conference on Medical Image Computing and Computer-Assisted Intervention , pages=. 2016 , organization=
work page 2016
-
[50]
Lim, Kyoung Yoon and Ko, Jae Eun and Hwang, Yoo Na and Lee, Sang Goo and Kim, Sung Min , journal=. 2025 , publisher=
work page 2025
-
[51]
Wittmann, Bastian and Wattenberg, Yannick and Amiranashvili, Tamaz and Shit, Suprosanna and Menze, Bjoern , booktitle=. 2025 , doi=
work page 2025
-
[52]
Chen, Jieneng and Lu, Yongyi and Yu, Qihang and Luo, Xiangde and Xie, Yutong and Liu, Yingtai and Yan, Yan and Chu, Lawrence and Zhu, Jiancheng and Zhou, Yuyin and Yuille, Alan and Zhou, Chang-An , journal=. 2024 , publisher=
work page 2024
-
[53]
Huang, Qing and Sun, Jinfeng and Ding, Hui and Wang, Xiaodong and Wang, Guangzhi , journal=. 2018 , doi=
work page 2018
-
[54]
Xu, Jichen and Jiang, Wei and Wu, Jiayi and Zhang, Wei and Zhu, Zhenyu and Xin, Jingmin and Zheng, Nanning and Wang, Bo , journal=. 2024 , doi=
work page 2024
-
[55]
Cavicchioli, Matteo and Moglia, Andrea and Garret, Guillaume and Puglia, Martina and Vacavant, Antoine and Pugliese, Giacomo and Cerveri, Pietro , journal=. 2025 , doi=
work page 2025
-
[56]
Zeng, Yu-Zhe and Zhao, Yu-Qian and Tang, Peng and Liao, Miao and Liang, Yi-Xiong and Liao, Sheng-Hui and Zou, Bei-Ji , journal=. 2017 , doi=
work page 2017
-
[57]
Sangsefidi, Nahid and Foruzan, Amir Hossein and Dolati, Ali , journal=. 2018 , doi=
work page 2018
-
[58]
Sato, Yoshinobu and Nakajima, Shin and Shiraga, Nobuyuki and Atsumi, Hideki and Yoshida, Shigeyuki and Koller, Thomas and Gerig, Guido and Kikinis, Ron , journal=. 1998 , doi=
work page 1998
-
[59]
Alirr, Omar Ibrahim and Abd Rahni, Ashrani Aizzuddin , journal=. 2023 , doi=
work page 2023
-
[60]
Kitrungrotsakul, Titinunt and Han, Xian-Hua and Iwamoto, Yutaro and Lin, Lanfen and Foruzan, Amir Hossein and Xiong, Wei and Chen, Yen-Wei , journal=. 2019 , doi=
work page 2019
-
[61]
Hao, Wanxing and Zhang, Juntao and Su, Junhui and Song, Yi and Liu, Zhi and Liu, Yaqiong and Qiu, Chengbiao and Han, Kai , journal=. 2022 , doi=
work page 2022
-
[62]
Yan, Qingsen and Wang, Bo and Zhang, Wei and Luo, Chuan and Xu, Wenqi and Xu, Zhengqing and Zhang, Yanning and Shi, Qinfeng and Zhang, Liang and You, Zheng , journal=. 2021 , doi=
work page 2021
-
[63]
and Sekuboyina, Anjany and Ezhov, Ivan and Unger, Alexander and Zhylka, Andrey and Pluim, Josien P
Shit, Suprosanna and Paetzold, Johannes C. and Sekuboyina, Anjany and Ezhov, Ivan and Unger, Alexander and Zhylka, Andrey and Pluim, Josien P. W. and Bauer, Ulrich and Menze, Bjoern H. , booktitle=. 2021 , doi=
work page 2021
-
[64]
Journal of Medical Internet Research , volume=
Nikolov, Stanislav and Blackwell, Sam and Zverovitch, Alexei and Mendes, Ruheena and Livne, Michelle and De Fauw, Jeffrey and Patel, Yojan and Meyer, Clemens and Askham, Harry and Romera-Paredes, Bernadino and Kelly, Christopher and Karthikesalingam, Alan and Chu, Carlton and Carnell, Dawn and Boon, Cheng and D'Souza, Derek and Moinuddin, Syed Ali and Gar...
work page 2021
-
[65]
The Journal of International Medical Research , volume=
Chierici, Andrea and Lareyre, Fabien and Salucki, Benjamin and Iannelli, Antonio and Delingette, Herv. The Journal of International Medical Research , volume=. 2024 , doi=
work page 2024
-
[66]
Zhao, Ziqi and Li, Wentao and Ding, Xiaoyi and Sun, Jianqi and Xu, Lisa X. , journal=. 2025 , doi=
work page 2025
-
[67]
2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP) , pages=
Toprak, Tu. 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP) , pages=. 2025 , doi=
work page 2025
-
[68]
Seker, Mustafa Ege and Kartal, Mustafa Said and Moniri, Ariorad and Ozkan, Orhan , booktitle=. 2025 , organization=
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.