REVIEW 4 major objections 5 minor 27 references
Multi-Task Learning for Integrated Automated Contouring and Voxel-Based Dose Prediction in Radiotherapy
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper argues that automated contouring and dose prediction in radiotherapy should be trained jointly, reporting DVH error reductions of 19.82% (prostate) and 16.33% (head and neck) versus a sequential automated pipeline.
desk verdict A useful sequential-vs-MTL comparison in automated radiotherapy planning, but capacity is unmatched and the headline gains are overstated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a cross-task attention network: one shared encoder extracts common CT features, while separate output branches predict contours and dose, and specialized attention modules within the encoder let each task weigh the shared features differently. Training balances a mean-absolute-error loss for dose with a combo loss (dice plus cross-entropy) for contours, and a dynamic weight average adjusts the relative loss weights during training. This mechanism is what lets the model output contours and dose simultaneously from CT alone, and it is the component the paper credits for the reported DVH-MAE improvements.
What would settle it
Train a single-task dose-prediction model with the same encoder capacity, pre-trained backbone, and attention machinery as Multi-Automated but without the contouring output branch; if its mean absolute error of the dose-volume histogram (DVH) metrics matches the reported 3.528 Gy and 10.109 Gy values, the improvement is due to model capacity rather than to joint contouring and dose training.
Extended reading notes
Core claim
The paper's central claim is that multi-task learning, rather than a sequential contour-then-plan workflow, is the right way to structure automated radiotherapy planning. In the proposed Multi-Automated model, a single shared encoder processes the CT image and feeds two task-specific output branches, one for organ contours and one for voxel-wise dose, linked by cross-task attention. The authors report that this integrated model outperforms both sequential baselines: it achieves a DVH-MAE of 3.528 Gy versus 4.301 Gy for sequential automated planning on prostate and 10.109 Gy versus 11.650 Gy on head and neck, and it improves average contouring Dice coefficients from 0.818 to 0.824 on prostate and from 0.674 to 0.716 on head and neck. The authors interpret this as evidence that sharing features between contouring and dose prediction lets the model learn anatomical relationships that a dose-only or contour-then-dose model cannot exploit.
Load-bearing premise
The central comparison assumes that the dose-prediction gain is caused by multi-task learning itself, but the multi-task model is a larger, more complex network than the single-task baselines, and no capacity-matched single-task model is tested.
Editorial extensions
If this is right
- A fully automated planning pipeline no longer needs a separate contouring model or pre-computed contours at inference time: CT goes in, contours and dose come out.
- Dose prediction becomes less sensitive to contour quality, because the model is not forced to trust imperfect input contours.
- The reported gains on prostate and head-and-neck datasets suggest the benefit is not site-specific and may transfer to other treatment sites.
- Integrated training can slightly improve contouring as well as dose prediction, so automation need not trade one task against the other.
Reading between the lines
- The paper does not compare Multi-Automated against a single-task dose predictor with the same parameter count or the same pre-trained encoder; until such a comparison is run, part of the reported gain could be attributed to model capacity rather than to multi-task learning.
- Because the OpenKBP CT-only baseline already beat both contour-based sequential models, the benefit of the integrated model there may partly come from its ability to ignore noisy contour labels; corrupting input contours in a controlled experiment would test this.
- The method is evaluated on 2D slices and a limited set of organs; translating it to full 3D plans with more anatomical structures is the natural next test of clinical value.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multi-task learning (MTL) framework that simultaneously performs automated contouring and voxel-based dose prediction for radiotherapy, using a shared encoder, two task-specific decoders, and cross-task attention taken from the authors' prior work [25]. The method is evaluated on an in-house prostate dataset (110 patients) and the public OpenKBP head-and-neck dataset (318 patients after exclusions), and is compared against a CT-only baseline, a sequential model using human contours (Seq-Human), and a sequential model using automated contours (Seq-Automated). The authors report average DVH-MAE improvements of Multi-Automated over Seq-Automated on both datasets, together with contouring Dice and Hausdorff distance results, and conclude that MTL can integrate contouring and dose prediction without compromising contouring accuracy.
Significance. If the central attribution to multi-task learning were established, this would be a practically relevant contribution: an integrated model that takes only CT as input and outputs both contours and dose could simplify the radiotherapy planning workflow and reduce sensitivity to contour quality. The use of the public OpenKBP dataset, the inclusion of both human-contour and automated-contour sequential baselines, and the reporting of two anatomically distinct sites are strengths. However, the manuscript as written does not establish that multi-task learning, rather than increased model capacity, drives the observed improvements; the abstract's headline percentages are not supported by the tables; and the statistical reporting is incomplete. The paper is a reasonable empirical study but requires substantial revision before the central claim can be accepted.
major comments (4)
- [Abstract; Section 3.1, Tables 2 and 3] The headline percentages do not follow from the reported averages. From Tables 2 and 3, Multi-Automated improves over Seq-Automated by (4.301−3.528)/4.301 = 17.97% on prostate and (11.650−10.109)/11.650 = 13.23% on OpenKBP, not the 19.82% and 16.33% stated in the abstract. The relative differences listed in the tables are computed against the Baseline column, not against Seq-Automated, so the abstract appears to use a different denominator. Because these percentages are the paper's primary quantitative claim, they must be corrected and the denominator explicitly defined.
- [Section 2.2.2-2.2.3; Tables 2 and 3] The comparison does not control for model capacity. Multi-Automated uses a shared encoder plus two task-specific decoders and cross-task attention, whereas Seq-Automated and the baseline are single-decoder U-Nets. No parameter-matched single-task dose predictor or MTL ablation is reported, so the observed DVH-MAE gains over Seq-Automated could be due to the larger model rather than to multi-task learning. The OpenKBP result is especially telling: the CT-only single-task Baseline achieves 9.436 Gy, beating Multi-Automated's 10.109 Gy on average, so the integrated model is not universally superior to a strong single-task predictor. A matched-capacity single-task dose model, or an MTL variant without cross-task attention, is required before the central MTL attribution can be accepted.
- [Section 2.3.3; Tables 2-5] Statistical support is underreported. The tables only mark significance with daggers, without reporting p-values, standard deviations, confidence intervals, number of random seeds, or training epochs. With test sizes of 10 (prostate) and 78 (OpenKBP) and many DVH metrics, the absence of uncertainty measures makes it impossible to assess the stability of the reported average improvements. In addition, the caption of Table 3 states 'p < 0.5' instead of 'p < 0.05', which appears to be a typo but should be corrected.
- [Section 3.2, Table 4] The claim that contouring accuracy is 'maintained or sometimes even improved' is only partially supported. On the prostate dataset, Multi-Automated's average Hausdorff distance is worse than the baseline (12.049 vs 7.549), and the right femur Hausdorff distance deteriorates significantly (36.441 vs 9.669), while Dice is essentially unchanged. On OpenKBP, Dice improves significantly for several ROIs, but the spinal cord Hausdorff distance worsens. The discussion should report these trade-offs explicitly rather than emphasizing the improvements only.
minor comments (5)
- [Section 2.1] The dataset arithmetic is inconsistent: the text says 328 patients were collected and 22 omitted, but the stated split of 200/40/78 sums to 318, whereas 328−22=306. Please reconcile these numbers and state how the omitted cases are distributed across the splits.
- [Throughout] There are several typographical errors: 'able able' in the abstract, 'respectivley' in the Figure 1 caption, 'radiotherpay' in Table 1, 'coeffcient' in Tables 4 and 5, and the 'p < 0.5' captions in Tables 3 and 5. These should be corrected.
- [Section 2.2.4] Training details are incomplete: the number of epochs, learning-rate schedule, dynamic weight average hyperparameters, and whether results come from a single run or multiple seeds are not reported. A code or data availability statement would also improve reproducibility.
- [Section 1] The paper says the framework is novel 'to the best of our knowledge,' but the architecture is from the authors' prior paper [25] and another multi-task dose-prediction/contouring study [24] is cited. Please state explicitly which components are new in this work, e.g., the systematic sequential-model comparison and dual-site validation, to avoid overclaiming novelty.
- [Section 2.3.1, Table 1] The description of D0.1cc is awkward: it is written as 'Maximum dose received by the smallest volume of 0.1 cc.' Standard D0.1cc is the dose received by 0.1 cc of the structure, not a maximum-dose definition; please rephrase for clarity.
Circularity Check
No circularity: the central MTL-vs-sequential claim is an empirical held-out comparison, not a construction that encodes its own conclusion.
full rationale
The paper's main claim is that Multi-Automated reduces DVH-MAE relative to sequential pipeline models. This is assessed on held-out test patients (10 prostate, 78 OpenKBP) with DVH-MAE computed from predicted versus clinical dose distributions. There is no fitted parameter that by construction equals the reported improvement; the loss functions (MAE, combo loss) and dynamic weight averaging are standard training objectives, not definitions of the evaluation metric. The self-citations to [25] supply the cross-task attention architecture and the baseline model, but the performance comparison is not derived from that citation: it is a direct experimental result, and the OpenKBP evaluation is externally benchmarked. The capacity-mismatch concern (Multi-Automated has an extra decoder and attention versus single-decoder baselines) is a legitimate experimental-design limitation, but it is not an instance of self-definition, fitted-input-as-prediction, or renaming. The abstract's 19.82%/16.33% figures use the CT-only baseline as denominator rather than Seq-Automated, and the paper's Tables 2-3 report different relative differences; this is a reporting issue, not a circular derivation. No load-bearing step reduces, by the paper's own equations or by self-citation, to its own inputs.
Assumptions & free parameters
free parameters (6)
- combo loss weight for dice loss =
0.3
- combo loss weight for cross-entropy loss =
0.7
- dynamic weight average parameters =
not specified
- batch size =
32 (Prostate), 8 (OpenKBP)
- learning rate =
1e-4
- weight decay =
1e-5
assumptions (5)
- domain assumption 2D axial CT slices are sufficient for dose prediction and contouring accuracy.
- domain assumption The four OpenKBP ROIs are consistently and correctly labeled across patients, and the excluded cases do not bias training.
- domain assumption DVH metrics computed using the clinical contours from the datasets are valid ground truth for dose evaluation.
- domain assumption The cross-task attention network from [25] is a valid and correctly instantiated MTL architecture.
- domain assumption Training with the same loss functions and encoder is fair across baseline, sequential, and multi-task models.
Cite this review
Pith. "Pith review of Multi-Task Learning for Integrated Automated Contouring and Voxel-Based Dose Prediction in Radiotherapy." pith.science (2026). https://pith.science/paper/W63XQK63
@misc{pith2026241118767,
author = {Pith},
title = {Pith review of: Multi-Task Learning for Integrated Automated Contouring and Voxel-Based Dose Prediction in Radiotherapy},
year = {2026},
howpublished = {\url{https://pith.science/paper/W63XQK63}},
note = {Machine review of arXiv:2411.18767}
}
read the original abstract
Deep learning-based automated contouring and treatment planning has been proven to improve the efficiency and accuracy of radiotherapy. However, conventional radiotherapy treatment planning process has the automated contouring and treatment planning as separate tasks. Moreover in deep learning (DL), the contouring and dose prediction tasks for automated treatment planning are done independently. In this study, we applied the multi-task learning (MTL) approach in order to seamlessly integrate automated contouring and voxel-based dose prediction tasks, as MTL can leverage common information between the two tasks and be able able to increase the efficiency of the automated tasks. We developed our MTL framework using the two datasets: in-house prostate cancer dataset and the publicly available head and neck cancer dataset, OpenKBP. Compared to the sequential DL contouring and treatment planning tasks, our proposed method using MTL improved the mean absolute difference of dose volume histogram metrics of prostate and head and neck sites by 19.82% and 16.33%, respectively. Our MTL model for automated contouring and dose prediction tasks demonstrated enhanced dose prediction performance while maintaining or sometimes even improving the contouring accuracy. Compared to the baseline automated contouring model with the dice score coefficients of 0.818 for prostate and 0.674 for head and neck datasets, our MTL approach achieved average scores of 0.824 and 0.716 for these datasets, respectively. Our study highlights the potential of the proposed automated contouring and planning using MTL to support the development of efficient and accurate automated treatment planning for radiotherapy.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[25]
Cross-task attention network: Improving multi-task learning for medical imaging applications
Sangwook Kim, Thomas G Purdie, and Chris McIntosh. Cross-task attention network: Improving multi-task learning for medical imaging applications. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 119–128. Springer, 2023
work page 2023
-
[1]
Chris McIntosh, Leigh Conroy, Michael C. Tjong, Tim Craig, Andrew Bayley, Charles Catton, Mary Gospodarowicz, Joelle Helou, Naghmeh Isfahanian, Vickie Kong, Tony Lam, Srinivas Raman, Padraig Warde, Peter Chung, Alejandro Berlin, and Thomas G. Purdie. Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate...
work page 2021
-
[2]
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18 , pages 234–241. Springer, 2015
2015
-
[3]
nnu-net: a self-configuring method for deep learning-based biomedical image segmentation
Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier- Hein. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021
2021
-
[4]
Geometric evaluations of ct and mri based deep learning segmentation for brain oars in radiotherapy
Nouf Alzahrani, Ann Henry, Anna Clark, Louise Murray, Michael Nix, and Bashar Al-Qaisieh. Geometric evaluations of ct and mri based deep learning segmentation for brain oars in radiotherapy. Physics in Medicine & Biology, 68(17):175035, 2023
work page 2023
-
[5]
Ruiyan Ni, Kathy Han, Benjamin Haibe-Kains, and Alexandra Rink. Generaliz- ability of deep learning in organ-at-risk segmentation: A transfer learning study in cervical brachytherapy. Radiotherapy and Oncology, 197:110332, 2024
work page 2024
-
[6]
Dan Nguyen, Troy Long, Xun Jia, Weiguo Lu, Xuejun Gu, Zohaib Iqbal, and Steve Jiang. A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning. Scientific Reports, 9(1):1076, 2019
work page 2019
-
[7]
Aaron Babier, Binghao Zhang, Rafid Mahmood, Kevin L. Moore, Thomas G. Purdie, Andrea L. McNiven, and Timothy C. Y. Chan. OpenKBP: The open- access knowledge-based planning grand challenge and dataset. Medical Physics , 48(9):5549–5561, 2021
work page 2021
Show all 27 references
-
[8]
OpenKBP-Opt: An international and reproducible evaluation of 76 knowledge-based planning pipelines
Aaron Babier, Rafid Mahmood, Binghao Zhang, Victor G L Alves, Ana Maria Barrag´ an-Montero, Joel Beaudry, Carlos E Cardenas, Yankui Chang, Zijie Chen, Jaehee Chun, Kelly Diaz, Harold David Eraso, Erik Faustmann, Sibaji Gaj, Skylar Gay, Mary Gronberg, Bingqi Guo, Junjun He, Ger...
2022
-
[9]
Deep learning–based dose prediction for automated, individualized quality assurance of head and neck radiation therapy plans
Mary P Gronberg, Beth M Beadle, Adam S Garden, Heath Skinner, Skylar Gay, Tucker Netherton, Wenhua Cao, Carlos E Cardenas, Christine Chung, David T Fuentes, et al. Deep learning–based dose prediction for automated, individualized quality assurance of head and neck radiation th...
2023
-
[10]
A transformer-embedded multi-task model for dose distribution prediction
Lu Wen, Jianghong Xiao, Shuai Tan, Xi Wu, Jiliu Zhou, Xingchen Peng, and Yan Wang. A transformer-embedded multi-task model for dose distribution prediction. International Journal of Neural Systems , 33(08):2350043, 2023
2023
-
[11]
A cascade 3d u-net for dose prediction in radiotherapy
Shuolin Liu, Jingjing Zhang, Teng Li, Hui Yan, and Jianfei Liu. A cascade 3d u-net for dose prediction in radiotherapy. Medical physics, 48(9):5574–5582, 2021
2021
-
[12]
3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture
Dan Nguyen, Xun Jia, David Sher, Mu-Han Lin, Zohaib Iqbal, Hui Liu, and Steve Jiang. 3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture. Physics in Medicine & Biology , 64(6):065020, 2019
2019
-
[13]
Flexible- cm gan: Towards precise 3d dose prediction in radiotherapy
Riqiang Gao, Bin Lou, Zhoubing Xu, Dorin Comaniciu, and Ali Kamen. Flexible- cm gan: Towards precise 3d dose prediction in radiotherapy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 715–725, 2023
2023
-
[14]
Diffdp: Radiotherapy dose prediction via a diffusion model
Zhenghao Feng, Lu Wen, Peng Wang, Binyu Yan, Xi Wu, Jiliu Zhou, and Yan Wang. Diffdp: Radiotherapy dose prediction via a diffusion model. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 191–201. Springer, 2023
2023
-
[15]
Dose distribution prediction for head-and-neck cancer radiotherapy using a generative adversarial network: influence of input data
Xiaojin Gu, Victor IJ Strijbis, Ben J Slotman, Max R Dahele, and Wilko F AR Verbakel. Dose distribution prediction for head-and-neck cancer radiotherapy using a generative adversarial network: influence of input data. Frontiers in Oncology, 13, 2023
2023
-
[16]
Multitask learning
Rich Caruana. Multitask learning. Machine learning, 28:41–75, 1997
1997
-
[17]
Generalized multi-task learning from substantially unlabeled multi-source medical image data
Ayaan Haque, Abdullah-Al-Zubaer Imran, Adam Wang, and Demetri Terzopoulos. Generalized multi-task learning from substantially unlabeled multi-source medical image data. arXiv preprint arXiv:2110.13185 , 2021
2021 arXiv
-
[18]
Multi-task deep learning based ct imaging analysis for covid-19 pneumonia: Classification and segmentation
Amine Amyar, Romain Modzelewski, Hua Li, and Su Ruan. Multi-task deep learning based ct imaging analysis for covid-19 pneumonia: Classification and segmentation. Computers in biology and medicine , 126:104037, 2020. REFERENCES 19
2020
-
[19]
Multi-task attention-based semi-supervised learning for medical image segmentation
Shuai Chen, Gerda Bortsova, Antonio Garc ´ ıa-Uceda Ju´ arez, Gijs Van Tulder, and Marleen De Bruijne. Multi-task attention-based semi-supervised learning for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd Internatio...
2019
-
[20]
Deep learning for multi-task medical image segmentation in multiple modalities
Pim Moeskops, Jelmer M Wolterink, Bas HM Van Der Velden, Kenneth GA Gilhuijs, Tim Leiner, Max A Viergever, and Ivana Iˇ sgum. Deep learning for multi-task medical image segmentation in multiple modalities. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 20...
2016
-
[21]
Multi-task deep learning for medical image computing and analysis: A review
Yan Zhao, Xiuying Wang, Tongtong Che, Guoqing Bao, and Shuyu Li. Multi-task deep learning for medical image computing and analysis: A review. Computers in Biology and Medicine , 153:106496, 2023
2023
-
[22]
Conversion of single-energy ct to parametric maps of dual-energy ct using convolutional neural network.British Journal of Radiology, 97(1158):1180– 1190, 2024
Sangwook Kim, Jimin Lee, Jungye Kim, Bitbyeol Kim, Chang Heon Choi, and Seongmoon Jung. Conversion of single-energy ct to parametric maps of dual-energy ct using convolutional neural network.British Journal of Radiology, 97(1158):1180– 1190, 2024
2024
-
[23]
Multimodal radiotherapy dose prediction using a multi-task deep learning model
Austen Maniscalco, Ezek Mathew, David Parsons, Justin Visak, Mona Arbab, Prasanna Alluri, Xingzhe Li, Narine Wandrey, Mu-Han Lin, Asal Rahimi, et al. Multimodal radiotherapy dose prediction using a multi-task deep learning model. Medical physics, 2024
2024
-
[24]
Mask-free radiotherapy dose prediction via multi-task learning
Zhengyang Jiao, Xingchen Peng, Jianghong Xiao, Xi Wu, Jiliu Zhou, and Yan Wang. Mask-free radiotherapy dose prediction via multi-task learning. In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI) , pages 1–5. IEEE, 2022
2022
-
[26]
Loss odyssey in medical image segmentation
Jun Ma, Jianan Chen, Matthew Ng, Rui Huang, Yu Li, Chen Li, Xiaoping Yang, and Anne L Martel. Loss odyssey in medical image segmentation. Medical Image Analysis, 71:102035, 2021
2021
-
[27]
End-to-end multi-task learning with attention
Shikun Liu, Edward Johns, and Andrew J Davison. End-to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1871–1880, 2019
2019
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.