Pith. sign in

REVIEW 4 major objections 6 minor 16 references

Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A learned Bayesian prior over convolutional filters transfers knowledge between MRI segmentation tasks better than pre-trained weights when the target dataset is small.

desk verdict Sensible DWP extension to cross-disease MRI transfer, but absent a random-prior control and significance tests, the transfer claim is not yet proven. read the letter →

arxiv 1908.05480 v1 pith:M4IBQGPW submitted 2019-08-15 eess.IV cs.CVcs.LGstat.ML

Anna Kuzina , Evgenii Egorov , Evgeny Burnaev This is my paper · ORCID
classification eess.IVcs.CVcs.LGstat.ML
keywords MRIsegmentationknowledgetransferDeepWeightPriorBayesianneuralnetworksvariationalautoencoder3DU-Netbraintumormultiplesclerosis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that when labelled MRI data are scarce, the most reliable way to borrow knowledge from a different disease is not to copy the source network's weights but to learn a probability distribution over its convolutional filters and use that distribution as a Bayesian prior. The authors train several 3D U-Nets on multiple-sclerosis lesion segmentation, fit a variational autoencoder to their kernels, and then train brain-tumor U-Nets on 5, 10, 15, or 20 BRATS18 volumes with this Deep Weight Prior. They report that this approach outperforms both random initialization and standard pre-training at every training size, with mean Dice of 0.64–0.74 against 0.61–0.69 for pre-training. If true, it matters because small medical datasets are the norm and conventional pre-training can fail when the source and target diseases look different.

What carries the argument

The carrying object is the Deep Weight Prior (DWP), a variational autoencoder trained on the $3\times3\times3$ kernels of U-Nets trained on the source dataset, used as an implicit prior $p(w)=\int p_\phi(w|z)p(z)\,dz$ over the target network's weights. Because the prior's density is not tractable, the KL-divergence term in the evidence lower bound is replaced by the DWP's upper-bound approximation, and stochastic variational inference with the reparameterization trick updates both the U-Net variational posterior $q_\theta(w)$ and the VAE encoder. Seven VAEs are trained, one for each group of layers that sees the same input resolution. Its job is to pull sampled target kernels toward the structured appearance of source kernels rather than the noisy kernels that overfit small datasets.

What would settle it

Run the same DWP training procedure with the source kernels replaced by random kernels of the same shape, keeping the target experiments on 5 and 10 BRATS18 volumes unchanged. If the Dice scores match the MS-trained DWP, then the improvement does not depend on the source disease; if they fall to the random-initialization level, the cross-disease prior is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that an implicit generative prior over filter weights, learned from a source segmentation network for a different disease, improves target segmentation more than the two standard baselines. On BRATS18 whole-tumor segmentation trained from 5, 10, 15, and 20 volumes, the DWP-trained U-Net reached mean DSC of 0.64, 0.71, 0.71, and 0.74 and IoU of 0.52, 0.58, 0.60, and 0.63; the pre-trained U-Net reached 0.61, 0.64, 0.67, and 0.69 Dice, while random initialization reached 0.62, 0.66, 0.70, and 0.70. The authors conclude that the prior supplies the structure that small training sets cannot, and that directly transferring weights from a different disease is not useful; frozen-layer pre-training was the worst setting. They further report preliminary CT results on liver and spleen segmentation where DWP is better than the baselines in most cases, and that at 100 target volumes all methods converge.

Load-bearing premise

The load-bearing assumption is that convolutional kernels from a network trained to segment multiple-sclerosis lesions share enough structure with kernels useful for brain-tumor segmentation that a VAE learned on the former is a good prior for the latter; the paper runs no control with a prior learned on random or irrelevant filters.

Editorial extensions

If this is right

  • For 5 to 20 training volumes, DWP is the best initialization strategy by Dice and IoU, and the gap over pre-training and random initialization grows as the training set becomes larger.
  • Standard pre-training is not reliably better than random initialization when source and target diseases differ, and freezing middle layers is reliably worse.
  • At 100 target volumes, DWP, pre-training, and random initialization converge to similar performance, so the benefit is specific to small-data settings.
  • The same DWP setup transfers to a different organ setting (liver and spleen CT), suggesting the mechanism is not brain-specific.
  • The authors state the approach is not limited to segmentation and could be applied to classification, detection, or any CNN-based task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: without a control prior trained on random or unrelated filters, the reported gain does not by itself prove cross-disease knowledge transfer; part or all of the improvement could be generic Bayesian regularization.
  • Inference: the design of one VAE per input-resolution group could be tested against a single shared prior or against a Gaussian prior with a learned covariance; such an ablation would show whether the VAE's expressiveness is the reason DWP works.
  • Inference: the method should extend to other transfer axes such as scanner type, acquisition protocol, or organ, and the preliminary liver and spleen results suggest it partially does, though those absolute Dice values are low.
  • Inference: a direct comparison against a prior learned from target-domain kernels would separate "the prior is useful because it is Bayesian" from "the prior is useful because it knows about brains and lesions."
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a knowledge-transfer method for MRI semantic segmentation when the target dataset is small. The authors train 3D U-Nets on a source task (multiple sclerosis lesion segmentation), collect the trained convolutional filters as data, learn a Deep Weight Prior (DWP) in the form of a VAE over those filters, and then train a target 3D U-Net for brain-tumor segmentation (BRATS18) with this implicit prior under a variational-inference objective. Experiments compare UNet-DWP against random initialization, pre-training on the source task, and pre-training with frozen layers, using 5, 10, 15, and 20 training volumes and reporting Dice and IoU. The paper claims that DWP noticeably outperforms the baselines for all training sizes, and it also presents preliminary liver/spleen CT experiments as evidence of general applicability.

Significance. If the central claim holds, the method would be a practically useful way to transfer knowledge between different diseases in medical image segmentation, a setting where conventional fine-tuning often fails. The idea of modeling the filter distribution with a generative prior is interesting and builds on a plausible inductive bias that well-trained kernels share structure. The paper provides a clear algorithmic description, uses public datasets, and reports standard segmentation metrics, which are positive features. However, the current evidence does not isolate the effect of source knowledge from generic Bayesian regularization, and the statistical support for the main claim is weak. The contribution is therefore promising but not yet established; the paper needs additional control experiments and more careful statistical evaluation before the central claim can be accepted.

major comments (4)
  1. [Section 3, Table 3] The statement that 'models trained with DWP noticeably outperformed both randomly initialized and pre-trained U-Net for all the training sizes' is not supported by the reported numbers. For train size 5, UNet-DWP (0.64 ± 0.05) and UNet-RI (0.62 ± 0.02) have overlapping error bars, and the same is true at train size 15 (0.71 ± 0.02 vs 0.70 ± 0.02). With only three random splits, the reported standard deviations do not establish a significant advantage at these sizes. The authors should either weaken the claim to specify the sizes where the separation is clear, or provide more splits and a statistical test to support superiority at all sizes.
  2. [Section 3, Algorithm 3, Eq. (2)] The experimental design does not include a control that isolates the source-specific knowledge. The DWP procedure introduces two coupled changes relative to UNet-RI: the variational Bayesian training with the KL term in Eq. (2), and a prior learned from MS-trained kernels. Without a control in which the same DWP pipeline is run with a prior learned from random filters or from filters of a network trained on an unrelated task, the improvements in Tables 3 and 4 cannot be attributed to knowledge transfer from the source disease; they could be a purely regularizing effect of the Bayesian framework. Such a control is necessary to support the paper's central claim of between-disease transfer.
  3. [Section 2.7.1, Algorithms 2 and 3] The comparison between UNet-DWP and UNet-PR is confounded by the amount of source data used: DWP trains the prior from 20 source networks (Section 2.7.1), while the pre-training baseline uses a single source network (Algorithm 2). This difference should be discussed explicitly. If the goal is to compare transfer methods, the baselines should be matched in the number of source models, or the effect of using an ensemble of source networks should be acknowledged as a separate factor that could explain some of the improvement.
  4. [Section 4.1, Table 5] The additional experiments on Task03_Liver and Task09_Spleen are presented as evidence that the method 'can be successfully applied not only to the brain but also to other organs.' However, these results report single runs without variance, and they are not consistently in favor of DWP (e.g., Task03_Liver at train size 5: UNet-DWP 0.275 vs UNet-RI 0.284). These preliminary outcomes should not be used to support the generalization claim; they should either be removed, clearly labeled as anecdotal, or replaced with experiments that include multiple splits and standard deviations.
minor comments (6)
  1. [Section 2.5] The text defines true positives as 'TR' but the acronym should be 'TP'; the same typo appears in the IoU formula.
  2. [Section 3] In the paragraph after Table 3, 'UNet-WDP' should be 'UNet-DWP' for consistency.
  3. [Section 2.2] 'the task burns down to the maximization' should be 'the task boils down to the maximization'.
  4. [Appendix A] The heading 'Stohastic Variational Inference' contains a typo; it should be 'Stochastic Variational Inference'.
  5. [Section 2.6 and Eq. (2)] The relationship between the segmentation loss L = 0.99 * LDICE + 0.01 * LCE and the ELBO data term in Eq. (2) is not explicitly defined; the authors should clarify how the segmentation loss enters the variational objective (e.g., whether it is used as the reconstruction log-likelihood).
  6. [Section 2.7.2] 'latent vector dimention 6' should be 'latent vector dimension 6'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the transfer prior is trained on a source dataset and evaluated on distinct target data, with no target-derived fitted value entering the derivation.

full rationale

The paper's central comparison is empirical rather than derivational. The Deep Weight Prior is fitted exclusively on kernels collected from U-Net models trained on the MS source dataset (Section 2.7.2), and the target BRATS18 experiments (Section 3, Tables 3 and 4) use no target-data information in constructing that prior. The VAE-based prior is a standard implicit prior from Atanov et al. (2018), which the paper cites rather than re-deriving; the ELBO objective in Eq. (2) is the usual variational bound with the DWP KL upper bound, and no equation in the paper reduces the reported Dice or IoU to a fitted parameter or to the prior by construction. The assumption that kernels from different segmentation networks share similar structure (Section 2.2) is an empirical hypothesis motivating the method, not a self-referential definition. The absence of a control prior trained on random or unrelated filters is a legitimate experimental-validity concern about whether the improvement is due to source-specific knowledge or generic Bayesian regularization, but it is not a circularity of the paper's derivation chain. Self-citations appear only as background references (e.g., prior Skoltech work on neuroimaging) and are not load-bearing for the transfer claim. The result is self-contained against external benchmarks, so the appropriate circularity finding is a non-finding.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the DWP framework inherited from Atanov et al. (2018), on the transferability of filter structure across diseases, and on a fully factorized variational approximation. Several hyperparameters (loss weights, latent dimension, number of VAEs) are chosen without sensitivity analysis. The paper introduces no invented entities; the DWP is a cited prior method.

free parameters (5)
  • CE loss weight (0.01) = 0.01
    Chosen experimentally after CE alone led to background prediction; no sensitivity analysis.
  • DWP VAE latent dimension = 6
    Fixed value; no ablation reported.
  • Number of VAEs (7) = 7
    One per layer group by input resolution; no justification for the grouping.
  • Number of source U-Nets (20) = 20
    10 converged models plus 10 from cyclical learning rate; arbitrary.
  • Network architecture (3D U-Net, 726480 parameters) = designed
    Chosen for popularity; no comparison to other architectures.
assumptions (5)
  • domain assumption The KL upper bound and training procedure of Deep Weight Prior from Atanov et al. (2018) are correct and tight enough for the target task.
    The paper adopts the DWP objective (Eq. 2) without re-deriving it; if the upper bound is loose, the variational optimization may not behave as intended.
  • domain assumption Convolutional filters from a network trained on MS lesion segmentation carry structure useful for brain tumor segmentation.
    Section 2.2 states that kernels from different segmentation networks share similar structure; this is the core transfer premise that the experiments aim to exploit.
  • domain assumption The fully factorized Gaussian variational posterior is a sufficient approximation for the 3D U-Net weights.
    Section 2.2 and Appendix A assume q factorizes over layers and channels; no evidence that this captures the posterior adequately.
  • domain assumption The MS (FLAIR, 448x512x22) and BRATS18 (T2, 240x240x155) datasets are sufficiently comparable after preprocessing for transfer to be meaningful.
    Section 2.4 describes the datasets; differences in modality, resolution, and protocol are known and the paper relies on the shared brain context.
  • standard math Variational inference with the reparametrization trick provides unbiased gradient estimates for the ELBO.
    Used throughout Appendix A as the basis for stochastic variational inference; not re-derived.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems." pith.science (2026). https://pith.science/paper/M4IBQGPW

@misc{pith2026190805480,
  author       = {Pith},
  title        = {Pith review of: Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M4IBQGPW}},
  note         = {Machine review of arXiv:1908.05480}
}
read the original abstract

Automatic segmentation methods based on deep learning have recently demonstrated state-of-the-art performance, outperforming the ordinary methods. Nevertheless, these methods are inapplicable for small datasets, which are very common in medical problems. To this end, we propose a knowledge transfer method between diseases via the Generative Bayesian Prior network. Our approach is compared to a pre-train approach and random initialization and obtains the best results in terms of Dice Similarity Coefficient metric for the small subsets of the Brain Tumor Segmentation 2018 database (BRATS2018).

Figures

Figures reproduced from arXiv: 1908.05480 by the authors.

Figure 1
Figure 1. U-Net architecture used in the experiments contains ResNet-like block both in Encoder and [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Full scheme of the training procedure with Deep Weight Prior. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Example of MRI slices and ground truth segmentation. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Segmentation accuracy on BRATS18 dataset for various train sample size, calculated for three [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Examples of trained kernels. The most popular model for semantic segmentation is volumetric U-Net (Deniz et al., 2018, Livne et al., 2019, Guerrero et al., 2018). The idea behind this architecture is quite simple; it is based on conventional U-Net model for semantic se…
Figure 6
Figure 6. Figure 6: Examples of models’ predictions on test samples, compared to ground truth segmentation [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 9 canonical work pages

  1. [5]

    doi: 10.1038/ s41598-018-34817-6

    ISSN 2045-2322. doi: 10.1038/ s41598-018-34817-6. URL https://www.nature.com/articles/s41598-018-34817-6 . 16 Gamaleldin F Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. Adversarial reprogramming of neural networks. arXiv preprint arXiv:1806.11146,

    arXiv

  2. [11]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 3D Vision (3DV), 2016 Fourth International Conference on, pages 565–571. IEEE,

    work page 2016

  3. [12]

    Deep learning with mixed supervision for brain tumor segmentation

    Pawel Mlynarski, Hervé Delingette, Antonio Criminisi, and Nicholas Ayache. Deep learning with mixed supervision for brain tumor segmentation. arXiv preprint arXiv:1812.04571,

    arXiv

  4. [13]

    3d MRI brain tumor segmentation using autoencoder regularization

    Andriy Myronenko. 3d MRI brain tumor segmentation using autoencoder regularization. arXiv preprint arXiv:1810.11654,

    arXiv

  5. [15]

    Cyclical learning rates for training neural networks

    Leslie N Smith. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 464–472. IEEE,

    work page 2017

  6. [16]

    doi: 10.1016/j.neuroimage.2017.02.035

    ISSN 1053-8119. doi: 10.1016/j.neuroimage.2017.02.035. URL http://www.sciencedirect.com/science/ article/pii/S1053811917301465. Garrett Wilson and Diane J Cook. Adversarial transfer learning. arXiv preprint arXiv:1812.02849,

    arXiv 2017

  7. [17]

    introduces approximate posterior distributionqθ(w) from some parametric family, e.g. fully factorized Gaussian, and solve optimization problem, minimizing Kull- back–Leibler divergence between true posterior distributionp(w|D) and variational approximationqθ(w) with respect to parametersθ. min KL (qθ(w))||p(w|D)). (3) Where Kullback–Leibler divergence, or...

    work page 2015

  8. [1999]

    Brain Tumor Segmentation and Tractographic Feature Extraction from Structural MR Images for Overall Survival Prediction

    Po-Yu Kao, Thuyen Ngo, Angela Zhang, Jefferson W Chen, and BS Manjunath. Brain tumor segmentation and tractographic feature extraction from structural mr images for overall survival prediction. arXiv preprint arXiv:1807.07716,

    work page Pith review arXiv

Show all 16 references
  1. [2002]

    doi: 10.1016/S1361-8415(02)00056-7

    ISSN 1361-8415. doi: 10.1016/S1361-8415(02)00056-7. URL http: //www.sciencedirect.com/science/article/pii/S1361841502000567. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medi...

    doi

  2. [2012]

    URL https://www.hindawi

    doi: 10.1155/2012/898430. URL https://www.hindawi. com/journals/cmmm/2012/898430/. Markus A Degel, Nassir Navab, and Shadi Albarqouni. Domain and geometry agnostic CNNs for left atrium segmentation in 3d ultrasound. arXiv preprint arXiv:1805.00357,

    2012 arXiv

  3. [2013]

    nnu-net: Self-adapting framework for u-net- based medical image segmentation

    Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, et al. nnu-net: Self-adapting framework for u-net- based medical image segmentation. arXiv preprint arXiv:1809.10486,

    arXiv

  4. [2015]

    Adversarial networks for the detection of aggres- sive prostate cancer

    Simon Kohl, David Bonekamp, Heinz-Peter Schlemmer, Kaneschka Yaqubi, Markus Hohenfellner, Boris Hadaschik, Jan-Philipp Radtke, and Klaus Maier-Hein. Adversarial networks for the detection of aggres- sive prostate cancer. arXiv preprint arXiv:1702.08014,

    arXiv

  5. [2016]

    SUNet: a deep learning architecture for acute stroke lesion segmentation and outcome prediction in multimodal mri

    Albert Clèrigues, Sergi Valverde, Jose Bernal, Jordi Freixenet, Arnau Oliver, and Xavier Lladó. SUNet: a deep learning architecture for acute stroke lesion segmentation and outcome prediction in multimodal mri. arXiv preprint arXiv:1810.13304,

    arXiv

  6. [2017]

    The multimodal brain tumor image segmentation benchmark (brats)

    Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging, 34(10):1993,

    1993

  7. [2018]

    Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin S Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos

    URL http://arxiv.org/abs/1810.06943. Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin S Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos. Advancing the cancer genome atlas glioma mri collections with expert segmentation l...

    arXiv

  8. [2019]

    doi: 10.3389/fnins.2019.00097

    ISSN 1662-453X. doi: 10.3389/fnins.2019.00097. URL https: //www.frontiersin.org/articles/10.3389/fnins.2019.00097/abstract. Yulong Lu. On the bernstein-von mises theorem for high dimensional nonlinear bayesian inverse problems. arXiv preprint arXiv:1706.00289,

    2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.