Pith. sign in

REVIEW 3 major objections 8 minor 56 references

VPBSD:Vessel-Pattern-Based Semi-Supervised Distillation for Efficient 3D Microscopic Cerebrovascular Segmentation

T0 review · 3 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A codebook of vessel patterns distilled from unlabeled brain scans lets a 0.12M-parameter model match near-teacher segmentation at a fraction of the cost.

desk verdict A sensible distillation pipeline for 3D vessel segmentation, but the missing train/test split on unlabeled data makes the headline DSC numbers hard to trust as evidence of generalization. read the letter →

arxiv 2411.09567 v1 pith:RORBCWVB submitted 2024-11-14 cs.CV

Xi Lin , Shixuan Zhao , Xinxu Wei , Amir Shmuel , Yongjie Li This is my paper · ORCID
classification cs.CV
keywords cerebrovascularsegmentation3Dmicroscopyknowledgedistillationsemi-supervisedlearningvectorquantizationcodebooklightweightmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes VpbSD, a two-stage pipeline for segmenting 3D microscopic cerebrovasculature. In the first stage, a large teacher model is pretrained on unlabeled microscopy volumes using a vector-quantized autoencoder, building a discrete codebook of vascular patterns. In the second stage, the frozen teacher's codebook guides a lightweight CNN student through an L1 distillation loss, while teacher-generated pseudo-labels provide semi-supervised supervision on unlabeled data. The paper claims the student reaches a DSC of 0.852, Jaccard of 0.751, and HD95 of 1.974, outperforming other distillation and semi-supervised methods under the same baseline, using only 0.12M parameters and 2.49 ms per 64x64x64 patch. The significance is that high-resolution whole-brain vessel segmentation becomes practical without requiring terabyte-scale manual annotation or the compute of large Transformer models.

What carries the argument

The vessel-pattern codebook is a vector-quantization codebook constructed during the teacher model's self-supervised pretraining stage. The teacher encoder's continuous features are mapped to a lower dimension via a 1x1 convolution, quantized to the nearest codebook vector, and then mapped back, with training losses following the VQ-VAE formulation plus Gumbel-Softmax to mitigate codebook collapse. During knowledge distillation, the student's encoder output is pulled toward the frozen discrete codebook embeddings using an L1 loss, and a semi-supervised Dice loss is computed between the student's predictions and the teacher's pseudo-labels on unlabeled volumes.

What would settle it

Train the teacher and the VpbSD student on unlabeled patches from one mouse strain and evaluate on labeled patches from a different strain; if the student's DSC falls below the plain Light UNet baseline or below standard intermediate distillation, then the codebook's transfer value is distribution-bound rather than general.

Watch

Extended reading notes

Core claim

The central claim is that vector-quantized vessel patterns, learned from unlabeled microscopy volumes, form an effective intermediate representation for knowledge transfer between a heterogeneous teacher and a lightweight student. The paper shows that aligning the student's deepest encoder layer to the teacher's discretized codebook outputs (vessel-pattern-based distillation) improves segmentation over standard intermediate-feature distillation, raising DSC from 0.842 to 0.846 when used alone, and to 0.852 when combined with the semi-supervised pseudo-label loss. The student achieves performance comparable to the much larger UNetR model while using roughly 1/800 of its parameters, and runs nearly 11 times faster per patch than the SwinUNetR teacher.

Load-bearing premise

The unlabeled volumes used for teacher pretraining and semi-supervised learning are assumed to be representative of the labeled evaluation data, so that the codebook and teacher pseudo-labels improve rather than mislead the student.

Editorial extensions

If this is right

  • A 0.12M-parameter student model can serve as a fast inference engine for whole-brain microscopic vessel segmentation, with 0.11 GFLOPs and 2.49 ms per 64x64x64 patch, making near-real-time deployment feasible.
  • The codebook-based distillation strategy removes the need for matching feature dimensions between heterogeneous teacher and student architectures, enabling distillation from Transformer-based teachers to light CNNs.
  • Each component contributes independently: removing the semi-supervised loss drops DSC from 0.852 to 0.846, and removing the distillation loss drops it to 0.842, showing both are needed for the reported gain.
  • Vessel-pattern-based distillation preserves vascular topology better than standard intermediate-feature distillation, as reflected by higher Cl Dice (0.900 vs 0.895) and lower GWD (0.231 vs 0.238).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same codebook-plus-distillation mechanism could transfer to other tubular-structure segmentation tasks such as retinal vessels, airways, or plant vasculature, wherever abundant unlabeled volumes exist.
  • The reported gains from pseudo-labels likely depend on pseudo-label quality; the absence of confidence filtering or noise analysis suggests that adding such filtering could further improve or stabilize the method.
  • The dimension-aligned low-dimensional codebook can be viewed as a learned vocabulary of vascular shapes, potentially enabling interpretable analysis of vessel morphology or out-of-distribution detection.
  • A fairer comparison would require consistent labeled-data fractions and model-capacity controls across all methods; the current experiments compare strategies under identical baseline architecture, which is encouraging but leaves capacity interactions unexplored.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes VpbSD, a two-stage pipeline for 3D microscopic cerebrovascular segmentation. In the first stage, a SwinUNetR teacher is self-supervised pretrained on unlabeled volumes while simultaneously building a vessel-pattern codebook via vector quantization. The teacher is then fine-tuned on the small labeled set. In the second stage, a lightweight UNet student is trained with a combination of supervised loss on labeled data, a semi-supervised loss using teacher pseudo-labels on unlabeled data, and a distillation loss that aligns the student's features with the discrete codebook outputs. Experiments on the VesSep2020 dataset with 5-fold cross-validation report DSC 0.852, HD95 1.974, and a student model with 0.12M parameters, 0.11 GFLOPs, and 2.49 ms per 64^3 patch. Ablations indicate that both the semi-supervised term and the codebook-based distillation contribute to the reported gains over the baseline and over standard intermediate-layer distillation.

Significance. If the reported results are valid, the proposed use of a vector-quantized codebook as an intermediate distillation target for heterogeneous teacher-student transfer is an interesting and potentially useful idea for vessel segmentation, where annotated data are scarce. The paper ships a public implementation link and uses a public dataset, which supports reproducibility. The efficiency claim (near-teacher accuracy at 1/500th the parameters and 1/900th the FLOPs) is attractive for high-resolution microscopy applications. However, the central empirical claim is currently undermined by ambiguity in the train/test/unlabeled split, and by the absence of any statistical significance assessment; these issues must be resolved before the results can be interpreted as evidence of generalization to unseen data.

major comments (3)
  1. [§4.1.1, §3.3.1, §3.5 (Eq. 10-12)] The manuscript never states that the unlabeled volumes used for teacher pretraining and for the semi-supervised loss L_semi are disjoint from the volumes of the subjects in the 5-fold evaluation. Section 4.1.1 says the unlabeled 2D images are stitched from 'each mouse' into 3D volumes, and the 11 annotated samples are subregions of the same light-sheet dataset. The default reading is therefore that patches from every test subject appear in the teacher's self-supervised pretraining, in the codebook construction, and in the student's semi-supervised training. This makes the evaluation a transductive protocol rather than a test of generalization to unseen volumes. The comparison against methods that do not use the test-subject unlabeled data, or that use it differently, is then not a controlled test of the proposed components. Please clarify the exact composition of the unlabeled set relative to the 5 folds. If the unlabeled set includes test-subject data, the experiments must be redone with a subject-disjoint unlabeled set, or the paper must be reframed as a transductive setting with appropriate baselines and disclaimers.
  2. [§4.2.1, Table 1, Table 2] The text claims 'significant improvements' over the compared methods, but no significance test is reported. In Table 1, the DSC difference between VpbSD (0.852±0.026) and UAMT (0.847±0.029), ME (0.847±0.024), or VID (0.846±0.019) is well within one standard deviation, and the same holds for several other metrics. Table 2 reports single numbers without error bars or fold-wise variability. Without paired significance tests across the five folds, the claim of significant superiority is not supported. Please report fold-wise results and add appropriate tests (e.g., Wilcoxon signed-rank), or soften the language to 'consistent mean improvements'.
  3. [§3.5, Eq. (10)-(12)] The overall student loss is written as L_stu = L_seg^S + β L_semi^S + γ L_dis^S = DICE(Y_stu, Y_gt) + β DICE(Y_stu, Y_pse) + γ L1(Z_stu, Z'_tea). As written, this is a sum of Dice coefficients, which are similarity measures to be maximized, not losses to be minimized. If the intent is to use 1−Dice or −Dice as the loss terms, that should be stated explicitly, because the current formulation makes the optimization objective ambiguous and hampers reproducibility.
minor comments (8)
  1. [§1, Contributions] The third contribution says 'conducting dilation study'; this appears to be a typo for 'ablation study'.
  2. [§4.2.1, §4.3.1] The cross-references are incorrect: the comparison table is Table 1 but the text says 'Table 3.5', and the ablation table is Table 3 but the text says 'Table 4.1.4'. Please fix all table references.
  3. [Table 1, §4.2.1] The text mentions Advent [33] as a semi-supervised baseline, but Table 1 lists 'ME'. Please clarify whether 'ME' denotes Advent or another method, and keep naming consistent between text and tables.
  4. [§3.3.1, Eq. (5)-(6)] The reconstruction loss is called MSE in Eq. (5), but Eq. (6) defines mean_i(|x_i - y_i|), which is mean absolute error. Please correct the notation to match the actual loss used.
  5. [§3.4, Eq. (8)] The formula for the Gumbel-Softmax quantization is unclear: the expression 'GumbelSoftmax_j || DA(z_i) - v_j ||_2 * DA(V)' mixes the distance computation and the codebook output without a precise definition of the operator. Please write the explicit softmax selection over codebook entries and define DA(V).
  6. [§4.1.3] The hyperparameters α, β, γ, the codebook size, and the embedding dimension are fixed without any sensitivity analysis or reference to a validation split. Please add a brief note on how these were chosen, even if only to state that they were held constant across all experiments.
  7. [§4.1.1] The preprocessing description 'retaining the values below the 95th percentile and truncating any values above this threshold' is vague; clarify whether this is clipping to the 95th percentile value or a percentile-based normalization.
  8. [§3.2, §4.1.2] The teacher is described as a 'self-attention CNN encoder', but SwinUNetR is a Transformer-based architecture. Please align the terminology to avoid confusion about the teacher's architecture.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity found; the only self-citation (SMD metric) is non-load-bearing, and the empirical claim is self-contained.

full rationale

The claimed derivation chain is not circular in the sense defined here. The teacher model and vessel-pattern codebook are learned from unlabeled volumes by reconstruction and codebook losses (Eq. 4-9); the student is trained with L_seg on labeled blocks, L_semi on teacher pseudo-labels, and L_dis aligning student features to the discrete codebook outputs (Eq. 10-12). None of these objectives is defined in terms of the reported test-fold Dice or HD95 values, so the headline 'DSC 0.852, HD95 1.974' (Sec 4.2.1) is not a fitted input renamed as a prediction. The evaluation is explicitly a 5-fold cross-validation (Sec 4.1.1) over the 11 annotated samples, and the comparisons in Table 1 use the same baseline architecture, so the central empirical claim is self-contained against external benchmarks. The only self-citation appearing in the evaluation pathway is the SMD metric [48], the authors' own prior ISBI paper; SMD is described in Sec 4.1.4 but is not reported in Tables 1-3, and the DSC/HD95/GWD/clDice claims do not depend on it, so the self-citation is minor and non-load-bearing. A separate concern is that the paper does not state whether the unlabeled patches used for teacher pretraining and L_semi are disjoint from the volumes of the test fold; if they overlap, the protocol would be transductive rather than a strict generalization test. That is a data-hygiene and experimental-design risk, not a demonstrated definitional circularity, because the text provides no equation or construction in which the reported metric is an input to the training objective.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claim rests on standard deep learning machinery (VQ-VAE, straight-through estimator, Gumbel-Softmax, Dice loss) plus hand-set hyperparameters (alpha, gamma, beta, learning rates, codebook size) whose sensitivity is not reported. The vessel-pattern codebook is a learned internal component with no independent evidence outside the paper's own ablation. The domain assumptions about pseudo-label quality and distribution matching are not empirically verified.

free parameters (5)
  • codebook loss weight alpha = 0.1
    Hand-set in Eq. 4; balances reconstruction and quantization; no sensitivity analysis provided.
  • distillation loss weight gamma = 1
    Hand-set in Eq. 10; sensitivity not reported.
  • semi-supervised loss weight beta = warm-up schedule
    Updated with a warming-up function after Laine and Aila [47]; exact schedule is not specified.
  • codebook size and embedding dimension = not reported
    N vectors of M=ds dimensions are described in Section 3.4, but actual values are omitted, affecting codebook quality.
  • learning rates and epoch counts = 1e-3, 1e-5, 1e-2; 80, 30, 300
    Chosen without sensitivity analysis in Section 4.1.3.
assumptions (5)
  • standard math Straight-through estimator and Gumbel-Softmax make the discrete quantization trainable.
    Inherited from VQ-VAE [38] and VQ-Wav2Vec [42], used in Section 3.4.
  • domain assumption Teacher pseudo-labels on unlabeled data are correct often enough to improve student training.
    Invoked in the semi-supervised loss in Eq. 11; no confidence filter or uncertainty weighting is applied.
  • domain assumption The distribution of the stitched unlabeled volumes matches the distribution of labeled patches used for evaluation.
    Required for the self-supervised pretraining and semi-supervised gains to transfer; not analyzed in Section 4.
  • domain assumption Dice loss is the appropriate training objective for vessel segmentation.
    Used in Eq. 7 and Eq. 11; standard in the field.
  • ad hoc to paper Dimension alignment via 1x1 convolution is sufficient because it performed comparably to linear projection and channel selection.
    Stated in Section 3.4 without reporting the comparison data.
invented entities (1)
  • vessel-pattern codebook
    purpose: Discrete latent embeddings that store vascular structure prototypes learned during teacher pretraining and serve as distillation targets.
    The codebook is a learned artifact with no external falsifiable handle beyond the paper's own segmentation metrics; its utility is shown only through internal ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VPBSD:Vessel-Pattern-Based Semi-Supervised Distillation for Efficient 3D Microscopic Cerebrovascular Segmentation." pith.science (2026). https://pith.science/paper/RORBCWVB

@misc{pith2026241109567,
  author       = {Pith},
  title        = {Pith review of: VPBSD:Vessel-Pattern-Based Semi-Supervised Distillation for Efficient 3D Microscopic Cerebrovascular Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RORBCWVB}},
  note         = {Machine review of arXiv:2411.09567}
}
read the original abstract

3D microscopic cerebrovascular images are characterized by their high resolution, presenting significant annotation challenges, large data volumes, and intricate variations in detail. Together, these factors make achieving high-quality, efficient whole-brain segmentation particularly demanding. In this paper, we propose a novel Vessel-Pattern-Based Semi-Supervised Distillation pipeline (VpbSD) to address the challenges of 3D microscopic cerebrovascular segmentation. This pipeline initially constructs a vessel-pattern codebook that captures diverse vascular structures from unlabeled data during the teacher model's pretraining phase. In the knowledge distillation stage, the codebook facilitates the transfer of rich knowledge from a heterogeneous teacher model to a student model, while the semi-supervised approach further enhances the student model's exposure to diverse learning samples. Experimental results on real-world data, including comparisons with state-of-the-art methods and ablation studies, demonstrate that our pipeline and its individual components effectively address the challenges inherent in microscopic cerebrovascular segmentation.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 38 canonical work pages

  1. [1]

    INTRODUCTION Cerebrovascular segmentation plays a pivotal role in visual- izing and extracting intricate details of cerebral blood vessel structures. As a fundamental aspect of neuroscience research, it facilitates comprehensive analysis of brain function and structural changes within cerebral vessels, while also en- abling the assessment of vascular abno...

    work page Pith review arXiv 2024

  2. [2]

    Microscopic Cerebrovascular Segmentation There have been some works at segmenting cerebral vessels in microscopic images

    RELA TED WORKS 2.1. Microscopic Cerebrovascular Segmentation There have been some works at segmenting cerebral vessels in microscopic images. [2] developed a pipeline based on transfer learning for cerebrovascular segmentation in light- sheet microscopic images of mice. By pretraining on syn- thetic datasets and fine-tuning on a small amount of real, an- ...

    work page 2015

  3. [3]

    To mitigate these challenges, we introduce a novel codebook-based semi- supervised knowledge distillation framework that addresses these issues concurrently

    METHODS High-resolution microscopic images present considerable challenges for 3D brain vascular segmentation due to their substantial storage requirements, increased annotation com- plexity, and prolonged computation times, all of which render the process more resource-intensive. To mitigate these challenges, we introduce a novel codebook-based semi- sup...

    work page

  4. [4]

    Implementation Details 4.1.1

    EXPERIMENTS 4.1. Implementation Details 4.1.1. Datasets In our experiments, we utilized the public VesSep2020 dataset, introduced in 2020 [2]. This dataset, derived from light-sheet microscopy, comprises a substantial collection of unlabeled microscopic images of mouse brains, comple- mented by a smaller subset of manually annotated data. The resolution o...

    work page 2020

  5. [5]

    Our strategies effectively leveraged feature learning from difficult- to-annotate microscopic images, resulting in a student model capable of fast and efficient segmentation

    CONCLUSION In this paper, we propose a semi-supervised knowledge dis- tillation model integrated with vector quantization to tackle the challenge of microscopic brain vessel segmentation. Our strategies effectively leveraged feature learning from difficult- to-annotate microscopic images, resulting in a student model capable of fast and efficient segmenta...

    work page

  6. [6]

    Angiogenesis in brain tumours,

    Rakesh K Jain, Emmanuelle Di Tomaso, Dan G Duda, Jay S Loeffler, A Gregory Sorensen, and Tracy T Batch- elor, “Angiogenesis in brain tumours,” Nature Reviews Neuroscience, vol. 8, no. 8, pp. 610–622, 2007

    work page 2007

  7. [7]

    A dataset of rodent cerebrovasculature from in vivo mul- tiphoton fluorescence microscopy imaging,

    Charissa Poon, Petteri Teikari, Muhammad Febrian Rachmadi, Henrik Skibbe, and Kullervo Hynynen, “A dataset of rodent cerebrovasculature from in vivo mul- tiphoton fluorescence microscopy imaging,” Scientific Data, vol. 10, no. 1, pp. 141, 2023

    work page 2023

  8. [8]

    Machine learning analysis of whole mouse brain vasculature,

    Mihail Ivilinov Todorov, Johannes Christian Paetzold, Oliver Schoppe, Giles Tetteh, Suprosanna Shit, Velizar Efremov, Katalin Todorov-V¨olgyi, Marco D¨uring, Mar- tin Dichgans, Marie Piraud, et al., “Machine learning analysis of whole mouse brain vasculature,” Nature methods, vol. 17, no. 4, pp. 442–449, 2020

    work page 2020

Show all 56 references
  1. [9]

    Orientation and context entangled network for retinal vessel segmentation,

    Xinxu Wei, Kaifu Yang, Danilo Bzdok, and Yongjie Li, “Orientation and context entangled network for retinal vessel segmentation,”Expert Systems with Applications, vol. 217, pp. 119443, 2023

    2023

  2. [10]

    Neuropathol- ogy simplified,

    David A Hilton and Aditya G Shivane, “Neuropathol- ogy simplified,” Cham, Switzerland: Springer Nature Switzerland, vol. 2, 2015

    2015

  3. [11]

    Cerebral small vessel disease,

    Qian Li, Yang Yang, Cesar Reis, Tao Tao, Wanwei Li, Xiaogang Li, and John H Zhang, “Cerebral small vessel disease,” Cell transplantation, vol. 27, no. 12, pp. 1711– 1722, 2018

    2018

  4. [12]

    Multiscale modelling of cerebrovas- cular injury reveals the role of vascular anatomy and parenchymal shear stresses,

    Siamak Farajzadeh Khosroshahi, Xianzhen Yin, Cor- nelius K. Donat, Aisling McGarry, Maria Yanez Lopez, Nicoleta Baxan, David J. Sharp, Magdalena Sastre, and Mazdak Ghajari, “Multiscale modelling of cerebrovas- cular injury reveals the role of vascular anatomy and parenchymal s...

    2021

  5. [13]

    All answers are in the im- ages: A review of deep learning for cerebrovascular seg- mentation,

    Cheng Chen, Kangneng Zhou, Zhiliang Wang, Qian Zhang, and Ruoxiu Xiao, “All answers are in the im- ages: A review of deep learning for cerebrovascular seg- mentation,” Computerized Medical Imaging and Graph- ics, vol. 107, pp. 102229, 2023

    2023

  6. [14]

    Retinal vessel segmentation with deep graph and capsule reasoning,

    Xinxu Wei, Xi Lin, Haiyun Liu, Shixuan Zhao, and Yongjie Li, “Retinal vessel segmentation with deep graph and capsule reasoning,” arXiv preprint arXiv:2409.11508, 2024

    2024 arXiv

  7. [15]

    Attention-assisted adversarial model for cerebrovascu- lar segmentation in 3d tof-mra volumes,

    Ying Chen, Darui Jin, Bin Guo, and Xiangzhi Bai, “Attention-assisted adversarial model for cerebrovascu- lar segmentation in 3d tof-mra volumes,” IEEE Trans- actions on Medical Imaging, vol. 41, no. 12, pp. 3520– 3532, 2022

    2022

  8. [16]

    Level set based cerebral vasculature segmentation and diameter quantification in ct angiog- raphy,

    Rashindra Manniesing, Birgitta K Velthuis, Maarten S van Leeuwen, Irene C van der Schaaf, PJ Van Laar, and Wiro J Niessen, “Level set based cerebral vasculature segmentation and diameter quantification in ct angiog- raphy,” Medical image analysis, vol. 10, no. 2, pp. 200– 214, 2006

    2006

  9. [17]

    Topology-aware learning for volumetric cerebrovascu- lar segmentation,

    Subhashis Banerjee, Dimitrios Toumpanakis, Ashis Ku- mar Dhara, Johan Wikstr ¨om, and Robin Strand, “Topology-aware learning for volumetric cerebrovascu- lar segmentation,” in 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE, 2022, pp. 1–4

    2022

  10. [18]

    cldice-a novel topology-preserving loss func- tion for tubular structure segmentation,

    Suprosanna Shit, Johannes C Paetzold, Anjany Sekuboyina, Ivan Ezhov, Alexander Unger, Andrey Zhylka, Josien PW Pluim, Ulrich Bauer, and Bjoern H Menze, “cldice-a novel topology-preserving loss func- tion for tubular structure segmentation,” in Proceedings of the IEEE/CVF confe...

    2021

  11. [19]

    Deepvesselnet: Vessel segmentation, centerline prediction, and bifurca- tion detection in 3-d angiographic volumes,

    Giles Tetteh, Velizar Efremov, Nils D Forkert, Matthias Schneider, Jan Kirschke, Bruno Weber, Claus Zimmer, Marie Piraud, and Bj ¨orn H Menze, “Deepvesselnet: Vessel segmentation, centerline prediction, and bifurca- tion detection in 3-d angiographic volumes,” Frontiers in Neu...

    2020

  12. [20]

    Self-supervised pre-training of swin transformers for 3d medical image analysis,

    Yucheng Tang, Dong Yang, Wenqi Li, Holger R Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh, “Self-supervised pre-training of swin transformers for 3d medical image analysis,” in Pro- ceedings of the IEEE/CVF conference on computer vi- sion and pattern re...

    2022

  13. [21]

    Masked autoencoders are scalable vision learners,

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Pi- otr Doll ´ar, and Ross Girshick, “Masked autoencoders are scalable vision learners,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16000–16009

    2022

  14. [22]

    Unsupervised learn- ing of visual representations by solving jigsaw puzzles,

    Mehdi Noroozi and Paolo Favaro, “Unsupervised learn- ing of visual representations by solving jigsaw puzzles,” in European conference on computer vision . Springer, 2016, pp. 69–84

    2016

  15. [23]

    A skeletonization algorithm for gradient-based optimization,

    Martin J Menten, Johannes C Paetzold, Veronika A Zimmer, Suprosanna Shit, Ivan Ezhov, Robbie Holland, Monika Probst, Julia A Schnabel, and Daniel Rueck- ert, “A skeletonization algorithm for gradient-based optimization,” in Proceedings of the IEEE/CVF In- ternational Conferenc...

    2023

  16. [24]

    Fitnets: Hints for thin deep nets,

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Ka- hou, Antoine Chassang, Carlo Gatta, and Yoshua Ben- gio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  17. [25]

    Distilling the knowledge in a neural network,

    Geoffrey Hinton, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  18. [26]

    Knowledge distillation: A survey,

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao, “Knowledge distillation: A survey,” In- ternational Journal of Computer Vision, vol. 129, no. 6, pp. 1789–1819, 2021

    2021

  19. [27]

    When does label smoothing help?,

    Rafael M ¨uller, Simon Kornblith, and Geoffrey E Hinton, “When does label smoothing help?,”Advances in neural information processing systems, vol. 32, 2019

    2019

  20. [28]

    Knowledge transfer via distillation of activation boundaries formed by hidden neurons,

    Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi, “Knowledge transfer via distillation of activation boundaries formed by hidden neurons,” in Proceedings of the AAAI conference on artificial intelli- gence, 2019, vol. 33, pp. 3779–3787

    2019

  21. [29]

    Variational information distillation for knowledge transfer,

    Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai, “Variational information distillation for knowledge transfer,” in Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, 2019, pp. 9163–9171

    2019

  22. [30]

    Learning with limited annotations: a survey on deep semi-supervised learning for medical image segmentation,

    Rushi Jiao, Yichi Zhang, Le Ding, Bingsen Xue, Ji- cong Zhang, Rong Cai, and Cheng Jin, “Learning with limited annotations: a survey on deep semi-supervised learning for medical image segmentation,” Computers in Biology and Medicine, p. 107840, 2023

    2023

  23. [31]

    Cross-architecture knowledge distillation,

    Yufan Liu, Jiajiong Cao, Bing Li, Weiming Hu, Jingt- ing Ding, and Liang Li, “Cross-architecture knowledge distillation,” in Proceedings of the Asian conference on computer vision, 2022, pp. 3396–3411

    2022

  24. [32]

    An introduction to autoen- coders,

    Umberto Michelucci, “An introduction to autoen- coders,” arXiv preprint arXiv:2201.03898, 2022

    2022 arXiv

  25. [33]

    Jigsaw clus- tering for unsupervised visual representation learning,

    Pengguang Chen, Shu Liu, and Jiaya Jia, “Jigsaw clus- tering for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , 2021, pp. 11526– 11535

    2021

  26. [34]

    3d self-supervised methods for medical imaging,

    Aiham Taleb, Winfried Loetzsch, Noel Danz, Julius Severin, Thomas Gaertner, Benjamin Bergner, and Christoph Lippert, “3d self-supervised methods for medical imaging,” Advances in neural information pro- cessing systems, vol. 33, pp. 18158–18172, 2020

    2020

  27. [35]

    Unsupervised representation learning by predicting im- age rotations,

    Spyros Gidaris, Praveer Singh, and Nikos Komodakis, “Unsupervised representation learning by predicting im- age rotations,” arXiv preprint arXiv:1803.07728, 2018

    2018 arXiv

  28. [36]

    Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation,

    Chenyu You, Ruihan Zhao, Lawrence H Staib, and James S Duncan, “Momentum contrastive voxel-wise representation learning for semi-supervised volumetric medical image segmentation,” in International Con- ference on Medical Image Computing and Computer- Assisted Intervention. Spr...

    2022

  29. [37]

    Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,

    Antti Tarvainen and Harri Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” Ad- vances in neural information processing systems , vol. 30, 2017

    2017

  30. [38]

    Uncertainty-aware self- ensembling model for semi-supervised 3d left atrium segmentation,

    Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ann Heng, “Uncertainty-aware self- ensembling model for semi-supervised 3d left atrium segmentation,” in Medical image computing and com- puter assisted intervention–MICCAI 2019: 22nd inter- national conference, Shenz...

    2019

  31. [39]

    Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation,

    Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick P´erez, “Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation,” in Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, 2019, pp. 2517–2526

    2019

  32. [40]

    Semi-supervised semantic segmentation with cross pseudo supervision,

    Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jing- dong Wang, “Semi-supervised semantic segmentation with cross pseudo supervision,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 2613–2622

    2021

  33. [41]

    Simcvd: Simple con- trastive voxel-wise representation distillation for semi- supervised medical image segmentation,

    Chenyu You, Yuan Zhou, Ruihan Zhao, Lawrence Staib, and James S Duncan, “Simcvd: Simple con- trastive voxel-wise representation distillation for semi- supervised medical image segmentation,” IEEE Trans- actions on Medical Imaging , vol. 41, no. 9, pp. 2228– 2237, 2022

    2022

  34. [42]

    vq-wav2vec: Self-supervised learning of discrete speech representations,

    Alexei Baevski, Steffen Schneider, and Michael Auli, “vq-wav2vec: Self-supervised learning of discrete speech representations,” arXiv preprint arXiv:1910.05453, 2019

    1910 arXiv

  35. [43]

    Thus, the quantization loss can be expressed as: Lquant = ∥sg[E] − vi∥2 2 + λ ∥E −sg[vi]∥2 2 (3) where sg(∗) is the stop-gradient operation

    is employed to address this issue. Thus, the quantization loss can be expressed as: Lquant = ∥sg[E] − vi∥2 2 + λ ∥E −sg[vi]∥2 2 (3) where sg(∗) is the stop-gradient operation. Nearest Embedding Replacing Light CNN Encoder 𝐸𝑠 Frozen Vessel-Pattern Codebook … Nearest Embedding R...

  36. [44]

    Pseudo-label guided contrastive learning for semi-supervised medical image segmentation,

    Hritam Basak and Zhaozheng Yin, “Pseudo-label guided contrastive learning for semi-supervised medical image segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 19786–19797

    2023

  37. [45]

    Neural dis- crete representation learning,

    Aaron Van Den Oord, Oriol Vinyals, et al., “Neural dis- crete representation learning,” Advances in neural infor- mation processing systems, vol. 30, 2017

    2017

  38. [46]

    Vector-quantized image modeling with improved vqgan,

    Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruom- ing Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu, “Vector-quantized image modeling with improved vqgan,” arXiv preprint arXiv:2110.04627, 2021

    2021 arXiv

  39. [47]

    Beit v2: Masked image modeling with vector-quantized visual tokenizers,

    Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei, “Beit v2: Masked image modeling with vector-quantized visual tokenizers,” arXiv preprint arXiv:2208.06366, 2022

    2022 arXiv

  40. [48]

    Regularized vector quantization for to- kenized image synthesis,

    Jiahui Zhang, Fangneng Zhan, Christian Theobalt, and Shijian Lu, “Regularized vector quantization for to- kenized image synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18467–18476

    2023

  41. [49]

    Estimating or propagating gradients through stochastic neurons for conditional computa- tion,

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville, “Estimating or propagating gradients through stochastic neurons for conditional computa- tion,” arXiv preprint arXiv:1308.3432, 2013

    2013 arXiv

  42. [50]

    Sub- space clustering for high dimensional data: a review,

    Lance Parsons, Ehtesham Haque, and Huan Liu, “Sub- space clustering for high dimensional data: a review,” Acm sigkdd explorations newsletter , vol. 6, no. 1, pp. 90–105, 2004

    2004

  43. [51]

    Terastitcher-a tool for fast automatic 3d-stitching of teravoxel-sized microscopy images,

    Alessandro Bria and Giulio Iannello, “Terastitcher-a tool for fast automatic 3d-stitching of teravoxel-sized microscopy images,” BMC bioinformatics, vol. 13, pp. 1–15, 2012

    2012

  44. [52]

    Attention is all you need,

    A Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017

    2017

  45. [53]

    Temporal ensem- bling for semi-supervised learning,

    Samuli Laine and Timo Aila, “Temporal ensem- bling for semi-supervised learning,” arXiv preprint arXiv:1610.02242, 2016

    2016 arXiv

  46. [54]

    Vas- cular skeleton deformation evaluation based on the met- ric of sinkhorn distance,

    Xi Lin, Xinxu Wei, Shixuan Zhao, and Yongjie Li, “Vas- cular skeleton deformation evaluation based on the met- ric of sinkhorn distance,” in 2024 IEEE International Symposium on Biomedical Imaging (ISBI). IEEE, 2024, pp. 1–5

    2024

  47. [55]

    Sinkhorn distances: Lightspeed compu- tation of optimal transport,

    Marco Cuturi, “Sinkhorn distances: Lightspeed compu- tation of optimal transport,” Advances in neural infor- mation processing systems, vol. 26, 2013

    2013

  48. [56]

    Unetr: Transformers for 3d medical image segmentation,

    Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R Roth, and Daguang Xu, “Unetr: Transformers for 3d medical image segmentation,” in Proceedings of the IEEE/CVF winter conference on applications of com- puter vision, 2022, pp. 574–584

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.