{"id":"1d47983b-9c16-4b80-9517-7ec9f2dc5107","arxiv_id":"2411.09567","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"VpbSD uses a vessel-pattern codebook trained on unlabeled microscopy data to distill knowledge from a large teacher into a 0.12M-parameter student, reaching DSC 0.852 on VesSep2020.","lead":"This paper proposes VpbSD, a training pipeline that combines a learned codebook, knowledge distillation, and semi-supervised learning to segment 3D microscopic brain vessel images with a small, fast student network. It reports improved Dice scores over baselines on a public mouse-brain dataset while shrinking model size and inference time.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential test-set leakage: the unlabeled volumes used in teacher pretraining and L_semi are not shown to be disjoint from the 5-fold evaluation volumes, which could invalidate the reported DSC 0.852 claim.","rationale":"The reader's weakest assumption focuses on pseudo-label quality and domain shift, but the more fundamental and testable problem is the absence of any demonstrated split between the unlabeled volumes used for training and the volumes used for evaluation. Section 4.1.1 says the unlabeled 2D slices were stitched per mouse into 3D volumes and divided into patches, and it describes the 5-fold split only for the 11 labeled samples. Because the labeled samples are subregions of the same light-sheet acquisitions, the unlabeled patch pool almost certainly contains the test-fold volumes unless explicitly excluded. If so, the student has already seen every test input during pretraining and semi-supervised training, and the teacher's pseudo-labels for those inputs are generated by a model that was fine-tuned on the training folds of the same data. This invalidates the central comparison against methods that either do not use unlabeled data or use it in a more restricted way. The proposed rerun with a subject-disjoint split directly settles whether the 0.852 DSC is a real generalization result or an artifact of the evaluation protocol. Until that check is performed, the paper should not be given a positive verdict; the appropriate status is unverified rather than conditionally accepted on the currently stated conditions. If the overlap is confirmed, the verdict should be rejection of the reported comparison, not merely a request for more ablations.","tokens_in":14383,"tokens_out":11583,"duration_ms":115296,"concrete_test":"Rerun VpbSD with a subject-disjoint split: for each of the 5 folds, restrict teacher pretraining, codebook construction, and L_semi to unlabeled volumes from the training subjects only, and keep the annotated test blocks entirely out of the unlabeled set. Then compare DSC, HD95, and Jaccard with Table 1. If the gap between VpbSD and FitNet/VID/MT disappears or DSC drops below roughly 0.842, the reported 0.852 result is attributable to the overlapping unlabeled set. Additionally, inspect the GitHub data-preprocessing script for an explicit removal of all annotated blocks from the unlabeled patches.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim (Sec. 4.2.1: VpbSD reaches DSC 0.852, HD95 1.974 with 0.12M parameters) rests on a 5-fold cross-validation of only 11 annotated samples. The pretraining stage (Sec. 3.3.1) and the semi-supervised term L_semi (Sec. 3.5, Eq. 10-12) use unlabeled 3D volumes stitched from 'each mouse' (Sec. 4.1.1), then cut into 64^3 patches. The paper never states that these unlabeled patches are disjoint from the 11 annotated blocks or from the volumes of the mice whose annotated blocks form the test fold. Since the 11 labeled samples are subregions of the same light-sheet volumes, the default reading is that unlabeled patches from every test subject are included in teacher pretraining, codebook construction, and L_semi. This means the student is trained on the test input, and the teacher, after fine-tuning on the labeled training fold, generates pseudo-labels for that same input. The evaluation then measures a transductive protocol rather than the claimed generalization to unseen data. The comparison against methods that do not exploit the test volumes, or that exploit them differently, is therefore not a controlled test of the proposed codebook distillation. Pseudo-label confidence and domain shift, the reader's weakest assumption, are secondary; even perfect pseudo-labels would not fix an overlapping train/test volume split.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VpbSD, a two-stage pipeline for 3D microscopic cerebrovascular segmentation. In the first stage, a SwinUNetR teacher is self-supervised pretrained on unlabeled volumes while simultaneously building a vessel-pattern codebook via vector quantization. The teacher is then fine-tuned on the small labeled set. In the second stage, a lightweight UNet student is trained with a combination of supervised loss on labeled data, a semi-supervised loss using teacher pseudo-labels on unlabeled data, and a distillation loss that aligns the student's features with the discrete codebook outputs. Experiments on the VesSep2020 dataset with 5-fold cross-validation report DSC 0.852, HD95 1.974, and a student model with 0.12M parameters, 0.11 GFLOPs, and 2.49 ms per 64^3 patch. Ablations indicate that both the semi-supervised term and the codebook-based distillation contribute to the reported gains over the baseline and over standard intermediate-layer distillation.","tokens_in":14695,"tokens_out":6165,"duration_ms":57039,"significance":"If the reported results are valid, the proposed use of a vector-quantized codebook as an intermediate distillation target for heterogeneous teacher-student transfer is an interesting and potentially useful idea for vessel segmentation, where annotated data are scarce. The paper ships a public implementation link and uses a public dataset, which supports reproducibility. The efficiency claim (near-teacher accuracy at 1/500th the parameters and 1/900th the FLOPs) is attractive for high-resolution microscopy applications. However, the central empirical claim is currently undermined by ambiguity in the train/test/unlabeled split, and by the absence of any statistical significance assessment; these issues must be resolved before the results can be interpreted as evidence of generalization to unseen data.","major_comments":[{"comment":"The manuscript never states that the unlabeled volumes used for teacher pretraining and for the semi-supervised loss L_semi are disjoint from the volumes of the subjects in the 5-fold evaluation. Section 4.1.1 says the unlabeled 2D images are stitched from 'each mouse' into 3D volumes, and the 11 annotated samples are subregions of the same light-sheet dataset. The default reading is therefore that patches from every test subject appear in the teacher's self-supervised pretraining, in the codebook construction, and in the student's semi-supervised training. This makes the evaluation a transductive protocol rather than a test of generalization to unseen volumes. The comparison against methods that do not use the test-subject unlabeled data, or that use it differently, is then not a controlled test of the proposed components. Please clarify the exact composition of the unlabeled set relative to the 5 folds. If the unlabeled set includes test-subject data, the experiments must be redone with a subject-disjoint unlabeled set, or the paper must be reframed as a transductive setting with appropriate baselines and disclaimers.","section":"§4.1.1, §3.3.1, §3.5 (Eq. 10-12)"},{"comment":"The text claims 'significant improvements' over the compared methods, but no significance test is reported. In Table 1, the DSC difference between VpbSD (0.852±0.026) and UAMT (0.847±0.029), ME (0.847±0.024), or VID (0.846±0.019) is well within one standard deviation, and the same holds for several other metrics. Table 2 reports single numbers without error bars or fold-wise variability. Without paired significance tests across the five folds, the claim of significant superiority is not supported. Please report fold-wise results and add appropriate tests (e.g., Wilcoxon signed-rank), or soften the language to 'consistent mean improvements'.","section":"§4.2.1, Table 1, Table 2"},{"comment":"The overall student loss is written as L_stu = L_seg^S + β L_semi^S + γ L_dis^S = DICE(Y_stu, Y_gt) + β DICE(Y_stu, Y_pse) + γ L1(Z_stu, Z'_tea). As written, this is a sum of Dice coefficients, which are similarity measures to be maximized, not losses to be minimized. If the intent is to use 1−Dice or −Dice as the loss terms, that should be stated explicitly, because the current formulation makes the optimization objective ambiguous and hampers reproducibility.","section":"§3.5, Eq. (10)-(12)"}],"minor_comments":[{"comment":"The third contribution says 'conducting dilation study'; this appears to be a typo for 'ablation study'.","section":"§1, Contributions"},{"comment":"The cross-references are incorrect: the comparison table is Table 1 but the text says 'Table 3.5', and the ablation table is Table 3 but the text says 'Table 4.1.4'. Please fix all table references.","section":"§4.2.1, §4.3.1"},{"comment":"The text mentions Advent [33] as a semi-supervised baseline, but Table 1 lists 'ME'. Please clarify whether 'ME' denotes Advent or another method, and keep naming consistent between text and tables.","section":"Table 1, §4.2.1"},{"comment":"The reconstruction loss is called MSE in Eq. (5), but Eq. (6) defines mean_i(|x_i - y_i|), which is mean absolute error. Please correct the notation to match the actual loss used.","section":"§3.3.1, Eq. (5)-(6)"},{"comment":"The formula for the Gumbel-Softmax quantization is unclear: the expression 'GumbelSoftmax_j || DA(z_i) - v_j ||_2 * DA(V)' mixes the distance computation and the codebook output without a precise definition of the operator. Please write the explicit softmax selection over codebook entries and define DA(V).","section":"§3.4, Eq. (8)"},{"comment":"The hyperparameters α, β, γ, the codebook size, and the embedding dimension are fixed without any sensitivity analysis or reference to a validation split. Please add a brief note on how these were chosen, even if only to state that they were held constant across all experiments.","section":"§4.1.3"},{"comment":"The preprocessing description 'retaining the values below the 95th percentile and truncating any values above this threshold' is vague; clarify whether this is clipping to the 95th percentile value or a percentile-based normalization.","section":"§4.1.1"},{"comment":"The teacher is described as a 'self-attention CNN encoder', but SwinUNetR is a Transformer-based architecture. Please align the terminology to avoid confusion about the teacher's architecture.","section":"§3.2, §4.1.2"}],"recommendation":"major_revision","confidential_remarks":"The main concern is the data-split ambiguity, which is potentially fatal to the generalization claim as currently written. I would encourage the editor to require the authors to disclose the exact relationship between the unlabeled set and the 5-fold evaluation, and to re-run or clearly reframe the experiments if there is any overlap. The novelty is moderate: the paper combines existing KD and SSL ideas with a vector-quantized codebook, and the efficiency numbers are attractive, but the current evidence is not convincing without a clean evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: VpbSD is a coherent combination of known pieces—a VQ-VAE codebook learned on unlabeled data, used as a discrete bridge for heterogeneous knowledge distillation, plus a Mean Teacher-style semi-supervised loss. The ablation shows each piece contributes, and the efficiency numbers (0.12M parameters, 0.11 GFLOPs, 2.49 ms per patch) are genuinely useful for deployment-minded readers. That part is solid.\n\nThe central empirical claim is on shakier ground. The 5-fold cross-validation uses 11 annotated samples, while the unlabeled volumes are stitched from all mice in the dataset. The paper never states that the unlabeled patches used in teacher pretraining and the semi-supervised term are disjoint from the subjects whose annotated blocks appear in the test fold. Since the 11 annotated samples are subregions of the same light-sheet volumes, the default reading is that the student sees test input during training—via codebook construction and pseudo-labels—making the evaluation transductive. That would invalidate the comparison to methods trained only on the labeled training fold. This is the first question a referee has to ask, and the paper does not answer it.\n\nSecondary issues are more standard: no significance tests despite overlapping error bars, a code link that is not a working artifact, and a few table-numbering and terminology slips. None of these are fatal alone, but they add noise.\n\nIf the authors can confirm that unlabeled data from test subjects was excluded, or if they reframe the claim as transductive and compare on that basis, the method would be a practical contribution to a narrow but real problem. As written, the generalization claim is not supported. I would still send it to peer review—the idea is worth the referee time, and the missing split information is checkable. It just should not be accepted as-is without that clarification.\n\nNet: worth a reading-group slot, worth a cite only after the split question is resolved, and worth a serious referee.","headline":"A sensible distillation pipeline for 3D vessel segmentation, but the missing train/test split on unlabeled data makes the headline DSC numbers hard to trust as evidence of generalization.","tokens_in":15238,"tokens_out":2632,"would_cite":false,"duration_ms":25671,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A codebook of vessel patterns distilled from unlabeled brain scans lets a 0.12M-parameter model match near-teacher segmentation at a fraction of the cost.","keywords":["cerebrovascular segmentation","3D microscopy","knowledge distillation","semi-supervised learning","vector quantization","codebook","lightweight model"],"falsifier":"Train the teacher and the VpbSD student on unlabeled patches from one mouse strain and evaluate on labeled patches from a different strain; if the student's DSC falls below the plain Light UNet baseline or below standard intermediate distillation, then the codebook's transfer value is distribution-bound rather than general.","tokens_in":14158,"feed_emoji":"🧠","tokens_out":3436,"duration_ms":28385,"temperature":0.7,"pith_summary":"This paper proposes VpbSD, a two-stage pipeline for segmenting 3D microscopic cerebrovasculature. In the first stage, a large teacher model is pretrained on unlabeled microscopy volumes using a vector-quantized autoencoder, building a discrete codebook of vascular patterns. In the second stage, the frozen teacher's codebook guides a lightweight CNN student through an L1 distillation loss, while teacher-generated pseudo-labels provide semi-supervised supervision on unlabeled data. The paper claims the student reaches a DSC of 0.852, Jaccard of 0.751, and HD95 of 1.974, outperforming other distillation and semi-supervised methods under the same baseline, using only 0.12M parameters and 2.49 ms per 64x64x64 patch. The significance is that high-resolution whole-brain vessel segmentation becomes practical without requiring terabyte-scale manual annotation or the compute of large Transformer models.","feed_headline":"Distilled vessel codebook yields near-teacher segmentation at 1/500 cost","feed_subtitle":"A codebook from unlabeled microscopy lets a 0.12M-parameter student reach DSC 0.852, close to SwinUNetR's 0.875.","key_machinery":"The vessel-pattern codebook is a vector-quantization codebook constructed during the teacher model's self-supervised pretraining stage. The teacher encoder's continuous features are mapped to a lower dimension via a 1x1 convolution, quantized to the nearest codebook vector, and then mapped back, with training losses following the VQ-VAE formulation plus Gumbel-Softmax to mitigate codebook collapse. During knowledge distillation, the student's encoder output is pulled toward the frozen discrete codebook embeddings using an L1 loss, and a semi-supervised Dice loss is computed between the student's predictions and the teacher's pseudo-labels on unlabeled volumes.","core_discovery":"The central claim is that vector-quantized vessel patterns, learned from unlabeled microscopy volumes, form an effective intermediate representation for knowledge transfer between a heterogeneous teacher and a lightweight student. The paper shows that aligning the student's deepest encoder layer to the teacher's discretized codebook outputs (vessel-pattern-based distillation) improves segmentation over standard intermediate-feature distillation, raising DSC from 0.842 to 0.846 when used alone, and to 0.852 when combined with the semi-supervised pseudo-label loss. The student achieves performance comparable to the much larger UNetR model while using roughly 1/800 of its parameters, and runs nearly 11 times faster per patch than the SwinUNetR teacher.","pith_inferences":["The same codebook-plus-distillation mechanism could transfer to other tubular-structure segmentation tasks such as retinal vessels, airways, or plant vasculature, wherever abundant unlabeled volumes exist.","The reported gains from pseudo-labels likely depend on pseudo-label quality; the absence of confidence filtering or noise analysis suggests that adding such filtering could further improve or stabilize the method.","The dimension-aligned low-dimensional codebook can be viewed as a learned vocabulary of vascular shapes, potentially enabling interpretable analysis of vessel morphology or out-of-distribution detection.","A fairer comparison would require consistent labeled-data fractions and model-capacity controls across all methods; the current experiments compare strategies under identical baseline architecture, which is encouraging but leaves capacity interactions unexplored."],"forward_implications":["A 0.12M-parameter student model can serve as a fast inference engine for whole-brain microscopic vessel segmentation, with 0.11 GFLOPs and 2.49 ms per 64x64x64 patch, making near-real-time deployment feasible.","The codebook-based distillation strategy removes the need for matching feature dimensions between heterogeneous teacher and student architectures, enabling distillation from Transformer-based teachers to light CNNs.","Each component contributes independently: removing the semi-supervised loss drops DSC from 0.852 to 0.846, and removing the distillation loss drops it to 0.842, showing both are needed for the reported gain.","Vessel-pattern-based distillation preserves vascular topology better than standard intermediate-feature distillation, as reflected by higher Cl Dice (0.900 vs 0.895) and lower GWD (0.231 vs 0.238)."],"supporting_citations":[{"why":"Supplies the VesSep2020 dataset of unlabeled and labeled mouse-brain microscopic vascular volumes used in all experiments.","marker":"[2]"},{"why":"Provides the Swin UNETR teacher architecture and the self-supervised pretraining paradigm the paper adapts.","marker":"[14]"},{"why":"Introduces VQ-VAE and the codebook-based discrete representation that underpins the vessel-pattern codebook.","marker":"[38]"},{"why":"Supplies the Gumbel-Softmax technique used to avoid codebook collapse during quantization.","marker":"[42]"},{"why":"Serves as a baseline intermediate-layer distillation method (FitNets) compared in the experiments.","marker":"[24]"},{"why":"Provides the vanilla knowledge distillation baseline (Hinton KD) used for comparison.","marker":"[19]"},{"why":"Supplies the Mean Teacher semi-supervised baseline compared against VpbSD.","marker":"[31]"},{"why":"Provides the warming-up schedule used to update the semi-supervised loss weight beta.","marker":"[47]"}],"fun_headline_variants":["Codebook distillation gives near-teacher 3D vessel segmentation","800x smaller model matches teacher via codebook distillation","11x faster, 800x smaller: codebook-based vessel segmentation","Semi-supervised codebook distillation for efficient 3D vessel segmentation","Student beats standard distillation with vessel codebook"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The unlabeled volumes used for teacher pretraining and semi-supervised learning are assumed to be representative of the labeled evaluation data, so that the codebook and teacher pseudo-labels improve rather than mislead the student.","fun_headline_variants_meta":{"raw":{"variants":["Codebook distillation gives near-teacher 3D vessel segmentation","800x smaller model matches teacher via codebook distillation","11x faster, 800x smaller: codebook-based vessel segmentation","Semi-supervised codebook distillation for efficient 3D vessel segmentation","Student beats standard distillation with vessel codebook"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1521,"prompt_tokens":879,"completion_tokens":642,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":558}},"tokens_in":495,"tokens_out":642,"duration_ms":6045,"temperature":1.0,"reasoning_tokens":558,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:31:07.525484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the teacher and the VpbSD student on unlabeled patches from one mouse strain and evaluate on labeled patches from a different strain; if the student's DSC falls below the plain Light UNet baseline or below standard intermediate distillation, then the codebook's transfer value is distribution-bound rather than general.","supporting_citations":[{"cited_title":"Microscopic Cerebrovascular Segmentation There have been some works at segmenting cerebral vessels in microscopic images","cited_arxiv_id":null,"evidence_quote":"Supplies the VesSep2020 dataset of unlabeled and labeled mouse-brain microscopic vascular volumes used in all experiments."},{"cited_title":"Uncertainty-aware self- ensembling model for semi-supervised 3d left atrium segmentation,","cited_arxiv_id":null,"evidence_quote":"Introduces VQ-VAE and the codebook-based discrete representation that underpins the vessel-pattern codebook."},{"cited_title":"Deepvesselnet: Vessel segmentation, centerline prediction, and bifurca- tion detection in 3-d angiographic volumes,","cited_arxiv_id":null,"evidence_quote":"Provides the vanilla knowledge distillation baseline (Hinton KD) used for comparison."},{"cited_title":"Cross-architecture knowledge distillation,","cited_arxiv_id":null,"evidence_quote":"Supplies the Mean Teacher semi-supervised baseline compared against VpbSD."}],"review_version":1}