REVIEW 5 major objections 6 minor 34 references
TractoEmbed: Modular Multi-level Embedding framework for white matter tract segmentation
T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper claims that combining three learned embeddings—from individual streamlines, their local point-cloud neighborhoods, and regional patches—improves white matter tract segmentation from diffusion MRI, especially in sparse…
desk verdict The modular idea is sensible, but the paper's best reported number comes from a two-embedding ablation, not the full three-embedding model, so the central claim is not backed by the paper's own tables. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Multi-Embedding Concat Layer, which concatenates three embeddings: a 256-dimensional streamline embedding produced by a CNN over a Fiber Descriptor representation; a 1024-dimensional cluster embedding from a PointNet over a hyperlocal point cloud of the streamline plus its five nearest neighbors; and a 1024-dimensional patch embedding from a mini-PointNet and discrete variational autoencoder that reconstructs 64 patches of 16 points sampled by farthest-point sampling and k-nearest neighbors. These three embeddings are concatenated into a 2304-dimensional vector that feeds a classifier MLP. During training, the streamline and patch encoders are frozen after pretraining, while the cluster encoder and classifier are trained end-to-end with focal loss to counter class imbalance.
What would settle it
Take a held-out subset of the pediatric and pathological datasets, have expert neuroanatomists manually label the tracts, and compare TractoEmbed's predictions against those manual labels. If the model's accuracy and F1 against manual labels are not higher than the comparison baselines, or if its errors cluster in tracts where the atlas disagrees with experts, the claim of cross-age generalization would be falsified.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a hierarchy of streamline-level, cluster-level, and patch-level representations, each processed by a task-specific encoder, can be fused into one classifier and outperforms single-representation and existing local-global point cloud baselines in white matter tract segmentation. The reported evaluation shows the full three-embedding model reaching 93.04% accuracy and 91.38% F1 in the hyperlocal point cloud setting, whereas the comparison method reaches 91.12% and 88.66%. The paper further reports that this advantage holds across datasets spanning pediatric, adolescent, adult, Parkinson's, and tumor subjects, all labeled through the same atlas-based parcellation pipeline.
Load-bearing premise
The pipeline treats the atlas-based parcellation as ground truth for all datasets; if the atlas, which was derived from young-adult HCP brains, mislabels developing or pathological brains, then the reported accuracy and F1 measure agreement with the atlas rather than anatomical truth, and the claimed generalization across age groups is not independently validated.
Editorial extensions
If this is right
- If correct, tract segmentation becomes effective with only five neighboring streamlines, reducing the data and compute needed for region-specific or time-sensitive analyses.
- The modular embedding design means new types of encoders can be added to the multi-embedding layer without reworking the existing components, allowing gradual performance gains as better embeddings are introduced.
- The reported improvements in F1 for structurally similar minor and projection fibers would support more reliable connectivity studies in those pathways.
- The claimed cross-dataset and cross-age generalization would make the method usable in clinical and developmental studies without per-dataset retraining, assuming the shared atlas label space is valid.
- The ablation pattern suggests a practical rule: denser point clouds need fewer embeddings, while sparser clouds benefit from all three, informing how to balance acquisition and modeling.
Reading between the lines
- A consequence the paper leaves implicit is that its evaluation protocol conflates label consistency with anatomical truth: because every dataset is labeled by the same young-adult-derived atlas, the reported cross-age generalization would only be independently confirmed by manual expert labels on pediatric and pathological brains.
- A testable extension would be to measure performance as the hyperlocal neighborhood shrinks from five streamlines to one or two, which would directly probe the method's ceiling for sparse clinical acquisitions.
- Adding a rotation-invariant embedding (e.g., based on streamline tangent frames) would be a natural experiment to test whether the current gains persist under arbitrary scanner orientation changes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes TractoEmbed, a modular multi-level embedding framework for white matter tract segmentation. Each streamline is represented at three levels: individual streamline points (15,3), local or hyperlocal point clouds of neighboring streamlines sampled by MDF/FSS, and regional patches obtained by farthest-point sampling and kNN. Three encoders are trained: a CNN on a fiber-descriptor representation of the streamline, a PointNet on the cluster point cloud, and a mini-PointNet+dVAE on patches. Their embeddings are concatenated at a Multi-Embedding Concat Layer and fed to an MLP classifier. Experiments use 1 million streamlines pooled from five public dMRI datasets with whitematteranalysis parcellation labels, and compare against DeepWMA, DCNN++, PointNet, DGCNN, and TractCloud. The paper claims state-of-the-art results, especially in sparse hyperlocal point clouds, and emphasizes modularity for future embedding extensions.
Significance. If the reported results are reproducible and correctly attributed, the framework is a useful contribution: it explores hierarchical streamline representations, reduces reliance on dense global context, and the modular design is conceptually easy to extend. The paper provides detailed architecture tables and an ablation over embedding combinations and point-cloud densities, and it reuses the TractCloud data pipeline to obtain comparable baselines. However, the main empirical claim is weakened by an internal inconsistency between the headline result in Table 5 and the ablation study in Table 6, by the lack of per-dataset validation despite the abstract's generalization claim, and by the absence of uncertainty estimates. In particular, the patch-embedding contribution does not show a consistent benefit in the ablation, which directly undermines the paper's central mechanism.
major comments (5)
- [Table 5 vs Table 6] The headline Hyperlocal PCD result for 'TractoEmbed (ours)' in Table 5 is 93.04% accuracy and 91.38% F1. In Table 6, the closest entries are the 'cluster + streamline' rows at nc=220 (93.038%, 91.381%) and nc=240 (93.020%, 91.346%); the full 'cluster + patch + streamline' rows are 92.946%/91.284% (nc=190), 92.837%/91.091% (nc=220), and 92.876%/91.164% (nc=240). Thus the reported SOTA number corresponds to a two-embedding configuration, not to the three-embedding method described in Section 4.4. In addition, the Local PCD row in Table 5 (92.09%/90.07%) matches no Local PCD row in Table 6; the best shown Local PCD entry is cluster+streamline at nc=190 with 91.494%/89.338%. The authors must state exactly which configuration produced each Table 5 row and explain why the full model is not the one reported.
- [Table 6] The ablation does not support the central claim that combining all three embeddings yields better segmentation. Adding the patch embedding to 'cluster + streamline' decreases accuracy and F1 in four of the six reported columns (hyperlocal nc=220, hyperlocal nc=240, local nc=190, local nc=240) and yields only marginal gains in the other two (accuracy +0.03 or less, with F1 gains up to 0.12). This contradicts the text in Section 5 that 'as the neighboring point cloud becomes sparser ... increasing the need for more embeddings.' Please provide a configuration in which the three-embedding model is consistently superior, or revise the contribution to reflect the two-embedding result.
- [Sections 3.2 and 5] The abstract claims generalization 'across different datasets' and 'various age groups,' but all results in Section 5 are reported on a single pooled test set of 20 subjects from the combined 100-subject corpus. No accuracy or F1 is given separately for dHCP, ABCD, HCP, PPMI, or BTP. Pooling hides domain shift and does not validate age-group or pathology-level generalization. Please report per-dataset and per-age-group results, or soften the claim accordingly.
- [Section 3.1] The labels are generated by whitematteranalysis parcellation using an atlas derived from 100 registered HCP young-adult tractograms, and the same atlas is applied to all datasets, including neonatal dHCP and pathological PPMI/BTP. If atlas registration is inaccurate for these populations, the reported accuracy and F1 measure agreement with the atlas rather than anatomical ground truth. The paper should explicitly discuss this limitation and, if possible, evaluate on a subset with expert or alternative labels.
- [Tables 5-7] All reported numbers are single point estimates from one train/validation/test split; there are no error bars, repeated runs, or significance tests. Many comparisons in Table 5 are within about 1% (e.g., Hyperlocal PCD: 93.04% vs 91.12%), and differences between embedding configurations in Table 6 are often smaller than 0.2%. Without uncertainty estimates, the claim to 'outperform' state-of-the-art methods is not statistically supported. Please add repeated-seed experiments or significance tests.
minor comments (6)
- [Table 1] Table 1 has formatting problems: the dHCP row reads '0 20 vol.' and the column headings (e.g., 'N Volumes (mm3)') are garbled. Please reformat the table so that subject counts, b-values, and volumes are clear.
- [Section 3.3] The FSS radius is reported only as '4mm-6mm'; please state the exact value used for all experiments, since this is a free parameter in the data-preparation pipeline.
- [Section 5] The paper says 'Code will be made available upon request,' which is not sufficient for reproducibility. Please release the code and the exact subject-wise data splits used for training, validation, and testing.
- [Figure 1 and Section 3.3] The symbols 'klocal' and 'khyperlocal' are used interchangeably in several places, and Table 5 uses 'k = 20' and 'k = 5' without a precise definition of which parameter they refer to. Please standardize the notation.
- [Table 7] The column header 'TractoEmbed (%)' in Table 7 is confusing because the rows show individual encoders, which are not the full TractoEmbed framework. Please relabel the table to make clear these are single-encoder baselines.
- [Throughout] The manuscript contains several grammatical errors, such as 'Diving even further' and 'can further be proven vital.' A careful proofreading pass is needed.
Circularity Check
No significant circularity: the central results are empirical evaluations on held-out test data, and the Table 5/Table 6 discrepancy is an internal consistency issue, not a derivational circle.
full rationale
TractoEmbed is an empirical machine-learning paper, not a derivation from first principles. The encoders are trained on labeled streamline data and evaluated on a subject-wise held-out test split, so the reported accuracy and F1 scores are not equivalent to the training inputs by construction. The use of the whitematteranalysis atlas to generate labels is an external labeling assumption, not a circular step: the model predictions are compared against those labels, and the atlas is not derived from the model. The comparative baseline numbers are sourced from TractCloud [30], but this is standard benchmark reuse, not circular reasoning. The self-citations in the related work section (e.g., BrainSegNet [10], FS2Net [11]) are contextual and are not load-bearing for the paper's central claim. The mismatch between Table 5's reported TractoEmbed Hyperlocal result and Table 6's ablation numbers is a real internal inconsistency, but it is not a case where a prediction reduces to an input by construction; it is a reproducibility or reporting concern. No fitted parameter is renamed as a prediction, and no uniqueness theorem or self-citation chain forces the result. Therefore, the paper's derivation chain is not circular, and the appropriate score is 0.
Assumptions & free parameters
free parameters (6)
- FSS radius =
4-6 mm
- k_hyperlocal =
5
- n_c (cluster point count) =
240 for hyperlocal; 190 and 220 in ablations
- k_local =
20
- patch parameters (p_f, p_local) =
64 patches, 16 points each
- training schedule =
40 epochs, learning rate 1e-4, focal loss
assumptions (4)
- domain assumption whitematteranalysis atlas parcellation yields reliable ground-truth tract labels for all five datasets
- domain assumption FSS with a 4-6 mm radius retrieves streamlines predominantly from the same tract class
- domain assumption downsampling to 15 points, interpolating to 40 points, and subsampling to n_c points preserves tract-discriminative geometry
- domain assumption held-out subjects from the same merged corpus support the age-group generalization claim
Cite this review
Pith. "Pith review of TractoEmbed: Modular Multi-level Embedding framework for white matter tract segmentation." pith.science (2026). https://pith.science/paper/TSTOP6ND
@misc{pith2026241108187,
author = {Pith},
title = {Pith review of: TractoEmbed: Modular Multi-level Embedding framework for white matter tract segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/TSTOP6ND}},
note = {Machine review of arXiv:2411.08187}
}
read the original abstract
White matter tract segmentation is crucial for studying brain structural connectivity and neurosurgical planning. However, segmentation remains challenging due to issues like class imbalance between major and minor tracts, structural similarity, subject variability, symmetric streamlines between hemispheres etc. To address these challenges, we propose TractoEmbed, a modular multi-level embedding framework, that encodes localized representations through learning tasks in respective encoders. In this paper, TractoEmbed introduces a novel hierarchical streamline data representation that captures maximum spatial information at each level i.e. individual streamlines, clusters, and patches. Experiments show that TractoEmbed outperforms state-of-the-art methods in white matter tract segmentation across different datasets, and spanning various age groups. The modular framework directly allows the integration of additional embeddings in future works.
Figures
Reference graph
Works this paper leans on
-
[1]
Biophysical journal66(1), 259–267 (1994)
Basser, P.J., Mattiello, J., LeBihan, D.: Mr diffusion tensor spectroscopy and imag- ing. Biophysical journal66(1), 259–267 (1994)
work page 1994
-
[2]
Magnetic reso- nance in medicine44(4), 625–632 (2000)
Basser, P.J., et al.: In vivo fiber tractography using dt-mri data. Magnetic reso- nance in medicine44(4), 625–632 (2000)
work page 2000
-
[3]
Behrens, T.E., et al.: Probabilistic diffusion tractography with multiple fibre ori- entations: What can we gain? neuroimage34(1), 144–155 (2007)
work page 2007
-
[4]
Bertò, G., et al.: Classifyber, a robust streamline-based linear classifier for white matter bundle segmentation. NeuroImage224, 117402 (2021)
work page 2021
-
[5]
Dumais, F., et al.: Fiesta: Autoencoders for accurate fiber segmentation in trac- tography. NeuroImage279, 120288 (2023)
work page 2023
-
[6]
Frontiers in neuroscience16, 886772 (2022)
Edwards, A.D., et al.: The developing human connectome project neonatal data release. Frontiers in neuroscience16, 886772 (2022)
work page 2022
-
[7]
Frontiers in neuroscience17, 1178473 (2023)
Funk,A.T.,Hassan,A.A.,Brüggemann,N.,Sharma,N.,Breiter,H.C.,Blood,A.J., Waugh, J.L.: In humans, striato-pallido-thalamic projections are largely segregated by their origin in either the striosome-like or matrix-like compartments. Frontiers in neuroscience17, 1178473 (2023)
work page 2023
-
[8]
Garyfallidis, E., Côté, M.A., Rheault, F., Sidhu, J., Hau, J., Petit, L., Fortin, D., Cunanne, S., Descoteaux, M.: Recognition of white matter bundles using local and global streamline-based registration and clustering. NeuroImage170, 283–295 (2018)
work page 2018
Show all 34 references
-
[9]
Frontiers in neuroscience6, 175 (2012) TractoEmbed 15
Garyfallidis, E., et al.: Quickbundles, a method for tractography simplification. Frontiers in neuroscience6, 175 (2012) TractoEmbed 15
2012
-
[10]
arXiv preprint arXiv:1710.05158 (2017)
Gupta,T.,Patil,S.M.,Tailor,M.,Thapar,D.,Nigam,A.:Brainsegnet:asegmenta- tion network for human brain fiber tractography data into anatomically meaningful clusters. arXiv preprint arXiv:1710.05158 (2017)
2017 arXiv
-
[11]
In: Computer Analysis of Images and Patterns: 18th International Conference, CAIP 2019, Salerno, Italy, September 3–5, 2019, Pro- ceedings, Part II 18
Jha, R.R., Patil, S., Nigam, A., Bhavsar, A.: Fs2net: fiber structural similarity net- work (fs2net) for rotation invariant brain tractography segmentation using stacked lstm based siamese network. In: Computer Analysis of Images and Patterns: 18th International Conference, CA...
2019
-
[12]
In: Medical Imaging 2018: Image Processing
Lam, P.D.N., Belhomme, G., Ferrall, J., Patterson, B., Styner, M., Prieto, J.C.: Trafic: fiber tract classification using deep learning. In: Medical Imaging 2018: Image Processing. vol. 10574, pp. 257–265. SPIE (2018)
2018
-
[13]
In: Graph Learning in Medical Imaging: First International Workshop, GLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Proceedings
Liu, F., Feng, J., Chen, G., Wu, Y., Hong, Y., Yap, P.T., Shen, D.: Deepbun- dle: fiber bundle parcellation with graph convolution neural networks. In: Graph Learning in Medical Imaging: First International Workshop, GLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, C...
2019
-
[14]
pp. 88–95. Springer (2019)
2019
-
[15]
Frontiers in radiology2, 866974 (2022)
Lucena, O., et al.: Informative and reliable tract segmentation for preoperative planning. Frontiers in radiology2, 866974 (2022)
2022
-
[16]
In: International Conference on Information Processing in Medical Imaging
Malcolm, J.G., Shenton, M.E., Rathi, Y.: Neural tractography using an unscented kalman filter. In: International Conference on Information Processing in Medical Imaging. pp. 126–138. Springer (2009)
2009
-
[17]
Progress in neurobiology 95(4), 629–635 (2011)
Marek, K., et al.: The parkinson progression marker initiative (ppmi). Progress in neurobiology 95(4), 629–635 (2011)
2011
-
[18]
IEEE transactions on medical imaging26(11), 1562–1575 (2007)
O’Donnell, L.J., Westin, C.F.: Automatic tractography segmentation using a high- dimensional white matter atlas. IEEE transactions on medical imaging26(11), 1562–1575 (2007)
2007
-
[19]
O’Donnell, L.J., Wells, W.M., Golby, A.J., Westin, C.F.: Unbiased groupwise regis- trationofwhitemattertractography.In:MedicalImageComputingandComputer- Assisted Intervention–MICCAI 2012: 15th International Conference, Nice, France, October 1-5, 2012, Proceedings, Part III 15....
2012
-
[20]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Qi, Charles, o.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 652–660 (2017)
2017
-
[21]
arXiv preprint arXiv:1609.02200 (2016)
Rolfe, J.T.: Discrete variational autoencoders. arXiv preprint arXiv:1609.02200 (2016)
2016 arXiv
-
[22]
Neuroinformatics20(4), 1093–1104 (2022)
St-Onge, E., et al.: Fast streamline search: an exact technique for diffusion mri tractography. Neuroinformatics20(4), 1093–1104 (2022)
2022
-
[23]
Neuroimage35(4), 1459–1472 (2007)
Tournier, J.D., Calamante, F., Connelly, A.: Robust determination of the fibre orientation distribution in diffusion mri: non-negativity constrained super-resolved spherical deconvolution. Neuroimage35(4), 1459–1472 (2007)
2007
-
[24]
Neuroimage23(3), 1176–1185 (2004)
Tournier, J.D., et al.: Direct estimation of the fiber orientation density function from diffusion-weighted mri data using spherical deconvolution. Neuroimage23(3), 1176–1185 (2004)
2004
-
[25]
Neuroimage 80, 62–79 (2013)
Van Essen, D.C., et al.: The wu-minn human connectome project: an overview. Neuroimage 80, 62–79 (2013)
2013
-
[26]
In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)
Vindas, N., et al.: Geolab: Geometry-based tractography parcellation of superficial white matter. In: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). pp. 1–5. IEEE (2023)
2023
-
[27]
Developmental cognitive neuroscience32, 4–7 (2018) 16 A
Volkow, N.D., et al.: The conception of the abcd study: From substance use to a broad nih collaboration. Developmental cognitive neuroscience32, 4–7 (2018) 16 A. Goel et al
2018
-
[28]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention
Wang, Z., et al.: Accurate corresponding fiber tract segmentation via fibergeomap learner. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 143–152. Springer (2022)
2022
-
[29]
NeuroImage183, 239–253 (2018)
Wasserthal, J., Neher, P., Maier-Hein, K.H.: Tractseg-fast and accurate white mat- ter tract segmentation. NeuroImage183, 239–253 (2018)
2018
-
[30]
IEEE transactions on medical imaging38(8), 1910–1922 (2019)
Xu, H., Dong, M., Lee, M.H., O’Hara, N., Asano, E., Jeong, J.W.: Objective de- tection of eloquent axonal pathways to minimize postoperative deficits in pediatric epilepsy surgery using diffusion tractography and convolutional neural networks. IEEE transactions on medical imag...
2019
-
[31]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Xue, T., et al.: Tractcloud: Registration-free tractography parcellation with a novel local-global streamline point cloud representation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 409–419. Springer (2023)
2023
-
[32]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Yu, X., Tang, L., Rao, Y., Huang, T., Zhou, J., Lu, J.: Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 19313– 19322 (2022)
2022
-
[33]
Neuroimage179, 429–447 (2018)
Zhang, F., Wu, Y., Norton, I., Rigolo, L., Rathi, Y., Makris, N., O’Donnell, L.J.: An anatomically curated fiber clustering white matter atlas for consistent white matter tract parcellation across the lifespan. Neuroimage179, 429–447 (2018)
2018
-
[34]
Medical Image Analysis65, 101761 (2020)
Zhang, F., et al.: Deep white matter analysis (deepwma): Fast and consistent tractography segmentation. Medical Image Analysis65, 101761 (2020)
2020
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.