REVIEW 3 major objections 6 minor 19 references
The Phantom of the Elytra -- Phylogenetic Trait Extraction from Images of Rove Beetles Using Deep Learning -- Is the Mask Enough?
T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Binary masks of rove beetle bodies extract phylogenetic traits better than full-color segmentations or Fourier outlines.
desk verdict A useful first three-way comparison of mask, Fourier, and segmentation representations for DL-based phylogenetic trait extraction, but the headline result is confounded by a large architecture gap and the abstract overstates what the statistics support. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the comparison of three morphological representations fed into deep metric learning: binary masks, Fourier epicycle descriptors (200 coefficients plus angular velocities), and full dorsal segmentations. The mask and segmentation models use a ResNet50 encoder with 23.8 million parameters, while the Fourier model uses a small two-layer encoder DSeqEnc with 103,028 parameters; all produce 128-dimensional embeddings trained with triplet loss. Embeddings are turned into estimated phylogenies, and the normalized Align score and normalized Robinson-Foulds score compare those trees to the reference Rove-Tree-11 phylogeny, whose train/validation/test split follows beetle subfamilies so the model faces wholly unseen taxa.
What would settle it
Train the Fourier descriptors with a ResNet50 encoder, or the masks with DSeqEnc, under the same triplet-loss regime; if Fourier at matched capacity still scores near 0.45 while masks stay near 0.33, the representation claim holds, but if Fourier matches the mask score, the architecture difference, not the representation, explains the result.
Extended reading notes
Core claim
The paper's central claim is that, for extracting phylogenetic traits from rove beetle images, a simple binary mask of the beetle's silhouette outperforms both the original full-color dorsal segmentation and a Fourier-descriptor outline. The mask model achieved normalized Align score 0.33 ± 0.02 on the test set, where lower is better, against 0.39 ± 0.07 for full segmentations and 0.45 ± 0.01 for Fourier descriptors, and also the best normalized Robinson-Foulds score. The authors conclude that shape representations are especially useful for automated phylogenetic trait extraction and that dorsal textural features may be of reduced phylogenetic relevance in this group, though they flag the need for further investigation. They attribute the Fourier model's shortfall to reduced model capacity and to outline approximations that distort fine structures such as legs.
Load-bearing premise
The paper's comparison assumes the input representation, not the model architecture, causes the performance gap between the Fourier and mask models; the Fourier model used a far smaller network, so the gap could be an artifact of capacity.
Editorial extensions
If this is right
- If masks suffice, large museum image collections can be turned into phylogenetic trait matrices without manual measurement or color annotation.
- The closest phylogeny came from the simplest input, suggesting that redundant texture and color may act as noise for this task.
- Fourier descriptors, a compact classic morphometric tool, need higher-fidelity outlines or larger networks before they can compete with mask-based models.
- The same embedding approach could extend to other insect groups and to fossil or pinned specimens where DNA is unavailable.
- Automated trait extraction with shape representations may help combine morphological and molecular data in total-evidence phylogenies.
Reading between the lines
- The representation comparison is confounded with architecture: our inference, not the paper's, is that an equal-capacity test is needed before concluding Fourier descriptors are inherently worse; the paper acknowledges but does not control the confound.
- A testable extension would be feeding Fourier descriptors through ResNet50 or masks through DSeqEnc to isolate representation from capacity.
- If texture is truly low-signal, combining masks with targeted trait annotations (for example, pronotum shape) might outperform raw color segmentation.
- The mask result may partly reflect that binary silhouettes are invariant to lighting and background, which could make them an easier learning problem than raw color images.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares three morphological representations—binary masks, Fourier descriptors of outlines, and full color segmentations—as inputs to deep metric learning models for phylogenetic trait extraction from dorsal images of rove beetles. Using the Rove-Tree-11 dataset and its published phylogeny as reference, the authors train a ResNet50 on masks and segmentations and a small fully connected encoder (DSeqEnc) on Fourier descriptors, then evaluate the resulting embeddings by building phylogenies and computing normalized Align and Robinson-Foulds scores on a held-out subfamily. They report that masks achieve the best normalized Align score (0.33 ± 0.02) and conclude that shape representations are especially useful and that dorsal texture may have lower phylogenetic relevance. The Fourier model's lower performance is attributed in the Discussion to its much smaller capacity and to outline approximation artifacts.
Significance. The study addresses a timely and practical question: whether automated morphological trait extraction for phylogenetics should use silhouettes, outlines, or full appearance. The Rove-Tree-11 dataset and its fixed reference phylogeny provide a concrete benchmark, and the evaluation protocol with a held-out subfamily and repeated runs is a sensible way to measure generalization. The manuscript is honest in acknowledging the capacity confound between the Fourier and mask models, and it makes a useful contribution by documenting that a simple representation paired with a strong architecture can beat a richer representation paired with the same architecture. However, the headline claims go beyond what the controlled comparisons establish: the only statistically clear difference is between models that differ in both input representation and architecture, and the mask-versus-segmentation comparison, which is architecture-matched, is not significant. If the authors add a controlled capacity-matched comparison and temper the conclusions accordingly, the result would be a solid empirical contribution to the emerging field of image-based phylogenetic trait extraction.
major comments (3)
- [§Results, Table 2; §Methodology, Model Architectures; §Discussion] The main claim that mask-based representations outperform Fourier-based representations is not supported by a controlled comparison. Table 2 reports normalized Align scores of 0.33 ± 0.02 for masks with ResNet50 (23,770,304 parameters, Table 1) and 0.45 ± 0.01 for Fourier descriptors with DSeqEnc (103,028 parameters, Table 1). Because architecture capacity and input representation vary simultaneously, the nAS difference could be due to model capacity rather than to the Fourier representation itself; the Discussion explicitly concedes this. To support the conclusion that masks are the better representation, the authors should include a capacity-matched control—for example, a ResNet50 trained on Fourier-derived features or a DSeqEnc-sized network trained on masks—or otherwise show that the representation, not the architecture, drives the difference.
- [§Results, Table 2; §Discussion; §Conclusion] The comparison between masks and full segmentations does not support the conclusion that textural features have lower phylogenetic relevance. Table 2 gives 0.33 ± 0.02 for masks and 0.39 ± 0.07 for segmentations, with the segmentations row underlined as being within the confidence interval of the best model; the two results are not statistically distinguishable. Moreover, full segmentations contain both shape and texture, so even a true null difference between masks and segmentations would not identify texture as the cause. The abstract and conclusion should be revised to state that masks were not worse than segmentations in this experiment, and that the role of texture remains an unresolved hypothesis.
- [§Results; §Evaluation Metrics] The statement that masks 'significantly outperformed' the Fourier model is not backed by an inferential test. The paper reports 95% confidence intervals from five runs but does not report paired tests, effect sizes, or any statistical procedure for comparing the nAS distributions. Given that the confidence intervals for masks and segmentations overlap, the word 'significantly' is particularly misleading for the mask-versus-segmentation comparison. The authors should either add appropriate significance tests on the paired runs or use non-overlapping confidence intervals with a clearly stated criterion before using 'significant' in the Results.
minor comments (6)
- [Abstract and §Conclusion] The conclusion that 'the mask-based model outperformed the others' is too strong given that the segmentation result is within the confidence interval of the mask result; the wording should be qualified to say the mask model achieved the best point estimate but was not significantly different from the segmentation model.
- [§Methodology, Fourier Descriptors] The choice of 200 Fourier coefficients is motivated only by visual inspection ('see figure 2'); it would strengthen the paper to report how outline reconstruction error or downstream nAS changes with the number of coefficients, or at least to describe the reconstruction-error criterion more explicitly.
- [§Methodology, Data Preprocessing] The threshold value 250 and the 3x3 blur kernel are presented without an ablation or sensitivity analysis; since these choices directly determine the mask, a brief statement of their effect on performance or a reference justifying the values would improve reproducibility.
- [§Methodology, Model Architectures] The gradient accumulation configuration (mini-batch 8, 14 accumulation steps) is described, but the effective batch size is stated as mimicking 112; it would be helpful to confirm that all three models use the same effective batch size and learning-rate schedule, since differences in training protocol could also affect the comparison.
- [§Evaluation Metrics] The normalized Align score (nAS) and normalized Robinson-Foulds score (nRF) are not defined in the manuscript; readers should be told how the normalization is computed and how the random baseline in Table 2 was generated, so that the absolute values can be interpreted.
- [Throughout] There are several small language and completeness issues: 'An comparison' should be 'A comparison', 'our the data preprocessing' should be 'our data preprocessing', 'Fourier descriptors reduces' should be 'reduce', and the sentence in the Conclusion beginning 'In contrast, Fourier descriptors and segmented features, while informative, faced limitations, however, also demonstrated...' is grammatically incomplete.
Circularity Check
No circularity: the mask-vs-Fourier-vs-segmentation comparison is evaluated against a fixed, externally available reference phylogeny, and no fitted parameter is renamed as a prediction.
full rationale
The paper's central claim is an empirical comparison of three input representations for phylogenetic trait extraction, evaluated with normalized Align and Robinson-Foulds scores against the fixed reference tree in the Rove-Tree-11 dataset. The models are trained with triplet loss and then used to estimate phylogenies from the learned embeddings; the evaluation scores are computed by comparing those estimated trees to the reference tree, not by reusing the training loss or any fitted parameter as the metric. The mask representation is derived from the segmentation by simple thresholding, and the Fourier descriptors are derived from the mask outline, but these preprocessing steps are not fit to the evaluation metric. The Fourier model uses a smaller architecture (DSeqEnc, 103,028 parameters) than the mask model (ResNet50, 23,770,304 parameters), and the paper explicitly acknowledges this capacity confound in the Discussion, so this is a weakness in experimental control rather than a circular derivation. References to the authors' prior Rove-Tree-11 paper provide the dataset, architecture, and code base, but the dataset and its reference tree are public, fixed, and external to the fitted models in this paper; the models are not fitting the evaluation metric. The conclusion that dorsal texture may be less phylogenetically relevant is an interpretive speculation from a non-significant mask-versus-segmentation difference, not a claim forced by definition. No self-definitional, fitted-input-as-prediction, self-citation-load-bearing, uniqueness-imported, ansatz-smuggled, or renaming circularity is present.
Assumptions & free parameters
free parameters (5)
- Number of Fourier coefficients =
200
- Latent embedding size =
128
- Mask threshold =
250 after 3x3 blur
- Number of training epochs =
50
- Gradient accumulation settings =
mini-batch 8, 14 accumulation steps (effective 112)
assumptions (4)
- domain assumption The Rove-Tree-11 ground-truth phylogeny is a valid reference for trait extraction.
- domain assumption Triplet-loss embeddings can serve as proxies for phylogenetic traits.
- domain assumption Train/validation/test split by subfamily isolates generalization to unseen taxa.
- ad hoc to paper 200 Fourier coefficients preserve enough outline information for phylogenetic signal.
Cite this review
Pith. "Pith review of The Phantom of the Elytra -- Phylogenetic Trait Extraction from Images of Rove Beetles Using Deep Learning -- Is the Mask Enough?." pith.science (2026). https://pith.science/paper/BWQ77XLU
@misc{pith2026250204541,
author = {Pith},
title = {Pith review of: The Phantom of the Elytra -- Phylogenetic Trait Extraction from Images of Rove Beetles Using Deep Learning -- Is the Mask Enough?},
year = {2026},
howpublished = {\url{https://pith.science/paper/BWQ77XLU}},
note = {Machine review of arXiv:2502.04541}
}
read the original abstract
Phylogenetic analysis traditionally relies on labor-intensive manual extraction of morphological traits, limiting its scalability for large datasets. Recent advances in deep learning offer the potential to automate this process, but the effectiveness of different morphological representations for phylogenetic trait extraction remains poorly understood. In this study, we compare the performance of deep learning models using three distinct morphological representations - full segmentations, binary masks, and Fourier descriptors of beetle outlines. We test this on the Rove-Tree-11 dataset, a curated collection of images from 215 rove beetle species. Our results demonstrate that the mask-based model outperformed the others, achieving a normalized Align Score of 0.33 plus/minus 0.02 on the test set, compared to 0.45 plus/minus 0.01 for the Fourier-based model and 0.39 plus/minus 0.07 for the segmentation-based model. The performance of the mask-based model likely reflects its ability to capture shape features while taking advantage of the depth and capacity of the ResNet50 architecture. These results also indicate that dorsal textural features, at least in this group of beetles, may be of lowered phylogenetic relevance, though further investigation is necessary to confirm this. In contrast, the Fourier-based model suffered from reduced capacity and occasional inaccuracies in outline approximations, particularly in fine structures like legs. These findings highlight the importance of selecting appropriate morphological representations for automated phylogenetic studies and the need for further research into explainability in automatic morphological trait extraction.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Borowiec, M. L.; Dikow, R. B.; Frandsen, P. B.; McKeeken, A.; Valentini, G.; and White, A. E. 2022. Deep learning as a tool for ecology and evolution. Methods in Ecology and Evolution, 13(8): 1640--1660
work page 2022
-
[4]
Caple, J.; Byrd, J.; and Stephan, C. N. 2017. Elliptical Fourier analysis: fundamentals, applications, and value for forensic anthropology. International Journal of Legal Medicine, 131(6): 1675--1690
work page 2017
-
[5]
Hermann, K.; Chen, T.; and Kornblith, S. 2020. The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33: 19000--19015
work page 2020
-
[6]
Hofmann, M.; Kiel, S.; K \"o sters, L. M.; W \"a ldchen, J.; and M \"a der, P. 2024. Inferring Taxonomic Affinities and Genetic Distances Using Morphological Features Extracted from Specimen Images: A Case Study with a Bivalve Data Set. Systematic Biology, 73(6): 920--940
work page 2024
-
[7]
F.; Guttenberg, N.; Ledger, S.; Crowther, R.; and Huertas, B
Hoyal Cuthill, J. F.; Guttenberg, N.; Ledger, S.; Crowther, R.; and Huertas, B. 2019. Deep learning on butterfly phenotypes tests evolution’s oldest mathematical model. Science advances, 5(8): eaaw4967
work page 2019
-
[8]
P.; Bull, J.; and Cunningham, C
Huelsenbeck, J. P.; Bull, J.; and Cunningham, C. W. 1996. Combining data in phylogenetic analysis. Trends in Ecology & Evolution, 11(4): 152--158
work page 1996
Show all 19 references
-
[9]
Hunt, R.; and Pedersen, K. S. 2022. Rove-Tree-11: The not-so-Wild Rover, A hierarchically structured image dataset for deep metric learning research. In Proceedings of the Asian Conference on Computer Vision, 2967--2983
2022
-
[10]
K.; and Yamato, J
Kuhner, M. K.; and Yamato, J. 2015. Practical performance of tree comparison metrics. Systematic biology, 64(2): 205--214
2015
-
[11]
S.; and Palci, A
Lee, M. S.; and Palci, A. 2015. Morphological phylogenetics in the genomic age. Current Biology, 25(19): R922--R929
2015
-
[12]
Lestrel, P. E. 1997. Fourier descriptors and their applications in biology. Cambridge University Press
1997
-
[13]
K.; Hahn, M
Mo, Y. K.; Hahn, M. W.; and Smith, M. L. 2024. Applications of machine learning in phylogenetics. Molecular Phylogenetics and Evolution, 196: 108066
2024
-
[14]
M.; Lio, P.; and Gilks, W
Nye, T. M.; Lio, P.; and Gilks, W. R. 2006. A novel algorithm and web-based tool for comparing two alternative phylogenetic trees. Bioinformatics, 22(1): 117--119
2006
-
[15]
J.; Ingraham, J
Riesselman, A. J.; Ingraham, J. B.; and Marks, D. S. 2017. Deep generative models of genetic variation capture mutation effects. arXiv preprint arXiv:1712.06527
2017 arXiv
-
[16]
Shi, W.; Lu, C.; Shao, M.; Zhang, Y.; Xia, S.; and Koniusz, P. 2024. Few-shot shape recognition by learning deep shape-aware features. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 1848--1859
2024
-
[17]
Tsutsumi, M.; Saito, N.; Koyabu, D.; and Furusawa, C. 2023. A deep learning approach for morphological feature extraction based on variational auto-encoder: an application to mandible shape. NPJ systems biology and applications, 9(1): 30
2023
-
[18]
Xie, T.; and Zhang, C. 2024. ARTree: a deep autoregressive model for phylogenetic inference. Advances in Neural Information Processing Systems, 36
2024
-
[19]
Zhou, M.; Yan, Z.; Layne, E.; Malkin, N.; Zhang, D.; Jain, M.; Blanchette, M.; and Bengio, Y. 2023. PhyloGFN: Phylogenetic inference with generative flow networks. arXiv preprint arXiv:2310.08774
2023 arXiv
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.