REVIEW 3 major objections 5 minor 44 references
Semantic Style Transfer for Enhancing Animal Facial Landmark Detection
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Style-transferred cat face images, selected for landmark accuracy, improve a 48-landmark cat face detector's Normalized Mean Error from 9.144 to 7.638 on CatFLW—an approximately 16.5 percent relative error reduction.
desk verdict Plausible augmentation idea for animal landmark detection, but the evidence is too thin: one split, inconsistent numbers, and the key assumption that style transfer preserves landmark positions is never directly tested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is Splice ViT, a vision-transformer style transfer method that encodes content and style separately to keep spatial structure, wrapped in two procedural safeguards. Cropped-Face Style Transfer (CF-ST) removes background before stylization, raising segmentation IoU from 0.463 to 0.857. Supervised Style Transfer (SST) selects style source images by lowest landmark NME, so the generated faces are stylistically varied but structurally aligned; this filter is what lets the original 48 landmark annotations be reused.
What would settle it
Run the baseline detector on original and stylized versions of the same faces, align them by eye corners, and measure each landmark's displacement; if the average displacement is close to the 1.5-point NME gain, or if retraining on generated images with freshly corrected landmarks removes the 7.638 advantage, the improvement is label-noise regularization rather than semantic style diversity.
Extended reading notes
Core claim
The central claim is that neural style transfer, normally used for artistic rendering, becomes a useful semantic augmentation tool when the transfer is restricted to cropped faces and style sources are chosen by landmark accuracy. The paper reports three connected results: cropped-face style transfer preserves facial structure much better than full-body transfer; training on stylized images alone degrades performance because annotations no longer align, but Supervised Style Transfer—picking style sources with the lowest NME—cuts that degradation from 14.6 percent to 3.2 percent; and augmenting the original training set with these curated stylized images beats both the baseline and rotation augmentation, with the best configuration reaching NME 7.638 and failure rate 11.
Load-bearing premise
The load-bearing premise is that style-transferred images keep all 48 landmarks close enough to their original positions that reusing the original annotations is safe; the paper checks this with segmentation overlap and training loss rather than with landmark displacement measured on the generated images.
Editorial extensions
If this is right
- Because Train + TrainSST(N=1) outperforms Train + TrainRotated (NME 7.638 versus 9.829), semantic texture diversity adds value beyond geometric augmentation for this detector.
- The ordering Train + TrainSST(N=1) < Train + TrainSST(N=10) < Train + TrainSST(N=250) implies that restricting style diversity to well-aligned sources is what preserves landmark accuracy.
- Cropped-face style transfer should be preferred over full-body style transfer whenever the downstream task needs precise facial structure.
- The paper argues the pipeline generalizes to other species and landmark detectors because neither the style transfer nor the selection criterion is cat-specific.
Reading between the lines
- A direct test would be to measure per-landmark displacement between original and stylized faces; if the 1.5-point NME gain comes from label noise, correcting the labels on generated images should erase the Train + TrainSST(N=1) advantage.
- The reported gains come from a 500-image training subset; on the full 2,091-image CatFLW dataset, the margin over baseline could shrink as the baseline itself improves with more data.
- The SST source set could be refreshed during training as the detector improves, turning a one-time selection into a closed-loop augmentation schedule.
- At roughly 20 minutes per style-transferred image pair on the reported hardware, practical adoption on larger datasets may require faster stylizers or precomputed style banks.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using semantic neural style transfer, specifically Splice ViT, to augment training data for a 48-landmark cat facial landmark detector built on the CatFLW dataset. The authors compare full-body and cropped-face style transfer, report that cropped-face transfer better preserves structure (IoU 0.8574 vs 0.4634 and lower Splice ViT loss), and then use style-transferred images either as replacements or as augmentations of the original training set. A Supervised Style Transfer (SST) strategy selects style sources according to landmark prediction accuracy, and the paper claims that augmenting with Train+SST(N=1) gives NME 7.638 and FR 11, outperforming the baseline (NME 9.144, FR 21) and rotation augmentation (NME 9.829, FR 14).
Significance. If the reported result is robust, the claimed 16.5% relative NME improvement over the baseline on CatFLW would be practically relevant for animal affective computing, where labeled facial data are scarce and landmark detectors underpin downstream pain and expression analysis. The work is novel in applying semantic style transfer to animal landmark detection, and the comparison of full-body versus cropped-face transfer is a sensible preprocessing study. The SST idea—selecting style sources by landmark accuracy—is a reasonable attempt to control annotation misalignment. However, the paper provides no code, no repeated evaluation, and no direct measurement of landmark preservation in generated images, so the central claim is currently supported mainly by a single run with internal inconsistencies.
major comments (3)
- [Section 2.3 and Table 1] The headline result, Train+TrainSST(N=1) with NME 7.638, is selected after evaluating on the same 100-image test set, and the paper reports only one random 500/100 split with no error bars or significance tests. Because differences among the style-augmented variants are small (7.638 vs 7.780 vs 7.838) and failure rates are small counts, the reported ranking may reflect noise. Please report results over multiple splits or seeds, select N using a validation set or a pre-registered criterion, and include error bars or statistical tests.
- [Section 2.1, Eq. (1), and Section 2.2] The augmentation reuses original landmark coordinates for stylized images G(I,S), but the paper does not directly verify that style transfer preserves landmark positions. The SST criterion in Eq. (1) selects style sources with low NME on the original source images, which does not measure whether G(I,S) displaces landmarks, and the CF-ST validation uses segmentation IoU and Lsplice, both insensitive to local landmark shifts. If G(I,S) moves facial features, the apparent improvement could come from label noise acting as regularization rather than from semantic style diversity. Please add a landmark-specific evaluation of displacement between I and G(I,S) using the 48 annotated landmarks, and include a control augmentation with equivalent label noise.
- [Section 3 and Table 1] The prose and Table 1 contain several inconsistent numerical values: TrainSST(N=10) is reported as 9.441 in the text but 9.870 in Table 1; TrainSST(N=1) as 10.046 vs 10.482; TrainSST(N=250) as 10.123 vs 9.962; and Train+TrainRotated as 9.829 vs 9.256. These discrepancies prevent reproduction and undermine the ranking claims. Please provide a single consistent set of numbers and state which values support each conclusion.
minor comments (5)
- [Eq. (1)] The set-builder notation in Eq. (1) is mathematically unclear; 'argmin_N(NME(I_i))' is not a boolean condition. Consider rewriting the set as the N images in Train with the lowest NME values.
- [Section 3.2] The text references 'Figure ??' when discussing the reduction of region-specific NME; the figure reference is unresolved.
- [Section 2.3 and Figure 5] Region-specific NME is listed as an evaluation metric, but Figure 5 is not analyzed quantitatively in the text; please report the region-level numbers and interpret them for the claimed improvements.
- [References] Reference [26] appears to duplicate reference [25]; both are cited for the same claim about full-body style-transfer artifacts. Please merge or differentiate them.
- [General] The paper does not include a code or data availability statement. Given that the core claim is empirical and the computational cost is high, making at least the evaluation protocol available would support reproducibility.
Circularity Check
No significant circularity; the central claim is an empirical comparison on a held-out CatFLW test set.
full rationale
This paper is an empirical augmentation study, not a derivation. The central claims (style-transferred augmentation improves NME/FR) are evaluated on a held-out subset of CatFLW, and the style-transfer model (Splice ViT) and dataset are external resources. The Supervised Style Transfer selection in Eq. (1) uses baseline NME on training images to choose style sources, but the reported improvements are NME on a separate test set; no fitted constant is reused as the prediction. The citations to Martvel et al. supply the dataset and detector that are the target of the study, not a premise that forces the conclusion. The only mild issues are non-circular: the Splice ViT training loss is used as one of two structural-preservation metrics, the hyperparameter N=1 appears selected after observing test performance, and there are numerical inconsistencies between the text and Table 1 (e.g., TrainSST(N=10) is 9.441 in Section 3.1 but 9.870 in Table 1). These are correctness or selection-bias concerns, not circular reasoning. Accordingly, the circularity score is low.
Assumptions & free parameters
free parameters (2)
- Number of style sources N in SST =
10 for replacement; 1 for augmentation
- Random seed for 500/100 CatFLW subset split =
not reported
assumptions (3)
- domain assumption CatFLW annotations are reliable ground truth for all 48 landmarks.
- domain assumption Style-transferred images keep landmark positions valid enough to reuse original labels.
- domain assumption NME on one fixed 100-image test set is a stable measure of detector accuracy.
Cite this review
Pith. "Pith review of Semantic Style Transfer for Enhancing Animal Facial Landmark Detection." pith.science (2026). https://pith.science/paper/54JXACFR
@misc{pith2026250505640,
author = {Pith},
title = {Pith review of: Semantic Style Transfer for Enhancing Animal Facial Landmark Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/54JXACFR}},
note = {Machine review of arXiv:2505.05640}
}
read the original abstract
Neural Style Transfer (NST) is a technique for applying the visual characteristics of one image onto another while preserving structural content. Traditionally used for artistic transformations, NST has recently been adapted, e.g., for domain adaptation and data augmentation. This study investigates the use of this technique for enhancing animal facial landmark detectors training. As a case study, we use a recently introduced Ensemble Landmark Detector for 48 anatomical cat facial landmarks and the CatFLW dataset it was trained on, making three main contributions. First, we demonstrate that applying style transfer to cropped facial images rather than full-body images enhances structural consistency, improving the quality of generated images. Secondly, replacing training images with style-transferred versions raised challenges of annotation misalignment, but Supervised Style Transfer (SST) - which selects style sources based on landmark accuracy - retained up to 98% of baseline accuracy. Finally, augmenting the dataset with style-transferred images further improved robustness, outperforming traditional augmentation methods. These findings establish semantic style transfer as an effective augmentation strategy for enhancing the performance of facial landmark detection models for animals and beyond. While this study focuses on cat facial landmarks, the proposed method can be generalized to other species and landmark detection models.
Figures
Reference graph
Works this paper leans on
-
[1]
Convolutional neural network-based technique for gaze estimation on mo- bile devices
Andronicus A Akinyelu and Pieter Blignaut. Convolutional neural network-based technique for gaze estimation on mo- bile devices. Frontiers in Artificial Intelligence , 4:796825,
-
[2]
Al-Eidan, Hend Suliman Al-Khalifa, and Ab- dulMalik S
Rasha M. Al-Eidan, Hend Suliman Al-Khalifa, and Ab- dulMalik S. Al-Salman. Deep-learning-based models for pain recognition: A systematic review. Applied Sciences , 10:5984, 2020. 1
work page 2020
-
[3]
Localizing parts of faces using a consensus of exemplars
Peter N Belhumeur, David W Jacobs, David J Kriegman, and Neeraj Kumar. Localizing parts of faces using a consensus of exemplars. IEEE transactions on pattern analysis and machine intelligence, 35(12):2930–2940, 2013. 1
work page 2013
-
[4]
Sara Broome, Karina B. Gleerup, Pia H. Andersen, and Hed- vig Kjellstrom. Going deeper than tracking: A survey of computer vision-based recognition of animal pain and affec- tive states. arXiv preprint arXiv:2206.08405, 2022. 1
work page Pith review arXiv 2022
-
[5]
Sofia Broome, Marcelo Feighelstein, Anna Zamansky, Car- reira G Lencioni, Haubro P Andersen, Francisca Pessanha, Marwa Mahmoud, Hedvig Kjellstr¨om, and Albert Ali Salah. Going deeper than tracking: A survey of computer-vision based recognition of animal pain and emotions.International Journal of Computer Vision, 131(2):572–590, 2023. 1
work page 2023
-
[6]
Cattleface-rgbt: Rgb-t cattle facial landmark benchmark
Ethan Coffman, Reagan Clark, Nhat-Tan Bui, Trong Thang Pham, Beth Kegley, Jeremy G Powell, Jiangchao Zhao, and Ngan Le. Cattleface-rgbt: Rgb-t cattle facial landmark benchmark. arXiv preprint arXiv:2406.03431, 2024. 1
arXiv 2024
-
[7]
An image is worth 16x16 words: Trans- formers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1
arXiv 2010
-
[8]
Predicting dog emotions based on posture analysis using deeplabcut
Kim Ferres, Timo Schloesser, and Peter A Gloor. Predicting dog emotions based on posture analysis using deeplabcut. Future Internet, 14(4):97, 2022. 1
work page 2022
Show all 44 references
-
[9]
Gatys, Alexander S
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06583, 2015. 1
2015 arXiv
-
[10]
Generative adversarial nets
Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 1
2014
-
[11]
Using machine learn- ing and deeplabcut in animal behavior
Abigail Hardin and Ingo Schlupp. Using machine learn- ing and deeplabcut in animal behavior. acta ethologica , 25(3):125–133, 2022. 1
2022
-
[12]
Pose-informed face alignment for extreme head pose variations in animals
Charlie Hewitt and Marwa Mahmoud. Pose-informed face alignment for extreme head pose variations in animals. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), pages 1–6. IEEE, 2019. 1
2019
-
[13]
Deep learning based landmark detection for measuring hock and knee angles in sows.Translational Animal Science, page txad033, 2023
RL Jeon, JM Peschel, BC Ramirez, JD Stock, and KJ Stalder. Deep learning based landmark detection for measuring hock and knee angles in sows.Translational Animal Science, page txad033, 2023. 1
2023
-
[14]
Neural style transfer: A review
Yongcheng Jing, Yezhou Yang, Zunlei Feng, Jingwen Ye, Yizhou Yu, and Mingli Song. Neural style transfer: A review. IEEE transactions on visualization and computer graphics , 26(11):3365–3385, 2019. 1
2019
-
[15]
Perceptual losses for real-time style transfer and super-resolution
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, pages 694–711. Springer, 2016. 1
2016
-
[16]
Primate markerless pose estimation and movement analysis using deeplabcut
Rollyn Labuguen, Dean Karlo Bardeloza, Salvador Blanco Negrete, Jumpei Matsumoto, Kenichi Inoue, and Tomohiro Shibata. Primate markerless pose estimation and movement analysis using deeplabcut. In 2019 Joint 8th International Conference on Informatics, Electronics & Vision (IC...
2019
-
[17]
Interactive facial feature localization
Vuong Le, Jonathan Brandt, Zhe Lin, Lubomir Bourdev, and Thomas S Huang. Interactive facial feature localization. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part III 12, pages 679–692. Springer, 2012. 1
2012
-
[18]
Dog breed classification using part local- ization
Jiongxin Liu, Angjoo Kanazawa, David Jacobs, and Pe- ter Belhumeur. Dog breed classification using part local- ization. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7- 13, 2012, Proceedings, Part I 12 , pages 172–185. Springer,
2012
-
[19]
Murine motion behavior recognition based on deeplabcut and convolutional long short-term memory network
Ruiqing Liu, Juncai Zhu, and Xiaoping Rao. Murine motion behavior recognition based on deeplabcut and convolutional long short-term memory network. Symmetry, 14(7):1340,
-
[20]
Landmark-based facial feature construction and action unit intensity prediction
Jialei Ma, Xiansheng Li, Yuanyuan Ren, Ran Yang, and Qichao Zhao. Landmark-based facial feature construction and action unit intensity prediction. Mathematical Problems in Engineering, 2021:1–12, 2021. 1
2021
-
[21]
Head pose estimation us- ing facial-landmarks classification for children rehabilitation games
Salim Malek and Silvia Rossi. Head pose estimation us- ing facial-landmarks classification for children rehabilitation games. Pattern Recognition Letters, 152:406–412, 2021. 1
2021
-
[22]
Dogflw: Dog facial landmarks in the wild dataset
George Martvel, Greta Abele, Annika Bremhorst, Chiara Canori, Nareed Farhat, Giulia Pedretti, Ilan Shimshoni, and Anna Zamansky. Dogflw: Dog facial landmarks in the wild dataset. arXiv preprint arXiv:2405.11501, 2024. 1, 2
2024 arXiv
-
[23]
Catflw: Cat facial landmarks in the wild dataset
George Martvel, Nareed Farhat, Ilan Shimshoni, and Anna Zamansky. Catflw: Cat facial landmarks in the wild dataset. arXiv preprint arXiv:2305.04232, 2023. 2
2023 arXiv
-
[24]
Au- tomated video-based pain recognition in cats using facial landmarks
George Martvel, Teddy Lazebnik, Marcelo Feighelstein, Lea Henze, Sebastian Meller, Ilan Shimshoni, Friederike Twele, Alexandra Sch¨utter, Nora Foraita, Sabine K¨astner, et al. Au- tomated video-based pain recognition in cats using facial landmarks. Scientific Reports, 14(1):28...
2024
-
[25]
Automated landmark-based cat facial analysis and its appli- cations
George Martvel, Teddy Lazebnik, Marcelo Feighelstein, Se- bastian Meller, Ilan Shimshoni, Lauren Finka, Stelio PL Luna, Daniel S Mills, Holger A V olk, and Anna Zamansky. Automated landmark-based cat facial analysis and its appli- cations. Frontiers in veterinary science, 11:1...
2024
-
[26]
Luna, Daniel S
George Martvel, Teddy Lazebnik, Marcelo Feighelstein, Se- bastian Meller, Ilan Shimshoni, Lauren Finka, Stelio P.L. Luna, Daniel S. Mills, Holger A. V olk, and Anna Zaman- sky. Automated landmark-based cat facial analysis and its applications. Frontiers in Veterinary Science, ...
-
[27]
Auto- mated detection of cat facial landmarks
George Martvel, Ilan Shimshoni, and Anna Zamansky. Auto- mated detection of cat facial landmarks. International Jour- nal of Computer Vision, 2024. 1, 2
2024
-
[28]
Auto- mated detection of cat facial landmarks
George Martvel, Ilan Shimshoni, and Anna Zamansky. Auto- mated detection of cat facial landmarks. International Jour- nal of Computer Vision, pages 1–16, 2024. 2
2024
-
[29]
Arbitrary style transfer with style-attentional networks
Dongmok Park and Kangwook Lee. Arbitrary style transfer with style-attentional networks. Computer Vision and Pat- tern Recognition, 2018. 1
2018
-
[30]
Facial image-based automatic assessment of equine pain
Francisca Pessanha, Albert Ali Salah, Thijs van Loon, and Remco Veltkamp. Facial image-based automatic assessment of equine pain. IEEE Transactions on Affective Computing,
-
[31]
S. Qian, K. Sun, W. Wu, C. Qian, and Jia J. Aggregation via separation: Boosting facial landmark detector with semi- supervised style translation. ICCV, 2019. 2
2019
-
[32]
Real-time closed-loop feedback in behavioral time scales using deeplabcut.eneuro, 8(2), 2021
Keisuke Sehara, Paul Zimmer-Harwood, Matthew E Larkum, and Robert NS Sachdev. Real-time closed-loop feedback in behavioral time scales using deeplabcut.eneuro, 8(2), 2021. 1
2021
-
[33]
Khoshgoftaar
Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 2019. 2
2019
-
[34]
Neural style transfer: A critical review
Akhil Singh, Vaibhav Jaiswal, Gaurav Joshi, Adith Sanjeeve, Shilpa Gite, and Ketan Kotecha. Neural style transfer: A critical review. IEEE Access, 9:131583–131613, 2021. 1
2021
-
[35]
Cafm: A 3d morphable model for animals
Yifan Sun and Noboru Murata. Cafm: A 3d morphable model for animals. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, pages 20–24, 2020. 1
2020
-
[36]
Using deeplabcut as a real-time and markerless tool for cardiac physiology as- sessment in zebrafish
Michael Edbert Suryanto, Ferry Saputra, Kevin Adi Kurnia, Ross D Vasquez, Marri Jmelou M Roldan, Kelvin H-C Chen, Jong-Chin Huang, and Chung-Der Hsiao. Using deeplabcut as a real-time and markerless tool for cardiac physiology as- sessment in zebrafish. Biology, 11(8):1243, 2022. 1
2022
-
[37]
Splice vit: Semantic appearance transfer via vision transformer
Narek Tumanyan, Alon Bar, and Shai Bagon. Splice vit: Semantic appearance transfer via vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 10787–10796. IEEE, 2022. 2
2022
-
[38]
Splicing vit features for semantic appearance transfer
Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10748–10757, 2022. 1
2022
-
[39]
Phy- logenetic relationships, evolution, and genetic diversity of the domestic dog
Carles Vil `a, Jesus E Maldonado, and Robert K Wayne. Phy- logenetic relationships, evolution, and genetic diversity of the domestic dog. Journal of Heredity, 90(1):71–77, 1999. 1
1999
-
[40]
Deepwild: Application of the pose estimation tool deeplabcut for behaviour tracking in wild chimpanzees and bonobos
Charlotte Wiltshire, James Lewis-Cheetham, Viola Kome- dov´a, Tetsuro Matsuzawa, Kirsty E Graham, and Cather- ine Hobaiter. Deepwild: Application of the pose estimation tool deeplabcut for behaviour tracking in wild chimpanzees and bonobos. Journal of Animal Ecology, 92(8):1560–1574,
-
[41]
Look at boundary: A boundary-aware face alignment algorithm
Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2129– 2138, 2018. 1
2018
-
[42]
Facial landmark detection: A lit- erature survey
Yue Wu and Qiang Ji. Facial landmark detection: A lit- erature survey. International Journal of Computer Vision , 127(2):115–142, 2019. 1
2019
-
[43]
Facial expression recognition based on facial action unit
Jiannan Yang, Fan Zhang, Bike Chen, and Samee U Khan. Facial expression recognition based on facial action unit. In 2019 Tenth International Green and Sustainable Computing Conference (IGSC), pages 1–6. IEEE, 2019. 1
2019
-
[44]
Key points tracking and grooming behavior recognition of bactrocera minax (diptera: Trypetidae) via deeplabcut
Wei Zhan, Yafeng Zou, Zhangzhang He, and Zhiliang Zhang. Key points tracking and grooming behavior recognition of bactrocera minax (diptera: Trypetidae) via deeplabcut. Mathematical problems in engineering , 2021(1):1392362, 2021. 1 6
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.