Pith. sign in

REVIEW 4 major objections 5 minor 76 references

Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single Y-shaped network predicts depth, pose, normals, segmentation, and captions from one RGB image.

desk verdict A genuinely unified five-task conv net that runs in real time, but the quantitative evaluation leans on a metric that rewards predicting zero on sparse maps and on teacher labels that measure distillation rather than world accuracy. read the letter →

arxiv 2411.10334 v1 pith:RMAZBVUT submitted 2024-11-15 cs.CV

Ammar Qammaz , Nikolaos Vasilikopoulos , Iason Oikonomidis , Antonis A. Argyros This is my paper · ORCID
classification cs.CV
keywords multi-tasklearningknowledgedistillationmonoculardepthestimationsurfacenormals2Dhumanposesemanticsegmentationimagecaptioningreal-timeinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Y-MAP-Net aims to show that a single convolutional network can replace four separate foundation models for scene understanding. Given one RGB image, it simultaneously outputs 2D human pose (joints and part affinity fields), depth, surface normals, an 11-class semantic segmentation, and an eight-token caption, all from one forward pass. The paper's claim is that this is the first monolithic CNN to do so, and that the distillation setup preserves enough of each teacher's capability to be practically useful. Because the model is 297M parameters and runs at 15.9 Hz on an RTX 4080 SUPER, the wager is that real-time robotics and interactive systems can get multi-modal scene understanding without stacking heavyweight models.

What carries the argument

The load-bearing object is the Y-shaped topology: one shared encoder compresses the RGB input down to a bridge of three densely connected layers, then two specialized decoder branches fan out. The pictorial branch up-samples to 44 image channels; the token branch regresses eight GloVe embeddings normalized to [-1,1], with residual connections between successive tokens scaled to 30% and layer normalization and dropout to prevent token repetition. A second mechanism is the multi-teacher, single-student training paradigm: Depth Anything V2 supervises depth and normals, Detectron2 supervises class segmentation, DPText supervises text segmentation, and VisionGPT2 supervises captions, with a weighted MSE loss (gains 10.0 for GloVe, 2.4 for joints, 0.8 for PAFs, 1.0 for depth/normals/text, 3.0 for segmentation). An iterative post-processing step refines the depth output by using the predicted normals to correct depth gradients over 35 iterations.

What would settle it

Feed Y-MAP-Net a set of images with known metric depth, for example from a LiDAR-equipped RGB-D dataset, and compare its depth and normal outputs against the sensor measurements; if the normalized-depth teacher's bias toward relative, non-metric depth carries through, the student's depth errors against the sensor will be large even though its agreement with teacher-generated targets is high.

Watch

Extended reading notes

Core claim

The central claim is that a Y-shaped encoder-bridge-decoder CNN can be trained, with weak supervision from four teacher models, to predict all five modalities at once without catastrophic interference. The spatial decoder branch emits 44 heatmaps covering 17 joints, 12 PAF channels, depth, three normal channels, and 11 grouped segmentation classes, while the token branch emits eight GloVe vectors that decode into multi-label captions. The authors report that on the COCO17 validation set the student matches its teachers closely on joints, PAFs, and text/class segmentation, with depth being the weakest modality, and that the model uses 41.88% of the combined teacher weights. The paper frames the contribution as a practical bridge between heavyweight foundation models and sparse real-time convolutional networks like YOLO.

Load-bearing premise

Every quantitative quality claim except for joints and PAFs is measured against outputs generated by the teacher models themselves, so the student's high agreement with its teachers is taken as evidence of high quality rather than as agreement with real-world ground truth.

Editorial extensions

If this is right

  • A single 1.1 GB model can replace four foundation models totaling roughly 709M weights for these tasks, at 41.88% of their combined size.
  • The model reaches real-time rates (15.9 Hz on an RTX 4080 SUPER, 5.6 Hz on a GTX 1070), making it suitable for closed-loop robotic perception on commodity hardware.
  • The ablation study shows that adding the captioning branch and grouped segmentation preserves or improves pictorial heatmap fidelity, indicating that shared features across modalities are compatible in a monolithic design.
  • The normal-guided depth refinement step sharpens depth maps and suppresses noise, showing that the network can use its own cross-modal consistency to improve its weakest output.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the teacher models are themselves imperfect, the reported teacher-matching scores may overstate real-world accuracy; a metric-depth benchmark or human caption evaluation would be a stricter test.
  • The fixed eight-token captioning scheme, with stopwords removed and GloVe cosine matching, is a compact alternative to autoregressive captioning; it could be extended to variable-length captions or to token sets tuned for a specific domain such as factory scenes.
  • The grouping of 183 COCO classes into 11 broad categories limits segmentation granularity; a future variant with per-group decoders or hierarchical labels could recover fine-grained classes without the large output bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Y-MAP-Net, a Y-shaped convolutional architecture that processes a single RGB image and outputs, in one monolithic network, 17 joint heatmaps, 12 PAFs, one depth map, three surface-normal maps, text and class segmentation maps, and eight GloVe-embedding caption tokens. Training uses a multi-teacher, single-student scheme in which Depth Anything V2, Detectron2, DPText, and VisionGPT2 supervise the student on COCO17. The authors claim this is the first convolutional network to jointly perform depth, normal, pose, segmentation, and captioning, and report a model of 297M parameters running at 15.9 Hz on an RTX 4080 SUPER. Quantitative evaluation is performed on the COCO17 validation set using teacher-generated labels, with HDM and MSE metrics plus per-token caption cosine similarity. The central quantitative evidence is impaired by the use of the same teacher models for supervision and evaluation, and by the HDM metric's insensitivity to trivial agreement on sparse maps.

Significance. If properly validated, the proposed architecture would be practically significant: a single 297M-parameter convolutional network producing dense geometric, semantic, and textual outputs in real time could be attractive for robotics and embedded vision. The multi-teacher distillation organization, the detailed ablation study, and the explicit runtime measurements are useful contributions. However, the current validation does not establish the accuracy or the claimed 'strong generalization' of the model. The stress-test concern about HDM being dominated by background agreement is confirmed by the numbers in Table 2. The paper's strengths are its clear architectural description and the breadth of tasks addressed; its main weakness is that the quantitative claims rest on self-referential and metric-insensitive evaluation.

major comments (4)
  1. [§4.2, Table 2] With the exception of joints and PAFs, which use human COCO annotations, all quantitative results are evaluated against labels generated by the same teacher models that supervised training: depth and normals against DAv2, text segmentation against DPText, class segmentation against Detectron2, and captions against VisionGPT2 (as stated in §4.2 and §3.6). These scores therefore measure distillation fidelity, not real-world task accuracy. If DAv2 has systematic depth biases, the student can score well while being wrong in absolute terms. The paper needs independent evaluation on standard benchmarks (for example NYUv2 or KITTI for depth, ADE20K or COCO-Stuff mIoU for segmentation, and standard pose metrics) together with direct comparison against the teacher models on the same data.
  2. [§3.6, Table 2] The HDM metric is not a meaningful accuracy measure for the sparse output modalities. Joint heatmaps, PAFs, text segmentation, and class segmentation maps are zero over the vast majority of pixels; with T=0.1 and outputs normalized to [0,1], a constant-zero prediction satisfies the HDM condition on every background pixel. The reported HDM values of 0.95 (joints), 0.96 (PAFs), 0.94 (text), and 0.91 (class) are therefore consistent with a model that has learned to output near-zero maps and do not demonstrate localization or segmentation quality. The invariance of the HDM values across thresholds 0.1–0.8 in Table 2 is consistent with this interpretation. The authors should report foreground-only HDM, true-positive rates, and standard metrics such as PCK/OKS for pose and mIoU for segmentation.
  3. [§4.1, §4.2] All quantitative results come from single training runs without error bars, multiple seeds, or statistical significance tests, yet each ablation requires roughly one week of training. The comparison in Table 1 is therefore hard to interpret, and differences of a few hundredths in HDM may not be reliable. Moreover, the paper makes no quantitative comparison with any prior method on a common benchmark; the 'first to jointly address' claim is supported only by an internal ablation. The captioning evaluation uses per-token cosine similarity against teacher captions, without standard captioning metrics such as CIDEr, BLEU, or SPICE, or human evaluation. These additions are necessary to support the central claims of generalization and of replacing the teacher models at comparable quality.
  4. [§3.2, Figure 4] The iterative depth-refinement procedure is claimed to 'produce noticeable improvements,' but no quantitative evaluation of the refinement is given. Figure 4 shows only a qualitative example. The normals themselves are derived from the depth teacher via Sobel gradients, so the refinement is not an independent source of information; its benefit should be measured, for example by comparing depth HDM or standard depth metrics before and after refinement over the validation set.
minor comments (5)
  1. [§2] The text cites 'Humans-300M [30]' but reference [30] is the Sapiens paper, not a dataset; this citation appears to be incorrect.
  2. [Table 1] The column header 'Pos.Dpth' is not defined anywhere in the text; please spell out the meaning.
  3. [§3.3] The paper states that COCO17 contains 183 segmentation label categories; please verify this against the standard COCO ontology (80 thing categories plus 91 stuff categories, or the 133 panoptic categories).
  4. [Figure 6] The caption says the 'Segment.' column contains the union of columns 7-12, but the paper defines 11 segmentation outputs; the correspondence between the figure columns and the output channels is unclear.
  5. [§3.4] The sentence 'This helps with ordering the various responses of the network while training it' is vague; a more precise explanation of how GloVe embeddings encode ordering would improve readability.

Circularity Check

1 steps flagged · score 4.0 of 10

Depth, normal, segmentation, and captioning scores are measured against the same teacher models that generated the training targets, making those quantitative results self-referential rather than independent; the architecture itself is not circular.

  1. fitted input called prediction [Section 3.2 (depth teacher), Section 4.2 (quantitative evaluation), Table 2]
    "COCO17 [35], much like virtually all non-synthetic, in-the-wild datasets, does not contain depth data for training. For this reason, we selected Depth Anything V2 (DAv2) [68, 69] as a teacher model to generate them. ... We evaluated our model on the COCO17 validation set, using ground truth data generated by our teacher models."

    The depth, normal, text-segmentation, class-segmentation, and captioning validation targets are outputs of the same teacher models (DAv2, DPText, Detectron2, VisionGPT2) that produced the training targets in the multi-teacher distillation setup. The paper calls these teacher outputs 'ground truth' and reports HDM/MSE/cosine against them. A student trained with an MSE loss to mimic a fixed teacher on COCO train will, if it generalizes, approximate that same teacher on COCO val. The scores therefore certify distillation fidelity to the supervision source, not accuracy against independent ground truth.

full rationale

The only load-bearing reduction I can exhibit is the evaluation protocol. Depth, normals, text segmentation, class segmentation, and captioning are trained against teacher-generated targets (DAv2, DPText, Detectron2, VisionGPT2) and then evaluated against 'ground truth data generated by our teacher models' (Sections 3.2 and 4.2, Table 2). Consequently, the reported HDM/MSE/cosine scores for these modalities certify self-consistency with the supervision source rather than independent accuracy; the claim that Y-MAP-Net 'exhibits strong generalization' and can replace the teachers is therefore not independently supported. Joints and PAFs use real COCO annotations, which is independent, though the HDM metric with T=0.1 on sparse heatmaps is dominated by background agreement (the paper itself notes 'large inactive areas' for these modalities in Section 4.2), so the 0.95/0.96 numbers should not be read as localization accuracy. No self-citation chain or uniqueness theorem is invoked; the architecture and distillation training are not circular. Overall: partial self-referential validation, central architecture still has independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities, forces, or dimensions are introduced; the method combines existing components such as heatmaps, PAFs, GloVe embeddings, and teacher models. The load-bearing free parameters are the manually tuned loss weights and iterative refinement constants, plus the VRAM-driven architectural choice. The methodological axioms center on trusting the teacher outputs and the custom evaluation metric.

free parameters (3)
  • Loss weights g_i = gG=10.0, gJ=2.4, gPAF=0.8, gD=1.0, gN=1.0, gT=1.0, gS=3.0
    Experimentally determined in Section 3.6 to promote harder tasks; no sensitivity analysis is provided, and the final caption token scores decay from 0.59 to 0.01, suggesting these weights were tuned on the validation set.
  • Iterative depth refinement parameters = 35 iterations, alpha=0.01, low-depth mask
    Hand-set values in Section 3.2 with no ablation or sensitivity study; these strongly affect the reported depth improvement shown in Figure 4.
  • 1x1 pixelwise layer width = 1500 filters
    Chosen because of available GPU VRAM (Section 3), not justified by a performance search.
assumptions (4)
  • domain assumption Teacher models (DAv2, Detectron2, DPText, VisionGPT2) provide accurate supervision for depth, normals, segmentation, and captions.
    Used to generate both training targets and test labels (Sections 3.2, 4.2); no independent verification of teacher accuracy on the evaluation set is given.
  • domain assumption COCO17 with 118K samples is sufficient to train a multi-task network with strong generalization when augmented.
    Stated in Section 3.5; this is orders of magnitude less data than the teacher models use, yet strong generalization is claimed in Section 4.3.
  • domain assumption A shared convolutional bridge can represent all five modalities without destructive interference.
    Central architectural premise of the Y-shape (Section 3); no negative-transfer analysis is provided.
  • ad hoc to paper HDM (fraction of pixels within threshold T of target) is an appropriate quality metric.
    Defined in Section 3.6; this is not a standard benchmark metric and can be trivially high for low-variance predictions, especially at T=0.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images." pith.science (2026). https://pith.science/paper/RMAZBVUT

@misc{pith2026241110334,
  author       = {Pith},
  title        = {Pith review of: Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RMAZBVUT}},
  note         = {Machine review of arXiv:2411.10334}
}
read the original abstract

We present Y-MAP-Net, a Y-shaped neural network architecture designed for real-time multi-task learning on RGB images. Y-MAP-Net, simultaneously predicts depth, surface normals, human pose, semantic segmentation and generates multi-label captions, all from a single network evaluation. To achieve this, we adopt a multi-teacher, single-student training paradigm, where task-specific foundation models supervise the network's learning, enabling it to distill their capabilities into a lightweight architecture suitable for real-time applications. Y-MAP-Net, exhibits strong generalization, simplicity and computational efficiency, making it ideal for robotics and other practical scenarios. To support future research, we will release our code publicly.

Figures

Figures reproduced from arXiv: 2411.10334 by the authors.

Figure 1
Figure 1. Given an RGB frame, Y-MAP-Net estimates human [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Architecture of the proposed Y-MAP-Net. The flow of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Encoder (green) and decoder (magenta) blocks of Y [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: NN output normals (right) enforced on NN depth output [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Qualitative results from COCO17 validation data show [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 40 canonical work pages

  1. [1]

    The history began from alexnet: A comprehensive survey on deep learning approaches

    Md Zahangir Alom, Tarek M Taha, Christopher Yakopcic, Stefan Westberg, Paheding Sidike, Mst Shamima Nasrin, Brian C Van Esesn, Abdul A S Awwal, and Vijayan K Asari. The history began from alexnet: A comprehensive survey on deep learning approaches. arXiv preprint arXiv:1803.01164,

    arXiv

  2. [2]

    Jyoti Aneja, Aditya Deshpande, and Alexander G. Schwing. Convolutional image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 2, 3, 4, 6

    work page 2018

  3. [3]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 2

    arXiv 2023

  4. [4]

    Multimodal fu- sion via teacher-student network for indoor action recogni- tion

    XB Bruce, Yan Liu, and Keith CC Chan. Multimodal fu- sion via teacher-student network for indoor action recogni- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3199–3207, 2021. 2

    work page 2021

  5. [5]

    Realtime multi-person 2d pose estimation using part affinity fields

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, 2017. 2, 3, 4, 7, 8

    work page 2017

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 2

    2021

  7. [7]

    Higherhrnet: Scale- aware representation learning for bottom-up human pose es- timation

    Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S Huang, and Lei Zhang. Higherhrnet: Scale- aware representation learning for bottom-up human pose es- timation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5386–5395,

  8. [8]

    Learning phrase representations using rnn encoder-decoder for statistical machine translation

    Kyunghyun Cho. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014. 2, 4

    arXiv 2014

Show all 76 references
  1. [9]

    Keras documentation - model checkpoint

    Franc ¸ois Chollet et al. Keras documentation - model checkpoint. https://keras.io/api/callbacks/ model_checkpoint/, 2024. 6

    2024

  2. [10]

    Keras documentation - early stopping

    Franc ¸ois Chollet et al. Keras documentation - early stopping. https : / / keras . io / api / callbacks / early _ stopping/, 2024. 6

    2024

  3. [11]

    Unihcp: A unified model for human-centric perceptions

    Yuanzheng Ci, Yizhou Wang, Meilin Chen, Shixiang Tang, Lei Bai, Feng Zhu, Rui Zhao, Fengwei Yu, Donglian Qi, and Wanli Ouyang. Unihcp: A unified model for human-centric perceptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 178...

    2023

  4. [12]

    Adam: A method for stochastic opti- mization

    P Kingma Diederik. Adam: A method for stochastic opti- mization. (No Title), 2014. 6

    2014

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 2

    2010 arXiv

  6. [14]

    Abhimanyu Dubey et. al. The llama 3 herd of models, 2024. 2

    2024

  7. [15]

    OpenAI et. al. Gpt-4 technical report, 2024. 2

    2024

  8. [16]

    A stop list for general text

    Christopher Fox. A stop list for general text. SIGIR Forum, 24(1–2):19–21, 1989. 4, 5

    1989

  9. [17]

    Visiongpt2

    Shreyas Daniel Gaddam. Visiongpt2. https://github. com/shreydan/VisionGPT2, 2023. 7, 8

    2023

  10. [18]

    Understanding the diffi- culty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the diffi- culty of training deep feedforward neural networks. In Pro- ceedings of the thirteenth international conference on artifi- cial intelligence and statistics, pages 249–256. JMLR Work- shop and Conference Proceedings, 2010. 6

    2010

  11. [19]

    Densepose: Dense human pose estimation in the wild

    Rıza Alp G ¨uler, Natalia Neverova, and Iasonas Kokkinos. Densepose: Dense human pose estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7297–7306, 2018. 2, 3

    2018

  12. [20]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

    2016

  13. [21]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 2

    2017

  14. [22]

    Long short-term memory

    Sepp Hochreiter and J ¨urgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. 2, 4

    1997

  15. [23]

    A multi-instance multi-label dual learning approach for video captioning

    Wanting Ji and Ruili Wang. A multi-instance multi-label dual learning approach for video captioning. ACM Trans. Multimedia Comput. Commun. Appl., 17(2s), 2021. 5

    2021

  16. [24]

    Densecap: Fully convolutional localization networks for dense caption- ing

    Justin Johnson, Andrej Karpathy, and Li Fei-Fei. Densecap: Fully convolutional localization networks for dense caption- ing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 2

    2016

  17. [25]

    Panoptic studio: A massively multiview system for social motion capture

    Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh. Panoptic studio: A massively multiview system for social motion capture. In Proceedings of the IEEE inter- national conference on computer vision , pages 3334–3342,

  18. [26]

    Bag of tricks for efficient text classification

    Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759, 2016. 2

    2016 arXiv

  19. [27]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  20. [28]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. arXiv preprint arXiv:2408.12569, 2024. 2

    2024 arXiv

  21. [29]

    Sapiens discrete models repository

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens discrete models repository. https://huggingface.co/facebook/sapiens ,

  22. [30]

    Sapiens: Foundation for human vision mod- els

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els. In European Conference on Computer Vision , pages 206–228. Springer, 2025. 2, 3

    2025

  23. [31]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2

    2023

  24. [32]

    Visual genome: 9 Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, et al. Visual genome: 9 Connecting language and vision using crowdsourced dense image annotations. International journal of compu...

    2017

  25. [33]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. Advances in neural information processing systems , 25, 2012. 2

    2012

  26. [34]

    Motion-x: A large- scale 3d expressive whole-body human motion dataset

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-x: A large- scale 3d expressive whole-body human motion dataset. Ad- vances in Neural Information Processing Systems, 36, 2024. 2

    2024

  27. [35]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  28. [36]

    Learning features combination for human action recognition from skeleton sequences

    Diogo Carbonera Luvizon, Hedi Tabia, and David Picard. Learning features combination for human action recognition from skeleton sequences. Pattern Recognition Letters, 99: 13–20, 2017. 2

    2017

  29. [37]

    Exploring the limits of weakly supervised pretraining

    Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In Proceedings of the European con- ference on computer vision (ECCV) , pages 181–196, 2018. 2

    2018

  30. [38]

    Amass: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019. 2

    2019

  31. [39]

    Y-net: joint segmen- tation and classification for diagnosis of breast biopsy im- ages

    Sachin Mehta, Ezgi Mercan, Jamen Bartlett, Donald Weaver, Joann G Elmore, and Linda Shapiro. Y-net: joint segmen- tation and classification for diagnosis of breast biopsy im- ages. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Co...

    2018

  32. [40]

    Efficient estimation of word representa- tions in vector space

    Tomas Mikolov. Efficient estimation of word representa- tions in vector space. arXiv preprint arXiv:1301.3781, 3781,

    arXiv

  33. [41]

    Distributed representations of words and phrases and their compositionality

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in neural in- formation processing systems, 26, 2013. 2

    2013

  34. [42]

    Stacked hour- glass networks for human pose estimation, 2016

    Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hour- glass networks for human pose estimation, 2016. 2

    2016

  35. [43]

    Derpa- nis, and Kostas Daniilidis

    Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpa- nis, and Kostas Daniilidis. Coarse-to-fine volumetric predic- tion for single-image 3d human pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2017. 2

    2017

  36. [44]

    Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representa- tion. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, 2014. 2, 3, 5

    2014

  37. [45]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10106–10116, 2024. 2

    2024

  38. [46]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  39. [47]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 2, 4

    2020

  40. [48]

    You only look once: Unified, real-time object de- tection

    J Redmon. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. 1, 2

    2016

  41. [49]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6):1137–1149, 2016. 2

    2016

  42. [50]

    Lcr-net: Localization-classification-regression for human pose

    Gregory Rogez, Philippe Weinzaepfel, and Cordelia Schmid. Lcr-net: Localization-classification-regression for human pose. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3433–3441, 2017. 2

    2017

  43. [51]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  44. [52]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural In- f...

    2022

  45. [53]

    Very deep convo- lutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 2

    2014 arXiv

  46. [54]

    Revisiting unreasonable effectiveness of data in deep learning era

    Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhi- nav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE international conference on computer vision, pages 843–852, 2017. 2

    2017

  47. [55]

    Mask-yolo

    Jianing Sun. Mask-yolo. https://github.com/ jianing-sun/Mask-YOLO, 2018. 2

    2018

  48. [56]

    Deep high-resolution representation learning for human pose es- timation

    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose es- timation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5693–5703,

  49. [57]

    Going deeper with convolutions

    Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1–9, 2015. 2

    2015

  50. [58]

    Joint training of a convolutional network and a graphical model for human pose estimation

    Jonathan J Tompson, Arjun Jain, Yann LeCun, and Christoph Bregler. Joint training of a convolutional network and a graphical model for human pose estimation. Advances in neural information processing systems, 27, 2014. 2

    2014

  51. [59]

    Deeppose: Human pose estimation via deep neural networks

    Alexander Toshev and Christian Szegedy. Deeppose: Human pose estimation via deep neural networks. In Proceedings of 10 the IEEE conference on computer vision and pattern recog- nition, pages 1653–1660, 2014. 2

    2014

  52. [60]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2, 4

    2017

  53. [61]

    Deep high-resolution repre- sentation learning for visual recognition

    Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al. Deep high-resolution repre- sentation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence , 43(10):3...

    2020

  54. [62]

    Y-net: a one-to-two deep learning framework for digital holographic reconstruction

    Kaiqiang Wang, Jiazhen Dou, Qian Kemao, Jianglei Di, and Jianlin Zhao. Y-net: a one-to-two deep learning framework for digital holographic reconstruction. Optics letters, 44(19): 4765–4768, 2019. 2

    2019

  55. [63]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697– 20709, 2024. 2

    2024

  56. [64]

    Detectron2

    Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2. https://github. com/facebookresearch/detectron2 , 2019. 2, 3, 7, 8

    2019

  57. [65]

    Empirical evaluation of rectified activations in con- volutional network

    Bing Xu. Empirical evaluation of rectified activations in con- volutional network. arXiv preprint arXiv:1505.00853, 2015. 3

    2015 arXiv

  58. [66]

    Unifying flow, stereo and depth estimation

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Unifying flow, stereo and depth estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 7

    2023

  59. [67]

    Zoomnas: search- ing for whole-body human pose estimation in the wild.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):5296–5313, 2022

    Lumin Xu, Sheng Jin, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, and Xiaogang Wang. Zoomnas: search- ing for whole-body human pose estimation in the wild.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):5296–5313, 2022. 2

    2022

  60. [68]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 2, 3, 4, 7, 8

    2024

  61. [69]

    Depth any- thing v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. arXiv:2406.09414, 2024. 2, 4, 7, 8

    2024 arXiv

  62. [70]

    End-to-end learning of deformable mixture of parts and deep convolutional neural networks for human pose esti- mation

    Wei Yang, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang. End-to-end learning of deformable mixture of parts and deep convolutional neural networks for human pose esti- mation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3073–3082, 2016. 2

    2016

  63. [71]

    Dptext-detr: Towards better scene text detection with dynamic points in transformer

    Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Bo Du, and Dacheng Tao. Dptext-detr: Towards better scene text detection with dynamic points in transformer. In Pro- ceedings of the AAAI Conference on Artificial Intelligence , pages 3241–3249, 2023. 2, 3, 7, 8

    2023

  64. [72]

    Differential transformer, 2024

    Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, and Furu Wei. Differential transformer, 2024. 4

    2024

  65. [73]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043–9053, 2023. 2

    2023

  66. [74]

    Learn- ing from multiple teacher networks

    Shan You, Chang Xu, Chao Xu, and Dacheng Tao. Learn- ing from multiple teacher networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1285–1294, 2017. 2

    2017

  67. [75]

    Lite-hrnet: A lightweight high-resolution network

    Changqian Yu, Bin Xiao, Changxin Gao, Lu Yuan, Lei Zhang, Nong Sang, and Jingdong Wang. Lite-hrnet: A lightweight high-resolution network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10440–10450, 2021. 2

    2021

  68. [76]

    Attention heads of large language models: A survey

    Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Bo Tang, Feiyu Xiong, and Zhiyu Li. Attention heads of large language models: A survey. arXiv preprint arXiv:2409.03752, 2024. 4 11

    2024 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.