Pith. sign in

REVIEW 5 major objections 6 minor 70 references

PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PoseBH trains one pose model on humans, animals, and whole bodies at once by matching keypoints to shared prototypes.

desk verdict Promising MDT recipe for pose estimation with heterogeneous skeletons, but the headline APT-36K gain is not yet attributable to the proposed mechanism because the training schedule is confounded. read the letter →

arxiv 2505.17475 v1 pith:A2JVVRJ4 submitted 2025-05-23 cs.CV

Uyoung Jeong , Jonathan Freer , Seungryul Baek , Hyung Jin Chang , Kwang In Kim This is my paper
classification cs.CV
keywords multi-datasettrainingposeestimationkeypointprototypesnonparametricprototypelearningcross-typeself-supervisionskeletonheterogeneitydomaintransferanimal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PoseBH is a recipe for training one pose-estimation network on many datasets whose skeletons do not match: humans with 17 or 16 keypoints, whole-body humans with 133, and animals with their own anatomies. The paper claims that representing every keypoint as an embedding matched against nonparametric prototypes lets these heterogeneous labels live in one space, and that a cross-type self-supervision loss fills in the keypoints that are unlabeled in any given image. If this works as described, multi-dataset training in pose estimation stops needing per-dataset heads, explicit keypoint correspondence, or relabeling, and the gains should carry over to unseen skeleton formats. On the reported benchmarks, the method beats the ViTPose++ multi-head baseline by 4.3 AP on average across animal and whole-body sets, with the largest jump on APT-36K, while keeping COCO, MPII, and AIC scores effectively unchanged.

What carries the argument

The central object is the nonparametric keypoint prototype bank $P \in \mathbb{R}^{J \times M \times F}$: one set of $M$ prototype vectors per keypoint class, across all $J = 214$ keypoints of the six training datasets, in an $F = 64$-dimensional normalized embedding space. Prototypes are never learned by gradient descent; each is a momentum average of the embedding vectors whose online Sinkhorn-Knopp assignment picks it, and pixel-prototype contrastive and distance losses train the embedding to land on the right prototype. The K-means clustering of all prototypes into 96 cross-dataset clusters is what turns the otherwise per-dataset contrastive objective into a cross-skeleton one, and the cosine-score heatmap generated by matching embeddings to prototypes is what makes the second prediction modality available for cross-type self-supervision.

What would settle it

Measure, on held-out images from COCO, AP-10K, APT-36K, and 3DPW, the mean cosine similarity between the embedding vectors of geometrically corresponding keypoints, such as COCO nose versus 3DPW jaw versus animal nose, after PoseBH training. If those similarities are not markedly higher than similarities to non-corresponding keypoints, or if removing the K-means cross-dataset negative term leaves the APT-36K gain unchanged, the claimed cross-skeleton unification is not what is driving the reported improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that keypoint heterogeneity is not a barrier to multi-dataset training if keypoints are treated as vectors in a shared embedding space rather than as dataset-specific output channels. A lightweight embedding head regresses a normalized keypoint embedding at every pixel; cosine similarity between that embedding and a bank of nonparametric prototypes produces a second heatmap, the prototype heatmap, for every dataset. During training the prototypes are updated from each dataset's own ground-truth foreground pixels by momentum averaging with Sinkhorn-Knopp online clustering, and one-shot K-means over all prototypes supplies cross-dataset negatives so that embeddings from different species are pushed apart unless they denote the same kind of landmark. A cross-type self-supervision signal then requires the dataset-specific head and the prototype head to agree, and confident, geometrically consistent predictions are blended into reliable heatmaps that supervise the keypoints a given image does not annotate. The paper reports that this joint design raises average AP from 69.1 to 73.4 on AP-10K, APT-36K, and COCO-WholeBody relative to ViTPose++, raises the average on COCO/AIC/MPII by 0.3, and that frozen-embedding transfer adds 0.9 AUC on InterHand2.6M and 1.9 AP on 3DPW.

Load-bearing premise

The load-bearing premise is that the unified embedding space spontaneously aligns anatomically matching keypoints from different skeletons, such as COCO nose with 3DPW jaw and COCO left hip with 3DPW pelvis, even though the training never receives any explicit cross-dataset keypoint correspondence.

Editorial extensions

If this is right

  • Multi-dataset pose training no longer needs a shared output label space; a new skeleton can be added by defining new keypoint classes and letting prototypes absorb them.
  • Models trained this way generalize to animal and whole-body benchmarks without sacrificing the standard human benchmarks, so practitioners can train one model instead of maintaining separate specialist models.
  • The learned embedding transfers to new domains such as hands and body shape by freezing the embedding module and only learning new prototypes, suggesting the embedding itself captures reusable semantic keypoint structure.
  • Because the self-supervision loss needs no teacher network or extra augmentations, the computational overhead of using unlabeled keypoints is small relative to distillation-based semi-supervised approaches.
  • A skeleton unseen at training time can still be predicted using an existing skeleton definition, though performance degrades under heavy occlusion and unusual poses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The strength of the result implies that a single embedding module could serve as a foundation for a much wider set of keypoint taxonomies, including medical or industrial landmarks, provided each new taxonomy contributes a few-shot sample to initialize its prototypes.
  • Inference: If the semantic alignment of prototypes is causal, then a direct evaluation of cross-dataset prototype alignment would be a cheap diagnostic: measuring whether anatomically corresponding keypoints across datasets land on nearby prototypes should predict transfer performance before fine-tuning.
  • Inference: The 11.2 AP gain on APT-36K compared with much smaller gains elsewhere suggests the main bottleneck being solved is not animal anatomy per se but video-specific pose variation; testing on another video-based animal set would separate those factors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces PoseBH, a multi-dataset training (MDT) framework for 2D pose estimation that addresses skeleton heterogeneity and label sparsity. The method has two main components: (i) nonparametric keypoint prototypes learned in a unified embedding space, which produce prototype-based keypoint heatmaps, and (ii) a cross-type self-supervision (CSS) loss that aligns the standard dataset-specific head predictions with the prototype-head predictions, generating reliable heatmaps for unlabeled keypoints. Experiments are conducted with a ViTPose++ multi-head baseline across COCO, AIC, MPII, AP-10K, APT-36K, and COCO-WholeBody, plus transfer experiments to InterHand2.6M and 3DPW. The central claim is that PoseBH substantially improves generalization on animal and whole-body pose datasets (AP-10K, APT-36K, COCO-WholeBody) while preserving performance on standard human benchmarks, and that the learned embeddings transfer to hand and body shape estimation.

Significance. If the results are reliable, PoseBH offers a practical recipe for multi-dataset pose training that goes beyond multi-head supervision, with a prototype-based unification of heterogeneous skeletons and a self-supervision signal that does not require teacher models or extra augmentations. The use of a shared experimental configuration with ViTPose++ (same backbone, datasets, and inference) is a methodological strength, as is the release of code. However, the current evidence does not fully support the attribution of the reported gains to the proposed components: the staged training schedule is not controlled, the ablation numbers are internally inconsistent, and all experiments are single-seed. These issues are fixable but need to be addressed before the central claim can be accepted.

major comments (5)
  1. [Sec. 4.2, Sec. 4.3, Supp. C.4] The staged training schedule is never ablated. PoseBH uses a 50/40/10 epoch curriculum in which the embedding module and prototypes are trained first, then heads and embeddings with prototypes frozen, and only later the full network with CSS. The ViTPose++ baseline is trained with the standard 100-epoch schedule. Because a curriculum that pretrains an auxiliary task and then fine-tunes the backbone can itself produce large gains on underrepresented datasets, the +11.2 AP on APT-36K and the +4.3 average in Table 2 cannot be attributed to keypoint prototypes or CSS without a schedule-matched baseline (e.g., ViTPose++ trained with the same 50/40/10 schedule and loss weighting, but without prototypes and CSS).
  2. [Sec. 4.6, Table 5] The ablation text and Table 5 are inconsistent. The text states that '+LProto' achieves an average score of 66.3, improving the baseline by 0.8, but Table 5 reports the baseline average as 68.2 and '+LProto' as 70.4, a gain of 2.2. The final configuration is described as outperforming the baseline by 2.4 points, which matches Table 5 (70.6 vs 68.2) but not the 66.3 figure. Please reconcile the text with the table and state which numbers are correct.
  3. [Sec. 4.2 vs. Supp. C.4] There is a direct contradiction about when the CSS loss is activated. The main text (Sec. 4.2) says 'during the last 10 epochs ... we introduce cross-type self-supervision loss,' while Supp. C.4 says 'At the start of the 50-th epoch, we set the backbone and the multi-heads to be trainable and freeze the prototypes. We then apply the L_CSS loss function.' Since the training schedule is central to the attribution of the results, please specify exactly at which epoch CSS is enabled and ensure the main text and supplementary agree.
  4. [Supp. C.4, Tables 1-4] All experiments use a single random seed (seed 0) with no error bars or repeated runs. The gains in Table 1 are small (e.g., +0.3 AP on COCO, +0.5 AP on AIC, +0.1 PCKh on MPII) and could easily be within run-to-run variation. Please provide results over multiple seeds (at least 3) with mean and standard deviation for the main tables, or otherwise justify that the observed differences are significant.
  5. [Sec. 3.2, Fig. 11] The claim that the unified keypoint embedding space is semantically consistent across heterogeneous skeletons is supported only by a qualitative t-SNE visualization (Fig. 11). Since prototypes are updated only from each dataset's own ground-truth foreground pixels and cross-dataset interaction is limited to one-time K-means cluster negatives, a quantitative evaluation of cross-dataset alignment is needed. For example, report the fraction of nearest-neighbor prototypes for each dataset's keypoints that correspond to semantically matching keypoints from other datasets, or show that cross-dataset contrastive learning changes prototype assignments in a meaningful way.
minor comments (6)
  1. [Sec. 3.2, Eq. (3)] The Sinkhorn-Knopp iteration in Eq. (3) is not fully explained. Please clarify the roles of u and v and the normalization steps, or refer the reader to the original derivation in [39] with enough detail to reproduce the computation.
  2. [Sec. 3.3, Eq. (7)] The confidence scores c^{kpt}_i and c^{emb}_i are not explicitly defined. Please state how they are computed from the heatmap predictions (e.g., the maximum value of the heatmap, or the value at the predicted location).
  3. [Sec. 4.1] The sentence 'COCO-WholeBody extends COCO with whole-body annotations covering 133 keypoints [52]' cites reference [52], which is the AI Challenger paper. Please correct this citation to the COCO-WholeBody paper.
  4. [Table 3] The ViTPose++-B row reports an AP_50 value of 73.4, which seems implausibly low for COCO (the HRNet baseline reports 90.6). Please check the column alignment and verify all numbers in this table.
  5. [Sec. 4.3] The text says 'UniHCP lags behind with an average score of 27.2,' but Table 2 reports UniHCP's average as 46.2. Please correct this inconsistent number.
  6. [Supp. C.1] The hyperparameters α, β, γ, δ are listed in the supplementary but never defined in the main text or the supplement. Please define each loss weight and which loss term it scales.

Circularity Check

1 steps flagged · score 4.0 of 10

Cross-type self-supervision (Eqs. 7-8) is a self-distillation loop whose targets are generated from the model's own outputs; the central prototype-based MDT claim remains independently grounded.

  1. self definitional [Section 3.3, Eqs. (7)-(8)]
    "To eliminate uncertain keypoint predictions, we apply two filtering conditions... The filtered predictions are then combined using a weighted average: ŷ_i = s_i ŷ^{kpt}_i + (1−s_i) ŷ^{emb}_i ... From ŷ, we generate a reliable heatmap k^{CSS} following the standard GT heatmap generation process. The loss for unlabeled samples is then computed as: L_{CSS} = Σ_d ζ [L_{hm}(k_d[u], k^{CSS}_d[u]) + L_{Proto}(e[u], k_d[u], k^{CSS}_d[u])]."

    The 'reliable heatmap' k^{CSS} that supervises unlabeled keypoints is constructed in Eq. (7) from the same model's two prediction streams, ŷ^{kpt} (multi-head keypoint output) and ŷ^{emb} (prototype/embedding output). Eq. (8) then trains those same streams to match this self-generated target, so the supervision signal is defined in terms of the very outputs it supervises: the model is its own teacher (the paper itself calls this 'self-distillation'). The gain attributed to L_{CSS} in Table 5 is only +0.2 AP on average, so this loop does not by itself force the headline multi-dataset results; the central prototype/MDT claim rests on the L_{Proto} branch, which is trained on ground-truth foreground pixels and external benchmark comparisons.

full rationale

The headline claim that PoseBH improves multi-dataset pose estimation is not circular: prototype learning (Eqs. 4-6) is driven by ground-truth foreground pixels and is evaluated against external benchmarks (COCO, MPII, AIC, AP-10K, APT-36K, COCO-WholeBody, InterHand2.6M, 3DPW) with ablations; no self-citation chain carries the argument. The one genuinely self-referential component is cross-type self-supervision (Eqs. 7-8), whose pseudo-labels are a filtered combination of the model's own two heads, making L_CSS a self-distillation loop; however its measured contribution is small (+0.2 AP), so it does not reduce the central claim. Separately, the staged 50/40/10 schedule is not ablated against ViTPose++, and the main text and Supp. C.4 disagree on when L_CSS starts; these are attribution/correctness issues rather than circularity and do not affect the circularity score. Score 4 reflects one partial self-supervision loop with the central claim independently grounded.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim rests on five premises the reader does not pay for upstream: the Sinkhorn-Knopp assignment (standard math inherited from ProtoSeg), the reliability of ground-truth foreground pixels as prototype samples (domain assumption), the emergence of cross-dataset semantic alignment without explicit correspondence (domain assumption, the true crux), the reliability of mutual head/prototype agreement as a pseudo-label source (ad hoc to this paper), and the validity of the author-chosen APT-36K split (domain assumption). The free parameters are standard hyperparameters, but several (c_thr, d_thr, M, K, zeta) directly control the two headline mechanisms, and the supplementary sensitivity tables show near-zero variation on COCO, so they do not demonstrate that the APT-36K-scale gains are stable under hyperparameter change.

free parameters (6)
  • confidence threshold c_thr = 0.25
    Hand-set threshold for the cross-type self-supervision mutual filtering (Sec 3.3); controls how many head/prototype predictions become pseudo-labels. No sensitivity analysis is reported on the datasets where gains are largest.
  • distance threshold d_thr = 2.1
    Hand-set RMS distance threshold for filtering head/prototype agreement in CSS (Sec 3.3); directly controls pseudo-label quantity and quality.
  • in-class prototypes per keypoint M = 3
    Number of prototypes per keypoint (supp C.1); central to the prototype representation in Eq. (1) and to prototype-update behavior in Eq. (4).
  • K-means cluster count K = 96
    Number of clusters used once during training to form cross-dataset negatives (Sec 3.2); the main cross-dataset alignment mechanism.
  • CSS loss weight zeta = 0.001
    Weight on the cross-type self-supervision loss in Eq. (8); supplementary Tab 8 shows COCO AP varies within 0.1 across values, so the choice is not strongly constrained by data.
  • embedding dimension F = 64
    Dimension of the unified keypoint embedding (supp C.1); Tab 6 shows COCO AP flat at 77.1 for F in {32, 64, 128}, so the chosen value is not data-driven.
assumptions (5)
  • standard math Sinkhorn-Knopp iteration converges to a doubly-stochastic assignment for online prototype assignment (Eq. 3).
    Inherited from ProtoSeg [69] and used without proof in Sec 3.2 for computing in-class prototype targets t_j.
  • domain assumption Ground-truth heatmap foreground pixels (nonzero values) are reliable, representative samples for each keypoint class.
    Prototype updates in Eq. (4) and contrastive losses in Eqs. (5)-(6) are computed only on foreground pixels; noisy or mislabeled GT pixels would corrupt prototypes and the embedding.
  • domain assumption Semantic alignment across skeletons emerges from shared weights and cross-dataset contrastive negatives without explicit keypoint correspondence.
    The core unification premise of Sec 3.2; no cross-dataset keypoint matching supervision is provided, so alignment must emerge from the shared backbone/embedding, which is only visualized via t-SNE in Fig 11.
  • ad hoc to paper Mutual agreement between the dataset-specific head and the prototype branch (confidence above c_thr, distance below d_thr) indicates a correct keypoint label.
    Defines the 'reliable heatmap' in Sec 3.3; if both branches share the same error, for example common backbone bias, the pseudo-labels reinforce it.
  • domain assumption The 7:1:2 APT-36K split matches the original paper's guidance and is comparable to previously reported numbers.
    APT-36K has no official train/val/test split; the authors partition it themselves (supp C.2), so all reported APT-36K comparisons (Tabs 2, 5) rest on this chosen split.
invented entities (2)
  • Nonparametric keypoint prototypes P in R^{J x M x F}
    purpose: Unified representation of all keypoints across datasets; classification via cosine similarity with pixel embeddings (Eq. 1).
    Internal model component, not a physical or external entity; its utility is measured only through the method's own benchmarks and t-SNE visualization, with no falsifiable handle outside the paper.
  • Reliable heatmap generated by cross-type self-supervision
    purpose: Pseudo-labels for unlabeled keypoints created by mutually filtering head and prototype predictions (Eq. 7, Sec 3.3).
    Labels generated by the model itself; no independent external verification of pseudo-label correctness is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation." pith.science (2026). https://pith.science/paper/A2JVVRJ4

@misc{pith2026250517475,
  author       = {Pith},
  title        = {Pith review of: PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A2JVVRJ4}},
  note         = {Machine review of arXiv:2505.17475}
}
read the original abstract

We study multi-dataset training (MDT) for pose estimation, where skeletal heterogeneity presents a unique challenge that existing methods have yet to address. In traditional domains, \eg regression and classification, MDT typically relies on dataset merging or multi-head supervision. However, the diversity of skeleton types and limited cross-dataset supervision complicate integration in pose estimation. To address these challenges, we introduce PoseBH, a new MDT framework that tackles keypoint heterogeneity and limited supervision through two key techniques. First, we propose nonparametric keypoint prototypes that learn within a unified embedding space, enabling seamless integration across skeleton types. Second, we develop a cross-type self-supervision mechanism that aligns keypoint predictions with keypoint embedding prototypes, providing supervision without relying on teacher-student models or additional augmentations. PoseBH substantially improves generalization across whole-body and animal pose datasets, including COCO-WholeBody, AP-10K, and APT-36K, while preserving performance on standard human pose benchmarks (COCO, MPII, and AIC). Furthermore, our learned keypoint embeddings transfer effectively to hand shape estimation (InterHand2.6M) and human body shape estimation (3DPW). The code for PoseBH is available at: https://github.com/uyoung-jeong/PoseBH.

Figures

Figures reproduced from arXiv: 2505.17475 by the authors.

Figure 1
Figure 1. PoseBH unifies diverse skeleton formats, including hu [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the PoseBH architecture. During training, the embedding head maps the backbone features into a unified keypoint [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustrative example of cross-type self-supervision. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparative pose estimation results on human (left) and animal (dog; right). [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparative results on InterHand2.6M (a–c) and 3DPW (e–f). [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Examples of SMPL fitting using KITRO and with our [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Pose estimation examples comparing ViTPose++ and our method. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Pose estimation examples for animals using ViTPose++ and our method. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Pose estimation examples from our algorithm on the InterHand2.6M dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Pose estimation examples from our algorithm on the 3DPW dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: t-SNE visualization of the prototypes. Best viewed when zoom-in. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Example pose predictions: (a-b) CrowdPose predictions using the COCO skeleton; (c–d) COP3D predictions using the AP-10K [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 67 canonical work pages

  1. [1]

    2D human pose estimation: New benchmark and state of the art analysis

    Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2D human pose estimation: New benchmark and state of the art analysis. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2014. 1, 5

    work page 2014

  2. [2]

    PoseTrack: A benchmark for human pose estima- tion and tracking

    Mykhaylo Andriluka, Umar Iqbal, Eldar Insafutdinov, Leonid Pishchulin, Anton Milan, Juergen Gall, and Bernt Schiele. PoseTrack: A benchmark for human pose estima- tion and tracking. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5167–5176,

    work page

  3. [3]

    This looks like that: Deep learn- ing for interpretable image recognition.Advances in Neural Information Processing Systems (NeurIPS), 32, 2019

    Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: Deep learn- ing for interpretable image recognition.Advances in Neural Information Processing Systems (NeurIPS), 32, 2019. 3

    work page 2019

  4. [4]

    ScaleDet: A scalable multi-dataset object detector

    Yanbei Chen, Manchen Wang, Abhay Mittal, Zhenlin Xu, Paolo Favaro, Joseph Tighe, and Davide Modolo. ScaleDet: A scalable multi-dataset object detector. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7288–7297, 2023. 2

    work page 2023

  5. [5]

    Learning to estimate robust 3D human mesh from in-the-wild crowded scenes

    Hongsuk Choi, Gyeongsik Moon, JoonKyu Park, and Ky- oung Mu Lee. Learning to estimate robust 3D human mesh from in-the-wild crowded scenes. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2

    work page 2022

  6. [6]

    UniHCP: A unified model for human-centric perceptions

    Yuanzheng Ci, Yizhou Wang, Meilin Chen, Shixiang Tang, Lei Bai, Feng Zhu, Rui Zhao, Fengwei Yu, Donglian Qi, and Wanli Ouyang. UniHCP: A unified model for human-centric perceptions. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17840–17852, 2023. 1, 2

    work page 2023

  7. [7]

    Where are we with human pose estimation in real- world surveillance? InIEEE/CVF Winter Conference on Ap- plications of Computer Vision, pages 591–601, 2022

    Mickael Cormier, Aris Clepe, Andreas Specker, and J ¨urgen Beyerer. Where are we with human pose estimation in real- world surveillance? InIEEE/CVF Winter Conference on Ap- plications of Computer Vision, pages 591–601, 2022. 1

    work page 2022

  8. [8]

    Weakly-supervised domain adaptive semantic segmentation with prototypical contrastive learning

    Anurag Das, Yongqin Xian, Dengxin Dai, and Bernt Schiele. Weakly-supervised domain adaptive semantic segmentation with prototypical contrastive learning. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 15434–15443, 2023. 3

    work page 2023

Show all 70 references
  1. [9]

    De- formable protopnet: An interpretable image classifier using deformable prototypes

    Jon Donnelly, Alina Jade Barnett, and Chaofan Chen. De- formable protopnet: An interpretable image classifier using deformable prototypes. InIEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 10265– 10275, 2022. 3

    2022

  2. [10]

    Weakly supervised semantic segmentation by pixel-to-prototype contrast

    Ye Du, Zehua Fu, Qingjie Liu, and Yunhong Wang. Weakly supervised semantic segmentation by pixel-to-prototype contrast. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4320–4329, 2022. 3

    2022

  3. [11]

    Sigmoid- weighted linear units for neural network function approxima- tion in reinforcement learning.Neural networks, 107:3–11,

    Stefan Elfwing, Eiji Uchibe, and Kenji Doya. Sigmoid- weighted linear units for neural network function approxima- tion in reinforcement learning.Neural networks, 107:3–11,

  4. [12]

    Human pose as compositional tokens

    Zigang Geng, Chunyu Wang, Yixuan Wei, Ze Liu, Houqiang Li, and Han Hu. Human pose as compositional tokens. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 6, 13

    2023

  5. [13]

    Human POSEitioning System (HPS): 3D human pose estimation and self-localization in large scenes from body-mounted sensors

    Vladimir Guzov, Aymen Mir, Torsten Sattler, and Gerard Pons-Moll. Human POSEitioning System (HPS): 3D human pose estimation and self-localization in large scenes from body-mounted sensors. InIEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 4318– 4...

    2021

  6. [14]

    A graph-based approach for category-agnostic pose estimation, 2024

    Or Hirschorn and Shai Avidan. A graph-based approach for category-agnostic pose estimation, 2024. 2

    2024

  7. [15]

    NeuMan: Neural human radiance field from a single video

    Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, and Anurag Ranjan. NeuMan: Neural human radiance field from a single video. InEuropean Conference on Computer Vision (ECCV), 2022. 1

    2022

  8. [16]

    MAS: Multi-view ancestral sampling for 3D mo- tion generation using 2D diffusion

    Roy Kapon, Guy Tevet, Daniel Cohen-Or, and Amit H Bermano. MAS: Multi-view ancestral sampling for 3D mo- tion generation using 2D diffusion. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1965–1974, 2024. 1

    1965

  9. [17]

    Sapiens: Foundation for human vision mod- els.arXiv preprint arXiv:2408.12569, 2024

    Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision mod- els.arXiv preprint arXiv:2408.12569, 2024. 2

    2024 arXiv

  10. [18]

    Huang, Otmar Hilliges, and Michael J

    Muhammed Kocabas, Chun-Hao P. Huang, Otmar Hilliges, and Michael J. Black. PARE: Part attention regressor for 3D human body estimation. InIEEE/CVF International Confer- ence on Computer Vision (ICCV), pages 11127–11137, 2021. 2

    2021

  11. [19]

    Human pose estimation for mitigating false negatives in weapon detection in video-surveillance

    Alberto Lamas, Siham Tabik, Antonio Cano Montes, Fran- cisco P ´erez-Hern´andez, Jorge Garc ´ıa, Roberto Olmos, and Francisco Herrera. Human pose estimation for mitigating false negatives in weapon detection in video-surveillance. Neurocomputing, 489:488–503, 2022. 1

    2022

  12. [20]

    MSeg: A composite dataset for multi- domain semantic segmentation

    John Lambert, Zhuang Liu, Ozan Sener, James Hays, and Vladlen Koltun. MSeg: A composite dataset for multi- domain semantic segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2

    2020

  13. [21]

    JRDB-PanoTrack: An open-world panoptic segmentation and tracking robotic dataset in crowded human environments

    Duy Tho Le, Chenhui Gou, Stavya Datta, Hengcan Shi, Ian Reid, Jianfei Cai, and Hamid Rezatofighi. JRDB-PanoTrack: An open-world panoptic segmentation and tracking robotic dataset in crowded human environments. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (C...

    2024

  14. [22]

    SimCC: A simple coordinate classification perspective for human pose estimation

    Yanjie Li, Sen Yang, Peidong Liu, Shoukui Zhang, Yunx- iao Wang, Zhicheng Wang, Wankou Yang, and Shu-Tao Xia. SimCC: A simple coordinate classification perspective for human pose estimation. InEuropean Conference on Com- puter Vision (ECCV), pages 89–106. Springer, 2022. 6, 13

    2022

  15. [23]

    Motion-X: A large- scale 3D expressive whole-body human motion dataset.Ad- vances in Neural Information Processing Systems (NeurIPS),

    Jing Lin, Ailing Zeng, Shunlin Lu, Yuanhao Cai, Ruimao Zhang, Haoqian Wang, and Lei Zhang. Motion-X: A large- scale 3D expressive whole-body human motion dataset.Ad- vances in Neural Information Processing Systems (NeurIPS),

  16. [24]

    Microsoft COCO: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In European Conference on Computer Vision (ECCV), pages 740–755. Springer, 2014. 1, 5

    2014

  17. [25]

    Learning orthogonal pro- totypes for generalized few-shot semantic segmentation

    Sun-Ao Liu, Yiheng Zhang, Zhaofan Qiu, Hongtao Xie, Yongdong Zhang, and Ting Yao. Learning orthogonal pro- totypes for generalized few-shot semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 3

    2023

  18. [26]

    HumanGaus- sian: Text-driven 3D human generation with gaussian splat- ting

    Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, and Ziwei Liu. HumanGaus- sian: Text-driven 3D human generation with gaussian splat- ting. InIEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 6646–6657, 2024. 1

    2024

  19. [27]

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi- person linear model.ACM Trans. Graphics (Proc. SIG- GRAPH Asia), 34(6):248:1–248:16, 2015. 6

    2015

  20. [28]

    ProMotion: Prototypes as motion learners

    Yawen Lu, Dongfang Liu, Qifan Wang, Cheng Han, Yim- ing Cui, Zhiwen Cao, Xueling Zhang, Yingjie Victor Chen, and Heng Fan. ProMotion: Prototypes as motion learners. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28109–28119, 2024. 3

    2024

  21. [29]

    InterHand2.6M: A dataset and baseline for 3D interacting hand pose estimation from a single rgb im- age

    Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee. InterHand2.6M: A dataset and baseline for 3D interacting hand pose estimation from a single rgb im- age. InEuropean Conference on Computer Vision (ECCV),

  22. [30]

    Neu- ralAnnot: Neural annotator for 3D human mesh training sets

    Gyeongsik Moon, Hongsuk Choi, and Kyoung Mu Lee. Neu- ralAnnot: Neural annotator for 3D human mesh training sets. InComputer Vision and Pattern Recognition Workshop (CVPRW), 2022. 2

    2022

  23. [31]

    Three Recipes for Better 3D Pseudo- GTs of 3D Human Mesh Estimation in the Wild

    Gyeongsik Moon, Hongsuk Choi, Sanghyuk Chun, Jiyoung Lee, and Sangdoo Yun. Three Recipes for Better 3D Pseudo- GTs of 3D Human Mesh Estimation in the Wild. InCom- puter Vision and Pattern Recognition Workshop (CVPRW),

  24. [32]

    Expressive whole-body 3D gaussian avatar

    Gyeongsik Moon, Takaaki Shiratori, and Shunsuke Saito. Expressive whole-body 3D gaussian avatar. InEuropean Conference on Computer Vision (ECCV), 2024. 1

    2024

  25. [33]

    TranSG: Transformer- based skeleton graph prototype contrastive learning with structure-trajectory prompted reconstruction for person re- identification

    Haocong Rao and Chunyan Miao. TranSG: Transformer- based skeleton graph prototype contrastive learning with structure-trajectory prompted reconstruction for person re- identification. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22118–22128, 2023. 3

    2023

  26. [34]

    Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, and Leonidas J. Guibas. HuMoR: 3D hu- man motion model for robust pose estimation. InIEEE/CVF International Conference on Computer Vision (ICCV), 2021. 1

    2021

  27. [35]

    Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bod- ies together.ACM Transactions on Graphics, (Proc. SIG- GRAPH Asia), 36(6), 2017. 6

    2017

  28. [36]

    Non- isotropy regularization for proxy-based deep metric learning

    Karsten Roth, Oriol Vinyals, and Zeynep Akata. Non- isotropy regularization for proxy-based deep metric learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7420–7430, 2022. 3

    2022

  29. [37]

    Learning 3D human pose estimation from dozens of datasets using a geometry-aware autoencoder to bridge between skeleton formats

    Istv ´an S ´ar´andi, Alexander Hermans, and Bastian Leibe. Learning 3D human pose estimation from dozens of datasets using a geometry-aware autoencoder to bridge between skeleton formats. InIEEE/CVF Winter Conference on Ap- plications of Computer Vision (WACV), 2023. 2

    2023

  30. [38]

    Common Pets in 3D: Dynamic new-view synthesis of real-life deformable categories.IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    Samarth Sinha, Roman Shapovalov, Jeremy Reizenstein, Ig- nacio Rocco, Natalia Neverova, Andrea Vedaldi, and David Novotny. Common Pets in 3D: Dynamic new-view synthesis of real-life deformable categories.IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 15

    2023

  31. [39]

    Concerning nonnegative matrices and doubly stochastic matrics.Pacific Journal of Mathematics, 21(2):343–348, 1967

    Richard Sinkhorn and Paul Knopp. Concerning nonnegative matrices and doubly stochastic matrics.Pacific Journal of Mathematics, 21(2):343–348, 1967. 4

    1967

  32. [40]

    Prototypical networks for few-shot learning.Advances in Neural Infor- mation Processing Systems (NeurIPS), 30, 2017

    Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning.Advances in Neural Infor- mation Processing Systems (NeurIPS), 30, 2017. 3

    2017

  33. [41]

    Vision-based fallen person detection for the elderly

    Markus D Solbach and John K Tsotsos. Vision-based fallen person detection for the elderly. InIEEE/CVF international conference on computer vision workshops (ICCVW), pages 1433–1442, 2017. 1

    2017

  34. [42]

    Deep high-resolution representation learning for human pose esti- mation

    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose esti- mation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 5, 6, 13

    2019

  35. [43]

    Monocular, one-stage, regression of multiple 3D people

    Yu Sun, Qian Bao, Wu Liu, Yili Fu, Black Michael J., and Tao Mei. Monocular, one-stage, regression of multiple 3D people. InIEEE/CVF International Conference on Computer Vision (ICCV), 2021. 2

    2021

  36. [44]

    Putting people in their place: Monocular regression of 3D people in depth

    Yu Sun, Wu Liu, Qian Bao, Yili Fu, Tao Mei, and Michael J Black. Putting people in their place: Monocular regression of 3D people in depth. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2

    2022

  37. [45]

    xR-EgoPose: Egocentric 3D human pose from an hmd camera

    Denis Tome, Patrick Peluse, Lourdes Agapito, and Hernan Badino. xR-EgoPose: Egocentric 3D human pose from an hmd camera. InIEEE/CVF International Conference on Computer Vision (ICCV), pages 7728–7738, 2019. 1

    2019

  38. [46]

    JRDB-Pose: A large-scale dataset for multi- person pose estimation and tracking

    Edward Vendrow, Duy Tho Le, Jianfei Cai, and Hamid Rezatofighi. JRDB-Pose: A large-scale dataset for multi- person pose estimation and tracking. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  39. [47]

    Recovering accurate 3D human pose in the wild using imus and a moving camera

    Timo von Marcard, Roberto Henschel, Michael Black, Bodo Rosenhahn, and Gerard Pons-Moll. Recovering accurate 3D human pose in the wild using imus and a moving camera. In European Conference on Computer Vision (ECCV), 2018. 5

    2018

  40. [48]

    Learning support and trivial prototypes for interpretable im- age classification

    Chong Wang, Yuyuan Liu, Yuanhong Chen, Fengbei Liu, Yu Tian, Davis McCarthy, Helen Frazer, and Gustavo Carneiro. Learning support and trivial prototypes for interpretable im- age classification. InIEEE/CVF International Conference on Computer Vision (ICCV), pages 2062–2072, 2023. 3

    2023

  41. [49]

    Scene-aware ego- centric 3D human pose estimation

    Jian Wang, Diogo Luvizon, Weipeng Xu, Lingjie Liu, Kri- pasindhu Sarkar, and Christian Theobalt. Scene-aware ego- centric 3D human pose estimation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13031–13040, 2023. 1

    2023

  42. [50]

    Towards universal object detection by domain at- tention

    Xudong Wang, Zhaowei Cai, Dashan Gao, and Nuno Vas- concelos. Towards universal object detection by domain at- tention. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7289–7298, 2019. 2

    2019

  43. [51]

    Do different tracking tasks require different appearance models?Advances in Neu- ral Information Processing Systems (NeurIPS), 34:726–738,

    Zhongdao Wang, Hengshuang Zhao, Ya-Li Li, Shengjin Wang, Philip Torr, and Luca Bertinetto. Do different tracking tasks require different appearance models?Advances in Neu- ral Information Processing Systems (NeurIPS), 34:726–738,

  44. [52]

    AI Challenger: A large-scale dataset for going deeper in image understanding.arXiv preprint arXiv:1711.06475, 2017

    Jiahong Wu, He Zheng, Bo Zhao, Yixin Li, Baoming Yan, Rui Liang, Wenjia Wang, Shipei Zhou, Guosen Lin, Yan- wei Fu, et al. AI Challenger: A large-scale dataset for going deeper in image understanding.arXiv preprint arXiv:1711.06475, 2017. 5

    2017 arXiv

  45. [53]

    Simple baselines for human pose estimation and tracking

    Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. InEuropean Con- ference on Computer Vision (ECCV), pages 466–481, 2018. 13

    2018

  46. [54]

    Universal-RCNN: Universal object detector via transferable graph r-cnn

    Hang Xu, Linpu Fang, Xiaodan Liang, Wenxiong Kang, and Zhenguo Li. Universal-RCNN: Universal object detector via transferable graph r-cnn. InAAAI Conference on Artificial Intelligence (AAAI), pages 12492–12499, 2020. 2

    2020

  47. [55]

    Pose for ev- erything: Towards category-agnostic pose estimation

    Lumin Xu, Sheng Jin, Wang Zeng, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, and Xiaogang Wang. Pose for ev- erything: Towards category-agnostic pose estimation. InEu- ropean Conference on Computer Vision (ECCV), pages 398–

  48. [56]

    ViTPose++: Vision transformer for generic body pose esti- mation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. ViTPose++: Vision transformer for generic body pose esti- mation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 2, 5, 12

    2023

  49. [57]

    ScoreHypo: Probabilistic human mesh estimation with hypothesis scoring

    Yuan Xu, Xiaoxuan Ma, Jiajun Su, Wentao Zhu, Yu Qiao, and Yizhou Wang. ScoreHypo: Probabilistic human mesh estimation with hypothesis scoring. InIEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 979–989, 2024. 12

    2024

  50. [58]

    UNIK: A unified framework for real-world skeleton-based action recognition.British Machine Visison Conference (BMVC),

    Di Yang, Yaohui Wang, Antitza Dantcheva, Lorenzo Garat- toni, Gianpiero Francesca, and Francois Bremond. UNIK: A unified framework for real-world skeleton-based action recognition.British Machine Visison Conference (BMVC),

  51. [59]

    KITRO: Re- fining human mesh by 2D clues and kinematic-tree rotation

    Fengyuan Yang, Kerui Gu, and Angela Yao. KITRO: Re- fining human mesh by 2D clues and kinematic-tree rotation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1052–1061, 2024. 8

    2024

  52. [60]

    APT-36K: A large-scale benchmark for animal pose estimation and tracking.Advances in Neural In- formation Processing Systems (NeurIPS), 35:17301–17313,

    Yuxiang Yang, Junjie Yang, Yufei Xu, Jing Zhang, Long Lan, and Dacheng Tao. APT-36K: A large-scale benchmark for animal pose estimation and tracking.Advances in Neural In- formation Processing Systems (NeurIPS), 35:17301–17313,

  53. [61]

    TapNet: Neural network augmented with task-adaptive projection for few-shot learning

    Sung Whan Yoon, Jun Seo, and Jaekyun Moon. TapNet: Neural network augmented with task-adaptive projection for few-shot learning. InInternational conference on machine learning, pages 7115–7123. PMLR, 2019. 3

    2019

  54. [62]

    AP-10K: A benchmark for animal pose es- timation in the wild

    Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. AP-10K: A benchmark for animal pose es- timation in the wild. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. 1, 5

    2021

  55. [63]

    Human– robot collaborative interaction with human perception and action recognition.Neurocomputing, 563:126827, 2024

    Xinyi Yu, Xin Zhang, Chengjun Xu, and Linlin Ou. Human– robot collaborative interaction with human perception and action recognition.Neurocomputing, 563:126827, 2024. 1

    2024

  56. [64]

    HRFormer: High- resolution transformer for dense prediction

    Yuhui Yuan, Rao Fu, Lang Huang, Weihong Lin, Chao Zhang, Xilin Chen, and Jingdong Wang. HRFormer: High- resolution transformer for dense prediction. InAdvances in Neural Information Processing Systems (NeurIPS), 2021. 6

    2021

  57. [65]

    Prototype completion with primitive knowl- edge for few-shot learning

    Baoquan Zhang, Xutao Li, Yunming Ye, Zhichao Huang, and Lisai Zhang. Prototype completion with primitive knowl- edge for few-shot learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3754–3762, 2021. 3

    2021

  58. [66]

    Uni3D: A unified baseline for multi-dataset 3D object detection

    Bo Zhang, Jiakang Yuan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Uni3D: A unified baseline for multi-dataset 3D object detection. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9253–9262,

  59. [67]

    Pose2Seg: Detection free human instance segmentation

    Song-Hai Zhang, Ruilong Li, Xin Dong, Paul Rosin, Zixi Cai, Xi Han, Dingcheng Yang, Haozhi Huang, and Shi-Min Hu. Pose2Seg: Detection free human instance segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 889–898, 2019. 13

    2019

  60. [68]

    Object detec- tion with a unified label space from multiple datasets

    Xiangyun Zhao, Samuel Schulter, Gaurav Sharma, Yi-Hsuan Tsai, Manmohan Chandraker, and Ying Wu. Object detec- tion with a unified label space from multiple datasets. In European Conference on Computer Vision (ECCV), pages 178–193. Springer, 2020. 2

    2020

  61. [69]

    Rethinking Semantic Segmentation: A Prototype View

    Tianfei Zhou, Wenguan Wang, Ender Konukoglu, and Luc Van Gool. Rethinking Semantic Segmentation: A Prototype View. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2582–2593, 2022. 3, 4, 12

    2022

  62. [70]

    Sim- ple multi-dataset detection

    Xingyi Zhou, Vladlen Koltun, and Philipp Kr¨ahenb¨uhl. Sim- ple multi-dataset detection. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7571–7580, 2022. 2, 13 F AP 32 77.1 64 77.1 128 77.1 Table 6. Impact of varying the embedding dimensionF(mean...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.