PCIE_Pose Solution for EgoExo4D Pose and Proficiency Estimation Challenge

Feng Chen; Jun Xie; Kanokphan Lertniphonphan; Qiancheng Yan; Tao Zhang; Xiaohui Fan; Zhepeng Wang

REVIEW 3 major objections 5 minor 10 references

PCIE_Pose Solution for EgoExo4D Pose and Proficiency Estimation Challenge

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that hybrid ViT+CNN hand pose networks and multimodal spatio-temporal body pose networks took first place in the EgoExo4D pose and proficiency challenges.

desk verdict Credible first-place challenge results undermined by a self-contradictory temporal-fusion ablation and sloppy copy-paste errors; worth a referee only after the authors clean up the internal narrative. read the letter →

arxiv 2505.24411 v1 pith:OGDQVFEU submitted 2025-05-30 cs.CV

Feng Chen , Kanokphan Lertniphonphan , Qiancheng Yan , Xiaohui Fan , Jun Xie , Tao Zhang , Zhepeng Wang This is my paper

classification cs.CV

keywords egocentricvisionhandposeestimationbodytransformerConvNeXtmultimodalfusionproficiencyEgoExo4D

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that two architectural decisions win the CVPR2025 EgoExo4D pose challenges: pairing a Vision Transformer with a ConvNeXt CNN backbone and fusing their predictions by weighted score for hand pose, and fusing head pose, egocentric video, and monocular depth features through transformer encoders for body pose. The reported test numbers are a Procrustes-aligned mean per-joint position error of 8.31 mm for hands and a mean per-joint position error of 11.25 cm for body, both in first place. The same body-pose framework, using only egocentric video and a VideoMAE v2 backbone, reports 0.53 top-1 accuracy on the demonstrator proficiency task, also first on the leaderboard. This matters because egocentric pose estimation struggles with small, occluded hands and a body that is often outside the camera view; the paper presents a practical recipe of complementary backbones and cross-modal temporal fusion for that setting. The report includes ablation tables and leaderboard comparisons supporting the claims.

What carries the argument

The load-bearing object for hand pose is HP-ViT+, a hybrid of a ViT path and a ConvNeXt path that each predict keypoints through a transformer decoder plus MLP head, with a final score-weighted fusion of the two predictions and horizontal-flip test-time augmentation. For body pose, the central mechanism is two-stage cross-modal fusion: temporal transformer encoders on head-pose sequences and depth maps, a 3D ResNet on egocentric video, concatenation of the attended embeddings, an MLP that mixes modalities, and a shared regression head for 3D joints. The proficiency model reuses this body-pose framework by swapping the head to an MLP classifier and the video encoder to VideoMAE v2, with an ensemble of different model sizes and frame counts.

What would settle it

Re-run the described configurations on the EgoExo4D validation set with the same data splits, or inspect the released challenge logs; if the reproduced hand PA-MPJPE is not near 8.31 mm or body MPJPE not near 11.25 cm, the central claim fails. A minimal internal check is resolving Table 4, where temporal fusion lowers MPJPE from 12.42 to 12.01, against the text's statement that enabling temporal fusion increases MPJPE by 0.4 cm; only one direction can match the winning configuration.

Watch

Extended reading notes

Core claim

The central claim is that HP-ViT+ estimates 21 3D hand joints by running a ViT and a ConvNeXt-Huge pathway in parallel and fusing their keypoint predictions by a score-weighted sum, reducing test MPJPE to 24.80 mm and PA-MPJPE to 8.31 mm. For the body, a multimodal spatio-temporal network encodes 16-frame head-pose sequences with a temporal transformer, 4-frame egocentric video with a ResNet3D encoder, and monocular depth sequences with another transformer, then concatenates the three feature streams through an MLP before regressing 17 body joints; the ensemble of variants reaches 11.25 cm MPJPE on the test set. For proficiency estimation, the same framework uses egocentric video only with a VideoMAE v2 backbone and an MLP classification head, reaching 0.53 top-1 accuracy. The paper presents these as first-place results on the official EgoExo4D leaderboards.

Load-bearing premise

The load-bearing premise is that the reported leaderboard scores were produced by the exact architectures and configurations described in the paper, since no code or training logs are released to confirm that link.

Editorial extensions

If this is right

Hand pose accuracy on the EgoExo4D test set improves from HP-ViT's 25.51 to HP-ViT+'s 24.80 MPJPE, with PA-MPJPE dropping from 8.49 to 8.31 mm.
Body pose single models reach 12.10 cm MPJPE on test, and the multi-modal, multi-sample ensemble lowers that to 11.25 cm, enough to lead the leaderboard.
The body-pose framework transfers across tasks: the same feature pipeline classifies proficiency at 0.53 top-1 accuracy when adapted to a classification head.
The ablation table attributes most of the body-pose gain to adding modalities rather than to any single encoder, consistent with the paper's emphasis on multimodal fusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Extension: an independent reimplementation of the described configurations on the same validation splits would test whether the reported numbers come from the architecture or from ensemble and leaderboard circumstances.
Extension: the text in Section 3.2.2 says temporal fusion increases MPJPE by 0.4 cm while Table 4 shows it decreasing from 12.42 to 12.01 cm; until that is resolved, the specific design lesson from the body-pose win is ambiguous.
Extension: if the proficiency result generalizes, it would imply egocentric video alone carries enough skill information for classification, which the paper does not test against exocentric input.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a CVPR challenge report, not a research paper, and it should be read as one. The headline result — three first-place finishes on EgoExo4D Hand Pose, Body Pose, and Proficiency Estimation — is externally anchored to the official leaderboard, so I take the numbers as real. The paper also does something useful: it shows that a fairly standard mix of ViT + ConvNeXt fusion, multimodal head-pose/video/depth fusion, and ensembling wins these benchmarks. For practitioners building egocentric AR or robotics systems, that is a genuinely informative data point. The hand pose ablation, modest as it is, gives some evidence that the CNN/Transformer fusion helps over HP-ViT alone.

The soft spots are real, but they are in the framing, not in the leaderboard claim. The temporal fusion ablation is self-contradictory: the text says enabling temporal fusion increases MPJPE by 0.4 cm, while Table 4 shows it dropping from 12.42 to 12.01. Either the table is right or the text is, and the paper never says which. That matters because the winning 11.25 number is attributed to ensembled variants, and the reader cannot tell whether temporal fusion was even enabled in the submission. Table 5 also has a copy-paste error: the Baseline row repeats 14.46 from the body-pose MPJPE table instead of a top-1 accuracy. And reference [6] cites a coronary-artery-stenosis paper for ConvNeXt-V2, which is just wrong. None of this makes the leaderboard results false, but it does mean the paper's own causal narrative about what contributed to the win is not supported as written. A further minor gap: the hand pose ablation in Table 2 never evaluates the full HP-ViT+ ensemble, so the value of the weighted fusion itself is asserted rather than shown.

Who gets value from this? Anyone who wants to know what it takes to win these specific benchmarks in 2025, and who is willing to treat the architecture as a recipe rather than a scientific claim. It is a credible technical report, but it would need a thorough revision — fix the temporal-fusion contradiction, correct the baseline and citation, show an HP-ViT+ ablation, and ideally release configuration files — before I'd trust it as a peer-reviewed paper. As is, a serious editor would be justified in asking for major revision rather than desk rejecting, because the external leaderboard anchor means the results are almost certainly genuine. I'd give it a conditional pass at best, and I would not cite it in its current form.

Referee Report

3 major / 5 minor

Summary. The paper reports the PCIE_EgoPose team's solutions for three EgoExo4D challenges at CVPR2025. For hand pose estimation, it describes HP-ViT+, a hybrid ViT plus ConvNeXt architecture with weighted fusion, and reports a championship PA-MPJPE of 8.31 mm. For body pose estimation, it proposes a multimodal spatio-temporal framework fusing head pose sequences, ego-view video, and depth maps, reporting a championship MPJPE of 11.25 cm. For demonstrator proficiency estimation, it extends the body pose framework with a VideoMAE-v2 backbone and reports a top-1 accuracy of 0.53. The paper is a competition report rather than a full research paper, with the main evidence being official leaderboard results.

Significance. If the reported leaderboard numbers are correct, these are strong results in three separate EgoExo4D tasks, and the system descriptions provide useful engineering recipes for egocentric pose estimation and proficiency classification. The paper's value is primarily archival and empirical; it does not claim a new formal method with theoretical guarantees. The absence of released code, checkpoints, or configuration files limits independent verification, and, as detailed below, several internal inconsistencies in the ablations and tables must be resolved before the causal claims about the winning configurations can be trusted.

major comments (3)

[§3.2.2, Table 4] The text and the table contradict each other on the effect of temporal fusion. The text states that 'Enabling temporal fusion increases MPJPE by 0.4 cm', but Table 4 shows MPJPE decreasing from 12.42 to 12.01 when temporal fusion is added, i.e., an improvement of 0.41 cm. Since the paper attributes the winning test score of 11.25 to ensembling 'multiple variants with different modal combinations and frame sampling strategies', the reader cannot tell whether temporal fusion was enabled in the submitted ensemble or whether the reported 11.25 corresponds to the architecture described. Please correct either the sentence or the table, and specify exactly which configuration or configurations produced the reported test numbers.
[§3.2.3, Table 5] The baseline row in Table 5 is impossible: under the 'Top 1 Accuracy (↑)' column it reports a value of 14.46, but an accuracy must be in [0,1]. The value appears to have been copied from the 'Camera Pose only' row of the body pose MPJPE table (Table 4). This makes the proficiency leaderboard comparison unusable as printed. Please replace it with the actual baseline accuracy, or remove the baseline if none was used, and recheck all numbers in the table.
[§3.2.1, Table 2] The hand pose ablation in Table 2 reports only individual backbones (ViT-Base/Large/Huge and ConvNeXt-V2-Huge), not the full HP-ViT+ model with weighted fusion. Consequently, the claimed benefit of the fusion mechanism is not directly demonstrated on the validation set. The text also shifts between describing HP-ViT+ as a 'hybrid architecture' and as a 'score-weighted ensemble of ConvNeXt-Huge and ViT outputs'. Please add a row for the complete HP-ViT+ configuration in Table 2 and clarify whether the championship test numbers come from the two-pathway architecture, the ensemble, or both.

minor comments (5)

[§2.2, Figure 2] The text refers to 'Figure ??', and the caption of Figure 2 describes a 'Multimodal Spatio-Temporal Fusion Framework for Hand Pose Estimation', although the figure appears to illustrate the body pose framework described in Section 2.2. Please fix the cross-reference and the caption.
[References [6]] Reference [6] is cited as the source for ConvNeXt-V2, but the cited paper is about coronary artery stenosis detection with Mask R-CNN, not the ConvNeXt-V2 architecture. Please cite the correct primary source for ConvNeXt-V2.
[§3.2.1] The text refers to 'ConvNeXt-V2-Huge' in one place and 'ConvNeXt-Huge' in another; please use the same name consistently.
[§3.2.2, Table 4] The row labels in Table 4 ('Camera Pose only', '+ Ego Camera', '+ Depth Estimation', '+ Temporal Fusion') do not match the modality names used in the text ('head pose', 'video', 'depth'). Please align the terminology so the reader can map each ablation row to the described input stream.
[Abstract] The body pose result is reported as '11.25 MPJPE' without units, whereas Section 3.1 states that body pose MPJPE is measured in centimeters; please state the unit consistently in the abstract and the results tables.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: results are externally benchmarked; minor self-citation and validation-tuned ensembling do not make the derivation circular.

full rationale

The paper's headline results (8.31 PA-MPJPE, 11.25 MPJPE, 0.53 top-1) are leaderboard numbers from the external EgoExo4D benchmark, so the central claims are not derived from the paper's own equations. HP-ViT+ is an extension of the authors' prior HP-ViT (Ref [1]), but the cited prior work is an independently published architecture rather than a uniqueness theorem or an unverified premise, and the present contribution (ConvNeXt pathway, weighted ensemble, multimodal fusion) is evaluated externally. The ensemble weights are optimized on the validation set, which is standard practice and does not by construction force the reported test-set numbers. The only significant flaw found is an internal inconsistency, not circularity: Section 3.2.2 states 'Enabling temporal fusion increases MPJPE by 0.4 cm, demonstrating the importance of modeling temporal dynamics,' while Table 4 shows temporal fusion reducing MPJPE from 12.42 to 12.01; this makes the causal attribution of the body-pose win unverifiable but does not reduce any claimed result to its input. Reference [6] also mis-cites a coronary-stenosis paper for ConvNeXt-V2, a citation-quality issue rather than circularity. Because the derivation is self-contained against an external benchmark, with only minor self-citation, the circularity score is 2.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. It is an empirical system report, but the exact settings of the ensembling and input configurations are free parameters fitted on validation data, and the reliability of the pipeline depends on the external benchmark and pretrained models.

free parameters (2)

Hand pose ensemble fusion weights = not reported
Ensemble of ConvNeXt-Huge and ViT outputs is described as 'optimized on the validation set' in Section 3.2.1, so the final weight is fitted to validation data.
Body pose ensemble configuration = not reported
Performance improved by 'training multiple variants with different modal combinations and frame sampling strategies, and ensembling their predictions' in Section 3.2.2; the selected combination is a hand-chosen model selection on validation.

assumptions (4)

domain assumption EgoExo4D benchmark annotations and official leaderboard results are accurate.
The paper relies entirely on the benchmark's evaluation for its reported test numbers; these are not reproduced.
domain assumption Off-the-shelf depth estimates from DepthAnything-v2 are reliable enough for pose regression.
Depth maps are generated by an external model and used as a modality in Section 2.2.1.
domain assumption Pre-trained backbones (ViT, ConvNeXt, VideoMAE v2) transfer to egocentric hand, body, and proficiency domains.
The presented ablations support the relative gains but do not measure transfer separately.
standard math MPJPE loss is the appropriate training objective for the pose tasks.
Used in Section 3.2.2 and standard in the field.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PCIE_Pose Solution for EgoExo4D Pose and Proficiency Estimation Challenge." pith.science (2026). https://pith.science/paper/OGDQVFEU

@misc{pith2026250524411,
  author       = {Pith},
  title        = {Pith review of: PCIE_Pose Solution for EgoExo4D Pose and Proficiency Estimation Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGDQVFEU}},
  note         = {Machine review of arXiv:2505.24411}
}

read the original abstract

This report introduces our team's (PCIE_EgoPose) solutions for the EgoExo4D Pose and Proficiency Estimation Challenges at CVPR2025. Focused on the intricate task of estimating 21 3D hand joints from RGB egocentric videos, which are complicated by subtle movements and frequent occlusions, we developed the Hand Pose Vision Transformer (HP-ViT+). This architecture synergizes a Vision Transformer and a CNN backbone, using weighted fusion to refine the hand pose predictions. For the EgoExo4D Body Pose Challenge, we adopted a multimodal spatio-temporal feature integration strategy to address the complexities of body pose estimation across dynamic contexts. Our methods achieved remarkable performance: 8.31 PA-MPJPE in the Hand Pose Challenge and 11.25 MPJPE in the Body Pose Challenge, securing championship titles in both competitions. We extended our pose estimation solutions to the Proficiency Estimation task, applying core technologies such as transformer-based architectures. This extension enabled us to achieve a top-1 accuracy of 0.53, a SOTA result, in the Demonstrator Proficiency Estimation competition.

Figures

Figures reproduced from arXiv: 2505.24411 by the authors.

**Figure 2.** Multimodal Spatio-Temporal Fusion Framework for Hand Pose Estimation. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

10 extracted references · 5 canonical work pages

[6]

Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence, 36 0 (7): 0 1325--1339, 2013

work page 2013
[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Pcie\_egohandpose solution for egoexo4d hand pose challenge

Feng Chen, Ling Ding, Kanokphan Lertniphonphan, Jian Li, Kaer Huang, and Zhepeng Wang. Pcie\_egohandpose solution for egoexo4d hand pose challenge. arXiv preprint arXiv:2406.12219, 2024

arXiv 2024
[3]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021

2021
[4]

Ego-exo4d: Understanding skilled human activity from first- and third-person perspectives

Kristen Grauman et al. Ego-exo4d: Understanding skilled human activity from first- and third-person perspectives. ArXiv, abs/2311.18259, 2023

arXiv 2023
[5]

Slowfast networks for video recognition

Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202--6211, 2019

work page 2019
[7]

ConvNeXtv2 Fusion with Mask R-CNN for Automatic Region Based Coronary Artery Stenosis Detection for Disease Diagnosis

Sandesh Pokhrel, Sanjay Bhandari, Eduard Vazquez, Yash Raj Shrestha, and Binod Bhattarai. Convnextv2 fusion with mask r-cnn for automatic region based coronary artery stenosis detection for disease diagnosis. arXiv preprint arXiv:2310.04749, 2023

work page Pith review arXiv
[8]

Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems, 2017

2017

Show all 10 references

[9]

Vi TP ose: Simple vision transformer baselines for human pose estimation

Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vi TP ose: Simple vision transformer baselines for human pose estimation. In Advances in Neural Information Processing Systems, 2022

2022
[10]

Depth anything v2

Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances in Neural Information Processing Systems, 37: 0 21875--21911, 2024

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [6]

Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence, 36 0 (7): 0 1325--1339, 2013

work page 2013

[2] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[3] [2]

Pcie\_egohandpose solution for egoexo4d hand pose challenge

Feng Chen, Ling Ding, Kanokphan Lertniphonphan, Jian Li, Kaer Huang, and Zhepeng Wang. Pcie\_egohandpose solution for egoexo4d hand pose challenge. arXiv preprint arXiv:2406.12219, 2024

arXiv 2024

[4] [3]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021

2021

[5] [4]

Ego-exo4d: Understanding skilled human activity from first- and third-person perspectives

Kristen Grauman et al. Ego-exo4d: Understanding skilled human activity from first- and third-person perspectives. ArXiv, abs/2311.18259, 2023

arXiv 2023

[6] [5]

Slowfast networks for video recognition

Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202--6211, 2019

work page 2019

[7] [7]

ConvNeXtv2 Fusion with Mask R-CNN for Automatic Region Based Coronary Artery Stenosis Detection for Disease Diagnosis

Sandesh Pokhrel, Sanjay Bhandari, Eduard Vazquez, Yash Raj Shrestha, and Binod Bhattarai. Convnextv2 fusion with mask r-cnn for automatic region based coronary artery stenosis detection for disease diagnosis. arXiv preprint arXiv:2310.04749, 2023

work page Pith review arXiv

[8] [8]

Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems, 2017

2017