{"id":"1b41e9c2-6b15-40a9-b58d-30e05d3a4ff4","arxiv_id":"2412.19676","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A hybrid local-global network using skeleton selective refine attention reports state-of-the-art 3D pose accuracy on Human3.6M and MPI-INF-3DHP.","lead":"This paper presents SSR-STF, a dual-stream network for 3D human pose estimation that combines a large-kernel attention module for local details with a Transformer for global context. It reports the lowest published pose errors on two standard benchmarks and shows that the learned motion features also improve human mesh recovery.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Load-bearing concern: the SOTA claim rests on an undocumented evaluation protocol; Tables I–III never state whether MPJPE is center-frame, all-frame, or with which window stride.","rationale":"I read the paper in good faith. The architecture is coherent: SSRA is a reasonable large-kernel attention module for skeleton sequences, the dual-stream design with adaptive fusion is clearly specified in Eqs. (1)-(11), and the ablations in Table V support the contribution of the local stream. The mesh-recovery experiment is a useful downstream check. There is no internal contradiction that makes the results impossible. However, the headline empirical claim cannot be verified from the text because the evaluation protocol is not documented. The reader's weakest assumption identifies exactly this gap, and I agree that it is load-bearing. A conditional verdict is appropriate: the method is promising, but the SOTA comparison needs an explicit protocol statement and a verification pass over the tables before the central claim should be accepted as stated. I therefore keep the reader's verdict unchanged rather than moving to accept or reject.","tokens_in":20780,"tokens_out":11538,"duration_ms":103087,"concrete_test":"Using the released code and checkpoints, evaluate SSR-STF on Human3.6M and MPI-INF-3DHP under three explicit protocols: (i) center-frame predictions with stride-1 sliding windows, (ii) all-frame predictions with stride-1 windows, and (iii) all-frame predictions with non-overlapping windows. Then rerun the official MotionBERT and MotionAGFormer evaluation scripts under the same three protocols. If the reported 37.4/12.8/13.2 mm values are reproduced only under one protocol while the baselines were cited under another, the SOTA claim must be restated; if the ranking is unchanged across all three protocols, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of state-of-the-art accuracy depends on comparing SSR-STF against baselines under an identical evaluation protocol. Eq. (12) trains the model to predict all T frames, so SSR-STF emits T poses per input window, whereas some cited baselines (PoseFormer, MHFormer, P-STMO) are central-frame predictors and others (MixSTE, MotionBERT) are sequence-to-sequence. Sections IV-A and IV-B describe datasets and training but never specify whether reported MPJPE is computed on the central frame only, on all frames, with a stride-1 sliding window, or with non-overlapping windows. This omission matters because all-frame evaluation with overlapping windows can average many predictions per frame and can be substantially easier than center-frame evaluation; a mismatch in either direction changes the comparison. The 37.4 mm detected-P1, 12.8 mm GT-P1, and 13.2 mm MPI-INF-3DHP numbers are the entire basis of the SOTA claim, so the missing protocol is the single most load-bearing assumption. Secondary table issues (e.g., the MotionBERT P-MPJPE WalkD./Walk entries appear swapped, and Table IV parameter counts are insensitive to Ch) are correctable and do not by themselves overturn the architecture's internal consistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SSR-STF, a dual-stream 2D-to-3D human pose estimation model that combines a Transformer stream (STFormer) capturing global spatio-temporal dependencies with a newly proposed SSRFormer stream using skeleton selective refine attention (SSRA), an irregular large-kernel attention mechanism, for local dependencies. The two streams are adaptively fused, and the model is trained to predict all T frames of an input window under a position plus velocity loss. Experiments on Human3.6M and MPI-INF-3DHP report state-of-the-art MPJPE/P-MPJPE results, including 37.4 mm on Human3.6M with detected 2D poses, 12.8 mm with ground-truth 2D poses at T=243, and 13.2 mm on MPI-INF-3DHP, together with ablations and a downstream human mesh recovery experiment showing improvements over MotionBERT.","tokens_in":21013,"tokens_out":3809,"duration_ms":39224,"significance":"If the reported numbers are obtained under a protocol comparable to that of the cited baselines, the work makes a solid empirical contribution: it presents a simple, modular way to complement global Transformer attention with local large-kernel attention in skeleton sequences, it includes extensive comparisons across two benchmarks and several temporal window sizes, it releases code, and it evaluates the learned motion representation on mesh recovery. The architecture is internally consistent and the ablation study supports the value of the dual-stream adaptive-fusion design. The main significance hinges on the comparability of the evaluation protocol, which is not currently documented.","major_comments":[{"comment":"The central claim of state-of-the-art accuracy is not verifiable because the evaluation protocol for the temporal window is never specified. Equation (12) trains the model to predict all T frames, so SSR-STF naturally produces T poses per input clip, whereas some baselines (PoseFormer, MHFormer, P-STMO) are central-frame predictors and others (MixSTE, MotionBERT) are sequence-to-sequence. The paper does not state whether reported MPJPE is computed on the central frame only, on all frames, with a stride-1 sliding window, or with non-overlapping windows. All-frame evaluation with overlapping windows can average multiple predictions per frame and is not directly comparable to center-frame evaluation. Please specify the exact protocol for every table and, ideally, re-evaluate the compared baselines under the identical protocol.","section":"Section IV-A/IV-B, Tables I-III"},{"comment":"The parameter counts in Table IV are internally inconsistent: the configurations with Ch=256 and Ch=512 both report 36.7M parameters, yet Ch is the motion embedding dimension used by the final linear layer and the regression head, so changing it should change the parameter count. The Ch=1024 row also increases by only 0.2M, which is implausible if the projection and head scale with Ch. Please correct the reported counts or explain why these parameters are shared or omitted.","section":"Table IV"},{"comment":"There is a citation/reference error in Table III: the MotionAGFormer row is cited as [28], but [28] in the reference list is KTPFormer, while MotionAGFormer is [8]. The text in Section IV-D also lists comparison baselines as [8], [17], [51], which does not match the table. This must be corrected, and the per-action breakdown for MotionAGFormer in Table II should be provided or explicitly marked as unavailable.","section":"Table III and Section IV-D"},{"comment":"No variance or repeated-run information is given for any reported number, yet several state-of-the-art claims rest on differences of about 1 mm or less (e.g., 37.4 vs. 38.4 for MotionAGFormer in Table I, and 31.7 vs. 31.9 in P-MPJPE). Without error bars, seeds, or at least a statement about the number of runs, the reader cannot assess whether these gaps are meaningful. Please add variance information or, if only one run was performed, state this explicitly and discuss the sensitivity of the conclusions.","section":"Tables I-VI"},{"comment":"The text discussing kernel shapes says that on Human3.6M the 35x11 kernel gives a 0.3 mm lower P1 error than the 11x1 kernel, but Table VI is titled as reporting results on MPI-INF-3DHP only and does not contain Human3.6M values. Please clarify which dataset Table VI reports, and either add the Human3.6M kernel comparisons or remove the unsupported claim.","section":"Section IV-E, Table VI"}],"minor_comments":[{"comment":"In the P-MPJPE block, the MotionBERT row lists 22.0 for WalkD. and 31.7 for Walk, which appears to be a swapped pair compared with the values reported in the original MotionBERT paper; please verify these entries against the source.","section":"Table I (bottom)"},{"comment":"Figure 3 contains the Chinese text '取名为不规则LK' and the undefined label 'LKFormer'; this is inconsistent with the English exposition and should be cleaned up before publication.","section":"Fig. 3"},{"comment":"The code link is given as a URL in the abstract but appears as the placeholder 'SSR-STF' in the full text; please use the actual repository URL consistently.","section":"Abstract and Section I"},{"comment":"The order of operators in Eq. (4) (DWD2(DWD1(DW2(DW1(X))))) is not clearly aligned with the surrounding text describing DW1/DW2 and DWD1/DWD2, and the kernel sizes in the text and in Fig. 3 do not match notationally; please harmonize the notation and the order of operations.","section":"Section III-B3, Eq. (4)"},{"comment":"Reference [5] is incomplete ('A. Vaswani, Attention is all you need', without venue or year), reference [35] lists the author as 'D. Alexey', and several other entries have inconsistent formatting; please normalize the bibliography.","section":"Reference list"},{"comment":"The MotionAGFormer row in Table II reports only the average (17.3 mm) with all per-action entries missing; either provide the full row or state that the per-action numbers are not available from the source.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is worth pursuing, but the missing evaluation protocol is a genuine load-bearing issue: the headline SOTA numbers could change substantially under a center-frame versus all-frame protocol. I would ask the authors to document the exact protocol, re-evaluate or carefully justify comparability with each baseline, and fix the table inconsistencies before considering acceptance. If the protocol turns out to be all-frame overlapping evaluation while key baselines are center-frame, the SOTA claim would need major revision or withdrawal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible incremental architecture paper that claims a large SOTA jump on three benchmarks, but the central comparison is unverifiable as written because the evaluation protocol is never specified. That's the main thing you should know.\n\nThe actual contribution: SSRFormer, a MetaFormer-style block whose token mixer is a large-kernel attention mechanism adapted to skeletons. The SSRA module decomposes a 2D kernel into depthwise and dilated depthwise convolutions, and the authors align the kernel along the spatial or temporal dimension depending on the stream. It's a genuine adaptation of VAN/LSKA to pose sequences, and as far as I can tell, the first such use in 3D HPE. The rest is a dual-stream assembly of STCFormer's criss-cross attention plus MotionBERT's adaptive fusion. The ablation study is clean: each component helps, parallel fusion beats sequential, and the kernel shape analysis is informative. The mesh recovery experiment is a small but nice bonus.\n\nWhere it gets shaky: the SOTA claim. Eq. (12) trains the model to predict all T frames, so the model emits T poses per input window. The paper never says whether the reported MPJPE is computed on the central frame only, on all frames, or with what sliding-window stride. This is not a pedantic point: the baselines split into central-frame predictors (PoseFormer, P-STMO) and seq2seq predictors (MixSTE, MotionBERT), and the two families are not compared the same way. A mismatch here would change the numbers by several millimeters and nullify the headline. The authors need to state the protocol explicitly and, ideally, re-run the baselines under the same protocol.\n\nThe smaller issues are correctable but real: Table IV reports identical parameter counts and identical MPJPE for Ch=256 and Ch=512 at depth 12, which can't be right; Table III labels MotionAGFormer as [28], which is KTPFormer; in Table I the MotionBERT P-MPJPE WalkD/Walk values appear swapped; Figure 3 contains leftover Chinese text; and the full text says 'Codes are available at SSR-STF' while the abstract has the actual URL. None of these overturn the method's internal consistency, but they suggest a rushed manuscript.\n\nBottom line: the architecture is plausible and the ablations are honest. The paper deserves a serious referee, but only after the authors document the evaluation protocol and clean up the tables. That's a revise-and-resubmit, not a desk reject.","headline":"Reasonable incremental architecture with a strong SOTA claim, but the missing evaluation protocol makes the headline numbers unverifiable as written.","tokens_in":21558,"tokens_out":4660,"would_cite":true,"duration_ms":365134,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a dual-stream model combining skeleton selective refine attention with a global transformer achieves state-of-the-art 3D pose accuracy, with a 12.8 mm MPJPE on Human3.6M ground-truth 2D poses.","keywords":["3D human pose estimation","skeleton selective refine attention","large-kernel attention","hybrid transformer","dual-stream network","motion representation","human mesh recovery","MPJPE"],"falsifier":"Run a single, explicitly stated evaluation on Human3.6M, for example central-frame MPJPE with a sliding window and fixed stride, for SSR-STF and the top baselines MotionAGFormer, MotionBERT, and KTPFormer, and check whether the reported margins, particularly the 4.5 mm ground-truth-2D gain, persist under identical frame scoring and sampling.","tokens_in":20588,"feed_emoji":"🦴","tokens_out":5352,"duration_ms":53821,"temperature":0.7,"pith_summary":"The paper tries to show that 3D human pose estimation from 2D skeletons improves when a model combines two parallel feature streams: a transformer that captures global spatio-temporal dependencies and a lightweight module, SSRFormer, that captures fine-grained local joint and motion details with a skeleton selective refine attention mechanism. If correct, the model sets a new accuracy standard on the two main benchmarks, with 37.4 mm mean per-joint position error on Human3.6M using detected 2D poses, 12.8 mm using ground-truth 2D poses at 243 input frames, and 13.2 mm on MPI-INF-3DHP. The paper also claims the learned motion representation transfers to human mesh recovery, improving on MotionBERT in vertex and joint error. The reason to care is that this would show that the bottleneck in 2D-to-3D lifting is not only long-range context but also local structure, and that a carefully designed large-kernel attention can supply that local structure at low parameter cost.","feed_headline":"Local-global dual stream cuts 3D pose error to 12.8 mm","feed_subtitle":"A skeleton-aware local attention branch paired with a transformer beats prior 3D pose models on Human3.6M and MPI-INF-3DHP.","key_machinery":"The skeleton selective refine attention (SSRA) mechanism, an irregular large-kernel attention built from depth-wise and dilated depth-wise convolutions decomposed into separable 1D kernels. By reshaping the input so the long kernel side aligns with either the joint dimension (spatial SSRFormer) or the time dimension (temporal SSRFormer), SSRA assigns different weights to spatial and temporal local aggregation. A pointwise convolution produces an attention map that modulates the input through a Hadamard product, making the local stream input-adaptive rather than a fixed convolution. This local stream runs in parallel with the global transformer stream, and adaptive fusion weights combine the two.","core_discovery":"SSR-STF is a dual-stream network in which each block runs an SSRFormer and an STFormer in parallel. The STFormer stream uses spatio-temporal criss-cross attention to model global dependencies, while the SSRFormer stream applies skeleton selective refine attention to selectively refine local dependencies along either the joint or time dimension, and the two streams are combined by an adaptive, learned weighted fusion. On Human3.6M the paper reports P1 errors of 37.4 mm with detected 2D poses and 12.8 mm with ground-truth 2D poses at T=243, and on MPI-INF-3DHP an MPJPE of 13.2 mm with PCK of 99.2 percent. In the ground-truth-2D Human3.6M setting, it claims a 4.5 mm improvement over MotionAGFormer, the previous best, and the same motion representation, when regressed to SMPL parameters, reportedly lowers mesh recovery error from 75.7 to 68.4 mm MPVE relative to MotionBERT.","pith_inferences":["A natural testable extension is to apply SSRA to other skeleton-based tasks, such as action recognition, where local joint correlations and temporal motion details matter as much as long-range context.","Because the paper does not state whether MPJPE is scored on the central frame only or on all frames in the clip, a matched-protocol re-evaluation could narrow the reported margin over baselines; this is an inference about the comparison, not a claim the paper makes.","The irregular kernel design suggests an architecture that adapts its receptive field to sequence length or joint count; one could vary the kernel shape per block and measure the accuracy-efficiency frontier.","The large gains on ground-truth 2D poses indicate the lifting model itself, not the 2D detector, drives the improvement; testing the same backbone with newer 2D detectors could show whether the margin compounds."],"forward_implications":["If the central claim holds, the practical upper bound of 2D-to-3D lifting improves substantially: 12.8 mm MPJPE from ground-truth 2D poses at T=243, about 26 percent below the previous best reported in the paper.","Hybrid local-global architectures would be validated over pure global transformers, giving a design pattern of parallel streams with adaptive fusion rather than sequential stacking.","The learned motion representation transfers to SMPL-based mesh recovery, suggesting the same backbone can support downstream human-centric tasks.","On MPI-INF-3DHP, the model generalizes across capture settings with 13.2 mm MPJPE and 99.2 percent PCK using ground-truth 2D poses, indicating robustness to background and camera variation."],"supporting_citations":[{"why":"Supplies the adaptive fusion mechanism and the motion-representation framework that SSR-STF extends, and serves as the main baseline for mesh recovery.","marker":"[18]"},{"why":"The previous state-of-the-art hybrid method that SSR-STF must beat; the reported 4.5 mm ground-truth-2D MPJPE gain is measured against it.","marker":"[8]"},{"why":"Supplies the spatio-temporal criss-cross attention used in the global STFormer stream and is a direct baseline.","marker":"[17]"},{"why":"Provides the large-kernel decomposition rule into depth-wise, dilated depth-wise, and pointwise convolutions that SSRA is built on.","marker":"[41]"},{"why":"Shows how 2D depth-wise kernels decompose into cascaded 1D separable kernels, the basis of the irregular kernel in SSRA.","marker":"[14]"},{"why":"A strong global-transformer baseline whose per-frame estimation setting frames the comparison.","marker":"[16]"},{"why":"The Human3.6M dataset and protocol on which the primary state-of-the-art numbers are reported.","marker":"[11]"},{"why":"The MPI-INF-3DHP dataset used to demonstrate generalization across capture settings.","marker":"[47]"},{"why":"Supplies the Stacked Hourglass 2D detections used as input for the Human3.6M detected-pose experiments.","marker":"[30]"}],"fun_headline_variants":["Dual-stream local-global net hits 12.8mm 3D pose error","Local-global fusion cuts 3D pose error to 12.8mm","SSR-STF dual-stream beats prior 3D pose models","Skeleton-aware attention improves 3D pose accuracy","Local-global network tops 3D pose benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported state-of-the-art result depends on the evaluation protocol for SSR-STF being identical to the protocols used for the cited baselines; the paper never states whether MPJPE is averaged over every frame in a clip or only the central frame, nor how input windows are sampled.","fun_headline_variants_meta":{"raw":{"variants":["Dual-stream local-global net hits 12.8mm 3D pose error","Local-global fusion cuts 3D pose error to 12.8mm","SSR-STF dual-stream beats prior 3D pose models","Skeleton-aware attention improves 3D pose accuracy","Local-global network tops 3D pose benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000476,"raw_usage":{"total_tokens":2394,"prompt_tokens":1009,"completion_tokens":1385,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1292}},"tokens_in":625,"tokens_out":1385,"duration_ms":10112,"temperature":1.0,"reasoning_tokens":1292,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:59:02.198633+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a single, explicitly stated evaluation on Human3.6M, for example central-frame MPJPE with a sliding window and fixed stride, for SSR-STF and the top baselines MotionAGFormer, MotionBERT, and KTPFormer, and check whether the reported margins, particularly the 4.5 mm ground-truth-2D gain, persist under identical frame scoring and sampling.","supporting_citations":[{"cited_title":"Motionbert: A unified perspective on learning human motion representations,","cited_arxiv_id":null,"evidence_quote":"Supplies the adaptive fusion mechanism and the motion-representation framework that SSR-STF extends, and serves as the main baseline for mesh recovery."},{"cited_title":"Motionagformer: Enhancing 3d human pose estimation with a transformer-gcnformer network,","cited_arxiv_id":null,"evidence_quote":"The previous state-of-the-art hybrid method that SSR-STF must beat; the reported 4.5 mm ground-truth-2D MPJPE gain is measured against it."},{"cited_title":"3d human pose estimation with spatio-temporal criss-cross attention,","cited_arxiv_id":null,"evidence_quote":"Supplies the spatio-temporal criss-cross attention used in the global STFormer stream and is a direct baseline."},{"cited_title":"Large separable kernel attention: Rethinking the large kernel attention design in cnn,","cited_arxiv_id":null,"evidence_quote":"Shows how 2D depth-wise kernels decompose into cascaded 1D separable kernels, the basis of the irregular kernel in SSRA."},{"cited_title":"Mixste: Seq2seq mixed spatio-temporal encoder for 3d human pose estimation in video,","cited_arxiv_id":null,"evidence_quote":"A strong global-transformer baseline whose per-frame estimation setting frames the comparison."},{"cited_title":"Monocular 3d human pose estimation in the wild using improved cnn supervision,","cited_arxiv_id":null,"evidence_quote":"The MPI-INF-3DHP dataset used to demonstrate generalization across capture settings."},{"cited_title":"Stacked hourglass networks for human pose estimation,","cited_arxiv_id":null,"evidence_quote":"Supplies the Stacked Hourglass 2D detections used as input for the Human3.6M detected-pose experiments."}],"review_version":1}