{"id":"22cb6744-014d-4aad-972c-932da692974f","arxiv_id":"2509.01095","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"An end-to-end video pose transformer built on PETR with spatio-temporal encoders and an instance consistency loss reaches 83.0 mAP on PoseTrack2017 and appears around 4x faster than DCPose.","lead":"VEPE is a transformer-based system that estimates multiple people's poses in video in one end-to-end pass, without separate person detectors. On the PoseTrack2017 benchmark it reaches 83.0 mAP, behind the strongest baseline but several times faster, while tables for PoseTrack2018 and 2021 results are missing from the text.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 300% efficiency claim rests on uncontrolled cross-paper timings; a controlled re-benchmark on identical hardware and software is needed before the central claim can stand.","rationale":"I read the paper in good faith: VEPE is a coherent end-to-end video pose estimation extension of PETR, with a plausible temporal encoder/decoder design and an instance-consistency mechanism. The ablation in Table 2 is internally consistent and shows incremental gains. However, the central claim that the paper advances is not just accuracy, but a 300% efficiency improvement over two-stage methods. That claim rests on Table 1 comparing inference times taken from different papers with different implementations, detectors, and measurement conventions. Section 4.2.3 gives only the single shared condition of a V100 GPU, which is far too weak to support a quantitative speedup factor. The reader's weakest assumption—cross-paper timing comparability—is exactly the load-bearing concern. The missing PoseTrack2018 and PoseTrack2021 tables are also a serious incompleteness: the text asserts improvements of 0.2 and 0.3 mAP over DCPose on those datasets but provides no data to verify them. This compounds the problem, but the efficiency claim is the primary differentiator and the least controlled. No code or checkpoints are provided, so the reported 334 ms cannot be independently reproduced. For these reasons, the current evidence does not support the central claim, and the reader's REJECT verdict remains appropriate. I am not claiming the method is wrong; I am claiming that the paper as submitted does not provide the controlled comparison needed to establish its main quantitative advantage.","tokens_in":13735,"tokens_out":4780,"duration_ms":59710,"concrete_test":"Obtain or reproduce the official DCPose and VEPE implementations, run both on the same NVIDIA V100 GPU with identical PyTorch/CUDA/OpenCV versions, identical batch size (1), identical input preprocessing (short side 800, long side ≤1333), and identical evaluation code on the PoseTrack2017 validation set. Measure median wall-clock latency per frame, including all preprocessing and post-processing. If VEPE is not at least 3x faster than DCPose under these identical conditions, the 300% efficiency claim fails.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's headline claim—'outperforms most two-stage models and improves inference efficiency by 300%'—depends almost entirely on Table 1, where VEPE reports 83.0 mAP at 334 ms/frame versus DCPose's 82.8 mAP at 1390 ms/frame. Section 4.2.3 only states that all methods were tested on a V100 GPU. It does not control for software versions, batching behavior, input resolution, whether timings include detection/NMS/ROI-cropping for top-down baselines, parameter counts, or measurement protocol. DCPose's 1390 ms may include a YOLOv3 detector and per-instance post-processing, while VEPE's 334 ms may reflect a different pipeline; without a unified measurement harness, the 1390-to-334 comparison does not establish a 300% efficiency gain. Additionally, Sections 4.3.2 and 4.3.3 refer to comparison tables for PoseTrack2018 and PoseTrack2021 that are absent (the tables are literally missing as 'Table ??'), so the accuracy component of the central claim is also unverifiable on two of the three datasets. The architecture itself is plausible, but the main empirical differentiator—speed—is the least secure part of the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VEPE, an end-to-end Transformer framework for video multi-person pose estimation, extending PETR with three spatio-temporal modules (STPE, STDME, STPD) and an instance consistency mechanism (ICM) with a learnable instance query. The method is evaluated on PoseTrack2017, and the authors claim it outperforms most two-stage models on PoseTrack datasets and improves inference efficiency by 300%. The manuscript includes PoseTrack2017 validation results (83.0 mAP, 334 ms/frame), component ablations, and a threshold study for pose query selection.","tokens_in":14071,"tokens_out":5063,"duration_ms":58408,"significance":"If the claims are validated, VEPE would be a meaningful step toward efficient end-to-end video pose estimation: it removes the separate detector and post-processing (NMS, RoI cropping) used by two-stage methods, and the proposed temporal modules are architecturally reasonable. The PoseTrack2017 validation result is competitive with DCPose, and the ablation study suggests each component contributes. However, the current evidence is incomplete: two of the three dataset comparisons are missing, and the headline 300% efficiency claim rests on uncontrolled cross-paper timings. These issues are fixable and do not invalidate the architectural idea, but they must be addressed before the central claims can be accepted.","major_comments":[{"comment":"The text refers to 'table ??' for the PoseTrack2018 and PoseTrack2021 comparisons, but no such tables are present in the manuscript. Since the abstract and conclusion claim superiority 'on the Posetrack dataset' and the conclusion repeats the claim, these results are load-bearing. Without the actual PoseTrack2018 and PoseTrack2021 numbers, the accuracy component of the central claim is unverifiable on two of the three datasets. Please include the tables or restrict the claims explicitly to PoseTrack2017.","section":"§4.3.2–4.3.3"},{"comment":"The 300% inference-efficiency claim is based on comparing VEPE's 334 ms/frame with DCPose's 1390 ms/frame and FAMI-Pose's 1577 ms/frame. The text only states that all methods were tested on a V100 GPU. This does not control for software versions, batching, input resolution, whether timings for top-down baselines include detector and post-processing, parameter counts, or measurement protocols. As reported, the speed advantage is not established. Provide a controlled benchmark on identical hardware/software, or rephrase the claim as a reported-per-paper comparison with explicit caveats.","section":"§4.2.3, Table 1"},{"comment":"All baselines in Table 1 except VEPE are two-stage models. Since the paper's stated contribution is an end-to-end framework and the efficiency argument is framed against two-stage pipelines, the comparison should also include recent end-to-end pose estimators (PETR, QueryPose, ED-POSE, GroupPose) on the same benchmark. Without such comparisons, the efficiency and accuracy advantages relative to the end-to-end family are not demonstrated.","section":"Table 1, §4.2.3"}],"minor_comments":[{"comment":"The relationship between Pose Query Selection (PQS) and the STPE ablation is unclear. Table 2 reports that adding STPE raises mAP from 77.2 to 80.7, and Table 3 reports threshold effects around 80.4–80.7. Is PQS included in the STPE row? What threshold is used for the full model in Table 1? Please clarify and consider adding a PQS row/column to the ablation table.","section":"§4.4.1–4.4.2"},{"comment":"The instance consistency loss uses a triplet formulation with cosine distance, but the construction of the candidate set, the choice of negatives, and the handling of multiple negatives are not specified. Please define how anchors, positives, and negatives are sampled across frames and how the loss is aggregated over the video clip.","section":"§3.6, Eq. (5)"},{"comment":"The instance tracking function is only supported by a qualitative similarity heatmap. No quantitative tracking metric (e.g., MOTA, IDF1) is reported. Either report such a metric or restrict the tracking claim to its effect on pose estimation mAP.","section":"§4.4.3, Figure 6"},{"comment":"There are several typos and inconsistent notations: 'HMA' in Eq. (2) should be 'MHA'; 'memroy' after Eq. (3); 'Posetrack' vs 'PoseTrack'; 'the row column' in the Figure 5 caption. Please proofread carefully.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The missing tables are literal in the manuscript text ('table ??'), not artifacts of the review process. If the authors cannot supply the PoseTrack2018/PoseTrack2021 results, the claims should be narrowed to PoseTrack2017. The speed comparison is the most fragile part of the empirical contribution; a controlled re-benchmark or a softened claim is needed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a coherent, incremental extension of PETR to video, with a sensible set of temporal modules and a contrastive instance-consistency loss. The PoseTrack2017 numbers are competitive (83.0 mAP) and the ablation (Table 2) shows the temporal pieces earn their keep—STPE alone gives +3.5 mAP over baseline. If that is real, it's a useful data point for the field.\n\nBut the paper as submitted is not finishable. Sections 4.3.2 and 4.3.3 refer to 'table ??' for PoseTrack2018 and PoseTrack2021, and those tables simply are not in the manuscript. The central accuracy claim—'outperforms most two-stage models'—is supposed to rest on all three benchmarks, so on two of them the paper asks the reader to take it on faith. That alone is grounds for rejection at this stage.\n\nThe speed claim is shakier. The abstract says 'improves inference efficiency by 300%,' and Table 1 compares VEPE's 334 ms/frame to DCPose's 1390 ms/frame. But those timings come from different papers, and section 4.2.3 only says all methods were run on a V100. No control for batch size, input resolution, software versions, or whether the top-down timings include the detector and post-processing (the last is explicitly noted to be YOLOv3, which is a big chunk of the cost). Without a unified benchmark, the 334-vs-1390 comparison does not establish a 4x gain. I'd want to see the authors benchmark both under the same protocol.\n\nOther soft spots are minor by comparison: no error bars, no code, and the ablation only sweeps one of the four free parameters. The instance consistency loss optimizes the same similarity metric used for matching—that's not circular in a damaging way, but it's worth a sentence acknowledging the loop.\n\nThe architecture itself is clear and the writing is straightforward. I'd trust the PoseTrack2017 ablation as an internally consistent result, even though I wouldn't trust the speed comparison.\n\nFor whom: this is a paper for people actively working on video pose estimation, particularly those curious whether the PETR-style query pipeline can be extended temporally. As a preprint, it's a useful technical report. As a submission, it needs the missing tables, a controlled speed benchmark, and ideally code or at least error bars, before a serious referee should spend time on it.\n\nMy recommendation: currently desk-reject or 'revise before review'—the missing tables are a basic completeness failure, and the speed claim is not supportable. The ideas deserve another look once the evidence is actually supplied.","headline":"A coherent PETR extension with honest ablations, but the manuscript is incomplete: two comparison tables are missing and the 300% speedup claim rests on uncontrolled cross-paper timings.","tokens_in":14514,"tokens_out":2968,"would_cite":false,"duration_ms":32399,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VEPE treats a video clip as a sequence-to-sequence pose problem: on PoseTrack2017 it reaches 83.0 mAP at 334 ms per frame, beating DCPose's 82.8 mAP at 1390 ms per frame.","keywords":["video multi-person pose estimation","end-to-end Transformer","spatio-temporal attention","deformable attention","instance consistency","PoseTrack","pose query","temporal modeling"],"falsifier":"Run VEPE and DCPose on the same machine, same software stack, same input resolution (shortest edge 800), and same batch size, timing PoseTrack2017 per frame on a single V100; if VEPE does not reproduce roughly 83.0 mAP at roughly 334 ms while DCPose sits near 82.8 mAP at 1390 ms, the central claim fails.","tokens_in":13657,"feed_emoji":"🏃","tokens_out":16101,"duration_ms":151252,"temperature":0.7,"pith_summary":"VEPE is an attempt to make video multi-person pose estimation fully end-to-end: instead of detecting and cropping each person per frame and then running a single-person temporal model, it treats the whole clip as a sequence-to-sequence problem solved by Transformer queries. The paper's central claim is that three spatio-temporal modules—a pose encoder, a deformable memory encoder, and a cascaded pose decoder—plus an instance-consistency loss can aggregate temporal context well enough to match or beat two-stage pipelines while removing detectors, RoI cropping, and NMS. On PoseTrack2017, VEPE reports 83.0 mAP at 334 ms per frame, compared with DCPose's 82.8 mAP at 1390 ms, which the paper summarizes as about a 300% inference-efficiency gain. If true, accurate video pose estimation does not need the expensive two-stage apparatus, and inference time is tied to a fixed set of pose queries rather than to the number of people in the frame.","feed_headline":"End-to-end video pose hits 83.0 mAP in 334 ms","feed_subtitle":"A query-based decoder tracks people across frames, matching two-stage models at about a quarter of the runtime.","key_machinery":"Four objects carry the argument. (1) STPE: self-attention over keyframe pose queries plus cross-attention with reference-frame queries, masked by instance-query similarity so a query attends to its match. (2) STDME: temporal multi-scale deformable attention that samples T×L×K points across frames and scales, pushing cross-frame visual cues into the keyframe feature memory. (3) STPD: three cascaded decoder layers refining normalized coordinates by Q_d = σ(σ^{-1}(Q_{d-1}) + ΔQ_p). (4) Instance consistency mechanism: learnable instance queries, one per pose query, trained with L_ic = Σ_i max(0, d(a_i,p_i) − d(a_i,n_i) + margin); the same similarity scores give tracking and the STPE attention ma","core_discovery":"On its own terms, the discovery is that temporal context for multi-person pose estimation can be folded into a single query-based Transformer. VEPE extends PETR with three spatio-temporal modules that align same-instance pose queries, aggregate cross-frame multi-scale visual features, and refine keyframe poses. A margin-based instance-consistency loss keeps the same person's instance queries close across frames and different people's apart; similarity between instance queries supplies tracking and an attention mask. The reported result: 83.0 mAP on PoseTrack2017 validation, up 5.8 over the spatial baseline, slightly above DCPose, at 334 ms per frame vs 1390 ms for DCPose.","pith_inferences":["The instance-consistency loss is a generic contrastive mechanism—learned per-identity queries plus a margin-based pull/push loss—so it could be lifted to other query-based video tasks such as multi-object tracking or video instance segmentation.","VEPE is trained and evaluated with a fixed window of 3 frames, so it remains untested whether the STPE/STDME modules keep helping over longer clips without retraining; extending the window is a natural stress test.","The paper's own spatial baseline (77.2 mAP) is the evidence for the temporal modules' gain; comparing against a stronger single-frame end-to-end model would further isolate what the spatio-temporal attention adds.","If the efficiency advantage holds in dense scenes, end-to-end query decoders could become the default choice for real-time multi-person video pose estimation, because one network handles spatial and temporal context simultaneously."],"forward_implications":["Video pose estimation can drop explicit person detection, RoI cropping, and NMS without sacrificing accuracy to two-stage models, at least on PoseTrack-style benchmarks.","The temporal modules are worth about 5.8 mAP over the spatial-only PETR baseline on PoseTrack2017, with STPE contributing the largest single gain in the ablation.","Because decoding cost is tied to a fixed number of pose queries rather than the number of detected persons, per-frame latency should grow much more slowly in crowded scenes than top-down pipelines' latency.","Instance queries learned for matching supply tracking as a by-product: similarity between instance queries identifies the same person across frames and guides cross-frame pose-query matching.","If the 334 ms/frame figure reproduces under controlled conditions, an end-to-end video pose estimator can operate in the same speed class as image pose models while still exploiting temporal context."],"supporting_citations":[{"why":"PETR, the fully end-to-end pose estimation framework that VEPE extends; supplies the spatial encoder, pose decoder, and pose queries.","marker":"[29]"},{"why":"Deformable DETR, source of the deformable multi-scale attention used in STDME and STPD.","marker":"[50]"},{"why":"PoseTrack2017, the benchmark whose validation set provides the main accuracy and timing results.","marker":"[14]"},{"why":"DCPose, the strongest two-stage baseline VEPE slightly outperforms in mAP and beats by roughly 4x in per-frame time.","marker":"[21]"},{"why":"FAMI-Pose, a two-stage video pose baseline with higher mAP and slower inference, used to frame the efficiency comparison.","marker":"[22]"},{"why":"PoseWarper, a representative learned temporal pose method that VEPE is compared against on PoseTrack.","marker":"[5]"},{"why":"HRNet-W48, the backbone chosen for VEPE and for several comparison methods.","marker":"[31]"},{"why":"YOLOv3, the human detector used by the top-down baselines whose inference times are compared with VEPE's.","marker":"[28]"},{"why":"Hungarian algorithm, used to establish one-to-one prediction-to-ground-truth correspondence for the instance consistency loss.","marker":"[17]"}],"fun_headline_variants":["One Transformer tracks and estimates video pose at 334 ms","Video pose hits 83.0 mAP with a single end-to-end model","4x faster video pose matching two-stage accuracy","Unified video pose Transformer: 334 ms per frame, 83.0 mAP","End-to-end video pose: no separate detector, 4x speedup"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The 300% speed gain assumes that per-frame inference times reported by different papers are directly comparable because they were all tested on a V100 GPU, without controlling for software versions, batching, input resolution, or model parameter counts.","fun_headline_variants_meta":{"raw":{"variants":["One Transformer tracks and estimates video pose at 334 ms","Video pose hits 83.0 mAP with a single end-to-end model","4x faster video pose matching two-stage accuracy","Unified video pose Transformer: 334 ms per frame, 83.0 mAP","End-to-end video pose: no separate detector, 4x speedup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1353,"prompt_tokens":803,"completion_tokens":550,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":455}},"tokens_in":547,"tokens_out":550,"duration_ms":7431,"temperature":1.0,"reasoning_tokens":455,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:52:26.558193+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run VEPE and DCPose on the same machine, same software stack, same input resolution (shortest edge 800), and same batch size, timing PoseTrack2017 per frame on a single V100; if VEPE does not reproduce roughly 83.0 mAP at roughly 334 ms while DCPose sits near 82.8 mAP at 1390 ms, the central claim fails.","supporting_citations":[{"cited_title":"End-to-end multi-person pose estimation with transformers","cited_arxiv_id":null,"evidence_quote":"PETR, the fully end-to-end pose estimation framework that VEPE extends; supplies the spatial encoder, pose decoder, and pose queries."},{"cited_title":"Posetrack: Joint multi-person pose estimation and tracking","cited_arxiv_id":null,"evidence_quote":"PoseTrack2017, the benchmark whose validation set provides the main accuracy and timing results."},{"cited_title":"Deep dual consec- utive network for human pose estimation","cited_arxiv_id":null,"evidence_quote":"DCPose, the strongest two-stage baseline VEPE slightly outperforms in mAP and beats by roughly 4x in per-frame time."},{"cited_title":"Tempo- ral feature alignment and mutual information maximization for video-based human pose estimation","cited_arxiv_id":null,"evidence_quote":"FAMI-Pose, a two-stage video pose baseline with higher mAP and slower inference, used to frame the efficiency comparison."},{"cited_title":"Learning temporal pose esti- mation from sparsely-labeled videos","cited_arxiv_id":null,"evidence_quote":"PoseWarper, a representative learned temporal pose method that VEPE is compared against on PoseTrack."},{"cited_title":"Deep high-resolution representation learning for human pose esti- mation","cited_arxiv_id":null,"evidence_quote":"HRNet-W48, the backbone chosen for VEPE and for several comparison methods."},{"cited_title":"The hungarian method for the assignment problem","cited_arxiv_id":null,"evidence_quote":"Hungarian algorithm, used to establish one-to-one prediction-to-ground-truth correspondence for the instance consistency loss."}],"review_version":1}