{"id":"da5a36c0-2b62-4681-9223-3c627026ba12","arxiv_id":"2412.02254","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ProbPose predicts calibrated per-pixel localization probabilities and a separate presence-in-window probability, improving out-of-image keypoint localization on a new CropCOCO benchmark while roughly matching standard COCO accuracy.","lead":"ProbPose replaces heatmap outputs in 2D pose estimation with calibrated probability maps plus a 'presence probability' that says whether each keypoint is inside the crop. It also introduces the CropCOCO dataset and the Extended OKS metric to evaluate out-of-image keypoints.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline out-of-image gains are driven by crop augmentation and measured only on CropCOCO, a synthetic set built with the same crop procedure used in training; the paper's own ablation shows probability maps add no CropCOCO mAP, and OCHuman shows no Ex-mAP gain.","rationale":"The most load-bearing condition for the central claim is that CropCOCO results demonstrate generalizable out-of-image keypoint localization and calibrated presence. That condition is insecure because CropCOCO is generated by the same random-crop process used as training augmentation (Secs. 3.4 and 4.2), so high CropCOCO scores may only show that the model fits its own augmentation distribution. Table 2 sharpens the reader's concern: the crop-augmented model without probability maps already reaches CropCOCO mAP 81.7, identical to the full model, so probability maps contribute nothing to the headline localization number. Because Ex-mAP is not reported for the ablations, the presence-probability contribution is likewise unquantified. The 45% error reduction in Sec. 5.1 compares against a ViTPose model not trained with crop augmentation, so it cannot separate the effect of the presence head from the effect of training on cropped images. OCHuman, which contains almost no out-of-image keypoints, shows no Ex-mAP gain for the standard model (60.2 vs 60.1), further indicating that the gains do not transfer to a different in-image occlusion domain. These are methodological gaps, not internal inconsistencies; the method is coherent, the ablation study is honest, and code and models are released. The appropriate verdict remains CONDITIONAL, pending the missing ablations and an evaluation on a non-synthetic out-of-image benchmark.","tokens_in":14109,"tokens_out":4421,"duration_ms":48919,"concrete_test":"Add an Ex-mAP column and in/out classification accuracy to Table 2, and train a ViTPose-s baseline using exactly the same crop augmentation and training schedule. Compare full ProbPose, the Table 2 row-3 model (crop augmentation without probability maps), and the crop-augmented ViTPose on CropCOCO and on a non-synthetic set of real partially cropped images. If row-3 or crop-augmented ViTPose matches full ProbPose, the claimed improvements come from the augmentation and the synthetic evaluation distribution, not from the probabilistic components.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, that ProbPose's probabilistic representation improves out-of-image keypoint localization and presence classification, rests on CropCOCO, a synthetic dataset created by randomly cropping COCO (Sec. 4.2) with the same augmentation used in training (Sec. 3.4). This makes the evaluation distribution nearly identical to the training augmentation, so the headline gain (CropCOCO mAP 81.7 vs 72.7 for ViTPose-s, Table 1) may reflect distribution matching rather than a general capability. Table 2 is the sharpest evidence: crop augmentation alone yields CropCOCO mAP 81.7, and adding probability maps leaves it at 81.7. The headline localization gain is therefore attributable to the augmentation, not to the new representation. On OCHuman, which has almost no out-of-image keypoints, the standard model shows no Ex-mAP gain (60.2 vs 60.1, Table 1). Moreover, the claimed 45% in/out error reduction (Sec. 5.1, Fig. 5) compares ProbPose's presence probability with ViTPose confidence, but the ViTPose baseline was not trained with crop augmentation, conflating the augmentation effect with the presence head. Ex-mAP also selects the optimal presence threshold per model on the test set (Sec. 4.1), which can inflate reported gains. The critical missing experiment is an Ex-mAP and in/out classification ablation for the crop-augmented model without probability maps, and for a ViTPose baseline trained with the same crop augmentation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"ProbPose proposes a top-down 2D human pose estimation method that replaces heatmaps with normalized probability maps, trains them with an OKS-based expected-risk loss, and adds a predicted presence probability for each keypoint indicating whether it lies inside the activation window. The paper also introduces crop-based data augmentation to create out-of-image keypoints during training, a double-heatmap variant with a wider field of view, a new synthetic CropCOCO evaluation dataset, and an Extended OKS (Ex-OKS) metric that assigns penalties for presence/absence disagreements. Experiments on COCO, CropCOCO, and OCHuman report improved out-of-image localization on CropCOCO (mAP 81.7 vs. 72.7 for ViTPose-s), a small in-image COCO gain (76.6 vs. 75.9), and a 45% reduction in in/out classification error relative to thresholding ViTPose confidence.","tokens_in":14396,"tokens_out":2573,"duration_ms":28533,"significance":"The paper introduces several genuinely useful ideas: a principled probabilistic treatment of keypoint localization, a loss aligned with the OKS evaluation metric, a presence-probability output that is calibrated and distinct from confidence, and a new evaluation protocol for out-of-image keypoints. The release of code and models is a strength, and the ablation study honestly isolates the effect of crop augmentation from the probability-map representation. However, the central claim that the probabilistic representation improves out-of-image localization is not supported by the paper's own ablation, and the headline ex-image gains rest almost entirely on CropCOCO, a synthetic set generated with the same crop procedure used in training. The significance will be established only after the evaluation is strengthened with the missing control experiments and independent validation.","major_comments":[{"comment":"The ablation shows that crop augmentation alone yields CropCOCO mAP 81.7 and adding probability maps leaves it at 81.7, while without crop augmentation probability maps actually reduce CropCOCO mAP (72.4 vs. 73.7). Therefore the headline CropCOCO localization gain in Table 1 (81.7 vs. 72.7 for ViTPose-s) is attributable to the crop augmentation, not to the probabilistic representation. The manuscript should either revise the central claim that the probabilistic approach improves out-of-image localization or provide an experiment where probability maps improve out-of-image localization beyond the augmentation effect.","section":"Table 2"},{"comment":"The claimed 45% in/out classification error reduction compares the presence probability of ProbPose (trained with crop augmentation) with the confidence of ViTPose (trained without crop augmentation). This conflates the effect of the presence head with the effect of the augmentation, because the augmentation provides the training signal that makes any in/out predictor learnable. A control with a ViTPose or ProbPose variant trained with the same crop augmentation but using heatmap confidence for the in/out decision is necessary to attribute the improvement to the presence probability.","section":"Sec. 5.1, Fig. 5"},{"comment":"Ex-mAP is computed by selecting the optimal presence threshold for each model on the test set, as stated in the manuscript. This is a form of test-set threshold selection and can inflate the reported Ex-mAP gains, especially when the compared models have different confidence distributions. The authors should select thresholds on a validation set or report Ex-mAP as a function of threshold, rather than using the test-set optimum.","section":"Sec. 4.1"},{"comment":"On OCHuman, the standard ProbPose shows no Ex-mAP improvement over ViTPose (60.2 vs. 60.1), and the main gain appears only in the double-heatmap variant (61.2). Since OCHuman is the only out-of-domain dataset in the paper, this result weakens the claim that the probabilistic representation improves out-of-image and presence-aware performance in general; the claim currently rests almost entirely on CropCOCO, whose distribution is generated by the same crop procedure used for training. The authors should discuss this limitation explicitly and, if possible, evaluate on an independently created out-of-image benchmark.","section":"Table 1, OCHuman rows"},{"comment":"The paper reports small differences, such as 0.7 mAP on COCO and 0.1 Ex-mAP on OCHuman, without error bars, multiple seeds, or statistical significance testing. Given that the central in-image improvement is only about 1%, the authors should provide variance estimates or repeated-run results to establish that the differences are not within training noise.","section":"Sec. 5.2 / Tables 1-2"}],"minor_comments":[{"comment":"The notation for the activation window is inconsistent: Eq. (1) and Eq. (2) use AW, while Eq. (5) and Appendix A use AM. Please unify.","section":"Throughout"},{"comment":"The definition of de(AM, x') is not precise; it should state explicitly that the distance is measured from the activation-window boundary, not from some interior point, and should specify the behavior when the point lies inside the window.","section":"Eq. (5)"},{"comment":"The description of CropCOCO does not specify the crop sampling distribution, the range of crop strengths, or how bounding boxes are recomputed; this information is needed to assess how representative the synthetic evaluation is and to reproduce the dataset.","section":"Sec. 4.2"},{"comment":"The table header says 'PropPose' while the paper uses 'ProbPose'; please correct the typo.","section":"Sec. 5.2, Table 2 caption"},{"comment":"The statement that 'probability maps always sum to 1' via Sparsemax is slightly misleading: Sparsemax maps produce nonnegative outputs that sum to 1 only over the entire activation window, but the paper later uses per-pixel losses; the relationship between the per-pixel interpretation and the global normalization should be clarified.","section":"Sec. 3.1"},{"comment":"The calibration procedure is described as ensuring each quantile contains approximately 5% of ground-truth keypoints, but the precise binning, temperature-scaling fitting procedure, and whether calibration is performed on the training or validation split are not fully specified.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a coherent and well-motivated method with a clear evaluation protocol contribution, but the current evidence does not support the strongest claims about out-of-image localization improvement. The missing crop-augmented baselines and the test-set threshold selection are fixable within the manuscript's scope, so I recommend major revision rather than rejection. The authors should also be encouraged to make the CropCOCO crop-generation procedure fully public and to report variance estimates for the small COCO/OCHuman differences."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [colleague],\n\nQuick take on arXiv:2412.02254 (ProbPose). The paper introduces a clean idea: represent keypoint locations as Sparsemax-normalized probability maps trained with a pixel-wise expected-OKS risk, plus a separate head that predicts the probability the keypoint is inside the activation window. It also releases CropCOCO and an Ex-OKS metric for evaluating out-of-image keypoints. These are useful artifacts for the pose estimation community, and the paper is honestly written—the ablation in Table 2 explicitly shows that probability maps alone don't change mAP; all localization gain comes from crop augmentation.\n\nWhat is actually new: the presence probability is a distinct concept from confidence, the pixel-wise OKS loss is a sensible extension of YOLO-Pose, and Ex-OKS/CropCOCO give a concrete way to talk about out-of-image keypoints. Calibration of probability maps via temperature scaling is a nice touch.\n\nWhere it falls short: the headline claim—that ProbPose improves out-of-image localization—is supported only on CropCOCO, a synthetic set built by cropping COCO with the same procedure used in training. Table 2 shows the gain over ViTPose-s on CropCOCO (72.7 to 81.7) is fully explained by crop augmentation. The probability maps add nothing on mAP; their value is supposed to be in calibration and presence probability, but the paper doesn't isolate that cleanly. The 45% error reduction in Fig. 5 compares ProbPose's presence head against ViTPose confidence where ViTPose wasn't trained with crop augmentation, so the comparison conflates augmentation with representation. A crop-augmented ViTPose baseline, or at least the paper's own heatmap model with crop augmentation, is needed for a fair comparison. On OCHuman—which has almost no out-of-image keypoints—the standard ProbPose shows no Ex-mAP gain (60.2 vs 60.1), so generalization to real out-of-image data is unproven. Also, Ex-mAP threshold is selected on the test set, and the '25% outside the box' and 'per-keypoint distribution adaptivity' claims lack quantitative support.\n\nThese are addressable issues, not fatal ones. The core representation is coherent, the metric is thought-provoking, and the ablation is honest. But the paper currently overclaims what is actually demonstrated.\n\nI'd send it to review, but expect the authors to add a controlled baseline and moderate the claims. For a reading group, it's a decent case study on evaluation design.\n\nBest,\n[Your name]","headline":"The presence-probability and Ex-OKS ideas are useful, but the paper's headline out-of-image gains are really crop augmentation, and it needs a controlled baseline before those claims hold.","tokens_in":15002,"tokens_out":5728,"would_cite":false,"duration_ms":49937,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ProbPose treats each joint as a distribution, so it can say when a keypoint is outside the image, and that calibrated 'don't know' output improves localization both in and beyond the frame.","keywords":["human pose estimation","probability maps","presence probability","out-of-image keypoints","CropCOCO","Ex-OKS metric","calibration","cropping augmentation"],"falsifier":"Build a test set of real images in which people are truncated by the frame or heavily occluded, manually annotate keypoints that lie outside the visible image or the activation window, and compare ProbPose's presence probability and out-of-image localization against the ViTPose baseline; if presence accuracy matches simple confidence thresholding or the out-of-image localization gain on this real data does not appear, the central transfer claim is falsified. A separate check: on those real truncated images, measure whether the calibrated top-x-percent probability-map bins actually contain x percent of ground-truth keypoints, since the probabilistic-query claim stands or falls with that calibration.","tokens_in":13836,"feed_emoji":"🦴","tokens_out":10969,"duration_ms":104581,"temperature":0.7,"pith_summary":"ProbPose is a top-down 2D human pose estimator that replaces heatmaps with calibrated probability maps and adds a per-keypoint presence probability: the chance that the joint actually falls inside the model's activation window. The paper argues that current pose models ignore out-of-image keypoints during training and evaluation, so they guess when a joint is cropped out, and their confidence scores are uncalibrated for in/out decisions. Training with randomly cropped images lets the model localize keypoints beyond the image border, lifting CropCOCO mAP from 72.7 to 81.7 for a ViTPose-scale model while also slightly improving in-image accuracy on COCO (76.6 vs 75.9). The paper also introduces Ex-OKS, an extension of OKS that scores presence and location together, plus the CropCOCO dataset. A sympathetic reader should care because calibrated 'I cannot see this joint' output is what robotics, interaction, and safety-critical applications need, not just a point estimate.","feed_headline":"Pose model finds joints cropped out of frame and flags when it cannot","feed_subtitle":"Beats confidence thresholds by 45% on in/out classification, gains 9 mAP on cropped COCO.","key_machinery":"The central machinery is the probability map: a per-keypoint distribution over pixels in the activation window, normalized to sum to 1 by a Sparsemax activation, so its shape is learned rather than forced to a Gaussian. These maps are trained by an expected-risk version of the OKS loss, with a Sobel-gradient regularizer, and decoded by expected OKS maximization, which favors regions with large probability mass over sharp isolated peaks. Presence probability is a separate head trained with binary cross-entropy on keypoints displaced out of the window by random cropping; temperature scaling calibrates both outputs. A double-heatmap variant adds a second, wider activation window at the same output resolution, extending the model's reach to keypoints up to 25% beyond the bounding box, and the new CropCOCO dataset with the Ex-OKS metric supplies the training and evaluation protocol for out-of-image keypoints.","core_discovery":"On the paper's own terms, the central discovery is that keypoint localization and keypoint presence are two separate, jointly trainable quantities, and both belong in the output representation. ProbPose outputs a probability map over the activation window normalized to sum to 1, a presence probability that the keypoint is inside that window, a predicted OKS quality score, and a visibility flag; at inference it picks the location that maximizes expected OKS rather than the heatmap argmax. Trained with an OKS risk-minimization loss and a cropping augmentation that pushes keypoints outside the image, this representation reaches 81.7 mAP and 73.9 Ex-mAP on CropCOCO versus 72.7 and 66.5 for the ViTPose-s baseline, while slightly improving COCO in-image localization. The paper further shows that presence probability classifies in/out status more accurately than confidence thresholding, cutting the error by 45% on a balanced subset, and that calibrating the outputs makes probabilistic queries such as 'the smallest region containing this keypoint with 95% probability' meaningful.","pith_inferences":["Beyond the paper: the same presence-probability signal could serve as an explicit uncertainty input for downstream systems, letting a robot or controller distinguish 'the joint is visible but I am unsure exactly where' from 'the joint is not in view at all'.","Beyond the paper: because the paper treats the image border as one form of occlusion, the cropping augmentation could be extended to object occlusion or truncation, potentially improving crowded-scene and self-occlusion benchmarks.","Beyond the paper: Ex-OKS could be adopted as a plug-in protocol for bottom-up and single-stage pose estimators, whose activation window is the whole image, making presence evaluation a general standard rather than a crop-specific metric.","Beyond the paper: a natural next stress test is a dataset of organically truncated people, such as sports or surveillance footage, where keypoints leave the frame without synthetic preprocessing; that would test whether the crop-trained distribution transfers to real-world out-of-image keypoints."],"forward_implications":["Pose estimators can be trained to localize joints that leave the visible image: on CropCOCO the ProbPose-s model reaches 81.7 mAP and 73.9 Ex-mAP against 72.7 and 66.5 for ViTPose-s with the same backbone scale and ground-truth boxes.","In/out decisions should use a dedicated presence probability rather than the heatmap peak, since the paper reports a 45% error reduction on the balanced CropCOCO subset and 30% on the full set.","Calibrated probability maps turn pose outputs into probabilistic statements, such as a confidence region that contains the keypoint with a chosen probability, which raw heatmap confidence cannot provide.","The double-heatmap design buys a wider field of view without the resolution loss of enlarging the input, localizing keypoints up to 25% outside the bounding box with only a 0.4% AP drop on COCO.","Because Ex-OKS penalizes false positives and wrong presence calls, models trained or evaluated with it are discouraged from guessing joints that are not visible in the activation window."],"supporting_citations":[{"why":"Baseline top-down model whose architecture and training ProbPose extends; all headline comparisons are against ViTPose-s.","marker":"[31]"},{"why":"Source of the OKS loss that ProbPose reformulates as per-pixel expected risk for probability maps.","marker":"[18]"},{"why":"Information-dropping augmentation whose in-image gains ProbPose matches while crop augmentation adds out-of-image localization.","marker":"[9]"},{"why":"UDP decoding, the standard heatmap decoder that expected OKS maximization is compared against and refined over.","marker":"[8]"},{"why":"Sparsemax activation that turns network outputs into probability maps summing to 1.","marker":"[19]"},{"why":"COCO supplies the training annotations, the base images for CropCOCO, and the OKS conventions Ex-OKS extends.","marker":"[13]"},{"why":"OCHuman provides the held-out occlusion dataset used to test generalization beyond the training distribution.","marker":"[33]"}],"fun_headline_variants":["Probabilistic pose model finds joints outside the frame and admits when lost","Pose AI now knows when a joint is out of view — and says so","Calibrated keypoint presence: new pose model beats confusion on cropped frames","ProbPose separates where a joint is from whether it is visible","New pose model cuts in/out errors by 45% with calibrated probability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Randomly cropping COCO images produces a training and evaluation distribution that faithfully represents how keypoints end up outside real images or bounding boxes, and the Ex-OKS distance penalty reflects how much real applications penalize presence errors; if those do not hold, the headline gains on CropCOCO may not transfer to real-world out-of-image keypoints.","fun_headline_variants_meta":{"raw":{"variants":["Probabilistic pose model finds joints outside the frame and admits when lost","Pose AI now knows when a joint is out of view — and says so","Calibrated keypoint presence: new pose model beats confusion on cropped frames","ProbPose separates where a joint is from whether it is visible","New pose model cuts in/out errors by 45% with calibrated probability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00024,"raw_usage":{"total_tokens":1525,"prompt_tokens":959,"completion_tokens":566,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":466}},"tokens_in":575,"tokens_out":566,"duration_ms":6217,"temperature":1.0,"reasoning_tokens":466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:40:15.476857+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a test set of real images in which people are truncated by the frame or heavily occluded, manually annotate keypoints that lie outside the visible image or the activation window, and compare ProbPose's presence probability and out-of-image localization against the ViTPose baseline; if presence accuracy matches simple confidence thresholding or the out-of-image localization gain on this real data does not appear, the central transfer claim is falsified. A separate check: on those real truncated images, measure whether the calibrated top-x-percent probability-map bins actually contain x percent of ground-truth keypoints, since the probabilistic-query claim stands or falls with that calibration.","supporting_citations":[{"cited_title":"ViTPose: Simple vision transformer baselines for human pose estimation","cited_arxiv_id":null,"evidence_quote":"Baseline top-down model whose architecture and training ProbPose extends; all headline comparisons are against ViTPose-s."},{"cited_title":"Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss","cited_arxiv_id":null,"evidence_quote":"Source of the OKS loss that ProbPose reformulates as per-pixel expected risk for probability maps."},{"cited_title":"AID: Pushing the Performance Boundary of Human Pose Estimation with Information Dropping Augmentation","cited_arxiv_id":"2008.07139","evidence_quote":"Information-dropping augmentation whose in-image gains ProbPose matches while crop augmentation adds out-of-image localization."},{"cited_title":"The devil is in the details: Delving into unbiased data processing for human pose estimation","cited_arxiv_id":null,"evidence_quote":"UDP decoding, the standard heatmap decoder that expected OKS maximization is compared against and refined over."},{"cited_title":"Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C","cited_arxiv_id":null,"evidence_quote":"COCO supplies the training annotations, the base images for CropCOCO, and the OKS conventions Ex-OKS extends."},{"cited_title":"Rosin, Zixi Cai, Han Xi, Dingcheng Yang, Hao-Zhi Huang, and Shi-Min Hu","cited_arxiv_id":null,"evidence_quote":"OCHuman provides the held-out occlusion dataset used to test generalization beyond the training distribution."}],"review_version":1}