{"id":"d184456e-5e07-44c5-a628-87fbcdb1935f","arxiv_id":"2505.16633","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Silhouette-only input to 3D-MuPPET keeps average error (RMSE) comparable to the textured version, but keypoint accuracy (PCK) drops sharply and zero-shot transfer to other birds is only preliminary.","lead":"This paper tests whether a multi-pigeon 3D pose tracking system can work from silhouettes instead of textured images, and reports mixed results: RMSE stays comparable, but keypoint accuracy drops sharply. It also tries a pigeon-trained model on four other bird species with no retraining, getting low but nonzero accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'comparable accuracy' claim rests solely on RMSE, while the paper's own preferred PCK and median metrics show large degradations; without a distributional analysis the central claim is unsupported.","rationale":"The reader's weakest assumption was the largest-connected-component mask selection in Sec. 3.2. That is a real limitation, especially for overlapping birds, but it is not the most load-bearing issue: even with perfect silhouettes and perfect masks, the central 'comparable accuracy' claim fails on the metrics the paper itself considers most meaningful. The key tension is internal to the reported numbers: RMSE improves while median error and PCK degrade substantially. Since RMSE is dominated by large errors, the result is consistent with a model that avoids catastrophic outliers but has worse typical accuracy. The paper does not provide error distributions, confidence intervals, or any downstream task evaluation that would justify privileging RMSE. The species-transfer section is explicitly preliminary and manually filtered, so it does not carry the central claim. The paper has real strengths: it builds directly on 3D-MuPPET, reports honest per-keypoint breakdowns, and the Wild-MuPPET RMSE improvement is notable. But the headline assertion of comparable accuracy needs additional evidence before it can be accepted. I would keep the conditional verdict, but sharpen the condition: the authors should either report distributional error statistics and confidence intervals, or reframe the claim from 'comparable accuracy' to 'fewer extreme outliers at the cost of lower typical accuracy'. The lack of code and data also prevents independent verification, though that alone is not a fatal flaw.","tokens_in":9455,"tokens_out":3729,"duration_ms":33543,"concrete_test":"Recompute Tables 2 and 3 as per-frame error distributions (CDFs and quantiles) with bootstrap 95% confidence intervals over test frames and keypoints. Also compute RMSE after removing the worst 5% of frames. If the bootstrap intervals for PCK05 and median error do not overlap between 3D-MuPPET and the silhouette methods, or if the error CDFs show the silhouette method is worse at every quantile above roughly the 20th percentile, then 'comparable accuracy' is falsified under the paper's own metric priority and the claim should be revised to 'more outlier-robust but less accurate'.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract and Sec. 6 claim 'comparable accuracy' to 3D-MuPPET, but the only metric favoring the silhouette methods is RMSE (Tab. 2: 20.5 vs 25.0 mm; Tab. 3: 32.1 vs 53.4 mm). Sec. 4.1 explicitly states that PCK is 'the more meaningful performance measure', and on every such metric the silhouette methods are substantially worse: on 3D-POP, median 13.5 vs 7.5 mm, PCK05 35.4% vs 66.1%, PCK10 74.5% vs 90.9%; on Wild-MuPPET, median 21.5 vs 15.0 mm, PCK05 8.8% vs 25.1%, PCK10 48.3% vs 74.4%. A lower RMSE together with a higher median indicates a distribution that is narrower around a worse center: fewer extreme outliers but systematically less accurate typical predictions. Calling this 'comparable accuracy' is misleading under the paper's own metric priority. The authors interpret the lower RMSE as a reduction in significant outliers, but no outlier analysis, error CDF, quantile breakdown, or task-level evaluation is provided to show that this outlier reduction is what matters for downstream use. The central claim is therefore not established by the reported evidence, independent of the largest-connected-component assumption that the reader flagged.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a texture-independent extension of the 3D-MuPPET framework for multi-pigeon 2D/3D pose estimation and tracking. Instead of feeding RGB crops to DeepLabCut, the method generates silhouettes via SAM (DLCSAM) or via SAM with largest-connected-component post-processing (DLCISO), and on Wild-MuPPET also via YOLOv8-seg (DLCISO-YOLO). The 2D keypoints are triangulated to 3D and tracked as in 3D-MuPPET. The authors report results on the 3D-POP and Wild-MuPPET benchmarks, an ablation of SAM variants, a keypoint-wise error analysis, and zero-shot species-transfer experiments on four bird species from Animal Kingdom. The central claim, stated in the abstract and Sec. 6, is that the texture-independent approach achieves accuracy comparable to the original texture-dependent 3D-MuPPET while being robust to texture and illumination changes.","tokens_in":9746,"tokens_out":2955,"duration_ms":24903,"significance":"If the central claim were fully supported, the paper would make a useful contribution: silhouette-only input would enable pose estimation to transfer across appearances and to new species without fine-tuning, which is valuable for bird behavior research. The paper has several strengths: it evaluates on held-out public benchmarks (3D-POP and Wild-MuPPET), uses zero-shot species transfer rather than retraining, includes a keypoint-level error analysis, and reports ablation studies on SAM variants and data augmentation. No circularity issue is present: the method is trained on standard annotations and evaluated on held-out splits, and the comparison with 3D-MuPPET uses the same splits. However, the main claim of \"comparable accuracy\" is not supported by the paper's own preferred metrics (PCK and median error), and several auxiliary claims, such as \"preliminary promising results\" for species transfer, are stated without quantitative baselines or thresholds. The paper is therefore a reasonable proof-of-concept but needs substantive revision of its claims and evidence before publication.","major_comments":[{"comment":"The central claim of \"comparable accuracy\" rests solely on RMSE. Sec. 4.1 states that PCK is \"the more meaningful performance measure,\" and on every such measure the silhouette methods are substantially worse: on 3D-POP (Tab. 2), median error rises from 7.5 mm to 13.5 mm, PCK05 drops from 66.1% to 35.4%, and PCK10 drops from 90.9% to 74.5%; on Wild-MuPPET (Tab. 3), median rises from 15.0 mm to 21.5 mm, PCK05 drops from 25.1% to 8.8%, and PCK10 drops from 74.4% to 48.3%. A lower RMSE combined with a higher median indicates a distribution that is more concentrated around a less accurate center, not a generally comparable predictor. The authors interpret the lower RMSE as a reduction in outliers, but they provide no outlier definition, no error CDF or quantile breakdown, and no task-level analysis showing that the RMSE improvement matters for downstream use. The \"comparable accuracy\" claim is therefore not established by the reported evidence and should be replaced with a claim that accurately reflects the metric trade-off, or supported with the missing distributional analysis.","section":"Sec. 4.1, Sec. 4.2, Tabs. 2 and 3"},{"comment":"The largest-connected-component assumption is load-bearing for the DLCISO variant and is acknowledged in Sec. 5 only as a general limitation. In multi-animal scenes with up to 10 pigeons, overlapping birds can split into disconnected mask fragments or merge into a single component, and the assumption that the largest component is the target pigeon has no direct validation. The paper reports aggregate results over all group sizes but no breakdown by number of animals, occlusion level, or mask quality, so the reader cannot tell when the assumption fails and how often it fails. This should be quantified or at least analyzed on a subset of frames, because the multi-animal claim is part of the paper's stated contribution.","section":"Sec. 3.2, Sec. 5"},{"comment":"The species-transfer results are presented as \"preliminary promising results,\" but the only numerical evidence is Table 6, where PCK10 ranges from 19.9% to 51.9% and PCK05 from 8.5% to 14.0%. Without a baseline (e.g., the texture-dependent model under the same protocol, or a chance-level bound) or a task-level demonstration of usefulness, these numbers do not by themselves establish that the model \"is capable of generalizing to new species.\" Additionally, the 73 raven images were manually selected for similar angle and behavior to 3D-POP; the selection criteria are subjective and not described in enough detail to judge whether the reported PCK10 of 51.9% is representative. The paper should either temper the conclusion or provide a more rigorous evaluation, including the number of images per species and a comparison against a suitable baseline.","section":"Sec. 4.4, Tab. 6"},{"comment":"The keypoint-wise analysis is informative but also reveals that the method is not yet reliable for a major body part: the tail has a median error of 21.0 mm and PCK05 of 1.9%, while left/right shoulder and eye confusion is admitted in Sec. 5. These are not merely minor imperfections; they directly affect the usability of the estimated pose for behavioral analysis. The manuscript should either discuss the consequences of these errors for the intended downstream use or restrict the claims to keypoints that can be localized reliably from silhouettes.","section":"Sec. 4.3, Tab. 4"}],"minor_comments":[{"comment":"There are several typos and inconsistencies: \"Waldman et al.\" should be \"Waldmann et al.\"; \"tine-tuning\" in Sec. 1 should be \"fine-tuning\"; \"apperance\" in Sec. 3.2 should be \"appearance\"; and the table captions use \"strived for\" rather than the more standard \"lower is better / higher is better.\"","section":"Throughout"},{"comment":"The definition of the PCK threshold in 3D is given as \"the maximum distance between any two keypoints,\" but it is not stated whether this is computed per frame from ground-truth keypoints and whether it matches the protocol used by 3D-MuPPET. Please clarify for reproducibility.","section":"Sec. 4.1"},{"comment":"Tables 2 and 3 report single aggregate values per metric, but the test set for 3D-POP consists of four sequences and Wild-MuPPET is a single pigeon; no variance or per-sequence breakdown is given. Please report per-sequence results or confidence intervals, especially given the small number of test sequences.","section":"Sec. 4.2"},{"comment":"The SAM ablation reports only three configurations (huge, base, base fine-tuned). It would be helpful to state the number of fine-tuning iterations and the fine-tuning data source, since the conclusion that fine-tuning has \"limited benefit\" depends on this experimental setup.","section":"Sec. 4.3, Tab. 5"},{"comment":"The species-transfer section does not report the number of images evaluated per species, and the text says \"we evaluate on all images\" without giving the count. Please include sample sizes and, if possible, the keypoint-level breakdown for the transfer results.","section":"Sec. 4.4"},{"comment":"The limitation that silhouettes make left-right keypoint assignment ambiguous is important, but the proposed solution (triangulating different left-right combinations and picking the smallest triangulation error) is not tested. Consider adding a small experiment or at least a more concrete description of how this would be implemented.","section":"Sec. 5"},{"comment":"The conclusion repeats the \"comparable accuracy\" claim without the caveat that it applies only to RMSE and not to the paper's own preferred metrics. The conclusion should be revised to be consistent with the metric analysis requested in the major comments.","section":"Sec. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable proof-of-concept for silhouette-based multi-animal pose estimation in pigeons, but the advertised central claim is not supported under the paper's own metric priority. The authors need to either perform the missing distributional/outlier analysis and demonstrate downstream relevance, or reframe the contribution as a texture-independent method with reduced accuracy on standard keypoint metrics. The species-transfer and multi-animal robustness claims also need strengthening. If the authors address these points, the paper could become a solid contribution; in its current form the evidence is not sufficient for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a straightforward but useful extension of 3D-MuPPET — replace the textured crop with a SAM-derived silhouette, retrain DeepLabCut, keep the multi-view triangulation and tracking as-is. The genuinely new elements are the systematic evaluation of silhouette-only 2D/3D pose estimation for multiple pigeons and the zero-shot species-transfer experiment on four bird species. Both are honestly scoped as preliminary.\n\nWhat it does well: the authors report all the standard metrics (RMSE, median, PCK05/10) instead of cherry-picking; the keypoint-wise breakdown in Table 4 is informative (tail is a disaster, head is fine); the SAM ablation is a reasonable check; and the limitations section is unusually candid about left-right ambiguity and silhouette quality. The benchmark against 3D-MuPPET on the same splits is fair, not circular.\n\nThe soft spot is the central claim. The abstract and conclusion say \"comparable accuracy,\" but that rests entirely on RMSE (20.5 vs 25.0 mm on 3D-POP; 32.1 vs 53.4 mm on Wild-MuPPET). The paper itself calls PCK \"the more meaningful performance measure,\" and on PCK the silhouette models lose badly: PCK05 drops from 66.1% to 35.4% on 3D-POP and from 25.1% to 8.8% on Wild-MuPPET; medians roughly double. A lower RMSE with a higher median means a narrower error distribution around a worse center — fewer extreme outliers but systematically less accurate typical predictions. Without an outlier analysis or task-level evaluation, \"comparable\" is an overstatement. The species-transfer results are weak (PCK10 20–52%) and the best number comes from a manually selected subset of ravens, which the authors disclose but which limits the conclusion. Also missing: code, data release plan, and error bars for any metric.\n\nNone of this kills the paper. The idea is plausible, the experiments are honest, and the Wild-MuPPET RMSE result is interesting. But the claims need to be pulled back to what the evidence supports: silhouette-based estimation trades typical accuracy for robustness to extreme outliers, and species transfer is suggestive, not proven.\n\nWho is this for? Behavioral ecologists who want texture-robust pose estimation without re-annotation, and anyone building on 3D-MuPPET. It deserves peer review with a request for revision — tighten the \"comparable\" language, add error bars and an outlier/CDF analysis, and either release code or clarify why not.","headline":"A useful but overclaimed extension of 3D-MuPPET: silhouette-only input trades typical keypoint accuracy for fewer extreme outliers, and the 'comparable accuracy' statement only holds under RMSE, not under the paper's own preferred PCK metrics.","tokens_in":10311,"tokens_out":2064,"would_cite":false,"duration_ms":17247,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Feeding a pose-estimation network a binary silhouette instead of a textured RGB image preserves 3D multi-pigeon tracking accuracy and lets the pigeon-trained model produce keypoints on other bird species without retraining.","keywords":["3D pose estimation","multi-animal tracking","silhouette-based keypoints","texture independence","species transfer","segmentation masks","multi-view triangulation","pigeons"],"falsifier":"Take frames in which two pigeons occlude each other so their masks merge into one connected component, and compare per-keypoint accuracy on those frames against accuracy on frames with isolated birds; if median keypoint error jumps and the fraction of keypoints within the strict accuracy threshold collapses specifically in merged-mask frames while the textured-input baseline does not, the assumption that the largest component equals the bird is falsified.","tokens_in":9213,"feed_emoji":"🐦","tokens_out":9003,"duration_ms":63891,"temperature":0.7,"pith_summary":"The paper tries to show that a bird's silhouette carries enough information to estimate and track 3D joint positions, so that texture is not needed. It replaces the textured image input of an existing multi-pigeon pose-and-tracking framework with binary masks produced by a segmentation model, then retrains the 2D keypoint network on those masks. On the 3D-POP benchmark the silhouette-only pipeline reaches accuracy comparable to the texture-based original, and on a harder outdoor dataset it reports a lower average 3D error. The same pigeon-trained model yields preliminary 2D keypoint estimates on four other bird species without fine-tuning. If this holds, researchers could track animal pose in new species and environments without collecting species-specific textured training data.","feed_headline":"Silhouette-only pipeline matches texture-based 3D bird pose tracking","feed_subtitle":"A binary mask in place of a textured image keeps multi-pigeon 3D accuracy and transfers to new bird species.","key_machinery":"The central object is the binary silhouette: a segmentation mask reduced to its largest connected component, an object the paper calls DLCISO. This object does the work of stripping away texture, color, and illumination while preserving the shape cues the keypoint network needs. The mask is generated by prompting a segmentation model with the detector's bounding box, and it is the only input seen by the pose network. The paper treats this abstraction as the mechanism that makes the framework both texture-independent and transferable across species.","core_discovery":"The central claim is that texture is not a necessary input for multi-animal 3D keypoint estimation in birds. By converting each detected bird into a binary silhouette and training a keypoint regressor on those silhouettes, the authors preserve the original framework's ability to estimate and track 2D and 3D poses of up to ten pigeons from four camera views. They report that this texture-independent version achieves comparable accuracy to the texture-dependent baseline, and on the Wild-MuPPET outdoor dataset it attains a lower average 3D error (32.1 mm versus 53.4 mm) while retaining a higher median error (21.5 mm versus 15.0 mm). The pigeon-trained silhouette model also produces 2D keypoints on pigeons, stock doves, turtle doves, and ravens without fine-tuning, with best results when the new species' pose and viewing angle resemble the training data.","pith_inferences":["The paper's logic suggests that synthetic silhouette data, such as rendered bird shapes with random poses, could be used to train the keypoint network and remove the annotation bottleneck entirely.","A testable extension: triangulating all four left/right keypoint combinations and keeping the one with the smallest reprojection error should resolve the left-right ambiguity the paper identifies in silhouette data.","Because the authors find mask quality is the main bottleneck, adding internal contour cues to the silhouette might push accuracy beyond the texture-based baseline rather than merely matching it.","The species-transfer results imply the real barrier to new species is pose and viewpoint distribution shift, not feather appearance, so viewpoint augmentation or a few annotated frames of the new species may close most of the gap."],"forward_implications":["Silhouette-only input is enough to keep 2D and 3D pose estimation and identity tracking working for up to 10 pigeons across four camera views.","Because the input is a mask, the pipeline is insensitive to plumage color, lighting, and background changes that leave the silhouette unchanged.","A model trained only on pigeon silhouettes can generate usable 2D keypoints on other bird species without additional labeled data.","On the outdoor Wild-MuPPET dataset, the silhouette variant with segmentation-model masks reports lower average 3D error than the texture-dependent baseline, indicating fewer large outliers."],"supporting_citations":[{"why":"Defines the 3D-MuPPET framework this paper extends and supplies the texture-dependent baseline accuracy that must be matched.","marker":"[27]"},{"why":"SAM is the segmentation model that generates the silhouettes from bounding-box prompts.","marker":"[14]"},{"why":"DeepLabCut is the keypoint network that is retrained on silhouettes to predict 2D joint positions.","marker":"[18]"},{"why":"YOLOv8 provides the bird detector whose bounding boxes prompt the segmentation model and whose segmentation variant produces masks in the outdoor experiments.","marker":"[25]"},{"why":"3D-POP is the multi-view pigeon dataset used for training and evaluating the 2D and 3D pose estimates.","marker":"[20]"},{"why":"Animal Kingdom supplies the four bird species used to test cross-species transfer without fine-tuning.","marker":"[21]"},{"why":"Supplies the dynamic matching method that assigns global identities across camera views in the first frame.","marker":"[10]"}],"fun_headline_variants":["Silhouette-only 3D bird pose matches textured baseline","Texture-free keypoint pipeline tracks up to 10 pigeons in 3D","Pigeon-trained silhouette keypoints generalize to four bird species","Texture-independent 3D pose for birds from silhouettes only","Binary masks replace textures in multi-bird 3D keypoint tracking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The single largest connected component of the segmentation mask is assumed to be the complete bird, so whenever birds overlap, occlude one another, or split into fragments, the silhouette no longer contains enough information to localize all nine keypoints.","fun_headline_variants_meta":{"raw":{"variants":["Silhouette-only 3D bird pose matches textured baseline","Texture-free keypoint pipeline tracks up to 10 pigeons in 3D","Pigeon-trained silhouette keypoints generalize to four bird species","Texture-independent 3D pose for birds from silhouettes only","Binary masks replace textures in multi-bird 3D keypoint tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000692,"raw_usage":{"total_tokens":3127,"prompt_tokens":934,"completion_tokens":2193,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":2102}},"tokens_in":550,"tokens_out":2193,"duration_ms":15521,"temperature":1.0,"reasoning_tokens":2102,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:56:57.320174+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take frames in which two pigeons occlude each other so their masks merge into one connected component, and compare per-keypoint accuracy on those frames against accuracy on frames with isolated birds; if median keypoint error jumps and the fraction of keypoints within the strict accuracy threshold collapses specifically in merged-mask frames while the textured-input baseline does not, the assumption that the largest component equals the bird is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the 3D-MuPPET framework this paper extends and supplies the texture-dependent baseline accuracy that must be matched."},{"cited_title":"Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick","cited_arxiv_id":null,"evidence_quote":"SAM is the segmentation model that generates the silhouettes from bounding-box prompts."},{"cited_title":"Cury, Taiga Abe, Venkatesh N","cited_arxiv_id":null,"evidence_quote":"DeepLabCut is the keypoint network that is retrained on silhouettes to predict 2D joint positions."},{"cited_title":"YOLOv8: A novel object detection algorithm with enhanced performance and robust- ness","cited_arxiv_id":null,"evidence_quote":"YOLOv8 provides the bird detector whose bounding boxes prompt the segmentation model and whose segmentation variant produces masks in the outdoor experiments."},{"cited_title":"Couzin, Fumihiro Kano, and M ´at´e Nagy","cited_arxiv_id":null,"evidence_quote":"3D-POP is the multi-view pigeon dataset used for training and evaluating the 2D and 3D pose estimates."},{"cited_title":"Animal kingdom: A large and diverse dataset for animal behavior understanding","cited_arxiv_id":null,"evidence_quote":"Animal Kingdom supplies the four bird species used to test cross-species transfer without fine-tuning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the dynamic matching method that assigns global identities across camera views in the first frame."}],"review_version":1}