{"id":"2f8dbb3f-45c2-4285-9da1-8cbf0c0b5548","arxiv_id":"2412.09599","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A new multi-camera dataset and a transformer-based method reconstruct a dense 3D rat body surface from 10 sparse keypoints, with reported mean errors around 5 to 7 mm.","lead":"This paper introduces RatDome, a 15-camera dome for capturing rats with temporary colored markers, and RatBodyFormer, a transformer that predicts a dense 3D rat body surface from 10 sparse keypoints. The work is a first step toward automated rat behavior analysis at the body-surface level, beyond the sparse pose estimates of tools like DeepLabCut.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"D3 generalization numbers are partly circular: shape-from-silhouette ground truth is built from the same masks that drive the test-time silhouette loss, so the reported 1.8–2.7 mm errors may only measure alignment with the visual hull.","rationale":"The reader's weakest assumption—that the keypoint-to-surface mapping is treated as deterministic despite being underdetermined in complex interactions—is a real limitation and is explicitly acknowledged in Sec. 6. However, the more load-bearing problem is that the D3 experiment, which provides the main evidence for generalization to markerless rats, uses a ground-truth protocol that is partly circular: shape-from-silhouette points are added to the ground truth, and the same silhouettes are used as a test-time supervision signal through the silhouette loss Ls. This can directly bias the reported D3 errors downward, especially because the error metric appears to be a nearest-neighbor distance to a dense point cloud. Therefore, while the method may still work, the empirical support for the central claim is weaker than the paper suggests. Since the reader's verdict is already CONDITIONAL, my concern reinforces that verdict rather than changing it. I would keep the paper conditional pending an independent evaluation of D3 accuracy, such as a leave-one-rat-out bead-based test or a LiDAR-only evaluation. The paper deserves credit for the RatDome dataset, the marker-based D1/D2 evaluations, and the ablation studies, including the error-propagation analysis in Appendix F; but the D3 numbers should not be used as evidence of markerless generalization until the circularity is resolved.","tokens_in":18458,"tokens_out":5545,"duration_ms":63153,"concrete_test":"Re-run the D3 protocol for the 7/9/11-week-old beaded rats in a leave-one-rat-out split, but compute errors only at the manually annotated bead positions, using the same test-time silhouette optimization. If the bead-position errors are closer to the D1/D2 range of 5–7 mm than to the reported 1.8–2.7 mm, the D3 generalization claim is an artifact of the silhouette-based ground truth. As a secondary check, recompute D3 errors using only filtered LiDAR points, excluding all shape-from-silhouette points, and compare the resulting errors with the published numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that RatBodyFormer reconstructs a dense rat body surface from sparse keypoints rests heavily on the D3 experiment, which is meant to show generalization to markerless rats. In Sec. 4.2, the ground-truth surface points are defined as the sum of LiDAR points and shape-from-silhouette points, and the same SAM-derived mask images are used both to build the shape-from-silhouette ground truth and to compute the silhouette loss Ls that optimizes the individual-dependent parameters C and T at inference time. Thus the evaluation target is partly generated by the same supervision signal used to fit the model to the test rat: the silhouette loss pushes predicted points inside the masks, while the visual-hull ground truth lies exactly on the boundary of those masks. This creates a leakage path that can inflate the reported D3 accuracy. The D3 errors (1.8–2.7 mm) are also much smaller than the marker-based D1/D2 errors (5–7 mm), which is suspicious because D3 is the harder generalization setting. Without an independent surface measurement, D3 does not establish accurate keypoint-only reconstruction for markerless rats; it may only demonstrate alignment with the visual hull. This concern is distinct from, and more immediately damaging than, the underdetermination limitation acknowledged in Sec. 6.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RatDome, a 15-camera multiview dome for capturing freely moving rats wearing colored beads, and RatBodyFormer, a transformer encoder-decoder that maps 10 3D keypoints (nose, eyes, ears, front/back paws, tail base) to a dense set of 3D body-surface points. Training pairs are obtained by triangulating bead and paint markers and mapping them to a canonical surface, with a semi-automatic annotation loop to enlarge the dataset. Experiments on single-rat (D1), two-rat (D2), and markerless all-rat (D3) settings report mean L2 errors of roughly 5.3–7.3 mm on the marker-based splits and 1.8–2.7 mm on D3, together with a large improvement over a MAMMAL-based baseline and an application to an animatable Gaussian splatting model called GaussianRat.","tokens_in":18742,"tokens_out":6298,"duration_ms":62317,"significance":"If the evaluation issues are resolved, this would be a useful first step toward dense surface reconstruction from sparse keypoints for rats, with a novel captured dataset and a network design that explicitly handles inconsistent marker annotations across individuals. The paper has several strengths: the RatDome capture rig and dataset are new assets; the canonical-surface formulation plus point-wise scaling/translation parameters is a sensible way to deal with cross-subject annotations; the ablation studies cover the position consistency loss, the individual-dependent parameters, and data normalization; and Appendix F shows robustness to injected keypoint noise. The main limitations are that the markerless-generalization experiment has a potential leakage between the silhouette loss and the shape-from-silhouette ground truth, and the practical claim of running from 'detected keypoints' is not tested with an actual keypoint detector. These issues currently temper the strength of the central claims but are addressable in a revision.","major_comments":[{"comment":"In D3, the ground-truth surface points are defined as the sum of LiDAR points and shape-from-silhouette points, and the same SAM-derived mask images are used both to construct the shape-from-silhouette component of that ground truth and to compute the silhouette loss Ls that optimizes the individual-dependent parameters C and T at inference time. Because the shape-from-silhouette target lies on the visual hull of these masks, optimizing Ls can improve agreement with that target for reasons that are not independent of the evaluation metric; this may explain why the reported D3 errors (1.80–2.69 mm) are much smaller than the D1/D2 marker-based errors (5.3–7.3 mm) even though D3 is the harder generalization setting. Please re-evaluate D3 against an independent surface measurement (e.g., marker triangulation on at least a subset of frames), or report errors separately for LiDAR-only and shape-from-silhouette-only points and demonstrate that the reported gains are not an artifact of the shared masks.","section":"Sec. 4.2 (D3)"},{"comment":"All quantitative results, including the claimed end-to-end use with 'detected keypoints', are obtained from ground-truth or marker-derived keypoints. Appendix F adds synthetic Gaussian noise to keypoint coordinates but does not run an actual keypoint detector; real detector errors are spatially structured and can differ from isotropic Gaussian noise. Since the abstract and introduction motivate the method by compatibility with past keypoint detectors, please evaluate the full pipeline on videos with an actual 3D keypoint detector (e.g., DANNCE or a DeepLabCut-based detector) and report surface errors conditioned on those detector outputs, or explicitly restrict the claims to keypoint coordinates supplied by other means.","section":"Sec. 4.2 and Appendix F"}],"minor_comments":[{"comment":"The abstract says RatBodyFormer is 'trained with masked-learning', but Sec. 3.2 does not describe any masking procedure; either add the missing definition or remove the term.","section":"Abstract / Sec. 3.2"},{"comment":"The caption contains the typo 'Transfomer'; it should read 'Transformer'.","section":"Fig. 1 caption"},{"comment":"Equation (1) uses both α′ and a′ for the barycentric coordinates; the notation should be unified.","section":"Appendix B, Eq. (1)"},{"comment":"The paper states that refraction by the acrylic tube is not modeled in the camera calibration; since all marker triangulations pass through this tube, a brief discussion of the expected magnitude of the resulting bias would help readers interpret the reported errors.","section":"Sec. 4.1"},{"comment":"The conclusion appropriately acknowledges that a single keypoint set may not uniquely determine a body surface in complex interactions; this scope limitation should also be stated in the abstract or introduction so that the headline claim is not broader than the experiments support.","section":"Sec. 6"},{"comment":"The MAMMAL comparison scales a mouse mesh to rat scale but does not retrain or fine-tune MAMMAL on rat data; a brief note on this limitation would help readers interpret the large accuracy gap.","section":"Sec. 4.2, MAMMAL comparison"},{"comment":"The paper does not state whether the RatDome dataset, trained models, or evaluation code will be released; please add a data-availability statement.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The D3 experiment is the main weakness: the shared use of SAM masks in both the ground-truth construction and the inference-time silhouette loss undermines the markerless-generalization claim as currently presented. The underlying dataset and model are potentially valuable, and the D1/D2 experiments provide a sounder basis for the core keypoint-to-surface mapping, so I see this as fixable via an independent D3 evaluation and an end-to-end keypoint-detector experiment rather than as a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the genuinely new asset here is RatDome, the first dataset pairing 3D rat keypoints with dense body-surface points, and a solid transformer mapping from 10 sparse keypoints to a dense surface that handles inconsistent marker layouts across rats. On the marker-based experiments D1 and D2 the evidence looks straight: mean L2 errors of 5-7 mm against manual marker triangulation, around one bead diameter, semi-automatic labels give a consistent ~1 mm gain, and the keypoint-noise test in the appendix shows the mapping degrades gently. Beating MAMMAL 6.5 vs 20.2 mm is real but weak evidence, since MAMMAL is a mouse-mesh model scaled to a rat and not designed for this task.\n\nThe soft spots, in order of real weight.\n\nOne: the abstract's 'trained with masked-learning' has no backing. I read Section 3.2 and the appendices; there is no masking anywhere in the training procedure. The semi-automatic annotation is self-training, not masked learning. That sentence should be corrected.\n\nTwo: D3 is the weak link. Markerless generalization is measured against ground truth made of LiDAR plus shape-from-silhouette, and the same SAM masks that build the visual hull feed the silhouette loss used to optimize the per-rat scale and translation at test time. The evaluation target is not independent of the inference-time supervision. The 1.8-2.7 mm numbers, well below the marker-based errors, likely reflect alignment with the visual hull plus dorsal-only scoring rather than true surface accuracy. I would want D3 re-run against an independent surface measurement, such as markers on one held-out rat or LiDAR-only regions where the depth is clean, before accepting the markerless claim.\n\nThree: all evaluation uses ground-truth keypoints, not a real detector. The noise simulation is a partial mitigation, but end-to-end with DeepLabCut or similar would be more convincing. And the dataset is three beaded rats, 33-66 markers each, with no code or data released. That limits how much anyone else can verify.\n\nThe underdetermination point the authors raise in the conclusion is real and they state it themselves; on the captured poses the deterministic mapping fits fine. The core mapping is not circular: it is trained on measured pairs and tested on held-out frames.\n\nBottom line: this is a useful, citable contribution for the marker-based setting, the dataset is valuable if it ships, and the D1/D2 results are credible. The D3 claim and the abstract need work. It deserves a serious referee; I would send it out expecting a substantive revision.","headline":"First dense rat body-surface dataset plus a workable keypoint-to-surface transformer; the marker-based results are honest, but the abstract overclaims masked learning and D3's markerless numbers lean on silhouettes that also supervise the test-time fit.","tokens_in":19301,"tokens_out":7180,"would_cite":true,"duration_ms":69099,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Ten sparse keypoints can be turned into a dense 3D rat body surface.","keywords":["rat behavior analysis","3D body surface reconstruction","keypoint-to-surface regression","multiview capture system","transformer encoder-decoder","non-rigid deformation","animal pose estimation","semi-automatic annotation"],"falsifier":"Record two different behaviors, such as curling versus stretching, in which the ten 3D keypoints are at nearly identical positions but the marker-covered body surfaces differ by more than the reported mean error; if those frames are held out, the deterministic keypoint-to-surface mapping will mispredict one of the two surfaces and the error will exceed the roughly 6.5 mm average.","tokens_in":18269,"feed_emoji":"🐀","tokens_out":9056,"duration_ms":80434,"temperature":0.7,"pith_summary":"RatBodyFormer is a method for turning the ten 3D keypoints that standard detectors already find on a rat—nose, eyes, ears, paws, tail base—into a dense cloud of body-surface points. The paper argues that the rat's featureless, highly non-rigid fur coat, which keypoint detectors cannot latch onto, can be modeled as a learned deformation of a canonical reference surface. To learn that deformation it introduces the RatDome capture system and dataset, in which colored beads and paint temporarily provide trackable surface features during training only. On real captures the method achieves mean $L^2$ error around 6.5 mm, close to the bead diameter, and outperforms a prior whole-body mesh-fitting approach by a large margin. The authors present this as the first foundation for dense, marker-free body-surface reconstruction of rats for automated behavior analysis.","feed_headline":"Ten keypoints yield a dense 3D rat body surface","feed_subtitle":"Sparse pose alone predicts the fur-covered body, with mean error close to the training bead diameter.","key_machinery":"The central object is a canonical 3D body surface: a reference standing-on-two-feet surface onto which all bead and paint marker positions are mapped with as-rigid-as-possible deformation. RatBodyFormer is a transformer encoder-decoder whose encoder tokens are the normalized displacement of each detected keypoint from its canonical position, and whose decoder queries are the normalized displacements of body-surface points from an analytically deformed initial guess; the decoder outputs corrected surface-point displacements. The surface points are therefore not indexed by fixed vertex identity, which lets one training set absorb differently-beaded rats. Per-rat scale and translation parameters are refined during training and at inference time using silhouette and position-consistency losses, and the model is trained with masked-learning plus semi-automatically annotated frames generated by the model itself.","core_discovery":"The paper claims that dense 3D body-surface reconstruction of a freely moving rat is achievable from sparse detected keypoints alone, without any image or silhouette input at inference time. Its core result is that a transformer encoder-decoder, trained on paired keypoint and surface-point data collected from marker-attached rats, can regress body-surface point displacements relative to a canonical surface with mean $L^2$ error of roughly 6.5 mm, comparable to the diameter of the training markers, and generalizes to unseen rats aged 5 to 14 weeks. The same architecture also supports a keypoint-driven animatable rat model. The paper positions this as the first method of its kind for rats, with accuracy well above the preceding mouse-mesh-based baseline.","pith_inferences":["Editorial: Conditioning the same architecture on images or silhouettes in addition to keypoints is the natural next step; the paper itself notes that keypoints alone underdetermine the surface in complex interactions, so visual cues should resolve that ambiguity.","Editorial: Because the training markers are 5.5 mm beads and reported errors are of the same scale, any behavior expressed at smaller spatial scales, such as fur ripples or piloerection, is likely invisible to this representation.","Editorial: The canonical-surface displacement formulation should transfer to other laboratory animals, such as mice or guinea pigs, provided a small set of shared keypoints and a reference pose can be defined.","Editorial: A testable extension is to output a distribution or set of candidate surfaces per keypoint configuration; comparing that multi-modal output on identical-keypoint, different-surface frames would quantify the remaining ambiguity."],"forward_implications":["A rat's dense body surface, not just its skeleton, becomes a measurable behavioral signal, so curling, stretching, and subtle torso changes can be quantified.","Because the surface is predicted from the same ten keypoints that existing detectors output, the method can be applied to keypoint-only recordings without any new markers.","The animatable surface model can synthesize novel views and poses, enabling analysis-by-synthesis and virtual-stimulus experiments.","The capture protocol generalizes across rat ages and body shapes, covering the 5-to-14-week age range common in biomedical experiments.","Semi-automatic annotation reduces the manual labeling burden for building paired keypoint-surface datasets for other featureless animals."],"supporting_citations":[{"why":"Established sparse keypoint detection pipeline whose outputs are the model's input.","marker":"[1]"},{"why":"Prior whole-body surface tracking method that serves as the main accuracy baseline.","marker":"[2]"},{"why":"Supplies the scanned mouse mesh used by the baseline, which the paper argues does not represent an awake rat.","marker":"[8]"},{"why":"Massively multiview capture design that the RatDome system adapts for rodent-scale capture.","marker":"[28]"},{"why":"As-rigid-as-possible deformation used to align all marker sets to the canonical surface and to compute the initial surface guess.","marker":"[55]"},{"why":"Transformer encoder-decoder architecture on which RatBodyFormer is built.","marker":"[56]"},{"why":"Object detector used to locate colored markers per view for semi-automatic annotation.","marker":"[16]"},{"why":"Assignment algorithm that matches triangulated marker points to model-estimated surface points for label transfer.","marker":"[35]"},{"why":"Differentiable silhouette renderer used in the silhouette loss for surface supervision.","marker":"[21]"},{"why":"Segmentation model used to obtain rat masks when evaluating on unseen rats without markers.","marker":"[33]"}],"fun_headline_variants":["Ten keypoints to dense rat body surface","RatBodyFormer: sparse keypoints to full 3D rat body","First dense rat body surface from keypoints only","From 10 keypoints to a rat's full body surface"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the body surface is a single deterministic function of the ten keypoint positions, so any pose in which two genuinely different body-surface shapes share the same keypoint configuration is outside what the trained network can represent.","fun_headline_variants_meta":{"raw":{"variants":["Ten keypoints to dense rat body surface","RatBodyFormer: sparse keypoints to full 3D rat body","First dense rat body surface from keypoints only","From 10 keypoints to a rat's full body surface"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000817,"raw_usage":{"total_tokens":3562,"prompt_tokens":913,"completion_tokens":2649,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":2583}},"tokens_in":529,"tokens_out":2649,"duration_ms":19067,"temperature":1.0,"reasoning_tokens":2583,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:52:32.878190+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record two different behaviors, such as curling versus stretching, in which the ten 3D keypoints are at nearly identical positions but the marker-covered body surfaces differ by more than the reported mean error; if those frames are held out, the deterministic keypoint-to-surface mapping will mispredict one of the two surfaces and the error will exceed the roughly 6.5 mm average.","supporting_citations":[{"cited_title":"Cury, Taiga Abe, Venkatesh N","cited_arxiv_id":null,"evidence_quote":"Established sparse keypoint detection pipeline whose outputs are the model's input."},{"cited_title":"Three-dimensional surface motion capture of multiple freely moving pigs using MAMMAL","cited_arxiv_id":null,"evidence_quote":"Prior whole-body surface tracking method that serves as the main accuracy baseline."},{"cited_title":"Bola ˜nos, Dongsheng Xiao, Nancy L","cited_arxiv_id":null,"evidence_quote":"Supplies the scanned mouse mesh used by the baseline, which the paper argues does not represent an awake rat."},{"cited_title":"Panoptic Studio: A Massively Multiview System for Social Motion Capture","cited_arxiv_id":null,"evidence_quote":"Massively multiview capture design that the RatDome system adapts for rodent-scale capture."},{"cited_title":"As-Rigid-As-Possible Sur- face Modeling","cited_arxiv_id":null,"evidence_quote":"As-rigid-as-possible deformation used to align all marker sets to the canonical surface and to compute the initial surface guess."},{"cited_title":"Attention is All you Need","cited_arxiv_id":null,"evidence_quote":"Transformer encoder-decoder architecture on which RatBodyFormer is built."},{"cited_title":"The Hungarian method for the assignment problem","cited_arxiv_id":null,"evidence_quote":"Assignment algorithm that matches triangulated marker points to model-estimated surface points for label transfer."},{"cited_title":"DRWR: A differentiable renderer without render- ing for unsupervised 3D structure learning from silhouette images","cited_arxiv_id":null,"evidence_quote":"Differentiable silhouette renderer used in the silhouette loss for surface supervision."},{"cited_title":"Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick","cited_arxiv_id":null,"evidence_quote":"Segmentation model used to obtain rat masks when evaluating on unseen rats without markers."}],"review_version":1}