{"id":"eae86e96-ff18-4d14-a419-3fd1ad80b28d","arxiv_id":"2507.20217","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A humanoid-specific multimodal occupancy perception system with a new dataset, sensor layout, and a fusion network that claims state-of-the-art results on its own benchmark.","lead":"This paper presents a complete multimodal occupancy perception system for humanoid robots, including a six-camera plus LiDAR sensor layout, a wearable data collection rig, and a dedicated annotation pipeline. It also introduces what it claims is the first panoramic occupancy dataset for humanoid robots and a baseline network that fuses camera and LiDAR features.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark's occupancy ground truth is only a voxelized LiDAR-visible surface shell (Poisson reconstruction explicitly skipped, Sec. 3.3), so the reported mIoU/rayIoU may certify visibility fitting rather than geometric occupancy understanding.","rationale":"I read the paper in good faith: the system is a competent integration of established components and the dataset collection effort is real. The reader's weakest assumption is exactly the load-bearing issue: if the occupancy label is only an uncompleted LiDAR surface shell, then the central dataset/benchmark contribution is not validated. I agree with that identification. The separate Table 2 versus Table 3 inconsistency (52.79/60.49 versus 47.92/53.97 for the same raw-distort configuration) is a real but fixable reporting problem; the ground-truth definition is more fundamental because it determines what the benchmark measures even after all numbers are cleaned up. I therefore do not propose moving the verdict away from CONDITIONAL; I would make the concrete ground-truth comparison test an explicit condition of acceptance. No claim of bad faith is intended; the missing validation could be supplied by the authors.","tokens_in":15220,"tokens_out":9352,"duration_ms":124873,"concrete_test":"Take a held-out subset of the 20 validation clips and build two occupancy ground truths from the same annotated point clouds: (i) the paper's direct voxelization, and (ii) voxelization after TSDF or Poisson surface completion with a fixed truncation/watertight threshold. Train or fine-tune the HumaniodOcc baseline with identical settings on each ground truth and compare the metrics in Tables 2 and 4. In addition, compute the same mIoU/rayIoU for a no-learning baseline that simply voxelizes the current frame's LiDAR points. If the completed-ground-truth numbers drop by more than a few mIoU points, or if the LiDAR-visibility baseline lands close to the published 52.79/60.49, the current benchmark is measuring surface-visibility recovery rather than occupancy understanding. This check decides whether the 'first panoramic occupancy dataset' claim is valid as a benchmark.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Humanoid Occupancy is a benchmark-quality multimodal occupancy perception system requires the ground-truth volumes to define occupancy, not merely observed surfaces. In Sec. 3.3 and Fig. 5, the annotation pipeline superimposes multi-frame static point clouds, stitches dynamic-object bounding-box points, and directly voxelizes the merged cloud; the text explicitly states that Poisson reconstruction is not performed on the superimposed and stitched point cloud. Consequently, every labeled occupied voxel is one that happened to be hit by a 40-line LiDAR ray after stacking, and unobserved space receives no label. There is no separation of free space from unknown space and no completion of surfaces behind occluders. The mIoU and rayIoU metrics defined in Sec. 4.1 are then computed against this shell. A model that learns to aggregate LiDAR visibility into the same scan pattern can score well without any scene-completion or semantic-geometry understanding, while a camera-only model is penalized for predicting plausible occluded geometry. The introduction's claim that occupancy captures not only the 2D distribution but also detailed structural and semantic attributes along the vertical dimension is therefore not actually tested by the benchmark. Because the dataset is the paper's main novel contribution, this ground-truth definition is the load-bearing validity assumption; no unobserved-volume statistics, no comparison against a completed TSDF/Poisson label, and no ground-truth release are provided to check it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Humanoid Occupancy, a multimodal occupancy perception system for humanoid robots. It describes a sensor layout for the Tienkung humanoid robot (6 cameras and a 40-line LiDAR), a wearable data-collection device to avoid the cost of robot-based collection, an annotation pipeline that produces voxelized semantic occupancy labels, and a fusion network called HumaniodOcc. The network combines PointPillar-based LiDAR BEV features, camera CNN features, deformable cross-attention fusion, temporal feature warping, and a FlashOcc-style prediction head. The authors collected a dataset of 180 training and 20 validation clips with three scene types and evaluate against BEVDet, FBOcc, and BEVFusion, with ablations on distortion handling, temporal frames, and modality inputs. The central claims are that this is the first panoramic occupancy dataset specifically for humanoid robots and that the proposed network achieves superior metrics with fewer parameters.","tokens_in":15362,"tokens_out":4754,"duration_ms":56545,"significance":"If the dataset and benchmark are valid, the work would address a real gap: there is no established humanoid-specific occupancy benchmark, and the integrated hardware-software system could be a useful template for future robot perception platforms. The lightweight network design and the explicit treatment of sensor placement and kinematic interference are practical contributions. The paper also creditably includes a temporal fusion ablation and a distortion-aware projection strategy. However, the benchmark validity rests on the ground-truth definition, and the current evaluation does not establish that the voxel labels represent occupancy rather than LiDAR-visible surfaces. The dataset is not released, no code is provided, and all results are single-run, so the 'generalized' and 'standardized benchmark' claims are not yet supported.","major_comments":[{"comment":"The occupancy ground truth is generated by directly voxelizing the superimposed static and stitched dynamic point clouds, and the text explicitly states that Poisson reconstruction is not performed. As a result, every labeled occupied voxel is a point hit by a LiDAR ray after multi-frame accumulation, with no separation of free space from unknown space and no completion of surfaces behind occluders. The mIoU and rayIoU metrics in Tables 2-5 therefore measure how well a model reproduces the LiDAR-visible surface shell, not geometric occupancy or scene completion. This is load-bearing for the paper's claim that the representation captures vertical structure and semantic attributes, and for the dataset's value as a benchmark. Please provide statistics on unobserved volume, compare against a completion-based label such as TSDF or Poisson reconstruction, or both, and adjust the claims accordingly.","section":"§3.3, Fig. 5"},{"comment":"The claim that the model 'achieves superior metrics' and the Fig. 7 statement that it 'significantly outperforms other methods in both mIoU and ray IoU' are contradicted by the 1-frame row: BEVFusion reports mIoU 53.98 versus HumaniodOcc 52.79, while rayIoU is essentially tied (60.24 vs 60.49). The advantage appears only in the 2-frame setting (55.73 vs 53.21). Please report per-configuration comparisons and correct the overclaim; with a single run and no error bars, 'significantly outperforms' is not supported.","section":"§4.2, Table 2 and Fig. 7"},{"comment":"All results come from a single training run on a self-collected dataset that is not released and is compared only against baselines trained in-house on the same data. No standard deviations, no multiple seeds, and no external benchmark are provided. Since the dataset is the paper's main novel contribution, this protocol is insufficient to support the 'standardized benchmark' and 'generalized' claims. At minimum, report variance across several seeds, describe the planned dataset release, and provide an external reference point such as evaluation on an established occupancy benchmark.","section":"§4.1–§4.2"},{"comment":"The distortion ablation reports mIoU 47.92 for the final distortion-aware projection method, whereas Table 2 reports 52.79 for the same 1-frame model with raw pinhole projection. The note 'for this comparison we use data with precisely calibrated projection alignment to reveal the distinct characteristics of each distortion handling method' does not specify what differs in data, alignment, or training. Without that clarification, the ablation cannot be compared with the main results, and the conclusion that the proposed method 'ensures that the performance is not affected' is not established.","section":"Table 3"}],"minor_comments":[{"comment":"There are frequent typos and inconsistent spellings, including 'HumaniodOcc' (should be 'HumanoidOcc'), 'evalidation' (§4.1), 'occulsion' and 'occlusion reasoning' (§4.3), 'A ablation' (Table 4), and 'the different number of temporal frames' (Table 4). The manuscript needs a careful proofreading pass.","section":"Throughout"},{"comment":"The wearable collection device is worn by humans, but no calibration or validation is reported to show that the wearable device's extrinsic calibration matches the robot-mounted configuration. Please specify how the transfer from human collector to robot is realized and what calibration checks are performed.","section":"§3.2"},{"comment":"The rayIoU metric is said to follow SparseOcc (Liu et al., 2023), but the cited reference [28] is 'Fully sparse 3d panoptic occupancy prediction.' Please clarify the exact ray sampling procedure and the correct citation.","section":"§4.1"},{"comment":"The semantic category sets differ across scenes (Table 1), yet Table 2 reports per-class numbers for a single pooled class list. Please specify how classes are aligned across scenes and how the per-class metrics are aggregated.","section":"§4.1, Table 2"},{"comment":"Several references are incomplete or malformed, such as [1], [2], and [30], and some citations have trailing commas (e.g., 'Sun et al., 2025,'). Please normalize the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper is a systems-and-benchmark contribution, but the dataset and code are not released and the benchmark's ground-truth validity is the central unresolved issue. The authors should be asked to either provide completion-based label validation or substantially temper the benchmark claims. The experimental protocol also needs multi-seed results before 'significantly outperforms' can be taken seriously."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading if you care about humanoid perception benchmarks. The new piece is the data package: a humanoid-head sensor layout (6 cameras + 40-line LiDAR), a wearable collection rig with a neck stabilizer, and what looks like the first panoramic occupancy dataset for humanoid robots. The network itself is a competent integration of known parts—PointPillar, ResNet-FPN, DeepFusion-style cross-attention, BEVDet4D temporal fusion, FlashOcc head—plus one small original detail: using distorted pinhole projection when sampling camera features, which avoids undistortion latency. That part is reasonable.\n\nThe main soft spot is the ground truth. The paper states outright that Poisson reconstruction is not performed on the merged point cloud before voxelization. So the occupancy labels are effectively a voxelized LiDAR-visible surface shell. The stress-test concern lands: mIoU and rayIoU computed against that shell reward models that reproduce the LiDAR visibility pattern, not models that understand occupied versus unoccupied space. A camera-only model is penalized for predicting plausible geometry behind occluders. This is a load-bearing validity issue for the benchmark, not a cosmetic one.\n\nOther soft spots are real but fixable. Table 2 contradicts the \"significantly outperforms\" claim in the 1-frame setting: BEVFusion gets 53.98 mIoU against 52.79 for HumaniodOcc. Table 3's distortion ablation reports 47.92 mIoU for the same \"raw distort\" method that Table 2 lists at 52.79 in the 1-frame row; no explanation is given. The dataset is not released, there are no error bars, and the conclusion's navigation claims are not backed by navigation experiments. The paper also skips comparisons against the closest related works, including the authors' own EmbodiedOcc, RoboOcc, and HumanoidPano.\n\nCredit where due: the paper is honest enough to mention the no-Poisson choice and the pose-error degradation with more temporal frames. The sensor layout discussion is practical, and the system integration is clean. But as a benchmark contribution it currently cannot be validated by anyone else.\n\nBottom line: this is a conditional dataset/benchmark paper, not a finished system paper. I would send it to peer review, because the dataset and sensor layout could matter to the humanoid perception community, but I would ask for major revision: release the data, separate free space from unknown space or compare against a completed occupancy label, fix the inconsistent tables, temper the overclaims, and add repeat runs. Without that, the benchmark numbers certify visibility fitting, not geometric understanding.","headline":"A potentially useful humanoid occupancy benchmark is undermined by a LiDAR-visibility ground truth and inconsistent numbers; the dataset release and a GT rework should decide its fate.","tokens_in":817,"tokens_out":914,"would_cite":false,"duration_ms":40422,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a voxel-based semantic occupancy grid, fused from six cameras and one LiDAR, can serve as a generalized perception module for humanoid robots, and it supplies the dataset, annotation pipeline, and lightweight…","keywords":["humanoid robots","3D semantic occupancy","multimodal perception","LiDAR-camera fusion","occupancy dataset","bird's-eye view","temporal fusion","sensor layout"],"falsifier":"Pick a held-out scene from the dataset and construct a dense completed reference that fills occluded space, such as a mesh or a second full-coverage scan; if the paper's voxel ground truth labels those filled regions as empty, then the reported mIoU and rayIoU measure how well models reproduce the 40-line LiDAR's visibility pattern, not true occupancy.","tokens_in":14882,"feed_emoji":"🤖","tokens_out":9379,"duration_ms":104444,"temperature":0.7,"pith_summary":"This paper is trying to establish that 3D semantic occupancy, previously a dominant perception representation in autonomous driving, can be transplanted to humanoid robots as a generalized perception module. It argues that the obstacles are engineering problems: a sensor layout that survives the robot's kinematics, a data collection and annotation scheme that yields dense voxel ground truth, and a lightweight multimodal fusion network. If the paper is right, the field gains the first panoramic occupancy dataset built specifically for humanoid robots, plus a benchmark and baseline that future work can compare against. The practical payoff for a general reader is that occupancy gives a humanoid robot the dense 3D geometry and semantics it needs for foot placement, manipulation, and path planning in human-scale spaces, not just the flat ground-plane view of bird's-eye representations.","feed_headline":"Humanoid robots get their own 3D occupancy perception system","feed_subtitle":"A six-camera LiDAR head, a wearable data rig, and a lightweight fusion network anchor the first humanoid occupancy benchmark.","key_machinery":"The machinery is the occupancy ground-truth generation pipeline plus the bird's-eye-view (BEV) fusion network. Static background points from many frames are motion-aligned and annotated point-wise; dynamic objects are labeled with 3D bounding boxes, with extra point-level annotation for special-posture pedestrians, and stitched into each frame; the merged cloud is voxelized into a 200 by 200 by 24 grid at 0.1 meter resolution, with Poisson reconstruction explicitly skipped. The network builds LiDAR BEV features from pillars, extracts image features from the six cameras with a shared convolutional backbone, fuses the two through deformable cross-attention keyed from LiDAR queries, warps in one historical BEV feature aligned by ego-motion, and reshapes fused BEV features into 3D voxels through a channel-to-height head. Training combines focal, lovasz-softmax, and scene- and class-scaled affinity losses, with temporal information introduced after the first epoch through a continuous-frame training schedule.","core_discovery":"The paper's central claim is that occupancy grids, in which each voxel carries both an occupied-or-free state and a semantic class, can serve as a unified environmental perception output for humanoid robots, and that a complete system requires solving sensor layout, data collection, annotation, and fusion together. It reports a full hardware and software stack: six cameras plus a 40-line 360-degree LiDAR arranged on the head to reduce kinematic occlusion, a wearable collection harness that matches the sensor height of the humanoid, and an annotation pipeline that accumulates multi-frame static points and stitches bounding-box-annotated dynamic points into one point cloud, which is then directly voxelized at 0.1 meter resolution into a 200 by 200 by 24 grid without volumetric completion. The network, named HumanoidOcc, fuses LiDAR and camera features through deformable cross-attention, warps in one historical bird's-eye-view feature frame, and reshapes the fused features into a 3D occupancy prediction. On the new dataset the two-frame model reports 55.73 mIoU and 61.32 rayIoU with 40.5 million parameters, higher than the camera-only and LiDAR-only variants and higher than the bird's-eye-view baselines it is compared against.","pith_inferences":["Editorial inference: the wearable data-collection harness could be reused on other robot morphologies by matching sensor height and adding neck stabilization, letting the dataset grow without operating a humanoid in every scene.","Editorial inference: because the ground truth is defined by visible laser points, the reported mIoU and rayIoU likely reward models that reproduce the 40-line LiDAR's visibility pattern; a benchmark with explicit unknown or occluded labels would measure true geometric understanding more directly.","Editorial inference: the finding that the optimal temporal window is exactly two frames points to pose-error accumulation as the binding constraint, so better odometry or learned alignment, rather than more historical frames, is the lever that would extend the temporal benefit.","Editorial inference: if the dataset is released, the immediate test is cross-platform generalization, training on the wearable-collected data and evaluating on the humanoid's own sensors, which would quantify the domain gap the height-matching design was intended to close."],"forward_implications":["A humanoid-specific occupancy benchmark with decimeter resolution over a 10 meter near-field range now exists, filling the gap left by autonomous-driving occupancy datasets that operate at 50 to 200 meters with meter-level resolution.","Multimodal fusion is worth its complexity on this task: the camera-plus-LiDAR model reaches 55.73 mIoU versus 50.37 for camera-only and 48.61 for LiDAR-only, confirming that geometry and semantics are complementary.","Temporal fusion of exactly one previous frame improves both mIoU and rayIoU, but windows of three or more frames degrade performance because pose errors accumulate during robotic motion.","A distortion-aware projection into BEV outperforms both raw pinhole projection and traditional image undistortion on the calibrated subset, so a latency-sensitive robot can avoid the cost of undistorting images.","The 40.5 million-parameter network establishes a lightweight baseline that future humanoid occupancy methods can be measured against on a common dataset."],"supporting_citations":[{"why":"supplies the surrounding-occupancy benchmark paradigm that the paper adapts to humanoid robots.","marker":"[56]"},{"why":"provides the visibility-aware label generation pipeline that motivates the humanoid annotation scheme.","marker":"[51]"},{"why":"contributes the LiDAR-to-camera cross-attention fusion design used in the multimodal network.","marker":"[24]"},{"why":"supplies the temporal BEV feature queue and ego-motion warping used for historical feature fusion.","marker":"[16]"},{"why":"provides the channel-to-height prediction head that turns BEV features into the 3D occupancy grid.","marker":"[59]"},{"why":"is the pillar-based LiDAR encoder that produces the geometric BEV features fused with camera features.","marker":"[22]"},{"why":"is the camera-only BEV baseline whose 75.3 million-parameter model is compared against the proposed network.","marker":"[17]"},{"why":"is the FB-Occ baseline that the paper's method is benchmarked against on the new dataset.","marker":"[25]"},{"why":"is the BEVFusion multimodal baseline that motivates and anchors the fusion comparison.","marker":"[29]"}],"fun_headline_variants":["First panoramic occupancy dataset for humanoid robots","6 cameras, one LiDAR: a head rig for humanoid occupancy","HumanoidOcc fuses LiDAR and camera features for better 3D occupancy","New multimodal occupancy system gives humanoid robots 3D understanding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The labels are built from multi-frame accumulated point clouds with no volumetric completion, so a voxel counts as empty whenever the 40-line laser did not put a point there; if real occupied space is silent to the sensor, every number in the tables inherits that blind spot.","fun_headline_variants_meta":{"raw":{"variants":["First panoramic occupancy dataset for humanoid robots","6 cameras, one LiDAR: a head rig for humanoid occupancy","HumanoidOcc fuses LiDAR and camera features for better 3D occupancy","New multimodal occupancy system gives humanoid robots 3D understanding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000498,"raw_usage":{"total_tokens":2481,"prompt_tokens":1032,"completion_tokens":1449,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":1385}},"tokens_in":648,"tokens_out":1449,"duration_ms":13413,"temperature":1.0,"reasoning_tokens":1385,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T13:42:16.542198+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick a held-out scene from the dataset and construct a dense completed reference that fills occluded space, such as a mesh or a second full-coverage scan; if the paper's voxel ground truth labels those filled regions as empty, then the reported mIoU and rayIoU measure how well models reproduce the 40-line LiDAR's visibility pattern, not true occupancy.","supporting_citations":[{"cited_title":"Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception","cited_arxiv_id":null,"evidence_quote":"supplies the surrounding-occupancy benchmark paradigm that the paper adapts to humanoid robots."},{"cited_title":"Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36:64318–64330, 2023","cited_arxiv_id":null,"evidence_quote":"provides the visibility-aware label generation pipeline that motivates the humanoid annotation scheme."},{"cited_title":"Le, Alan L","cited_arxiv_id":null,"evidence_quote":"contributes the LiDAR-to-camera cross-attention fusion design used in the multimodal network."}],"review_version":1}