{"id":"cbdcbe73-1e3a-4cab-91b9-a0b2d926e242","arxiv_id":"2412.08096","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"THUD++ is a 13-scene dynamic indoor RGB-D and trajectory dataset with benchmarks showing existing algorithms struggle in crowded mobile-robot environments.","lead":"THUD++ adds a large dynamic indoor RGB-D dataset, pedestrian trajectories, and a Unity3D simulator for mobile robot research, with benchmarks across detection, segmentation, relocalization, trajectory prediction, and navigation. If the data are reliable, it gives robot teams a missing resource for testing in crowded indoor spaces.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Trajectory and navigation benchmarks rely on synthetic pedestrians with unvalidated motion models; without evidence that these match real indoor dynamics, the 'realistic testbed' claim is not supported.","rationale":"I read the paper as a dataset and benchmark contribution: THUD++ claims to provide a large-scale, dynamic indoor testbed for mobile robots, with the synthetic components intended as realistic proxies. For that central claim to hold, the synthetic pedestrian behavior in the trajectory dataset and navigation emulator must behave sufficiently like real indoor pedestrians. The paper asserts this in Section V.A ('designed to capture realistic pedestrian movement patterns') and Section I ('mimic the real-world environments'), but the only description of the generation process is a kinematic parameterization with no explicit obstacle avoidance, social forces, or interaction model. The experiments in Sections VI.D and VI.E benchmark on these synthetic data, so any unrealistic artifact in the motion model could produce the observed performance gaps without reflecting genuine indoor difficulty. The real data portion is too small (5,191 of 90,175 frames) to independently validate realism. This matches the reader's weakest assumption exactly. I recommend keeping the CONDITIONAL verdict: if the authors add a validation study comparing their synthetic trajectories to real indoor datasets, or clearly scope the benchmarks as synthetic-only without transfer claims, the paper would be supportable. My proposed test would directly probe the realism assumption by checking for obstacle penetration and comparing motion statistics to established real-world datasets.","tokens_in":15256,"tokens_out":5931,"duration_ms":60587,"concrete_test":"Compute the obstacle-clearance and pairwise-distance statistics in the released THUD++ trajectory files (Section V.B format) and compare them with real indoor datasets (THÖR or L-CAS). Specifically, calculate the fraction of timesteps where a pedestrian is inside a static obstacle polygon or closer than 0.3 m to another pedestrian. If THUD++ has a non-negligible violation rate (e.g., >1%) or if its speed/curvature distributions differ significantly from real data (two-sample test), the 'realistic pedestrian movement' claim is falsified and the benchmark transferability is in doubt.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that THUD++ is a reusable testbed for dynamic scene understanding in real mobile-robot settings depends on the synthetic components being representative of real indoor dynamics, but this is not established. In Section V.A, synthetic pedestrian trajectories are generated by parameterizing start/end points, direction, speed, angular velocity, and acceleration, and each pedestrian disappears upon reaching the target. The paper does not describe any collision-avoidance or social-interaction model for these pedestrians, nor any validation against real indoor trajectory data. Sections VI.D and VI.E then use these synthetic trajectories and scenes to benchmark trajectory prediction and navigation; the resulting performance degradations (Tables V and VI) are attributed to indoor dynamic complexity, but could instead reflect artifacts of the synthetic motion model, such as trajectories that pass through obstacles or have unrealistic spacing. The real RGB-D frames are only 5,191 of 90,175 (Section IV.B), so they do not validate the synthetic dynamics. Thus the load-bearing assumption is unverified realism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces THUD++, a mobile-robot-oriented dataset that combines real and synthetic RGB-D data from 13 indoor scenes, with 90,175 frames annotated with 2D/3D bounding boxes, semantic and instance labels, camera poses, and IMU data; a synthetic pedestrian trajectory dataset for three indoor scenes; and a Unity3D-based navigation emulator. The authors benchmark several published methods on 3D object detection, semantic segmentation, robot relocalization, pedestrian trajectory prediction, and navigation, reporting performance degradation with increasing scene dynamics. The stated goal is to provide a reusable testbed for dynamic indoor scene understanding for mobile robots.","tokens_in":15461,"tokens_out":6165,"duration_ms":56026,"significance":"If the dataset is released as described and the benchmark results are reproducible, THUD++ would be a useful community resource: it is one of the few RGB-D datasets with explicit dynamic-object annotations spanning multiple real and synthetic scenes, and it offers a closed-loop navigation emulator plus trajectory data for indoor settings, where existing trajectory datasets are scarce. The paper is transparent about building on the authors' previous ICRA dataset [26], and the release of the dataset and platform is an asset. The main caveats are that the headline numbers in the abstract are not consistent with the body, and that the synthetic trajectory and navigation components carry an unvalidated realism assumption on which the causal claims in Section VI depend.","major_comments":[{"comment":"The abstract states that the trajectory dataset 'covers over 6,000 pedestrian trajectories', but Section V.B reports 1,257 automatically annotated pedestrian tracks across 6,363 frames, and Table II lists 1,257 pedestrians. This is a factor-of-five overstatement of the number of trajectories and must be corrected; the 'over 6,000' number appears to refer to frames, not pedestrians.","section":"Abstract; Section V.B; Table II"},{"comment":"The abstract advertises 'more than 20 million 2D/3D bounding boxes', while Section IV.B says the dataset has 'over 20M labels' across four annotation types (2D boxes, 3D boxes, semantic segmentation, instance segmentation) and gives an average of 176 data labels per frame. With 90,175 frames, 176 labels per frame gives about 15.9M labels, not 20M. Please report per-annotation-type counts and reconcile these numbers; the current phrasing makes the dataset scale ambiguous.","section":"Abstract; Section IV.B"},{"comment":"The trajectory prediction and navigation benchmarks are built entirely on synthetic pedestrians whose motion model is described only by start/end points, direction, speed, angular velocity, and acceleration, with no collision-avoidance or social-interaction model, and with pedestrians disappearing at their target. Section VI.D nevertheless attributes the indoor performance gap to static obstacles and to human interactions occurring more frequently, and Section VI.E treats the synthetic scenes as representative of real crowded indoor environments. This is a load-bearing inference that needs support: either document and validate the pedestrian model (for example, by comparison with real indoor trajectory statistics), or explicitly restrict the claims to synthetic-domain benchmarking.","section":"Section V.A; Sections VI.D and VI.E"}],"minor_comments":[{"comment":"The sentence 'the ADE metric for Social-GAN drops from 1.08 to 1.81 (67.6%)' uses 'drops' for an increase in error; it should read 'increases by 67.6%'.","section":"Section VI.D"},{"comment":"The stated duration of 60 minutes is inconsistent with 6,363 frames recorded at 0.4-second intervals (about 42 minutes); please clarify the sampling rate and duration.","section":"Table II; Section V.B"},{"comment":"ImVoteNet's dynamic-object mAP (17.49) exceeds its static-object mAP (17.29) in the Supermarket row, which is at odds with the unqualified statement that there are 'varying degrees of decrease' for dynamic objects; the subsequent caveat about object categories should be integrated into the conclusion.","section":"Table III"},{"comment":"There is a typo 'ACNett' in the Canteen block of Table IV, and Figure 9 has garbled axis labels that should be regenerated.","section":"Table IV; Figure 9"},{"comment":"The abstract and Section IV.B should use the same terminology for the annotation counts ('bounding boxes' versus 'labels') to avoid confusion.","section":"Abstract; Section IV.B"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the manuscript is a dataset and benchmark contribution rather than a methods paper, and the core resource is potentially valuable. The main blockers are internal numerical inconsistencies in the headline statistics and the unvalidated synthetic-realism assumption; both are addressable in revision. I saw no circularity or attribution concerns in the relation to [26], and the self-citation is appropriate and transparent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a genuine dataset extension, not a conceptual advance. The authors are transparent about what is new over their ICRA paper: the Unity3D platform and navigation emulator, the synthetic trajectory set, and benchmark runs for trajectory prediction and navigation. That transparency is good, and the RGB-D portion with 5,191 real frames and 84,984 synthetic frames, densely annotated, is a usable resource for dynamic indoor scene understanding.\n\nThe soft spots are real and need to be fixed. First, the headline numbers do not reconcile. The abstract says \"over 6,000 pedestrian trajectories\"; Section V.B says 1,257 tracks over 6,363 frames. The abstract says \"over 20M labels\"; at 176 labels per frame over 90,175 frames, that is about 15.9M. Section V says 60 minutes of data, but 6,363 frames at 0.4s intervals is 42 minutes. These are not minor typos; they undermine the scale claims that are the paper's main selling point. Second, the new trajectory and navigation components are entirely synthetic, and the paper does not validate them against real indoor pedestrian motion. The trajectory generation is parameterized start/end points, direction, speed, angular velocity, and acceleration, with pedestrians disappearing at the target. No collision avoidance or social interaction is described. Training and testing navigation in the same synthetic environment with ORCA-simulated humans means the performance degradations in Tables V and VI could reflect simulation artifacts just as much as scene complexity. Third, the benchmark tables have no uncertainty measures; the differences between methods are often small, and no variance or multiple seeds are reported.\n\nNone of this makes the paper worthless. The dataset and emulator are likely to be useful to people working on dynamic indoor robot perception. The citation pattern is fine; the self-citation to [26] is appropriate and clearly announced. The authors are honest about scope and do not hide the synthetic origin. The central problem is that they then assert realism without evidence.\n\nSend it to peer review. A serious referee can push for corrected statistics, uncertainty reporting, and at least an honest limitation statement about synthetic-to-real transfer. Do not accept it in its current form.","headline":"A useful but inconsistently-numbered dataset extension; the new trajectory and navigation components are unvalidated synthetic simulations.","tokens_in":15952,"tokens_out":4495,"would_cite":true,"duration_ms":47015,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"THUD++ is a 90,175-frame indoor RGB-D dataset with over 20 million labels for benchmarking how robot perception and navigation degrade as scenes get more dynamic.","keywords":["dynamic indoor scenes","mobile robot dataset","RGB-D dataset","pedestrian trajectory prediction","robot navigation emulator","scene understanding benchmark","synthetic and real data","dynamic complexity"],"falsifier":"Run one of the benchmarked navigation policies in a real crowded indoor space, measure the same success, collision, and navigation-time metrics with the same per-frame pedestrian counts, and compare them with the synthetic results; systematic divergence would show that the simulated dynamics do not transfer.","tokens_in":15097,"feed_emoji":"🤖","tokens_out":10169,"duration_ms":94212,"temperature":0.7,"pith_summary":"THUD++ is a dataset and benchmark aimed at one gap: most indoor RGB-D datasets are static, while real service robots operate among moving people. The paper contributes 90,175 frames from 13 real and simulated indoor scenes, more than 20 million labels over 91 categories, a 6,363-frame pedestrian-trajectory set with 1,257 tracks, and a closed-loop navigation emulator. Benchmarks on representative state-of-the-art methods show consistent degradation as scene dynamics increase: dynamic objects are detected worse than static ones, relocalization errors grow with pedestrian density, trajectory prediction errors are larger indoors than outdoors, and navigation success falls as crowd size rises. If the dataset is adopted, it gives the mobile-robotics community a shared way to train and test dynamic scene understanding.","feed_headline":"New dataset packs 90k frames of moving people into robot benchmarks","feed_subtitle":"Five robot tasks lose accuracy as the new THUD++ scenes get crowded.","key_machinery":"The load-bearing object is the dataset itself, organized in three synchronized parts: RGB-D frames with 2D/3D bounding boxes, semantic and instance segmentation, camera poses and IMU; pedestrian trajectories recorded in confined indoor layouts; and a physics-based simulation platform with a navigation emulator for closed-loop experiments. The argument-carrying device is the dynamic-complexity metric, defined as the average number of pedestrians or dynamic objects per frame. Every benchmark in the paper is arranged around that quantity: static versus dynamic object categories, scenes with low and high pedestrian density, and navigation trials with K=10, 15, or 20 pedestrians.","core_discovery":"The central claim, stated on the paper's own terms, is that THUD++ is the large-scale dynamic indoor testbed that existing RGB-D and trajectory datasets lack. The paper demonstrates the claim by benchmarking five tasks at different levels of dynamic complexity, quantified as average pedestrians per frame. On THUD++, detectors lose mAP on dynamic objects, RGB-D segmentation mIoU drops on the real crowded canteen scene, relocalization translation and rotation errors increase with pedestrian count, trajectory-prediction ADE and FDE rise substantially compared with an outdoor public set, and navigation policies show lower success and higher collision rates as K grows from 10 to 20 pedestrians. Density of annotation, about 176 labels per frame, is what lets the benchmarks separate the effect of motion from the effect of clutter.","pith_inferences":["Not tested in the paper: whether the 84,984 synthetic frames transfer to real robot operation; a real-crowd validation run would settle that.","The dynamic-complexity metric could be applied to existing indoor datasets, letting the community rank benchmarks by motion content and compare methods at matched difficulty.","A natural extension is to use the trajectory and navigation components together to train socially aware planners that both predict pedestrians and avoid static obstacles.","The gap between global- and local-feature relocalization suggests fusing both feature types could be a cheap robustness gain that THUD++'s scene coverage could test."],"forward_implications":["THUD++ can serve as a stress test: methods that keep their accuracy as pedestrians per frame rises are the ones likely to survive real service-robot deployments.","Evaluating detectors separately on static and dynamic objects makes it possible to quantify how much performance loss is caused by motion rather than by object category.","Trajectory prediction gets an indoor benchmark with static obstacles and confined spaces, where current methods show markedly larger errors than on open outdoor scenes.","The emulator allows closed-loop navigation experiments under controlled crowd density, with success rate, collision rate, navigation time, and social path efficiency as comparable metrics.","The synthetic pipeline can generate additional labeled scenes on demand, so researchers can expand the benchmark without new physical sensors."],"supporting_citations":[{"why":"Representative static RGB-D reconstruction dataset that THUD++ contrasts with in the dataset comparison table, establishing the missing dynamic-object gap.","marker":"[10]"},{"why":"Earlier conference version of THUD++ that this paper explicitly extends with the simulation platform, trajectory dataset, and additional benchmark tasks.","marker":"[26]"},{"why":"Outdoor pedestrian-trajectory set used as the comparison baseline in the trajectory prediction benchmark.","marker":"[28]"},{"why":"ImVoteNet baseline used in the 3D object detection benchmark on THUD++ scenes.","marker":"[36]"},{"why":"ESANet baseline used in the semantic segmentation benchmark, including an ablation with and without dynamic objects.","marker":"[47]"},{"why":"NetVLAD baseline used in the robot relocalization benchmark; its error grows with pedestrian density.","marker":"[52]"},{"why":"Social-GAN baseline used in the trajectory prediction benchmark; its error rises steeply in THUD++ indoor scenes.","marker":"[54]"},{"why":"ORCA baseline used in the navigation emulator benchmark; its navigation time rises with pedestrian count and exhibits the freezing-robot problem.","marker":"[57]"}],"fun_headline_variants":["90k frames, 20M boxes: THUD++ puts robots in crowded scenes","New benchmark makes robots face crowded indoor chaos","THUD++ adds 90k dynamic frames to robot benchmarks","Crowded scenes drop robot accuracy across five tasks","Indoor robot dataset THUD++ raises the stakes on dynamics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the simulated pedestrians and indoor layouts in THUD++ are representative enough of real crowded canteens and supermarkets that benchmark rankings obtained in simulation will also hold on physical robots.","fun_headline_variants_meta":{"raw":{"variants":["90k frames, 20M boxes: THUD++ puts robots in crowded scenes","New benchmark makes robots face crowded indoor chaos","THUD++ adds 90k dynamic frames to robot benchmarks","Crowded scenes drop robot accuracy across five tasks","Indoor robot dataset THUD++ raises the stakes on dynamics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000313,"raw_usage":{"total_tokens":1772,"prompt_tokens":930,"completion_tokens":842,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":757}},"tokens_in":546,"tokens_out":842,"duration_ms":8219,"temperature":1.0,"reasoning_tokens":757,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:13:44.011553+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run one of the benchmarked navigation policies in a real crowded indoor space, measure the same success, collision, and navigation-time metrics with the same per-frame pedestrian counts, and compare them with the synthetic results; systematic divergence would show that the simulated dynamics do not transfer.","supporting_citations":[{"cited_title":"Scannet: Richly-annotated 3d reconstructions of indoor scenes,","cited_arxiv_id":null,"evidence_quote":"Representative static RGB-D reconstruction dataset that THUD++ contrasts with in the dataset comparison table, establishing the missing dynamic-object gap."},{"cited_title":"Mobile robot oriented large-scale indoor dataset for dynamic scene understanding,","cited_arxiv_id":null,"evidence_quote":"Earlier conference version of THUD++ that this paper explicitly extends with the simulation platform, trajectory dataset, and additional benchmark tasks."},{"cited_title":"Imvotenet: Boosting 3d object detection in point clouds with image votes,","cited_arxiv_id":null,"evidence_quote":"ImVoteNet baseline used in the 3D object detection benchmark on THUD++ scenes."},{"cited_title":"Efficient rgb-d semantic segmentation for indoor scene analysis,","cited_arxiv_id":null,"evidence_quote":"ESANet baseline used in the semantic segmentation benchmark, including an ablation with and without dynamic objects."},{"cited_title":"Reciprocal n- body collision avoidance,","cited_arxiv_id":null,"evidence_quote":"ORCA baseline used in the navigation emulator benchmark; its navigation time rises with pedestrian count and exhibits the freezing-robot problem."}],"review_version":1}