{"id":"c671ecc5-769c-4da0-9aa0-91e58417d687","arxiv_id":"2605.28237","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"POINav-Bench provides the first high-fidelity real-world benchmark for POI-goal VLN using 3DGS reconstructions of 126k m² with 163 POIs, supported by a Brain-Action framework and 70K real signage-entrance dataset.","lead":"The paper introduces POINav-Bench, the first closed-loop benchmark for real-world POI-goal navigation built from 3D Gaussian Splatting reconstructions of 11 commercial areas, plus the POINav Brain-Action Framework and a 70K signage-entrance dataset. A smart generalist might read it to see how final-meters precision in real environments could be tested and improved for practical robot navigation.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"3DGS sim fidelity for traversability/dynamics unvalidated vs physical execution","rationale":"The reader's weakest assumption is precisely the load-bearing point; the provided abstract supplies no counter-evidence (physical trials or fidelity metrics), so the UNVERDICTED verdict with low confidence is appropriate and unchanged by this analysis.","tokens_in":1730,"tokens_out":315,"duration_ms":15461,"concrete_test":"Run the released POINav agent on a physical robot in at least one of the 11 commercial areas using the same POIs and reference trajectories; measure success rate and final-meter error against the POINav-Bench numbers. A gap >15% absolute success or >0.5 m median error falsifies the transfer assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that POINav-Bench + Brain-Action Framework provides a viable path to real-world POI navigation. This rests on the assumption that 3DGS reconstructions of the 11 areas (with traversability annotations) enable closed-loop evaluation whose results transfer without large gaps. The abstract reports experiments only inside the benchmark; no physical robot deployment, no quantitative sim-to-real comparison on success rate or trajectory error, and no ablation on lighting/dynamics mismatch are described. If the reconstruction omits dynamic obstacles, specular reflections on commercial surfaces, or fine-grained traversability (e.g., curbs, wet floors), the reported gains remain internal to simulation and do not establish viability for physical execution.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces POINav-Bench, the first closed-loop benchmark for real-world POI-goal navigation, consisting of 11 commercial areas (126,398 m² total) reconstructed via 3D Gaussian Splatting with traversability-aware annotations and reference trajectories spanning 163 POIs. It proposes the POINav Brain-Action Framework, in which a Brain module performs POI-grounded reasoning to guide an Action module that outputs continuous waypoints, and curates POINav-Dataset (70K real-world signage-entrance pairs). The abstract states that experiments demonstrate the framework provides a viable path toward refining real-world POI-goal navigation.","tokens_in":1860,"tokens_out":351,"duration_ms":19993,"significance":"If the sim-to-real transfer holds, the benchmark and framework could meaningfully advance VLN research by supplying high-fidelity, POI-rich environments that address coarse granularity and sim-to-real gaps in existing benchmarks, with potential downstream impact on practical robotic navigation in commercial spaces. The scale of the 3DGS reconstructions and the introduction of traversability annotations represent concrete contributions to evaluation infrastructure.","major_comments":[{"comment":"Abstract: The central claim that the framework 'provides a viable path toward refining real-world POI-goal navigation' is load-bearing yet unsupported by any described physical robot deployment, quantitative sim-to-real comparison (success rate, trajectory error, etc.), or ablation on unmodeled factors such as dynamic obstacles, specular reflections, or fine-grained traversability (curbs, wet floors). All experiments appear confined to the benchmark, leaving the transfer assumption untested.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive review. We address the major comment below and will make corresponding revisions to strengthen the manuscript.","responses":[{"response":"We agree that the current experiments are performed within the POINav-Bench rather than on physical robots. The benchmark itself is constructed from real-world captures of 11 commercial areas (126,398 m²) using 3D Gaussian Splatting, with traversability annotations and reference trajectories derived directly from those captures; the 70K signage-entrance pairs are likewise real-world data. The Brain-Action framework is evaluated in closed-loop on this high-fidelity benchmark to demonstrate improved POI-grounded reasoning and waypoint prediction. We acknowledge that no physical robot deployment, explicit sim-to-real quantitative metrics, or ablations on dynamic obstacles, specular reflections, or fine-grained surface conditions are reported. To address this, we will revise the abstract (and relevant sections) to state that the framework shows promise on the high-fidelity real-world-derived benchmark as a concrete step toward real-world POI-goal navigation, rather than claiming direct refinement of physical systems without further validation.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The central claim that the framework 'provides a viable path toward refining real-world POI-goal navigation' is load-bearing yet unsupported by any described physical robot deployment, quantitative sim-to-real comparison (success rate, trajectory error, etc.), or ablation on unmodeled factors such as dynamic obstacles, specular reflections, or fine-grained traversability (curbs, wet floors). All experiments appear confined to the benchmark, leaving the transfer assumption untested."}],"tokens_in":1370,"tokens_out":354,"duration_ms":25172,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main contribution is POINav-Bench: 11 real commercial areas reconstructed via 3D Gaussian Splatting, totaling over 126k square meters and 163 POIs, with traversability annotations and reference trajectories for closed-loop evaluation. They also release a 70k real-world signage-entrance dataset and a Brain-Action framework that splits POI reasoning from waypoint prediction.\n\nThis is genuinely new. Most existing VLN work either uses generated scenes or stops at coarse goals, so a high-fidelity real-capture benchmark focused on the final meters fills a clear gap. The scale and the explicit POI emphasis are useful additions that prior datasets do not directly provide.\n\nThe framework itself is straightforward but reasonable: the Brain module grounds language in the scene, the Action module outputs continuous controls. The dataset curation from real signage is a practical step.\n\nThe soft spot is the evaluation. All reported results run inside the benchmark; the abstract gives no physical robot deployments, no quantitative sim-to-real transfer numbers on success rate or path error, and no tests on lighting changes, moving obstacles, or surface variations that 3DGS often struggles with. The claim that the framework offers a viable path to real-world use therefore rests on an untested transfer assumption. If the reconstructions miss curbs, wet floors, or dynamic elements, the internal gains may not carry over.\n\nThis paper is for groups working on real-world VLN or sim-to-real transfer who need better testbeds. The benchmark and dataset artifacts are the parts most likely to see reuse. It deserves a serious referee because new, grounded evaluation resources can organize a subfield even when the accompanying method is still early. I would send it to review with the expectation that the authors add at least one physical deployment or a clear sim-to-real ablation.","headline":"New 3DGS-based benchmark and 70k dataset for real POI navigation in commercial spaces, but experiments stay inside simulation with no physical robot tests or sim-to-real checks.","tokens_in":2397,"tokens_out":449,"would_cite":false,"duration_ms":16081,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"POINav-Bench and the Brain-Action Framework enable precise POI-goal navigation in reconstructed real-world environments.","keywords":["vision language navigation","point of interest","robot navigation","3d gaussian splatting","benchmark","final meters","real world navigation","waypoints"],"falsifier":"A physical robot executing the framework in one of the 11 commercial areas showing substantially different success rates or paths compared to its performance on the corresponding POINav-Bench reconstruction.","tokens_in":2638,"feed_emoji":"🗺️","tokens_out":496,"duration_ms":31166,"temperature":0.7,"pith_summary":"The paper presents POINav-Bench, a benchmark with 11 commercial areas reconstructed via 3D Gaussian Splatting spanning over 126,000 square meters and 163 points of interest. It includes traversability annotations and reference paths for closed-loop testing of navigation agents. The authors also introduce the POINav Brain-Action Framework that uses a reasoning module to guide waypoint prediction based on POIs. This matters because current vision-language navigation often fails to reach exact destinations in the final meters despite succeeding at higher-level tasks. The work aims to close the gap between simulation and real-world execution for practical robot navigation in stores and similar spaces.","feed_headline":"Benchmark tests final-meters robot navigation to store entrances","feed_subtitle":"11 real commercial areas reconstructed in 3D and a reasoning framework guide precise waypoint following.","key_machinery":"The POINav Brain-Action Framework, with its Brain module performing POI-grounded reasoning and Action module predicting continuous waypoints.","core_discovery":"POINav-Bench provides the first closed-loop evaluation platform for real-world POI-goal navigation using high-fidelity 3DGS reconstructions of 11 commercial areas with traversability-aware annotations and reference trajectories. The POINav Brain-Action Framework employs a Brain module for POI-grounded reasoning to direct an Action module in generating continuous waypoints. Together with the POINav-Dataset of 70K real-world signage-entrance pairs, these tools demonstrate a viable approach to refining final-meters arrival in POI-rich environments.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["POINav-Bench evaluates final-meters POI navigation in real 3D scenes","Brain-Action framework directs waypoint prediction for store entrances","11 3DGS areas enable closed-loop testing of real POI navigation","70K signage pairs support precise entrance waypoint generation"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The 3D Gaussian Splatting models of the commercial areas accurately reflect real-world traversability, lighting, and dynamic conditions so that results transfer to physical robots.","fun_headline_variants_meta":{"raw":{"variants":["POINav-Bench evaluates final-meters POI navigation in real 3D scenes","Brain-Action framework directs waypoint prediction for store entrances","11 3DGS areas enable closed-loop testing of real POI navigation","70K signage pairs support precise entrance waypoint generation"]},"model":"grok-4.3","cost_usd":0.004985,"raw_usage":{"total_tokens":2445,"prompt_tokens":687,"num_sources_used":0,"completion_tokens":71,"cost_in_usd_ticks":49849500,"prompt_tokens_details":{"text_tokens":687,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1687,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":687,"tokens_out":71,"duration_ms":13627,"temperature":1.0,"reasoning_tokens":1687,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T11:46:03.299268+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A physical robot executing the framework in one of the 11 commercial areas showing substantially different success rates or paths compared to its performance on the corresponding POINav-Bench reconstruction.","supporting_citations":[],"review_version":1}