{"id":"cbc28bb6-6624-4c0e-8b2a-1d8339ed56a6","arxiv_id":"2608.11901","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new outdoor vision-and-language navigation dataset that pairs driving instructions with continuous, partially dynamic simulated streets, plus a reinforcement learning baseline.","lead":"The authors introduce a new outdoor navigation dataset, built in the CARLA simulator, where an agent follows natural language driving directions while moving freely and avoiding randomly placed cars and pedestrians. The paper argues that this combination of continuous movement and changing traffic is closer to real streets than previous navigation benchmarks, though the supporting experiments are thin.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 'continuous actions' novelty is contradicted by the benchmark itself: Section 4.2.1 defines a seven-symbol discrete action space with fixed 4 m steps, so the paper's own experiments do not exercise continuous action control.","rationale":"The reader's assigned weakest assumption is the realism of programmatic dynamic elements, which I think is a real concern but not the most load-bearing one. A stronger and more direct problem is that the paper's central 'continuous actions' contribution is not realized in its own experimental setup. This undermines the headline novelty claim independently of how one judges the CARLA dynamic elements. The reader's rationale does mention the mismatch between the 'continuous actions' framing and the discrete seven-action space, so there is partial agreement, but the reader's formal weakest_assumption focuses elsewhere. My stress-test therefore identifies the action-space contradiction as the primary issue. I keep the reader's CONDITIONAL verdict rather than moving to REJECT because the flaw is concrete and potentially fixable: the authors could either re-scope the novelty claim to 'continuous environment with discrete high-level commands' or implement and evaluate a genuinely continuous action space. The still-unreleased data/code, lack of seed variance in the RL experiments, and the confounded discrete comparison between maps of different granularity further support the need for revision, but none of these is as directly tied to the central claim as the continuous-action contradiction.","tokens_in":11093,"tokens_out":8139,"duration_ms":85148,"concrete_test":"Ask the authors to release the CARLA environment wrapper and training code, then inspect the action interface. If the policy outputs one of seven categorical action IDs and the simulator advances the vehicle by a fixed 4 m (or 3 m/5 m) per command, the 'continuous actions' claim is false. A direct behavioral check: replace the categorical policy head with a Gaussian policy over continuous steering/throttle commands and attempt to step the environment; if the environment rejects arbitrary continuous values and only accepts the seven discrete symbols, then the benchmark is discrete rather than continuous.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that DaViNCi is the first outdoor VLN dataset with both continuous and dynamic factors. The dynamic factor depends on the realism of randomly spawned CARLA vehicles and pedestrians, which is arguable but at least implemented as claimed. The continuous-action factor, however, is contradicted by the paper's own benchmark. Section 4.2.1 defines the action space as A in {sharpleft, left, straight, right, sharpright, lanechange, stop}, and Section 4.2.3 states that each action is executed with a fixed 4 m step. A policy over seven categorical symbols is a discrete action space; the dataset may contain continuous vehicle trajectories and online visual rendering, but the agent interface evaluated in Tables 3-5 is not continuous action control. The abstract and Figure 5 nevertheless claim 'continuous actions,' and the central contribution asserts a paradigm shift from discrete to continuous. The granularity ablation in Table 4 varies step sizes (3 m, 4 m, 5 m), which changes the distance per discrete command, not the continuity of the action parameterization. Thus the claimed continuous-action factor is unsupported by the paper's own experiments. If 'continuous' is meant only to describe uninterrupted physical motion or non-graph-based waypoints, that weaker meaning must replace the abstract's 'continuous actions,' and the novelty claim needs to be re-scoped. This is not a matter of external consensus; it is an internal contradiction between the stated action space and the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DaViNCi, a CARLA-based outdoor Vision-and-Language Navigation (VLN) dataset containing 6,933 trajectories across six maps, with instructions generated by a VLM and then manually fine-tuned. The authors claim this is the first outdoor VLN dataset to simultaneously provide continuous action control and dynamic environmental elements. They build a discrete version of four maps to compare existing outdoor VLN methods against Touchdown and Map2Seq, and they propose an RL-based baseline (COVL-RL) for a continuous experimental setting, also studying the effect of action granularity and the number of dynamic elements. The central claims are that DaViNCi is more challenging than previous datasets, with success rates dropping by more than 10% in discrete settings and by even more in continuous settings, and that this will advance sim-to-real transfer for VLN agents.","tokens_in":11342,"tokens_out":4670,"duration_ms":49587,"significance":"If the central claims are validated, DaViNCi would be a useful contribution: it provides a relatively large outdoor VLN dataset with diverse maps, dynamic objects, and a reproducible benchmark pipeline, including a random-policy baseline and an RL baseline. The authors also give credit for collecting first-person video and using trajectory-derived turn events to constrain instruction generation. However, the paper's principal novelty—continuous actions—is not actually exercised in the experiments: the action space in Section 4.2.1 is a seven-symbol discrete set with fixed 4 m steps, so the claimed 'continuous actions' paradigm shift is not demonstrated. The abstract's '10% success rate decrease' claim is also not precisely supported by the reported metrics (TC, SPD, nDTW) and is confounded by different map granularities. The dynamic-elements contribution is plausible but rests on unvalidated assumptions about realism. These issues are fixable within the scope of the manuscript, but they are load-bearing and require substantive revision.","major_comments":[{"comment":"The paper claims 'continuous actions' as a central novelty (Abstract, Section 1, Figure 5), but the actual experimental action space is discrete: A ∈ {sharpleft, left, straight, right, sharpright, lanechange, stop}, executed with a fixed 4 m step. A policy over seven categorical symbols is a discrete action space, regardless of whether the underlying simulator state is continuous. Varying the step size (3/4/5 m in Table 4) does not make the action parameterization continuous. If 'continuous environment' is meant to refer to free movement without a topological graph, rather than continuous action control, the text must be re-scoped and the 'paradigm shift from discrete to continuous' claim must be revised accordingly.","section":"Section 4.2.1, Section 4.2.3, Figure 5"},{"comment":"The abstract and conclusion claim that 'the success rate on DaViNCi decreased by more than 10% in discrete environments compared to previous datasets,' but Table 2 reports TC, SPD, and nDTW, not success rate. TC is not defined in the evaluation-metrics paragraph, and if TC is intended to be the success rate, that should be stated explicitly. Moreover, the comparison is confounded: town03 uses a 12 m discretization while other maps use 5 m, and the Touchdown/Map2Seq columns are from different datasets and map structures. Please provide per-model, per-map success rates with clear definitions, or remove the unsupported 'success rate' wording.","section":"Abstract, Section 4.1, Table 2"},{"comment":"The RL baseline results in Tables 3–5 are reported without standard deviations or the number of seeds. Given that the random-policy baseline achieves near-zero success rates, single-seed results are insufficient to support the conclusions about granularity (Table 4) and dynamic-element impact (Table 5), where the differences between rows are often small (e.g., town03 SR 17.1 vs 17.6 vs 17.9). Please run multiple seeds and report mean ± std, or explicitly state that these are single-seed results and temper the corresponding conclusions.","section":"Section 4.2.4, Tables 3, 4, 5"},{"comment":"The dynamic elements—randomly spawned vehicles and pedestrians acting 'based on programmatic rules and operate randomly'—are the core of the 'dynamic' novelty, but no evidence is presented that they represent realistic unpredictable urban traffic. The paper later attributes performance decline to 'obstruction of the partial field of view' without a causal analysis. To support the claim that DaViNCi 'will significantly advance the sim-to-real transfer of VLN agents,' please validate the dynamic actors (e.g., report collision rates, interaction statistics, or a comparison with scripted traffic scenarios) or explicitly narrow the claim to 'dynamic elements in simulation.'","section":"Section 3.1.3, Section 5"}],"minor_comments":[{"comment":"There are multiple grammatical and typographical issues: 'the figure. 2' and 'the figure. 3' should be 'Figure 2' and 'Figure 3'; 'we introduces' in Related Work should be 'we introduce'; Figure 5 contains 'LTSM', which should be 'LSTM'.","section":"Throughout"},{"comment":"The metrics TC, SPD, and nDTW are named as 'standard metrics' but not defined. Please provide explicit formulas or references, especially because the abstract refers to 'success rate,' which may or may not correspond to TC.","section":"Section 4.1.2"},{"comment":"The term 'granularity' is used for two different concepts: discrete map discretization (5 m vs 12 m) in Section 4.1.1 and continuous action step size (3/4/5 m) in Table 4. This is confusing; please use distinct terms such as 'map discretization' and 'action step size.'","section":"Section 4.1.1"},{"comment":"The citation for the 'CARLA AD Leaderboard' is given as [6], which is the NAVSIM paper (Dauner et al., 2024), not the CARLA Leaderboard. Please cite the correct source or remove the citation.","section":"Section 3.1.2"},{"comment":"The equation defines Path Completion (PC), but the metric used in tables is called Average Path Completion (APC). Please clarify the relationship between PC and APC.","section":"Section 4.2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's central novelty claim—'first outdoor VLN dataset with continuous and dynamic factors'—is internally contradicted by its own action-space definition in Section 4.2.1. This is not a matter of external consensus but a direct mismatch between the stated contribution and the experimental interface. The dataset itself may still be valuable as an outdoor dynamic VLN benchmark, and the authors should re-scope the claims accordingly. The absence of multi-seed RL results and the undefined 'success rate' claim also need to be addressed before the paper can be recommended for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on DaViNCi. The dataset itself is a plausible resource: 6,933 CARLA trajectories across six maps, generated by tracking autopilot vehicles and then annotated with VLM-generated instructions plus manual cleanup. That is real work, and the combination of outdoor VLN with non-graph-based continuous paths and dynamic obstacles (programmatic vehicles/pedestrians) is genuinely new relative to Touchdown, Talk2Nav and Map2Seq. The discrete benchmark includes several strong baselines (GA, VELMA, FLAME) and the numbers show the dataset is non-trivial.\n\nThe problem is the paper's central 'continuous actions' claim. Section 4.2.1 defines the agent's action space as seven symbols — {sharpleft, left, straight, right, sharpright, lanechange, stop} — and each action is executed with a fixed 4 m step. That is a discrete action space, not a continuous one. The dataset contains continuous ground-truth trajectories, and the rendering is online, but the evaluated agent interface is discrete. The abstract nevertheless says 'continuous actions' and the paper draws a paradigm-shift conclusion from it. That is an internal inconsistency, not a matter of external debate. If the authors meant 'continuous' as 'not constrained to a topological graph,' they should say that and re-scope the novelty claim.\n\nThe experimental evidence has a few more soft spots. The random baselines are fine, but the RL results come from a single seed with no error bars or variance reporting, so the success rates (roughly 12–18%) are hard to interpret. The discrete comparison across datasets mixes map and granularity: town03 uses 12 m waypoints while the others use 5 m, so the claim that DaViNCi is >10% harder than Touchdown/Map2Seq is not a controlled comparison. The dynamic elements are programmatic and simple; whether they represent real-world unpredictability is asserted, not demonstrated. And the paper does not say whether the dataset and code will be released, which limits reproducibility of the benchmark itself.\n\nNone of this makes the dataset worthless. The construction pipeline and the two benchmarks are a reasonable starting point for outdoor VLN. But the abstract oversells what the experiments actually evaluate. I would send this to a serious referee with a request for revision: re-scope the 'continuous' terminology, add seeds and error bars, control the discrete comparison or acknowledge its confounds, state the release plan, and show a few qualitative examples that the dynamic elements cause the visual obstruction they claim.\n\nNet: worth engaging, not worth accepting as-is.","headline":"A useful outdoor VLN dataset whose central 'continuous actions' claim is contradicted by its own seven-symbol action space.","tokens_in":11934,"tokens_out":3251,"would_cite":false,"duration_ms":31310,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DaViNCi is the first outdoor VLN dataset to combine continuous driving actions with moving traffic, and experiments show this combination lowers success rates by more than 10 percentage points compared with static discrete datasets.","keywords":["Outdoor Vision-and-Language Navigation","Continuous Environment","Dynamic Elements","Reinforcement Learning","CARLA simulator","dataset benchmark","sim-to-real transfer"],"falsifier":"Run the same continuous baseline twice on the same routes: once with moving vehicles and pedestrians, and once with identical vehicles and pedestrians placed at fixed positions. If success rates and path completion are the same in both conditions, then the dynamic property itself is not what makes DaViNCi hard, only the presence of extra objects; equivalently, replay trained agents against real recorded traffic in the same CARLA towns and check whether the baseline ranking of methods changes.","tokens_in":10845,"feed_emoji":"🚗","tokens_out":9870,"duration_ms":97956,"temperature":0.7,"pith_summary":"Vision-and-language navigation (VLN) research has mostly moved indoors and, outdoors, has relied on fixed waypoint graphs in static scenes. The paper argues this is the wrong abstraction for vehicles: real streets are continuous and filled with moving traffic. To address this, DaViNCi collects 6,933 path-instruction pairs across six CARLA towns where the agent drives with continuous actions and must share the road with randomly moving vehicles and pedestrians. The central empirical claim is that this combination is genuinely harder: existing outdoor VLN methods lose more than 10 percentage points in success when transferred to a discretized version of DaViNCi, and a purpose-built continuous baseline reaches only 5.8–18.2 percent success. The experiments also isolate the difficulty, showing that increasing the number of dynamic elements lowers performance monotonically while action granularity has no consistent effect.","feed_headline":"Outdoor navigation benchmark adds moving cars and pedestrians","feed_subtitle":"DaViNCi replaces static waypoints with continuous driving and live traffic; success rates drop by over 10%.","key_machinery":"The load-bearing mechanism is the pairing of a continuous action space with a dynamic rendering loop inside the CARLA simulator. Instead of pre-collected panoramic snapshots at fixed waypoints, the agent requests first-person RGB images in real time based on its current pose, so the same route looks different whenever obstacles are present. Instruction generation uses a two-stage pipeline: turn directions are first computed from the recorded trajectory as hard constraints, and a vision-language model converts first-person video plus those constraints into landmark-grounded text, which is then manually cleaned. The continuous-mode baseline COVL-RL supplies the policy that closes the loop: at each timestep it takes the global instruction, the current image, and a history of past images and actions, and outputs one of seven driving actions; this lets the authors measure how much of the difficulty comes from actions (granularity) versus perception of moving objects (dynamic elements).","core_discovery":"DaViNCi is the first outdoor VLN dataset that simultaneously introduces continuous and dynamic factors. Paths are generated by tracking autonomous vehicles driving freely in CARLA, instructions are produced from first-person video and trajectory-derived turn events by a two-stage VLM pipeline followed by manual clean-up, and at execution time the same path is rendered with randomly spawned vehicles and pedestrians so that visual input changes from trial to trial. The authors validate the dataset in two modes: discrete, where existing outdoor VLN methods score 30–40 percent success versus 48–60 percent on Touchdown and Map2Seq; and continuous, where the COVL-RL baseline, an online RL agent conditioned on instruction, current image, and trajectory history, scores 5.8–18.2 percent success. They further show that adding dynamic elements from 0 to 30 vehicles and pedestrians degrades success on every tested map, while step size (3, 4, or 5 meters) has only a weak, non-linear effect. The conclusion is that DaViNCi raises the bar for outdoor VLN and offers a benchmark closer to the conditions a deployed driving agent would face.","pith_inferences":["The two-stage instruction-generation design (trajectory-derived turn constraints first, VLM text grounding second) could be reused outside DaViNCi, for instance to produce navigational instructions from dashcam logs or other simulators, reducing hallucinated landmarks.","Because the paper finds granularity nearly irrelevant while dynamic elements cost performance, a cheaper way to stress-test static outdoor VLN agents may be to inject moving obstacles into existing discrete datasets rather than building a fully continuous one.","The monotonic loss with added traffic suggests that agents with explicit memory for object positions or ability to predict future obstacle motion could recover much of the gap; DaViNCi is a natural benchmark for testing such predictive policies.","Since every observation is generated inside CARLA with rule-based actors, the sim-to-real conclusion is only as strong as the simulator's fidelity; verification in higher-fidelity simulators or real closed-loop settings would be needed to confirm the transfer claim."],"forward_implications":["Existing outdoor VLN agents trained on static discrete datasets should expect success rates more than 10 percentage points lower when evaluated on DaViNCi in discrete mode, giving the field a harder yardstick.","In continuous mode, an agent trained on one town does not automatically generalize: success on unseen towns is generally below the seen town, so cross-map generalization remains an open problem.","Increasing dynamic elements monotonically reduces success and path completion, so traffic density is a first-order difficulty parameter for outdoor VLN benchmarks.","Action granularity (3, 4, or 5 meters per step) has little consistent effect, which suggests the continuous action space is not the primary bottleneck; visual variability caused by moving entities is.","The dataset and its COVL-RL baseline provide a reusable testbed for online, closed-loop VLN agents that must react to a changing visual world."],"supporting_citations":[{"why":"Supplies the six CARLA towns, continuous vehicle control, and the dynamic vehicle and pedestrian actors that define the dataset's continuous and dynamic properties.","marker":"[7]"},{"why":"Defines the vision-and-language navigation task and provides the R2R dataset that DaViNCi builds on as its task framing.","marker":"[2]"},{"why":"Touchdown is the main prior outdoor VLN dataset compared against; its discrete static format is the baseline DaViNCi claims to surpass.","marker":"[3]"},{"why":"Talk2Nav is an earlier outdoor VLN dataset with discrete topological graphs, supporting the paper's claim that prior outdoor work is discrete and static.","marker":"[22]"},{"why":"VLN-CE established continuous indoor VLN, the precedent DaViNCi extends to outdoor, dynamic scenes.","marker":"[11]"},{"why":"Map2Seq is an additional outdoor dataset used as a comparison benchmark for the discrete-mode experiments.","marker":"[17]"},{"why":"Mem4Nav is one of the strongest methods evaluated on DaViNCi, providing the best discrete-mode numbers the dataset must beat.","marker":"[9]"},{"why":"FLAME is another leading outdoor VLN method evaluated in the discrete benchmark, part of the comparison showing the dataset's challenge.","marker":"[26]"}],"fun_headline_variants":["First outdoor VLN dataset with dynamic obstacles and continuous actions","New outdoor VLN benchmark adds live traffic and continuous driving","DaViNCi: outdoor VLN with moving cars, pedestrians, and continuous steps","Outdoor navigation benchmark tests agents with dynamic elements","DaViNCi: continuous-action VLN dataset with unpredictable traffic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that DaViNCi captures the challenge of real outdoor driving rests on randomly spawned, rule-based vehicles and pedestrians standing in for real, unpredictable traffic; if those actors behave too simply, the measured difficulty drops may not transfer to real roads.","fun_headline_variants_meta":{"raw":{"variants":["First outdoor VLN dataset with dynamic obstacles and continuous actions","New outdoor VLN benchmark adds live traffic and continuous driving","DaViNCi: outdoor VLN with moving cars, pedestrians, and continuous steps","Outdoor navigation benchmark tests agents with dynamic elements","DaViNCi: continuous-action VLN dataset with unpredictable traffic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1414,"prompt_tokens":1017,"completion_tokens":397,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":310}},"tokens_in":633,"tokens_out":397,"duration_ms":4364,"temperature":1.0,"reasoning_tokens":310,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:22:52.972116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same continuous baseline twice on the same routes: once with moving vehicles and pedestrians, and once with identical vehicles and pedestrians placed at fixed positions. If success rates and path completion are the same in both conditions, then the dynamic property itself is not what makes DaViNCi hard, only the presence of extra objects; equivalently, replay trained agents against real recorded traffic in the same CARLA towns and check whether the baseline ranking of methods changes.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the six CARLA towns, continuous vehicle control, and the dynamic vehicle and pedestrian actors that define the dataset's continuous and dynamic properties."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Touchdown is the main prior outdoor VLN dataset compared against; its discrete static format is the baseline DaViNCi claims to surpass."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Talk2Nav is an earlier outdoor VLN dataset with discrete topological graphs, supporting the paper's claim that prior outdoor work is discrete and static."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Map2Seq is an additional outdoor dataset used as a comparison benchmark for the discrete-mode experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Mem4Nav is one of the strongest methods evaluated on DaViNCi, providing the best discrete-mode numbers the dataset must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FLAME is another leading outdoor VLN method evaluated in the discrete benchmark, part of the comparison showing the dataset's challenge."}],"review_version":1}