{"id":"134fcc48-7805-4e98-83cd-79053c69ecd0","arxiv_id":"2501.06132","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A vision-language model chooses which self-driving cars serve which passengers and which cars should plan together, then consensus ADMM plans their trajectories in parallel, beating heuristic dispatch rules in CARLA simulations.","lead":"This paper builds a simulated self-driving ride-hailing system that uses a vision-language model to assign cars to passengers and to split cars into small collision-risk groups, then solves each group's trajectory optimization in parallel. The result is a demonstration that combining a language model with an optimization-based planner can improve response and completion rates in a simulated city.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Safety claim rests on unverified assumption that the VLM grouping catches every collision-risk CAV pair; a single missed pair removes all coupling constraints, and the paper provides no completeness evidence.","rationale":"The reader's weakest assumption is exactly the load-bearing concern I identify: the safety of the overall system reduces to the VLM's ability to identify every potentially colliding CAV pair, because only grouped pairs receive collision-avoidance constraints through the subgraph OCP. My review of the algorithm (Section V-C and Algorithm 1) confirms there is no independent geometric verification or fallback coupling between ungrouped vehicles. The experimental evidence does not close this gap: Fig. 11 shows a single run per method with no error bars, Table II reports no repeated-seed trials or statistical tests, and no rule-based grouping baseline is compared, so we cannot attribute the favorable distance penalty to the VLM's grouping ability versus the specific scenario geometry. The paper is otherwise coherent and the system architecture is plausible; the issue is addressable with additional logging and controlled baselines. Therefore the conditional verdict is appropriate, and my read does not change it.","tokens_in":25797,"tokens_out":4737,"duration_ms":49001,"concrete_test":"Log the VLM grouping output at every planning step for all five Table II settings. Post-hoc, compute the set of CAV pairs whose minimum distance over the next planning horizon (2 s, 20 steps) falls below a safety threshold (e.g., 2 m) and verify every such pair was placed in a common subgraph at the preceding grouping step. Additionally, run the same scenarios with a deterministic rule-based grouping (e.g., all pairs with predicted closest approach below a threshold or within 15 m) and compare distance penalties and collision rates. If any near-collision pair was ungrouped, or if the rule-based grouping achieves equal or lower distance penalty, the safety benefit of the VLM grouping is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central safety claim—smallest distance penalty and robustness for future AMoD—depends entirely on the VLM-based collision risk evaluation in Section V-C. The VLM partitions CAVs into subgraphs by outputting lists of IDs; only CAVs placed in the same subgraph receive mutual collision-avoidance constraints in the OCP (Eq. 9) and subsequent ADMM iterations. Any pair not grouped together is planned independently, so a missed pair has no constraint that could prevent a collision. The paper gives no evidence that the VLM is complete: it does not compare against a rule-based grouping baseline, it does not report post-hoc checks of whether all near-colliding pairs were grouped, and the distance penalty in Fig. 11 is a single-run metric without error bars. Furthermore, GPT-4o is a closed, non-deterministic model, and no formal verification or fallback geometric check is provided. The ablation study (Table III, Fig. 12) isolates BEV input and foundation model choice, but not the grouping mechanism itself, so the contribution of VLM-guided graph evolution to safety is not isolated. Without completeness evidence, the framework cannot be claimed to be robustly safe, even if the specific logged runs happened to avoid collisions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoDriveVLM, a framework that couples a VLM-based dispatching coordinator with VLM-guided subgraph generation and consensus ADMM for decentralized cooperative motion planning of connected autonomous vehicles in an AMoD system. The dispatching coordinator receives annotated BEV images and textual prompts, uses chain-of-thought reasoning and top-K memory retrieval, and outputs CAV–passenger assignments. A separate VLM invocation analyzes the current BEV and outputs lists of CAV IDs that should be grouped into subgraphs; only vehicles in the same subgraph share collision-avoidance constraints in the OCP solved by ADMM. The authors evaluate the framework in CARLA Town10 across five vehicle/request settings, report response and completion rates, average task response/completion times, and a distance-penalty metric, and compare against four rule-based dispatching baselines. They also ablate the BEV input and foundation model choice and study the few-shot memory count. The central claim is that CoDriveVLM achieves the best response and completion rates in most settings and the smallest distance penalty, demonstrating feasibility and robustness for future AMoD systems.","tokens_in":26114,"tokens_out":3569,"duration_ms":36303,"significance":"If the results are substantiated, the paper makes a useful contribution by showing a concrete way to insert a VLM into both the dispatching and the motion-planning loop of an AMoD system, and by combining that with a scalable decentralized ADMM scheme. The open-source release of the code is a strength that supports reproducibility. However, the significance is currently tempered by the evaluation design: the VLM's role in the safety-critical collision-risk grouping is not isolated against a rule-based grouping baseline, the numerical comparisons are based on single simulation runs without seed variation, and the distance-penalty safety metric lacks error bars. These limitations mean the paper currently demonstrates feasibility in specific logged scenarios rather than robust superiority.","major_comments":[{"comment":"The safety claim depends on the VLM grouping being complete: only CAV pairs placed in the same subgraph receive mutual collision-avoidance constraints in the OCP of Eq. (9), so a single missed pair is planned without any coupling constraint and may collide. The paper provides no evidence of grouping completeness, no comparison against a rule-based or geometric grouping baseline, and no post-hoc check of whether all near-colliding pairs were grouped. Moreover, Fig. 11 reports the distance penalty from what appears to be a single simulation run without error bars or statistical tests. Without this evidence, the statement that CoDriveVLM achieves the best safety performance and robustness is not supported beyond the particular logged runs.","section":"Section V-C and Fig. 11"},{"comment":"All reported results appear to come from one simulation trial per environmental setting. The '±' values in Table II are standard deviations across passenger requests within a single run, not trial-to-trial variability, so they do not quantify stochastic variation of the simulation or of the VLM. There are no reruns with different random seeds, no confidence intervals, and no significance tests. Consequently, claims such as 'best response rate and completion rate in most of the environment settings' are not backed by statistical evidence, especially where the margins over baselines are small (e.g., the 15×30 and 25×40 rows). The authors should provide multiple seeds per setting, report mean and variance across seeds, and state the number of runs.","section":"Table II and Fig. 11"},{"comment":"The few-shot number K is selected by comparing performance on the same evaluation settings that are later used for the main comparison in Table II. For example, Fig. 12 shows that K=3 gives the best overall performance, and that value is then used in the comparative study. This constitutes test-set tuning, which can inflate the reported performance. A separate validation split or cross-validation over scenarios should be used for selecting K, or the paper should explicitly disclose that the reported numbers include hyperparameter selection on the test scenarios.","section":"Section VI-C and Fig. 12"},{"comment":"The distance definition is inconsistent. Eq. (6) defines d_ij as the Euclidean distance between vehicle i and passenger j, but the human messages in Figs. 9 and 10 state that 'The distance matrix is calculated based on the Manhattan distance,' and the worked example in Fig. 9 uses Manhattan distances (e.g., vehicle 6 at [77.5,24.9] to passenger 17 at [106.0,50.9] yields 54.5 by Manhattan but 38.6 by Euclidean). Section VI-B also refers to 'conditional Manhattan distance' for the baseline from [47]. This inconsistency affects the reproducibility of the dispatching prompts and makes it unclear which distance the VLM actually receives, which in turn limits the comparability of the dispatching results.","section":"Eq. (6) and Figs. 9–10"}],"minor_comments":[{"comment":"The notation in Eq. (2) is inconsistent: the set R includes 'Gj' in the text but the tuple lists 'Aj' twice and omits 'Gj' in the formal expression; please correct this and define all boolean flags.","section":"Section III-B, Eq. (2)"},{"comment":"The ablation in Table III uses the 20-vehicle/40-request setting, but it is not stated whether this is the same single trial as in Table II or a separate run; please clarify the experimental protocol and report variability.","section":"Table III and Section VI-C"},{"comment":"The hardware description is unclear: 'NVIDIA RTX 4090 GPU with 24 × 4 GB of graphic memory, spread across four units' should be rephrased to state the number of GPUs and their total memory per GPU.","section":"Section VI-A"},{"comment":"The summation index in Eq. (10) is written as 'PN i=1' without defining N at that point; the sum should be over the vehicles in the subgraph, so 'Ns' or a clearly defined set should be used.","section":"Section V-D, Eq. (10)"},{"comment":"Several typos appear in the reprinted prompts: 'numers' should be 'numbers', 'pasenger' should be 'passenger', 'analize' should be 'analyze', and 'spacial' should be 'spatial'.","section":"Figs. 9 and 10"},{"comment":"The description of the 'conditional Manhattan distance' grouping baseline attributed to [47] is too sparse to be replicated; please specify the exact rule used for the baseline grouping.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the idea of using VLMs for both dispatching and subgraph formation is timely. The main risk is not the mathematical formulation, which follows standard consensus ADMM, but the strength of the empirical evidence for the central safety and efficiency claims. The missing rule-based grouping baseline and the absence of multiple-seed trials are the load-bearing gaps; these can be addressed experimentally and would substantially improve the paper. The inconsistent distance definition should also be fixed before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a serious integration paper, not a breakthrough. The new bit is using a VLM for both dispatching and collision-risk subgraph generation feeding consensus ADMM cooperative planning. That combination is not in the cited literature, and the paper gives a detailed algorithm, a CARLA evaluation, ablations on BEV input and foundation models, and a code link. The central architecture is coherent.\n\nCredit where due: the authors are clear about what is theirs versus prior work. The ADMM planner builds on their own improved consensus ADMM; the VLM dispatching builds on LMMCoDrive. The contribution is the closed-loop integration and the VLM-guided graph evolution. The math in the OCP/ADMM section looks consistent, aside from a few notation typos (e.g., the index slip in Eq. 13b).\n\nThe soft spots are real but not fatal. The safety claim depends on the VLM flagging every CAV pair that might collide; only grouped pairs receive mutual avoidance constraints. The paper provides no completeness evidence: no rule-based grouping baseline, no post-hoc check of near-collision pairs, no fallback geometric check. Given that the baseline comparison explicitly attributes baseline collisions to missed pairs, this is a load-bearing assumption for the safety result. Also, the quantitative evaluation is single-run: no seed variation, no statistical tests, and Fig. 11's distance penalty is one run without error bars. The ablation selects K (few-shot number) using Fig. 12 on the same evaluation, which is a mild hyperparameter tuning on test. There is also a distance-definition inconsistency: Eq. (6) defines Euclidean distance, but the human message in Fig. 9 says Manhattan. Minor, but easy to fix.\n\nThese are all addressable. The paper would be stronger with repeated seeds, a non-VLM grouping baseline, and a clear limitation statement about VLM completeness plus a geometric fallback check.\n\nThis is for researchers working on AMoD, LLM/VLM decision-making, or distributed motion planning. A serious referee should spend time on it; the integration is novel and the code is available. It should go to peer review, with a request for the missing baselines and statistical rigor. I would not cite it in my own work in the next year, but it deserves a proper review.","headline":"A plausible VLM+ADMM integration for AMoD dispatching and cooperative planning, but the safety claim rests on an untested assumption that the VLM groups all collision-risk pairs.","tokens_in":26622,"tokens_out":2362,"would_cite":false,"duration_ms":24192,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoDriveVLM claims that a single vision-language model can both dispatch robotaxis and decide which pairs of them must avoid each other, and that feeding only those pairs into a parallel ADMM trajectory optimizer beats rule-based…","keywords":["vision-language model","autonomous mobility-on-demand","cooperative motion planning","consensus ADMM","collision risk evaluation","dispatching","bird's-eye view","in-context learning"],"falsifier":"Re-run the reported urban scenarios with the VLM collision grouping replaced by a deterministic checker that flags every CAV pair within a set distance and closing-speed threshold, then compare the minimum inter-vehicle distance in each time slot; also scan the VLM-generated trajectories for any two vehicles that came closer than the safety margin while their IDs never appeared together in an output list. Either finding would show whether the VLM grouping, rather than the ADMM layer, produces the reported safety margin.","tokens_in":25636,"feed_emoji":"🚖","tokens_out":10914,"duration_ms":99531,"temperature":0.7,"pith_summary":"The paper claims that one vision-language model (VLM) can run the two decisions an Autonomous Mobility-on-Demand (AMoD) system must make continuously: which connected autonomous vehicle (CAV) should serve which passenger request, and which pairs of CAVs are at risk of colliding. The dispatching assignment comes from a VLM that reads an annotated top-down image plus text describing distances, waiting times, and road layout; the collision-risk evaluation comes from a second VLM pass that outputs lists of CAV IDs, and each list becomes a subgraph whose vehicles solve a cooperative trajectory optimization together. Only vehicles in the same subgraph exchange information, and the optimization is carried out by a parallel consensus ADMM solver. The authors report that this integrated pipeline gets the best response and completion rates in most of their urban simulator settings and keeps the smallest inter-vehicle distance penalty, arguing that VLM reasoning can make simultaneous dispatching and collision avoidance both feasible and robust for future robotaxi fleets.","feed_headline":"Robotaxi fleets get one AI brain for dispatch and collision checks","feed_subtitle":"CoDriveVLM pairs visual AI dispatching with parallel planning and beats rule-based schedulers on speed and safety.","key_machinery":"The load-bearing mechanism is VLM-guided subgraph evolution: a VLM reads a bird's-eye-view image containing all active CAVs with their IDs, headings, velocities, and the road layout, and outputs lists of CAV IDs whose spatial relationship suggests possible collision. Each list becomes a subgraph, and only CAVs inside the same subgraph receive mutual collision-avoidance constraints in the trajectory optimization, which keeps the optimization sparse and the communication local. The optimization engine is dual consensus ADMM, which decomposes each subgraph's optimal control problem into per-vehicle LQR subproblems coupled only through consensus variables exchanged with neighboring vehicles; subgraphs are solved in parallel with no information exchange between them. The dispatching coordinator uses the same VLM architecture with chain-of-thought prompting and top-K memory retrieval to assign free vehicles to passenger requests, so the same perception stack feeds both the scheduling and the risk-grouping decisions.","core_discovery":"The central claim is that a VLM-ADMM hybrid can replace the usual sequential simplification, schedule first and then plan paths on precomputed routes, with a closed loop that handles scheduling, routing, and mutual avoidance together. On the dispatching side, the VLM uses chain-of-thought prompting with few-shot memory retrieval to weigh distance, waiting time, and road geometry, and it outputs CAV-passenger pairs. On the planning side, the same VLM reads a bird's-eye-view image of active CAVs and outputs groups of vehicles that may collide; those groups define subgraphs, and each subgraph's trajectory optimization is solved independently in parallel by dual consensus ADMM with iterative LQR updates. The loop re-plans on a receding horizon and triggers new dispatching only when free vehicles and pending requests coexist. The paper's experimental claim is that this integrated system outperforms distance-first, idle-first, first-come-first-served, and mixed-first dispatch on response rate and completion rate in most tested configurations, and reports the smallest distance penalty, meaning the closest approach between any two CAVs stayed farthest from collision range.","pith_inferences":["Inference: replacing the VLM collision grouping with a deterministic geometric pair-checker would isolate where the safety margin comes from; if the checker matches the reported distance penalty, the VLM is not the source of the improvement.","Inference: the framework's real-world viability depends on VLM inference latency and cost, so measuring per-call response time against the 2-second planning horizon would test whether the closed loop can run online rather than in batch simulation.","Inference: the same 'let a VLM choose which agents interact, then optimize only those subgraphs' pattern transfers to other multi-robot coordination problems, such as warehouse fleets or intersection management, wherever full pairwise coupling is too expensive."],"forward_implications":["If the VLM reliably flags every collision-prone pair, the ADMM layer only has to couple small subgraphs, so fleet-wide motion planning remains parallel and tractable as the number of vehicles grows.","Because dispatching is event-triggered and re-evaluated whenever free vehicles and pending requests coexist, assignments can adapt to newly spawned requests rather than following a fixed precomputed schedule.","The ablation without the bird's-eye-view image shows a sharp drop in response and completion rates, which implies that the multimodal image input is doing real work for the dispatching agent.","The few-shot memory experiments imply that in-context learning improves dispatching up to a point, with the best results at three retrieved memories and degradation beyond that.","The distance-penalty results imply that the VLM's subgraph selection, rather than the optimizer alone, is what prevents close approaches between CAVs in the tested scenarios."],"supporting_citations":[{"why":"It supplies the sparse consensus-ADMM formulation for large-scale CAV cooperative motion planning, and its conditional-Manhattan-distance grouping is the comparison baseline for the safety evaluation.","marker":"[47]"},{"why":"It provides the dual-consensus-ADMM update rules that the iterative LQR subproblems in the paper follow.","marker":"[54]"},{"why":"It provides the open urban driving simulator used for all closed-loop dispatching and motion-planning experiments.","marker":"[55]"},{"why":"It supplies the pre-trained image and text encoders used to compute retrieval similarity for few-shot memory selection.","marker":"[50]"},{"why":"It contributes the chain-of-thought prompting template that structures the VLM's reasoning before it outputs dispatching pairs and risk subgraphs.","marker":"[51]"},{"why":"It inspires the fast/adaptive-frequency hybrid architecture that combines VLM reasoning with optimization-based planning.","marker":"[49]"}],"fun_headline_variants":["AI brain unifies robotaxi dispatch and collision avoidance","CoDriveVLM merges VLM scheduling with ADMM path planning","Robotaxis get one-loop dispatch and cooperative motion planning","Vision-language model coordinates robotaxi fleets end-to-end","Unified AMoD dispatching and planning via VLM and ADMM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The safety result rests on the vision-language model never missing a pair of cars that could collide, because a missed pair receives no collision-avoidance constraint and the paper does not compare the VLM's grouping with a deterministic rule-based grouping.","fun_headline_variants_meta":{"raw":{"variants":["AI brain unifies robotaxi dispatch and collision avoidance","CoDriveVLM merges VLM scheduling with ADMM path planning","Robotaxis get one-loop dispatch and cooperative motion planning","Vision-language model coordinates robotaxi fleets end-to-end","Unified AMoD dispatching and planning via VLM and ADMM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000719,"raw_usage":{"total_tokens":3287,"prompt_tokens":1065,"completion_tokens":2222,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":681,"completion_tokens_details":{"reasoning_tokens":2136}},"tokens_in":681,"tokens_out":2222,"duration_ms":17283,"temperature":1.0,"reasoning_tokens":2136,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:05:33.052877+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the reported urban scenarios with the VLM collision grouping replaced by a deterministic checker that flags every CAV pair within a set distance and closing-speed threshold, then compare the minimum inter-vehicle distance in each time slot; also scan the VLM-generated trajectories for any two vehicles that came closer than the safety margin while their IDs never appeared together in an output list. Either finding would show whether the VLM grouping, rather than the ADMM layer, produces the reported safety margin.","supporting_citations":[{"cited_title":"Improved con- sensus admm for cooperative motion planning of large-scale connected autonomous vehicles with limited communication,","cited_arxiv_id":null,"evidence_quote":"It supplies the sparse consensus-ADMM formulation for large-scale CAV cooperative motion planning, and its conditional-Manhattan-distance grouping is the comparison baseline for the safety evaluation."},{"cited_title":"Distributed and constrained H2 control design via system level synthesis and dual consensus ADMM,","cited_arxiv_id":null,"evidence_quote":"It provides the dual-consensus-ADMM update rules that the iterative LQR subproblems in the paper follow."},{"cited_title":"CARLA: An open urban driving simulator,","cited_arxiv_id":null,"evidence_quote":"It provides the open urban driving simulator used for all closed-loop dispatching and motion-planning experiments."},{"cited_title":"Learning transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"It supplies the pre-trained image and text encoders used to compute retrieval similarity for few-shot memory selection."},{"cited_title":"Chain-of-thought prompting elicits reasoning in large language models,","cited_arxiv_id":null,"evidence_quote":"It contributes the chain-of-thought prompting template that structures the VLM's reasoning before it outputs dispatching pairs and risk subgraphs."}],"review_version":1}