{"id":"380971bd-c635-4c5e-87f6-daf92b40ec6c","arxiv_id":"2501.06113","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper proposes and partially demonstrates a sequential MIL, HIL, and Vehicle-in-Virtual-Environment testing pipeline for autonomous driving functions aimed at pedestrian safety.","lead":"This paper describes a staged testing pipeline for autonomous driving safety functions that runs the same algorithm through simulation, hardware-in-the-loop, and a real vehicle inside a virtual world before public roads. It shows preliminary tests of a deep reinforcement learning braking agent at the first two stages, but the full pipeline has not yet been run end to end.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed MIL-HIL-VVE pipeline is never exercised end-to-end: the VVE stage tests only manual-driving synchronization, and the conclusion's 'all tests confirm' claim is contradicted by the paper's own statement that the full DRL collision-avoidance test is still in progress.","rationale":"The reader's stated weakest assumption is the fidelity of the single-track/Modified Dugoff vehicle model for transfer from MIL to HIL to VVE. That is a real downstream risk, but the more immediate load-bearing problem is that the pipeline is never completed in the paper: the VVE stage contains no evaluation of the DDQN agent at all. The authors explicitly acknowledge this in the Conclusion and in the VVE Test section ('additional tests will be conducted and progressively incorporated into the paper'). This is a claim-without-verification that breaks the central claim as stated, and it is located in the manuscript itself. I therefore differ from the reader on which assumption is weakest, while agreeing with the overall REJECT verdict. The proposed concrete test is the missing experiment that would settle the concern: if the trained agent is run through VVE on the same scenario and meets safety metrics comparable to MIL/HIL, the pipeline claim becomes supported; if not, the transfer/fidelity concern becomes decisive. Because the paper currently contains neither the experiment nor a defensible substitute, the verdict remains REJECT and no verdict change is needed.","tokens_in":15608,"tokens_out":3122,"duration_ms":28851,"concrete_test":"Run the already-trained DDQN collision-avoidance agent in the VVE architecture shown in Figure 9 on the same two-pedestrian crossing scenario used in the MIL and HIL tests, replacing manual driving with agent-generated control commands while virtual pedestrian motion is fed into the real-vehicle MABX. Report safety metrics such as minimum TTZ, minimum distance to pedestrian, and stop-before-crosswalk success rate, and compare them with the MIL and HIL results. If the agent completes the VVE scenario with comparable safety metrics, the integrated-pipeline claim would gain direct support; if it cannot, or if the metrics degrade substantially, the transfer gap and the model-fidelity concern would be confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the sequentially integrated MIL, HIL, and VVE pipeline comprehensively develops and evaluates autonomous driving functions, with the supporting conclusion that 'All these tests confirm the feasibility and effectiveness of our proposed experimental methods.' This support is missing at the load-bearing joint: the VVE test reported in 'Vehicle-In-Virtual-Environment Test' (Figure 18) evaluates only manual-driving trajectory synchronization between the real and virtual vehicles. The DDQN collision-avoidance agent that was trained in the MIL stage and evaluated in the HIL stage is never run in the VVE stage. The manuscript itself states in the Conclusion: 'Due to time constraints, a full test of the deep reinforcement learning (DRL)-based collision avoidance algorithm is still in progress and will be included in the updated version of the paper.' Thus the paper explicitly flags the missing end-to-end demonstration. Without the VVE evaluation of the trained agent, the sequential pipeline MIL→HIL→VVE is not completed for the same AD function, so the claimed effectiveness of the integrated methodology rests on a future experiment rather than on reported evidence. This is an internal inconsistency between the claim and the presented results, independent of any question about model fidelity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a staged testing and development pipeline for autonomous driving functions, combining Model-in-the-Loop (MIL), Hardware-in-the-Loop (HIL), and Vehicle-in-Virtual-Environment (VVE) testing before public road deployment, with a focus on vulnerable road user (VRU) safety. The authors present an extended single-track vehicle model with a Modified Dugoff tire model, train a Double Deep Q-Network (DDQN) collision-avoidance agent in a CARLA-based MIL setup, evaluate it in a HIL setup, and demonstrate a VVE motion-synchronization test using manual driving. The conclusion states that all tests confirm the feasibility and effectiveness of the proposed pipeline, while also noting that a full test of the DRL-based collision-avoidance algorithm in the VVE stage is still in progress.","tokens_in":15820,"tokens_out":3641,"duration_ms":35405,"significance":"If the integrated MIL-HIL-VVE pipeline were fully demonstrated, it would be a practically valuable contribution to safe and resource-efficient validation of VRU-related autonomous driving functions, potentially reducing the need for risky public-road testing during development. The manuscript gives a detailed account of the vehicle dynamic model and the DDQN training setup, and it is transparent about the current lack of a complete DRL-in-VVE experiment. However, the paper's central claim of pipeline effectiveness is not yet supported by the reported evidence, because the VVE stage only demonstrates manual-driving synchronization and does not exercise the trained collision-avoidance agent that was used in the MIL and HIL stages.","major_comments":[{"comment":"The VVE test reported in the 'Vehicle-In-Virtual-Environment Test' section (Figure 18) evaluates only manual-driving trajectory synchronization between the real and virtual vehicles. The DDQN collision-avoidance agent trained in the MIL stage and evaluated in the HIL stage is never run in the VVE stage. The Conclusion explicitly states: 'Due to time constraints, a full test of the deep reinforcement learning (DRL)-based collision avoidance algorithm is still in progress and will be included in the updated version of the paper.' Because the same AD function is not carried through MIL, HIL, and VVE, the claimed MIL-to-HIL-to-VVE integration is not demonstrated. The statement 'All these tests confirm the feasibility and effectiveness of our proposed experimental methods' is therefore internally inconsistent with the reported results. The authors should either include the VVE evaluation of the trained DRL agent or clearly restrict the paper's claims to a partial demonstration of the pipeline components.","section":"Vehicle-In-Virtual-Environment Test; Conclusion"},{"comment":"The MIL section states that the vehicle dynamic model 'closely mirrors the dynamics of a real vehicle' and that training results are 'highly reliable and can be easily applied to HIL and VVE testing environments.' The Future Work section, however, acknowledges that the bicycle model was chosen 'because our primary objective was to demonstrate the effectiveness of the proposed testing pipeline rather than to achieve high-fidelity vehicle dynamics simulation' and that singularities at zero speed 'restrict the effectiveness of MIL testing.' These statements are in tension, and the claimed transferability from MIL to HIL and VVE is not supported by any quantitative comparison of the simulation model against real-vehicle data. The authors should either provide such a validation or temper the transferability claims to match the acknowledged low-fidelity modeling choice.","section":"Model-in-Loop Test; Future Work"},{"comment":"The Conclusion says 'We then showcased how to evaluate traditional path-following control algorithms using the HIL setup,' but the HIL experiment reported in the 'Hardware-in-Loop Test' section (Figure 17) is an evaluation of the DDQN-based agent's speed-tracking and braking behavior, not a traditional path-following controller. This discrepancy makes it unclear which AD function actually completed the HIL stage and which function is claimed to have progressed toward VVE. The authors should clarify whether the HIL results correspond to the DRL agent or to a separate classical controller, and align the Conclusion accordingly.","section":"Conclusion; Hardware-in-Loop Test"}],"minor_comments":[{"comment":"The opening sentence of the Conclusion repeats the phrase 'In order to address challenges caused by urbanization and' twice; this should be corrected.","section":"Conclusion"},{"comment":"The DQN citations appear as '[30-31]' in the text, but the actual Mnih et al. references are numbered [35] and [36]; the in-text citation numbers should be corrected.","section":"References; Deep Reinforcement Learning Based Collision Avoidance"},{"comment":"The manuscript introduces 'time-to-collision-zones (TTZ)' earlier but later defines 'TTZ' as 'Time-To-Collision'; please use one consistent definition and abbreviation throughout.","section":"Deep Reinforcement Learning Based Collision Avoidance; Experiments and Results"},{"comment":"Several figures (Figures 7, 8, 9, 10, 11, 12, and 18) are explicitly taken from the authors' prior work (references [16] and [38]). The manuscript should state more clearly which elements are new in this submission and which are reused from earlier publications.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's incremental contribution is the sequential integration of MIL, HIL, and VVE, but that integration is exactly the part not yet demonstrated: the VVE test does not run the trained DRL agent. The reliance on the authors' own prior work for both the VVE architecture and the DDQN agent is acceptable if the new contribution is the integrated demonstration, which is currently missing. A revision that adds the promised VVE DRL experiment, clarifies the HIL subject, and tempers the 'all tests confirm' claim could make the paper publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. The reader's assessment is right, and the stress-test note lands: the paper's central claim is the sequential MIL-HIL-VVE pipeline, but the pipeline is never executed as a whole. The VVE section reports only manual-driving trajectory synchronization (Figure 18), and the conclusion says \"All these tests confirm the feasibility and effectiveness,\" then immediately says the full DRL-based collision avoidance test is still in progress. That is a claim-without-verification. The joint that would connect HIL to VVE -- running the trained DDQN agent in the VVE setup -- is exactly the piece missing.\n\nWhat is actually new and good: the authors lay out a sensible four-stage testing sequence (MIL -> HIL -> VVE -> public road) and this explicit sequencing is not in their cited prior work. The extended single-track model with Modified Dugoff tire model and wheel-rotation dynamics is described carefully with equations, and the MIL training of the DDQN agent is shown with a reward curve and TTZ plots. The HIL speed-tracking result is a reasonable sanity check. The VVE manual synchronization result, while only a trajectory overlap plot, validates the coordinate-frame synchronization mechanism. For a reader wanting to see how a lab puts these pieces together, the implementation details (Scalexio, MABX, Ethernet UDP, CARLA) are informative.\n\nWhere it's soft: the central claim is not supported. The paper is an engineering status report, not a validated methodology. The vehicle model fidelity is also acknowledged by the authors as simplified (bicycle model, singularities at zero speed), which weakens the claim that MIL training transfers to the real vehicle. That caveat is in the Future Work section, so it isn't hidden, but it does undercut the \"all tests confirm\" statement. There is also heavy reliance on the group's own prior VVE papers [15,16] and the DDQN agent [38]; the new contribution is the sequencing, which is legitimate but incremental.\n\nWho is this for? Practitioners in automated-driving testing labs who want a concrete description of a MIL-HIL-VVE setup. It is not a refereed validation of the integrated pipeline. I would not cite it as evidence that the pipeline works, but I might point someone to it as a starting point for VVE implementation details.\n\nRecommendation: desk-reject in current form, but invite a resubmission after the DRL-in-VVE test is actually run and reported. It's a work-in-progress paper that overclaims its current results. If an editor sends it to review, the reviewers would likely ask for exactly the missing end-to-end experiment. Save the referee time until that exists.","headline":"The paper is an honest status report, but the load-bearing result -- the integrated MIL-HIL-VVE pipeline -- is never run end-to-end; the VVE stage only shows manual trajectory sync, and the authors admit the DRL-in-VVE test is still in progress.","tokens_in":16397,"tokens_out":2841,"would_cite":false,"duration_ms":26136,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A staged MIL-HIL-VVE pipeline can validate pedestrian-safe autonomous driving before public road testing.","keywords":["autonomous driving","vulnerable road user safety","Vehicle-in-Virtual-Environment","hardware-in-the-loop","model-in-the-loop","deep reinforcement learning","collision avoidance","vehicle dynamics"],"falsifier":"Run the MIL-trained DDQN agent in the VVE setup with a real pedestrian walking across the crosswalk, and measure the virtual time-to-collision and the real braking profile; if the vehicle fails to keep time-to-collision above four seconds as it did in MIL, or if the real trajectory diverges from the simulated one at low approach speed, the claimed transfer and pipeline effectiveness are contradicted.","tokens_in":15381,"feed_emoji":"🚗","tokens_out":9470,"duration_ms":77996,"temperature":0.7,"pith_summary":"This paper proposes a development and evaluation pipeline for autonomous driving functions that runs sequentially through Model-in-the-Loop (MIL) simulation, Hardware-in-the-Loop (HIL) testing, and Vehicle-in-Virtual-Environment (VVE) testing before any public road deployment. The intended payoff is that vulnerable road users—pedestrians in particular—are never involuntarily exposed to experimental self-driving algorithms. The pipeline is demonstrated with a double deep Q-network (DDQN) agent trained to brake for pedestrians in the MIL stage, then checked on real control hardware in HIL, while the VVE stage synchronizes a real vehicle's motion into a virtual traffic scenario. The paper claims all these tests confirm the feasibility and effectiveness of the proposed methods, with the DRL-based collision avoidance test in VVE stated as still in progress.","feed_headline":"Three-stage pipeline takes pedestrian crash tests off public roads","feed_subtitle":"Simulator-trained agents face real hardware and a real car in one virtual world, protecting pedestrians.","key_machinery":"The key mechanism is the Vehicle-in-Virtual-Environment (VVE) synchronization loop: a real vehicle operating in a safe open space has its position and heading measured by an RTK GPS unit, passed to the onboard MicroAutobox controller, and streamed over Ethernet UDP to an in-vehicle PC running an Unreal Engine-based CARLA virtual environment, where a frame transformation maps the real motion into the virtual vehicle's motion; virtual sensor data then flows back to the real control unit so the vehicle reacts to virtual pedestrians and other actors. This loop is what lets real vehicle dynamics and real pedestrian behavior participate in repeatable, editable traffic scenarios without any public-road exposure. The supporting machinery includes the extended single-track vehicle model with longitudinal and lateral dynamics, a Modified Dugoff tire model and wheel rotation model that generate tire forces, and the DDQN agent whose state space combines an occupancy grid, ego-vehicle status, path waypoints, and time-to-collision zones.","core_discovery":"The central claim is that combining MIL, HIL, and VVE into one sequential testing pipeline can comprehensively develop and validate autonomous driving functions for vulnerable road user safety, replacing the risky practice of continuing development on public roads. In the MIL stage, a DDQN-based agent trained in a CARLA virtual environment against an extended single-track vehicle model with a Modified Dugoff tire model learns to decelerate and stop for crossing pedestrians, converging after roughly 800 episodes and keeping time-to-collision above four seconds. In HIL, the same trained agent tracks the desired braking profile in real time on a MicroAutobox controller connected to the vehicle model over CAN and Ethernet UDP. In VVE, a real vehicle's RTK-GPS position and heading are transformed into the virtual world so its trajectory overlaps with the virtual vehicle's, allowing real pedestrians equipped with a Bluetooth low-energy mobile app to act as virtual road users. The paper states that all these tests confirm the feasibility and effectiveness of the proposed experimental methods, and that the full DRL collision avoidance test in the VVE environment is still in progress.","pith_inferences":["The paper's results demonstrate each stage individually, but the end-to-end claim that one DRL agent transfers unchanged through MIL, HIL, and VVE still depends on the in-progress VVE collision avoidance experiment; until that runs, the pipeline's full transfer is an inference.","A quantitative synchronization error between the real and virtual vehicle trajectories would strengthen the VVE validation; the paper currently reports only a visual overlap.","If the near-zero-speed singularities of the bicycle model appear in practice, the agent's emergency braking behavior at low approach speeds could differ between simulation and the real vehicle, so a higher-fidelity vehicle model would make the pipeline's transfer claim more robust."],"forward_implications":["Developers can validate VRU collision avoidance functions through simulation, hardware, and real-vehicle-in-virtual-environment stages before any public road testing, cutting cost and risk.","Rare and safety-critical traffic scenarios (e.g., a pedestrian stepping out) can be tested repeatably by editing the virtual environment rather than waiting for real-world encounters.","The pipeline supports multi-actor tests: real pedestrians using a Bluetooth low-energy phone app can be synchronized into the virtual scene for V2P collision avoidance.","If the transfer claim holds, a DRL agent trained entirely in MIL can be evaluated on real hardware (HIL) and in a real vehicle (VVE) without retraining or performance loss.","Passing all three stages would raise confidence that an algorithm is ready for a final, shorter public road validation phase."],"supporting_citations":[{"why":"Supplies the hardware-in-the-loop simulator architecture with MicroAutobox, CAN, and Ethernet UDP used in the HIL stage.","marker":"[14]"},{"why":"Origin of the Vehicle-in-Virtual-Environment method and its synchronization of real vehicle motion into a virtual environment.","marker":"[15]"},{"why":"Source of the VVE architecture, implementation structure, and test vehicle used for the VVE experiments.","marker":"[16]"},{"why":"Provides the Bluetooth low-energy mobile pedestrian app that broadcasts pedestrian position and heading into the virtual V2P scenario.","marker":"[17]"},{"why":"Source vehicle dynamic model equations that the extended single-track lateral model adapts.","marker":"[18]"},{"why":"Defines the modified Dugoff tire formula used to generate longitudinal and lateral tire forces.","marker":"[19]"},{"why":"Introduces Double Q-learning, the base algorithm that the DDQN agent's overestimation mitigation builds on.","marker":"[37]"},{"why":"Describes the proposed DDQN neural network structure and collision avoidance algorithm used as the demonstration use case.","marker":"[38]"}],"fun_headline_variants":["Real car, virtual world, real pedestrians: no road risk","VVE: real car meets virtual pedestrians, no road risk","Safe crash tests: real car in virtual world with real pedestrians","Three-stage pipeline: real car, virtual pedestrians, no public-road tests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline depends on the assumption that the low-order bicycle vehicle model with Modified Dugoff tires is accurate enough that a braking policy trained on it in MIL will still behave correctly on the real vehicle in HIL and VVE without retraining.","fun_headline_variants_meta":{"raw":{"variants":["Real car, virtual world, real pedestrians: no road risk","VVE: real car meets virtual pedestrians, no road risk","Safe crash tests: real car in virtual world with real pedestrians","Three-stage pipeline: real car, virtual pedestrians, no public-road tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000967,"raw_usage":{"total_tokens":4141,"prompt_tokens":997,"completion_tokens":3144,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":3071}},"tokens_in":613,"tokens_out":3144,"duration_ms":23682,"temperature":1.0,"reasoning_tokens":3071,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:05:06.924583+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the MIL-trained DDQN agent in the VVE setup with a real pedestrian walking across the crosswalk, and measure the virtual time-to-collision and the real braking profile; if the vehicle fails to keep time-to-collision above four seconds as it did in MIL, or if the real trajectory diverges from the simulated one at low approach speed, the claimed transfer and pipeline effectiveness are contradicted.","supporting_citations":[{"cited_title":"Guvenc, B","cited_arxiv_id":null,"evidence_quote":"Source vehicle dynamic model equations that the extended single-track lateral model adapts."}],"review_version":1}