{"id":"27233da0-dd65-407b-8931-78ddd63b474a","arxiv_id":"1908.01046","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Reward augmentation with RSS safety rules and a trajectory dissimilarity metric helps Adaptive Stress Testing find more diverse and relevant failure scenarios for autonomous vehicles.","lead":"This paper improves a simulation-based method for finding failures in autonomous driving systems. It adds two tweaks to the search reward so that the failures found are more varied and more often the vehicle's fault.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-run comparison in Table III does not establish that generic AST discovers zero vehicle-induced or vehicle/vehicle failures; repeated-seed evaluation is needed.","rationale":"I read the paper as making an empirical claim: reward augmentation lets AST discover a larger and more expressive subset of the failure space than generic AST, supported primarily by Table III and Figure 4. The most load-bearing vulnerability is that the evidence comes from what appears to be a single stochastic run of each configuration, with no error bars or significance testing. This matters more than the RSS parameter assumption for the strongest claim, because even if RSS parameters are valid, the observed differences in failure-type counts might vanish under repeated trials. The reader noted the lack of error bars in their rationale but designated the RSS parameter validity as the weakest assumption; I partially agree with the reader but prioritize the statistical robustness. A secondary concern is circularity: the TD reward explicitly optimizes a dissimilarity objective, so finding more diverse failures is partially by construction. However, the categorical breakdown (vehicle-induced vs. pedestrian-induced vs. vehicle/vehicle) is not the exact optimized quantity, and the comparison with generic AST is still informative if the counts are reproducible. The proposed concrete test directly addresses whether the zero counts are robust; this is a practical, low-cost check that could be added in a revision. I do not think the paper should be rejected, but the conditional acceptance should explicitly require repeated-seed results for both the TD and RSS experiments. My recommendation is UNCHANGED because the reader's verdict is already CONDITIONAL, and my concern reinforces the need for that condition rather than moving the verdict to a different category.","tokens_in":7705,"tokens_out":2814,"duration_ms":33901,"concrete_test":"Re-run both the generic AST and TD-augmented AST experiments from Section IV-B using at least 20 independent random seeds, keeping all MCTS parameters, environment settings, and termination criteria identical. Record the number of failures in each category of Table III per seed. Report the mean, standard deviation, and the fraction of seeds in which generic AST finds at least one vehicle-induced vehicle/pedestrian failure or vehicle/vehicle failure. If that fraction is substantially above zero (e.g., >5%), the '0 vs 4/6' contrast is not robust, and the central claim should be weakened to a probabilistic statement. Also report the same statistics for the RSS experiment (Figure 4) to assess whether the fimp distribution shift is consistent across seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on Table III, which reports counts of failure types from what appears to be a single MCTS run for each reward condition. AST is stochastic: MCTS relies on random rollouts and the TRPO/MCTS policies are randomly initialized. Without multiple seeds, confidence intervals, or statistical tests, the zero counts in the Generic Reward column (0 vehicle-induced VP, 0 vehicle/vehicle) could be artifacts of one unlucky run. If generic AST produces those failure types in other seeds, the strongest claim (that generic AST 'did not discover them at all') is false; the correct statement would be that they are less frequent or less probable. Additionally, the TD reward explicitly maximizes a dissimilarity term among the returned trajectories, so the discovery of a more diverse set is partly by construction. The categorical breakdown in Table III is the independent evidence for 'expressiveness,' but that evidence is statistically unanchored. The RSS experiment in Section IV-A similarly compares aggregate histograms without error bars, though the RSS parameter assumption (Table I, including ρ=0 and acceleration bounds) is a secondary concern about whether the discovered improper-behavior failures are truly relevant. The lack of repeated trials is the most load-bearing issue because it directly undermines the quantitative basis for the paper's headline contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses a known limitation of Adaptive Stress Testing (AST) for autonomous vehicle validation: generic AST tends to find unavoidable collisions and repeatedly discovers similar failure modes. The authors propose two reward augmentations: (1) an RSS-based reward that gives zero terminal reward only to collisions where the vehicle's trajectory has an improper fraction above a threshold fcrit, and (2) a trajectory dissimilarity reward that adds a bonus proportional to the average distance between a new failure trajectory and previously found top failures. They evaluate the RSS augmentation with TRPO in a single-pedestrian crosswalk scenario, reporting a shift in the distribution of the improper-timestep fraction, and evaluate the dissimilarity reward with MCTS in a two-vehicle/two-pedestrian scenario, reporting counts of failure types in Table III. The authors conclude that augmented AST discovers a larger and more expressive subset of the failure space compared with generic AST, and that these failures are more relevant for validating an autonomous vehicle policy.","tokens_in":7990,"tokens_out":4097,"duration_ms":44731,"significance":"If the empirical claims hold, this is a practical and useful contribution. The paper clearly identifies a genuine bottleneck of AST (mode collapse and discovery of unavoidable collisions) and proposes two well-motivated, easy-to-implement reward modifications. The specific new failure types in Table III (vehicle-induced vehicle/pedestrian and vehicle/vehicle failures) are exactly the kinds of outcomes that matter for AV validation, and the paper states a falsifiable empirical prediction: the augmented reward will discover failure modes that generic AST does not. The manuscript does not provide formal guarantees, machine-checked proofs, or a code release, but the method description is sufficiently detailed to reimplement. The main weakness is statistical: the headline comparison rests on a single run per condition, and the quantitative claims are partly evaluated using metrics that are also the optimized objectives.","major_comments":[{"comment":"The strongest claim in the paper, that generic AST 'did not discover' vehicle-induced vehicle/pedestrian or vehicle/vehicle failures, rests on counts that appear to come from a single MCTS run per reward condition. AST is stochastic: MCTS uses random rollouts and the policies or tree statistics are randomly initialized. A single run cannot establish that generic AST produces zero failures of those types. Please repeat the experiment over multiple seeds and report means with confidence intervals, or apply a statistical test (e.g., a Poisson or bootstrap comparison) to the failure-type counts. If repeated runs yield nonzero counts for generic AST, the conclusion must be softened from 'did not discover at all' to 'discovered significantly less frequently.' This is load-bearing because the abstract and Section V directly claim a larger and more expressive subset of the failure space.","section":"Section IV-B, Table III"},{"comment":"The RSS reward directly optimizes the fraction of improper timesteps fimp: the terminal reward is zero only if fimp exceeds fcrit, and the non-terminal penalty is proportional to fimp. Observing that RSS-augmented AST produces trajectories with higher fimp is therefore partly a consequence of the objective rather than independent evidence of improved relevance. To support the claim that these failures are 'more useful,' evaluate relevance with a measure not contained in the reward, such as an independent avoidability label, human expert judgment, or a quantitative metric of whether the vehicle could have acted differently. In addition, Figure 4 shows aggregate histograms without error bars or a statistical comparison; statements such as 'the most common type of failure involves vehicle trajectories where the vehicle is not at fault' are not supported without repeated trials and a quantitative comparison.","section":"Section IV-A, Figure 4 and Eq. (6)"},{"comment":"The reported results depend on several free parameters that are not analyzed: fcrit, gamma, n, alpha, beta, and the RSS parameters in Table I (rho = 0 and the chosen acceleration bounds). These choices directly determine which trajectories are classified as improper and how much weight is placed on diversity. In particular, rho = 0 and the acceleration bounds are plausible but not validated, and the paper itself notes that 'for industrial applications, these parameters should be chosen with care.' A sensitivity analysis, or at least a discussion of how the qualitative conclusions change with these parameters, is needed to support the generality of the contribution. The stated assumption in Section III-A that higher fimp implies higher likelihood of collision is also asserted rather than tested; please provide evidence for this correlation or soften the claim.","section":"Section III, Table I"}],"minor_comments":[{"comment":"The displayed formula for the head-on safe longitudinal distance contains ambiguous notation ('v1 +v1,rho 2 rho' appears to be a typesetting error); please re-typeset it so that the product terms are clear and consistent with the definitions of v1,rho and v2,rho.","section":"Section II-B, Eq. (3)"},{"comment":"The decrease in pedestrian-induced failures from 25 (generic) to 15 (TD reward) is not discussed. Since the dissimilarity reward changes the composition of the returned set, a short discussion of this trade-off would help readers understand whether the augmentation sacrifices coverage of one failure mode to gain another.","section":"Section IV-B, Table III"},{"comment":"The paper states that the MCTS solver is run for 'the same number of iterations' but does not report the number of iterations, the UCT exploration constant, the rollout policy, or the TRPO network architecture and training hyperparameters. Please include these details for reproducibility.","section":"Section IV-B"},{"comment":"Consider presenting the fimp distributions as normalized empirical CDFs or as histograms with bin counts and error bars, rather than smoothed density curves, so that the comparison between the generic and RSS-augmented conditions is more quantitative and easier to assess.","section":"Section IV-A, Figure 4"},{"comment":"The phrases 'more expressive subset of the failure space' and 'relevant failures' are used without operational definitions. Adding a precise definition of expressiveness and a criterion for failure relevance would sharpen the claims and make them testable.","section":"Abstract and Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper is reasonably positioned as an empirical methods contribution, and the proposed reward augmentations are sensible. The main issue is statistical: the central 'did not discover at all' claim in Table III is based on a single run, and the RSS evaluation is partly circular because the metric being measured is also the objective being optimized. These are fixable with additional experiments and careful rewording. I would support publication after major revision; the scope of the fix (repeated runs, error bars, sensitivity analysis, an independent relevance measure) is well within the manuscript's reach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"One thing you should know: this is a modest, well-scoped paper. It takes the AST framework and adds two reward terms—one based on RSS responsibility classification, one based on trajectory dissimilarity—to push AST toward failures where the AV is at fault and toward a more varied set of failures. The RSS experiment is the stronger half. The trajectory diversity experiment is the weaker half: the central claim that generic AST discovers zero vehicle-induced or vehicle-vehicle failures comes from what looks like a single MCTS run, and that needs replication.\n\nWhat's new: the specific combination. RSS has been used for responsibility rules before; trajectory dissimilarity is borrowed from Liu and Schneider. Putting them into the AST reward is a straightforward but reasonable engineering move. The paper is clearly written, the experimental setup is described well enough to reproduce, and the qualitative trajectory examples in Figures 5 and 6 make the RSS effect vivid. The authors also note that RSS parameters should be chosen with care, which is honest.\n\nThe RSS results show a large shift in the f_imp histogram, and that shift is plausible even without error bars. The TD results are where I'd push back. In Table III, the zero counts are load-bearing, and a single run of MCTS is not enough to establish that generic AST 'did not discover' those failure types. It could just be an unlucky seed. Also, the TD reward explicitly maximizes trajectory dissimilarity, so the finding that TD produces more diverse trajectories is partly baked in. The categorical breakdown (vehicle-induced VP, vehicle/vehicle) is the independent evidence, but that evidence needs statistical grounding. I would like to see multiple seeds, confidence intervals, and maybe a sensitivity analysis for gamma and n. The RSS parameter assumption is a secondary concern; it's a modeling choice and the authors flag it.\n\nWho is this for: people actively working on AST or simulation-based AV validation. It's not a breakthrough, but it's a useful practical tweak. I think it deserves peer review—the idea is testable and the claim matters—but it should come back with repeated-seed data. Without that, the strong wording of the diversity claim should be softened.","headline":"Practical AST reward tweaks with a plausible RSS demonstration, but the diversity claim needs repeated-seed evidence before it can be taken literally.","tokens_in":8501,"tokens_out":2753,"would_cite":true,"duration_ms":26859,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By augmenting the Adaptive Stress Testing reward with Responsibility-Sensitive Safety and trajectory-dissimilarity terms, this paper shows that simulation-based failure search can recover a broader, more relevant set of autonomous-vehicle…","keywords":["Adaptive Stress Testing","autonomous vehicle validation","reward augmentation","Reinforcement Learning","Responsibility-Sensitive Safety","trajectory dissimilarity","failure scenario search","simulation-based testing"],"falsifier":"Run a dense uniform random search over the same environment-action space in the same simulator and count how often the vehicle-induced pedestrian collisions and vehicle-vehicle collisions occur. If those modes appear at comparable rates under random search, then the augmented-reward result is a statement about search efficiency rather than about the existence of a larger failure space; if they never appear, the claim that these are genuine failures of the AV policy would need to be re-examined.","tokens_in":7502,"feed_emoji":"🚗","tokens_out":7283,"duration_ms":67662,"temperature":0.7,"pith_summary":"Adaptive Stress Testing (AST) searches for likely failure scenarios of an autonomous vehicle by framing the scenario search as a Markov decision process and solving it with reinforcement learning. The paper's claim is that AST alone concentrates on collisions the vehicle cannot avoid and keeps rediscovering the same failure mode, so the authors modify the reward function in two ways: they use Responsibility-Sensitive Safety (RSS) rules to reward collisions in which the vehicle behaved improperly, and they add a trajectory-dissimilarity term that rewards each new failure for being unlike failures already found. In a simulated crosswalk scenario, these augmentations produce a wider and more useful set of failures: with the RSS reward, nearly all collisions involve a nontrivial fraction of improper vehicle responses, and with the dissimilarity reward, the search finds vehicle-caused pedestrian collisions and vehicle-vehicle collisions that the generic reward never found. If correct, this gives validation engineers a way to turn simulation testing toward the failure modes that actually reveal policy weaknesses.","feed_headline":"Stress-test reward tweaks uncover failures generic search misses","feed_subtitle":"With RSS and trajectory-dissimilarity rewards, adaptive stress testing finds crashes the vehicle caused.","key_machinery":"The load-bearing object is the augmented reward function of the AST Markov decision process. Generic AST gives $R(s)=0$ for a critical state, $-\\alpha-\\beta D(r_v,r_p)$ when the episode times out without a collision, and $-M(a|s)$ during the episode, where $M$ is the Mahalanobis distance penalizing unlikely environment actions. The RSS augmentation replaces the critical set with $E_\\mathrm{RSS}=\\{\\tau \\mid \\tau\\in E,\\ f_\\mathrm{imp}(\\tau)>f_\\mathrm{crit}\\}$ and replaces the terminal distance term with a penalty proportional to the fraction of improper timesteps, biasing the solver toward collisions the vehicle could have avoided. The trajectory-dissimilarity augmentation rewards a failure trajectory with $\\frac{\\gamma}{\\mu}\\sum_i D(\\tau_s,\\tau_i)$, where $D$ averages the Euclidean distance between $n$ normalized segment centers of mass of the new trajectory and each previously returned failure trajectory, so the solver is pushed away from failure modes it has already found. These two mechanisms, alone or in combination, are what carry the argument that reward augmentation changes the discovered failure space.","core_discovery":"The central discovery, stated on the paper's own terms, is that the reward function in AST controls not only whether failures are found but which failures are found, and that domain knowledge can be injected through that reward to make the search return failures that matter. The RSS augmentation restricts the critical set to trajectories whose fraction of improper vehicle timesteps, $f_\\mathrm{imp}(\\tau)$, exceeds a threshold $f_\\mathrm{crit}$, and uses $\\beta f_\\mathrm{imp}(\\tau)$ as the penalty for trajectories that do not reach that set; in the single-pedestrian TRPO experiment, the trajectories found by this reward have substantially higher improper-response fractions than those found by generic AST. The trajectory-dissimilarity augmentation changes the reward for reaching the critical set to $\\frac{\\gamma}{\\mu}\\sum_{i=1}^{\\mu} D(\\tau_s,\\tau_i)$, the average dissimilarity between the new failure trajectory and previously found failures, where $D$ is computed from normalized segment centers of mass. In the two-vehicle, two-pedestrian MCTS experiment, generic AST returned 25 failures, all pedestrian-induced collisions with a stopped vehicle, while the dissimilarity-augmented reward returned 15 pedestrian-induced collisions, 4 vehicle-induced vehicle/pedestrian collisions, and 6 vehicle/vehicle collisions, showing a larger and more expressive subset of the failure space.","pith_inferences":["A testable extension the authors leave implicit is to apply the RSS reward to label the other agents' behavior as well as the AV's, so AST could be used to find failures caused by pedestrians or other vehicles, not only failures the AV should have prevented.","The fixed coefficient $\\gamma$ in the dissimilarity reward could be annealed over the search horizon; a time-varying weight would let the solver first explore broadly and then concentrate on separating the most similar remaining failure modes, which the paper's constant-weight setup does not address.","The paper measures diversity by spatial trajectory shape; a complementary metric would define diversity by failure mechanism (sensor-noise-induced versus control-induced), which might correlate more directly with which subsystem of the autonomy stack needs fixing.","Because the RSS parameters (including $\\rho=0$ and the chosen acceleration bounds) are simulator-specific, a practical deployment would need to calibrate them to the actual vehicle and pedestrian dynamics; the set of 'relevant' failures will shift if those parameters change."],"forward_implications":["Validation runs with AST can now report a portfolio of failures across multiple modes, so an engineer learns about several weaknesses in one campaign rather than one repeated weakness.","Failures found with the RSS reward are more likely to be cases where the autonomous vehicle's own behavior is blameworthy, which points directly at places where the driving policy needs improvement.","The trajectory-dissimilarity term can be applied to any black-box simulator that exposes trajectories, so the diversity mechanism is not tied to the specific crosswalk scenario.","The number of failures returned by AST is an incomplete measure of validation value; the distribution of failure types and their responsibility classification matters as much as the count.","Combining the RSS and dissimilarity rewards in one framework should, if the paper's mechanism is right, yield a set of failures that is simultaneously diverse and vehicle-induced, which is the natural next step the authors identify."],"supporting_citations":[{"why":"Defines the Adaptive Stress Testing method as an MDP-based search solved with reinforcement learning.","marker":"[8]"},{"why":"Provides the autonomous-vehicle crosswalk scenario and the IDM-controlled system-under-test that the experiments extend.","marker":"[11]"},{"why":"Supplies the Responsibility-Sensitive Safety rules used to classify vehicle behavior as proper or improper.","marker":"[9]"},{"why":"Supplies the trajectory dissimilarity measure based on normalized segment centers of mass used in the diversity reward.","marker":"[16]"},{"why":"Provides the Monte Carlo Tree Search solver used in the trajectory-dissimilarity experiments.","marker":"[14]"},{"why":"Provides the Trust Region Policy Optimization solver used in the RSS reward experiments.","marker":"[15]"},{"why":"Defines the Intelligent Driver Model that controls the autonomous vehicle under test.","marker":"[12]"},{"why":"Defines the Mahalanobis distance used in the negative action-likelihood term of the reward.","marker":"[13]"}],"fun_headline_variants":["Smarter stress-test rewards expose failure scenarios generic search misses","Reward tweaks in adaptive stress testing unearth diverse vehicle failures","Augmented rewards expand the failure set found by self-driving stress tests","Domain-aware rewards improve adaptive stress testing for self-driving validation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The RSS reward depends on the assumption that the chosen Responsibility-Sensitive Safety rules and parameters—including a zero response time and the specified acceleration bounds—correctly say when the autonomous vehicle is to blame in a collision; if that responsibility label is wrong, the 'relevant' failures it highlights are mislabeled and no more useful than the generic ones.","fun_headline_variants_meta":{"raw":{"variants":["Smarter stress-test rewards expose failure scenarios generic search misses","Reward tweaks in adaptive stress testing unearth diverse vehicle failures","Augmented rewards expand the failure set found by self-driving stress tests","Domain-aware rewards improve adaptive stress testing for self-driving validation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000763,"raw_usage":{"total_tokens":3391,"prompt_tokens":957,"completion_tokens":2434,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":2363}},"tokens_in":573,"tokens_out":2434,"duration_ms":18194,"temperature":1.0,"reasoning_tokens":2363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:23:57.656252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a dense uniform random search over the same environment-action space in the same simulator and count how often the vehicle-induced pedestrian collisions and vehicle-vehicle collisions occur. If those modes appear at comparable rates under random search, then the augmented-reward result is a statement about search efficiency rather than about the existence of a larger failure space; if they never appear, the claim that these are genuine failures of the AV policy would need to be re-examined.","supporting_citations":[{"cited_title":"Adaptive Stress Testing of Airborne Collision Avoidance Systems,","cited_arxiv_id":null,"evidence_quote":"Defines the Adaptive Stress Testing method as an MDP-based search solved with reinforcement learning."},{"cited_title":"Adaptive Stress Testing for Autonomous Vehicles,","cited_arxiv_id":null,"evidence_quote":"Provides the autonomous-vehicle crosswalk scenario and the IDM-controlled system-under-test that the experiments extend."},{"cited_title":"Similarity Measurement of Moving Object Trajectories,","cited_arxiv_id":null,"evidence_quote":"Supplies the trajectory dissimilarity measure based on normalized segment centers of mass used in the diversity reward."},{"cited_title":"A Survey of Monte Carlo Tree Search Methods,","cited_arxiv_id":null,"evidence_quote":"Provides the Monte Carlo Tree Search solver used in the trajectory-dissimilarity experiments."},{"cited_title":"Trust Region Policy Optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the Trust Region Policy Optimization solver used in the RSS reward experiments."},{"cited_title":"Con- gested Traffic States in Empirical Observations and Microscopic Simulations,","cited_arxiv_id":null,"evidence_quote":"Defines the Intelligent Driver Model that controls the autonomous vehicle under test."},{"cited_title":"On the generalized distance in statis- tics,","cited_arxiv_id":null,"evidence_quote":"Defines the Mahalanobis distance used in the negative action-likelihood term of the reward."}],"review_version":1}