Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single trained policy can switch a vision-based driving agent's style at runtime, no retraining required.

desk verdict Solid first application of preference-driven MORL to vision-based end-to-end driving, but the 'dynamic adaptation' claim is not actually tested—every episode fixes the preference vector. read the letter →

arxiv 2505.05223 v2 pith:OIPKJ5NG submitted 2025-05-08 cs.RO cs.CVcs.LG

Hendrik Surmann , Jorge de Heuvel , Maren Bennewitz This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords multi-objectivereinforcementlearningpreference-drivenadaptationend-to-endautonomousdrivingstylepersonalizationruntimepreferencevision-basedPD-MORL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that one reinforcement-learned driving policy can serve many users: a continuous preference vector—weights for comfort, speed, aggressiveness, and efficiency—is fed to the same trained network, and the car changes how it drives immediately. That matters because current end-to-end driving agents typically freeze one style at training time, require repeated human feedback, or must be re-optimized for each new preference. The authors train a vision-based agent in dense mixed-traffic urban simulation and show that raising the comfort weight lowers jerk and acceleration, raising the speed weight raises velocities, and raising the aggressiveness weight increases lateral acceleration and yaw rate, while collisions and route failures stay low. If the result is right, personalization becomes a runtime input rather than a retraining problem, and one model could cover a whole spectrum of driving styles.

What carries the argument

The load-bearing mechanism is the preference-conditioned critic of the PD-MORL algorithm, with two ingredients that make one policy span all preferences. First, a preference interpolator $I(\lambda) = \lambda^p$ projects the user vector into a normalized latent space, and an angle loss $L_{\text{angle}}(\lambda_p, Q)$ penalizes directional mismatch between the projected preference and the critic's Q-vector, so the critic learns which behaviors the preference points to. Second, preference-aware hindsight experience replay resamples stored transitions with alternative preference vectors, so the policy sees every preference region during training. Preferences enter the update through element-wise multiplication $\lambda \odot Q_j$. Around this, the paper wraps a five-dimensional reward vector: the first dimension is a static core reward for collision avoidance, lane keeping, and route progress, and the other four are the style rewards, which lets preference modulation happen on top of a fixed safety and navigation base.

What would settle it

A concrete check would be to run the trained agent on an unseen map of the same simulator with identical traffic seeds under comfort weight 0 versus 1; if mean jerk no longer drops significantly at the high comfort setting, while route completion remains unchanged, the preference adaptation fails to generalize beyond the training towns. A sharper version is a forced-choice video study: if human raters cannot distinguish high-aggressiveness from high-comfort runs at better than chance, the behavior changes are artifacts of the reward shaper rather than perceived driving style.

Watch

Extended reading notes

Core claim

The central discovery is that a preference-conditioned multi-objective reinforcement learning agent, trained once on a five-dimensional reward vector, learns a single policy that covers the entire continuous preference space rather than one operating point. At evaluation, the user's four preference weights are part of the state, the actor-critic acts accordingly, and the resulting behavior shifts in the intended directions: the reported differences are statistically significant, with speed weight raising mean velocity, comfort weight lowering mean jerk, and aggressiveness weight raising acceleration and yaw rate, while core driving metrics remain intact. The best configuration, which uses semantic-segmentation input and a truncated ResNet-18 encoder, attains a Driving Score of 42±17, route completion of 57±16 percent, and Preference Alignment of 11.3 degrees, indicating that the critic's Q-vector points close to the user's preference direction. In the paper's own terms, this is the first demonstration of preference-driven MORL applied to vision-based end-to-end autonomous driving.

Load-bearing premise

The load-bearing assumption is that the hand-written reward terms for aggressiveness, comfort, speed, and efficiency faithfully capture what humans mean by those driving styles; if users' comfort or sportiness is not reflected in weighted jerk, throttle variability, and yaw rate, the demonstrated behavioral changes do not amount to personalization.

Editorial extensions

If this is right

  • A passenger or route planner can change the preference weights mid-journey and the same policy will change its driving style, with no retraining and no swap to a different model.
  • A single model trained once on the preference space can serve many users with differing tastes, avoiding the need to store or train one policy per style.
  • Because each style objective is tied to concrete measurable behavior, an operator can audit the personalization: higher comfort should visibly lower jerk, higher speed should raise velocity, and higher aggressiveness should raise yaw rate.
  • The reward decomposition into a static core plus dynamic preferences means the safety-critical parts of driving are not re-negotiated when the style changes, which is what the low collision and lane-invasion rates support.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The current evidence for personalization is measured with the same reward terms the training optimizes; a forced-choice human study comparing recorded runs would test whether passengers actually perceive the intended style differences.
  • Because the policy responds across a continuous weight interval, one could define a context scheduler that maps road type or traffic density to preference weights and switches automatically segment by segment; the paper does not evaluate that use.
  • A natural extension is to learn the preference vector itself from sparse user feedback, since the policy already parameterizes the full preference manifold and only the mapping from user to weights is missing.
  • The Preference Alignment angle, which measures how well the critic's Q-values point along the user preference, could serve as a general diagnostic in other multi-objective control tasks beyond driving.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes a preference-driven multi-objective reinforcement learning (PD-MORL) agent for end-to-end, vision-based autonomous driving in CARLA. Preferences over aggressiveness, comfort, speed, and efficiency are encoded as a four-dimensional weight vector that conditions a single TD3-based policy, trained with an angle loss and hindsight experience replay. The evaluation has two parts: a 'preference reflection' analysis showing that varying a preference weight shifts mean velocity, acceleration, and jerk in the expected directions, and an ablation over visual encoders and input modalities reporting driving and preference metrics. The paper claims runtime adaptation to changing preferences without retraining.

Significance. The contribution is potentially useful: continuous preference conditioning in a high-dimensional, vision-based driving task with code release is a timely extension of PD-MORL, and the use of statistical tests for behavioral shifts is a positive feature. However, the evidence as presented supports preference-conditioned behavior under fixed per-episode preferences rather than true runtime adaptation during an episode, and the preference-related metrics are constructed from the same reward and Q-values that training optimizes. The significance of the personalization claim therefore depends on additional evidence.

major comments (4)
  1. [III-E, IV-B, Fig. 4] All experiments fix the preference vector for the entire episode: training samples lambda once at episode start, and Figure 4's x-axis is the fixed sampled weight. No experiment changes lambda mid-episode, and the HER mechanism relabels experiences with alternative constant vectors rather than exposing the policy to within-episode switches. Since lambda is part of the state, the policy could in principle react to a change, but the abstract and conclusion promise 'dynamically adapts according to changing preferences,' and this central claim is not demonstrated. Please add intra-episode switching experiments (e.g., changing lambda at a known waypoint and comparing behavior before and after the switch, including safety and route-completion metrics) or revise the claims to inter-episode adaptation.
  2. [III-F, III-G] Preference Score is defined as the preference-weighted return of the hand-designed reward vector, and Preference Alignment is the cosine angle between the preference vector and critic Q-values, which the angle loss explicitly aligns during training. Both metrics therefore measure consistency with the reward design, not whether the behavior matches an independent notion of the corresponding driving style. The behavioral shifts in Figure 4 largely confirm that the reward shaping influences control, which is a weaker statement than personalized driving. Please either provide an external validation (user ratings, independent style metrics, or human-driver comparisons) or clearly restrict the claims to reward-defined style objectives.
  3. [IV-B, Table I] The evaluation contains no comparison against a non-preference-conditioned baseline, a fixed-preference policy trained for a single weight, or an existing MORL/end-to-end driving method. The ablations show that encoder and modality choices matter, but they do not establish that the proposed single-policy approach is preferable to simpler alternatives or that it is competitive with prior work. At minimum, a fixed-preference TD3 and a single-objective baseline should be included with the same scenarios and metrics.
  4. [III-B, III-D] The state and critic description is internally inconsistent: the reward vector is five-dimensional and the critic outputs Q in R^5, while the preference vector is in R^4 and the paper refers to element-wise multiplication lambda \odot Q. It is not stated how the core reward is included in the scalarization or how a four-dimensional lambda multiplies a five-dimensional Q. Please specify the exact preference-conditioning mechanism, including the treatment of the core objective, or the implementation cannot be assessed.
minor comments (4)
  1. [Fig. 2] Figure 2 caption states a 'six-dimensional reward vector,' whereas Section III-F defines a five-dimensional vector; please reconcile.
  2. [Table I] The column header 'Ours Feature Extractor Ablation Vision Modality Ablation' combines three panels into one line; the table should separate the two ablations with clear panel headers.
  3. [III-F] The efficiency reward formula uses vmax and amax without defining their values or units; the source of these constants should be given, for example in Table II.
  4. [Fig. 4] Define the exact quantities plotted on the y-axes (mean per-episode velocity and acceleration magnitudes, and jerk magnitude), including the unit of jerk, and state whether the 480 episodes were collected in one scenario or across all seven evaluation scenarios.

Circularity Check

2 steps flagged · score 5.0 of 10

Preference Score and Preference Alignment are defined from the exact reward and Q-objectives that training optimizes, but the core behavioral-adaptation claim rests on independent velocity/acceleration/jerk measurements; circularity is partial, not central.

  1. self definitional [Section III-G, 'Driving and Preference Metrics' (Preference Score definition)]
    "To assess preference adherence, we report the Preference Score, defined as PS = 1 T PT t=1 ω⊤rPrefs,t. It is calculated as the preference-weighted return over the episode, where ω is the user-defined preference vector and rPrefs,t the vector of preference-specific rewards at time t."

    PS is the undiscounted average of the very preference-reward vector rPrefs whose lambda-weighted sum the PD-MORL policy is trained to maximize (Section III-B: preferences are incorporated via element-wise multiplication lambda⊙Q, with an extra angle loss). A high PS therefore mostly confirms that reward optimization succeeded on the hand-designed terms; calling it 'preference adherence' presupposes that rPrefs faithfully operationalizes human comfort, speed, aggressiveness, and efficiency. The metric is self-definitional with respect to the training objective, not an independent behavioral or user-level validation.

  2. self definitional [Section III-B (angle loss) and Section III-G (Preference Alignment definition)]
    "This allows augmenting the loss function with an angle loss term Langle(λp, Q), which penalizes directional mismatches between the interpolated preference vector λp and the critic's output Q. ... Preference Alignment quantifies how well the critic's Q-values reflect the intended preferences, computed as PA = 1 N PN t=1 g(ωt,Q(st,at,ωt)), where g(·) is the cosine-based angular deviation (in degrees) between preference vector and Q-values."

    PA is exactly the cosine-based angular deviation between lambda and Q-values, and the angle loss Langle(lambda_p, Q) is added to the training loss to penalize that deviation. Reporting low PA as evidence that the critic 'faithfully encodes' user preferences is therefore a check that the training objective was optimized, not a test of whether the learned behavior matches preferences. It is a convergence diagnostic that reduces by construction to the definition of the angle loss.

full rationale

The paper's central claim—that a single policy conditioned on the preference vector lambda modulates driving style—is supported by non-circular behavioral evidence. Figure 4 and the qualitative trajectories measure raw velocity, acceleration, and jerk as functions of the preference weights, with Welch t-tests; these measurements are not defined in terms of the training objective. The circularity risk is concentrated in two self-reported metrics. Preference Score (PS) is the episode-average of the exact preference reward vector the policy is trained to maximize, so high PS confirms reward optimization rather than independently validating preference satisfaction. Preference Alignment (PA) is the cosine angle between lambda and the critic Q-values, and the training loss explicitly minimizes that angle via Langle(lambda_p, Q); low PA is therefore a convergence diagnostic. Neither metric is the sole basis of the main claim, and there is no load-bearing self-citation chain: reference [28] is only related-work support, and reference [27] is an external algorithm. The paper's 'runtime ... changing preferences' wording is stronger than what the experiments show (every evaluation episode fixes lambda), but that is an evidence gap about intra-episode changes, not a circularity. Overall: partial circularity in the preference metrics, independent core behavioral evidence, so score 5.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a large set of hand-chosen reward coefficients (Table II) that operationalize driving style preferences, plus the assumption that PD-MORL transfers to vision-based continuous control and that CARLA is a valid proxy for real driving. No sensitivity analysis is provided for the reward weights. No new physical entities are introduced.

free parameters (6)
  • Speed reward deviation weight δ_speed = 1.75
    Hand-chosen weight for the speed objective's penalty on deviation from target velocity (Table II).
  • Comfort constant bias β_b = 1.20
    Hand-chosen constant added to the comfort reward to keep it non-negative and to shape the comfort/speed trade-off (Table II).
  • Aggressiveness yaw-rate weight α_yaw = 0.30
    Hand-chosen weight on yaw rate magnitude in the aggressiveness reward (Table II).
  • Comfort jerk penalty β_jerk = 0.03
    Hand-chosen penalty on jerk magnitude in the comfort reward (Table II).
  • Preference reward auxiliary weights (α_l, α_l_acc, β_steer, β_throttle, β_v, β_long) = α_l=0.10, α_l_acc=0.20, β_steer=0.10, β_throttle=0.05, β_v=0.30, β_long=0.30
    Additional hand-chosen coefficients for acceleration, steering, and throttle variability terms in the preference rewards (Table II).
  • Core reward coefficient set (c_col, c_acc, c_brake, c_spd_high, c_idle, c_osc, c_steer, c_throttle, c_off, c_inv… = c_col=5, c_acc=0.1, c_brake=2.75, w_type=1.7, c_spd_high=0.3, c_idle=3.5, c_osc=0.2, c_steer=0.6, c_throttle=0.4…
    Hand-tuned weights that define the static core driving reward; mis-tuning could dominate or suppress preference effects (Table II).
assumptions (5)
  • domain assumption The reward hypothesis: any user preference can be adequately expressed as a weighted linear combination of scalar reward terms
    The method encodes aggressiveness, comfort, speed, and efficiency as linear reward components and a convex preference vector; if user preferences are not linearly separable or not captured by these terms, the approach fails. Section III-F.
  • domain assumption CARLA simulator is a valid proxy for real-world driving behavior and perception
    All training and evaluation are done in CARLA; the paper acknowledges real-world validation as future work. Sections III-A, IV.
  • domain assumption The PD-MORL algorithm (preference interpolator, angle loss, HER) from Basaklar et al. transfers to high-dimensional vision-based continuous control
    The paper applies PD-MORL (MO-TD3-HER) to a vision-based task with a high-dimensional state; transferability is assumed from [27]. Sections III-B, III-D.
  • domain assumption Pretrained ImageNet features in a truncated ResNet18 provide useful representations for CARLA semantic segmentation images
    The frozen-pretrained-then-finetuned ResNet is assumed to extract features sufficient for driving; this is a standard but unverified-for-CARLA assumption. Section III-D.
  • domain assumption Traffic rules (traffic lights, stop signs) can be ignored without invalidating the driving evaluation
    The paper explicitly excludes traffic light adherence; this limits the claim to scenarios without such rules. Sections III-A, III-F.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving." pith.science (2026). https://pith.science/paper/OIPKJ5NG

@misc{pith2026250505223,
  author       = {Pith},
  title        = {Pith review of: Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OIPKJ5NG}},
  note         = {Machine review of arXiv:2505.05223}
}
abstract

Human drivers exhibit individual preferences regarding driving style. Adapting autonomous vehicles to these preferences is essential for user trust and satisfaction. However, existing end-to-end driving approaches often rely on predefined driving styles or require continuous user feedback for adaptation, limiting their ability to support dynamic, context-dependent preferences. We propose a novel approach using multi-objective reinforcement learning (MORL) with preference-driven optimization for end-to-end autonomous driving that enables runtime adaptation to driving style preferences. Preferences are encoded as continuous weight vectors to modulate behavior along interpretable style objectives$\unicode{x2013}$including efficiency, comfort, speed, and aggressiveness$\unicode{x2013}$without requiring policy retraining. Our single-policy agent integrates vision-based perception in complex mixed-traffic scenarios and is evaluated in diverse urban environments using the CARLA simulator. Experimental results demonstrate that the agent dynamically adapts its driving behavior according to changing preferences while maintaining performance in terms of collision avoidance and route completion.

Figures

Figures reproduced from arXiv: 2505.05223 by the authors.

Figure 1
Figure 1. Diverse driving situations necessitate distinct driving styles, mo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our approach enables personalized autonomous driving by integrating user preferences into a multi-objective reinforcement learning setup. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. a) Bird’s-eye view of the agent’s trajectories while navigating a T-intersection. Each trajectory reflects a distinct driving style, achieved by setting one preference weight (for speed, efficiency, aggressiveness, or com￾fort) to one, with all others set to zero. Subfigures b) - e) display additional metrics including steering, throttle, velocity, and lateral acceleration. Values are smoothed using exponential smoo… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Effect of increasing preference weights on key driving behavior metrics. Each subplot illustrates the relationship between a specific driving behavior [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Objective Reinforcement Learning for Tactical Decision Making for Trucks in Highway Traffic

    cs.LG 2026-01 conditional novelty 5.0 of 10

    A PPO-based multi-objective RL method learns a set of truck highway-driving policies spanning the trade-off among safety, driver time, and energy cost in SUMO simulation.

Reference graph

Works this paper leans on

42 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [1]

    Emma: End-to-end multimodal model for autonomous driving,

    J.-J. Hwang, R. Xu, H. Lin, W.-C. Hung, J. Ji, K. Choi, D. Huang, T. He, P. Covington, B. Sapp, J. Guo, D. Anguelov, and M. Tan, “Emma: End-to-end multimodal model for autonomous driving,” arXiv:2410.23262, 2024

    arXiv 2024

  2. [2]

    Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation,

    Z. Li, K. Li, S. Wang, S. Lan, Z. Yu, Y . Ji, Z. Li, Z. Zhu, J. Kautz, Z. Wu, et al. , “Hydra-mdp: End-to-end multimodal planning with multi-target hydra-distillation,” arXiv:2406.06978, 2024

    arXiv 2024

  3. [3]

    Exploration of the acceptability of different behaviors of an autonomous vehicle in so- called conflict situations,

    G. Nativel-Fontaine, V . Lespinet-Najib, R. Cazes, C. Dupetit, C. De Gasquet, M. Chevrie, F. A ¨ıoun, and L. Ojeda, “Exploration of the acceptability of different behaviors of an autonomous vehicle in so- called conflict situations,” Accident Analysis and Prevention, vol. 186, p. 107041, 2023

    work page 2023

  4. [4]

    Toward adaptive driving styles for automated driving with users’ trust and preferences,

    M. Natarajan, A. Kumar, and T. Misu, “Toward adaptive driving styles for automated driving with users’ trust and preferences,” in 2022 17th ACM/IEEE Intl. Conf. on Human-Robot Interaction (HRI) , 2022, pp. 940–944

    work page 2022

  5. [5]

    Dynamic preferences in multi-criteria reinforcement learning,

    S. Natarajan and P. Tadepalli, “Dynamic preferences in multi-criteria reinforcement learning,” in Proc. of the 22nd Intl. Conf. on Machine Learning, ser. ICML ’05. Association for Computing Machinery, 2005, p. 601–608

    work page 2005

  6. [6]

    User-driven adaptation: Tailoring autonomous driving systems with dynamic preferences,

    M. Zhang, J. Li, N. Li, E. Kang, and K. Tei, “User-driven adaptation: Tailoring autonomous driving systems with dynamic preferences,” in Extended Abstracts of the CHI Conference on Human Factors in Computing Systems , ser. CHI EA ’24. Association for Computing Machinery, 2024

    work page 2024

  7. [7]

    Lexicographic actor-critic deep reinforcement learning for urban autonomous driving,

    H. Zhang, Y . Lin, S. Han, and K. Lv, “Lexicographic actor-critic deep reinforcement learning for urban autonomous driving,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 4308–4319, 2023

    work page 2023

  8. [8]

    Urban driving with multi-objective deep reinforcement learning,

    C. Li and K. Czarnecki, “Urban driving with multi-objective deep reinforcement learning,” in Intl. Conf. on Autonomous Agents and Multiagent Systems (AAMAS) , IFAAMAS. IFAAMAS, 2019

    work page 2019

Show all 42 references
  1. [9]

    Multi-objective optimization for autonomous driving strategy based on deep q network,

    T. Hu, B. Luo, and C. Yang, “Multi-objective optimization for autonomous driving strategy based on deep q network,” Discover Artificial Intelligence, vol. 1, p. 11, 2021

    2021

  2. [10]

    Porf-ddpg: Learning per- sonalized autonomous driving behavior with progressively optimized reward function,

    J. Chen, T. Wu, M. Shi, and W. Jiang, “Porf-ddpg: Learning per- sonalized autonomous driving behavior with progressively optimized reward function,” Sensors, vol. 20, no. 19, 2020

    2020

  3. [11]

    Toward personalized decision making for au- tonomous vehicles: A constrained multi-objective reinforcement learn- ing technique,

    X. He and C. Lv, “Toward personalized decision making for au- tonomous vehicles: A constrained multi-objective reinforcement learn- ing technique,” Transportation Research Part C Emerging Technolo- gies, vol. 156, pp. 1–17, 2023

    2023

  4. [12]

    Navigation in urban environments amongst pedestrians using multi-objective deep reinforcement learning,

    N. Deshpande, D. Vaufreydaz, and A. Spalanzani, “Navigation in urban environments amongst pedestrians using multi-objective deep reinforcement learning,” in 2021 IEEE International Intelligent Trans- portation Systems Conference (ITSC) , 2021, pp. 923–928

    2021

  5. [13]

    Learning driving styles for autonomous vehicles from demonstration,

    M. Kuderer, S. Gulati, and W. Burgard, “Learning driving styles for autonomous vehicles from demonstration,” in 2015 IEEE Intl. Conf. on Robotics and Automation (ICRA) , 2015, pp. 2641–2646

    2015

  6. [14]

    Driving behavior modeling using naturalistic human driving data with inverse reinforcement learning,

    Z. Huang, J. Wu, and C. Lv, “Driving behavior modeling using naturalistic human driving data with inverse reinforcement learning,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 8, pp. 10 239–10 251, 2022

    2022

  7. [15]

    Driving with style: Inverse reinforcement learning in general-purpose planning for automated driving,

    S. Rosbach, V . James, S. Grosjohann, S. Homoceanu, and S. Roth, “Driving with style: Inverse reinforcement learning in general-purpose planning for automated driving,” in 2019 IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS) . IEEE, 2019, p. 2658–2665

    2019

  8. [16]

    Driving style alignment for llm-powered driver agent,

    R. Yang, X. Zhang, A. Fernandez-Laaksonen, X. Ding, and J. Gong, “Driving style alignment for llm-powered driver agent,” in 2024 IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 11 318–11 324

    2024

  9. [17]

    From words to wheels: Automated style-customized policy generation for autonomous driving,

    X. Han, X. Chen, Z. Cai, P. Cai, M. Zhu, and X. Chu, “From words to wheels: Automated style-customized policy generation for autonomous driving,” arXiv:2409.11694, 2024

    2024 arXiv

  10. [18]

    On-board vision-language models for personalized autonomous vehicle motion control: System design and real-world validation,

    C. Cui, Z. Yang, Y . Zhou, J. Peng, S.-Y . Park, C. Zhang, Y . Ma, X. Cao, W. Ye, Y . Feng, J. H. Panchal, L. Li, Y . Chen, and Z. Wang, “On-board vision-language models for personalized autonomous vehicle motion control: System design and real-world validation,” arXiv:2411.11...

    2024 arXiv

  11. [19]

    A review of personalization in driving behavior: Dataset, modeling, and validation,

    X. Liao, Z. Zhao, M. J. Barth, A. Abdelraouf, R. Gupta, K. Han, J. Ma, and G. Wu, “A review of personalization in driving behavior: Dataset, modeling, and validation,” IEEE Transactions on Intelligent Vehicles, pp. 1–22, 2024

    2024

  12. [20]

    Self-driving like a human driver instead of a robocar: Personalized comfortable driving experience for autonomous vehicles,

    I. Bae, J. Moon, J. Jhung, H. Suk, T. Kim, H. Park, J. Cha, J. Kim, D. Kim, and S. Kim, “Self-driving like a human driver instead of a robocar: Personalized comfortable driving experience for autonomous vehicles,” in Machine Learning for Autonomous Driving Workshop, Conf. on N...

    2019

  13. [21]

    Toward safe and personal- ized autonomous driving: Decision-making and motion control with dpf and cdt techniques,

    C. Huang, C. Lv, P. Hang, and Y . Xing, “Toward safe and personal- ized autonomous driving: Decision-making and motion control with dpf and cdt techniques,” IEEE/ASME Transactions on Mechatronics , vol. 26, no. 2, pp. 611–620, 2021

    2021

  14. [22]

    Personalized driving behavior oriented autonomous vehicle control for typical traffic situations,

    H. Li, W. Wei, S. Zheng, C. Sun, Y . Lu, and T. Zhou, “Personalized driving behavior oriented autonomous vehicle control for typical traffic situations,” Journal of the Franklin Institute , vol. 361, no. 10, p. 106924, 2024

    2024

  15. [23]

    Recent advancements in end-to-end au- tonomous driving using deep learning: A survey,

    P. S. Chib and P. Singh, “Recent advancements in end-to-end au- tonomous driving using deep learning: A survey,” IEEE Transactions on Intelligent Vehicles, vol. 9, pp. 103–118, 2023

    2023

  16. [24]

    Personalized car following for autonomous driving with in- verse reinforcement learning,

    Z. Zhao, Z. Wang, K. Han, R. Gupta, P. Tiwari, G. Wu, and M. J. Barth, “Personalized car following for autonomous driving with in- verse reinforcement learning,” in 2022 Intl. Conf. on Robotics and Automation (ICRA), 2022, pp. 2891–2897

    2022

  17. [25]

    Text-to-drive: Diverse driving behavior synthesis via large language models,

    P. Nguyen, T.-H. J. Wang, Z.-W. Hong, S. Karaman, and D. Rus, “Text-to-drive: Diverse driving behavior synthesis via large language models,” 2024 IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), pp. 10 495–10 502, 2024

    2024

  18. [26]

    Multi-objective end-to-end self- driving based on pareto-optimal actor-critic approach,

    T. Wang, Y . Luo, J. Liu, and K. Li, “Multi-objective end-to-end self- driving based on pareto-optimal actor-critic approach,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) , 2021, pp. 473–478

    2021

  19. [27]

    PD-MORL: Preference-driven multi-objective reinforcement learning algorithm,

    T. Basaklar, S. Gumussoy, and ¨Umit Y . Ogras, “PD-MORL: Preference-driven multi-objective reinforcement learning algorithm,” in Proc. of the 11th Intl. Conf. on Learning Representations (ICLR) , 2022

    2022

  20. [28]

    Demonstration- enhanced adaptable multi-objective robot navigation,

    J. de Heuvel, T. Sethuraman, and M. Bennewitz, “Demonstration- enhanced adaptable multi-objective robot navigation,” in Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS) , 2025

    2025

  21. [29]

    CARLA: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “CARLA: An open urban driving simulator,” inProc. of the 1st Annual Conference on Robot Learning , 2017, pp. 1–16

    2017

  22. [30]

    A practical guide to multi-objective reinforcement learning and plan- ning,

    C. F. Hayes, R. Radulescu, E. Bargiacchi, J. K ¨allstr¨om, M. Mac- farlane, M. Reymond, T. Verstraeten, L. M. Zintgraf, R. Dazeley, F. Heintz, E. Howley, A. A. Irissappane, P. Mannion, A. Now ´e, G. de Oliveira Ramos, M. Restelli, P. Vamplew, and D. M. Roijers, “A practical gu...

    2022

  23. [31]

    Prediction- guided multi-objective reinforcement learning for continuous robot control,

    J. Xu, Y . Tian, P. Ma, D. Rus, S. Sueda, and W. Matusik, “Prediction- guided multi-objective reinforcement learning for continuous robot control,” in Proc. of the 37th Intl. Conf. on Machine Learning , ser. ICML’20, 2020

    2020

  24. [32]

    End-to- end reinforcement learning for autonomous longitudinal control using advantage actor critic with temporal context,

    S. Kuutti, R. Bowden, H. Joshi, R. d. Temple, and S. Fallah, “End-to- end reinforcement learning for autonomous longitudinal control using advantage actor critic with temporal context,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC) , 2019, pp. 2456–2462

    2019

  25. [33]

    PASRL: Stabilising reinforcement learning with past action-state representation learning,

    T. Endrei, A. Lozano-Diez, and G. Cserey, “PASRL: Stabilising reinforcement learning with past action-state representation learning,” Preprint submitted to ICLR, 2024

    2024

  26. [34]

    Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,

    K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger, “Transfuser: Imitation with transformer-based sensor fusion for au- tonomous driving,” IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, vol. 45, pp. 12 878–12 895, 2022

    2022

  27. [35]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016, pp. 770–778

    2016

  28. [36]

    Model-free deep reinforcement learning for urban autonomous driving,

    J. Chen, B. Yuan, and M. Tomizuka, “Model-free deep reinforcement learning for urban autonomous driving,” 2019 IEEE Intelligent Trans- portation Systems Conference (ITSC) , pp. 2765–2771, 2019

    2019

  29. [37]

    End-to-end model-free reinforcement learning for urban driving using implicit affordances,

    M. Toromanoff, E. Wirbel, and F. Moutarde, “End-to-end model-free reinforcement learning for urban driving using implicit affordances,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 7151–7160, 2019

    2020

  30. [38]

    Prioritized experience-based reinforcement learning with human guidance for autonomous driving,

    J. Wu, Z. Huang, W. Huang, and C. Lv, “Prioritized experience-based reinforcement learning with human guidance for autonomous driving,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 1, pp. 855–869, 2024

    2024

  31. [39]

    Carla leaderboard,

    CARLA Autonomous Driving Challenge, “Carla leaderboard,” https: //leaderboard.carla.org, 2023, accessed: 2024-08-19

    2023

  32. [40]

    Privileged sensing scaffolds reinforcement learning,

    E. Hu, J. Springer, and D. Jayaraman, “Privileged sensing scaffolds reinforcement learning,” in Proc. of the Intl. Conf. on Learning Representations (ICLR), 2024

    2024

  33. [41]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. A. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through d...

    2015

  34. [42]

    Searching for MobileNetV3,

    A. G. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y . Zhu, R. Pang, V . Vasudevan, Q. V . Le, and H. Adam, “Searching for MobileNetV3,” 2019 IEEE/CVF Intl. Conf. on Com- puter Vision (ICCV) , pp. 1314–1324, 2019. Notation Value Description Preference Rewa...

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.