Pith. sign in

REVIEW 4 major objections 7 minor 28 references

Developement of Reinforcement Learning based Optimisation Method for Side-Sill Design

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Reinforcement learning beats genetic algorithm for side-sill design

desk verdict A worthwhile engineering case study, but the central RL-vs-GA claim is undermined by a sign error in the objective/reward definition. read the letter →

arxiv 2411.09499 v1 pith:ID3I7Q6C submitted 2024-11-14 cs.LG

Aditya Borse , Rutwik Gulakala , Marcus Stoffel This is my paper · ORCID
classification cs.LG
keywords reinforcementlearningcrashworthinesssidesillmulti-objectiveoptimizationsurrogatemodelfiniteelementsimulationinversedesignwallthickness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a reinforcement-learning optimizer can handle an inverse multi-parameter, multi-objective crashworthiness design problem: choosing seven wall thicknesses of a multi-cell side sill to maximize energy absorption while limiting mass. The authors train an advantage actor-critic (A2C) agent in a custom environment where the reward rewards higher combined energy absorption and penalizes mass, first using a regression surrogate trained on 310 finite-element simulations, then coupling the environment directly to the finite-element solver. If the approach is right, engineers could replace expensive population-based search over crash simulations with an agent that explores thickness parameters directly and adapts to new objective targets without restarting the whole optimization. The reported result is a design absorbing 1417 J at 14.49 kg, higher on energy than the genetic-algorithm baseline (1370 J at 15.59 kg) and comparable to network inversion.

What carries the argument

The central mechanism is a custom reinforcement-learning environment in which the state is the seven wall thicknesses and each action changes one thickness by a fixed step. The reward function is $R=(\bar f_1-M_1)-0.5\bar f_2$, with both objectives scaled to [1,100] before the reward is computed, so the agent maximizes absorbed energy relative to a user-supplied ideal target while penalizing mass. The agent is an advantage actor-critic (A2C) policy-gradient method trained with a model-free reinforcement-learning implementation: first against a three-hidden-layer regression surrogate (trained on 310 finite-element simulations with the Adam optimizer), and later in a coupled mode where the environment calls the finite-element solver directly for each state, with the termination step limit reduced to 20 and the ideal objective array raised to [825, 625, 14].

What would settle it

Take the trained regression surrogate and evaluate it on a grid of thickness combinations that were not part of the 310 training simulations, then run those exact designs through the finite-element solver: if the surrogate's ranking of candidate designs disagrees with the finite-element rankings often enough that the RL agent's chosen path changes, the claimed optimization gain would be an artifact of the surrogate rather than a real improvement.

Watch

Extended reading notes

Core claim

The central claim is that the reinforcement-learning environment, coupled with the explicit finite-element solver, can solve the side-sill optimization problem. In the environment, the state is the vector of seven wall thicknesses, each action increments or decrements one thickness by its manufacturing step size, and the reward is $R=(\bar f_1-M_1)-0.5\bar f_2$, where $\bar f_1$ is the scaled combined energy absorbed by the inner sill and front part, $M_1$ is the scaled ideal energy target, and $\bar f_2$ is the scaled side-sill mass. The A2C agent first learns from a regression surrogate whose prediction residuals on a held-out test set lie within $\pm5\%$, and then the same agent is evaluated with the environment calling the finite-element solver directly, with the termination step limit reduced to 20 and the ideal objective array raised to [825, 625, 14]. The final thickness vector [1.7, 2.0, 2.0, 1.0, 3.0, 3.4, 4.0] mm gives 1417.36 J absorbed and 14.49 kg mass, which the paper reports as better than the genetic-algorithm baseline and comparable to network inversion while giving energy absorption the higher weight.

Load-bearing premise

The entire optimization stands on the regression surrogate, trained on only 310 finite-element simulations with residuals checked only on a held-out 20%, being accurate enough to guide the RL agent's policy across the full seven-dimensional thickness range, and the paper provides no coverage or sensitivity analysis to show this.

Editorial extensions

If this is right

  • Because the RL environment separates the objective array from the agent, the same trained agent can be pointed at new energy and mass targets and re-evaluated without retraining the surrogate, as the paper demonstrates with different ideal objective arrays.
  • Coupling the environment to the finite-element solver removes the surrogate's extrapolation limit, allowing the optimizer to seek energy absorption above the roughly 1400 J average present in the training database.
  • The final thickness values were identical under three different termination tolerances (T2=3, 4, and 5), suggesting the coupled RL run settled on a stable point in the parameter space.
  • For similar multi-objective problems, the pipeline is not gradient-based and learns by exploring the parameter space, so it could be applied where evolutionary algorithms struggle with more than three parameters.
  • The paper states that only A2C was tested and that other model-free RL agents and reward functions could plausibly improve the results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The scalarized reward with fixed weights $w_1=1$ and $w_2=-0.5$ merges two objectives into one; a natural extension would be to run multiple weight vectors and trace a Pareto front of thickness designs, which the single run does not deliver.
  • The peak contact force is extracted from the simulations but never used as an objective; since occupant safety is the real goal, constraining or minimizing peak contact force alongside energy and mass would be a direct test of the method's practical value.
  • Several final thicknesses (t2, t3, t4, t7) sit at the boundary of their allowed ranges, so the true optimum may lie outside the studied ranges; extending those ranges and retraining would show whether the agent is converging or simply walking to a constraint boundary.
  • The 310 training simulations were chosen randomly without a coverage or design-of-experiments analysis; an active-learning or space-filling sampling plan could make the surrogate trustworthy with fewer simulations or reveal regions where the current agent is effectively guessing.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript proposes an inverse multi-objective optimization method for a seven-parameter multi-cell side sill design. It combines finite element (FE) simulations (310 Abaqus runs), a neural-network regression surrogate, and a reinforcement learning (A2C) agent to maximize energy absorption while minimizing mass. The agent is first trained on the surrogate and later coupled directly with Abaqus for final evaluation. Results are compared with a genetic algorithm and network inversion; the coupled RL reports 1417.36 J absorbed at 14.49 kg, versus 1370.35 J at 15.59 kg for the GA.

Significance. If the methodological issues are resolved, the paper would offer a useful demonstration of RL-based multi-objective crashworthiness optimization with FE-in-the-loop validation. The combination of a regression surrogate for initial training and an FE-coupled refinement is practical and relevant, and the residual analysis on a held-out test set is a reasonable first check. However, the current evidence does not yet support the central claim: the sign inconsistency in the objective function undermines the GA comparison, the surrogate's accuracy in unexplored regions of the 7-dimensional space is not assessed, and the RL results are based on a single 20-step episode. The paper does not provide code or data, though the FE setup is described in sufficient detail for partial reproducibility.

major comments (4)
  1. [Section 3, Eq. (3.2)] The optimization problem is stated as min_T O(T) = w1*(f1(T) - M1(T)) + w2*f2(T) with w1=1 and w2=-0.5. Since f2 is the mass and the problem is minimized, the negative coefficient on f2 makes larger mass reduce O, which contradicts the text's assertion that 'the mass has to be minimised as much as possible.' If the genetic algorithm baseline described in Section 4.3 uses this expression as its fitness, then the comparison in Table 4 is invalid because the GA would be rewarded for increasing mass. The sign convention should be corrected (e.g., O = (M1 - f1) + c*f2 with c > 0) and all baselines should be re-run with the same objective used for the RL reward.
  2. [Section 4.5, Eq. (4.3)] The reward R = (f1(T) - M1(T)) - 0.5*f2(T) is maximized by the RL agent, which is equivalent to minimizing (M1(T) - f1(T)) + 0.5*f2(T). This is not the same objective as Eq. (3.2), where O is minimized with a negative f2 coefficient. The manuscript does not reconcile these two formulations. If the GA and the RL use different scalarizations, the reported advantage of RL over GA in Table 4 may be an artifact of the inconsistency rather than a genuine property of the method. The paper should state explicitly how each method's objective is derived from the corrected Eq. (3.2).
  3. [Section 5.2 and Section 4.5] The regression surrogate is validated only through residuals within ±5% on a 20% hold-out (62 of 310 simulations). The RL agent, however, queries the surrogate at arbitrary points in the 7-dimensional thickness space, and the paper itself acknowledges in Section 4.5 that the agent 'cannot estimate better energy absorption values while reducing the mass' because of the extrapolation limitations of the regression model. No coverage assessment, uncertainty quantification, or sensitivity analysis is provided. This is load-bearing for the initial RL training, because if the surrogate misranks candidate designs, the agent optimizes the wrong function. Please add an evaluation of surrogate accuracy along the RL agent's exploration trajectories or an uncertainty-aware reward.
  4. [Section 4.5 and Table 4] The coupled RL results are identical for T2=3, 4, and 5, and the evaluation consists of a single 20-step episode. This does not establish convergence or robustness; identical outputs across different termination thresholds suggest that the agent follows a deterministic trajectory rather than optimizing a stochastic policy. Repeated runs with different random seeds, and a report of variance or a convergence criterion, are needed to support the claim that the RL environment 'provided encouraging results.'
minor comments (7)
  1. [Title and Section 2 heading] The title contains a typo: 'Developement' should be 'Development', and the Section 2 heading 'optimsiation' should be 'optimisation'.
  2. [Abstract] The abstract contains a duplicated sentence: 'This challenge leads to the need for inverse multi-parameter multi-objective optimisation.' appears twice with slightly different wording.
  3. [Section 5.3] Section 5.3 uses 'KJ' and 'kJ' while Table 4 and the FE database use 'J'; please standardize the energy units throughout.
  4. [Figures 4-6] Figure 4 axis labels and Figures 5-6 labels appear garbled (strings such as '/uni00000035/...' are visible); the figures should be regenerated with proper text rendering.
  5. [Section 4.5] The ideal energy-mass arrays [800,600,13] and [825,625,14] are chosen without explaining how these values were derived; add a justification or a sensitivity study.
  6. [Section 4.5] The second termination condition T2 is described as 'the mean sum of the absolute difference'; it is unclear whether this is a mean or a sum, and the condition 'less than 5' should specify over which steps the mean is taken.
  7. [Section 4.4] The phrase 'the inverse network is then trained using the loss of the forward network' is confusing, since network inversion is implemented through gradient descent in the input space (Eq. 4.2); please revise the wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the optimization chain is self-contained and externally validated against FE simulations.

full rationale

The paper's derivation chain is not circular. The regression surrogate is trained on 310 FE simulations and checked on a held-out 20% test set (Section 5.2), so its reported +/-5% residuals are an independent performance measure rather than a fitted input. The RL reward (Eq. 4.3) is a user-defined objective that combines energy absorption and mass; the ideal energy-mass arrays are targets chosen by the user, not quantities derived from the final result. Crucially, Table 4 reports FE-validated outcomes: the RL designs (1417.36 J, 14.49 kg), network inversion (1412.95 J, 13.16 kg), and GA (1370.35 J, 15.59 kg) are all evaluated by running the actual Abaqus simulations with the obtained thickness values. Thus the central comparative claim does not reduce to the surrogate's own predictions. The self-citation to the authors' previous framework [4] is contextual and not load-bearing: the present paper describes its own surrogate, reward function, environment, and FE coupling. The sign inconsistency between Eq. 3.2 (minimize O = (f1-M1) - 0.5 f2, which rewards larger mass) and Eq. 4.3 (maximize R = (f1-M1) - 0.5 f2, which penalizes larger mass) is a correctness risk and a likely error in the stated problem formulation, but it is not a circularity: the fitted surrogate, the reward, and the FE validation remain distinct stages, and the final design is not equivalent to the input by definition.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the fidelity of the FE surrogate and the choice of the scalar objective. The weights and ideal targets are hand-picked, and the surrogate's accuracy is only checked on a held-out fraction of the same 310 simulations.

free parameters (3)
  • Objective weights w1 and w2 = w1=1, w2=-0.5
    Chosen to emphasize energy absorption over mass reduction; no justification or sensitivity analysis given.
  • Ideal objective array for coupled RL = [825, 625, 14]
    User-defined target energy (combined 1450 J) and mass (14 kg); set above the average in the FE database, leading to extrapolation.
  • Termination thresholds T2 = 3, 4, 5
    The mean absolute difference threshold that stops the episode; varied values all yielded the same design, but no analysis of sensitivity is provided.
assumptions (3)
  • domain assumption FE simulations with a 10 mm shell mesh and bilinear elastoplastic material accurately represent crash behavior of the side sill.
    The entire optimization is evaluated against these simulations; no mesh convergence or experimental validation is reported.
  • domain assumption The regression surrogate trained on 310 randomly sampled designs is accurate enough to guide the RL agent.
    Residuals are within +/-5% on the test set, but the agent is trained on the surrogate and must generalize across the design space.
  • ad hoc to paper The reward function in Eq 4.3 is a valid scalarization of the two crashworthiness objectives.
    The weights and the linear combination are chosen by the authors; alternative scalarizations are not explored.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Developement of Reinforcement Learning based Optimisation Method for Side-Sill Design." pith.science (2026). https://pith.science/paper/ID3I7Q6C

@misc{pith2026241109499,
  author       = {Pith},
  title        = {Pith review of: Developement of Reinforcement Learning based Optimisation Method for Side-Sill Design},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ID3I7Q6C}},
  note         = {Machine review of arXiv:2411.09499}
}
read the original abstract

Optimisation for crashworthiness is a critical part of the vehicle development process. Due to stringent regulations and increasing market demands, multiple factors must be considered within a limited timeframe. However, for optimal crashworthiness design, multiobjective optimisation is necessary, and for complex parts, multiple design parameters must be evaluated. This crashworthiness analysis requires computationally intensive finite element simulations. This challenge leads to the need for inverse multi-parameter multi-objective optimisation. This challenge leads to the need for multi-parameter, multi-objective inverse optimisation. This article investigates a machine learning-based method for this type of optimisation, focusing on the design optimisation of a multi-cell side sill to improve crashworthiness results. Furthermore, the optimiser is coupled with an FE solver to achieve improved results.

Figures

Figures reproduced from arXiv: 2411.09499 by the authors.

Figure 1
Figure 1. Cross-section of the side sill with the varied wall thickness [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Assmebly view of one of the models of side sill and oblique pole impact [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Data-correlation matrix of the FE simulation database [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Residual distribution for the validation of the trained regression model [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Training of SB3- A2C agent- Episodic rewards over episode [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Results of different optimisation methods and evaluation of coupled RL environment [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 20 canonical work pages

  1. [4]

    Borse, R

    A. Borse, R. Gulakala, M. Stoffel, Development of a machine learning-based design optimiza- tion method for crashworthiness analysis, Archives of Mechanics 76 (2024) 61–92.arXiv: https://am.ippt.pan.pl/am/article/viewFile/4454/pdf, doi:10.24423/aom.4454. URL https://am.ippt.pan.pl/am/article/view/4454

    doi:10.24423/aom.4454 2024

  2. [1]

    Duddeck, Multidisciplinary optimization of car bodies, Structural and Multidisciplinary Optimization 35 (2008) 375–389.doi:10.1007/s00158-007-0130-6

    F. Duddeck, Multidisciplinary optimization of car bodies, Structural and Multidisciplinary Optimization 35 (2008) 375–389.doi:10.1007/s00158-007-0130-6

    work page doi:10.1007/s00158-007-0130-6 2008

  3. [2]

    Zhang, J

    B. Zhang, J. Yang, Z. Zhong, Optimisation of vehicle side interior panels for occupant safety in side impact, International Journal of Crashworthiness 15 (2010) 617–623.doi: 10.1080/13588265.2010.484193

    work page arXiv 2010

  4. [3]

    Schäffer, R

    M. Schäffer, R. Sturm, H. Friedrich, Automated generation of physical surrogate vehicle models for crash optimization, International Journal of Mechanics and Materials in Design 15 (2019) 1–18. doi:10.1007/s10999-018-9407-8

    work page doi:10.1007/s10999-018-9407-8 2019

  5. [5]

    Q. Li, L. Wu, T. Chen, E. Li, L. Hu, F. Wang, T. Zou, Multi-objective optimization design of b-pillar and rocker sub-systems of battery electric vehicle, Structural and Multidisciplinary Optimization 64 (2021) 3999–4023.doi:10.1007/s00158-021-03073-0

    work page doi:10.1007/s00158-021-03073-0 2021

  6. [6]

    D. Xia, L. Pan, S. Di, T. Hu, J. Zhang, Crashworthiness characteristics and structural optimization of electric vehicles’ side pole impact, Proceedings of China SAE Congress 2020: Selected Papers (2022).doi:https://doi.org/10.1007/978-981-16-2090-4_33

    work page doi:10.1007/978-981-16-2090-4_33 2022

  7. [7]

    Belingardi, A

    G. Belingardi, A. Scattina, Battery pack and underbody: Integration in the structure design for battery electric vehicles—challenges and solutions, Vehicles 5 (2023) 498–514. doi: 10.3390/vehicles5020028

    work page doi:10.3390/vehicles5020028 2023

  8. [8]

    Nicoletti, F

    L. Nicoletti, F. Ostermann, M. Heinrich, A. Stauber, X. Lin, Topology analysis of electric ve- hicles, with a focus on the traction battery, Forschung auf dem Gebiete des Ingenieurwesens 84 (06 2021). doi:10.1007/s10010-020-00422-1

    work page doi:10.1007/s10010-020-00422-1 2021

Show all 28 references
  1. [9]

    C. Long, S. Chung Kim Yuen, G. Nurick, Analysis of a car door subjected to side pole impact, Latin American Journal of Solids and Structures 16 (01 2019). doi:10.1590/ 1679-78255753

    2019

  2. [10]

    S. Wang, D. Wang, Research on crashworthiness and lightweight of b-pillar based on mpso with topsis method, Journal of the Brazilian Society of Mechanical Sciences and Engineering 41 (2019) 498. doi:10.1007/s40430-019-2019-x. 13

    2019 doi

  3. [11]

    Estrada, D

    Q. Estrada, D. Szwedowicz, T. Tran, A. Rodriguez Mendez, M. C. Espinosa, O. Gómez- Vargas, G. Partida-Ochoa, Bending crashworthiness of elliptical tubes with different aspect ratio and stiffeners, The International Journal of Advanced Manufacturing Technology 120 (06 2022). do...

    2022 doi

  4. [12]

    S. Li, D. Zhou, A.-X. Pan, Integrated lightweight optimization design of wall thickness, ma- terial, and performance of automobile body side structure, Structural and Multidisciplinary Optimization 67 (05 2024).doi:10.1007/s00158-024-03810-1

    2024 doi

  5. [13]

    Horstemeyer, X

    M. Horstemeyer, X. Ren, H. Fang, E. Acar, P. Wang, A comparative study of design opti- misation methodologies for side-impact crashworthiness, using injury-based versus energy- based criterion, International Journal of Crashworthiness - INT J CRASHWORTHINESS 14 (2009) 125–138....

    2009 doi

  6. [14]

    Djamaluddin, Optimization of foam filled door sill for pure electric vehicle, International Journal of Mechanics 17 (2023) 31–37.doi:10.46300/9104.2023.17.5

    F. Djamaluddin, Optimization of foam filled door sill for pure electric vehicle, International Journal of Mechanics 17 (2023) 31–37.doi:10.46300/9104.2023.17.5

    2023

  7. [15]

    Shetty, L

    S. Shetty, L. Nilsson, Multiobjective reliability-based and robust design optimisation for crashworthiness of a vehicle side impact, International Journal of Vehicle Design 67 (2015)

    2015

  8. [16]

    Hayashi, M

    K. Hayashi, M. Ohsaki, Graph-based reinforcement learning for discrete cross-section op- timization of planar steel frames, Advanced Engineering Informatics 51 (2022) 101512. doi:10.1016/j.aei.2021.101512

    2022

  9. [17]

    K. Li, T. Zhang, R. Wang, Deep reinforcement learning for multi-objective optimization, IEEE Transactions on Cybernetics 51 (2020) 3103 – 3114. doi:10.1109/TCYB.2020. 2977661

    2020 doi

  10. [18]

    Z. Wang, S. Yao, G. Li, Q. Zhang, Multiobjective combinatorial optimization using a single deep reinforcement learning model, IEEE Transactions on Cybernetics PP (2023) 1–13. doi:10.1109/TCYB.2023.3312476

    2023

  11. [19]

    F. Zou, G. Yen, C. Zhao, Dynamic multiobjective optimization driven by inverse reinforce- ment learning, Information Sciences 575 (06 2021).doi:10.1016/j.ins.2021.06.054

    2021 doi

  12. [20]

    S. Doi, H. Sasaki, H. Igarashi, Multi-objective topology optimization of rotating machines using deep learning, IEEE Transactions on Magnetics PP (2019) 1–5.doi:10.1109/TMAG. 2019.2899934

    2019

  13. [21]

    J. Lim, C. You, I. Dayyani, Multi-objective topology optimization and structural analysis of periodic spaceframe structures, Materials & Design 190 (2020) 108552. doi:https: //doi.org/10.1016/j.matdes.2020.108552. URL https://www.sciencedirect.com/science/article/pii/S026412752030085X

    2020

  14. [22]

    Smith, ABAQUS/ Standard User’s Manual, Version 6.9, Dassault Systèmes Simulia Corp, United States, 2009

    M. Smith, ABAQUS/ Standard User’s Manual, Version 6.9, Dassault Systèmes Simulia Corp, United States, 2009

    2009

  15. [23]

    Chollet, et al., Keras,https://keras.io (2015)

    F. Chollet, et al., Keras,https://keras.io (2015). URL https://github.com/fchollet/keras

    2015

  16. [24]

    Kingma, J

    D. Kingma, J. Ba, Adam: A method for stochastic optimization, International Conference on Learning Representations (12 2014).doi:10.48550/arXiv.1412.6980. 14

    2014 doi arXiv

  17. [25]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blon- del, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning R...

    2011

  18. [26]

    Kindermann, A

    J. Kindermann, A. Linden, Inversion of neural networks by gradient descent, Parallel Com- puting 14 (3) (1990) 277–286.doi:https://doi.org/10.1016/0167-8191(90)90081-J. URL https://www.sciencedirect.com/science/article/pii/016781919090081J

    1990

  19. [27]

    Raffin, A

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, N. Dormann, Stable-baselines3: Reliable reinforcement learning implementations, Journal of Machine Learning Research 22 (268) (2021) 1–8. URL http://jmlr.org/papers/v22/20-1364.html 15

    2021

  20. [347]

    doi:10.1504/IJVD.2015.070410

    2015

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.