Pith. sign in

REVIEW 4 major objections 4 minor 60 references

A model-free nested reinforcement-learning loop jointly optimises wing geometry and flight-control policy for a flexible thermal-soaring glider, and guided design search outperforms random-design training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:04 UTC pith:TEXEBOGR

load-bearing objection A competent application of Schaff et al.'s nested GMM/PPO co-design loop to aeroelastic systems, with a useful new MuJoCo flexible-wing environment, but the flagship thermal-soaring result depends on a wing model that omits induced drag and therefore should not be trusted until that is fixed. the 4 major comments →

arxiv 2607.15329 v1 pith:TEXEBOGR submitted 2026-07-16 eess.SY cs.SY

Co-Design of Aeroelastic Systems with Deep Reinforcement Learning

Yao Cheng Li , Urban Fasel This is my paper
classification eess.SY cs.SY
keywords control co-designaeroelasticitydeep reinforcement learningproximal policy optimisationthermal soaringflexible gliderreward shapingdesign-conditioned policy
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that aircraft structure and controller should be designed together, not sequentially, and that this co-design can be achieved without a model when the system is too complex to optimise analytically. Its central demonstration is a flexible high-aspect-ratio glider whose wing area and aspect ratio are optimised jointly with a flight-control policy for a thermal-soaring mission under atmospheric uncertainty. The method trains one control policy conditioned on the sampled design parameters, while an outer loop maintains and progressively prunes a distribution over candidate designs. On three case studies—a spring-mass-damper, a pitch-plunge-flap aerofoil, and the glider—guided co-design concentrates search toward high-reward regions and achieves better reward than policies trained on randomly sampled designs. The paper also shows that reward shaping is the ingredient that makes learning stable in partially observed, stochastic environments.

Core claim

On its own terms, the paper's discovery is that a model-free nested optimisation loop can solve a control-co-design problem in aeroelasticity where design choices alter stability, controllability, and mission capability simultaneously. In the final case study, the framework jointly selects wing area and aspect ratio, which in turn set the wing's mass, inertia, and stiffness, and learns a soaring policy that explores thermals, avoids sinks, and tracks waypoints. The optimised design (wing area 0.133 m², aspect ratio 6.05) deviates sharply from the energy-minimum baseline glider (0.240 m², aspect ratio 17.25) and achieves higher mission reward. The evidence is the reward curves across five tra

What carries the argument

The load-bearing mechanism is a nested loop. The inner loop is a design-conditioned Markov decision process: the transition dynamics and the policy both depend on the sampled design parameters, and the policy is trained with proximal policy optimisation. The outer loop keeps a Gaussian mixture model over the design space, updates the mixture with a gradient step on weighted negative log-likelihood of achieved rewards, and periodically prunes the worst-performing clusters so that training focuses on promising designs. Underneath sits a parameterisable low-fidelity aeroelastic glider implemented as a multi-body physics simulation: a discretised wing with sectional masses, torsional springs for

Load-bearing premise

The load-bearing premise is that the low-fidelity discretised-wing simulation used for training is a faithful stand-in for a real flexible glider, because the paper never validates that model against a higher-fidelity aeroelastic solver, wind-tunnel data, or flight tests.

What would settle it

Take the co-designed glider (wing area 0.133 m², aspect ratio 6.05) with its trained policy and measure its thermal-soaring waypoint performance in a higher-fidelity aeroelastic simulation (for example, a geometrically nonlinear beam model with unsteady aerodynamics) or on a real small UAV in thermals; if the guided co-design policy does not beat a policy trained on randomly sampled designs in that setting, the reported advantage is an artifact of the training simulator.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the framework works in this setting, the same loop can be applied to other flexible-aircraft co-design problems—morphing wings, folding-wing UAVs, or high-altitude long-endurance platforms—where analytic design optimisation is intractable.
  • The result implies that mission-level behaviour, not just cruise efficiency, can drive structural design: the optimised glider sacrifices lift-to-drag ratio for roll agility and thermal-search capability.
  • Reward shaping should be treated as a core design input of the method, not a tuning detail; the paper shows learning collapses without it in partially observed, stochastic environments.
  • The comparison against random sampling indicates that concentrating the design distribution is what makes deep-RL co-design sample-efficient; a fixed uniform design distribution is not enough.
  • The same procedure may transfer to other black-box co-design problems where system dynamics are hard to model, such as fleet-level aircraft rollout planning, as the paper itself suggests.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • I would not bet on the specific optimised numbers (wing area 0.133 m², aspect ratio 6.05); the low-fidelity environment is not validated, so the qualitative conclusion—guided co-design outperforms random sampling—carries more weight than the particular design.
  • A direct testable extension would be to re-run the same framework with a different mission reward weighting (for example, penalising sink rate more heavily) and see whether the optimum shifts toward higher aspect ratio; if it does not, the method may be overfitting reward-shaping choices rather than trading efficiency against agility.
  • The paper's framing suggests the outer-loop design search itself could be replaced by a learned design generator or an off-policy RL algorithm, which might reduce sensitivity to reward shaping—an avenue the paper leaves open.
  • Because the glider model assumes no spanwise flow, no wake effects, and a fixed quarter-chord aerodynamic centre, the framework's real-world applicability depends on whether those simplifications hold near stall and in tight thermalling turns; measured flight data from a small flexible UAV would be the natural check.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a model-free nested co-design framework for aeroelastic systems. A design-conditioned control policy is trained with PPO in an inner loop, while an outer loop maintains a Gaussian mixture model over design parameters, updating and pruning the mixture based on achieved rewards (Algorithm 1). The framework is evaluated on three case studies of increasing complexity: a spring-mass-damper, a pitch-plunge-flap aerofoil, and a MuJoCo-based flexible glider performing a stochastic thermal-soaring waypoint mission. The authors claim that guided co-design progressively concentrates the design search toward high-performing regions and outperforms policies trained on randomly sampled designs, and that reward shaping is important for stable learning in partially observed, stochastic environments. The paper is an adaptation of Schaff et al. with aeroelastic applications as the novel contribution.

Significance. If the reported results are robust, the paper makes a useful empirical contribution by demonstrating that an existing nested RL co-design loop can be applied to aeroelastic flight systems, including a mission-level thermal-soaring task. The study has several strengths: a transparent design-conditioned MDP formulation, a warm-start mechanism to stabilize training, a comparison against random-sampling baselines in most case studies, and an appendix providing implementation details of the MuJoCo flexible-wing model. The pitch-plunge-flap case, with flutter, divergence, and control-reversal boundaries inside the design space, is a sensible stress test. However, the central 'across case studies' claim is overstated because the spring-mass-damper results show little difference from random sampling, and the flagship glider result rests on a low-fidelity aerodynamic model that omits induced drag and is not validated against a higher-fidelity reference. These issues are correctable in principle, but they affect the robustness of the main claims rather than being purely presentational.

major comments (4)
  1. [Abstract; §4.3; §5] The abstract and conclusions state that 'across these case studies' the framework outperforms policies trained on randomly sampled designs. This is contradicted by §4.1, which reports 'little difference between the reward curves' for the spring-mass-damper system, and by Figure 2, where the 95% confidence intervals overlap substantially. Please qualify the claim to the specific case studies for which an improvement is actually demonstrated, or remove the cross-case wording.
  2. [Appendix A, Eq. (32); §4.3] The sectional aerodynamic force model in Eq. (32) contains profile lift and drag only: CLi and CDi are computed from Viterna coefficients (Eqs. 33–40), with no finite-wing induced-drag term of the form C_Di = C_L^2/(π e AR). Aspect ratio enters the drag model only through Cdmax in Eq. (35), which affects post-stall behaviour, not pre-stall induced drag. In the glider case, this omission is load-bearing: the optimizer can choose AR = 6.05 because it obtains reduced roll damping and a higher optimal glide speed without incurring the induced-drag penalty that high-AR wings are designed to reduce. Since the introduction itself invokes Prandtl's lifting-line theory, the omission is an internal inconsistency in the stated aerodynamics. Please add a finite-wing induced-drag term or explicitly justify its absence, and rerun the glider optimization to report whether the optimum and the gap to the
  3. [Appendix A; §4.3] The discretised MuJoCo wing model is not validated. Equations (50)–(55) define equivalent spring stiffnesses from beam theory, but the paper does not show that the 10-section discretisation reproduces the continuous flexible-wing behaviour: no comparison is given for bending and twist convergence, trim state, flutter onset, or free-flight dynamics against a continuous aeroelastic solver or experimental data. The optimised design (Sref = 0.133, AR = 6.05) and the claim that the framework 'jointly addresses wing design, flight control, and mission-level behaviour' are therefore tied to an unvalidated simulator. At minimum, provide a convergence study in the number of spanwise sections and a representative validation of the model against the Aerosandbox baseline from [55].
  4. [§4.3, Figure 11] The glider reward curve is described as repeated over five separate training runs, but Figure 11 shows no confidence intervals or individual run curves, unlike Figures 2 and 6 for the earlier case studies. The text selects 'the best trained policy' and reports its design parameters without dispersion statistics. Given the stochastic environment and the variance visible in the other case studies, this omission makes it difficult to assess whether the co-designed glider reliably outperforms the random-sampling baseline or the baseline glider. Please report mean/CI or per-run curves for the glider, and report the distribution of optimized design parameters across runs.
minor comments (4)
  1. [Eq. (20)] In the rotation matrix R12(θ_i), the bottom-right entry is written as cos(φ_i), which appears to be a typo; it should presumably be cos(θ_i).
  2. [Algorithm 1] The line 'if step > N_freeze N_step N_env' is missing multiplication signs or parentheses, making the intended comparison ambiguous.
  3. [Appendix A; throughout] Minor typographical issues: 'dynamcis' in the first paragraph of Appendix A; 'Mujoco' is inconsistently capitalised; 'thermals' appears as 'thermals' in several places.
  4. [§4.3] The sentence 'a smaller wing reduces the wing loading and allows the aircraft to operate at a higher optimal glide speed' is physically confusing as written; a smaller wing area increases wing loading, not reduces it. Please clarify the intended mechanism.

Circularity Check

0 steps flagged

No significant circularity: the co-design loop's concentration on high-reward designs is its own objective rather than a derived prediction, and the paper's empirical comparisons are not forced by the equations.

full rationale

This is an optimization-methods paper. The outer-loop loss (Eq. 6) is a reward-weighted negative log-likelihood, so the GMM is by construction attracted to high-reward designs; the abstract's statement that the search 'progressively concentrate[s] the design search towards high-performing regions' restates this objective rather than deriving an independent prediction. The substantive claims---that the guided GMM/PPO procedure outperforms a policy trained on LHS-random designs and that reward shaping stabilizes learning in the glider case---are empirical comparisons supported by the reported training curves and baselines; they are not entailed by the definitions. The nested framework is imported from an independent prior source [32], and the aeroelastic/aerodynamic models are based on external references [53,54,58,59,60]. The only self-citation ([29], involving the corresponding author Paul Li) appears as an example of prior DRL application and as a future-work suggestion; it is not load-bearing for any technical result. The skeptic's concern about omitted induced drag in the MuJoCo model is a model-fidelity and correctness risk, not a circularity: no prediction in the paper reduces to an input by construction. Accordingly, no circular step can be exhibited under the required standard.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claim rests on the reward specification and simulator fidelity: reward weights (Table 1) are hand-tuned; the MuJoCo wing model is an unvalidated discretisation; the GMM outer loop is assumed to track high-performing designs under nonstationary rewards. No new physical entities are introduced.

free parameters (3)
  • Reward weighting coefficients for glider (Table 1) = λ_end=1.00; λ_α=1.00; λ_θ=1.00; λ_β=2.00; λ_ψ=0.10; λ_q=0.01; λ_wW=1.00; λ_hdot=2.00; λ_Δd=10.00
    Hand-tuned ('meticulous tuning', Appendix B) to prevent training collapse; these weights shape what the co-design optimizes, so the final design is contingent on them.
  • Flap-rate penalty weight λ_βdot = 0.01
    Added to aerofoil reward (Eq. 16) to damp oscillatory/flutter behaviour; without it, training would not account for flutter, per §4.2.
  • Co-design hyperparameters (N_design, N_inner, N_prune, α_prune, N_freeze, N_step) = Appendix C, Table 2
    Chosen by hand; outer-loop pruning and warm-start configuration materially affect whether guided search beats random sampling.
axioms (5)
  • domain assumption PPO's clipped surrogate objective (Eq. 4) converges to a good control policy on each design-conditioned MDP under chosen hyperparameters.
    The inner-loop success is assumed; no convergence proof or hyperparameter sensitivity analysis is given (§4).
  • domain assumption Reward-weighted GMM update (Eq. 6) drives the design distribution toward high-performing regions despite nonstationary returns from an undertrained policy.
    The outer loop weights the negative log-likelihood by episode returns; there is no analysis showing this is unbiased or that pruning selects true high performers rather than noise (§3, Algorithm 1).
  • domain assumption Quasi-steady aeroelastic model of [54] (Eqs. 13–14) with six hidden aerodynamic states is valid for the pitch-plunge-flap case.
    Borrowed without re-derivation; controls the flutter/divergence/reversal boundaries that the design optimization exploits (§4.2).
  • ad hoc to paper The discretised MuJoCo wing with equivalent torsional stiffnesses (Eqs. 50–55) and uniform sectional lift approximates the continuous flexible wing adequately for design optimization.
    Introduced in Appendix A without validation against a continuous aeroelastic solver or experimental data; this is the final case study's environment.
  • ad hoc to paper Composite reward (Eq. 17, Appendix B) encodes the mission objective without distortion; shaping terms do not dominate the waypoint-reaching objective.
    Reward weights were tuned to make training work; the paper states reward shaping is key but does not ablate the weights or show the learned behaviour is robust to them.

pith-pipeline@v1.3.0-alltime-deepseek · 19295 in / 16940 out tokens · 250714 ms · 2026-08-02T00:04:21.964875+00:00 · methodology

0 comments
read the original abstract

Control co-design considers the physical system and its controller together, enabling the strong coupling between system design and control to be uncovered and exploited. This is especially relevant in aeroelastic flight systems, where structural, aerodynamic, and control design choices jointly determine manoeuvrability and efficiency. This paper presents a model-free nested co-design framework for aeroelastic systems using deep reinforcement learning, in which a design-conditioned control policy is trained with proximal policy optimisation while an outer loop updates a distribution over candidate design parameters. The approach is evaluated on three case studies of increasing complexity: a spring-mass-damper system, a pitch-plunge-flap aerofoil, and a highly flexible high-aspect-ratio glider performing a thermal-soaring mission in a stochastic environment. Across these case studies, the framework is shown to progressively concentrate the design search towards high-performing regions and to outperform policies trained on randomly sampled designs. The results also show that reward shaping plays an important role in enabling stable learning in partially observed and stochastic environments. In the final glider case, the method jointly addresses wing design, flight control, and mission-level behaviour in the presence of aeroelastic coupling and atmospheric uncertainty. These results highlight the potential of model-free co-design for complex aeroelastic systems in which design, control, and mission objectives are tightly coupled.

Figures

Figures reproduced from arXiv: 2607.15329 by Urban Fasel, Yao Cheng Li.

Figure 1
Figure 1. Figure 1: Nested co-design framework for joint glider and controller optimisation. A design [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Smoothed reward curve of the mass-spring-damper system repeated over 5 training [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Co-design evaluation stage for pruning at training step 61440. Damping, 𝑐 Stiffness, k Stiffness, k [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Schematic of the pitch-plunge-flap aerofoil. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Smoothed reward curve of the pitch-plunge-flap system repeated over 5 training at [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The training reward landscape before the final design distribution update of the pitch [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Evolution of flutter, divergence and control reversal speeds over the iterations of the [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Root locus plot of the optimised aeroelastic system across velocity range [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Evaluating a trained policy on an example generated reference signal. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Reward curve of the thermal soaring glider repeated over five training attempts. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: A sample flight trajectory using the trained policy with its optimised system param [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Training reward landscape before the final design distribution update of the glider in soaring application at step 6029312. Figures 13 and 14 show the evolution of the design distributions during optimisation. Even at an early stage of training, the distributions shift clearly away from higher-aspect-ratio and larger wing area designs. This may be attributed both to increased roll damping and to higher in… view at source ↗
Figure 15
Figure 15. Figure 15: Example discretisation of a clamped wing in Mujoco. [PITH_FULL_IMAGE:figures/full_fig_p021_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Rear view of a 2 section discretisation of the port side wing. [PITH_FULL_IMAGE:figures/full_fig_p022_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Port side view of a 2 section discretisation. [PITH_FULL_IMAGE:figures/full_fig_p022_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Definition of transformation between body axes (black), stability axes (blue) and [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Deflection of a continuous beam under a force and moment load at its midspan. [PITH_FULL_IMAGE:figures/full_fig_p027_19.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 8 linked inside Pith

  1. [1]

    Control co-design: an engineering game changer.Advanced Control for Applications: Engineering and Industrial Systems, 2019

    Mario Garcia-Sanz. Control co-design: an engineering game changer.Advanced Control for Applications: Engineering and Industrial Systems, 2019

    2019

  2. [2]

    On the coupling be- tween the plant and controller optimization problems

    Hosam K Fathy, Julie A Reyer, Panos Y Papalambros, and AG Ulsov. On the coupling be- tween the plant and controller optimization problems. InProceedings of the 2001 American Control Conference.IEEE, 2001

    2001

  3. [3]

    Nested and simultaneous solution strategies for general combined plant and control design problems.Journal of Mechanical Design, 2019

    Daniel R Herber and James T Allison. Nested and simultaneous solution strategies for general combined plant and control design problems.Journal of Mechanical Design, 2019

    2019

  4. [4]

    Energy modeling and control for improved engine stability and efficiency in air vehicles.Annual Reviews in Control, 2021

    Rupamathi Jaddivada, Marija D Ilic, and Eyad H Abed. Energy modeling and control for improved engine stability and efficiency in air vehicles.Annual Reviews in Control, 2021

    2021

  5. [5]

    Co-design of autonomous systems: From hardwareselectiontocontrolsynthesis

    Gioele Zardini, Andrea Censi, and Emilio Frazzoli. Co-design of autonomous systems: From hardwareselectiontocontrolsynthesis. In2021 European Control Conference (ECC).IEEE, 2021

    2021

  6. [6]

    Co-design of distributed model-based control andevent-triggeringschemeforloadfrequencyregulationinsmartgrids.IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018

    Shichao Liu, Wensheng Luo, and Ligang Wu. Co-design of distributed model-based control andevent-triggeringschemeforloadfrequencyregulationinsmartgrids.IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018

    2018

  7. [7]

    Control for societal-scale challenges: Road map 2030.IEEE Control Systems Magazine, 2024

    Andrew Alleyne, Frank Allgower, Aaron D Ames, Saurabh Amin, James Anderson, Anu- radha M Annaswamy, Panos J Antsaklis, Neda Bagheri, Hamsa Balakrishnan, and Bassam Bamieh. Control for societal-scale challenges: Road map 2030.IEEE Control Systems Magazine, 2024

    2030

  8. [8]

    Engineering and technology applications of control co-design: a survey

    Josefredo Gadelha Da Silva, Thalita Nazare, Matheus Costa, Marcio J Lacerda, and Eriv- elton Nepomuceno. Engineering and technology applications of control co-design: a survey. IEEE Access, 2024

    2024

  9. [9]

    The expanding domain of aeroelasticity.The Aeronautical Journal, 1946

    AR Collar. The expanding domain of aeroelasticity.The Aeronautical Journal, 1946

    1946

  10. [10]

    Pathways to net-zero emissions from aviation.Nature Sustainability, 2023

    Candelaria Bergero, Greer Gosnell, Dolf Gielen, Seungwoo Kang, Morgan Bazilian, and Steven J Davis. Pathways to net-zero emissions from aviation.Nature Sustainability, 2023. 17 REFERENCES REFERENCES

    2023

  11. [11]

    Reducing the environmental impact of international aviation through sustainable aviation fuel with integrated carbon capture and storage.Energy Conversion and Management, 2024

    Alberto Almena, Regina Siu, Katie Chong, Patricia Thornley, and Mirjam Roder. Reducing the environmental impact of international aviation through sustainable aviation fuel with integrated carbon capture and storage.Energy Conversion and Management, 2024

    2024

  12. [12]

    Multidisciplinary optimization of a transonic truss-braced wing aircraft using aviary

    Eliot Aretskin-Hariton, Jennifer Gratz, John Jasa, Kenneth Moore, Robert Falck, Caroline Kuhnle, Mark Leader, Jeff Chapman, Eric Hendricks, and Jason Kirk. Multidisciplinary optimization of a transonic truss-braced wing aircraft using aviary. InAIAA SciTech Forum and Exposition, 2024

    2024

  13. [13]

    Overview of semi aeroelastic hinge simulation research at airbus

    Thomas Wilson, James Kirk, Robert Mills, Andrea Castrichini, Paolo Mastracci, Bede Geoghegan Hart, Theo Meyer, Joaquin Alvarez-Rementeria Reina, and Jean-Philippe Boin. Overview of semi aeroelastic hinge simulation research at airbus. InAIAA SCITECH 2025 Forum, 2025

    2025

  14. [14]

    Ludwig prandtl’s 1933 paper concerning wings for minimum induced drag, translation and commentary

    Douglas F Hunsaker and Warren Phillips. Ludwig prandtl’s 1933 paper concerning wings for minimum induced drag, translation and commentary. InAIAA Scitech 2020 Forum, 2020

    1933

  15. [15]

    Evaluation of the mass and aerodynamic efficiency of a high aspect ratio wing for prospective passenger aircraft.Aerospace, 2022

    Anatolii Kretov and Dmytro Tiniakov. Evaluation of the mass and aerodynamic efficiency of a high aspect ratio wing for prospective passenger aircraft.Aerospace, 2022

    2022

  16. [16]

    X-hale: A very flexible unmanned aerial vehicle for nonlinear aeroelastic tests

    Carlos ES Cesnik, Patrick J Senatore, Weihua Su, Ella M Atkins, and Christopher M Shearer. X-hale: A very flexible unmanned aerial vehicle for nonlinear aeroelastic tests. AIAA journal, 2012

    2012

  17. [17]

    How do albatrosses fly around the world without flapping their wings? Progress in Oceanography, 2011

    Philip L Richardson. How do albatrosses fly around the world without flapping their wings? Progress in Oceanography, 2011

    2011

  18. [18]

    Adult vultures outperform juveniles in challenging thermal soaring conditions.Scientific reports, 2016

    Roi Harel, Nir Horvitz, and Ran Nathan. Adult vultures outperform juveniles in challenging thermal soaring conditions.Scientific reports, 2016

    2016

  19. [19]

    Field observations of thermals and thermal streets, and the theory of cross-country soaring flight.Journal of Avian Biology, 1998

    C Ji Pennycuick. Field observations of thermals and thermal streets, and the theory of cross-country soaring flight.Journal of Avian Biology, 1998

    1998

  20. [20]

    Increasing persis- tence of uavs and mavs through thermal soaring.The Aeronautical Journal, 2009

    Ian D Cowling, Simon Willcox, Yoge Patel, Phill Smith, and M Roberts. Increasing persis- tence of uavs and mavs through thermal soaring.The Aeronautical Journal, 2009

    2009

  21. [21]

    Optimal patterns of glider dynamic soaring.Optimal control applications and methods, 2004

    Yiyuan J Zhao. Optimal patterns of glider dynamic soaring.Optimal control applications and methods, 2004

    2004

  22. [22]

    Deep reinforcement learning: An overview.arXiv preprint arXiv:1701.07274, 2017

    Yuxi Li. Deep reinforcement learning: An overview.arXiv preprint arXiv:1701.07274, 2017

    Pith/arXiv arXiv 2017

  23. [23]

    Benchmarking reinforcement learning algorithms on real-world robots

    A Rupam Mahmood, Dmytro Korenkevych, Gautham Vasan, William Ma, and James Bergstra. Benchmarking reinforcement learning algorithms on real-world robots. InCon- ference on robot learning. PMLR, 2018

    2018

  24. [24]

    Mastering the game of go with deep neural networks and tree search.nature, 2016

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, and Marc Lanctot. Mastering the game of go with deep neural networks and tree search.nature, 2016

    2016

  25. [25]

    Grand- master level in starcraft ii using multi-agent reinforcement learning.nature, 2019

    Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, JunyoungChung, DavidHChoi, RichardPowell, TimoEwalds, andPetkoGeorgiev. Grand- master level in starcraft ii using multi-agent reinforcement learning.nature, 2019

    2019

  26. [26]

    Playing atari with deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. 18 REFERENCES REFERENCES

    Pith/arXiv arXiv 2013

  27. [27]

    Highly accurate protein structure prediction with alphafold.nature, 2021

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, and Anna Potapenko. Highly accurate protein structure prediction with alphafold.nature, 2021

    2021

  28. [28]

    Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 2024

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, and Joshua Bambrick. Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 2024

    2024

  29. [29]

    Multi-objective, multi-agent based approaches for predication of sustainable aviation roll- out

    Martin J Muir, Stephen Rolston, Zenudeen Shehbaz, Paul Li, Lyuxi Wang, and Ian Marr. Multi-objective, multi-agent based approaches for predication of sustainable aviation roll- out. InAIAA SCITECH 2024 Forum, 2024

    2024

  30. [30]

    A two-timescale reinforcement learning approach for control co-design problems

    Eddieb Sadat, Mostaan Lotfalian Saremi, and Alparslan Emrah Bayrak. A two-timescale reinforcement learning approach for control co-design problems. InInternational Design Engineering Technical Conferences and Computers and Information in Engineering Con- ference. American Society of Mechanical Engineers, 2023

    2023

  31. [31]

    N-limb: Neural limb optimization for efficient morphological design.arXiv preprint arXiv:2207.11773, 2022

    Charles Schaff and Matthew R Walter. N-limb: Neural limb optimization for efficient morphological design.arXiv preprint arXiv:2207.11773, 2022

    Pith/arXiv arXiv 2022

  32. [32]

    Jointly learning to construct and control agents using deep reinforcement learning

    Charles Schaff, David Yunis, Ayan Chakrabarti, and Matthew R Walter. Jointly learning to construct and control agents using deep reinforcement learning. In2019 international conference on robotics and automation (ICRA). IEEE, 2019

    2019

  33. [33]

    Revealing principles of autonomous thermal soaring in windy conditions using vulture-inspired deep reinforcement- learning.Nature Communications, 2024

    Yoav Flato, Roi Harel, Aviv Tamar, Ran Nathan, and Tsevi Beatus. Revealing principles of autonomous thermal soaring in windy conditions using vulture-inspired deep reinforcement- learning.Nature Communications, 2024

    2024

  34. [34]

    Study on the glider soaring strategy in random location thermal updraft via reinforcement learning.Aerospace, 2023

    Yunxiang Cui, De Yan, and Zhiqiang Wan. Study on the glider soaring strategy in random location thermal updraft via reinforcement learning.Aerospace, 2023

    2023

  35. [35]

    Deep reinforcement learning approach for integrated updraft mapping and exploitation.Journal of Guidance, Control, and Dynamics, 2023

    Stefan Notter, Christian Gall, Gregor Muller, Aamir Ahmad, and Walter Fichter. Deep reinforcement learning approach for integrated updraft mapping and exploitation.Journal of Guidance, Control, and Dynamics, 2023

    2023

  36. [36]

    Embodied intelligence via learning and evolution.Nature communications, 2021

    Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution.Nature communications, 2021

    2021

  37. [37]

    Robogrammar: graph grammar for terrain-optimized robot design.ACM Transactions on Graphics (TOG), 2020

    AllanZhao, JieXu, MinaKonakovic-Lukovic, JosephineHughes, AndrewSpielberg, Daniela Rus, and Wojciech Matusik. Robogrammar: graph grammar for terrain-optimized robot design.ACM Transactions on Graphics (TOG), 2020

    2020

  38. [38]

    Hardware as policy: Mechanical and computational co-optimization using deep reinforcement learning.arXiv preprint arXiv:2008.04460, 2020

    Tianjian Chen, Zhanpeng He, and Matei Ciocarlie. Hardware as policy: Mechanical and computational co-optimization using deep reinforcement learning.arXiv preprint arXiv:2008.04460, 2020

    Pith/arXiv arXiv 2008

  39. [39]

    Co-design optimisation of morphing topology and con- trol of winged drones

    Fabio Bergonti, Gabriele Nava, Valentin Wüest, Antonello Paolino, Giuseppe L’Erario, Daniele Pucci, and Dario Floreano. Co-design optimisation of morphing topology and con- trol of winged drones. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 8679–8685. IEEE, 2024

    2024

  40. [40]

    Automatic co-design of aerial robots using a graph grammar

    Allan Zhao, Tao Du, Jie Xu, Josie Hughes, Juan Salazar, Pingchuan Ma, Wei Wang, Daniela Rus, and Wojciech Matusik. Automatic co-design of aerial robots using a graph grammar. In2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022

    2022

  41. [41]

    Sharp turning maneu- vers with avian-inspired wing and tail morphing.Communications Engineering, 2022

    Enrico Ajanic, Mir Feroskhan, Valentin Wüest, and Dario Floreano. Sharp turning maneu- vers with avian-inspired wing and tail morphing.Communications Engineering, 2022. 19 REFERENCES REFERENCES

    2022

  42. [42]

    Agile perching maneuvers in birds and morphing-wing drones.Nature Commu- nications, 2024

    Valentin Wuest, Simon Jeger, Mir Feroskhan, Enrico Ajanic, Fabio Bergonti, and Dario Floreano. Agile perching maneuvers in birds and morphing-wing drones.Nature Commu- nications, 2024

    2024

  43. [43]

    Optimal vibration control and co-design of very flexible actuated structures.Journal of Sound and Vibration, 2016

    S Maraniello and R Palacios. Optimal vibration control and co-design of very flexible actuated structures.Journal of Sound and Vibration, 2016

    2016

  44. [44]

    Effectiveness of warm-start ppo for guidance with highly constrained nonlinear fixed-wing dynamics

    Christian T Coletti, Kyle A Williams, Hannah C Lehman, Zahi M Kakish, Daniel Whitten, and Julie J Parish. Effectiveness of warm-start ppo for guidance with highly constrained nonlinear fixed-wing dynamics. In2023 American Control Conference (ACC). IEEE, 2023

    2023

  45. [45]

    Flightmare: A flexible quadrotor simulator

    Yunlong Song, Selim Naji, Elia Kaufmann, Antonio Loquercio, and Davide Scaramuzza. Flightmare: A flexible quadrotor simulator. InConference on Robot Learning. PMLR, 2021

    2021

  46. [46]

    Pyflyt–uav simulation environments for reinforcement learning research.arXiv preprint arXiv:2304.01305, 2023

    Jun Jet Tai, Jim Wong, Mauro Innocente, Nadjim Horri, James Brusey, and Swee King Phang. Pyflyt–uav simulation environments for reinforcement learning research.arXiv preprint arXiv:2304.01305, 2023

    Pith/arXiv arXiv 2023

  47. [47]

    A review of nine physics engines for reinforcement learning research.arXiv preprint arXiv:2407.08590, 2024

    Michael Kaup, Cornelius Wolff, Hyerim Hwang, Julius Mayer, and Elia Bruni. A review of nine physics engines for reinforcement learning research.arXiv preprint arXiv:2407.08590, 2024

    Pith/arXiv arXiv 2024

  48. [48]

    Simulation tools for model-based robotics: Comparisonofbullet, havok, mujoco, odeandphysx

    Tom Erez, Yuval Tassa, and Emanuel Todorov. Simulation tools for model-based robotics: Comparisonofbullet, havok, mujoco, odeandphysx. In2015 IEEE international conference on robotics and automation (ICRA). IEEE, 2015

    2015

  49. [49]

    Whole- body simulation of realistic fruit fly locomotion with deep reinforcement learning.bioRxiv, 2024

    Roman Vaxenburg, Igor Siwanowicz, Josh Merel, Alice A Robie, Carmen Morrow, Guido Novati, Zinovia Stefanidi, Gert-Jan Both, Gwyneth M Card, Michael B Reiser, et al. Whole- body simulation of realistic fruit fly locomotion with deep reinforcement learning.bioRxiv, 2024

    2024

  50. [50]

    Simulating a human fetus in soft uterus

    Dongmin Kim, Hoshinori Kanazawa, and Yasuo Kuniyoshi. Simulating a human fetus in soft uterus. In2022 IEEE International Conference on Development and Learning (ICDL). IEEE, 2022

    2022

  51. [51]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  52. [52]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Pith/arXiv arXiv 2014

  53. [53]

    General theory of aerodynamic instability and the mechanism of flutter

    Theodore Theodorsen. General theory of aerodynamic instability and the mechanism of flutter. Technical report, National Advisory Committee for Aeronautics, 1949

    1949

  54. [54]

    John Wiley & Sons, 2017

    Grigorios Dimitriadis.Introduction to nonlinear aeroelasticity. John Wiley & Sons, 2017

    2017

  55. [55]

    Thesis, Massachusetts Institute of Technology, 2021

    Peter D Sharpe and R John Hansman.Aerosandbox: A differentiable framework for aircraft design optimization. Thesis, Massachusetts Institute of Technology, 2021

    2021

  56. [56]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2012

    2012

  57. [57]

    Springer, 2014

    Roy Featherstone.Rigid body dynamics algorithms. Springer, 2014

    2014

  58. [58]

    Mathematical method for prediction aeroelastic phenomena and multidisciplinary optimization lifting surfaces of flight vehicle at prelimi- nary stage design, 2024

    Vitalii Sukhov Havaza and Ruslan Nikitin. Mathematical method for prediction aeroelastic phenomena and multidisciplinary optimization lifting surfaces of flight vehicle at prelimi- nary stage design, 2024. 20 A MUJOCO FLIGHT ENVIRONMENT

    2024

  59. [59]

    Wind turbine post-stall airfoil performance character- istics guidelines for blade-element momentum methods

    James Tangler and David Kocurek. Wind turbine post-stall airfoil performance character- istics guidelines for blade-element momentum methods. In43rd AIAA Aerospace Sciences Meeting and Exhibit, 2005

    2005

  60. [60]

    !" 𝜙"!# 𝐿$!# 𝑘%&

    Randal W Beard and Timothy W McLain.Small unmanned aircraft: Theory and practice. Princeton university press, 2012. A Mujoco Flight Environment Mujoco, which stands for multi-joint dynamcis with contact, is a physics engine developed to facilitate model-based control developments [56]. Physical objects known asbodywithin Mujoco are defined in a kinematic ...

    2012