Pith. sign in

REVIEW 3 major objections 8 minor 61 references

A single compact free-flyer controller, steered by a five-number physics priority vector through a hypernetwork, masters four orbital GNC tasks and composes new ones without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 19:00 UTC pith:DQQ5A3CT

load-bearing objection Solid multi-task GNC systems paper with real hardware and code; the zero-shot lifetime claim is thinner than the abstract sells because the composition tests stay in the trained corner of the embedding. the 3 major comments →

arxiv 2607.24292 v1 pith:DQQ5A3CT submitted 2026-07-27 cs.RO

Learning Adaptive Multi-Task Guidance, Navigation, and Control via Hypernetworks

Ricard Marsal I Castan , Aman Arora , Antoine Richard , Andrej Orsula , C\'edric Pradalier , Miguel A. Olivares-M\'endez This is my paper
classification cs.RO
keywords reinforcement learninghypernetworksspace roboticsmulti-task RLguidance navigation and controlsim-to-realfree-flying robots
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Orbital free-flyers today carry brittle libraries of separate controllers for docking, velocity tracking, inspection, and obstacle navigation; each new mission mix forces a ground upload. This paper argues that one small shared actor-critic is enough if a hypernetwork writes its weights from a continuous five-dimensional embedding of physical priorities (position, attitude, velocity, braking, safety). That embedding lets the same policy match specialist sample efficiency on the four training tasks, interpolate held-out combinations such as docking among obstacles or pure stabilization, and transfer to a physical air-bearing satellite emulator under mass, center-of-mass, and wrench randomization. The practical stake is longer autonomous lifetime: novel behaviors can be dialed in at deployment time with no on-orbit learning and no new reward design.

Core claim

HYPER-GNC shows that mapping a continuous physics-informed task embedding through a rank-1 hypernetwork onto the weights of a shared Beta-policy actor-critic yields a single compact GNC controller that matches single-task PPO specialists on velocity tracking, docking, inspection, and obstacle navigation, supports zero-shot composition of novel mission mixes by embedding interpolation alone, and transfers across those profiles to a real floating-platform emulator when trained with domain randomization.

What carries the argument

Physics-informed hypernetwork modulation: a 5-D priority vector e_k = [α_pos, α_att, α_vel, α_brake, α_safe] is encoded and factored into rank-1 scale-and-shift heads that rewrite every adaptive layer of a shared actor-critic at inference, turning discrete tasks into points on a continuous behavioral manifold.

Load-bearing premise

The hand-chosen five priority numbers, sampled in fixed ranges per task, form a rich enough continuous manifold that simple interpolation at test time truly produces safe novel GNC behaviors without further learning or redesign.

What would settle it

Train only on the four base tasks, then measure docking-with-obstacles, stabilization, and point-navigation error on the physical emulator under the same mass/CoM/wrench randomization used in the paper; if position or orientation error systematically exceeds the single-task specialist upper bound or collides, the composition claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • One onboard weight set can replace an ensemble of specialist GNC policies, cutting memory and simplifying flight software.
  • Operators or a higher-level planner can request new mission mixes by editing five scalars, without reward redesign or gradient steps on orbit.
  • Domain-randomized training of the same hypernetwork architecture is sufficient to close the sim-to-real gap for planar free-flyer thruster control.
  • Continuous priority embeddings can serve as a standard interface between classical GNC mode logic and learned low-level thruster policies.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the five-axis manifold generalizes, the same hypernetwork could later condition on manipulator or multi-body priorities without a full multi-task retrain from scratch.
  • The observed thruster sparsity on the emulator suggests the embedding regularizes fuel use; an explicit propellant term in the embedding could turn that into a controllable trade-off.
  • Failure modes under large external wrenches point to a natural next stress test: whether α_safe and α_brake alone can absorb unmodeled disturbance magnitudes outside the training ranges.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper presents HYPER-GNC, a multi-task reinforcement learning framework in which a hypernetwork maps a continuous 5-D physics-informed task embedding e_k = [α_pos, α_att, α_vel, α_brake, α_safe] to rank-1 scale-and-shift modulations of the weights of a shared actor-critic policy (Eqs. 2–5). A single compact policy is trained with PPO in Isaac Lab to perform four GNC tasks for a model of JAXA's Int-Ball2 (velocity tracking, docking, inspection, navigation with obstacle avoidance), is evaluated zero-shot on three composed tasks (docking-with-obstacles, stabilization, point navigation), and is transferred — with a Beta-to-Gaussian output-head switch and domain randomization over mass, CoM, and external wrenches — to a physical air-bearing floating platform. Reported results (Tables I–III, 5 seeds × 128 episodes) show HYPER-GNC matching or beating single-task PPO specialists on the four trained tasks, remaining competitive on compositions, and benefiting clearly from domain randomization on hardware. Code, weights, and deployment scripts are released.

Significance. If the composition claim is substantiated, this is a solid and useful contribution to spacecraft GNC learning: it demonstrates that parameter-efficient hypernetwork modulation (rank-1, ~129k parameters total vs. ~306k for the specialist ensemble, Table VII) can replace a brittle library of discrete controllers, with honest multi-seed evaluation and physical-platform evidence. The release of code, trained weights, and deployment scripts materially raises the paper's value to the community. The zero-shot composition idea — treating mission profiles as points on a physics-informed manifold — is conceptually appealing and, if the interior-interpolation behavior is shown to be sound, would be a genuinely enabling capability for long-duration orbital autonomy.

major comments (3)
  1. [§V-C, Table II vs. Table V (zero-shot composition evaluation)] The abstract, §I, and §IV-A.2 claim that the continuous embedding manifold lets the controller 'generalize to novel mission configurations at deployment time without any retraining,' and this underwrites the extended-mission-lifetime framing. However, cross-checking Table V against the evaluated compositions shows that all three 'novel' tasks sample their active embedding components from U(0.8,1.0) — the same high-value corner in which those components were already active (and in the case of (α_pos, α_brake), co-active) during training. The evaluation therefore tests recombination of co-trained corner regions, not interpolation through the manifold interior, and never tests the conflicting-priority regime the paper itself identifies as the hard case in §IV-A.1 (α_vel high together with α_brake high). For a rank-1 scale-and-shift hypernetwork, behavior at untrained embedding points is unc
  2. [§IV-A.3 and §VI (Beta→Gaussian switch for binary thrusters)] The switch from a Beta policy (Int-Ball2, continuous) to a Gaussian policy with discretized outputs (Floating Platform, binary thrusters) is stated as a design choice but the mechanism of transfer is underspecified: is the FP policy retrained from scratch on FP dynamics with a Gaussian head, or fine-tuned from the Int-Ball2-trained actor, and is the embedding→weight map preserved across the swap? This matters because Table X(b) shows the Gaussian ablation degrades sim performance by ~51% on average, yet Table III shows strong FP results; the reader cannot tell why the degradation documented in Table X(b) does not carry over, or whether it does and is masked by the easier planar task. A short description of the transfer procedure plus one control (e.g., Beta-trained FP policy with thresholding) would close this gap, since sim-to-real transfer 'across all mission profiles' is one of the th
  3. [§IV-B, Eq. (5) — rank-1 modulation expressivity] The rank-1 key/query outer product (Eq. 5) is justified purely by parameter efficiency (16× reduction) and by analogy to LoRA/FiLM, but no ablation varies the rank or compares against an unstructured FiLM-style conditioning of the same parameter count. Since the architecture is the paper's core contribution and the claim is that rank-1 modulation is 'expressive enough to resolve conflicting GNC gradients,' a rank sweep (1, 4, 16, full) on at least two tasks would substantiate that the efficiency gain is not bought with a performance loss that a slightly larger budget would recover. This is a modest experimental addition within the existing framework.
minor comments (8)
  1. [Table I (Inspection column)] The mean orientation error for Inspection is ~0.6 rad for essentially all methods including single-task experts, yet success rates reach 85–91%. Please clarify what e_o measures here (final target? average over sequence?) and what angular threshold defines success in r_seq; as written the metric and the success rate appear contradictory.
  2. [Fig. 4 (robustness analysis)] The caption states 'a relative error of 1.0 represents the performance of the non-randomized baseline,' but the y-axis semantics (numerator/denominator of the ratio) are unclear, and §V-D says the ratio is computed between the policy 'trained without randomizations' and perturbed evaluations — yet the figure is used to justify the DR ranges deployed on hardware. Please define the ratio precisely.
  3. [Table III (real-robot results)] Only three runs per task are reported on hardware. Given the safety-relevance framing, please state this explicitly in the table caption and, if feasible, expand n, or at least report per-run values rather than only mean/std over n=3.
  4. [References] Several reference pairs are duplicates: [8]/[9], [10]/[11], [52]/[53].
  5. [Table VII (compute/parameters)] The single-task parameter count is presented as the sum of four networks (~306k) vs. HYPER-GNC's 128,737, which is the right comparison for the deployment-footprint argument; but the per-task inference footprint of a specialist (~74k) is smaller than HYPER-GNC's. The text should acknowledge that the advantage is in total on-board storage and architectural simplicity, not per-inference FLOPs, which Table VII partly contradicts (HYPER-GNC training time 423 min vs. MTRL 291 min is fine, but inference cost per forward pass including the hypernetwork modulation is not reported).
  6. [§IV-B (observation zero-padding)] Observation spaces differ across tasks (Table VI) and are handled by zero-padding to a fixed maximum length; for the composition tasks (e.g., Docking with Obstacles), please state explicitly which observation layout is used at evaluation time and how obstacle dimensions are populated for a task whose closest trained analog (Docking) had no obstacle observations.
  7. [Eq. (5)] The normalization by √d_out in Eq. (5) is unconventional (LoRA-style outer products typically scale by 1/r or a learnable α); a one-line justification or citation would help, particularly given the near-zero-variance initialization described in the same section.
  8. [§VI-C, Table IV] The payload/CoM sensitivity table mixes single-run results (Fig. 6 caption says 'single run') with multi-run-looking error bars; please state n for each cell.

Circularity Check

1 steps flagged

Empirical MTRL systems paper; no derivation reduces by construction to its inputs. Minor design circularity only in hand-chosen embedding ranges that match the trained tasks.

specific steps
  1. self definitional [Section IV-C / Table V (Semantic Embedding Selection)]
    "Instead of discrete task IDs, we project all mission objectives into a continuous semantic manifold. We define the context vector e_k ∈ [0,1]^5 as a set of normalized physical control priorities: e_k = [α_pos, α_att, α_vel, α_brake, α_safe] ... Continuous sampling as regularization. Each dimension is uniformly sampled from a range during training (see Table V)"

    The continuous manifold and the claim that interpolating it yields novel behaviors are defined by the authors' hand-chosen per-task sampling boxes in Table V, which are set to the same high-value corners (U(0.8,1.0)) that the four training tasks and the three composition tasks later use. This is ordinary design circularity for a conditioned policy (ranges chosen to match the tasks), not a fitted-input-called-prediction or a derivation that forces the Table I/II numbers by construction. Flagged only for completeness; it does not elevate the overall score.

full rationale

HYPER-GNC is an empirical multi-task RL systems paper. The claimed results (sample efficiency vs single-task PPO, zero-shot composition on three held-out mixes, sim-to-real on a floating platform) are measured performance numbers on held-out initial conditions and physical hardware, not analytic predictions. The 5-D physics-informed embedding e_k, the per-task uniform sampling ranges in Table V, the rank-1 scale-and-shift hypernetwork, the Beta policy head, and the domain-randomization ranges are all explicit design choices; none is derived from a uniqueness theorem, a fitted identity, or a self-citation chain that forces the reported numbers. Composition tasks in Table II are recombinations of the same high-value corners already sampled in training (as the skeptic notes), so the generalization claim is weaker than advertised, but that is an overclaim / insufficient stress-test issue, not circularity: the paper does not assert that the composition errors equal a fitted parameter by construction. Rewards (Table IX) are hand-specified and evaluated, not reverse-engineered from the metrics. No load-bearing self-citation uniqueness result appears. Score 1 reflects only the ordinary, non-load-bearing design circularity that the semantic ranges were chosen to cover the tasks being trained—standard for conditioned policies and not equivalent to claiming a first-principles derivation.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 2 invented entities

Load-bearing content is mostly standard RL + rigid-body free-flyer modeling plus paper-specific design choices (5-D priority embedding, rank-1 hypernet heads, reward weights, DR ranges). No new physical entities. The central claim rests on those design axioms being rich enough for composition and on DR closing the FP reality gap—not on a formal derivation.

free parameters (6)
  • Semantic embedding sampling ranges per task (Table V) = e.g. Docking α_pos,α_att,α_brake ~ U(0.8,1.0); Inspection α_pos ~ U(0.7,1.0)
    Hand-chosen U(0.7/0.8,1.0) boxes define which region of the 5-D manifold each named task occupies; composition quality depends on these boxes.
  • Reward term weights and length-scales (Table IX) = Listed per-term in Table IX
    Dozens of scalar weights (α_pose=2.0, collision -50, progress gains, velocity clips, etc.) shape what ‘success’ means and are tuned by authors.
  • Domain-randomization magnitudes for sim-to-real = U(0.0,0.15) mass/CoM; U(0.0,0.2) wrench/torque
    Mass/CoM U(0,0.15) and wrench/torque U(0,0.2) chosen from robustness sweep (Fig. 4) then frozen for hardware deploy.
  • Network widths and rank-1 hypernet head init = As in Appendix B / §IV-B
    Actor [32,32], critic [512,512], hyper MLP 32, σ=1e-4 near-zero init and +1 scale shift are architectural knobs affecting stability claims.
  • PPO hyperparameters (γ, clip, entropy, parallel envs) = Appendix A
    Standard but claim-relevant training knobs (4096 envs, γ=0.99, clip 0.2, entropy 0.005).
  • Curriculum sigmoid obstacle schedule = n = (n_max-n_min)/(1+e^{-7(x-0.5)})+n_min
    n(x) sigmoid steepness 7 and midpoint 0.5 chosen empirically to aid convergence.
axioms (6)
  • domain assumption Each GNC mission is an MDP and joint multi-task return (Eq. 1) is a valid training objective under shared dynamics family.
    §III problem formulation; standard RL assumption.
  • domain assumption Int-Ball2 / floating-platform rigid-body thruster dynamics in Isaac Lab plus listed DR cover the deploy distribution well enough for zero-shot hardware transfer.
    §III-A, §IV-D, §VI; sim-to-real claim depends on this.
  • ad hoc to paper Rank-1 key/query outer-product modulation (Eq. 5) plus residual scale-and-shift (Eqs. 3–4) is expressive enough to resolve conflicting GNC gradients without full-rank heads or MoE routing.
    §IV-B efficiency argument; motivated by LoRA analogy but not proved.
  • ad hoc to paper The five fixed physical priorities fully parameterize the behavioral manifold needed for trained and novel mission mixes.
    §IV-C; core inductive bias for zero-shot composition.
  • domain assumption Obstacle-avoidance observations include true relative obstacle states (perfect information).
    Table VI; explicitly flagged in Limitations as not addressing perception.
  • ad hoc to paper Switching actor output from Beta (sim continuous) to Gaussian then discretizing is valid for binary FP thrusters without destroying the learned embedding→weight map.
    §IV-A.3 / sim-to-real setup.
invented entities (2)
  • HYPER-GNC architecture (semantic encoder + per-layer rank-1 scale/add hyperheads over shared adaptive MLP actor-critic) independent evidence
    purpose: Single compact multi-task GNC policy with deploy-time behavioral composition.
    Named system contribution; composition of known parts (hypernets, FiLM-like modulation, PPO).
  • Physics-informed 5-D GNC priority embedding (α_pos, α_att, α_vel, α_brake, α_safe) independent evidence
    purpose: Replace categorical TaskIDs with a continuous manifold aligned to classical GNC priorities.
    Hand-specified basis analogous to mission design priority vectors; not learned latents. Falsifiable via composition/ablation tables.

pith-pipeline@v1.2.0-grok45-kimik3 · 25200 in / 4216 out tokens · 93197 ms · 2026-07-31T19:00:13.387444+00:00 · methodology

0 comments
read the original abstract

Autonomous free-flying robots in orbital environments require controllers that are both versatile and resource-efficient, yet maintaining a separate, task-specific policy for each mission profile is architecturally brittle and limits operational flexibility as requirements evolve. We introduce HYPER-GNC, a multi-task reinforcement learning framework in which a hypernetwork maps physics-informed task embeddings to the weights of a shared actor-critic policy, enabling a single compact controller to master four distinct GNC tasks: velocity tracking, docking, inspection, and navigation with obstacle avoidance. The continuous embedding space allows the controller to generalize to novel mission configurations at deployment time without any retraining. Extensive experiments demonstrate that HYPER-GNC achieves sample efficiency comparable to single-task specialists while maintaining stability under significant inertial perturbations and external body wrenches. We further validate the framework on a physical satellite emulator, successfully bridging the simulation-to-reality gap across all mission profiles. Code, trained models, and deployment scripts are made publicly available to support reproducibility.

Figures

Figures reproduced from arXiv: 2607.24292 by Aman Arora, Andrej Orsula, Antoine Richard, C\'edric Pradalier, Miguel A. Olivares-M\'endez, Ricard Marsal I Castan.

Figure 1
Figure 1. Figure 1: Overview. Our proposed framework, HYPER-GNC, performing distinct GNC tasks in simulated environments (top two rows) and transferring the framework for sim-to-real in a satellite emulator (bottom row). without ground intervention, directly extending its effective operational lifetime. Reinforcement Learning (RL) offers a data-driven alterna￾tive capable of mastering complex, non-linear dynamics. Yet, standa… view at source ↗
Figure 2
Figure 2. Figure 2: Framework Overview. We train a single model to perform multiple tasks. Depending on the objective, the weights of the model are modulated by the hypernetwork. The entire system is trained end-to-end using Proximal Policy Optimization (PPO) [40]. IV. METHODOLOGY This section introduces HYPER-GNC, an RL-based frame￾work for learning a single policy that can adapt to mul￾tiple GNC tasks, as summarized in [PI… view at source ↗
Figure 3
Figure 3. Figure 3: Comparative Returns for Int-Ball2 GNC Tasks. The proposed Hypernetwork architecture (Orange) demonstrates competitive sample efficiency, matching the convergence rate of Single-Task Experts (Blue) across all four mission profiles. TABLE I: INT-BALL2 FOUR GNC TASKS COMPARISON PERFORMANCE. Methods Docking Track Velocities Inspection Navigation with Obstacles ep[m] ↓ eo[rad] ↓ ev[m s−1 ] ↓ eω[rad s−1 ] ↓ eo[r… view at source ↗
Figure 4
Figure 4. Figure 4: Floating Platform robustness analysis in simulation. A relative error of 1.0 represents the performance of the non￾randomized baseline; values above 1.0 indicate performance degradation under perturbation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: HYPER-GNC in the real world. From left to right, top to bottom, target and actual velocities of the Track Velocities task, docking into a satellite, inspection of a satellite, and navigation through obstacles. D. More Analyses Combining HYPER-GNC with MTCR and PCGrad (Ta￾ble X). We evaluated the compatibility of HYPER-GNC with auxiliary multi-task optimization techniques. Surprisingly, the addition of PCGr… view at source ↗
Figure 6
Figure 6. Figure 6: Sensitivity analysis. Single run of an external force ap￾plied during the docking task. a) Position error. b) Orientation error. c) Visual of the moment of contact. d) Full trajectory with orientation. Metric Payload Mass CoM w/ 2kg 1kg 2kg 3kg 10cm 20cm 30cm ep[m] ↓ 0.143 (±0.177) 0.059 (±0.043) 0.107 (±0.093) 0.116 (±0.016) 0.262 (±0.258) 0.218 (±0.069) eo[rad] ↓ 0.204 (±0.119) 0.177 (±0.154) 0.176 (±0.1… view at source ↗
Figure 7
Figure 7. Figure 7: Navigation with Obstacles actions. Individual thruster actions for a single navigation with obstacles run. HYPER￾GNC is able to learn policies that are fuel efficient. TABLE VIII: PERFORMANCE ON NOVEL COMPOSITION TASKS WITH THE SAME ABLATIONS OF TABLE X. Methods Docking with Obstacles Stabilization Point Navigation ep[m] ↓ eo[rad] ↓ ev[m s−1 ] ↓ eω[rad s−1 ] ↓ ep[m] ↓ Single-Task RL 0.057 (±0.076) 0.141 (±… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 3 canonical work pages

  1. [1]

    Modular multitask reinforcement learning with policy sketches,

    Jacob Andreas, Dan Klein, and Sergey Levine. Modular multitask reinforcement learning with policy sketches,

  2. [2]

    Intelligent motion planning for collision free autonomous docking of satellite em- ulation platform using reinforcement learning.IFAC- PapersOnLine, 56(2):3354–3359, 2023

    D Athauda, A Banerjee, S Satpute, A Agha-Mohammadi, and G Nikolakopoulos. Intelligent motion planning for collision free autonomous docking of satellite em- ulation platform using reinforcement learning.IFAC- PapersOnLine, 56(2):3354–3359, 2023

    2023

  3. [3]

    Hypernetworks in meta-reinforcement learning

    Jacob Beck, Matthew Thomas Jackson, Risto Vuorio, and Shimon Whiteson. Hypernetworks in meta-reinforcement learning. InConference on Robot Learning, pages 1478–

  4. [4]

    A tutorial on meta-reinforcement learning, 2025

    Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shimon White- son. A tutorial on meta-reinforcement learning, 2025. URL https://arxiv.org/abs/2301.08028

    Pith/arXiv arXiv 2025

  5. [5]

    Multitask learning.Machine Learning, 28:41–75, 1997

    Rich Caruana. Multitask learning.Machine Learning, 28:41–75, 1997. doi: 10.1023/A:1007379606734. 9

    doi:10.1023/a:1007379606734 1997

  6. [6]

    Au- tonomous imaging and mapping of small bodies using deep reinforcement learning

    David M Chan and Ali-akbar Agha-mohammadi. Au- tonomous imaging and mapping of small bodies using deep reinforcement learning. In2019 IEEE aerospace conference, pages 1–12. IEEE, 2019

    2019

  7. [7]

    Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,

    Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,

  8. [8]

    Quantifying generalization in re- inforcement learning, 2019

    Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in re- inforcement learning, 2019. URL https://arxiv.org/abs/ 1812.02341

    Pith/arXiv arXiv 2019

  9. [9]

    Quantifying generalization in rein- forcement learning

    Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in rein- forcement learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors,Proceedings of the 36th Interna- tional Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1282–

  10. [10]

    Learning modular neural network policies for multi-task and multi-robot transfer,

    Coline Devin, Abhishek Gupta, Trevor Darrell, Pieter Abbeel, and Sergey Levine. Learning modular neural network policies for multi-task and multi-robot transfer,

  11. [11]

    Learning modular neural network policies for multi-task and multi-robot transfer

    Coline Devin, Abhishek Gupta, Trevor Darrell, Pieter Abbeel, and Sergey Levine. Learning modular neural network policies for multi-task and multi-robot transfer. In2017 IEEE international conference on robotics and automation (ICRA), pages 2169–2176. IEEE, 2017

    2017

  12. [12]

    Drift: Deep reinforcement learning for intelligent floating plat- forms trajectories

    Matteo El-Hariry, Antoine Richard, Vivek Muralidharan, Matthieu Geist, and Miguel Olivares-Mendez. Drift: Deep reinforcement learning for intelligent floating plat- forms trajectories. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), page 14034–14041. IEEE, October 2024. doi: 10. 1109/iros58592.2024.10801927. URL http://d...

    arXiv 2024

  13. [13]

    Hu et al

    Edward J. Hu et al. Lora: Low-rank adaptation of large language models, 2021. URL arXiv:2106.09685

    Pith/arXiv arXiv 2021

  14. [14]

    Film: Visual reasoning with a general conditioning layer, 2017

    Ethan Perez et al. Film: Visual reasoning with a general conditioning layer, 2017. URL arXiv:1709.07871

    Pith/arXiv arXiv 2017

  15. [15]

    Isaac lab: A gpu-accelerated simulation framework for multi-modal robot learning

    Mayank Mittal et al. Isaac lab: A gpu-accelerated simulation framework for multi-modal robot learning. arXiv preprint arXiv:2511.04831, 2025. URL https: //arxiv.org/abs/2511.04831

    Pith/arXiv arXiv 2025

  16. [16]

    Bag of tricks for image classification with convolutional neural networks, 2018

    Tong He et al. Bag of tricks for image classification with convolutional neural networks, 2018. URL arXiv: 1812.01187

    Pith/arXiv arXiv 2018

  17. [17]

    Federici, A

    L. Federici, A. Scorsoglio, and R. Furfaro. Optical bench validation of meta-reinforcement learning for image- based spacecraft guidance and navigation.Authorea Preprints, 2025

    2025

  18. [18]

    Wigbert Fehse.Automated Rendezvous and Docking of Spacecraft. 2003. doi: 10.1017/cbo9780511543388

    work page doi:10.1017/cbo9780511543388 2003

  19. [19]

    J. Feng, M. Chen, Z. Pu, et al. Efficient multi-task reinforcement learning via task-specific action correction. IEEE Transactions on Neural Networks and Learning Systems, 2025

    2025

  20. [20]

    Model- agnostic meta-learning for fast adaptation of deep net- works, 2017

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep net- works, 2017. URL https://arxiv.org/abs/1703.03400

    Pith/arXiv arXiv 2017

  21. [21]

    K. Fu, S. A. Jain, P. Howell, and H. Ravichandar. Capability-aware shared hypernetworks for flexible het- erogeneous multi-robot coordination.arXiv preprint arXiv:2501.06058, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Six degree-of-freedom body-fixed hovering over unmapped asteroids via lidar altimetry and reinforcement meta- learning.Acta Astronautica, 172:90–99, July 2020

    Brian Gaudet, Richard Linares, and Roberto Furfaro. Six degree-of-freedom body-fixed hovering over unmapped asteroids via lidar altimetry and reinforcement meta- learning.Acta Astronautica, 172:90–99, July 2020. ISSN 0094-5765. doi: 10.1016/j.actaastro.2020.03.026. URL http://dx.doi.org/10.1016/j.actaastro.2020.03.026

    work page doi:10.1016/j.actaastro.2020.03.026 2020

  23. [23]

    Deep reinforcement learning for six degree-of-freedom planetary landing.Advances in Space Research, 65(7): 1723–1741, 2020

    Brian Gaudet, Richard Linares, and Roberto Furfaro. Deep reinforcement learning for six degree-of-freedom planetary landing.Advances in Space Research, 65(7): 1723–1741, 2020

    2020

  24. [24]

    Adams, and Sergey Levine

    Dibya Ghosh, Jad Rahme, Aviral Kumar, Amy Zhang, Ryan P. Adams, and Sergey Levine. Why generalization in rl is difficult: Epistemic pomdps and implicit par- tial observability, 2021. URL https://arxiv.org/abs/2107. 06277

    2021

  25. [25]

    The phi-sat-1 mission: The first on-board deep neural network demonstrator for satellite earth observation.IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022

    Gianluca Giuffrida, Luca Fanucci, Gabriele Meoni, Matej Batiˇc, L ´eonie Buckley, Aubrey Dunne, Chris van Dijk, Marco Esposito, John Hefele, Nathan Vercruyssen, Gi- anluca Furano, Massimiliano Pastena, and Josef As- chbacher. The phi-sat-1 mission: The first on-board deep neural network demonstrator for satellite earth observation.IEEE Transactions on Geo...

    arXiv 2022

  26. [26]

    Hypernetworks

    David Ha, Andrew Dai, and Quoc V Le. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016

    Pith/arXiv arXiv 2016

  27. [27]

    Multi- task reinforcement learning with mixture of orthogonal experts.arXiv preprint arXiv:2311.11385, 2023

    Ahmed Hendawy, Jan Peters, and Carlo D’Eramo. Multi- task reinforcement learning with mixture of orthogonal experts.arXiv preprint arXiv:2311.11385, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Deep reinforcement learning for spacecraft proximity operations guidance

    Kirk Hovell and Steve Ulrich. Deep reinforcement learning for spacecraft proximity operations guidance. Journal of spacecraft and rockets, 58(2):254–264, 2021

    2021

  29. [29]

    A survey on artificial intelligence trends in spacecraft guidance dynamics and control.Astrodynamics, 3(4):287–299, 2019

    Dario Izzo, Marcus M ¨artens, and Binfeng Pan. A survey on artificial intelligence trends in spacecraft guidance dynamics and control.Astrodynamics, 3(4):287–299, 2019

    2019

  30. [30]

    Learning to achieve goals

    Leslie Pack Kaelbling. Learning to achieve goals. In International Joint Conference on Artificial Intelligence,

  31. [31]

    Scaling up multi- task robotic reinforcement learning

    Dmitry Kalashnikov, Jake Varley, Yevgen Chebotar, Benjamin Swanson, Rico Jonschkowski, Chelsea Finn, Sergey Levine, and Karol Hausman. Scaling up multi- task robotic reinforcement learning. In5th Annual Conference on Robot Learning, 2021

    2021

  32. [32]

    Mitani, T

    S. Mitani, T. Nishishita, and D. Hirano. Int-ball2: Compact high-torque propulsion system actively utilizes propeller air drag polarity.Proceedings of the 33rd Astrodynamics Symposium (in Japanese), 2023

    2023

  33. [33]

    Multi-critic actor learning: 10 Teaching RL policies to act with style

    Siddharth Mysore, George Cheng, Yunqi Zhao, Kate Saenko, and Meng Wu. Multi-critic actor learning: 10 Teaching RL policies to act with style. InInternational Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=rJvY 5OzoI

    2022

  34. [34]

    Orsula, M

    A. Orsula, M. Geist, M. Olivares-Mendez, et al. Space robotics bench: Robot learning beyond earth.arXiv preprint arXiv:2509.23328, 2025

    arXiv 2025

  35. [35]

    Actor-mimic: Deep multitask and transfer re- inforcement learning, 2016

    Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhut- dinov. Actor-mimic: Deep multitask and transfer re- inforcement learning, 2016. URL https://arxiv.org/abs/ 1511.06342

    Pith/arXiv arXiv 2016

  36. [36]

    Irving G. B. Petrazzini and Eric A. Antonelo. Proxi- mal policy optimization with continuous bounded action space via the beta distribution, 2021. URL https://arxiv. org/abs/2111.02202

    Pith/arXiv arXiv 2021

  37. [37]

    An overview of multi-task learning in deep neural networks, 2017

    Sebastian Ruder. An overview of multi-task learning in deep neural networks, 2017. URL https://arxiv.org/abs/ 1706.05098

    Pith/arXiv arXiv 2017

  38. [38]

    Hypernetwork-ppo for continual reinforcement learning

    Philemon Sch ¨opf, Sayantan Auddy, Jakob Hollenstein, and Antonio Rodriguez-Sanchez. Hypernetwork-ppo for continual reinforcement learning. InDeep Reinforcement Learning Workshop NeurIPS 2022, 2022

    2022

  39. [39]

    Jordan, and Pieter Abbeel

    John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel. Trust region policy optimization, 2017. URL https://arxiv.org/abs/1502.05477

    Pith/arXiv arXiv 2017

  40. [40]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  41. [41]

    Rsl-rl: A learning library for robotics research.arXiv preprint arXiv:2509.10771, 2025

    Clemens Schwarke, Mayank Mittal, Nikita Rudin, David Hoeller, and Marco Hutter. Rsl-rl: A learning library for robotics research.arXiv preprint arXiv:2509.10771, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    Projected task-specific layers for multi-task reinforcement learning

    Josselin Somerville Roberts and Julia Di. Projected task-specific layers for multi-task reinforcement learning. arXiv e-prints, pages arXiv–2309, 2023

    2023

  43. [43]

    L. Sun, J. Wu, F. Chen, et al. Hypertasr: Hypernetwork- driven task-aware scene representations for robust ma- nipulation.arXiv preprint arXiv:2508.18802, 2025

    arXiv 2025

  44. [44]

    Paco: Parameter-compositional multi-task reinforcement learning.Advances in Neural Information Processing Systems, 35:21495–21507, 2022

    Lingfeng Sun, Haichao Zhang, Wei Xu, and Masayoshi Tomizuka. Paco: Parameter-compositional multi-task reinforcement learning.Advances in Neural Information Processing Systems, 35:21495–21507, 2022

    2022

  45. [45]

    Discretizing continu- ous action space for on-policy optimization, 2020

    Yunhao Tang and Shipra Agrawal. Discretizing continu- ous action space for on-policy optimization, 2020. URL arXiv:1901.10500

    Pith/arXiv arXiv 2020

  46. [46]

    Distral: Robust multitask reinforcement learning, 2017

    Yee Whye Teh, Victor Bapst, Wojciech Marian Czar- necki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, and Razvan Pascanu. Distral: Robust multitask reinforcement learning, 2017. URL https: //arxiv.org/abs/1707.04175

    Pith/arXiv arXiv 2017

  47. [47]

    Domain random- ization for transferring deep neural networks from simu- lation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain random- ization for transferring deep neural networks from simu- lation to the real world. In2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 23–30, 2017. doi: 10.1109/IROS.2017.8202133

    arXiv 2017

  48. [48]

    Bong Wie.Space Vehicle Dynamics and Control. AIAA,

  49. [49]

    Rein- forcement learning for spacecraft maneuvering near small bodies

    Stefan Willis, Dario Izzo, and Daniel Hennes. Rein- forcement learning for spacecraft maneuvering near small bodies. InAAS/AIAA Space Flight Mechanics Meeting, volume 158, pages 1351–1368, 2016

    2016

  50. [50]

    S. P. Yamaguchi, T. Yamamoto, et al. Int-ball2: Iss jem internal camera robot with increased degree of autonomy design and initial checkout.IEEE Robotics and Automation Magazine, 2024

    2024

  51. [51]

    Multi-task reinforcement learning with soft modular- ization.Advances in Neural Information Processing Systems, 33:4767–4777, 2020

    Ruihan Yang, Huazhe Xu, Yi Wu, and Xiaolong Wang. Multi-task reinforcement learning with soft modular- ization.Advances in Neural Information Processing Systems, 33:4767–4777, 2020

    2020

  52. [52]

    Gradient surgery for multi-task learning.Advances in neural information processing systems, 33:5824–5836, 2020

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning.Advances in neural information processing systems, 33:5824–5836, 2020

    2020

  53. [53]

    Gradient surgery for multi-task learning, 2020

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning, 2020. URL https://arxiv. org/abs/2001.06782

    Pith/arXiv arXiv 2020

  54. [54]

    Meta- world: A benchmark and evaluation for multi-task and meta reinforcement learning, 2021

    Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta- world: A benchmark and evaluation for multi-task and meta reinforcement learning, 2021. URL https://arxiv. org/abs/1910.10897

    Pith/arXiv arXiv 2021

  55. [55]

    A survey on multi-task learning, 2021

    Yu Zhang and Qiang Yang. A survey on multi-task learning, 2021. URL https://arxiv.org/abs/1707.08114. APPENDIX A. More Experimental Details Hardware Setup.We train our policies on a single Nvidia RTX 4090. The Floating Platform has a mass of 5kg, a height of 0.5m, radius of 30cm, and 8 binary thruster actuation. We modeled the Int-Ball2 following [32]. Ev...

    Pith/arXiv arXiv 2021

  56. [1289]

    URL https://proceedings

    PMLR, 09–15 Jun 2019. URL https://proceedings. mlr.press/v97/cobbe19a.html

    2019

  57. [1993]

    URL https://api.semanticscholar.org/CorpusID: 5538688

  58. [2008]

    doi: 10.2514/4.860119

    work page doi:10.2514/4.860119

  59. [2016]

    URL https://arxiv.org/abs/1609.07088

    Pith/arXiv arXiv

  60. [2017]

    URL https://arxiv.org/abs/1611.01796

    Pith/arXiv arXiv

  61. [2018]

    URL https://arxiv.org/abs/1711.02257

    Pith/arXiv arXiv