Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance

Andreas Hadjipieris; Andrei Starodubov; Ioannis Kyriakides; Roberto Galeazzi; Yaqub Aris Prabowo

REVIEW 3 major objections 4 minor 16 references

Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance

T0 review · 3 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A trained policy that activates one sensor per step can track vessels almost as well as always-on sensing, while avoiding the online entropy search of expected-information-gain selection.

desk verdict A careful, honest RL sensor-scheduling paper whose central result is real but explicitly tied to its limited-overlap sensor geometry; deserves peer review with requests for code, seeds, and a second geometry. read the letter →

arxiv 2607.22667 v1 pith:USJFJ5H6 submitted 2026-07-03 cs.AI cs.ITcs.LGcs.ROcs.SYeess.SPeess.SYmath.IT

Andrei Starodubov , Yaqub Aris Prabowo , Andreas Hadjipieris , Roberto Galeazzi , Ioannis Kyriakides This is my paper

classification cs.AIcs.ITcs.LGcs.ROcs.SYeess.SPeess.SYmath.IT

keywords sensorschedulingreinforcementlearningparticlefilterinformationgainmaritimesurveillancetargettrackingheterogeneousnetworkproximalpolicyoptimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a reinforcement-learning policy can decide which single sensor to activate at each decision step in a fixed heterogeneous maritime sensor network and still track a vessel nearly as well as activating all sensors at once. The proposed framework, IG-RLSS, trains a proximal-policy-optimization agent on particle-filter belief features, rewarding the realized reduction in posterior entropy. On 1,000 held-out test routes in a simulated marina, the learned single-sensor policy is statistically equivalent to expected-information-gain selection within 2 m RMSE and 2% lost-track margins in all zones, and non-inferior to always-on sensing at the same margins, while costing fewer FLOPs than the online entropy-search baseline. The practical point is that a fixed camera-LiDAR network can run at a fraction of the sensing and computation cost without sacrificing tracking quality, at least when fields of view overlap only sparsely.

What carries the argument

The load-bearing mechanism is an amortized information-gain scheduler: a PPO policy that maps a belief-derived observation vector to a one-of-five sensor action, trained with a reward equal to the realized posterior entropy reduction of the particle filter, multiplied by an observability mask indicating whether the vessel is seen by at least one sensor. The particle filter (500 particles, kinematic motion with a turn-rate constraint, Gaussian Cartesian likelihoods, missed-detection updates) supplies the belief and the information-gain signal, while the policy supplies the selection decision. This replaces the explicit counterfactual entropy search of EIG-SS with a single forward pass of a tw

What would settle it

Run the identical IG-RLSS training and test procedure on a simulated sensor layout with heavily overlapping fields of view (for example, co-located cameras and LiDARs covering the same area) and check the TOST margins: if IG-RLSS is no longer within 2 m RMSE / 2% lost-track of always-on sensing in the covered regions, the claimed near-equivalence is specific to sparse coverage and does not generalize.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that explicit online information-gain search is not necessary to obtain near-optimal sensor scheduling: a policy trained offline with a realized-information-gain reward gated by an observability mask can amortize the selection decision. Using a 500-particle sequential Monte Carlo tracker as the belief source and a 56-dimensional observation vector encoding belief spread, detection history, coverage, and sensor geometry, the PPO agent selects one of five sensors each second. Over the held-out test split, IG-RLSS is TOST-equivalent to EIG-SS within 2 m RMSE / 2% lost-track in all zones, non-inferior to always-on sensing at the same margins, an

Load-bearing premise

The central claim rests on the sensor layout being one where a single sensor dominates the particle-filter update for most target locations; in a denser, higher-overlap layout the equivalence to always-on sensing could break, and the simulator's fidelity to a real marina remains unvalidated.

Editorial extensions

If this is right

In networks with sparse field-of-view overlap, single-sensor scheduling can recover nearly all of the tracking benefit of always-on sensing; the residual difference is within 2 m RMSE and 2% lost track.
A learned policy can serve as a low-compute substitute for expected-information-gain selection, avoiding repeated counterfactual particle-filter updates at run time.
Tracking quality in the far field is limited mainly by coverage rather than by the scheduling rule: all non-random methods degrade similarly, and no sensor covers the true position during about 18% of far-zone timesteps.
The framework establishes a reusable belief-based, single-sensor scheduling baseline and simulation environment for extending to multi-sensor subset selection and energy-aware policies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If this carries over to other sparse-overlap networks, it implies that the expensive part of information-driven sensing is the online search, not the information criterion; the same realized-IG reward could be used to train subset-selection policies directly.
A natural stress test is a high-overlap layout: the paper's own geometry explanation predicts that the equivalence to always-on should break down as redundant sensors begin contributing independent information; running the same training procedure on such a layout would either confirm the boundary or falsify the generalization.
The reward is posterior-entropy reduction, not tracking error, so the policy may be choosing sensors that reduce uncertainty even when that does not reduce RMSE; comparing against an error-reward variant could separate informative from accurate behavior.
Because all results are simulation-based, the strongest testable extension is deployment on a real camera-LiDAR marina testbed; unmodeled effects such as occlusion, weather, and calibration drift could change which sensor is genuinely informative.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid, honest engineering paper, not a paradigm shift. The new thing is the specific integration — a PPO policy selecting one fixed sensor per second from a particle-filter belief, trained with a realized-IG reward, and compared against an explicit EIG-SS baseline. Prior RL scheduling work mostly used non-SMC beliefs or mobile-agent control, so this is a legitimate new combination. The paper does not oversell it, which I appreciate.

What it does well: the statistical work is unusually careful for this area. Matched-pair tests, Holm correction, TOST equivalence to EIG-SS and non-inferiority to always-on at pre-specified 2 m / 2% margins on 1,000 held-out routes. They also disclose the Zone 3 degradation honestly — all methods lose track about 45% of the time there, and they show it is a coverage problem, not a scheduling problem. The mechanistic explanation for why one-sensor selection nearly matches always-on — limited FOV overlap in this particular marina layout — is clear and plausible. The FLOP estimate is rough, but they label it as arithmetic cost only.

The soft spots, in proportion. Biggest scoping gap: the "close to always-on" claim is conditional on the sensor geometry. The authors say so themselves, and they list alternative geometries as future work, but they never test one. With a denser, redundant layout the equivalence could collapse. That is not a fatal flaw — it is a scoping limitation they explicitly acknowledge. Second, all results are simulation-only, with no code or data release and no multi-seed training variance reported. That limits reproducibility and makes it hard to judge how robust the policy is. Third, the equivalence margins (2 m, 2%) are asserted as application-level tolerances without derivation from operational requirements. Minor: the reward is defined the same way as in their prior EIG-SS work [5], which is an overlapping-author citation, but that is by design — the learned policy is meant to amortize that very criterion.

Who this is for: anyone working on practical sensor management, especially maritime surveillance with fixed camera/LiDAR networks, and anyone who wants a clean example of combining SMC belief representations with RL. It deserves a serious referee. My recommendation: send it to peer review, but require the authors to release the simulator and evaluation code, report multiple training seeds, justify the margins, and ideally evaluate at least one alternative sensor geometry — even a stylized one. Those are reasonable conditions, not showstoppers.

Referee Report

3 major / 4 minor

Summary. The paper proposes IG-RLSS, a reinforcement-learning framework for selecting one of five fixed camera/LiDAR sensors at each decision step during single-vessel tracking in a simulated georeferenced marina. A particle filter maintains the belief state, and a PPO agent is trained with a reward equal to realized posterior information gain gated by an observability mask. The method is compared against random single-sensor selection, always-on sensing, and an expected-information-gain baseline (EIG-SS) on 1,000 held-out test routes. The authors report that IG-RLSS is TOST-equivalent to EIG-SS within 2 m RMSE and 2% lost-track margins in all three spatial zones, non-inferior to always-on at the same margins, and cheaper in FLOPs than EIG-SS. They also explain that the close performance is tied to the limited overlap of the sensor layout and propose more redundant configurations as future work.

Significance. If the results are reproduced, the paper offers a practical amortized alternative to online expected-information-gain sensor scheduling in resource-constrained maritime surveillance, with a clear computational advantage. The study is carefully designed in several respects: matched-pair statistics on pre-generated test routes, pre-specified equivalence and non-inferiority margins, held-out evaluation, and an explicit FLOP comparison. The authors also honestly flag the dependence of the result on the specific sensor geometry and the absence of real-world validation. These strengths make the contribution worthwhile, provided that the definitional and reporting gaps identified below are addressed.

major comments (3)

[Section II, reward paragraph; Section III, evaluation metrics] The realized information gain I_real^t, the EIG-SS baseline, and the lost-track percentage are all defined by reference to the authors' prior work [5] rather than in this manuscript. These quantities are load-bearing: the reward drives training, EIG-SS is the central comparison, and lost-track is a headline metric. Without explicit definitions the method is not reproducible and the reader cannot assess whether the reported equivalence is meaningful. The manuscript should include self-contained definitions of I_real^t, the EIG-SS selection rule, and the lost-track metric.
[Section III, statistical tests paragraph] The paper states that TOST equivalence and one-sided non-inferiority tests show IG-RLSS remains within the 2 m / 2% margins in all zones, but it does not report the test statistics, confidence intervals, or p-values. The reader cannot verify the central claim. Provide a table with, for each zone and comparison, the estimated paired difference, the 90% (or 95%) confidence interval, the pre-specified margin, and the TOST/non-inferiority conclusion.
[Abstract and Conclusion vs. Section III close-performance discussion] The abstract and conclusion claim that the learned policy achieves tracking performance close to always-on sensing as a general result, but the paper's own mechanistic explanation is that this holds because the fixed sensors' fields of view overlap only over a limited portion of the operational area; for most target locations a single sensor dominates the update. The only evaluated layout is the low-overlap marina sensor configuration. The claim should be explicitly scoped to this geometry, or a complementary experiment with a redundant, high-overlap layout should be added. This is a scoping gap, not an internal inconsistency, but it is central to how the result is presented.

minor comments (4)

[Section IV, Conclusion] Typo: 'rather than from an ground-truth tracking-error penalty' should be 'rather than from a ground-truth tracking-error penalty'.
[Throughout] Spelling of LiDAR is inconsistent (both 'LiDAR' and 'LIDAR' appear). Use one standard form.
[Section III, statistical tests paragraph] The phrase 'one-sided non-inferiority tests (TOST)' is imprecise: TOST is a two one-sided tests procedure for equivalence, whereas non-inferiority is a one-sided test. Clarify the terminology, e.g. 'one-sided non-inferiority tests' without labeling them TOST.
[Section III, Fig. 4] The FLOP comparison is useful but the paper acknowledges it excludes runtime, memory, and energy. Consider adding a sentence in the caption or text noting that these are implementation-dependent; the current text already does this, so this is only a presentational suggestion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the RL-vs-EIG-SS resemblance is by design and the always-on comparison is an empirical held-out result.

full rationale

The derivation chain is self-contained. The learned policy is trained by PPO on a reward that is explicitly realized information gain (Section II), and the EIG-SS baseline greedily maximizes expected information gain; the paper openly frames IG-RLSS as an amortized approximation to EIG-SS, so the resemblance is by design rather than a hidden reduction. The headline comparison to always-on sensing is an empirical result on 1,000 held-out test routes using pre-specified TOST/non-inferiority margins, and the reward is not the evaluation metric (RMSE/lost-track). The reference to [5] for the exact IG definition and zoning is a normal self-citation; it is not used to rule out alternatives or as a substitute for the current experiment. The acknowledged limitations (tested geometry with limited FOV overlap, simulation-only validation, no sensitivity study) are scoping gaps, not circular steps. No equation reduces to its input, and no fitting target is renamed as a prediction.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

Free parameters are mostly standard RL/filter tuning; the two load-bearing ones are the asserted equivalence margins (which define the headline claims) and the limited-overlap sensor geometry (which the paper itself credits for the single-sensor vs always-on closeness). Axioms are standard Bayesian filtering math plus a chain of domain assumptions about sensor and motion modeling and simulator fidelity to the real testbed. No new physical entities are introduced.

free parameters (6)

PPO hyperparameters and schedules = alpha 1e-3->5e-5; entropy 2e-2->2e-3; gamma 0.995; lambda 0.95; clip 0.2
Taken from standard PPO/GAE practice then hand-adapted; the paper states a systematic sensitivity study is future work.
Equivalence margins = 2 m RMSE; 2% lost-track
Claimed pre-specified 'application-level tolerances' but no derivation from operational requirements is given; the TOST/non-inferiority conclusions are defined by these margins.
Base detection probabilities p_d = 0.95 cameras; 0.99 LiDAR
Inputs to the detection and missed-detection models (Section II); they shape the realized-IG reward and the PF updates.
Particle filter tuning = N_p=500; resample at N_eff<0.5N_p; turn-rate 30 deg/s; Mahalanobis gate
Tracker settings that directly affect RMSE; no sensitivity analysis reported.
Zone radii = 350 m / 600 m boundaries
Evaluation partition taken from the authors' prior work [5]; zone-wise conclusions depend on it.
Sensor layout and FOV geometry = positions per Fig. 1; LiDAR range 150 m
The limited FOV overlap is what makes one-sensor selection nearly as good as always-on; treated as fixed, with no geometry-variation experiment.

assumptions (6)

standard math SIR particle filter with 500 particles adequately approximates the posterior for policy training and RMSE evaluation
Invoked throughout Section II via the standard PF formulation [3]; all tracking and reward computations inherit this approximation.
domain assumption Gaussian Cartesian likelihoods with pinhole/range-bearing sensor models represent the real cameras and LiDARs
Measurement generation in Section II; the missed-detection update and observability-gated reward depend on p_d=0.95/0.99 and the stated noise models. Real-sensor validation is deferred.
domain assumption Vessel motion follows Dubins-smoothed routes with speeds 2.5-15 km/h and a 30 deg/s turn-rate constraint
Particle propagation model in Section II; RMSE and coverage statistics are conditional on this motion model.
domain assumption The georeferenced simulator faithfully represents the CMMI Smart Marina testbed
All results are simulation-only; the conclusion states validation 'under real maritime conditions' is future work.
domain assumption Training, validation, and test route splits are independent and identically distributed
8000/1000/1000 path split with fixed per-episode seeds (Section III); no distribution-shift or leakage analysis is provided.
domain assumption The 56-dimensional belief/history observation is sufficient for a near-optimal selection policy
Implicit in the PPO design (Section II); no feature ablation or observation-sufficiency analysis is reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance." pith.science (2026). https://pith.science/paper/USJFJ5H6

@misc{pith2026260722667,
  author       = {Pith},
  title        = {Pith review of: Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/USJFJ5H6}},
  note         = {Machine review of arXiv:2607.22667}
}

read the original abstract

This paper presents an information-gain-guided reinforcement-learning sensor-selection framework for single-vessel tracking in heterogeneous maritime sensor networks. The proposed approach is motivated by information-theoretic sensor management: instead of activating all sensors or repeatedly performing computationally expensive online expected-information-gain evaluation, a learned policy selects one tracking-relevant sensor at each decision epoch. A Bayesian sequential Monte Carlo tracker estimates the vessel state from noisy measurements and provides a belief representation for scheduling under nonlinear and non-Gaussian conditions. A Proximal Policy Optimization agent selects one of five sensors deployed in a georeferenced simulation of the CMMI Smart Marina testbed at Ayia Napa Marina, Cyprus. The agent observes belief-state, detection-history, coverage, sensor-geometry, and realized-information-gain features. The reward is defined as a realized-information-gain term gated by an observability mask. Final-test simulations compare the proposed framework with random single-sensor selection, always-on sensing using all sensors simultaneously, and the expected-information-gain sensor-selection baseline proposed in our previous work. Results show that the learned policy achieves tracking performance close to always-on sensing while activating only one sensor per decision time step and avoiding the computationally expensive online entropy search required by expected-information-gain selection.

Figures

Figures reproduced from arXiv: 2607.22667 by the authors.

**Figure 2.** Overview of the proposed IG-RLSS framework. At each decision step, the PPO agent selects one of 5 fixed sensors. The [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** Estimated arithmetic cost per decision step for IG [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 3.** Figure 3: Zone-wise tracking results on the final test split for IG [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 4 linked inside Pith

[5]

Adaptive entropy-driven sensor selection in a camera-lidar particle filter for single-vessel tracking,

A. Starodubov, Y . A. Prabowo, A. Hadjipieris, I. Kyriakides, and R. Galeazzi, “Adaptive entropy-driven sensor selection in a camera-lidar particle filter for single-vessel tracking,”arXiv preprint arXiv:2603.08457, 2026

arXiv 2026
[1]

Vessel identification and monitoring systems for maritime security,

S.-J. Chang, “Vessel identification and monitoring systems for maritime security,” inIEEE 37th Annual 2003 International Carnahan Conference on Security Technology, 2003. Proceedings.IEEE, 2003, pp. 66–70

2003
[2]

Sea-surface object detection based on electro-optical sensors: A review,

H. Lyu, Z. Shao, T. Cheng, Y . Yin, and X. Gao, “Sea-surface object detection based on electro-optical sensors: A review,”IEEE Intelligent Transportation Systems Magazine, vol. 15, no. 2, pp. 190–216, 2022

2022
[3]

A tutorial on particle filters for online nonlinear/non-Gaussian bayesian tracking,

M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for online nonlinear/non-Gaussian bayesian tracking,” IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174–188, Feb. 2002

2002
[4]

Agile target tracking based on greedy information gain,

I. Kyriakides, “Agile target tracking based on greedy information gain,” in2021 IEEE 12th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). IEEE, 2021, pp. 0204– 0208

2021
[6]

Prox- imal policy optimization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

arXiv 2017
[7]

Discriminative particle filter reinforcement learning for complex partial observations,

X. Ma, P. Karkus, D. Hsu, W. S. Lee, and N. Ye, “Discriminative particle filter reinforcement learning for complex partial observations,”ArXiv, vol. abs/2002.09884, 2020

arXiv 2002
[8]

Differentiable particle filters: End-to-end learning with algorithmic priors,

R. Jonschkowski, D. Rastogi, and O. Brock, “Differentiable particle filters: End-to-end learning with algorithmic priors,” inProceedings of Robotics: Science and Systems, Pittsburgh, Pennsylvania, Jun. 2018

2018

Show all 16 references

[9]

Information particle filter tree: An on- line algorithm for POMDPs with belief-based rewards on continuous domains,

J. Fischer and O. S. Tas, “Information particle filter tree: An on- line algorithm for POMDPs with belief-based rewards on continuous domains,” inProceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, H. Daum’e III ...

2020
[10]

A novel sensor scheduling algorithm based on deep reinforcement learning for bearing-only target tracking in UWSNs,

L. Zheng, M. Liu, S. Zhang, and J. Lan, “A novel sensor scheduling algorithm based on deep reinforcement learning for bearing-only target tracking in UWSNs,”IEEE/CAA Journal of Automatica Sinica, vol. 10, no. 4, pp. 1077–1079, 2023

2023
[11]

Sensor path planning using reinforcement learning,

F. Hoffmann, A. Charlish, M. Ritchie, and H. Griffiths, “Sensor path planning using reinforcement learning,” inProceedings of the IEEE 23rd International Conference on Information Fusion, 2020, pp. 1–8

2020
[12]

An improved Q- Learning-Based sensor-scheduling algorithm for multi-target tracking,

Z. Qu, X. Zhao, H. Xu, H. Tang, J. Wang, and B. Li, “An improved Q- Learning-Based sensor-scheduling algorithm for multi-target tracking,” Sensors, vol. 22, no. 18, p. 6972, 2022

2022
[13]

Learning to track dynamic targets in partially known environments,

H. Jeong, H. Hassani, M. Morari, D. D. Lee, and G. J. Pappas, “Learning to track dynamic targets in partially known environments,” 2020

2020
[14]

Policy learning for active target tracking over continuousse(3)trajectories,

P. Yang, S. Koga, A. Asgharivaskasi, and N. Atanasov, “Policy learning for active target tracking over continuousse(3)trajectories,” inProceed- ings of The 5th Annual Learning for Dynamics and Control Conference, ser. Proceedings of Machine Learning Research, N. Matni, M. Mora...

2023
[15]

Stable-baselines3: Reliable reinforcement learning implementa- tions,

A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dor- mann, “Stable-baselines3: Reliable reinforcement learning implementa- tions,”Journal of machine learning research, vol. 22, no. 268, pp. 1–8, 2021

2021
[16]

High- dimensional continuous control using generalized advantage estimation,

J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv:1506.02438, 2015

2015 arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [5]

Adaptive entropy-driven sensor selection in a camera-lidar particle filter for single-vessel tracking,

A. Starodubov, Y . A. Prabowo, A. Hadjipieris, I. Kyriakides, and R. Galeazzi, “Adaptive entropy-driven sensor selection in a camera-lidar particle filter for single-vessel tracking,”arXiv preprint arXiv:2603.08457, 2026

arXiv 2026

[2] [1]

Vessel identification and monitoring systems for maritime security,

S.-J. Chang, “Vessel identification and monitoring systems for maritime security,” inIEEE 37th Annual 2003 International Carnahan Conference on Security Technology, 2003. Proceedings.IEEE, 2003, pp. 66–70

2003

[3] [2]

Sea-surface object detection based on electro-optical sensors: A review,

H. Lyu, Z. Shao, T. Cheng, Y . Yin, and X. Gao, “Sea-surface object detection based on electro-optical sensors: A review,”IEEE Intelligent Transportation Systems Magazine, vol. 15, no. 2, pp. 190–216, 2022

2022

[4] [3]

A tutorial on particle filters for online nonlinear/non-Gaussian bayesian tracking,

M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorial on particle filters for online nonlinear/non-Gaussian bayesian tracking,” IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174–188, Feb. 2002

2002

[5] [4]

Agile target tracking based on greedy information gain,

I. Kyriakides, “Agile target tracking based on greedy information gain,” in2021 IEEE 12th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). IEEE, 2021, pp. 0204– 0208

2021

[6] [6]

Prox- imal policy optimization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

arXiv 2017

[7] [7]

Discriminative particle filter reinforcement learning for complex partial observations,

X. Ma, P. Karkus, D. Hsu, W. S. Lee, and N. Ye, “Discriminative particle filter reinforcement learning for complex partial observations,”ArXiv, vol. abs/2002.09884, 2020

arXiv 2002

[8] [8]

Differentiable particle filters: End-to-end learning with algorithmic priors,

R. Jonschkowski, D. Rastogi, and O. Brock, “Differentiable particle filters: End-to-end learning with algorithmic priors,” inProceedings of Robotics: Science and Systems, Pittsburgh, Pennsylvania, Jun. 2018

2018