Pith. sign in

REVIEW 3 major objections 5 minor 28 references

Optimizing Plastic Waste Collection in Water Bodies Using Heterogeneous Autonomous Surface Vehicles with Deep Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A deep-reinforcement-learned fleet of scout and cleaner boats collects more floating plastic than greedy, PSO, or pattern-based planners, especially in narrow, obstacle-heavy ports.

desk verdict A useful simulation study of DRL for heterogeneous cleanup fleets, but the headline performance gap may be inflated by selecting the DRL checkpoint on the same 100 episodes used to report final metrics. read the letter →

arxiv 2412.02316 v1 pith:C7JDAW32 submitted 2024-12-03 cs.RO cs.LG

Alejandro Mendoza Barrionuevo , Samuel Yanes Luis , Daniel Gutiérrez Reina , Sergio L. Toral Marín This is my paper · ORCID
classification cs.ROcs.LG
keywords heterogeneousmulti-robotsystemsautonomoussurfacevehiclesdeepreinforcementlearninginformativepathplanningDuelingDoubleQ-Learningplasticwastecollectionmulti-agentcoordinationwaterwaymonitoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a heterogeneous fleet of autonomous surface vehicles—fast scouts that map floating plastic with wide-range cameras and slower cleaners that collect it—coordinates more effectively under deep reinforcement learning than under classical heuristics. The proposed framework, Dueling Double Deep Q-Learning with a shared per-team network, uses the trash model as the nexus between teams: scouts are rewarded for updating the map and covering new ground, cleaners for collecting trash and staying near known waste. In the harder of two simulated layouts, a narrow-corridor sport wharf, the best learned policy cleans 91.13% of the trash against 72.11% for the greedy heuristic and 67.19% for particle swarm optimization, with smaller confidence intervals across random episodes. The authors conclude that model-free learned policies generalize across layouts without per-scenario heuristic tuning, which matters for real cleanup missions where waste drifts with wind and currents.

What carries the argument

The load-bearing mechanism is the shared contamination model $\hat{Y}$, updated at every step by any vehicle whose sensor radius $\rho$ reaches a node, and this model is part of the state input to both teams' neural networks, making it the nexus of cooperation. Policies are learned by Dueling Double Deep Q-Learning: two estimates of the action-value function reduce Q-value overestimation, a dueling stream separates state value from action advantage, and prioritized experience replay, shared within each team, stabilizes training. The reward design encodes the division of labor—cleaners receive a large weight for trash collected and a penalty proportional to distance to known trash, while scouts receive bonuses for covering unexplored nodes and for producing changes in the model. The state representation is a stack of six image-like matrices: navigability with coverage, the trash model at times $t$, $t-1$, and $t-2$, the observing agent's fading ten-step trail, and the positions of the other agents.

What would settle it

A concrete test: run the trained policy on a physical test course with known trash placements and measure PTC against both the simulated value and a greedy baseline; if detection misses or radio delays push the real learned-policy PTC at or below the greedy policy's in a corridor layout, the adaptability claim fails. A cheaper simulation-based check is to inject a realistic detection probability (for example 90% within radius $\rho$, consistent with the stated vision-model accuracy) and re-run the 100-episode evaluation; if the DDDQL+Greedy margin over greedy collapses, the result depends on the exact-perception assumption rather than on the learned coordination.

Watch

Extended reading notes

Core claim

The paper's central claim is that inter-team cooperation between scouts and cleaners can be learned end-to-end with model-free deep reinforcement learning, and that the learned policy outperforms benchmark heuristics on both the percentage of trash cleaned (PTC) and the accuracy of the shared contamination model. Cooperation is deliberately engineered through the trash model $\hat{Y}$: the cleaner team's decisions depend on the map the scout team produces, so the better the scouts map, the easier the cleaners' job becomes. In the challenging Scenario B, the DDDQL variant trained with greedy actions reaches 91.13% PTC with a 95% confidence interval of $\pm 2.38$, versus 72.11% ($\pm 7.10$) for greedy and 67.19% ($\pm 6.80$) for PSO, while also achieving the lowest final mean squared error (MSE) of all compared methods. The authors further claim that training with greedy actions and pre-filling the replay buffer improves final performance in intricate layouts, whereas in the open Scenario A the plain DDDQL already matches it, and that the millisecond-level inference time is negligible at real ASV speeds.

Load-bearing premise

The load-bearing premise is that every piece of trash inside a vehicle's sensor radius is detected exactly and shared instantly, since the simulation ignores detection errors, missed detections, communication delays, and range limits; if the real perception and networking chain deviates from this, the reported cleaning and mapping advantages may not reproduce on physical boats.

Editorial extensions

If this is right

  • A fixed, model-free DRL policy trained once transfers across structurally different port layouts without per-scenario tuning of heuristics.
  • In corridor-heavy environments the learned policy beats greedy by up to 26 percentage points in the percentage of trash cleaned (PTC), so cluttered waterways are exactly where this approach earns its complexity.
  • Fleet size can grow with little added learning cost, since agents within a team share one network and one experience replay.
  • The narrower confidence intervals of the DRL results mean more predictable performance across random starting positions, trash distributions, and wind conditions.
  • The roughly 19 ms per-fleet decision time is far below the second-scale movement of real ASVs, so the computational overhead does not block real-time use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the simulation treats detection within radius $\rho$ as exact, an immediate extension is to feed the scout reward with detector confidence or false-negative rates; if the 91% result relies on perfect perception, the gap over greedy may close when detection noise is injected.
  • The MSE advantage may partly be a side effect of faster cleaning—fewer remaining trash items make the map trivially easier to estimate—so an ablation with scouts mapping but cleaners disabled would isolate genuine mapping quality.
  • The greedy-seeding training recipe generalizes beyond trash collection: for off-policy DRL in sparse-reward robotic tasks, pre-filling the replay buffer with a strong heuristic's trajectories may speed learning more than random exploration.
  • The problem is framed as a finite cleanup, but persistent waste influx (a river continuously depositing plastic) would turn it into sustained monitoring; the current reward structure would need a coverage-ratio term to avoid the fleet quitting after the initial batch.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes a model-free deep reinforcement learning framework for coordinating two teams of autonomous surface vehicles: scouts that maintain a trash-density model and cleaners that collect waste. The method uses DDDQL with team-shared neural networks, an image-like state representation, and role-specific reward functions. Experiments compare two DRL training variants against lawn mower, random walker, PSO, and greedy baselines in two grid scenarios, reporting PTC and MSE over 100 episodes. The authors conclude that DRL-based algorithms outperform the benchmark heuristics, with the largest gap in the narrow-corridor Scenario B (DDDQL+Greedy reaches 91.13% PTC versus 72.11% for the greedy baseline).

Significance. If the empirical comparison is valid, the paper makes a useful contribution to multi-robot informative path planning: it demonstrates a concrete heterogeneous ASV coordination mechanism, provides a detailed simulation setup, and makes the code publicly available. The comparison against external heuristics gives the claim independent grounding, and the inclusion of confidence intervals is a positive feature. However, the headline quantitative claim rests on an evaluation protocol that appears to use the same 100 episodes for both checkpoint selection and final metric reporting, which can inflate the DRL advantage. The simulation also makes strong idealizations about perception and communication. The work is therefore of interest to the community, but the central claim needs a sounder evaluation before it can be accepted.

major comments (3)
  1. [Section IV, Table I and Fig. 6] The checkpoint selection protocol appears to use the same 100 episodes that are later used to compute all reported metrics. Section IV states: "The policy chosen is the one that obtains the best performance on average. For the evaluation process, the average results of the same 100 episodes have been recorded for each algorithm." If the 100 evaluation episodes are also used to select the DRL checkpoint, then the DRL results in Table I are maxima over however many checkpoints were considered, whereas each baseline receives a single score. This alone can inflate the reported DRL advantage and shrink the confidence intervals in Fig. 6, because the variability across checkpoints is not included. Please separate a validation set for checkpoint selection from a fresh test set for final metrics, and report the number of checkpoints considered and the variance over training seeds.
  2. [Section IV, baselines] The baseline algorithms are not specified sufficiently for reproduction. There is no description of the PSO swarm size, update rule, objective function, or replanning horizon, and the greedy baseline's behavior in the heterogeneous multi-agent setting is not defined (for example, how it breaks ties, how it coordinates the four agents, and how it uses the shared trash model). This matters because the claimed margin over these baselines is the main evidence for the DRL advantage. Please provide precise parameter settings or pseudocode for all baselines, including the lawn mower and random walker route-generation details.
  3. [Section III-B and Section I] The simulator assumes that every vehicle within radius rho updates the trash model exactly, with Y_hat[i,j] = Y[i,j], and that centralized communication has no delay or range limits. This is stated in Section III-B, but the introduction and contributions claim operation "under realistic constraints" and the conclusion generalizes to real ASV deployments. Because detection errors, missed detections, false positives, communication latency, and water currents beyond wind/random fluctuations are absent, the reported PTC and MSE margins should be interpreted as upper bounds for an idealized setting. The generalization claim should be tempered, or supported by a sensitivity analysis or a set of physical trials.
minor comments (5)
  1. [Section IV, paragraph after Table I] The text uses the acronym "PCT" in one sentence ("both in terms of average MSE and PCT"), while the metric is defined and used elsewhere as PTC.
  2. [Equation (1)] The target Q-value expression has an extra closing parenthesis after the argmax; please check the notation: Q_target(s', argmax_{a'}(Q(s'; theta)); theta-) should be written unambiguously.
  3. [Section III-B] The trash count K is sampled from N(60, 10^2), but a normal distribution can produce negative values; please specify how negative samples are handled.
  4. [Section IV, hyperparameters] The reward weights c_alpha, c_beta, c_gamma, c_delta and the greedy buffer prefill ratio are tuned by trial and error on the two test scenarios; a sensitivity analysis or validation on at least one unseen scenario would make the generalization claim more convincing.
  5. [Figure 2 caption] The caption contains a grammatical error: "two discretized scenario maps which differences in complexity" should read "with differences in complexity."

Circularity Check

1 steps flagged · score 6.0 of 10

DRL advantage in Table I is partly an artifact of selecting the best policy on the same 100 episodes used for evaluation.

  1. fitted input called prediction [Section IV (Experiments), paragraph on training duration, policy selection, and evaluation]
    "The policy chosen is the one that obtains the best performance on average. For the evaluation process, the average results of the same 100 episodes have been recorded for each algorithm. This diversity is what allows for generalization in the evaluation of the effectiveness of the algorithms. These episodes have never been seen during the DRL training process."

    The policy is chosen as the one with the best average performance, and the reported metrics are the average results on the same 100 episodes. Under the protocol as written, the DRL numbers in Table I and Fig. 6 are selected maxima over however many checkpoints or policy variants were compared on that fixed set, while each baseline (Greedy, PSO, etc.) is scored once on the same set. The headline DRL advantage (e.g., 91.13% vs 72.11% PTC in Scenario B) is therefore at least partly an artifact of fitting the policy to the evaluation episodes rather than a prediction on unseen data. The number of candidate policies evaluated is not reported, so the inflation cannot be quantified or ruled out.

full rationale

The methodological core of the paper—reward design, Double DQN updates, prioritized replay, and the comparison against Greedy, PSO, lawn-mower, and random-walker baselines—is not circular. The cleaner reward r_delta = C(p_n, Y) is aligned with PTC and the scout rewards encourage coverage and model updates, but this is standard reward engineering, and the external heuristics give the performance comparison independent grounding. The self-citations ([11], [15]) are to the authors' prior DDDQL/IPP work and are used for architectural precedent, not as a uniqueness or existence theorem, so they are not load-bearing. The one significant circularity is the evaluation protocol: the policy is chosen as the one with the best average performance and the reported metrics are the average results of the same 100 episodes. Under the natural reading, the DRL checkpoint is selected on the evaluation set, making the reported PTC/MSE a selected maximum over the candidate policies while each baseline gets a single score; the confidence intervals are then computed on the very episodes used for selection. This does not make the method's superiority true by definition, but it means the headline numbers in Table I are partly forced by the selection step and cannot be interpreted as an unbiased prediction on unseen episodes.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a simulated environment that abstracts away perception and communication errors, and on six hyperparameters (reward weights, buffer prefill ratio, and greedy action ratio) that were set by trial and error. These are the main things the reader is asked to accept without independent evidence.

free parameters (6)
  • c_alpha = 1
    Reward weight for the negative distance to nearest trash term; tuned by trial and error (Section IV).
  • c_beta = 2
    Reward weight for scout coverage reward; tuned by trial and error (Section IV).
  • c_gamma = 1
    Reward weight for scout model-change reward; tuned by trial and error (Section IV).
  • c_delta = 50
    Reward weight for trash collected by cleaners; tuned by trial and error (Section IV).
  • greedy_prefill_ratio = 20%
    Fraction of the replay buffer pre-filled with greedy experiences; evaluated through trial and error (Section IV).
  • greedy_action_ratio = 0.5
    Half of training actions selected by greedy, half random; a training choice described in Section IV.
assumptions (5)
  • domain assumption The environment is an 8-connected grid with discrete time; a vehicle moves one or two nodes per step and each step is one time unit.
    Section III-B defines the grid and movement.
  • domain assumption Trash detection is exact within each vehicle's sensor radius (rho=6 for scouts, rho=1 for cleaners).
    Section III-B sets Y_hat equal to Y within the radius, ignoring detection noise or missed detections.
  • domain assumption Communication is centralized, delay-free, and unlimited in range.
    Section III-B states 'shared among all agents through a central server, with communication and data processing delays and range being ignored.'
  • domain assumption All trash is present at the start of the episode and moves according to a constant wind plus random fluctuations.
    Section III-B: 'all trash is generated at the start of the episode, and no new items are introduced.'
  • ad hoc to paper The reward weights chosen by trial and error are sufficient for generalization across scenarios.
    Section IV: 'The reward function ponderations ... were determined by trial and error until satisfactory results were achieved.'

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimizing Plastic Waste Collection in Water Bodies Using Heterogeneous Autonomous Surface Vehicles with Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/C7JDAW32

@misc{pith2026241202316,
  author       = {Pith},
  title        = {Pith review of: Optimizing Plastic Waste Collection in Water Bodies Using Heterogeneous Autonomous Surface Vehicles with Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C7JDAW32}},
  note         = {Machine review of arXiv:2412.02316}
}
read the original abstract

This paper presents a model-free deep reinforcement learning framework for informative path planning with heterogeneous fleets of autonomous surface vehicles to locate and collect plastic waste. The system employs two teams of vehicles: scouts and cleaners. Coordination between these teams is achieved through a deep reinforcement approach, allowing agents to learn strategies to maximize cleaning efficiency. The primary objective is for the scout team to provide an up-to-date contamination model, while the cleaner team collects as much waste as possible following this model. This strategy leads to heterogeneous teams that optimize fleet efficiency through inter-team cooperation supported by a tailored reward function. Different trainings of the proposed algorithm are compared with other state-of-the-art heuristics in two distinct scenarios, one with high convexity and another with narrow corridors and challenging access. According to the obtained results, it is demonstrated that deep reinforcement learning based algorithms outperform other benchmark heuristics, exhibiting superior adaptability. In addition, training with greedy actions further enhances performance, particularly in scenarios with intricate layouts.

Figures

Figures reproduced from arXiv: 2412.02316 by the authors.

Figure 1
Figure 1. Prototype of a scout ASV equipped with a Zed 2i stereo [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Representation of the two discretized scenario maps [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example of a state representation, composed of six [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Visual representations of the terms used in the reward [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Graphical comparison over time of the percentage [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Comparison paths in a random episode at Scenario B [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 27 canonical work pages

  1. [1]

    The New Plastics Economy: Rethinking the future of plastics,

    World Economic Forum, Ellen MacArthur Foundation and McKinsey & Company, “The New Plastics Economy: Rethinking the future of plastics,” 2016

    work page 2016

  2. [2]

    A survey on multi-robot systems,

    Y . Cai and S. X. Yang, “A survey on multi-robot systems,” in World Automation Congress 2012 , pp. 1–6, IEEE, 2012

    work page 2012

  3. [3]

    Cooperative heterogeneous multi- robot systems: A survey,

    Y . Rizk, M. Awad, and E. W. Tunstel, “Cooperative heterogeneous multi- robot systems: A survey,” ACM Computing Surveys (CSUR) , vol. 52, no. 2, pp. 1–31, 2019

    work page 2019

  4. [4]

    A survey and critique of multiagent deep reinforcement learning,

    P. Hernandez-Leal, B. Kartal, and M. E. Taylor, “A survey and critique of multiagent deep reinforcement learning,” Autonomous Agents and Multi- Agent Systems, vol. 33, no. 6, pp. 750–797, 2019

    work page 2019

  5. [5]

    Learning- based methods for adaptive informative path planning,

    M. Popovi ´c, J. Ott, J. R ¨uckin, and M. J. Kochenderfer, “Learning- based methods for adaptive informative path planning,” Robotics and Autonomous Systems, vol. 179, p. 104727, 2024

    work page 2024

  6. [6]

    Aquafel-pso: An informative path planning for water resources monitoring using autonomous surface vehicles based on multi-modal pso and federated learning,

    M. Jara Ten Kathen, F. Peralta, P. Johnson, I. Jurado Flores, and D. G. Reina, “Aquafel-pso: An informative path planning for water resources monitoring using autonomous surface vehicles based on multi-modal pso and federated learning,” Ocean Engineering , vol. 311, p. 118787, 2024

    work page 2024

  7. [7]

    Water quality online modeling using multi-objective and multi-agent bayesian optimization with region partitioning,

    F. Peralta, D. G. Reina, and S. L. Toral, “Water quality online modeling using multi-objective and multi-agent bayesian optimization with region partitioning,” Mechatronics, vol. 91, p. 102953, 2023

    work page 2023

  8. [8]

    Deep reinforcement learning algorithms for path planning domain in grid-like environment,

    M. Grzelczak and P. Duch, “Deep reinforcement learning algorithms for path planning domain in grid-like environment,” Applied Sciences , vol. 11, no. 23, p. 11335, 2021

    work page 2021

Show all 28 references
  1. [9]

    Deep reinforcement learning with dynamic graphs for adaptive informative path planning,

    A. Vashisth, J. R ¨uckin, F. Magistri, C. Stachniss, and M. Popovi´c, “Deep reinforcement learning with dynamic graphs for adaptive informative path planning,” IEEE Robotics and Automation Letters , vol. 9, no. 9, pp. 7747–7754, 2024

    2024

  2. [10]

    Dynamic path planning of unknown environment based on deep reinforcement learning,

    X. Lei, Z. Zhang, and P. Dong, “Dynamic path planning of unknown environment based on deep reinforcement learning,”Journal of Robotics, vol. 2018, no. 1, p. 5781591, 2018

    2018

  3. [11]

    Deep reinforcement multiagent learning framework for infor- mation gathering with local gaussian processes for water monitoring,

    S. Yanes Luis, D. Shutin, J. Marchal G ´omez, D. G. Reina, and S. L. Toral, “Deep reinforcement multiagent learning framework for infor- mation gathering with local gaussian processes for water monitoring,” Advanced Intelligent Systems , vol. 6, no. 8, p. 2300850, 2024

    2024

  4. [12]

    Multi-robot path planning based on a deep reinforcement learning dqn algorithm,

    Y . Yang, L. Juntao, and P. Lingling, “Multi-robot path planning based on a deep reinforcement learning dqn algorithm,” CAAI Transactions on Intelligence Technology, vol. 5, no. 3, pp. 177–183, 2020

    2020

  5. [13]

    Collision avoidance for an unmanned surface ve- hicle using deep reinforcement learning,

    J. Woo and N. Kim, “Collision avoidance for an unmanned surface ve- hicle using deep reinforcement learning,” Ocean Engineering, vol. 199, p. 107001, 2020

    2020

  6. [14]

    Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,

    S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” in 2017 IEEE international conference on robotics and automation (ICRA), pp. 3389–3396, IEEE, 2017

    2017

  7. [15]

    Informative deep reinforcement path planning for heterogeneous au- tonomous surface vehicles in large water resources,

    A. M. Barrionuevo, S. Yanes Luis, D. G. Reina, and S. L. Toral, “Informative deep reinforcement path planning for heterogeneous au- tonomous surface vehicles in large water resources,” IEEE Access , vol. 12, pp. 71835–71852, 2024

    2024

  8. [16]

    Heterogeneous multi-agent deep reinforce- ment learning for traffic lights control,

    J. A. Calvo and I. Dusparic, “Heterogeneous multi-agent deep reinforce- ment learning for traffic lights control,” in Irish Conference on Artificial Intelligence and Cognitive Science , vol. 2259 of CEUR Workshop Proceedings, pp. 2–13, CEUR-WS.org, 2018

    2018

  9. [17]

    Asymmetric self-play-enabled intelligent heterogeneous multi- robot catching system using deep multiagent reinforcement learning,

    Y . Gao, J. Chen, X. Chen, C. Wang, J. Hu, F. Deng, and T. L. Lam, “Asymmetric self-play-enabled intelligent heterogeneous multi- robot catching system using deep multiagent reinforcement learning,” IEEE Transactions on Robotics , vol. 39, no. 4, pp. 2603–2622, 2023

    2023

  10. [18]

    Unmanned floating waste collecting robot,

    A. Akib, F. Tasnim, D. Biswas, M. B. Hashem, K. Rahman, A. Bhat- tacharjee, and S. A. Fattah, “Unmanned floating waste collecting robot,” in TENCON 2019 - 2019 IEEE Region 10 Conference (TENCON) , pp. 2645–2650, 2019

    2019

  11. [19]

    Development of water surface mobile garbage collector robot,

    N. Kamarudin, I. N. A. Mohd Nordin, D. Misman, N. Khamis, M. Razif, and F. Hanim, “Development of water surface mobile garbage collector robot,” Alinteri Journal of Agriculture Sciences , vol. 36, pp. 534–540, 06 2021

    2021

  12. [20]

    Automatic collaborative water surface coverage and cleaning strategy of UA V and USVs,

    T. Deng, X. Xu, Z. Ding, X. Xiao, M. Zhu, and K. Peng, “Automatic collaborative water surface coverage and cleaning strategy of UA V and USVs,” Digital Communications and Networks , 12 2022. doi: 10.1016/j.dcan.2022.12.014

    2022 doi

  13. [21]

    Flow: A dataset and bench- mark for floating waste detection in inland waters,

    Y . Cheng, J. Zhu, M. Jiang, J. Fu, C. Pang, P. Wang, K. Sankaran, O. Onabola, Y . Liu, D. Liu, and Y . Bengio, “Flow: A dataset and bench- mark for floating waste detection in inland waters,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10933–10942, 2021

    2021

  14. [22]

    Deep reinforcement learning with double q-learning,

    H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, p. 2094–2100, 2016

    2016

  15. [23]

    Bellman, Dynamic Programming

    R. Bellman, Dynamic Programming. Princeton University, NJ, USA: Princeton University Press, 1 ed., 1957

    1957

  16. [24]

    Prioritized experience replay,

    T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” in 4th International Conference on Learning Representations (ICLR), 2016

    2016

  17. [25]

    Dueling network architectures for deep reinforcement learning,

    Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas, “Dueling network architectures for deep reinforcement learning,” in 33rd International Conference on Machine Learning (ICML), pp. 1995–2003, 2016

    1995

  18. [26]

    Reward (mis)design for autonomous driving,

    W. B. Knox, A. Allievi, H. Banzhaf, F. Schmitt, and P. Stone, “Reward (mis)design for autonomous driving,” Artificial Intelligence , vol. 316, p. 103829, 2023

    2023

  19. [27]

    Designing rewards for fast learning

    H. Sowerby, Z. Zhou, and M. L. Littman, “Designing rewards for fast learning.” arXiv preprint arXiv:2205.15400, 2022. Available: https://arxiv.org/abs/2205.15400

    2022 arXiv

  20. [28]

    Greed is good: Near-optimal submodular maximization via greedy optimization,

    M. Feldman, C. Harshaw, and A. Karbasi, “Greed is good: Near-optimal submodular maximization via greedy optimization,” in Proceedings of the 2017 Conference on Learning Theory , vol. 65, pp. 758–784, 2017

    2017

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.