Pith. sign in

REVIEW 4 major objections 5 minor 149 references

Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Online transfer learning works between imperfect agents, no expert needed

desk verdict A promising but not yet established framework for expert-free online experience sharing; the core mechanism rests on an unverified cross-agent comparability of uncertainty scores. read the letter →

arxiv 2501.15495 v1 pith:5L46BCDE submitted 2025-01-26 cs.AI cs.LGcs.MA

Alberto Castagna This is my paper · ORCID
classification cs.AIcs.LGcs.MA
keywords multi-agentreinforcementlearningonlinetransferexperiencesharingepistemicuncertaintyrandomnetworkdistillationsourceselectiondeep
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This thesis claims that online transfer learning in multi-agent reinforcement learning does not require a fixed expert. It proposes EF-OnTL, a framework in which each learning agent periodically shares a buffer of past interactions, each labelled with epistemic uncertainty, and every receiving agent filters those interactions to fill its own knowledge gaps. Source agents are selected on the fly by average uncertainty or recent performance, and transferred tuples are chosen by the gap between source and target uncertainty or by expected surprise. The author argues that EF-OnTL improves on or matches action-advice baselines across four environments of increasing complexity, with larger gains in harder tasks. If correct, the framework gives a practical way for imperfect agents to accelerate each other's convergence without a designated teacher.

What carries the argument

The load-bearing component is the Transfer Core Engine, a per-agent module that exchanges transfer buffers, chooses a temporary source by Source Selection Criteria (lowest average uncertainty or best recent performance), and filters the incoming buffer by Transfer Content Selection Criteria based on delta-confidence and expected surprise approximated by TD-error. The second load-bearing component is sars-RND, an estimator that distils a random target network and reports prediction error conditioned on state, action, reward, and next state, which gives finer-grained epistemic uncertainty than state-only RND during online learning.

What would settle it

Run the same task with two agents whose sars-RND networks are initialised to produce deliberately different uncertainty scales, for example by initialising one predictor near zero and the other with large weights; if EF-OnTL's source selection or tuple ranking shifts substantially and performance degrades, the cross-agent comparability assumption fails. A second check is to compute delta-confidence after normalising each agent's uncertainties to a common distribution and measure how much the selected transferred batch changes; if the selected tuples differ, the raw scores are not doing the claimed work.

Watch

Extended reading notes

Core claim

EF-OnTL is an online experience-sharing framework where no agent is assumed to be an expert. At transfer steps every agent publishes its uncertainty-labelled interaction buffer; a common source is selected using either lowest average epistemic uncertainty or best recent cumulative reward. Each target then scores the source's tuples by delta-confidence, defined as target uncertainty minus source uncertainty on the same tuple, or by a combined normalised loss-and-confidence measure, and optimises its policy on the top B tuples. The thesis also introduces sars-RND, a Random Network Distillation variant that estimates epistemic uncertainty from the full state-action-reward-next-state tuple rather than from the state alone, so that rarely tried actions in familiar states still register as uncertain. Experiments across Cart-Pole, Multi-Team Predator-Prey, Half Field Offense, and a ride-sharing simulator show better or equal performance against no-transfer, OCMAS action-advice sharing, and RCMP expert-advice baselines, with improvements growing with environment complexity.

Load-bearing premise

The framework assumes that the uncertainty numbers produced by different agents' sars-RND estimators can be compared directly, so that a large gap between a target's and a source's uncertainty on the same tuple reliably marks the knowledge the target is missing.

Editorial extensions

If this is right

  • Agents can transfer knowledge online without any fixed expert, so target performance is not capped by a single teacher's policy quality.
  • Experience sharing can match or beat action-advice methods, meaning targets do not need their policies overridden to benefit from others' knowledge.
  • The reported performance gap between EF-OnTL and baselines grows with environment complexity, so the framework matters most where exploration is hardest.
  • sars-RND provides a usable fine-grained uncertainty signal during online training, unlike state-visit counters or state-only RND.
  • Each target receives a personalised batch tailored to its own knowledge gaps rather than a uniform broadcast from the source.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a direct test of the cross-agent comparability assumption would be to calibrate each agent's sars-RND uncertainty to a common scale before computing delta-confidence; if the transferred-batch ranking changes materially, the raw gap is not measuring the intended knowledge gap.
  • Beyond the paper: EF-OnTL could be extended to heterogeneous agents with different reward models by inserting a state and reward mapping function, a direction the thesis leaves open.
  • Beyond the paper: in large fleets, global source selection could be localised to communication neighbourhoods, and whether the loss in source quality is acceptable is an empirical question the thesis does not answer.
  • Beyond the paper: because the reported benefit scales with task difficulty, one would expect EF-OnTL to show its largest advantage in sparse-reward, high-dimensional MARL problems, which the four benchmarks only partially cover.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The thesis proposes Expert-Free Online Transfer Learning (EF-OnTL), a framework for online experience sharing among homogeneous RL agents without a fixed expert. Each agent maintains a transfer buffer of (s,a,r,s') tuples labelled by a new uncertainty estimator, sars-RND. At regular transfer steps, a source agent is chosen by average uncertainty or best performance, and each target samples B tuples selected by delta-confidence and/or TD-error (TCS). The thesis also introduces sars-RND, an extension of RND that takes action, reward, and next state into account. The evaluation compares EF-OnTL against no-transfer, OCMAS, RCMP, QMIX, and multi-agent PA-DDPG in Cart-Pole, MT-PP, HFO, and 3R2S, reporting better or equal convergence and final performance in these benchmarks.

Significance. EF-OnTL addresses a real gap: agent-to-agent transfer learning has mostly been teacher-student action advice, often with an expert, while experience sharing has mostly been offline pretraining. Showing that imperfect agents can mutually accelerate learning through filtered experience, with a public code repository (Section 5.3) and ablations of SS, TCS, and B (Section 6.3.2), would be a useful empirical contribution, especially for the real-world 3R2S demand scenario. No machine-checked proofs or parameter-free derivations are involved; the contribution is algorithmic and empirical, and the code is openly available. The claim is not derived from fitted equations, so the external circularity burden is low; the main concerns are internal validity: the uncertainty-filtering mechanism is not isolated from data-quantity effects, and the cross-agent comparability of sars-RND outputs is assumed rather than demonstrated.

major comments (4)
  1. [Section 4.5.2 and Eq. (4.1)] The central mechanism presupposes that raw sars-RND outputs from different agents are directly comparable, but no evidence establishes this. In Eq. (4.1) the source is selected by minimizing average uncertainty u_i, and in Section 4.5.2 the content filter is delta-confidence = u_target(oi_s) - u_source(oi_s). Each agent runs its own sars-RND predictor/target pair, initialized randomly and trained only on that agent's experience; RND-type prediction errors are uncalibrated and their scale depends on network initialization, optimizer trajectory, and data distribution. A 'common uncertainty estimator methodology' (Section 1.3) and a 'standardised architecture' (Section 4.5.2) do not by themselves put the estimators on a common scale. If one agent's estimator is systematically lower, that agent will always be selected as source and tuples will be filtered by estimator scale rather than by a transferable knowledge gap. Section 7.2 flags communication and homogeneity limitations but does not flag this calibration assumption. The internal comparison of sars-RND and RND in Section 3.4.1 only shows sensitivity to action changes within one estimator; it never checks whether scores are comparable across agents. I request either a cross-agent calibration study (for example, evaluating all estimators on a common held-out tuple set and reporting the scale distributions) or an explicit normalization step, together with a validation that delta-confidence ordering correlates with measured transfer utility.
  2. [Section 6.3.1, Figures 6.5-6.10] No random-experience transfer control is included. EF-OnTL is compared with no-transfer, OCMAS, RCMP, QMIX, and multi-agent PA-DDPG, but none of these isolates the effect of TCS-based uncertainty filtering from the simple effect of adding B extra tuples to the target's learning process. A control that transfers B uniformly random tuples from the source buffer would show whether delta-confidence and sars-RND provide benefit beyond data quantity or implicit regularization. Because the central claim is specifically that uncertainty-filtered experience accelerates convergence, this control is load-bearing and should be added.
  3. [Section 6.1.2 and Figures 6.5-6.10] The headline comparisons do not report confidence intervals or significance tests. Figure 6.7 reports evaluation metrics on 500 test episodes but no variance or seed-level test, and the number of independent runs used in Chapter 6 is not stated in the reviewed text. With per-environment tuned configurations in Table 6.3, point-estimate learning curves are insufficient to support the abstract's 'better or equal performance' claim across four benchmarks. The 50-run average curves in Section 3.3 show that the authors can compute such statistics; they should do so for the Chapter 6 comparisons.
  4. [Tables 6.3-6.4 and Section 6.3.2] EF-OnTL's hyperparameters are tuned per environment, and the sensitivity analysis covers only Cart-Pole and MT-PP. Transfer budget B varies from 5,000 to 100 across environments, and TF, SS, and TCS also change; for HFO and 3R2S only one configuration is evaluated. The ablation in Section 6.3.2 therefore does not establish that the reported improvements are robust to B and TF in the more complex environments, which weakens the generality of the 'better or equal' claim as stated in the abstract.
minor comments (5)
  1. [Section 3.4.1] The normalization used in Figure 3.12 is not defined; the text should state exactly how RND and sars-RND uncertainties are aligned before comparison, because the two estimators have different input dimensions and are not obviously on the same scale.
  2. [Abstract and throughout] There are numerous spacing and typographical errors, such as 'inEF-OnTL' and 'frameworkEF-OnTL' in the abstract, and inconsistent rendering of 'sars-RND' across the text; a careful copy-edit is needed.
  3. [Table 3.2] The column header 'sigma pass' is not defined; it appears to denote the standard deviation of passengers served per vehicle, but this should be stated explicitly in the caption or text.
  4. [Section 5.1.1] The sentence 'which is more than doubled on the server' is ambiguous: if the MT-PP training and testing phase takes about 7 hours on the laptop, the server should take less time, so either the direction or the wording needs correction.
  5. [Section 2.4 and Section 6.1.1] The RCMP baseline is described in Section 2.4 as a modification that replaces the single optimal teacher with multiple trained agents and majority voting, but Section 6.1.1 does not restate this modification; the evaluation section should define the exact RCMP setup used so the comparison is reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: EF-OnTL's transfer criteria are heuristics evaluated against external baselines; sars-RND is an estimator tested by a synthetic experiment, not a predicted quantity derived from itself.

full rationale

The thesis reports an empirical framework rather than a derivation chain that reduces to its own inputs. Source selection uses average uncertainty (Eq. 4.1) or best performance (Eq. 4.2), and content selection uses delta-confidence or surprise; these are stated design heuristics, not parameters fitted to the target result. The central claim—that EF-OnTL achieves better or equal performance than TL baselines—is supported by comparisons against OCMAS, RCMP, no-transfer, and other baselines on Cart-Pole, MT-PP, HFO, and 3R2S. No equation in the paper defines successful transfer in terms of delta-confidence itself, and no fitted value is later renamed as a prediction. The sars-RND estimator is introduced and tested in §3.4.1 through an artificial action-sensitivity experiment; that experiment is a sanity check of the estimator's behaviour, not a circular validation of EF-OnTL's final performance. Self-citations appear only in the dissemination section (§1.5) and are not load-bearing in any technical claim. The concern about cross-agent comparability of sars-RND uncertainty values is a legitimate assumption/validity issue, but it is not circularity: the paper does not define the transfer benefit to be identical to the delta-confidence score. Therefore no specific circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central claim relies on homogeneous-agent assumptions, fault-free communication, cross-agent uncertainty comparability, TD-error as surprise, and off-policy reuse of external data. Main free parameters are transfer budget, transfer frequency, sars-RND encoder size, and the TCS/SS choices; these are tuned per environment, not derived.

free parameters (6)
  • Transfer budget B per benchmark = Cart-Pole 5000, MT-PP 500, HFO 100, 3R2S 500 (Figs. 6.5-6.10, Table 6.3)
    Chosen by parameter sweeps; the main comparison uses a different budget in each environment, so outcome-dependent tuning affects the central claim.
  • Transfer frequency TF = Set per environment in the EF-OnTL configuration
    Controls how often transfer happens and therefore how much external experience is consumed; no sensitivity analysis is reported in the reproduced text.
  • sars-RND encoder size = 1024 features
    Selected from a sensitivity analysis in §3.4 (Fig. 3.9); affects uncertainty granularity and therefore source selection and filtering.
  • Delta-confidence median threshold (rnd delta-conf TCS) = Median of incoming batch
    Defines which tuples pass the random delta-confidence filter; no principled derivation is given.
  • Offline transfer uncertainty thresholds = 0.05, 0.02, 0.015 plus mean and median
    Preliminary study in §3.3; the best threshold changes when the budget doubles, so the threshold choice is not robust.
  • Loss and confidence equal weighting = 0.5 / 0.5 after normalization
    Used in the loss and conf TCS; no sensitivity analysis for the weighting is reported.
assumptions (5)
  • domain assumption Agents are homogeneous: same state and action space, same reward model, and equal learning capacity.
    Stated in §1.2: 'Agents are homogeneously defined over state and action space with same goal... equal capacity and ability to learn a policy.'
  • domain assumption Communication among agents is fault-free and effectively unlimited (n-to-n) with fixed cost.
    Stated in §1.1.1: 'This thesis assumes fault-free transmission between agents, disregarding potential breakdowns in communication channels.'
  • ad hoc to paper sars-RND uncertainty values are comparable across agents, so delta-conf is a meaningful knowledge-gap signal.
    Introduced in §4.5.2 as a subtraction of per-agent uncertainties without scale calibration or external validation.
  • domain assumption TD-error is a valid operationalization of expected surprise for selecting transferable tuples.
    Used in §4.5.2, following Gerstgrasser et al. and prioritized experience replay; not re-validated in this thesis.
  • domain assumption Off-policy updates with transferred external tuples do not systematically destabilize the target DQN or PA-DDPG.
    The design moves from PPO to off-policy methods so external experience can be replayed (§3.1, §5.2); the assumption is implicit and untested.
invented entities (1)
  • State Action Reward Next-state Random Network Distillation (sars-RND)
    purpose: Estimates epistemic uncertainty from the full (s, a, r, s') tuple so agents can compare confidence and filter transfer content.
    Demonstrated only through the paper's own synthetic MT-PP experiment (§3.4.1) and its use within EF-OnTL; no external calibration or formal check is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning." pith.science (2026). https://pith.science/paper/5L46BCDE

@misc{pith2026250115495,
  author       = {Pith},
  title        = {Pith review of: Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5L46BCDE}},
  note         = {Machine review of arXiv:2501.15495}
}
read the original abstract

Reinforcement Learning (RL) enables an intelligent agent to optimise its performance in a task by continuously taking action from an observed state and receiving a feedback from the environment in form of rewards. RL typically uses tables or linear approximators to map state-action tuples that maximises the reward. Combining RL with deep neural networks (DRL) significantly increases its scalability and enables it to address more complex problems than before. However, DRL also inherits downsides from both RL and deep learning. Despite DRL improves generalisation across similar state-action pairs when compared to simpler RL policy representations like tabular methods, it still requires the agent to adequately explore the state-action space. Additionally, deep methods require more training data, with the volume of data escalating with the complexity and size of the neural network. As a result, deep RL requires a long time to collect enough agent-environment samples and to successfully learn the underlying policy. Furthermore, often even a slight alteration to the task invalidates any previous acquired knowledge. To address these shortcomings, Transfer Learning (TL) has been introduced, which enables the use of external knowledge from other tasks or agents to enhance a learning process. The goal of TL is to reduce the learning complexity for an agent dealing with an unfamiliar task by simplifying the exploration process. This is achieved by lowering the amount of new information required by its learning model, resulting in a reduced overall convergence time...

Figures

Figures reproduced from arXiv: 2501.15495 by the authors.

Figure 1.1
Figure 1.1. RL research work published over the latest years (2000 - 2022) and available through [PITH_FULL_IMAGE:figures/full_fig_p024_1_1.png] view at source ↗
Figure 2.1
Figure 2.1. Single step of Reinforcement Learning process [6]. [PITH_FULL_IMAGE:figures/full_fig_p036_2_1.png] view at source ↗
Figure 2.2
Figure 2.2. Single step DRL process with deep NN. 2.1.3.1 Value-based Methods Value-based DRL methods use a NN to approximate a value function used to select the best action to take given a state. One of the first algorithms is Deep Q-Network (DQN), Alberto Castagna PhD Thesis [PITH_FULL_IMAGE:figures/full_fig_p043_2_2.png] view at source ↗
Figures from the paper (48 more)
Figure 2.3
Figure 2.3. Figure 2.3: DQN architecture at the top and dueling DQN architecture at the bottom. [PITH_FULL_IMAGE:figures/full_fig_p046_2_3.png]
Figure 2.4
Figure 2.4. Figure 2.4: PPO architecture [7]. DDPG is an actor-critic off-policy method used for tasks with continuous control space. PhD Thesis Alberto Castagna [PITH_FULL_IMAGE:figures/full_fig_p048_2_4.png]
Figure 2.5
Figure 2.5. Figure 2.5: Parameterised Action Deep Deterministic Policy Gradient (PA-DDPG) [8] Actor-Critic [PITH_FULL_IMAGE:figures/full_fig_p049_2_5.png]
Figure 3.1
Figure 3.1. Figure 3.1: PPO extended model [PITH_FULL_IMAGE:figures/full_fig_p071_3_1.png]
Figure 3.2
Figure 3.2. Figure 3.2: 9x9 predator-prey grid with one predator and two prey. The configuration used for this study consists of a 9x9 obstacle-free grid with a single predator and two prey, as shown in [PITH_FULL_IMAGE:figures/full_fig_p072_3_2.png]
Figure 3.3
Figure 3.3. Figure 3.3: SUMO simulator screenshots. single taxi vehicle, and each taxi vehicle is exclusively controlled by an agent. From a single agent perspective, the goal is to maximise the amount of ride-requests served throughout its life time while minimising the cumulated delay for…
Figure 3.4
Figure 3.4. Figure 3.4: Vehicle internal state and perception. PhD Thesis Alberto Castagna [PITH_FULL_IMAGE:figures/full_fig_p076_3_4.png]
Figure 3.5
Figure 3.5. Figure 3.5: Observed demand imbalance in New York Taxi dataset [9] trips between morning (7- [PITH_FULL_IMAGE:figures/full_fig_p078_3_5.png]
Figure 3.6
Figure 3.6. Figure 3.6: Comparison of offline experience sharing in ST-PP across different filtering thresholds. [PITH_FULL_IMAGE:figures/full_fig_p080_3_6.png]
Figure 3.7
Figure 3.7. Figure 3.7: Comparison of offline experience sharing in ST-PP against the teacher-student frame [PITH_FULL_IMAGE:figures/full_fig_p081_3_7.png]
Figure 3.8
Figure 3.8. Figure 3.8: Evaluation metrics in the 3R2S environment across 4 different scenarios. [PITH_FULL_IMAGE:figures/full_fig_p084_3_8.png]
Figure 3.9
Figure 3.9. Figure 3.9: Uncertainty estimator with different encoder size for RND model in MT-PP. [PITH_FULL_IMAGE:figures/full_fig_p085_3_9.png]
Figure 3.10
Figure 3.10. Figure 3.10: Two states in MT-PP environments. To ensure a fair comparison, the two estimator models share the same architecture except for the input layer. sars-RND input layer is composed by a few extra neurons to accommodate action, reward and next state. To assess the uncert…
Figure 3.11
Figure 3.11. Figure 3.11: 0 50 100 150 200 250 300 Step 0 40 80 120 Uncertainty State 1 RND sars-RND 0 50 100 150 200 250 300 Step State 2 [PITH_FULL_IMAGE:figures/full_fig_p088_3_11.png]
Figure 3.12
Figure 3.12. Figure 3.12: Zoomed in normalised uncertainties curves, with their 95% confidence interval, while [PITH_FULL_IMAGE:figures/full_fig_p088_3_12.png]
Figure 4
Figure 4. Figure 4: depicts [PITH_FULL_IMAGE:figures/full_fig_p093_4.png]
Figure 4.1
Figure 4.1. Figure 4.1: EF-OnTL agent components diagram. approximate epistemic uncertainty. State Action Reward Next-state Random Network Distillation (sars-RND) is covered in Section 4.6. The data processed by an EF-OnTL agent come from two distinct sources: an environ￾ment accessed onlin…
Figure 4.2
Figure 4.2. Figure 4.2: EF-OnTL simplified workflow in a two agents scenario. In detail, the source agent As and the related source transfer buffer T Bs are globally se￾lected, as the SS metrics, further presented in Section 4.5.1, do not consider the target agent to identify the source of …
Figure 4
Figure 4. Figure 4: divides the agent into fine-grained sub-modules. [PITH_FULL_IMAGE:figures/full_fig_p097_4.png]
Figure 4.3
Figure 4.3. Figure 4.3: divides the agent into fine-grained sub-modules. Learning Process LP Uncertainty Estimator UE Uncertainties Surprises Transfer Buffer o t and uncertainty u t Interactions in TBs Batch Received Transfer Core Engine Other Agents EF-OnTL Agent Interface State s t s t Ob…
Figure 4.4
Figure 4.4. Figure 4.4: Detail of EF-OnTL Transfer Core diagram. 4.5.1 Source Selection In TL approaches where an expert is readily available for unidirectional knowledge transfer, as the one described in Chapter 3, the source selection task is not required. However, in EF-OnTL, given the c…
Figure 4.5
Figure 4.5. Figure 4.5: sars-RND architecture. When used in an online TL scenario, RND is expected to lack crucial informations that are necessary to estimate the epistemic uncertainty of an agent, i.e., action taken and the Alberto Castagna PhD Thesis [PITH_FULL_IMAGE:figures/full_fig_p10…
Figure 6.1
Figure 6.1. Figure 6.1: Cart-Pole environment [10]. The Cart-Pole problem consists of an unstable pole placed vertically on top of a moving cart [142]. At initialisation time, the pole is balanced but subjected to gravity forces. The goal consists of preventing the pole from falling by movi…
Figure 6.2
Figure 6.2. Figure 6.2: Screenshot of Multi-Team Predator-Prey (MT-PP). [PITH_FULL_IMAGE:figures/full_fig_p122_6_2.png]
Figure 6.3
Figure 6.3. Figure 6.3: Screenshot of HFO game. The setup used in this work is a 3 vs 2 situation consisting of 3 offense players and 2 for the defense. The defense consist of a goalie and a single defender [PITH_FULL_IMAGE:figures/full_fig_p124_6_3.png]
Figure 6.4
Figure 6.4. Figure 6.4: Ride-requests membership over the 4 clusters identified by K-means [143]. [PITH_FULL_IMAGE:figures/full_fig_p126_6_4.png]
Figure 6.5
Figure 6.5. Figure 6.5: EF-OnTL (transfer based on Higher Delta Confidence (high ∆-conf ), budget: 5, 000 and SS : U) compared against RCMP, OCMAS and no-transfer in Cart-Pole. 6.3.1.2 Multi-Team Predator-Prey The second environment used to benchmark EF-OnTL against the baselines is MT-PP, …
Figure 6.6
Figure 6.6. Figure 6.6: Learning curves of EF-OnTL (high ∆-conf , budget: 500 and SS : U) compared against no-transfer, OCMAS, RCMP and QMIX in MT-PP. thanks to the expert teacher, achieves positive rewards in the first 200 episodes, while oth￾ers techniques experience severe negative rewar…
Figure 6.7
Figure 6.7. Figure 6.7: EF-OnTL and baselines evaluation metrics on 500 test episodes in MT-PP Both OCMAS and EF-OnTL show an improvement over no-transfer method across all the three metrics. On the contrary, RCMP agents score a lower average reward in the test episodes while having a winni…
Figure 6
Figure 6. Figure 6: reports the results. The [PITH_FULL_IMAGE:figures/full_fig_p133_6.png]
Figure 6.8
Figure 6.8. Figure 6.8: EF-OnTL(high ∆-conf , budget: 100 and SS : U) compared against no-transfer, RCMP and multi-agent PA-DDPG in HFO. Furthermore, the positive effect of EF-OnTL becomes noticeable during the second-half of the training phase. Sharing a restrained number of selected exper…
Figure 6
Figure 6. Figure 6: reports the average budget used by [PITH_FULL_IMAGE:figures/full_fig_p134_6.png]
Figure 6.9
Figure 6.9. Figure 6.9: RCMP analysis in HFO with uncertainty trend and budget consumption. network which outputs a single value and as such, it is impossible to predict a range and hence to normalise the estimated uncertainty as done previously in Cart-Pole and MT-PP. In fact, the RCMP unc…
Figure 6.10
Figure 6.10. Figure 6.10: EF-OnTL compared versus no-transfer, OCMAS and RCMP in the multi-brains 3R2S benchmark. EF-OnTL has a lower negative impact due to the transfer of experiences compared to the other TL baselines. Interestingly, both EF-OnTL and OCMAS exhibit a similar perform￾ance pr…
Figure 6.11
Figure 6.11. Figure 6.11: EF-OnTL learning curves in Cart-Pole with different parameters. In the top chart the source of transfer is selected by U, the agent with lower average uncertainty within the transfer buffer while, in the bottom one the source is selected by BP, the performance achie…
Figure 6.12
Figure 6.12. Figure 6.12: EF-OnTL learning curves organised by TCS used by a target agent to filter incoming knowledge in Cart-Pole. these scenarios, rnd ∆-conf generally exhibit the mildest impact on the overall system performance. While high ∆-conf and loss & conf demonstrate comparable pe…
Figure 6.13
Figure 6.13. Figure 6.13: EF-OnTL learning curves organised by the transfer budget B in Cart-Pole. environment, where despite the different transfer settings all configurations followed a similar trajectory. Therefore, to maintain the flow of the section, we present the results once, based o…
Figure 6.14
Figure 6.14. Figure 6.14: EF-OnTL learning curves in MT-PP with different parameters. In the top chart the source of transfer is selected by U, while, in the bottom chart the source is selected by BP. contains rows representing different evaluation metrics. The top row displays the average a…
Figure 6.15
Figure 6.15. Figure 6.15: Evaluation metrics on the final 500 episodes in the MT-PP environment across the 18 EF-OnTL configurations. in improved performance for the target agents, in MT-PP transferring 5, 000 tuples within each transfer step leads to an overall decrease in performance. For …
Figure 6.16
Figure 6.16. Figure 6.16: Single-Agent Same Train Data - train on 7 − 10am test on 6 − 9pm. per vehicle is higher for EF-OnTL than for the no-transfer method, while the distance trav￾elled in ride-sharing and the number of satisfied requests are slightly lower. In conclusion, the difference …
Figure 6.17
Figure 6.17. Figure 6.17: Single-Agent Mixed Train Data - train on both 7 − 10am and 6 − 9pm and test on 6 − 9pm. In the Single-Agent Mixed Train Data scenario, where transfer occurs between agents Alberto Castagna PhD Thesis [PITH_FULL_IMAGE:figures/full_fig_p149_6_17.png]
Figure 7.2
Figure 7.2. Figure 7.2: 0 1000 2000 3000 4000 5000 6000 7000 8000 Episode -11 -9 -7 -5 -3 -1 1 Per Episode Reward SS : U - B : 500 SS : BP - B : 500 SS : U - B : 1, 500 SS : BP - B : 1, 500 SS : U - B : 5, 000 SS : BP - B : 5, 000 (a) rnd ∆-conf 0 1000 2000 3000 4000 5000 6000 7000 8000 Epi…
Figure 7.1
Figure 7.1. Figure 7.1: EF-OnTL learning curves organised by TCS used by a target agent to filter incoming knowledge. Alberto Castagna PhD Thesis [PITH_FULL_IMAGE:figures/full_fig_p163_7_1.png]
Figure 7.2
Figure 7.2. Figure 7.2: EF-OnTL learning curves organised by the transfer budget B in MT-PP. PhD Thesis Alberto Castagna [PITH_FULL_IMAGE:figures/full_fig_p164_7_2.png]
Figure 7.3
Figure 7.3. Figure 7.3: Budget utilisation of OCMAS and RCMP in the Cart-Pole environment. This graph complements [PITH_FULL_IMAGE:figures/full_fig_p165_7_3.png]
Figure 7.4
Figure 7.4. Figure 7.4: Budget utilisation of OCMAS and RCMP in the MT-PP environment. This graph complements [PITH_FULL_IMAGE:figures/full_fig_p166_7_4.png]
Figure 6
Figure 6. Figure 6: , are based on a threshold of [PITH_FULL_IMAGE:figures/full_fig_p166_6.png]
Figure 7.5
Figure 7.5. Figure 7.5: RCMP performance in the MT-PP benchmark environment with different thresholds. higher reduces the prioritisation of the catch action and therefore prevents the capturing of adversarial prey. From the analysis presented in this section, it is evident that RCMP necessi…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

149 extracted references · 51 canonical work pages

  1. [1]

    Demand-responsive zone gen- eration for real-time vehicle rebalancing in ride-sharing fleets,

    A. Castagna, M. Guériau, G. Vizzari, and I. Dusparic, “Demand-responsive zone gen- eration for real-time vehicle rebalancing in ride-sharing fleets,” in11th International Workshop on Agents in Traffic and Transportation (ATT 2020) at ECAI 2020, 2020

    2020

  2. [2]

    Demand-responsive rebalancing zone generation for reinforcement learning- based on-demand mobility,

    ——, “Demand-responsive rebalancing zone generation for reinforcement learning- based on-demand mobility,”AI Communications, vol. 34, no. 1, pp. 73–88, 2021

    2021

  3. [3]

    Multi-agent transfer learning in reinforcement learning- based ride-sharing systems,

    A. Castagna and I. Dusparic, “Multi-agent transfer learning in reinforcement learning- based ride-sharing systems,” inInternational Conference on Agents and Artificial In- telligence (ICAART), 2022

    2022

  4. [4]

    Expert-free online transfer learning in multi-agent reinforcement learning,

    ——, “Expert-free online transfer learning in multi-agent reinforcement learning,” in (Accepted to) European Conference on Artificial Intelligence ECAI 2023, 2023

    2023

  5. [5]

    Continual model-based reinforcement learning for data efficient wireless network optimisation,

    C. Hasan, A. Agapitos, D. Lynch, A. Castagna, G. Cruciata, H. Wang, and A. Milen- ovic, “Continual model-based reinforcement learning for data efficient wireless network optimisation,” in(Accepted to) European Conference on Machine Learning and Prin- ciples and Practice of Knowledge Discovery in Databases, 06 2023

    2023

  6. [6]

    R. S. Sutton and A. G. Barto,Reinforcement learning: An introduction. MIT press, 2018

    2018

  7. [7]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    arXiv 2017

  8. [8]

    Deep reinforcement learning in parameterized action space,

    M. Hausknecht and P. Stone, “Deep reinforcement learning in parameterized action space,” arXiv preprint arXiv:1511.04143, 2015

    arXiv 2015

Show all 149 references
  1. [9]

    Tlc trip record data,

    NYC Taxi and Limousine Commission, “Tlc trip record data,” 2020. [Online]. Available: https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page

    2020

  2. [10]

    Openai gym,

    G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016

    2016

  3. [11]

    Sim-to-real transfer in deep reinforce- ment learning for robotics: a survey,

    W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforce- ment learning for robotics: a survey,” in2020 IEEE symposium series on computa- tional intelligence (SSCI). IEEE, 2020, pp. 737–744

    2020

  4. [12]

    Reinforcement learning in robotics: A survey,

    J. Kober, J. A. Bagnell, and J. Peters, “Reinforcement learning in robotics: A survey,” The International Journal of Robotics Research, vol. 32, no. 11, pp. 1238–1274, 2013

    2013

  5. [13]

    Survey of model-based reinforcement learning: Applications on robotics,

    A. S. Polydoros and L. Nalpantidis, “Survey of model-based reinforcement learning: Applications on robotics,”Journal of Intelligent & Robotic Systems, vol. 86, no. 2, pp. 153–173, 2017. Alberto Castagna PhD Thesis 150 BIBLIOGRAPHY

    2017

  6. [14]

    Reinforcement learning in economics and finance,

    A. Charpentier, R. Elie, and C. Remlinger, “Reinforcement learning in economics and finance,” Computational Economics, pp. 1–38, 2021

    2021

  7. [15]

    Recent advances in reinforcement learning in finance,

    B. M. Hambly, R. Xu, and H. Yang, “Recent advances in reinforcement learning in finance,” Recent Advances in Reinforcement Learning in Finance (November 24, 2021), 2021

    2021

  8. [16]

    Multi-agent reinforcement learning for traffic light con- trol,

    M. A. Wiering et al., “Multi-agent reinforcement learning for traffic light con- trol,” in Machine Learning: Proceedings of the Seventeenth International Conference (ICML’2000), 2000, pp. 1151–1158

    2000

  9. [17]

    Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation,

    H. Wei, G. Zheng, V. Gayah, and Z. Li, “Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation,”ACM SIGKDD Explorations Newsletter, vol. 22, no. 2, pp. 12–18, 2021

    2021

  10. [18]

    Shared autonomous mobilityondemand: Alearning-basedapproachanditsperformanceinthepresenceof traffic congestion,

    M. Guériau, F. Cugurullo, R. A. Acheampong, and I. Dusparic, “Shared autonomous mobilityondemand: Alearning-basedapproachanditsperformanceinthepresenceof traffic congestion,”IEEE Intelligent Transportation Systems Magazine, vol. 12, no. 4, pp. 208–218, 2020

    2020

  11. [19]

    Deep reinforce- ment learning for mobile 5g and beyond: Fundamentals, applications, and challenges,

    Z. Xiong, Y. Zhang, D. Niyato, R. Deng, P. Wang, and L.-C. Wang, “Deep reinforce- ment learning for mobile 5g and beyond: Fundamentals, applications, and challenges,” IEEE Vehicular Technology Magazine, vol. 14, no. 2, pp. 44–52, 2019

    2019

  12. [20]

    Rl-nsb: Reinforcement learning- based 5g network slice broker,

    V. Sciancalepore, X. Costa-Perez, and A. Banchs, “Rl-nsb: Reinforcement learning- based 5g network slice broker,” IEEE/ACM Transactions on Networking, vol. 27, no. 4, pp. 1543–1557, 2019

    2019

  13. [21]

    Deep reinforcement learning in medical imaging: A literature review,

    S. K. Zhou, H. N. Le, K. Luu, H. V. Nguyen, and N. Ayache, “Deep reinforcement learning in medical imaging: A literature review,”Medical image analysis, vol. 73, p. 102193, 2021

    2021

  14. [22]

    Reinforcement learning for intelligent healthcare applications: A survey,

    A. Coronato, M. Naeem, G. De Pietro, and G. Paragliola, “Reinforcement learning for intelligent healthcare applications: A survey,”Artificial Intelligence in Medicine, vol. 109, p. 101964, 2020

    2020

  15. [23]

    Deep reinforcement learning in medicine,

    A. Jonsson, “Deep reinforcement learning in medicine,”Kidney diseases, vol. 5, no. 1, pp. 18–22, 2019

    2019

  16. [24]

    Literature survey of statistical, deep and reinforce- ment learning in natural language processing,

    A. R. Sharma and P. Kaushik, “Literature survey of statistical, deep and reinforce- ment learning in natural language processing,” in2017 International Conference on Computing, Communication and Automation (ICCCA). IEEE, 2017, pp. 350–354

    2017

  17. [25]

    Survey on reinforcement learning for language processing,

    V. Uc-Cetina, N. Navarro-Guerrero, A. Martin-Gonzalez, C. Weber, and S. Wermter, “Survey on reinforcement learning for language processing,”Artificial Intelligence Re- view, pp. 1–33, 2022

    2022

  18. [26]

    Progressive neural networks,

    A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavuk- cuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,”arXiv preprint arXiv:1606.04671, 2016

    2016 arXiv

  19. [27]

    Progress & compress: A scalable framework for continual learning,

    J.Schwarz, W.Czarnecki, J.Luketina, A.Grabska-Barwinska, Y.W.Teh, R.Pascanu, and R. Hadsell, “Progress & compress: A scalable framework for continual learning,” in International conference on machine learning. PMLR, 2018, pp. 4528–4537. PhD Thesis Your Name BIBLIOGRAPHY 151

    2018

  20. [28]

    Deep reinforcement learning with knowledge transfer for online rides order dispatching,

    Z. Wang, Z. Qin, X. Tang, J. Ye, and H. Zhu, “Deep reinforcement learning with knowledge transfer for online rides order dispatching,” in2018 IEEE International Conference on Data Mining (ICDM). IEEE, 2018, pp. 617–626

    2018

  21. [29]

    Sharing knowledge in multi-task deep reinforcement learning,

    C. D’Eramo, D. Tateo, A. Bonarini, M. Restelli, J. Peterset al., “Sharing knowledge in multi-task deep reinforcement learning,” in8th International Conference on Learning Representations,{ICLR} 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view. net, 2020, pp. 1–11

    2020

  22. [30]

    Learning modular neural network policies for multi-task and multi-robot transfer,

    C. Devin, A. Gupta, T. Darrell, P. Abbeel, and S. Levine, “Learning modular neural network policies for multi-task and multi-robot transfer,” in2017 IEEE international conference on robotics and automation (ICRA). IEEE, 2017, pp. 2169–2176

    2017

  23. [31]

    Deep successor reinforce- ment learning,

    T. D. Kulkarni, A. Saeedi, S. Gautam, and S. J. Gershman, “Deep successor reinforce- ment learning,”arXiv preprint arXiv:1606.02396, 2016

    2016 arXiv

  24. [32]

    Overcoming cata- strophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming cata- strophic forgetting in neural networks,”Proceedings of the national academy of sci- ences, vol. 114, no. 13, pp. 3521–...

    2017

  25. [33]

    Towards knowledge transfer in deep re- inforcement learning,

    R. Glatt, F. L. Da Silva, and A. H. R. Costa, “Towards knowledge transfer in deep re- inforcement learning,” in2016 5th Brazilian Conference on Intelligent Systems (BRA- CIS). IEEE, 2016, pp. 91–96

    2016

  26. [34]

    Knowledge transfer for deep reinforcement learning with hier- archical experience replay,

    H. Yin and S. Pan, “Knowledge transfer for deep reinforcement learning with hier- archical experience replay,” inProceedings of the AAAI Conference on Artificial In- telligence, vol. 31, no. 1, 2017

    2017

  27. [35]

    Policy distillation,

    A. A. Rusu, S. G. Colmenarejo, C. Gulcehre, G. Desjardins, J. Kirkpatrick, R. Pas- canu, V. Mnih, K. Kavukcuoglu, and R. Hadsell, “Policy distillation,”arXiv preprint arXiv:1511.06295, 2015

    2015 arXiv

  28. [36]

    Discorl: Continual reinforcement learning via policy distillation,

    R. Traoré, H. Caselles-Dupré, T. Lesort, T. Sun, G. Cai, N. Díaz-Rodríguez, and D. Filliat, “Discorl: Continual reinforcement learning via policy distillation,”arXiv preprint arXiv:1907.05855, 2019

    1907 arXiv

  29. [37]

    Successor features for transfer in reinforcement learning,

    A. Barreto, W. Dabney, R. Munos, J. J. Hunt, T. Schaul, H. P. van Hasselt, and D. Sil- ver, “Successor features for transfer in reinforcement learning,”Advances in neural information processing systems, vol. 30, 2017

    2017

  30. [38]

    Deep q-learning from demonstrations,

    T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osbandet al., “Deep q-learning from demonstrations,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018

    2018

  31. [39]

    Transfer of samples in batch reinforcement learning,

    A. Lazaric, M. Restelli, and A. Bonarini, “Transfer of samples in batch reinforcement learning,” in Proceedings of the 25th international conference on Machine learning, 2008, pp. 544–551

    2008

  32. [40]

    Don’t start from scratch: Leveraging prior data to automate robotic reinforcement learning,

    H. R. Walke, J. H. Yang, A. Yu, A. Kumar, J. Orbik, A. Singh, and S. Levine, “Don’t start from scratch: Leveraging prior data to automate robotic reinforcement learning,” in Conference on Robot Learning. PMLR, 2023, pp. 1652–1662. Your Name PhD Thesis 152 BIBLIOGRAPHY

    2023

  33. [41]

    Reinforcement learning agents providing advice in complex video games,

    M. E. Taylor, N. Carboni, A. Fachantidis, I. Vlahavas, and L. Torrey, “Reinforcement learning agents providing advice in complex video games,”Connection Science, vol. 26, no. 1, pp. 45–63, 2014

    2014

  34. [42]

    Uncertainty-aware action advising for deep reinforcement learning agents,

    F. L. Da Silva, P. Hernandez-Leal, B. Kartal, and M. E. Taylor, “Uncertainty-aware action advising for deep reinforcement learning agents,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 04, 2020, pp. 5792–5799

    2020

  35. [43]

    Learning by reusing previous advice in teacher-student paradigm,

    C. Zhu, Y. Cai, H.-f. Leung, and S. Hu, “Learning by reusing previous advice in teacher-student paradigm,” in Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, 2020, pp. 1674–1682

    2020

  36. [44]

    Experience classification for transfer learning in traffic signal control,

    M. Norouzi, M. Abdoos, and A. L. Bazzan, “Experience classification for transfer learning in traffic signal control,”The Journal of Supercomputing, vol. 77, pp. 780– 795, 2021

    2021

  37. [45]

    Teaching on a budget: Agents advising agents in reinforce- ment learning,

    L. Torrey and M. Taylor, “Teaching on a budget: Agents advising agents in reinforce- ment learning,” in Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, 2013, pp. 1053–1060

    2013

  38. [46]

    A q-values sharing framework for multi-agent reinforcement learning under budget constraint,

    C. Zhu, H.-F. Leung, S. Hu, and Y. Cai, “A q-values sharing framework for multi-agent reinforcement learning under budget constraint,”ACM Transactions on Autonomous and Adaptive Systems (TAAS), vol. 15, no. 2, pp. 1–28, 2021

    2021

  39. [47]

    Parallel knowledge transfer in multi-agent reinforcement learn- ing,

    Y. Liang and B. Li, “Parallel knowledge transfer in multi-agent reinforcement learn- ing,” arXiv preprint arXiv:2003.13085, 2020

    2003 arXiv

  40. [48]

    Simultaneously learning and advising in multiagent reinforcement learning,

    F. L. Da Silva, R. Glatt, and A. H. R. Costa, “Simultaneously learning and advising in multiagent reinforcement learning,” inProceedings of the 16th conference on autonom- ous agents and multiagent systems, 2017, pp. 1100–1108

    2017

  41. [49]

    Parallel transfer learning in multi- agent systems: What, when and how to transfer?

    A. Taylor, I. Dusparic, M. Guériau, and S. Clarke, “Parallel transfer learning in multi- agent systems: What, when and how to transfer?” in2019 International Joint Con- ference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–8

    2019

  42. [50]

    Knowledge transfer in multi-agent reinforcement learning with incremental number of agents,

    W. Liu, L. Dong, J. Liu, and C. Sun, “Knowledge transfer in multi-agent reinforcement learning with incremental number of agents,” Journal of Systems Engineering and Electronics, vol. 33, no. 2, pp. 447–460, 2022

    2022

  43. [51]

    Multi-agent advisor q-learning,

    S. G. Subramanian, M. E. Taylor, K. Larson, and M. Crowley, “Multi-agent advisor q-learning,” Journal of Artificial Intelligence Research, vol. 74, pp. 1–74, 2022

    2022

  44. [52]

    Teaching on a budget in multi-agent deep reinforcement learning,

    E. Ilhan, J. Gow, and D. Perez-Liebana, “Teaching on a budget in multi-agent deep reinforcement learning,” in 2019 IEEE Conference on Games (CoG). IEEE, 2019, pp. 1–8

    2019

  45. [53]

    Selectively sharing experiences improves multi-agent reinforcement learning,

    M. Gerstgrasser, T. Danino, and S. Keren, “Selectively sharing experiences improves multi-agent reinforcement learning,” 2022

    2022

  46. [54]

    Ferber,Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence, 1st ed

    J. Ferber,Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence, 1st ed. USA: Addison-Wesley Longman Publishing Co., Inc., 1999

    1999

  47. [55]

    Multi-agent deep reinforcement learning for large-scale traffic signal control,

    T. Chu, J. Wang, L. Codecà , and Z. Li, “Multi-agent deep reinforcement learning for large-scale traffic signal control,”IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 3, pp. 1086–1095, 2020. PhD Thesis Your Name BIBLIOGRAPHY 153

    2020

  48. [56]

    Tar- mac: Targeted multi-agent communication,

    A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, and J. Pineau, “Tar- mac: Targeted multi-agent communication,” inInternational Conference on Machine Learning. PMLR, 2019, pp. 1538–1546

    2019

  49. [57]

    Learning attentional communication for multi-agent cooperation,

    J. Jiang and Z. Lu, “Learning attentional communication for multi-agent cooperation,” Advances in neural information processing systems, vol. 31, 2018

    2018

  50. [58]

    ⨿⌈-learning: A collaborative distributed strategy for multi-agent reinforcement learning through consensus + innovations,

    S. Kar, J. M. Moura, and H. V. Poor, “ ⨿⌈-learning: A collaborative distributed strategy for multi-agent reinforcement learning through consensus + innovations,” IEEE Transactions on Signal Processing, vol. 61, no. 7, pp. 1848–1862, 2013

    2013

  51. [59]

    Monotonic value function factorisation for deep multi-agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic value function factorisation for deep multi-agent reinforcement learning,” The Journal of Machine Learning Research, vol. 21, no. 1, pp. 7234–7284, 2020

    2020

  52. [60]

    Value-decomposition networks for cooperative multi-agent learning,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuylset al., “Value-decomposition networks for cooperative multi-agent learning,”arXiv preprint arXiv:1706.05296, 2017

    2017 arXiv

  53. [61]

    Multi-agent actor-critic for mixed cooperative-competitive environments,

    R.Lowe, Y.I.Wu, A.Tamar, J.Harb, O.PieterAbbeel, andI.Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,”Advances in neural in- formation processing systems, vol. 30, 2017

    2017

  54. [62]

    Q-rts: a real-time swarm intelligence based on multi-agent q-learning,

    M. Matta, G. C. Cardarilli, L. Di Nunzio, R. Fazzolari, D. Giardino, M. Re, F. Sil- vestri, and S. Spanò, “Q-rts: a real-time swarm intelligence based on multi-agent q-learning,” Electronics Letters, vol. 55, no. 10, pp. 589–591, 2019

    2019

  55. [63]

    Multi-agent reinforcement learning: Independent vs. cooperative agents,

    M. Tan, “Multi-agent reinforcement learning: Independent vs. cooperative agents,” in Proceedings of the tenth international conference on machine learning, 1993, pp. 330–337

    1993

  56. [64]

    A data-driven multi-agent autonomous voltage control framework using deep reinforcement learn- ing,

    S. Wang, J. Duan, D. Shi, C. Xu, H. Li, R. Diao, and Z. Wang, “A data-driven multi-agent autonomous voltage control framework using deep reinforcement learn- ing,” IEEE Transactions on Power Systems, vol. 35, no. 6, pp. 4644–4654, 2020

    2020

  57. [65]

    A centralized reinforcement learning method for multi-agent job schedul- ing in grid,

    M. Moradi, “A centralized reinforcement learning method for multi-agent job schedul- ing in grid,” in2016 6th International Conference on Computer and Knowledge En- gineering (ICCKE). IEEE, 2016, pp. 171–176

    2016

  58. [66]

    A new framework for multi-agent reinforcement learning–centralized train- ingandexplorationwithdecentralizedexecutionviapolicydistillation,

    G. Chen, “A new framework for multi-agent reinforcement learning–centralized train- ingandexplorationwithdecentralizedexecutionviapolicydistillation,” arXiv preprint arXiv:1910.09152, 2019

    1910 arXiv

  59. [67]

    Counterfactual multi-agent policy gradients,

    J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” inProceedings of the AAAI conference on artificial in- telligence, vol. 32, no. 1, 2018

    2018

  60. [68]

    Scalable central- ized deep multi-agent reinforcement learning via policy gradients,

    A. Khan, C. Zhang, D. D. Lee, V. Kumar, and A. Ribeiro, “Scalable central- ized deep multi-agent reinforcement learning via policy gradients,” arXiv preprint arXiv:1805.08776, 2018

    2018 arXiv

  61. [69]

    Cooperative multi-agent control us- ing deep reinforcement learning,

    J. K. Gupta, M. Egorov, and M. Kochenderfer, “Cooperative multi-agent control us- ing deep reinforcement learning,” in Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, Best Papers, São Paulo, Brazil, May 8-12, 2017, Revised Selected Papers 16. Springer, 2017, pp...

    2017

  62. [70]

    Exploration by random network distillation,

    Y. Burda, H. Edwards, A. Storkey, and O. Klimov, “Exploration by random network distillation,” arXiv preprint arXiv:1810.12894, 2018

    2018 arXiv

  63. [71]

    Surprise and curiosity for big data robotics,

    A. White, J. Modayil, and R. S. Sutton, “Surprise and curiosity for big data robotics,” in Workshops at the Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014

    2014

  64. [72]

    Minigrid & miniworld: Modular & custom- izable reinforcement learning environments for goal-oriented tasks,

    M. Chevalier-Boisvert, B. Dai, M. Towers, R. de Lazcano, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry, “Minigrid & miniworld: Modular & custom- izable reinforcement learning environments for goal-oriented tasks,” CoRR, vol. abs/2306.13831, 2023

    2023 arXiv

  65. [73]

    Half field offense: An environment for multiagent learning and ad hoc teamwork,

    M. Hausknecht, P. Mupparaju, S. Subramanian, S. Kalyanakrishnan, and P. Stone, “Half field offense: An environment for multiagent learning and ad hoc teamwork,” in AAMAS Adaptive Learning Agents (ALA) Workshop, vol. 3. sn, 2016

    2016

  66. [74]

    Microscopic traffic simulation using sumo,

    P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wießner, “Microscopic traffic simulation using sumo,” inThe 21st IEEE International Conference on Intelligent Transportation Systems. IEEE, 2018. [Online...

    2018

  67. [75]

    Multi-agent transfer learning in reinforcement learning- based ride-sharing systems,

    A. Castagna and I. Dusparic, “Multi-agent transfer learning in reinforcement learning- based ride-sharing systems,”arXiv preprint arXiv:2112.00424, 2021

    2021 arXiv

  68. [76]

    Markov decision processes,

    M. L. Puterman, “Markov decision processes,”Handbooks in operations research and management science, vol. 2, pp. 331–434, 1990

    1990

  69. [77]

    Planning and acting in partially observable stochastic domains,

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,”Artificial intelligence, vol. 101, no. 1-2, pp. 99–134, 1998

    1998

  70. [78]

    Actor-critic algorithms,

    V. Konda and J. Tsitsiklis, “Actor-critic algorithms,”Advances in neural information processing systems, vol. 12, 1999

    1999

  71. [79]

    Q-learning,

    C. J. Watkins and P. Dayan, “Q-learning,”Machine learning, vol. 8, pp. 279–292, 1992

    1992

  72. [80]

    Robot juggling: implementation of memory-based learn- ing,

    S. Schaal and C. G. Atkeson, “Robot juggling: implementation of memory-based learn- ing,” IEEE Control Systems Magazine, vol. 14, no. 1, pp. 57–71, 1994

    1994

  73. [81]

    Automatic programming of behavior-based robots using reinforcement learning,

    S. Mahadevan and J. Connell, “Automatic programming of behavior-based robots using reinforcement learning,” Artificial intelligence, vol. 55, no. 2-3, pp. 311–365, 1992

    1992

  74. [82]

    Q-learning for robot control,

    C. Gaskettet al., “Q-learning for robot control,” 2002

    2002

  75. [83]

    Reinforcement learning in the multi-robot domain,

    M. J. Matarić, “Reinforcement learning in the multi-robot domain,”Robot colonies, pp. 73–83, 1997

    1997

  76. [84]

    A reinforcement learning algorithm to train a tetris playing agent,

    P. Thiam, V. Kessler, and F. Schwenker, “A reinforcement learning algorithm to train a tetris playing agent,” inArtificial Neural Networks in Pattern Recognition: 6th IAPR TC 3 International Workshop, ANNPR 2014, Montreal, QC, Canada, October 6-8,

    2014

  77. [85]

    Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria,

    I. Erev and A. E. Roth, “Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria,”American economic review, pp. 848–881, 1998. PhD Thesis Your Name BIBLIOGRAPHY 155

    1998

  78. [86]

    Samod: Shared autonomous mobility-on-demand using decentralized reinforcement learning,

    M. Gueriau and I. Dusparic, “Samod: Shared autonomous mobility-on-demand using decentralized reinforcement learning,” inThe 21st IEEE International Conference on Intelligent Transportation Systems, 2018

    2018

  79. [87]

    Openai gym,

    G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,”arXiv preprint arXiv:1606.01540, 2016

    2016 arXiv

  80. [88]

    Challenges of real-world reinforcement learning: definitions, benchmarks and analysis,

    G. Dulac-Arnold, N. Levine, D. J. Mankowitz, J. Li, C. Paduraru, S. Gowal, and T. Hester, “Challenges of real-world reinforcement learning: definitions, benchmarks and analysis,”Machine Learning, vol. 110, no. 9, pp. 2419–2468, 2021

    2021

  81. [89]

    Challenges of real-world reinforce- ment learning,

    G. Dulac-Arnold, D. Mankowitz, and T. Hester, “Challenges of real-world reinforce- ment learning,”arXiv preprint arXiv:1904.12901, 2019

    1904 arXiv

  82. [90]

    A logical calculus of the ideas immanent in nervous activity,

    W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” The bulletin of mathematical biophysics, vol. 5, pp. 115–133, 1943

    1943

  83. [91]

    Learning representations by back-propagating errors,

    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,”nature, vol. 323, no. 6088, pp. 533–536, 1986

    1986

  84. [92]

    State-of-the-art in artificial neural network applications: A survey,

    O.I.Abiodun, A.Jantan, A.E.Omolara, K.V.Dada, N.A.Mohamed, andH.Arshad, “State-of-the-art in artificial neural network applications: A survey,”Heliyon, vol. 4, no. 11, p. e00938, 2018

    2018

  85. [93]

    Deep sparse rectifier neural networks,

    X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2011, pp. 315–323

    2011

  86. [94]

    A fast learning algorithm for deep belief nets,

    G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural computation, vol. 18, no. 7, pp. 1527–1554, 2006

    2006

  87. [95]

    Gradient-based learning applied to document recognition,

    Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998

    1998

  88. [96]

    Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,

    J. S. Bridle, “Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,” inNeurocomputing: Algorithms, architectures and applications. Springer, 1990, pp. 227–236

    1990

  89. [97]

    Adadelta: an adaptive learning rate method,

    M. D. Zeiler, “Adadelta: an adaptive learning rate method,” arXiv preprint arXiv:1212.5701, 2012

    2012 arXiv

  90. [98]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”arXiv pre- print arXiv:1412.6980, 2014

    2014 arXiv

  91. [99]

    Neural networks for machine learning lecture 6a overview of mini-batch gradient descent,

    G. Hinton, N. Srivastava, and K. Swersky, “Neural networks for machine learning lecture 6a overview of mini-batch gradient descent,”Cited on, vol. 14, no. 8, p. 2, 2012

    2012

  92. [100]

    Deep learning,

    Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,”nature, vol. 521, no. 7553, pp. 436–444, 2015

    2015

  93. [101]

    Playing atari with deep reinforcement learning,

    V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013. Your Name PhD Thesis 156 BIBLIOGRAPHY

    2013 arXiv

  94. [102]

    Human-level con- trol through deep reinforcement learning,

    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovskiet al., “Human-level con- trol through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  95. [103]

    Deep reinforcement learning with double q-learning,

    H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016

    2016

  96. [104]

    Prioritized experience replay,

    T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,”arXiv preprint arXiv:1511.05952, 2015

    2015 arXiv

  97. [105]

    Dueling network architectures for deep reinforcement learning,

    Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” inInternational conference on machine learning. PMLR, 2016, pp. 1995–2003

    2016

  98. [106]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,”arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  99. [107]

    Transfer in reinforcement learning: a framework and a survey,

    A. Lazaric, “Transfer in reinforcement learning: a framework and a survey,”Reinforce- ment Learning: State-of-the-Art, pp. 143–173, 2012

    2012

  100. [108]

    Playing games in the dark: An approach for cross-modality transfer in reinforcement learning,

    R. Silva, M. Vasco, F. S. Melo, A. Paiva, and M. Veloso, “Playing games in the dark: An approach for cross-modality transfer in reinforcement learning,”arXiv preprint arXiv:1911.12851, 2019

    1911 arXiv

  101. [109]

    Transferring instances for model-based reinforcement learning,

    M. E. Taylor, N. K. Jong, and P. Stone, “Transferring instances for model-based reinforcement learning,” inMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II 19. Springer, 20...

    2008

  102. [110]

    Using advice to transfer knowledge acquired in one reinforcement learning task to another,

    L. Torrey, T. Walker, J. Shavlik, and R. Maclin, “Using advice to transfer knowledge acquired in one reinforcement learning task to another,” inMachine Learning: ECML 2005: 16th European Conference on Machine Learning, Porto, Portugal, October 3-7,

    2005

  103. [111]

    Integrating reinforcement learning with human demonstrations of varying ability,

    M. E. Taylor, H. B. Suay, and S. Chernova, “Integrating reinforcement learning with human demonstrations of varying ability,” inThe 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, 2011, pp. 617–624

    2011

  104. [112]

    Improving reinforcement learning with human input

    M. E. Taylor and A. Borealis, “Improving reinforcement learning with human input.” in IJCAI, vol. 328, 2018, pp. 5724–5728

    2018

  105. [113]

    Reinforcement learning from demonstration through shaping,

    T. Brys, A. Harutyunyan, H. B. Suay, S. Chernova, M. E. Taylor, and A. Nowé, “Reinforcement learning from demonstration through shaping,” inTwenty-fourth in- ternational joint conference on artificial intelligence, 2015

    2015

  106. [114]

    Interactive reinforcement learning with dynamic reuse of prior knowledge from human/agent’s demonstration,

    Z. Wang and M. E. Taylor, “Interactive reinforcement learning with dynamic reuse of prior knowledge from human/agent’s demonstration,” arXiv preprint arXiv:1805.04493, 2018

    2018 arXiv

  107. [115]

    Improving reinforcement learning with confidence-based demonstrations

    ——, “Improving reinforcement learning with confidence-based demonstrations.” in IJCAI, 2017, pp. 3027–3033. PhD Thesis Your Name BIBLIOGRAPHY 157

    2017

  108. [116]

    Active deep q-learning with demonstration,

    S.-A. Chen, V. Tangkaratt, H.-T. Lin, and M. Sugiyama, “Active deep q-learning with demonstration,” Machine Learning, vol. 109, pp. 1699–1725, 2020

    2020

  109. [117]

    Overcoming ex- ploration in reinforcement learning with demonstrations,

    A. Nair, B. McGrew, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Overcoming ex- ploration in reinforcement learning with demonstrations,” in2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, pp. 6292–6299

    2018

  110. [118]

    Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards,

    M. Vecerik, T. Hester, J. Scholz, F. Wang, O. Pietquin, B. Piot, N. Heess, T. Rothörl, T. Lampe, and M. Riedmiller, “Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards,”arXiv preprint arXiv:1707.08817, 2017

    2017 arXiv

  111. [119]

    Learningcomplexdexterousmanipulationwithdeepreinforcementlearning and demonstrations,

    A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S.Levine, “Learningcomplexdexterousmanipulationwithdeepreinforcementlearning and demonstrations,” arXiv preprint arXiv:1709.10087, 2017

    2017 arXiv

  112. [120]

    Pre-training neural networks with human demonstrations for deep reinforcement learning,

    G. V. Cruz Jr, Y. Du, and M. E. Taylor, “Pre-training neural networks with human demonstrations for deep reinforcement learning,” arXiv preprint arXiv:1709.04083, 2017

    2017 arXiv

  113. [121]

    Pre-training with non-expert human demon- stration for deep reinforcement learning,

    V. Gabriel, Y. Du, and M. E. Taylor, “Pre-training with non-expert human demon- stration for deep reinforcement learning,”The Knowledge Engineering Review, vol. 34, 2019

    2019

  114. [122]

    Reinforcement learning from imperfect demonstrations,

    Y. Gao, H. Xu, J. Lin, F. Yu, S. Levine, and T. Darrell, “Reinforcement learning from imperfect demonstrations,” arXiv preprint arXiv:1802.05313, 2018

    2018 arXiv

  115. [123]

    Decaf: A deep convolutional activation feature for generic visual recognition,

    J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, “Decaf: A deep convolutional activation feature for generic visual recognition,” in International conference on machine learning. PMLR, 2014, pp. 647–655

    2014

  116. [124]

    Deep learning of representations for unsupervised and transfer learning,

    Y. Bengio, “Deep learning of representations for unsupervised and transfer learning,” in Proceedings of ICML workshop on unsupervised and transfer learning. JMLR Workshop and Conference Proceedings, 2012, pp. 17–36

    2012

  117. [125]

    Deep neural network with generative adversarial networks pre-training for brain tumor classification based on mr images,

    N. Ghassemi, A. Shoeibi, and M. Rouhani, “Deep neural network with generative adversarial networks pre-training for brain tumor classification based on mr images,” Biomedical Signal Processing and Control, vol. 57, p. 101678, 2020

    2020

  118. [126]

    Distributed rep- resentations of words and phrases and their compositionality,

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed rep- resentations of words and phrases and their compositionality,”Advances in neural information processing systems, vol. 26, 2013

    2013

  119. [127]

    Glove: Global vectors for word rep- resentation,

    J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word rep- resentation,” in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532–1543

    2014

  120. [128]

    Reinforcement learning with action-free pre-training from videos,

    Y. Seo, K. Lee, S. L. James, and P. Abbeel, “Reinforcement learning with action-free pre-training from videos,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesv...

    2022

  121. [129]

    Decoupling representation learning from reinforcement learning,

    A. Stooke, K. Lee, P. Abbeel, and M. Laskin, “Decoupling representation learning from reinforcement learning,” inInternational Conference on Machine Learning. PMLR, 2021, pp. 9870–9879

    2021

  122. [130]

    Faster reinforcement learning after pre- training deep networks to predict state dynamics,

    C. W. Anderson, M. Lee, and D. L. Elliott, “Faster reinforcement learning after pre- training deep networks to predict state dynamics,” in2015 International Joint Con- ference on Neural Networks (IJCNN). IEEE, 2015, pp. 1–7

    2015

  123. [131]

    Pretraining representations for data-efficient rein- forcement learning,

    M. Schwarzer, N. Rajkumar, M. Noukhovitch, A. Anand, L. Charlin, R. D. Hjelm, P. Bachman, and A. C. Courville, “Pretraining representations for data-efficient rein- forcement learning,”Advances in Neural Information Processing Systems, vol. 34, pp. 12686–12699, 2021

    2021

  124. [132]

    Awac: Accelerating online reinforcement learning with offline datasets,

    A. Nair, A. Gupta, M. Dalal, and S. Levine, “Awac: Accelerating online reinforcement learning with offline datasets,”arXiv preprint arXiv:2006.09359, 2020

    2006 arXiv

  125. [133]

    Aw-opt: Learning robotic skills with imitation andreinforcement at scale,

    Y. Lu, K. Hausman, Y. Chebotar, M. Yan, E. Jang, A. Herzog, T. Xiao, A. Irpan, M. Khansari, D. Kalashnikovet al., “Aw-opt: Learning robotic skills with imitation andreinforcement at scale,” in Conference on Robot Learning. PMLR, 2022, pp. 1078–1088

    2022

  126. [134]

    Offline-to-online reinforcement learning via balanced replay and pessimistic q-ensemble,

    S. Lee, Y. Seo, K. Lee, P. Abbeel, and J. Shin, “Offline-to-online reinforcement learning via balanced replay and pessimistic q-ensemble,” inConference on Robot Learning. PMLR, 2022, pp. 1702–1712

    2022

  127. [135]

    Deep exploration via boot- strapped dqn,

    I. Osband, C. Blundell, A. Pritzel, and B. Van Roy, “Deep exploration via boot- strapped dqn,” Advances in neural information processing systems, vol. 29, 2016

    2016

  128. [136]

    Unifying count-based exploration and intrinsic motivation,

    M. Bellemare, S. Srinivasan, G. Ostrovski, T. Schaul, D. Saxton, and R. Munos, “Unifying count-based exploration and intrinsic motivation,”Advances in neural in- formation processing systems, vol. 29, 2016

    2016

  129. [137]

    # exploration: A study of count-based exploration for deep reinforcement learning,

    H.Tang, R.Houthooft, D.Foote, A.Stooke, O.XiChen, Y.Duan, J.Schulman, F.De- Turck, and P. Abbeel, “# exploration: A study of count-based exploration for deep reinforcement learning,” Advances in neural information processing systems, vol. 30, 2017

    2017

  130. [138]

    Noisy networks for exploration,

    M. Fortunato, M. G. Azar, B. Piot, J. Menick, I. Osband, A. Graves, V. Mnih, R. Munos, D. Hassabis, O. Pietquinet al., “Noisy networks for exploration,”arXiv preprint arXiv:1706.10295, 2017

    2017 arXiv

  131. [139]

    Curiosity-driven exploration by self-supervised prediction,

    D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” in Proceedings of the 34th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., vol. 70. PML...

    2017

  132. [140]

    The orgins and evolution of predator-prey theory,

    A. A. Berryman, “The orgins and evolution of predator-prey theory,”Ecology, vol. 73, no. 5, pp. 1530–1535, 1992

    1992

  133. [141]

    Minimalistic gridworld environment for openai gym,

    M. Chevalier-Boisvert, L. Willems, and S. Pal, “Minimalistic gridworld environment for openai gym,” https://github.com/maximecb/gym-minigrid, 2018

    2018

  134. [142]

    Introduction to reinforcement learning,

    R. S. Sutton, A. G. Bartoet al., “Introduction to reinforcement learning,” 1998. PhD Thesis Your Name BIBLIOGRAPHY 159

    1998

  135. [143]

    Algorithm as 136: A k-means clustering algorithm,

    J. A. Hartigan and M. A. Wong, “Algorithm as 136: A k-means clustering algorithm,” Journal of the royal statistical society. series c (applied statistics), vol. 28, no. 1, pp. 100–108, 1979

    1979

  136. [144]

    Deeppool: Distributed model-free algorithm for ride-sharing using deep reinforcement learning,

    A. O. Al-Abbasi, A. Ghosh, and V. Aggarwal, “Deeppool: Distributed model-free algorithm for ride-sharing using deep reinforcement learning,”IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 12, pp. 4714–4727, 2019

    2019

  137. [145]

    On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment,

    J. Alonso-Mora, S. Samaranayake, A. Wallar, E. Frazzoli, and D. Rus, “On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment,” Proceedings of the National Academy of Sciences, vol. 114, no. 3, pp. 462–467, 2017. [Online]. Available: https://www.pnas.org/conte...

    2017

  138. [146]

    Proactive rebalancing and speed-up techniques for on-demand high capacity vehicle pooling,

    Y. Liu and S. Samaranayake, “Proactive rebalancing and speed-up techniques for on-demand high capacity vehicle pooling,” CoRR, 2019. [Online]. Available: http://arxiv.org/abs/1902.03374

    2019 arXiv

  139. [147]

    Larg/hfo: Robocup 2d half field offense,

    DurgeshSamant, “Larg/hfo: Robocup 2d half field offense,” ht- tps://github.com/LARG/HFO/blob/master/doc/manual.pdf, Accessed: 22 May 2023. Alberto Castagna PhD Thesis

    2023

  140. [2005]

    Springer, 2005, pp

    Proceedings 16. Springer, 2005, pp. 412–424

    2005

  141. [2014]

    Springer, 2014, pp

    Proceedings 6. Springer, 2014, pp. 165–170

    2014

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.