Pith. sign in

REVIEW 4 major objections 4 minor 53 references

Blockchain-assisted Demonstration Cloning for Multi-Agent Deep Reinforcement Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A probability-filtered expert-suggestion rule lets multi-agent PPO learn faster under sparse rewards while staying resilient to faulty and malicious shared models.

desk verdict The multi-expert action-suggestion idea is plausible and the experiments are consistent, but the paper's on-policy justification is wrong and the evidence lacks seeds, error bars, and code. read the letter →

arxiv 2501.10938 v1 pith:KRQWDQXH submitted 2025-01-19 cs.LG cs.AI

Ahmed Alagha , Jamal Bentahar , Hadi Otrok , Shakti Singh , Rabeb Mizouni This is my paper · ORCID
classification cs.LGcs.AI
keywords Multi-AgentDeepReinforcementLearningDemonstrationCloningProximalPolicyOptimizationSparseRewardsFederatedBlockchainSmartContractsImitation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MEDC, a method that lets previously trained expert models suggest actions to agents during Proximal Policy Optimization training, but accepts a suggestion only when the learner's own policy assigns that action a probability above a threshold $Q$. The claim is that this accelerates learning under sparse rewards compared with plain multi-agent reinforcement learning, and that it outperforms federated reinforcement learning, reward shaping, and imitation-learning-assisted reinforcement learning while staying resilient to faulty and malicious experts. Because agents still learn from the rewards they collect after executing expert-suggested actions, bad suggestions do not poison the policy; at worst the method matches a sparse-reward baseline. A companion blockchain-based model-sharing framework with smart contracts allocates shared models to requesters by reputation and environmental similarity.

What carries the argument

The load-bearing mechanism is the $Q$-threshold acceptance rule in Algorithm 2: given the agent's policy distribution $P$ over actions, an expert-suggested action $a_e$ is executed only if $P(a_e) > Q$, and the agent then takes the highest-probability surviving suggestion. This converts expert knowledge into exploration guidance while staying within the on-policy PPO framework, and it self-attenuates as the policy sharpens. The model-sharing side is carried by a smart-contract allocation mechanism that ranks shared models by the quality metric $\mathrm{QoS}_m = (\mathrm{Rep}_i \times \mathrm{Rep}_m) / D_m$, where $\mathrm{Rep}_i$ is the owner's reputation, $\mathrm{Rep}_m$ is the model's reputation, and $D_m$ is a weighted distance between the model's training environment and the requester's environment.

What would settle it

Record the importance ratios $r_t(\theta)$ from Equation (2) during MEDC episodes and check whether expert-suggested actions routinely push $r_t$ outside $[1-\varepsilon, 1+\varepsilon]$; if clipping activates mainly on expert-influenced steps, the reported speed-ups could stem from biased updates rather than better exploration. A cleaner control is to replace the expert suggestions in the same training schedule with actions sampled from the agent's own policy and see whether the learning-speed advantage disappears.

Watch

Extended reading notes

Core claim

The central discovery is that an expert's action suggestion can serve as a safe exploration hint for an on-policy learner if it passes a probability filter under the agent's current policy. By following only suggestions with $P(a_e) > Q$ and then learning from the rewards those actions actually produce, MEDC keeps the PPO update nominally on-policy while making sparse rewards reachable early in training. The threshold automatically reduces expert influence as the policy becomes confident, since fewer expert actions will clear $Q$. The paper reports that across target localization, fleet coordination, and maze cleaning, MEDC learns faster than federated averaging, reward shaping, and behavioral-cloning-assisted RL, and in the presence of faulty or malicious experts its performance degrades at most to the no-expert sparse-reward case.

Load-bearing premise

The method assumes that accepting an expert-chosen action whenever the agent's own policy assigns it probability above $Q$ keeps the PPO updates valid, even though that action was not sampled from the agent's policy and therefore makes the clipped objective in Equation (1) a biased estimate of the policy gradient.

Editorial extensions

If this is right

  • Different neural-network architectures can be reused as experts, since no weight averaging or distillation is needed.
  • Simple sparse rewards become viable, avoiding the engineering cost and local-optima risk of shaped reward functions.
  • Expert influence shrinks automatically over training, since a confident policy assigns low probability to suggested actions and they fail the $Q$ filter.
  • Faulty, random, biased, or malicious experts do not degrade learning below the plain sparse-reward baseline in the tested settings.
  • The blockchain-based sharing and allocation provides traceable, autonomous model exchange without a single trusted server, at gas costs the paper reports as low.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed threshold $Q=0.05$ is a hyperparameter; an adaptive $Q$ schedule that starts low and rises with policy confidence could extract more early guidance while further reducing bias later in training.
  • The acceptance rule is discrete-action-specific (argmax over surviving actions); for continuous action spaces it would need a density- or divergence-based filter, which is not addressed.
  • If the on-policy bias is material, a variant with off-policy correction or importance sampling for expert-suggested actions would make the speed-up claim robust to the distribution-shift objection.
  • The QoS allocation depends on the environment-attribute distance $D_m$ with application-specific weights, so applying the framework to a new problem requires defining those attributes and weights.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes MEDC, a multi-agent RL training protocol in which an agent's PPO actor occasionally (with probability R_E per episode) follows actions suggested by externally shared expert or semi-expert models, subject to a probability threshold Q under the current policy. The same framework includes a consortium-blockchain/IPFS model-sharing system with smart contracts for reputation-based allocation. The authors evaluate MEDC on target localization, fleet coordination, and maze cleaning, comparing it with MDRL using sparse rewards, FRL, reward shaping, and IL-assisted RL, and report faster learning and resilience to faulty/malicious experts.

Significance. The practical direction is useful: rather than averaging models, agents can use heterogeneous expert models as action proposers, which sidesteps architecture restrictions and reduces the impact of poisoned models. Strong points include the explicit Algorithms 1 and 2, the concrete smart-contract data structures and gas-cost analysis, and evaluation across three different MDRL applications. However, the core algorithmic claim that the Q-threshold preserves PPO's on-policy update is not justified, and the empirical evidence lacks multiple seeds and error bars, so the quantitative outperformance claims are not yet established.

major comments (4)
  1. [Section III-B, Algorithm 2, Eqs. (1)-(2)] The threshold Q does not restore on-policy validity. In Algorithm 2, during expert episodes the action stored and used in the PPO update is a_t = argmax(ExpActions) after zeroing actions with P_k[x] < Q; it is not sampled from the agent's old policy π_θ_old. The clipped surrogate in Eq. (1) is an unbiased estimator of the policy-gradient objective only when the stored actions are drawn from π_θ_old, but here the behavior policy is a mixture of π_θ_old and an expert-dependent deterministic selection with probability R_E. Consequently r_t(θ) in Eq. (2) is not a valid likelihood ratio and the gradient estimate is biased. The condition P_k[a_t] > Q only bounds the probability of the selected action under the current policy; it does not make the action distribution equal to P_k. With Q = 0.05 and a small discrete action space, early in training P_k is near-uniform, so most expert suggestions pass the threshold and the bias can be large precisely in the phase where Fig. 6 reports the largest speedup. The paper offers no off-policy correction (importance sampling, Retrace, or similar) and no ablation in which the same expert suggestions are fed back through on-policy sampling. This issue must be resolved before the learning-speed claim is accepted.
  2. [Section V-B, Section V-C, Figs. 6-11] The empirical evidence for the central claim is based on single learning curves. Figures 6-11 show one trace per condition, and the text does not state the number of independent runs, standard deviations, confidence intervals, or statistical tests. Given that the differences between MEDC and the reward-shaping baseline in Fig. 8 are modest and the curves are visibly noisy, the reported 'outperformance' and 'resiliency' conclusions are not statistically supported. The experiments should be repeated over multiple random seeds and reported with means and error bars, and the resilience claim should include a measure of variance over the faulty-expert configurations.
  3. [Table III, Section III-B (R_E and Q)] The method's two central hyperparameters, expert rate R_E = 0.1 and action threshold Q = 0.05, are fixed to single values with no sensitivity analysis. Section III-B describes qualitatively how Q should balance expert involvement and on-policy learning, but no experiment varies Q or R_E, and the resilience claim that MEDC is 'at worst' as good as MDRL-sparse is conditional on these values. Since both hyperparameters control how much expert behavior enters the policy update, the robustness of the reported gains is undemonstrated; a sweep over at least Q and R_E is needed.
  4. [Section V-C] The FRL comparison is set up in a way that may disadvantage the baseline by construction. In the target-localization benchmark, the global FRL model is formed by averaging models from eight users with deliberately different wall and agent counts (A1W0 through A3W3). This is a known failure mode for FedAvg-style aggregation, so the conclusion that MEDC 'outperforms FRL' is only established for a heterogeneous setting with naive aggregation, not as a general statement. The paper should either compare with an FRL variant designed for heterogeneous environments or explicitly scope the claim to naive FedAvg-style FRL.
minor comments (4)
  1. [Algorithms 1-2] Algorithm 1 and Algorithm 2 use `i % H == 0` inside the inner episode loop, but `i` is the outer loop counter over episodes, not a count of environment timesteps; this makes the update schedule inconsistent with the text that H = 4000 timesteps per update. The pseudocode should be corrected.
  2. [Algorithm 2] Algorithm 2 does not define the fallback when no expert action satisfies P_k[x] >= Q; in that case all entries of ExpActions are zero and argmax is undefined.
  3. [Eqs. (2)-(3)] The notation r_t is used both for the PPO probability ratio in Eq. (2) and for the reward in Eq. (3), which is confusing and should be disambiguated.
  4. [Section V-D] Section V-D repeatedly says 'costumers' instead of 'customers'; this typo should be fixed throughout the two application descriptions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MEDC's reported gains are empirical and do not reduce to fitted parameters or to the authors' prior citations.

full rationale

MEDC's learning procedure (Algorithm 2) is an algorithmic proposal evaluated empirically; the paper does not derive the reported learning curves from any equation whose inputs are fitted to those same curves. The Q threshold and RE are fixed hyperparameters (Table III), not calibrated to the episodic-length results, and no equation such as Eqs. (1)-(3) is shown to encode expert suggestions as a predicted speedup. The QoS allocation (Eq. 6) and reputation updates (Eqs. 4-5) are operational definitions for model selection, not predictions of the outcome metric. The target-localization testbed, CNN architecture, and semi-expert models are inherited from the authors' prior work [7], [28], but those citations supply an environment and a network, not the paper's central claim, so they are not load-bearing circular support. The main validity concern raised by a skeptical reading—that Algorithm 2 stores expert-chosen actions without an off-policy correction, so PPO's clipped surrogate may be biased—is a correctness risk, not a circularity step: nothing in the paper defines the speedup to be equal to the threshold or to the expert-action probability by construction. No uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known pattern was found. The comparison baselines (MDRL, FRL, RS, IL) are external benchmarks; even if one disagrees with their fairness or with the on-policy claim, that disagreement does not amount to circular reasoning.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method depends on two hand-set hyperparameters (RE=0.1, Q=0.05), on an unproved claim that thresholded expert actions keep PPO on-policy, and on assumptions that similar-environment experts transfer and that smart-contract reputation selects useful models. No new physical entities are introduced.

free parameters (3)
  • Expert Rate RE = 0.1
    Fraction of episodes in which expert-suggested actions are used; set by hand with no sensitivity analysis, and it directly shapes the resiliency result.
  • Action Probability Threshold Q = 0.05
    Minimum probability an expert-suggested action must have under the agent's policy for it to be followed; set by hand and asserted to preserve on-policy PPO without proof.
  • QoS attribute weights w_i = equal weights by default
    Weights on environment attributes in Eq. (7); not tuned or reported for the experiments, although they change model rankings.
assumptions (4)
  • domain assumption Trained expert policies from similar environments provide useful exploration signals in a new MDRL environment.
    Section III-B defines experts as trained on the exact or similar environment and uses them to guide exploration; no formal transfer guarantee is given.
  • ad hoc to paper Following expert actions with P(ae)>Q keeps PPO updates approximately on-policy.
    Section III-B and Algorithm 2 claim the threshold avoids unstable on-policy learning, but stored actions come from the expert, not from the agent's policy, so the PPO objective is not unbiased.
  • domain assumption Blockchain smart-contract reputation and QoS allocation produce suitable expert models for requesters.
    Section IV-A defines Rep and QoS; the paper provides gas costs but no experiments on allocation quality or adversarial reputation attacks.
  • domain assumption Sparse rewards plus 10% expert-guided episodes are sufficient to learn the three tasks.
    Hyperparameters RE=0.1 and Q=0.05 are used across all experiments without ablation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Blockchain-assisted Demonstration Cloning for Multi-Agent Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/KRQWDQXH

@misc{pith2026250110938,
  author       = {Pith},
  title        = {Pith review of: Blockchain-assisted Demonstration Cloning for Multi-Agent Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KRQWDQXH}},
  note         = {Machine review of arXiv:2501.10938}
}
read the original abstract

Multi-Agent Deep Reinforcement Learning (MDRL) is a promising research area in which agents learn complex behaviors in cooperative or competitive environments. However, MDRL comes with several challenges that hinder its usability, including sample efficiency, curse of dimensionality, and environment exploration. Recent works proposing Federated Reinforcement Learning (FRL) to tackle these issues suffer from problems related to model restrictions and maliciousness. Other proposals using reward shaping require considerable engineering and could lead to local optima. In this paper, we propose a novel Blockchain-assisted Multi-Expert Demonstration Cloning (MEDC) framework for MDRL. The proposed method utilizes expert demonstrations in guiding the learning of new MDRL agents, by suggesting exploration actions in the environment. A model sharing framework on Blockchain is designed to allow users to share their trained models, which can be allocated as expert models to requesting users to aid in training MDRL systems. A Consortium Blockchain is adopted to enable traceable and autonomous execution without the need for a single trusted entity. Smart Contracts are designed to manage users and models allocation, which are shared using IPFS. The proposed framework is tested on several applications, and is benchmarked against existing methods in FRL, Reward Shaping, and Imitation Learning-assisted RL. The results show the outperformance of the proposed framework in terms of learning speed and resiliency to faulty and malicious models.

Figures

Figures reproduced from arXiv: 2501.10938 by the authors.

Figure 1
Figure 1. A general overview of the proposed framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. At the beginning of each episode, an expert probability [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The framework is built using two smart contracts: [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 3
Figure 3. Figure 3: The proposed Blockchain-assisted model sharing framework for [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 4
Figure 4. Figure 4: The interactions between the users and smart contracts as part of the proposed framework. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The set of observations (b)-(f) given the snapshot of the environment [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The episodic length throughout the learning for an environment of 3 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: The episodic length throughout the learning for an environment of 3 [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 9
Figure 9. Figure 9: The episodic length throughout the learning for an environment of 3 [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 11
Figure 11. Figure 11: The episodic length throughout the learning for the (a) fleet [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 10
Figure 10. Figure 10: The episodic length throughout the learning for the (a) fleet [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 41 canonical work pages

  1. [1]

    A reinforcement learning method for human-robot collaboration in assembly tasks,

    R. Zhang, Q. Lv, J. Li, J. Bao, T. Liu, and S. Liu, “A reinforcement learning method for human-robot collaboration in assembly tasks,” Robotics and Computer-Integrated Manufacturing , vol. 73, p. 102227, 2022

    work page 2022

  2. [2]

    IoT sensor selection for target localization: A reinforcement learning based approach,

    M. Shurrab, S. Singh, R. Mizouni, and H. Otrok, “IoT sensor selection for target localization: A reinforcement learning based approach,” Ad Hoc Networks, vol. 134, p. 102927, 2022

    work page 2022

  3. [3]

    Self- supervised online and light-weight anomaly and event detection for iot devices,

    M. Abououf, R. Mizouni, S. Singh, H. Otrok, and E. Damiani, “Self- supervised online and light-weight anomaly and event detection for iot devices,” IEEE Internet of Things Journal , 2022

    work page 2022

  4. [4]

    Mastering the game of go without human knowledge,

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton et al. , “Mastering the game of go without human knowledge,” nature, vol. 550, no. 7676, pp. 354–359, 2017

    2017

  5. [5]

    Dota 2 with large scale deep reinforcement learning,

    C. Berner et al., “Dota 2 with large scale deep reinforcement learning,” arXiv preprint arXiv:1912.06680 , 2019

    arXiv 1912

  6. [6]

    Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s inter- sections,

    G.-P. Antonio and C. Maria-Dolores, “Multi-agent deep reinforcement learning to manage connected autonomous vehicles at tomorrow’s inter- sections,” IEEE Transactions on Vehicular Technology , vol. 71, no. 7, pp. 7033–7043, 2022

    2022

  7. [7]

    Target lo- calization using multi-agent deep reinforcement learning with proximal policy optimization,

    A. Alagha, S. Singh, R. Mizouni, J. Bentahar, and H. Otrok, “Target lo- calization using multi-agent deep reinforcement learning with proximal policy optimization,” Future Generation Computer Systems , vol. 136, pp. 342–357, 2022

    2022

  8. [8]

    Fault- tolerant federated reinforcement learning with theoretical guarantee,

    X. Fan, Y . Ma, Z. Dai, W. Jing, C. Tan, and B. K. H. Low, “Fault- tolerant federated reinforcement learning with theoretical guarantee,” Advances in Neural Information Processing Systems , vol. 34, pp. 1007– 1021, 2021

    work page 2021

Show all 53 references
  1. [9]

    Resource allocation in iot edge computing via concurrent federated reinforcement learning,

    Z. Tianqing, W. Zhou, D. Ye, Z. Cheng, and J. Li, “Resource allocation in iot edge computing via concurrent federated reinforcement learning,” IEEE Internet of Things Journal , vol. 9, no. 2, pp. 1414–1426, 2021

    2021

  2. [10]

    Federated reinforcement learn- ing for fast personalization,

    C. Nadiger, A. Kumar, and S. Abdelhak, “Federated reinforcement learn- ing for fast personalization,” in 2019 IEEE Second International Con- ference on Artificial Intelligence and Knowledge Engineering (AIKE) . IEEE, 2019, pp. 123–127

    2019

  3. [11]

    Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications,

    T. T. Nguyen et al. , “Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications,” IEEE transactions on cybernetics , vol. 50, no. 9, pp. 3826–3839, 2020

    2020

  4. [12]

    Learning to utilize shaping rewards: A new approach of reward shaping,

    Y . Hu, W. Wang, H. Jia, Y . Wang, Y . Chen, J. Hao, F. Wu, and C. Fan, “Learning to utilize shaping rewards: A new approach of reward shaping,” Advances in Neural Information Processing Systems , vol. 33, pp. 15 931–15 941, 2020

    2020

  5. [13]

    Graph convolutional recurrent networks for reward shaping in reinforcement learning,

    H. Sami, J. Bentahar, A. Mourad, H. Otrok, and E. Damiani, “Graph convolutional recurrent networks for reward shaping in reinforcement learning,” Information Sciences, vol. 608, pp. 63–80, 2022

    2022

  6. [14]

    Re- ward shaping using convolutional neural network,

    H. Sami, H. Otrok, J. Bentahar, A. Mourad, and E. Damiani, “Re- ward shaping using convolutional neural network,” arXiv preprint arXiv:2210.16956, 2022

    2022 arXiv

  7. [15]

    Federated learning for internet of things: A comprehensive survey,

    D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, and H. V . Poor, “Federated learning for internet of things: A comprehensive survey,” IEEE Communications Surveys & Tutorials, vol. 23, no. 3, pp. 1622–1658, 2021

    2021

  8. [16]

    Knowledge distillation: A survey,

    J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision , vol. 129, pp. 1789– 1819, 2021

    2021

  9. [17]

    Federated reinforcement learning: Techniques, applications, and open challenges,

    J. Qi, Q. Zhou, L. Lei, and K. Zheng, “Federated reinforcement learning: Techniques, applications, and open challenges,” arXiv preprint arXiv:2108.11887, 2021

    2021 arXiv

  10. [18]

    Overcoming exploration in reinforcement learning with demonstrations,

    A. Nair et al., “Overcoming exploration in reinforcement learning with demonstrations,” in 2018 IEEE international conference on robotics and automation (ICRA), 2018, pp. 6292–6299

    2018

  11. [19]

    Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards,

    M. Vecerik, T. Hester, J. Scholz, F. Wang, O. Pietquin, B. Piot, N. Heess, T. Roth ¨orl, T. Lampe, and M. Riedmiller, “Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards,” arXiv preprint arXiv:1707.08817 , 2017

    2017 arXiv

  12. [20]

    Federated learning meets blockchain in edge computing: Opportunities and challenges,

    D. C. Nguyen, M. Ding, Q.-V . Pham, P. N. Pathirana, L. B. Le, A. Seneviratne, J. Li, D. Niyato, and H. V . Poor, “Federated learning meets blockchain in edge computing: Opportunities and challenges,” IEEE Internet of Things Journal , vol. 8, no. 16, pp. 12 806–12 825, 2021

    2021

  13. [21]

    A context-aware blockchain-based crowdsourcing framework: Open challenges and op- portunities,

    M. Kadadha, S. Singh, R. Mizouni, and H. Otrok, “A context-aware blockchain-based crowdsourcing framework: Open challenges and op- portunities,” IEEE Access, 2022

    2022

  14. [22]

    Federated multiagent actor– critic learning for age sensitive mobile-edge computing,

    Z. Zhu, S. Wan, P. Fan, and K. B. Letaief, “Federated multiagent actor– critic learning for age sensitive mobile-edge computing,” IEEE Internet of Things Journal , vol. 9, no. 2, pp. 1053–1067, 2021

    2021

  15. [23]

    In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning,

    X. Wang, Y . Han, C. Wang, Q. Zhao, X. Chen, and M. Chen, “In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning,” Ieee Network, vol. 33, no. 5, pp. 156–165, 2019

    2019

  16. [24]

    When deep rein- forcement learning meets federated learning: Intelligent multitimescale resource management for multiaccess edge computing in 5g ultradense network,

    S. Yu, X. Chen, Z. Zhou, X. Gong, and D. Wu, “When deep rein- forcement learning meets federated learning: Intelligent multitimescale resource management for multiaccess edge computing in 5g ultradense network,” IEEE Internet of Things Journal, vol. 8, no. 4, pp. 2238–2251, 2020

    2020

  17. [25]

    Lifelong federated reinforcement learn- ing: a learning architecture for navigation in cloud robotic systems,

    B. Liu, L. Wang, and M. Liu, “Lifelong federated reinforcement learn- ing: a learning architecture for navigation in cloud robotic systems,” IEEE Robotics and Automation Letters , vol. 4, no. 4, pp. 4555–4562, 2019

    2019

  18. [26]

    Federated transfer reinforcement learning for autonomous driving,

    X. Liang, Y . Liu, T. Chen, M. Liu, and Q. Yang, “Federated transfer reinforcement learning for autonomous driving,” in Federated and Transfer Learning. Springer, 2022, pp. 357–371

    2022

  19. [27]

    Double Q-learning for radiation source detection,

    Z. Liu and S. Abbaszadeh, “Double Q-learning for radiation source detection,” Sensors, vol. 19, no. 4, p. 960, 2019

    2019

  20. [28]

    Multi- agent deep reinforcement learning with demonstration cloning for target localization,

    A. Alagha, R. Mizouni, J. Bentahar, H. Otrok, and S. Singh, “Multi- agent deep reinforcement learning with demonstration cloning for target localization,” IEEE Internet of Things Journal , 2023

    2023

  21. [29]

    Emergent tool use from multi-agent autocurricula,

    B. Baker, I. Kanitscheider, T. Markov, Y . Wu, G. Powell, B. McGrew, and I. Mordatch, “Emergent tool use from multi-agent autocurricula,” in 2020 Proc. Int. Conf. on Learning Representations (ICLR) , 2020

    2020

  22. [30]

    Principled reward shaping for rein- forcement learning via lyapunov stability theory,

    Y . Dong, X. Tang, and Y . Yuan, “Principled reward shaping for rein- forcement learning via lyapunov stability theory,” Neurocomputing, vol. 393, pp. 83–90, 2020

    2020

  23. [31]

    Primal: Pathfinding via reinforcement and imitation multi- agent learning,

    G. Sartoretti, J. Kerr, Y . Shi, G. Wagner, T. S. Kumar, S. Koenig, and H. Choset, “Primal: Pathfinding via reinforcement and imitation multi- agent learning,” IEEE Robotics and Automation Letters , vol. 4, no. 3, pp. 2378–2385, 2019

    2019

  24. [32]

    Primal 2: Pathfind- ing via reinforcement and imitation multi-agent learning-lifelong,

    M. Damani, Z. Luo, E. Wenzel, and G. Sartoretti, “Primal 2: Pathfind- ing via reinforcement and imitation multi-agent learning-lifelong,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 2666–2673, 2021

    2021

  25. [33]

    Multi-agent deep reinforcement learning: a survey,

    S. Gronauer and K. Diepold, “Multi-agent deep reinforcement learning: a survey,” Artificial Intelligence Review , pp. 1–49, 2021

    2021

  26. [34]

    Proximal policy optimization algorithms,

    J. Schulman et al. , “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  27. [35]

    High- dimensional continuous control using generalized advantage estimation,

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” in 2016 Proc. Int. Conf. on Learning Representations (ICLR) , 2016

    2016

  28. [36]

    Roulette-wheel selection via stochastic acceptance,

    A. Lipowski and D. Lipowska, “Roulette-wheel selection via stochastic acceptance,” Physica A: Statistical Mechanics and its Applications , vol. 391, no. 6, pp. 2193–2196, 2012

    2012

  29. [37]

    Influence-and interest- based worker recruitment in crowdsourcing using online social net- works,

    A. Alagha, S. Singh, H. Otrok, and R. Mizouni, “Influence-and interest- based worker recruitment in crowdsourcing using online social net- works,” IEEE Transactions on Network and Service Management , 2022

    2022

  30. [38]

    On- chain behavior prediction machine learning model for blockchain-based 14 crowdsourcing,

    M. Kadadha, H. Otrok, R. Mizouni, S. Singh, and A. Ouali, “On- chain behavior prediction machine learning model for blockchain-based 14 crowdsourcing,” Future Generation Computer Systems , vol. 136, pp. 170–181, 2022

    2022

  31. [39]

    Ipfs-content addressed, versioned, p2p file system,

    J. Benet, “Ipfs-content addressed, versioned, p2p file system,” arXiv preprint arXiv:1407.3561, 2014

    2014 arXiv

  32. [40]

    An optimization and auction-based incentive mechanism to maximize social welfare for mobile crowdsourcing,

    Y . Wang, Z. Cai, Z.-H. Zhan, Y .-J. Gong, and X. Tong, “An optimization and auction-based incentive mechanism to maximize social welfare for mobile crowdsourcing,” IEEE Transactions on Computational Social Systems, vol. 6, no. 3, pp. 414–429, 2019

    2019

  33. [41]

    A worker-selection incentive mechanism for optimizing platform-centric mobile crowdsourcing sys- tems,

    Y . Wang, Y . Gao, Y . Li, and X. Tong, “A worker-selection incentive mechanism for optimizing platform-centric mobile crowdsourcing sys- tems,” Computer Networks, vol. 171, p. 107144, 2020

    2020

  34. [42]

    Auction fever: Rising revenue in second-price auction formats,

    K.-M. Ehrhart, M. Ott, and S. Abele, “Auction fever: Rising revenue in second-price auction formats,” Games and Economic Behavior , vol. 92, pp. 206–227, 2015

    2015

  35. [43]

    Path planning and scheduling for a fleet of autonomous vehicles,

    E. Xidias, P. Zacharia, and A. Nearchou, “Path planning and scheduling for a fleet of autonomous vehicles,” Robotica, vol. 34, no. 10, pp. 2257– 2273, 2016

    2016

  36. [44]

    Autonomous vehicle fleet coordination with deep reinforce- ment learning,

    C. Punma, “Autonomous vehicle fleet coordination with deep reinforce- ment learning,” 2018

    2018

  37. [45]

    Multi-agent reinforcement learning with di- rected exploration and selective memory reuse,

    S. Jiang and C. Amato, “Multi-agent reinforcement learning with di- rected exploration and selective memory reuse,” in Proceedings of the 36th annual ACM symposium on applied computing, 2021, pp. 777–784

    2021

  38. [46]

    Data-driven dynamic active node selection for event localization in IoT applications-a case study of radiation localization,

    A. Alagha, S. Singh, R. Mizouni, A. Ouali, and H. Otrok, “Data-driven dynamic active node selection for event localization in IoT applications-a case study of radiation localization,” IEEE Access , vol. 7, pp. 16 168– 16 183, 2019

    2019

  39. [47]

    Reinforcement learn- ing framework for uav-based target localization applications,

    M. Shurrab, R. Mizouni, S. Singh, and H. Otrok, “Reinforcement learn- ing framework for uav-based target localization applications,” Internet of Things, p. 100867, 2023

    2023

  40. [48]

    A predictive target tracking framework for iot using cnn–lstm,

    L. A. Hussain, S. Singh, R. Mizouni, H. Otrok, and E. Damiani, “A predictive target tracking framework for iot using cnn–lstm,” Internet of Things, vol. 22, p. 100744, 2023

    2023

  41. [49]

    RFLS-resilient fault- proof localization system in IoT and crowd-based sensing applications,

    A. Alagha, S. Singh, H. Otrok, and R. Mizouni, “RFLS-resilient fault- proof localization system in IoT and crowd-based sensing applications,” Journal of Network and Computer Applications , vol. 170, 2020

    2020

  42. [50]

    SDRS: A stable data-based recruitment system in IoT crowdsensing for localiza- tion tasks,

    A. Alagha, R. Mizouni, S. Singh, H. Otrok, and A. Ouali, “SDRS: A stable data-based recruitment system in IoT crowdsensing for localiza- tion tasks,” Journal of Network and Computer Applications , vol. 177, p. 102968, 2021

    2021

  43. [51]

    A uav- assisted search and localization strategy in non-line-of-sight scenarios,

    B. Yuan, R. He, B. Ai, R. Chen, G. Wang, J. Ding, and Z. Zhong, “A uav- assisted search and localization strategy in non-line-of-sight scenarios,” IEEE Internet of Things Journal , 2022

    2022

  44. [52]

    A matching game-based crowdsourcing framework for last-mile delivery: Ground-vehicles and unmanned-aerial vehicles,

    H. Abualola, R. Mizouni, H. Otrok, S. Singh, and H. Barada, “A matching game-based crowdsourcing framework for last-mile delivery: Ground-vehicles and unmanned-aerial vehicles,” Journal of Network and Computer Applications, vol. 213, p. 103601, 2023

    2023

  45. [53]

    Lenet-5, convolutional neural networks,

    Y . LeCun et al. , “Lenet-5, convolutional neural networks,” URL: http://yann. lecun. com /exdb/lenet, vol. 20, no. 5, p. 14, 2015

    2015

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.