Pith. sign in

REVIEW 12 cited by

Federated Reinforcement Learning: Techniques, Applications, and Open Challenges

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2108.11887 v2 pith:VW2FL7ID submitted 2021-08-26 cs.LG cs.AI

Jiaju Qi , Qihao Zhou , Lei Lei , Kan Zheng This is my paper · ORCID
classification cs.LGcs.AI
keywords learningfederatedreinforcementalgorithmsopenaccordingadditionadvantages
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

This paper presents a comprehensive survey of Federated Reinforcement Learning (FRL), an emerging and promising field in Reinforcement Learning (RL). Starting with a tutorial of Federated Learning (FL) and RL, we then focus on the introduction of FRL as a new method with great potential by leveraging the basic idea of FL to improve the performance of RL while preserving data-privacy. According to the distribution characteristics of the agents in the framework, FRL algorithms can be divided into two categories, i.e. Horizontal Federated Reinforcement Learning (HFRL) and Vertical Federated Reinforcement Learning (VFRL). We provide the detailed definitions of each category by formulas, investigate the evolution of FRL from a technical perspective, and highlight its advantages over previous RL algorithms. In addition, the existing works on FRL are summarized by application fields, including edge computing, communication, control optimization, and attack detection. Finally, we describe and discuss several key research directions that are crucial to solving the open problems within FRL.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Safe-EF: Error Feedback for Nonsmooth Constrained Optimization

    cs.LG 2025-05 conditional novelty 7.0 of 10

    Safe-EF achieves the optimal O(RM/√(δT)) rate, up to constants, for non-smooth convex distributed optimization with contractive compression and safety constraints, and the matching lower bound is established.

  2. On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representations

    cs.LG 2024-11 conditional novelty 7.0 of 10

    The paper proves that personalized federated temporal-difference learning with a shared linear representation converges at rate O(1/(N^{2/3} T^{2/3})), yielding linear speedup in the number of agents under Markovian noise.

  3. Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity

    cs.LG 2026-04 conditional novelty 6.0 of 10

    Personalized local running-mean/variance observation normalization prevents weight-norm overshadowing in FedAvg and improves FedRL-PPO on heterogeneous MuJoCo morphology variants.

  4. Federated Reinforcement Learning in Heterogeneous Environments

    cs.LG 2025-07 reject novelty 5.0 of 10

    FedRQ adds a robustness term to federated Q-learning and claims convergence to an optimal worst-case policy over heterogeneous local environments, but the proof has a reversed inequality.

  5. Heterogeneous Federated Reinforcement Learning Using Wasserstein Barycenters

    cs.LG 2025-06 conditional novelty 4.0 of 10

    FedWB aggregates local models by computing Wasserstein barycenters of flattened, normalized weight matrices, yielding faster early convergence than FedAvg on MNIST and on heterogeneous CartPole DQN training.

  6. Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning

    cs.LG 2025-05 conditional novelty 4.0 of 10

    Federated averaging of behavior-metric-based state projection networks improves cross-environment generalization in federated reinforcement learning, while the claimed privacy protection is not demonstrated.

  7. E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing

    cs.LG 2025-02 conditional novelty 4.0 of 10

    A federated-learning gradient compressor that sends tiny synthetic features instead of gradients, plus a download-phase compressor and a budget scheduler, with conditional convergence proofs and broad experiments.

  8. Blockchain-assisted Demonstration Cloning for Multi-Agent Deep Reinforcement Learning

    cs.LG 2025-01 conditional novelty 4.0 of 10

    A multi-expert action-advice method plus a blockchain model marketplace speeds up multi-agent reinforcement learning under sparse rewards and tolerates faulty experts.

  9. Federated Multi-Agent Reinforcement Learning for Privacy-Preserving and Energy-Aware Resource Management in 6G Edge Networks

    cs.LG 2025-09 reject novelty 3.0 of 10

    FERMI-6G, a federated multi-agent DRQN framework with secure aggregation, reportedly improves latency, energy, reliability, and fairness over centralized and heuristic baselines in a simulated 6G edge network.

  10. A Survey of Multi Agent Reinforcement Learning: Federated Learning and Cooperative and Noncooperative Decentralized Regimes

    cs.MA 2025-07 reject novelty 3.0 of 10

    A review of multi-agent reinforcement learning that catalogues federated, decentralized cooperative, and noncooperative regimes from the existing literature.

  11. Modular Federated Learning: A Meta-Framework Perspective

    cs.LG 2025-05 conditional novelty 3.0 of 10

    A 63-page survey that reframes federated learning as a composition of eight modules and proposes an 'alignment operator' taxonomy, while surveying Python FL frameworks and open challenges.

  12. Machine Learning for Spectrum Sharing: A Survey

    eess.SP 2024-11 conditional novelty 2.0 of 10

    A review of machine learning applied to spectrum sharing, covering sensing, allocation, access, handoff, beamforming, and security, with summary tables of the literature.

Pith tools