Pith. sign in

REVIEW 3 major objections 5 minor 40 references

TinyMA-IEI-PPO: Exploration Incentive-Driven Multi-Agent DRL with Self-Adaptive Pruning for Vehicular Embodied AI Agent Twins Migration

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TinyMA-IEI-PPO shows that a multi-agent PPO algorithm whose structured pruning threshold is driven by individual exploration incentives can keep VEAAT migration decisions close to the Stackelberg equilibrium of the AV-RSU bandwidth game…

desk verdict Eq. (38) makes the pruning rate identically equal to the static schedule, so the paper's central self-adaptive pruning claim is not implemented; the rest is a competent but internally evaluated MARL/Stackelberg paper. read the letter →

arxiv 2505.00055 v1 pith:ZFPSV4MR submitted 2025-04-30 cs.MA cs.GT

Zhuoqi Zeng , Yuxiang Wei , Jiawen Kang This is my paper · ORCID
classification cs.MAcs.GT
keywords embodiedAIVEAATmigrationStackelberggamemulti-agentdeepreinforcementlearningself-adaptivestructuredpruningexplorationincentivevehicularnetworksresourceallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that migrating Vehicular Embodied Agent AI Twins (VEAATs) between roadside units can be solved by a tiny multi-agent PPO algorithm that prunes its own neural network during training, and that the resulting bandwidth and pricing decisions stay close to the equilibrium of a Stackelberg game between vehicles and roadside units. A sympathetic reader would care because autonomous vehicles have limited onboard compute: if pruning removes redundant neurons without sacrificing decision quality, migration choices can be made in real time on the vehicle rather than offloaded to the cloud. The paper pairs the algorithm with a utility model that folds in social influence among vehicles, complementarity and substitutability of virtual services, and an immersion-based matching probability, then argues this game has a unique Stackelberg equilibrium that the learned policies approximate.

What carries the argument

The load-bearing object is the exploration-incentive signal: the Bayesian surprise rate $r_{k,\mathrm{int}}^t = I(z_{t+1}; a_k^t \mid s_t, a_{-k}^t) = D_{\mathrm{KL}}(p(z_{t+1}|s_t,a^t)\|p(z_{t+1}|s_t,a_{-k}^t))$, estimated by a conditional variational autoencoder. This signal enters the PPO objective as an intrinsic reward and, after conversion to a bounded JS divergence $r'_{k,\mathrm{int}}$, adaptively raises or lowers the pruning rate $p_t$ that sets the neuron-importance threshold $\psi = \sum_n \sum_l S_n^{(l)} p_t$. The importance metric $S_n^{(l)}$ is a time-windowed, decayed sum of products of incoming and outgoing weight norms, and pruning is executed by a binary mask updated against $\psi$, with a group-sparse regularizer tying mask updates to actor updates. The rest of the machinery, including the MLMF Stackelberg game, the $\theta_{ij}$ matching probability built from A-DISTS image quality, and the migration delay model, feeds the rewards that the pruned policies optimize.

What would settle it

Compute the full Hessian of $U_i^F$ with respect to the bandwidth vector $b_i$, including the off-diagonal terms $\partial^2 U_i^F/\partial b_{ij}\partial b_{is}$ for $j\neq s$, under the parameter ranges used in Section VI; if any principal minor is positive or the Hessian is indefinite at a candidate best response, quasi-concavity fails and the uniqueness argument collapses. A direct numerical check would be to grid-search the best-response correspondence for a two-RSU, two-AV instance with strong social influence $\zeta_{ik}$ and substitutability $\eta_{js}<0$ and look for two distinct local maxima of $U_i^F$.

Watch

Extended reading notes

Core claim

The central claim is that individual exploration incentives can serve double duty: they improve exploration during training and, once converted to a bounded Jensen-Shannon divergence signal, they modulate the pruning threshold so that neurons are removed only when exploration pressure is low. The resulting TinyMA-IEI-PPO converges at speeds comparable to baseline algorithms, runs at pruning rates as high as 85 percent, and produces bandwidth-demand and pricing strategies whose average rewards sit closest to the theoretical Stackelberg equilibrium among all compared methods. The paper further claims that the underlying multi-leader multi-follower Stackelberg game, with vehicle utilities containing social-influence and service-interconnection terms, has a unique equilibrium established through standard-function best responses.

Load-bearing premise

The argument that the game has a unique equilibrium rests on the vehicle's utility being single-peaked (quasi-concave) across all of its bandwidth purchases at once, but the proof checks only the curvature of each purchase separately and ignores the cross-terms between purchases; if those cross-terms create multiple peaks, the unique equilibrium that the algorithm is said to approximate may not exist.

Editorial extensions

If this is right

  • Deploying TinyMA-IEI-PPO on a computation-constrained platform keeps training and inference feasible on autonomous vehicles, since the pruned actor network is physically smaller and the exploration module is removed after training.
  • At an 85 percent pruning rate the algorithm still converges, and in the reported runs it outperforms the unpruned MA-IEI-PPO baseline, so aggressive sparsity does not automatically sacrifice solution quality.
  • Across varying numbers of AVs and RSUs, the learned average rewards for both followers and leaders stay closest to the theoretical Stackelberg equilibrium compared with MAPPO, MAA2C, MAA3C, Greedy, and Random, supporting the claim that the game model and learner are consistent.
  • The hybrid reward with an annealing intrinsic weight preserves exploration early and exploitation late, which is why the algorithm converges faster than baselines despite larger early fluctuations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recipe, using an intrinsic novelty or impact signal to modulate a structured-pruning threshold, should transfer to any edge-deployed multi-agent reinforcement learning setting, not just Stackelberg migration games, because the mechanism only depends on the reward signal being bounded and reflecting exploration need.
  • Because the uniqueness proof never checks cross-partial derivatives of the AV utility, the claim of closely approximating the Stackelberg equilibrium should be read as conditional on that equilibrium being unique; if quasi-concavity fails in some parameter regime, the algorithm may still converge to a reasonable policy but the equilibrium benchmark loses its meaning.
  • A cheaper alternative to the CVAE estimator would be any bounded surprise proxy, such as prediction error in a learned forward model; if such a proxy also keeps pruning stable, the CVAE could be dropped, making the pipeline lighter still.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses the problem of migrating Vehicular Embodied Agent AI Twins (VEAATs) between roadside units in vehicular embodied AI networks. It proposes a multi-leader multi-follower Stackelberg game that couples AV and RSU utilities with social influence, service complementarity/substitutability, and a matching probability based on an image-quality immersion metric. On the learning side, it introduces TinyMA-IEI-PPO, a MAPPO variant that adds an intrinsic exploration incentive (measured as Bayesian surprise estimated by a CVAE) and a self-adaptive dynamic structured pruning scheme claimed to adjust the pruning threshold based on individual exploration incentives. The authors prove (or claim) existence and uniqueness of the Stackelberg equilibrium via standard-function arguments, and present simulations on an embedded platform showing convergence and closeness to the computed equilibrium. The core algorithmic novelty is the self-adaptive pruning mechanism described by Eq. (38).

Significance. If the pruning mechanism worked as described and the equilibrium proofs were complete, the paper would offer a practical way to deploy lightweight multi-agent DRL on resource-constrained vehicles, which is a real need in vehicular edge intelligence. The intrinsic-exploration module and the time-window neuron-importance metric are useful ideas. However, the central contribution is internally inconsistent: Eq. (38) reduces the adaptive pruning rate to the static progressive schedule, so the headline claim of exploration-driven pruning is not implemented by the algorithm as written. In addition, the game-theoretic uniqueness proofs are incomplete. These are load-bearing issues, and the numerical evaluation cannot compensate because it is largely a self-consistency check against the paper's own equilibrium model. The manuscript in its current form does not support its central claims.

major comments (3)
  1. [Section V-C, Eq. (38)] The self-adaptive pruning mechanism is a mathematical no-op. In Eq. (38), p^t = min(max(p^{t1}*(1 + φ r'_k,int), p^{t2}*(1 + φ r'^{t-1}_k,int)), p^{t1}). Since Eq. (41) states that the JS divergence r'_k,int lies in [0,1], and φ is a sensitivity hyperparameter that is presumably nonnegative, p^{t1}*(1 + φ r'_k,int) ≥ p^{t1}. Therefore the inner max is always at least p^{t1}, and the outer min with p^{t1} forces p^t = p^{t1} identically for every time step. Consequently, the pruning rate never depends on the exploration incentive, and the threshold ψ in Eq. (37) follows only the static progressive schedule given by p^{t1}. This invalidates the paper's central claim that the algorithm 'adapts to changes in individual exploration incentives during training, dynamically formulating pruning strategies.' The mechanism must be redesigned so that p^t can actually vary with r'_k,int, and the experimental results in Fig. 10 must be re-run against the corrected mechanism.
  2. [Section IV-B-1, Theorem 1 and Theorem 2] The proof of existence and uniqueness of the follower-level Nash equilibrium is incomplete. Theorem 1 asserts quasi-concavity of U_i^F based only on the diagonal second derivative ∂²U_i^F/∂b_ij² < 0 in Eq. (15). The utility in Eq. (9) contains bilinear cross-terms such as ζ_ik b_ij b_kj and η_js b_ij b_is; the Hessian has nonzero off-diagonal entries, and a negative diagonal does not imply quasi-concavity of the full function over the vector b_i. Furthermore, the constraint Σ_j θ_ij T_ij ≤ T_i^max in problem P1 involves T_ij, which depends on b_ij through a logarithm in the denominator (and on the nonconvex M/M/1 queuing term), so the feasible strategy set is not shown to be convex. Theorem 2's standard-function argument also assumes without proof that the positive social/complementarity terms dominate the negative substitutability terms, and Eq. (18) does not provide a clean proof of scalability. Because the uniqueness of the follower equilibrium is a load-bearing step for Theorem 3, the Stackelberg equilibrium uniqueness claim is not established.
  3. [Section VI, Figs. 3-10] The numerical evaluation is not sufficient to support the claimed contributions. The baselines are compared against 'the SE value' computed from the paper's own utility model, which makes the approximation-to-equilibrium claim a self-consistency check rather than a validation against an external optimum; the authors should also compare with a direct numerical solver (e.g., iterative best response) or report the gap to the true equilibrium of the system. More importantly, since Eq. (38) collapses to p^t = p^{t1}, the experiments in Fig. 10 cannot demonstrate exploration-adaptive pruning; they only compare a fixed progressive-pruning schedule against other fixed-schedule baselines. The paper would need new experiments that directly measure how the pruning rate and threshold change with the intrinsic reward, and that compare the corrected adaptive mechanism against static schedules.
minor comments (5)
  1. [Throughout (abstract/title vs. Sections V-VI)] The algorithm name is used inconsistently: the title and abstract say TinyMA-IEI-PPO, while the main text (e.g., Section V and Fig. 3 caption) refers to TinyMA-IMI-PPO. Please standardize.
  2. [Eq. (35)] There is a typo in the displayed equation: 'Ωt,(l)n t' should read 'Ωt,(l)n'.
  3. [Section VI, first paragraph] The text says 'the key parameters of the experiment are similar to [12] [23]' but does not provide a table of parameter values. Please include a table with the simulation parameters and their settings, including the hyperparameters c1, c3, φ, λ, and the pruning schedule values.
  4. [Section VI, Fig. 10 caption] The caption mentions an '85& pruning rate'; this is presumably a typo for '85% pruning rate'.
  5. [Section V-B-2, Eq. (31)] The CVAE objective in Eq. (31) is introduced without derivation or reference to the exact formulation used in prior work [26]. Please clarify how the variational lower bound is obtained and how the two KL terms are combined, since this objective is central to the intrinsic-reward definition.

Circularity Check

1 steps flagged · score 8.0 of 10

Equation (38) forces p^t = p^{t1} whenever the intrinsic reward and sensitivity are nonnegative, so the headline self-adaptive pruning is vacuous by construction.

  1. self definitional [Section V-C-2, Eqs. (37)-(41); Algorithm 3]
    "pt = min{max(pt1·(1+φr′k,int), pt2·(1+φr′t−1k,int)), pt1} ... where ψ is the pruning threshold, ϕ is a hyperparameter used to control the sensitivity of the pruning threshold to the individual exploration degree, and pt is the pruning rate after the adaptive adjustment ... The JS divergence is symmetric, and its value range is between 0 and 1."

    The adaptive pruning rate p^t is defined by an inner max followed by an outer min with p^{t1}. Since r'_k,int, the JS divergence representing the exploration incentive, lies in [0,1], for any nonnegative sensitivity φ we have p^{t1}(1+φ r'_k,int) ≥ p^{t1}, so the inner max is at least p^{t1}; the outer min then selects p^{t1} exactly. Hence p^t ≡ p^{t1} regardless of the exploration incentive, and Eq. (37)'s threshold ψ inherits only the static progressive-pruning schedule from Eq. (39). The claimed dependence of pruning on individual exploration incentives, which is the central novelty of TinyMA-IEI-PPO, is therefore not implemented by the algorithm as written. Fig. 10 consequently evaluates a fixed progressive-pruning PPO variant, not an exploration-adaptive mechanism.

full rationale

The only load-bearing degeneracy is in the pruning-rate update: Eq. (38) defines p^t with min(max(..., p^{t1}), ...) and, since r'_k,int ∈ [0,1], p^t is identically p^{t1}; hence the threshold in Eq. (37) and the mask updates in Eq. (44) never respond to exploration incentives. This is not a tuning issue, because every nonnegative φ gives the same collapse. I did not count as circular the reuse of [22], [23], and [31] for the game model and neuron-importance metric, because the relevant equations (1), (9), (10), (35), and (45) are restated in the paper and the equilibrium target is computed from the paper's own utility model; that is an external-validity concern rather than a derivation from the target. The quasi-concavity proof in Theorem 1 checks only ∂²U/∂b²_ij < 0 and leaves cross-partials of Φ_i and β_i unexamined; that is a correctness gap, not circularity. Because Eq. (38)'s collapse makes the central self-adaptive-pruning claim vacuous by construction, the circularity score is 8.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The ledger shows that the paper's central claims rest on unproven game-theoretic properties (quasi-concavity and standard-function best responses), an M/M/1 queueing assumption, and a pruning adaptivity mechanism that Eq. (38) actually prevents. All numerical inputs are inherited from prior work without being reported. No new physical or conceptual entities are introduced beyond the algorithm itself.

free parameters (6)
  • c1 (intrinsic reward weight)
    Balances intrinsic exploration reward against external reward in Eq. (32); no value reported.
  • Annealing hyperparameters c3, alpha, N0
    Control the decay of exploration incentive weight in Eq. (34); no values reported.
  • Pruning sensitivity phi
    Controls how strongly the exploration incentive modifies the pruning rate in Eq. (38); no value reported.
  • Pruning schedule (p_i, p_f, t0, N, Delta) = target sparsity 85% in Fig. 10
    Define the progressive pruning rates in Eqs. (39)-(40); only the final sparsity is mentioned.
  • Sparse regularizer weight lambda
    Weight of the neuron-importance group sparse penalty in Eq. (45); no value reported.
  • Simulation environment parameters (delta_i, alpha_i, k_j, c_j, lambda_j, mu_j)
    The numerical section says parameters are similar to [12][23] but does not list them; the curves cannot be reproduced without these values.
assumptions (6)
  • ad hoc to paper The AV utility function U_i^F is quasi-concave over the full bandwidth vector b_i.
    Theorem 1 uses Lemma 1 to claim existence of Nash equilibrium, but the proof only shows diagonal second derivatives are negative (Eq. 15); cross-partials from Phi_i and beta_i are ignored.
  • ad hoc to paper The AV best-response function satisfies the standard-function property under Theorem 2's condition.
    The stated condition 'pj < delta_i/(alpha_i Tth_i)-e' is garbled, and the scalability algebra in Eq. (18) is not rigorously justified.
  • ad hoc to paper The RSU best-response function H_j(Y) is a standard function.
    The monotonicity derivation in Eq. (24) and the scalability inequality in Eq. (25) contain index errors and unproven steps, so uniqueness of the leader-level equilibrium is not established.
  • domain assumption Queuing delay at each RSU follows an M/M/1 queue with arrival rate lambda_j and service rate mu_j.
    Standard queueing model inherited from [13],[23]; not validated in this paper.
  • ad hoc to paper The exploration incentive r'_k,int is able to increase the pruning rate p^t.
    This is the core of the claimed self-adaptive pruning, but Eq. (38) as written always returns p^t1 because the min caps the max at p^t1; the assumption is violated.
  • domain assumption The environment parameters used in Section VI are the same as in [12],[23].
    The paper provides no explicit settings, so the comparison to baselines depends on unspecified external configurations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TinyMA-IEI-PPO: Exploration Incentive-Driven Multi-Agent DRL with Self-Adaptive Pruning for Vehicular Embodied AI Agent Twins Migration." pith.science (2026). https://pith.science/paper/ZFPSV4MR

@misc{pith2026250500055,
  author       = {Pith},
  title        = {Pith review of: TinyMA-IEI-PPO: Exploration Incentive-Driven Multi-Agent DRL with Self-Adaptive Pruning for Vehicular Embodied AI Agent Twins Migration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZFPSV4MR}},
  note         = {Machine review of arXiv:2505.00055}
}
read the original abstract

Embodied Artificial Intelligence (EAI) addresses autonomous driving challenges in Vehicular Embodied AI Networks (VEANETs) through multi-modal perception, adaptive decision-making, and hardware-software co-scheduling. However, the computational demands of virtual services and the inherent mobility of autonomous vehicles (AVs) necessitate real-time migration of Vehicular Embodied Agent AI Twins (VEAATs) between resource-constrained Roadside Units (RSUs). This paper proposes a novel framework for efficient VEAAT migration in VEANETs, combining a multi-leader multi-follower (MLMF) Stackelberg game-theoretic incentive mechanism with a tiny multi-agent deep reinforcement learning (MADRL) algorithm. First, We propose an virtual immersive experience-driven utility model that captures AV-RSU dynamic interactions by integrating AVs' social influence, service complementarity and substitutability, and RSUs' resource allocation strategies to optimize VEAAT migration decisions. Second, to enhance training efficiency and enable efficient deployment on computation-constrained AVs while preserving exploration-exploitation performance, we propose TinyMA-IEI-PPO, a self-adaptive dynamic structured pruning algorithm that dynamically adjusts neuron importance based on agents' exploration incentives. Numerical results demonstrate that our approach achieves convergence comparable to baseline models and closely approximates the Stackelberg equilibrium.

Figures

Figures reproduced from arXiv: 2505.00055 by the authors.

Figure 1
Figure 1. The system model for VEAAT migration. AV i is denoted as Ji = {Di , Ci , T max i , αi}, where Di is the total amount of migrated VEAAT data, including vehicle configuration, historical interaction data and real VEAAT state, Ci is the number of CPU cycles required to re-instantiate VEAAT, T max i is the maximum tolerated delay and αi is used to describe their heterogeneity to measure the delay sensitivity of maintain… view at source ↗
Figure 2
Figure 2. TinyIEI-MAPPO algorithm’s Architecture for the VEAAT migration. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Comparison of episode reward curves of TinyMA-IMI [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The pricing strategies of RSUs and the bandwidth [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The average reward of AVs for different numbers of AVs underdifferent algo￾rithms [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 10
Figure 10. Figure 10: The Episode reward performance of each method with [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 27 canonical work pages

  1. [26]

    Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning

    X. Li, Z. Liu, S. Chen, and J. Zhang, “Individual contributions as intrinsic exploration scaffolds for multi-agent reinforcement learning,” arXiv preprint arXiv:2405.18110 , 2024

    work page Pith review arXiv 2024

  2. [1]

    Habitat: A platform for embodied ai research,

    M. Savva, A. Kadian, O. Maksymets, Y . Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V . Koltun, J. Malik et al. , “Habitat: A platform for embodied ai research,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 9339–9347

    2019

  3. [2]

    A call for embodied ai,

    G. Paolo, J. Gonzalez-Billandon, and B. K ´egl, “A call for embodied ai,” arXiv preprint arXiv:2402.03824 , 2024

    arXiv 2024

  4. [3]

    Artificial intelligence based object detection and traffic prediction by autonomous vehicles–a review,

    P. Sharma and C. Rana, “Artificial intelligence based object detection and traffic prediction by autonomous vehicles–a review,” Expert Systems with Applications, p. 124664, 2024

    2024

  5. [4]

    Aligning cyber space with physical world: A comprehensive survey on embodied ai,

    Y . Liu, W. Chen, Y . Bai, X. Liang, G. Li, W. Gao, and L. Lin, “Aligning cyber space with physical world: A comprehensive survey on embodied ai,” arXiv preprint arXiv:2407.06886 , 2024

    arXiv 2024

  6. [5]

    Generative diffusion-based contract design for efficient ai twin migration in vehicular embodied ai networks,

    Y . Zhong, J. Kang, J. Wen, D. Ye, J. Nie, D. Niyato, X. Gao, and S. Xie, “Generative diffusion-based contract design for efficient ai twin migration in vehicular embodied ai networks,” IEEE Transactions on Mobile Computing, 2025

    work page 2025

  7. [6]

    Embodied ai-enhanced vehicular networks: An integrated large language models and reinforcement learning method,

    R. Zhang, C. Zhao, H. Du, D. Niyato, J. Wang, S. Sawadsitang, X. Shen, and D. I. Kim, “Embodied ai-enhanced vehicular networks: An integrated large language models and reinforcement learning method,” arXiv preprint arXiv:2501.01141 , 2025

    arXiv 2025

  8. [7]

    Language models meet world models: Embodied experiences enhance language models,

    J. Xiang, T. Tao, Y . Gu, T. Shu, Z. Wang, Z. Yang, and Z. Hu, “Language models meet world models: Embodied experiences enhance language models,” Advances in neural information processing systems , vol. 36, pp. 75 392–75 412, 2023

    work page 2023

Show all 40 references
  1. [8]

    Intelligent cockpit for intelligent connected vehicles: Definition, taxonomy, technology and evaluation,

    W. Li, D. Cao, R. Tan, T. Shi, Z. Gao, J. Ma, G. Guo, H. Hu, J. Feng, and L. Wang, “Intelligent cockpit for intelligent connected vehicles: Definition, taxonomy, technology and evaluation,” IEEE Transactions on Intelligent Vehicles, vol. 9, no. 2, pp. 3140–3153, 2023

    2023

  2. [9]

    Scenario-function system for automotive intelligent cockpits: Framework, research progress and perspectives,

    H. Chen, R. Gao, L. Fan, E. Liu, W. Li, R. Tan, Y . Li, L. He, and D. Cao, “Scenario-function system for automotive intelligent cockpits: Framework, research progress and perspectives,” IEEE Transactions on Intelligent Vehicles, 2024

    2024

  3. [10]

    Learning-based incentive mechanism for task freshness-aware vehic- ular twin migration,

    J. Zhang, J. Nie, J. Wen, J. Kang, M. Xu, X. Luo, and D. Niyato, “Learning-based incentive mechanism for task freshness-aware vehic- ular twin migration,” in 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW) . IEEE, 2023, pp. 103–108

    2023

  4. [11]

    Multiagent deep reinforcement learning for dynamic avatar migration in aiot-enabled vehicular metaverses with trajectory prediction,

    J. Chen, J. Kang, M. Xu, Z. Xiong, D. Niyato, C. Chen, A. Jamalipour, and S. Xie, “Multiagent deep reinforcement learning for dynamic avatar migration in aiot-enabled vehicular metaverses with trajectory prediction,” IEEE Internet of Things Journal , vol. 11, no. 1, pp. 70–83, 2023

    2023

  5. [12]

    A multi- leader multi-follower game-based analysis for incentive mechanisms in socially-aware mobile crowdsensing,

    J. Nie, J. Luo, Z. Xiong, D. Niyato, P. Wang, and H. V . Poor, “A multi- leader multi-follower game-based analysis for incentive mechanisms in socially-aware mobile crowdsensing,” IEEE Transactions on Wireless Communications, vol. 20, no. 3, pp. 1457–1471, 2020

    2020

  6. [13]

    Stackelberg game-based computation offloading in social and cognitive industrial internet of things,

    F. Li, H. Yao, J. Du, C. Jiang, and Y . Qian, “Stackelberg game-based computation offloading in social and cognitive industrial internet of things,” IEEE Transactions on Industrial Informatics , vol. 16, no. 8, pp. 5444–5455, 2019

    2019

  7. [14]

    Multiagent federated reinforcement learning for secure incentive mechanism in intelligent cyber–physical systems,

    M. Xu, J. Peng, B. B. Gupta, J. Kang, Z. Xiong, Z. Li, and A. A. A. El-Latif, “Multiagent federated reinforcement learning for secure incentive mechanism in intelligent cyber–physical systems,” IEEE Internet of Things Journal , p. 22095–22108, Nov 2022. [Online]. Available: ht...

    2022

  8. [15]

    Tiny machine learning for concept drift,

    S. Disabato and M. Roveri, “Tiny machine learning for concept drift,” IEEE Transactions on Neural Networks and Learning Systems , 2022

    2022

  9. [16]

    A compre- hensive survey of few-shot learning: Evolution, applications, challenges, and opportunities,

    Y . Song, T. Wang, P. Cai, S. K. Mondal, and J. P. Sahoo, “A compre- hensive survey of few-shot learning: Evolution, applications, challenges, and opportunities,” ACM Computing Surveys, vol. 55, no. 13s, pp. 1–40, 2023

    2023

  10. [17]

    Strategically efficient exploration in competitive multi-agent reinforcement learning,

    R. Loftin, A. Saha, S. Devlin, and K. Hofmann, “Strategically efficient exploration in competitive multi-agent reinforcement learning,” in Un- certainty in Artificial Intelligence . PMLR, 2021, pp. 1587–1596

    2021

  11. [18]

    Embodied ai- empowered low altitude economy: Integrated sensing, communications, computation, and control (isc3),

    Y . Yang, Y . Chen, J. Wang, G. Sun, and D. Niyato, “Embodied ai- empowered low altitude economy: Integrated sensing, communications, computation, and control (isc3),” arXiv preprint arXiv:2412.19996 , 2024

    2024 arXiv

  12. [19]

    Embodied understanding of driving scenarios,

    Y . Zhou, L. Huang, Q. Bu, J. Zeng, T. Li, H. Qiu, H. Zhu, M. Guo, Y . Qiao, and H. Li, “Embodied understanding of driving scenarios,” arXiv preprint arXiv:2403.04593 , 2024

    2024 arXiv

  13. [20]

    Multi-attribute auction-based resource allocation for twins migration in vehicular metaverses: A gpt-based drl approach,

    Y . Tong, J. Chen, M. Xu, J. Kang, Z. Xiong, D. Niyato, C. Yuen, and Z. Han, “Multi-attribute auction-based resource allocation for twins migration in vehicular metaverses: A gpt-based drl approach,” IEEE Transactions on Cognitive Communications and Networking , vol. 11, no. 1...

    2025

  14. [21]

    Learning-based incentive mechanism for task freshness-aware vehic- ular twin migration,

    J. Zhang, J. Nie, J. Wen, J. Kang, M. Xu, X. Luo, and D. Niyato, “Learning-based incentive mechanism for task freshness-aware vehic- ular twin migration,” in 2023 IEEE 43rd International Conference on Distributed Computing Systems Workshops (ICDCSW) , 2023, pp. 103– 108

    2023

  15. [22]

    Tiny multiagent drl for twins migration in uav metaverses: A multileader multifollower stackelberg game approach,

    J. Kang, Y . Zhong, M. Xu, J. Nie, J. Wen, H. Du, D. Ye, X. Huang, D. Niyato, and S. Xie, “Tiny multiagent drl for twins migration in uav metaverses: A multileader multifollower stackelberg game approach,” IEEE Internet of Things Journal , vol. 11, no. 12, pp. 21 021–21 036, 2024

    2024

  16. [23]

    When metaverses meet vehicle road cooperation: Multiagent drl-based stackelberg game for vehicular twins migration,

    J. Kang, J. Zhang, H. Yang, D. Ye, and M. S. Hossain, “When metaverses meet vehicle road cooperation: Multiagent drl-based stackelberg game for vehicular twins migration,” IEEE Internet of Things Journal, vol. 11, no. 22, pp. 35 928–35 941, 2024

    2024

  17. [24]

    End-to-end multitarget flexible job shop scheduling with deep reinforcement learning,

    R. Wang, Y . Jing, C. Gu, S. He, and J. Chen, “End-to-end multitarget flexible job shop scheduling with deep reinforcement learning,” IEEE Internet of Things Journal , vol. 12, no. 4, pp. 4420–4434, 2025

    2025

  18. [25]

    Strangeness-driven exploration in multi-agent reinforcement learning,

    J.-B. Kim, H.-B. Choi, and Y .-H. Han, “Strangeness-driven exploration in multi-agent reinforcement learning,” Neural Networks , vol. 172, p. 106149, 2024

    2024

  19. [27]

    Episodic multi-agent reinforcement learning with curiosity- driven exploration,

    L. Zheng, J. Chen, J. Wang, J. He, Y . Hu, Y . Chen, C. Fan, Y . Gao, and C. Zhang, “Episodic multi-agent reinforcement learning with curiosity- driven exploration,” Advances in Neural Information Processing Sys- tems, vol. 34, pp. 3757–3769, 2021

    2021

  20. [28]

    Self-motivated multi-agent exploration,

    S. Zhang, J. Cao, L. Yuan, Y . Yu, and D.-C. Zhan, “Self-motivated multi-agent exploration,” arXiv preprint arXiv:2301.02083 , 2023

    2023 arXiv

  21. [29]

    Online-s2t: A lightweight distributed online reinforcement learning training framework for resource-constrained devices,

    F. Zhou, X. Qiu, Z. Cai, W. Chen, H. Zhao, and Z. Li, “Online-s2t: A lightweight distributed online reinforcement learning training framework for resource-constrained devices,” in 2023 Asia Conference on Advanced Robotics, Automation, and Control Engineering (ARACE). IEEE, 202...

    2023

  22. [30]

    Pops: Policy pruning and shrinking for deep reinforcement learning,

    D. Livne and K. Cohen, “Pops: Policy pruning and shrinking for deep reinforcement learning,” IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 4, pp. 789–801, 2020

    2020

  23. [31]

    Compressing deep reinforcement learning networks with a dynamic structured prun- ing method for autonomous driving,

    W. Su, Z. Li, M. Xu, J. Kang, D. Niyato, and S. Xie, “Compressing deep reinforcement learning networks with a dynamic structured prun- ing method for autonomous driving,” IEEE Transactions on Vehicular Technology, 2024

    2024

  24. [32]

    Embodied artificial intelligence,

    R. Chrisley, “Embodied artificial intelligence,” Artificial Intelligence , vol. 149, no. 1, pp. 131–150, 2003. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0004370203000559

    2003

  25. [33]

    Hybrid reconfigurable intelligent meta- surfaces: Enabling simultaneous tunable reflections and sensing for 6g wireless communications,

    G. C. Alexandropoulos, N. Shlezinger, I. Alamzadeh, M. F. Imani, H. Zhang, and Y . C. Eldar, “Hybrid reconfigurable intelligent meta- surfaces: Enabling simultaneous tunable reflections and sensing for 6g wireless communications,” IEEE Vehicular Technology Magazine , vol. 19, ...

    2024

  26. [34]

    Pre-equalization aided grant-free massive access in massive mimo system,

    Y . Wang, Y . Mei, Z. Gao, Z. Wan, B. Ning, D. Mi, and S. Muhaidat, “Pre-equalization aided grant-free massive access in massive mimo system,” arXiv preprint arXiv:2502.06239 , 2025

    2025 arXiv

  27. [35]

    Joint user association and resource pricing for metaverse: Distributed and centralized approaches,

    X. Huang, W. Zhong, J. Nie, Q. Hu, Z. Xiong, J. Kang, and T. Quek, “Joint user association and resource pricing for metaverse: Distributed and centralized approaches,” Aug 2022

    2022

  28. [36]

    Locally adaptive struc- ture and texture similarity for image quality assessment,

    K. Ding, Y . Liu, X. Zou, S. Wang, and K. Ma, “Locally adaptive struc- ture and texture similarity for image quality assessment,” in Proceedings of the 29th ACM International Conference on multimedia , 2021, pp. 2483–2491

    2021

  29. [37]

    Attention-based qoe-aware digital twin empowered edge computing for immersive virtual reality,

    J. Yu, A. Alhilal, T. Zhou, P. Hui, and D. H. Tsang, “Attention-based qoe-aware digital twin empowered edge computing for immersive virtual reality,” IEEE Transactions on Wireless Communications , 2024

    2024

  30. [38]

    Attention-aware resource allocation and qoe analysis for metaverse xurllc services,

    H. Du, J. Liu, D. Niyato, J. Kang, Z. Xiong, J. Zhang, and D. Kim, “Attention-aware resource allocation and qoe analysis for metaverse xurllc services,” Aug 2022

    2022

  31. [39]

    R. B. MYERSON, Game Theory: Analysis of Conflict . Harvard University Press, 1991. [Online]. Available: http://www.jstor.org/stable/ j.ctvjsf522

    1991

  32. [40]

    Privacy- preserving incentive mechanism for multi-leader multi-follower iot-edge computing market: A reinforcement learning approach,

    H. Xu, X. Qiu, W. Zhang, K. Liu, S. Liu, and W. Chen, “Privacy- preserving incentive mechanism for multi-leader multi-follower iot-edge computing market: A reinforcement learning approach,” Journal of Systems Architecture , p. 101932, Mar 2021. [Online]. Available: http://dx.d...

    2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.