Pith. sign in

REVIEW 4 major objections 6 minor 60 references

The paper claims that a language-model-guided curriculum with diversity-based context blending improves generalization and sample efficiency over value-based and random curriculum baselines in multi-agent traffic signal control.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

cMALC-D uses an LLM to generate training contexts for multi-agent RL and a diversity-blending mechanism to avoid mode collapse, claiming improved generalization on traffic signal control.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A plausible LLM-driven curriculum idea for cMARL, but the reported 'significant' gains don't survive close reading—the LLM's causal role is never isolated, and the evaluation has gaps. the 4 major comments →

arxiv 2508.20818 v1 pith:NLYI2ADK submitted 2025-08-28 cs.LG cs.MA

cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending

Anirudh Satheesh , Keenan Powell , Hua Wei This is my paper
classification cs.LG cs.MA
keywords large language modelscurriculum learningmulti-agent reinforcement learningcontextual MDPtraffic signal controlgeneralizationsample efficiencydiversity-based context blending
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

cMALC-D claims that a large language model can act as the curriculum designer for multi-agent reinforcement learning, replacing noisy value estimates with a single performance metric per training context. The framework alternates policy training with LLM context generation: the LLM reads a sliding window of recent contexts and their scores, proposes the next traffic configuration, and a similarity-triggered blending mechanism mixes in a randomly sampled past context to prevent mode collapse. On three real-world traffic signal control environments, the authors report that this approach matches or beats domain randomization, prioritized level replay, ACCEL, SPACE, and training without a curriculum on held-out test reward, delay, wait time, and throughput, while converging with fewer samples. The diversity mechanism is the part the ablation supports directly; it accelerates convergence and stabilizes test performance relative to the no-diversity variant.

Core claim

In its own terms, the paper's central claim is that a contextual MARL policy can be trained more effectively when a large language model, rather than a noisy value estimate, decides which environment context comes next. cMALC-D alternates training the MAPPO policy on the current traffic context and querying the LLM with a sliding window of recent contexts and their scalar performance metrics; the LLM proposes the next context as a JSON configuration of car-following parameters. When consecutive LLM proposals are too similar, the algorithm blends the proposal with a randomly sampled historical context to inject diversity. Across Jinan 1×3, Hangzhou, and Jinan 3×4 traffic signal datasets, the

What carries the argument

The central mechanism is an evolutionary LLM-guided curriculum loop with a diversity-based context blending gate. The LLM is prompted with a sliding window of recent contexts and their performance metrics and asked to propose the next car-following parameter set; a similarity counter compares the proposal to the recent window, and once several consecutive similar proposals occur, the next context becomes a linear blend of the LLM proposal and a uniformly sampled prior context. This replaces the value-estimate or TD-error proxies used by self-paced baselines with language-model reasoning over one scalar metric, and the blending gate is the part that prevents mode collapse.

Load-bearing premise

The load-bearing premise is that the single scalar performance metric collected on the current training context gives the LLM enough information to propose the next useful context, and that the observed gains come from those proposals rather than from smooth or diverse context sampling.

What would settle it

Run the same alternating training loop with the LLM replaced by a uniform random sampler over the context feature ranges, keeping the similarity-based blending gate and all other hyperparameters; if test reward on the held-out contexts matches cMALC-D, then the LLM's semantic reasoning is not what drives the reported gains. A second check is to record the scalar performance metric on the training context alongside policy value estimates and test correlation with actual held-out returns.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The curriculum generator no longer needs a learned value function or advantage estimate; a scalar per-context performance metric plus LLM reasoning is claimed sufficient for effective context selection.
  • Policies trained under cMALC-D generalize to held-out and randomly generated traffic contexts with lower delay, higher throughput, and higher test reward than the four named curriculum baselines.
  • The diversity-based blending mechanism is the component credited with improved sample efficiency and with avoiding the test-reward decline observed in the no-diversity variant.
  • Because the framework only assumes a context space and a performance metric, it can in principle wrap any MARL algorithm; the paper demonstrates this with MAPPO.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper never runs a random-context-generation control, so its experiments cannot separate the LLM's semantic reasoning from the smoothness and diversity that the curriculum loop provides; replacing the LLM with a random proposer while keeping the blending gate would isolate that effect.
  • The reported feature correlations are consistent with the LLM exploiting domain semantics; a testable extension is to prompt the LLM with explicitly declared feature relationships and measure whether curricula and test returns improve further.
  • If one scalar metric is enough, a non-LLM optimizer using the same sliding window might achieve similar gains; comparing against such a cheap baseline would clarify whether language-model semantics are the active ingredient.
  • The method is restricted to traffic signal control in the paper; transferring the same prompting template to other parameterized MARL domains, such as navigation or supply chain, would test whether the claimed generalization benefit is domain-specific.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes cMALC-D, a curriculum learning framework for contextual multi-agent reinforcement learning (cMARL) in which an LLM proposes new environment contexts from a sliding window of past contexts and their performance metrics, augmented by a diversity-based context blending mechanism that interpolates between LLM proposals and randomly sampled historical contexts when consecutive proposals are too similar. The method is evaluated against No Curriculum, Domain Randomization, PLR, ACCEL, and SPACE on three traffic signal control CityFlow datasets using MAPPO. The central claim is that cMALC-D 'significantly improves both generalization and sample efficiency' relative to existing curriculum baselines, based on test reward and traffic metrics such as delay, wait time, and throughput. The paper also presents an ablation of the diversity mechanism and an analysis of generated context trajectories.

Significance. If the claims were fully supported, the paper would make a useful contribution: it demonstrates a concrete integration of LLM-based curriculum generation into a multi-agent traffic control setting, ships code, and evaluates on three real-world datasets. The idea of replacing noisy value estimates with a scalar performance signal plus LLM reasoning is interesting and potentially valuable for cMARL. However, the current evidence is not sufficient to establish the central claim. The missing LLM ablation, the decision to report fine-tuned rather than zero-shot generalization, the absence of statistical significance testing, and the underspecification of key algorithmic components (m_t and Sim) mean that the empirical contribution is not yet demonstrated at the level claimed.

major comments (4)
  1. [Section 6.2 / Algorithm 1] The ablation never isolates the LLM. cMALC-D is compared only to cMALC (no diversity mechanism) and cMALC-ε (random blending), both of which still use the LLM to propose contexts. There is no control in which the LLM is replaced by a non-semantic, comparably smooth context proposer (e.g., random sampling with the same sliding-window constraints, or a simple interpolation of past contexts) while retaining the diversity-blending mechanism. Without this control, the observed gains over PLR/ACCEL/SPACE cannot be attributed to LLM-guided reasoning; they could be due to the diversity-based blending heuristic or the smoothness of the curriculum. This is load-bearing for the paper's main claim that LLM-generated curricula improve generalization and sample efficiency.
  2. [Section 6.1 / Table 1] The claim of 'significant' improvement is not backed by any statistical test. The tables report mean ± std over 5 seeds, but no paired significance tests, confidence intervals, or effect sizes are provided. Moreover, in the HZ environment cMALC-D's test reward (172.87 ± 1.03) and throughput (2440.09 ± 35.45) are effectively identical to SPACE's (172.90 ± 1.05 and 2440.18 ± 32.02), so the claim that cMALC-D consistently outperforms or matches is not even clearly established as a substantive advantage. Similar overlap appears in other rows. The paper should report appropriate statistical comparisons across seeds and discuss practical significance.
  3. [Section 5.1] The evaluation protocol conflicts with the generalization claim. The paper states that after training, 10 additional random contexts are generated and the policy is evaluated 'in both a zero-shot setting and after a brief finetuning phase of 5 episodes,' but then says 'we report performance metrics ... based on the fine-tuned policy.' The abstract and introduction claim generalization to unseen contexts, yet the reported numbers are fine-tuned results. The assertion that 'minimal differences' exist between zero-shot and fine-tuned policies is not accompanied by any zero-shot results. Fine-tuning on test contexts can mask curriculum differences and makes the headline claim difficult to interpret. Zero-shot results should be reported and used for the main generalization claim, or the claim should be revised.
  4. [Algorithm 1 / Section 4.2] Two central components of the algorithm are undefined. Algorithm 1 line 5 requires a 'performance metric m_t' to be collected, but the manuscript never defines how m_t is computed from the raw traffic metrics (reward, delay, throughput, wait time) or how it is presented to the LLM. Line 8 uses a 'similarity function Sim' without specifying it; the text only says 'monitor the similarity between successive contexts.' These are not minor omissions because the LLM's input and the diversity mechanism's triggering condition both depend on these definitions. Without them, the method is not reproducible and the analysis of 'what kinds of contexts are generated' is unmoored from a precise algorithmic specification.
minor comments (6)
  1. [Section 5.1] Typo: 'neglible' should be 'negligible.'
  2. [Section 4.2] The phrase 'task arithmetic from continual learning' is used to motivate the blending mechanism, but no citation or formal definition is provided. Please add a reference and a precise statement of the arithmetic operation.
  3. [Figures 1 and Tables 1a-c] In Tables 1b (HZ), cMALC-D and SPACE have nearly identical throughput and test reward; bold formatting plus the text 'outperforms or matches' can mislead readers. Consider marking statistical ties explicitly.
  4. [Section 6.3 / Figures 6-8] The correlation analysis is presented as evidence that the LLM 'identifies and exploits meaningful semantic patterns,' but correlations among generated context features can arise from the prompt constraints, the blending mechanism, or random drift. This interpretive claim should be softened or supported with a baseline correlation analysis (e.g., for random context generation).
  5. [Appendix A.4 / Table 6] Table 6 says 'MAX New Tokens 400'; use consistent capitalization, e.g., 'Max New Tokens.'
  6. [Abstract / Section 6.1] The phrase 'significantly improves' is used in the abstract, but the results section does not provide significance tests; consider replacing with 'improves' or 'tends to improve' until statistical evidence is provided.

Circularity Check

0 steps flagged

No significant circularity: cMALC-D's curriculum is generated from measured training performance and evaluated on held-out contexts; the LLM is an external proposer, not a fitted parameter.

full rationale

The paper's claimed derivation chain is empirical rather than definitional. At each curriculum step, the LLM receives a sliding window of past contexts and their associated performance metrics {m_t} (Algorithm 1, Section 4.2). These metrics are external measurements collected while training the MARL policy on the current context, not fitted values or quantities defined in terms of the test outcome. The LLM proposes the next context, and the diversity mechanism optionally blends this proposal with a randomly sampled past context. Test performance is measured on held-out contexts (Section 5.1: 'We reserve a held-out test set of 5 contexts'), so the generalization claim is not equivalent to the training inputs by construction. The only self-citation (Satheesh & Powell 2025) appears in a related-work list for traffic signal control and is not load-bearing. The absence of an LLM-free random-context baseline with the same blending is a legitimate attribution concern but concerns experimental control, not circularity. Missing definitions of the performance metric m_t and similarity function Sim similarly affect reproducibility and correctness, but they do not make the result reduce to its inputs. No step satisfies the hard requirement of exhibiting a specific reduction by equation or definition.

Axiom & Free-Parameter Ledger

6 free parameters · 3 axioms · 0 invented entities

The central loop has a modest number of hand-chosen hyperparameters (alpha, delta, k, w), an undefined similarity function, an unspecified performance metric, and three domain assumptions about signal reliability, blending validity, and LLM reasoning. No new physical entities are introduced.

free parameters (6)
  • blending factor alpha = 0.5
    Hand-chosen in Table 5; no sensitivity analysis.
  • similarity threshold delta = 0.1
    Hand-chosen in Table 5; controls when blending activates.
  • max similar count k = 3
    Hand-chosen in Table 5; triggers blending after k similar proposals.
  • sliding window size w = 3
    Hand-chosen in Table 5; controls how much history the LLM sees.
  • similarity function Sim
    Used in Algorithm 1 line 8 but never defined; the choice materially changes when blending triggers.
  • performance metric m_t
    Passed to the LLM as 'results given in json format' (Section B.1) but never precisely specified; any reward formulation would change the curriculum.
axioms (3)
  • domain assumption A single scalar performance metric m_t on the current training context is a reliable signal for curriculum progression.
    This is the core input to the LLM prompt; the paper criticizes value estimates as noisy and unstable (Section 4.1) but does not validate m_t.
  • domain assumption Linear interpolation c_{t+1} = alpha * c_r + (1-alpha) * c_M produces valid traffic contexts.
    Context parameters have heterogeneous ranges and headwayTime is an integer; the paper does not address how blending respects these constraints (Algorithm 1, line 16).
  • domain assumption The LLM (Qwen2.5-7B) can reason over the context space from a small history and propose beneficial next contexts.
    The entire method rests on this capability; no verification is provided beyond the overall performance numbers.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending." pith.science (2026). https://pith.science/paper/NLYI2ADK

@misc{pith2026250820818,
  author       = {Pith},
  title        = {Pith review of: cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NLYI2ADK}},
  note         = {Machine review of arXiv:2508.20818}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Many multi-agent reinforcement learning (MARL) algorithms are trained in fixed simulation environments, making them brittle when deployed in real-world scenarios with more complex and uncertain conditions. Contextual MARL (cMARL) addresses this by parameterizing environments with context variables and training a context-agnostic policy that performs well across all environment configurations. Existing cMARL methods attempt to use curriculum learning to help train and evaluate context-agnostic policies, but they often rely on unreliable proxy signals, such as value estimates or generalized advantage estimates that are noisy and unstable in multi-agent settings due to inter-agent dynamics and partial observability. To address these issues, we propose Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending (cMALC-D), a framework that uses Large Language Models (LLMs) to generate semantically meaningful curricula and provide a more robust evaluation signal. To prevent mode collapse and encourage exploration, we introduce a novel diversity-based context blending mechanism that creates new training scenarios by combining features from prior contexts. Experiments in traffic signal control domains demonstrate that cMALC-D significantly improves both generalization and sample efficiency compared to existing curriculum learning baselines. We provide code at https://github.com/DaRL-LibSignal/cMALC-D.

Figures

Figures reproduced from arXiv: 2508.20818 by Anirudh Satheesh, Hua Wei, Keenan Powell.

Figure 1
Figure 1. Figure 1: Test return over training timesteps across three datasets (columns) and five held-out test tasks (rows). Each plot shows the performance of three algorithm variants (cMALC-D, cMALC, and cMALC-ϵ) averaged over five random seeds. The left column shows results for the Jinan (1×3) dataset, the middle column for Hangzhou, and the right column for the more challenging Jinan (3×4) dataset. hances MARL policy gene… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of all 8 phases for each intersection. Phases are designed so that only two lanes can be active at any time. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Individual feature values over time for cMALC-D for JN 1 × 3 [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Individual feature values over time for cMALC-D for HZ [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Individual feature values over time for cMALC-D for JN 3 × 4 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Correlation matrix for cMALC-D generated curriculum on the JN 1 × 3 environment. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Correlation matrix for cMALC-D generated curriculum on the HZ environment [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Correlation matrix for cMALC-D generated curriculum on the JN environment. B.1. LLM Prompts [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 33 canonical work pages · 6 internal anchors

  1. [1]

    Curriculum learning

    Bengio, Y., Louradour, J., Collobert, R., and Weston, J. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pp.\ 41–48, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585161. doi:10.1145/1553374.1553380. URL https://doi.org/10.1145/1553374.1553380

    arXiv 2009

  2. [2]

    J., and Welling, M

    Brandstetter, J., Hesselink, R., van der Pol, E., Bekkers, E. J., and Welling, M. Geometric and physical quantities improve e (3) equivariant message passing. arXiv preprint arXiv:2110.02905, 2021

    Pith/arXiv arXiv 2021

  3. [3]

    Openai gym

    Brockman, G. Openai gym. arXiv preprint arXiv:1606.01540, 2016

    Pith/arXiv arXiv 2016

  4. [4]

    Robust multi-agent reinforcement learning via adversarial regularization: Theoretical foundation and stable algorithms

    Bukharin, A., Li, Y., Yu, Y., Zhang, Q., Chen, Z., Zuo, S., Zhang, C., Zhang, S., and Zhao, T. Robust multi-agent reinforcement learning via adversarial regularization: Theoretical foundation and stable algorithms. Advances in Neural Information Processing Systems, 36: 0 68121--68133, 2023

    work page 2023

  5. [5]

    ${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning

    Chen, D. and Zhang, Q. e (3) -equivariant actor-critic methods for cooperative multi-agent reinforcement learning, 2024. URL https://arxiv.org/abs/2308.11842

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    Multi-agent deep reinforcement learning for large-scale traffic signal control

    Chu, T., Wang, J., Codec \`a , L., and Li, Z. Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE transactions on intelligent transportation systems, 21 0 (3): 0 1086--1095, 2019

    work page 2019

  7. [7]

    Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning

    Da, L., Gao, M., Mei, H., and Wei, H. Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 82--90, 2024

    work page 2024

  8. [8]

    S., Gupta, T., Makoviichuk, D., Makoviychuk, V., Torr, P

    De Witt, C. S., Gupta, T., Makoviichuk, D., Makoviychuk, V., Torr, P. H., Sun, M., and Whiteson, S. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020

    Pith/arXiv arXiv 2011

  9. [9]

    Goal-gan: Multimodal trajectory prediction based on goal position estimation

    Dendorfer, P., Osep, A., and Leal-Taix \'e , L. Goal-gan: Multimodal trajectory prediction based on goal position estimation. In Proceedings of the Asian Conference on Computer Vision, 2020

    work page 2020

  10. [10]

    A survey on in-context learning, 2024

    Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Liu, T., Chang, B., Sun, X., Li, L., and Sui, Z. A survey on in-context learning, 2024. URL https://arxiv.org/abs/2301.00234

    Pith/arXiv arXiv 2024

  11. [11]

    J., Li, J., Paduraru, C., Gowal, S., and Hester, T

    Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., and Hester, T. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning, 110 0 (9): 0 2419--2468, Sep 2021. ISSN 1573-0565. doi:10.1007/s10994-021-05961-4. URL https://doi.org/10.1007/s10994-021-05961-4

    doi:10.1007/s10994-021-05961-4 2021

  12. [12]

    Self-paced context evaluation for contextual reinforcement learning

    Eimer, T., Biedenkapp, A., Hutter, F., and Lindauer, M. Self-paced context evaluation for contextual reinforcement learning. In International Conference on Machine Learning, pp.\ 2948--2958. PMLR, 2021

    work page 2021

  13. [13]

    On the convergence theory of debiased model-agnostic meta-reinforcement learning

    Fallah, A., Georgiev, K., Mokhtari, A., and Ozdaglar, A. On the convergence theory of debiased model-agnostic meta-reinforcement learning. Advances in Neural Information Processing Systems, 34: 0 3096--3107, 2021

    work page 2021

  14. [14]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp.\ 1126--1135. PMLR, 2017

    2017

  15. [15]

    Automatic goal generation for reinforcement learning agents

    Florensa, C., Held, D., Geng, X., and Abbeel, P. Automatic goal generation for reinforcement learning agents. In International conference on machine learning, pp.\ 1515--1528. PMLR, 2018

    work page 2018

  16. [16]

    Contextual markov decision processes

    Hallak, A., Di Castro, D., and Mannor, S. Contextual markov decision processes. arXiv preprint arXiv:1502.02259, 2015

    Pith/arXiv arXiv 2015

  17. [17]

    Meta-Learning in Games

    Harris, K., Anagnostides, I., Farina, G., Khodak, M., Wu, Z. S., and Sandholm, T. Meta-learning in games. arXiv preprint arXiv:2209.14110, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  18. [18]

    Robust multi-agent reinforcement learning with state uncertainty

    He, S., Han, S., Su, S., Han, S., Zou, S., and Miao, F. Robust multi-agent reinforcement learning with state uncertainty. arXiv preprint arXiv:2307.16212, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Intersectionzoo: Eco-driving for benchmarking multi-agent contextual reinforcement learning

    Jayawardana, V., Freydt, B., Qu, A., Hickert, C., Yan, Z., and Wu, C. Intersectionzoo: Eco-driving for benchmarking multi-agent contextual reinforcement learning. arXiv preprint arXiv:2410.15221, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Prioritized level replay

    Jiang, M., Grefenstette, E., and Rockt \"a schel, T. Prioritized level replay. In International Conference on Machine Learning, pp.\ 4940--4950. PMLR, 2021

    work page 2021

  21. [21]

    Multi-agent reinforcement learning for traffic signal control through universal communication method

    Jiang, Q., Qin, M., Shi, S., Sun, W., and Zheng, B. Multi-agent reinforcement learning for traffic signal control through universal communication method. arXiv preprint arXiv:2204.12190, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    R., and Pajarinen, J

    Klink, P., D'Eramo, C., Peters, J. R., and Pajarinen, J. Self-paced deep reinforcement learning. Advances in Neural Information Processing Systems, 33: 0 9216--9227, 2020

    work page 2020

  23. [23]

    Google research football: A novel reinforcement learning environment

    Kurach, K., Raichuk, A., Stańczyk, P., Zając, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bousquet, O., and Gelly, S. Google research football: A novel reinforcement learning environment. Proceedings of the AAAI Conference on Artificial Intelligence, 34 0 (04): 0 4501--4510, Apr. 2020. doi:10.1609/aaai.v34i04.5878. URL https://...

    doi:10.1609/aaai.v34i04.5878 2020

  24. [24]

    H., Gonzalez, J

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    2023

  25. [25]

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration

    Lin, J., Tang, J., Tang, H., Yang, S., Chen, W.-M., Wang, W.-C., Xiao, G., Dang, X., Gan, C., and Han, S. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6: 0 87--100, 2024

    work page 2024

  26. [26]

    Multi-agent deep reinforcement learning for multi-echelon inventory management

    Liu, X., Hu, M., Peng, Y., and Yang, Y. Multi-agent deep reinforcement learning for multi-echelon inventory management. Production and Operations Management, pp.\ 10591478241305863, 2022

    work page 2022

  27. [27]

    J., Liang, W., Wang, G., Huang, D.-A., Bastani, O., Jayaraman, D., Zhu, Y., Fan, L., and Anandkumar, A

    Ma, Y. J., Liang, W., Wang, G., Huang, D.-A., Bastani, O., Jayaraman, D., Zhu, Y., Fan, L., and Anandkumar, A. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    J., Liang, W., Wang, H.-J., Wang, S., Zhu, Y., Fan, L., Bastani, O., and Jayaraman, D

    Ma, Y. J., Liang, W., Wang, H.-J., Wang, S., Zhu, Y., Fan, L., Bastani, O., and Jayaraman, D. Dreureka: Language model guided sim-to-real transfer, 2024. URL https://arxiv.org/abs/2406.01967

    Pith/arXiv arXiv 2024

  29. [29]

    Multi-agent meta-reinforcement learning: Sharper convergence rates with task similarity

    Mao, W., Qiu, H., Wang, C., Franke, H., Kalbarczyk, Z., Iyer, R., and Basar, T. Multi-agent meta-reinforcement learning: Sharper convergence rates with task similarity. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 66556--66570. Curran Associates, Inc...

    work page 2023

  30. [30]

    Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance

    McClellan, J., Haghani, N., Winder, J., Huang, F., and Tokekar, P. Boosting sample efficiency and generalization in multi-agent reinforcement learning via equivariance. arXiv preprint arXiv:2410.02581, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARL

    McClellan, J., Brothers, G., Huang, F., and Tokekar, P. Penguin: Partially equivariant graph neural networks for sample efficient marl. arXiv preprint arXiv:2503.15615, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  32. [32]

    A., and Mowbray, M

    Mousa, M., van de Berg, D., Kotecha, N., del Rio Chanona, E. A., and Mowbray, M. An analysis of multi-agent reinforcement learning for decentralized inventory control systems. Computers & Chemical Engineering, 188: 0 108783, 2024

    work page 2024

  33. [33]

    On first-order meta-learning algorithms

    Nichol, A., Achiam, J., and Schulman, J. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018

    Pith/arXiv arXiv 2018

  34. [34]

    Papoudakis, G., Christianos, F., Schäfer, L., and Albrecht, S. V. Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS), 2021. URL http://arxiv.org/abs/2006.07869

    Pith/arXiv arXiv 2021

  35. [35]

    Evolving curricula with regret-based environment design, 2023

    Parker-Holder, J., Jiang, M., Dennis, M., Samvelyan, M., Foerster, J., Grefenstette, E., and Rocktäschel, T. Evolving curricula with regret-based environment design, 2023. URL https://arxiv.org/abs/2203.01302

    Pith/arXiv arXiv 2023

  36. [36]

    Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments

    Portelas, R., Colas, C., Hofmann, K., and Oudeyer, P.-Y. Teacher algorithms for curriculum learning of deep rl in continuously parameterized environments. In Conference on Robot Learning, pp.\ 835--853. PMLR, 2020

    work page 2020

  37. [37]

    S., Farquhar, G., Nardelli, N., Rudner, T

    Samvelyan, M., Rashid, T., de Witt, C. S., Farquhar, G., Nardelli, N., Rudner, T. G. J., Hung, C.-M., Torr, P. H. S., Foerster, J., and Whiteson, S. The StarCraft Multi - Agent Challenge . CoRR, abs/1902.04043, 2019

    Pith/arXiv arXiv 1902

  38. [38]

    A Constrained Multi-Agent Reinforcement Learning Approach to Autonomous Traffic Signal Control

    Satheesh, A. and Powell, K. A constrained multi-agent reinforcement learning approach to autonomous traffic signal control. arXiv preprint arXiv:2503.23626, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  39. [39]

    G., Hoogeboom, E., and Welling, M

    Satorras, V. G., Hoogeboom, E., and Welling, M. E(n) equivariant graph neural networks. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.\ 9323--9332. PMLR, 18--24 Jul 2021. URL https://proceedings.mlr.press/v139/satorras21a.html

    work page 2021

  40. [40]

    Learn to follow: Decentralized lifelong multi-agent pathfinding via planning and learning

    Skrynnik, A., Andreychuk, A., Nesterova, M., Yakovlev, K., and Panov, A. Learn to follow: Decentralized lifelong multi-agent pathfinding via planning and learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 17541--17549, 2024

    work page 2024

  41. [41]

    H., Wu, J., Washington, C., Sadler, B

    Song, C. H., Wu, J., Washington, C., Sadler, B. M., Chao, W.-L., and Su, Y. Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023

    work page 2023

  42. [42]

    Intrinsic motivation and automatic curricula via asymmetric self-play

    Sukhbaatar, S., Kostrikov, I., Szlam, A., and Fergus, R. Intrinsic motivation and automatic curricula via asymmetric self-play. 03 2017. doi:10.48550/arXiv.1703.05407

    Pith/arXiv arXiv doi:10.48550/arxiv.1703.05407 2017

  43. [43]

    Qwen2.5: A party of foundation models, September 2024

    Team, Q. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  44. [44]

    Domain randomization for transferring deep neural networks from simulation to the real world

    Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp.\ 23--30. IEEE, 2017

    work page 2017

  45. [45]

    Adapting deep visuomotor representations with weak pairwise constraints

    Tzeng, E., Devin, C., Hoffman, J., Finn, C., Abbeel, P., Levine, S., Saenko, K., and Darrell, T. Adapting deep visuomotor representations with weak pairwise constraints. In Algorithmic Foundations of Robotics XII: Proceedings of the Twelfth Workshop on the Algorithmic Foundations of Robotics, pp.\ 688--703. Springer, 2020

    work page 2020

  46. [46]

    Presslight: Learning max pressure control to coordinate traffic signals in arterial network

    Wei, H., Chen, C., Zheng, G., Wu, K., Gayah, V., Xu, K., and Li, Z. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, pp.\ 1290–1298, New York, NY, USA, 2019 a . Association for Computing Machinery. ISBN ...

    arXiv 2019

  47. [47]

    Colight: Learning network-level cooperation for traffic signal control

    Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., and Li, Z. Colight: Learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM '19, pp.\ 1913–1922, New York, NY, USA, 2019 b . Association for Computing Machinery....

    work page arXiv 1913

  48. [48]

    H., Peng, H., and Zhang, S

    Wen, L., Tseng, E. H., Peng, H., and Zhang, S. Dream to adapt: Meta reinforcement learning by latent context imagination and mdp imagination. IEEE Robotics and Automation Letters, 2024

    work page 2024

  49. [49]

    Xu, Y., Li, W., Vaezipoor, P., Sanner, S., and Khalil, E. B. Llms and the abstraction and reasoning corpus: Successes, failures, and the importance of object-based representations. arXiv preprint arXiv:2305.18354, 2023

    Pith/arXiv arXiv 2023

  50. [50]

    MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctions

    Yang, Q., Xie, Z., Wei, H., Zhang, D., and Yang, Y. Mallight: Influence-aware coordinated traffic signal control for traffic signal malfunctions, 2024. URL https://arxiv.org/abs/2408.09768

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  51. [51]

    Webshop: Towards scalable real-world web interaction with grounded language agents

    Yao, S., Chen, H., Yang, J., and Narasimhan, K. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35: 0 20744--20757, 2022

    work page 2022

  52. [52]

    The surprising effectiveness of ppo in cooperative multi-agent games

    Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., and Wu, Y. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems, 35: 0 24611--24624, 2022

    2022

  53. [53]

    Envgen: Generating and adapting environments via llms for training embodied agents, 2024

    Zala, A., Cho, J., Lin, H., Yoon, J., and Bansal, M. Envgen: Generating and adapting environments via llms for training embodied agents, 2024. URL https://arxiv.org/abs/2403.12014

    Pith/arXiv arXiv 2024

  54. [54]

    Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario

    Zhang, H., Feng, S., Liu, C., Ding, Y., Zhu, Y., Zhou, Z., Zhang, W., Yu, Y., Jin, H., and Li, Z. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The world wide web conference, pp.\ 3620--3624, 2019

    work page 2019

  55. [55]

    Robust multi-agent reinforcement learning with model uncertainty

    Zhang, K., SUN, T., Tao, Y., Genc, S., Mallya, S., and Basar, T. Robust multi-agent reinforcement learning with model uncertainty. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp.\ 10571--10583. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/pa...

    work page arXiv 2020

  56. [56]

    No-regret learning in time-varying zero-sum games

    Zhang, M., Zhao, P., Luo, H., and Zhou, Z.-H. No-regret learning in time-varying zero-sum games. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.\ 26772--26808. PMLR, 17--23 Jul 2022. URL...

    work page 2022

  57. [57]

    Learning meta representations for agents in multi-agent reinforcement learning

    Zhang, S., Shen, L., and Han, L. Learning meta representations for agents in multi-agent reinforcement learning. arXiv preprint arXiv:2108.12988, 2021

    work page arXiv 2021

  58. [58]

    Met-mapf: A metamorphic testing approach for multi-agent path finding algorithms

    Zhang, X.-Y., Liu, Y., Arcaini, P., Jiang, M., and Zheng, Z. Met-mapf: A metamorphic testing approach for multi-agent path finding algorithms. ACM Transactions on Software Engineering and Methodology, 33 0 (8): 0 1--37, 2024

    work page 2024

  59. [59]

    P., and Westerlund, T

    Zhao, W., Queralta, J. P., and Westerlund, T. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp.\ 737--744, 2020. doi:10.1109/SSCI47803.2020.9308468

    arXiv 2020

  60. [60]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.