Pith. sign in

REVIEW 3 major objections 4 minor 60 references

Multi-agent Path Finding for Timed Tasks using Evolutionary Games

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A replicator-dynamics policy update, trained on weighted-automaton rewards, outperforms deep RL by nearly 30 percent in path length on large multi-agent grid worlds.

desk verdict Interesting combination of weighted automata and evolutionary policy updates, but the published Algorithm 1 is internally inconsistent with the paper's own replicator equation, so the headline empirical claims cannot be trusted as written. read the letter →

arxiv 2411.10558 v1 pith:3RAORDLC submitted 2024-11-15 cs.MA cs.GTcs.NE

Sheryl Paul , Anand Balakrishnan , Xin Qin , Jyotirmoy V. Deshmukh This is my paper · ORCID
classification cs.MAcs.GTcs.NE MSC 91A2268T4268Q45
keywords multi-agentpathfindingweightedautomataevolutionarygametheoryreplicatordynamicsreinforcementlearningtemporallogicrewardshapingplanning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to solve multi-agent path finding when agents do not know their goal locations in advance and must satisfy timed safety and performance objectives. It argues that weighted automata can specify trajectory-level preferences such as expeditiousness that ordinary deadline-based temporal logic cannot capture, and that evolutionary game theory offers a sample-efficient alternative to deep reinforcement learning for training homogeneous teams. The authors claim that their algorithm, MAPF-EGT, reduces path length by nearly 30% versus state-of-the-art RL on large grids, is at least an order of magnitude faster to train, and scales better with agent count. A sympathetic reader would take the central contribution to be the demonstration that a simple tabular replicator-dynamics update, fed with shared batch trajectories and automaton-derived rewards, can outperform deep RL on this class of tasks.

What carries the argument

The engine of the method is the replicator update $\pi_{k+1}(s,a) = \pi_k(s,a) f(s,a) / \sum_{a'} \pi_k(s,a') f(s,a')$, where $f(s,a)$ is the batch-estimated expected return for taking action $a$ in state $s$ and $f(s)$ is the state's expected return. Actions whose estimated fitness exceeds the average are amplified, actions below average are suppressed, and the updated distribution is blended with an annealed uniform exploration policy. The reward signal comes from a four-state weighted automaton that gives $+b$ for a first reach of the goal, $-a$ for each step before reaching it, and $-c$ for a collision, with $b \ge c > aT$, so that expeditious goal-reaching dominates the optimization.

What would settle it

Run MAPF-EGT on a 100x100 grid with 50 agents, record the batch return $\eta_k$ after each iteration, and check whether consecutive batches ever satisfy $\eta_{k+1} < \eta_k$; the paper's termination guarantee rests on monotonic improvement, so a single observed decrease while the algorithm is still running would falsify that premise. Also, re-running the paper's comparison with a well-tuned PPO over the same wall-clock budget would test whether the ~30% path-length advantage is an artifact of PPO's hyperparameter defaults.

Watch

Extended reading notes

Core claim

The paper claims that trajectory-level temporally extended tasks for homogeneous multi-agent teams can be specified by deterministic weighted automata with discounted-sum valuations, and that maximizing the resulting weighted return via a replicator-dynamics policy update yields a shared stochastic policy that is both safer and more expeditious than policies learned by PPO, Q-learning, Monte Carlo search, or A* in large grid worlds. Concretely, the authors report that MAPF-EGT shortens path length by nearly 30% relative to state-of-the-art RL methods on large grids, trains at least an order of magnitude faster, and scales better as the number of agents grows. They prove that any trajectory with positive weighted-automaton reward satisfies the reach-avoid specification, and that among collision-free trajectories, earlier arrival yields higher reward.

Load-bearing premise

The training loop assumes that the average return computed from a batch of sampled trajectories is an unbiased and accurate estimate of each state's and action's true expected return, so that promoting above-average actions strictly increases the value of every visited state.

Editorial extensions

If this is right

  • If the reported gains hold, homogeneous multi-agent teams can be trained on tabular policies without neural networks, dramatically lowering compute and hardware requirements for warehouse and delivery fleets.
  • Weighted-automaton rewards provide a language for specifying performance objectives such as 'reach fast, then stay safe' that goes beyond hard deadlines, making reward design more expressive for timed tasks.
  • Because the policy is shared and updated from pooled trajectories, increasing the number of agents enriches the batch data, so larger fleets should train faster relative to RL baselines.
  • The convergence guarantee is only to a positive-return, specification-satisfying policy, not to optimality; practitioners must monitor the per-iteration return to detect stalls or oscillation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The replicator update is a form of natural policy gradient where the step size is implicit in the fitness ratio; formalizing this connection could yield convergence-rate bounds and variance-reduction techniques drawn from policy-gradient theory.
  • The monotonicity assumption could be tested directly: replacing batch returns with bootstrapped or importance-sampled estimates may fix the noisy-fitness failure mode and is a natural empirical next step.
  • A direct head-to-head on a standard benchmark suite with the same reward and wall-clock budget would tell whether the ~30% path-length advantage persists outside the paper's grid-world configuration.
  • The framework's restriction to deterministic automata is not essential; resolving nondeterministic runs with a max over valuations, as the paper notes, would let it handle partial observability or multiple accepting behaviors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes MAPF-EGT, a training method for homogeneous multi-agent path finding where goal locations are unknown a priori. The method uses weighted automata to define trajectory-level rewards that reward expeditious, collision-free goal reaching, and updates a shared stochastic policy using an evolutionary game theory replicator-style rule (Section 4.1, Eq. (4)). The authors claim that, compared with deep RL baselines (PPO), tabular Q-learning, Monte Carlo, and A*, their algorithm reduces path length by nearly 30% in large grids, is at least an order of magnitude faster, and scales better with the number of agents (Abstract, Section 5).

Significance. If the algorithm and experiments are correct, the paper makes two useful contributions: weighted automata rewards that go beyond deadline satisfaction to expeditiousness, and an evolutionary, tabular, shared-policy training method that avoids neural networks and large sample counts. The paper explicitly ships heuristic convergence reasoning and acknowledges sub-optimality, which is honest. However, the central algorithmic description in Algorithm 1 is mathematically inconsistent with the stated replicator update, and the empirical section does not provide enough numerical detail to verify the headline claims; the significance is therefore conditional on correcting these issues.

major comments (3)
  1. [Section 4.1, Algorithm 1, line 13] The update rule as written is not a valid stochastic policy update. It computes πnew_{k+1}(s,a) ← πnew_k(s,a) · α · (πnew_k(s,a) f(s,a) / Σ_{a′} πnew_k(s,a′) f(s,a′)). Summing this expression over a gives α · Σ_a π_k(a)^2 f(a) / Σ_b π_k(b) f(b), which is not 1 in general, even for α = 1. The subsequent mixing with the uniform distribution in line 14 cannot restore normalization, because a convex combination of a distribution and a sub-distribution is still a sub-distribution. Thus the output of Algorithm 1 is not a probability distribution over actions, and the described training procedure cannot be executed as published. The authors must correct line 13 to match Eq. (4)—that is, remove the extra π_k(s,a) factor and either set α = 1 or provide a normalization step that makes the update a valid distribution.
  2. [Section 4.1, 'Termination and convergence'] The key convergence argument is asserted, not proved. The text claims 'The evolutionary update guarantees that ... the value of the state s increases,' but provides no proof that the batch-sampled fitness estimates f(s,a) and f(s) satisfy the conditions needed for replicator-style monotonicity. Since these fitnesses are empirical batch returns, sampling noise or estimation bias can easily break the monotonicity property. The manuscript itself concedes that optimality would require a contraction mapping and defers that proof. Given that the termination condition (η_k − η_{k−1} < δ) and the complexity bound O((η*−η0)/δ · T · B) both depend on this asserted monotone improvement, the authors should either supply a rigorous proof under explicit assumptions or clearly state that termination and convergence are heuristic.
  3. [Section 5 and Figure 5] The central empirical claims—'nearly 30%' path-length reduction, 'order-of-magnitude' speedup, and better scaling with agent count—are not supported by reproducible numerical evidence. The paper reports only qualitative plots and does not provide means, standard deviations, number of seeds, or a table of the exact values behind Figure 5. It also does not report the hyperparameters used for MAPF-EGT (α, ν, ε, δ, batch size B, horizon T, discount γ) or the reward weights a, b, c. Without these details, the reader cannot verify the headline numbers or reproduce the experiments. Please include full experimental settings and quantitative results, preferably with error bars.
minor comments (4)
  1. [Figure 5 caption] There is a typo in the caption: 'Time steps to reach the goaal' should read 'goal'; also, 'Fig. d' should be 'Fig. (d)' for consistency with the other subfigures.
  2. [Section 2, Definition 1] The transition sampling notation contains a typo: 'si t ∼ ∆(s′|s, a)' should read 'si_{t+1} ∼ ∆(s′|s, a)', since the next state is sampled, not the current one.
  3. [Section 3, Eq. (2)] The constraint 'b ≥ c > a·T' appears, but the paper does not explicitly state that these weights must also ensure that a successful, collision-free trajectory has a positive total reward and that any trajectory with a collision or a missed deadline has non-positive reward; making this explicit would sharpen Proposition 1.
  4. [Section 1, related work on AvSTL] The text cites reference [2] as 'AvSTL', but the bibliography entry is about 'Time Robustness in MTL and Expressivity in Hybrid System Falsification'; please clarify the connection or correct the citation.

Circularity Check

1 steps flagged · score 1.0 of 10

One definitional reward lemma aside, no significant circularity: the central 30% path-length and order-of-magnitude speedup claims are external empirical comparisons against shared-reward baselines.

  1. self definitional [Section 3, Lemma 1; reward Eqs. (1)-(2) and Figure 3]
    "Lemma 1. Trajectories that are more expeditious i.e. that reach the goal sooner (assuming they do not have collisions) have higher rewards returned by the weighted automata. toa(τ ) < toa(τ ′) ⇒ wA(τ ) > wA(τ ′). Proof. The proof follows from the definitions of the reward function and wA(τ)."

    The reward automaton is hand-designed to encode expeditiousness: Eq. (2) assigns −a to every pre-goal step and +b only at the first goal visit, 'to hasten the process'. Under DiscountedSum, an earlier time of arrival removes negative −a terms while preserving +b, so wA(τ)>wA(τ′) follows from the chosen weights by construction. Lemma 1 thus restates the reward-design objective as a formal consequence; it does not show an independent property of weighted automata. This definitional lemma is not used as the basis of the empirical MAPF-EGT comparison, so it is minor rather than central circularity.

full rationale

The paper's central empirical claims—nearly 30% shorter paths on large grids and at least an order-of-magnitude lower computation time—are external comparisons against A*, Monte-Carlo search, Q-learning, and PPO under the same reward function, so no fitted parameter is renamed as a prediction and no benchmark result is forced by construction. The weighted-automaton reward is an explicit design choice; Proposition 1 and Lemma 1 are sanity properties of that hand-chosen reward, with Lemma 1 being definitional as noted. Self-citations (e.g., Refs. 7, 8, 25, 46, 59) appear in related-work and background contexts and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors. The admitted lack of a contraction-mapping optimality guarantee (Section 4) and the Algorithm 1 line 13 update, which as written does not preserve the normalization of Eq. (4), are proof-rigor and reproducibility concerns, but they are correctness issues rather than circular reasoning. Overall, the derivation chain is self-contained for its main empirical contribution, with only minor definitional circularity in the expeditiousness lemma.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The algorithmic claims rest on hand-designed reward constants, unstated hyperparameters, and the assertion that sampled-batch replicator updates improve the shared policy. No new physical or ontological entities are introduced.

free parameters (2)
  • reward weights a, b, c = not reported
    The shaped reward in Eq. 1-2 depends on per-step penalty a, goal reward b, and obstacle penalty c, constrained by b >= c > a*T, but no numeric values are given in the paper.
  • hyperparameters alpha, delta, nu, epsilon, batch size B, horizon T, discount gamma = not reported
    Algorithm 1 lists these inputs, but the experiments never state their values; the empirical claims depend on them.
assumptions (4)
  • ad hoc to paper The replicator update with sampled fitness estimates monotonically increases the value of visited states (Section 4, 'Termination and convergence').
    This is the load-bearing convergence premise; it is asserted rather than proven and depends on unbiased fitness estimates from finite batches.
  • domain assumption A single shared policy over individual agent states is sufficient for homogeneous teams with identical tasks (Sections 2 and 4).
    This reduces the multi-agent problem to single-agent learning, but assumes away the need for coordination or for modeling other agents' positions.
  • standard math Weighted automata with the DiscountedSum valuation capture the desired trajectory-level objective (Section 3).
    Weighted automata semantics are established; the specific automaton in Fig. 3 is a direct encoding of the hand-designed reward.
  • domain assumption The reward constraint b >= c > a*T creates the intended hierarchy: goal reachability first, then safety, then speed (Section 3).
    This ordering of priorities is a modeling choice; different task priorities would need different constants.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-agent Path Finding for Timed Tasks using Evolutionary Games." pith.science (2026). https://pith.science/paper/3RAORDLC

@misc{pith2026241110558,
  author       = {Pith},
  title        = {Pith review of: Multi-agent Path Finding for Timed Tasks using Evolutionary Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3RAORDLC}},
  note         = {Machine review of arXiv:2411.10558}
}
read the original abstract

Autonomous multi-agent systems such as hospital robots and package delivery drones often operate in highly uncertain environments and are expected to achieve complex temporal task objectives while ensuring safety. While learning-based methods such as reinforcement learning are popular methods to train single and multi-agent autonomous systems under user-specified and state-based reward functions, applying these methods to satisfy trajectory-level task objectives is a challenging problem. Our first contribution is the use of weighted automata to specify trajectory-level objectives, such that, maximal paths induced in the weighted automaton correspond to desired trajectory-level behaviors. We show how weighted automata-based specifications go beyond timeliness properties focused on deadlines to performance properties such as expeditiousness. Our second contribution is the use of evolutionary game theory (EGT) principles to train homogeneous multi-agent teams targeting homogeneous task objectives. We show how shared experiences of agents and EGT-based policy updates allow us to outperform state-of-the-art reinforcement learning (RL) methods in minimizing path length by nearly 30\% in large spaces. We also show that our algorithm is computationally faster than deep RL methods by at least an order of magnitude. Additionally our results indicate that it scales better with an increase in the number of agents as compared to other methods.

Figures

Figures reproduced from arXiv: 2411.10558 by the authors.

Figure 1
Figure 1. Shortest paths in the grid en￾vironment: agents a1 and a2 must get to designated final states (black circles) while avoiding red obstacle regions. Red dashed trajectories τ 1 1 of agent a1 and τ 2 1 of a2 satisfy the task but are not the shortest paths, green solid trajectories τ 1 2 and τ 2 2 are the shortest paths. Arrows point to￾wards goals. Traditional AI approaches such as the multi-agent pathfinding (MAPF) pr… view at source ↗
Figure 2
Figure 2. Counter-example: An agent (O) tries to go from the initial location (yel￾low) to the goal location (green). Under general expeditious semantics (such as in AvSTL) the trajectory in the left figure where the agent reaches the goal quickly but wanders after would receive a lower reward (area under the curve) than the one in the right figure where the agent takes a longer path to reach the goal but stays in the goal fo… view at source ↗
Figure 3
Figure 3. A deterministic weighted automaton defining the reach-avoid task that needs to be completed by each agent in the system. In the figure, s i refers to the current state of agent i in the multi-agent system. collisions (−c) at any point which could be smaller than or equal to the reward for reaching the goal, depending on whether or not the system wants to permit [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of the evolutionary based learning approach used in our algo￾rithms. rewards by combining efficiency with safety. The learning rate α ensures that the update remains bounded and allows for fine-tuning of the learning process. Proposed algorithm: Our algorithm …
Figure 5
Figure 5. Figure 5: MAPF-EGT benchmarked against the algorithms: A∗ , Monte-Carlo search, PPO, and Q learning. Timesteps to reach the goaal (Fig. a), Expected minimum distance from obstacles (greater distance indicates safer paths)(Fig. b), and clock time (seconds) required for computatio…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 42 canonical work pages

  1. [1]

    In: Proceedings of the AAAI Conference on Artificial Intel- ligence

    Agmon, N., Urieli, D., Stone, P.: Multiagent patrol generalized to complex envi- ronmental conditions. In: Proceedings of the AAAI Conference on Artificial Intel- ligence. vol. 25, pp. 1090–1095 (2011)

    work page 2011

  2. [2]

    Akazaki, T., Hasuo, I.: Time Robustness in MTL and Expressivity in Hybrid Sys- tem Falsification (Extended Version) (2015)

    work page 2015

  3. [3]

    In: 2016 IEEE 55th Conference on Decision and Control (CDC)

    Aksaray, D., Jones, A., Kong, Z., Schwager, M., Belta, C.: Q-Learning for Ro- bust Satisfaction of Signal Temporal Logic Specifications. In: 2016 IEEE 55th Conference on Decision and Control (CDC). pp. 6565–6570 (Dec 2016).https: //doi.org/10.1109/cdc.2016.7799279

    arXiv 2016

  4. [4]

    In: Zalta, E.N

    Alexander, J.M.: Evolutionary Game Theory. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Sum- mer 2021 edn. (2021)

    work page 2021

  5. [5]

    Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Con- crete Problems in AI Safety (2016)

    work page 2016

  6. [6]

    Badings, T., Romao, L., Abate, A., Jansen, N.: A Stability-Based Abstraction Framework for Reach-Avoid Control of Stochastic Dynamical Systems with Un- known Noise Distributions (2024)

    work page 2024

  7. [7]

    In: 2019 IEEE/RSJ International Conference on Intel- ligent Robots and Systems (IROS)

    Balakrishnan, A., Deshmukh, J.V.: Structured Reward Shaping Using Signal Tem- poral Logic Specifications. In: 2019 IEEE/RSJ International Conference on Intel- ligent Robots and Systems (IROS). pp. 3481–3486 (Nov 2019).https://doi.org/ 10.1109/IROS40897.2019.8968254

    arXiv 2019

  8. [8]

    In: 2023 62nd IEEE Conference on Decision and Control (CDC)

    Balakrishnan, A., Jakšić, S., Aguilar, E.A., Ničković, D., Deshmukh, J.V.: Model- Free Reinforcement Learning for Spatiotemporal Tasks Using Symbolic Automata. In: 2023 62nd IEEE Conference on Decision and Control (CDC). pp. 6834–6840 (Dec 2023). https://doi.org/10.1109/CDC49753.2023.10383559

    arXiv 2023

Show all 60 references
  1. [9]

    Bellusci, M., Basilico, N., Amigoni, F.: Multi-agent path finding in configurable environments.In:Proceedingsofthe19thInternationalConferenceonAutonomous Agents and MultiAgent Systems. pp. 159–167 (2020)

    2020

  2. [10]

    Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. I. Athena Sci- entific, Belmont, MA, USA, 3rd edn. (2005)

    2005

  3. [11]

    (eds.) Reachability Problems, vol

    Boker,U.:Quantitativevs.WeightedAutomata.In:Bell,P.C.,Totzke,P.,Potapov, I. (eds.) Reachability Problems, vol. 13035, pp. 3–18. Springer International Pub- lishing, Cham (2021).https://doi.org/10.1007/978-3-030-89716-1_1

    2021 doi

  4. [12]

    arXiv preprint arXiv:1606.01540 (2016)

    Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)

    2016 arXiv

  5. [13]

    In: Goal- sRL, a workshop collocated with ICML/IJCAI/AAMAS (2018)

    Camacho, A., Chen, O., Sanner, S., McIlraith, S.A.: Non-Markovian rewards ex- pressed in LTL: Guiding search via reward shaping (extended version). In: Goal- sRL, a workshop collocated with ICML/IJCAI/AAMAS (2018)

    2018

  6. [14]

    In: Kamin- ski, M., Martini, S

    Chatterjee, K., Doyen, L., Henzinger, T.A.: Quantitative Languages. In: Kamin- ski, M., Martini, S. (eds.) Computer Science Logic, vol. 5213, pp. 385–400. Springer Berlin Heidelberg, Berlin, Heidelberg (2008).https://doi.org/10.1007/ 978-3-540-87531-4_28

    2008

  7. [15]

    In: Rival, X

    Chatterjee, K., Henzinger, T.A., Otop, J.: Quantitative Monitor Automata. In: Rival, X. (ed.) Static Analysis, vol. 9837, pp. 23–38. Springer Berlin Heidelberg, Berlin, Heidelberg (2016).https://doi.org/10.1007/978-3-662-53413-7_2

    2016 doi

  8. [16]

    In: Adaptive and Learning-Based Control of Safety-Critical Systems, pp

    Cohen, M., Belta, C.: Temporal Logic Guided Safe Model-Based Reinforcement Learning. In: Adaptive and Learning-Based Control of Safety-Critical Systems, pp. 165–192. Springer (2023) 18 Paul et al

    2023

  9. [17]

    In: Proceedings of the 24th In- ternational Conference on Hybrid Systems: Computation and Control

    Cohen, M.H., Belta, C.: Model-based reinforcement learning for approximate op- timal control with temporal logic specifications. In: Proceedings of the 24th In- ternational Conference on Hybrid Systems: Computation and Control. pp. 1–11 (2021)

    2021

  10. [18]

    In: Droste, M., Kuich, W., Vogler, H

    Droste, M., Gastin, P.: Weighted Automata and Weighted Logics. In: Droste, M., Kuich, W., Vogler, H. (eds.) Handbook of Weighted Automata, pp. 175–211. Springer Berlin Heidelberg, Berlin, Heidelberg (2009).https://doi.org/10.1007/ 978-3-642-01492-5_5

    2009

  11. [19]

    Annals of Mathematics and Artificial Intelligence57, 293–320 (2009)

    Elmaliach,Y.,Agmon,N.,Kaminka,G.A.:Multi-robotareapatrolunderfrequency constraints. Annals of Mathematics and Artificial Intelligence57, 293–320 (2009)

    2009

  12. [20]

    IEEE Control Systems Letters5(1), 241–246 (2020)

    Gilpin, Y., Kurtz, V., Lin, H.: A smooth robustness measure of signal temporal logic for symbolic control. IEEE Control Systems Letters5(1), 241–246 (2020)

    2020

  13. [21]

    IEEE Transactions on Automatic Control63(12), 4051–4066 (2018)

    Guo, M., Zavlanos, M.M.: Probabilistic motion planning under temporal tasks and soft constraints. IEEE Transactions on Automatic Control63(12), 4051–4066 (2018)

    2018

  14. [22]

    IFAC-PapersOnLine51(16), 73–78 (2018)

    Haesaert, S., Soudjani, S., Abate, A.: Temporal logic control of general markov decision processes by approximate policy refinement. IFAC-PapersOnLine51(16), 73–78 (2018)

    2018

  15. [23]

    IEEE Transactions on Systems Science and Cybernetics 4(2), 100–107 (1968).https://doi.org/10.1109/TSSC.1968.300136

    Hart, P.E., Nilsson, N.J., Raphael, B.: A Formal Basis for the Heuristic Deter- mination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics 4(2), 100–107 (1968).https://doi.org/10.1109/TSSC.1968.300136

    1968

  16. [24]

    In: 2019 IEEE 58th conference on decision and control (CDC)

    Hasanbeig, M., Kantaros, Y., Abate, A., Kroening, D., Pappas, G.J., Lee, I.: Re- inforcement learning for temporal logic control synthesis with probabilistic satis- faction guarantees. In: 2019 IEEE 58th conference on decision and control (CDC). pp. 5338–5343. IEEE (2019)

    2019

  17. [25]

    Hashemi, N., Hoxha, B., Prokhorov, D., Fainekos, G., Deshmukh, J.: Scaling Learn- ing based Policy Optimization for Temporal Tasks via Dropout (2024)

    2024

  18. [26]

    In: Proceedings of the 35th International Conference on Machine Learning

    Icarte, R.T., Klassen, T., Valenzano, R., McIlraith, S.: Using Reward Machines for High-Level Task Specification and Decomposition in Reinforcement Learning. In: Proceedings of the 35th International Conference on Machine Learning. pp. 2107–2116. PMLR (Jul 2018)

    2018

  19. [27]

    Journal of Ar- tificial Intelligence Research73, 173–208 (Jan 2022).https://doi.org/10.1613/ jair.1.12440

    Icarte, R.T., Klassen, T.Q., Valenzano, R., McIlraith, S.A.: Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning. Journal of Ar- tificial Intelligence Research73, 173–208 (Jan 2022).https://doi.org/10.1613/ jair.1.12440

    2022

  20. [28]

    Formal Methods in System Design53(1), 83–112 (Aug 2018)

    Jakšić, S., Bartocci, E., Grosu, R., Nguyen, T., Ničković, D.: Quantitative moni- toring of STL with edit distance. Formal Methods in System Design53(1), 83–112 (Aug 2018). https://doi.org/10.1007/s10703-018-0319-x

    2018 doi

  21. [29]

    Jaksic, S., Bartocci, E., Grosu, R., Nickovic, D.: An Algebraic Framework for Run- time Verification (2018)

    2018

  22. [30]

    Advances in Neural Information Processing Systems 34 (2021)

    Jothimurugan, K., Bansal, S., Bastani, O., Alur, R.: Compositional Reinforcement Learning from Logical Specifications. Advances in Neural Information Processing Systems 34 (2021)

    2021

  23. [31]

    arXiv preprint arXiv:2011.00632 (2020)

    Kalagarla, K.C., Jain, R., Nuzzo, P.: Synthesis of discounted-reward optimal poli- cies for Markov decision processes under linear temporal logic specifications. arXiv preprint arXiv:2011.00632 (2020)

    2020 arXiv

  24. [32]

    In: AAAI 2022 Spring Sympo- sium Series (2022)

    Kempa, B., Cramer, N.B., Frank, J.D.: Swarm Mentality: Toward Automatic Swarm State Awareness with Runtime Verification. In: AAAI 2022 Spring Sympo- sium Series (2022)

    2022

  25. [33]

    IEEE Transactions on Robotics21(3), 354–363 (2005) Multi-agent Path Finding for Timed Tasks using Evolutionary Games 19

    Koenig, S., Likhachev, M.: Fast replanning for navigation in unknown terrain. IEEE Transactions on Robotics21(3), 354–363 (2005) Multi-agent Path Finding for Timed Tasks using Evolutionary Games 19

    2005

  26. [34]

    In: 2018 Annual American Control Conference (ACC)

    Li, X., Ma, Y., Belta, C.: A policy search method for temporal logic specified re- inforcement learning tasks. In: 2018 Annual American Control Conference (ACC). pp. 240–245. IEEE (2018)

    2018

  27. [35]

    In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

    Li, X., Vasile, C.I., Belta, C.: Reinforcement learning with temporal logic rewards. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 3834–3839. IEEE (2017)

    2017

  28. [36]

    In: Pettersson, P., Yi, W

    Maler, O., Nickovic, D., Pnueli, A.: Real Time Temporal Logic: Past, Present, Future. In: Pettersson, P., Yi, W. (eds.) Formal Modeling and Analysis of Timed Systems.pp.2–16.LectureNotesinComputerScience,Springer,Berlin,Heidelberg (2005). https://doi.org/10.1007/11603009_2

    2005 doi

  29. [37]

    Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous Methods for Deep Reinforcement Learning (2016)

    2016

  30. [38]

    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing Atari with Deep Reinforcement Learning (2013)

    2013

  31. [39]

    In: Droste, M., Kuich, W., Vogler, H

    Mohri, M.: Weighted Automata Algorithms. In: Droste, M., Kuich, W., Vogler, H. (eds.) Handbook of Weighted Automata, pp. 213–254. Springer Berlin Heidelberg, Berlin, Heidelberg (2009).https://doi.org/10.1007/978-3-642-01492-5_6

    2009 doi

  32. [40]

    In: Work- shops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

    Morris, R., Pasareanu, C.S., Luckow, K., Malik, W., Ma, H., Kumar, T.S., Koenig, S.: Planning, scheduling and monitoring for airport surface operations. In: Work- shops at the Thirtieth AAAI Conference on Artificial Intelligence (2016)

    2016

  33. [41]

    Chaos: An Inter- disciplinary Journal of Nonlinear Science31(2) (Feb 2021).https://doi.org/10

    Mukhopadhyay, A., Chakraborty, S.: Replicator equations induced by microscopic processes in nonoverlapping population playing bimatrix games. Chaos: An Inter- disciplinary Journal of Nonlinear Science31(2) (Feb 2021).https://doi.org/10. 1063/5.0032311

    2021

  34. [42]

    In: 2017 IEEE Conference on Control Tech- nology and Applications (CCTA)

    Pant, Y.V., Abbas, H., Mangharam, R.: Smooth operator: Control using the smooth robustness of temporal logic. In: 2017 IEEE Conference on Control Tech- nology and Applications (CCTA). pp. 1235–1240 (2017). https://doi.org/10. 1109/CCTA.2017.8062628

    2017

  35. [43]

    In: 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS)

    Pant, Y.V., Abbas, H., Quaye, R.A., Mangharam, R.: Fly-by-Logic: Control of Multi-Drone Fleets with Temporal Logic Objectives. In: 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS). pp. 186–197 (2018). https://doi.org/10.1109/ICCPS.2018.00026

    2018

  36. [44]

    In: Proceedings of the International Conference on Cyber-Physical Systems (ICCPS)

    Pant,Y.V.,Abbas,H.,Quaye,R.A.,Mangharam,R.:Fly-by-logic:controlofmulti- drone fleets with temporal logic objectives. In: Proceedings of the International Conference on Cyber-Physical Systems (ICCPS). pp. 186–197 (2018)

    2018

  37. [45]

    In: 18th Annual Symposium on Foun- dations of Computer Science (Sfcs 1977)

    Pnueli, A.: The temporal logic of programs. In: 18th Annual Symposium on Foun- dations of Computer Science (Sfcs 1977). pp. 46–57. ieee (1977)

    1977

  38. [46]

    In: 2023 21st ACM-IEEE International Symposium on Formal Methods and Models for System Design (MEMOCODE)

    Qin, X., Aréchiga, N., Deshmukh, J., Best, A.: Robust Testing for Cyber-Physical Systems using Reinforcement Learning. In: 2023 21st ACM-IEEE International Symposium on Formal Methods and Models for System Design (MEMOCODE). pp. 36–46 (2023)

    2023

  39. [47]

    Journal of Machine Learning Research (2021)

    Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable- baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research (2021)

    2021

  40. [48]

    Reinbacher, T., Rozier, K.Y., Schumann, J.: Temporal-Logic Based Runtime Ob- server Pairs for System Health Management of Real-Time Systems. In: Hutchison, D., Kanade, T., Kittler, J., Kleinberg, J.M., Mattern, F., Mitchell, J.C., Naor, M., Nierstrasz, O., Pandu Rangan, C., St...

    2014

  41. [49]

    In: Proceedings of Robotics: Science and Systems XII (2016)

    Sadigh, D., Kapoor, A.: Safe control under uncertainty with probabilistic signal temporal logic. In: Proceedings of Robotics: Science and Systems XII (2016)

    2016

  42. [50]

    In: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems

    Salzman, O., Stern, R.: Research challenges and opportunities in multi-agent path finding and multi-agent pickup and delivery problems. In: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems. pp. 1711–1715 (2020)

    2020

  43. [51]

    3176–3205

    Sandholm, W.H.: Evolutionary Game Theory, pp. 3176–3205. Springer New York, New York, NY (2009).https://doi.org/10.1007/978-0-387-30440-3_188

    2009 doi

  44. [52]

    arXiv preprint arXiv:1707.06347 (2017)

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  45. [53]

    Cambridge University Press (1982)

    Smith, J.: Evolution and the Theory of Games. Cambridge University Press (1982)

    1982

  46. [54]

    In: Intelligent unmanned ground vehicles, pp

    Stentz, A.: Optimal and efficient path planning for partially known environments. In: Intelligent unmanned ground vehicles, pp. 203–220. Springer (1997)

    1997

  47. [55]

    In: Twelfth Annual Symposium on Combinatorial Search (2019)

    Stern,R.,Sturtevant,N.R.,Felner,A.,Koenig,S.,Ma,H.,Walker,T.T.,Li,J.,Atz- mon, D., Cohen, L., Kumar, T.S., et al.: Multi-agent pathfinding: Definitions, vari- ants, and benchmarks. In: Twelfth Annual Symposium on Combinatorial Search (2019)

    2019

  48. [56]

    MIT press (2018)

    Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT press (2018)

    2018

  49. [57]

    Varambally, S., Li, J., Koenig, S.: Which MAPF Model Works Best for Automated Warehousing? In: Proceedings of the International Symposium on Combinatorial Search. vol. 15, pp. 190–198 (2022)

    2022

  50. [58]

    Machine learning8(3), 279–292 (1992)

    Watkins, C.J., Dayan, P.: Q-learning. Machine learning8(3), 279–292 (1992)

    1992

  51. [59]

    In: 2024 International Conference on Autonomous Agents and Multiagent Systems

    Williams, S., Deshmukh, J.: Potential Games on Cubic Splines for Multi-Agent Motion Planning of Autonomous Agents. In: 2024 International Conference on Autonomous Agents and Multiagent Systems. University of Southern California, Los Angeles, CA, USA (2024)

    2024

  52. [60]

    https://doi.org/10

    Zhou, W., Li, W.: A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machines (Apr 2022). https://doi.org/10. 48550/arXiv.2204.09772

    2022 doi arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.