RECORD
Benchmarking projective simulation in navigation problems
Machine Learning · Submitted 23 Apr 2018
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our comparison demonstrates that the performance of PS and standard learning approaches are qualitatively and quantitatively similar, while it is much easier to choose optimal model parameters in case of projective simulation, with a reduced computational effort of one to two orders of magnitude.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Projective simulation (PS) is a model for intelligent agents with a deliberation capacity that is based on episodic memory. The model has been shown to provide a flexible framework for constructing reinforcement-learning agents, and it allows for quantum mechanical generalization, which leads to a speed-up in deliberation time. PS agents have been applied successfully in the context of complex skill learning in robotics, and in the design of state-of-the-art quantum experiments. In this paper, we study the performance of projective simulation in two benchmarking problems in navigation, namely the grid world and the mountain car problem. The performance of PS is compared to standard tabular reinforcement learning approaches, Q-learning and SARSA. Our comparison demonstrates that the performance of PS and standard learning approaches are qualitatively and quantitatively similar, while it is much easier to choose optimal model parameters in case of projective simulation, with a reduced computational effort of one to two orders of magnitude. Our results show that the projective simulation model stands out for its simplicity in terms of the number of model parameters, which makes it simple to set up the learning agent in unknown task environments.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Projective simulation (PS) is a model for intelligent agents with a deliberation capacity that is based on episodic memory. | background 0.99 | 0.05 | 0.02 | asserted 1.00 |
| s1 | The model has been shown to provide a flexible framework for constructing reinforcement-learning agents, and it allows for quantum mechanical generalization, which leads to a speed-up in deliberation time. | background 1.00 | 0.09 | 0.03 | asserted 1.00 |
| s2 | PS agents have been applied successfully in the context of complex skill learning in robotics, and in the design of state-of-the-art quantum experiments. | background 1.00 | 0.10 | 0.02 | asserted 1.00 |
| s3 | In this paper, we study the performance of projective simulation in two benchmarking problems in navigation, namely the grid world and the mountain car problem. | contribution 0.97 | 0.81 | 0.03 | asserted 0.89 |
| s4 | The performance of PS is compared to standard tabular reinforcement learning approaches, Q-learning and SARSA. | method 0.91 | 0.78 | 0.03 | asserted 0.95 |
| s5 | Our comparison demonstrates that the performance of PS and standard learning approaches are qualitatively and quantitatively similar, while it is much easier to choose optimal model parameters in case of projective simulation, with a reduced computational effort of one to two orders of magnitude. | result 0.99 | 0.96 | 0.78 | asserted 1.00 |
| s6 | Our results show that the projective simulation model stands out for its simplicity in terms of the number of model parameters, which makes it simple to set up the learning agent in unknown task environments. | interpretation 0.61 | 0.88 | 0.28 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Projective simulation (PS) is a model for intelligent agents with a deliberation capacity that is based on episodic memory. The model has been shown to provide a flexible framework for constructing reinforcement-learning agents, and it allows for quantum mechanical generalization, which leads to a speed-up in deliberation time. PS agents have been applied successfully in the context of complex skill learning in robotics, and in the design of state-of-the-art quantum experiments. In this paper, we study the performance of projective simulation in two benchmarking problems in navigation, namely the grid world and the mountain car problem. The performance of PS is compared to standard tabular reinforcement learning approaches, Q-learning and SARSA. Our comparison demonstrates that the performance of PS and standard learning approaches are qualitatively and quantitatively similar, while it is much easier to choose optimal model parameters in case of projective simulation, with a reduced computational effort of one to two orders of magnitude. Our results show that the projective simulation model stands out for its simplicity in terms of the number of model parameters, which makes it simple to set up the learning agent in unknown task environments.
Human review
How the literature uses it
1 arXiv paper cite this in 2 sentences
Papers citing it · 1 sentence without a confident label
Uses its method or tool 1 paper
-
As a specific learning model we employ Projective Simulation (PS) [ref] , which has been shown to perform well in standard RL problems [ref] , in advanced robotics applications [ref] , and recently PS has been used to design new quantum experiments [ref] .
Optimizing Quantum Error Correction Codes with Reinforcement Learning · Framework & Overview · 0.88
What it cites · 4 sentences without a confident label
Cites as background 2 papers
-
The second navigation problem that we consider is the mountain car problem, which was first introduced in Ref. [ref] and is a part of a modern benchmarking framework [ref] .
OpenAI Gym · The mountain car problem · 0.91
-
[ref] and later modified in many other works, see, e.g., refs [ref] .
Learning to Navigate in Complex Environments · The grid world problem · 0.95
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.