RECORD
Estimating scale-invariant future in continuous time
Artificial Intelligence · Submitted 18 Feb 2018
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially-discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. On the other hand, an important drawback of model-free algorithms is the need to select a time-scale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically-compressed scale, and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation which generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| s1 | Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially-discounted future reward using the Bellman equation (model-free algorithms). | background 1.00 | 0.04 | 0.03 | asserted 1.00 |
| s2 | An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. | background 0.78 | 0.08 | 0.02 | asserted 1.00 |
| s3 | On the other hand, an important drawback of model-free algorithms is the need to select a time-scale required for exponential discounting. | background 0.70 | 0.05 | 0.02 | asserted 1.00 |
| s4 | We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. | contribution 0.99 | 0.90 | 0.33 | asserted 1.00 |
| s5 | This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically-compressed scale, and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. | method 0.61 | 0.82 | 0.47 | asserted 0.87 |
| s6 | The representation of future time retains information about what will happen when. | result 0.71 | 0.57 | 0.11 | asserted 1.00 |
| s7 | The entire timeline can be constructed in a single parallel operation which generates concrete behavioral and neural predictions. | result 0.40 | 0.72 | 0.18 | asserted 0.99 |
| s8 | This computational mechanism could be incorporated into future reinforcement learning algorithms. | future work 0.98 | 0.13 | 0.13 | hedged possibility 1.00 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially-discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. On the other hand, an important drawback of model-free algorithms is the need to select a time-scale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically-compressed scale, and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation which generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.