pith. sign in

arxiv: 1907.04060 · v1 · pith:L5PE25H7new · submitted 2019-07-09 · 💻 cs.NE · eess.IV

Event-based attention and tracking on neuromorphic hardware

Alpha Renner , Matthew Evanusa , Yulia Sandamirskaya This is my paper

Pith reviewed 2026-05-24 23:59 UTC · model grok-4.3

classification 💻 cs.NE eess.IV
keywords event-based visionneuromorphic hardwareselective attentionobject trackingspiking neural networksdynamic neural fieldsattractor dynamics
0
0 comments X

The pith

A recurrent spiking neural network sustains object tracking from sparse event input despite distractors or pauses.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a fully event-driven vision system for selective attention and tracking built on neuromorphic hardware. Input comes from an event-based camera and is processed by a recurrent spiking network that realizes attractor dynamics of dynamic neural fields. The central demonstration is that this network produces sustained activation capable of following an object even when the object slows or stops and when distractors appear, while also lowering the overall event rate. A sympathetic reader would care because the setup shows how spiking networks can maintain focus using only intermittent data rather than continuous dense input.

Core claim

We demonstrate capability of the system to create sustained activation that supports object tracking when distractors are present or when the object slows down or stops, reducing the number of generated events.

What carries the argument

recurrent spiking neural network implementing attractor-dynamics of dynamic neural fields, which generates and holds a stable focus state from incoming events

If this is right

  • Tracking continues during periods when the object produces few or no events.
  • Presence of distractors does not break the sustained focus state.
  • Overall number of events required for ongoing tracking is reduced.
  • The entire attention and tracking pipeline runs in a fully event-driven manner.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same attractor mechanism could support tracking in robotic platforms that must operate for long periods on limited power.
  • Scaling the network might allow simultaneous tracking of several objects if the field dynamics remain stable.
  • Coupling this attention stage with additional event-based processing layers on the same hardware could create end-to-end perception without frame-based conversion.

Load-bearing premise

The recurrent spiking neural network will maintain a stable focus state from sparse, intermittent event input without requiring continuous stimulation or external reset mechanisms.

What would settle it

A recording showing the activation pattern collapsing or drifting away from the object's location once events become sparse because the object has stopped moving.

Figures

Figures reproduced from arXiv: 1907.04060 by Alpha Renner, Matthew Evanusa, Yulia Sandamirskaya.

Figure 1
Figure 1. Figure 1: Schematic representation of a 1-dimensional winner [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: The plots show activity of spiking neurons on the Loihi chip, configured as a DNF in an input-driven (a, b, c) and self-sustained (d) [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Trajectory of the selected object (star), obtained from [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Object tracking experiment: (a) snapshots of input DAVIS frames (top); (b) DAVIS on (green) and off (red) events binned into [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Object tracking experiment 2: DAVIS on (green) and off (red) events binned into 10ms frames (top); firing rate of the first [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
read the original abstract

We present a fully event-driven vision and processing system for selective attention and tracking, realized on a neuromorphic processor Loihi interfaced to an event-based Dynamic Vision Sensor DAVIS. The attention mechanism is realized as a recurrent spiking neural network that implements attractor-dynamics of dynamic neural fields. We demonstrate capability of the system to create sustained activation that supports object tracking when distractors are present or when the object slows down or stops, reducing the number of generated events.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper presents a fully event-driven vision and processing system for selective attention and tracking realized on the Loihi neuromorphic processor interfaced to a DAVIS event-based sensor. The attention mechanism is implemented as a recurrent spiking neural network realizing the attractor dynamics of dynamic neural fields. The central claim is that the system produces sustained activation supporting object tracking in the presence of distractors or when the tracked object slows down or stops (thereby reducing the rate of generated events).

Significance. If the result holds, the work would demonstrate a hardware realization of event-driven attention that maintains a stable focus state from sparse, intermittent input without continuous stimulation. This would be a concrete contribution to neuromorphic vision, building on established dynamic neural field models and real Loihi/DAVIS hardware rather than simulation-only results.

major comments (2)
  1. [Implementation / Results (recurrent SNN and attractor dynamics)] The central claim requires that the recurrent SNN on Loihi implements stable DNF attractor dynamics that sustain a localized bump from event streams whose rate drops sharply or is corrupted by distractors. No explicit recurrent kernel, weight values, eigenvalue analysis, or Lyapunov stability confirmation is supplied to show that the operating point lies inside the known narrow parameter regime for bump stability.
  2. [Experimental evaluation / tracking demonstrations] The demonstrations of sustained activation when the object stops or distractors appear are reported, but the manuscript contains no systematic trials at event rates below the demonstrated cases, no parameter-robustness sweeps, and no failure-mode tests. Without these, it remains possible that the observed behavior is tied to the particular input statistics rather than a general property of the implemented dynamics.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed review. The comments highlight important aspects of the implementation and evaluation that we address below. We will incorporate clarifications and additional material in a revised manuscript where feasible.

read point-by-point responses

  1. Referee: [Implementation / Results (recurrent SNN and attractor dynamics)] The central claim requires that the recurrent SNN on Loihi implements stable DNF attractor dynamics that sustain a localized bump from event streams whose rate drops sharply or is corrupted by distractors. No explicit recurrent kernel, weight values, eigenvalue analysis, or Lyapunov stability confirmation is supplied to show that the operating point lies inside the known narrow parameter regime for bump stability.

    Authors: We agree that the manuscript would benefit from an explicit description of the recurrent kernel. The connectivity follows the standard DNF Mexican-hat profile (local excitation, global inhibition) with parameters chosen from the literature to place the system in the stable bump regime. We will add the precise weight values and a brief description of the kernel in the revised manuscript. A full eigenvalue or Lyapunov analysis was not included because the work emphasizes the hardware realization and empirical behavior rather than theoretical derivation; the sustained tracking results provide supporting evidence for stability under the tested inputs. revision: partial

  2. Referee: [Experimental evaluation / tracking demonstrations] The demonstrations of sustained activation when the object stops or distractors appear are reported, but the manuscript contains no systematic trials at event rates below the demonstrated cases, no parameter-robustness sweeps, and no failure-mode tests. Without these, it remains possible that the observed behavior is tied to the particular input statistics rather than a general property of the implemented dynamics.

    Authors: We acknowledge that the current demonstrations are limited to specific input conditions and that additional systematic testing would strengthen the claims. We will include further trials at lower event rates and a short robustness discussion in the revision. Comprehensive parameter sweeps and exhaustive failure-mode analysis are constrained by the hardware setup and experiment duration, but the added trials will help address the concern about input-statistic dependence. revision: yes

Circularity Check

0 steps flagged

No circularity detected; empirical hardware demonstration is self-contained

full rationale

The paper describes a hardware implementation of known dynamic neural field attractor dynamics in a recurrent SNN on Loihi, interfaced with a DAVIS event sensor. Claims of sustained activation for tracking rest on experimental results with real event streams rather than any mathematical derivation, parameter fitting, or self-citation chain that reduces outputs to inputs by construction. DNF models are treated as established external foundations, and no load-bearing step equates a 'prediction' to a fitted quantity or renames an input as a result.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that dynamic neural fields can be realized as recurrent spiking networks on neuromorphic hardware and will exhibit attractor behavior from event input. No free parameters or invented entities are identifiable from the abstract.

axioms (1)
  • domain assumption Dynamic neural fields can be implemented as recurrent spiking neural networks that produce sustained attractor states from sparse event input.
    Invoked to support the sustained activation claim in the abstract.

pith-pipeline@v0.9.0 · 5595 in / 1111 out tokens · 17300 ms · 2026-05-24T23:59:28.086628+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 22 canonical work pages

  1. [1]

    STICK: Spike Time Inter- val Computational Kernal, a Framework for General Pur- pose Computation Using Neurons, Precise Timing and Syn- chrony,

    X. Lagorce and R. Benosman, “STICK: Spike Time Inter- val Computational Kernal, a Framework for General Pur- pose Computation Using Neurons, Precise Timing and Syn- chrony,”Neural computation, vol. 27, pp. 2261–2317, 2015. 1

    work page 2015

  2. [2]

    HOTS: A Hierarchy Of event-based Time- Surfaces for pattern recognition,

    X. Lagorce, G. Orchard, F. Gallupi, B. E. Shi, and R. Benosman, “HOTS: A Hierarchy Of event-based Time- Surfaces for pattern recognition,” IEEE Transactions on (a) (b) (c) Figure 6. Object tracking experiment 2: DA VIS on (green) and off (red) events binned into 10ms frames (top); firing rate of the first non-selective WTA layer on Loihi (middle); output of...

    work page 2016

  3. [3]

    A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems,

    M. Osswald, S.-H. Ieng, R. Benosman, and G. Indiveri, “A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems,” Scientific Reports , vol. 7, no. December 2016, p. 40703,

    work page 2016

  4. [4]

    Available: http://www.nature.com/articles/ srep40703 1

    [Online]. Available: http://www.nature.com/articles/ srep40703 1

    work page

  5. [5]

    Neuromorphic Event-Based 3D Pose Estimation,

    D. R. Valeiras, G. Orchard, S. H. Ieng, and R. B. Benosman, “Neuromorphic Event-Based 3D Pose Estimation,” vol. 9, no. January, pp. 1–15, 2015. 1

    work page 2015

  6. [6]

    Event- based 3D reconstruction from neuromorphic retinas,

    J. Carneiro, S. H. Ieng, C. Posch, and R. Benosman, “Event- based 3D reconstruction from neuromorphic retinas,”Neural Networks, vol. 45, pp. 27–38, 2013. [Online]. Available: http://dx.doi.org/10.1016/j.neunet.2013.03.006 1

    work page doi:10.1016/j.neunet.2013.03.006 2013

  7. [7]

    Asynchronous frameless event-based optical flow

    R. Benosman, S.-H. Ieng, C. Clercq, C. Bartolozzi, and M. Srinivasan, “Asynchronous frameless event-based optical flow.” Neural networks : the official journal of the International Neural Network Society , vol. 27, pp. 32–7, mar 2012. [Online]. Available: http://www.ncbi.nlm.nih. gov/pubmed/22154354 1

    work page arXiv 2012

  8. [8]

    Memory and information pro- cessing in neuromorphic systems,

    G. Indiveri and S.-C. Liu, “Memory and information pro- cessing in neuromorphic systems,”Proceedings of the IEEE, vol. 103, no. 8, pp. 1379–1397, 2015. 1

    work page 2015

  9. [9]

    Loihi: a Neuromorphic Manycore Processor with On-Chip Learning,

    M. Davies, N. Srinivasa, T.-h. Lin, G. Chinya, Y . Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, Y . Liao, C.-k. Lin, A. Lines, R. Liu, D. Mathaikutty, S. Mccoy, A. Paul, J. Tse, G. Venkataramanan, Y .-h. Weng, A. Wild, Y . Yang, and H. Wang, “Loihi: a Neuromorphic Manycore Processor with On-Chip Learning,”IEEE Micro, vol. 38, no. 1, pp. 82– 99...

    work page 2018

  10. [10]

    Artificial brains. A million spiking-neuron integrated circuit with a scalable communication network and interface

    P. A. Merolla, J. V . Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y . Nakamura, B. Brezzo, I. V o, S. K. Esser, R. Appuswamy, B. Taba, A. Amir, M. D. Flickner, W. P. Risk, R. Manohar, D. S. Modha, C. Eliasmith, T. C. Stewart, X. Choo, T. Bekolay, T. DeWolf, Y . Tang, D. Rasmussen, M. Mishkin, L. G. Unger...

    work page arXiv 2014

  11. [11]

    Overview of the SpiNNaker System Architecture,

    S. B. Furber, D. R. Lester, L. A. Plana, J. D. Garside, E. Painkras, S. Temple, and A. D. Brown, “Overview of the SpiNNaker System Architecture,” IEEE Transactions on Computers, vol. 62, no. 12, pp. 2454–2467, 2012. 1

    work page 2012

  12. [12]

    Modeling Selective Attention Using a Neuromorphic Analog VLSI Device,

    G. Indiveri, “Modeling Selective Attention Using a Neuromorphic Analog VLSI Device,” Neural Computation, vol. 12, no. 12, 2000. [Online]. Available: http://dx.doi.org/ 10.1162/089976600300014755 1

    work page doi:10.1162/089976600300014755 2000

  13. [13]

    An accelerated LIF neuronal network array for a large-scale mixed-signal neu- romorphic architecture,

    S. A. Aamir, Y . Stradmann, P. Muller, C. Pehle, A. Hartel, A. Grubl, J. Schemmel, and K. Meier, “An accelerated LIF neuronal network array for a large-scale mixed-signal neu- romorphic architecture,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 65, no. 12, pp. 4299–4312,

    work page

  14. [14]

    Point-to-Point Connectivity Between Neu- romorphic Chips using Address-Events,

    K. A. Boahen, “Point-to-Point Connectivity Between Neu- romorphic Chips using Address-Events,” Ieee Transactions on Circuits & Systems , vol. 47, no. 5, pp. 416–434,

    work page

  15. [15]

    Available: https://web.stanford.edu/group/ brainsinsilicon/pdf/00{ }journ{ }IEEEtsc{ }Point.pdf 1

    [Online]. Available: https://web.stanford.edu/group/ brainsinsilicon/pdf/00{ }journ{ }IEEEtsc{ }Point.pdf 1

    work page

  16. [16]

    A 240 180 130 dB 3 µs latency global shutter spatiotem- poral vision sensor,

    C. Brandli, R. Berner, M. Yang, S. C. Liu, and T. Delbruck, “A 240 180 130 dB 3 µs latency global shutter spatiotem- poral vision sensor,” IEEE Journal of Solid-State Circuits , vol. 49, no. 10, pp. 2333–2341, 2014. 1, 2

    work page 2014

  17. [17]

    Sch ¨oner and J

    G. Sch ¨oner and J. P. Spencer, Eds., Dynamic Thinking: A Primer on Dynamic Field Theory . Oxford University Press, 2015. [Online]. Available: https://books.google. ch/books?id=iLVPAwAAQBAJ{&}printsec=frontcover{&} hl=de{&}source=gbs{ }ge{ }summary{ }r{&}cad=0 2, 3, 4

    work page 2015

  18. [18]

    Dynamic Neural Fields as a Step To- wards Cognitive Neuromorphic Architectures,

    Y . Sandamirskaya, “Dynamic Neural Fields as a Step To- wards Cognitive Neuromorphic Architectures,” Frontiers in Neuroscience, vol. 7, p. 276, 2013. 2

    work page 2013

  19. [19]

    A neuromorphic ap- proach for tracking using dynamic neural fields on a pro- grammable vision-chip,

    J. N. P. Martel and Y . Sandamirskaya, “A neuromorphic ap- proach for tracking using dynamic neural fields on a pro- grammable vision-chip,” in Proceedings of the 10th Interna- tional Conference on Distributed Smart Camera (ICDSC), ACM, 2016. 2

    work page 2016

  20. [20]

    Mapping spiking neural networks onto a many- core neuromorphic architecture,

    T.-H. Lin, G. N. Chinya, M. Davies, C.-K. Lin, A. Wild, and H. Wang, “Mapping spiking neural networks onto a many- core neuromorphic architecture,” ACM SIGPLAN Notices , vol. 53, no. 4, pp. 78–89, 2018. 2

    work page 2018

  21. [21]

    Using Dynamic Field Theory to extend the em- bodiment stance toward higher cognition,

    Y . Sandamirskaya, S. K. U. Zibner, S. Schneegans, and G. Sch¨oner, “Using Dynamic Field Theory to extend the em- bodiment stance toward higher cognition,”New Ideas in Psy- chology, vol. 31, no. 3, pp. 322–339, 2013. 3

    work page 2013

  22. [22]

    The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM,

    E. Mueggler, H. Rebecq, G. Gallego, T. Delbruck, and D. Scaramuzza, “The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM,” International Journal of Robotics Research, vol. 36, no. 2, pp. 142–149, 2017. 5

    work page 2017