Pith. sign in

REVIEW 4 major objections 6 minor 69 references

Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retina

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Neural Ganglion Sensors beat DVS at equal or lower bandwidth

desk verdict A well-motivated differentiable event-camera simulator with learned spatial kernels, but the experiments don't isolate the spatial-kernel contribution from threshold learning, so the headline claim is suggestive rather than proven. read the letter →

arxiv 2504.13457 v1 pith:Z32SV4GM submitted 2025-04-18 cs.CV cs.ETeess.IV

Haley M. So , Gordon Wetzstein This is my paper · ORCID
classification cs.CVcs.ETeess.IV
keywords eventcamerasretinalganglioncellsspatialkernelsdifferentiablesimulationvideointerpolationopticalflowbandwidthin-pixelcompute
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an event camera can be improved by replacing its per-pixel trigger rule with a learned spatio-temporal kernel, mimicking the way retinal ganglion cells pool light from a neighborhood before firing. The authors build a differentiable event simulator that supports such spatial kernels, then train the kernels together with downstream vision models for two tasks: video interpolation and optical flow. They report that the learned kernels outperform a conventional event camera at matched or lower event counts—for example, 35.4 dB PSNR versus 33.8 dB at roughly equal bandwidth for interpolation, and end-point error 2.42 versus 2.80 with about half the events for optical flow. If the result holds on physical sensors, it suggests that moving a modest amount of spatial computation into the sensor plane can buy both accuracy and bandwidth savings.

What carries the argument

The central object is the learned spatio-temporal event kernel $W$ in the RGC trigger $P=\mathbf{1}(f(W*(I_{\mathrm{curr}}-I_{\mathrm{mem}}))>\delta)$, combined with the closed-form differentiable binning (equations 7--10) that maps events between a frame pair into the two nearest time bins with arithmetic-sequence weights. Because the bin values are computed in closed form, gradients can flow from the event voxel grid back to the kernel, using the straight-through estimator for the quantization of event count and thresholding. This kernel-plus-binning loop is what allows task-specific sensing to be optimized for both accuracy and event bandwidth.

What would settle it

Run the learned kernels on real events from a physical DAVIS-style sensor facing a calibration target with known displacement, and compare PSNR or EPE against a conventional DVS at the same event count; if the learned kernel's advantage disappears or reverses, the central claim falls.

Watch

Extended reading notes

Core claim

Conventional event cameras trigger an event when the temporal intensity change at a single pixel crosses a threshold. This paper's central claim is that the per-pixel trigger is unnecessarily impoverished: a linear-nonlinear cascade over a small spatial patch, like the receptive field of an RGC, can decide which events are worth sending. The paper defines the RGC event model $\mathbf{1}(f([W*(I_{\mathrm{curr}}-I_{\mathrm{mem}})](x,y,t))>\delta)$ and learns the kernel $W$ and thresholds end-to-end, backpropagating through a closed-form differentiable binning of events into voxel grids. On video interpolation, the learned linear-domain kernel with spatially varying parameters reaches $35.4$ dB PSNR at $32.2$k events per bin, against $33.8$ dB for DVS at $33.0$k; on optical flow, the learned kernel reaches EPE $2.42$ at $3.80$M events versus $2.80$ at $7.30$M for the best DVS baseline. Learning multiple complementary kernels per pixel adds further gains at fixed bandwidth, echoing the parallel RGC types in the retina.

Load-bearing premise

The quantitative gains are measured on events simulated from video, and for optical flow the video is itself generated by interpolation, so the whole comparison assumes those synthetic events behave like a physical event camera's output.

Editorial extensions

If this is right

  • The same learning loop can be retargeted to other event-based tasks such as depth estimation, recognition, or state estimation, with the sensor tuned to the task's loss.
  • Because the gains appear at matched bandwidth, a hardware sensor implementing a few learned kernels per pixel could deliver better outputs than a conventional DVS at no increase in readout cost.
  • The learned kernels differ visibly between interpolation (nearly symmetric, contrast-sensitive) and optical flow (asymmetric, direction-selective), showing that a single fixed sensing kernel leaves performance on the table.
  • Spatially varying kernels that assign one of four learned kernels per pixel match the performance of four kernels per pixel, suggesting a cheaper hardware route to most of the benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The synthetic-events testbed means the reported PSNR/EPE ratios are upper bounds on what a physical sensor would deliver; the real advantage will only be known once a sensor with learned spatial kernels is built and tested on real scenes.
  • The learned kernels may be partially replicable in software by applying a fixed learned convolution to the intensity stream before the standard DVS threshold—if the full gain persists under that condition, the hardware could be simplified to a per-pixel threshold after a spatial pre-filter.
  • A straightforward testable extension is to train the same kernels on real event recordings (e.g., from a DAVIS camera) with ground-truth flow from projected patterns, and compare the learned kernel shapes against the ones for simulated events.
  • The approach implies a sensor-design spectrum between general-purpose DVS and task-specialized chips, with the bandwidth-equalized Pareto front of this paper as a way to choose where to land.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Neural Ganglion Sensors, an extension of event cameras in which the event trigger uses a learned spatial-temporal kernel W instead of the per-pixel identity, together with learnable positive and negative contrast thresholds. The authors build a differentiable event simulator based on ESIM/V2E, with closed-form binning so that gradients can flow from the task loss back to W and the thresholds. They evaluate the approach on video interpolation (REFID on GoPRO) and optical flow (IDNet on the TartanAir 'Hard' subset, with EMA-VFI interpolation), comparing against conventional DVS and handcrafted center-surround baselines. They report that learned RGC events improve PSNR and EPE at lower or equal event counts, and that multiple learned kernels per pixel further improve interpolation quality.

Significance. If the result holds, this is a valuable demonstration of task-specific sensing co-optimized with a downstream network, and the differentiable simulator is a potentially reusable contribution. The biological motivation is well connected to existing retinal models. The paper also uses scene-level held-out splits for optical flow, which strengthens the generalization claim relative to random splits. However, the central causal claim—that spatial aggregation, rather than threshold learning or end-to-end sensor optimization, drives the gains—is not isolated, and all experiments are conducted in simulation. These issues must be addressed before the paper's main conclusion is fully supported.

major comments (4)
  1. [Sec. 3.2, Eq. (2)-(4); Fig. 2; Table 1] The framework jointly learns the spatial kernel W and the contrast thresholds δ, and the headline comparisons are against conventional DVS with fixed thresholds. There is no control condition in which W is fixed to the identity kernel while thresholds are learned under the same pipeline, sparsity loss, and task network. Without such a control, the improvements in Fig. 2 and Table 1 could be caused by threshold learning, by end-to-end sensor/network optimization, or by the interaction of the sparsity loss with the task loss, rather than by spatial aggregation. This concern is reinforced by the observation that the handcrafted spatial CSDVS baselines perform about the same as DVS; only the jointly learned conditions beat DVS. Please add a learned-DVS baseline (W fixed to identity) to isolate the effect of the spatial kernel.
  2. [Table 1, Fig. 2, Sec. 5.4] All quantitative results appear to come from single runs without error bars or multiple seeds. Some reported differences are small (for example, the 0.40 dB gain of RGC2 over RGC1 and the 0.35 dB gain of RGC4 over RGC2 in Sec. 5.4, and the 0.4 dB range across kernel sizes in Supplement S1.4.1), so run-to-run variance could change the conclusions. Please report means and standard deviations over at least three seeds, and preferably statistical significance tests, for the main bandwidth-vs-performance comparisons.
  3. [Sec. 4.2, Supplement S1.2] For the optical flow experiments, the input high-speed frames are produced by EMA-VFI interpolation of TartanAir frames, so the simulated events are generated from synthetic, twice-processed intensity dynamics. The paper states that the proposed sensing improves performance relative to conventional event cameras, but no experiment uses real event camera data or a realistic sensor noise model beyond additive Gaussian noise. Because the paper's topic is a physical sensor design, the transfer of these results to hardware is not established. Please either add a real-data validation, strengthen the noise and non-ideality model, or explicitly limit the claims to the simulated setting and discuss the transfer risk.
  4. [Sec. 5.4, Supplement S1.5.3] The multi-channel comparisons use event count as the measure of bandwidth. In a physical multi-channel event sensor, each event must also encode which channel fired, which requires additional bits; the supplement's suggested Huffman encoding is not quantified. Consequently, the statement that RGC4 and RGC2 achieve their gains 'at the same bandwidth' as RGC1 is not established in bitrate terms, and the improvement could shrink once channel-identification bits are counted. Please report bitrate or a byte-equivalent measure, or revise the claim to refer specifically to event count rather than bandwidth.
minor comments (6)
  1. [Eq. (2) vs. Eq. (5)] Equation (2) uses a strict inequality ('>') for the event trigger while Eq. (5) uses '≥'; please unify the threshold comparison across the paper.
  2. [Eq. (7)] The definition N = IRGC // δ should clarify the floor/rounding semantics for negative values of IRGC and how polarity is assigned before quantization; this is important because IRGC can be negative.
  3. [Supplement S1.1, Eq. (3)] Equation (3) in the supplement repeats the same expression for βr twice; the redundancy looks like a typo and should be cleaned up.
  4. [Fig. 2 caption] The DVS curve in Fig. 2 appears to be obtained by sweeping contrast thresholds; please state this explicitly in the caption and mark the discrete threshold settings used.
  5. [Sec. 3.2.1 / Code availability] The paper says the differentiable simulator 'will be open-sourced' but no link or release plan is given; please add a data/code availability statement.
  6. [Sec. 6, Hardware Feasibility] The hardware feasibility paragraph acknowledges that current in-pixel processors require quantized or ternary kernels, but the experiments use full-precision kernels; a brief note on whether the reported gains survive quantization constraints would help.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: learned kernels are fitted on training data and evaluated on held-out test sets; the biological analogy and self-citations are not load-bearing.

full rationale

We walked the derivation chain from Eqs. (2)-(4) and (10). The RGC trigger, binary output, memory update, and closed-form differentiable binning define a sensor model with learnable W and thresholds; W and delta are optimized on training splits with a task loss plus sparsity loss, and all headline numbers are evaluations on held-out test splits (e.g., Sec. 5.1: DVS 33.8 dB at 33.0k events vs. RGC-lin-sv 35.4 dB at 32.2k; Table 1: RGC-lin EPE 2.42 at 3.80M events vs. DVS 0.1T EPE 2.80 at 7.30M). No reported metric is fed back as an input, and no fitted parameter is renamed as a prediction. Bandwidth is tuned via the sparsity-loss weight and then measured, not derived from the performance numbers. The paper does invoke prior self-work (e.g., refs. [37], [62], [63]) and the center-surround hardware idea [33], but these are background and hardware-feasibility citations; none is used as a uniqueness theorem or as the justification that learned spatial kernels improve performance. The biological analogy is motivational framing rather than evidence. The absence of a control with an identity kernel and learned thresholds is a legitimate experimental-design concern about attributing gains specifically to spatial aggregation, but it is not circularity under the stated criteria: no equation in the paper reduces the reported result to a fitted value or to a self-citation. We therefore find no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces a sensor design concept, not a new physical entity. The load-bearing assumptions are all about the fidelity of the simulated testbed: the event camera model, the differentiable binning approximation, and the use of interpolated video frames. The learned kernels and thresholds are fitted parameters, which is the intended method rather than hidden fitting.

free parameters (4)
  • Spatial event kernel W = Learned per task and per channel; visualized in Fig. 5 and supplement, not numerically tabulated
    The central trainable object: convolved with Icurr-Imem to decide when events fire. Fitted end-to-end with task loss plus sparsity loss.
  • Positive and negative contrast thresholds = Not reported numerically
    Learnable thresholds in the event trigger; they directly control event bandwidth and are tuned jointly with the kernel.
  • Sparsity loss weighting = Varied across runs to produce the bandwidths in Fig. 2 and Table 1
    Hand-chosen hyperparameter that moves each model along the performance-bandwidth Pareto front; the headline comparisons are made at bandwidths selected by this weight.
  • Kernel size k = 5x5 default; swept over 3, 5, 7, 9, 11
    Model choice selected by the authors; the sweep shows performance is roughly invariant at fixed bandwidth, so this is not load-bearing.
assumptions (5)
  • domain assumption The linear-nonlinear cascade model approximates retinal ganglion cell responses closely enough to motivate the event-generation model.
    Used in Section 3.1, Eq. (1), to derive the spatio-temporal event trigger from biological RGC models.
  • domain assumption The threshold-and-memory event camera model faithfully represents physical event camera behavior.
    Equations (3)-(5) in Section 3.1.1 define event output and memory update; the entire evaluation assumes this model captures real DVS behavior.
  • standard math Linear event binning into the two nearest time bins, with straight-through gradient estimation, is a valid differentiable approximation for training.
    Adopted from [54], [55], and [56] in Section 3.2.2; the closed-form arithmetic-series solution is derived inside the paper.
  • domain assumption EMA-VFI-interpolated frames stand in for true high-speed intensity frames when simulating optical-flow events.
    In Section 4.2 and Supplement S1.2, TartanAir frames are interpolated 15x with EMA-VFI to generate simulated events; if these frames are inaccurate, the simulated events do not match real scene dynamics.
  • domain assumption The differentiable simulator, including shot noise, refractory period, and non-ideal thresholds, reproduces the behavior of real event cameras sufficiently for conclusions to transfer.
    Section 3.2.1 and Supplement S1.1 extend ESIM/V2E-style simulation; no real event camera recordings are used to validate this transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retina." pith.science (2026). https://pith.science/paper/Z32SV4GM

@misc{pith2026250413457,
  author       = {Pith},
  title        = {Pith review of: Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retina},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z32SV4GM}},
  note         = {Machine review of arXiv:2504.13457}
}
read the original abstract

Inspired by the data-efficient spiking mechanism of neurons in the human eye, event cameras were created to achieve high temporal resolution with minimal power and bandwidth requirements by emitting asynchronous, per-pixel intensity changes rather than conventional fixed-frame rate images. Unlike retinal ganglion cells (RGCs) in the human eye, however, which integrate signals from multiple photoreceptors within a receptive field to extract spatio-temporal features, conventional event cameras do not leverage local spatial context when deciding which events to fire. Moreover, the eye contains around 20 different kinds of RGCs operating in parallel, each attuned to different features or conditions. Inspired by this biological design, we introduce Neural Ganglion Sensors, an extension of traditional event cameras that learns task-specific spatio-temporal retinal kernels (i.e., RGC "events"). We evaluate our design on two challenging tasks: video interpolation and optical flow. Our results demonstrate that our biologically inspired sensing improves performance relative to conventional event cameras while reducing overall event bandwidth. These findings highlight the promise of RGC-inspired event sensors for edge devices and other low-power, real-time applications requiring efficient, high-resolution visual streams.

Figures

Figures reproduced from arXiv: 2504.13457 by the authors.

Figure 1
Figure 1. Analogy between Neural Ganglion Sensors and the human retina: On the left, we show a simplified diagram of different layers in the human retina. Light hits the photoreceptors (rods and cones), of which there are about 100 million per eye. The signals get transferred and modulated through Bipolar cells along with additional Horizontal and Amacrine cells. In the end, the roughly 1 million Retinal Ganglion Cells (RGCs)… view at source ↗
Figure 2
Figure 2. Video Interpolation Performance vs Bandwidth Trade-off. We perform video interpolation using DVS, CSDVS, CSDVS-Delbruck, ¨ RGC-log (learned, log regime), RGC-lin (learned, linear space), and RGC-lin-sv (learned, linear space, and spatially varying). For any given bandwidth, RGC-lin-sv provides the best performance. additional weighted L1 loss on the number of events to push the model to learn RGC events that maximiz… view at source ↗
Figure 3
Figure 3. Video Interpolation Qualitative Results. For each scene, we compare the reconstructions of the middle frame in the sequence for DVS, CSDVS, and RGC-lin. The top row shows the generated events, binned into the corresponding middle time bin, the second is the predicted image and the bottom row shows zoom-ins. The right-most column shows the start and end frames, alpha-blended, ground truth frame, and zoom-ins. PSNR(↑)… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: shows three samples of the optical flow reconstruc￾tion from DVS and from our Learned RGC-lin events. The DVS model corresponds to DVS 0.1T from tab. 1, as it had the best performance among the DVS models. Learning the kernel end-to-end allows RGC-lin to better reconst…
Figure 5
Figure 5. Figure 5: Comparison of Kernels. We show the 5 × 5 kernels for DVS, CSDVS, the learned RGC-lin kernels for video interpolation, and the learned RGC-lin kernels for optical flow. captures something akin to an edge detection or bandpass filter, which can be a powerful cue for figu…
Figure 6
Figure 6. Figure 6: Comparison between learned single and multi-event. We compare the learned kernels of a single RGC model, two RGC, and four RGC for interpolation and show the generated events for each. Each column is a different scene At the same bandwidth, the 2-kernel model had a 0.4…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 57 canonical work pages

  1. [1]

    E2(go)motion: Motion augmented event stream for egocentric action recognition,

    C. Plizzari, M. Planamente, G. Goletto, M. Cannici, E. Gusso, M. Matteucci, and B. Caputo, “E2(go)motion: Motion augmented event stream for egocentric action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 19 935–19 947

    work page 2022

  2. [2]

    Action recog- nition and benchmark using event cameras,

    Y. Gao, J. Lu, S. Li, N. Ma, S. Du, Y. Li, and Q. Dai, “Action recog- nition and benchmark using event cameras,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 14 081– 14 097, 2023

    work page 2023

  3. [3]

    Spatiotemporal filtering for event-based action recognition,

    R. Ghosh, A. K. Gupta, A. N. Silva, A. B. Soares, and N. V . Thakor, “Spatiotemporal filtering for event-based action recognition,” ArXiv, vol. abs/1903.07067, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:81978239

    arXiv 1903

  4. [4]

    Event-based background-oriented schlieren,

    S. Shiba, , F. Hamann, Y. Aoki, and G. Gallego, “Event-based background-oriented schlieren,” 2023

    work page 2023

  5. [5]

    Secrets of event-based optical flow,

    S. Shiba, Y. Aoki, and G. Gallego, “Secrets of event-based optical flow,” in European Conference on Computer Vision (ECCV), 2022, pp. 628–645

    work page 2022

  6. [6]

    Event transformer flownet for optical flow estimation,

    Y. Tian and J. Andrade-Cetto, “Event transformer flownet for optical flow estimation,” in 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21- 24, 2022 . BMVA Press, 2022. [Online]. Available: https: //bmvc2022.mpi-inf.mpg.de/0577.pdf

    work page 2022

  7. [7]

    Learn- ing an event sequence embedding for dense event-based deep stereo,

    S. Tulyakov, F. Fleuret, M. Kiefel, P . Gehler, and M. Hirsch, “Learn- ing an event sequence embedding for dense event-based deep stereo,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    work page 2019

  8. [8]

    Self-supervised intensity-event stereo matching,

    J. Gu, J. Zhou, R. S. W. Chu, Y. Chen, J. Zhang, X. Cheng, S. Zhang, and J. S. Ren, “Self-supervised intensity-event stereo matching,”

    work page

Show all 69 references
  1. [9]

    Event- based shape from polarization,

    M. Muglikar, L. Bauersfeld, D. Moeys, and D. Scaramuzza, “Event- based shape from polarization,” in IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), Jun 2023

    2023

  2. [10]

    Event-based vision: A survey,

    G. Gallego, T. Delbruck, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis, and D. Scaramuzza, “Event-based vision: A survey,”IEEE Transac- tions on Pattern Analysis & Machine Intelligence , vol. 44, no. 01, pp. 154–180,...

    2022

  3. [11]

    B. A. Wandell, Foundations of vision. Sinauer Associates, 1995

    1995

  4. [12]

    Discharge patterns and functional organization of mammalian retina,

    S. W. Kuffler, “Discharge patterns and functional organization of mammalian retina,” Journal of neurophysiology , vol. 16, no. 1, pp. 37–68, 1953

    1953

  5. [13]

    Information processing in the primate retina: Circuitry and coding,

    G. Field and E. Chichilnisky, “Information processing in the primate retina: Circuitry and coding,” Annual Review of Neuroscience, vol. 30, no. 1, pp. 1–30, 2007, pMID: 17335403. [Online]. Available: https://doi.org/10.1146/annurev.neuro.30. 051606.094252

    2007 arXiv

  6. [14]

    Understanding the retina: A review of computational models of the retina from the single cell to the network level,

    T. Guo, D. Tsai, S. Bai, J. Morley, G. Suaning, N. Lovell, and S. Dokos, “Understanding the retina: A review of computational models of the retina from the single cell to the network level,” Critical reviews in biomedical engineering, vol. 42, pp. 419–36, 01 2014

    2014

  7. [15]

    The retina dissects the visual scene into distinct features,

    B. Roska and M. Meister, “The retina dissects the visual scene into distinct features,” in The New Visual Neurosciences (Werner, JS, Chalupa, LM, eds), M. Press., Ed., 2014, pp. 163–182

    2014

  8. [16]

    A simple white noise analysis of neuronal light responses,

    E. Chichilnisky, “A simple white noise analysis of neuronal light responses,” in Network (Bristol, England), vol. 12, 2001, pp. 199–213

    2001

  9. [17]

    Reconstruction of natural images from responses of primate retinal ganglion cells,

    N. Brackbill, C. Rhoades, A. Kling, N. P . Shah, A. Sher, A. M. Litke, and E. Chichilnisky, “Reconstruction of natural images from responses of primate retinal ganglion cells,” eLife, vol. 9, p. e58516, nov 2020. [Online]. Available: https: //doi.org/10.7554/eLife.58516

    2020 doi

  10. [18]

    A quantitative description of membrane current and its application to conduction and excitation in nerve,

    A. Hodgkin and A. Huxley, “A quantitative description of membrane current and its application to conduction and excitation in nerve,” Bulletin of Mathematical Biology, vol. 52, no. 1, pp. 25–71, 1990. [Online]. Available: https://www.sciencedirect. com/science/article/pii/S009...

    1990

  11. [19]

    Physiological Gain Leads to High ISI Variability in a Simple Model of a Cortical Regular Spiking Cell,

    T. W. Troyer and K. D. Miller, “Physiological Gain Leads to High ISI Variability in a Simple Model of a Cortical Regular Spiking Cell,” Neural Computation , vol. 9, no. 5, pp. 971–983, 07 1997. [Online]. Available: https://doi.org/10.1162/neco.1997.9.5.971

    1997 doi

  12. [20]

    The spike response model: A framework to predict neuronal spike trains,

    R. Jolivet, T. J., and W. Gerstner, “The spike response model: A framework to predict neuronal spike trains,” in Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003, O. Kaynak, E. Alpaydin, E. Oja, and L. Xu, Eds. Berlin, Heidel- berg: Springer Ber...

    2003

  13. [21]

    Prediction and decoding of retinal ganglion cell responses with a probabilistic spiking model,

    J. W. Pillow, L. Paninski, V . J. Uzzell, E. P . Simoncelli, and E. J. Chichilnisky, “Prediction and decoding of retinal ganglion cell responses with a probabilistic spiking model,” Journal of Neuroscience, vol. 25, no. 47, pp. 11 003–11 013, 2005. [Online]. Available: https:/...

    2005

  14. [22]

    Deep learning models of the retinal response to natural scenes,

    L. T. McIntosh, N. Maheswaranathan, A. Nayebi, S. Ganguli, and S. A. Baccus, “Deep learning models of the retinal response to natural scenes,” in Proceedings of the 30th International Conference on Neural Information Processing Systems, ser. NIPS’16. Red Hook, NY, USA: Curran ...

    2016

  15. [23]

    Nonlinear decoding of a complex movie from the mammalian retina,

    V . Botella-Soler, S. Deny, G. Martius, O. Marre, and G. Tka ˇcik, “Nonlinear decoding of a complex movie from the mammalian retina,” PLOS Computational Biology , vol. 14, no. 5, pp. 1–27, 05

  16. [24]

    Neural networks for efficient bayesian decoding of natural images from retinal neurons,

    N. Parthasarathy, E. Batty, W. Falcon, T. Rutten, M. Rajpal, E. Chichilnisky, and L. Paninski, “Neural networks for efficient bayesian decoding of natural images from retinal neurons,” in Advances in Neural Information Processing Systems , I. Guyon, U. V . Luxburg, S. Bengio, ...

    2017

  17. [25]

    Mahowald, VLSI analogs of neuronal visual processing: a synthesis of form and function, Jul 2008

    M. Mahowald, VLSI analogs of neuronal visual processing: a synthesis of form and function, Jul 2008

    2008

  18. [26]

    A 128× 128 120 db 15 µs latency asynchronous temporal contrast vision sensor,

    P . Lichtsteiner, C. Posch, and T. Delbruck, “A 128× 128 120 db 15 µs latency asynchronous temporal contrast vision sensor,” IEEE Journal of Solid-State Circuits, vol. 43, no. 2, pp. 566–576, 2008

    2008

  19. [27]

    Retinomorphic event-based vision sensors: Bioinspired cameras with spiking output,

    C. Posch, T. Serrano-Gotarredona, B. Linares-Barranco, and T. Del- bruck, “Retinomorphic event-based vision sensors: Bioinspired cameras with spiking output,” Proceedings of the IEEE , vol. 102, no. 10, pp. 1470–1484, 2014

    2014

  20. [28]

    Neuromorophic vision sensing and pro- cessing,

    T. Delbruck, “Neuromorophic vision sensing and pro- cessing,” 2016 46th European Solid-State Device Research Conference (ESSDERC) , pp. 7–14, 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:44003126

    2016

  21. [29]

    A qvga 143 db dynamic range frame-free pwm image sensor with lossless pixel- level video compression and time-domain cds,

    C. Posch, D. Matolin, and R. Wohlgenannt, “A qvga 143 db dynamic range frame-free pwm image sensor with lossless pixel- level video compression and time-domain cds,” IEEE Journal of Solid-State Circuits, vol. 46, no. 1, pp. 259–275, 2011

    2011

  22. [30]

    A biomor- phic digital image sensor,

    E. Culurciello, R. Etienne-Cummings, and K. Boahen, “A biomor- phic digital image sensor,” IEEE Journal of Solid-State Circuits , vol. 38, no. 2, pp. 281–294, 2003

    2003

  23. [31]

    A 240x180 120db 10mw 12us-latency sparse output vision sensor for mobile applications,

    R. Berner, C. Brandli, M. Yang, S.-C. Liu, and T. Delbruck, “A 240x180 120db 10mw 12us-latency sparse output vision sensor for mobile applications,” 2013. [Online]. Available: https://api.semanticscholar.org/CorpusID:17870509

    2013

  24. [32]

    A 240 × 180 130 db 3 µs latency global shutter spatiotemporal vision sensor,

    C. Brandli, R. Berner, M. Yang, S.-C. Liu, and T. Delbruck, “A 240 × 180 130 db 3 µs latency global shutter spatiotemporal vision sensor,” IEEE Journal of Solid-State Circuits, vol. 49, no. 10, pp. 2333– 2341, 2014

    2014

  25. [33]

    Utility and feasibility of a center surround event camera,

    T. Delbruck, C. Li, R. Graca, and B. Mcreynolds, “Utility and feasibility of a center surround event camera,” Feb. 2022. [Online]. Available: http://arxiv.org/abs/2202.13076

    2022 arXiv

  26. [34]

    Two-stream vision sensors,

    C. Li, “Two-stream vision sensors,” Jun. 2017. [Online]. Available: https://doi.org/10.3929/ethz-b-000164862

    2017 doi

  27. [35]

    Sensing diversity and sparsity models for event generation and video reconstruction from events,

    S. Liu and P . L. Dragotti, “Sensing diversity and sparsity models for event generation and video reconstruction from events,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 10, pp. 12 444–12 458, 2023

    2023

  28. [36]

    High dynamic range imaging: spatially varying pixel exposures,

    S. K. Nayar and T. Mitsunaga, “High dynamic range imaging: spatially varying pixel exposures,” Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662), vol. 1, pp. 472–479 vol.1, 2000. [Online]. Available: https://api.semanticscholar...

    2000

  29. [37]

    Neural sensors: Learning pixel exposures for hdr imaging and video compressive sensing with programmable sensors,

    J. N. P . Martel, L. K. M¨uller, S. J. Carey, P . Dudek, and G. Wetzstein, “Neural sensors: Learning pixel exposures for hdr imaging and video compressive sensing with programmable sensors,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 42, no. 7, pp. 1...

    2020

  30. [38]

    Learning spatially varying pixel exposures for motion deblurring,

    C. M. Nguyen, J. N. Martel, and G. Wetzstein, “Learning spatially varying pixel exposures for motion deblurring,” IEEE International Conference on Computational Photography (ICCP), 2022

    2022

  31. [39]

    Generalized event cameras,

    V . Sundar, M. Dutson, A. Ardelean, C. Bruschini, E. Charbon, and M. Gupta, “Generalized event cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024. 10

    2024

  32. [41]

    The cnn vs. snn event-camera dichotomy and perspectives for event-graph neural networks,

    T. Dalgaty, T. Mesquida, D. Joubert, A. Sironi, C. Soubeyrat, P . Vivet, and C. Posch, “The cnn vs. snn event-camera dichotomy and perspectives for event-graph neural networks,” in 2023 De- sign, Automation & Test in Europe Conference & Exhibition (DATE) , 2023, pp. 1–6

    2023

  33. [42]

    Spikformer: When spiking neural network meets transformer,

    Z. Zhou, Y. Zhu, C. He, Y. Wang, S. YAN, Y. Tian, and L. Yuan, “Spikformer: When spiking neural network meets transformer,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https: //openreview.net/forum?id=frE4fUwz h

    2023

  34. [43]

    Deep directly-trained spiking neural networks for object detection,

    Q. Su, Y. Chou, Y. Hu, J. Li, S. Mei, Z. Zhang, and G. Li, “Deep directly-trained spiking neural networks for object detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 6555–6565

    2023

  35. [44]

    Event- based video reconstruction via potential-assisted spiking neural network,

    L. Zhu, X. Wang, Y. Chang, J. Li, T. Huang, and Y. Tian, “Event- based video reconstruction via potential-assisted spiking neural network,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 3584–3594

    2022

  36. [45]

    Tactilesgnet: A spiking graph neural network for event-based tactile object recognition,

    F. Gu, W. Sng, T. Taunyazov, and H. Soh, “Tactilesgnet: A spiking graph neural network for event-based tactile object recognition,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020

    2020

  37. [46]

    A low power, fully event-based gesture recognition system,

    A. Amir, B. Taba, D. Berg, T. Melano, J. McKinstry, C. Di Nolfo, T. Nayak, A. Andreopoulos, G. Garreau, M. Mendoza, J. Kusnitz, M. Debole, S. Esser, T. Delbruck, M. Flickner, and D. Modha, “A low power, fully event-based gesture recognition system,” in 2017 IEEE Conference on ...

    2017

  38. [47]

    Human gesture recognition for robot partners by spiking neural network and classification learning,

    J. Botzheim, T. Obo, and N. Kubota, “Human gesture recognition for robot partners by spiking neural network and classification learning,” in The 6th International Conference on Soft Computing and Intelligent Systems, and The 13th International Symposium on Advanced Intelligenc...

    2012

  39. [48]

    Deep learning for event-based vision: A comprehensive survey and benchmarks,

    X. Zheng, Y.-P . Liu, Y. Lu, T. Hua, T. Pan, W. Zhang, D. Tao, and L. Wang, “Deep learning for event-based vision: A comprehensive survey and benchmarks,” ArXiv, vol. abs/2302.08890, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID: 257019827

    2023 arXiv

  40. [49]

    High speed and high dynamic range video with an event camera,

    H. Rebecq, R. Ranftl, V . Koltun, and D. Scaramuzza, “High speed and high dynamic range video with an event camera,” IEEE Trans. Pattern Anal. Mach. Intell. (T-P AMI) , 2019. [Online]. Available: http://rpg.ifi.uzh.ch/docs/TPAMI19 Rebecq.pdf

    2019

  41. [50]

    Event-based vision meets deep learning on steering prediction for self-driving cars,

    A. M. I. Maqueda, A. Loquercio, G. Gallego, N. Garc ´ıa, and D. Scaramuzza, “Event-based vision meets deep learning on steering prediction for self-driving cars,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 5419– 5427, 2018. [Online]. Available: h...

    2018

  42. [51]

    Ev-flownet: Self- supervised optical flow estimation for event-based cameras,

    A. Zhu, L. Yuan, K. Chaney, and K. Daniilidis, “Ev-flownet: Self- supervised optical flow estimation for event-based cameras,” in Proceedings of Robotics: Science and Systems , Pittsburgh, Pennsylva- nia, June 2018

    2018

  43. [52]

    Amae: Adaptive motion-agnostic encoder for event-based object classification,

    Y. Deng, Y. Li, and H. Chen, “Amae: Adaptive motion-agnostic encoder for event-based object classification,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4596–4603, 2020

    2020

  44. [53]

    v2e: From video frames to realistic DVS events,

    Y. Hu, S. C. Liu, and T. Delbruck, “v2e: From video frames to realistic DVS events,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . IEEE, 2021. [Online]. Available: http://arxiv.org/abs/2006.07722

    2021 arXiv

  45. [54]

    Event-based frame interpolation with ad-hoc deblurring,

    L. Sun, C. Sakaridis, J. Liang, P . Sun, J. Cao, K. Zhang, Q. Jiang, K. Wang, and L. Van Gool, “Event-based frame interpolation with ad-hoc deblurring,” arXiv preprint arXiv:2301.05191, 2023

    2023 arXiv

  46. [55]

    Lightweight event-based optical flow estimation via iterative deblurring,

    Y. Wu, F. Paredes-Vall ´es, and G. C. H. E. de Croon, “Lightweight event-based optical flow estimation via iterative deblurring,” in Proceedings of IEEE International Conference on Robotics and Automa- tion (ICRA’24), May 2024, to Appear

    2024

  47. [56]

    Estimating or propagating gradients through stochastic neurons for conditional computation,

    Y. Bengio, N. L ´eonard, and A. C. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,” ArXiv, vol. abs/1308.3432, 2013. [Online]. Available: https://api.semanticscholar.org/CorpusID: 18406556

    2013 arXiv

  48. [57]

    Deep multi-scale convolutional neural network for dynamic scene deblurring,

    S. Nah, T. H. Kim, and K. M. Lee, “Deep multi-scale convolutional neural network for dynamic scene deblurring,” in CVPR, July 2017

    2017

  49. [58]

    Tartanair: A dataset to push the limits of visual slam,

    W. Wang, D. Zhu, X. Wang, Y. Hu, Y. Qiu, C. Wang, Y. Hu, A. Kapoor, and S. Scherer, “Tartanair: A dataset to push the limits of visual slam,” 2020

    2020

  50. [59]

    Event camera data pre- training,

    Y. Yang, L. Pan, and L. Liu, “Event camera data pre- training,” CoRR, vol. abs/2301.01928, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2301.01928

    2023 doi arXiv

  51. [60]

    Extracting motion and appearance via inter-frame attention for efficient video frame interpolation,

    G. Zhang, Y. Zhu, H. Wang, Y. Chen, G. Wu, and L. Wang, “Extracting motion and appearance via inter-frame attention for efficient video frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 5682–5692

    2023

  52. [61]

    A 100,000 fps vision sensor with embedded 535gops/w 256×256 simd processor array,

    S. J. Carey, A. Lopich, D. R. Barr, B. Wang, and P . Dudek, “A 100,000 fps vision sensor with embedded 535gops/w 256×256 simd processor array,” in 2013 Symposium on VLSI Circuits , 2013, pp. C182–C183

    2013

  53. [62]

    Mantissacam: Learning snapshot high-dynamic-range imaging with perceptually-based in-pixel irradiance encoding,

    H. M. So, J. P . Martel, G. Wetzstein, and P . Dudek, “Mantissacam: Learning snapshot high-dynamic-range imaging with perceptually-based in-pixel irradiance encoding,” in 2022 IEEE International Conference on Computational Photography (ICCP) . Los Alamitos, CA, USA: IEEE Compu...

    2022

  54. [63]

    Pixelrnn: In-pixel recurrent neural networks for end-to-end-optimized perception with neural sensors,

    H. M. So, L. Bose, P . Dudek, and G. Wetzstein, “Pixelrnn: In-pixel recurrent neural networks for end-to-end-optimized perception with neural sensors,” in CVPR, June 2024, pp. 25 233–25 244

    2024

  55. [64]

    A camera that cnns: Towards embedded neural networks on pixel processor arrays,

    L. Bose, P . Dudek, J. Chen, S. Carey, and W. Mayol-Cuevas, “A camera that cnns: Towards embedded neural networks on pixel processor arrays,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Los Alamitos, CA, USA: IEEE Computer Society, nov 2019, pp. 1335–...

    2019

  56. [65]

    Fully embedding fast convolutional networks on pixel processor arrays,

    L. Bose, P . Dudek, J. Chen, S. J. Carey, and W. W. Mayol-Cuevas, “Fully embedding fast convolutional networks on pixel processor arrays,” in Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX . Berlin, Heidelberg: Sp...

    2020 doi

  57. [66]

    High-speed light-weight cnn inference via strided convolutions on a pixel processor array,

    Y. Liu, L. Bose, J. Chen, S. J. Carey, P . Dudek, and W. Mayol-Cuevas, “High-speed light-weight cnn inference via strided convolutions on a pixel processor array,” in British Machine Vision Conference , 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:221671340

    2020

  58. [67]

    Sensing diversity and sparsity models for event generation and video reconstruction from events,

    S. Liu and P . L. Dragotti, “Sensing diversity and sparsity models for event generation and video reconstruction from events,” IEEE Transactions on Pattern Analysis and Machine Intelligence , pp. 1–16, 2023

    2023

  59. [68]

    Event camera data dense pre-training,

    Y. Yang, L. Pan, and L. Liu, “Event camera data dense pre-training,” 2024. [Online]. Available: https://arxiv.org/abs/ 2311.11533 11 NEURAL GANGLION SENSORS : SUPPLEMENTAL MATERIAL S1 A DDITIONAL EXPERIMENTAL DETAILS S1.1 Closed Form Binning, with refractory period In the main...

    2024 arXiv

  60. [2018]

    Available: https://doi.org/10.1371/journal.pcbi

    [Online]. Available: https://doi.org/10.1371/journal.pcbi. 1006057

    doi

  61. [2022]

    Available: https://arxiv.org/abs/2211.00509

    [Online]. Available: https://arxiv.org/abs/2211.00509

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.