Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets through Strongly Scattering Media

Arto Nurmikko; Ning Zhang; Timothy Shea

REVIEW 4 major objections 4 minor 94 references

Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets through Strongly Scattering Media

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An end-to-end neuromorphic pipeline—event camera plus deep spiking neural network—can track and reconstruct randomly moving hidden objects through strongly scattering media, achieving image similarity above 0.95 in transmission experiments.

desk verdict A genuinely integrated DVS-plus-SNN demonstration through calibrated dense phantoms, but the empirical evidence is thinner than the claims — track metrics, error bars, and a baseline are missing. read the letter →

arxiv 2501.03874 v2 pith:PFAI4OEM submitted 2025-01-07 cs.NE cs.CVcs.LGeess.IV

Ning Zhang , Timothy Shea , Arto Nurmikko This is my paper · ORCID

classification cs.NEcs.CVcs.LGeess.IV

keywords neuromorphicimagingeventcameradynamicvisionsensorspikingneuralnetworkscatteringmediaopticaltrackingimagereconstructionturbid

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that a fully neuromorphic imaging pipeline—an event camera feeding a deep spiking neural network—can track and reconstruct visually hidden moving objects through strongly scattering media. The authors argue that the event camera acts as a hardware attention filter: it emits spikes only when light changes, so a moving target creates a signal while the static scattering background stays silent. They combine that sparse spike stream with a two-module SNN that simultaneously estimates the target's coordinates and reconstructs its image, using temporal memory to converge on a sharp picture over time. In benchtop experiments the system tracks randomly moving handwritten digits through a phantom with 72 scattering mean free paths (SSIM 0.9577) and reconstructs time-varying Kanji characters in a 144-MFP reflection geometry (SSIM 0.8068). If correct, this offers a path to real-time, low-power imaging in fog, tissue, and other turbid settings where conventional cameras fail.

What carries the argument

The load-bearing mechanism is the event camera's change-detection front end plus the deep SNN's temporal memory. A dynamic vision sensor (DVS) fires asynchronous per-pixel spikes only when log-intensity crosses a threshold, so a moving target in a static scattering medium produces events while the diffuse background produces almost none; this is the hardware-level attention that isolates the target. The SNN then processes those spikes through leaky integrate-and-fire neurons with trainable membrane decay, stateful synapse filters, and a last-layer reset in the tracking module, which lets the network accumulate evidence over time and produce both coordinates and images.

What would settle it

A decisive test is to make the phantom itself fluctuate—slowly stirring the scattering particles or vibrating the slab while the target moves—and then measure whether the DVS event stream still encodes the target and whether tracking MSE and reconstruction SSIM hold; if they collapse, the static-medium assumption is load-bearing. A complementary test replaces the DMD-projected 2D character with a solid 3D object moving in depth and checks whether a recognizable reconstruction still emerges.

Watch

Extended reading notes

Core claim

The central claim is that combining event-driven sensing with a deep spiking neural network yields simultaneous tracking and image reconstruction of targets that are optically invisible through dense scattering media. Photons that have traversed a turbid phantom are detected by a dynamic vision sensor, which converts intensity changes into asynchronous positive and negative spikes; preprocessing turns these into spiking tensors of shape [time, polarity, height, width]. A tracking module (convolutional SNN encoder plus linear mapping) outputs normalized [x,y] center coordinates, while a reconstruction module (spiking U-Net encoder-decoder with skip connections, followed by a residual refinement module) outputs a probability map of the target; both run in parallel over time steps. The network is trained end-to-end with a surrogate gradient and a hybrid loss combining BCE, SSIM, and IoU for reconstruction and MSE for tracking. On the test set, transmission experiments with random MNIST digit motion at 72 mean free paths reach SSIM 0.9577 and MSE 0.0108, and reflection experiments with Kanji characters at 144 mean free paths reach SSIM 0.8068 and MSE 0.0583. The authors also estimate the SNN consumes about 20x less energy than an architecture-matched ANN (7.99 mJ vs 142.32 mJ).

Load-bearing premise

The load-bearing premise is that the scattering medium stays static while the target is the only dynamic light source; if the medium fluctuates or background motion creates comparable events, the event camera's attention mechanism no longer isolates target information and the learned spiking network loses its input signal.

Editorial extensions

If this is right

In transmission through a phantom with 72 scattering mean free paths, the system tracks random x-y trajectories and reconstructs the moving character with SSIM 0.9577, so the same pipeline should apply to other optically hidden moving targets such as vehicles in fog.
In reflection with 144 mean free paths, spatially fixed but optically time-varying characters are reconstructed with SSIM 0.8068, indicating the method works when the illuminating light is incoherent and the detector sees double-pass scattering.
The measured energy estimate for the SNN engine is about 20 times lower than an equivalent ANN (7.99 mJ vs 142.32 mJ), which makes real-time, battery-limited field deployment plausible.
Because spikes are the sole currency of information from sensor to output, the approach is compatible with emerging neuromorphic hardware and with wireless transmission of sparse events.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the static-medium condition is relaxed, a slowly varying phantom would inject its own events into the DVS; the attention mechanism would then need a motion prior or contrast filter to separate target from medium dynamics, which the paper does not address.
The DMD-projected characters are planar and pre-specified; a real solid object moving in three dimensions would add occlusion, perspective, and depth motion that the current two-module network has not been shown to handle.
The energy advantage is tied to spike sparsity; in scenes with dense, continuous background dynamics the AC/MAC ratio and the 20x figure would shrink, so the comparison should be re-measured at higher background event rates.
A natural extension is to feed the tracking module's coordinate output back into the reconstruction module as a positional prior, which could sharpen reconstructions at early time steps where the current system is blurry.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this one before reading: the paper is a real benchtop demonstration, not vaporware, and the system-level integration is new. The authors combine a DVS event camera with a two-module spiking neural network (tracking + reconstruction) and show results through silicone phantoms with MFP 72 (transmission) and 144 (reflection). That specific combination is not in the prior art they cite, though their own refs 42–44 already did DVS-plus-SNN in turbid media, so it is an extension of an established research program rather than a leap. They also give held-out test metrics (SSIM 0.9577 and 0.8068), which is real evidence.

What the paper does well: the optical engineering is careful — calibrated phantoms, MFP quantified, DMD and E-ink proxies acknowledged as such, and the static-medium assumption stated plainly in the introduction. The incremental nature of the architecture (U-Net, BASNet-style refinement) is not hidden. The energy comparison is clearly an estimate from spike rates and standard per-op costs, not a measured result, and they say so.

The soft spots are mostly about evidence strength, not method. Tracking accuracy is never quantified — the OTM outputs coordinates, but there is no error metric, no plot of tracked vs. true trajectories with numbers. The 5 training iterations are mentioned but no variance or error bars are reported. There is no baseline comparison against a frame-based camera with an ANN, or against a non-spiking reconstruction, which weakens the "fully neuromorphic" efficiency claim beyond the paper estimate. The "fully neuromorphic" label also overstates what was run: the SNN is simulated on an RTX 3090, not on Loihi or equivalent hardware.

The stress-test concern about the static-medium assumption is valid but not fatal. The paper explicitly says the medium is static and "could in principle be slowly varying," so this is a scope limitation, not a hidden flaw. The claim "through strongly scattering media" should be read as "through static strongly scattering media." Generalization to fog, tissue, or turbid water with dynamic optical density is unproven.

Who should read it: people working in computational imaging through turbid media or event-driven vision who want a concrete integration example. It deserves a serious referee: the experimental setup is solid, the limitation statements are honest, and the missing pieces (tracking metric, error bars, baseline) are additions, not fundamental corrections. I would send it to peer review with a request for those additions, plus code and data — the GitHub link is still a promise.

Referee Report

4 major / 4 minor

Summary. The manuscript presents an end-to-end neuromorphic optical imaging system that combines a dynamic vision sensor (DVS) with a multistage deep spiking neural network (SNN) to track and reconstruct randomly moving or time-varying targets hidden behind strongly scattering silicone phantoms. In transmission geometry, MNIST-like projected characters move randomly behind a phantom with MFP=72; in reflection geometry, Kanji-MNIST characters with time-varying contrast are imaged behind a phantom with MFP=144. The network has a tracking module (OTM) and a reconstruction module (ORM plus residual refinement), trained end-to-end with surrogate gradients and a hybrid loss. The reported reconstruction metrics are SSIM 0.9577 and MSE 0.0108 (transmission) and SSIM 0.8068 and MSE 0.0583 (reflection), and the energy estimate claims up to 20x lower power than an equivalent ANN. The paper frames the DVS as a hardware attention mechanism that suppresses the static scattering background, with the SNN performing tracking and reconstruction in parallel.

Significance. If the empirical claims are fully supported, the work introduces a genuinely new combination of hardware and algorithmic ideas: event-based sensing as a physical attention layer for scattering media, paired with a spiking reconstruction/tracking network. The benchtop experiments use quantitatively characterized phantoms (MFP=72 and 144), standardized character sets, held-out test samples, and a surrogate-gradient training pipeline. These are concrete and reproducible elements that go beyond a purely conceptual proposal. The energy comparison with an ANN of identical architecture is a useful first-order estimate. However, the current evidence is incomplete in several load-bearing respects: tracking accuracy is never quantified, reported metrics lack variance across the five training iterations, the static-medium assumption is untested, and no baseline comparison is provided. The central idea is promising and largely defensible, but the manuscript as written overstates its validated scope.

major comments (4)

[Object Tracking Module / Figure 4(b)] The central claim includes tracking, yet tracking accuracy is never quantitatively evaluated. Figure 4(b) shows inferred positions as green crosses versus ground truth as red crosses, but no metric such as center-coordinate root-mean-square error, mean absolute error, or per-time-step error is reported anywhere in the text. Without a quantitative tracking metric on the held-out test set, the tracking claim is supported only by a single visual example. Please add numeric tracking errors over all test trajectories, ideally with per-time-step distributions.
[Methods, Training and Evaluation / Figures 5 and 6] The paper states that five training iterations were conducted with different initializations, but all reported reconstruction metrics are single point estimates (SSIM 0.9577, MSE 0.0108; SSIM 0.8068, MSE 0.0583). With only five runs, the standard deviation is easy to compute and is needed to assess whether the differences between geometries and configurations are meaningful. Please report mean and standard deviation (or individual values) for all headline metrics across the five iterations.
[Introduction (near Fig. 1) and Methods, Phantom Preparation] The method's core hardware-attention mechanism relies on the scattering medium being static: the DVS emits events only for temporal intensity changes, so if the phantom itself had fluctuating optical density, motion, or decorrelation, those events would be indistinguishable from target events at the sensor. The manuscript explicitly states 'The turbid medium is taken as static though could in principle be slowly varying in its optical density,' and all benchtop experiments use a fixed silicone phantom. Yet the abstract and discussion generalize to fog, tissue, smoke, and other strongly scattering media. This assumption is load-bearing: either add experiments with a deliberately perturbed or slowly fluctuating phantom to test robustness, or restrict the claims in the abstract and title to static scattering media.
[Discussion / Results comparison] No baseline comparison is provided to calibrate the reported reconstruction and tracking results. The paper does not compare against a frame-based camera with a conventional CNN, against the same DVS data processed by a non-spiking network, or against existing speckle-correlation methods for tracking through scattering media. The energy comparison is only between the SNN and an ANN of the same architecture, which is informative but does not address whether the full neuromorphic pipeline outperforms simpler alternatives in accuracy. Please add at least one appropriate baseline on the same benchtop data, or temper the comparative claims of advantage in the Introduction and Discussion.

minor comments (4)

[Methods, Eq. (12)] In Eq. (12), the third loss term is written with weight 'a' (BCE weight) rather than a distinct weight, and the preceding paragraph says the loss comprises MSE and SSIM, while the equation uses BCE, SSIM, and IoU. Please correct the typo and align the text with the equation.
[Figure 4 caption and Results text] The Fig. 4 caption reports SSIM 0.9568 and MSE 0.0108 for the single digit '4', while the main text reports SSIM 0.9577 across the full testing dataset; please clarify which number corresponds to which evaluation set so the reader can distinguish the example from the aggregate result.
[Various locations] Several typos and formatting errors appear: 'grpup' in the Fig. 4 caption, 'repectedly' after the SSIM/MSE values, 'Kanji MINST' in the Results section, and 'CONVERGENC' in the section heading. These should be corrected.
[Data Availability] The data availability statement says data 'will be available in a provided GitHub repository' but no repository link is given in the manuscript; please include the actual link or a clear statement of how to obtain the code and data.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the demonstration is a supervised-learning experiment evaluated on held-out data, and no prediction reduces to a fitted input or self-citation chain.

full rationale

The paper makes no first-principles derivation whose output is equivalent to its input. The tracking module is trained with MSE against ground-truth DMD/E-ink target positions and evaluated on held-out MNIST/K-MNIST test objects; the random trajectories are generated independently for training and testing, so the reported tracking is a genuine held-out regression result rather than a fit renamed as prediction. Image reconstruction is trained against ground-truth characters and measured with SSIM/MSE on the same held-out set, again a conventional supervised evaluation. The energy comparison (Table I) is an estimate from measured spike rates and standard per-operation costs (Horowitz 2014) applied equally to the SNN and an architecture-matched ANN; it is not forced by construction. The paper's self-citations (refs 42-44) are related prior DVS-plus-SNN works but are not invoked as load-bearing justification for the architecture or results; no uniqueness theorem or ansatz is imported from them. The stated assumption of a static turbid medium is an acknowledged limitation of generality, not a circular step: the experiments genuinely pass light through MFP=72 and MFP=144 phantoms and the DVS responses are measured, not defined to match the output. Some novelty claims about prior work in transparent media are arguable given the authors' own earlier turbid-media papers, but that is a positioning concern, not circularity: no equation is defined in terms of the claimed result, and no fitted parameter is relabeled as a prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central contribution is an empirical deep-learning pipeline; the ledger captures learned weights and hand-chosen preprocessing choices rather than a derivation. The main load-bearing assumptions are the static-medium condition and the projection-based proxy for moving targets.

free parameters (3)

SNN synaptic weights = 3.69M parameters, AdamW, 50 epochs
Tracking and reconstruction outputs are entirely produced by supervised training on 10,000 or 25,000 labeled examples; there is no closed-form model of scattering or motion.
DVS contrast threshold, ROI, and temporal binning = 25% nominal contrast threshold; ROI selected manually; input tensors [18,2,64,64]
These hand-chosen preprocessing choices define what information the SNN receives and directly affect the reported SSIM and MSE values.
Hybrid loss weights a, b, c in Eq. 12 = not reported
The BCE, SSIM, and IoU combination requires weighting coefficients that are not specified; the balance is chosen by hand and affects reconstruction quality.

assumptions (4)

domain assumption The DVS event stream follows the fixed log-contrast threshold model of Eq. 11.
The entire input representation depends on this event-generation model; any deviation changes the spike statistics fed to the SNN.
ad hoc to paper The scattering medium is static and the target is the only dynamic source in the scene.
Stated in the Introduction: 'The turbid medium is taken as static though could in principle be slowly varying'; the DVS attention mechanism relies on temporal contrast being dominated by the target.
ad hoc to paper DMD-projected structured light and E-ink contrast changes are valid proxies for real physical moving or reflective targets.
The paper uses projected 2D characters and brightness changes instead of a solid object moving in 3D; generalization to vehicles or tissue is assumed, not demonstrated.
standard math LIF neuron dynamics and surrogate-gradient training faithfully account for neuromorphic computation.
These are standard SNN modeling choices; simulations run on GPU, not on neuromorphic hardware.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets through Strongly Scattering Media." pith.science (2026). https://pith.science/paper/PFAI4OEM

@misc{pith2026250103874,
  author       = {Pith},
  title        = {Pith review of: Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets through Strongly Scattering Media},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PFAI4OEM}},
  note         = {Machine review of arXiv:2501.03874}
}

read the original abstract

Tracking and acquiring simultaneous optical images of randomly moving targets obscured by scattering media remains a challenging problem of importance to many applications that require precise object localization and identification. In this work we develop an end-to-end neuromorphic optical engineering and computational approach to demonstrate how to track and image normally invisible objects by combining an event detecting camera with a multistage neuromorphic deep learning strategy. Photons emerging from dense scattering media are detected by the event camera and converted to pixel-wise asynchronized spike trains - a first step in isolating object-specific information from the dominant uninformative background. Spiking data is fed into a deep spiking neural network (SNN) engine where object tracking and image reconstruction are performed by two separate yet interconnected modules running in parallel in discrete time steps over the event duration. Through benchtop experiments we demonstrate tracking and imaging randomly moving objects in dense turbid media as well as image reconstruction of spatially stationary but optically dynamic objects. Standardized character sets serve as representative proxies for geometrically complex objects, underscoring the method's generality. The results highlight the advantages of a fully neuromorphic approach in meeting a major imaging technology with high computational efficiency and low power consumption.

Figures

Figures reproduced from arXiv: 2501.03874 by the authors.

**Figure 1.** High level view of the end-to-end neuromorphic optical imaging approach for tracking and reconstructing images of dynamic targets obscured by turbid media (here cartoon of a jumping frog). The direction of data flow reflects the integration of the benchtop event-driven optical subsystem in either transmission of reflection mode (left) with the outline of the computational architecture of the deep spiking neural netw… view at source ↗

**Figure 2.** Optical setup for tracking and reconstruction of dynamical objects through scattering media. (a) Transmission geometry used for experiments with MNIST characters as objects. Structured light was projected via a digital micromirror device (DMD) to represent randomly moving objects. (b) Example of random trajectories for objects in the transmission setup, showing unpredictable, random patterns of movement in the x-y p… view at source ↗

**Figure 3.** SNN Architecture for Dynamic Object Tracking and Reconstruction. The architecture is designed for event [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Summary of reconstruction results obtained in transmission geometry for randomly selected examples from MNIST, displayed as an 8×8 array where each image represents a distinct object. The figure compares the "ground truth" data, the raw spiking data recorded by the DVS…

**Figure 6.** Figure 6: Results of experiments conducted in reflection (bakscattering) geometry using Kanji characters as targets. Each cha [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 64 canonical work pages

[1]

Kijima, D. et al. Time-of-flight imaging in fog using multiple time-gated exposures. Opt. Express 29, 6453 (2021)

2021
[2]

Maccarone, A. et al. Submerged single-photon LiDAR imaging sensor used for real-time 3D scene reconstruction in scattering underwater environments. Opt. Express 31, 16690 (2023)

2023
[3]

Lyons, A. et al. Computational time-of-flight diffuse optical tomography. Nat. Photonics 13, 575–579 (2019)

2019
[4]

Zhao, Y. et al. High Resolution, Deep Imaging Using Confocal Time-of-Flight Diffuse Optical Tomography. IEEE Trans. Pattern Anal. Mach. Intell. 43, 2206–2219 (2021)

2021
[5]

& Liu, Y

Shi, Y., Sheng, W., Fu, Y. & Liu, Y. Overlapping speckle correlation algorithm for high-resolution imaging and tracking of objects in unknown scattering media. Nat. Commun. 14, 7742 (2023)

2023
[6]

Chen, W.-Y., O’Toole, M., Sankaranarayanan, A. C. & Levin, A. Enhancing speckle statistics for imaging inside scattering media. Optica 9, 1408–1416 (2022)

2022
[7]

& Bertolotti, J

Jauregui-Sánchez, Y., Penketh, H. & Bertolotti, J. Tracking moving objects through scattering media via speckle correlations. Nat. Commun. 13, 5779 (2022)

2022
[8]

& Zhang, Y

Ma, K., Wang, X., He, S., Zhang, X. & Zhang, Y. Learning to image and track moving objects through scattering media via speckle difference. Opt. Laser Technol. 159, 108925 (2023)

2023

Show all 94 references

[9]

Shi, T. et al. Computational imaging of moving objects obscured by a random corridor via speckle correlations. Nat. Commun. 13, 4081 (2022)

2022
[10]

Xu, Q. et al. Imaging and Tracking Through Scattering Medium With Low Bit Depth Speckle. IEEE Photonics J. 12, 1–7 (2020)

2020
[11]

Heide, F., Xiao, L., Kolb, A., Hullin, M. B. & Heidrich, W. Imaging in scattering media using correlation image sensors and sparse convolutional coding. Opt. Express 22, 26338–26350 (2014). 18

2014
[12]

& Wang, A

Rawat, S., Wendoloski, J. & Wang, A. Deep learning-assisted imaging through stationary scattering media. Opt. Express 30, 18145 (2022)

2022
[13]

McShane, E. P. et al. High resolution TCSPC imaging of diffuse light with a one-dimensional SPAD array scanning system. Opt. Express 30, 27926 (2022)

2022
[14]

Scholes, S. et al. Fundamental limits to depth imaging with single-photon detector array sensors. Sci. Rep. 13, 176 (2023)

2023
[15]

M., Burri, S

Bruschini, C., Homulle, H., Antolovic, I. M., Burri, S. & Charbon, E. Single-photon avalanche diode imagers in biophotonics: review and outlook. Light Sci. Appl. 8, 87 (2019)

2019
[16]

Lindell, D. B. & Wetzstein, G. Three-dimensional imaging through scattering media based on confocal diffuse tomography. Nat. Commun. 11, 4517 (2020)

2020
[17]

& Raskar, R

Satat, G., Tancik, M. & Raskar, R. Towards photography through realistic fog. in 2018 IEEE International Conference on Computational Photography (ICCP) 1–10 (IEEE, Pittsburgh, PA, 2018). doi:10.1109/ICCPHOT.2018.8368463

2018
[18]

Jiang, P.-Y. et al. Long range 3D imaging through atmospheric obscurants using array-based single-photon LiDAR. Opt. Express 31, 16054 (2023)

2023
[19]

& Katz, O

Weinberg, G., Sunray, E. & Katz, O. Noninvasive megapixel fluorescence microscopy through scattering layers by a virtual incoherent reflection matrix. Sci. Adv. 10, eadl5218 (2024)

2024
[20]

& Razansky, D

Zhou, Q., Chen, Z., Robin, J., Deán-Ben, X.-L. & Razansky, D. Diffuse optical localization imaging for noninvasive deep brain microangiography in the NIR-II window. Optica 8, 796 (2021)

2021
[21]

Sanjeev, A. et al. Non-Invasive Imaging Through Scattering Medium by Using a Reverse Response Wavefront Shaping Technique. Sci. Rep. 9, 12275 (2019)

2019
[22]

Feng, B. Y. et al. NeuWS: Neural wavefront shaping for guidestar-free imaging through static and dynamic scattering media. Sci. Adv. 9, eadg4671 (2023). 19

2023
[23]

& Hao, Q

Lin, L.-X., Cao, J., Zhou, D., Cui, H. & Hao, Q. Ghost imaging through scattering medium by utilizing scattered light. Opt. Express 30, 11243 (2022)

2022
[24]

Kang, S. et al. Tracing multiple scattering trajectories for deep optical imaging in scattering media. Nat. Commun. 14, 6871 (2023)

2023
[25]

Kanaev, A. V. et al. Imaging through extreme scattering in extended dynamic media. Opt. Lett. 43, 3088 (2018)

2018
[26]

K., Zhu, X., Adamo, G

Wang, D., Sahoo, S. K., Zhu, X., Adamo, G. & Dang, C. Non-invasive super-resolution imaging through dynamic scattering media. Nat. Commun. 12, 3150 (2021)

2021
[27]

Liu, H. et al. Learning-based real-time imaging through dynamic scattering media. Light Sci. Appl. 13, 194 (2024)

2024
[28]

& Delbruck, T

Lichtsteiner, P., Posch, C. & Delbruck, T. A 128\times 128 120 dB 15 μs Latency Asynchronous Temporal Contrast Vision Sensor. IEEE J. Solid-State Circuits 43, 566–576 (2008)

2008
[29]

Gallego, G. et al. Event-Based Vision: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 44, 154– 180 (2022)

2022
[30]

Guo, R. et al. EventLFM: event camera integrated Fourier light field microscopy for ultrafast 3D imaging. Light Sci. Appl. 13, 144 (2024)

2024
[31]

Muglikar, M. et al. Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging. Preprint at http://arxiv.org/abs/2404.11511 (2024)

2024 arXiv
[32]

& Mukhopadhyay, S

Chakraborty, B., She, X. & Mukhopadhyay, S. A Fully Spiking Hybrid Neural Network for Energy- Efficient Object Detection. IEEE Trans. Image Process. 30, 9014–9029 (2021)

2021
[33]

Liu, B.-C. et al. Spiking Neuron Networks based Energy-Efficient Object Detection for Mobile Robot. in 2021 China Automation Congress (CAC) 3224–3229 (IEEE, Beijing, China, 2021). doi:10.1109/CAC53003.2021.9727350. 20

2021
[34]

H., Makkiabadi, B

Amiri, M., Jafari, A. H., Makkiabadi, B. & Nazari, S. Recognizing intertwined patterns using a network of spiking pattern recognition platforms. Sci. Rep. 12, 19436 (2022)

2022
[35]

& Taylor, D

Kumarasinghe, K., Kasabov, N. & Taylor, D. Brain-inspired spiking neural networks for decoding and understanding muscle activity and kinematics from electroencephalography signals during hand movements. Sci. Rep. 11, 2486 (2021)

2021
[36]

Schuman, C. D. et al. Opportunities for neuromorphic computing algorithms and applications. Nat. Comput. Sci. 2, 10–19 (2022)

2022
[37]

& Panda, P

Roy, K., Jaiswal, A. & Panda, P. Towards spike-based machine intelligence with neuromorphic computing. Nature 575, 607–617 (2019)

2019
[38]

Mitchell, J. P. et al. NeoN: Neuromorphic control for autonomous robotic navigation. in 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS) 136–142 (2017). doi:10.1109/IRIS.2017.8250111

2017
[39]

Liao, J. et al. A Spiking Neural Network Decoder for Implantable Brain Machine Interfaces and its Sparsity-aware Deployment on RISC-V Microcontrollers. Preprint at http://arxiv.org/abs/2405.02146 (2024)

2024 arXiv
[40]

Yamazaki, K., Vo-Ho, V.-K., Bulsara, D. & Le, N. Spiking Neural Networks and Their Applications: A Review. Brain Sci. 12, 863 (2022)

2022
[41]

Wu, J., Wang, Y., Li, Z., Lu, L. & Li, Q. A Review of Computing with Spiking Neural Networks. Comput. Mater. Contin. 78, 2909–2939 (2024)

2024
[42]

& Nurmikko, A

Zhang, N., Shea, T. & Nurmikko, A. Event-Driven Imaging in Turbid Media: A Confluence of Optoelectronics and Neuromorphic Computation. Preprint at https://doi.org/10.48550/arXiv.2309.06652 (2023). 21

2023 doi arXiv
[43]

& Nurmikko, A

Zhang, N., Watts, J. & Nurmikko, A. Combining a Dynamic Vision Sensor with Neuromorphic Computing for Imaging in Turbid Media. in Computational Optical Sensing and Imaging CTu2B-4 (Optica Publishing Group, 2023)

2023
[44]

& Nurmikko, A

Zhang, N. & Nurmikko, A. Brain-inspired optical imaging: a neuromorphic computing approach for image reconstruction of dynamic targets obscured by dense turbid media. in Computational Optical Imaging and Artificial Intelligence in Biomedical Sciences vol. 12857 16–19 (SPIE, 2024)

2024
[45]

Debat, G. et al. Event-Based Trajectory Prediction Using Spiking Neural Networks. Front. Comput. Neurosci. 15, 658764 (2021)

2021
[46]

Ziegler, A. et al. Detection of Fast-Moving Objects with Neuromorphic Hardware. Preprint at http://arxiv.org/abs/2403.10677 (2024)

2024 arXiv
[47]

& Bennamoun, M

Barchid, S., Mennesson, J., Eshraghian, J., Djéraba, C. & Bennamoun, M. Spiking neural networks for frame-based and event-based single object localization. Neurocomputing 559, 126805 (2023)

2023
[48]

& Panda, P

Kim, Y. & Panda, P. Optimizing Deeper Spiking Neural Networks for Dynamic Vision Sensing. Neural Netw. 144, 686–698 (2021)

2021
[49]

R., Barranco, F

Cuadrado, J., Rançon, U., Cottereau, B. R., Barranco, F. & Masquelier, T. Optical flow estimation from event-based cameras and spiking neural networks. Front. Neurosci. 17, 1160034 (2023)

2023
[50]

& Roy, K

Roy, A., Nagaraj, M., Mihiranga Liyanagedera, C. & Roy, K. Live Demonstration: Real-time Event- based Speed Detection using Spiking Neural Networks. in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 4081–4082 (IEEE, Vancouver, BC, Canada,...

2023
[51]

& Benosman, R

Haessig, G., Berthelon, X., Ieng, S.-H. & Benosman, R. A Spiking Neural Network Model of Depth from Defocus for Event-based Neuromorphic Vision. Sci. Rep. 9, 3744 (2019)

2019
[52]

& Indiveri, G

Osswald, M., Ieng, S.-H., Benosman, R. & Indiveri, G. A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems. Sci. Rep. 7, 40703 (2017). 22

2017
[53]

& Huang, T

Dong, S., Bi, Z., Tian, Y. & Huang, T. Spike Coding for Dynamic Vision Sensor in Intelligent Driving. IEEE Internet Things J. 6, 60–71 (2019)

2019
[54]

& Di Caterina, G

Kirkland, P. & Di Caterina, G. Movement Classification and Segmentation Using Event-Based Sensing and Spiking Neural Networks. in 2022 Sensor Signal Processing for Defence Conference (SSPD) 1–5 (IEEE, London, United Kingdom, 2022). doi:10.1109/SSPD54131.2022.9896217

2022
[55]

Amir, A. et al. A Low Power, Fully Event-Based Gesture Recognition System. in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 7388–7397 (IEEE, Honolulu, HI, 2017). doi:10.1109/CVPR.2017.781

2017 doi
[56]

Lee, J. et al. An asynchronous wireless network for capturing event-driven data from large populations of autonomous sensors. Nat. Electron. 7, 313–324 (2024)

2024
[57]

Yan, Z. et al. Scattering imaging of moving targets based on recurrent neural networks: a fusion of temporal and phase information. Opt. Express 32, 45978–45994 (2024)

2024
[58]

& Situ, G

Lyu, M., Wang, H., Li, G., Zheng, S. & Situ, G. Learning-based lensless imaging through optically thick scattering media. Adv. Photonics 1, 036002 (2019)

2019
[59]

& Han, J

Shi, Y., Guo, E., Bai, L. & Han, J. Prior-free imaging unknown target through unknown scattering medium. Opt. Express 30, 17635 (2022)

2022
[60]

Zhao, Y. et al. Unrolled-DOT: an interpretable deep network for diffuse optical tomography. J. Biomed. Opt. 28, 036002 (2023)

2023
[61]

O., Mostafa, H

Neftci, E. O., Mostafa, H. & Zenke, F. Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks. IEEE Signal Process. Mag. 36, 51–63 (2019)

2019
[62]

Eshraghian, J. K. et al. Training Spiking Neural Networks Using Lessons From Deep Learning. Proc. IEEE 111, 1016–1054 (2023). 23

2023
[63]

& Liu, S.-C

Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M. & Liu, S.-C. Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification. Front. Neurosci. 11, 682 (2017)

2017
[64]

& Thorpe, S

Masquelier, T. & Thorpe, S. J. Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity. PLoS Comput. Biol. 3, e31 (2007)

2007
[65]

& Brox, T

Ronneberger, O., Fischer, P. & Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Preprint at http://arxiv.org/abs/1505.04597 (2015)

2015 arXiv
[66]

Qin, X. et al. BASNet: Boundary-Aware Salient Object Detection. in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 7471–7481 (IEEE, Long Beach, CA, USA, 2019). doi:10.1109/CVPR.2019.00766

2019
[67]

& Qiu, Q

Fang, H., Shrestha, A., Zhao, Z. & Qiu, Q. Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network. Preprint at http://arxiv.org/abs/2003.02944 (2020)

2020 arXiv
[68]

Xie, S. & Tu, Z. Holistically-Nested Edge Detection
[69]

P., Mannor, S

de Boer, P.-T., Kroese, D. P., Mannor, S. & Rubinstein, R. Y. A Tutorial on the Cross-Entropy Method. Ann. Oper. Res. 134, 19–67 (2005)

2005
[70]

Wang, Z., Simoncelli, E. P. & Bovik, A. C. Multiscale structural similarity for image quality assessment. in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 vol. 2 1398-1402 Vol.2 (2003)

2003
[71]

& Urtasun, R

Mattyus, G., Luo, W. & Urtasun, R. DeepRoadMapper: Extracting Road Topology from Aerial Images. in 2017 IEEE International Conference on Computer Vision (ICCV) 3458–3466 (IEEE, Venice, 2017). doi:10.1109/ICCV.2017.372

2017 doi
[72]

& Tian, Y

Chen, G., Peng, P., Li, G. & Tian, Y. Training Full Spike Neural Networks via Auxiliary Accumulation Pathway. Preprint at http://arxiv.org/abs/2301.11929 (2023). 24

2023 arXiv
[73]

Panda, P., Aketi, S. A. & Roy, K. Toward Scalable, Efficient, and Accurate Deep Spiking Neural Networks With Backward Residual Connections, Stochastic Softmax, and Hybridization. Front. Neurosci. 14, (2020)

2020
[74]

& Beerel, P

Kundu, S., Pedram, M. & Beerel, P. A. HIRE-SNN: Harnessing the Inherent Robustness of Energy- Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise. in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) 5189–5198 (IEEE, Montreal, QC, Canada,...

2021
[75]

Yao, M. et al. SPIKE-DRIVEN TRANSFORMER V2: META SPIKING NEURAL NETWORK ARCHITECTURE INSPIRING THE DESIGN OF NEXT-GENERATION NEUROMORPHIC CHIPS. (2024)

2024
[76]

1.1 Computing’s energy problem (and what we can do about it)

Horowitz, M. 1.1 Computing’s energy problem (and what we can do about it). in 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) 10–14 (2014). doi:10.1109/ISSCC.2014.6757323

2014
[77]

Muglikar, M. et al. Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging. Preprint at https://doi.org/10.48550/arXiv.2404.11511 (2024)

2024 doi arXiv
[78]

Zhang, N., Zhang, Q., Choquette, K. D. & Nurmikko, A. VCSEL Arrays as Chip Scale Sources for Ultra-High Density Diffuse Optical Tomography. in 2022 IEEE Photonics Conference (IPC) 1–2 (2022). doi:10.1109/IPC53466.2022.9975480

2022
[79]

& Nurmikko, A

Zhang, N., Zhang, Q. & Nurmikko, A. Sub-mm resolution tomographic imaging in turbid media by an ultra-high density multichannel approach. Biomed. Opt. Express 13, 5926–5936 (2022)

2022
[80]

Zhou, C. et al. Spikingformer: Spike-driven Residual Learning for Transformer-based Spiking Neural Network. Preprint at http://arxiv.org/abs/2304.11954 (2023)

2023
[81]

Yue, Y. et al. Spiking neural networks fine-tuning for brain image segmentation. Front. Neurosci. 17, 1267639 (2023). 25

2023
[82]

& Roy, K

Rathi, N. & Roy, K. DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks. Preprint at http://arxiv.org/abs/2008.03658 (2020)

2020 arXiv
[83]

Shi, X., Hao, Z. & Yu, Z. SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks. in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 5610–5619 (IEEE, Seattle, WA, USA, 2024). doi:10.1109/CVPR52733.2024.00536

2024
[84]

& Shi, L

Li, H., Liu, H., Ji, X., Li, G. & Shi, L. CIFAR10-DVS: An Event-Stream Dataset for Object Classification. Front. Neurosci. 11, 309 (2017)

2017
[85]

Orchard, G., Jayawant, A., Cohen, G. K. & Thakor, N. Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades. Front. Neurosci. 9, (2015)

2015
[86]

Neurophotonic tools for microscopic measurements and manipulation: status report. https://www.spiedigitallibrary.org/journals/neurophotonics/volume-9/issue- S1/013001/Neurophotonic-tools-for-microscopic-measurements-and-manipulation-status- report/10.1117/1.NPh.9.S1.013001.full

doi
[87]

Willomitzer, F. et al. Fast non-line-of-sight imaging with high-resolution and wide field of view using synthetic wavelength holography. Nat. Commun. 12, 6647 (2021)

2021
[88]

Orchard, G. et al. Efficient Neuromorphic Signal Processing with Loihi 2. Preprint at https://doi.org/10.48550/arXiv.2111.03746 (2021)

2021 doi arXiv
[89]

Insect-inspired Spatio-temporal Downsampling of Event-based Input

Ghosh, A. Insect-inspired Spatio-temporal Downsampling of Event-based Input. (2023)

2023
[90]

Sekar, S. K. V. et al. Solid phantom recipe for diffuse optics in biophotonics applications: a step towards anatomically correct 3D tissue phantoms. Biomed. Opt. Express 10, 2090 (2019)

2019
[91]

Naglič, P. et al. Optical properties of PlatSil SiliGlass tissue-mimicking phantoms. Biomed. Opt. Express 11, 3753 (2020)

2020
[92]

https://docs.prophesee.ai/stable/hw/manuals/biases.html. 26
[93]

Fang, W. et al. SpikingJelly: An open-source machine learning infrastructure platform for spike- based intelligence. Sci. Adv. 9, eadi1480 (2023)

2023
[94]

https://lava-nc.org/

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Kijima, D. et al. Time-of-flight imaging in fog using multiple time-gated exposures. Opt. Express 29, 6453 (2021)

2021

[2] [2]

Maccarone, A. et al. Submerged single-photon LiDAR imaging sensor used for real-time 3D scene reconstruction in scattering underwater environments. Opt. Express 31, 16690 (2023)

2023

[3] [3]

Lyons, A. et al. Computational time-of-flight diffuse optical tomography. Nat. Photonics 13, 575–579 (2019)

2019

[4] [4]

Zhao, Y. et al. High Resolution, Deep Imaging Using Confocal Time-of-Flight Diffuse Optical Tomography. IEEE Trans. Pattern Anal. Mach. Intell. 43, 2206–2219 (2021)

2021

[5] [5]

& Liu, Y

Shi, Y., Sheng, W., Fu, Y. & Liu, Y. Overlapping speckle correlation algorithm for high-resolution imaging and tracking of objects in unknown scattering media. Nat. Commun. 14, 7742 (2023)

2023

[6] [6]

Chen, W.-Y., O’Toole, M., Sankaranarayanan, A. C. & Levin, A. Enhancing speckle statistics for imaging inside scattering media. Optica 9, 1408–1416 (2022)

2022

[7] [7]

& Bertolotti, J

Jauregui-Sánchez, Y., Penketh, H. & Bertolotti, J. Tracking moving objects through scattering media via speckle correlations. Nat. Commun. 13, 5779 (2022)

2022

[8] [8]

& Zhang, Y

Ma, K., Wang, X., He, S., Zhang, X. & Zhang, Y. Learning to image and track moving objects through scattering media via speckle difference. Opt. Laser Technol. 159, 108925 (2023)

2023