Pith. sign in

REVIEW 3 major objections 5 minor 58 references

DELTA: Dense Depth from Events and LiDAR using Transformer's Attention

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DELTA uses self- and cross-attention to fuse event-camera and LiDAR data, cutting close-range depth error by up to four times.

desk verdict A credible attention-based fusion architecture with honest limitations, but the '4x better' claim hinges on comparing ALED only on its D_bf output, so the SOTA claim is broader than the evidence. read the letter →

arxiv 2505.02593 v1 pith:GO2SYSGL submitted 2025-05-05 cs.CV

Vincent Brebion , Julien Moreau , Franck Davoine This is my paper · ORCID
classification cs.CV
keywords eventcameraLiDARfusiondensedepthestimationtransformerattentioncross-attentionrecurrentmemoryautonomousdrivingfromevents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an attention-based network, DELTA, can fuse asynchronous event-camera data with sparse LiDAR depth to produce dense depth maps, and that this fusion strategy beats the previous state of the art, especially close to the sensor. On the synthetic SLED dataset, the mean depth error at the 10 m cutoff drops from 2.01 m to 0.54 m on the Town03 test sequence, roughly a fourfold reduction. The authors argue the gain comes from replacing convolutional fusion with self- and cross-attention, supported by two memory mechanisms: a propagation memory that updates stale LiDAR with newer events and a central recurrent memory that stabilizes predictions when the platform stops. If the claim holds, event-plus-LiDAR depth estimation becomes substantially more accurate for the close-range driving scenarios where pedestrians and obstacles matter most.

What carries the argument

The load-bearing machinery is attention with two memories. Event volumes and projected LiDAR point clouds are split into patches and self-attended separately; a central cross-attention module then fuses the two modalities; a propagation memory, queried by the previous LiDAR representation, updates the LiDAR features with the newest events even when no new LiDAR sweep has arrived; and a GRU maintains a central memory state that carries context across time. The propagation memory is what lets fusion happen at the event rate rather than the LiDAR rate, and the ablation study identifies the central cross-attention module as the component whose removal costs the most accuracy.

What would settle it

Run DELTA and the previous method on a dataset with dense, synchronized, error-free ground truth, or on MVSEC after correcting its known moving-object depth errors, and compare every output, including the previous method's after-event depth maps. If the 10 m cutoff mean error on SLED Town03 does not stay near 0.54 m versus 2.01 m, the central improvement claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that dense depth from events and LiDAR is better solved by attention-based fusion than by the convolutional fusion used in the prior state of the art. DELTA tokenizes event volumes and projected LiDAR into patches, applies self-attention within each modality and cross-attention between them, and maintains two memories: a small propagation memory that lets stale LiDAR be updated by newer events, and a gated recurrent unit (GRU) central memory that condenses the past and smooths the output. Evaluated on SLED, MVSEC, and M3ED, the paper reports a new state of the art, with the largest gains at close range, up to about four times lower mean depth error at the 10 m cutoff on SLED. The paper also revises its earlier two-depth-map formulation, arguing that only the before-event depth map per temporal window is needed because the after-event map can be read from the next window.

Load-bearing premise

The state-of-the-art claim rests on the evaluation protocol: if the imperfect ground truth in MVSEC, the redefined M3ED splits, or the choice to compare the previous method only on its before-event depth outputs unfairly favors DELTA, the fourfold close-range improvement is overstated.

Editorial extensions

If this is right

  • At the 10 m cutoff on the SLED Town03 sequence, mean depth error drops from 2.01 m with the previous best method to 0.54 m with DELTA, roughly a fourfold reduction, with smaller but consistent gains on Town01.
  • Because the propagation memory updates LiDAR features with each incoming event window, fusion can run at the event stream's temporal resolution instead of waiting for the next LiDAR sweep.
  • Ablations attribute the improvement to specific components: removing the central cross-attention hurts most, removing the propagation memory adds roughly 0.2–0.4 m of error, and removing the central memory adds up to about 0.3 m at close range.
  • DELTA remains competitive on real-world MVSEC and M3ED data despite acknowledged errors and sparsity in their ground truth, and pretraining on synthetic SLED transfers reasonably to real sensors.
  • On high-resolution 1280x720 input, DELTA runs at about 6.3 Hz with roughly 1.79 TFLOPs on a single GPU, with FLOPs and memory usage comparable to the previous convolutional method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the close-range gain should depend on LiDAR density; sparser LiDAR would likely shrink the gap between attention and convolutional fusion, a testable prediction the paper does not run.
  • Because DSEC was excluded on the grounds that its ground truth is timestamped to RGB frames rather than LiDAR, resynchronizing DSEC depth to LiDAR timestamps would provide a larger real-world benchmark for testing whether the SLED gains transfer.
  • The revision from two depth maps to one implies after-event depth can be recovered from the next window's before-event map, so a production version could halve the output head's cost without losing information.
  • The reported failure cases, such as event-volume saturation during fast turns and dark objects missed by the event camera, suggest the event representation, not the attention mechanism, is the next bottleneck; time-ordered or asynchronous event representations are a natural extension.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DELTA, a recurrent attention-based network that fuses projected LiDAR point clouds with temporal windows of event data to produce dense depth maps. The architecture uses self- and cross-attention in a U-Net-like structure, with a propagation memory for temporally propagating LiDAR data using events and a GRU-based central memory for temporal smoothing. The method is evaluated on three datasets (SLED, MVSEC, and a redefined subset of M3ED) and compared with prior event-and-LiDAR depth estimation methods. The main claims are that DELTA sets a new state of the art and that it reduces depth errors up to four times at close ranges compared to the previous state of the art, based on SLED Town03 results against ALED.

Significance. The core architectural idea is well motivated: attention-based cross-modal fusion with two memory mechanisms is a sensible and timely contribution, and the ablation study in Sec. 4.6 provides evidence that each component matters. The paper is also commendable for releasing source code and trained models, for performing ablations beyond the main comparison, and for reporting computational complexity in the supplementary material. If the reported gains hold under a fair evaluation protocol, the SLED results represent a substantial improvement in close-range depth estimation. However, the headline state-of-the-art claim and the factor-of-four improvement are conditional on a comparison protocol whose fairness is not fully established: the primary baseline is evaluated only on one of its two output depth maps, and no variance estimates are provided. The acknowledged ground-truth limitations of MVSEC and M3ED further temper the strength of the cross-dataset conclusions.

major comments (3)
  1. [Sec. 4.2, Tables 2 and 3] The paper states that 'for fairness of evaluation, comparisons with ALED [3] are only made on its D_bf depth maps,' but ALED was designed to output two depth maps per temporal window and to use event timestamps to assign each event its d_bf or d_af. The headline claim of a roughly four-fold error reduction at 10m on SLED Town03 (2.01m vs. 0.54m) is based entirely on this D_bf-only comparison. If ALED's intended event-level output is more accurate, then both the factor-of-four improvement and the 'new state of the art' claim are overstated. Please report ALED's full output (or its best available output) on the same SLED and MVSEC splits, or provide a principled argument for why D_bf-only is the correct comparison target.
  2. [Sec. 4.1, Tab. 1, Sec. 4.5] The M3ED dataset is subsampled and the train/val/test splits are redefined because LiDAR data is not provided for the original test set. The ALED results in Tab. 5 are stated to have been 'computed for this article,' which is appropriate, but the paper must confirm that ALED and DELTA were evaluated on the exact same redefined split and with the same subsampling and preprocessing. Without this confirmation, the M3ED comparison is not reproducible and the improvements reported there may be split-dependent. The exact split definition should be provided in the supplementary material.
  3. [Secs. 4.3-4.6, Tables 2-7] All quantitative results appear to come from a single training run. Several conclusions rely on small differences between methods, for example in Tab. 5 where DELTA_M3 and ALED_M3 differ by only 0.005 to 0.01 in the delta_1/delta_2 metrics, and in Tab. 6 where the central-memory ablation differs by 0.09m at the 200m cutoff. Without error bars, multiple seeds, or statistical significance tests, the robustness of these gains cannot be assessed. Please provide mean and standard deviation over at least three training seeds for the main tables, or explicitly justify why a single run is sufficient.
minor comments (5)
  1. [Sec. 4.3, Fig. 3] The text refers to the SLED test sequences as 'Town01' and 'Town03', while the figure captions use 'Town01_08' and 'Town03_19'; please be consistent.
  2. [Sec. 4.2] The patch size P and dimensionality D are given, but the relationship between the number of patches N and the input resolution is not stated. This matters for interpreting the propagation memory size of 128, which is much smaller than the typical number of patches for a 512x512 input. A short sentence explaining how N is computed would improve clarity.
  3. [Supplementary Material, Tab. 8] The table uses 'FLOPS' where the standard unit is 'FLOPs'; also, inference-time measurements should state whether they include GPU warm-up and which PyTorch version was used, to make the comparison with ALED more meaningful.
  4. [References] The bibliography entry for Cui et al. [10] contains stray page numbers after the year ('2022. 2, 5, 6, 7, 8, 1, 4'); these appear to be citation markers and should be removed.
  5. [Sec. 4.4] The paper acknowledges that MVSEC ground truth is 'erroneous when there are moving objects in the scene,' yet MVSEC is still used for quantitative evaluation. This is a known limitation of the dataset, but it would be helpful to state explicitly in the main text whether any filtering of erroneous ground-truth pixels was applied, and to discuss how this could affect the reported numbers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the SOTA claim rests on held-out empirical comparisons, not on a self-referential derivation.

full rationale

I traced the paper's claimed derivation chain. DELTA's central claims are empirical: the architecture combines event volumes and projected LiDAR with self-/cross-attention, propagation memory, and central memory, and the reported gains are measured on held-out test sequences (SLED, MVSEC, M3ED) against published baselines, chiefly the authors' earlier ALED. No equation or fitted constant defines the predicted depth error reduction in terms of the inputs; the 4x close-range figure is a table comparison (Tab. 2), not a construction. Self-citations to [3] provide the ALED baseline, a prior published model, and a loss formulation, but these are not load-bearing circular evidence: ALED's outputs are externally measured on the same test data, and the loss formulation is a standard multiscale gradient-matching loss. The D_bf-only comparison with ALED and the redefined M3ED splits are evaluation-protocol concerns that could affect the strength of the SOTA claim, but they are not reductions of the prediction to the model's own fitted values. The ablation studies further support the architectural attributions independently. I therefore find no circular step requiring a nonzero score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The main numerical choices are hyperparameters (window length, patch size, memory size, dimensionality, normalization range) rather than fitted physical constants. The modeling rests on assumptions about the sufficiency of event volumes, the adequacy of LiDAR projection, the well-posedness of single-map estimation, and the reliability of imperfect ground truth, all of which are acknowledged in the paper.

free parameters (5)
  • Temporal event window delta_t = 50 ms
    Chosen based on the rate of ground truth across datasets (Sec. 4.2). It controls how much event information is accumulated per input volume.
  • Patch size P = 16 for SLED and M3ED, 12 for MVSEC
    Chosen by resolution to capture details (Sec. 4.2). It affects the model's receptive field and the number of patches.
  • Propagation memory size = 128 elements
    Empirically chosen (Sec. 4.2). It controls the capacity of the propagated LiDAR representation.
  • Patch dimensionality D = 1024
    Standard dimensionality set for all attention modules (Sec. 4.2). It determines the width of the transformer layers.
  • Depth normalization maximum range = 200m for SLED, 100m for MVSEC, 120m for M3ED
    LiDAR projections and ground truth are normalized to [0,1] using the dataset maximum LiDAR range (Sec. 4.2). This scaling affects error magnitudes and the loss balance.
assumptions (5)
  • domain assumption Event volumes accumulated over a 50ms window retain sufficient temporal and spatial information for depth densification.
    Sec. 3.2 and 4.2 use a fixed delta_t of 50ms and the event volume representation of Zhu et al. The paper acknowledges in the conclusion that this representation can lose information under very fast motion.
  • domain assumption Projecting LiDAR points onto the camera image plane is an adequate input representation for the fusion task.
    Sec. 3.2 uses projected LiDAR as input. The conclusion notes that this projection makes the 3D structure disappear, which is a known limitation.
  • domain assumption Estimating a single depth map D_bf per temporal window is well-posed, and the earlier two-depth (D_bf/D_af) formulation is ill-posed.
    Sec. 3.1 argues this modeling choice. If the single-map assumption is wrong, the method could fail to represent depth changes inside a window.
  • domain assumption The ground truth depth maps on MVSEC and M3ED, despite acknowledged errors and sparsity, are reliable enough to validate the method and rank methods.
    Sec. 4.1 states MVSEC ground truth is erroneous for moving objects and calibration is approximate, and M3ED ground truth is very sparse. The paper still uses these datasets for its main comparisons.
  • domain assumption The redefined M3ED train/val/test split is representative of the dataset and does not bias results.
    Sec. 4.1 and Tab. 1 define new splits because the original test set lacks LiDAR. The selection of sequences is not justified as random or difficulty-balanced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DELTA: Dense Depth from Events and LiDAR using Transformer's Attention." pith.science (2026). https://pith.science/paper/GO2SYSGL

@misc{pith2026250502593,
  author       = {Pith},
  title        = {Pith review of: DELTA: Dense Depth from Events and LiDAR using Transformer's Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GO2SYSGL}},
  note         = {Machine review of arXiv:2505.02593}
}
read the original abstract

Event cameras and LiDARs provide complementary yet distinct data: respectively, asynchronous detections of changes in lighting versus sparse but accurate depth information at a fixed rate. To this day, few works have explored the combination of these two modalities. In this article, we propose a novel neural-network-based method for fusing event and LiDAR data in order to estimate dense depth maps. Our architecture, DELTA, exploits the concepts of self- and cross-attention to model the spatial and temporal relations within and between the event and LiDAR data. Following a thorough evaluation, we demonstrate that DELTA sets a new state of the art in the event-based depth estimation problem, and that it is able to reduce the errors up to four times for close ranges compared to the previous SOTA.

Figures

Figures reproduced from arXiv: 2505.02593 by the authors.

Figure 1
Figure 1. Overall principle of DELTA. Sparse and low-rate pro [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The complete architecture of our DELTA network. Unless noted, data is of shape [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Results on the Town01_08 ((a), (b)) and Town03_19 ((c), (d)) sequences of SLED. Rows (a) and (c), left to right: ground truth depth map; result from ALEDSL; our result (DELTASL). Dif￾ferences between ALEDSL and DELTASL are better seen in rows (b) and (d), showing the error maps of ALEDSL and DELTASL (where pixels with an error inferior to 0.5m are in gray). For a better visualization, an enlarged version of this fig… view at source ↗
Figures from the paper (20 more)
Figure 4
Figure 4. Figure 4: Qualitative results on the MVSEC dataset. Left to right: events; LiDAR projection (with larger points for a better readability); [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results for the city_hall_day sequence from M3ED. The size of points was increased for both the LiDAR projection and the ground truth. For a better visualization, an enlarged version of this figure is given in the Supplementary Material. Cutoff DELTASL DELT…
Figure 6
Figure 6. Figure 6: Qualitative results for the DELTANL and DELTANE vari￾ants, on the Town01_19 sequence of SLED. uation including ablation studies was conducted on three datasets of the state of the art to demonstrate the relevance of these propositions. On the synthetic SLED dataset, la…
Figure 7
Figure 7. Figure 7: Comparison on the Town01_08 (top) and Town03_19 (bottom) sequences of SLED (enlarged version of [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Comparison on the outdoor_day_1, outdoor_night_1, and outdoor_night_2 sequences of MVSEC (enlarged version of [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Comparison on the city_hall_day sequence of M3ED (enlarged version of [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: The alternative architecture without propagation memory, DELTA [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: The alternative architecture without central memory, DELTA [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: The alternative architecture without the central cross-attention, DELTA [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: The alternative architecture without the LiDAR input, DELTA [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: The alternative architecture without the event input, DELTA [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: The alternative architecture where the convolutional encoding heads are replaced by a simple splitting into patches, DELTA [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Results on the Town01_08 (top) and Town03_19 (bottom) sequences of SLED, for DELTASL and DELTASL NEH. Zoom on the numerical version may be required to better see the individual patches and artifacts for DELTASL NEH [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Additional results on the SLED dataset, on sequences [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Additional results on the SLED dataset, on sequences [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]
Figure 19
Figure 19. Figure 19: Additional results on the SLED dataset, on sequences [PITH_FULL_IMAGE:figures/full_fig_p021_19.png]
Figure 20
Figure 20. Figure 20: Additional results on the SLED dataset, on sequences [PITH_FULL_IMAGE:figures/full_fig_p022_20.png]
Figure 21
Figure 21. Figure 21: Additional results on the MVSEC dataset. Sequences shown, from left to right: [PITH_FULL_IMAGE:figures/full_fig_p023_21.png]
Figure 22
Figure 22. Figure 22: Additional results on the M3ED dataset, for the [PITH_FULL_IMAGE:figures/full_fig_p024_22.png]
Figure 23
Figure 23. Figure 23: Additional results on the M3ED dataset, for the [PITH_FULL_IMAGE:figures/full_fig_p025_23.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 54 canonical work pages

  1. [3]

    Learning to estimate two dense depths from LiDAR and event data

    Vincent Brebion, Julien Moreau, and Franck Davoine. Learning to estimate two dense depths from LiDAR and event data. InImage Analysis - 22nd Scandinavian Confer- ence, SCIA 2023, Sirkka, Finland, April 18-21, 2023, Pro- ceedings, Part II, pages 517–533. Springer, 2023. 2, 3, 4, 5, 6, 7, 8

    work page 2023

  2. [1]

    R. W. Baldwin, Ruixu Liu, Mohammed Almatrafi, Vi- jayan K. Asari, and Keigo Hirakawa. Time-ordered recent event (TORE) volumes for event cameras.IEEE TPAMI, 45: 2519–2532, 2021. 8

    work page 2021

  3. [2]

    A 240×180 130 dB 3 µs latency global shutter spatiotemporal vision sensor.IEEE Journal of Solid-State Circuits, 49:2333–2341, 2014

    Christian Brandli, Raphael Berner, Minhao Yang, Shih-Chii Liu, and Tobi Delbruck. A 240×180 130 dB 3 µs latency global shutter spatiotemporal vision sensor.IEEE Journal of Solid-State Circuits, 49:2333–2341, 2014. 2

    work page 2014

  4. [4]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InECCV, 2020. 3

    2020

  5. [5]

    Kenneth Chaney, Fernando Cladera Ojeda, Ziyun Wang, An- thony Bisulco, M. A. Hsieh, Christopher M. Korpela, Vi- jay R. Kumar, Camillo Jose Taylor, and Kostas Daniilidis. M3ED: Multi-robot, multi-sensor, multi-environment event dataset.CVPRW, pages 4016–4023, 2023. 2, 4

    work page 2023

  6. [6]

    Low-latency monocular depth estimation us- ing event timing on neuromorphic hardware.CVPRW, pages 4071–4080, 2023

    Stefano Chiavazza, Svea Marie Meyer, and Yulia San- damirskaya. Low-latency monocular depth estimation us- ing event timing on neuromorphic hardware.CVPRW, pages 4071–4080, 2023. 2

    work page 2023

  7. [7]

    Learning adaptive dense event stereo from the image domain.CVPR, pages 17797–17807, 2023

    Hoonhee Cho, Jegyeong Cho, and Kuk-Jin Yoon. Learning adaptive dense event stereo from the image domain.CVPR, pages 17797–17807, 2023. 2

    work page 2023

  8. [8]

    Learning phrase representations using RNN encoder-decoder for statistical machine translation

    Kyunghyun Cho, Bart van Merrienboer, C ¸ aglar G ¨ulc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. InCon- ference on Empirical Methods in Natural Language Process- ing, 2014. 4

    work page 2014

Show all 58 references
  1. [9]

    MULi-Ev: Maintaining unperturbed LiDAR-event calibra- tion.CVPRW, pages 4579–4586, 2024

    Mathieu Cocheteux, Julien Moreau, and Franck Davoine. MULi-Ev: Maintaining unperturbed LiDAR-event calibra- tion.CVPRW, pages 4579–4586, 2024. 2

    2024

  2. [10]

    Dense depth-map estimation based on fusion of event camera and sparse LiDAR.IEEE Transactions on Instrumentation and Measurement, 71:1– 11, 2022

    Mingyue Cui, Yuzhang Zhu, Yechang Liu, Yun-Meng Liu, Gang Chen, and Kai Huang. Dense depth-map estimation based on fusion of event camera and sparse LiDAR.IEEE Transactions on Instrumentation and Measurement, 71:1– 11, 2022. 2, 5, 6, 7, 8, 1, 4

    2022

  3. [11]

    CARLA: An open urban driving simulator

    Alexey Dosovitskiy, Germ ´an Ros, Felipe Codevilla, Antonio L´opez, and Vladlen Koltun. CARLA: An open urban driving simulator. InProceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017. 4

    2017

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  5. [13]

    Depth map prediction from a single image using a multi-scale deep net- work

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work. InNeurIPS, 2014. 5

    2014

  6. [14]

    Kerr Fitzgerald, Meng Law, Jarrel C. Y . Seah, Jennifer Tang, and Bogdan J. Matuszewski. Multi-resolution fine-tuning of vision transformers. InAnnual Conference on Medical Im- age Understanding and Analysis, 2022. 6

    2022

  7. [15]

    Derpa- nis, and Davide Scaramuzza

    Daniel Gehrig, Antonio Loquercio, Konstantinos G. Derpa- nis, and Davide Scaramuzza. End-to-end learning of repre- sentations for asynchronous event-based data.ICCV, pages 5632–5642, 2019. 8

    2019

  8. [16]

    Combining events and frames using recurrent asynchronous multimodal net- works for monocular depth prediction.IEEE Robotics and Automation Letters, 6:2822–2829, 2021

    Daniel Gehrig, Michelle R ¨uegg, Mathias Gehrig, Javier Hidalgo-Carri´o, and Davide Scaramuzza. Combining events and frames using recurrent asynchronous multimodal net- works for monocular depth prediction.IEEE Robotics and Automation Letters, 6:2822–2829, 2021. 2, 3, 5, 6

    2021

  9. [17]

    Recurrent vision transformers for object detection with event cameras.CVPR, pages 13884–13893, 2023

    Mathias Gehrig and Davide Scaramuzza. Recurrent vision transformers for object detection with event cameras.CVPR, pages 13884–13893, 2023. 2

    2023

  10. [18]

    DSEC: A stereo event camera dataset for driv- ing scenarios.IEEE Robotics and Automation Letters, 6: 4947–4954, 2021

    Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza. DSEC: A stereo event camera dataset for driv- ing scenarios.IEEE Robotics and Automation Letters, 6: 4947–4954, 2021. 2, 4

    2021

  11. [19]

    Multi-event-camera depth estimation and outlier rejection by refocused events fusion.Advanced Intelligent Systems, 4, 2022

    Suman Ghosh and Guillermo Gallego. Multi-event-camera depth estimation and outlier rejection by refocused events fusion.Advanced Intelligent Systems, 4, 2022. 2

    2022

  12. [20]

    Hierarchical neural memory network for low latency event processing.CVPR, pages 22867–22876, 2023

    Ryuhei Hamaguchi, Yasutaka Furukawa, Masaki Onishi, and Ken Sakurada. Hierarchical neural memory network for low latency event processing.CVPR, pages 22867–22876, 2023. 2, 6

    2023

  13. [21]

    Learning monocular dense depth from events.3DV, pages 534–542, 2020

    Javier Hidalgo-Carri ´o, Daniel Gehrig, and Davide Scara- muzza. Learning monocular dense depth from events.3DV, pages 534–542, 2020. 2

    2020

  14. [22]

    LCE-Calib: Automatic LiDAR-frame/event camera ex- trinsic calibration with a globally optimal solution.ArXiv, abs/2303.09825, 2023

    Jianhao Jiao, Feiyi Chen, He Wei, Jin Wu, and Mingming Liu. LCE-Calib: Automatic LiDAR-frame/event camera ex- trinsic calibration with a globally optimal solution.ArXiv, abs/2303.09825, 2023. 2

    2023 arXiv

  15. [23]

    Mukhopadhyay

    Uday Kamal, Saurabh Dash, and S. Mukhopadhyay. As- sociative memory augmented asynchronous spatiotemporal representation learning for event-based perception. InICLR,

  16. [24]

    Hanme Kim, Stefan Leutenegger, and Andrew J. Davison. Real-time 3D reconstruction and 6-DoF tracking with an event camera. InECCV, 2016. 2

    2016

  17. [25]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization.CoRR, abs/1412.6980, 2015. 5

    2015 arXiv

  18. [26]

    Enhancing 3-D LiDAR point clouds with event-based camera.IEEE Transactions on Instrumentation and Measurement, 70:1–12, 2021

    Boyang Li, Hao Meng, Yuzhang Zhu, Rihui Song, Mingyue Cui, Gang Chen, and Kai Huang. Enhancing 3-D LiDAR point clouds with event-based camera.IEEE Transactions on Instrumentation and Measurement, 70:1–12, 2021. 2

    2021

  19. [27]

    Salman Asif, and Zhan Ma

    Zhihao Li, M. Salman Asif, and Zhan Ma. Event transformer. ArXiv, abs/2204.05172, 2022. 2

    2022 arXiv

  20. [28]

    PCDepth: Pattern-based complementary learning for monocular depth estimation by best of both worlds.ArXiv, abs/2402.18925,

    Haotian Liu, Sanqing Qu, Fan Lu, Zongtao Bu, Florian Roehrbein, Alois Knoll, and Guang Chen. PCDepth: Pattern-based complementary learning for monocular depth estimation by best of both worlds.ArXiv, abs/2402.18925,

    arXiv

  21. [29]

    Event-based monocular dense depth estimation with recur- rent transformers.ArXiv, abs/2212.02791, 2022

    Xu Liu, Jianing Li, Xiaopeng Fan, and Yonghong Tian. Event-based monocular dense depth estimation with recur- rent transformers.ArXiv, abs/2212.02791, 2022. 2

    2022 arXiv

  22. [30]

    Stereo depth from events cameras: Concentrate and focus on the future.CVPR, pages 6104–6113, 2022

    Yeongwoo Nam, Sayed Mohammad Mostafavi Isfahani, Kuk-Jin Yoon, and Jonghyun Choi. Stereo depth from events cameras: Concentrate and focus on the future.CVPR, pages 6104–6113, 2022. 2

    2022

  23. [31]

    Time-to-contact map by joint estimation of up-to-scale inverse depth and global motion using a single event camera

    Urbano Miguel Nunes, Laurent Udo Perrinet, and Sio-Hoi Ieng. Time-to-contact map by joint estimation of up-to-scale inverse depth and global motion using a single event camera. ICCV, 2023. 2

    2023

  24. [32]

    GET: Group event transformer for event-based vision.ICCV, 2023

    Yansong Peng, Yueyi Zhang, Zhiwei Xiong, Xiaoyan Sun, and Feng Wu. GET: Group event transformer for event-based vision.ICCV, 2023. 2

    2023

  25. [33]

    Cot- tereau, and Timoth´ee Masquelier

    Ulysse Ranc ¸on, Javier Cuadrado-Anibarro, Benoit R. Cot- tereau, and Timoth´ee Masquelier. Stereospike: Depth learn- ing with a spiking neural network.IEEE Access, 10:127428– 127439, 2021. 2, 5, 6

    2021

  26. [34]

    U- Net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- Net: Convolutional networks for biomedical image segmen- tation. InMedical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241. Springer In- ternational Publishing, 2015. 3

    2015

  27. [35]

    Event transformer

    Alberto Sabater, Luis Montesano, and Ana Cristina Murillo. Event transformer. A sparse-aware solution for efficient event data processing.CVPRW, pages 2676–2685, 2022. 2

    2022

  28. [36]

    Event transformer+

    Alberto Sabater, Luis Montesano, and Ana Cristina Murillo. Event transformer+. A multi-purpose solution for efficient event data processing.ArXiv, abs/2211.12222, 2022. 2, 5, 6

    2022 arXiv

  29. [37]

    Mario A. V . Saucedo, Akash Patel, Rucha Sawlekar, Ak- shit Saradagi, Christoforos Kanellakis, Ali akbar Agha- mohammadi, and George Nikolakopoulos. Event camera and lidar based human tracking for adverse lighting conditions in subterranean environments.ArXiv, abs/2304.08908, 2023. 2

    2023 arXiv

  30. [38]

    Dynamic stereo vision system for real-time tracking.Proceedings of 2010 IEEE International Sympo- sium on Circuits and Systems, pages 1409–1412, 2010

    Stephan Schraml, Ahmed Nabil Belbachir, Nenad Milosevic, and Peter Sch¨on. Dynamic stereo vision system for real-time tracking.Proceedings of 2010 IEEE International Sympo- sium on Circuits and Systems, pages 1409–1412, 2010. 2

    2010

  31. [39]

    An event-driven stereo system for real-time 3-D 360◦ panoramic vision.IEEE Transactions on Industrial Electronics, 63:418–428, 2016

    Stephan Schraml, Ahmed Nabil Belbachir, and Horst Bischof. An event-driven stereo system for real-time 3-D 360◦ panoramic vision.IEEE Transactions on Industrial Electronics, 63:418–428, 2016. 2

    2016

  32. [40]

    Calibration of event-based camera and 3D Li- DAR.2018 WRC Symposium on Advanced Robotics and Automation (WRC SARA), pages 289–295, 2018

    Rihui Song, Zhihua Jiang, Yanghao Li, Yunxiao Shan, and Kai Huang. Calibration of event-based camera and 3D Li- DAR.2018 WRC Symposium on Advanced Robotics and Automation (WRC SARA), pages 289–295, 2018. 2

    2018

  33. [41]

    L2E: Lasers to events for 6- DoF extrinsic calibration of lidars and event cameras.ICRA, pages 11425–11431, 2023

    Kevin Ta, David Br ¨uggemann, Tim Br ¨odermann, Christos Sakaridis, and Luc Van Gool. L2E: Lasers to events for 6- DoF extrinsic calibration of lidars and event cameras.ICRA, pages 11425–11431, 2023. 2

    2023

  34. [42]

    RAFT: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jun Deng. RAFT: Recurrent all-pairs field transforms for optical flow. InECCV, pages 402–419, 2020. 4, 1

    2020

  35. [43]

    Andrade-Cetto

    Yi Tian and J. Andrade-Cetto. Event transformer flownet for optical flow estimation. InBMVC, 2022. 2

    2022

  36. [44]

    DeMoN: Depth and motion network for learning monocular stereo.CVPR, pages 5622–5631, 2016

    Benjamin Ummenhofer, Huizhong Zhou, Jonas Uhrig, Niko- laus Mayer, Eddy Ilg, Alexey Dosovitskiy, and Thomas Brox. DeMoN: Depth and motion network for learning monocular stereo.CVPR, pages 5622–5631, 2016. 4

    2016

  37. [45]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS, 2017. 1

    2017

  38. [46]

    Exploiting spatial sparsity for event cameras with visual transformers

    Zuowen Wang, Yuhuang Hu, and Shih-Chii Liu. Exploiting spatial sparsity for event cameras with visual transformers. ICIP, pages 411–415, 2022. 2

    2022

  39. [47]

    Adrian, Daniel Cremers, and J ¨org Conradt

    David Weikersdorfer, David B. Adrian, Daniel Cremers, and J ¨org Conradt. Event-based 3D SLAM with a depth- augmented dynamic vision sensor.ICRA, pages 359–364,

  40. [48]

    Event- based video reconstruction using transformer.ICCV, pages 2543–2552, 2021

    Wenming Weng, Yueyi Zhang, and Zhiwei Xiong. Event- based video reconstruction using transformer.ICCV, pages 2543–2552, 2021. 2

    2021

  41. [49]

    Kumar, and Kostas Daniilidis

    Alex Zihao Zhu, Dinesh Thakur, Tolga ¨Ozaslan, Bernd Pfrommer, Vijay R. Kumar, and Kostas Daniilidis. The multivehicle stereo event camera dataset: An event camera dataset for 3D perception.IEEE Robotics and Automation Letters, 3:2032–2039, 2018. 2, 4, 5

    2018

  42. [50]

    Unsupervised event-based learning of op- tical flow, depth, and egomotion.CVPR, pages 989–997,

    Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, and Kostas Daniilidis. Unsupervised event-based learning of op- tical flow, depth, and egomotion.CVPR, pages 989–997,

  43. [51]

    From chaos comes order: Ordering event repre- sentations for object recognition and detection.ICCV, 2023

    Nikola Zubi ´c, Daniel Gehrig, Mathias Gehrig, and Davide Scaramuzza. From chaos comes order: Ordering event repre- sentations for object recognition and detection.ICCV, 2023. 8 DELTA: Dense Depth from Events and LiDAR using Transformer’s Attention Supplementary Material

    2023

  44. [52]

    3 is given in Fig

    Enlarged Views of the Results on SLED, MVSEC, and M3ED As described in the main article, an enlarged version of Fig. 3 is given in Fig. 7 (this version also includes the in- put LiDAR and event data), an enlarged version of Fig. 4 is given in Fig. 8, and an enlarged version of...

  45. [53]

    10 to 15

    Alternative V ersions of DELTA Alternative versions of our DELTA network are given in Figs. 10 to 15. Versions of the network illustrated in Figs. 10 to 14 are used as part of the ablation study in Sec. 4.6 of the main article, while the version illustrated in Fig. 15 is used ...

  46. [54]

    Ablation Study on Encoding Heads In addition to the ablation studies conducted in the main ar- ticle, we propose here an additional variant of the network, DELTANEH, showcased in Fig. 15. Here, the convolutional encoding heads are replaced by a more direct splitting into patch...

  47. [55]

    No Encoding Head

    Computational Complexity We report in Tab. 8 several metrics of the computational complexity of DELTA, computed on a single NVIDIA L40 Map Cutoff DELTA SL DELTASLNEH Town01 10m0.661.58(+0.92) 20m1.332.89(+1.56) 30m1.913.90(+1.99) 100m3.226.73(+3.51) 200m4.549.85(+5.41) Town03 ...

  48. [56]

    17 to 20

    Additional Visual Results on the SLED Dataset Additional qualitative results on the SLED dataset are given in Figs. 17 to 20. We showcase in Figs. 17 and 18 scenes with accurate estimations, but also some small and larger failure cases in Figs. 19 and 20. Model Resolution (wit...

  49. [57]

    21, showing the quality of the results for both day and night scenes despite the sparse and low-resolution input event and LiDAR data

    Additional Visual Results on the MVSEC Dataset Additional qualitative results on the MVSEC dataset are given in Fig. 21, showing the quality of the results for both day and night scenes despite the sparse and low-resolution input event and LiDAR data

  50. [58]

    Additional Visual Results on the M3ED Dataset Additional qualitative results on the M3ED dataset are given in Figs. 22 and 23, where the sparsity of the ground truth depth maps (especially compared to the density of the LiDAR data) and the blob-like appearance of the objects i...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.