Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Dense Depth from Event Focal Stack

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dense depth maps from an event camera focus sweep

desk verdict A credible dense-depth-from-event-focal-stack pipeline with a strong synthetic comparison, but the real-data claim rests on unaligned pseudo-depth and qualitative evaluation. read the letter →

arxiv 2412.08120 v1 pith:RV42WAOO submitted 2024-12-11 cs.CV cs.AIcs.LGcs.RO

Kenta Horikawa , Mariko Isogawa , Hideo Saito , Shohei Mori This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.RO
keywords eventcameradensedepthestimationfromdefocusfocalstacksweepsimulationlow-lightvisionencoder-decodernetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a dense inverse depth map can be predicted from the event stream produced while an event camera's lens sweeps its focal plane. It turns the stream into a five-bin event focal stack and decodes it with a U-Net trained on simulated events. On synthetic scenes the method roughly halves the error of an image-domain depth-from-defocus baseline, and on real scenes it still produces usable depth in low light after fine-tuning on a small set of real events. The practical point is a depth sensor that works where ordinary cameras fail, using the event camera's high dynamic range and low power draw.

What carries the argument

The load-bearing object is the event focal stack: a $B=5$ channel voxel grid of time-weighted event counts, built by normalizing event timestamps into bins and distributing each event's polarity between the two closest bins. This compact representation lets a U-Net-style encoder-decoder with skip connections see the temporal order of defocus-induced events and produce a dense depth map with mean squared error loss against inverse depth. Lens breathing from real focus sweeps is corrected by homography warps computed from a defocus-robust circular checkerboard. Synthetic training data comes from rendered focal stacks of random 3D scenes converted to events by two event simulators, and the real-data step fine-tunes on fifty scenes with pseudo ground truth from a pre-trained monocular depth estimator.

What would settle it

Record a focus-sweep event stream in a scene with independent measured depth (for example, a LiDAR or structured-light scan) in low-light conditions, run the proposed network, and compare inverse depth predictions against the measured values; if the predictions do not beat the image-domain baseline, or if the MAE against true depth is no better than chance, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the spatial and temporal structure of a focus-sweep event stream encodes depth densely enough to regress an inverse depth map at every pixel, not just at event locations. The paper defends this by constructing event focal stacks from events and training a convolutional encoder-decoder to map the stack directly to inverse depth, using synthetic focal stacks rendered from arbitrary 3D scenes and converted to events through event simulators. The reported result is that this event-based method outperforms a comparable image-domain depth-from-defocus network (MAE 0.0762 vs 0.1606 in inverse depth on synthetic data), and that after fine-tuning on real events it gives plausible qualitative depth in office and low-light scenes.

Load-bearing premise

The real-world evaluation and fine-tuning use pseudo depth from a pre-trained monocular depth estimator applied to well-lit images, not sensor-measured depth; if that pseudo depth is biased, the fine-tuned model inherits the bias and the real-data claim is not demonstrated.

Editorial extensions

If this is right

  • Depth sensing can continue under low light or rapid motion, where frame-based cameras lose the signal.
  • The dense-depth task can be trained without real depth sensors, since synthetic focal stacks and event simulation provide labeled data at scale.
  • A focus-tunable lens, if it can sweep faster than a mechanical lens, should feed the same network with more events per unit time and improve accuracy.
  • The success of the event focal stack suggests treating a focus sweep as a time-coded volume rather than a set of images, which may also help other event-based vision tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to add sensor-measured ground truth (stereo or LiDAR) to the real benchmark; the paper's real-data evaluation currently rests on pseudo depth from a monocular estimator, so a true depth comparison would settle the real-world claim.
  • The authors observe that simulators produce few negative events; building a simulator with better noise and polarity statistics could remove much of the need for fine-tuning.
  • Applying the same event focal stack to dynamic scenes would require aligning events on moving objects to a common sweep; the paper leaves that as a limitation, so a natural follow-up is a recurrent or motion-compensated variant.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents a method for dense depth estimation from an event stream produced by sweeping the focal plane of a lens attached to an event camera. Events are voxelized into a B=5 'event focal stack' and decoded by a U-Net into a dense inverse depth map. Training uses Blender-rendered focal stacks passed through ESIM or DVS-Voltmeter event simulators, with metric depth ground truth; real-world adaptation is done by fine-tuning on a small real dataset with pseudo ground truth from Depth Anything, after correcting for lens breathing via homographies. The method is compared against a depth-from-defocus baseline (Focus on Defocus) on 20 synthetic test scenes and on 10 real scenes (qualitatively). Quantitative synthetic results show MAE 0.0762 (DVS-Voltmeter) vs 0.1606 for the baseline, with ablations of bin size and polarity integration.

Significance. The idea of using a focus-sweep event stream as a dense depth cue is novel and well-motivated; the event focal stack representation is a natural extension of image focal stacks, and the synthetic-to-real fine-tuning pipeline with lens breathing correction is a practical contribution that goes beyond simple naive domain transfer. The synthetic evaluation is quantitative, uses 20 test scenes, averages over 10 training runs, and clearly separates the proposed configuration from the image-domain baseline. However, the real-world component, which is the abstract's second half of the central claim, is supported only by qualitative depth maps and relies on unaligned pseudo-labels, so the claim of 'superior performance on real datasets' is not yet established.

major comments (2)
  1. [Sec. 5 (Real dataset) and Fig. 7] The abstract claims superior performance over the image-domain baseline on real datasets, but the real-world evaluation is entirely qualitative. No MAE/RMSE or any quantitative metric is reported for the 10 held-out real scenes, and the statement that the method 'remains usable in low light' is a visual judgment. Since the real-data comparison is a load-bearing part of the central claim, please add quantitative evaluation on the real test scenes, ideally against sensor-measured depth (e.g., LiDAR or structured light) or at least against aligned pseudo-labels with clearly stated limitations.
  2. [Sec. 3.2, Eq. (1) and Sec. 4.1 (Real Dataset)] The real-data fine-tuning and evaluation use pseudo ground truth Dgt generated by Depth Anything, which outputs affine-invariant (scale- and shift-ambiguous) depth, yet Eq. (1) is an MSE loss in metric inverse depth (1/m). No per-scene scale/shift alignment, normalization to the synthetic inverse-depth range, or filtering of unreliable pseudo-labels is described. This means the fine-tuning target does not match the training target, and the network can internalize an arbitrary scale/offset per scene, confounding both the fine-tuned model and any qualitative comparison. Please describe the alignment procedure, or report results with a properly aligned pseudo-label baseline, or use sensor-measured ground truth.
minor comments (5)
  1. [Sec. 4.4 and Tables 1-5] The paper states that all evaluations were repeated 10 times, but only point estimates are reported. Please add means and standard deviations (or confidence intervals) so the reader can judge the significance of differences, especially for the close B=5 vs B=10 values in Table 2.
  2. [Sec. 4.5 and Table 2] The choice of B=5 for DVS-Voltmeter is inconsistent with the quantitative results: Table 2 gives B=10 the lowest MAE (0.0760) and RMSE (0.1006), while B=5 is second (0.0762, 0.1022). The justification 'plausibly lower errors with B=5' refers to a qualitative figure. Please either adopt the quantitatively best configuration for the main comparison or provide a non-qualitative rationale (e.g., compactness) for choosing B=5, and state how this affects the reported results.
  3. [Sec. 3.2] It is unclear whether the network output is constrained to be non-negative (as inverse depth should be) and how pixels with no events or invalid ground-truth depth are handled in Eq. (1). Please specify the output activation and the masking/weighting of invalid pixels.
  4. [Sec. 5 and Fig. 7] The claim that fine-tuning 'mitigates both the reversed depth and holes' would be strengthened by reporting at least per-scene quantitative errors on the real data after aligning the pseudo-labelled ground truth, rather than relying solely on visual inspection.
  5. [Figure 1 and Sec. 3.1] The citation '[27]' is referred to in the text as 'Alex et al.'; for consistency with the reference list it should be 'Zhu et al.' Also, the figure captions contain inline code-like references such as '(Sec 3.1)' that should be formatted consistently.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the synthetic pipeline trains on Blender ground-truth depth and is evaluated against an external image-domain baseline; the real-data component uses external pseudo-labels and is qualitative.

full rationale

The paper's central claim is empirical: a U-Net maps a voxelized event focal stack to a dense inverse depth map. The training signal is Blender-rendered ground-truth depth (Sec. 4.1: 'We also rendered corresponding depth images as the ground truth'), and the main quantitative comparison is against Focus on Defocus [16], an external baseline trained under the same protocol. The voxelization (Sec. 3.1) and U-Net architecture (Sec. 3.2) are generic components adopted from prior work [12, 27]; the event-to-depth mapping is learned from data and is not defined in terms of the target depth. No fitted parameter is renamed as a prediction: the only tuned hyperparameter, bin size B=5 (Tables 1 and 2), is selected on an evaluation set, which is a mild selection-on-test concern but does not make the depth output equal to any training input. The real-data fine-tuning uses pseudo labels from the external Depth Anything estimator (Secs. 3.2 and 4.1); these are not the network's own outputs, and the real results are reported qualitatively (Fig. 7), so they do not force the result. The cited prior work by the same group ([14]) is related-work context on focal stacks and is not load-bearing. The acknowledged limitations (textureless surfaces, static scenes) further indicate the authors do not claim a forced or tautological result. The derivation chain is therefore self-contained; the pseudo-depth and test-set-selection concerns belong to correctness risk rather than circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method is an empirical pipeline, so the ledger is mostly domain assumptions about event simulation, lens optics, and pseudo labels rather than free-floating mathematical entities. The main unverified input is the pseudo-depth source (Depth Anything); the main tunable hyperparameters are the voxel bin count, ESIM threshold, and data split. No new physical entities are introduced.

free parameters (3)
  • Number of time bins B = 5
    Selected by comparing MAE/RMSE on synthetic scenes in Tables 1 and 2; the optimal value by metric for DVS-Voltmeter is B=10 but B=5 is chosen for compactness and consistency with prior work.
  • ESIM contrast threshold = 0.08
    Set to 0.08 in Sec. 4.5 to make synthetic events visually noisier and closer to real events; this threshold affects the synthetic training distribution.
  • Fine-tuning set size = 50 scenes
    Real scenes are split 50 for fine-tuning and 10 for inference; the 50/10 split is a design choice that directly affects the reported real-data result.
assumptions (5)
  • domain assumption A focus sweep of the lens produces a time-ordered event stream whose local event timing encodes depth (defocus-to-depth cue).
    This is the core physical assumption that an event focal stack carries depth information; it motivates the entire pipeline and is not proven by the paper.
  • domain assumption Events synthesized by ESIM or DVS-Voltmeter from Blender focal stacks are sufficiently similar to real events for pretraining a depth network.
    The paper shows a domain gap and uses fine-tuning to close it, implicitly assuming that the synthetic distribution still provides useful structure for the real task.
  • ad hoc to paper Depth Anything pseudo-labels from well-lit grayscale images are accurate enough to serve as ground truth for fine-tuning and for evaluating real scenes, including low-light scenes.
    Sec. 4.1 explicitly uses Depth Anything as an alternative ground truth because no real depth sensor was used; this assumption is load-bearing for the real part of the claim.
  • domain assumption Lens breathing can be corrected by warping event coordinates with planar homographies computed from a calibration checkerboard at fixed distances up to two meters.
    Sec. 3.3 applies homography H[k] to correct FOV change; this is exact only for points on the calibration plane, while the actual scenes are 3D, so residual distortion is assumed small.
  • domain assumption The scene is static during the focus sweep; events are due only to the focus change.
    The authors list this as a limitation in Sec. 5, but the network and datasets are built on it during both training and inference.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dense Depth from Event Focal Stack." pith.science (2026). https://pith.science/paper/RV42WAOO

@misc{pith2026241208120,
  author       = {Pith},
  title        = {Pith review of: Dense Depth from Event Focal Stack},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RV42WAOO}},
  note         = {Machine review of arXiv:2412.08120}
}
read the original abstract

We propose a method for dense depth estimation from an event stream generated when sweeping the focal plane of the driving lens attached to an event camera. In this method, a depth map is inferred from an ``event focal stack'' composed of the event stream using a convolutional neural network trained with synthesized event focal stacks. The synthesized event stream is created from a focal stack generated by Blender for any arbitrary 3D scene. This allows for training on scenes with diverse structures. Additionally, we explored methods to eliminate the domain gap between real event streams and synthetic event streams. Our method demonstrates superior performance over a depth-from-defocus method in the image domain on synthetic and real datasets.

Figures

Figures reproduced from arXiv: 2412.08120 by the authors.

Figure 1
Figure 1. The proposed framework for a dense depth map only from an event focal stack. We collected the datasets in both synthetic and real-world (Sec 4.1) environments for this framework. By voxelizing the events from the focus sweep into an event focal stack (Sec 3.1), the data is transformed into a format compatible with a U-Net like CNN architecture and then input into the network (Sec 3.2). We aim to bridge the domain ga… view at source ↗
Figure 2
Figure 2. Collecting real-captured data. We captured the focal sweep events by event camera and computer-controlled lens. To avoid the impact of breathing, we use homography matrices H[k] = R 3×3 calculated by 330 images of a circular checkerboard for the correction. generation via an event simulator. We used the synthetic data to leverage ground truth depth maps. Note that image focal stacks are generated for training. Our m… view at source ↗
Figure 3
Figure 3. Qualitative comparison of bin impacts. Inverse depth images indicate that the distance increases as the color transitions from orange to purple. Differential images show that the error increases as the color transitions from blue to red. Both ESIM and DVS-Voltmeter, bin = 5 shows the smallest error. breathing is a phenomenon that involves varying fields of view as the lens focus distance changes. It is challenging t… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of events using ESIM, DVS￾Voltmeter and real-captured event. The captured scene is one in which boxes are arranged to become progressively more dis￾tant from left to right. Although the real-captured events can be observed negative events (red do…
Figure 6
Figure 6. Figure 6: Qualitative comparison using synthetic data generated by DVS-Voltmeter. Ours have a small margin of error, and the depth images do not become blurred. 5. Experiments and Discussion Synthetic dataset [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison using real-captured data at the condition of two light environment. The rows show the same scene. We prepared models trained on synthetic events generated by ESIM and DVS-Voltmeter, and compared the results with and without fine￾tuning on real da…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Static in Frames, Dynamic in Events: Rethinking Features in Event Cameras as Motion Cues

    cs.CV 2026-08 conditional novelty 5.0 of 10

    Harris eigenvalues and spatiotemporal density values from event cameras encode motion direction and, when added to an optical flow network, improve accuracy in data-scarce settings.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [4]

    Learning Monocular Depth from Focus with Event Focal Stack

    Jiang Chenxu, Lin Mingyuan, Zhang Chi, Wang Zhenghai, and Yu Lei. Learning monocular depth from focus with event focal stack. arXiv preprint arXiv:2405.06944, 2024. 1, 2

    work page Pith review arXiv 2024

  2. [1]

    End-to-end learning of geometry and context for deep stereo regression

    Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Pe- ter Henry, Ryan Kennedy, Abraham Bachrach, Adam Bry. End-to-end learning of geometry and context for deep stereo regression. In ICCV, pages 66–75, 2017. 1

    work page 2017

  3. [2]

    A new sense for depth

    Alex Paul Pentland. A new sense for depth. IEEE TPAMI, PAMI-9(4):523–531, 1987. 1

    work page 1987

  4. [3]

    The Blender project - free and open 3d creation software, Accesed: 2023

    Blender Foundation. The Blender project - free and open 3d creation software, Accesed: 2023. 4

    work page 2023

  5. [5]

    Eigen and R

    D. Eigen and R. Fergus. Predicting depth, surface normals and semantic labels with a common multi-scale convolu- tional architecture. In ICCV, pages 2650–2658, 2015. 2

    work page 2015

  6. [6]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2014. 5

    work page 2014

  7. [7]

    Depthlab: Real-time 3d interaction with depth maps for mobile augmented reality

    Ruofei Du, Eric Lee Turner, Maksym Dzitsiuk, Luca Prasso, Ivo Duarte, Jason Dourgarian, Joao Afonso, Jose Pascoal, Josh Gladstone, Nuno Moura e Silva Cruces, Shahram Izadi, Adarsh Kowdle, Konstantine Nicholas John Tsotsos, and David Kim. Depthlab: Real-time 3d interaction with depth maps for mobile augmented reality. In Proceedings of the 33rd Annual ACM ...

    work page 2020

  8. [8]

    Deep depth from focus with differential focus volume

    Fengting Yang, Xiaolei Huang, and Zihan Zhou. Deep depth from focus with differential focus volume. In CVPR, pages 12632–12641, 2022. 2

    work page 2022

Show all 27 references
  1. [9]

    Davison, J¨org Conradt, Kostas Dani- ilidis, Davide Scaramuzza

    Guillermo Gallego, Tobi Delbru ¨uck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J. Davison, J¨org Conradt, Kostas Dani- ilidis, Davide Scaramuzza. Event-based vision: A survey. IEEE TPAMI, 44(1):154–180, 2019. 1

    2019

  2. [10]

    A spiking neural network model of depth from defocus for event-based neuromorphic vision

    Germain Haessig, Xavier Berthelon, Sio-Hoi Ieng, and Ryad Benosman. A spiking neural network model of depth from defocus for event-based neuromorphic vision. Scientific re- ports, 9(1):3744, 2019. 1

    2019

  3. [11]

    All- in-focus imaging from event focal stack

    Hanyue Lou, Minggui Teng, Yixin Yang, and Boxin Shi. All- in-focus imaging from event focal stack. In CVPR, pages 17366–17375, 2023. 2

    2023

  4. [12]

    High speed and high dynamic range video with an event camera

    Henri Rebecq, Ren ´e Ranftl, Vladlen Koltun, and Davide Scaramuzza. High speed and high dynamic range video with an event camera. IEEE TPAMI, 43(6):1964–1980, 2019. 2, 3, 5, 6

    1964

  5. [13]

    Co-attention- guided bilinear model for echo-based depth estimation

    Go Irie, Takashi Shibata, and Akisato Kimura. Co-attention- guided bilinear model for echo-based depth estimation. In ICASSP, pages 4648–4652, 2022. 1

    2022

  6. [14]

    Multi-layer scene representation from composed focal stacks

    Reina Ishikawa, Hideo Saito, Denis Kalkofen, and Shohei Mori. Multi-layer scene representation from composed focal stacks. IEEE TVCG, 29(11):4719–4729, 2023. 2

    2023

  7. [15]

    Learning monocular dense depth from events

    Javier Hidalgo-Carri ´o, Daniel Gehrig and Davide Scara- muzza. Learning monocular dense depth from events. In 3DV, pages 534–542, 2020. 1, 2

    2020

  8. [16]

    Fo- cus on defocus: Bridging the synthetic to real domain gap for depth estimation

    Maxim Maximov, Kevin Galim, and Laura Leal-Taixe. Fo- cus on defocus: Bridging the synthetic to real domain gap for depth estimation. In CVPR, pages 1071–1080, 2020. 1, 2, 5, 6, 7

    2020

  9. [17]

    M4depth: Monocular depth estimation for autonomous ve- hicles in unseen environments

    Micha ¨el Fonder, Damien Ernst, Marc Van Droogenbroeck. M4depth: Monocular depth estimation for autonomous ve- hicles in unseen environments. Sensors, 22(23):1–22, 2022. 1

    2022

  10. [18]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In International Conference on Medical image com- puting and computer-assisted intervention , pages 234–241,

  11. [19]

    Zhou and A

    Q. Zhou and A. Jacobson. Thingi10k: A dataset of 10,000 3d-printing models. arXiv preprint, 2016. 4

    2016

  12. [20]

    ESIM: an open event camera simulator

    Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. ESIM: an open event camera simulator. Conf. on Robotics Learning (CoRL), Oct. 2018. 1, 5

    2018

  13. [21]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE TPAMI, 44(10):1623–1637, 2022. 2

    2022

  14. [22]

    Dvs- voltmeter: Stochastic process-based event simulator for dy- namic vision sensors

    Songnan Lin, Ye Ma, Zhenhua Guo, and Bihan Wen. Dvs- voltmeter: Stochastic process-based event simulator for dy- namic vision sensors. In ECCV, pages 578–593, 2022. 1, 5

    2022

  15. [23]

    Computer Vision: Algorithms and Appli- cations

    Richard Szeliski. Computer Vision: Algorithms and Appli- cations. Springer Nature, 2022. 2

    2022

  16. [24]

    Event-based monocular dense depth estimation with recur- rent transformers

    Xu Liu, Jianing Li, Xiaopeng Fan and Yonghong Tian. Event-based monocular dense depth estimation with recur- rent transformers. arXiv preprint, 2022. 2

    2022

  17. [25]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, pages 10371–10381, 2024. 3, 5

    2024

  18. [26]

    Motion deblurring and depth estimation from multiple images

    Zhen, Ruiwen and Stevenson, Robert L. Motion deblurring and depth estimation from multiple images. In ICIP, pages 2688–2692, 2016. 1

    2016

  19. [27]

    Unsupervised event-based optical flow using mo- tion compensation

    Alex Zihao Zhu, Liangzhe Yuan, Keneth Chaney, and Kostas Daniilidis. Unsupervised event-based optical flow using mo- tion compensation. In ECCVW, 2018. 3

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.