pith. sign in

arxiv: 2606.18826 · v1 · pith:ZKUSMCA7new · submitted 2026-06-17 · ⚛️ physics.optics · cs.CV· eess.IV

EDoF-NeRF: extended depth-of-field neural radiance fields using a coded aperture camera

Yoshiyuki Shirasaki , Ryoichi Horisaki This is my paper

Pith reviewed 2026-06-26 20:19 UTC · model grok-4.3

classification ⚛️ physics.optics cs.CVeess.IV
keywords neural radiance fieldscoded aperturedepth of fieldnovel view synthesiscamera modeldefocusphotorealistic renderingextended depth of field
0
0 comments X

The pith

Coded aperture integration into NeRF allows direct use of defocused images for high-fidelity novel views with extended depth of field.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

NeRF training datasets inherit the depth-of-field versus light-quantity trade-off from conventional cameras. The paper introduces a coded aperture at the pupil that preserves spatial frequency content even when images are defocused. A new camera model is built inside the NeRF framework so that these coded images can be fed directly into the training process. The resulting EDoF-NeRF then renders novel views that remain sharp over a larger depth range than standard aperture models permit. Simulations and real-camera experiments show improved fidelity relative to conventional capture.

Core claim

We develop a camera model that incorporates coded apertures into NeRF, allowing direct input of coded images and enabling the generation of novel views with an extended DoF while maintaining high fidelity.

What carries the argument

Coded aperture placed at the camera pupil, integrated into the NeRF forward imaging model to handle defocus without frequency loss.

If this is right

  • NeRF training sets can be acquired with wider apertures, shortening exposure times while still covering larger scene depths.
  • Novel-view synthesis quality remains high even when input images contain intentional defocus from the coded pattern.
  • The same coded-aperture model can be swapped into existing NeRF pipelines without changing the underlying radiance-field representation.
  • Real-world experiments confirm the simulated gains, indicating the method transfers from rendered to physical cameras.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach may generalize to other implicit scene representations that rely on differentiable camera models.
  • Optimized aperture codes could be scene-specific, learned jointly with the radiance field rather than fixed in advance.
  • Extending the method to dynamic scenes would require the coded pattern to preserve frequencies across both space and time.

Load-bearing premise

The chosen coded aperture pattern must continue to transmit the scene's spatial frequencies even when the sensor plane lies outside the focal range.

What would settle it

Capture the same scene with both a conventional aperture and the coded aperture at identical focus settings, then compare the peak signal-to-noise ratio of novel views synthesized by each NeRF; a drop below the conventional case would falsify the claim.

Figures

Figures reproduced from arXiv: 2606.18826 by Ryoichi Horisaki, Yoshiyuki Shirasaki.

Figure 1
Figure 1. Figure 1: Pinhole camera model. 3.2. Lens camera The lens camera model proposed in DoF-NeRF [10], as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Lens camera model. Proxy point Ray Coded aperture Image plane Sampling point Blur diameter: Dproxy Pixel position: Lens-sensor distance: s Proxy depth: Rproxy Interval: ∆I−1 C1 1 σ1 CI I σI CI−1 I − 1 σI−1 ・・・ Index:i Density: σi Aperture diameter: d Color radiance: Ci Lens (Focal length: ) f ・・・ ・・・ p CCA(p) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Coded aperture camera model. 4. Network architecture 4.1. Positional encoding Following the original NeRF framework, we employ multi-layer perceptrons (MLPs) to predict the color radiance 𝑪𝑖( 𝒑) and the density 𝜎𝑖( 𝒑) at the 𝑖-th sampling point along the ray corresponding to the pixel position 𝒑, as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the network architecture and training pipeline. FC layer: fully [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Numerical demonstration results. (a) Reference novel-view image generated [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Depth maps in the numerical demonstration. (a) Reference depth map rendered [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Experimental camera setup for optical validation. (a) Camera equipped with the [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Optical demonstration results. (a) Reference all-in-focus image captured [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Depth maps in the optical demonstration. (a) Reference depth map rendered [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
read the original abstract

We propose a method for extending the depth-of-field (DoF) to construct high-fidelity neural radiance fields (NeRF) -- an emerging technique for rendering photorealistic novel views from a dataset of images captured at different viewpoints, based on implicit neural representations. The trade-off between DoF and light quantity is inherent not only in conventional cameras but also in NeRF, since the datasets used by NeRF are captured by these cameras. To address this issue, we introduce a coded aperture placed at the camera pupil, preserving spatial frequency components under defocused conditions. We develop a camera model incorporating coded apertures into NeRF, allowing direct input of coded images and enabling the generation of novel views with an extended DoF. We validate the proposed method, termed extended DoF-NeRF (EDoF-NeRF), through simulations and experiments, demonstrating its superior performance compared to conventional aperture cameras.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes EDoF-NeRF, a NeRF variant that incorporates a coded aperture placed at the camera pupil into the forward camera model. This allows direct use of defocused coded images as input while generating novel views with extended depth-of-field, by asserting that the coded aperture preserves spatial frequency content under defocus. Validation is claimed via simulations and real experiments showing superior performance over conventional circular apertures.

Significance. If the optical model is shown to be invertible across depths, the approach would address a practical bottleneck in NeRF data capture—the inherent DoF versus light-throughput trade-off—potentially enabling larger apertures without focus stacking. The work introduces a concrete hardware modification (coded aperture) directly into the differentiable rendering pipeline, which is a clear engineering contribution even if the frequency-preservation claim requires further substantiation.

major comments (2)
  1. [Abstract / camera model] Abstract and camera-model description: the central claim that the coded aperture 'preserving spatial frequency components under defocused conditions' is asserted without derivation of the pupil function, its Fourier transform, or the resulting depth-dependent OTF/PSF. This assumption is load-bearing for the statement that coded images can be fed directly into NeRF to recover a sharp radiance field; without an explicit expression replacing the standard pinhole integral, it is impossible to verify that high-frequency content remains recoverable for out-of-focus points.
  2. [Validation / experiments] Validation section: the abstract states that superiority is demonstrated 'through simulations and experiments,' yet no error metrics (PSNR, SSIM, depth-range coverage), scene parameters, or comparison baselines (e.g., focus-stacking NeRF or small-aperture capture) are supplied. Without these, the empirical support for the extended-DoF claim cannot be assessed and the result remains unverifiable from the given text.
minor comments (2)
  1. [Method] Notation for the coded pupil function and its integration into the volume-rendering equation should be introduced with an explicit equation number rather than left at the level of prose description.
  2. [Abstract] The abstract would benefit from a single quantitative statement (e.g., 'X dB PSNR gain over Y mm DoF range') to allow readers to gauge the magnitude of the improvement before reading further.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the optical model and validation. We address each major comment below and will revise the manuscript to improve clarity and verifiability.

read point-by-point responses

  1. Referee: [Abstract / camera model] Abstract and camera-model description: the central claim that the coded aperture 'preserving spatial frequency components under defocused conditions' is asserted without derivation of the pupil function, its Fourier transform, or the resulting depth-dependent OTF/PSF. This assumption is load-bearing for the statement that coded images can be fed directly into NeRF to recover a sharp radiance field; without an explicit expression replacing the standard pinhole integral, it is impossible to verify that high-frequency content remains recoverable for out-of-focus points.

    Authors: We agree that the current presentation would benefit from an explicit derivation. In the revised manuscript we will add a new subsection deriving the pupil function of the coded aperture, its Fourier transform, and the resulting depth-dependent OTF. This will include the modified forward imaging integral that replaces the standard pinhole model and will explicitly show the frequency-preservation property relative to a circular aperture. revision: yes

  2. Referee: [Validation / experiments] Validation section: the abstract states that superiority is demonstrated 'through simulations and experiments,' yet no error metrics (PSNR, SSIM, depth-range coverage), scene parameters, or comparison baselines (e.g., focus-stacking NeRF or small-aperture capture) are supplied. Without these, the empirical support for the extended-DoF claim cannot be assessed and the result remains unverifiable from the given text.

    Authors: We acknowledge that the abstract alone does not contain quantitative details. The full manuscript reports simulation and real-world results; however, to address the concern we will expand the experiments section with a summary table of PSNR, SSIM, and depth-range metrics, explicit scene parameters, and direct numerical comparisons against focus-stacking NeRF and small-aperture baselines. revision: yes

Circularity Check

0 steps flagged

No circularity: forward camera model is constructed independently of the NeRF optimization target

full rationale

The paper describes a forward model that inserts a coded-aperture pupil function into the standard NeRF volume-rendering integral. No equation is shown in which a fitted parameter (e.g., a scale, ratio, or loss term) is later renamed as a prediction, nor is any uniqueness theorem imported from the authors’ prior work to force the model choice. The claim that the coded aperture “preserves spatial frequency components under defocused conditions” is presented as an optical property motivating the design rather than a quantity derived from the NeRF loss itself. Because the derivation chain remains self-contained against external optical benchmarks and does not reduce any reported performance metric to a tautological fit, the circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; the central claim rests on the unelaborated statement that the coded aperture preserves spatial frequencies.

pith-pipeline@v0.9.1-grok · 5694 in / 1114 out tokens · 22850 ms · 2026-06-26T20:19:56.200244+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    NeRF: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik,et al., “NeRF: Representing scenes as neural radiance fields for view synthesis,” Commun. ACM65, 99–106 (2021)

    2021

  2. [2]

    RT-NeRF: Real-time on-device neural radiance fields towards immersive AR/VR rendering,

    C. Li, S. Li, Y. Zhao,et al., “RT-NeRF: Real-time on-device neural radiance fields towards immersive AR/VR rendering,” inProceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design,(2022), pp. 1–9

    2022

  3. [3]

    ARthroNeRF: Field of view enhancement of arthroscopic surgeries using augmented reality and neural radiance fields,

    X. Zou, Z. Zhang, A. Schwarz,et al., “ARthroNeRF: Field of view enhancement of arthroscopic surgeries using augmented reality and neural radiance fields,” in2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR),(IEEE, 2024), pp. 1197–1205

    2024

  4. [4]

    Neural radiance fields (NeRF): Review and potential applications to digital cultural heritage,

    V. Croce, G. Caroti, L. De Luca,et al., “Neural radiance fields (NeRF): Review and potential applications to digital cultural heritage,” The Int. Arch. Photogramm. Remote. Sens. Spatial Inf. Sci.48, 453–460 (2023)

    2023

  5. [5]

    NeRF for heritage 3D reconstruction,

    G. Mazzacca, A. Karami, S. Rigon,et al., “NeRF for heritage 3D reconstruction,” Int. Arch. Photogramm. Remote. Sens. Spatial Inf. Sci.48, 1051–1058 (2023)

    2023

  6. [6]

    Lightning NeRF: Efficient hybrid scene representation for autonomous driving,

    J. Cao, Z. Li, N. Wang, and C. Ma, “Lightning NeRF: Efficient hybrid scene representation for autonomous driving,” in2024 IEEE International Conference on Robotics and Automation (ICRA),(IEEE, 2024), pp. 16803–16809

    2024

  7. [7]

    DriveEnv-NeRF: Exploration of a NeRF-based autonomous driving environment for real-world performance validation,

    M.-Y. Shen, C.-C. Hsu, H.-Y. Hou,et al., “DriveEnv-NeRF: Exploration of a NeRF-based autonomous driving environment for real-world performance validation,” ArXiv Prepr. ArXiv:2403.15791 (2024)

    work page arXiv 2024

  8. [8]

    Deblur-NeRF: Neural radiance fields from blurry images,

    L. Ma, X. Li, J. Liao,et al., “Deblur-NeRF: Neural radiance fields from blurry images,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,(2022), pp. 12861–12870

    2022

  9. [9]

    AR-NeRF: Unsupervised learning of depth and defocus effects from natural images with aperture rendering neural radiance fields,

    T. Kaneko, “AR-NeRF: Unsupervised learning of depth and defocus effects from natural images with aperture rendering neural radiance fields,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,(2022), pp. 18387–18397

    2022

  10. [10]

    DoF-NeRF: Depth-of-field meets neural radiance fields,

    Z. Wu, X. Li, J. Peng,et al., “DoF-NeRF: Depth-of-field meets neural radiance fields,” inProceedings of the 30th ACM International Conference on Multimedia,(2022), pp. 1718–1729

    2022

  11. [11]

    J. W. Goodman,Introduction to Fourier Optics(Roberts and Company publishers, 2005)

    2005

  12. [12]

    Single-shotcompressivespectralimagingwithadual-disperserarchitecture,

    M.E.Gehm,R.John,D.J.Brady,et al.,“Single-shotcompressivespectralimagingwithadual-disperserarchitecture,” Opt. Express15, 14013–14027 (2007)

    2007

  13. [13]

    Video rate spectral imaging using a coded aperture snapshot spectral imager,

    A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express17, 6368–6388 (2009)

    2009

  14. [14]

    Coded aperture compressive temporal imaging,

    P. Llull, X. Liao, X. Yuan,et al., “Coded aperture compressive temporal imaging,” Opt. Express21, 10526–10545 (2013)

    2013

  15. [15]

    Coded aperture detector: an image sensor with sub 20-nm pixel resolution,

    R. Miyakawa, R. Mayer, A. Wojdyla,et al., “Coded aperture detector: an image sensor with sub 20-nm pixel resolution,” Opt. Express22, 19803–19809 (2014)

    2014

  16. [16]

    Coupled deep learning coded aperture design for compressive image classification,

    J. Bacca, L. Galvis, and H. Arguello, “Coupled deep learning coded aperture design for compressive image classification,” Opt. Express28, 8528–8540 (2020)

    2020

  17. [17]

    Coded aperture compression temporal imaging based on a dual-mask and deep denoiser,

    Y. Ge, G. Qu, Y. Huang, and D. Liu, “Coded aperture compression temporal imaging based on a dual-mask and deep denoiser,” J. Opt. Soc. Am. A40, 1468–1477 (2023)

    2023

  18. [18]

    Dual-optical-path coded aperture compressive temporal imaging,

    G. Wang, X. Liu, and X. Yuan, “Dual-optical-path coded aperture compressive temporal imaging,” Opt. Lett.50, 1865–1868 (2025)

    2025

  19. [19]

    Fourier transform photography: a new method for x-ray astronomy,

    J. Ables, “Fourier transform photography: a new method for x-ray astronomy,” Publ. Astron. Soc. Aust.1, 172–173 (1968)

    1968

  20. [20]

    Scatter-hole cameras for x-rays and gamma rays,

    R. Dicke, “Scatter-hole cameras for x-rays and gamma rays,” Astrophys. Journal, vol. 153, p. L101153, L101 (1968)

    1968

  21. [21]

    Coded aperture imaging with uniformly redundant arrays,

    E. E. Fenimore and T. M. Cannon, “Coded aperture imaging with uniformly redundant arrays,” Appl. Opt.17, 337–347 (1978)

    1978

  22. [22]

    Image and depth from a conventional camera with a coded aperture,

    A. Levin, R. Fergus, F. Durand, and W. T. Freeman, “Image and depth from a conventional camera with a coded aperture,” ACM Trans. on Graph. (TOG)26, 70–es (2007)

    2007

  23. [23]

    Coded aperture pairs for depth from defocus,

    C. Zhou, S. Lin, and S. Nayar, “Coded aperture pairs for depth from defocus,” in2009 IEEE 12th International Conference on Computer Vision,(IEEE, 2009), pp. 325–332

    2009

  24. [24]

    Deeply coded aperture for lensless imaging,

    R. Horisaki, Y. Okamoto, and J. Tanida, “Deeply coded aperture for lensless imaging,” Opt. Lett.45, 3131–3134 (2020)

    2020

  25. [25]

    Toward all-in-focus lensless imaging with full-aperture radial masks,

    J. R. C. S. A. Silva Neto, H. Kawachi, Y. Yagi, and T. Nakamura, “Toward all-in-focus lensless imaging with full-aperture radial masks,” Opt. Express33, 48112–48129 (2025)

    2025

  26. [26]

    Adam: A Method for Stochastic Optimization

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” ArXiv Prepr. ArXiv:1412.6980 (2014)

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  27. [27]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing13, 600–612 (2004)

    2004

  28. [28]

    Structure-from-motion revisited,

    J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,(2016), pp. 4104–4113

    2016

  29. [29]

    Learned phase coded aperture for the benefit of depth of field extension,

    S. Elmalem, R. Giryes, and E. Marom, “Learned phase coded aperture for the benefit of depth of field extension,” Opt. Express26, 15316–15331 (2018)

    2018

  30. [30]

    Learned rotationally symmetric diffractive achromat for full-spectrum computational imaging,

    X. Dun, H. Ikoma, G. Wetzstein,et al., “Learned rotationally symmetric diffractive achromat for full-spectrum computational imaging,” Optica7, 913–922 (2020)

    2020

  31. [31]

    End-to-endcomputationalopticswithasingletlensforlargedepth-of-fieldimaging,

    Y.Liu,C.Zhang,T.Kou,et al.,“End-to-endcomputationalopticswithasingletlensforlargedepth-of-fieldimaging,” Opt. Express29, 28530–28548 (2021)

    2021

  32. [32]

    Hybrid diffractive optics design via hardware-in-the-loop methodology for achromatic extended-depth-of-field imaging,

    S. Pinilla, S. R. Miri Rostami, I. Shevkunov,et al., “Hybrid diffractive optics design via hardware-in-the-loop methodology for achromatic extended-depth-of-field imaging,” Opt. Express30, 32633–32649 (2022)

    2022

  33. [33]

    Optimizing wavefront coding for extended depth of field: a synchronous algorithm for optical element and decoding optimization,

    Y. Li, Y. Lyu, J. Wang,et al., “Optimizing wavefront coding for extended depth of field: a synchronous algorithm for optical element and decoding optimization,” Opt. Lett.48, 5847–5850 (2023)

    2023

  34. [34]

    Extended depth of field through wave-front coding,

    E. R. Dowski Jr and W. T. Cathey, “Extended depth of field through wave-front coding,” Appl. Opt.34, 1859–1866 (1995)

    1995

  35. [35]

    High-efficiency rotating point spread functions,

    S. R. P. Pavani and R. Piestun, “High-efficiency rotating point spread functions,” Opt. Express16, 3484–3489 (2008)

    2008

  36. [36]

    Depth estimation and image recovery using broadband, incoherent illumination with engineered point spread functions,

    S. Quirin and R. Piestun, “Depth estimation and image recovery using broadband, incoherent illumination with engineered point spread functions,” Appl. Opt.52, A367–76 (2013)

    2013

  37. [37]

    Optimized asymmetrical tangent phase mask to obtain defocus invariant modulation transfer function in incoherent imaging systems,

    V. N. Le, S. Chen, and Z. Fan, “Optimized asymmetrical tangent phase mask to obtain defocus invariant modulation transfer function in incoherent imaging systems,” Opt. Lett.39, 2171–2174 (2014)

    2014