REVIEW 4 major objections 4 minor 34 references
A zero-shot diffusion method deblurs smartphone photos using Lidar depth maps, beating state-of-the-art on a new real-image benchmark.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 19:25 UTC pith:SXC6TQR3
load-bearing objection The dataset is the real contribution; the zero-shot method is plausible but the SOTA claim rests on thin, alignment-sensitive evaluation. the 4 major comments →
A novel method and dataset for depth-guided image deblurring from smartphone Lidar
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that ZSLDB, a zero-shot deblurring method using a pretrained Stable Diffusion model conditioned on Lidar depth through ControlNet, outperforms state-of-the-art blind deblurring methods on real smartphone images in terms of perceptual quality (LPIPS). On their new dataset, ZSLDB scores 0.1643 LPIPS versus 0.1884 for DeblurGAN, 0.1783 for Restormer, and 0.1819 for J-MKPD. Ablations show that depth guidance improves quality over the no-depth variant (0.1821) and that using the raw depth map is more effective than using an edge map extracted from it (0.1682), suggesting depth gradients carry additional useful information beyond object boundaries.
What carries the argument
The key mechanism is a latent diffusion model (Stable Diffusion) combined with a ControlNet subnetwork that takes the Lidar depth map as conditioning side information. ZSLDB first inverts the blurred image into the diffusion latent space using DDIM, then optimizes the latent and an estimated blur kernel by backpropagating through a short DDIM sampling chain, measuring consistency with the blurred observation via L2 loss in the VAE latent space plus LPIPS and an aesthetic reward. This allows blind zero-shot deblurring that leverages depth without any paired training data.
Load-bearing premise
The central claim rests on the assumption that the blurred images, Lidar depth maps, and sharp ground truth are registered well enough that LPIPS differences reflect true deblurring quality rather than alignment artifacts.
What would settle it
A concrete check would be to compute pixel-level alignment error between the depth maps, blurred images, and ground truth (e.g., using optical flow or manual correspondence) and to re-evaluate LPIPS on tightly aligned crops. If the LPIPS gap between ZSLDB and baselines shrinks or reverses after alignment correction, the claimed benefit of Lidar guidance would be called into question. Alternatively, testing on synthetic blurred images with known kernels and perfect registration would directly validate whether the optimization recovers the true kernel and sharp image.
If this is right
- Smartphone Lidar depth can serve as a practical conditioning signal for image restoration, improving perceptual quality on real low-light motion blur.
- Zero-shot diffusion approaches can avoid the need for large paired datasets of blurred images and depth maps, reducing the domain-gap problem of synthetic training data.
- The new dataset provides a benchmark with real blurred images, registered Lidar depth, and sharp ground truth, enabling future research on depth-guided deblurring.
- Depth guidance is more effective than edge guidance, indicating that full depth gradients regularize the deblurring problem better than binary boundaries alone.
Where Pith is reading between the lines
- The method is likely extensible to other side-information modalities (e.g., normal maps, segmentation, or multi-spectral cues) using the same ControlNet conditioning and latent optimization framework.
- Because the evaluation relies on LPIPS rather than PSNR, the reported gains may partly reflect perceptual alignment rather than pixel-accurate restoration; a more complete comparison would include alignment-corrected PSNR or human studies.
- The zero-shot nature suggests the approach could generalize to unseen camera and blur characteristics, provided the pretrained diffusion prior matches the input domain; testing on additional devices would clarify this.
- The dataset's 45 indoor scenes, mostly in Lidar range, may favor depth guidance; extending to outdoor or long-range scenes would test whether the benefit persists when depth boundaries are sparse or noisy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ZSLDB, a zero-shot blind image deblurring method that uses a pretrained Stable Diffusion model with a ControlNet branch conditioned on smartphone Lidar depth maps. The method formulates deblurring as optimization over a latent code and an unknown blur kernel, with data fidelity measured in VAE latent space and with LPIPS, plus an aesthetic reward regularizer. The paper also introduces a new dataset of 45 real low-light motion-blurred smartphone images with paired Lidar depth maps and sharp ground truths, acquired with an iPhone 15 Pro. In experiments, ZSLDB reports LPIPS 0.1643, outperforming DeblurGAN (0.1884), Restormer (0.1783), and J-MKPD (0.1819). An ablation without depth guidance gives 0.1821, and an edge-map-conditioned variant gives 0.1682. The authors conclude that Lidar depth guidance improves perceptual quality and that ZSLDB achieves state-of-the-art results on real data.
Significance. If the claims hold, the paper makes two useful contributions: it provides the first real-world smartphone dataset pairing motion-blurred images, Lidar depth maps, and sharp ground truth, and it demonstrates a training-free way to inject depth side information into a diffusion-based deblurring pipeline. The zero-shot formulation avoids the expensive paired-data requirement of prior supervised Lidar-guided deblurring, and the code/dataset release is a concrete asset for the community. However, the central empirical claims rest on a single perceptual metric over 45 scenes, and the paper itself flags the alignment problem that makes that metric difficult to interpret. The qualitative and architectural ideas are promising, but the evaluation needs substantial strengthening before the state-of-the-art claim can be accepted.
major comments (4)
- [§5.1, Table 1 and dataset description in §4] The paper explicitly states that 'imperfect alignment at a single-pixel level' makes PSNR unreliable, and therefore uses LPIPS as the sole quantitative metric. However, no alignment-error analysis is provided. The blurred and sharp images are separate exposures; any residual parallax, small camera motion, or depth-map misregistration will produce systematic spatial shifts. Since ZSLDB conditions on a depth map that is registered to the blurred frame, its output edges may align to a different coordinate frame than the sharp ground truth, while baselines producing softer edges may be less penalized by misalignment. The reported LPIPS gap between ZSLDB and Restormer is 0.014, and the depth ablation gap is 0.018; these are small enough that a 1–2 pixel registration bias could plausibly account for them. The manuscript must provide quantitative alignment analysis (e.g., estimated residual dis
- [§3.2, Eq. (7)] The blur kernel A is central to the 'blind' deblurring claim, but the paper never specifies its parametric form, dimensionality, initialization, or optimization procedure. Is it a single global kernel, or spatially varying? What support size is used? Is it constrained (e.g., nonnegative, sum-to-one)? How is the minimization over A performed jointly with z? Without these details, the method cannot be reproduced and the 'blind' aspect is not substantiated. At minimum, the authors should state the kernel model and the optimization schedule, and ideally visualize estimated kernels on representative scenes.
- [§5.1, Table 1] The quantitative evaluation uses a single metric (LPIPS) on only 45 scenes, with no error bars, confidence intervals, or significance tests. The difference between ZSLDB (0.1643) and the closest baseline Restormer (0.1783) is 0.014; without per-scene paired statistics we cannot know whether this is meaningful. Add a paired significance test (e.g., Wilcoxon signed-rank) and report variance or confidence intervals. Additional perceptual metrics (e.g., DISTS, FID, or a user study) would also strengthen the claim, especially because LPIPS itself is used as a loss term in Eq. (7), creating a favorable-bias concern for the proposed method.
- [§5.1, baselines] The three comparison methods—DeblurGAN, Restormer, and J-MKPD—are used off-the-shelf with no adaptation to real low-light smartphone blur. These models were trained on synthetic or different-domain data, so their LPIPS scores may reflect domain shift rather than deblurring ability. The paper should either fine-tune/adapt the baselines on the new data (e.g., with a small amount of supervised tuning or test-time adaptation), or explicitly discuss and justify that off-the-shelf performance is the intended comparison. As it stands, the state-of-the-art claim is weakened by this asymmetry.
minor comments (4)
- [Abstract and §1] The phrase 'zero-shot' is used in a specific sense (no training with paired blurred/depth/ground-truth data), which is clarified in §5.1. Consider stating this definition earlier to avoid confusion, since the method still relies on pretrained Stable Diffusion and ControlNet.
- [§2] Typographical issues: 'Morevoer' should be 'Moreover', 'denosing' should be 'denoising', and 'V AE' spacing is inconsistent. Also Eq. (5) uses both $\hat\epsilon_t$ and $\epsilon_t$ without explicitly defining $\hat\epsilon_t$; please clarify.
- [§4] The DSLR acquisitions are mentioned as 'coarsely registered' and usable only for distribution-level assessment, but no such assessment is reported. Either add the distribution-level analysis or remove the sentence to avoid promising an evaluation that is not delivered.
- [§5.3, Table 3] The edge-map ablation compares depth conditioning to Canny edges on the depth map. The conclusion that 'depth gradients' provide extra information is plausible but speculative; consider analyzing which regions benefit most (e.g., textureless vs. textured) to support this interpretation.
Circularity Check
No circular derivation: ZSLDB's objective does not coincide with its evaluation metric, and the depth ablation is an empirical comparison on a novel dataset.
full rationale
The paper's central claims are empirical rather than derivationally circular. ZSLDB is defined by the optimization in Eq. (7), which minimizes a VAE-latent L2 fidelity term, an aesthetic reward, and an LPIPS term evaluated between the output and the blurred observation y. The reported headline result in Table 1 is LPIPS between the output and the sharp ground truth. Thus the evaluation metric is not the same object as any term in the objective: LPIPS(y, G(z,d)) is not LPIPS(GT, G(z,d)). The depth ablation (Table 2) compares the same pipeline with and without the ControlNet branch, so the improvement attributed to depth is a measured difference, not an identity forced by construction. Reference [2] shares an author with the present paper and is used as motivation, but it is not load-bearing: no theorem, uniqueness claim, or ansatz is imported from [2] to justify the central result. The paper does note a real limitation—'imperfect alignment at a single-pixel level'—and therefore chooses LPIPS over PSNR; this is a validity concern about the benchmark, not a circular step. The lack of an external benchmark and the authors' own dataset are independence and reproducibility limitations, but they do not make the reported numbers equivalent to the method's inputs. No equation in the paper reduces to its own input, and no fitted parameter is renamed as a prediction.
Axiom & Free-Parameter Ledger
free parameters (4)
- gamma =
0.1
- lambda =
1.5
- blur kernel A parametrization =
unspecified
- optimization schedule =
unspecified
axioms (5)
- domain assumption The forward degradation is a linear operator: y = Ax + n (Eq. 1).
- domain assumption DDIM inversion of a blurred image into Stable Diffusion's latent space is a valid starting point for deblurring.
- ad hoc to paper Pretrained Stable Diffusion and ControlNet provide a general sharp-image prior for real low-light smartphone images.
- ad hoc to paper The aesthetic reward model from [26] is a useful regularizer for perceptual quality.
- domain assumption Depth guidance through ControlNet improves boundary sharpness in the restored image.
Cite this review
Pith. "Pith review of A novel method and dataset for depth-guided image deblurring from smartphone Lidar." pith.science (2026). https://pith.science/paper/SXC6TQR3
@misc{pith2026250909241,
author = {Pith},
title = {Pith review of: A novel method and dataset for depth-guided image deblurring from smartphone Lidar},
year = {2026},
howpublished = {\url{https://pith.science/paper/SXC6TQR3}},
note = {Machine review of arXiv:2509.09241}
}
read the original abstract
Modern smartphones are equipped with Lidar sensors providing depth-sensing capabilities. Recent works have shown that this complementary sensor allows to improve various tasks in image processing, including deblurring. However, there is a current lack of datasets with realistic blurred images and paired mobile Lidar depth maps to further study the topic. At the same time, there is also a lack of blind zero-shot methods that can deblur a real image using the depth guidance without requiring extensive training sets of paired data. In this paper, we propose an image deblurring method based on denoising diffusion models that can leverage the Lidar depth guidance and does not require training data with paired Lidar depth maps. We also present the first dataset with real blurred images with corresponding Lidar depth maps and sharp ground truth images, acquired with an Apple iPhone 15 Pro, for the purpose of studying Lidar-guided deblurring. Experimental results on this novel dataset show that Lidar guidance is effective and the proposed method outperforms state-of-the-art deblurring methods in terms of perceptual quality.
Reference graph
Works this paper leans on
-
[1]
AI-powered LIDAR fusion for next-generation smartphone cameras (LICAM)
INTRODUCTION Modern smartphones are increasingly becoming multimodal imaging devices with the inclusion of Lidar sensors in recent consumer devices, such as the Apple iPhone. Despite its lim- ited resolution due to space and cost constraints, the active Lidar instrument can serve as complementary source of infor- mation to passive optical cameras, even in...
Pith/arXiv arXiv 2022
-
[2]
Several solutions have been developed over the years, including optimization- based methods carefully modeling priors suitable to describe natural images
BACKGROUND Image restoration, including deblurring, is a longstanding problem in the image processing field. Several solutions have been developed over the years, including optimization- based methods carefully modeling priors suitable to describe natural images. Deep learning has then shown that neural networks could effectively capture more sophisticate...
-
[3]
METHOD The overall goal of this paper is to investigate the problem of image deblurring from smartphone cameras, when side infor- mation in the form of a depth map acquired by the Lidar on the same device is available. Compared to [2] that worked on synthetically degraded data, we focus on real data by both devising a novel depth-guided zero-shot deblurri...
-
[4]
DA TASET Current research efforts into the investigation of the use of smartphone Lidar sensors for the regularization of inverse problems in imaging are limited by the lack of available data. In particular, the only existing dataset with paired smartphone images and Lidar depth maps captured by the same device is ARKitScenes [3]. However, this dataset wa...
-
[5]
zero-shot
EXPERIMENTAL RESULTS 5.1. Experimental setting and main result In this section we present deblurring results using the new dataset presented in Sec. 4. We remark that since our focus is on real images and Lidar depth maps, this is the only existing dataset that allows such investigation. The proposed ZSLDB uses a pretrained Stable Diffusion with ControlNe...
-
[6]
CONCLUSIONS We presented an investigation into image deblurring guided by smartphone Lidar depth information with a focus on two contributions: the first dataset of real blurred smartphone pho- tos with registered depth information and a novel method for zero-shot Lidar-guided deblurring based on denoising diffu- sion models. Results on the new dataset sh...
-
[7]
Lidar depth map guided image compression model,
Alessandro Gnutti, Stefano Della Fiore, Mattia Savardi, Yi- Hsin Chen, Riccardo Leonardi, and Wen-Hsiao Peng, “Lidar depth map guided image compression model,” in2024 IEEE International Conference on Image Processing (ICIP), 2024, pp. 1890–1896
2024
-
[8]
Deep lidar-guided image deblurring,
Ziyao Yi, Diego Valsesia, Tiziano Bianchi, and Enrico Magli, “Deep lidar-guided image deblurring,”arXiv preprint arXiv:2412.07262, 2024
Pith/arXiv arXiv 2024
-
[9]
ARKitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data,
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman, “ARKitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data,” inThirty-fifth Conference on Neu- ral Information Processing Systems Datasets and Benchmarks Trac...
2021
-
[10]
Plug-and-play methods for integrating physical and learned models in computational imaging: The- ory, algorithms, and applications,
Ulugbek S Kamilov, Charles A Bouman, Gregery T Buzzard, and Brendt Wohlberg, “Plug-and-play methods for integrating physical and learned models in computational imaging: The- ory, algorithms, and applications,”IEEE Signal Processing Magazine, vol. 40, no. 1, pp. 85–97, 2023
2023
-
[11]
Auto-encoding variational bayes,
Diederik P Kingma, “Auto-encoding variational bayes,”arXiv preprint arXiv:1312.6114, 2013
Pith/arXiv arXiv 2013
-
[12]
Generative adversarial nets,
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, “Generative adversarial nets,”Advances in neural information processing systems, vol. 27, 2014
2014
-
[13]
Generative modeling by esti- mating gradients of the data distribution,
Yang Song and Stefano Ermon, “Generative modeling by esti- mating gradients of the data distribution,”Advances in Neural Information Processing Systems, vol. 32, 2019
2019
-
[14]
Restormer: Efficient transformer for high-resolution image restoration,
Syed Waqas Zamir, Aditya Arora, Salman Khan, Mu- nawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang, “Restormer: Efficient transformer for high-resolution image restoration,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5728– 5739
2022
-
[15]
Simple baselines for image restoration,
Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun, “Simple baselines for image restoration,” inEuropean confer- ence on computer vision. Springer, 2022, pp. 17–33
2022
-
[16]
Blind motion deblurring with pixel-wise kernel esti- mation via kernel prediction networks,
Guillermo Carbajal, Patricia Vitoria, Jos ´e Lezama, and Pablo Mus´e, “Blind motion deblurring with pixel-wise kernel esti- mation via kernel prediction networks,”IEEE Transactions on Computational Imaging, 2023
2023
-
[17]
Pulse: Self-supervised photo upsampling via latent space exploration of generative models,
Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin, “Pulse: Self-supervised photo upsampling via latent space exploration of generative models,” inProceed- ings of the ieee/cvf conference on computer vision and pattern recognition, 2020, pp. 2437–2445
2020
-
[18]
Deblurgan: Blind motion deblurring using conditional adversarial networks,
Orest Kupyn, V olodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiri Matas, “Deblurgan: Blind motion deblurring using conditional adversarial networks,” inPro- ceedings of the IEEE conference on computer vision and pat- tern recognition, 2018, pp. 8183–8192
2018
-
[19]
Diffusion posterior sam- pling for general noisy inverse problems,
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye, “Diffusion posterior sam- pling for general noisy inverse problems,”arXiv preprint arXiv:2209.14687, 2022
Pith/arXiv arXiv 2022
-
[20]
Denoising diffusion restoration models,
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song, “Denoising diffusion restoration models,”Advances in Neural Information Processing Systems, vol. 35, pp. 23593– 23606, 2022
2022
-
[21]
Zero-shot image restoration using denoising diffusion null-space model,
Yinhuai Wang, Jiwen Yu, and Jian Zhang, “Zero-shot image restoration using denoising diffusion null-space model,”arXiv preprint arXiv:2212.00490, 2022
Pith/arXiv arXiv 2022
-
[22]
Denoising diffusion models for plug-and-play image restoration,
Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool, “Denoising diffusion models for plug-and-play image restoration,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1219–1229
2023
-
[23]
Solving linear in- verse problems provably via posterior sampling with latent dif- fusion models,
Litu Rout, Negin Raoof, Giannis Daras, Constantine Carama- nis, Alex Dimakis, and Sanjay Shakkottai, “Solving linear in- verse problems provably via posterior sampling with latent dif- fusion models,”Advances in Neural Information Processing Systems, vol. 36, 2024
2024
-
[24]
Solving inverse problems with la- tent diffusion models via hard data consistency,
Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen, “Solving inverse problems with la- tent diffusion models via hard data consistency,”arXiv preprint arXiv:2307.08123, 2023
Pith/arXiv arXiv 2023
-
[25]
Guided image fil- tering,
Kaiming He, Jian Sun, and Xiaoou Tang, “Guided image fil- tering,”IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 6, pp. 1397–1409, 2012
2012
-
[26]
Adding conditional control to text-to-image diffusion models,
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala, “Adding conditional control to text-to-image diffusion models,” inPro- ceedings of the IEEE/CVF International Conference on Com- puter Vision, 2023, pp. 3836–3847
2023
-
[27]
U- net: Convolutional networks for biomedical image segmen- tation,
Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U- net: Convolutional networks for biomedical image segmen- tation,” inMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III
2015
-
[28]
Springer, 2015, pp. 234–241
2015
-
[29]
Denoising Diffusion Implicit Models,
Jiaming Song, Chenlin Meng, and Stefano Ermon, “Denoising Diffusion Implicit Models,” inInternational Conference on Learning Representations, 2020
2020
-
[30]
High-resolution image synthe- sis with latent diffusion models,
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer, “High-resolution image synthe- sis with latent diffusion models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2022, pp. 10684–10695
2022
-
[31]
Neural discrete representation learning,
Aaron Van Den Oord, Oriol Vinyals, et al., “Neural discrete representation learning,”Advances in neural information pro- cessing systems, vol. 30, 2017
2017
-
[32]
The unreasonable effectiveness of deep fea- tures as a perceptual metric,
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang, “The unreasonable effectiveness of deep fea- tures as a perceptual metric,” inProceedings of the IEEE con- ference on computer vision and pattern recognition, 2018, pp. 586–595
2018
-
[33]
Aligning text-to-image diffusion models with reward backpropagation,
Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki, “Aligning text-to-image diffusion models with reward backpropagation,”arXiv preprint arXiv:2310.03739, 2023
Pith/arXiv arXiv 2023
-
[34]
Memory-efficient backpropagation through time,
Audrunas Gruslys, R ´emi Munos, Ivo Danihelka, Marc Lanctot, and Alex Graves, “Memory-efficient backpropagation through time,”Advances in neural information processing systems, vol. 29, 2016
2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.