{"id":"983bdcf6-3fe7-496f-bd5b-0d7160cf8c42","arxiv_id":"2412.07262","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Feeding real smartphone Lidar depth through lightweight adapters improves deblurring PSNR by up to 2.10 dB on simulated ARKitScenes blur.","lead":"This paper tests whether depth maps from a smartphone Lidar can help neural networks remove blur from photos. It adds small trainable adapters to existing deblurring models so they can use depth information, and reports sharper edges on simulated blur from the ARKitScenes dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported depth gains are confounded with added adapter capacity: no control with uninformative depth is included, so the causal role of depth content in the +2.10 dB improvement is not yet established.","rationale":"The reader's CONDITIONAL verdict is appropriate, but the weakest point is not only the synthetic-blur external validity; it is the absence of a control that isolates depth content from the added adapter architecture. The paper's strongest claim is causal: 'utilizing true depth information can significantly boost.' To support that, the comparison must vary only the information content of the depth signal while holding everything else fixed. The current tables vary architecture and parameters at the same time. Table I shows a parameter increase; Table II compares Lidar depth vs estimated depth, both with the adapter; and Table IV compares fusion designs, not informative vs uninformative depth. A constant-depth or shuffled-depth control would directly test whether the adapter extracts useful guidance from depth evidence. This is a feasible experiment for the authors, unlike a full real-blur dataset collection, and it is a standard confound check. The NAFNet row suggests capacity alone is not the whole story, but it does not prove the result; the controlled test does. The reader's focus on real-world blur is also valid, but that concern is about generality, while the constant-depth control is about the interpretation of the existing numbers. Thus, I would keep the verdict CONDITIONAL, with the revision request to add the uninformative-depth control.","tokens_in":14521,"tokens_out":8913,"duration_ms":94101,"concrete_test":"Retrain Depth-Restormer and Depth-Stripformer under the exact Section IV-A protocol with the same adapter and training schedule, but replace the Lidar depth map with (a) a constant depth map set to the dataset mean depth, and (b) a spatially shuffled version of the true depth map that preserves local depth statistics but destroys correspondence with image edges. Compare PSNR/SSIM/LPIPS against Table I. If either control reproduces the majority of the reported gain (e.g., >=1.5 dB for Restormer), the gain is not specifically due to depth information and the claim should be weakened; if both controls stay within a small margin of the no-depth baseline (34.52 dB for Restormer, 35.17 dB for Stripformer), the causal role of depth content is supported. Report parameter counts for the controls to confirm capacity is matched.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that true depth information boosts deblurring rests on comparisons in Tables I–IV where 'None' is the original architecture without adapters and without depth. For Depth-Restormer the comparison is Restormer (26.1M params) vs Depth-Restormer (30.0M params): the +2.10 dB gain includes a 15% parameter increase plus new depth-super-resolution and adapter modules. No experiment keeps the adapter and training protocol identical but supplies an uninformative depth signal (constant depth, scene-shuffled depth, or noise), so the improvement cannot yet be attributed to depth content rather than to additional capacity or to a learned feature-refinement module that would help even with garbage input. The NAFNet result (+0.04 dB with +6.6M params) makes pure-capacity explanations less likely but does not substitute for the missing control. Likewise, the 'None' row in Table II conflates depth absence with architecture absence. This is an internal-validity gap: the abstract's causal wording ('utilizing true depth information can significantly boost') is stronger than the controlled evidence supports.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method to integrate low-resolution mobile Lidar depth maps into existing neural image-deblurring models. The core ingredients are a lightweight depth super-resolution network that maps iPad-Lidar depth to the image resolution while preserving edges, a set of neural adapters that modulate decoder features with depth features in a way inspired by the guided filter, and a continual-learning strategy that freezes the encoder of a pretrained deblurring network and trains the decoder plus adapters. Experiments are run on a subset of ARKitScenes with real iPad Lidar depth and blur simulated by convolution with Levin et al. kernels. The method is applied to Restormer, Stripformer, DeblurDiNATL, and NAFNet, with reported PSNR gains of +2.10, +1.17, +0.19, and +0.04 dB, respectively. Ablations address the value of neural super-resolution over bicubic interpolation, the comparison between Lidar depth and depth estimated from the blurred image, the adapter versus input concatenation, and the choice of freezing the encoder.","tokens_in":14625,"tokens_out":4691,"duration_ms":49725,"significance":"If the central claim holds, the paper would provide useful evidence that an independent, blur-free depth measurement from a smartphone-class Lidar can regularize neural deblurring, which is a novel and practically relevant direction. The approach is architecture-agnostic and is demonstrated on four recent deblurring models, which is a genuine strength. The use of real consumer-Lidar depth data from ARKitScenes, rather than synthetic depth, is also commendable, as is the inclusion of ablations that isolate the super-resolution, fusion, and training-strategy choices. However, the evidence base is narrower than the abstract suggests: the blur is synthetic and spatially invariant, the causal role of depth content versus added network capacity is not established, there are no error bars or statistical tests, and there is no real-blur validation. These gaps prevent the paper from fully supporting its stated conclusions, but they are addressable with additional experiments, so the contribution is potentially sound rather than fundamentally flawed.","major_comments":[{"comment":"The comparison that supports the central claim (\"None\" vs. \"Depth-*\") changes both the input modality and the network architecture at the same time. For Depth-Restormer, the parameter count rises from 26.1M to 30.0M, and for Depth-NAFNet from 17.1M to 23.7M, while the training protocol also changes through the addition of adapter modules and a depth-super-resolution network. The reported gain of +2.10 dB for Depth-Restormer therefore conflates the effect of depth content with the effect of added capacity and of a learned feature-refinement module that might improve performance even with an uninformative depth input. A control experiment that keeps the adapter and training protocol identical but supplies an uninformative depth signal (e.g., a constant map, Gaussian noise, or scene-shuffled depth) is needed to attribute the gain to depth content. The NAFNet result (+0.04 dB with +6.6M parameters) makes a pure-capacity explanation less likely, but it does not substitute for the missing control.","section":"§IV-A, Table I"},{"comment":"All quantitative evidence is obtained on synthetically blurred images: sharp ARKitScenes frames are convolved with a set of standard benchmark kernels following Levin et al. This is a spatially invariant, linear degradation, whereas the motivating application is handheld motion blur in low light, which is typically spatially varying and accompanied by sensor noise and rolling-shutter artifacts. The abstract claims validation on \"a dataset with real-world depth data,\" but the depth is real while the blur is not. To support the practical claim, the authors should add an evaluation on real blurred images with associated Lidar depth, or at least on a realistic blur benchmark (e.g., GoPro-style frame averaging or a recent blur-synthesis method), and report whether the depth-guided gains persist. Section V mentions limitations of the Lidar sensor but does not test them experimentally.","section":"§IV-A"},{"comment":"All reported PSNR/SSIM/LPIPS values come from a single run on a validation set of 500 randomly sampled pairs, with no error bars, no multiple seeds, and no statistical significance tests. Differences such as +0.04 dB (Depth-NAFNet) and +0.19 dB (Depth-DeblurDiNATL) may be within run-to-run variation, and the \"+0.0012\" delta for DeblurDiNATL in Table I is actually an LPIPS increase (worse). The paper should report mean and standard deviation over at least three independent training runs, or per-image confidence intervals, so that the smaller gains can be assessed. This is particularly important because the paper's main claim is a causal one about the utility of depth, not merely a qualitative architectural comparison.","section":"§IV-C, Tables I–IV"},{"comment":"The proposed adapter mechanism relies on the assumption that the depth map and the RGB image are accurately registered and that invalid depth pixels are filled without introducing bias. The paper states that invalid pixels are discarded during depth-super-resolution pretraining but does not describe the alignment preprocessing between the iPad Lidar depth and the RGB camera, nor does it report any analysis of depth-RGB misalignment or its effect on deblurring. Since the adapters are designed to align depth edges with image edges, this is a load-bearing assumption. The authors should add a sensitivity experiment with, for example, small translations or rotations of the depth map relative to the RGB image, to quantify the robustness of the method to registration errors.","section":"§III-B2, §IV-A"}],"minor_comments":[{"comment":"There is a citation inconsistency: Restormer is cited as [26] in Section II-A but as [37] in Section IV-A and Table I. Reference [37] in the bibliography is a different paper (Zhong et al., video deblurring).","section":"§II-A, §IV-A"},{"comment":"The LPIPS deltas for DeblurDiNATL and NAFNet are reported as positive numbers although the depth-enhanced models have higher (worse) LPIPS values. The sign should be negative, or the table should indicate that lower is better and deltas reflect an increase.","section":"Table I"},{"comment":"Several typos should be corrected: \"belurring\" (Section II-A), \"inpaiting\" (Section III-B1), \"demostrate\" (Section IV-B), and \"finetuning\" (used inconsistently as \"fine-tuning\").","section":"Throughout"},{"comment":"The description of the depth super-resolution pretraining omits important details: the input and output resolutions are not explicitly stated, and it is unclear at which feature resolution the last projection layer is removed. Please specify the architecture dimensions and the resolution at which deep features are extracted.","section":"§III-B1"},{"comment":"The term \"continual learning\" is used loosely; the proposed method freezes the encoder and trains the decoder and adapters, which is closer to standard fine-tuning or parameter-efficient adaptation. The authors should clarify the relationship to continual learning and, if they intend it as continual learning, describe how catastrophic forgetting is avoided beyond the partial freezing.","section":"§II-B"},{"comment":"The architecture diagram is dense and the text labels are small; consider enlarging the figure or splitting it into panels for the depth block and the adapter so that the flow of depth features and image features is clearer.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the problem is timely. The main technical direction is original to the best of my knowledge, and the use of real consumer Lidar depth is a positive feature. However, the absence of code and the lack of a capacity-matched control, error bars, and real-blur validation are significant reproducibility and internal-validity concerns. I recommend major revision rather than rejection because the identified issues are addressable with additional experiments within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is the first paper I know to use real smartphone/iPad Lidar depth to condition neural deblurring. That alone makes it worth a look. The adapter design is sensible, the depth super-resolution is well motivated, and the authors test four different backbones rather than just one. The ablations comparing neural SR vs bicubic, and real Lidar vs estimated depth, are informative and give the paper more substance than the average 'add a modality' submission.\n\nThe soft spots are real, though not fatal. The headline claim is causal — true depth helps deblurring — but the evidence is entirely on simulated blur (Levin kernels on ARKitScenes sharp frames). Real handheld motion blur is spatially varying and often noisier, so the +2.10 dB on synthetic blur may not transfer. Second, the stress-test is right: the depth-augmented models add parameters and new modules, and there is no control with uninformative depth (constant, shuffled, or noise). So part of the gain could come from capacity, not from depth content. The NAFNet result (+0.04 dB) makes a pure-capacity story less likely, and the depth-resolution ablation (bicubic vs NN×4 vs NN×8 vs HR) does show that better depth helps. But a constant-depth control would have settled it. Also no error bars, no code, and one small citation error: Restormer is [26] in the text but [37] in Table I / Section IV-A. Minor.\n\nThe limitations paragraph is honest about flat surfaces and range, which I appreciate.\n\nBottom line: the paper's narrow claim — that mobile Lidar depth can be fused into existing deblurring networks with a modest adapter and give a real improvement on simulated blur — is supported. The broad claim in the abstract overstates the evidence. This should get peer review with requests for a real-blur validation or at least a clear discussion of why synthetic blur is sufficient, plus the uninformative-depth control. The method is novel enough and the experiments careful enough that it deserves a shot.","headline":"First real-Lidar deblurring study with a sensible adapter and informative ablations, but the causal claim outruns the synthetic-blur evidence and the missing uninformative-depth control.","tokens_in":15267,"tokens_out":5256,"would_cite":true,"duration_ms":48842,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Depth maps from a smartphone Lidar, fused through lightweight adapters, measurably improve neural image deblurring, with Depth-Restormer gaining 2.10 dB PSNR on ARKitScenes.","keywords":["image deblurring","Lidar depth","depth-guided deblurring","continual learning","multimodal fusion","depth super-resolution","adapters","ARKitScenes"],"falsifier":"Capture real low-light handheld-blurred photos with a phone that also records Lidar depth (using a tripod sharp reference or a burst-aligned reference), run Depth-Restormer against Restormer on those frames, and check whether the reported 2.10 dB PSNR advantage survives; if it collapses, the synthetic-blur premise is the breaking point.","tokens_in":14222,"feed_emoji":"📸","tokens_out":11693,"duration_ms":97036,"temperature":0.7,"pith_summary":"This paper asks whether the depth maps produced by the small Lidar sensors now found in smartphones can improve the quality of neural image deblurring. The authors argue that they can, because Lidar measures scene geometry actively and is therefore immune to the motion blur that corrupts the RGB exposure. They build a generic recipe: take any state-of-the-art encoder-decoder deblurring model, freeze its encoder, attach lightweight depth adapters in the decoder that modulate image features with depth features, and train the adapters plus decoder on paired RGB-depth data. On ARKitScenes frames blurred with standard benchmark kernels, the depth-enhanced models consistently beat their RGB-only baselines, with Depth-Restormer gaining 2.10 dB PSNR. The claim matters because it identifies a practical, already-deployed sensor as a way to regularize an ill-posed inverse problem without retraining large models from scratch.","feed_headline":"Smartphone Lidar depth boosts deblurring by 2.1 dB","feed_subtitle":"Universal depth adapters turn any pretrained deblurring network into a depth-guided one, tested on real iPad Lidar data.","key_machinery":"The machinery has three pieces. The depth super-resolution block is a small network that upsamples the mobile Lidar map (e.g., $256\\times192$) to full image resolution while preserving sharp, piecewise-constant depth edges; it is pretrained on ARKitScenes pairs of iPad Lidar and Faro Focus S70 depth, and its final projection layer is removed so it outputs deep depth features rather than a pixel map. The depth adapter sits at each decoder scale of the pretrained model: it processes depth features with a convolutional attention that mimics the second-order statistics of the guided filter, passes them through a sigmoid, multiplies them onto the image features, and concatenates the conditioned features back through a lightweight transformer block. The continual learning protocol freezes the encoder of the deblurring model, keeps the decoder trainable, and trains the adapters on paired multimodal data, requiring only a small update to $\\theta_g$ and learning the new $\\theta_h$. Together these components implement the factorized estimator $f_\\theta(y,d)=g_{\\theta_g}(y)\\circ h_{\\theta_h}(d)$, making the depth-guidance mechanism portable to any encoder-decoder restoration network.","core_discovery":"The central claim is that an independent, blur-free depth measurement from a mobile Lidar—even at the low $256\\times192$ resolution of the iPad Pro sensor—carries genuine regularizing information for image deblurring, and that this information can be injected into any state-of-the-art neural deblurring model through a universal adapter. The paper demonstrates this by factorizing the joint estimator as $f_\\theta(y,d)=g_{\\theta_g}(y) \\circ h_{\\theta_h}(d)$, where the unimodal deblurrer $g_{\\theta_g}(y)$ is kept largely pretrained and the depth processor $h_{\\theta_h}(d)$ is learned. The depth branch super-resolves the Lidar map with a network trained to preserve piecewise-constant edges, then uses convolutional attention adapters that generalize the classical guided filter to modulate decoder features at multiple scales. On ARKitScenes with simulated blur, depth-enhanced models outperform their baselines across four architectures, and $8\\times$ super-resolved mobile depth matches the guidance quality of a high-end Faro Focus S70 Lidar. An ablation shows real Lidar depth (+2.10 dB) beats depth estimated from the blurry image itself (+1.27 dB), supporting the argument that true independent side information, not just extra image features, drives the gain.","pith_inferences":["Because all quantitative results use synthetic blur applied to sharp ARKitScenes frames, the 2.10 dB advantage has not yet been demonstrated on real handheld motion blur; a natural next step is a real low-light capture experiment with paired depth.","The architecture-agnostic design suggests the same adapter recipe could extend Lidar guidance to other restoration tasks such as denoising, super-resolution, or video deblurring, where an independent depth stream is available.","The paper's own limitation note predicts that gains should shrink on flat, textureless surfaces and on distant scenes; this could be tested by stratifying deblurring results by scene depth range and edge density.","The reported sensitivity to depth-RGB misalignment is not quantified; a practical robustness test would jitter the depth map by a few pixels and measure how quickly the PSNR advantage degrades."],"forward_implications":["Any encoder-decoder deblurring model, including Restormer, Stripformer, NAFNet, and DeblurDiNAT, can be converted into a depth-guided version with a few extra parameters and a small training budget, without training from scratch.","Mobile Lidar depth, after learned $8\\times$ super-resolution, provides guidance equivalent to a high-end Faro Focus S70 Lidar for deblurring, at 36.62 versus 36.59 dB PSNR.","Depth estimated from the blurry image itself helps less than true Lidar depth (+1.27 versus +2.10 dB), suggesting that independent geometric side information, not just extra image features, is the source of the improvement.","Freezing the encoder while training the decoder and adapters outperforms full fine-tuning by 0.45 dB, so the continual-learning recipe is the effective way to add the modality.","The gain is not uniform: it is large for Restormer, modest for DeblurDiNAT, and nearly null for NAFNet, so depth guidance is most valuable when the base model is not already near its performance ceiling."],"supporting_citations":[{"why":"supplies the ARKitScenes dataset with paired iPad Lidar depth, high-end Faro depth, and sharp RGB frames used for training and evaluation.","marker":"[39]"},{"why":"provides the standard benchmark blur kernels used to simulate blur on the sharp ARKitScenes frames.","marker":"[36]"},{"why":"defines the Restormer architecture, the primary base model adapted into Depth-Restormer.","marker":"[26]"},{"why":"defines the NAFNet baseline, one of the four state-of-the-art models tested with and without depth.","marker":"[27]"},{"why":"defines the Stripformer baseline, another of the four models tested with and without depth.","marker":"[30]"},{"why":"defines the DeblurDiNAT baseline, the fourth model tested with and without depth.","marker":"[57]"},{"why":"provides the guided filter whose second-order-statistics mechanism the adapters generalize.","marker":"[17]"},{"why":"supplies the Depth Anything model used to estimate depth from blurry images in the comparison ablation.","marker":"[59]"},{"why":"offers the depth-guided deblurring approach that estimates depth from the blurry image, the prior work the authors compare against.","marker":"[32]"}],"fun_headline_variants":["Lidar depth sharpens phone photos: +2.1 dB","Depth-aware deblurring: 2.1 dB gain from Lidar","Phone Lidar depth guides neural deblurring to +2.1 dB","Universal depth adapter lifts deblurring by 2.1 dB"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All measured gains come from synthetic blur applied to sharp ARKitScenes frames, so the paper assumes real handheld blur behaves like those benchmark kernels and that the iPad Lidar depth is accurately registered and filled without bias.","fun_headline_variants_meta":{"raw":{"variants":["Lidar depth sharpens phone photos: +2.1 dB","Depth-aware deblurring: 2.1 dB gain from Lidar","Phone Lidar depth guides neural deblurring to +2.1 dB","Universal depth adapter lifts deblurring by 2.1 dB"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000754,"raw_usage":{"total_tokens":3380,"prompt_tokens":995,"completion_tokens":2385,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":2302}},"tokens_in":611,"tokens_out":2385,"duration_ms":39580,"temperature":1.0,"reasoning_tokens":2302,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:55:50.309085+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Capture real low-light handheld-blurred photos with a phone that also records Lidar depth (using a tripod sharp reference or a burst-aligned reference), run Depth-Restormer against Restormer on those frames, and check whether the reported 2.10 dB PSNR advantage survives; if it collapses, the synthetic-blur premise is the breaking point.","supporting_citations":[{"cited_title":"ARKitscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data,","cited_arxiv_id":null,"evidence_quote":"supplies the ARKitScenes dataset with paired iPad Lidar depth, high-end Faro depth, and sharp RGB frames used for training and evaluation."},{"cited_title":"Understanding and evaluating blind deconvolution algorithms,","cited_arxiv_id":null,"evidence_quote":"provides the standard benchmark blur kernels used to simulate blur on the sharp ARKitScenes frames."},{"cited_title":"Simple baselines for image restoration,","cited_arxiv_id":null,"evidence_quote":"defines the NAFNet baseline, one of the four state-of-the-art models tested with and without depth."},{"cited_title":"Stripformer: Strip transformer for fast image deblurring,","cited_arxiv_id":null,"evidence_quote":"defines the Stripformer baseline, another of the four models tested with and without depth."},{"cited_title":"Deblurdinat: A lightweight and effective transformer for image deblurring,","cited_arxiv_id":null,"evidence_quote":"defines the DeblurDiNAT baseline, the fourth model tested with and without depth."},{"cited_title":"Guided image filtering,","cited_arxiv_id":null,"evidence_quote":"provides the guided filter whose second-order-statistics mechanism the adapters generalize."},{"cited_title":"Dynamic scene deblurring by depth guided model,","cited_arxiv_id":null,"evidence_quote":"offers the depth-guided deblurring approach that estimates depth from the blurry image, the prior work the authors compare against."}],"review_version":1}