{"id":"134bce88-4bff-45b1-b3b8-5fee126e25ca","arxiv_id":"2506.14709","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DiFuse-Net pairs an RGB encoder with a dual-pixel encoder that uses windowed bi-directional parallax attention, and contributes a new 5,700-image RGB-DP-D dataset for training and benchmarking.","lead":"DiFuse-Net estimates depth from a normal RGB image plus the dual-pixel (DP) autofocus signal already inside many smartphone cameras, using separate neural encoders and a windowed attention mechanism. It also introduces a new dual-pixel depth dataset created from a two-smartphone stereo rig.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DCDP pseudo ground truth from an undisclosed synthetic-trained stereo estimator is the weakest load-bearing link; without independent validation, Table IV and the DCDP-trained claims are uninterpretable.","rationale":"The reader's weakest-assumption analysis correctly identifies the DCDP pseudo-ground-truth pipeline as the most load-bearing uncertainty. I re-read the dataset generation and experiments sections and found no additional internal inconsistency that changes the verdict. The architecture claim on the Google DP dataset is supported by a plausible evaluation, but the DCDP dataset is one of the paper's five stated contributions and underpins Table IV and the qualitative DCDP-training claims. Because the pseudo-GT estimator is undisclosed and no independent accuracy validation is provided, the conditional verdict is the right call: the paper should be accepted only if the authors release the data, masks, and estimator details, and demonstrate that the pseudo-GT agrees with an independent reference on a held-out subset. I agree with the reader that this is addressable, so no movement to REJECT is warranted.","tokens_in":11182,"tokens_out":10480,"duration_ms":115305,"concrete_test":"Hold out 50-100 DCDP scenes and re-capture them with an independent reference depth sensor (e.g., a calibrated LiDAR/ToF camera or a high-accuracy active stereo system); align the reference to the RGB-DP frame and compute per-pixel inverse-depth error of the pseudo-GT, both inside and outside the manual masks. If the median error on unmasked pixels is comparable to or larger than the DP disparity-induced inverse-depth differences the network is supposed to learn, or if the masked-out fraction is large, then the DCDP benchmark cannot support the claimed improvements and Table IV should be re-run on the independently verified subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DiFuse-Net is superior on DCDP (Table IV) and that DCDP training improves boundary quality (Fig. 12) depends entirely on the pseudo ground-truth depth described in Section IV-C. That GT comes from an AI stereo disparity estimator trained on synthetic data [27]; neither the exact estimator, its training recipe, nor its error rate on real 2.5 cm-baseline smartphone stereo is disclosed. With a 2.5 cm baseline, disparity magnitudes are small, so a stereo model trained on synthetic large-disparity data can easily carry a systematic bias that dominates relative inverse-depth error. The manual masks are described only as visually incorrect regions and are excluded from training loss; if the same selective masks are used in evaluation, the DCDP metrics in Table IV are not full-image accuracy, and if they are not, invalid pseudo-GT regions contaminate the test labels. Either way, the reported gains are not separable from the quality of the undisclosed pseudo-GT. The Google DP results are less affected, but the dataset contribution and all DCDP-based experiments are load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DiFuse-Net, a two-branch encoder-decoder for depth estimation from RGB and dual-pixel (DP) images, with a windowed bi-directional parallax attention module (WBiPAM), a dynamic fusion module, and a three-stage cross-modal transfer learning (CmTL) strategy. The authors also introduce a new real-world RGB-DP-D dataset called DCDP, created with two Galaxy S23 Ultra phones in a symmetric stereo setup, where ground-truth depth is obtained from an AI stereo disparity estimator trained on synthetic data. The method is evaluated on the Google DP dataset and on the DCDP dataset, reporting improved metrics over DPNet, a parameter-matched baseline, a RAFT-stereo-based baseline, and the monocular models MiDaS and ZoeDepth. Ablations analyze the contributions of WBiPAM, fusion granularity, DP encoder depth, and CmTL.","tokens_in":11422,"tokens_out":5745,"duration_ms":58731,"significance":"If the claims are validated, the paper makes a useful contribution: it addresses the scarcity of dense RGB-DP-D training data, proposes a dedicated attention module for the small disparities characteristic of smartphone DP sensors, and demonstrates that a decoupled RGB/DP design can outperform concatenation-based architectures. The comparative ablations and the effort to create a new dataset with a careful capture and rectification protocol are strengths. However, the DCDP ground-truth generation and the benchmarking on it are the central load-bearing pieces, and they currently rely on an undisclosed learned stereo model with only qualitative masking; until that pseudo-ground truth is independently validated and fully disclosed, the dataset contribution and the DCDP results cannot be taken as established.","major_comments":[{"comment":"The DCDP ground-truth depth is generated by an undisclosed \"AI stereo disparity estimation\" model trained on a synthetic dataset [27], with manual masks used to exclude visually incorrect regions from the training loss. The paper does not name the model, specify its training details, quantify its accuracy on real 2.5 cm-baseline stereo images, or state whether the same masks are applied to the DCDP test set used in Table IV. Because a 2.5 cm baseline yields very small disparity magnitudes, a synthetic-trained stereo model can carry a systematic bias that dominates the relative inverse-depth error, so the DCDP metrics in Table IV and the boundary-quality claims in Fig. 12 are not currently interpretable. The authors should disclose the estimator, validate it against an independent depth source (e.g., structure-from-motion or an active sensor), and clarify the exact masking protocol for test labels.","section":"Section IV-C and Table IV"},{"comment":"The DPNet baseline numbers are taken \"from their official GitHub page\" after the authors' \"release of modified train and test datasets,\" but the modification is never described, and no version or date is cited. In addition, all quantitative results in Tables I, II, and IV appear to be single-run numbers with no error bars, multiple seeds, or statistical significance testing; the gap between DiFuse-Net (0.0799) and the Stereo Baseline (0.0911) on the Google DP dataset is small in absolute terms. The authors should specify the exact DPNet variant and dataset split, and they should report variance across at least three training runs or otherwise demonstrate that the reported ranking is robust.","section":"Section VI-A and Table I"},{"comment":"The CmTL description is internally inconsistent: Stage 1 and Stage 2 train the decoder together with the DP and RGB encoders respectively, but Stage 3 states that \"The Fusion module and the Decoder weights are initialized randomly.\" This discards the decoder knowledge learned in Stages 1 and 2, making the purpose of the decoder pretraining unclear and weakening the claimed transfer mechanism. The authors should either reinitialize only the fusion module, justify why the decoder pretraining is unnecessary, or redesign the stage description to match the actual implementation.","section":"Section III-F"},{"comment":"The \"Stereo Baseline\" is introduced as replacing WBiPAM with a traditional stereo matching cost-volume approach using RAFT-Stereo [30], but the paper does not describe how this baseline is adapted to dual-pixel inputs. It is unspecified whether the DP left/right images are fed as a stereo pair, how the network is trained on the Google DP and DCDP data, whether the baseline sees the same training labels, and how the disparity range and vertical/horizontal orientation are handled. Without this information, the comparison against the Stereo Baseline in Tables I and IV cannot be reproduced or properly interpreted.","section":"Section VI-A, Table I (Stereo Baseline)"}],"minor_comments":[{"comment":"The window size k used in WBiPAM is never given a numerical value, nor is there an ablation over k; since k defines the parallax search range and the attention window, the authors should report the chosen value and its sensitivity.","section":"Section III-C"},{"comment":"The caption reads \"DPNet trained (second)\" while subfigures (b) and (c) are described as \"DiFuse-Net trained on our DCDP dataset\" and \"DiFuse-Net trained on Google DP dataset\"; the caption text appears to be a typo.","section":"Fig. 12 caption"},{"comment":"There is a typo: \"captured in the begging of a capture session\" should be \"beginning.\"","section":"Fig. 6 caption"},{"comment":"The method name is \"Cross-modal Transfer Learning\" (CmTL), but Section III-F introduces it as \"cross-model transfer learning\"; the terminology should be made consistent.","section":"Section III-F"},{"comment":"The metrics AIWE 1 and AIWE 2 are used without being defined in the text; the authors should state the formulas or explicitly cite the definitions from [11].","section":"Section V-C"}],"recommendation":"major_revision","confidential_remarks":"The main risk to the manuscript is the DCDP pseudo-ground-truth pipeline: if the AI stereo estimator and masking protocol cannot be validated or disclosed, the dataset contribution and all DCDP-based results should be treated as unverified. I would encourage the editor to request, at a minimum, a description of the estimator's architecture and training data, quantitative validation against independent depth measurements, and clarification of whether masks are applied at test time. The Google DP results alone are promising but the small margins and lack of variance reporting would still need strengthening."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a genuine engineering contribution: a modality-decoupled RGB+DP depth network with a windowed bi-directional parallax attention module (WBiPAM), a three-stage cross-modal transfer learning (CmTL) scheme, and a new 5700-image DCDP dataset captured with a symmetric stereo rig. The architecture is new as a combination, the ablations on the Google DP dataset are clean and support the design choices, and the Google DP results (1-SRCC 0.0799 vs 0.1520 for DPNet) are plausible. The per-session stereo calibration and the border-crop handling of DP cues show careful thought. Credit where it is due: the dataset capture setup and the staged transfer strategy are genuine contributions, and the citation pattern looks fine.\n\nThe soft spot is exactly where the stress-test put it: the DCDP ground truth. It comes from an undisclosed AI stereo disparity model trained on synthetic data, with no error analysis on real 2.5 cm-baseline smartphone stereo. Small baseline means small disparities, so a synthetic-trained model can carry systematic bias that dominates the inverse-depth signal. The manual masks are described only as 'visually incorrect regions.' If those same masks are used during evaluation, then Table IV is not full-image accuracy; if they are not, invalid pseudo-GT regions contaminate the test labels. Either way, the DCDP gains are not separable from the quality of the undisclosed pseudo-GT. This is load-bearing for the DCDP claims and for the dataset's value as a benchmark.\n\nOther concerns are minor-to-moderate: no error bars anywhere, so we cannot tell if the small gaps over the Stereo Baseline are noise; the 'Stereo Baseline' uses RAFT-Stereo but the paper does not say whether it was fine-tuned or frozen; and the DPNet comparison uses unofficial GitHub numbers rather than retraining or the original paper values. None of these kill the Google DP story, but they need to be fixed in a revision.\n\nWho is this for: researchers working on smartphone depth, dual-pixel sensing, or dataset construction for defocus disparity. It deserves a serious referee. The weaknesses are real but addressable. I would send it to review with a strong request to disclose the stereo model, validate it on known geometry or an independent sensor, clarify the DCDP evaluation protocol, and release code/data. The architecture itself is worth publishing even if the DCDP numbers need to be walked back.","headline":"A solid architecture-and-dataset paper whose Google DP results are plausible but whose DCDP benchmark rests on an undisclosed AI-stereo ground truth; the DCDP numbers need a disclosed and validated GT pipeline before they are meaningful.","tokens_in":11944,"tokens_out":1961,"would_cite":false,"duration_ms":21514,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Separate RGB and dual-pixel encoders with windowed bi-directional parallax attention cut ordinal depth error to 0.0799 on the public test set and add a dense dataset.","keywords":["dual-pixel depth estimation","defocus disparity","window bi-directional parallax attention","cross-modal transfer learning","RGB-DP-D dataset","stereo calibration","smartphone depth sensing"],"falsifier":"Measure a sample of DCDP scenes with a high-precision depth sensor (e.g., time-of-flight or structured light) and compare those depths against the AI-estimated ground truth, especially at distances where the 2.5 cm stereo baseline produces sub-pixel disparities; if large systematic deviations appear, the dataset's claimed ground-truth quality and the DCDP benchmark numbers would be undermined.","tokens_in":11022,"feed_emoji":"📷","tokens_out":9306,"duration_ms":78879,"temperature":0.7,"pith_summary":"The paper claims that depth estimation from dual-pixel (DP) sensor data works better when the RGB image and the DP pair are processed by separate encoders that are later fused, rather than concatenated into one input as prior work does. At the heart of the DP branch is a window bi-directional parallax attention module that matches left and right DP features only within a small window along the disparity axis, matching the tiny defocus disparities (-8 to +8 pixels) that smartphone apertures produce. To compensate for the lack of large RGB-DP-D training sets, the paper proposes a three-stage cross-modal transfer learning scheme that first trains the DP branch on DP-D pairs, then the RGB branch on large RGB-D datasets, and finally fine-tunes the whole network together. It also introduces a new dataset, DCDP, with 5,000 training and 700 test samples whose dense ground truth comes from an AI stereo disparity estimator with manual masking of bad regions. The paper reports that DiFuse-Net beats DPNet, a parameter-matched baseline, and a stereo baseline on both the public DP dataset and DCDP.","feed_headline":"Two-stream attention network halves dual-pixel depth error","feed_subtitle":"Processing RGB context and DP defocus separately beats prior dual-pixel and stereo baselines on both benchmarks.","key_machinery":"The central object is the Window Bi-directional Parallax Attention Module (WBiPAM), an adaptation of stereo parallax attention for dual-pixel defocus disparity. It partitions the left and right DP feature maps into non-overlapping k-by-1 windows along the epipolar direction, computes cross-attention scores from left-to-right as softmax(QK^T) and obtains the right-to-left scores as the transpose, then merges the windows back to the original spatial layout. Because the DP disparity range in smartphone images is small and localized, the windowed attention keeps the matching local and preserves cues that would be lost under global attention or heavy downsampling; the paper limits the DP encoder to two blocks for the same reason. The fusion module then estimates a feature-wise score per source (left DP, right DP, RGB) and recalibrates the concatenated features before convolution.","core_discovery":"DiFuse-Net's central claim is that disentangling the two modalities is the key to good smartphone dual-pixel depth: an RGB encoder supplies global scene context while a shallow siamese DP encoder, equipped with WBiPAM, extracts local defocus disparity cues, and a dynamic fusion module weights the three feature streams feature-wise. The paper reports an ordinal-error metric, 1 minus the Spearman rank correlation coefficient (1-SRCC, lower is better), of 0.0799 on the public dual-pixel test set, versus 0.1520 for DPNet, 0.0927 for a parameter-matched baseline, and 0.0911 for a stereo baseline; on the new DCDP dataset the same ordering holds (0.0878 versus 0.1522 for DPNet). It further claims that the two-layer DP encoder is optimal because deeper downsampling destroys the subtle disparity cues, and that CmTL adds a measurable gain over training from scratch.","pith_inferences":["If DCDP's AI stereo ground truth is systematically biased in textureless or far regions, then DiFuse-Net's margin over baselines on DCDP could partly reflect agreement with that estimator rather than true geometric accuracy; this is my inference, not the paper's claim.","The WBiPAM window size k is a de facto maximum-disparity prior; adapting k (or the number of windows) during inference is a testable extension for cameras with different apertures or for foveated processing.","Because the paper notes that many smartphone DP sensors have vertical rather than horizontal disparity, an orientation-aware or rotation-invariant WBiPAM variant is a natural next experiment that could generalize the method across sensor configurations.","The staged CmTL procedure could be applied to other sensor-pair tasks with scarce co-registered data, such as event-camera depth or RGB plus thermal, by pretraining each modality branch on large single-modality corpora."],"forward_implications":["DiFuse-Net's decoupled design can be adopted in phone and drone depth pipelines without additional sensors or calibration.","CmTL shows that large RGB-D datasets can substitute for scarce RGB-DP-D data, lowering the data-collection barrier for dual-pixel depth.","The DCDP dataset and its capture protocol provide a dense, manually cleaned training resource plus a benchmark for future RGB-DP methods.","The two-layer DP encoder finding implies that preserving low-level disparity detail matters more than very deep feature extraction for DP cues."],"supporting_citations":[{"why":"the prior dual-pixel depth method and public dataset that serve as the primary baseline and evaluation set.","marker":"[11]"},{"why":"the parallax attention formulation that WBiPAM adapts into windowed bidirectional cross-attention.","marker":"[24]"},{"why":"the stereo matching approach used to form the parameter-matched Stereo Baseline.","marker":"[30]"},{"why":"an AI stereo disparity estimation method used to generate ground-truth depth for DCDP.","marker":"[4]"},{"why":"an AI stereo disparity estimation method used alongside [4] for ground-truth depth generation.","marker":"[5]"},{"why":"the large synthetic dataset on which the stereo disparity model is trained for DCDP ground truth.","marker":"[27]"},{"why":"source of the scale-invariant loss function and the MiDaS model used in the monocular comparison.","marker":"[13]"}],"fun_headline_variants":["Disentangled RGB and DP features slash dual-pixel depth error","Window parallax attention of DiFuse-Net beats stereo in depth","Cross-modal transfer lifts dual-pixel depth from small datasets","New DCDP dataset drives dual-pixel depth accuracy gains","DiFuse-Net: RGB context plus DP cues cuts depth error in half"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The DCDP dataset's ground-truth depth is not measured by a range sensor but produced by an AI stereo disparity estimator trained on synthetic data, with only manual masking of visibly wrong regions, and the authors assume that estimator is accurate enough to train and evaluate depth models without a disclosed validation of its errors.","fun_headline_variants_meta":{"raw":{"variants":["Disentangled RGB and DP features slash dual-pixel depth error","Window parallax attention of DiFuse-Net beats stereo in depth","Cross-modal transfer lifts dual-pixel depth from small datasets","New DCDP dataset drives dual-pixel depth accuracy gains","DiFuse-Net: RGB context plus DP cues cuts depth error in half"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1313,"prompt_tokens":978,"completion_tokens":335,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":243}},"tokens_in":594,"tokens_out":335,"duration_ms":3972,"temperature":1.0,"reasoning_tokens":243,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:47:32.968115+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure a sample of DCDP scenes with a high-precision depth sensor (e.g., time-of-flight or structured light) and compare those depths against the AI-estimated ground truth, especially at distances where the 2.5 cm stereo baseline produces sub-pixel disparities; if large systematic deviations appear, the dataset's claimed ground-truth quality and the DCDP benchmark numbers would be undermined.","supporting_citations":[{"cited_title":"Learning single camera depth estimation using dual-pixels,","cited_arxiv_id":null,"evidence_quote":"the prior dual-pixel depth method and public dataset that serve as the primary baseline and evaluation set."},{"cited_title":"Parallax attention for unsupervised stereo correspondence learning,","cited_arxiv_id":null,"evidence_quote":"the parallax attention formulation that WBiPAM adapts into windowed bidirectional cross-attention."},{"cited_title":"Raft-stereo: Multilevel recurrent field transforms for stereo matching,","cited_arxiv_id":null,"evidence_quote":"the stereo matching approach used to form the parameter-matched Stereo Baseline."},{"cited_title":"Practical stereo matching via cascaded recurrent network with adaptive correlation,","cited_arxiv_id":null,"evidence_quote":"an AI stereo disparity estimation method used to generate ground-truth depth for DCDP."},{"cited_title":"Wavelet synthesis net for disparity estimation to synthesize DSLR calibre bokeh effect on smartphones,","cited_arxiv_id":null,"evidence_quote":"an AI stereo disparity estimation method used alongside [4] for ground-truth depth generation."},{"cited_title":"A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation,","cited_arxiv_id":null,"evidence_quote":"the large synthetic dataset on which the stereo disparity model is trained for DCDP ground truth."},{"cited_title":"Towards robust monocular depth estimation: Mixing datasets for zero-shot cross- dataset transfer,","cited_arxiv_id":null,"evidence_quote":"source of the scale-invariant loss function and the MiDaS model used in the monocular comparison."}],"review_version":2}