{"id":"ec95e8bd-dcaa-4acd-ab16-6a47fc161bad","arxiv_id":"2502.05055","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DMDPS reconstructs surface normals and albedos using only a phone display and front camera, with learned patterns and HDR capture for use in normal indoor lighting.","lead":"This paper turns a mobile phone into a portable 3D scanner by using the screen as a programmable light source and the front camera as the sensor. The authors built a custom app that displays learned light patterns and captures HDR photos, enabling photometric stereo in ordinary rooms instead of a dark lab.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Leaf reconstructions depend on a planar 10 cm model (§5.2) that leaf curvature violates; the paper gives no ground-truth normal evaluation for non-planar objects, so the central natural-objects claim is unvalidated.","rationale":"The reader identified the Lambertian and planar assumptions as the weakest point. I agree that the linear model is load-bearing, but I focus on the planar component specifically because it is explicitly assumed in §5.2 and directly conflicts with the paper's own application to fallen leaves, which have curvature. The paper provides no quantitative validation for non-planar objects, so the leaf dataset and the associated claim of handling curvature are unsupported. This is an internal tension, not a disagreement with external consensus: the same model that is stated as planar is later claimed to work on curved objects. The reader's CONDITIONAL verdict already captures the need for additional validation, and my concern is one of the conditions that should be added. Therefore, I do not propose changing the verdict, but I recommend adding this specific check to the list of required evidence. I partially agree with the reader because they grouped planar and Lambertian together; I isolate the planar aspect as the more concrete and testable failure mode for the leaf dataset.","tokens_in":8620,"tokens_out":5175,"duration_ms":53786,"concrete_test":"Use a 3D-printed object with known non-planar geometry (e.g., a hemisphere or sinusoidal relief) and ground-truth normals from its CAD mesh. Place it at the same 10 cm distance, run the DMDPS capture and reconstruction, and compute angular error as a function of local surface normal deviation from the assumed plane. If the error grows with curvature, the planar assumption is violated in practice and the leaf reconstructions are unreliable; alternatively, obtain ground-truth normals for a real leaf via structured-light scanning and compare directly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—that DMDPS yields surface normals for natural objects such as fallen leaves with accuracy comparable to DDPS—rests on the reconstruction model in Section 5.2: I = ρ ⊙ P l N, where l is the illumination direction matrix computed from superpixel positions and 'assuming a planar surface at a fixed distance of 10 cm from the camera.' The normal is estimated by the pseudo-inverse N ← (ρ ⊙ P l)† I. For any non-planar surface, the true illumination direction to a scene point differs from the assumed planar-direction l, introducing a systematic bias that grows with surface slope or curvature. Section 6.3 claims that DMDPS 'handle[s] variations in curvature' and achieves 'high angular accuracy' on fallen leaves, but this is supported only by qualitative images; no ground-truth normals for leaves or any other non-planar object are provided. The glossy-object failure in Section 6.2 shows that the method is sensitive to violations of its linear model; non-planarity is an analogous, unquantified violation. Because the leaf dataset is one of the two headline contributions, the unsupported planar-to-curved extrapolation is a load-bearing gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Differentiable Mobile Display Photometric Stereo (DMDPS), a photometric stereo system that uses a mobile phone's display as a programmable light source and its front camera for capture. The authors develop a custom mobile app for synchronized pattern display and RAW/HDR image capture, apply Gaussian filtering and camera undistortion, and adapt the differentiable pattern-learning framework of DDPS to the mobile setting. Experiments on 3D-printed objects show that learned patterns reduce reconstruction error relative to initial patterns, and the authors introduce a new dataset of fallen leaves with reconstructed surface normals and albedos.","tokens_in":8841,"tokens_out":7507,"duration_ms":68653,"significance":"If the central claims hold, DMDPS would be a practical and portable photometric stereo solution requiring only a commodity smartphone, enabling capture in general environments rather than a dark room and without polarization hardware. The contributions include the mobile app, the adaptation of learned illumination patterns, and a new natural-object dataset. However, the quantitative evaluation is compromised by a likely train/test overlap, and the natural-object validation is strictly qualitative; these issues must be resolved before the paper's main claims are convincing.","major_comments":[{"comment":"The paper does not state whether the 3D-printed objects used to compute the reconstruction errors in Table 2 are the same as those used in the pattern-learning procedure described in Section 5.1. Since Section 5.1 refers to a \"3D-printed training dataset\" and Table 2 reports errors for \"the 3D-printed database,\" it appears that the evaluation may be performed on the training set itself. If so, the improvements in Table 2 (e.g., from 0.4109 to 0.05754 for the Flat gray pattern) could be inflated by overfitting, and the claim that learned patterns generalize is not supported. The authors should either report results on a clearly disjoint held-out set of 3D-printed objects or explicitly state that Table 2 is a training-set evaluation and provide additional test-set numbers.","section":"Section 6.1 / Table 2; Section 5.1"},{"comment":"The reconstruction model I = ρ ⊙ P l N assumes a planar surface at a fixed distance of 10 cm, and the illumination directions l are computed under this assumption. Fallen leaves are non-planar and exhibit curvature, so the true illumination directions at each scene point differ from l, causing a systematic bias in the pseudo-inverse normal estimate. Section 6.3 claims that the method \"handle[s] variations in curvature\" and achieves \"high angular accuracy\" on leaves, but no quantitative evaluation with ground-truth normals is provided; the supporting evidence is limited to qualitative images in Figure 7. This is a load-bearing gap because the fallen-leaf dataset is one of the paper's headline contributions. The authors should provide either ground-truth normal maps for a set of leaves (e.g., obtained by 3D scanning) or a quantitative sensitivity analysis that estimates the error introduced by the planar assumption on curved surfaces.","section":"Section 5.2, Eq. (1); Section 6.3"},{"comment":"The paper claims that HDR and RAW imaging enable capture in general environments instead of a dark room, but no controlled experiment is reported to substantiate this claim. Specifically, there is no comparison of reconstruction error with and without HDR, nor a comparison of captures in a standard room versus a dark room. Without such an ablation, it is unclear whether the claimed advantage of \"general environments\" is actually attributable to the HDR/RAW pipeline, and whether the method would work without these components. The authors should add an experiment that isolates the effect of HDR/RAW on reconstruction accuracy, or tone down the claim accordingly.","section":"Section 4.3 / Section 3.3"},{"comment":"The differentiable image formation model f_I(P_i, B) = Σ_j B_j P_{i,j} is a linear function of the basis image B and the pattern P_i. This model does not account for external ambient light, specular reflections, or the nonlinear merging used in HDR imaging, all of which are present in the actual capture pipeline. The paper does not explain how B is captured (e.g., whether it includes external light components), nor why this linear simulation is adequate for learning patterns that are then used on real HDR captures. The authors should describe the acquisition of B and discuss the potential domain gap between the simulated training images and real captured images, or provide an experiment that tests whether patterns learned with the linear model transfer to real captures with HDR.","section":"Section 5.1, Eq. (2) and pattern-learning objective"}],"minor_comments":[{"comment":"The label \"OLA T\" is a typo and should be \"OLAT.\"","section":"Table 1"},{"comment":"The equation for tangential distortion appears incorrect: x_distorted = x(2p1 y + p2(r^2 + 2x^2)) is missing the additive identity term. The standard model is x_distorted = x + [2p1 xy + p2(r^2 + 2x^2)] (and similarly for y). Please correct the formula.","section":"Section 4.3, tangential distortion formula"},{"comment":"The weight function W(I_ti) in the HDR merging equation is not defined. Please specify the weighting scheme used.","section":"Section 4.3, HDR merging equation"},{"comment":"The paper does not report the number of 3D-printed training objects, the number of fallen leaves, or the number of patterns K used in the experiments. These details are necessary for reproducibility and for assessing the scale of the evaluation.","section":"Section 5.1 / Table 2"},{"comment":"The statement that the average reconstruction error is \"approximately 0.01 higher than that of DDPS\" would be more informative if the DDPS error on a comparable object set were reported explicitly, preferably in the same table.","section":"Section 6.1"},{"comment":"The leaf reconstructions in Figure 7 are qualitative. Adding quantitative metrics or a comparison with a baseline photometric stereo method would strengthen the claims in Section 6.3.","section":"Figure 7"},{"comment":"There are several typos, e.g., \"capture s\" in the abstract, \"normal s\" in the abstract, \"sperate\" in Section 6.1, and inconsistent use of spacing. A careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper builds on the authors' own CVPR 2023 DDPS work and demonstrates a credible mobile imaging system. The main weakness is the evaluation: Table 2 likely reports training-set performance, and the natural-object results lack ground truth. If the authors can provide a held-out 3D-printed test set and some quantitative validation on non-planar objects, the contribution would be substantially strengthened. The novel dataset of fallen leaves is a useful resource for the community. I would support publication after major revisions addressing the train/test separation and the planar-assumption validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The key thing to know: this paper shows a working mobile display photometric stereo system with learned patterns, and the qualitative leaf reconstructions look plausible. But the quantitative evidence is weaker than the presentation suggests. Table 2 evaluates on the same 3D-printed objects used to learn the patterns, with no train/test split described, so part of the improvement is expected from fitting. And the leaf results—one of the two headline contributions—have no ground-truth normals to back the claim of \"high angular accuracy\" for curved objects.\n\nWhat is genuinely new is the engineering: a custom app that synchronizes display and RAW capture, HDR merging, and undistortion. That is real work, and it moves display photometric stereo from a desktop instrument to a phone. The fallen-leaf dataset is a nice idea, though only a small set and not yet released.\n\nThe soft spots are load-bearing. First, the reconstruction model in Section 5.2 assumes a planar surface at a fixed 10 cm distance, but leaves are curved. The stress-test note is right: no quantitative validation on non-planar objects means the central natural-objects claim is unsubstantiated. Second, there is no direct comparison to DDPS under a shared protocol; the statement that error is about 0.01 higher is not backed by an experiment the reader can check. Third, no code, app, or data is released, so none of this is independently verifiable.\n\nTo the paper's credit, the Discussion honestly acknowledges the specular-separation failure and the glossy-object limitation. That narrows the scope but also makes the claims more trustworthy. The citation pattern looks fine; the core learning pipeline is properly attributed to the authors' own DDPS work.\n\nThis deserves a serious referee, but I would ask for: a clear train/test split for the 3D-printed evaluation, a direct DDPS comparison, quantitative validation on non-planar objects (e.g., 3D-printed curved shapes with known normals), and a commitment to release the app and dataset. As is, it is a nice demo with promising engineering, but the scientific claims need shoring up.","headline":"A promising mobile photometric stereo demo whose main experimental evidence is partially circular and whose natural-object claims lack ground truth; the engineering is real, the evaluation needs work.","tokens_in":9402,"tokens_out":1754,"would_cite":true,"duration_ms":18330,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a consumer mobile phone, using its own display as the light source and its front camera as the sensor, can reconstruct surface normals and albedos of matte objects in ordinary rooms rather than a dark room, reaching…","keywords":["photometric stereo","mobile phone","differentiable pattern learning","HDR imaging","surface normal reconstruction","display illumination","fallen leaves dataset"],"falsifier":"Photograph a curved, non-planar object at distances other than 10 cm, or a matte object under strong ambient light, and compare the reconstructed normal map to a ground-truth 3D scan; if the angular error rises sharply, the planar Lambertian and fixed-distance assumptions are the cause.","tokens_in":8413,"feed_emoji":"📱","tokens_out":3054,"duration_ms":31907,"temperature":0.7,"pith_summary":"This paper tries to establish that display photometric stereo, which normally requires a fixed desktop monitor, a polarization camera, and a dark room, can be carried out with a single mobile phone. The authors build a custom app that displays learned illumination patterns on the phone screen while the front camera captures RAW and HDR images, then reconstruct normals with a differentiable pattern-learning pipeline. If correct, high-quality surface-orientation capture becomes portable and works under general indoor lighting, and it can be applied in the wild to objects such as fallen leaves. The reported reconstruction error is roughly 0.01 higher than the desktop DDPS baseline, and the method fails on glossy objects.","feed_headline":"Phone screen and front camera rebuild surface normals anywhere","feed_subtitle":"Mobile photometric stereo works in ordinary rooms and lands within 0.01 of desktop accuracy, enabling new leaf datasets.","key_machinery":"The load-bearing mechanism is a learned display-pattern matrix $\\mathbf{P}$ optimized through a differentiable pipeline: a differentiable image formation function $f_I(\\mathbf{P}_i, \\mathbf{B}) = \\sum_j \\mathbf{B}_j \\mathbf{P}_{i,j}$ synthesizes captured images from basis images $\\mathbf{B}$, and a differentiable photometric stereo function $f_n$ estimates normals, with loss $(1 - \\mathbf{N} \\cdot \\mathbf{N}_{GT})/2$ penalizing angular error. Optimization uses Adam, with a sigmoid constraint keeping patterns in $[0,1]$. Reconstruction at test time assumes a Lambertian, planar surface at a fixed 10 cm distance and solves $\\mathbf{I} = \\boldsymbol{\\rho} \\odot \\mathbf{P} \\mathbf{l} \\mathbf{N}$ by pseudo-inverse, with albedo set to the maximum observed intensity. Supporting image processing includes Gaussian filtering, camera undistortion, and HDR merging from exposure-bracketed RAW captures.","core_discovery":"The central claim is that a mobile phone with a screen and a front camera is a sufficient imaging system for practical physics-based photometric stereo. The paper introduces differentiable mobile display photometric stereo (DMDPS), which learns display patterns end-to-end using a differentiable image formation model and a differentiable photometric stereo solver, then uses those learned patterns to reconstruct surface normals on real objects. The key measurement is that, for 3D-printed training objects, DMDPS achieves average angular reconstruction error about 0.01 higher than the desktop DDPS baseline, despite using a weaker, non-polarized light source and capturing in general environments instead of a dark room. The authors also produce a first dataset of fallen leaves with reconstructed surface normals and albedos.","pith_inferences":["The fixed 10 cm planar assumption could be relaxed by estimating scene depth per pixel or by capturing at multiple distances, which would extend the method to curved objects; the paper does not explore this.","Because the learned patterns are optimized for one phone's display and camera, transferring them to another phone model would likely require recalibration or re-learning, a step the paper leaves implicit.","The HDR approach suggests that carefully chosen exposure brackets could also compensate for mild object motion or varying ambient flicker, though the paper does not test these conditions.","A natural extension is to attach a polarizing film to the front camera and use a polarized display mode, which the authors mention as future work and which would directly address their main error source."],"forward_implications":["Surface normal and albedo capture no longer requires a dedicated desktop setup, enabling photometric stereo in homes, offices, or field conditions.","The method's error being only about 0.01 above the desktop baseline suggests that learned display patterns remain effective when the light source is much smaller and weaker.","The fallen-leaves dataset demonstrates that the approach can be applied to unstructured natural objects, yielding normals that capture veins and surface undulations.","A phone-based system makes it straightforward to build new photometric-stereo datasets by photographing objects as they are encountered, rather than transporting them to a lab.","The glossy-object failure indicates that extending the method to reflective materials would require diffuse-specular separation, which the current phone camera cannot perform."],"supporting_citations":[{"why":"Introduces differentiable display photometric stereo (DDPS), the desktop baseline that DMDPS extends and compares against.","marker":"[1]"},{"why":"Supplies the differentiable image formation and differentiable photometric stereo pipeline, as well as the basis-image capture procedure that DMDPS adapts to a mobile phone.","marker":"[2]"},{"why":"Shows prior use of mobile devices for surface reflectometry, establishing the portability precedent that DMDPS pushes further.","marker":"[5]"},{"why":"Provides reflectance-capture methodology using univariate BRDF sampling, one of the mobile-capture approaches the paper situates itself against.","marker":"[6]"},{"why":"The Adam optimizer is the optimization algorithm used to minimize the angular normal loss during display-pattern learning.","marker":"[11]"}],"fun_headline_variants":["Phone screen and camera do photometric stereo anywhere","Differentiable mobile photometric stereo within 0.01 of desktop","First fallen-leaf normals dataset from a phone-based system","Practical photometric stereo: just a phone display and camera","Mobile differentiable photometric stereo for real-world objects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reconstruction assumes each surface patch is matte, flat, and fixed at a 10 cm distance, with all non-display light treated as removable noise; if ambient light, specular reflection, or curved geometry violates that model, the pseudo-inverse normal estimate is biased.","fun_headline_variants_meta":{"raw":{"variants":["Phone screen and camera do photometric stereo anywhere","Differentiable mobile photometric stereo within 0.01 of desktop","First fallen-leaf normals dataset from a phone-based system","Practical photometric stereo: just a phone display and camera","Mobile differentiable photometric stereo for real-world objects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1343,"prompt_tokens":887,"completion_tokens":456,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":377}},"tokens_in":503,"tokens_out":456,"duration_ms":5575,"temperature":1.0,"reasoning_tokens":377,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T20:24:29.851759+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Photograph a curved, non-planar object at distances other than 10 cm, or a matte object under strong ambient light, and compare the reconstructed normal map to a ground-truth 3D scan; if the angular error rises sharply, the planar Lambertian and fixed-distance assumptions are the cause.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces differentiable display photometric stereo (DDPS), the desktop baseline that DMDPS extends and compares against."},{"cited_title":"4-6,2023","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable image formation and differentiable photometric stereo pipeline, as well as the basis-image capture procedure that DMDPS adapts to a mobile phone."},{"cited_title":"Mobile surface reflectometry","cited_arxiv_id":null,"evidence_quote":"Shows prior use of mobile devices for surface reflectometry, establishing the portability precedent that DMDPS pushes further."},{"cited_title":"Reflectance capture using univariate sampling of brdfs","cited_arxiv_id":null,"evidence_quote":"Provides reflectance-capture methodology using univariate BRDF sampling, one of the mobile-capture approaches the paper situates itself against."},{"cited_title":"Kingma and Jimmy Ba","cited_arxiv_id":null,"evidence_quote":"The Adam optimizer is the optimization algorithm used to minimize the angular normal loss during display-pattern learning."}],"review_version":1}