{"id":"c4acba76-7d78-4c18-8e93-e6e703f3c68e","arxiv_id":"2504.17067","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"PPS-Ctrl uses a Per-Pixel Shading map as a ControlNet condition for Stable Diffusion, improving sim-to-real colonoscopy translation and downstream depth estimation.","lead":"This paper presents PPS-Ctrl, a framework that translates synthetic colonoscopy images into realistic clinical-looking images using Stable Diffusion guided by a Per-Pixel Shading map. The authors report that this improves downstream depth estimation on phantom and clinical datasets compared to a GAN-based baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The PPS condition in Eq. 2 depends on an unspecified light-source position pc; if the near-field lighting model is miscalibrated, the claimed advantage over depth conditioning may not be robust.","rationale":"The reader's weakest assumption—the near-field point-light model with unknown pc—is the key assumption that the paper's novelty depends on. The ablation in Table 1 provides the direct evidence for the central claim, but the PPS condition is the only difference between Ours-PPS and Ours-Depth, so the validity of the PPS construction is what the entire contribution rests on. I agree with the reader's focus. A secondary observation is that the FID values in Table 2 are implausibly low, but the depth-estimation result is the quantitative core and is independent of FID; I therefore do not pivot the verdict on the FID issue. The concern about pc is a correctness/reproducibility risk, not a disagreement with consensus: Eq. 2 is under-specified. The proposed sensitivity analysis would settle whether the method is robust to this degree of freedom. Since the reader already marked the paper CONDITIONAL, this concern reinforces that verdict without moving it; the required condition is to specify and validate pc (and run variations).","tokens_in":7566,"tokens_out":7718,"duration_ms":76418,"concrete_test":"Using the released code, recompute Table 1 for Ours-PPS w.DC while varying the light-source position pc: (i) pc = camera optical center, (ii) pc = the known simulator light position used to render SimCol3D, and (iii) a tip-offset position typical of colonoscopes (e.g., 5–10 mm from the optical center). Also run Ours-Depth w.DC as control. If all pc variants keep an RMSE improvement over Ours-Depth w.DC of roughly the reported 9% and within the seed variance, the concern is weak; if the advantage shrinks, reverses, or is within run-to-run noise, the paper must specify and justify pc before the superiority claim can be accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that PPS conditioning preserves structure better than depth conditioning—rests on Eq. 2 (Sec. 3.2), where PPS(x)=La(x)(Ld(x)^T N(x)) with Ld and La defined by a point light at pc. The paper never specifies how pc is set. In colonoscopes, the light source is at the tip, offset from the optical center and spatially extended, so a single point light at the camera center is a poor approximation, particularly in near-field regions where inverse-square falloff is strongest. Since the paper motivates PPS precisely by failures in near-field/far-field shading, an arbitrary or miscalibrated pc changes the conditioning signal in exactly the regions where the method claims its main benefit. This is not merely a realism issue: the PPS map is the sole structural control during generation, and the downstream depth gain (Table 1) could depend on a favorable pc choice rather than on the PPS representation itself. If pc is a hidden tuned parameter, the 20% RMSE improvement over MI-CycleGAN and the 9% improvement over depth-with-decoder conditioning may not generalize to other scopes with different tip geometry. The text gives no calibration or sensitivity analysis, so the central claim is not yet reproducible as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PPS-Ctrl, a sim-to-real image translation framework for colonoscopy that uses Stable Diffusion with a ControlNet conditioned on a Per-Pixel Shading (PPS) map rather than on a depth map. The PPS map is computed from a depth map under a near-field point-light model, and an encoder-decoder is introduced to extract latent control features. The authors evaluate downstream depth estimation by training DepthAnything on translated SimCol3D-to-C3VD data, report FID scores for translation quality, and provide qualitative results on C3VD-to-Colon10K translation. The central claim is that PPS conditioning better preserves structure than depth conditioning, yielding a 20% RMSE improvement over MI-CycleGAN. Code is publicly available.","tokens_in":7785,"tokens_out":4161,"duration_ms":39740,"significance":"If the claims hold, the idea of using a physically motivated per-pixel shading field as the structural control signal in a diffusion-based translation pipeline is a useful contribution to endoscopic domain adaptation, where structure preservation is critical. The paper tests the central claim with a downstream depth-estimation task on out-of-domain phantom data, which avoids circularity, and it includes an ablation separating the PPS condition from the control decoder. The code release is a strength for reproducibility. However, the empirical support is currently incomplete: the light-source parameter in the PPS model is unspecified, the FID numbers are implausibly low and lack methodological details, and the downstream results are reported without variance or significance testing against more than one baseline.","major_comments":[{"comment":"The PPS map is the sole structural control signal, but the paper never specifies how the light-source position pc is set. Eq. (2) defines PPS(x) = La(x)(Ld(x)^T N(x)) with Ld and La depending on pc; for a colonoscope, the light source is at the tip, offset from the optical center and spatially extended, so an arbitrary choice of pc changes the conditioning exactly in the near-field regions the method is designed to improve. Since the Table 1 advantage of Ours-PPS over Ours-Depth could depend on a favorable pc choice, the manuscript should state the value used for each dataset, justify it from the scope geometry, and include a sensitivity analysis over pc or a calibration procedure. Without this, the central claim is not reproducible as written.","section":"Sec. 3.2, Eq. (2)"},{"comment":"The reported FID values (0.437 for SimCol3D-to-C3VD and 0.297 for C3VD-to-Colon10K) are orders of magnitude lower than typical FID values for full image distributions and are also lower than the corresponding no-translation FID values (0.545 and 0.527), which is suspicious because the untranslated source and target are different domains. The paper does not specify the feature layer, the number of samples, or the implementation used for FID; a mismatch in implementation would invalidate the realism claim. Please recompute FID with a standard implementation and report the sample size and confidence intervals.","section":"Table 2"},{"comment":"The central quantitative claim rests on a single table without error bars, number of runs, or significance testing. The 20% RMSE improvement over MI-CycleGAN (4.662 vs. 3.740) and the 9% improvement of Ours-PPS w.DC over Ours-Depth w.DC (4.099 vs. 3.740) are plausible but unverified. Because the downstream depth metric is the main evidence for the PPS representation, please report mean and standard deviation over multiple training runs, perform a significance test, and add at least one additional strong baseline beyond MI-CycleGAN, such as a diffusion-only translation method without explicit structure conditioning, to support the comparison.","section":"Table 1"},{"comment":"There is a train/inference text-prompt mismatch that is not discussed. Stage 2 trains the ControlNet with text prompts corresponding to the Sim domain (psource), while inference uses the Real-domain prompt (ptarget). Since the ControlNet is conditioned on text embeddings, this domain shift could affect conditioning behavior. The paper should either train with both prompts at appropriate sampling probabilities or justify why the prompt change does not interact with the PPS control signal.","section":"Sec. 3.2 and Sec. 3.3"}],"minor_comments":[{"comment":"The notation for the inverse-square falloff is ambiguous: 'La(x) = 1 / ||x−pc||2^2' should be written as La(x) = ||x - pc||_2^{-2}, and the unit direction Ld should be defined explicitly as (x - pc)/||x - pc||_2.","section":"Eq. (2)"},{"comment":"There are several typos: 'Image-to-imagetranslation', 'Depthestimation', 'Colonscopy' in the keywords, 'Freceht Inception Distance' in Sec. 4.1, and 'Ous-PPS withDC' in Sec. 4.4. These should be corrected.","section":"Keywords and Abstract"},{"comment":"The Control Encoder-Decoder architecture is described only as '4 residual blocks' and 'a transposed architecture of the control encoder EC'; please provide the encoder architecture, feature resolutions, and whether the decoder is applied at multiple scales, since this module is central to the ablation.","section":"Sec. 3.3"},{"comment":"The caption reports absolute relative errors for PPS maps but does not explain how those errors are computed or which regions they aggregate over; please add a precise description in the caption or text.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the scope of the venue and the core idea is worth pursuing, but the missing specification of pc and the anomalous FID values are load-bearing and need to be resolved before the claims can be accepted. The comparison against only one baseline is thin, though the internal ablation is informative."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing about this paper is that it tests a concrete idea: instead of feeding a depth map to ControlNet for structure-preserving sim-to-real translation, feed a per-pixel shading (PPS) map computed from that depth under a near-field lighting model. The ablations in Table 1 back the idea up—PPS beats depth, and adding the control decoder helps further, with the best config getting RMSE 3.740 vs. 4.662 for MI-CycleGAN on C3VD. That is a meaningful, if contained, improvement for downstream depth estimation.\n\nWhat's genuinely new is the combination. PPS maps come from near-field photometric stereo work, and ControlNet is established, but the specific use of PPS as a structural condition in a diffusion translation pipeline for endoscopy is not in the cited literature. The encoder-decoder that reconstructs PPS from the encoded feature is a small but sensible addition, and the paper correctly acknowledges that specularity and fine details remain uncontrolled.\n\nThe soft spots are matters of evidence, not conception. First, the FID values are implausibly low: 0.545, 0.437, 0.297. FIDs on real image distributions are usually in the tens, even for near-identical domains. Either they computed something else or the dataset is so small that the metric is meaningless. The paper doesn't explain this. Second, there are no error bars or multiple seeds in Table 1, and the only baseline is MI-CycleGAN. A 20% RMSE improvement is easy to over-read without a sense of variance. Third, the PPS computation in Eq. 2 depends on the light source position pc, which is never specified. In real colonoscopes the light is at the tip, offset from the optical center and not a point source. If pc is arbitrarily set to the camera center, the conditioning signal is miscalibrated exactly in the near-field regions that motivate the method. A sensitivity analysis of pc and at least a qualitative comparison with an alternative placement would make the claim reproducible. The stress-test note gets this right.\n\nNone of this sinks the paper. The depth estimation results are internally consistent and the ablation logic is sound. The paper deserves a serious referee, but it needs revisions: clarify the FID computation, add uncertainty estimates, expand the baseline set, and address the sensitivity of pc.\n\nFor a reading group on medical-image synthesis, I'd bring it. I wouldn't cite it in my own work yet.","headline":"PPS-Ctrl shows a plausible and internally consistent win for shading-conditioned sim-to-real in colonoscopy, but the evaluation needs to grow up before the claims are taken at face value.","tokens_in":8344,"tokens_out":2360,"would_cite":false,"duration_ms":21122,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that conditioning a diffusion-based translator on a per-pixel shading map rather than a raw depth map preserves source geometry while adding realistic textures, and that this improves downstream colonoscopy depth…","keywords":["image-to-image translation","depth estimation","colonoscopy","diffusion models","ControlNet","per-pixel shading","sim-to-real transfer","structural conditioning"],"falsifier":"Measure how well the computed PPS map predicts actual image intensity (after albedo normalization) in near-field frames of a real colonoscopy video; a large mismatch would mean the conditioning signal is partly fictional. A second decisive experiment: train PPS-Ctrl with the light position shifted to the scope tip's true offset from the optical center and compare downstream RMSE on the phantom target; no improvement would suggest the specific lighting model is not what delivers the gain.","tokens_in":7352,"feed_emoji":"🖼️","tokens_out":11275,"duration_ms":91223,"temperature":0.7,"pith_summary":"The paper is trying to establish that, when translating synthetic colonoscopy images into realistic ones, the structural condition passed to a diffusion model should be a per-pixel shading map rather than a depth map. Depth alone, the authors argue, lets the generator alter surface colors inconsistently in near-field and far-field regions, while shading encodes how light falls on each surface point and keeps the translation aligned with the source geometry. They build a Stable Diffusion pipeline with a ControlNet that takes an encoded PPS map as its conditioning input, and they show that depth estimators trained on their translated images beat those trained on a GAN-based baseline. If true, this offers a practical way to train depth estimators for real colonoscopy without collecting ground-truth clinical depth.","feed_headline":"Shading maps beat depth maps for colonoscopy sim-to-real translation","feed_subtitle":"Diffusion model conditioned on per-pixel shading cuts depth-estimation error by 20% versus the GAN baseline.","key_machinery":"The central object is the Per-Pixel Shading (PPS) map, defined from a depth map as $\\mathrm{PPS}(x) = L_a(x)\\,(L_d(x)^{\\mathsf T} N(x))$, with $L_d(x) = (x-p_c)/\\|x-p_c\\|$ and $L_a(x) = \\|x-p_c\\|^{-2}$ for an assumed point light at $p_c$. The map is computed from the synthetic depth map $D(u,v)$, camera intrinsics $K$, and the derived surface normal $N(x)$, so it encodes how strongly each surface point would be lit by a near-field source. The machinery feeds this map through a learnable control encoder $E_C$ into a ControlNet attached to a fine-tuned latent diffusion model, and adds a control decoder $D_C$ that reconstructs the PPS map from the latent feature map; the reconstruction loss in Eq. (3) is summed with the diffusion denoising objective in Eq. (4). The load-bearing idea is that shading, not raw depth, is what the generator must respect to keep near-field and far-field geometry consistent while swapping textures across domains.","core_discovery":"The central claim is that a Per-Pixel Shading (PPS) map, computed from a depth map as inverse-square light falloff times the dot product of surface normal and light direction, is a stronger structural condition for sim-to-real translation than the depth map itself. The authors build a latent diffusion pipeline in which a ControlNet takes an encoded PPS map as its conditioning input, with a control decoder that reconstructs the PPS from the latent features to keep the conditioning signal intact. On SimCol3D-to-C3VD translation, training the downstream depth estimator DepthAnything on the translated data yields RMSE 3.740 with the full PPS-plus-decoder configuration, versus 4.662 for MI-CycleGAN and 4.753 for no translation; the paper reports this as a 20% improvement over MI-CycleGAN. On C3VD-to-Colon10K translation to clinical video, FID improves from 0.498 for MI-CycleGAN to 0.297, and every ablation row shows PPS conditioning outperforming depth conditioning, with the control decoder helping further.","pith_inferences":["One extension the paper leaves implicit is that PPS conditioning should transfer to other near-field endoscopic settings, such as laparoscopy or arthroscopy, wherever a usable depth map and camera-light geometry exist.","A decisive ablation the paper does not run is to randomize the assumed light position $p_c$ during training; if downstream depth accuracy does not degrade, the benefit may come from the encoder-decoder regularizer rather than the physical shading model.","The control decoder's PPS reconstruction could be turned into a self-supervised signal on real clinical video, allowing the shading model to be refined without paired depth."],"forward_implications":["Depth estimators can be trained on translated synthetic images paired with the original synthetic depth maps, so the method does not need clinical ground-truth depth to improve target-domain depth prediction.","Using the PPS map instead of the depth map as the ControlNet condition improves downstream depth metrics in every ablation setting, and adding the control decoder improves them further.","The best configuration lowers RMSE on the phantom target from 4.662 (MI-CycleGAN) to 3.740 and lowers FID on clinical translation from 0.498 to 0.297.","The paper's qualitative results show fewer checkerboard artifacts and fewer false dark down-the-barrel textures than the GAN baseline, which matters for clinical usability."],"supporting_citations":[{"why":"Supplies the near-field lighting model (inverse-square falloff and light direction) that defines the PPS map.","marker":"[13]"},{"why":"Introduces the PPS representation and its prior use for endoscopy 3D estimation, the structural cue the paper adapts.","marker":"[15]"},{"why":"Extends the near-field point-light model to endoscopic SLAM, supporting the paper's assumption that this lighting model fits colonoscopy.","marker":"[3]"},{"why":"Introduces the latent diffusion model that the paper fine-tunes and uses as the generative base.","marker":"[19]"},{"why":"Introduces ControlNet, the conditioning mechanism the paper adapts to accept PPS instead of depth.","marker":"[27]"},{"why":"Provides the MI-CycleGAN baseline that the paper compares against for both depth accuracy and image translation quality.","marker":"[22]"},{"why":"Supplies the downstream monocular depth estimator trained on translated data and evaluated on the target domain.","marker":"[25]"},{"why":"Provides the synthetic source dataset SimCol3D with paired depth used to train the translator and the downstream estimator.","marker":"[16]"},{"why":"Provides the phantom target dataset C3VD with paired depth used for quantitative depth evaluation.","marker":"[4]"},{"why":"Provides the clinical target dataset Colon10K used for qualitative translation quality and depth prediction evaluation.","marker":"[14]"}],"fun_headline_variants":["PPS-Ctrl: Shading-conditioned diffusion beats GAN for colonoscopy depth","Per-pixel shading as condition improves colonoscopy depth estimation","Diffusion with shading maps outperforms CycleGAN for colonoscopy depth","Shading-conditioned diffusion cuts colonoscopy depth error by 20%","ControlNet on shading maps yields better colonoscopy depth than GAN"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that colonoscopy lighting behaves like a single point light at a known position near the camera; if the real scope's light geometry differs enough, the shading map will encode wrong structure and the downstream gains could shrink.","fun_headline_variants_meta":{"raw":{"variants":["PPS-Ctrl: Shading-conditioned diffusion beats GAN for colonoscopy depth","Per-pixel shading as condition improves colonoscopy depth estimation","Diffusion with shading maps outperforms CycleGAN for colonoscopy depth","Shading-conditioned diffusion cuts colonoscopy depth error by 20%","ControlNet on shading maps yields better colonoscopy depth than GAN"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000754,"raw_usage":{"total_tokens":3328,"prompt_tokens":894,"completion_tokens":2434,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":2340}},"tokens_in":510,"tokens_out":2434,"duration_ms":15936,"temperature":1.0,"reasoning_tokens":2340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:49:55.424055+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure how well the computed PPS map predicts actual image intensity (after albedo normalization) in near-field frames of a real colonoscopy video; a large mismatch would mean the conditioning signal is partly fictional. A second decisive experiment: train PPS-Ctrl with the light position shifted to the scope tip's true offset from the optical center and compare downstream RMSE on the phantom target; no improvement would suggest the specific lighting model is not what delivers the gain.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the near-field lighting model (inverse-square falloff and light direction) that defines the PPS map."},{"cited_title":"In: International Conference on Medical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"Provides the MI-CycleGAN baseline that the paper compares against for both depth accuracy and image translation quality."},{"cited_title":"Medical image analysis 90, 102956 (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the phantom target dataset C3VD with paired depth used for quantitative depth evaluation."},{"cited_title":"In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)","cited_arxiv_id":null,"evidence_quote":"Provides the clinical target dataset Colon10K used for qualitative translation quality and depth prediction evaluation."}],"review_version":1}