{"id":"3d2fb971-8a58-474a-bb88-485b2bd5a232","arxiv_id":"2505.02654","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A CycleGAN variant with a depth-consistency loss translates simulated colonoscopy images to look real while preserving fold geometry, enabling synthetic-only training of a fold segmentation model that outperforms prior baselines on real data.","lead":"This paper adds a depth-consistency constraint to a CycleGAN so simulated colonoscopy images get realistic textures without bending the folds, and uses the translated images to train a fold segmentation network with no real labeled images. It also releases a new synthetic benchmark and beats an earlier model by about 11 points of mean IoU on real EndoMapper frames.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Depth-consistency loss rests on an unvalidated assumption that the fixed monocular depth estimator is reliable on translated images; no direct structure-preservation metric is reported.","rationale":"The reader's weakest assumption identifies exactly the load-bearing point: the pretrained depth estimator's reliability on translated images is assumed, not shown, and Eq. (1) depends on it. My reading of the full text confirms that no validation of depth accuracy on translated images is provided, and the qualitative figures plus downstream mIoU are the only support for structure preservation. This is a genuine concern because SCSfM-Learner was pretrained on VR-CAPS simulated images, making the translated-image distribution out-of-distribution for the depth estimator. The concern is addressable with a focused experiment, and it does not invalidate the empirical comparisons; it just leaves the central mechanism under-verified. Therefore I do not move the verdict: it should remain conditional, with the added validation as the condition. I agree with the reader that the missing quantitative structure-preservation evidence and the significance-testing gap are the main weaknesses, and the depth-estimator assumption is the sharpest form of that weakness.","tokens_in":7849,"tokens_out":4875,"duration_ms":61863,"concrete_test":"Use the held-out 500-frame Sim. test set to generate translations with Ours and with plain CycleGAN under identical training data and segmentation backbone. Compute the scale-invariant depth error of Eq. (1) between the fixed estimator's prediction on each translated image and the available ground-truth depth, and repeat with an independent estimator such as DepthAnything. Also report a direct structural metric, e.g., the IoU between fold instance masks propagated from the source through a robust keypoint or optical-flow alignment, or the depth-error gap between Ours and CycleGAN. If Ours does not show substantially lower scale-invariant depth error than CycleGAN, or if the two depth estimators disagree on which translations are structure-preserving, then the depth-consistency loss is not demonstrably the cause of the reported segmentation improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism is Eq. (1), which uses the depth prediction d of the translated image y as the arbiter of structural preservation. The paper never validates that the fixed monocular depth models (SCSfM-Learner pretrained on VR-CAPS for the main colonoscopy experiment, DepthAnything otherwise) produce accurate relative depth on the translated-image distribution. Translation adds realistic texture, lighting, and artifacts that these estimators, especially SCSfM-Learner trained on simulated VR-CAPS images, were not trained on. If the estimator is biased on translated images, Eq. (1) enforces the wrong geometry, or simply adds noisy gradients that do not reliably penalize structural distortion. The scale-invariant form removes only a global log-depth offset; it does not correct estimator-specific, spatially varying bias. The only evidence offered for structure preservation is qualitative figures plus downstream segmentation mIoU in Table 2. That evidence is indirect: the best setting, Sim-Aug. & Sim., also feeds untranslated simulated images in the same training batch, so the segmentation model has another route to the source geometry, and no reported metric directly measures fold-shape or fold-location preservation on the translated images. Without a direct check of depth prediction accuracy on the translated distribution, the paper's core claim that the new depth-consistency loss is what preserves structure remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a CycleGAN-based image translation method that adds a depth-consistency loss (Eq. 1) to preserve scene structure (specifically the shape and location of haustral folds) when transferring realistic texture from real colonoscopy images (EndoMapper) to simulated images (VR-CAPS). The translated images, optionally mixed with the original simulated images, are then used to train an EndoFM-TransUNet model for fold segmentation. The authors also release a new simulated benchmark with ground-truth fold labels and depth maps, plus manual annotations on 100 real EndoMapper test frames. In the main comparison, the best configuration reaches 44.3 mean IoU on the EM test set versus 32.64 for the best FoldIt baseline, an absolute difference of 11.66 points. Additional qualitative results are shown for NBI colonoscopy and laparoscopy style transfer.","tokens_in":8064,"tokens_out":6556,"duration_ms":73708,"significance":"If the mechanism is substantiated, the paper would provide a practical and data-efficient recipe for sim2real in endoscopy: a lightweight depth-consistency term that prevents the structural distortion typical of CycleGAN, plus a strong segmentation backbone, allowing training on synthetic data alone to transfer to real images. The strongest evidence is the within-model comparison under EndoFM-TransUNet: the proposed translation (42.5 IoU) clearly outperforms CycleGAN (26.9) and I2I (37.6) when training only on translated images, indicating that the depth loss brings a real benefit. The release of the simulated benchmark, ground-truth depth maps, and EM metadata is a valuable community contribution. However, the paper does not yet directly validate the depth estimator on the translated distribution, does not compare against the closest prior work (SPIGAN), and does not provide significance testing, so the central claim that depth consistency is what preserves structure remains partially unsupported.","major_comments":[{"comment":"The depth-consistency loss relies on a fixed pretrained monocular depth estimator (SCSfM-Learner pretrained on VR-CAPS) to judge whether the translated image y preserves the geometry of the simulated input x. The paper provides no evidence that this estimator produces reliable relative depth on the translated-image distribution, which contains realistic textures, lighting, and artifacts that the estimator may not have seen. A direct validation is required: for example, compute a scale-invariant depth error between the predicted depth of the translated image and the ground-truth depth of the corresponding simulated image, and report it for each translation baseline (CycleGAN, I2I, Ours). Without such a check, the central claim that Eq. (1) is what preserves fold shape and location remains unsupported.","section":"Sec. 3, Eq. (1)"},{"comment":"The most closely related prior work, SPIGAN [12], also uses depth estimation to maintain scene layout during CycleGAN-based sim2real translation and trains a segmentation model jointly. It is discussed in Related Work but never compared experimentally. Since the paper's novelty centers on the depth-consistency loss, a comparison to SPIGAN, or at least a principled explanation of why it is not applicable to this setting, is necessary to position the contribution and rule out that the proposed loss is a re-implementation of an existing idea.","section":"Sec. 2 and Table 2"},{"comment":"The reported means and standard deviations are over test frames, not over multiple training runs, and no significance testing is performed. The differences between, e.g., Sim-Aug. Ours (42.5 ± 11.59) and Sim-Aug. & Sim. Ours (44.3 ± 11.47) are within one standard deviation; it is unclear whether the 1.8-point gain from the paired-batch strategy is meaningful. The authors should provide confidence intervals, paired tests across multiple seeds, or per-run scatter plots to support the claim of significant improvement over baselines.","section":"Table 2 and Sec. 4.2"},{"comment":"The headline comparison between the proposed pipeline and FoldIt conflates two change factors: the segmentation architecture (EndoFM-TransUNet vs. FoldIt) and the translation method. The stated 11.66% improvement is an absolute difference on the EM test set (44.3 vs. 32.64) between different models trained on different data. The paper should clearly separate the within-model gain produced by the translation (4.6 points over the no-translation EndoFM-TransUNet baseline) from the cross-model gain, and should also isolate the paired-batch effect by reporting a single ablation that varies only whether the Sim. set is included alongside Sim-Aug.","section":"Sec. 4.2 and Table 2"}],"minor_comments":[{"comment":"The table captions contain a formatting error: 'T able' should be 'Table'.","section":"Tables 1 and 2"},{"comment":"The sentence 'We show an overall 11.66% improvement ... of which 4.6% is the result of our proposed image translation' should specify that these are absolute percentage points, not relative improvements.","section":"Sec. 4.2"},{"comment":"The weight given to the depth-consistency loss (lambda_depth) in the total CycleGAN objective is never reported. Since this is the key free parameter of the method, its value and ideally a sensitivity analysis should be provided for reproducibility.","section":"Sec. 3"},{"comment":"The comparison with I2I from [14] uses different training data sizes (the caption says I2I was trained on 20,000 simulated and 74,000 real images, while Ours uses 2,000 simulated and 2,840 real). The text should explicitly state that this is a data-efficiency comparison, as the qualitative parity in the figure could otherwise be misread.","section":"Fig. 5(b)"},{"comment":"The claim that the method 'produces realistic images in different endoscopy scenarios' is supported only qualitatively for NBI colonoscopy and laparoscopy; the quantitative evaluation is limited to fold segmentation in white-light colonoscopy. The authors should either soften this claim or provide a quantitative metric for the additional scenarios.","section":"Abstract and Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and application-driven problem, and the released data and code are a positive contribution. However, the novelty relative to SPIGAN is incremental, and the current evaluation is not yet rigorous enough to establish the claimed structural-preservation mechanism. The authors should be encouraged to add a direct depth-validation experiment, a SPIGAN comparison, and significance tests; these are feasible within the manuscript's scope and would substantially strengthen the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe practical contribution here is real: a labeled simulated colonoscopy dataset for fold segmentation, automatic labels from mesh curvature, manual annotations on 100 real EndoMapper frames, and code/data released. The task previously had no public evaluation set, so the benchmark alone is worth having. The end-task results are also credible: the best EndoFM-TransUNet trained only on translated synthetic data reaches 44.3 mIoU on real EM frames, beating the strongest FoldIt variant by 11.66 points. I checked the decomposition and it is consistent: 7.06 from the backbone itself, 2.8 from their translation, 1.8 from the paired-batch training. So the paper is honest about where the gains come from.\n\nThe soft spots are the ones you'd expect. The depth-consistency loss is not new in spirit — SPIGAN already used depth supervision to preserve layout in CycleGAN-based sim2real translation, and the paper cites it. Not comparing directly against SPIGAN is a real gap, especially since SPIGAN is a one-step method and the paper is explicitly pitching a two-step pipeline. Second, the core mechanism in Eq. (1) assumes the fixed pretrained depth estimator gives reliable relative depth on the translated image distribution. The paper never validates this on translated images. SCSfM-Learner was pretrained on VR-CAPS simulations; feeding it CycleGAN-realistic textures could bias the loss. The scale-invariant term removes global offset, not spatial bias. No direct structure-preservation metric (e.g., depth error or label IoU on translated images) is reported; the evidence is qualitative figures plus downstream segmentation. The downstream result is encouraging, but it does not isolate the depth loss specifically — the paired-batch trick and the backbone choice are confounds, and the fold-supervised end task may tolerate mild geometric distortion.\n\nThese issues are all addressable: add SPIGAN as a baseline, report depth error or label agreement on translated images, run an ablation without the depth term while keeping paired batches, and report whether the depth estimator's accuracy degrades on the translated distribution. I'd like to see those before accepting the novelty claim at face value, but I don't think the paper is wrong. The benchmark and the empirical improvement are genuine.\n\nI'd send it to review. The weaknesses are experimental gaps, not a load-bearing flaw in the core idea. The data release and the first public real test set for fold segmentation make it worth referee time.","headline":"A useful new benchmark for colonoscopy fold segmentation and a plausible sim2real pipeline, but the depth-consistency mechanism is under-validated and the novelty claim overlaps with SPIGAN.","tokens_in":8605,"tokens_out":2760,"would_cite":true,"duration_ms":30396,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding depth consistency to CycleGAN preserves colon fold structure and lifts real fold segmentation by 11.66 mean-IoU points.","keywords":["endoscopy","colonoscopy","image translation","CycleGAN","depth consistency","fold segmentation","sim-to-real","synthetic data"],"falsifier":"On the released benchmark, compare the depth predicted from a translated image against the ground-truth depth of its simulated source; if the predicted depth is systematically wrong in fold regions, or if replacing the depth estimator with a deliberately poor one still yields the same 11.66-point IoU gain, then the depth-consistency term is not the mechanism responsible for the improvement.","tokens_in":7629,"feed_emoji":"🩺","tokens_out":8491,"duration_ms":97585,"temperature":0.7,"pith_summary":"This paper claims that the standard CycleGAN style transfer that makes synthetic colonoscopy images look real also deforms the scene structure, so the automatically generated labels no longer match. The authors propose adding a depth-consistency loss that compares a monocular depth estimate of the translated image with the known depth of the synthetic input, forcing the generator to keep the colon layout and folds intact. Training a fold-segmentation model only on these translated synthetic images, paired with the original synthetic images, yields better segmentation on real EndoMapper colonoscopy frames than the existing FoldIt baseline. This matters because folds can hide polyps, and reliable annotated real data are hard to obtain.","feed_headline":"Style transfer that preserves depth lifts colon fold segmentation 11.66 IoU points","feed_subtitle":"Adding a depth loss to CycleGAN keeps folds intact, so synthetic-only training beats FoldIt on real data.","key_machinery":"The load-bearing object is the depth-consistency loss of Eq. (1), a scale-invariant log-depth error between the depth predicted by a fixed pretrained estimator on the translated image and the ground-truth depth of the simulated input. Added to the CycleGAN objective, this loss penalizes the generator whenever texture transfer moves surfaces closer or farther in depth, so geometric ridges such as colon folds are preserved. A second component is the paired training structure, where the downstream segmentation model sees batches containing both original and translated simulated images, which compensates for residual distortion introduced by translation.","core_discovery":"The central claim is that a CycleGAN-based translation model supervised with a scale-invariant depth-consistency loss can add realistic texture to simulated endoscopy images while preserving the shape and location of anatomical structures, and that the resulting images are sufficient to train a fold-segmentation model that generalizes to real colonoscopy data without any real labels. In the authors' reported experiments, their full pipeline reaches 44.3 mean IoU on the EM test set, an 11.66-point improvement over the best FoldIt variant, with the depth-consistency translation and paired-batch training each contributing measurable gains.","pith_inferences":["The depth-consistency mechanism is only as good as the fixed depth estimator, so the obvious next check is to validate depth accuracy on translated images; a biased estimator could silently enforce wrong geometry while downstream IoU still improves for unrelated reasons.","Because the method only needs a simulator that can render depth, it likely generalizes as a structure-preserving sim-to-real recipe beyond colonoscopy, for example to laparoscopy or capsule endoscopy.","A direct measurement of fold-shape preservation, such as mask IoU between the original simulated label and the label re-projected on the translated image, would isolate how much of the downstream gain comes from translation fidelity versus the stronger segmentation backbone.","The paired-batch training suggests some residual distortion remains even with the depth loss, so adapting the loss over training or adding a local shape-consistency term could close the remaining gap."],"forward_implications":["Fold segmentation on real colonoscopy can be trained with zero real annotations, using only automatically labeled simulated frames processed by the proposed translation approach.","The released simulated benchmark with ground-truth fold masks and depth maps provides a public testbed for fold segmentation, which previously lacked one.","The same depth-consistency translation recipe transfers to other endoscopy styles, such as NBI lighting and laparoscopy, where it also reduces CycleGAN structure distortion.","Because the downstream model trains only on simulated data, the approach can be extended to other anatomical landmarks with automatic labels, such as polyps or lesions.","In the paper's ablations, adding the depth-consistency translation and the paired-batch structure each improve IoU beyond the already stronger EndoFM-TransUNet baseline."],"supporting_citations":[{"why":"Supplies the real colonoscopy images used as translation target and the 100-frame manually annotated evaluation set.","marker":"[1]"},{"why":"Pretrained depth estimation model (SCSFM-Learner) used in the main experiment to compute the depth-consistency loss on translated images.","marker":"[2]"},{"why":"Base encoder-decoder architecture for the downstream fold-segmentation model.","marker":"[4]"},{"why":"Source of the scale-invariant depth log loss used in Eq. (1).","marker":"[6]"},{"why":"Source of the simulated colon images and their ground-truth depth and fold labels used for training and testing.","marker":"[9]"},{"why":"Existing fold-segmentation model for optical colonoscopy that serves as the main quantitative baseline.","marker":"[13]"},{"why":"Comparison image-translation method evaluated in the quantitative fold-segmentation experiments.","marker":"[14]"},{"why":"Pretrained endoscopy-specific backbone integrated into the downstream segmentation model.","marker":"[19]"},{"why":"Alternative fixed depth estimator used in the NBI and laparoscopy translation experiments.","marker":"[20]"},{"why":"Base image-translation method whose loss the depth-consistency term extends and whose structure distortion motivates the approach.","marker":"[24]"}],"fun_headline_variants":["Structure-aware style transfer preserves folds, boosts colon segmentation 11.66 IoU","Depth-guided style transfer keeps folds in place, lifts IoU 11.66 on real colonoscopy","Sim2real endoscopy: structure-aware translation lifts fold segmentation by 11.66 IoU","No real labels needed: structure-preserving sim-to-real lifts fold IoU by 11.66","Structure-aware image translation yields 11.66 IoU gain for colon fold segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the fixed pretrained depth estimator is trustworthy on CycleGAN-translated images; if its depth predictions on translated images are biased, the depth-consistency loss enforces the wrong geometry and the claimed fold preservation may not hold.","fun_headline_variants_meta":{"raw":{"variants":["Structure-aware style transfer preserves folds, boosts colon segmentation 11.66 IoU","Depth-guided style transfer keeps folds in place, lifts IoU 11.66 on real colonoscopy","Sim2real endoscopy: structure-aware translation lifts fold segmentation by 11.66 IoU","No real labels needed: structure-preserving sim-to-real lifts fold IoU by 11.66","Structure-aware image translation yields 11.66 IoU gain for colon fold segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001182,"raw_usage":{"total_tokens":4870,"prompt_tokens":922,"completion_tokens":3948,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":3831}},"tokens_in":538,"tokens_out":3948,"duration_ms":34693,"temperature":1.0,"reasoning_tokens":3831,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:44:45.965625+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the released benchmark, compare the depth predicted from a translated image against the ground-truth depth of its simulated source; if the predicted depth is systematically wrong in fold regions, or if replacing the depth estimator with a deliberately poor one still yields the same 11.66-point IoU gain, then the depth-consistency term is not the mechanism responsible for the improvement.","supporting_citations":[{"cited_title":"Murillo S T S T S T Fig","cited_arxiv_id":null,"evidence_quote":"Supplies the real colonoscopy images used as translation target and the 100-frame manually annotated evaluation set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Pretrained depth estimation model (SCSFM-Learner) used in the main experiment to compute the depth-consistency loss on translated images."},{"cited_title":"In: Proceedings of the IEEE Int","cited_arxiv_id":null,"evidence_quote":"Source of the scale-invariant depth log loss used in Eq. (1)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the simulated colon images and their ground-truth depth and fold labels used for training and testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Existing fold-segmentation model for optical colonoscopy that serves as the main quantitative baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Comparison image-translation method evaluated in the quantitative fold-segmentation experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Pretrained endoscopy-specific backbone integrated into the downstream segmentation model."},{"cited_title":"In: Proceedings of the IEEE Int","cited_arxiv_id":null,"evidence_quote":"Base image-translation method whose loss the depth-consistency term extends and whose structure distortion motivates the approach."}],"review_version":1}