{"id":"7cd6ec4e-20fc-4ad5-89b8-ad0c1e28544c","arxiv_id":"2411.10309","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RDIStitcher fuses and rectangles stitched images by reference-driven inpainting with a LoRA-fine-tuned Stable Diffusion model, trained without labeled stitching data.","lead":"An image stitching method fills the fusion area using a text-to-image diffusion model fine-tuned with a self-supervised scheme that needs no labeled stitching pairs, producing smoother panoramas in hard cases like uneven lighting and large parallax. The paper also proposes using vision-language models as automated quality judges for stitched images.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on an untested transfer step: Sec. 3.3’s pseudo-pairs are masks of one image, not two true views; the model is never forced to resolve genuine parallax or occlusion, so the claimed generalization to real multi-view stitching is not established.","rationale":"The reader's weakest assumption identified both the self-supervised transfer and the MLLM metric validation. I agree that the transfer step is the most load-bearing, and the reader's CONDITIONAL verdict already reflects this. My concern is more specific than the reader's broad statement: the pseudo-pair construction in Eq. 3 does not simulate genuine multi-view conflict, and the paper supplies no controlled evidence that the learned operation transfers to true two-view inputs. The MLLM metrics are a secondary issue; even if the metrics align with human perception, they cannot validate geometric correctness of the fusion. I therefore keep the verdict at CONDITIONAL but would make the condition concrete: add a synthetic or known-homography experiment that directly tests the transfer hypothesis, and report the augmentation-ablation comparison suggested above. The paper has real strengths: a clean reference-driven inpainting formulation, a small-parameter LoRA fine-tuning scheme, released code, and a new evaluation direction. Those strengths do not remove the need for a direct test of the central generalization claim.","tokens_in":17817,"tokens_out":4596,"duration_ms":51520,"concrete_test":"Construct a synthetic multi-view test set from a 3D scene (or a calibrated stereo pair): render two images with a known homography plus controlled parallax and unequal illumination, compute the ground-truth stitched composite, then run RDIStitcher's released inference. Measure overlap-region PSNR/SSIM against the ground truth and reprojection error of known feature points. Separately retrain RDIStitcher under three conditions: (a) the Sec. 3.3 pseudo-pair procedure; (b) the same procedure but with true two-view synthetic pairs instead of single-image masks; (c) the Sec. 3.3 procedure with color jitter and affine augmentation probabilities set to zero. If (a) does not outperform (c) on the UDIS-D test set, the augmentations are not driving transfer; if (b) outperforms (a), the single-image pseudo-pair construction is the bottleneck.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is Sec. 3.3's self-supervised transfer hypothesis. In Eq. 3, both pseudo-views are produced from the same single image Isg by different masks (M_i_wr, M_i_wt) borrowed from UDIS-D; the diffusion target x0 is the original Isg. Thus training always presents a reference and target that are two masked copies of one image. The only simulated stitching difficulties are color jitter on the reference (p=0.25) and a global affine translation of the reference (p=0.25). No experiment forces the model to handle local parallax, object occlusion, independent illumination changes, or content that exists in one view but not the other. In pseudo-training, the reference contains exactly the pixels to be recovered, so the model can succeed by copying or by relying on its generative prior; it is never required to combine two genuinely different observations. The abstract's claim of 'strong generalization capabilities' in uneven-hue and large-parallax scenarios therefore rests on an unverified assumption: that Stable Diffusion's prior converts single-image masked inpainting into true two-view stitching at inference. The qualitative zero-shot results are suggestive but not a controlled test of this transfer; no quantitative comparison against ground-truth geometry is reported. If the transfer fails, the main advantage over SRStitcher disappears, because SRStitcher also uses a frozen T2I prior but with a smaller modification area. The consistency scores in Table 4 measure similarity to UDIS, not correctness of the underlying geometric fusion. This is the most load-bearing concern because it directly supports the paper's central claim of seamless, coherent stitching in difficult cases.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes RDIStitcher, a diffusion-based image stitching method that reformulates fusion and rectangling as reference-driven inpainting. A Stable Diffusion inpainting model is fine-tuned with LoRA using a self-supervised scheme in which single-view images are combined with masks from UDIS-D to create pseudo-stitching pairs; color jitter and affine shifts simulate hue differences and misalignment. The authors also propose two MLLM-based evaluation metrics (SIQS and MICQS) and report experiments on UDIS-D and zero-shot traditional datasets, claiming state-of-the-art performance in uneven-hue and large-parallax scenarios.","tokens_in":18129,"tokens_out":6371,"duration_ms":57122,"significance":"If the claims are borne out, the paper would contribute a new training paradigm for inpainting-based stitching, including what the authors describe as the first unsupervised training method for the rectangling problem, as well as a reusable automated evaluation metric for stitched images. The release of code and the use of five-seed reporting with standard deviations are positive aspects of the work. However, the significance is conditional on two load-bearing premises: that the self-supervised pseudo-pairs teach a concept that transfers to true multi-view stitching, and that the proposed MLLM metrics are a reliable proxy for human perception. The current evidence for both premises is suggestive but not conclusive.","major_comments":[{"comment":"The self-supervised training signal is structurally different from the test-time task. In Eq. (3), both pseudo-views are produced by masking the same single image Isg, so the reference always contains the exact pixels to be recovered. The only simulated difficulties are color jitter (Eq. 4) and a global affine shift (Eq. 5); there is no local parallax, no object occlusion, and no content that exists in only one of the two views. The model can therefore succeed by copying from the reference or by invoking its generative prior, without ever learning to combine two genuinely different observations. The paper's claim of strong generalization capabilities to real uneven-hue and large-parallax pairs (Abstract, Sec. 4.4) rests on the unverified assumption that the Stable Diffusion prior converts this single-image masking task into true two-view stitching. I recommend a quantitative test of this transfer, for example on real pairs with known correspondences or with objects visible in only one view, comparing RDIStitcher against a model trained on actual two-view pseudo-pairs.","section":"Sec. 3.3, Eqs. (3)-(5)"},{"comment":"The main quantitative evidence for superiority uses the authors' own MLLM metrics. These metrics are validated on only 50 hand-crafted images rated by four volunteers (Supp. Sec. G), and the reported correlations are moderate (SRCC 0.728 for SIQS-Q, 0.634 for SIQS-G; PLCC 0.685 and 0.525). On the UDIS-D test set, the gaps between RDIStitcher and the strongest baseline are 0.05-0.30 points on a 10-point scale, with overlapping standard deviations (e.g., SIQS-Q 9.54±0.07 vs 9.49±0.05; SIQS-G 9.39±0.05 vs 9.33±0.04). This does not support the claim of significant improvement. The paper should report paired statistical tests, effect sizes, and ideally validate the metrics on a larger, more diverse set of stitched images before using them as the primary endpoint.","section":"Table 3; Sec. 4.2 and 4.6"},{"comment":"The comparison with SRStitcher is unequal: SRStitcher is a frozen, zero-shot method, while RDIStitcher is fine-tuned on UDIS-D training data. The authors acknowledge this (the comparison is somewhat less fair), but the central claim of outperforming the inpainting-based SOTA is drawn from this comparison. Since SRStitcher also uses a smaller modification area, the comparison cannot isolate the effect of modification area or intensity, which is the paper's organizing principle. To make the claim convincing, the authors should fine-tune SRStitcher under equivalent conditions (or a controlled variant with matched modification area) or explicitly restrict the claim to zero-shot transfer.","section":"Sec. 4.1"},{"comment":"The consistency evaluation measures PSNR/SSIM between RDIStitcher's output and UDIS's output, not against ground truth or the original input images. High agreement with UDIS is not by itself evidence of content preservation; it may indicate that RDIStitcher makes only small changes relative to a particular baseline. The conclusion that the method preserves content consistency is therefore not supported by these numbers. I suggest evaluating fidelity to the original reference and target images in non-overlapping regions, or using correspondence-based metrics, and reporting the variance across the five seeds for these results as well.","section":"Sec. 4.5, Table 4"}],"minor_comments":[{"comment":"There are typos in the supplementary material: 'cnfiguration' should be 'configuration' and 'we don not test' should be 'we do not test'.","section":"Supp. E.1 and Supp. D.2"},{"comment":"The notation Concat(·) and the exact tensor layout of the model input are not formally defined; please specify the channel dimensions and ordering explicitly rather than relying on Fig. 3.","section":"Sec. 3.2"},{"comment":"The meaning of the 'Both Good' and 'Both Bad' categories in the stacked bar charts is not defined in the caption; please state how these labels were assigned.","section":"Fig. 4"},{"comment":"The paper claims to be the first unsupervised training method for the rectangling problem, but the related work section does not discuss prior unsupervised rectangling attempts; if none exist, state this explicitly in the related work section.","section":"Sec. 1 and 3.3"},{"comment":"The symbol I is used both for the input image and for the identity matrix; please use a distinct symbol such as I_3 for the identity matrix to avoid ambiguity.","section":"Eq. (1)"},{"comment":"The use of red highlighting to indicate the best result is not accessible in printed or color-blind formats; please add boldface or symbols in addition to color.","section":"Tables 3 and 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is transparent about several limitations (small user study, unequal SRStitcher comparison, limited MLLM validation), and the code release is a strength. My main concern is evidential: the central empirical claim depends on metrics and a training hypothesis that are not yet rigorously validated. I do not think this warrants rejection, but the revision should provide the additional evidence described in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The self-supervised training recipe is the real contribution here. Simulating pseudo-stitched pairs from a single image with mask priors and then fine-tuning a T2I diffusion model is genuinely new, and the MLLM-based quality metrics (SIQS/MICQS) are a useful prompt for a field that badly needs cheap evaluation. The code is shipped, the method is reproducible, and the qualitative results on hard cases look real.\n\nThe soft spots are proportionate to how much weight they carry. The load-bearing assumption is that training on two masked copies of one image, with color jitter or an affine shift, teaches a stitching concept that transfers to genuine two-view inputs. The stress-test note gets this right: the model is never forced to resolve true parallax or occlusion, and the paper offers no quantitative zero-shot evaluation on datasets with ground truth. That leaves the central generalization claim resting on the generative prior of Stable Diffusion, which is plausible but unverified.\n\nThe quantitative evidence in Table 3 is also thinner than the prose suggests. The differences over the best baseline are 0.05 to 0.3 points on a 10-point scale, and the standard deviations overlap for several comparisons. The word \"significantly\" does not follow from those numbers. The comparison with SRStitcher is unequal, as the authors admit, since SRStitcher is not fine-tuned on the target dataset. The MLLM metric calibration uses 50 images and four volunteers, and the supplementary notes that MLLMs struggle with the concept of \"artifact\"—so the metric is preliminary, not settled.\n\nThat said, the paper earns credit for honesty. It lists limitations, it compares against LeftRefill as an additional baseline, and the consistency evaluations on DS/DL are concrete and favorable. The self-supervised method is worth discussing even if the current evidence is not conclusive.\n\nMy recommendation: send it to peer review. The idea is strong enough to warrant referee time, but a major revision should demand a direct test of the transfer hypothesis—either train on real multi-view pairs or evaluate zero-shot quantitatively on traditional datasets—and a larger, independent human study for the metrics. I would not cite the current version's strong claims, but I would cite the self-supervised training idea once it is properly validated.","headline":"A plausible stitching paper with a novel self-supervised training idea, but the claimed generalization to real parallax scenes is not yet supported.","tokens_in":18705,"tokens_out":2636,"would_cite":false,"duration_ms":27620,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that treating image fusion and border straightening as reference-driven inpainting, trained on pseudo-stitched single photos, removes visible seams in uneven-hue and large-parallax stitching, and that multimodal language…","keywords":["image stitching","reference-driven inpainting","diffusion model","self-supervised learning","text-to-image","image quality assessment","MLLM-based metrics","large parallax"],"falsifier":"Collect a few hundred real multi-view image pairs covering uneven hue and large parallax, have independent human raters choose between RDIStitcher and each baseline without knowing which is which, and check whether the automated scores agree with those choices; if RDIStitcher does not win a clear majority, or if its automated scores disagree with the raters, the central claims are falsified.","tokens_in":17621,"feed_emoji":"🖼️","tokens_out":8440,"duration_ms":76465,"temperature":0.7,"pith_summary":"Stitched panoramas often fail where the input photos differ in lighting or viewpoint: a visible seam or a ghosted object marks the boundary. The paper's central claim is that this failure disappears if the fusion and border-straightening steps are treated as one reference-driven inpainting task, with a deliberately large and strongly modified fusion region. To make that training feasible without labeled stitching data, the paper builds pseudo-stitched images from single photos by cutting out stitching-shaped masks and adding color shifts and affine shifts, then fine-tunes a text-to-image diffusion model to restore the original photo. It also argues that multimodal language models can score stitched-image quality automatically. A sympathetic reader would care because the method offers a path to seamless stitching in hard real-world cases and an automated way to measure what previous methods could only judge by small user studies.","feed_headline":"Bold inpainting removes seams from hard photo stitching","feed_subtitle":"A diffusion model fine-tuned on pseudo-stitched photos handles uneven lighting and large viewpoint gaps.","key_machinery":"The load-bearing object is the reference-driven inpainting model $\\epsilon_\\theta(\\cdot)$, implemented as a pretrained text-to-image diffusion model fine-tuned on a small number of trainable parameters. Its input concatenates a coarse-rectangled reference image with the target image, a mask marks the missing seam and border regions, and a special text prompt binds the generation to the stitching task; the output keeps the target side and fills the masked side. Training is powered by a pseudo-stitching scheme: the mask distribution of a real unlabeled stitching dataset is cut onto single-view images, with color jitter and random affine transforms simulating the two failure modes, and the model minimizes the standard denoising diffusion objective. The evaluation machinery is a pair of multimodal-language-model prompts, one scoring a single image on seams, brightness transitions, distortion, clarity, and abnormal content, and one comparing two stitched images.","core_discovery":"The paper's central discovery is that the fusion and rectangling stages of stitching can be collapsed into a reference-conditioned inpainting problem and that this model can be taught without ground-truth stitched pairs. Masks sampled from a real stitching dataset are applied to single-view images, augmented with color jitter to imitate uneven hue and random affine translations to imitate parallax misalignment, and the model is trained to recover the original image from this pseudo-stitched input. At inference, the target image is kept fixed while the model inpaints the seam and border regions using the reference image as guidance, accepting a larger modification area and stronger modification intensity than previous inpainting-based stitching. The claimed payoff is higher content coherence and seamless transitions in uneven-hue and large-parallax scenes, including zero-shot transfer to datasets never seen in training. For evaluation, the paper introduces a multimodal-large-language-model protocol: a single-image quality score covering seams, brightness transitions, distortion, clarity, and abnormal content, and a comparative score that chooses the better of two stitched images.","pith_inferences":["The two data augmentations could be isolated in ablation: training with color jitter only and with affine shifts only would test whether each simulated failure mode transfers independently to its real counterpart.","The same pseudo-stitching recipe should transfer to other generative backbones, since the mask-distribution trick is not tied to the chosen text-to-image model.","If the central claim holds, the practical trade-off is that users accept generative edits inside the fusion region; the consistency numbers in the paper measure global similarity, not whether thin structures such as wires or pillars are semantically preserved.","The multimodal-language-model metrics, once validated on a larger human study than the 50-image pilot described in the supplement, could become the default evaluation instrument for unsupervised stitching research."],"forward_implications":["Stitching pipelines can drop the separate supervised rectangling stage: fusion and border straightening become one inpainting step trained from unlabeled photo collections.","Hard cases that previously forced a choice between visible seams and distorted content can be handled by permitting a larger edited region, as long as the reference image stays coherent with the target.","The self-supervised concept transfers zero-shot to unseen stitching datasets, so new domains such as aerial, medical, or remote-sensing mosaics could be stitched without collected ground-truth pairs.","Stitched-image quality can be assessed automatically at scale with multimodal-language-model scores, replacing small manual user studies in future work."],"supporting_citations":[{"why":"Supplies the unlabeled UDIS-D stitching dataset, the pretrained registration model whose warp is used for alignment, and the mask distribution that drives pseudo-stitching training.","marker":"[23]"},{"why":"Provides the pretrained text-to-image model whose U-Net and text encoder are fine-tuned as the inpainting backbone.","marker":"[2]"},{"why":"Defines the inpainting-based fusion-rectangling framework and the content consistency score that RDIStitcher extends and is compared against.","marker":"[51]"},{"why":"Supplies the reference-left/target-right input construction that RDIStitcher adapts to stitching.","marker":"[6]"},{"why":"Provides low-rank adaptation, the parameter-efficient fine-tuning mechanism that keeps training feasible.","marker":"[14]"},{"why":"Defines the denoising diffusion objective used as the training loss.","marker":"[13]"},{"why":"Contributes the unique-identifier prompt trick used to bind the generation to the stitching task.","marker":"[30]"},{"why":"Provides the fast-marching inpainting routine that builds the coarse rectangling prior.","marker":"[33]"}],"fun_headline_variants":["Reference-driven inpainting removes stitching seams","Diffusion model stitches without labeled data","Seamless stitching via self-supervised inpainting","RDIStitcher: reference inpainting for seamless panoramas","Zero-shot stitching with diffusion inpainting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a model trained to restore a single photo from pseudo-stitched versions of that same photo—masked, color-shifted, and affine-warped using masks sampled from real stitching data—learns a stitching concept that transfers to genuinely different multi-view pairs.","fun_headline_variants_meta":{"raw":{"variants":["Reference-driven inpainting removes stitching seams","Diffusion model stitches without labeled data","Seamless stitching via self-supervised inpainting","RDIStitcher: reference inpainting for seamless panoramas","Zero-shot stitching with diffusion inpainting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000588,"raw_usage":{"total_tokens":2761,"prompt_tokens":946,"completion_tokens":1815,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":1741}},"tokens_in":562,"tokens_out":1815,"duration_ms":13036,"temperature":1.0,"reasoning_tokens":1741,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:45:09.767193+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a few hundred real multi-view image pairs covering uneven hue and large parallax, have independent human raters choose between RDIStitcher and each baseline without knowing which is which, and check whether the automated scores agree with those choices; if RDIStitcher does not win a clear majority, or if its automated scores disagree with the raters, the central claims are falsified.","supporting_citations":[{"cited_title":"Unsupervised deep image stitching: Reconstructing stitched features to images","cited_arxiv_id":null,"evidence_quote":"Supplies the unlabeled UDIS-D stitching dataset, the pretrained registration model whose warp is used for alignment, and the mask distribution that drives pseudo-stitching training."},{"cited_title":"Stable diffusion 2 inpainting","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained text-to-image model whose U-Net and text encoder are fine-tuned as the inpainting backbone."},{"cited_title":"Reconstructing the image stitching pipeline: Integrat- ing fusion and rectangling into a unifiedinpainting model","cited_arxiv_id":null,"evidence_quote":"Defines the inpainting-based fusion-rectangling framework and the content consistency score that RDIStitcher extends and is compared against."},{"cited_title":"Leftrefill: Filling right canvas based on left refer- ence through generalized text-to-image diffusion model","cited_arxiv_id":null,"evidence_quote":"Supplies the reference-left/target-right input construction that RDIStitcher adapts to stitching."},{"cited_title":"LoRA: Low-rank adaptation of large language models","cited_arxiv_id":null,"evidence_quote":"Provides low-rank adaptation, the parameter-efficient fine-tuning mechanism that keeps training feasible."},{"cited_title":"An image inpainting technique based on the fast marching method","cited_arxiv_id":null,"evidence_quote":"Provides the fast-marching inpainting routine that builds the coarse rectangling prior."}],"review_version":1}