{"id":"5bf5155c-9e2b-4181-b5c6-b82c29b111c5","arxiv_id":"2505.06668","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Repurposing Stable Diffusion as an image-to-motion model yields state-of-the-art single-image rectangling and rolling-shutter correction with one-step inference.","lead":"StableMotion adapts a pretrained Stable Diffusion image generator into a one-step motion estimator for fixing distorted images. It reports state-of-the-art results on stitched-image rectangling and rolling-shutter correction, while running over 100 times faster than prior diffusion-based methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SOTA margins are small and unquantified, and Table 1 contains an implausible baseline LPIPS that may invalidate the perceptual-quality claim.","rationale":"I read the paper as making two linked claims: (a) repurposing Stable Diffusion as an image-to-motion estimator gives state-of-the-art quality on DIR-D and RS-Real, and (b) one-step inference delivers that quality at ~200x speedup. The speed claim is well supported by the architecture and implementation details, and the from-scratch ablation in Table 3 provides independent evidence that the pretrained prior matters. The SSD theory is weak and its derivation in Section 3.3/Appendix is internally inconsistent (the T=1 case does not follow from Eq. 17, and the sum includes a Delta_0 term that should not appear), but the method would still be valid if one-step inference is chosen empirically. The most load-bearing uncertainty is therefore whether the reported quality gains are genuine. Several margins are small, no uncertainty quantification is provided, and the RecDiffusion LPIPS of 0.789 in Table 1 is a red flag that the comparison protocol may be inconsistent. The abstract also promises 'no-reference evaluations' on OOD benchmarks, but the full text only shows qualitative examples, so the generalization claim is overstated. These issues do not disprove the method, but they leave the SOTA claim conditional on verification. The reader's conditional verdict is appropriate; my concern shifts the emphasis from SSD to the reliability of the empirical comparison.","tokens_in":13228,"tokens_out":15058,"duration_ms":161173,"concrete_test":"Using the released weights and official test splits, recompute Tables 1 and 2 under one fixed protocol: same LPIPS backbone (e.g., VGG or AlexNet as originally defined), same FID reference set, and mean +/- 95% CI over at least 5 inference seeds for StableMotion and over the released checkpoints for RecDiffusion and RS-Diffusion. Specifically, (1) get RecDiffusion's raw outputs and recompute LPIPS; if it is not close to 0.789, the published baseline is misreported. (2) Bootstrap the RS-Real SSIM and FID differences; if the 0.02 SSIM and 0.5 FID margins overlap with RS-Diffusion, the SOTA claim on that benchmark is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that StableMotion achieves state-of-the-art quality on DIR-D and RS-Real while being ~200x faster. The speed claim is credible from the one-step design and the reported 32 ms inference time, but the quality claim is not yet secured. The margins over the strongest diffusion baselines are small and reported without error bars, test-set sizes, seeds, or significance tests: on RS-Real, SSIM is 0.724 vs 0.704, FID is 5.18 vs 5.68, and StableMotion is actually worse on LPIPS (0.068 vs 0.067); on DIR-D, the PSNR gain over RecDiffusion is only 0.85 dB. More concretely, Table 1 lists RecDiffusion's LPIPS as 0.789, which is implausible for a method whose PSNR is 22.21 dB and whose SSIM is 0.773; this strongly suggests a different LPIPS variant, a typo, or an inconsistent evaluation protocol. If that baseline number is corrected, the DIR-D perceptual SOTA margin could shrink or even invert. The SSD analysis in Section 3.3 is also heuristic and its Taylor derivation is not rigorous, but it is not the load-bearing part: the one-step choice is an empirical design decision and would survive even if SSD were wrong. The load-bearing issue is whether the empirical superiority over prior methods is real and statistically stable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"StableMotion repurposes Stable Diffusion 2.0 into an image-to-motion estimator for two single-image rectification tasks: stitched image rectangling (SIR) and rolling shutter correction (RSC). The VAE is adapted as a flow refiner and the UNet as a motion estimator, with the UNet's first layer replicated and scaled to accept the concatenated condition and flow latents. Training combines diffusion reconstruction loss, pixel-space condition loss, and perceptual loss. Inference uses a one-step DDIM schedule, optionally followed by an Adaptive Ensemble Strategy (AES). The paper reports state-of-the-art PSNR, SSIM, LPIPS, and FID on DIR-D and RS-Real, attributes a substantial speedup (about 200x) to one-step sampling, and introduces Sampling Steps Disaster (SSD) to explain why increasing sampling steps degrades performance. Generalization to unseen OOD data is demonstrated qualitatively and with no-reference evaluations.","tokens_in":13587,"tokens_out":4033,"duration_ms":39798,"significance":"If the empirical results hold, the paper makes a valuable contribution: it shows that a pretrained text-to-image diffusion backbone can be adapted to dense geometric motion estimation with one-step inference, yielding both quality and efficiency gains over task-specific diffusion models trained from scratch. The from-scratch ablation in Table 3 supports the claim that diffusion priors are essential for convergence on a single task dataset, and the one-step inference is validated not only on the proposed model but also on the external MDM model in Table 6. The code/weights release and the reported 32 ms inference time are concrete strengths. However, the central quality claim is not yet statistically secured: the reported margins over baselines are small, no error bars or significance tests are provided, and one baseline LPIPS value in Table 1 is implausible and may invalidate the perceptual SOTA claim on DIR-D.","major_comments":[{"comment":"The LPIPS value reported for RecDiffusion (0.789) is implausible for a method whose PSNR is 22.21 dB and SSIM is 0.773; every other entry in that table lies in the 0.14-0.15 range. This strongly suggests a different LPIPS variant, a typo, or an inconsistent evaluation protocol. Because the DIR-D perceptual-quality SOTA claim (LPIPS 0.136 vs 0.789) depends on this number, the authors must report the exact LPIPS configuration (backbone network, spatial normalization, preprocessing), re-evaluate RecDiffusion with the same code, or correct the table; otherwise the perceptual SOTA margin may shrink or even invert.","section":"Table 1"},{"comment":"The reported SOTA margins are small and no error bars, repeated-run variance, seeds, test-set sizes, or statistical significance tests are provided. On RS-Real, StableMotion is actually worse than RS-Diffusion on LPIPS (0.068 vs 0.067) and its SSIM lead is only 0.02; on DIR-D, the PSNR gain over RecDiffusion is 0.85 dB. Without variance estimates, the claim of state-of-the-art quality is not statistically supported. The authors should add bootstrap confidence intervals across test images or multiple training seeds and state the evaluation protocol explicitly.","section":"Section 4.2, Tables 1-2"},{"comment":"The SSD argument is not a rigorous derivation. Definition 2 defines p as a mapping from PD_t to GD_{t-1}, but Eq. (15) composes p with itself without specifying the time-indexed domains or showing that composition is well-defined; the Taylor expansion in Eq. (17) assumes differentiability and small perturbations, and the conclusion that the error rises exponentially is not established from the sum in Eq. (25). The key premise that joint training with reconstruction and condition losses yields a distribution intermediate between pseudo-label and ground truth is illustrated in Fig. 4 but never verified. The one-step design is empirically supported by Table 6 and Fig. 1, so the paper's core empirical claim does not collapse, but the SSD explanation should be reframed as a hypothesis or made mathematically precise.","section":"Section 3.3, Eq. (17), and Appendix"}],"minor_comments":[{"comment":"The phrase 'a novel framework leverages knowledge' is missing the relative pronoun 'that'; please revise to 'a novel framework that leverages knowledge'.","section":"Abstract"},{"comment":"There is a typo in 'DM-basd methods'; it should read 'DM-based methods'.","section":"Section 1"},{"comment":"The table caption reads 'Ablaiton on loss items'; this should be 'Ablation on loss items'.","section":"Section 4.6, Table 4"},{"comment":"The implementation details report 32 ms inference on an NVIDIA H100, but the speedup comparison in the abstract and introduction does not state the hardware used for the baseline methods; please specify the hardware and measurement protocol.","section":"Section 4.1"},{"comment":"The sentence 'For SIR, the conditions C include stitched images alone with its masks' should read 'along with its masks'.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and addresses a timely problem, but the evaluation section needs substantial strengthening before the SOTA claim can be accepted. The implausible RecDiffusion LPIPS value and the absence of error bars are the main risks; both are fixable within a revision. I do not see grounds for rejection, provided the authors can verify the corrected metrics and either tighten or reframe the SSD explanation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe paper to know: StableMotion takes a pretrained Stable Diffusion backbone, re-wires it to output dense motion fields instead of images, and with one-step DDIM sampling solves stitched-image rectangling and rolling-shutter correction. The speed claim is credible: one-step inference at 32 ms per image, roughly 200x faster than prior diffusion rectification methods. The core repurposing idea is genuinely new in this subfield, and the from-scratch ablation (Table 3) gives real support that the pretrained prior is doing the work.\n\nThe soft spots, in order of seriousness. First, the main quantitative comparison has an internal contradiction. Table 2 reports StableMotion at 22.65 dB PSNR on RS-Real, but the loss ablation in Table 4 reports the full model at 25.50 dB on the same dataset. That is a 2.85 dB gap with no explanation. It could be a different evaluation split or protocol, but as presented the two tables cannot both describe the final model, and the reader has no way to tell which number is the real SOTA. Second, the suspicion about RecDiffusion's LPIPS of 0.789 in Table 1 looks right: for a method with 22.21 dB PSNR and 0.773 SSIM, an LPIPS that high is implausible, which suggests the DIR-D perceptual comparison uses incompatible evaluation code. Third, no error bars, seeds, or significance tests anywhere, and the margins over the strongest diffusion baselines are small (0.02 SSIM on RS-Real; 0.85 dB PSNR on DIR-D). Fourth, the SSD theory in Section 3.3 is heuristic; the Taylor expansion in Eq. 17 is stated rather than justified. That said, the one-step choice does not depend on SSD being correct as a theory, since the empirical step-count degradation is shown in their own experiments and independently on MDM in Table 6.\n\nThe paper is worth a serious referee. A good referee would ask for the RS-Real discrepancy to be resolved, the baseline LPIPS recalculated, and a small battery of seeds to show the margins are stable. If those hold, this is a solid within-subfield contribution.\n\nRecommendation: send it to peer review, but with the evaluation issues front and center in the comments.","headline":"One-step repurposing of Stable Diffusion for motion estimation is a genuinely new idea, but the reported SOTA quality is undercut by an unexplained 2.85 dB discrepancy between the main table and the loss ablation.","tokens_in":14030,"tokens_out":3383,"would_cite":false,"duration_ms":30797,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"StableMotion repurposes Stable Diffusion into a one-step motion estimator that beats task-specific diffusion models with 200x speedup.","keywords":["diffusion prior","motion estimation","stitched image rectangling","rolling shutter correction","one-step inference","sampling steps disaster","adaptive ensemble strategy","Stable Diffusion repurposing"],"falsifier":"Train the same two-loss recipe on a third synthetic flow dataset with ground-truth flows, then run DDIM with $T=1,4,16,64,256$ steps and compare both final PSNR and the per-step latent distribution shift $\\Delta_t=\\mathbb{E}[x_t-y_t]$ (for example, via MMD or FID between the model's intermediate latents and training-time latents). If quality does not degrade with step count, or if the per-step shift does not track the error growth in Eq. (17), SSD as stated is falsified. A simpler disconfirmation would be a task where the same training losses produce better multi-step results, contradicting the claimed universality.","tokens_in":13030,"feed_emoji":"🖼️","tokens_out":7426,"duration_ms":71007,"temperature":0.7,"pith_summary":"StableMotion claims that a pretrained text-to-image diffusion model, Stable Diffusion, can be repurposed as a one-step image-to-motion estimator, and that this beats task-specific diffusion models trained from scratch on two single-image rectification tasks. The authors report state-of-the-art PSNR, SSIM, LPIPS, and FID numbers on the DIR-D stitched-image rectangling benchmark and the RS-Real rolling-shutter benchmark, with roughly 200x faster inference than prior diffusion approaches. The framework maps the input (and, for rectangling, a mask) to a per-pixel flow field that warps the input into the corrected image, using the pretrained VAE as a flow refiner and the adapted UNet as the motion estimator. To justify one-step inference, the paper introduces Sampling Steps Disaster (SSD): with multiple training losses, the model's learned conditional distribution lies between the pseudo-label and ground-truth distributions, so multi-step sampling feeds the model inputs off its training distribution and errors compound. An optional Adaptive Ensemble Strategy (AES) averages several one-step outputs to reduce generative inconsistency and boundary artifacts. If correct, the framework would make large diffusion priors a general, cheap source of geometric supervision for dense prediction tasks.","feed_headline":"One-step diffusion prior beats task-specific models 200x faster","feed_subtitle":"Repurposed Stable Diffusion estimates warp fields directly, with top scores on rectangling and rolling-shutter benchmarks.","key_machinery":"The load-bearing object is the predicted flow field $\\hat{F}=\\theta(C,\\epsilon)$ produced by a repurposed Stable Diffusion backbone, with the VAE re-used as a flow refiner (encoding homogeneous normalized flows $f'=\\mathrm{cat}(F/\\gamma,1)$ and decoding predicted flow latents) and the UNet re-initialized to accept $4(N+1)$ input channels: the latent condition(s) plus the noisy flow latent. The argument also rides on the one-step DDIM sampling recipe, whose justification is the SSD error expansion of Eq. (17): the composite mapping $p=s\\circ\\theta$ carries the training-time distribution $PD_t$ to $GD_{t-1}$, and any gap $\\Delta_t=x_t-y_t$ introduced at one step is multiplied by Jacobians in later steps. AES, the third component, is an adaptive minimum-median filter whose mask is the product of a warped mask $M_{\\mathrm{warp}}=W(M,\\hat{F}_{0|t})$ and a fixed-edge mask $M_{\\mathrm{edge}}$, used to ensemble multiple one-step outputs for the rectangling task.","core_discovery":"The paper's central claim is that the architecture and weights of Stable Diffusion can be adapted, with minimal input-channel surgery, into an image-to-motion model: the UNet takes the concatenation of latent condition features and a noised latent flow, and predicts a denoised flow feature; the VAE decodes it into a normalized homogeneous flow $f'=\\operatorname{cat}(F/\\gamma,1)$, which after denormalization warps the condition image via $W(I_{\\mathrm{cond}},\\hat{F})$ to produce the rectified output. Training combines the diffusion reconstruction loss with a pixel-space warp-condition loss and a perceptual loss, which together steer the model toward a conditional distribution intermediate between the pseudo-label and ground-truth distributions. Inference is a single DDIM step from pure noise, a choice justified by the paper's SSD analysis: the first-order Taylor expansion of the error between a multi-step chain and a corrected chain yields an error that sums products of the distribution shifts $\\Delta_t$ and Jacobians, growing with step count and vanishing when $T=1$. The empirical backbone is a reported state-of-the-art result on DIR-D and RS-Real, plus an ablation showing the same architecture trained from scratch on one dataset fails to converge, and that an image-to-image variant of SD converges to an inferior PSNR. This combination—pretrained prior, auxiliary-loss training, and one-step sampling—is what the authors say accounts for both the quality and the approximately 200x speedup over prior diffusion methods.","pith_inferences":["If SSD is a general property, then many conditional diffusion models trained with perceptual, adversarial, or task losses—not just motion estimators—should also show non-monotonic step-count behavior, so practitioners should sweep step counts rather than assume more steps help.","One-step inference plus a random noise draw effectively turns the adapted diffusion model into a near-deterministic flow regressor with stochastic perturbation; ensembling over runs, as AES does, averages out that perturbation and may be essential for stable outputs.","The SSD argument's Taylor expansion assumes the composite map $p$ is smooth enough for first-order propagation; if the denoiser is highly non-Lipschitz near the data manifold, multi-step errors could grow faster than the formula suggests, which is a testable prediction about the shape of the error curve.","A natural testable extension is to apply the same two-loss recipe to other dense prediction tasks such as optical flow or depth refinement; the paper's ablation predicts that without pretrained priors the architecture will fail to converge on small datasets."],"forward_implications":["On DIR-D and RS-Real, a pretrained Stable Diffusion backbone outperforms diffusion models trained from scratch for the same tasks, suggesting that task-specific diffusion training may no longer be the default for geometric correction.","One-step DDIM sampling brings per-image inference to about 32 ms on an H100, roughly 200x faster than prior diffusion-based motion estimators even with AES post-processing.","The SSD phenomenon implies that when a diffusion model is trained with auxiliary losses, sampling steps should be tuned downward rather than upward; the model's best output can come from a single step.","The same weights transfer without fine-tuning to unseen stitched-image and real rolling-shutter data, indicating the pretrained semantic and geometric prior is what generalizes.","The framework could in principle be re-targeted to other single-image correction tasks by changing the condition channel count and retraining on their flow datasets."],"supporting_citations":[{"why":"Supplies the pretrained latent diffusion backbone (VAE and UNet) whose weights and architecture the framework adapts.","marker":"[33]"},{"why":"Provides the strongest diffusion-based baseline for stitched-image rectangling that StableMotion claims to beat, and its numbers in the appendix give independent evidence of SSD.","marker":"[52]"},{"why":"Provides the diffusion-based rolling-shutter correction baseline and the RS-Real test set used for the comparison.","marker":"[46]"},{"why":"Provides the DIR-D dataset and the deep-learning rectangling baseline used for training and evaluation.","marker":"[28]"},{"why":"Provides the DDIM scheduler that enables the single-step inference recipe.","marker":"[36]"},{"why":"Supplies the conditional-sampling formalism the framework builds on, though the final system uses the full-condition setting.","marker":"[9]"},{"why":"Supplies the DDPM forward/noising process used to train the adapted UNet.","marker":"[10]"}],"fun_headline_variants":["Diffusion prior enables 200x faster one-step motion estimation","Stable Diffusion repurposed for motion: 200x speedup","One-step motion from Stable Diffusion prior, 200x faster","Pretrained diffusion prior cuts motion steps to one, 200x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole justification for one-step inference rests on the assumption that training with both reconstruction and pixel-space losses puts the learned conditional distribution strictly between the pseudo-label and ground-truth distributions at every noise level, so that multi-step sampling feeds the model inputs it never saw in training; if that in-between property or the first-order Taylor picture fails, one-step may still work empirically but the paper's stated mechanism would collapse.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion prior enables 200x faster one-step motion estimation","Stable Diffusion repurposed for motion: 200x speedup","One-step motion from Stable Diffusion prior, 200x faster","Pretrained diffusion prior cuts motion steps to one, 200x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1440,"prompt_tokens":1077,"completion_tokens":363,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":289}},"tokens_in":693,"tokens_out":363,"duration_ms":3642,"temperature":1.0,"reasoning_tokens":289,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:36:01.124801+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same two-loss recipe on a third synthetic flow dataset with ground-truth flows, then run DDIM with $T=1,4,16,64,256$ steps and compare both final PSNR and the per-step latent distribution shift $\\Delta_t=\\mathbb{E}[x_t-y_t]$ (for example, via MMD or FID between the model's intermediate latents and training-time latents). If quality does not degrade with step count, or if the per-step shift does not track the error growth in Eq. (17), SSD as stated is falsified. A simpler disconfirmation would be a task where the same training losses produce better multi-step results, contradicting the claimed universality.","supporting_citations":[{"cited_title":"High-resolution image syn- 9 thesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained latent diffusion backbone (VAE and UNet) whose weights and architecture the framework adapts."},{"cited_title":"Recdiffu- sion: Rectangling for image stitching with diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the strongest diffusion-based baseline for stitched-image rectangling that StableMotion claims to beat, and its numbers in the appendix give independent evidence of SSD."},{"cited_title":"Single Image Rolling Shutter Removal with Diffusion Models","cited_arxiv_id":"2407.02906","evidence_quote":"Provides the diffusion-based rolling-shutter correction baseline and the RS-Real test set used for the comparison."},{"cited_title":"Deep rectangling for image stitching: A learning base- line","cited_arxiv_id":null,"evidence_quote":"Provides the DIR-D dataset and the deep-learning rectangling baseline used for training and evaluation."},{"cited_title":"Denois- ing diffusion implicit models","cited_arxiv_id":null,"evidence_quote":"Provides the DDIM scheduler that enables the single-step inference recipe."},{"cited_title":"Classifier-free diffusion guidance","cited_arxiv_id":null,"evidence_quote":"Supplies the conditional-sampling formalism the framework builds on, though the final system uses the full-condition setting."}],"review_version":1}