Pith. sign in

REVIEW 5 major objections 6 minor 105 references

Inpainting models that invent objects and shift colors can be fixed by injecting a reconstruction prior into the frozen backbone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

ASUKA uses MAE priors and a harmonization VAE decoder to reduce hallucinated objects and color shifts in latent diffusion inpainting.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection ASUKA-II is a credible, well-engineered follow-up to the authors' CVPR paper — the MAE-prior training trick has a real leakage concern, but the tables show it isn't the whole story; send to referees. the 5 major comments →

arxiv 2601.15368 v3 pith:KGSZGTZO submitted 2026-01-21 cs.CV eess.IV

Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency

Yikai Wang , Junqiu Yu , Chenjie Cao , Xiangyang Xue , Yanwei Fu This is my paper
classification cs.CV eess.IV MSC 68T45
keywords image inpaintingobject hallucinationcolor consistencymasked autoencoder priorlatent diffusionrectified flowdecoder harmonizationevaluation benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing diffusion- and rectified-flow inpainting models still hallucinate objects in masked regions and produce color-shifted outputs. This paper argues that both flaws can be mitigated without retraining the generative backbone, by injecting a Masked Auto-Encoder (MAE) reconstruction prior into the frozen model through a lightweight alignment module, and by re-training the VAE decoder as a local harmonizer conditioned on visible pixels. The paper claims that this post-hoc framework, ASUKA, works on both U-Net-based Stable Diffusion and transformer-based FLUX, and that it outperforms standard inpainting methods on Places2 and on a new benchmark, MISATO, including on explicit hallucination and color-consistency metrics.

Core claim

The central claim is that unwanted object insertion and color inconsistency in latent inpainting models come from (1) reliance on spurious semantic noise and text conditioning, and (2) the VAE decoder's low-frequency reconstruction loss and the gap between generated and real latents. ASUKA addresses the first by replacing text conditioning with a MAE-derived context-stable prior, aligned to the frozen generator's conditioning space via a linear dimension mapping plus self-attention distribution alignment; the second by fine-tuning the VAE decoder on color- and latent-augmented inputs, reformulating decoding as local harmonization. On Places2 and the new MISATO benchmark, ASUKA reports lower

What carries the argument

The two load-bearing components are (i) the MAE context-stable prior—a masked autoencoder fine-tuned on inpainting-style masks—whose predictions are aligned to the frozen generator by an alignment module (a linear layer for dimension matching followed by self-attention blocks), and (ii) a color-consistent decoder, a VAE decoder fine-tuned as a local harmonizer with color and latent augmentation. For transformer backbones, ASUKA-II adds cross-attention condition injection with LoRA and gating, and scaled positional encoding to map the low-resolution MAE grid onto the high-resolution latent.

Load-bearing premise

The training trick of Section IV-A, where the alignment module is trained on MAE reconstructions of the fully unmasked image with probability p decaying from 100% to 10%, assumes that the module's learned use of that leaked information transfers to the masked-only predicted prior at inference; if the frozen generator relies on content that only exists in the full-image reconstruction, hallucination suppression will weaken on genuinely novel masks.

What would settle it

A controlled experiment on a set of images where the masked object is semantically incongruent with the surroundings (e.g., a rare object in a common scene), comparing ASUKA with p decay vs. p=0 throughout training: if hallucination counts on these novel masks do not drop below the baseline, the leaked-prior training is load-bearing. More directly, ablation at inference with the reconstructed prior removed (i.e., only the masked-predicted prior available) should show whether the model still suppresses hallucination.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Object hallucination in frozen inpainting models can be suppressed by substituting a reconstruction-based prior for text conditioning, so the fix is plug-and-play and does not require re-training the generator.
  • A single harmonizing decoder improves color consistency and transfers to text-guided inpainting, suggesting decoder-side corrections are a cheap, generalizable repair for latent generative models.
  • The approach transfers from U-Net to MMDiT-style backbones with only lightweight modifications, indicating the same recipe may apply to future latent generators.
  • The proposed VLM-based hallucination metric and G@e color metric provide evaluation protocols that are more aligned with human judgment than CLIP similarity and can be reused by the community.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 'handle misalignment' training trick, which feeds the MAE reconstruction of the full unmasked image with probability decaying from 100% to 10%, may leak information about the masked region into the alignment module; if so, the reported hallucination suppression at inference could be optimistic on truly novel masks. This is an inference, not a claim proven in the paper.
  • Because the backbone is frozen, the upper bound on quality is set by the original generator; ASUKA's improvements are corrections on top of that bound, so the framework will be most effective when the base model is already strong.
  • The MAE prior is a mean estimate, so replacing it with a sharper reconstruction prior (e.g., a Diffusion prior or a learned regressor) might trade off hallucination suppression against texture fidelity—a direction the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes ASUKA, a post-hoc framework to reduce unwanted object insertion and color inconsistency in latent inpainting models. It injects a Masked Auto-Encoder (MAE) reconstruction prior into frozen SD1.5 and FLUX.1-Fill-dev backbones via a lightweight alignment module (ASUKA-I-SD, ASUKA-I-FLUX, ASUKA-II-FLUX) and introduces a VAE decoder fine-tuned as a local harmonizer conditioned on the masked image. The paper also introduces a new evaluation set, MISATO, and two new metrics (G@e and a VLM-based hallucination judge). Experiments on Places2, MISATO, CelebA-HQ, and FFHQ report improvements over GAN, diffusion, and rectified-flow baselines on LPIPS/FID/IDS/G@e, reductions in VLM/human hallucination counts, and user-study preference.

Significance. If the results hold, this is a practical contribution: it shows that frozen latent inpainting backbones can be steered with a reconstruction prior to reduce hallucination while preserving generative quality, and that a harmonization-trained decoder can fix color shifts. The paper includes extensive ablations (decoder, alignment module, p-schedule, module contributions) and tests on a new multi-domain benchmark, which strengthens the empirical case. The p=0 control in Table VII is a good-faith check that the core idea has merit without the training leak. However, several load-bearing training and evaluation details are missing, and the training leakage from full-image MAE priors clouds the causal interpretation of the hallucination reduction. The proposed evaluation methodology (MISATO, VLM judge, G@e) is potentially useful but needs precise specification and validation.

major comments (5)
  1. [Section IV-A, Table VII] The alignment module is trained with the MAE reconstructed prior obtained by running MAE on the full unmasked image, with probability p decaying from 100% to 10% and then frozen. This prior has access to the masked-region content, whereas at inference only the masked-input predicted prior is available. The central claim is that a prior derived from the unmasked region suppresses hallucination, but a 10% leak remains at convergence. The p=0 ablation shows the method improves over SD without the leak, but Table VII does not report hallucination counts for p=0/p=1, and the gap between p=0 and Ours (U-IDS 0.403 vs 0.423; FID 11.804 vs 11.495) could be inflated by the leak. Please report VLM/human hallucination numbers for p=0, p=1, decay-to-0, and Ours, or remove the residual 10% leak.
  2. [Section V-A, Tables I-II] ASUKA-I-FLUX is absent from the main quantitative tables and appears only in Table III (VLM counts). The text claims ASUKA-I-FLUX 'consistently outperforms' FLUX-Fill, but no LPIPS/FID/IDS/G@e values are reported. Since ASUKA-II is motivated by the limitations of ASUKA-I-FLUX, the numerical case is incomplete. Please add full ASUKA-I-FLUX rows to Tables I and II (with error bars).
  3. [Section V, Object hallucination evaluation] The VLM protocol is the primary quantitative evidence for hallucination suppression, but it is under-specified. The manuscript does not give the exact prompt, the composite-image layout, output parsing, number of VLM calls, temperature, or whether counts are number of flagged images. No agreement statistic (e.g., Cohen's kappa) between VLM and human judgments is reported. Because the central claim rests on this metric, please provide the full protocol and validation.
  4. [Section V, Color-consistency evaluation (G@e)] The G@e definition is only verbal: 'average pixel gradient difference along the boundary of the masked region with respect to the ground-truth image.' The exact computation is absent, and the measure may reward blur (a smoothed boundary trivially lowers gradients). Please provide the formula, kernel/radius, normalization, and validate against the human CC preferences or a blurred baseline to show that G@e captures color consistency rather than boundary smoothness.
  5. [Tables I-X] All quantitative results are single-run, with no error bars or significance tests. For stochastic latent diffusion/flow models, small differences (e.g., Places2 LPIPS 0.174 vs 0.178; U-IDS 0.410 vs 0.403) may be within run-to-run noise. Please report mean±std over at least 3 seeds and, for main comparisons, paired significance tests, or explicitly state that the differences are not statistically assessed.
minor comments (6)
  1. [Abstract and Conclusion] The claim of 'outperforming existing diffusion- and rectified flow-based inpainting methods' is too broad; Table I shows FLUX-Fill beats ASUKA-I-SD on Places2 LPIPS (0.178 vs 0.183), and ASUKA-I-SD beats ASUKA-II-FLUX on Places2 FID and U-IDS. Qualify with 'on most metrics' and cite the tables.
  2. [Eq. (2)-(3)] The gate argument in Eq. (3) writes f_mae + f_task without layer indices i; align notation with Eq. (2).
  3. [Table III] Human judgment is reported only for FLUX-based models. Add a human-evaluated subset for SD baselines so the VLM-vs-human agreement can be assessed for the SD family too.
  4. [Tables I-II] G@e values differ by an order of magnitude between Places2 and MISATO; explain whether this reflects mask sizes, resolutions, or metric calibration.
  5. [Section IV-A] Specify which dataset(s) are used for MAE fine-tuning and the exact number of MAE fine-tuning steps; 'batch size 1024' alone is not sufficient for reproduction.
  6. [Section V, Implementation Details] The sentence 'set p as 100% and linearly decay it to 10% in the first 2K training steps and then freeze' is ambiguous: clarify whether p or the alignment module weights are frozen.

Circularity Check

0 steps flagged

No significant circularity: ASUKA's main claims are validated by external benchmarks, ablations, and independent evaluations; the disclosed train/inference mismatch in the MAE prior is a correctness risk, not a definitional or by-construction circularity.

full rationale

The paper's core derivation chain is an empirical post-training pipeline: an MAE prior is injected into frozen SD1.5/FLUX inpainting models via an alignment module, and a VAE decoder is fine-tuned as a local harmonizer. The claimed improvements are measured on held-out Places2, MISATO, CelebA-HQ, and FFHQ data against external baselines (LaMa, Co-Mod, MAT, MAE-FAR, SD-Repaint, FLUX-Fill, IP-Adapter, T2I-Adapter, CAEv2), plus VLM and human hallucination judgments and a user study. No prediction in the paper reduces by construction to a fitted parameter or to an input equation. The closest concern is the 'Handle misalignment' training trick (Section IV-A), where the alignment module is trained with probability p on the MAE reconstructed prior obtained by running MAE on the full unmasked image, which contains the masked-out content. This is an explicit train/inference distribution mismatch and is disclosed by the authors: 'The MAE reconstructed prior is obtained by running MAE on the full image without masking, so it has access to all information for reconstruction.' That is a legitimate correctness/leak concern about causal attribution, but it is not circularity: at inference only the masked-only predicted MAE prior is used, and the paper's own p=0 ablation (Table VII) still yields large gains over SD (FID 11.804 vs. 12.812 on MISATO), so the leak is not the sole driver and the final claim is not equivalent to its input by construction. Self-citations to the conference version [19], SEELE [73], and MAE-FAR [32] are present, but they are used as prior published building blocks or as baselines, not as unverified load-bearing uniqueness theorems. The result is therefore a self-contained empirical contribution with no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 6 invented entities

Central result rests on a small set of hand-chosen training designs (MAE-prior mixing schedule p, mask distribution, augmentation ranges) rather than on a closed-form derivation. These designs are tuned on the authors' own ablations and are coupled to the evaluation protocol (same mask distribution for training and MISATO). The architecture components (alignment module, gated LoRA injection, harmonization decoder) are the paper's engineering contributions; the invented entities are benchmark/metrics, all with independent evidence False because none is yet released or externally validated.

free parameters (5)
  • MAE-prior mixing probability p = 100% linear decay to 10% over 2K training steps, then frozen
    Probability of using the MAE reconstructed prior instead of the predicted prior when training the alignment module (Sec. IV-A). The linear-decay schedule is selected via ablation (Tab. VII).
  • Mask sampling distribution for MAE/decoder training = 50% object-shape, 40% irregular, 10% regular; ratio [0.1,0.75] enlarged to 75%
    Hand-designed mask prior (Sec. IV-A); the same distribution is reused for MISATO evaluation masks, coupling train and test.
  • Color augmentation magnitudes = brightness 0.15, contrast 0.2, saturation 0.1, hue 0.03
    ColorJitter ranges for decoder harmonization training (Sec. V Implementation); no ablation reported.
  • Latent augmentation configuration = applied to 50% of images; t sampled from [500,1000)
    One-step reconstruction gap simulation (Eq. 1); probability and timestep range chosen without ablation.
  • Alignment module capacity = 4 self-attention blocks
    Chosen from ablation of linear/attn/cross x4/self-attn variants (Tab. V).
axioms (5)
  • domain assumption MAE trained with L2 loss produces a mean estimate of the masked region; this mean estimate acts as a context-stable prior that suppresses hallucination when guiding a frozen generative model.
    Sec. IV-A ('MAE as a context-stable prior'). The paper asserts rather than proves that mean-estimate guidance is the right regularizer; empirical gains are the only evidence.
  • domain assumption The one-step latent augmentation z0_hat = (1/a)(zt - b eps_theta([zt; z0; O], t)) with true z0 and an all-zero mask approximates the distribution gap between generated and real latents.
    Sec. IV-B Eq. (1). The approximation is justified by large-step denoising; no statistical comparison to real generated latents is given.
  • ad hoc to paper Training the alignment module with the MAE reconstructed prior (full-image MAE) at p=100% decaying to 10% transfers to inference with only the masked-input MAE predicted prior.
    Sec. IV-A 'Handle misalignment'. This injects masked-region information during training; if the module learns to lean on that information, test-time hallucination suppression would be overestimated.
  • domain assumption Qwen3-VL-235B pairwise VLM judgments are a valid proxy for human perception of object hallucination, and the relative rankings transfer.
    Sec. V 'Object hallucination evaluation'. Only aggregate counts are reported; no prompt, sampling temperature, or inter-rater agreement is given.
  • domain assumption The frozen SD1.5 and FLUX.1-Fill-dev generators can faithfully represent MAE-conditioned control without any backbone weight updates.
    Required for the 'post-hoc, frozen model' claim; tested only indirectly by all experiments, which rely on point estimates without error bars.
invented entities (6)
  • ASUKA alignment module (linear projection + self-attention blocks) no independent evidence
    purpose: Bridges MAE features (dimension/distribution) into the frozen generator's conditioning space
    New module introduced to inject MAE prior; its effectiveness is measured only on the paper's own benchmarks.
  • Per-layer gated LoRA MAE-condition injection for DiT (ASUKA-II) no independent evidence
    purpose: Provides per-transformer-layer MAE control without modifying sequence length or the text-conditioning path
    ASUKA-II design evaluated in the paper's experiments only; no external replication yet.
  • ASUKA local-harmonization decoder no independent evidence
    purpose: Replaces the vanilla latent decoder to make decoded colors match visible regions
    Fine-tuned from the Zhu et al. [51] decoder; its standalone value is shown in Tab. IX for text-guided inpainting, but that is still the authors' own evaluation.
  • MISATO benchmark no independent evidence
    purpose: Multi-domain evaluation set (indoor, landscape, building, background) with 2,000 test images
    Constructed by the authors from Matterport3D, Flickr-Landscape, MegaDepth, COCO; not released, so no external use or scrutiny yet.
  • Gradient@edge (G@e) metric no independent evidence
    purpose: Measures color consistency as average pixel gradient difference at the mask boundary
    New metric defined in Sec. V; validated only indirectly via the paper's tables and not against an external human-labeled color-quality set.
  • VLM object-hallucination evaluation protocol no independent evidence
    purpose: Uses Qwen3-VL to judge whether hallucinated objects appear in inpainted regions
    New protocol adopted after the authors found their CLIP@mask metric unreliable; the prompt and implementation are not specified, preventing independent audit.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency." pith.science (2026). https://pith.science/paper/KGSZGTZO

@misc{pith2026260115368,
  author       = {Pith},
  title        = {Pith review of: Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KGSZGTZO}},
  note         = {Machine review of arXiv:2601.15368}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Generative image inpainting can produce realistic results even with large, irregular masks, but existing methods still suffer from two common problems: (1) Unwanted object insertion: hallucinate artifacts that do not match the surrounding context. (2) Color inconsistency: noticeable color shifts that lead to smeared textures. We analyze the causes of these issues and propose Aligned Stable inpainting with UnKnown Areas prior (ASUKA), a post-hoc framework for pre-trained inpainting models. To reduce unwanted object insertion, we use reconstruction-based priors to guide the generative model, suppressing hallucinated objects while preserving generative flexibility. To address color inconsistency, we design a specialized VAE decoder that formulates latent-to-image decoding as a local harmonization task. We implement ASUKA on both U-Net-based and DiT-based inpainting models with lightweight modifications. Experiments on Places2 and MISATO, our proposed benchmark, show that ASUKA effectively suppresses object hallucination and improves color consistency, outperforming existing diffusion- and rectified flow-based inpainting methods. The dataset, models, and code will be released on GitHub.

Figures

Figures reproduced from arXiv: 2601.15368 by Chenjie Cao, Junqiu Yu, Xiangyang Xue, Yanwei Fu, Yikai Wang.

Figure 1
Figure 1. Figure 1: Image inpainting results obtained using standard SD and FLUX [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The color shift exists in all kinds of scenarios in inpainted images, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) The color of the reconstructed image is shifted, where larger [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: ASUKA tackles the unwanted object insertion issue by adopting [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Use MAE prior for image-to-image translation (start from 80% noise [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Decoder trained by local harmonization task, enhancing mask-unmask [PITH_FULL_IMAGE:figures/full_fig_p006_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: SD1.5 inpainting results decoded by (b) vanilla decoder of SD [16], [PITH_FULL_IMAGE:figures/full_fig_p007_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Illustration of the conditioning mechanism in ASUKA-II-FLUX. We [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
Figure 12
Figure 12. Figure 12: Inpainting results for 5122 images. GANs generate blurred results; SD variants hallucinate unreasonable objects and suffer from color shift. ASUKA-I-SD achieves unwanted-object-mitigated and color-consistent inpainting. object-shaped masks, irregular masks, and regular masks. These mask types are designed to cover a wide range of plausible occlusion patterns and work well in most settings. However, in pra… view at source ↗
Figure 13
Figure 13. Figure 13: Our ASUKA-II largely improves the unwanted object insertion behavior compared with FLUX-Fill and ASUKA-I. [PITH_FULL_IMAGE:figures/full_fig_p010_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Different image domains in MISATO. the foreground with provided segmentation and remove it from the generated masks, yielding a dataset specified for purely background inpainting. General evaluation metrics We use the Learned Perceptual Image Patch Similarity (LPIPS) [96] to calculate the patch￾level image distances, Frechet Inception Distance (FID) [97] ´ to compare the distribution distance between gene… view at source ↗
Figure 15
Figure 15. Figure 15: Object hallucination evaluation results using the VLM-based metric [PITH_FULL_IMAGE:figures/full_fig_p013_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

105 extracted references · 5 linked inside Pith

  1. [1]

    Image in- painting,

    M. Bertalmio, G. Sapiro, V . Caselles, and C. Ballester, “Image in- painting,” inProceedings of the 27th annual conference on Computer graphics and interactive techniques, 2000, pp. 417–424

    2000

  2. [2]

    Object removal by exemplar- based inpainting,

    A. Criminisi, P. P ´erez, and K. Toyama, “Object removal by exemplar- based inpainting,”2003 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition, 2003. Proceedings., vol. 2, pp. II–II, 2003

    2003

  3. [3]

    Scene completion using millions of pho- tographs,

    J. Hays and A. A. Efros, “Scene completion using millions of pho- tographs,”ACM Transactions on Graphics (ToG), vol. 26, no. 3, pp. 4–es, 2007

    2007

  4. [4]

    Learning how to inpaint from global image statistics,

    A. Levin, A. Zomet, and Y . Weiss, “Learning how to inpaint from global image statistics,”Proceedings Ninth IEEE International Con- ference on Computer Vision, pp. 305–312 vol.1, 2003

    2003

  5. [5]

    Fields of experts: a framework for learning image priors,

    S. Roth and M. J. Black, “Fields of experts: a framework for learning image priors,”2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 860–867 vol. 2, 2005

    2005

  6. [6]

    Context encoders: Feature learning by inpainting,

    D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2536–2544

    2016

  7. [7]

    Edge- connect: Structure guided image inpainting using edge prediction,

    K. Nazeri, E. Ng, T. Joseph, F. Qureshi, and M. Ebrahimi, “Edge- connect: Structure guided image inpainting using edge prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019

    2019

  8. [8]

    Guidance and evaluation: Semantic-aware image inpainting for mixed scenes,

    L. Liao, J. Xiao, Z. Wang, C.-W. Lin, and S. Satoh, “Guidance and evaluation: Semantic-aware image inpainting for mixed scenes,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVII 16. Springer, 2020, pp. 683–700

    2020

  9. [9]

    Learning a sketch tensor space for image inpainting of man-made scenes,

    C. Cao and Y . Fu, “Learning a sketch tensor space for image inpainting of man-made scenes,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 14 509–14 518

    2021

  10. [10]

    Resolution-robust large mask inpainting with fourier convolutions,

    R. Suvorov, E. Logacheva, A. Mashikhin, A. Remizova, A. Ashukha, A. Silvestrov, N. Kong, H. Goka, K. Park, and V . Lempitsky, “Resolution-robust large mask inpainting with fourier convolutions,” inProceedings of the IEEE/CVF winter conference on applications of computer vision, 2022, pp. 2149–2159

    2022

  11. [11]

    High-fidelity pluralistic image completion with transformers,

    Z. Wan, J. Zhang, D. Chen, and J. Liao, “High-fidelity pluralistic image completion with transformers,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 4692–4701

    2021

  12. [12]

    Large scale image completion via co-modulated generative adversarial networks,

    S. Zhao, J. Cui, Y . Sheng, Y . Dong, X. Liang, I. Eric, C. Chang, and Y . Xu, “Large scale image completion via co-modulated generative adversarial networks,” inInternational Conference on Learning Rep- resentations, 2020

    2020

  13. [13]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014

    2014

  14. [14]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,”Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020

    2020

  15. [15]

    Taming transformers for high-resolution image synthesis,

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high-resolution image synthesis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12 873–12 883

    2021

  16. [16]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 10 684–10 695

    2022

  17. [17]

    B. F. Labs, “Flux.1,” 2024. [Online]. Available: https://blackforestlabs. ai/announcing-black-forest-labs/

    2024

  18. [18]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. Girshick, “Masked autoencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009

    2022

  19. [19]

    Towards enhanced image inpainting: Mitigating unwanted object insertion and preserving color consistency,

    Y . Wang, C. Cao, J. Yu, K. Fan, X. Xue, and Y . Fu, “Towards enhanced image inpainting: Mitigating unwanted object insertion and preserving color consistency,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 23 237–23 248

    2025

  20. [20]

    Places: A 10 million image database for scene recognition,

    B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,”IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 6, pp. 1452– 1464, 2017

    2017

  21. [21]

    Matterport3D: Learning from RGB- D data in indoor environments,

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang, “Matterport3D: Learning from RGB- D data in indoor environments,”International Conference on 3D Vision (3DV), 2017

    2017

  22. [22]

    InfinityGAN: Towards infinite-pixel image synthesis,

    C. H. Lin, Y .-C. Cheng, H.-Y . Lee, S. Tulyakov, and M.-H. Yang, “InfinityGAN: Towards infinite-pixel image synthesis,” inInternational Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=ufGMqIM0a4b

    2022

  23. [23]

    Megadepth: Learning single-view depth predic- tion from internet photos,

    Z. Li and N. Snavely, “Megadepth: Learning single-view depth predic- tion from internet photos,” inComputer Vision and Pattern Recognition (CVPR), 2018

    2018

  24. [24]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” inEuropean conference on computer vision. Springer, 2014, pp. 740–755

    2014

  25. [25]

    Region filling and object removal by exemplar-based image inpainting,

    A. Criminisi, P. P ´erez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,”IEEE Transactions on image processing, vol. 13, no. 9, pp. 1200–1212, 2004

    2004

  26. [26]

    Patch- match: A randomized correspondence algorithm for structural image editing,

    C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman, “Patch- match: A randomized correspondence algorithm for structural image editing,”ACM Trans. Graph., vol. 28, no. 3, p. 24, 2009

    2009

  27. [27]

    A robust forgery detection algorithm for object removal by exemplar-based image inpainting,

    D. Zhang, Z. Liang, G. Yang, Q. Li, L. Li, and X. Sun, “A robust forgery detection algorithm for object removal by exemplar-based image inpainting,”Multimedia Tools and Applications, vol. 77, pp. 11 823–11 842, 2018

    2018

  28. [28]

    Nontexture inpainting by curvature-driven diffusions,

    T. F. Chan and J. Shen, “Nontexture inpainting by curvature-driven diffusions,”Journal of visual communication and image representation, vol. 12, no. 4, pp. 436–449, 2001

    2001

  29. [29]

    Simultaneous structure and texture image inpainting,

    M. Bertalmio, L. Vese, G. Sapiro, and S. Osher, “Simultaneous structure and texture image inpainting,”IEEE transactions on image processing, vol. 12, no. 8, pp. 882–889, 2003

    2003

  30. [30]

    Free- form image inpainting with gated convolution,

    J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang, “Free- form image inpainting with gated convolution,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4471–4480

    2019

  31. [31]

    Mat: Mask-aware transformer for large hole image inpainting,

    W. Li, Z. Lin, K. Zhou, L. Qi, Y . Wang, and J. Jia, “Mat: Mask-aware transformer for large hole image inpainting,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

    2022

  32. [32]

    Learning prior feature and attention enhanced image inpainting,

    C. Cao, Q. Dong, and Y . Fu, “Learning prior feature and attention enhanced image inpainting,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 306–322

    2022

  33. [33]

    Image inpainting for irregular holes using partial convolutions,

    G. Liu, F. A. Reda, K. J. Shih, T.-C. Wang, A. Tao, and B. Catanzaro, “Image inpainting for irregular holes using partial convolutions,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 85–100

    2018

  34. [34]

    Aggregated contextual transfor- mations for high-resolution image inpainting,

    Y . Zeng, J. Fu, H. Chao, and B. Guo, “Aggregated contextual transfor- mations for high-resolution image inpainting,”IEEE Transactions on Visualization and Computer Graphics, 2022

    2022

  35. [35]

    Generative image inpainting with contextual attention,

    J. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang, “Generative image inpainting with contextual attention,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5505–5514

    2018

  36. [36]

    Contextual residual aggregation for ultra high-resolution image inpainting,

    Z. Yi, Q. Tang, S. Azizi, D. Jang, and Z. Xu, “Contextual residual aggregation for ultra high-resolution image inpainting,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2020, pp. 7508–7517

    2020

  37. [37]

    High- resolution image inpainting with iterative confidence feedback and guided upsampling,

    Y . Zeng, Z. Lin, J. Yang, J. Zhang, E. Shechtman, and H. Lu, “High- resolution image inpainting with iterative confidence feedback and guided upsampling,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 1–17

    2020

  38. [38]

    Continuously masked transformer for image inpainting,

    K. Ko and C.-S. Kim, “Continuously masked transformer for image inpainting,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 169–13 178

    2023

  39. [39]

    Image completion with heterogeneously filtered spectral hints,

    X. Xu, S. Navasardyan, V . Tadevosyan, A. Sargsyan, Y . Mu, and H. Shi, “Image completion with heterogeneously filtered spectral hints,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 4591–4601

    2023

  40. [40]

    Rethinking fast fourier convolution in image inpainting,

    T. Chu, J. Chen, J. Sun, S. Lian, Z. Wang, Z. Zuo, L. Zhao, W. Xing, and D. Lu, “Rethinking fast fourier convolution in image inpainting,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23 195–23 205. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 15

    2023

  41. [41]

    Image inpainting with cascaded modulation gan and object-aware training,

    H. Zheng, Z. Lin, J. Lu, S. Cohen, E. Shechtman, C. Barnes, J. Zhang, N. Xu, S. Amirghodsi, and J. Luo, “Image inpainting with cascaded modulation gan and object-aware training,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 277–296

    2022

  42. [42]

    Palette: Image-to-image diffusion models,

    C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet, and M. Norouzi, “Palette: Image-to-image diffusion models,” inACM SIGGRAPH 2022 Conference Proceedings, 2022, pp. 1–10

    2022

  43. [43]

    Kandinsky 3.0 technical report,

    V . Arkhipkin, A. Filatov, V . Vasilev, A. Maltseva, S. Azizov, I. Pavlov, J. Agafonova, A. Kuznetsov, and D. Dimitrov, “Kandinsky 3.0 technical report,” 2023

    2023

  44. [44]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boeselet al., “Scaling rectified flow transformers for high-resolution image synthesis,” inForty-first International Conference on Machine Learning, 2024

    2024

  45. [45]

    Sdedit: Guided image synthesis and editing with stochastic differential equations,

    C. Meng, Y . He, Y . Song, J. Song, J. Wu, J.-Y . Zhu, and S. Ermon, “Sdedit: Guided image synthesis and editing with stochastic differential equations,” inInternational Conference on Learning Representations, 2021

    2021

  46. [46]

    An image is worth one word: Personal- izing text-to-image generation using textual inversion,

    R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-or, “An image is worth one word: Personal- izing text-to-image generation using textual inversion,” inThe Eleventh International Conference on Learning Representations, 2022

    2022

  47. [47]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2021

    2021

  48. [48]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 3836–3847

    2023

  49. [49]

    Openai’s consistency decoder,

    OpenAI, “Openai’s consistency decoder,” 2023

    2023

  50. [50]

    Image super- resolution via latent diffusion: A sampling-space mixture of ex- perts and frequency-augmented decoder approach,

    F. Luo, J. Xiang, J. Zhang, X. Han, and W. Yang, “Image super- resolution via latent diffusion: A sampling-space mixture of ex- perts and frequency-augmented decoder approach,”arXiv preprint arXiv:2310.12004, 2023

    Pith/arXiv arXiv 2023

  51. [51]

    Designing a better asymmetric vqgan for stablediffusion,

    Z. Zhu, X. Feng, D. Chen, J. Bao, L. Wang, Y . Chen, L. Yuan, and G. Hua, “Designing a better asymmetric vqgan for stablediffusion,” arXiv preprint arXiv:2306.04632, 2023

    Pith/arXiv arXiv 2023

  52. [52]

    Beit: Bert pre-training of image trans- formers,

    H. Bao, L. Dong, and F. Wei, “Beit: Bert pre-training of image trans- formers,” inInternational Conference on Learning Representations, 2022

    2022

  53. [53]

    Simmim: A simple framework for masked image modeling,

    Z. Xie, Z. Zhang, Y . Cao, Y . Lin, J. Bao, Z. Yao, Q. Dai, and H. Hu, “Simmim: A simple framework for masked image modeling,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9653–9663

    2022

  54. [54]

    Context autoencoder for self-supervised representation learning,

    X. Chen, M. Ding, X. Wang, Y . Xin, S. Mo, Y . Wang, S. Han, P. Luo, G. Zeng, and J. Wang, “Context autoencoder for self-supervised representation learning,”International Journal of Computer Vision, pp. 1–16, 2023

    2023

  55. [55]

    Masked feature prediction for self-supervised visual pre-training,

    C. Wei, H. Fan, S. Xie, C.-Y . Wu, A. Yuille, and C. Feichtenhofer, “Masked feature prediction for self-supervised visual pre-training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 14 668–14 678

    2022

  56. [56]

    Mvp: Multimodality- guided visual pre-training,

    L. Wei, L. Xie, W. Zhou, H. Li, and Q. Tian, “Mvp: Multimodality- guided visual pre-training,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 337–353

    2022

  57. [57]

    Deep image harmonization,

    Y .-H. Tsai, X. Shen, Z. Lin, K. Sunkavalli, X. Lu, and M.-H. Yang, “Deep image harmonization,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3789–3797

    2017

  58. [58]

    Learning a discriminative model for the perception of realism in composite images,

    J.-Y . Zhu, P. Krahenbuhl, E. Shechtman, and A. A. Efros, “Learning a discriminative model for the perception of realism in composite images,” inProceedings of the IEEE International Conference on Computer Vision, 2015, pp. 3943–3951

    2015

  59. [59]

    Dovenet: Deep image harmonization via domain verification,

    W. Cong, J. Zhang, L. Niu, L. Liu, Z. Ling, W. Li, and L. Zhang, “Dovenet: Deep image harmonization via domain verification,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 8394–8403

    2020

  60. [60]

    Intrinsic image harmonization,

    Z. Guo, H. Zheng, Y . Jiang, Z. Gu, and B. Zheng, “Intrinsic image harmonization,” inProceedings of the ieee/cvf conference on computer vision and pattern recognition, 2021, pp. 16 367–16 376

    2021

  61. [61]

    High-resolution image harmonization via collaborative dual transfor- mations,

    W. Cong, X. Tao, L. Niu, J. Liang, X. Gao, Q. Sun, and L. Zhang, “High-resolution image harmonization via collaborative dual transfor- mations,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 470–18 479

    2022

  62. [62]

    Transformer for image harmonization and beyond,

    Z. Guo, Z. Gu, B. Zheng, J. Dong, and H. Zheng, “Transformer for image harmonization and beyond,”IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 11, pp. 12 960–12 977, 2022

    2022

  63. [63]

    Deep image harmonization with learnable augmentation,

    L. Niu, J. Cao, W. Cong, and L. Zhang, “Deep image harmonization with learnable augmentation,” inProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, 2023, pp. 7482–7491

    2023

  64. [64]

    Semi- supervised parametric real-world image harmonization,

    K. Wang, M. Gharbi, H. Zhang, Z. Xia, and E. Shechtman, “Semi- supervised parametric real-world image harmonization,” inProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 5927–5936

    2023

  65. [65]

    Lemart: Label-efficient masked region transform for image harmonization,

    S. Liu, C. P. Huynh, C. Chen, M. Arap, and R. Hamid, “Lemart: Label-efficient masked region transform for image harmonization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18 290–18 299

    2023

  66. [66]

    High- resolution image harmonization with adaptive-interval color transfor- mation,

    Q. Meng, L. Qinglin, Z. Li, X. Lan, S. Zhang, and L. Nie, “High- resolution image harmonization with adaptive-interval color transfor- mation,”Advances in Neural Information Processing Systems, vol. 37, pp. 13 769–13 793, 2024

    2024

  67. [67]

    Relightful harmonization: Lighting-aware portrait back- ground replacement,

    M. Ren, W. Xiong, J. S. Yoon, Z. Shu, J. Zhang, H. Jung, G. Gerig, and H. Zhang, “Relightful harmonization: Lighting-aware portrait back- ground replacement,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 6452–6462

    2024

  68. [68]

    Shape-guided object inpainting,

    Y . Zeng, Z. Lin, and V . M. Patel, “Shape-guided object inpainting,” arXiv preprint arXiv:2204.07845, 2022

    Pith/arXiv arXiv 2022

  69. [69]

    Smartbrush: Text and shape guided object inpainting with diffusion model,

    S. Xie, Z. Zhang, Z. Lin, T. Hinz, and K. Zhang, “Smartbrush: Text and shape guided object inpainting with diffusion model,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 22 428–22 437

    2023

  70. [70]

    Imagen editor and editbench: Advancing and evaluating text-guided image inpainting,

    S. Wang, C. Saharia, C. Montgomery, J. Pont-Tuset, S. Noy, S. Pel- legrini, Y . Onoe, S. Laszlo, D. J. Fleet, R. Soricutet al., “Imagen editor and editbench: Advancing and evaluating text-guided image inpainting,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 18 359–18 369

    2023

  71. [71]

    Erasedraw: Learning to insert objects by erasing them from images,

    A. Canberk, M. Bondarenko, E. Ozguroglu, R. Liu, and C. V ondrick, “Erasedraw: Learning to insert objects by erasing them from images,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 144–160

    2024

  72. [72]

    Brush2prompt: Contextual prompt genera- tor for object inpainting,

    M. T. Chiu, Y . Zhou, L. Zhang, Z. Lin, C. Barnes, S. Amirghodsi, E. Shechtman, and H. Shi, “Brush2prompt: Contextual prompt genera- tor for object inpainting,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 12 636–12 645

    2024

  73. [73]

    Repositioning the subject within image,

    Y . Wang, C. Cao, K. Fan, Q. Dong, Y . Li, X. Xue, and Y . Fu, “Repositioning the subject within image,”Transactions on Machine Learning Research, 2024

    2024

  74. [74]

    A task is worth one word: Learning with task prompts for high-quality versatile image inpainting,

    J. Zhuang, Y . Zeng, W. Liu, C. Yuan, and K. Chen, “A task is worth one word: Learning with task prompts for high-quality versatile image inpainting,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 195–211

    2024

  75. [75]

    Improving text-guided object inpainting with semantic pre-inpainting,

    Y . Chen, J. Chen, Y . Pan, Y . Li, T. Yao, Z. Chen, and T. Mei, “Improving text-guided object inpainting with semantic pre-inpainting,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 110–126

    2024

  76. [76]

    Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion,

    X. Ju, X. Liu, X. Wang, Y . Bian, Y . Shan, and Q. Xu, “Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 150–168

    2024

  77. [77]

    Invi: Object insertion in videos us- ing off-the-shelf diffusion models,

    N. Saini, N. Bodla, A. Shrivastava, A. Ravichandran, X. Zhang, A. Shrivastava, and B. Singh, “Invi: Object insertion in videos us- ing off-the-shelf diffusion models,”arXiv preprint arXiv:2407.10958, 2024

    Pith/arXiv arXiv 2024

  78. [78]

    Leftrefill: Filling right canvas based on left reference through generalized text-to-image diffusion model,

    C. Cao, Y . Cai, Q. Dong, Y . Wang, and Y . Fu, “Leftrefill: Filling right canvas based on left reference through generalized text-to-image diffusion model,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 7705–7715

    2024

  79. [79]

    Magiceraser: Erasing any objects via semantics-aware control,

    F. Li, Z. Zhang, Y . Huang, J. Liu, R. Pei, B. Shao, and S. Xu, “Magiceraser: Erasing any objects via semantics-aware control,” in European Conference on Computer Vision. Springer, 2024, pp. 215– 231

    2024

  80. [80]

    Placing objects in context via inpainting for out-of-distribution segmentation,

    P. de Jorge, R. V olpi, P. K. Dokania, P. H. Torr, and G. Rogez, “Placing objects in context via inpainting for out-of-distribution segmentation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 456–473

    2024

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.