Pith. sign in

REVIEW 4 major objections 6 minor 30 references

MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that masking-augmented diffusion training plus inference-time pause tokens jointly improve visual editing faithfulness and instruction adherence.

desk verdict MAgD is a plausible training trick with consistent but small gains, but the pause-token scaling claim is confounded by best-of-n, and the paper needs error bars and a fix to its own objective equation. read the letter →

arxiv 2507.13401 v1 pith:D5X3AOS7 submitted 2025-07-16 cs.CV cs.LG

Shreya Kadambi , Risheek Garrepalli , Shubhankar Borse , Munawar Hyatt , Fatih Porikli This is my paper
classification cs.CVcs.LG
keywords visualeditingdiffusionmodelsmaskedreconstructioninference-timescalingpausetokensinstructionfollowingimagefaithfulness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that diffusion models for visual editing can be made more controllable by changing both how they are trained and how they are run. It introduces MAgD, a training objective that adds a masked-reconstruction term to standard denoising score matching by masking the noisy input during the forward process at high noise levels. It also introduces an inference-only mechanism in which pause tokens are inserted into the prompt, giving the model extra computation before it generates the edited image. The paper reports that these mechanisms together improve both source-image faithfulness and instruction adherence, and that pause-token scaling lets practitioners trade one against the other without retraining. A sympathetic reader would care because editing is an inverse problem where faithfulness and instruction-following usually compete, and the paper claims to relax that trade-off.

What carries the argument

The central object is the MAgD objective, which applies a random binary mask to the noisy latent $x_t$ from the forward process, replacing masked entries with a mask embedding, and trains the denoiser with the standard score-matching loss on this dual-corrupted input. Masking is applied stochastically (with probability $p_{\mathrm{magd}}$) and only at high noise levels ($t \ge \tau_{\mathrm{MAgD}}$), which the paper argues targets the regime where the network builds global structure. The second mechanism is the pause token $\langle\mathrm{pause}\rangle$, inserted at inference time between the instruction or reference image and the generation segment, which effectively grants extra transformer computation before output. Together, the masking objective is intended to instill localized, compositional representations, and the pause tokens are intended to let the model spend more compute on hard edits.

What would settle it

Fine-tune the same base architecture on the same corpus twice, once with the MAgD objective and once with the standard denoising objective, holding data, steps, and seeds fixed; if the DINO and CLIP-Dir gaps disappear, the masking objective is not the cause. For pause tokens, generate multiple samples per prompt with 0, 8, 16, and 32 pause tokens and apply the paper's best-CLIP-Dir selection within each condition; if the pause=0 best-of-n matches the best pause-token CLIP-Dir, the scaling effect reduces to best-of-n sampling.

Watch

Extended reading notes

Core claim

MAgD's dual-corruption objective—standard Gaussian denoising plus stochastic masking of the noisy input at high noise levels—produces visual representations that are more compositional and more finely grounded in text, so that a unified in-context diffusion model performs better at instruction-guided editing. The paper also claims that pause tokens, special placeholder tokens inserted into the prompt only at inference, give the model additional computational capacity that improves instruction adherence, and that by varying the number of pause tokens a user can navigate the trade-off between source-image faithfulness and instruction following without retraining. Empirical support comes from fine-tuning a single base architecture on about 400K samples and reporting that MAgD raises DINO from 0.882 to 0.927 and CLIP-Dir from 0.122 to 0.126 on Emu-Edit, with pause-token scaling further raising CLIP-Dir to 0.134 at a small DINO cost.

Load-bearing premise

The central claim would collapse if the measured improvements came from the extra fine-tuning data or from picking the best of several random samples, rather than from the masking objective and pause tokens themselves.

Editorial extensions

If this is right

  • If MAgD's representation gains are real, the same training objective should transfer to any unified in-context diffusion architecture, improving image editing without architectural changes.
  • Pause-token scaling would let practitioners select an operating point on the faithfulness-instruction trade-off at inference, with no retraining, and the paper reports higher recall of edits meeting a DINO $\ge 0.91$ target when scaling is used.
  • Fine-tuning with expressive, step-wise prompts appears to further improve instruction adherence, and the gains are larger when combined with MAgD than with the base objective.
  • The dual-corruption objective also improves scores on compositional text-to-image benchmarks such as GenEval, suggesting the representation benefit is not limited to editing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if pause tokens add real capacity, then holding total compute fixed, a run with pause tokens should beat a run with extra denoising steps of the same cost; this comparison is not in the paper.
  • Beyond the paper: the high-noise-only masking schedule suggests MAgD's benefit should concentrate on edits that require global restructuring, such as background or global-style changes, and be smaller for localized texture swaps; a task-sliced ablation would test this.
  • Beyond the paper: because pause-token scaling is inference-only, it could be stacked with other zero-retraining controls such as guidance-scale adjustment to trace a full Pareto frontier of faithfulness versus instruction adherence per prompt.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a framework, MADI, built on top of the OmniGen in-context diffusion architecture for instruction-based image editing. It introduces MAgD, a training objective that adds a random token-masking corruption on the Gaussian-noised input and supervises the denoiser on the masked input, and it introduces inference-time pause tokens inserted into the prompt to provide extra computational capacity without retraining. On Emu-Edit, MagicBrush, Complex-Edit, IdeaBench, and GenEval, the authors report that MAgD improves both source faithfulness (DINO 0.882 to 0.927) and instruction adherence (CLIP-Dir 0.122 to 0.126) over the OmniGen baseline, and that pause-token inference scaling further shifts CLIP-Dir to 0.134.

Significance. If the training-objective results are reproducible, MAgD is a simple, computationally cheap modification to the standard diffusion objective, and the paper's explicit treatment of the faithfulness-versus-instruction-following trade-off is useful to the editing community. The pause-token mechanism is an interesting transfer of an LLM idea to diffusion-based visual editing, but its causal contribution is not yet established. Strengths include the breadth of benchmarks, component-level ablations in Tables 3, 7, and 8, and a candid discussion of the limitations of CLIP-based metrics. The paper does not, however, provide code, checkpoints, or repeated-seed variability for its headline editing numbers, which matters because several of the key reported differences are small.

major comments (4)
  1. [Section 4.1, Table 4, Fig. 6] The inference-time scaling evidence is confounded with best-of-n selection. The protocol in Fig. 6 selects, per sample, the generation with the best CLIP-Dir among pause-token counts {0, 8, 16, 32}, while the comparison baseline is a single generation at pause=0. Under this protocol, the pause-0 arm samples one image while the pause-token arm samples up to four, so the observed gain can be explained by the maximum of a larger sample rather than by the added capacity of the pause tokens. Table 4's 'Best CLIP' and 'Best DINO' columns are selected in the same way. A valid test requires a matched multi-sample baseline: for example, generate K images with pause=0 and select the best under the same criterion, then compare best-of-K at pause=0 with best-of-{0, 8, 16, 32}; the paper should also report results under a fixed, non-oracle pause count. Without such a baseline, the causal claim that 'pause tokens provide additional capacity' in Sections 3.3 and 4.1 is not supported.
  2. [Eq. (6) / Eq. (9), Sections 3.1.3 and 7] The masking schedule is internally inconsistent. Section 3.1.3 states that masking is restricted to high noise levels, t greater than or equal to tau_MAgD, but Eq. (6) applies the masked loss when t is less than tau_MAgD; Section 7's Eq. (9) repeats the same inconsistency, and Table 7's caption uses 't < tau_MAgD' while the surrounding text says 't >= tau_MAgD'. This is not a typographical detail: it changes which training steps receive the dual corruption and therefore changes the trained model. The authors should choose one condition and align the equation, the prose, and the table.
  3. [Table 3 and text after it] The text claims that 'MAgD alone boosts both CLIP-Dir and DINO over OmniGen and finetuned omnigen.' The table contradicts this for the fine-tuned comparison: row 4 (MAgD without expressive prompting) has DINO 0.913, which is below the 0.917 DINO of the fine-tuned baseline in row 2, even though CLIP-Dir improves from 0.111 to 0.127. The claim should be restricted to the comparison with the untuned OmniGen checkpoint, or new data are needed to support the original statement.
  4. [Tables 1, 3, and 5] All headline editing results are single-run point estimates. The main claimed CLIP-Dir improvement from 0.122 to 0.126 is a difference of 0.004, and the inference-scaling gain is 0.008; without standard deviations or confidence intervals, these differences may be within run-to-run or evaluation noise. Table 5 averages GenEval over five seeds, so the authors already have the infrastructure for seed-averaged reporting; the editing metrics should be reported with the same practice.
minor comments (6)
  1. [Section 3.1.3, Eq. (4)] The product m circle x_t circle empty-set is not well-defined because the empty-set is described as a mask token (a vector) while m circle x_t is a vector; please specify how the mask embedding replaces the masked entries rather than multiplying them.
  2. [Baselines paragraph, Section 4] The baselines paragraph cites Show-O as [13], but reference [13] in the bibliography is the scratchpad paper; Show-O is reference [24]. Please correct the citation.
  3. [Table 1 and Figure 2] The naming of rows in Table 1 ('MAgD-I' vs. 'w/o Inference Scaling') is inconsistent with Figure 2 ('MAgD-I (w/o inference scaling)' and 'MAgD-I'), making it difficult to tell which model is the training-objective ablation; please use a single naming convention throughout.
  4. [Table 4] The column header 'Best CLIP' is ambiguous: the first block reports DINO and CLIP-Dir under a selection rule that likely means best CLIP-Dir; please clarify which metric is used for selection and which metric is reported.
  5. [Section 9, Table 9] Table 9 lists 'Timestep conditioning T-0.3', which is inconsistent with the tau >= 0.7 description in Sections 3.1.3 and 7; this should be updated after the masking-schedule inconsistency in the major comments is resolved.
  6. [Section 4 and supplementary Table 6] The paper states that MLLM(Aggregate) is a weighted average, but the exact weighting and the exclusion rule described for Table 6 are not summarized in the main text; readers should be able to interpret the headline MLLM values without consulting the appendix.

Circularity Check

1 steps flagged · score 6.0 of 10

Inference-time scaling evidence reduces to best-of-n selection: the reported CLIP-DIR/DINO gains are per-sample maxima over pause counts that include pause=0, so improvement over vanilla is guaranteed by construction.

  1. fitted input called prediction [Section 4.1 'Inference time scaling', Table 4, and Figure 6 caption]
    "Compared to vanilla inference, choosing best CLIP-DIR among different generations with varying pause tokens = 0,8,16,32 we can see significant boost in CLIP-DIR with some loss in faithfulness (DINO) Score. ... Evaluation metrics resulting from different think token selection criteria per sample."

    The reported inference-scaling gains are computed by per-sample selection of the best CLIP-DIR (or best DINO, or best CLIP-DIR subject to a DINO threshold) among generations with pause counts 0, 8, 16, and 32. Because pause=0 is one of the candidates, the selected metric for each sample is definitionally at least the pause=0 value, so the average improvement over the 'without scaling' baseline is guaranteed by the protocol rather than by any capacity added through pause tokens. The pause count is effectively fit to maximize the reported metric, and the resulting score is then presented as evidence of inference-time capacity scaling, making the comparison circular.

full rationale

The MAgD training-objective results (Tables 1, 3, 5) are not circular: the mask-augmented loss is a heuristic modification of the diffusion forward process, trained and evaluated on external benchmarks (Emu-Edit, MagicBrush, Complex-Edit, IdeaBench, GenEval), and the reported gains are not derived from fitted constants. The central circularity is confined to the inference-time scaling story. In Figure 6, Table 4, and the Section 4.1 summary, the paper selects, per sample, the best CLIP-DIR (or DINO/recall under a DINO threshold) over pause-token counts 0, 8, 16, and 32, and then attributes the resulting improvement to pause-token capacity. Since pause=0 is included among the candidates, the selected score is at least the pause=0 score sample-wise, so the reported gain over the no-scaling baseline is a mathematical consequence of the selection protocol, not evidence that pause tokens add inference capacity. The paper is transparent that metrics 'result from different think token selection criteria,' but the causal claim that scaling improved CLIP-Dir from 0.126 to 0.134 relies on this selection. A non-circular comparison would require a matched multi-seed pause=0 baseline or per-pause-count distributions. Separately, Eq. (6)/(9) conditions (if u < p_magd and t < tau_MAgD) contradict the surrounding text and Table 7, which say masking is applied only for t >= tau_MAgD; this is a reproducibility inconsistency, not a circularity. Also, Section 9 says a single pause token is appended after each expressive prompt during training, which conflicts with the claim that pause tokens are inserted 'exclusively at inference time'; this undermines the inference-only framing but is not itself circular. Overall, because one of the paper's two core contributions has its headline evidence reduce by construction to best-of-n selection, the analysis is partially circular; the training-objective claims remain independent. Score 6.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central method introduces two main invented components, pause tokens and learnable mask embeddings, both without external falsifiable handles. The free parameters are training and inference hyperparameters selected via ablations on the same benchmarks used for final evaluation. The axioms include standard diffusion training assumptions and two paper-specific hypotheses (masking improves compositions, pause tokens improve capacity) that are empirically asserted rather than independently verified.

free parameters (5)
  • masking rate r_mask = 0.25
    Chosen via ablation on Emu-Edit (Table 7); lower rate found more stable.
  • timestep threshold tau_MAgD = 0.7 (t >= 0.7)
    Chosen via ablation; higher threshold gives more stable training (Table 7).
  • masking probability p_magd = 0.5
    Applied with 50 percent probability per optimization step; not ablated, stated in Section 3.1.3.
  • CFG guidance scale and image guidance = 2.5 and 1.5
    Used at inference; no ablation provided.
  • pause token counts = 0, 8, 16, 32
    Inference-time choices; results reported as best across these counts (Table 4).
assumptions (5)
  • standard math Denoising score matching with noise prediction is a valid training objective for generative modeling.
    Used in Eq. (3), from Ho et al. and Song et al.
  • domain assumption OmniGen's pretrained unified image-text transformer provides a suitable base that preserves editing capability after fine-tuning.
    Section 4: 'We build upon OmniGen as our base architecture...'.
  • ad hoc to paper Masking noisy input at high noise levels encourages compositional representations that transfer to editing.
    Core hypothesis of MAgD, Section 3.1.3; no direct measurement of representation quality.
  • ad hoc to paper Pause tokens inserted at inference provide additional capacity and improve grounding without retraining.
    Section 3.3. Empirical only; confounded by best-of-n selection in evaluation.
  • domain assumption MLLM (Gemini Flash 2.0) judgments approximate human preference for editing quality.
    Section 8: MLLM-based evaluation used as aggregate metric; acknowledged as imperfect.
invented entities (2)
  • Pause tokens (langle pause rangle)
    purpose: Inserted into the prompt at inference to allocate extra computation for editing.
    The paper's experiments do not isolate the token's effect from best-of-n selection; no external evidence.
  • Learnable mask embedding pool of 1024 tokens
    purpose: Replaces masked tokens with a learned embedding during MAgD training.
    Design choice ablated in Table 8; no independent evidence of benefit.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing." pith.science (2026). https://pith.science/paper/D5X3AOS7

@misc{pith2026250713401,
  author       = {Pith},
  title        = {Pith review of: MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D5X3AOS7}},
  note         = {Machine review of arXiv:2507.13401}
}
read the original abstract

Despite the remarkable success of diffusion models in text-to-image generation, their effectiveness in grounded visual editing and compositional control remains challenging. Motivated by advances in self-supervised learning and in-context generative modeling, we propose a series of simple yet powerful design choices that significantly enhance diffusion model capacity for structured, controllable generation and editing. We introduce Masking-Augmented Diffusion with Inference-Time Scaling (MADI), a framework that improves the editability, compositionality and controllability of diffusion models through two core innovations. First, we introduce Masking-Augmented gaussian Diffusion (MAgD), a novel training strategy with dual corruption process which combines standard denoising score matching and masked reconstruction by masking noisy input from forward process. MAgD encourages the model to learn discriminative and compositional visual representations, thus enabling localized and structure-aware editing. Second, we introduce an inference-time capacity scaling mechanism based on Pause Tokens, which act as special placeholders inserted into the prompt for increasing computational capacity at inference time. Our findings show that adopting expressive and dense prompts during training further enhances performance, particularly for MAgD. Together, these contributions in MADI substantially enhance the editability of diffusion models, paving the way toward their integration into more general-purpose, in-context generative diffusion architectures.

Figures

Figures reproduced from arXiv: 2507.13401 by the authors.

Figure 1
Figure 1. a) Comparison of Standard Denoising (Top) and MAgD Objective (Bottom). The top row illustrates standard denoising, The bottom row depicts the MAgD objective, where the target image is randomly masked, and noise is then added to the entire image before denoising. b) Encapsulating Pause tokens for Inference time scaling: The red box in the figure denotes the impact of additional "thinking time" on latent processing st… view at source ↗
Figure 2
Figure 2. Baseline vs MAgD-I (w/o inference scaling) vs MAgD-I. While baseline follows the prompt, it destroys the scene composition in (top) and (bottom). In (middle) we observe that the object appearance is also modified. While MAgD-I w/o scaling alleviates this problem and retains the scene composition. We observe that with inference scaling, model makes localized updates on the latents more closely following the prompt. I… view at source ↗
Figure 3
Figure 3. Comparisons between finetuned-Omnigen vs MAgD w/o inference scaling across different tasks including local, global, background, remove. When using our objective the model retains the intra image composition on global edit task while changing the weather. While finetuned omnigen loses the structure. In the local edit, MAGD improves the semantic representations for example, MAGD generates a good composition and semant… view at source ↗
Figures from the paper (23 more)
Figure 4
Figure 4. Figure 4: [Top]:Finetuned model vs [bottom] MAgD-I for increasing complexities of same instruction prompt and source image. On complexity 1, the finetuned model changes the color of horse but unicorn is clearly generated for all complexities for MAgD-I. At higher complexities (C…
Figure 5
Figure 5. Figure 5: Finetuned model vs MAgD We show the Qualitative results on count and two object tasks on GenEval. In the first figure, sometimes Base model fails to generate two distinct objects as evidenced in the first figure of "A photo of a horse and giraffe" 5 Limitations and Fut…
Figure 6
Figure 6. Figure 6: Compared to vanilla inference, choosing best CLIP-DIR among different generations with varying [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Compared to vanilla inference, choosing best CLIP-DIR among different generations with varying [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Compared to vanilla inference, choosing best CLIP-DIR among different generations with varying [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Effect of think tokens vs different seeds The figure illustrates the effect of varying the number of pause tokens on an image editing task, prompt: ‘turn her hair white.’ It presents a ‘Source image’ , edited images for different seeds showing the results of the edit w…
Figure 10
Figure 10. Figure 10: Qualitative visualizations across different masking rate and noise level threshold.This figure provides qualitative examples illustrating the influence of the noise level threshold τ and masking rate (r) on image editing quality. The top row (Giraffe example) for "Cha…
Figure 11
Figure 11. Figure 11: Our method on Text-Image generation tasks. We visualize the efficacy on our model on T2I tasks. Across two object, counting, colors and position the Base Omnigens performance does not deteriorate. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: This figure presents image generations with progressively increasing CLIP DIR for LOCAL task (a) Squirrel: While visually accurate and semantically aligned, this generation exhibits a low CLIP DIR despite a high CLIP-T score. (b) & (c) Cake-to-Bread Transformation & s…
Figure 13
Figure 13. Figure 13: This figure presents image generations with progressively increasing CLIP DIR for Background task. CLIP DIR accurately reflects the scores based on generations. Its important to notice that DINO scores for these tasks are generally low, a higher DINO scores might most…
Figure 14
Figure 14. Figure 14: Metrics comparisons between Finetuned generations and MAgD-I. Top: Although both generations add a cowboy on the hors, MAgD-I is perceptually better composed into the image. However CLIP DIR scores are lower compared to the finetuned image. CLIP-T scores also being lo…
Figure 15
Figure 15. Figure 15: Examples of CLIP DIR not reflecting the edits: Top: For the color task the image generated adds artifacts while not really reflecting the bike structure. However, the CLIP DIR is more biased towards instruction. Here Instruction following metric and Faithfullness also…
Figure 16
Figure 16. Figure 16: MLLM-based evaluation questions and answer schema used for assessing alignment, faithfulness, [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: Building expressive prompt corpus for Training: Instruction, source image prompt and source image is passed to the MLLM to analyse the inconsistancy between the text and source prompt and image. Further MLLM is instructed to generate upto 5 sequential concise prompts …
Figure 18
Figure 18. Figure 18: Synthetic editing instructions decomposed into expressive multi-step prompts to guide MAgD [PITH_FULL_IMAGE:figures/full_fig_p024_18.png]
Figure 19
Figure 19. Figure 19: Prompt: Change the background of this image featuring a man in a blue suit and a black hat. Replace the current plain, weathered wall with a modern cityscape, showing tall glass buildings and a busy street in the distance. Ensure the new background integrates smoothly…
Figure 20
Figure 20. Figure 20: Ideabench Prompt: Redesign the appearance of the square tissue boxes to reflect a clean and natural aesthetic. The body of the boxes should be a soft off-white color, adorned with light green leaf patterns, conveying a sense of nature and eco-friendliness. The surface…
Figure 21
Figure 21. Figure 21: Ideabench Prompt for Images to image : Generate an image where the two people playing chess in the first image are replaced by Iron Man from the second image and Captain America from the third image. Keep the primary elements of the original image, such as the chessbo…
Figure 22
Figure 22. Figure 22: Qualitative results on Emu-Edit for Local Task: "Replace the net with a brick wall" 26 [PITH_FULL_IMAGE:figures/full_fig_p026_22.png]
Figure 23
Figure 23. Figure 23: Qualitative results on Emu-Edit for Background Task: "Make the background a race track" [PITH_FULL_IMAGE:figures/full_fig_p027_23.png]
Figure 24
Figure 24. Figure 24: Qualitative results on Emu-Edit for Global Task: "Set this to look like it is floating in the sky surrounded by white fluffy clouds" [PITH_FULL_IMAGE:figures/full_fig_p027_24.png]
Figure 25
Figure 25. Figure 25: Qualitative results on Emu-Edit for Add task: "Add a straw to the drink" 27 [PITH_FULL_IMAGE:figures/full_fig_p027_25.png]
Figure 26
Figure 26. Figure 26: Complex edit Evaluations Comparisons between Finetuned and MAgD across different complexities 28 [PITH_FULL_IMAGE:figures/full_fig_p028_26.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 8 canonical work pages

  1. [1]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions, 2023

    2023

  2. [2]

    Muse: Text-to-image generation via masked generative transformers

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transformers. arXiv preprint arXiv:2301.00704, 2023

    arXiv 2023

  3. [3]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11315–11325, 2022

    work page 2022

  4. [4]

    Bert: Pre-training of deep bidi- rectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019

    2019

  5. [5]

    Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing, 2025

    Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Xihui Liu, and Hongsheng Li. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing, 2025

    work page 2025

  6. [6]

    Geneval: An object-focused framework for evaluating text-to-image alignment, 2023

    Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment, 2023

    2023

  7. [7]

    Think before you speak: Training language models with pause tokens

    Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens. arXiv preprint arXiv:2310.02226, 2023

    arXiv 2023

  8. [8]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022

    2022

Show all 30 references
  1. [9]

    Denoising diffusion probabilistic models, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020

    2020

  2. [10]

    [mask] is all you need

    Vincent Tao Hu and Björn Ommer. [mask] is all you need. arXiv preprint arXiv:2412.06787, 2024

    2024 arXiv

  3. [11]

    Learning action and reasoning-centric image editing from videos and simulations, 2024

    Benno Krojer, Dheeraj Vattikonda, Luis Lara, Varun Jampani, Eva Portelance, Christopher Pal, and Siva Reddy. Learning action and reasoning-centric image editing from videos and simulations, 2024

    2024

  4. [12]

    Idea-bench: How far are generative models from professional designing? arXiv preprint arXiv:2412.11767, 2024

    Chen Liang, Lianghua Huang, Jingwu Fang, Huanzhang Dou, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Junge Zhang, Xin Zhao, and Yu Liu. Idea-bench: How far are generative models from professional designing? arXiv preprint arXiv:2412.11767, 2024

    2024 arXiv

  5. [13]

    Show your work: Scratchpads for intermediate computation with language models

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. 2021

    2021

  6. [14]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  7. [15]

    Cogcom: A visual language model with chain-of-manipulations reasoning

    Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. Cogcom: A visual language model with chain-of-manipulations reasoning. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  8. [16]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    2022

  9. [17]

    Emu edit: Precise image editing via recognition and generation tasks

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2024

  10. [18]

    Seededit: Align image re-generation to image editing, 2024

    Yichun Shi, Peng Wang, and Weilin Huang. Seededit: Align image re-generation to image editing, 2024

    2024

  11. [19]

    Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, Nov 2020. 11

    2020

  12. [20]

    VectorSpaceLab. Omnigen. https://github.com/VectorSpaceLab/OmniGen?tab= readme-ov-file, 2023

    2023

  13. [21]

    Omniedit: Building image editing generalist models through specialist supervision

    Cong Wei, Zheyang Xiong, Weiming Ren, Xeron Du, Ge Zhang, and Wenhu Chen. Omniedit: Building image editing generalist models through specialist supervision. In The Thirteenth International Conference on Learning Representations, 2024

    2024

  14. [22]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  15. [23]

    Omnigen: Unified image generation

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409.11340, 2024

    2024 arXiv

  16. [24]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024

    2024 arXiv

  17. [25]

    Llava-cot: Let vision language models reason step-by-step, 2025

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025

    2025

  18. [26]

    Complex-Edit: Cot-like instruction generation for complexity-controllable image editing benchmark, 2025

    Siwei Yang, Mude Hui, Bingchen Zhao, Yuyin Zhou, Nataniel Ruiz, and Cihang Xie. Complex-Edit: Cot-like instruction generation for complexity-controllable image editing benchmark, 2025

    2025

  19. [27]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36:45533–45547, 2023

    2023

  20. [28]

    Magicbrush: A manually annotated dataset for instruction-guided image editing

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing. Advances in Neural Information Processing Systems, 36:31428–31449, 2023

    2023

  21. [29]

    Ultraedit: Instruction-based fine-grained image editing at scale

    Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-based fine-grained image editing at scale. Advances in Neural Information Processing Systems, 37:3058–3093, 2024

    2024

  22. [30]

    pause tokens,

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.09305, 2023. 12 6 Supplementary Contents As part of the supplementary materials for this paper, we share our Implementation detai...

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.