REVIEW 7 cited by
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present Imagen Editor, a cascaded diffusion model built, by fine-tuning Imagen on text-guided image inpainting. Imagen Editor's edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. In addition, Imagen Editor captures fine details in the input image by conditioning the cascaded pipeline on the original high resolution image. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting. EditBench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes. Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and Stable Diffusion -- and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.
Forward citations
Cited by 7 Pith papers
-
D-Fusion: Direct Preference Optimization for Aligning Diffusion Models with Visually Consistent Samples
Mask-guided self-attention fusion creates well-aligned target images that stay visually close to poorly-aligned base images, with full denoising trajectories, and DPO on these pairs improves alignment.
-
EditInspector: A Benchmark for Evaluation of Text-Guided Image Edits
A new human-labeled benchmark shows leading vision-language models are unreliable at judging image edits, and the authors' methods improve artifact detection and difference captioning.
-
Towards Reliable Identification of Diffusion-based Image Manipulations
RADAR combines semantic and geometric vision features with contrastive learning to detect and localize diffusion-based image edits, outperforming prior methods on a new 28-model benchmark.
-
LightLab: Controlling Light Sources in Images with Diffusion Models
A diffusion model fine-tuned on paired real and synthetic images can edit the intensity, color, and ambient level of visible light sources in a single photograph while keeping shadows and reflections plausible.
-
VideoDPO: Omni-Preference Alignment for Video Diffusion Generation
VideoDPO shows that DPO-style training on automatically selected best and worst video pairs improves overall VBench scores on three open text-to-video models, with some sub-metrics degrading and weak gains on external...
-
VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models
VBench++ is a benchmark that scores text-to-video and image-to-video models on 16 quality dimensions plus trustworthiness, reporting human-alignment correlations for each.
-
Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews
A client-side landmark mask and feathered composite confined image-editing API outputs to the requested facial region in a 15-face pilot, at the cost of little on-target change, while no tested editor moved identity e...
Discussion (0). Continue with ORCID to comment.