Pith. sign in

REVIEW 22 cited by

Re-Imagen: Retrieval-Augmented Text-to-Image Generator

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.14491 v3 pith:DX7SWPYQ submitted 2022-09-29 cs.CV cs.AIcs.LG

Wenhu Chen , Hexiang Hu , Chitwan Saharia , William W. Cohen This is my paper
classification cs.CVcs.AIcs.LG
keywords entitiesre-imagentextimagesretrievalimagegeneratingmodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Research on text-to-image generation has witnessed significant progress in generating diverse and photo-realistic images, driven by diffusion and auto-regressive models trained on large-scale image-text data. Though state-of-the-art models can generate high-quality images of common entities, they often have difficulty generating images of uncommon entities, such as `Chortai (dog)' or `Picarones (food)'. To tackle this issue, we present the Retrieval-Augmented Text-to-Image Generator (Re-Imagen), a generative model that uses retrieved information to produce high-fidelity and faithful images, even for rare or unseen entities. Given a text prompt, Re-Imagen accesses an external multi-modal knowledge base to retrieve relevant (image, text) pairs and uses them as references to generate the image. With this retrieval step, Re-Imagen is augmented with the knowledge of high-level semantics and low-level visual details of the mentioned entities, and thus improves its accuracy in generating the entities' visual appearances. We train Re-Imagen on a constructed dataset containing (image, text, retrieval) triples to teach the model to ground on both text prompt and retrieval. Furthermore, we develop a new sampling strategy to interleave the classifier-free guidance for text and retrieval conditions to balance the text and retrieval alignment. Re-Imagen achieves significant gain on FID score over COCO and WikiImage. To further evaluate the capabilities of the model, we introduce EntityDrawBench, a new benchmark that evaluates image generation for diverse entities, from frequent to rare, across multiple object categories including dogs, foods, landmarks, birds, and characters. Human evaluation on EntityDrawBench shows that Re-Imagen can significantly improve the fidelity of generated images, especially on less frequent entities.

Discussion (0). Sign in to comment.

Forward citations

Cited by 22 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ImageAuditor: Membership Inference Attack against Image-based Retrieval-Augmented Generation

    cs.CR 2026-06 unverdicted novelty 7.0 of 10

    ImageAuditor is the first MIA for IRAG that achieves over 80% AUROC with four queries by using reward-guided policy optimization for cross-modal retrieval and task-specific prompting for signal extraction.

  2. MemoGen: Can Past Experience Improve Future Text-to-Image Generation?

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    MemoGen is a training-free agentic framework that stores task understanding, references, visual feedback, and lessons from past generations as reusable memory to improve text-to-image output over evolution rounds.

  3. DrawMotion: Generating 3D Human Motions by Freehand Drawing

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    DrawMotion is a diffusion-based framework that fuses text and hand-drawn stickman conditions via a Multi-Condition Module and training-free guidance to generate 3D human motions.

  4. Similar Users-Augmented Interest Network

    cs.IR 2026-04 unverdicted novelty 7.0 of 10

    SUIN improves CTR prediction by augmenting target user sequences with similar users' behaviors via embedding-based retrieval, user-specific position encoding, and user-aware target attention.

  5. ASTRA: Enhancing Multi-Subject Generation with Retrieval-Augmented Pose Guidance and Disentangled Position Embedding

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    ASTRA disentangles subject identity from pose structure in diffusion transformers via retrieval-augmented pose guidance, asymmetric EURoPE embeddings, and a DSM adapter to improve multi-subject generation.

  6. Gen-Searcher: Reinforcing Agentic Search for Image Generation

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    Gen-Searcher is the first trained search-augmented image generation agent using SFT followed by GRPO reinforcement learning with dual text-image rewards, delivering 15-16 point gains on knowledge-intensive benchmarks.

  7. MV-RAG: Retrieval Augmented Multiview Diffusion

    cs.CV 2025-08 conditional novelty 7.0 of 10

    A retrieval-augmented multiview diffusion model conditions on web images to generate 3D-consistent views of rare concepts, trained with a hybrid 3D/2D objective and evaluated on a new OOD benchmark.

  8. From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems

    cs.MA 2025-06 accept novelty 7.0 of 10

    A survey that defines Compound AI Systems, proposes a multi-dimensional taxonomy based on component roles and orchestration strategies, reviews four foundational paradigms, and identifies key challenges for future research.

  9. Polaris: Scaling Up Instruction-Guided Image Generation Towards Millions of Personalized Style Needs

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Polaris retrieves and integrates relevant models from a large library of checkpoints and adapters to enable scalable instruction-guided image generation and editing without additional training.

  10. SegRAG: Training-Free Retrieval-Augmented Semantic Segmentation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SegRAG is a training-free retrieval-augmented framework that extracts class-specific point prompts from a filtered DINOv3 feature bank to boost SAM3 semantic segmentation performance on standard and agricultural benchmarks.

  11. SegRAG: Training-Free Retrieval-Augmented Semantic Segmentation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SegRAG augments SAM3 with class-specific point prompts retrieved via DINOv3 features and filtered by ICCD, using TSG at inference to improve open-vocabulary segmentation.

  12. Gen-Searcher: Reinforcing Agentic Search for Image Generation

    cs.CV 2026-03 unverdicted novelty 6.0 of 10

    Gen-Searcher is the first search-augmented image generation agent trained with SFT followed by agentic RL using dual text and image rewards on custom datasets and the KnowGen benchmark.

  13. USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning

    cs.CV 2025-08 conditional novelty 6.0 of 10

    USO trains one DiT model for subject-driven, style-driven, and joint generation by disentangling content and style from triplet data and adding a style-reward objective, claiming SOTA on USO-Bench.

  14. FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space

    cs.GR 2025-06 unverdicted novelty 6.0 of 10

    FLUX.1 Kontext unifies image generation and editing via flow matching and sequence concatenation, delivering improved multi-turn consistency and speed on the new KontextBench benchmark.

  15. RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction

    cs.CV 2025-04 unverdicted novelty 6.0 of 10

    RA-RRG extracts key phrases with LLMs, retrieves them via multimodal similarity, and conditions report generation on them to achieve SOTA CheXbert scores and competitive RadGraph F1 on MIMIC-CXR and IU X-ray while sup...

  16. Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    BadRDM is a backdoor attack on retrieval-augmented diffusion models that poisons the retrieval database with toxicity surrogates and uses multimodal contrastive learning to force toxic generations from text triggers w...

  17. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

    cs.CV 2023-08 unverdicted novelty 6.0 of 10

    IP-Adapter adds effective image prompting to text-to-image diffusion models using a lightweight decoupled cross-attention adapter that works alongside text prompts and other controls.

  18. Prompt Evolution for Generative AI: A Classifier-Guided Approach

    cs.LG 2023-05 unverdicted novelty 6.0 of 10

    The paper introduces a classifier-guided multi-objective evolutionary algorithm for prompt evolution in generative AI that uses the model's stochastic generation as implicit mutations to create Pareto-optimized images...

  19. eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers

    cs.CV 2022-11 unverdicted novelty 6.0 of 10

    An ensemble of stage-specialized text-to-image diffusion models improves prompt alignment over single shared-parameter models while preserving visual quality and inference speed.

  20. PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    PhysRAG curates 7K videos from WISA-80K, builds a physical video database, and injects knowledge via learnable queries into a diffusion model to reach SOTA visual quality and physical compliance on PhyGenBench and VBench.

  21. Animalbooth: multimodal feature enhancement for animal subject personalization

    cs.CV 2025-09 unverdicted novelty 5.0 of 10

    AnimalBooth introduces an Animal Net, adaptive attention module, and frequency-controlled DCT feature integration to improve identity preservation and perceptual quality in personalized animal image generation, suppor...

  22. RAVA: Retrieval-Augmented Viewpoint Alignment for Subject-Driven Image Generation

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    RAVA retrieves view-consistent target-subject images via a learned cross-instance embedding and LogDet subset selection, then uses them in a multi-reference generator to improve cross-subject viewpoint alignment.

Pith tools