Pith. sign in

REVIEW 8 cited by

PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.15278 v4 pith:GWTJ2MGI submitted 2024-09-23 cs.CV

PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions

Weifeng Lin , Xinyu Wei , Renrui Zhang , Le Zhuo , Shitian Zhao , Siyuan Huang , Huan Teng , Junlin Xie
show 3 more authors
Yu Qiao Peng Gao Hongsheng Li
This is my paper
classification cs.CV
keywords imagegenerationpixwizardinstructionsmodeltasksassistantcapabilities
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X LinkedIn Reddit HN
read the original abstract

This paper presents a versatile image-to-image visual assistant, PixWizard, designed for image generation, manipulation, and translation based on free-from language instructions. To this end, we tackle a variety of vision tasks into a unified image-text-to-image generation framework and curate an Omni Pixel-to-Pixel Instruction-Tuning Dataset. By constructing detailed instruction templates in natural language, we comprehensively include a large set of diverse vision tasks such as text-to-image generation, image restoration, image grounding, dense image prediction, image editing, controllable generation, inpainting/outpainting, and more. Furthermore, we adopt Diffusion Transformers (DiT) as our foundation model and extend its capabilities with a flexible any resolution mechanism, enabling the model to dynamically process images based on the aspect ratio of the input, closely aligning with human perceptual processes. The model also incorporates structure-aware and semantic-aware guidance to facilitate effective fusion of information from the input image. Our experiments demonstrate that PixWizard not only shows impressive generative and understanding abilities for images with diverse resolutions but also exhibits promising generalization capabilities with unseen tasks and human instructions. The code and related resources are available at https://github.com/AFeng-x/PixWizard

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Masked Generative Transformer Is What You Need for Image Editing

    cs.CV 2026-05 unverdicted novelty 8.0

    EditMGT applies masked generative transformers with attention consolidation and region-hold sampling to deliver state-of-the-art localized image editing at 6x the speed of diffusion methods.

  2. MetaEarth-MM: Unified Multimodal Remote Sensing Image Generation with Scene-centered Joint Modeling

    cs.CV 2026-05 conditional novelty 7.0

    MetaEarth-MM unifies multi-modal remote sensing image generation and any-to-any translation across five modalities via scene-centered joint modeling on the new EarthMM dataset.

  3. Edit Where You Mean: Region-Aware Adapter Injection for Mask-Free Local Image Editing

    cs.CV 2026-04 unverdicted novelty 7.0

    A co-trained adapter framework enables mask-free local editing in DiTs by factorizing edit semantics from spatial location and jointly learning a mask predictor.

  4. Your Pre-trained Diffusion Model Secretly Knows Restoration

    cs.CV 2026-04 unverdicted novelty 7.0

    Pre-trained diffusion models inherently support image restoration that can be unlocked by optimizing prompt embeddings at the text encoder output using a diffusion bridge formulation, achieving competitive results on ...

  5. MICo-150K: A Comprehensive Dataset Advancing Multi-Image Composition

    cs.CV 2025-12 unverdicted novelty 7.0

    MICo-150K is a new 150K-image dataset with 7 tasks, a De&Re real-image subset, MICo-Bench, and Weighted-Ref-VIEScore metric that improves AI models for generating consistent composites from arbitrary numbers of refere...

  6. Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning

    cs.CV 2025-07 unverdicted novelty 7.0

    Presents Reason50K dataset and ReasonBrain framework for hypothetical instruction-based image editing that requires physical, temporal, causal, and story reasoning.

  7. Video models are zero-shot learners and reasoners

    cs.LG 2025-09 unverdicted novelty 6.0

    Generative video models exhibit emergent zero-shot capabilities across perception, manipulation, and basic reasoning tasks.

  8. Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Planning Study

    cs.CV 2026-05 unverdicted novelty 5.0

    DiffKT3D transfers priors from video diffusion models to 3D radiotherapy dose prediction via modality-specific embeddings and clinically guided RL, reducing voxel MAE from 2.07 to 1.93 and claiming SOTA over the GDP-H...