REVIEW 16 cited by
JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present JanusFlow, a powerful framework that unifies image understanding and generation in a single model. JanusFlow introduces a minimalist architecture that integrates autoregressive language models with rectified flow, a state-of-the-art method in generative modeling. Our key finding demonstrates that rectified flow can be straightforwardly trained within the large language model framework, eliminating the need for complex architectural modifications. To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training. Extensive experiments show that JanusFlow achieves comparable or superior performance to specialized models in their respective domains, while significantly outperforming existing unified approaches across standard benchmarks. This work represents a step toward more efficient and versatile vision-language models.
Forward citations
Cited by 16 Pith papers
-
Flow Straight and Fast in Hilbert Space: Functional Rectified Flow
Functional rectified flow is defined and proved to preserve marginals in separable Hilbert spaces, with functional flow matching and probability-flow ODEs as special cases.
-
D-AR: Diffusion via Autoregressive Models
D-AR recasts pixel-space diffusion as vanilla autoregressive next-token prediction using a diffusion-ordered discrete tokenizer, reaching 2.09 FID on ImageNet 256x256 with a 775M Llama backbone.
-
Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer
Z-Image is an efficient 6B-parameter foundation model for image generation that rivals larger commercial systems in photorealism and bilingual text rendering through a new single-stream diffusion transformer and strea...
-
Interleaving Reasoning for Better Text-to-Image Generation
A text-image-text-image multi-turn pipeline improves text-to-image generation, gaining up to 8 points over the base model on several benchmarks while adding an image-conditioned reflection step.
-
MPCC: A Novel Benchmark for Multimodal Planning with Complex Constraints in Multimodal Large Language Models
A new 2,700-task benchmark with budget, time, and distance constraints shows that even the best multimodal LLMs produce feasible plans less than 22% of the time.
-
Epona: Autoregressive Diffusion World Model for Autonomous Driving
An autoregressive diffusion world model jointly generates the next camera frame and a multi-step trajectory, enabling long videos and real-time planning for autonomous driving.
-
MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings
A two-stage training recipe that converts causal VLMs into bidirectional multimodal embedding models, achieving SOTA on MMEB.
-
UniCode$^2$: Cascaded Large-scale Codebooks for Unified Multimodal Understanding and Generation
UniCode² builds a 500K-entry codebook from clustered SigLIP embeddings and uses a cascaded frozen-plus-trainable codebook to unify multimodal understanding and generation with stable training and high token utilization.
-
FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities
A 1.5B unified multimodal model trained with discrete flow matching and metric-induced probability paths matches autoregressive baselines of similar size on generation and understanding benchmarks.
-
MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models
MMIG-Bench is a unified benchmark of 4,850 prompts and 1,750 reference images with a three-level evaluation suite, including the VQA-based Aspect Matching Score that correlates with human ratings.
-
UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation
UniGen shows a 1.5B model trained on open data can beat larger systems on image understanding and generation once it verifies its own outputs with chain-of-thought and Best-of-N selection.
-
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths
UniMoD prunes tokens with task-specific routers in unified multimodal transformers, cutting training FLOPs by 15-40% while roughly maintaining benchmark performance.
-
PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework
PosterCraft improves text-to-poster generation by cascading four stages of training (text rendering, region-weighted fine-tuning, preference optimization, and vision-language feedback), outperforming open-source basel...
-
UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning
A self-improving post-training method that uses a model's own generated images as training data, with SFT and GRPO, improves generation and understanding and reduces task imbalance.
-
ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
A ComfyUI-based multi-agent system with semantic workflow modules and tree-based local-feedback planning reports near-perfect pass rates on ComfyBench and competitive scores on GenEval and Reason-Edit.
-
Towards Evaluating Robustness of Prompt Adherence in Text to Image Models
New benchmark results show that Stable Diffusion 3.x and Janus Pro models struggle to place simple geometric shapes in the correct image quadrant, with best F1 scores around 0.41 to 0.5.
Discussion (0). Continue with ORCID to comment.