REVIEW 2 major objections 18 cited by
Image Generators are Generalist Vision Learners
T0 review · 2 major / 0 minor · reviewed 2026-07-05 · glm-5.2
Pith's one-line read Image Generation Training Builds General Vision Understanding
desk verdict Bold paradigm-shift claim from abstract-only review; SOTA numbers and contamination safety cannot be verified without full text. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Vision Banana (instruction-tuned Nano Banana Pro); RGB-image output parameterization for vision tasks; instruction-tuning on mixed generative and vision-task data
What would settle it
A discriminatively pretrained model of matching scale and instruction-tuning data that achieves comparable segmentation and depth results would undermine the claim that generative pretraining is the key driver of visual understanding.
Extended reading notes
Core claim
The paper's central discovery is that an image generator, when lightly instruction-tuned, can achieve state-of-the-art performance on perception tasks by treating those tasks' outputs as images to be generated. The load-bearing mechanism is the reframing of perception outputs (segmentation masks, depth maps) as RGB images, so that the same generative training objective that produces images also produces structured visual understanding. The authors present this as evidence that generation pretraining itself, not task-specific architectural specialisation, is the source of the model's understanding capability.
Load-bearing premise
The paper attributes the strong perception results to the generative pretraining objective, but without an ablation comparing against a discriminatively pretrained model of equivalent scale and data, it is unclear whether generation training is the necessary cause or simply one sufficient path among others.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces Vision Banana, a generalist vision model built by instruction-tuning the Nano Banana Pro (NBP) image generator on a mixture of its original training data and vision task data. Perception outputs are parameterized as RGB images, reframing segmentation and depth estimation as image generation. The authors claim state-of-the-art results on segmentation (rivaling SAM 3) and metric depth (rivaling Depth Anything), and argue that image generation pretraining plays a role analogous to LLM pretraining for visual understanding.
Significance. The central thesis—that generative pretraining can serve as a unified foundation for both image generation and visual understanding—is provocative and, if correct, would represent a meaningful paradigm shift in computer vision. The reframing of perception outputs as RGB images is an elegant conceptual contribution with clear practical appeal. However, this assessment is based solely on the abstract; the full manuscript was not available for review, so the empirical and methodological details underpinning the SOTA claims could not be examined.
major comments (2)
- The paper's central empirical claims (SOTA on segmentation and depth) rest on fine-tuning a web-scale generative model (NBP). A load-bearing concern is whether standard benchmark test images (e.g., COCO, NYUv2, KITTI) were present in NBP's pretraining corpus. Generative diffusion models are known to memorize training images; if test images appeared in pretraining, the reported SOTA numbers could be inflated by memorization rather than genuine understanding. The manuscript must explicitly address data contamination: either by documenting that test images were excluded from NBP's training data, by evaluating on held-out splits, or by providing memorization analysis. Without this, the SOTA claims are not securely established.
- The causal claim that 'image generation pretraining is a generalist vision learner' requires isolating the contribution of the generative pretraining objective. The abstract does not mention ablations against a discriminative pretraining baseline of equivalent scale and capacity. Without such a comparison, the results could be driven by model scale or the instruction-tuning data mixture rather than by the generative objective itself. The full manuscript should include this ablation to support the paradigm-shift argument.
Simulated Author's Rebuttal
We thank the referee for a thoughtful assessment of the central thesis. We address both major concerns below. We note that the referee's review is based on the abstract alone; the full manuscript (which was submitted alongside the abstract) contains additional experiments and discussion relevant to both points. We summarize what is already in the manuscript and what revisions we will make in response.
read point-by-point responses
-
Referee: Data contamination: benchmark test images (COCO, NYUv2, KITTI) may have appeared in NBP's web-scale pretraining corpus, inflating SOTA numbers via memorization. The manuscript must address this.
Authors: This is a legitimate and important concern. We acknowledge that NBP is trained on a large-scale web image corpus, and we cannot guarantee zero overlap between that corpus and standard benchmark test sets. The full manuscript (Section 5.3) includes a preliminary memorization analysis: we compute nearest-neighbor retrieval between benchmark test images and a subsample of NBP's training data, and we also measure whether the model reproduces near-duplicate test images at the pixel level during generation. The results indicate low overlap rates and no evidence of systematic memorization driving the reported metrics. However, we agree this analysis is not exhaustive. In the revision we will: (1) expand the nearest-neighbor analysis to cover the full training corpus rather than a subsample, (2) report performance on a newly held-out split of images confirmed absent from NBP's training data, and (3) include a comparison of per-image performance against retrieval distance to the nearest training image, to show that performance does not degrade for images far from the training distribution. We believe these additions will substantially strengthen the empirical claims. revision: yes
-
Referee: The causal claim that generative pretraining is responsible for the results requires an ablation against a discriminative pretraining baseline of equivalent scale and capacity. Without this, results could be driven by model scale or instruction-tuning data rather than the generative objective.
Authors: We agree that isolating the contribution of the generative pretraining objective is essential for the paradigm-shift argument. The full manuscript includes a partial ablation: we compare Vision Banana against (a) NBP without instruction-tuning (zero-shot transfer to vision tasks via prompting only), and (b) a smaller discriminative backbone (ViT-L/14, CLIP-pretrained) instruction-tuned on the same vision task data mixture. These comparisons show that the generative pretrained model substantially outperforms the discriminative baseline at smaller scale, and that instruction-tuning provides a large boost over zero-shot prompting. However, we acknowledge that the referee's specific request—a discriminative model of equivalent scale and capacity to NBP—is not included, and we agree this would be the strongest possible control. The practical obstacle is that training a discriminative model at the scale of NBP (which is a multi-billion-parameter diffusion model trained on billions of images) requires computational resources comparable to the original NBP pretraining, which is beyond what we can provision. In the revision, we will (1) explicitly acknowledge this limitation in the discussion, (2) add comparison to the largest publicly available discriminative foundation models (e.g., DINOv2-giant, OpenCLIP-G) instruction-tuned on the same data, which narrows the scale gap as much as feasible, and (3) frame the causal claim more carefully, noting that while our evidence is consistent with the generative objective being the key factor, a fully controlled ablation at matched scale remains future work. We will soften the claim from 'image generation pretraining is a generalist vision learner' to 'image generation pretraining can serve as an effective foundation for generalist视觉理解,' revision: partial
- We cannot provide a discriminative pretraining ablation at scale fully matched to NBP. Training such a model requires resources comparable to the original NBP pretraining run, which is beyond our means. We will provide the closest feasible approximation (large publicly available discriminative models) and will be transparent about this limitation, but the fully controlled comparison the referee requests is not something we can deliver in this revision.
Circularity Check
No circularity detected; the paper makes an empirical claim supported by fine-tuning experiments, not a derivation that reduces to its inputs.
full rationale
The paper's central claim — that image generation pretraining produces generalist visual representations — is an empirical hypothesis tested by instruction-tuning a generative model (NBP) on vision task data and evaluating on standard benchmarks. The argument chain is: (1) take a pretrained image generator, (2) fine-tune it on perception tasks reformulated as RGB image generation, (3) measure performance on held-out benchmarks. No step in this chain is defined in terms of its output. The paper does not fit a parameter to evaluation data and then call the fitted value a prediction, does not invoke a self-cited uniqueness theorem to force its conclusion, and does not rename a known result as a new derivation. The reader's concern about missing ablations (no discriminative baseline at equivalent scale) and the skeptic's concern about potential test-set contamination in NBP's pretraining data are legitimate methodological and validity risks, but they are correctness concerns, not circularity — the paper's logic does not reduce to its inputs by construction. This is a standard empirical paper whose claims rest on experimental evidence that could in principle be falsified independently.
Assumptions & free parameters
free parameters (2)
- Instruction-tuning data mixture ratio
- RGB task output encoding scheme
assumptions (2)
- domain assumption Image generation pretraining produces representations that are transferable to understanding tasks.
- ad hoc to paper Parameterizing perception outputs as RGB images is a sufficient interface for SOTA performance.
invented entities (2)
-
Vision Banana
independent evidence
-
Nano Banana Pro (NBP)
Cite this review
Pith. "Pith review of Image Generators are Generalist Vision Learners." pith.science (2026). https://pith.science/paper/7VITVVLL
@misc{pith2026260420329,
author = {Pith},
title = {Pith review of: Image Generators are Generalist Vision Learners},
year = {2026},
howpublished = {\url{https://pith.science/paper/7VITVVLL}},
note = {Machine review of arXiv:2604.20329}
}
read the original abstract
Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining. While it has long been conjectured that the ability to create visual content implies an ability to understand it, there has been limited evidence that generative vision models have developed strong understanding capabilities. In this work, we demonstrate that image generation training serves a role similar to LLM pretraining, and lets models learn powerful and general visual representations that enable SOTA performance on various vision tasks. We introduce Vision Banana, a generalist model built by instruction-tuning Nano Banana Pro (NBP) on a mixture of its original training data alongside a small amount of vision task data. By parameterizing the output space of vision tasks as RGB images, we seamlessly reframe perception as image generation. Our generalist model, Vision Banana, achieves SOTA results on a variety of vision tasks involving both 2D and 3D understanding, beating or rivaling zero-shot domain-specialists, including Segment Anything Model 3 on segmentation tasks, and the Depth Anything series on metric depth estimation. We show that these results can be achieved with lightweight instruction-tuning without sacrificing the base model's image generation capabilities. The superior results suggest that image generation pretraining is a generalist vision learner. It also shows that image generation serves as a unified and universal interface for vision tasks, similar to text generation's role in language understanding and reasoning. We could be witnessing a major paradigm shift for computer vision, where generative vision pretraining takes a central role in building Foundational Vision Models for both generation and understanding.
Forward citations
Cited by 18 Pith papers
-
The generator is the tracker: Multi-object tracking by painting persistent identity colours
A video generator fine-tuned to emit persistent per-person colors tracks DanceTrack dancers at 40.3 HOTA with no detector or tracker.
-
Vision as Unified Multimodal Generation
A single unified multimodal model matches leading task-specialized vision systems across detection, segmentation, dense geometry, and multi-view 3D by casting all outputs as native text or image generation.
-
From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models
A frozen DiT plus task LoRA and a ~33K-parameter token-local linear head reads out pixel-space dense fields and sets SOTA on matting, KITTI depth, and referring segmentation while running up to 2.48 imes faster than e...
-
Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing
A unified 3D multimodal model combines understanding, text-to-3D generation, instruction-guided editing, and part generation in one architecture, trained on an 87M-sample corpus, with claimed state-of-the-art results.
-
Image-Space Rule Discovery
A new benchmark shows frontier image-editing models can sometimes follow instructions rendered inside an image and answer on the same canvas, with the best model, Nano Banana Pro, reaching 48.7% strict proxy accuracy ...
-
MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities
A decoder-only transformer with two internal experts achieves competitive any-to-any translation across text and 14 vision modalities in one model, using a staged curriculum and uniform timestep sampling.
-
Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text
Image-generation models become competitive on spatial benchmarks when answers are expressed as protocol-constrained pixels, while text-output VLMs retain an edge on compositional reasoning — with a parser-sensitivity ...
-
Video Generation Models are General-Purpose Vision Learners
A video-diffusion backbone fine-tuned as a single-step multi-task perceiver matches or beats specialists on depth, normals, pose and segmentation, with high data efficiency and sim-to-real transfer.
-
Prompting Diffusion Models for Zero-Shot Instance Segmentation
Prompt2Seg augments diffusion models with an explicit spatial prompt conditioning branch, enabling zero-shot instance segmentation that generalizes from limited synthetic category training to diverse unseen objects an...
-
ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?
ImageWAM shows image editing models can replace video generation in world action models, delivering better performance with 6x lower FLOPs and 4x lower latency by using edit-derived KV caches as compact context.
-
Modality Forcing for Scalable Spatial Generation
Modality Forcing lets a single DiT produce image and depth outputs in any order after training on sparse real-world depth, with larger image-pretrained models yielding better depth accuracy and a 57% AbsRel reduction ...
-
iMaC: Translating Actions into Motion and Contact Images for Embodied World Models
iMaC introduces image-based action tokens in a dual-branch architecture to improve future state prediction and control in embodied world models over vector-based baselines.
-
PathPainter: Transferring the Generalization Ability of Image Generation Models to Embodied Navigation
PathPainter transfers image generation models to embodied navigation by generating traversability masks from BEV images and language instructions while using cross-view localization to reduce odometry drift.
-
Leveraging Image Generators to Address Data Scarcity: The Gen4Regen Dataset for Forest Regeneration Mapping
Prompt-generated image-mask pairs, mixed with real UAV imagery at a 40:60 ratio, lift forest-regeneration segmentation by >15 F1 points over supervised baselines and sharply improve rare-species F1.
-
Open-Source Image Editing Models Are Zero-Shot Vision Learners
Open-source image-editing models show competitive zero-shot performance on monocular depth, surface normals, and semantic segmentation, sometimes matching tuned models.
-
Diffusion Model as a Generalist Segmentation Learner
DiGSeg repurposes diffusion U-Nets as generalist segmentation learners by conditioning on image-mask latents and multi-scale CLIP text features, achieving strong cross-domain performance.
-
PaintBench: Deterministic Evaluation of Precise Visual Editing
PaintBench provides a scalable deterministic benchmark for precise visual editing operations, revealing that even the best of 11 models achieves only 17.1% mIoU and that scores correlate strongly with applied data vis...
-
Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence
Integrating generative novel-view synthesis into LMM reasoning loops improves accuracy on spatial subtasks by 1.3 to 3.9 percentage points across multiple models and tasks.
Reviewed July 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.