Image-Space Rule Discovery

Hirokatsu Kataoka; Misora Sugiyama; Toya Oyama

REVIEW 3 major objections 4 minor 40 references

Image-Space Rule Discovery

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The strongest image-editing model solves 48.7% of a new visual-rule worksheet benchmark, far below human solvability.

desk verdict WISRD is a genuinely useful benchmark with an honest evaluation protocol, but its headline rates rest on a glyph-blind proxy and the only human audit skips the model that tops the table. read the letter →

arxiv 2608.00490 v1 pith:KLBYXCFY submitted 2026-08-01 cs.CV

Misora Sugiyama , Toya Oyama , Hirokatsu Kataoka This is my paper

classification cs.CV

keywords image-spacerulediscoveryimageeditingworksheetbenchmarkvisualreasoningin-imageinstructionspreservation-constrainedinstructionfollowinggenerativemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether image-editing models can discover and apply visual rules directly in image space, the way a human reads a worksheet, infers the rule, and writes the answer onto the page. To test this, it introduces WISRD, a benchmark of 11 synthetic worksheet tasks under eight information conditions that require reading rendered in-image instructions, localizing the answer, controlling output count, and preserving everything else. The central finding is that current systems can partially do this: the best model, Nano Banana Pro, achieves 48.7% strict automatic pass rate, while other editors score in the low teens or zero, and human solvers reach 96.1%. The paper also shows that the best model can sometimes rely on instructions rendered inside the image even when the external text prompt is generic, and that success varies sharply across reasoning abilities such as symbolic tracking versus abstract analogy. A sympathetic reader would care because the benchmark turns image editing from open-ended generation into a constrained operational problem-solving task, exposing failure modes that text-answering benchmarks miss.

What carries the argument

WISRD is the central instrument: a worksheet-based benchmark with a fixed 1024x1024 layout, 11 task templates grouped into five families (marking, filling, copying, counting, suppression), and eight controlled information conditions that vary whether the external prompt, rendered image text, and visual references are present. Success is judged by task-specific automatic proxies, Auto-Strict and Auto-Loose, that check red-mask placement, output count, preservation of black geometry and structure, and format validity, along with eight operational failure dimensions such as destination binding and unnecessary-edit suppression. The benchmark's design separates where task information comes from a

What would settle it

Run the WISRD letter and digit items through a scorer that OCR-reads the produced glyph and compares it to the ground-truth symbol; if Auto-Strict pass rates drop materially, the proxy is the load-bearing part of the result. A smaller direct check: have a panel re-judge a random 200-output sample of Nano Banana Pro Auto-Strict 'successes' for semantic glyph identity and count correctness.

Watch

Extended reading notes

Core claim

The paper claims that image-editing models can, to a limited but measurable degree, perform image-space rule discovery: given a worksheet image containing rendered instructions, the model must infer the requested operation, compute the answer, and write it back as a localized red edit on the same canvas while preserving all non-target content. On the shared V0–V3 no-reference subset, Nano Banana Pro reaches 48.7% Auto-Strict and 64.0% Auto-Loose proxy pass rates, Qwen-Image-Edit reaches 13.4% and 20.4%, FLUX.2 Klein 4B variants reach about 11% Auto-Strict, and InstructPix2Pix scores 0.0%. The paper further shows that Nano Banana Pro can often use rendered in-image text as an operational task

Load-bearing premise

The headline pass rates rest on automatic proxies that measure red-mask placement, output count, and preservation but never check whether a produced letter or digit is the correct symbol; if those proxies overstate true task success, the reported numbers overstate rule discovery.

Editorial extensions

If this is right

If the central claim holds, image-editing capability should be evaluated as constrained problem solving with separate scores for correctness, preservation, suppression, and format validity, because a model that does nothing can score high on preservation.
Rendered in-image instructions can act as part of the task specification, meaning evaluation protocols should not assume the external prompt is the only source of task information.
Visual reference images can interfere with task interpretation and target binding, so adding in-context examples is not a reliable way to improve worksheet-style rule discovery.
Reasoning abilities do not transfer uniformly: a model may track hundreds of sequential symbol operations yet fail abstract visual analogy, so single aggregate scores misrepresent capability.
Explicit instruction compliance does not imply spontaneous detection of problems such as unsatisfiable puzzles, suggesting current models rely heavily on literal prompt following.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported 48.7% and 63.3% Auto-Strict numbers likely overstate true semantic success on letter and digit tasks, since the automatic scorer does not recognize whether the written glyph is actually the correct symbol; an OCR-augmented or multi-rater human audit on a random sample of 'successes' would give a more honest estimate.
The external-prompt-dominates-conflict result implies a practical robustness risk: in real deployment, a contradictory text prompt may override correct visual instructions in the image, so future systems need explicit mechanisms for reconciling multiple instruction sources.
WISRD-style renderable worksheets could serve as a scalable training signal for image-native models: procedurally generated puzzles with known ground-truth edits could be used for post-training, but whether that improves genuine rule induction rather than template retrieval remains untested.
The sharp divergence between sequential symbol chains (94%) and RAVEN analogy (22.9%) suggests that 'visual reasoning' is not a single capacity, and benchmark designers should separate symbolic tracking from abstract relational inference when characterizing model progress.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

WISRD is a real benchmark contribution, but the headline numbers should be read as upper bounds on the capability being claimed. The paper introduces a sensible protocol: 11 worksheet tasks, eight information conditions that cleanly separate external prompt, rendered in-image text, and visual references, and a same-canvas answer format that forces a model to bind its answer to a location and preserve the rest. That measurement infrastructure is new and worth having. The strongest empirical finding—Nano Banana Pro can follow instructions rendered inside the image even with a generic external prompt (63.3% Auto-Strict on the TXT slice)—is genuinely interesting and mostly well argued. The paper also does good failure analysis: the Wason probe shows a permissive red-mask proxy would score 99% when exact card selection is 49%, which is a nice demonstration of why proxy choice matters.

The soft spots are real but not fatal. The letter/digit Auto-Strict proxy does not check glyph identity; it uses red-content and mask-overlap heuristics. The paper discloses this, but the disclosure doesn't quantify the bias, and the top model is strongest on exactly the copying/counting tasks where glyph semantics matter. The only human audit is of FLUX, not of Nano Banana Pro, so we have no direct evidence of true glyph accuracy for the model carrying the paper's main claim. The abstract also overreaches: 'current image-editing models can partially rely' is supported by one proprietary model; Qwen gets 15.1% and the FLUX variants 0% on TXT. That's 'a proprietary model can,' not a general finding. The release artifacts are promised only after review, thresholds aren't given, and there are no error bars—minor for a first benchmark paper, but it limits independent verification.

On balance, the benchmark protocol holds up. The measurement is a good idea, the execution is transparent, and the limitations are acknowledged in the text. What needs work is the gap between the careful caveats and the bold phrasing in the abstract, plus an audit of the top model or a tighter glyph-aware scorer. This deserves peer review. I'd encourage the editor to send it with a request that the authors release the renderer/evaluator (or at least the exact scoring thresholds) and add a human audit covering Nano Banana Pro before accepting. I'd cite it if the artifacts land.

Referee Report

3 major / 4 minor

Summary. The paper introduces WISRD, a worksheet-style benchmark that asks image-editing models to solve visual problems end-to-end in image space: read rendered instructions, infer the rule, and write the answer as a localized red edit into the same canvas while preserving everything else. The benchmark includes 11 core tasks under eight information conditions plus four supplementary reasoning-stress probes. The main empirical claims are: (i) among the evaluated frontier editors, Nano Banana Pro achieves the highest Auto-Strict proxy pass rate (48.7% on the shared V0–V3 no-reference subset), with Qwen-Image-Edit at 13.4%, FLUX.2 Klein 4B variants around 11%, and InstructPix2Pix at 0.0%; (ii) rendered in-image instructions can partially serve as operational task specifications, with Nano Banana Pro reaching 63.3% Auto-Strict on the all-task TXT slice; and (iii) supplementary diagnostics show strong performance on sequential symbol manipulation and 4×4 Sudoku but weak performance on RAVEN-style analogy. The paper is careful to describe its automatic scoring protocol, first-shot policy, and many limitations.

Significance. If the results hold, WISRD is a useful and timely benchmark. It operationalizes a visual-to-visual interface for rule discovery, measures behaviors that text-based VQA benchmarks cannot see (destination binding, output-count control, unnecessary-edit suppression, preservation), and provides a shared protocol for comparing image-editing models. The paper also gives a structured failure taxonomy and several controlled ablation probes. The main strengths are the detailed evaluation protocol, the explicit first-shot policy, the separation of proxy and human-audit evidence, and the unusually candid limitation statements. The central claim, however, depends on automatic proxies whose semantic fidelity for letter/digit tasks is not directly established for the model carrying the headline result.

major comments (3)

[Section 4, 'Correctness and preservation'; Section 5, human audit] The Auto-Strict proxy for letter and digit tasks uses answer-region red-content measures and red-mask overlap against the ground-truth answer mask, but the paper states that it does not explicitly recognize the semantic identity of the produced glyph. The only human audit is for FLUX.2 Klein 4B API outputs, where wrong glyphs accounted for 5.8% of Audit-Strict failures; no comparable audit is reported for Nano Banana Pro. Because the headline numbers (48.7% overall, 63.3% on the TXT slice) are driven substantially by copying and counting tasks, the proxy could be rewarding red marks that are locally plausible but semantically incorrect glyphs. The limitation is disclosed, but the disclosure does not bound the potential inflation for the model that carries the central empirical claim. Please provide either a semantic audit of Nano Banana Pro's letter/digit outputs (even on a subset) or a
[Section 5, 'Human task-solvability check'] The comparison to the human reference is potentially misleading. The human task-solvability check used clicks or typed responses, not pixel-level image editing, and the authors acknowledge it is not a direct human image-editing baseline. Yet the abstract and text frame the result as 'far below human task solvability' (96.1% Solve-Strict vs. 48.7%). Since the human interface differs in exactly the dimensions WISRD is designed to test (destination binding, output-count control, preservation), the human number is not an apples-to-apples ceiling. This does not invalidate the model comparison, but the framing should be softened or accompanied by a human image-editing pilot, even a small one.
[Section 6, reference-condition ablations] The conclusion that visual references 'do not explain Nano Banana Pro's advantage' is based on a five-task spatial marking/filling subset (V4–V7: 8.0/15.6 to 8.4/16.4% Auto-Strict/Auto-Loose). The paper does disclose this is a subset, but the sentence in the main text reads as a general claim about reference conditions across WISRD. Given that Target-Shape Filling carries most of the reference-condition success, the claim should be explicitly scoped to the five-task subset throughout, and the possibility that reference conditions behave differently on copying/counting/suppression tasks should be stated.

minor comments (4)

[Throughout] Several section headings and figure captions are missing spaces (e.g., 'NanoBananaProStandsOutAndCanUseRenderedWISRDInstructions.', 'Fig. 2:Overview', 'Fig. 3:FLUX.2'). These are typographical and should be fixed in the camera-ready version.
[Section 4, 'Correctness and preservation'] The description of Auto-Strict for letter/digit tasks would benefit from one concrete example of how red-mask overlap is thresholded (e.g., Dice coefficient threshold). Currently the reader cannot assess how tight the proxy is.
[Section 6, CREF probe] The CREF results are described as 'canonical-target proxy match rates' but the text also says they show that an incompatible reference 'does not provide a robust solution path.' This is a reasonable interpretation, but the lack of manual source-following labels means the claim about which source the model followed is not directly supported; the paper acknowledges this, but it could be stated in the main text as well.
[Section 5, 'Human audit'] The human audit is single-rater and covers only FLUX.2 Klein 4B API. Please report inter-rater agreement if any second-rater data were collected, and clearly label the audit as exploratory; the paper does the latter, but a confidence interval on the 7.3% Audit-Strict success rate would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a measurement study whose pass rates are computed from fixed proxies and model outputs, not fitted or derived from self-citations.

full rationale

The paper's central claims are empirical benchmark results. The Auto-Strict and Auto-Loose pass rates are obtained by applying fixed, pre-specified task-specific scoring proxies to first-shot model outputs against ground-truth answer masks; no parameter is fitted to model outputs to produce the reported ranking. The letter/digit proxy limitation (not checking semantic glyph identity) is explicitly disclosed and affects construct validity, not derivation circularity. The human audit is explicitly separate from the main ranking and is single-rater, which is a methodological caveat rather than a circular step. The paper contains no load-bearing self-citations: the cited prior works are external to the authors, and no uniqueness theorem or ansatz is imported from the same authors to force a conclusion. The OCR+Gemini diagnostic is clearly labeled as a diagnostic pipeline with a deterministic renderer, not an end-to-end baseline. Overall, the measurement chain is transparent and non-circular, with any concerns belonging to evaluation validity or generalizability rather than circular reasoning.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central claims are empirical measurements; they rely on the validity of the paper's own scoring proxies and on the representativeness of the chosen model endpoints. No parameters are fitted to data in a derivation sense, so the ledger lists the benchmark's hand-chosen thresholds and domain assumptions.

free parameters (2)

Auto-Strict task-specific thresholds
Hand-chosen spatial/pixel-overlap thresholds and red-content limits define pass/fail for each of the 11 task families; values are not reported or varied in a sensitivity analysis (Section 4).
Auto-Loose relaxation parameters
Relaxed spatial or pixel-overlap thresholds permit aligned or resized outputs to pass; the relaxation amount is not quantified (Section 4).

assumptions (3)

domain assumption Proxy metrics (red-mask overlap, black F1, structure F1) approximate the operational requirements of image-space rule discovery.
Used throughout Sections 4 and 5 to convert outputs into pass/fail; the paper discloses that letter/digit proxies do not recognize glyph semantics.
domain assumption The fixed 1024x1024 synthetic worksheet layout and the 11 task templates instantiate 'image-space rule discovery' as defined in Section 3.
The benchmark definition relies on this operationalization; generalization to natural images and scientific diagrams is untested (Section 7, Limitations).
domain assumption The tested API endpoints (gemini-3-pro-image-preview, FLUX.2 Klein 4B, etc.) are representative of the published models at the stated evaluation period.
Exact identifiers and dates are reported, but proprietary endpoints can change; the paper itself notes this in Limitations.

invented entities (1)

WISRD (Worksheet Image-Space Rule Discovery benchmark)
purpose: Provides tasks, conditions, and scoring proxies for measuring whether image editors can perform same-canvas rule discovery.
The benchmark and its scoring are defined by the authors; the renderer and evaluator are planned for public release after review, so there is no independently verifiable artifact outside the paper at this time.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Image-Space Rule Discovery." pith.science (2026). https://pith.science/paper/KLBYXCFY

@misc{pith2026260800490,
  author       = {Pith},
  title        = {Pith review of: Image-Space Rule Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KLBYXCFY}},
  note         = {Machine review of arXiv:2608.00490}
}

read the original abstract

Can image-editing models discover visual rules in image space and complete problem-solving end-to-end? We tackle this question in the spirit of a human worksheet test (e.g., an IQ test), using problems that require models to read image-based instructions, recognize the problem, infer the answer, bind it to the correct destination, control output count, suppress unnecessary edits, and preserve the input and format. We introduce WISRD, a Worksheet Image-Space Rule Discovery benchmark with 11 core tasks under eight information conditions, spanning localized marking, filling, copying, counting, and no-edit suppression, together with four supplementary reasoning-stress probes for multi-step spatial manipulation, abstract pattern reasoning, logical inference, and constraint-based problem solving. We identify three key findings as follows. (i) Among the frontier image-editing models evaluated, Nano Banana Pro achieves the highest score. On the shared V0--V3 no-reference subset, the Auto-Strict proxy pass rates are 48.7% for Nano Banana Pro, 13.4\% for Qwen-Image-Edit, 11.5% for FLUX.2 Klein 4B API, 11.3% for FLUX.2 Klein 4B open-weight, and 0.0% for InstructPix2Pix. (ii) Analysis reveals that current image-editing models can partially rely on rendered in-image instructions even when the external prompt is absent or merely generic. (iii) In small supplementary diagnostics, Nano Banana Pro achieves 70.0% on 4-by-4 Sudoku and 22.9% on public RAVEN pattern-discovery items in image space.

Figures

Figures reproduced from arXiv: 2608.00490 by the authors.

**Figure 1.** Overview of WISRD. (a) Task-family Auto-Strict pass rates and overall AutoStrict/Auto-Loose pass rates on the shared V0–V3 no-reference subset. Within each model group, the first five bars aggregate the 11 tasks by family: Marking (Line Intersections, Line Midpoint Mark, Circle Every Dot, and Overlapping-Shape Center Marking), Filling (Target-Shape Filling), Copying (Circled-Letter Copying, All-A Control, and Sin… view at source ↗

**Figure 2.** Overview of WISRD, our benchmark for Image-Space Rule Discovery. (a) Representative problem–answer pairs, with task labels color-coded by family: Marking, Filling, Copying, Counting, and Suppression. Red is reserved for the required output edit. (b) Shared 1024×1024 WISRD worksheet format, where rendered in-image instructions appear above the light-gray frame and problem elements plus answer boxes appear inside th… view at source ↗

**Figure 3.** FLUX.2 Klein 4B API Auto-Strict and Auto-Loose proxy pass rates across WISRD tasks and information conditions. Low-information conditions can appear to preserve the input, but this often reflects inaction rather than successful problem solving. score, because Gemini predicts a structured edit plan and a deterministic renderer draws the final red edit. The gap between this diagnostic pipeline and human solve-only p… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Motivation of the WISRD P-series example for instruction-region red-signal analysis. produced little instruction-region red signal under the automatic proxy. Redrendered phrases triggered instruction-region red flags, but because those phrases were already red in the …

**Figure 5.** Figure 5: Additional WISRD reasoning-stress diagnostics and representative problem– answer examples for Nano Banana Pro. Task accuracy measures task-specific correctness. For Wason, this is exact selection of the correct card set, not the looser red-mask overlap proxy. Task+Fmt…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 29 canonical work pages

[1]

Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al.: Flamingo: A visual language model for few-shot learning. In: Adv. Neural Inform. Process. Syst. (2022)

work page 2022
[2]

Black Forest Labs: FLUX.2 [klein]: Towards interactive visual intelligence (2026), accessed: 4 July 2026

work page 2026
[3]

In: IEEE Conf

Brooks, T., Holynski, A., Efros, A.A.: InstructPix2Pix: Learning to follow image editing instructions. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

work page 2023
[4]

Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. In: Adv. Neural Inform. Process. Syst. (2021)

work page 2021
[5]

Dice,L.R.:Measuresoftheamountofecologicassociationbetweenspecies.Ecology 26(3), 297–302 (1945)

1945
[6]

Cognition123(3), 392–403 (2012)

Fayol, M., Thevenot, C.: The use of procedural knowledge in simple addition and subtraction problems. Cognition123(3), 392–403 (2012)

work page 2012
[7]

arXiv preprint arXiv:2604.20329 (2026) Image-Space Rule Discovery 19

Gabeur, V., Long, S., Peng, S., et al.: Image generators are generalist vision learn- ers. arXiv preprint arXiv:2604.20329 (2026) Image-Space Rule Discovery 19

arXiv 2026
[8]

Ghosh, D., Hajishirzi, H., Schmidt, L.: GenEval: An object-focused framework for evaluating text-to-image alignment. In: Adv. Neural Inform. Process. Syst. (2023)

work page 2023

Show all 40 references

[9]

Google AI for Developers: Gemini 3 developer guide (2026), accessed July 2026

2026
[10]

Neuropsychologia47(2), 604–608 (2009)

Grabner, R.H., Ansari, D., Koschutnig, K., Reishofer, G., Ebner, F., Neuper, C.: To retrieve or to calculate? left angular gyrus mediates the retrieval of arithmetic facts during problem solving. Neuropsychologia47(2), 604–608 (2009)

2009
[11]

arXiv preprint arXiv:2511.01295 (2025)

Han, F., Wang, Y., Li, C., Liang, Z., Wang, D., Jiao, Y., Wei, Z., Gong, C., Jin, C., Chen, J., Wang, J.: UniREditBench: A unified reasoning-based image editing benchmark. arXiv preprint arXiv:2511.01295 (2025)

2025
[12]

Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-Prompt image editing with Cross-Attention control. In: Int. Conf. Learn. Represent. (2023)

2023
[13]

Ho, J., Jain, A.N., Abbeel, P.: Denoising diffusion probabilistic models. In: Adv. Neural Inform. Process. Syst. (2020)

2020
[14]

Hu, Y., Liu, B., Kasai, J., Wang, Y., Ostendorf, M., Krishna, R., Smith, N.A.: TIFA: Accurate and interpretable text-to-image faithfulness evaluation with ques- tion answering. In: Int. Conf. Comput. Vis. (2023)

2023
[15]

IEEE Trans

Huang, K., Duan, C., Sun, K., Xie, E., Li, Z., Liu, X.: T2I-CompBench++: An en- hanced and comprehensive benchmark for compositional text-to-image generation. IEEE Trans. Pattern Anal. Mach. Intell.47(5), 3563–3579 (2025)

2025
[16]

In: IEEE Conf

Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C.L., Girshick, R.:CLEVR:Adiagnosticdatasetforcompositionallanguageandelementaryvisual reasoning. In: IEEE Conf. Comput. Vis. Pattern Recog. (2017)

2017
[17]

arXiv preprint arXiv:2510.11026 (2025)

Li, H., Li, Y., Lin, B., Niu, Y., Yang, Y., Huang, X., Cai, J., Jiang, X., Hu, Y., Chen, L.: GIR-Bench: Versatile benchmark for generating images with reasoning. arXiv preprint arXiv:2510.11026 (2025)

2025
[18]

Li, J., Li, D., Savarese, S., Hoi, S.C.H.: BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In: Int. Conf. Mach. Learn. (2023)

2023
[19]

Liu,H.,Li,C.,Wu,Q.,Lee,Y.J.:Visualinstructiontuning.In:Adv.NeuralInform. Process. Syst. (2023)

2023
[20]

Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.Y., Ermon, S.: SDEdit: Guided image synthesis and editing with stochastic differential equations. In: Int. Conf. Learn. Represent. (2022)

2022
[21]

arXiv preprint arXiv:2607.18787 (2026)

Mizrahi, U.: Image editing models are numerical solvers. arXiv preprint arXiv:2607.18787 (2026)

2026 arXiv
[22]

In: IEEE Conf

Mokady, R., Hertz, A., Aberman, K., Pritch, Y., Cohen-Or, D.: Null-text inversion for editing real images using guided diffusion models. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

2023
[23]

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visualmodelsfromnaturallanguagesupervision.In:Int.Conf.Mach.Learn.(2021)

2021
[24]

In: Asian Conf

Rahmanzadehgervi, P., Bolton, L., Taesiri, M.R., Nguyen, A.T.: Vision language models are blind. In: Asian Conf. Comput. Vis. (2024)

2024
[25]

In: IEEE Conf

Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: IEEE Conf. Comput. Vis. Pattern Recog. (2022)

2022
[26]

arXiv preprint arXiv:2410.22029 (2024) 20 M

Singh, A., Gupta, M., Garg, S.: Are VLMs really blind. arXiv preprint arXiv:2410.22029 (2024) 20 M. Sugiyama et al

2024 arXiv
[27]

Sohl-Dickstein, J., Weiss, E.A., Maheswaranathan, N., Ganguli, S.: Deep unsuper- vised learning using nonequilibrium thermodynamics. In: Int. Conf. Mach. Learn. (2015)

2015
[28]

Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. In: Int. Conf. Learn. Represent. (2021)

2021
[29]

Vo, A., Nguyen, K.N., Taesiri, M.R., Dang, V.T., Nguyen, A.T., Kim, D.: Vision language models are biased. In: Int. Conf. Learn. Represent. (2026)

2026
[30]

arXiv preprint arXiv:2409.18869 (2024)

Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y., Wang, J., Zhang, F., Wang, Y., Li, Z., Yu, Q., Zhao, Y., Ao, Y., Min, X., Li, T., Wu, B., Zhao, B., Zhang, B., Wang, L., Liu, G., He, Z., Yang, X., Liu, J., Lin, Y., Huang, T., Wang, Z.: Emu3: Next-token prediction is all you nee...

2024 arXiv
[31]

arXiv preprint arXiv:2607.21072 (2026)

Wang, X., Yao, K., Pan, M., Zhou, X., Liu, X., Zhang, W., Zhang, X.: Show, don’t tell: Evaluating spatial cognition in generative pixels rather than LLM text. arXiv preprint arXiv:2607.21072 (2026)

2026 arXiv
[32]

Quarterly Journal of Experimental Psychol- ogy20(3), 273–281 (1968).https://doi.org/10.1080/14640746808400161

Wason, P.C.: Reasoning about a rule. Quarterly Journal of Experimental Psychol- ogy20(3), 273–281 (1968).https://doi.org/10.1080/14640746808400161

1968 doi
[33]

arXiv preprint arXiv:2509.20328 (2025)

Wiedemer, T., Li, Y., Vicol, P., Gu, S.S., Matarese, N., Swersky, K., Kim, B., Jaini, P., Geirhos, R.: Video models are zero-shot learners and reasoners. arXiv preprint arXiv:2509.20328 (2025)

2025 arXiv
[34]

arXiv preprint arXiv:2508.02324 (2025)

Wu, C., Li, J., Zhou, J., Lin, J., Gao, K., Yan, K., et al.: Qwen-image technical report. arXiv preprint arXiv:2508.02324 (2025)

2025 arXiv
[35]

arXiv preprint arXiv:2505.16707 (2025)

Wu, Y., Li, Z., Hu, X., Ye, X., Zeng, X., Yu, G., Zhu, W., Schiele, B., Yang, M.H., Yang, X.: KRIS-Bench: Benchmarking next-level intelligent image editing models. arXiv preprint arXiv:2505.16707 (2025)

2025 arXiv
[36]

In: IEEE Conf

Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., Chen, W.: MMMU: A massive multi- discipline multimodal understanding and ...

2024
[37]

In: IEEE Conf

Zhang, C., Gao, F., Jia, B., Zhu, Y., Zhu, S.C.: RAVEN: A dataset for relational and analogical visual reasoning. In: IEEE Conf. Comput. Vis. Pattern Recog. (2019)

2019
[38]

Zhang, K., Mo, L., Chen, W., Sun, H., Su, Y.: MagicBrush: A manually annotated dataset for instruction-guided image editing. In: Adv. Neural Inform. Process. Syst. (2023)

2023
[39]

arXiv preprint arXiv:2504.02826 (2025)

Zhao, X., Zhang, P., Tang, K., Li, H., Zhang, Z., Zhai, G., Yan, J., Yang, H., Yang, X., Duan, H.: Envisioning beyond the pixels: Benchmarking reasoning-informed visual editing. arXiv preprint arXiv:2504.02826 (2025)

2025 arXiv
[40]

arXiv preprint arXiv:2512.15110 (2025)

Zuo, J., Deng, H., Zhou, H., Zhu, J., Zhang, Y., Zhang, Y., Yan, Y., Huang, K., Chen, W., Deng, Y., Jin, R., Sang, N., Gao, C.: Is Nano Banana Pro a low-level vision all-rounder? a comprehensive evaluation on 14 tasks and 40 datasets. arXiv preprint arXiv:2512.15110 (2025)

2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al.: Flamingo: A visual language model for few-shot learning. In: Adv. Neural Inform. Process. Syst. (2022)

work page 2022

[2] [2]

Black Forest Labs: FLUX.2 [klein]: Towards interactive visual intelligence (2026), accessed: 4 July 2026

work page 2026

[3] [3]

In: IEEE Conf

Brooks, T., Holynski, A., Efros, A.A.: InstructPix2Pix: Learning to follow image editing instructions. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

work page 2023

[4] [4]

Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. In: Adv. Neural Inform. Process. Syst. (2021)

work page 2021

[5] [5]

Dice,L.R.:Measuresoftheamountofecologicassociationbetweenspecies.Ecology 26(3), 297–302 (1945)

1945

[6] [6]

Cognition123(3), 392–403 (2012)

Fayol, M., Thevenot, C.: The use of procedural knowledge in simple addition and subtraction problems. Cognition123(3), 392–403 (2012)

work page 2012

[7] [7]

arXiv preprint arXiv:2604.20329 (2026) Image-Space Rule Discovery 19

Gabeur, V., Long, S., Peng, S., et al.: Image generators are generalist vision learn- ers. arXiv preprint arXiv:2604.20329 (2026) Image-Space Rule Discovery 19

arXiv 2026

[8] [8]

Ghosh, D., Hajishirzi, H., Schmidt, L.: GenEval: An object-focused framework for evaluating text-to-image alignment. In: Adv. Neural Inform. Process. Syst. (2023)

work page 2023