Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Towards Physics-Faithful Generation of Scientific Diagrams

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing flat captions with structured physics reasoning chains — not changing the architecture — yields diagrams whose forces, states, and equations are physically consistent.

desk verdict Real contribution is a structured representation and data pipeline, and the in-house benchmark is clever, but the paper hasn't isolated training supervision from inference-time prompt expansion. read the letter →

arxiv 2608.13112 v1 pith:6X3YPYNC submitted 2026-08-13 cs.CV

Minghui Zhang , Jinxin Shi , Yifan Chang , Liangliang Zhao , Yuandong Pu , Qian Yu , Ming Hu , Hanxiao Zhang
show 7 more authors
Yun Gu Yirong Chen Yu Qiao Bo Zhang Xiangchao Yan Bin Fu Yihao Liu
This is my paper · ORCID
classification cs.CV
keywords physics-faithfuldiagramgenerationstructuredchain-of-thoughttext-to-imagescientificdiagramsmultimodalsupervisionphysicalfaithfulnessevaluationSP-CoTVeriphyT2IBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-to-image models can draw a plausible pendulum but routinely mis-point its forces, because web captions describe appearance, not physics. This paper argues that the bottleneck is supervision, not architecture: it replaces flat captions with Structured Physical Chain-of-Thought (SP-CoT), a fixed five-step template that separates what is visibly drawn from what is physically inferred and types all mathematics symbolically. On 4.3 million physics images, 115,037 of them expert-verified, the authors train Princigram, a generator conditioned on these structured annotations during both training and inference. The result is a generator that beats generic open and closed models on physics-diagram faithfulness, including a roughly fourfold gain over its own backbone on the physics subset of a public text-to-image exam. The paper also introduces a benchmark, VeriphyT2IBench, whose yes/no questions are compiled from each diagram's own annotation, so scores name which physical facts are wrong rather than giving one opaque number.

What carries the argument

The load-bearing object is Structured Physical Chain-of-Thought (SP-CoT), a fixed JSON schema instantiated per subdiscipline. Each field carries a type (entity, relation, or value) and a grounding tag: visible fields may record only elements actually drawn in the image, while inferred fields may be filled by physical reasoning from those drawn elements, and missing information must be left empty rather than guessed; all mathematics is stored as valid LaTeX. The same serialized annotation serves three purposes: pre-training and fine-tuning supervision for the generator, a structured thinking prompt that a language model populates from a user request at inference, and the source of the binary yes/no checklist used by the evaluator, because the serializer preserves keys and values verbatim. That single representation makes data construction, training, and evaluation share one machine-parseable object.

What would settle it

Have two trained experts independently score a random sample of generated images against the same binary checklists used in VeriphyT2IBench, then compare their answers with the vision-language judge's; if judge–expert agreement is much smaller than the roughly 29-point margin Princigram shows over its untrained backbone, the measured faithfulness gain is largely a property of the judge rather than of the diagrams.

Watch

Extended reading notes

Core claim

The central claim is that replacing flat image–caption pairs with SP-CoT — a per-subdiscipline, five-step reasoning chain covering scenario, parameters, structure, laws, and synthesis, with a strict visible/inferred split and symbolic LaTeX — yields supervision that is dense, auditable, and aligned across six physics subdisciplines, and that a generator trained on it produces diagrams more consistent with the underlying physics than generic baselines. The evidence is the measured jump in faithfulness: on the in-house benchmark, the same unified multimodal backbone rises from 46.38 to 75.69 on local per-attribute accuracy and from 62.15 to 82.54 on global accuracy; on the physics subset of a public text-to-image exam, its relaxed score rises from 13.8 to 54.8, about fourfold. The gains concentrate on the middle reasoning steps (parameters, structure, laws), widen as prompts get harder, and appear on a second, architecturally different backbone, which the paper takes as evidence that the structured supervision, not the network, is the active ingredient. The paper also states plainly that fully faithful generation is unsolved: under zero-tolerance strict scoring, every model, including Princigram, scores at or near zero on almost every subject.

Load-bearing premise

The reported gains rest on a vision-language judge correctly answering the yes/no physics questions and on expert-annotated gold answers whose inter-annotator consistency is not measured, so systematic judge or annotation error could inflate the score gaps.

Editorial extensions

If this is right

  • On the physics subset of a public text-to-image exam, structured supervision lifts the relaxed score about fourfold (13.8 to 54.8) on the paper's primary backbone, bringing an open-weight model ahead of closed reference systems on several subjects.
  • On the in-house benchmark, the gain transfers across two mechanistically different unified backbones, with local-score improvements of roughly 29 and 35 points, which suggests the supervision rather than the architecture is doing the work.
  • The advantage is largest on the schema steps that require physical inference, such as Laws and Synthesis, and on hard prompts with many simultaneous constraints, exactly where appearance-based baselines collapse.
  • No current system, including Princigram, reliably produces a diagram in which every checked physical attribute is correct: zero-tolerance strict scores are near zero everywhere, so fully faithful generation remains open.
  • Because the checklist is compiled from each diagram's annotation, a low score names the specific physical facts a model gets wrong, such as a missing $mg\sin\theta$ component, rather than giving one holistic number.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The visible/inferred tags are currently unused in scoring; a natural next step, left implicit by the paper, is to weight grounded picture checks more heavily than inferred reasoning checks, or to use the per-fact scores as a reward for preference optimization.
  • The same five-step structure should transplant to other quantitative sciences: chemistry, geometric optics, and biology diagrams have their own visually carried relations, so per-discipline schemas could let this style of supervision generalize without new architecture.
  • Because each benchmark question is tied to a named physical fact, the same machinery could be repurposed as automatic item-specific feedback for students or as a filter that rejects physically wrong generated figures before publication; the paper does not pursue these uses.
  • The evaluation rests on a single vision-language judge; a checkable extension is to measure judge–judge and judge–expert agreement on a sample, since cross-judge consistency is not reported.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Princigram, a physics-faithful scientific-diagram generator built around Structured Physical Chain-of-Thought (SP-CoT), a per-subdiscipline JSON schema that decomposes a physics diagram into five reasoning steps (scenario, parameters, structure, laws, synthesis) with a strict visible/inferred grounding split. The authors curate a corpus of 4.3 million physics images, of which 115,037 receive expert-level structured annotations, and train two unified multimodal backbones (BAGEL and DiMOO) on these annotations. At inference, a language model expands a raw user prompt into the SP-CoT schema before generation. They evaluate on the physics subset of GenExam and on a new benchmark, VeriphyT2IBench, whose binary yes/no questions are derived from each held-out diagram's own structured annotation. The central claim is that structured physics supervision, rather than architecture or scale, is the key lever for physically faithful diagram generation.

Significance. If the central claim holds, the paper makes a substantial contribution: a reusable, machine-parseable annotation framework for physics diagrams, a large corpus to support it, and evidence that dense structured supervision can improve physical faithfulness across two mechanistically different backbones. The formal specification in S1.3 (typed fields, grounding map, fidelity rule, serialization, and the shared training/evaluation objective) is a genuine strength and provides a clear substrate for future work. The external GenExam evaluation and the two-backbone comparison are also valuable design choices that partially mitigate concerns about benchmark circularity. However, because the main quantitative comparisons are confounded by inference-time prompting, the magnitude of the claimed supervision effect is not yet established; the qualitative same-training ablation supports the direction but does not quantify it.

major comments (3)
  1. [S1.7 / Tables 3-4] The comparison between Princigram and the baselines in Tables 3 and 4 varies two factors simultaneously: training supervision and inference-time structured prompting. Section S1.7 states that "for any baseline run in structured mode" the original prompt is expanded into the subdiscipline schema, but no structured-mode baseline scores are reported anywhere in the paper. The raw BAGEL and DiMOO baselines are prompted with the unexpanded prompt, while Princigram uses the SP-CoT expansion of Eq. (13) at inference. Consequently, the reported gains (BAGEL from 46.38 to 75.69 Local on VeriphyT2IBench and from 13.8 to 54.8 relaxed on GenExam) may be inflated by the inference-time structured prompt alone. The central claim in Sections 1 and 3 that "closing the supervision gap is the key lever" requires that an untrained backbone prompted with the same SP-CoT expansion does not already reach Princigram-level scores. Please report structured-mode baseline results (e.g., BAGEL and DiMOO with the SP-CoT prompt expansion but without SP-CoT training) or provide a quantitative version of the Section S3.3 ablation on the full benchmarks.
  2. [2.5 / S1.8] VeriphyT2IBench is constructed from the same structured annotations used to train Princigram: the questions and gold answers are compiled from the SP-CoT fields (Section S1.8). The risk is that the benchmark measures adherence to the training schema rather than independent physical correctness, since the model has been trained to produce images consistent with that schema. The external GenExam results mitigate this concern, but the paper's largest and most detailed claims (Tables 4-6, Figure 3) rest on VeriphyT2IBench. To rule out that the scores are inflated by schema familiarity, the authors should report performance of structured-mode baselines on VeriphyT2IBench or add a human validation subset in which expert raters judge physical correctness and the judge's answers are compared against human judgments. In addition, the inter-annotator consistency of the expert gold annotations is not reported (Section S1.1); since every benchmark gold answer depends on these annotations, a consistency measure (e.g., Cohen's kappa on a sample) would strengthen the reliability of the benchmark.
  3. [S3.3 / Figure 14] The only same-training ablation (Figure 14) is qualitative and limited to three mechanics prompts. It supports the direction of the claim but does not quantify the effect of SP-CoT supervision on the benchmarks reported in the main tables. This is not a flaw by itself, but it underscores that the quantitative evidence for the "supervision, not architecture" claim currently comes from the confounded comparison in Tables 3 and 4. A quantitative version of this ablation (with and without SP-CoT, same pipeline) on VeriphyT2IBench and GenExam would directly test the central claim.
minor comments (5)
  1. [Abstract / Figure 2a / Table 2] The abstract says "4.3 million physics images," Figure 2a says "4.2M," and Table 2 totals 4,313,866; please harmonize the numbers.
  2. [S1.3 / Section 2.3] Section 2.3 states that "the same model that reasons over the physics also renders the diagram," but inference (S1.3, Eq. 13) uses a separate prompt-expansion model Φ to populate the schema; please clarify whether Φ is the same backbone or a distinct model.
  3. [Table 6 / Eq. (16)] The strict scoring tolerance is defined as a fraction of wrong answers per item, but the column labels "Tol. 0%, 5%, 10%" could be misread as a required accuracy; consider renaming the columns to indicate the maximum allowed error fraction.
  4. [Figure 3] The per-step faithfulness scores in Figure 3 are presented without error bars or sample sizes; reporting confidence intervals or the number of items per step would help assess the significance of the 26-42 point improvements claimed in the text.
  5. [Section 3.3] The limitation that corpus-level annotations are machine-generated and unverified is correctly acknowledged in Section 3.3; it would be helpful to mention this caveat in the abstract or introduction where the 4.3M corpus is first described.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the in-house VeriphyT2IBench compiles its gold answers from the same SP-CoT annotations used to train Princigram, though the external GenExam physics benchmark provides independent support.

  1. self definitional [Section S1.3 (Equations 8-16) and S1.9; Section 2.5 (VeriphyT2IBench construction)]
    "We give a single formal object (the structured annotation, one instance of SP-CoT) and show that pre-training (Section S1.5), supervised fine-tuning (Section S1.6) and the VeriphyT2IBench verification protocol (Section S1.8) are three uses of that same object."

    The paper's own equations make the identity explicit: the training loss in Eq. (10) conditions on the serialized annotation sigma(a), and Section S1.9 states that "each question's gold answer y_q is fixed by the corresponding field." Equations (14)-(16) then score the judge's answers against those gold answers, which are compiled from the very same structured annotation fields used as training supervision. A generator trained to produce diagrams from SP-CoT annotations is therefore advantaged by construction on a benchmark whose correctness is defined by SP-CoT fields. Consequently, the large Local/Global/Strict gains on VeriphyT2IBench partly measure adherence to the training schema rather than independent physical correctness.

full rationale

The central derivation of SP-CoT as a representation is not itself circular: the schema, fidelity rule (Eq. 7), serializer (Eq. 9), and training objectives (Eqs. 10-12) are self-contained definitions, and the reported GenExam improvements are measured on an externally authored benchmark whose scoring points are not derived from the authors' annotations. The main circular concern is the in-house VeriphyT2IBench: its gold answers are generated from the same structured annotations that define the training signal, making the benchmark a partial self-evaluation of schema adherence. The paper even states that pre-training, fine-tuning, and VeriphyT2IBench verification are "three uses of that same object," which is an explicit reduction of the evaluation to the training representation. This is mitigated by the independent GenExam results and by the held-out nature of the benchmark images, so the overall circularity score is moderate rather than high. Self-citation overlaps (e.g., Lumina-DiMOO and GenExam authors) are not load-bearing here, since the cited backbones are used as baselines and GenExam is an external benchmark. The inference-time prompt-expansion confound between raw-prompt baselines and structured-prompt Princigram is a real attribution risk but is a methodological confound, not a by-construction circularity.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the coverage of the SP-CoT schema, the reliability of the VLM judge, and the correctness of expert annotations; no new physical entities are introduced.

free parameters (3)
  • per-subdiscipline mixture weights w_s
    Hand-chosen weights in the training objective (Eq. 10) rebalance the heavily imbalanced corpus; values not reported, and they affect the exact training mixture.
  • fraction of generic image-text data retained during pre-training
    Retained to preserve general rendering ability; the fraction is stated qualitatively as a portion but not quantified (S1.5).
  • strict scoring tolerances tau = 0%, 5%, 10%
    Evaluation thresholds chosen by the authors for strict scoring (Table 6); not fitted, but they determine the headline strict scores.
assumptions (6)
  • domain assumption The SP-CoT five-step schema adequately captures the physical content that makes a diagram correct or incorrect.
    This is the central representation choice; if a diagram's correctness depends on information outside the schema, the method and benchmark miss it. Introduced in Section 2.1.
  • domain assumption GPT-4o answers the binary physics questions about an image at least as accurately as a human expert would.
    All benchmark scores in Tables 4-6 are computed from this judge's answers, with no validation against human ratings. Stated as a proxy in Section 3.3.
  • domain assumption The expert-verified annotations are correct and consistent across annotators.
    No inter-annotator agreement is reported; gold answers for VeriphyT2IBench come directly from these annotations (Section 2.5, S1.8).
  • domain assumption The corpus-level annotations produced by Qwen3-VL are physically reasonable enough for pre-training.
    Acknowledged as unverified in Section 3.3; if systematic physics errors exist at scale, they could be baked into the pretrained model.
  • domain assumption The subdiscipline classifier correctly routes images and discards non-physics content.
    Routing is a pipeline step (S1.1) with no reported accuracy; misrouted images would corrupt annotations.
  • standard math Standard machine learning and probability background is sound.
    Training losses (Eq. 11, 12) are standard rectified-flow and masked-diffusion objectives; no new mathematics is claimed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Physics-Faithful Generation of Scientific Diagrams." pith.science (2026). https://pith.science/paper/6X3YPYNC

@misc{pith2026260813112,
  author       = {Pith},
  title        = {Pith review of: Towards Physics-Faithful Generation of Scientific Diagrams},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6X3YPYNC}},
  note         = {Machine review of arXiv:2608.13112}
}
read the original abstract

Text-to-image generation has reached photorealistic quality, yet state-of-the-art systems remain unreliable at producing scientific diagrams, whose value depends not on appearance but on physical faithfulness: correct force directions, valid coordinate systems, consistent thermodynamic states, and equations matching the depicted scenario. Trained on web imagery with physically shallow captions, generic models produce diagrams that look plausible but are physically wrong, harmful in education and scientific communication. We present Princigram, a physics-faithful scientific-diagram generator, and its data pipeline. Our central advance is Structured Physical Chain-of-Thought (SP-CoT): a per-subdiscipline schema that decomposes a physics diagram into an explicit multi-step reasoning chain across six subdisciplines, from scene identification through force or process analysis to governing laws and synthesis. Unlike free-form chain-of-thought, SP-CoT follows a fixed schema with strict fidelity rules that separate visually grounded facts from physically inferred reasoning and type all mathematics symbolically; it serves both as dense training supervision and, at inference, as a structured "thinking" prompt. With it we curate and structurally annotate 4.3 million physics images, of which 115,037 carry expert-level annotation, and adapt a unified multimodal backbone. We further introduce VeriphyT2IBench, whose questions are derived from each held-out diagram's own structured annotation: each diagram becomes an item-specific bank of binary questions about its objects, forces, and states, so a judge model's score decomposes into named physical facts rather than one holistic number. On the physics subset of GenExam and on VeriphyT2IBench, Princigram shows that explicit physics-structured supervision improves the physical faithfulness of generated scientific diagrams.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 35 canonical work pages

  1. [1]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. “Zero-shot text-to-image generation”. In:International conference on machine learning. Pmlr. 2021, pp. 8821–8831

    work page 2021

  2. [2]

    Denoising Diffusion Probabilistic Models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. “Denoising Diffusion Probabilistic Models”. In: Advances in Neural Information Processing Systems(2020)

    work page 2020

  3. [3]

    Score-Based Generative Modeling through Stochastic Differential Equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. “Score-Based Generative Modeling through Stochastic Differential Equations”. In: International Conference on Learning Representations. 2021

    work page 2021

  4. [4]

    GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. “GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models”. In:International Conference on Machine Learning. 2022

    work page 2022

  5. [5]

    High- Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. “High- Resolution Image Synthesis with Latent Diffusion Models”. In:IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. 2022

    work page 2022

  6. [6]

    Scalable Diffusion Models with Transformers

    William Peebles and Saining Xie. “Scalable Diffusion Models with Transformers”. In:IEEE/CVF International Conference on Computer Vision. 2023

    work page 2023

  7. [7]

    Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. “Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding”. In:Advances in Neural Information Proce...

    work page 2022

  8. [8]

    Hierarchical Text-Conditional Image Generation with CLIP Latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. “Hierarchical Text-Conditional Image Generation with CLIP Latents”. In:arXiv preprint arXiv:2204.06125 (2022)

    arXiv 2022

Show all 51 references
  1. [9]

    SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis”. In:International Conference on Learning Representations. 2024

    2024

  2. [10]

    PixArt-𝛼: Fast Training of Diffusion TransformerforPhotorealisticText-to-ImageSynthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. “PixArt-𝛼: Fast Training of Diffusion TransformerforPhotorealisticText-to-ImageSynthesis”.In:InternationalConferenceonLearning Representations...

    2024

  3. [11]

    Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

    Lei Bai, Zongsheng Cao, Yang Chen, Zhiyao Cui, Shangheng Du, Yue Fan, Shiyang Feng, Zijie Guo, Haonan He, Liang He, et al. “Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent”. In:arXiv preprint arXiv:2606.30616(2026)

    2026 arXiv

  4. [12]

    Flow Matching for Generative Modeling

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. “Flow Matching for Generative Modeling”. In:International Conference on Learning Representations. 2023

    2023

  5. [13]

    Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. “Scaling Rectified Flow Trans...

    2024

  6. [14]

    Black Forest Labs.FLUX.1.https://blackforestlabs.ai. 2024

    2024

  7. [15]

    Qwen-Image Technical Report

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai...

    2025 arXiv

  8. [16]

    HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformer

    Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang,FengbinGao,PeihanXu,YimengWang,KaiYu,WenxuanChen,ZiweiFeng,ZijianGong, Jianzhuang Pan, Yi Peng, Rui Tian, Siyu Wang, Bo Zhao, Ting Yao, and Tao Mei. “HiDream-I1: A High-Efficient Ima...

    2025 arXiv

  9. [17]

    Chameleon: Mixed-Modal Early-Fusion Foundation Models

    Chameleon Team. “Chameleon: Mixed-Modal Early-Fusion Foundation Models”. In:arXiv preprint arXiv:2405.09818(2024)

    2024 arXiv

  10. [18]

    Emu3: Next-Token Prediction is All You Need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tie...

    2024 arXiv

  11. [19]

    Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. “Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model”. In:arXiv preprint arXiv:2408.11039 (2024)

    2024 arXiv

  12. [20]

    Show-o: One Sin- gle Transformer to Unify Multimodal Understanding and Generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. “Show-o: One Sin- gle Transformer to Unify Multimodal Understanding and Generation”. In:arXiv preprint arXiv:2408.12528(2024)

    2024 arXiv

  13. [21]

    Show-o2: Improved Native Unified Multimodal Models

    Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. “Show-o2: Improved Native Unified Multimodal Models”. In:arXiv preprint arXiv:2506.15564(2025)

    2025 arXiv

  14. [22]

    Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understanding

    Yi Xin, Qi Qin, Siqi Luo, Kaiwen Zhu, Juncheng Yan, Yan Tai, Jiayi Lei, Yuewen Cao, Keqi Wang, Yibin Wang, Jinbin Bai, Qian Yu, Dengyang Jiang, Yuandong Pu, Haoxing Chen, Le Zhuo, Junjun He, Gen Luo, Tianbin Li, Ming Hu, Jin Ye, Shenglong Ye, Bo Zhang, Chang Xu, Wenhai Wang, H...

    2025

  15. [23]

    Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. “Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling”. In:arXiv preprint arXiv:2501.17811(2025)

    2025 arXiv

  16. [24]

    Emerging Properties in Unified Multimodal Pretraining

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. “Emerging Properties in Unified Multimodal Pretraining”. In:arXiv preprint arXiv:2505.14683(2025)

    2025 arXiv

  17. [25]

    Learning Transferable Visual Models from Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, GirishSastry,AmandaAskell,PamelaMishkin,JackClark,GretchenKrueger,andIlyaSutskever. “Learning Transferable Visual Models from Natural Language Supervision”. In:International Conference on...

    2021

  18. [26]

    LAION-5B: An Open Large-Scale Dataset for Training Next Generation Image-Text Models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. “LAIO...

    2022

  19. [27]

    TextDiffuser: Diffusion Models as Text Painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. “TextDiffuser: Diffusion Models as Text Painters”. In:Advances in Neural Information Processing Systems (2023)

    2023

  20. [28]

    AnyText: Multilingual Visual Text Generation and Editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. “AnyText: Multilingual Visual Text Generation and Editing”. In:International Conference on Learning Representations. 2024

    2024

  21. [29]

    GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment”. In:Advances in Neural Information Processing Systems(2023)

    2023

  22. [30]

    T2I-CompBench: A Compre- hensive Benchmark for Open-world Compositional Text-to-image Generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. “T2I-CompBench: A Compre- hensive Benchmark for Open-world Compositional Text-to-image Generation”. In:Advances in Neural Information Processing Systems(2023)

    2023

  23. [31]

    Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. “Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models”. In:ACM SIGGRAPH. 2023

    2023

  24. [32]

    TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. “TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering”. In:IEEE/CVF International Conference on Computer Vision. 2023

    2023

  25. [33]

    ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yux- iao Dong. “ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation”. In:Advances in Neural Information Processing Systems(2023)

    2023

  26. [34]

    Science-t2i: Addressing scientific illusions in image synthesis

    Jialuo Li, Wenhao Chai, Xingyu Fu, Haiyang Xu, and Saining Xie. “Science-t2i: Addressing scientific illusions in image synthesis”. In:Proceedings of the Computer Vision and Pattern Recognition Conference. 2025, pp. 2734–2744

    2025

  27. [35]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. “Grounding dino: Marrying dino with grounded pre-training for open-set object detection”. In:European conference on computer vision. Springer. 2024, pp....

    2024

  28. [36]

    A Diagram is Worth a Dozen Images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. “A Diagram is Worth a Dozen Images”. In:European Conference on Computer Vision. 2016

    2016

  29. [37]

    SciCap: Generating Captions for Scientific Figures

    Ting-Yao Hsu, C. Lee Giles, and Ting-Hao K. Huang. “SciCap: Generating Captions for Scientific Figures”. In:Findings of the Association for Computational Linguistics: EMNLP. 2021

    2021

  30. [38]

    ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. “ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning”. In: Findings of the Association for Computational Linguistics: ACL. 2022

    2022

  31. [39]

    Mul- timodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models

    Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. “Mul- timodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models”. In:Annual Meeting of the Association for Computational Linguistics. 2024

    2024

  32. [40]

    MathVista: Evaluating Mathematical Rea- soning of Foundation Models in Visual Contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. “MathVista: Evaluating Mathematical Rea- soning of Foundation Models in Visual Contexts”. In:International Conference on Learning Represe...

    2024

  33. [41]

    Improving Image Generation with Better Captions

    JamesBetker,GabrielGoh,LiJing,TimBrooks,JianfengWang,LinjieLi,LongOuyang,Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. “Improving Image Generation with Better Captions”. In:OpenAI Technical Report(2023)

    2023

  34. [42]

    What If We Recaption Billions of Web Images with LLaMA-3?

    Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, Yuyin Zhou, and Cihang Xie. “What If We Recaption Billions of Web Images with LLaMA-3?” In:arXiv preprint arXiv:2406.08478(2024)

    2024 arXiv

  35. [43]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”. In:Advances in Neural Information Processing Systems(2022)

    2022

  36. [44]

    GenExam: A Multidisciplinary Text-to-Image Exam

    Zhaokai Wang, Penghao Yin, Xiangyu Zhao, Changyao Tian, Yu Qiao, Wenhai Wang, Jifeng Dai, and Gen Luo. “GenExam: A Multidisciplinary Text-to-Image Exam”. In:arXiv preprint arXiv:2509.14232(2025)

    2025 arXiv

  37. [45]

    OpenDataLab: Empow- ering General Artificial Intelligence with Open Datasets

    Conghui He, Wei Li, Zhenjiang Jin, Chao Xu, Bin Wang, and Dahua Lin. “OpenDataLab: Empow- ering General Artificial Intelligence with Open Datasets”. In:arXiv preprint arXiv:2407.13773 (2024)

    2024 arXiv

  38. [46]

    Seedream 4.0: Toward Next-generation Multimodal Image Generation

    Team Seedream. “Seedream 4.0: Toward Next-generation Multimodal Image Generation”. In: arXiv preprint arXiv:2509.20427(2025)

    2025 arXiv

  39. [47]

    BLIP3-o: A Family of Fully Open Unified Multimodal Models—Architecture, Training and Dataset

    Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, Le Xue, Caiming Xiong, and Ran Xu. “BLIP3-o: A Family of Fully Open Unified Multimodal Models—Architecture, Training and Dataset”. In:arXiv preprint ar...

    2025 arXiv

  40. [48]

    Google DeepMind.Gemini 2.5 Flash Image.https://ai.google.dev/gemini-api/docs/ models/gemini-2.5-flash-image. 2025

    2025

  41. [49]

    OpenAI.Addendum to GPT-4o System Card: 4o Image Generation.https://openai.com/ index/gpt-4o-image-generation-system-card-addendum/. 2025

    2025

  42. [50]

    does the image show𝑋?

    Qwen Team.Qwen3-VL.https://github.com/QwenLM/Qwen3-VL. 2025. 19 Supplementary Information S1. Methods This Methods section provides full procedural detail for the data pipeline, the annotation framework, the Princigram model, training, inference, and evaluation summarized in t...

    2025

  43. [51]

    Brw Tal”, “a

    and scattering states (E > 0). The vertical axis represents energy, while the horizontal axis shows different angular momentum states labeled as \ell = 0, 1, 2, 3, 4, 5. Bound states are shown for N = 1 and N = 2, while scattering states are shown for N = 3, 4, 5. Each energy ...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.