Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

Current multimodal AI models fall far short of human experts when interpreting materials characterization images, a new 1,500-question benchmark shows.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:19 UTC pith:ZUKE5XBZ

load-bearing objection MatCha is a genuinely useful new materials-characterization benchmark and the main finding—MLLMs trail human experts by a wide margin—probably holds, but the AI-generated half needs a label audit before the exact numbers are trusted. the 4 major comments →

arxiv 2509.09307 v1 pith:ZUKE5XBZ submitted 2025-09-11 cs.CV cs.AIcs.CLcs.MM

Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization

Zhengzhao Lai , Youbin Zheng , Zhenyang Cai , Haonan Lyu , Jinpu Yang , Hongqing Liang , Yan Hu , Benyou Wang This is my paper
classification cs.CV cs.AIcs.CLcs.MM
keywords multimodal large language modelsmaterials characterizationbenchmarkvisual question answeringelectron microscopydomain-specific evaluationmaterials scienceMLLM evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MatCha is the first benchmark that tests multimodal large language models on what materials scientists actually do when they look at characterization data: identify techniques, read microstructures, interpret spectra and diffraction patterns, and connect structure to properties. It contains 1,500 expert-level multiple-choice questions across 21 tasks and four research stages, drawn from real published figures and three human-annotated microscopy datasets. On the paper-generated half, the best model (GPT-4o) scores 62.58% against 88.87% for human experts; on the converted real-data half, the best model (LLaMA-4-Maverick) scores 57.71% against 88.93%. The paper argues this gap shows that current MLLMs lack the domain knowledge and fine-grained visual perception that real materials characterization requires, and that few-shot and chain-of-thought prompting do not reliably close it. The benchmark is intended as a diagnostic tool to guide the development of scientific AI and autonomous discovery agents.

Core claim

The paper's central claim is that state-of-the-art multimodal large language models, despite strong performance on natural images and some scientific benchmarks, cannot yet interpret materials characterization imagery at an expert level. On MatCha's two subsets—994 GPT-4o-generated questions reviewed by materials experts, and 506 questions converted from human-annotated electron microscopy datasets—the best models achieve only 62.58% and 57.71% accuracy, respectively, while human experts score 88.87% and 88.93%. The gap widens as tasks progress from basic characterization-technique identification to structure and property analysis, and error attribution shows that a lack of materials knowled

What carries the argument

The central object is MatCha, a closed-ended visual question answering benchmark. Its load-bearing structure is a four-stage task taxonomy—Processing Correlation, Morphology Analysis, Structure Analysis, Property Analysis—that mirrors the real workflow of materials scientists, so each of its 21 sub-tasks maps to a concrete step in that workflow. Questions are generated in two ways: GPT-4o generates multiple-choice questions from figure–caption–context triplets extracted from CC-BY Nature-Platform articles, followed by AI filtering and review by two materials-science PhD candidates; and three existing human-annotated microscopy datasets are converted into multiple-choice questions by template

Load-bearing premise

The headline model-human gap rests on two unverified premises: that GPT-4o-generated answers certified by two PhD candidates, with no reported inter-annotator agreement, are all correct and visually grounded, and that the human baseline scores of 88.87% and 88.93% come from a sufficiently large and representative panel.

What would settle it

Re-annotate the 994 generated questions with a third independent materials expert (or a panel), blinded to GPT-4o's answers and the original labels, and measure agreement; if a substantial fraction—say more than 10%—of original answers are judged wrong, ambiguous, or answerable only from article context, then MatCha's model accuracies and the 26-point human gap are mismeasured. Similarly, report the number of human test-takers and the variance of their scores; if the human panel is small, the 88.9% baseline is not a stable benchmark.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If MatCha's results are representative, no current MLLM is reliable enough for autonomous materials characterization: even the best model sits about 26–31 points below human experts.
  • Performance declines systematically from Processing Correlation through Morphology, Structure, and Property Analysis, meaning tasks that require deeper materials expertise and visual reasoning are precisely where models fail.
  • Few-shot and chain-of-thought prompting give inconsistent, often negative results, so improving prompt strategy alone will not close the gap; future gains must come from domain knowledge and perception itself.
  • Error analysis attributes 59–71% of model failures to lack of material knowledge, pointing to knowledge injection, such as retrieval augmentation, as a more promising direction than pure prompting.
  • Open-source models, while generally behind proprietary ones by about 10 points, occasionally outperform specific proprietary models on advanced stages, suggesting targeted training on scientific imagery can narrow the gap.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the paper's label reliability assumption fails—that is, if a meaningful share of the 994 GPT-4o-generated answers are wrong or answerable from article context rather than the image alone—the reported 26–31 point model-human gap would compress accordingly.
  • The human baseline of 88.87%/88.93% is presented without participant count or variance; a third-party replication with a larger, more diverse panel and reported dispersion would clarify how stable the human reference actually is.
  • Because the paper's no-image ablation shows some models can partly answer from text, a stricter variant of MatCha could add unanswerable or correspondence-based controls to separate genuine visual understanding from language priors.
  • A testable extension of the paper's error analysis is to fine-tune or retrieve materials corpus text before evaluation and measure whether accuracy on Structure and Property stages rises more than on Morphology; if so, knowledge injection is the primary lever rather than visual encoders.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces MatCha, a multiple-choice benchmark for materials characterization image understanding, containing 1,500 questions across 21 sub-tasks organized into four research stages: Processing Correlation, Morphology Analysis, Structure Analysis, and Property Analysis. The benchmark is built from 340 Nature-platform articles (generated VQA, 994 questions) and three externally labeled microscopy datasets (converted VQA, 506 questions). The authors evaluate 6 proprietary and 9 open-source MLLMs, report that the best models reach 62.58% (GPT-4o) on the generated subset and 57.71% (LLaMA-4-Maverick) on the converted subset, versus 88.87% and 88.93% for human experts, respectively. A no-image ablation shows substantial accuracy drops, and few-shot and chain-of-thought prompting do not close the gap. The paper concludes that current MLLMs lack expert-level materials characterization understanding.

Significance. If the benchmark labels and human baseline are validated, MatCha would be a useful contribution: the task taxonomy reflects a real scientific workflow, the coverage of characterization techniques and material types is broad, and the public release with a large model evaluation is valuable. The two-source construction (generated + converted) is a strength because the converted subset relies on independently human-annotated datasets rather than GPT-4o outputs. The no-image ablation is a sensible control for visual grounding. However, the headline quantitative claims depend on two load-bearing elements that are not yet adequately supported: the quality of the generated VQA labels and the statistical grounding of the human baseline. These are fixable with additional validation and reporting, but they currently prevent the paper from being accepted as is.

major comments (4)
  1. [§4.1, Table 1] The human expert scores (88.87% and 88.93%) are reported as point estimates with no participant count, per-participant variance, or confidence intervals. The central claim of a large human-model gap depends entirely on this baseline. Please report the number of doctoral researchers, how their expertise was verified, per-participant scores, and variance. If the sample is small, provide bootstrap confidence intervals and avoid strong claims of a fixed gap.
  2. [§3.3] Ground truth for the generated VQA subset is produced by GPT-4o from (subfigure, sub-caption, context) triplets and reviewed by only two Ph.D. candidates. No inter-annotator agreement, independent error audit, or evidence of blindness to GPT-4o's proposed answers is reported. The review criterion 'answerable solely through visual cues' is hard to enforce because the generator also saw the caption and article context; a question can pass review while still being answerable from text or in-figure annotations. The no-image ablation in §E shows models use images but cannot certify label correctness. Please report annotation instructions, agreement statistics, and an independent audit with external experts; quantify how many retained questions are answerable from the subcaption/context alone.
  3. [§G.2] The first error case in the appendix illustrates a possible label problem: for the FFT question, the stated correct answer is C ('three distinct lattice structures'), yet the in-image labels are '2H' and '1T' – Grain 1 / 1T' – Grain 2', which name two distinct lattice structures (with two grains of 1T'). The model's reasoning that the answer should be B is at least plausible. If the benchmark's own exemplar has an arguable answer, label noise may be non-negligible. Please audit and remove or revise such ambiguous items, and report the proportion of items with reviewer disagreement.
  4. [Table 1, §E] The Random Choice row for Generated VQA 'All' is 15.79%, which is inconsistent with the per-stage random baselines (19.61–26.64%) and with the option-count distribution in Fig. 2; the expected random accuracy for the generated subset is approximately 25.3%. This numerical error affects the interpretation of the 'challenging benchmark' claim. In addition, Table 9 appears to report only the no-image score and the drop, with the full 'MatCha' column omitted; the LLaMA-4-Maverick row implies a no-image accuracy of 5.93% against a random baseline of 24.73%, contradicting the text that says 'other models' are 'marginally above random guessing.' Please correct the random baselines and clarify Table 9.
minor comments (4)
  1. [Figures 2 and 3] The statistical text in Figure 2 and the sub-task proportions in Figure 3 are difficult to read because of small font sizes and dense layout. Please reformat for legibility and provide exact numeric tables in the appendix.
  2. [§4.5] The error analysis uses GPT-4o to classify 100 model errors into four categories. This is acceptable as an exploratory analysis, but should be reported as model-generated and, ideally, validated by human annotation on at least a subsample.
  3. [§3.2] Please provide more reproducibility details for the crawling and parsing pipeline: Exsclaim version and parameters, the regular-expression matching function, and the prompt used for GPT-4o sub-caption segmentation. Also state the random seeds used for question generation and filtering.
  4. [Tables 4–8] Several models output 0% at 8-shot and 16-shot because they fail to produce a valid option. This is reported, but it would be helpful to state explicitly that those zeros are non-answers rather than systematic wrong answers, and to show the proportion of valid outputs.

Circularity Check

0 steps flagged

No circularity: MatCha is an empirical benchmark; reported accuracies are measurements on a constructed test set, not quantities derived from fitted parameters or self-citations.

full rationale

MatCha is a benchmark-construction and evaluation paper, not a derivation chain. The headline claims—that MLLMs underperform human experts and that performance degrades across research stages—are empirical accuracy numbers measured on 1,500 multiple-choice items. There is no fitted parameter later renamed as a prediction, and no equation in which an output is identical to an input by construction. The generated-VQA ground truth is authored by GPT-4o and then reviewed by two PhD candidates (Section 3.3); the fact that GPT-4o is also among the evaluated models does not make its 62.58% score definitionally forced, because at test time the model must answer from the image alone without the caption/context used in generation, and it in fact errs on 37.42% of the items. The AI-filtering step (Section 3.3) removes questions that three open MLLMs answer correctly in all attempts; this is a difficulty-selection design choice, not a circular reduction, since no single evaluated model's output defines the surviving set and the reported accuracies remain empirical. The no-image ablation (Appendix E) provides independent evidence that visual content drives responses. There are no load-bearing self-citations; the cited prior work is external (e.g., Exsclaim, MMMU, MaCBench, external microscopy datasets). The reader-raised concerns—two unreported PhD reviewers, no inter-annotator agreement, no external label audit, and a human baseline without participant count or variance—are validity and measurement-quality risks, not circularity in the sense of a claim being equivalent to its inputs by definition. Accordingly, no circular step is identified.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The benchmark's validity rests on ground-truth quality, human baseline fairness, and model-independent difficulty. No physical free parameters are fitted. The main hand-chosen design choice is the AI filtering threshold, and the core assumptions all concern the trustworthiness of labels and human scores.

free parameters (1)
  • AI filter pass threshold = all three AI models correct on all three attempts
    Hand-chosen rule in Section 3.3 that removes easy generated questions; it shapes the benchmark's difficulty and the reported model-human gap, though it is not a numerical model parameter.
axioms (4)
  • domain assumption The 1,500 ground-truth labels are correct and answerable from the image alone with intrinsic domain knowledge.
    Only two PhD candidates reviewed the 994 GPT-4o-generated questions; no inter-annotator agreement or independent error audit is reported (Section 3.3). Label noise would directly bias the measured model accuracies.
  • domain assumption The human expert baseline is a fair and reliable upper bound.
    Human scores of 88.87% and 88.93% are reported without participant count, variance, or confidence intervals (Section 4.1, Table 1). The size and composition of the human group are unknown.
  • domain assumption GPT-4o-generated distractors are plausible and do not leak the correct answer through language priors.
    The no-image ablation suggests language priors alone are weak, but distractors were generated by the same model family that is later evaluated (Section 3.3). No systematic leakage test is reported.
  • ad hoc to paper Filtering out questions that three smaller MLLMs answer correctly is a valid way to define expert-level difficulty.
    Section 3.3 retains only questions that Qwen2.5-VL, InternVL3, and LLaMA-3.2-Vision all failed, making the generated subset hard by construction for models trained with similar multimodal objectives.

pith-pipeline@v1.3.0-alltime-deepseek · 24128 in / 12519 out tokens · 140829 ms · 2026-08-04T19:19:05.604063+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization." pith.science (2026). https://pith.science/paper/ZUKE5XBZ

@misc{pith2026250909307,
  author       = {Pith},
  title        = {Pith review of: Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZUKE5XBZ}},
  note         = {Machine review of arXiv:2509.09307}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Materials characterization is fundamental to acquiring materials information, revealing the processing-microstructure-property relationships that guide material design and optimization. While multimodal large language models (MLLMs) have recently shown promise in generative and predictive tasks within materials science, their capacity to understand real-world characterization imaging data remains underexplored. To bridge this gap, we present MatCha, the first benchmark for materials characterization image understanding, comprising 1,500 questions that demand expert-level domain expertise. MatCha encompasses four key stages of materials research comprising 21 distinct tasks, each designed to reflect authentic challenges faced by materials scientists. Our evaluation of state-of-the-art MLLMs on MatCha reveals a significant performance gap compared to human experts. These models exhibit degradation when addressing questions requiring higher-level expertise and sophisticated visual perception. Simple few-shot and chain-of-thought prompting struggle to alleviate these limitations. These findings highlight that existing MLLMs still exhibit limited adaptability to real-world materials characterization scenarios. We hope MatCha will facilitate future research in areas such as new material discovery and autonomous scientific agents. MatCha is available at https://github.com/FreedomIntelligence/MatCha.

Figures

Figures reproduced from arXiv: 2509.09307 by Benyou Wang, Haonan Lyu, Hongqing Liang, Jinpu Yang, Yan Hu, Youbin Zheng, Zhengzhao Lai, Zhenyang Cai.

Figure 1
Figure 1. Figure 1: MatCha construction pipeline. First, experts define scientifically meaningful, practical tasks and extract key [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Statistical overview of MatCha samples. Superscripts indicate the number of multiple￾choice options for those questions. Multiphase Interface Assessment Defect Type Classification Grain/Pore Size Classification Material Classification Image Content Analysis Supplementary Defect Type Classification Supplementary Surface Microstructure Assessment Supplementary Image Content Analysis (b) Morphology Analysis S… view at source ↗
Figure 4
Figure 4. Figure 4: Performance analysis. Left: Confusion matrices of several models on the Suppl. DTC task. Right: [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PolyReal: A Benchmark for Real-World Polymer Science Workflows

    cs.CV 2026-04 unverdicted novelty 7.0

    PolyReal benchmark shows leading MLLMs perform well on polymer knowledge reasoning but drop sharply on practical tasks like lab safety analysis and raw data extraction.

  2. TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards

    cs.CV 2026-05 unverdicted novelty 6.0

    TextAlign applies hierarchical VLM-based rewards to align text-to-image models for better glyph-level text rendering via GRPO and DPO.

  3. TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards

    cs.CV 2026-05 unverdicted novelty 6.0

    TextAlign uses a hierarchical VLM reward for preference alignment to boost text accuracy in generative models like FLUX.1-dev.

Reference graph

Works this paper leans on

16 extracted references · 8 linked inside Pith · cited by 2 Pith papers

  1. [1]

    These correspond to different lattice structures or orientations

    Observation of the FFT Pattern: The inset FFT pattern shows three distinct sets of diffraction spots, each marked with different colored dashed lines (blue, orange, and green). These correspond to different lattice structures or orientations

  2. [2]

    2H," "1T’ – Grain 1,

    Identification of Lattice Structures in the Image: - The main image shows three regions labeled as "2H," "1T’ – Grain 1," and "1T’ – Grain 2." - The "2H" region corresponds to one lattice structure. - The "1T’ – Grain 1" and "1T’ – Grain 2" regions correspond to the same lattice structure (1T’) but with different grain orientations

  3. [3]

    2H" lattice structure. 19 - The

    Conclusion: From the FFT pattern, we can identify two distinct lattice structures: - The "2H" lattice structure. 19 - The "1T’" lattice structure (with two grains but the same structure). Thus, the correct answer is: (B) Two Error Analysis: The model incorrectly concluded that there are only two distinct lattice structures based on the FFT pattern and the...

  4. [4]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh

    Matexpert: Decomposing materials discov- ery by mimicking human experts.arXiv preprint arXiv:2410.21317. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. InProceedings of the IEEE conference on computer vision and pattern re...

    Pith/arXiv arXiv 2017

  5. [8]

    Prateek Verma, Minh-Hao Van, and Xintao Wu

    Accelerated scheme to predict ring-opening polymerization enthalpy: simulation-experimental data fusion and multitask machine learning.The Journal of Physical Chemistry A, 127(50):10709– 10716. Prateek Verma, Minh-Hao Van, and Xintao Wu. 2024. Beyond human vision: The role of large vision lan- guage models in microscope image analysis. In2024 IEEE Interna...

    Pith/arXiv arXiv 2024

  6. [9]

    Tong Xie, Yuwei Wan, Yixuan Liu, Yuchen Zeng, Shaozhou Wang, Wenjie Zhang, Clara Grazian, Chunyu Kit, Wanli Ouyang, Dongzhan Zhou, and 1 others

    Darwin series: Domain specific large lan- guage models for natural science.arXiv preprint arXiv:2308.13565. Tong Xie, Yuwei Wan, Yixuan Liu, Yuchen Zeng, Shaozhou Wang, Wenjie Zhang, Clara Grazian, Chunyu Kit, Wanli Ouyang, Dongzhan Zhou, and 1 others. 2024. Darwin 1.5: Large language models as materials science adapted learners.arXiv preprint arXiv:2412....

    Pith/arXiv arXiv 2024

  7. [14]

    - These structures resemble lamellae, which are characteristic of a lamellar microstructure

    Observation of the Image: - The image shows a clear pattern of elongated, parallel structures within larger grains. - These structures resemble lamellae, which are characteristic of a lamellar microstructure

  8. [15]

    - Lamellar: The image shows α lamellae within the grains, matching the description of a lamellar microstructure

    Comparison with Descriptions: - Acicular: Acicular morphology typically consists of needle-like structures, which are not observed in this image. - Lamellar: The image shows α lamellae within the grains, matching the description of a lamellar microstructure. - Duplex: A duplex microstructure would show a combination of equiaxed grains and α lamellae. Howe...

  9. [16]

    vqas": [

    Conclusion: - The microstructure in the SEM image corresponds to (B) lamellar:αlamellae. Error Analysis: The model incorrectly identified the microstructure as lamellar (B) instead of duplex (C). The SEM image clearly shows a bimodal morphology consisting of equiaxed grains and α-lamellae, which is characteristic of a duplex microstructure. The model fail...

  10. [2017]

    Brian L DeCost and Elizabeth A Holm

    Exploring the microstructure manifold: image texture representations applied to ultrahigh carbon steel microstructures.Acta Materialia, 133:30–40. Brian L DeCost and Elizabeth A Holm. 2015. A com- puter vision approach for automated analysis and classification of microstructural image data.Compu- tational materials science, 110:126–133. Brian L DeCost, Bo...

    2015

  11. [2019]

    Jonathan Roberts, Kai Han, Neil Houlsby, and Samuel Albanie

    Deep learning for semantic segmentation of defects in advanced stem images of steels.Scientific reports, 9(1):12744. Jonathan Roberts, Kai Han, Neil Houlsby, and Samuel Albanie. 2024. Scifibench: Benchmarking large mul- timodal models for scientific figure interpretation. Advances in Neural Information Processing Systems, 37:18695–18728. Ian M Robertson, ...

    Pith/arXiv arXiv 2024

  12. [2020]

    Let’s think step by step

    Crystal symmetry classification from pow- der x-ray diffraction patterns using a convolutional neural network.Materials Today Communications, 25:101662. Huan Zhang, Yu Song, Ziyu Hou, Santiago Miret, and Bang Liu. 2024. Honeycomb: A flexible llm-based agent system for materials science.arXiv preprint arXiv:2409.00135. Yaowei Zheng, Richong Zhang, Junhao Z...

    Pith/arXiv arXiv 2024

  13. [2021]

    Mingren Shen, Guanzhao Li, Dongxia Wu, Yuhan Liu, Jacob RC Greaves, Wei Hao, Nathaniel J Krakauer, Leah Krudy, Jacob Perez, Varun Sreenivasan, and 1 others

    Exsclaim!–an automated pipeline for the con- struction of labeled materials imaging datasets from literature.arXiv preprint arXiv:2103.10631. Mingren Shen, Guanzhao Li, Dongxia Wu, Yuhan Liu, Jacob RC Greaves, Wei Hao, Nathaniel J Krakauer, Leah Krudy, Jacob Perez, Varun Sreenivasan, and 1 others. 2021. Multi defect detection and analysis of electron micr...

    Pith/arXiv arXiv 2021

  14. [2023]

    InInternational conference on ma- chine learning, pages 19730–19742

    Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large lan- guage models. InInternational conference on ma- chine learning, pages 19730–19742. PMLR. Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. 2024a. Multi- modal arxiv: A dataset for improving scientific com- prehension of large visi...

    Pith/arXiv arXiv 2018

  15. [2024]

    InProceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

    Macbench: a multimodal chemistry and ma- terials science benchmark. InProceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. https://www-cdn.anthropic.com/ de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/ Model_Card_Claude_3.pdf. Luis M Antunes, Keith T Butler, ...

    2024

  16. [2025]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, and 1 others

    Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, and 1 others. 2024b. Expanding performance boundaries of open-source multimodal models with model, data, and test-ti...

    Pith/arXiv arXiv 2016