Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

This paper introduces MMOral, a large instruction dataset and benchmark for panoramic dental X-rays, and claims that a single epoch of fine-tuning lifts a 7-billion-parameter vision-language model's benchmark score by 24.73 percentage point

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:23 UTC pith:SCYOBG2R

load-bearing objection The dataset is a real contribution, but the headline 24.73% SFT gain is not credible because MMOral-Bench is drawn from the same image pool the model was trained on. the 3 major comments →

arxiv 2509.09254 v1 pith:SCYOBG2R submitted 2025-09-11 cs.CV cs.MM

Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis

Jing Hao , Yuxuan Fan , Yanpeng Sun , Kaixin Guo , Lizhuo Lin , Jinrong Yang , Qi Yong H. Ai , Lun M. Wong
show 2 more authors
Hao Tang Kuo Feng Hung
This is my paper
classification cs.CV cs.MM
keywords panoramic X-raydental AIvision-language modelsinstruction tuningbenchmarkvisual question answeringmedical report generationOralGPT
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MMOral is a new dataset and evaluation suite built for panoramic dental X-rays, a modality with dense anatomy and subtle disease cues that general medical benchmarks skip. The paper claims that a single epoch of supervised fine-tuning on MMOral's instruction data lifts a 7-billion-parameter vision-language model from 21.5% to 46.2% average accuracy on its own benchmark. It also reports that 64 current vision-language models, including the strongest proprietary ones, score at most 41.45%, so the field is far from reliable automated interpretation. If these numbers hold, MMOral provides both the reading curriculum and the measuring stick that dental AI has been missing.

Core claim

The central claim is that panoramic X-ray interpretation can be taught to a general vision-language model with one training epoch using automatically generated instruction data, and that the same data can expose how far current models are from clinical competence. MMOral contains 20,563 annotated images, 1.3 million instruction instances, 904k bounding boxes, and report, VQA, and chat subsets. MMOral-Bench offers 1,100 questions over 100 images across five diagnostic dimensions: condition of teeth, pathological findings, historical treatments, jawbone observations, and clinical summary and recommendations. On this benchmark the fine-tuned OralGPT improves from 21.46% to 46.19% average score,

What carries the argument

The pipeline starts with ten visual specialist models detecting 49 categories of anatomical structures in panoramic X-rays. A post-processing chain deduplicates overlapping detections with threshold filtering and non-maximum suppression, links findings to FDI tooth numbers through spatial relationship matching, and inserts clinical rules such as recommending extraction when a wisdom tooth lacks an opposing counterpart. From this structured output, template-based grounding captions and two-stage LLM-generated medical reports are built, and then GPT-style prompting produces closed-ended and open-ended question-answer pairs plus patient-radiologist dialogues. All instruction data is organized a

Load-bearing premise

The load-bearing premise is that MMOral-Bench measures generalization: Section 4.1 draws every benchmark case from the same MMOral pool used for fine-tuning in Section 5.3, and the supplementary notes the underlying labels have not been independently validated.

What would settle it

Pick 100 panoramic X-rays from a second imaging center that were never part of MMOral, have two dentists label the same five diagnostic dimensions, and run OralGPT zero-shot; if its score falls near the 21–25% baseline instead of the 46% reported, the large gain came from memorizing the benchmark pool.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • On MMOral-Bench, the strongest proprietary model reaches only 41.45% average accuracy, so current vision-language models are not yet reliable readers of panoramic X-rays.
  • A single epoch of supervised fine-tuning on MMOral instruction data lifts a 7-billion-parameter model from 21.46% to 46.19%, meaning large gains come from domain instruction data rather than architecture changes.
  • Open-ended questions are disproportionately hard: a large share of open-source models score below 25%, so models can recognize options but cannot generate accurate descriptions.
  • Models perform best on coarse jaw-level anatomy and worst on fine-grained tooth-level questions, locating the specific weakness in current multimodal encoders.
  • Medical-specific vision-language models do not outperform general-purpose ones on this benchmark, suggesting dental skill is not inherited from generic medical training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the benchmark is selected from the same MMOral pool used for fine-tuning, the 24.73% gain likely measures how well the model fits that pool; an external set from another scanner would be needed to know whether the skill generalizes.
  • The paper's supplementary section concedes that the underlying anatomical labels have not been independently validated; if those labels are noisy, both the instruction data and the benchmark inherit that noise.
  • The visual-specialist plus LLM pipeline could be reused for other dense-anatomy imaging such as periapical X-rays or CBCT, but the bottleneck is the quality of the specialist labels.
  • If the five-dimension diagnostic taxonomy proves robust, it could become a standard reporting structure for dental AI, letting models output per-tooth findings plus summary recommendations in a form clinicians can audit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MMOral, a large multimodal instruction dataset for panoramic X-ray analysis (20,563 images, 1.3M instruction instances across attribute extraction, report generation, VQA, and chat), together with MMOral-Bench, an evaluation suite of 100 images and 1,100 closed/open QA pairs spanning five dental diagnostic dimensions. The authors evaluate 64 LVLMs zero-shot, reporting that even GPT-4o reaches only 41.45% average accuracy, and propose OralGPT, a Qwen2.5-VL-7B model fine-tuned for one epoch on MMOral instruction data, claiming a 24.73% average improvement on MMOral-Bench. The main contributions are the dataset, the benchmark, the large-scale model evaluation, and the SFT result.

Significance. If MMOral-Bench were an independent, expertly validated test set, this would be a valuable contribution to dental AI: the instruction dataset is large, the evaluation covers 64 models with standard tools (VLMEvalKit), the report-generation pipeline includes human evaluation, and the LLM-as-judge for open-ended answers is accompanied by repeatability and dentist-agreement checks. The zero-shot leaderboard, in particular, is a useful resource irrespective of the SFT claims. However, the central SFT claim is not supported because the benchmark is drawn from the same image pool and instruction data used for training, and the anatomical ground truth is generated by automated specialist models without independent validation. These two issues undermine the paper's headline conclusion about the value of MMOral for improving dental LVLMs.

major comments (3)
  1. [§4.1, §5.3, Table 9] The headline result—one epoch of SFT on MMOral improves Qwen2.5-VL-7B by 24.73% (21.46→46.19, Table 9)—is not supported because MMOral-Bench is not disjoint from the MMOral training data. §4.1 states that all MMOral-Bench cases are 'manually chosen and checked from the MMOral,' and its 100 images are selected from Hoang Viet Do [19], one of the two image sources of MMOral (§2.1). The SFT in §5.3 trains on MMOral-Report, MMOral-VQA, and MMOral-Chat, which are generated from the same grounding captions and medical reports that underlie the benchmark QA pairs (§2.4). The model can therefore memorize the benchmark images and near-identical questions; the reported improvement measures train-set fit, not generalization to new panoramic X-rays. The zero-shot ranking of 64 LVLMs is unaffected by this overlap, but the paper's central claim about MMOral's effectiveness is not established. The auth
  2. [Appendix F, §2.1–§2.4, §4.1] The benchmark ground truth is not independently validated. Appendix F concedes that the accuracy of the ground-truth labels from the constituent public datasets 'has not been validated by independent third-party organizations,' and the visual specialist models are trained on those labels. MMOral-Bench QA pairs are then built from the resulting grounding captions and LLM-generated reports (§4.1), so a model's score partly reflects agreement with potentially noisy automated annotations. Human evaluation in §3.2(3) covers report quality, not the anatomical labels or benchmark answers; §4.1 does not specify that the 'manually chosen and checked' step involved dental experts or measured inter-rater agreement. The claim that GPT-4o's 41.45% represents a clinically meaningful performance ceiling requires an expert-reviewed sample of MMOral-Bench QA pairs with reported agreement statistics.
  3. [§5.2, Finding 1] The paper's Finding 1 states that MMOral-Bench 'poses significant challenges to ALL LVLMs' based on the 41.45% GPT-4o score. Given the ground-truth derivation issues above, the absolute difficulty of the benchmark is unclear: low scores could indicate either genuine model weakness or annotation noise/ambiguity in the benchmark itself. The authors should quantify label quality, e.g., by having two or more dentists independently answer a random sample of MMOral-Bench questions and reporting human performance and agreement. Without this, the interpretation of all absolute scores on MMOral-Bench is ambiguous.
minor comments (5)
  1. [§1] The term 'Oral-Bench' appears once in the introduction and in §5.2 ('on our Oral-Bench'); elsewhere it is 'MMOral-Bench.' Please standardize.
  2. [§5.3] The main text refers to 'Table 9' for the SFT results on Qwen2.5-VL-7B, but the table in the main text is numbered Table 3; Table 9 appears in the supplementary. Please fix the cross-reference.
  3. [§2.1] The text calls the OCR model 'OpenOCR' and cites reference [21], but the reference is titled 'Svtrv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition.' Please clarify whether OpenOCR is the deployed model and cite the appropriate source.
  4. [§3.1 / Figure 3] Figure 3(c) shows the diagnostic distribution of MMOral-Bench, but the figure caption is terse. Please list the exact counts or percentages for the five categories in the caption or text.
  5. [§4.2, Eq. (1)–(2)] The equations are standard, but the definition of N_c in Eq. (2) should be explicit about how samples assigned to multiple categories are handled in per-category scores.

Circularity Check

1 steps flagged

MMOral-Bench is a subset of the MMOral training pool, so the reported 24.73% OralGPT SFT gain reflects memorization rather than generalization.

specific steps
  1. self definitional [Section 4.1 (Benchmark Construction) and Section 5.3 / Table 3 (Efficacy Validation)]
    "All cases in MMOral-Bench are manually chosen and checked from the MMOral to ensure their quality and reliability. ... We implement supervised fine-tuning (SFT) on the Qwen2.5-VL-7B model using our MMOral instruction data, and the results are presented in Table 9."

    By the paper's own description, MMOral-Bench is a curated subset of MMOral. OralGPT is then fine-tuned on MMOral-Report, MMOral-VQA, and MMOral-Chat for one epoch and evaluated on MMOral-Bench. No train/test split is described, and the benchmark images come from one of the two sources used to build the full 20,563-image MMOral. The benchmark QA pairs are generated from the same grounding captions and medical reports that underlie MMOral-VQA/Report, so the SFT model has seen the images and near-identical questions. The headline 24.73% improvement (21.46 to 46.19) is therefore an evaluation on the training distribution by construction; it measures memorization of the benchmark content, not transfer or generalization. The zero-shot ranking of 64 LVLMs is independent, but the central claim tha

full rationale

The paper's strongest empirical claim—that a single epoch of SFT on MMOral improves OralGPT by 24.73% on MMOral-Bench—is circular in effect. Section 4.1 states that 'All cases in MMOral-Bench are manually chosen and checked from the MMOral', and Section 5.3 fine-tunes on the MMOral instruction data with no exclusion of benchmark images or benchmark-derived QA pairs. Since the benchmark is a subset of the training pool, the measured gain is forced by construction and does not demonstrate generalization to unseen panoramic X-rays. The zero-shot evaluation of 64 existing LVLMs is not affected by this leakage and provides independent benchmark value. Appendix F's concession that ground-truth labels lack independent third-party validation is a separate correctness concern, not itself circularity, but it compounds the issue because even a held-out split would inherit potentially noisy labels. Overall, the circularity is partial: the benchmark contribution and zero-shot results stand, while the SFT efficacy conclusion reduces to training-set performance.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper's contributions rest on automated annotation pipelines and an overlapping train/test benchmark. The key axioms are that the automated detections are accurate, that LLM-based scoring is a faithful proxy for experts, and that the benchmark is independent of training data. The last is violated, creating a circular evaluation for OralGPT.

axioms (4)
  • domain assumption Visual specialist models produce sufficiently accurate anatomical ground truth
    All subsequent reports and QA pairs inherit these detections; Appendix F acknowledges the labels have not been independently validated.
  • domain assumption GPT-4-turbo scoring of open-ended answers approximates expert dentist scoring
    The paper validates this in Sec D.2, but per-category absolute differences up to 9.79 points remain.
  • domain assumption The two public source datasets represent the clinical diversity of panoramic X-rays
    The benchmark and training images come from the same two sources, so findings may not transfer to other acquisition settings.
  • ad hoc to paper MMOral-Bench is disjoint from the MMOral training distribution
    This assumption is implicit in using the benchmark to measure SFT gains, but it is explicitly violated in Sec 4.1 where benchmark cases are selected from MMOral.

pith-pipeline@v1.3.0-alltime-deepseek · 37769 in / 10274 out tokens · 108111 ms · 2026-08-04T19:23:42.348373+00:00 · methodology

0 comments
read the original abstract

Recent advances in large vision-language models (LVLMs) have demonstrated strong performance on general-purpose medical tasks. However, their effectiveness in specialized domains such as dentistry remains underexplored. In particular, panoramic X-rays, a widely used imaging modality in oral radiology, pose interpretative challenges due to dense anatomical structures and subtle pathological cues, which are not captured by existing medical benchmarks or instruction datasets. To this end, we introduce MMOral, the first large-scale multimodal instruction dataset and benchmark tailored for panoramic X-ray interpretation. MMOral consists of 20,563 annotated images paired with 1.3 million instruction-following instances across diverse task types, including attribute extraction, report generation, visual question answering, and image-grounded dialogue. In addition, we present MMOral-Bench, a comprehensive evaluation suite covering five key diagnostic dimensions in dentistry. We evaluate 64 LVLMs on MMOral-Bench and find that even the best-performing model, i.e., GPT-4o, only achieves 41.45% accuracy, revealing significant limitations of current models in this domain. To promote the progress of this specific domain, we also propose OralGPT, which conducts supervised fine-tuning (SFT) upon Qwen2.5-VL-7B with our meticulously curated MMOral instruction dataset. Remarkably, a single epoch of SFT yields substantial performance enhancements for LVLMs, e.g., OralGPT demonstrates a 24.73% improvement. Both MMOral and OralGPT hold significant potential as a critical foundation for intelligent dentistry and enable more clinically impactful multimodal AI systems in the dental field. The dataset, model, benchmark, and evaluation suite are available at https://github.com/isbrycee/OralGPT.

Figures

Figures reproduced from arXiv: 2509.09254 by Hao Tang, Jing Hao, Jinrong Yang, Kaixin Guo, Kuo Feng Hung, Lizhuo Lin, Lun M. Wong, Qi Yong H. Ai, Yanpeng Sun, Yuxuan Fan.

Figure 1
Figure 1. Figure 1: Overview of the MMOral. It consists of four sub-datasets: MMOral-Attribute, MMOral [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The MMOral dataset curation pipeline, which consists of four sequential steps. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The data statistic distribution and human evaluation results. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance comparison on both closed-ended and open-ended QA across multiple LVLMs. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Three examples of case studies on closed-ended QA and open-ended QA, respectively. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The prompt for DeepSeek-R1-Distill-Llama-70B to generate the medical report of [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The manually designed in-context examples for medical report generation. Given a [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The prompt for GPT-4-Turbo to revise the generated report. We manually check the [PITH_FULL_IMAGE:figures/full_fig_p021_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The prompt for GPT-4-Turbo to generate both closed-ended and open-ended question [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The prompt for GPT-4-Turbo to generate a multi-turn conversation between the assistant [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The word cloud maps for MMOral-Report, MMOral-VQA, and MMOral-Chat sub [PITH_FULL_IMAGE:figures/full_fig_p024_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Few-shot prompt for evaluating model predictions using GPT-4-Turbo, where [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: The means and standard deviations of each category on 5 repeated evaluations across four [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: An example of MMOral-Attribute and MMOral-Report. [PITH_FULL_IMAGE:figures/full_fig_p028_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: An example of MMOral-Attribute and MMOral-Report. [PITH_FULL_IMAGE:figures/full_fig_p029_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: An example of MMOral-Attribute and MMOral-Report. [PITH_FULL_IMAGE:figures/full_fig_p030_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: An example of MMOral-Attribute and MMOral-Report. [PITH_FULL_IMAGE:figures/full_fig_p031_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: A closed-ended QA example. Red highlights the right answer. 32 [PITH_FULL_IMAGE:figures/full_fig_p032_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: A closed-ended QA example. Red highlights the right answer. Blue highlights the wrong answer. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: A closed-ended QA example. Red highlights the right answer. 34 [PITH_FULL_IMAGE:figures/full_fig_p034_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: A closed-ended QA example. Red highlights the right answer. Blue highlights the wrong answer. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: A closed-ended QA example. Red highlights the right answer. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: A closed-ended QA example. Red highlights the right answer. Blue highlights the wrong answer. 37 [PITH_FULL_IMAGE:figures/full_fig_p037_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: An open-ended QA example. Red highlights the right description. 38 [PITH_FULL_IMAGE:figures/full_fig_p038_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: An open-ended QA example. Red highlights the right description. 39 [PITH_FULL_IMAGE:figures/full_fig_p039_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: An open-ended QA example. Red highlights the right description. 40 [PITH_FULL_IMAGE:figures/full_fig_p040_26.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice

    cs.CL 2026-05 unverdicted novelty 7.0

    OralMLLM-Bench is a new benchmark with 27 tasks in four cognitive categories that evaluates six MLLMs on dental radiographs and shows clear performance gaps versus clinicians.

  2. OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice

    cs.CL 2026-05 unverdicted novelty 7.0

    OralMLLM-Bench reveals performance gaps between multimodal large language models and clinicians on cognitive tasks for dental radiographic analysis across periapical, panoramic, and cephalometric images.

  3. ArchMap: Arch-Flattening and Knowledge-Guided Vision Language Model for Tooth Counting and Structured Dental Understanding

    cs.CV 2025-11 unverdicted novelty 7.0

    ArchMap combines geometric arch-flattening with a dental knowledge base to guide VLMs for accurate tooth counting and structured understanding of 3D intraoral scans without training.

  4. Large AI Models in Dental Healthcare: From General-Purpose Systems to Domain-Specific Foundation Models

    cs.AI 2026-06 unverdicted novelty 4.0

    A PRISMA-ScR scoping review of 97 studies classifies AI models in dentistry into language, vision, and domain-specific types and concludes integrated pipelines outperform single models while noting data and benchmark gaps.

Reference graph

Works this paper leans on

88 extracted references · 40 linked inside Pith · cited by 3 Pith papers

  1. [1]

    URL https://universe.roboflow.com/ nanyang-technological-university-kdgtt/dentistry-vibir

    Dentistry computer vision project, 2023. URL https://universe.roboflow.com/ nanyang-technological-university-kdgtt/dentistry-vibir

    2023

  2. [2]

    URL https://universe.roboflow.com/ panoramic-xray-images/dental-caries-detection

    Dental caries detection computer vision project, 2023. URL https://universe.roboflow.com/ panoramic-xray-images/dental-caries-detection

    2023

  3. [3]

    URL https://www.kaggle.com/datasets/reemsalahshehab/dental?select=data

    2023. URL https://www.kaggle.com/datasets/reemsalahshehab/dental?select=data. yaml

    2023

  4. [4]

    URL https://www.kaggle.com/datasets/ayaalialnozahyy/ dental-diseases/data

    Dental diseases, 2024. URL https://www.kaggle.com/datasets/ayaalialnozahyy/ dental-diseases/data

    2024

  5. [5]

    URL https://universe.roboflow.com/ arshs-workspace-radio/vzrad2

    vzrad2 computer vision project, 2024. URL https://universe.roboflow.com/ arshs-workspace-radio/vzrad2

    2024

  6. [6]

    URLhttps://universe.roboflow.com/arshs-workspace-radio/vzrad2

    2024. URLhttps://universe.roboflow.com/arshs-workspace-radio/vzrad2

    2024

  7. [7]

    URLhttps://www.kaggle.com/datasets/nadaaglan/dental-periapical-x-rayss

    2024. URLhttps://www.kaggle.com/datasets/nadaaglan/dental-periapical-x-rayss

    2024

  8. [8]

    Abdin, J

    M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, H. Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Abdin, J

    M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, et al. Phi-4 technical report.arXiv preprint arXiv:2412.08905, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  11. [11]

    Claude 3.7 sonnet system card

    Anthropic. Claude 3.7 sonnet system card. URL https://api.semanticscholar.org/CorpusID: 276612236

  12. [12]

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou. Qwen-vl: A versatile vision- language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Beyer, A

    L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Alabdulmohsin, M. Tschannen, E. Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer.arXiv preprint arXiv:2407.07726, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    J. Chen, C. Gui, R. Ouyang, A. Gao, S. Chen, G. H. Chen, X. Wang, R. Zhang, Z. Cai, K. Ji, et al. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024. 10

    Pith/arXiv arXiv 2024

  16. [16]

    Z. Chen, W. Wang, Y . Cao, Y . Liu, Z. Gao, E. Cui, J. Zhu, S. Ye, H. Tian, Z. Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    T. S. de Oliveira Capote, M. de Almeida Gonçalves, A. Gonçalves, and M. Gonçalves. Panoramic radiography—diagnosis of relevant structures that might compromise oral and general health of the patient. InEmerging Trends in Oral Health Sciences and Dentistry. IntechOpen, 2015

    2015

  18. [18]

    Deitke, C

    M. Deitke, C. Clark, S. Lee, R. Tripathi, Y . Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models.arXiv preprint arXiv:2409.17146, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    H. V . Do, T. N. N. V o, P. T. Nguyen, T. H. L. Luong, N. G. Cu, and H. S. Le. A dataset of apical periodontitis lesions in panoramic radiographs for deep-learning-based classification and detection.Data in Brief, 54:110486, 2024

    2024

  20. [20]

    R. Dong, C. Han, Y . Peng, Z. Qi, Z. Ge, J. Yang, L. Zhao, J. Sun, H. Zhou, H. Wei, et al. Dreamllm: Syner- gistic multimodal comprehension and creation.International Conference on Learning Representations, 2024

    2024

  21. [21]

    Y . Du, Z. Chen, H. Xie, C. Jia, and Y .-G. Jiang. Svtrv2: Ctc beats encoder-decoder models in scene text recognition.CoRR, abs/2411.15858, 2024. URLhttps://arxiv.org/abs/2411.15858

    Pith/arXiv arXiv 2024

  22. [22]

    H. Duan, J. Yang, Y . Qiao, X. Fang, L. Chen, Y . Liu, X. Dong, Y . Zang, P. Zhang, J. Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM International Conference on Multimedia, pages 11198–11201, 2024

    2024

  23. [23]

    T. GLM, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Rojas, G. Feng, H. Zhao, H. Lai, H. Yu, H. Wang, J. Sun, J. Zhang, J. Cheng, J. Gui, J. Tang, J. Zhang, J. Li, L. Zhao, L. Wu, L. Zhong, M. Liu, M. Huang, P. Zhang, Q. Zheng, R. Lu, S. Duan, S. Zhang, S. Cao, S. Yang, W. L. Tam, W. Zhao, X. Liu, X. Xia, X. Zhang, X. Gu, X. Lv, X. Liu, X. Liu, X. Yang, ...

    2024

  24. [24]

    Grattafiori, A

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Guo and W

    Y . Guo and W. Huang. Llava-next-med: Medical multimodal large language model. In2025 Asia-Europe Conference on Cybersecurity, Internet of Things and Soft Computing (CITSC), pages 474–477. IEEE, 2025

    2025

  26. [26]

    I. E. Hamamci, S. Er, E. Simsar, A. E. Yuksel, S. Gultekin, S. D. Ozdemir, K. Yang, H. B. Li, S. Pati, B. Stadlinger, et al. Dentex: An abnormal tooth detection with dental enumeration and diagnosis benchmark for panoramic x-rays.arXiv preprint arXiv:2305.19112, 2023

    arXiv 2023

  27. [27]

    J. Hao, Y . Zhu, L. He, M. Liu, J. K. H. Tsoi, and K. F. Hung. T-mamba: A unified framework with long-range dependency in dual-domain for 2d & 3d tooth segmentation.arXiv preprint arXiv:2404.01065, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    S. He, Y . Nie, Z. Chen, Z. Cai, H. Wang, S. Yang, and H. Chen. Meddr: Diagnosis-guided bootstrapping for large-scale medical vision-language learning.arXiv e-prints, pages arXiv–2404, 2024

    2024

  29. [29]

    X. He, Y . Zhang, L. Mou, E. Xing, and P. Xie. Pathvqa: 30000+ questions for medical visual question answering.arXiv preprint arXiv:2003.10286, 2020

    Pith/arXiv arXiv 2003

  30. [30]

    Z. He, Y . Wang, A. Yan, Y . Liu, E. Y . Chang, A. Gentili, J. McAuley, and C.-N. Hsu. Medeval: a multi- level, multi-task, and multi-domain medical benchmark for language model evaluation.arXiv preprint arXiv:2310.14088, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Y . Hu, T. Li, Q. Lu, W. Shao, J. He, Y . Qiao, and P. Luo. Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22170–22183, 2024

    2024

  32. [32]

    Hurst, A

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    J. J. Lau, S. Gayen, A. Ben Abacha, and D. Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images.Scientific data, 5(1):1–10, 2018. 11

    2018

  34. [34]

    B. Li, Y . Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y . Li, Z. Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    2023

  36. [36]

    S. Li, T. Lin, L. Lin, W. Zhang, J. Liu, X. Yang, J. Li, Y . He, X. Song, J. Xiao, et al. Eyecaregpt: Boosting comprehensive ophthalmology understanding with tailored dataset, benchmark and model.arXiv preprint arXiv:2504.13650, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    T. Li, Y . Su, W. Li, B. Fu, Z. Chen, Z. Huang, G. Wang, C. Ma, Y . Chen, M. Hu, et al. Gmai-vl & gmai-vl-5.5 m: A large vision-language model and a comprehensive multimodal dataset towards general medical ai.arXiv preprint arXiv:2411.14522, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    T. Lin, W. Zhang, S. Li, Y . Yuan, B. Yu, H. Li, W. He, H. Jiang, M. Li, X. Song, et al. Healthgpt: A medical large vision-language model for unifying comprehension and generation via heterogeneous knowledge adaptation.arXiv preprint arXiv:2502.09838, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Liu, L.-M

    B. Liu, L.-M. Zhan, L. Xu, L. Ma, Y . Yang, and X.-M. Wu. Slake: A semantically-labeled knowledge- enhanced dataset for medical visual question answering. In2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650–1654. IEEE, 2021

    2021

  40. [40]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  41. [41]

    H. Liu, C. Li, Y . Li, B. Li, Y . Zhang, S. Shen, and Y . J. Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URLhttps://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  42. [42]

    M. Liu, W. Hu, J. Ding, J. Xu, X. Li, L. Zhu, Z. Bai, X. Shi, B. Wang, H. Song, et al. Medbench: A comprehensive, standardized, and reliable benchmarking system for evaluating chinese medical large language models.Big Data Mining and Analytics, 7(4):1116–1128, 2024

    2024

  43. [43]

    H. Lu, W. Liu, B. Zhang, B. Wang, K. Dong, B. Liu, J. Sun, T. Ren, Z. Li, H. Yang, et al. Deepseek-vl: towards real-world vision-language understanding.arXiv preprint arXiv:2403.05525, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    P. Lu, B. Peng, H. Cheng, M. Galley, K.-W. Chang, Y . N. Wu, S.-C. Zhu, and J. Gao. Chameleon: Plug-and-play compositional reasoning with large language models.Advances in Neural Information Processing Systems, 36:43447–43478, 2023

    2023

  45. [45]

    S. Lu, Y . Li, Q.-G. Chen, Z. Xu, W. Luo, K. Zhang, and H.-J. Ye. Ovis: Structural embedding alignment for multimodal large language model.arXiv preprint arXiv:2405.20797, 2024

    Pith/arXiv arXiv 2024

  46. [46]

    M. Moor, Q. Huang, S. Wu, M. Yasunaga, Y . Dalmia, J. Leskovec, C. Zakka, E. P. Reis, and P. Rajpurkar. Med-flamingo: a multimodal medical few-shot learner. InMachine Learning for Health (ML4H), pages 353–367. PMLR, 2023

    2023

  47. [47]

    J. Pan, C. Liu, J. Wu, F. Liu, J. Zhu, H. B. Li, C. Chen, C. Ouyang, and D. Rueckert. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. arXiv preprint arXiv:2502.19634, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Panetta, R

    K. Panetta, R. Rajendran, A. Ramesh, S. P. Rao, and S. Agaian. Tufts dental database: a multimodal panoramic x-ray dataset for benchmarking diagnostic systems.IEEE journal of biomedical and health informatics, 26(4):1650–1659, 2021

    2021

  49. [49]

    J. Park, S. Kim, B. Yoon, J. Hyun, and K. Choi. M4cxr: Exploring multi-task potentials of multi-modal large language models for chest x-ray interpretation.arXiv preprint arXiv:2408.16213, 2024

    Pith/arXiv arXiv 2024

  50. [50]

    Z. Peng, W. Wang, L. Dong, Y . Hao, S. Huang, S. Ma, and F. Wei. Kosmos-2: Grounding multimodal large language models to the world.arXiv preprint arXiv:2306.14824, 2023

    Pith/arXiv arXiv 2023

  51. [51]

    Z. Qin, Y . Yin, D. Campbell, X. Wu, K. Zou, Y .-C. Tham, N. Liu, X. Zhang, and Q. Chen. Lmod: A large multimodal ophthalmology dataset and benchmark for large vision-language models.arXiv preprint arXiv:2410.01620, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Y . Sun, H. Wu, C. Zhu, S. Zheng, Q. Chen, K. Zhang, Y . Zhang, D. Wan, X. Lan, M. Zheng, et al. Pathmmu: A massive multimodal expert-level benchmark for understanding and reasoning in pathology. InEuropean Conference on Computer Vision, pages 56–73. Springer, 2024. 12

    2024

  53. [53]

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  54. [54]

    G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    K. Team, A. Du, B. Yin, B. Xing, B. Qu, B. Wang, C. Chen, C. Zhang, C. Du, C. Wei, et al. Kimi-vl technical report.arXiv preprint arXiv:2504.07491, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    Q. Team. Qvq: To see the world with wisdom, December 2024. URL https://qwenlm.github.io/ blog/qvq-72b-preview/

    2024

  57. [57]

    P. Tong, E. Brown, P. Wu, S. Woo, A. J. V . IYER, S. C. Akula, S. Yang, J. Yang, M. Middepogu, Z. Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    2024

  58. [58]

    Turosz, K

    N. Turosz, K. Ch˛ eci´nska, M. Ch˛ eci´nski, A. Brzozowska, Z. Nowak, and M. Sikora. Applications of artificial intelligence in the analysis of dental panoramic radiographs: an overview of systematic reviews. Dentomaxillofacial Radiology, 52(7):20230284, 2023

    2023

  59. [59]

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al. Qwen2-vl: Enhanc- ing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Pith/arXiv arXiv 2024

  60. [60]

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, S. XiXuan, et al. Cogvlm: Visual expert for pretrained language models.Advances in Neural Information Processing Systems, 37: 121475–121499, 2024

    2024

  61. [61]

    X. Wang, F. Wang, Y . Li, Q. Ma, S. Wang, B. Jiang, C. Li, and J. Tang. Cxpmrg-bench: Pre- training and benchmarking for x-ray medical report generation on chexpert plus dataset.arXiv preprint arXiv:2410.00379, 2024

    Pith/arXiv arXiv 2024

  62. [62]

    X. Wang, X. Zhang, Z. Luo, Q. Sun, Y . Cui, J. Wang, F. Zhang, Y . Wang, Z. Li, Q. Yu, et al. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    Pith/arXiv arXiv 2024

  63. [63]

    H. Wei, L. Kong, J. Chen, L. Zhao, Z. Ge, J. Yang, J. Sun, C. Han, and X. Zhang. Vary: Scaling up the vision vocabulary for large vision-language model. InEuropean Conference on Computer Vision, pages 408–424. Springer, 2024

    2024

  64. [64]

    Wong and J.-L

    T. Wong and J.-L. Eiselé. Fdi world dental federation: Responding to new realities of oral health, 2015

    2015

  65. [65]

    Wright and P

    C. Wright and P. Reeves. Radbench: benchmarking image interpretation skills.Radiography, 22(2): e131–e136, 2016

    2016

  66. [66]

    Y . Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    Pith/arXiv arXiv 2024

  67. [67]

    J. Ye, G. Wang, Y . Li, Z. Deng, W. Li, T. Li, H. Duan, Z. Huang, Y . Su, B. Wang, et al. Gmai-mmbench: A comprehensive multimodal evaluation benchmark towards general medical ai.Advances in Neural Information Processing Systems, 37:94327–94427, 2024

    2024

  68. [68]

    J. Ye, H. Xu, H. Liu, A. Hu, M. Yan, Q. Qian, J. Zhang, F. Huang, and J. Zhou. mplug-owl3: Towards long image-sequence understanding in multi-modal large language models.arXiv preprint arXiv:2408.04840, 2024

    Pith/arXiv arXiv 2024

  69. [69]

    Q. Ye, H. Xu, J. Ye, M. Yan, A. Hu, H. Liu, Q. Qian, J. Zhang, and F. Huang. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. InProceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 13040–13051, 2024

    2024

  70. [70]

    K. Ying, F. Meng, J. Wang, Z. Li, H. Lin, Y . Yang, H. Zhang, W. Zhang, Y . Lin, S. Liu, et al. Mmt-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi.arXiv preprint arXiv:2404.16006, 2024

    Pith/arXiv arXiv 2024

  71. [71]

    Young, B

    A. Young, B. Chen, C. Li, C. Huang, G. Zhang, G. Zhang, G. Wang, H. Li, J. Zhu, J. Chen, et al. Yi: Open foundation models by 01. ai.arXiv preprint arXiv:2403.04652, 2024

    Pith/arXiv arXiv 2024

  72. [72]

    E. Yu, L. Zhao, Y . Wei, J. Yang, D. Wu, L. Kong, H. Wei, T. Wang, Z. Ge, X. Zhang, et al. Merlin: Empowering multimodal llms with foresight minds. InEuropean Conference on Computer Vision, pages 425–443. Springer, 2024. 13

    2024

  73. [73]

    W. Yu, Z. Yang, L. Li, J. Wang, K. Lin, Z. Liu, X. Wang, and L. Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490, 2023

    Pith/arXiv arXiv 2023

  74. [74]

    W. Yu, Z. Yang, L. Ren, L. Li, J. Wang, K. Lin, C.-C. Lin, Z. Liu, L. Wang, and X. Wang. Mm-vet v2: A challenging benchmark to evaluate large multimodal models for integrated capabilities.arXiv preprint arXiv:2408.00765, 2024

    Pith/arXiv arXiv 2024

  75. [75]

    X. Yue, Y . Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y . Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556–9567, 2024

    2024

  76. [76]

    Zhang, X

    P. Zhang, X. Dong, Y . Zang, Y . Cao, R. Qian, L. Chen, Q. Guo, H. Duan, B. Wang, L. Ouyang, et al. Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output.arXiv preprint arXiv:2407.03320, 2024

    Pith/arXiv arXiv 2024

  77. [77]

    L. Zhao, E. Yu, Z. Ge, J. Yang, H. Wei, H. Zhou, J. Sun, Y . Peng, R. Dong, C. Han, et al. Chatspot: Bootstrapping multimodal llms via precise referring instruction tuning.arXiv preprint arXiv:2307.09474, 2023

    Pith/arXiv arXiv 2023

  78. [78]

    Zheng, R

    Y . Zheng, R. Zhang, J. Zhang, Y . Ye, Z. Luo, Z. Feng, and Y . Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics. URLhttp://arxiv.org/abs/240...

    Pith/arXiv arXiv 2024

  79. [79]

    Object Detection 4 Quadrants 4 634

  80. [80]

    Object Detection Caries, Deep Caries, Periapical le- sions, Impacted tooth 4 705

Showing first 80 references.