Pith. sign in

REVIEW 3 major objections 4 minor 36 references

CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a 100,000-image synthetic dataset of digital measurement devices, with procedurally rewritten displays and realistic placement, teaches large vision-language models to read real instruments: InternVL2.5-26B's ANLS rise

desk verdict A useful synthetic-data pipeline for a real OCR/VQA niche, but the headline causal claim is undercut by the missing 0%-synthetic control. read the letter →

arxiv 2508.21732 v1 pith:PZESWHOC submitted 2025-08-29 cs.CV cs.AI

João Valente , Atabak Dehban , Rodrigo Ventura This is my paper
classification cs.CVcs.AI
keywords digitalmeasurementdevicessyntheticdatagenerationvisualquestionansweringOCRfine-tuningDMDBenchCAD2DMD-SETlargevision-languagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large vision-language models can talk fluently about images yet frequently misread a humble digital multimeter or pulse oximeter, especially under the oblique viewpoints and motion blur that head-mounted cameras produce. This paper claims that this failure is fixable with synthetic data: a 100,000-image, VQA-labelled dataset of six measurement devices, generated by rendering photogrammetric 3D models whose displays are procedurally rewritten with new readings and then placing the renders into realistic indoor scenes, can replace large amounts of real instrument data in fine-tuning. The evidence is a before/after comparison on DMDBench, a new 1,000-image real-world validation set. InternVL2.5-26B's ANLS jumps from 32.92% to 95.16% with only a 10% mixture of synthetic data, while its TextVQA score stays flat; LLaVA improves from 25.06% to 71.21%. A reader should care because it suggests a practical, open-source path to training AR and OCR systems for instrument reading without collecting tens of thousands of hard-to-obtain real photos.

What carries the argument

Three coupled modules carry the argument. The Display Generator Otsu-binarizes one real photo of each device's screen and rewrites the readings inside predefined regions using the segmented-font family DSEG, with values sampled from per-mode dictionaries. The Renderer maps that display via UV coordinates onto a single face of a photogrammetrically reconstructed CAD mesh, randomizes camera distance, object rotation, focal length, and light color and intensity, adds motion blur, and exports depth-derived masks. The Image Composer places the foreground with the FOPA placement model, corrects the bounding box to preserve aspect ratio, pastes it onto an ADE20K indoor background, and logs metadata

What would settle it

Collect a held-out set of real DMD photos that includes brands, display types, and scenes absent from the six reconstructed devices, fine-tune with the identical 10% LoRA mixture, and measure ANLS. If the gain on that set is near zero while DMDBench remains high, the effect is overfit to the synthetic distribution; a complementary check is to compute a distribution distance between synthetic and real display crops, such as Fréchet Inception Distance, and correlate it with the transfer gain.

Watch

Extended reading notes

Core claim

The paper establishes that a fully synthetic training distribution can teach an LVLM to read real digital measurement devices. Each synthetic image is built by Otsu-thresholding a real display photo, clearing the digits, drawing new values from per-device dictionaries in a segmented-display font, UV-mapping that display onto a photogrammetrically reconstructed CAD mesh, and pasting the rendered device into an indoor background at a contextually plausible location, with motion blur and lighting randomization. Fine-tuning LoRA adapters on a 10% mixture of these images lifts InternVL2.5-26B's ANLS on the real-world DMDBench from 32.92% to 95.16%, and to 96.04% at 50%, while TextVQA moves by les

Load-bearing premise

The load-bearing premise is that synthetic displays rendered with segmented fonts and mapped onto six photogrammetric meshes look enough like real LCD/LED screens under blur, reflections, and oblique viewing that a model fine-tuned on them will read real instruments it has not seen; the paper's only support is the final DMDBench score, with no direct measurement of the synthetic-to-real gap.

Editorial extensions

If this is right

  • A LoRA fine-tune with just a 10% synthetic mixture can take InternVL2.5-26B from 32.92% to 95.16% ANLS on real-world DMD reading, and a 50% mixture reaches 96.04%.
  • The recipe is partially device-general: excluding power supply images from training still raises power-supply ANLS from 41.58% to 72.94%, so the synthetic data transfers across devices within the reconstructed set.
  • Realistic placement is a major source of the gain: FOPA-guided composition gives 95.16% versus 78.17% for random placement, at a generation-time cost of 1.4s vs 0.2s per image.
  • Putting motion blur into roughly 20% of training images adds 6.3 points over a no-blur dataset, matching the blur-dominated failure mode of head-mounted cameras.
  • One-word VQA labels improve unit accuracy from 84.5% to 99.6% over full-sentence labels on the prompt-independent DMDBench, showing label format matters for OCR-level learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never measures the synthetic-to-real domain gap directly; the ANLS gains could in part be specific to the six photogrammetrically reconstructed devices and the author-collected DMDBench. A natural test is to fine-tune with the same recipe and evaluate on an independently collected set of different DMD brands and display types; if the gain shrinks much, the learned prior is narrower than
  • Because a 10% synthetic mixture nearly matches 50% for InternVL while LLaVA degrades at 50%, the optimal mixture ratio appears model-dependent; practitioners would need a small real validation set to choose the ratio rather than assuming more synthetic data is better.
  • Since the photogrammetry route requires physical access to each device, the pipeline as described cannot create novel or unobtainable instruments; replacing photogrammetric meshes with synthetic CAD models would test whether the display-rendering and placement steps, not the scan textures, are what transfers.
  • The one-word label result suggests that part of the measured 'instrument reading' improvement is about answer formatting and unit vocabulary, not only visual perception; an OCR-specific metric on display crops would separate the two effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces CAD2DMD-SET, a synthetic data generation tool for digital measurement device (DMD) reading, together with DMDBench, a manually annotated benchmark of 1,000 real-world DMD images. The pipeline reconstructs 3D CAD models of six DMDs via photogrammetry, renders them in Blender with synthetic display textures generated from DSEG fonts, and composes the rendered foregrounds into ADE20k backgrounds using the FOPA placement model. VQA labels are produced automatically. The authors benchmark Pixtral-12B, LLaVA-1.5-13B, and InternVL2.5-26B on DMDBench and TextVQA, and fine-tune LoRA modules on mixtures of standard instruction-tuning data and the synthetic dataset. They report large DMDBench ANLS improvements, most notably InternVL2.5-26B rising from 32.92% to 95.16% (10% mixture) and 96.04% (50% mixture), with TextVQA staying roughly constant. Four ablations examine object placement, motion blur, label format, and inter-device generalization.

Significance. If the central causal claim holds, the paper makes a useful practical contribution: it provides an extensible open-source-style tool for generating large synthetic DMD datasets, a real-world benchmark, and evidence that fine-tuning on such data transfers to real images. The authors are careful to include TextVQA as a no-forgetting check, and the inter-device ablation in A.3.4 partially supports generalization beyond the exact training devices. The tool addresses a genuine data-scarcity problem for AR/head-mounted-camera applications. However, the headline result is not yet fully established because the experimental design lacks a critical control, and the evaluation set appears to share device instances/types with the synthetic training set. The quantitative claims also rest on single training runs with no variance estimates.

major comments (3)
  1. [§4.3, Table 1] The central causal claim — that CAD2DMD-SET synthetic data drives the DMDBench improvement — is not isolated. The table compares non-fine-tuned baselines to models fine-tuned on a mixture of synthetic data and the model's standard instruction-tuning data. There is no 0%-synthetic LoRA control. The improvement could in principle come from additional instruction tuning / OCR exposure rather than from the synthetic DMD data. Since the ablations in A.3.1–A.3.3 all vary settings among synthetic-data-containing runs, they do not identify the synthetic contribution. A LoRA fine-tuned on the standard instruction data alone, with the same data budget and training protocol, is necessary to support the abstract's attribution.
  2. [§4.2, §4.3, A.3.4] DMDBench appears to be built from the same six device types (and possibly the same physical instances) for which CAD models were reconstructed for the synthetic training set. The paper never states whether the same device instances appear in both. If they do, the large ANLS gains may reflect instance/type alignment rather than general DMD-reading ability. The inter-device ablation (A.3.4) is encouraging — excluding power supply images still yields 72.94% vs 41.58% on power-supply-only DMDBench — but it covers only one held-out device type and still uses the same test distribution construction. The authors should state the device overlap explicitly and ideally add a held-out device category, or provide synthetic-to-real domain-gap statistics, to support the claim of real-world generalization.
  3. [§4.3, Tables 1–5] All fine-tuning results are single runs with no confidence intervals, error bars, or significance tests. The headline gap (32.92% to 95.16% ANLS) is the basis for the paper's main claim, but LoRA fine-tuning is stochastic in data sampling, seed, and training order, and the differences between the 10% and 50% conditions are small. Reporting mean ± standard deviation over at least three seeds, or otherwise quantifying variability, is needed before the quantitative superiority of one condition over another can be assessed.
minor comments (4)
  1. [§4.3] The sentence 'The ANLS score improvement /indicates the model learned proper sentence structures from the synthetic dataset Fig. 1.' contains a stray slash and an unclear figure reference. Also, 'DMD-Bench' is used inconsistently with 'DMDBench'.
  2. [§4.3] The description 'trained with 10% and 50%, respectively, of a mixture of our synthetic dataset and the standard instruction tuning data' is ambiguous. It could mean the total fine-tuning data is a 10%/50% subsample, or that 10%/50% of the mixture is synthetic. Please clarify the composition of the fine-tuning set.
  3. [§3.2] There are several typos: 'rotation around access' should be 'axis', 'Eeeve' should be 'Eevee', and 'Furthemore' in §5 should be 'Furthermore'.
  4. [§4.1] The paper states approximately 200 images per device for photogrammetry but does not report reconstruction quality, number of mesh vertices, or any manual cleanup steps beyond 'manual refinement'. A sentence on model fidelity would help readers judge the synthetic-to-real gap.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the synthetic-training improvement claim is empirical and not forced by construction, though a 0%-synthetic control is missing.

full rationale

The paper's central claim is empirical and does not reduce to its inputs by construction. CAD2DMD-SET synthesizes images through parametric rendering (display dictionaries, DSEG fonts, photogrammetric 3D meshes, Blender rendering, FOPA-based composition), while DMDBench is a separately collected set of 1,000 real images annotated by the authors. No equation defines the benchmark outputs in terms of the training outputs, and no fitted parameter is renamed as a prediction. The fine-tuning evaluation does not use DMDBench during training; the inter-device ablation (A.3.4) shows improvement on a held-out device, and TextVQA provides an external no-forgetting check. The only self-citation (Ref. [3]) is motivational and not load-bearing. A legitimate internal-validity concern is that Table 1 lacks a 0%-synthetic LoRA control, so the causal attribution to the synthetic component rather than to the standard instruction-tuning mixture is not fully isolated; however, this is an experimental confound, not a constructional circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on the quality and transferability of the synthetic pipeline. The free parameters are experimental settings that affect the reported magnitudes. The axioms are domain assumptions about photogrammetry, font realism, composition, and transfer; none are formally verified, and the paper provides no independent evidence for them beyond the final benchmark numbers.

free parameters (3)
  • synthetic_data_mixture_fraction = 10% and 50%
    Hand-chosen fraction of CAD2DMD-SET synthetic data in the LoRA fine-tuning mixture; the headline InternVL result uses 50% (95.16% for 10%, 96.04% for 50%), and LLaVA performs worse at 50% (53.55 vs 71.21), so the chosen fraction materially affects the reported improvement.
  • motion_blur_training_fraction = ~20%
    The main training dataset includes approximately 20% motion-blurred images, selected after an ablation showing better ANLS than 0% (95.16 vs 88.89). The central result embeds this hand-set fraction.
  • foreground_min_area_fraction = 10%
    Bounding boxes covering less than 10% of the background are scaled up to keep devices readable; this hand-set rule changes the scale distribution of training composites and is not derived from any principle.
assumptions (4)
  • domain assumption Photogrammetry via Object Capture produces CAD meshes with display faces flat and accurate enough for UV mapping of synthetic displays.
    Invoked in Section 3.2; the renderer requires a single rectangular display face, possibly after manual mesh editing. If the reconstructed geometry distorts the display area, the synthetic images will not faithfully resemble real devices.
  • domain assumption DSEG fonts faithfully reproduce the appearance of the real devices' digital displays.
    Invoked in Section 3.1; measurement values are rendered with randomly chosen DSEG fonts. The fine-tuning transfer claim depends on these synthetic displays being visually similar to real LCD/LED displays in DMDBench.
  • domain assumption FOPA placement computed on downscaled backgrounds (longest side 256 px) remains valid after rescaling to high-resolution, and copy-paste composition preserves measurement legibility.
    Invoked in Section 3.3; the authors note aspect-ratio corrections reduce realism and harmonization was disabled because it blurred digits. If placement or scaling distorts the foreground, training samples drift from real-world appearance.
  • domain assumption LoRA fine-tuning on a mixture of synthetic data and instruction-tuning data transfers to real-world DMD images without catastrophic forgetting.
    This is the transfer premise of the empirical study; it is tested on DMDBench and TextVQA, but no formal guarantee or error analysis is provided, and it could fail for devices or conditions outside the training distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models." pith.science (2026). https://pith.science/paper/PZESWHOC

@misc{pith2026250821732,
  author       = {Pith},
  title        = {Pith review of: CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PZESWHOC}},
  note         = {Machine review of arXiv:2508.21732}
}
read the original abstract

Recent advancements in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities across various multimodal tasks. They continue, however, to struggle with trivial scenarios such as reading values from Digital Measurement Devices (DMDs), particularly in real-world conditions involving clutter, occlusions, extreme viewpoints, and motion blur; common in head-mounted cameras and Augmented Reality (AR) applications. Motivated by these limitations, this work introduces CAD2DMD-SET, a synthetic data generation tool designed to support visual question answering (VQA) tasks involving DMDs. By leveraging 3D CAD models, advanced rendering, and high-fidelity image composition, our tool produces diverse, VQA-labelled synthetic DMD datasets suitable for fine-tuning LVLMs. Additionally, we present DMDBench, a curated validation set of 1,000 annotated real-world images designed to evaluate model performance under practical constraints. Benchmarking three state-of-the-art LVLMs using Average Normalised Levenshtein Similarity (ANLS) and further fine-tuning LoRA's of these models with CAD2DMD-SET's generated dataset yielded substantial improvements, with InternVL showcasing a score increase of 200% without degrading on other tasks. This demonstrates that the CAD2DMD-SET training dataset substantially improves the robustness and performance of LVLMs when operating under the previously stated challenging conditions. The CAD2DMD-SET tool is expected to be released as open-source once the final version of this manuscript is prepared, allowing the community to add different measurement devices and generate their own datasets.

Figures

Figures reproduced from arXiv: 2508.21732 by the authors.

Figure 2
Figure 2. CAD2DMD-SET Pipeline - Pulse Oximeter Example [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Synthetic Multimeter Display [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Foreground Object Renders and Masks (Thermometer and Multimeter). [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figures from the paper (8 more)
Figure 6
Figure 6. Figure 6: Examples of the Metronome, Power Supply and Blood Pressure Device. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: DMDBench Image Examples To assess the relative difficulty that DMDBench images present to LVLMs, the same models were also evaluated on the TextVQA dataset [23]. This a benchmark specifically designed to test the ability of models to read and reason about text within i…
Figure 8
Figure 8. Figure 8: Challenging Motion Blur DMDBench Examples [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Challenging Lighting DMDBench Examples 13 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Challenging Orientation DMDBench Examples [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Training Dataset Examples A.2 Training Improvements and Limitations Fine-tuning experiments demonstrated the effectiveness of the CAD2DMD-SET training dataset in improving the robustness and performance of LVLMs under challenging real-world conditions. Nonetheless, ce…
Figure 13
Figure 13. Figure 13: DMDBench Improvement Examples for LLaVA-v1.5-13B. [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 15
Figure 15. Figure 15: DMDBench Challenging Examples for LLaVA-v1.5-13B. [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 27 canonical work pages

  1. [1]

    Effectiveness assessment of recent large vision-language models

    Yao Jiang, Xinyu Yan, Ge-Peng Ji, Keren Fu, Meijun Sun, Huan Xiong, Deng-Ping Fan, and Fahad Shahbaz Khan. Effectiveness assessment of recent large vision-language models. Visual Intelligence, 2024

    work page 2024

  2. [2]

    Vision language models are biased

    An V o, Khai-Nguyen Nguyen, Mohammad Reza Taesiri, Vy Tuong Dang, Anh Totti Nguyen, and Daeyoung Kim. Vision language models are biased. arXiv preprint arXiv:2505.23941, 2025

    arXiv 2025

  3. [3]

    Ai-powered augmented reality for satellite assembly, integration and test

    Álvaro Patrício, João Valente, Atabak Dehban, Inês Cadilha, Daniel Reis, and Rodrigo Ventura. Ai-powered augmented reality for satellite assembly, integration and test. In IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), 2025

    work page 2025

  4. [4]

    High-fidelity synthetic data generation framework for unique objects detection

    Nataliya Shakhovska, Bohdan Sydor, Solomiia Liaskovska, Olga Duran, Yevgen Martyn, and V olodymyr Vira. High-fidelity synthetic data generation framework for unique objects detection. Computation, 2025

    work page 2025

  5. [5]

    Unity perception: Generate synthetic data for computer vision

    Steve Borkman, Adam Crespi, Saurav Dhakad, Sujoy Ganguly, Jonathan Hogins, You-Cyuan Jhang, Mohsen Kamalzadeh, Bowen Li, Steven Leal, Pete Parisi, Cesar Romero, Wesley Smith, Alex Thaman, Samuel Warren, and Nupur Yadav. Unity perception: Generate synthetic data for computer vision. arXiv preprint arXiv:2107.04259, 2021

    arXiv 2021

  6. [6]

    Libcom: Image composition toolbox

    Bcmi. Libcom: Image composition toolbox. https://github.com/bcmi/libcom. Accessed: 2025-05-24

    work page 2025

  7. [7]

    Controlcom: Controllable image composition using diffusion model

    Bo Zhang, Yuxuan Duan, Jun Lan, Yan Hong, Huijia Zhu, Weiqiang Wang, and Li Niu. Controlcom: Controllable image composition using diffusion model. arXiv preprint arXiv:2308.10040, 2023

    arXiv 2023

  8. [8]

    Fast object placement assessment

    Li Niu, Qingyang Liu, Zhenchen Liu, and Jiangtong Li. Fast object placement assessment. arXiv preprint arXiv:2205.14280, 2022

    arXiv 2022

Show all 36 references
  1. [9]

    Opa: Object placement assessment dataset

    Liu Liu, Zhenchen Liu, Bo Zhang, Jiangtong Li, Li Niu, Qingyang Liu, and Liqing Zhang. Opa: Object placement assessment dataset. arXiv preprint arXiv:2107.01889, 2022. 10

    2022 arXiv

  2. [10]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM international conference on multimedia, 2024

    2024

  3. [11]

    JaidedAI. Easyocr. https://github.com/JaidedAI/EasyOCR. Accessed: 2025-06-24

    2025

  4. [12]

    A threshold selection method from gray-level histograms

    Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 1979

    1979

  5. [13]

    Dseg: Font series for seven-segment display

    Keshikan. Dseg: Font series for seven-segment display. https://github.com/keshikan/DSEG, 2019. Ac- cessed: 2025-06-19

    2019

  6. [14]

    Recognizing indoor scenes

    Ariadna Quattoni and Antonio Torralba. Recognizing indoor scenes. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009

    2009

  7. [15]

    Semantic understanding of scenes through the ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision, 2019

    2019

  8. [16]

    Image harmonization: A review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization

    Fengling Hu, Andrew A Chen, Hannah Horng, Vishnu Bashyam, Christos Davatzikos, Aaron Alexander-Bloch, Mingyao Li, Haochang Shou, Theodore D Satterthwaite, Meichen Yu, et al. Image harmonization: A review of statistical and deep learning methods for removing batch effects and e...

    2023

  9. [17]

    Pixtral 12b

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073, 2024

    2024 arXiv

  10. [18]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  11. [19]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXi...

    2025 arXiv

  12. [20]

    Xiaorui Ma, Haoran Xie, and S. Joe Qin. Efficiently integrate large language models with visual perception: A survey from the training paradigm perspective. Information Fusion, 2025

    2025

  13. [21]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Represen- tations (ICLR), 2022

    2022

  14. [22]

    Anls* – a universal document processing metric for generative large language models

    David Peer, Philemon Schöpf, V olckmar Nebendahl, Alexander Rietzler, and Sebastian Stabinger. Anls* – a universal document processing metric for generative large language models. arXiv preprint arXiv:2402.03848, 2025

    2025 arXiv

  15. [23]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019

    2019

  16. [24]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision (ECCV), 2014

    2014

  17. [25]

    Gqa: A new dataset for real-world visual reasoning and com- positional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and com- positional question answering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2019

    2019

  18. [26]

    Ocr-vqa: Visual question answering by reading text in images

    Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. InInternational Conference on Document Analysis and Recognition (ICDAR), 2019

    2019

  19. [27]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer ...

    2017

  20. [28]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In European Conference on Computer Vision (ECCV). Springer, 2016

    2016

  21. [29]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics, 2022. 11

    2022

  22. [30]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In IEEE Winter Conference on Applications of Computer Vision (WACV), 2021

    2021

  23. [31]

    Dvqa: Understanding data visualizations via question answering

    Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. Dvqa: Understanding data visualizations via question answering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  24. [32]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In IEEE International Conference on Computer Vision (ICCV), 2023

    2023

  25. [33]

    Ocr-free document understanding transformer

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. Ocr-free document understanding transformer. In European Conference on Computer Vision (ECCV). Springer, 2022

    2022

  26. [34]

    Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning

    Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. In Findings of the Association for Computational Linguistics, 2021

    2021

  27. [35]

    Robust transfer learning with pretrained language models through adapters

    Wenjuan Han, Bo Pang, and Ying Nian Wu. Robust transfer learning with pretrained language models through adapters. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Process...

    2021

  28. [36]

    A systematic survey of prompt engineering on vision-language foundation models

    Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, V olker Tresp, and Philip Torr. A systematic survey of prompt engineering on vision-language foundation models. arXiv preprint arXiv:2307.12980, 2023. 12 A Supplementary Material ...

    2023 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.