Pith. sign in

REVIEW 3 major objections 7 minor 60 references

Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By scoring both modalities as one text problem, SCALE selects a 50K instruction-tuning subset that outperforms the full 500K training pool.

desk verdict A sensible data-curation recipe, but the '10% beats 100%' claim is a single-run artifact in the current write-up, with a possible A-OKVQA train/eval overlap. read the letter →

arxiv 2506.08429 v1 pith:R6I7W24C submitted 2025-06-10 cs.CV

Mingjie Xu , Andrew Estornell , Hongzheng Yang , Yuzhi Zhao , Zhaowei Zhu , Qi Xuan , Jiaheng Wei This is my paper
classification cs.CV
keywords vision-languagemodelsinstructiontuningdataselectionqualitymultimodalalignmentimagecaptioningsupervisedfine-tuningSCALE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SCALE is a data-selection pipeline for vision-language instruction tuning that claims a 10% subset can outperform training on the full dataset. It rates each image-text entry by first converting the image into general and task-specific captions, then having a text-only judge score clarity, relevance, and task rarity. The paper argues that unimodal quality scores alone are biased, because they overlook samples that matter for robustness and specific tasks, while text-rated captions provide a unified way to evaluate multimodal alignment. On eight benchmarks, a 7B vision-language model fine-tuned on the selected 50K subset averages 84.23 versus 83.94 for the full 500K pool. The largest gains appear in math reasoning and science knowledge, making the case that curation can replace scale.

What carries the argument

The load-bearing mechanism is the unified-modality scoring step at the center of SCALE. Each image is turned into two captions—a general description and a task-specific description selected from a benchmark-derived task taxonomy—so that the image disappears from the comparison and a text-only judge can rate clarity, relevance, and task rarity of the original text against the captions. These ratings are combined with image and text quality scores into a single weighted score $S = 0.2 S_I + 0.2 S_T + 0.6 S_{MM}$, and entries are ranked and the top 10% kept. The captions are what make cross-modal alignment measurable as a text task.

What would settle it

Take 100 randomly sampled entries from the 500K pool, have human annotators rate image-text alignment and textual clarity, and compare their rankings with SCALE's scores. If human agreement with SCALE's ordering is near zero, or if fine-tuning a different vision-language model on SCALE's 50K subset fails to beat the full-data baseline, the central claim would fail.

Watch

Extended reading notes

Core claim

The central claim is that a quality score formed from three components—image fidelity, text quality, and a multimodal alignment score computed entirely in text—can identify the instruction-tuning examples that matter, so that 10% of a 500K pool yields better fine-tuned performance than the whole pool. The multimodal score is obtained by generating a general caption and a task-specific caption for each image and asking a text-only judge to rate whether the original text is clear relative to those captions, relevant to them, and rare as a task. The paper reports that this pipeline (SCALE) beats random selection, image-only scoring, text-only scoring, their simple sum, and embedding-based image-text similarity on every benchmark, and that it outperforms the full-data baseline by 0.29 points on average, with the biggest single gain on MME (+1.1). It also argues two general lessons: unimodal evaluation can undervalue images or text that are essential for robustness, and generated captions are a viable bridge that turns image-text alignment into a text-modality problem.

Load-bearing premise

The load-bearing premise is that the ratings from the judge models and the captions from the vision-language model faithfully measure the quality of an image-text example, yet the paper reports no human agreement study, no caption-accuracy check, and no experiment with a different judge family.

Editorial extensions

If this is right

  • Fine-tuning a vision-language model on a curated 50K subset can match or surpass a 500K pool, so dataset size is not the main driver of post-training quality.
  • Math and science reasoning tasks benefit the most from clean, well-aligned examples, since the largest gains appear there.
  • Unimodal quality filters can harm performance by discarding blurry or terse examples that teach robustness and by demoting valid but short answers.
  • Caption-based scoring offers a low-cost text-only route to auditing image-text datasets, replacing expensive human alignment checks with language-model judgments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the same model families are used for scoring, captioning, and the final fine-tuned model, the selection may encode what one model family already understands well; testing on a different base model or with human-written captions would show how transferable the gain is.
  • The task-rarity score depends on a text judge recognizing a task as rare, so it may favor unusual wording rather than genuinely hard visual content; a distributional task-frequency measure could separate the two.
  • If caption generation is accurate enough, the same pipeline could be applied to raw web-scale image-text data before any instruction tuning, turning data selection into a general data-cleaning filter.
  • The reported gains are averages over eight benchmarks with a single base model, so the practical advantage of the 10% subset could be smaller or larger once judge and captioner choices change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. SCALE is a quality-driven data-selection pipeline for VLM instruction tuning. It evaluates each image-text entry in a 500K pool using unimodal image quality judged by Q-Sit, unimodal text quality judged by Qwen2.5-32B-Instruct, a predicted MMBench task label, general and task-specific captions generated by Qwen2.5-VL-7B, and a multimodal rating (clarity, relevance, task rarity) produced by Qwen2.5-32B-Instruct from the captions and original text. Entries are ranked by S = 0.2*SI + 0.2*ST + 0.6*SMM and the top 10% (~50K) is used for supervised fine-tuning of Qwen2.5-VL-7B. The paper compares against random selection, IQA-only, TQA-only, joint I+T QA, CLIP similarity, and retrieval baselines, and evaluates on A-OKVQA, CRPE-Exist, CRPE-Relation, LLaVA-Wild, MMBench-EN, MME, ScienceQA, and SeedBench. Table 4 reports a +0.29 average gain over full-data training (84.23 vs. 83.94), with the largest gains on MME (+1.1) and LLaVA-Wild (+0.7) and a -0.1 drop on CRPE-Exist. The paper interprets this as evidence that a 10% curated subset can surpass full-data training.

Significance. Should the headline result hold, the practical value is high: SFT on one tenth of a 500K pool would cut GPU hours and storage by roughly 90% while improving average benchmark performance, and the proposed unified-modality scoring would offer a concrete mechanism for multimodal data curation. I credit the authors for specifying fixed rather than learned selection weights (0.2/0.2/0.6), for validating task prediction against MMBench ground truth with >80% accuracy on all subtasks (Table 3), and for comparing against several reasonable baselines, including a three-seed random baseline and unimodal-only and retrieval baselines. The qualitative examples and ablations (Tables 4 and 5) are consistent with the paper's central narrative. The significance, however, is currently conditional: the central empirical claim rests on a single SFT run per condition, and the aggregate score in Table 4 uses an undocumented normalization for MME. If multi-seed results confirm the +0.29 average with reasonable confidence intervals, this would be a solid contribution; without them, the 'less is more' claim is not yet established.

major comments (3)
  1. [Section 4.2.2, Table 4] The claim that the SCALE-curated 10% subset outperforms full-data training is based on a single supervised fine-tuning run for the full-data and SCALE conditions. Only the Random Selection baseline is repeated (three times, mean reported). On a 7B VLM such as Qwen2.5-VL-7B, run-to-run variability on these benchmarks is typically larger than the reported +0.29 average difference, and the paper reports no confidence intervals, no training steps/epochs, and no statement that the two conditions consumed equal compute. Please report multiple seeds for the SCALE and full-data conditions, report exact training budgets (steps, epochs, batch size, learning rate), and state whether each benchmark difference is consistent across seeds.
  2. [Table 4, Avg Score column] The 'Avg Score' column mixes MME's raw score (approximately 2250) with percentage accuracies without stating the normalization. For the Full Data row, averaging the seven percentage columns gives approximately 84.46, while the reported Avg Score is 83.94; for Random Selection the seven-percentage average is approximately 83.00, while the reported Avg Score is 82.65. This implies an undocumented transformation involving MME. Because the +0.29 advantage over Full Data could plausibly be an artifact of this transformation, the paper must state the exact aggregation formula (or better, report MME separately and use a mean of percentage-normalized scores).
  3. [Sections 3.2.3-3.2.5 and Appendix A] The entire selection depends on the fidelity of Qwen2.5-VL-7B-generated captions and on the ratings of Qwen2.5-32B-Instruct for clarity, relevance, and task rarity. The paper reports no human agreement study, no caption-accuracy audit, and no experiment with a different captioner or judge, and the same model family is later fine-tuned, so the 'quality' score may reflect properties specific to that model family rather than intrinsic data quality. The limitation paragraph in Appendix A names bias risk qualitatively but does not address this validation gap. Please add (i) a human audit of, e.g., 100-200 selected and rejected entries, and (ii) a stability check in which the pipeline is rerun with a different captioning model; even a small-scale version of these experiments would substantially strengthen the central quality-based interpretation.
minor comments (7)
  1. [Section 3.2.1] The sentence beginning 'With gratefully acknowledge' contains a typo and should read 'We gratefully acknowledge'.
  2. [Table 2] Please clarify whether 'Size' denotes the original dataset size and 'Selected size' denotes the downsampled pool count; the current labeling makes it easy to confuse the 500K pool with the SCALE-selected 50K subset.
  3. [Section 3.2.3] The text mentions comparing image and text 'whether using Pearson Correlation or by an LLM', but no Pearson-correlation results are reported anywhere; either add this analysis or remove the mention.
  4. [Section 4.2.2, Table 4] The sentence 'the full data baseline uses the entire pool of 500,000 unfiltered multimodal examples, serving as an example for the impact of dataset size, only marginally outperforms the base model' is grammatically awkward and should be reworded.
  5. [Table 4 caption] The caption states that improvements of at least +0.5 points are highlighted in green, but the plain-text version does not show any highlighting; please ensure the final PDF includes the color coding and add a legend.
  6. [Tables 2 and 4] Please specify whether the A-OKVQA entries in the data pool are from the training split and whether the reported A-OKVQA accuracy is on the validation/test split; without this, readers cannot rule out data leakage.
  7. [Section 5] The sentence 'Empirically, VLMs fine-tuned on a 10% SCALE selected subset ... demonstrate the less-is-more hypothesis' has subject-verb disagreement and should read 'demonstrates'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SCALE's quality score is a fixed benchmark-independent composition; the less-is-more result is an empirical SFT comparison, not a self-referential fit.

full rationale

SCALE is an empirical data-selection pipeline rather than a derived prediction. The composite score is fixed in Section 3.2.5 as S = 0.2S_I + 0.2S_T + 0.6S_MM, with component scores produced by pre-trained judges (Q-Sit for images, Qwen2.5-32B-Instruct for text and multimodal ratings) and captions from Qwen2.5-VL-7B-Instruct; no component is fitted to the evaluation benchmarks or defined in terms of final benchmark scores. The central less-is-more claim is a direct SFT comparison in Table 4 against the full 500K pool, with all methods evaluated on external benchmarks through VLMEvalKit. The task-prediction module is independently checked against MMBench ground-truth labels (Table 3). The paper contains many self-citations in related work, but none are load-bearing: no uniqueness theorem or prior author result is invoked to force the selection choice. The most plausible concern is methodological rather than circular: the captioner, judge, and fine-tuned model all come from the Qwen2.5 family, so selection may encode that family's preferences, and Table 4 reports single SFT runs without error bars or a stated MME normalization. These are validity and robustness limitations, not reductions of the conclusion to its inputs. No circular step can be exhibited, so the score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper contributes an empirical pipeline, so the ledger is dominated by modeling assumptions about judge reliability and caption fidelity rather than free parameters fitting data. The only explicit hand-chosen numbers are the weighting coefficients and the 10% threshold.

free parameters (2)
  • Selection weights = 0.2 (image), 0.2 (text), 0.6 (multimodal)
    Hand-chosen fixed weights in S = 0.2*SI + 0.2*ST + 0.6*SMM; no sensitivity analysis or grid search reported, yet they determine which entries are kept.
  • Selection threshold (top-k) = 10% (about 50K of 500K)
    The top 10 percentile is chosen; no experiment varies this ratio, so the 'less than 10%' framing is an untested hyperparameter.
assumptions (4)
  • domain assumption Qwen2.5-32B-Instruct provided ratings for informativeness, complexity, completeness, clarity, relevance, and task rarity are valid proxies for data quality.
    Section 3.2.1 and 3.2.4; the whole pipeline depends on the judge's ratings, and no human agreement study is provided.
  • domain assumption Qwen2.5-VL-7B generated general and task-specific captions that faithfully represent visual content.
    Section 3.2.3; caption fidelity is the bridge for unified modality scoring; no caption correctness evaluation is reported.
  • domain assumption Fine-tuning on the selected 10% for a comparable number of steps/epochs to the full-data baseline is implicitly assumed by the comparison.
    Section 4.1; training duration is not reported, so the comparison may confound data quality with optimization effort.
  • domain assumption Qwen2.5-VL-7B is a representative base model; results generalize to other VLM architectures.
    All experiments use Qwen2.5-VL-7B as the base and the captioner; no other family is tested, supporting the selection-bias concern.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring." pith.science (2026). https://pith.science/paper/R6I7W24C

@misc{pith2026250608429,
  author       = {Pith},
  title        = {Pith review of: Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R6I7W24C}},
  note         = {Machine review of arXiv:2506.08429}
}
read the original abstract

The application of visual instruction tuning and other post-training techniques has significantly enhanced the capabilities of Large Language Models (LLMs) in visual understanding, enriching Vision-Language Models (VLMs) with more comprehensive visual language datasets. However, the effectiveness of VLMs is highly dependent on large-scale, high-quality datasets that ensure precise recognition and accurate reasoning. Two key challenges hinder progress: (1) noisy alignments between images and the corresponding text, which leads to misinterpretation, and (2) ambiguous or misleading text, which obscures visual content. To address these challenges, we propose SCALE (Single modality data quality and Cross modality Alignment Evaluation), a novel quality-driven data selection pipeline for VLM instruction tuning datasets. Specifically, SCALE integrates a cross-modality assessment framework that first assigns each data entry to its appropriate vision-language task, generates general and task-specific captions (covering scenes, objects, style, etc.), and evaluates the alignment, clarity, task rarity, text coherence, and image clarity of each entry based on the generated captions. We reveal that: (1) current unimodal quality assessment methods evaluate one modality while overlooking the rest, which can underestimate samples essential for specific tasks and discard the lower-quality instances that help build model robustness; and (2) appropriately generated image captions provide an efficient way to transfer the image-text multimodal task into a unified text modality.

Figures

Figures reproduced from arXiv: 2506.08429 by the authors.

Figure 3
Figure 3. Unimodal quality assessment stage illustration. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Illustration of the visual understanding stage and the multimodal quality assessment stage. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 4
Figure 4. Data task prediction stage illustration. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Ranking and selecting data stage illustration. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 9
Figure 9. Figure 9: Examples of high-rated multimodal data entries. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Additional qualitative results illustrating multimodal rating score. Text highlighted in red [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Additional qualitative results showing image-only and text-only quality scores. These [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Model responses from the full data baseline and [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 28 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. a...

    arXiv 2025

  3. [3]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. InEuropean Conference on Computer Vision, pages 370–387. Springer, 2025

    work page 2025

  4. [4]

    Sharegpt4video: Improving video understanding and generation with better captions.Advances in Neural Information Processing Systems, 37:19472– 19495, 2024

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions.Advances in Neural Information Processing Systems, 37:19472– 19495, 2024

    work page 2024

  5. [5]

    Learning with instance-dependent label noise: A sample sieve approach.arXiv preprint arXiv:2010.02347, 2020

    Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance-dependent label noise: A sample sieve approach.arXiv preprint arXiv:2010.02347, 2020

    arXiv 2010

  6. [6]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  7. [7]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM International Conference on Multimedia, pages 11198–11201, 2024

    2024

  8. [8]

    Vector-valued property elicitation

    Rafael Frongillo and Ian A Kash. Vector-valued property elicitation. InConference on Learning Theory, pages 710–727. PMLR, 2015

    work page 2015

Show all 60 references
  1. [9]

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.arXiv preprint arXiv:2209.07858, 2022

    Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.arXiv preprint arXiv:2209.07858, 2022

    2022 arXiv

  2. [10]

    Strictly proper scoring rules, prediction, and estimation

    Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359–378, 2007

    2007

  3. [11]

    Scaling laws and interpretability of learning from repeated data.arXiv preprint arXiv:2205.10487, 2022

    Danny Hernandez, Tom Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Tom Henighan, Tristan Hume, et al. Scaling laws and interpretability of learning from repeated data.arXiv preprint arXiv:2205.10487, 2022

    2022 arXiv

  4. [12]

    An information theoretic framework for designing information elicitation mechanisms that reward truth-telling.ACM Transactions on Economics and Computation (TEAC), 7(1):1–33, 2019

    Yuqing Kong and Grant Schoenebeck. An information theoretic framework for designing information elicitation mechanisms that reward truth-telling.ACM Transactions on Economics and Computation (TEAC), 7(1):1–33, 2019

    2019

  5. [13]

    Deduplicating training data makes language mod- els better

    Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. Deduplicating training data makes language mod- els better. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume...

    2022

  6. [14]

    Seed-bench: Benchmarking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13299–13308, 2024

    2024

  7. [15]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  8. [16]

    BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors,Proceedings of...

    2022

  9. [17]

    Align before fuse: Vision and language representation learning with momentum distillation.Advances in neural information processing systems, 34:9694–9705, 2021

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation.Advances in neural information processing systems, 34:9694–9705, 2021

    2021

  10. [18]

    Georeasoner: Geo-localization with reasoning in street views using a large vision-language model

    Ling Li, Yu Ye, Bingchuan Jiang, and Wei Zeng. Georeasoner: Geo-localization with reasoning in street views using a large vision-language model. InInternational Conference on Machine Learning, pages 29222–29233. PMLR, 2024

    2024

  11. [19]

    A survey of multimodel large language models

    Zijing Liang, Yanjie Xu, Yifan Hong, Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. A survey of multimodel large language models. InProceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering, pages 405–409, 2024

    2024

  12. [20]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26296–26306, June 2024

    2024

  13. [21]

    Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    2024

  14. [22]

    Automatic dataset construction (adc): Sample collection, data curation, and beyond.arXiv preprint arXiv:2408.11338, 2024

    Minghao Liu, Zonglin Di, Jiaheng Wei, Zhongruo Wang, Hengxiang Zhang, Ruixuan Xiao, Haoyu Wang, Jinlong Pang, Hao Chen, Ankit Shah, et al. Automatic dataset construction (adc): Sample collection, data curation, and beyond.arXiv preprint arXiv:2408.11338, 2024

    2024 arXiv

  15. [23]

    Classification with noisy labels by importance reweighting

    Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 38(3):447–461, 2015

    2015

  16. [24]

    Peer loss functions: Learning from noisy labels without knowing noise rates

    Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. InInternational conference on machine learning, pages 6226–6236. PMLR, 2020

    2020

  17. [25]

    Auditing for federated learning: A model elicitation approach

    Yang Liu, Rixing Lou, and Jiaheng Wei. Auditing for federated learning: A model elicitation approach. InProceedings of the Fifth International Conference on Distributed Artificial Intelligence, pages 1–9, 2023

    2023

  18. [26]

    Incentives for federated learning: A hypothesis elicitation approach

    Yang Liu and Jiaheng Wei. Incentives for federated learning: A hypothesis elicitation approach. arXiv preprint arXiv:2007.10596, 2020

    2007 arXiv

  19. [27]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233. Springer, 2025

    2025

  20. [28]

    Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning

    Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics ...

    2021

  21. [29]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–252...

    2022

  22. [30]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors,Findings of the Association for Computationa...

    2022

  23. [31]

    Infographicvqa

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawa- har. Infographicvqa. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706, 2022

    2022

  24. [32]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209, 2021

    2021

  25. [33]

    Measures of the value of information.Proceedings of the National Academy of Sciences, 42(9):654–655, 1956

    John McCarthy. Measures of the value of information.Proceedings of the National Academy of Sciences, 42(9):654–655, 1956

    1956

  26. [34]

    Mm1: methods, analysis and insights from multimodal llm pre-training

    Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Anton Belyi, et al. Mm1: methods, analysis and insights from multimodal llm pre-training. InEuropean Conference on Computer Vision, pages 304–323....

    2025

  27. [35]

    Learning with noisy labels.Advances in neural information processing systems, 26, 2013

    Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels.Advances in neural information processing systems, 26, 2013

    2013

  28. [36]

    GPT-4o system card.arXiv preprint arXiv:2410.21276, 2024

    OpenAI. GPT-4o system card.arXiv preprint arXiv:2410.21276, 2024. Submitted on 25 Oct 2024

    2024 arXiv

  29. [37]

    Token cleaning: Fine-grained data selection for llm supervised fine-tuning.arXiv preprint arXiv:2502.01968, 2025

    Jinlong Pang, Na Di, Zhaowei Zhu, Jiaheng Wei, Hao Cheng, Chen Qian, and Yang Liu. Token cleaning: Fine-grained data selection for llm supervised fine-tuning.arXiv preprint arXiv:2502.01968, 2025

    2025

  30. [38]

    Improving data efficiency via curating LLM-driven rating systems

    Jinlong Pang, Jiaheng Wei, Ankit Shah, Zhaowei Zhu, Yaxuan Wang, Chen Qian, Yang Liu, Yujia Bao, and Wei Wei. Improving data efficiency via curating LLM-driven rating systems. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  31. [39]

    Deep learning on a data diet: Finding important examples early in training.Advances in neural information processing systems, 34:20596–20607, 2021

    Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training.Advances in neural information processing systems, 34:20596–20607, 2021

    2021

  32. [40]

    Estimating training data influence by tracing gradient descent.Advances in Neural Information Processing Systems, 33:19920–19930, 2020

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent.Advances in Neural Information Processing Systems, 33:19920–19930, 2020

    2020

  33. [41]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  34. [42]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1–67, 2020

    2020

  35. [43]

    A-okvqa: A benchmark for visual question answering using world knowledge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowledge. In European conference on computer vision, pages 146–162. Springer, 2022

    2022

  36. [44]

    On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm

    Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9390–9399, 2024

    2024

  37. [45]

    Data pruning via moving-one-sample-out.Advances in Neural Information Processing Systems, 36, 2024

    Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, and Xiaojuan Qi. Data pruning via moving-one-sample-out.Advances in Neural Information Processing Systems, 36, 2024. 12

    2024

  38. [46]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processin...

    2024

  39. [47]

    The all-seeing project v2: Towards general relation comprehension of the open world

    Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-seeing project v2: Towards general relation comprehension of the open world. InEuropean Conference on Computer Vision, pages 471–490, 2024

    2024

  40. [48]

    Sample elicitation

    Jiaheng Wei, Zuyue Fu, Yang Liu, Xingyu Li, Zhuoran Yang, and Zhaoran Wang. Sample elicitation. InInternational Conference on Artificial Intelligence and Statistics, pages 2692–

  41. [49]

    To smooth or not? when label smoothing meets noisy labels

    Jiaheng Wei, Hangyu Liu, Tongliang Liu, Gang Niu, Masashi Sugiyama, and Yang Liu. To smooth or not? when label smoothing meets noisy labels. InInternational Conference on Machine Learning, pages 23589–23614. PMLR, 2022

    2022

  42. [50]

    Measuring and reducing llm hallucination without gold-standard answers.arXiv preprint arXiv:2402.10412, 2024

    Jiaheng Wei, Yuanshun Yao, Jean-Francois Ton, Hongyi Guo, Andrew Estornell, and Yang Liu. Measuring and reducing llm hallucination without gold-standard answers.arXiv preprint arXiv:2402.10412, 2024

    2024 arXiv

  43. [51]

    Learning with noisy labels revisited: A study using real-world human annotations

    Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. InInternational Conference on Learning Representations, 2022

    2022

  44. [52]

    Icm-assistant: Instruction-tuning multimodal large language models for rule-based explainable image content moderation

    Mengyang Wu, Yuzhi Zhao, Jialun Cao, Mingjie Xu, Zhongming Jiang, Xuehui Wang, Qinbin Li, Guangneng Hu, Shengchao Qin, and Chi-Wing Fu. Icm-assistant: Instruction-tuning multimodal large language models for rule-based explainable image content moderation. In Proceedings of the...

    2025

  45. [53]

    Less: Selecting influential data for targeted instruction tuning

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. InInternational Conference on Machine Learning, pages 54104–54132. PMLR, 2024

    2024

  46. [54]

    LESS: Selecting influential data for targeted instruction tuning

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, edit...

    2024

  47. [55]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  48. [56]

    Robust noisy correspondence learning with equivariant similarity consistency

    Yuchen Yang, Likai Wang, Erkun Yang, and Cheng Deng. Robust noisy correspondence learning with equivariant similarity consistency. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17700–17709, 2024

    2024

  49. [57]

    Teaching lmms for image quality scoring and interpreting.arXiv preprint arXiv:2503.09197, 2025

    Zicheng Zhang, Haoning Wu, Ziheng Jia, Weisi Lin, and Guangtao Zhai. Teaching lmms for image quality scoring and interpreting.arXiv preprint arXiv:2503.09197, 2025

    2025 arXiv

  50. [58]

    Mitigating noisy correspondence by geometrical structure consistency learning

    Zihua Zhao, Mengxi Chen, Tianjie Dai, Jiangchao Yao, Bo Han, Ya Zhang, and Yanfeng Wang. Mitigating noisy correspondence by geometrical structure consistency learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27381–27390, 2024. 13

    2024

  51. [59]

    Unmasking and improving data credibility: A study with datasets for training harmless language models

    Zhaowei Zhu, Jialu Wang, Hao Cheng, and Yang Liu. Unmasking and improving data credibility: A study with datasets for training harmless language models. InThe Twelfth International Conference on Learning Representations

  52. [60]

    Spatial Relationship

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neural Information Proces...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.