Pith. sign in

REVIEW 4 major objections 6 minor 51 references

HAIBU-ReMUD: Reasoning Multimodal Ultrasound Dataset and Model Bridging to General Specific Domains

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Auto-generated ultrasound data lets a 7B model beat GPT-4o and Gemini.

desk verdict Useful dataset pipeline, but the headline accuracy gaps are not yet trustworthy given train/test source overlap. read the letter →

arxiv 2506.07837 v1 pith:4CPHGOVP submitted 2025-06-09 cs.AI

Shijie Wang , Yilun Zhang , Zeyu Lai , Dexing Kong This is my paper
classification cs.AI
keywords multimodallargelanguagemodelsultrasounddomain-specificdatasetvisualquestionansweringsupervisedfine-tuningreasoningbudgetforcingmedicalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a fully automated pipeline can turn scattered ultrasound textbooks, guidelines, reports, and images into a supervised fine-tuning dataset of image-question-thinking-answer quadruplets, and that fine-tuning a small open multimodal model on that data is enough to beat much larger general-purpose models on ultrasound knowledge and diagnosis tasks. The resulting model, ReMUD-7B, built on Qwen2.5-VL-7B-Instruct, scores 80.1 on the text-only USTQ-Knowledge set versus 78.3 for GPT-4o, and 90.1 overall on the image-plus-text UVQA-Diagnosis set versus 66.6 for Gemini-2.5-Pro-Exp. If correct, this shows that domain-specific expertise can be injected into small models cheaply, without manual annotation and without a domain-specific pretraining corpus. It also suggests the same data-generation recipe could transfer to other specialty domains whose materials exist as PDFs and images.

What carries the argument

The load-bearing mechanism is the quadruplet generation pipeline: it converts a PDF page into a JSON list of bounding boxes (coordinates, label, caption, and related context) detected by Qwen2.5-VL, and then converts each boxed image plus its surrounding text into a question, thinking trace, and answer triplet via GPT-4o or Gemini-2.0-Flash-Thinking-Exp API calls with prompts that demand answers be traceable to the provided material. This machinery replaces manual expert annotation with automated extraction, which is what makes a 45,000-item supervised fine-tuning dataset affordable. The second piece of machinery is budget forcing: when a generated chain of thought is too short, the model is forced to insert a wait tag to continue reasoning, and overlong chains are truncated, so test-time compute stays in a beneficial window.

What would settle it

Run ReMUD-7B on an independently constructed ultrasound benchmark, such as questions written by practicing sonographers from hospital cases or an existing exam bank not used in training, and compare its accuracy with GPT-4o and Gemini; if the margin collapses to near zero on that held-out set, the claimed domain gain is largely an artifact of training and test distribution overlap. A cheaper check is to compute n-gram or embedding overlap between ReMUD's training questions and the test questions and show that reported test questions are near-duplicates of training items.

Watch

Extended reading notes

Core claim

The central discovery is that a small 7-billion-parameter open multimodal model, tuned only on automatically generated ultrasound instruction data, can surpass far larger closed general-domain models on domain-specific multiple-choice exams. The paper's pipeline first uses Qwen2.5-VL's object-grounding ability to detect images on PDF pages and produce bounding boxes with captions and context, then asks the GPT-4o or Gemini-2.0-Flash-Thinking-Exp APIs to write questions, thinking traces, and answers tied strictly to that page content, and finally saves each cropped image with its triplet to build the ReMUD dataset. Fine-tuning Qwen2.5-VL-7B-Instruct on this 45,000-plus item dataset with a mix of reasoning and non-reasoning data, plus a budget-forcing wait tag at inference time, yields ReMUD-7B, which outperforms all tested general multimodal models on both of the paper's constructed test sets.

Load-bearing premise

The comparison assumes that USTQ-Knowledge and UVQA-Diagnosis are valid, unbiased measures of ultrasound ability that do not overlap with the ReMUD training distribution; both test sets are built from the same types of public ultrasound materials and the same API models used to create the training data, and no train/test deduplication is reported.

Editorial extensions

If this is right

  • If the central claim is correct, automatically generated domain-specific instruction data can close the domain gap for small multimodal models without manual annotation or pretraining.
  • The same pipeline, applied to other PDF- and image-based specialties such as radiology, pathology, or ophthalmology, could produce comparable supervised fine-tuning datasets.
  • Budget forcing provides a cheap test-time scaling lever: accuracy rises when thinking traces are lengthened up to a point, then falls with over-long traces, as shown in the paper's Figure 6.
  • The released dataset, code, and model parameters allow others to reproduce and extend the result without relying on proprietary training data.
  • A small open model fine-tuned this way could serve as a practical alternative to closed general-domain APIs in settings where data privacy or cost matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The reported accuracy gaps may overstate the true domain gain, because the test sets are generated from the same types of public ultrasound materials and the same API models used to create the training data, and no train/test deduplication is reported; an independent, clinician-authored ultrasound benchmark would be needed to confirm how much transfers.
  • Editorial inference: Because the pipeline uses GPT-4o and Gemini to create the training data and also to score data quality, the dataset inherits the biases of those two systems; using the same systems later to evaluate the model creates a feedback loop that could inflate apparent progress.
  • Editorial inference: A testable implication of the paper's logic is that the pipeline transfers to other PDF-rich specialties, and that the performance gain over general models should shrink when the specialty's public materials are already well represented in the base model's pretraining data.
  • Editorial inference: The budget-forcing curve in Figure 6 implies there is an optimal thinking-trace length per question; predicting that optimum per item difficulty could yield further accuracy gains without extra training, a direction the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes an automated pipeline for generating multimodal reasoning supervised fine-tuning (SFT) data in a specific domain, instantiated for medical ultrasound. The authors construct the ReMUD dataset, containing over 45,000 text QA and image-text VQA items with reasoning and non-reasoning splits, generated from publicly available ultrasound books, guidelines, papers, and datasets via OCR, Qwen2.5-VL grounding, and GPT-4o/Gemini API calls. They fine-tune Qwen2.5-VL-7B-Instruct on ReMUD with budget forcing to produce ReMUD-7B, and evaluate it on two self-built benchmarks, USTQ-Knowledge (text) and UVQA-Diagnosis (image-text), reporting that ReMUD-7B outperforms general-domain MLLMs such as GPT-4o and Gemini-2.5-Pro-Exp. The paper also reports test-time scaling behavior with budget forcing and releases dataset, code, and model weights.

Significance. If the evaluation is valid, the contribution is practically significant: it demonstrates a scalable, low-cost recipe for creating domain-specific multimodal instruction data without manual annotation, and shows that a 7B open-weight model can beat much larger closed models on a narrow medical task. The open release of the dataset, generation code, and model parameters is a concrete asset for reproducibility and for future domain adaptation research. The budget-forcing/ReMUD analysis is also a useful empirical data point. However, the central performance claim rests entirely on the validity of two self-built test sets that appear to overlap in source material with the training data; until this is resolved, the headline numbers cannot be interpreted as evidence of general ultrasound competence.

major comments (4)
  1. [Section III-B.3, III-C, V-B] The evaluation sets are not shown to be independent of the training distribution. UVQA-Diagnosis is compiled from the Chinese Ultrasound Report Dataset [46], which is cited in Section III-B.3 as a training source transformed into triplets. USTQ-Knowledge is generated from "ultrasound books, guidelines, and papers" using GPT-4o, which are the same source categories and the same API model used in Section III-A for training data generation. The deduplication described in Section III-C (n-gram n=12 and sentence transformer) is stated only to ensure uniqueness within the test data, not to remove overlap between training and test sets. No train/test overlap analysis is reported. Consequently, the reported gaps (80.1 vs 78.3 on USTQ-Knowledge; 90.1 vs 66.6 on UVQA-Diagnosis) could be explained by memorization of training items rather than by domain competence. The authors should (a) quantify overlap between training and test sets at the image, report, and question-paraphrase levels, (b) remove or quarantine overlapping/near-duplicate items, and (c) re-report all numbers after this cleaning.
  2. [Section IV-C, V-C] The pass@1 metric is defined inconsistently. Equation (1) defines pass@1 as (1/k) * sum(p_i), which is the mean correctness over k generated responses. Section V-C, however, describes pass@1 as "at least one completely correct answer across multiple attempts." These are different quantities; for k=4 and a typical per-sample accuracy p, the "at least one correct" estimator 1 - (1-p)^k is substantially larger than the mean p. The authors must state which estimator was actually used for all models in Table II and correct either the formula or the prose. If the intended estimator is the probability of at least one correct response, the correct expression is pass@1 = 1 - prod(1-p_i).
  3. [Section V-B, Table II] The USTQ-Knowledge advantage of ReMUD-7B (80.1) over GPT-4o (78.3) is only 1.8 percentage points on 371 questions, which corresponds to roughly 7 questions. At this sample size, a binary proportion near 0.8 has a 95% confidence interval of about ±4 percentage points, so the observed difference is well within sampling error. Since the USTQ-Knowledge result is part of the central "outperforms general-domain MLLMs" claim, the authors should report confidence intervals or a significance test, or restrict the claim to the UVQA-Diagnosis task where the gap is much larger.
  4. [Section V-C] The evaluation protocol for the baselines is underspecified. The paper says that "all baseline models adopted identical sampling strategies and evaluation protocols," but it does not report the exact prompt given to each model, whether the thinking-trace field was included in the prompt, how final multiple-choice answers were extracted from free-form generated responses, or how responses that failed to follow the requested format were scored. This information is needed to assess whether the comparison in Table II is fair and to reproduce the numbers. The authors should provide the full evaluation prompts and answer-extraction code in the supplementary material.
minor comments (6)
  1. [Abstract and Introduction] The word "vedio" appears in the abstract and should be corrected to "video."
  2. [Section III-C, V-C] The second test set name is spelled inconsistently: "UVQA-Diagonis," "UVQA-diagnos," and "UVQA-Diagonsis" appear in different places; the intended name is "UVQA-Diagnosis."
  3. [Section III-C, Table I] The text states that ReMUD contains "1k+ test data," but Table I lists 371 + 386 = 757 test items; please correct this count or provide the actual total.
  4. [Section IV-C, V-C] The phrase "multi-choice" is used; the standard term is "multiple-choice."
  5. [Title and Abstract] The acronym "HAIBU" in the title is not explained anywhere in the paper; the authors should define it or remove it from the title.
  6. [Section V-D, Figure 6] Figure 6 is described as showing test-time scaling, but the text does not define the axes or specify the budget-forcing protocol used (e.g., the length thresholds for inserting "Wait" versus truncating). Please clarify so the experiment is reproducible.

Circularity Check

1 steps flagged · score 6.0 of 10

UVQA-Diagnosis is built from the same ultrasound report dataset [46] used to form ReMUD training triplets, and no train/test deduplication is reported, so the headline comparison partly measures fit to the training distribution.

  1. fitted input called prediction [Section III-B.3 (Diversity); Section V-B (UVQA-Diagnosis); Table I caption.]
    "we gather publicly accessible ultrasound datasets from the Internet, such as those for breast nodule benign and malignant classifications and ultrasound report generation [46], [47], and transform them into triplets for better utilization. ... UVQA-Diagnosis is compiled based on the publicly available dataset, Chinese Ultrasound Report Dataset [46], which contains 7k+ ultrasound reports with 30k+ images. ... ReMUD consists of four train dataset and two test dataset. Both of them are generated from open-sourced ultrasound materials."

    ReMUD-7B is supervised-fine-tuned on ReMUD, whose construction includes transforming the report dataset [46] into training triplets. The same [46] dataset is then used as the basis of the UVQA-Diagnosis test set. The paper only describes n-gram and sentence-transformer deduplication to ensure uniqueness within the test data ('ensures the uniqueness and quality of the test data'); no train/test deduplication relative to [46] is described. Consequently the reported UVQA-Diagnosis gap (ReMUD-7B 90.1 vs Gemini-2.5-Pro-Exp 66.6, Table II) can be explained substantially by the model reproducing the distribution it was trained on, rather than by a general ultrasound capability. The evaluation 'prediction' is partly a measurement of training-source fit.

full rationale

The paper is an empirical dataset-and-finetuning contribution, so the circularity question concerns whether the evaluation benchmarks are independent of the training distribution. The strongest reduction is in the UVQA-Diagnosis loop: Section III-B.3 says the authors gather open ultrasound datasets, including ultrasound report generation [46], and transform them into training triplets; Section V-B says UVQA-Diagnosis is compiled from the same Chinese Ultrasound Report Dataset [46]; and Table I's caption concedes that both train and test sets are generated from open-sourced ultrasound materials. The only deduplication described is within each test set, not between train and test. Thus the headline claim that ReMUD-7B beats general-domain MLLMs on UVQA-Diagnosis is partially circular. A similar but less specifically documented risk applies to USTQ-Knowledge, which is generated from ultrasound books/guidelines/papers by the same API models used to generate the training corpus. There is no load-bearing self-citation chain or imported uniqueness theorem; the issue is benchmark dependence, not citation circularity. Because the tests are not identical to the training items by construction and the baselines are evaluated under the same protocol, the paper is not entirely circular, but the central empirical claim is not independently secured as written. Score 6: one key 'prediction' (UVQA-Diagnosis superiority) reduces in part to fit on the training source.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

No invented entities. The free parameters are hand-chosen training and evaluation settings; there is no derivation that depends on fitted constants. The axioms are domain assumptions about data quality and test validity; the third is the most fragile because train/test contamination is not quantified.

free parameters (6)
  • Learning rate = 3e-5
    Hand-chosen; no sweep reported in Section V-A.
  • Training epochs = 3
    Hand-chosen; no early stopping or ablation reported.
  • Sampling temperature = 0.6
    Used for evaluation generations; affects pass@1 estimates.
  • Top-p = 0.7
    Evaluation sampling parameter.
  • Number of samples k = 4
    Used in pass@1 evaluation; number of generated responses per question.
  • n-gram n = 12
    Deduplication threshold, hand-chosen.
assumptions (3)
  • domain assumption Public ultrasound textbooks, guidelines, papers, and the Chinese Ultrasound Report Dataset are reliable sources of professional knowledge and labels.
    The whole dataset is crawled from these public sources; errors in sources propagate into training and test data.
  • domain assumption GPT-4o and Gemini-2.0-Flash-Thinking-Exp generate sufficiently accurate questions, answers, and thinking traces without item-level expert review of all samples.
    Only a multiple-choice question bank subset received doctor review; most generated triplets are kept after API-based scoring.
  • ad hoc to paper The two self-built test sets are valid, non-overlapping measures of ultrasound capability.
    The test sets are constructed by the authors from the same source types and generation pipeline as training; no train/test overlap check is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HAIBU-ReMUD: Reasoning Multimodal Ultrasound Dataset and Model Bridging to General Specific Domains." pith.science (2026). https://pith.science/paper/4CPHGOVP

@misc{pith2026250607837,
  author       = {Pith},
  title        = {Pith review of: HAIBU-ReMUD: Reasoning Multimodal Ultrasound Dataset and Model Bridging to General Specific Domains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CPHGOVP}},
  note         = {Machine review of arXiv:2506.07837}
}
read the original abstract

Multimodal large language models (MLLMs) have shown great potential in general domains but perform poorly in some specific domains due to a lack of domain-specific data, such as image-text data or vedio-text data. In some specific domains, there is abundant graphic and textual data scattered around, but lacks standardized arrangement. In the field of medical ultrasound, there are ultrasonic diagnostic books, ultrasonic clinical guidelines, ultrasonic diagnostic reports, and so on. However, these ultrasonic materials are often saved in the forms of PDF, images, etc., and cannot be directly used for the training of MLLMs. This paper proposes a novel image-text reasoning supervised fine-tuning data generation pipeline to create specific domain quadruplets (image, question, thinking trace, and answer) from domain-specific materials. A medical ultrasound domain dataset ReMUD is established, containing over 45,000 reasoning and non-reasoning supervised fine-tuning Question Answering (QA) and Visual Question Answering (VQA) data. The ReMUD-7B model, fine-tuned on Qwen2.5-VL-7B-Instruct, outperforms general-domain MLLMs in medical ultrasound field. To facilitate research, the ReMUD dataset, data generation codebase, and ReMUD-7B parameters will be released at https://github.com/ShiDaizi/ReMUD, addressing the data shortage issue in specific domain MLLMs.

Figures

Figures reproduced from arXiv: 2506.07837 by the authors.

Figure 1
Figure 1. Statistical overview of ReMUD. ReMUD incorporates ultrasound knowledge across various anatomical regions, enabling the model to acquire comprehensive and extensive ultrasound expertise. ultrasound domain supervised fine-tuning (SFT) dataset ReMUD, which contains 45,000+ reasoning and non￾reasoning SFT QA and VQA data (see [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Flowchart illustrating the process of visual question answering (VQA) data generation, incorporating data collection, image grounding, VQA creation, and Data Cleaning. ability to answer the question. For instance, Path-VQA [32] is obtained by extracting pathology images and captions from publicly-available textbooks and digital libraries, using natural language processing techniques to generate question￾answer pairs… view at source ↗
Figure 3
Figure 3. An example for VQA generation. By leveraging bounding box and page image, GPT-4o or Gemini-2.0-Flash-Thinking-Exp APIs can generate required data. • After successfully converting the unformatted data into text format, we harnessed the power of the GPT-4o or Gemini-2.0-Flash-Thinking-Exp APIs. These advanced language model APIs were used to generate triplets consisting of a question, a thinking trace, and a generated… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Messages we use to prompt GPT-4o to generate VQA instruction-following data. Different adjustments to the prompts may be made when handling different data. See [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Budget-forcing with Qwen2.5-VL-Ultra-7B. Add the marker ”Wait, ” to the chain of thought to prompt the model to think again. After the model had incorrect thinking, it thought again and corrected the diagnostic result. and UVQA-Diagonis. USTQ-Knowledge automatically ge…
Figure 6
Figure 6. Figure 6: Test-time scaling with ReMUD-7B. We evaluate ReMUD-7B on UVQA-Diagnosis and USTQ-Knowledge using pass@1 and varying test￾time compute length. each of these three anatomical regions. Specifically, we set the temperature parameter at 0.6 and the top-p parameter at 0.7 to…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 44 canonical work pages

  1. [46]

    Ultrasound report genera- tion with cross-modality feature alignment via unsupervised guidance,

    J. Li, T. Su, B. Zhao, F. Lv, Q. Wang, et al., “Ultrasound report genera- tion with cross-modality feature alignment via unsupervised guidance,” 2024

    work page 2024

  2. [1]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, Y . J. Lee, et al., “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34892– 34916, 2023

    work page 2023

  3. [2]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, et al., “Gpt-4o system card,” 2024

    work page 2024

  4. [3]

    How to bridge the gap between modalities: Survey on multimodal large language model,

    S. Song, X. Li, S. Li, S. Zhao, J. Yu, et al., “How to bridge the gap between modalities: Survey on multimodal large language model,” 2025

    work page 2025

  5. [4]

    A survey of multimodel large language models,

    Z. Liang, Y . Xu, Y . Hong, et al., “A survey of multimodel large language models,” in Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering, pp. 405–409, 2024

    work page 2024

  6. [5]

    The revolu- tion of multimodal large language models: A survey,

    D. Caffagni, F. Cocchi, L. Barsellotti, N. Moratelli, et al., “The revolu- tion of multimodal large language models: A survey,” 2024

    work page 2024

  7. [6]

    Multimodal large language models in health care: applications, challenges, and future outlook,

    R. AlSaad, A. Abd-Alrazaq, S. Boughorbel, et al., “Multimodal large language models in health care: applications, challenges, and future outlook,” Journal of medical Internet research, vol. 26, p. e59505, 2024

    work page 2024

  8. [7]

    Hallucination of multimodal large language models: A survey,

    Z. Bai, P. Wang, T. Xiao, T. He, et al., “Hallucination of multimodal large language models: A survey,” arXiv preprint arXiv:2404.18930, 2024

    arXiv 2024

Show all 51 references
  1. [8]

    Qwen2.5-vl technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, et al., “Qwen2.5-vl technical report,” 2025

    2025

  2. [9]

    Google. gemini 2.0 flash thinking mode (gemini-2.0-flash-thinking-exp- 1219),

    “Google. gemini 2.0 flash thinking mode (gemini-2.0-flash-thinking-exp- 1219),” december 2024

    2024

  3. [10]

    Kimi-vl technical report,

    K. Team, A. Du, B. Yin, B. Xing, B. Qu, et al., “Kimi-vl technical report,” 2025

    2025

  4. [11]

    Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understand- ing,

    Z. Wu, X. Chen, Z. Pan, X. Liu, et al., “Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understand- ing,” 2024

    2024

  5. [12]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,

    J. Zhu, W. Wang, Z. Chen, Z. Liu, et al., “Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,” 2025

    2025

  6. [13]

    Gemma 3 technical report,

    G. Team, A. Kamath, J. Ferret, S. Pathak, et al., “Gemma 3 technical report,” 2025

    2025

  7. [14]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  8. [15]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, et al., “Learning transferable visual models from natural language supervision,” 2021

    2021

  9. [16]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, et al., “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022, 2021

    2021

  10. [17]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, et al., “Llama: Open and efficient foundation language models,” 2023

    2023

  11. [18]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, et al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” March 2023

    2023

  12. [19]

    Qwen2.5 technical report,

    Qwen, :, A. Yang, B. Yang, B. Zhang, et al., “Qwen2.5 technical report,” 2025

    2025

  13. [20]

    Qwen2 technical report,

    A. Yang, B. Yang, B. Hui, et al., “Qwen2 technical report,” 2024

    2024

  14. [21]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” 2023

    2023

  15. [22]

    Towards multimodal in-context learning for vision & language models,

    S. Doveh, S. Perek, M. J. Mirza, et al., “Towards multimodal in-context learning for vision & language models,” arXiv preprint arXiv:2403.12736, 2024

    2024 arXiv

  16. [23]

    Multimodal chain-of-thought reasoning in language models,

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola, “Multimodal chain-of-thought reasoning in language models,” 2024

    2024

  17. [24]

    Visual programming: Compositional visual reasoning without training,

    T. Gupta and A. Kembhavi, “Visual programming: Compositional visual reasoning without training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14953– 14962, 2023

    2023

  18. [25]

    Mm-narrator: Narrating long-form videos with multimodal in-context learning,

    C. Zhang, K. Lin, Z. Yang, et al., “Mm-narrator: Narrating long-form videos with multimodal in-context learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13647–13657, 2024

    2024

  19. [26]

    Mia- bench: Towards better instruction following evaluation of multimodal llms,

    Y . Qian, H. Ye, J.-P. Fauconnier, P. Grasch, Y . Yang, and Z. Gan, “Mia- bench: Towards better instruction following evaluation of multimodal llms,” arXiv preprint arXiv:2407.01509, 2024

    2024 arXiv

  20. [27]

    Llava-ultra: Large chinese language and vision assistant for ultrasound,

    X. Guo, W. Chai, S.-Y . Li, and G. Wang, “Llava-ultra: Large chinese language and vision assistant for ultrasound,” 2024

    2024

  21. [28]

    Gpqa: A graduate-level google-proof q&a benchmark,

    D. Rein, B. L. Hou, A. C. Stickland, et al., “Gpqa: A graduate-level google-proof q&a benchmark,” 2023

    2023

  22. [29]

    Clevr: A diagnostic dataset for compositional language and elementary visual reasoning,

    J. Johnson, B. Hariharan, et al., “Clevr: A diagnostic dataset for compositional language and elementary visual reasoning,” 2016

    2016

  23. [30]

    Expert knowledge-aware image difference graph representation learning for difference-aware medical visual question answering,

    X. Hu, L. Gu, Q. An, M. Zhang, et al., “Expert knowledge-aware image difference graph representation learning for difference-aware medical visual question answering,” in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4156–4165, 2023

    2023

  24. [31]

    Ai models collapse when trained on recursively generated data,

    I. Shumailov, Z. Shumaylov, et al., “Ai models collapse when trained on recursively generated data,” Nature, vol. 631, no. 8022, pp. 755–759, 2024

    2024

  25. [32]

    Pathvqa: 30000+ questions for medical visual question answering,

    X. He, Y . Zhang, L. Mou, E. Xing, and P. Xie, “Pathvqa: 30000+ questions for medical visual question answering,” 2020

    2020

  26. [33]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters,

    C. Snell, J. Lee, K. Xu, and A. Kumar, “Scaling llm test-time compute optimally can be more effective than scaling model parameters,” 2024

    2024

  27. [34]

    Large language monkeys: Scaling inference compute with repeated sampling,

    B. Brown, J. Juravsky, R. Ehrlich, R. Clark, et al., “Large language monkeys: Scaling inference compute with repeated sampling,” arXiv preprint arXiv:2407.21787, 2024

    2024 arXiv

  28. [35]

    A simple model of inference scaling laws,

    N. Levi, “A simple model of inference scaling laws,” 2024

    2024

  29. [36]

    Rewarding chatbots for real-world en- gagement with millions of users,

    R. Irvine, D. Boubert, et al., “Rewarding chatbots for real-world en- gagement with millions of users,” 2023

    2023

  30. [37]

    Interpretable contrastive monte carlo tree search reasoning,

    Z. Gao, B. Niu, X. He, H. Xu, H. Liu, et al., “Interpretable contrastive monte carlo tree search reasoning,” 2024

    2024

  31. [38]

    Self-evaluation guided beam search for reasoning,

    Y . Xie, K. Kawaguchi, Y . Zhao, X. Zhao, M.-Y . Kan, J. He, and Q. Xie, “Self-evaluation guided beam search for reasoning,” 2023

    2023

  32. [39]

    Openai. learning to reason with llms,

    “Openai. learning to reason with llms,” september 2024

    2024

  33. [40]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, et al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025

    2025

  34. [41]

    Synthetic continued pretraining,

    Z. Yang, N. Band, S. Li, E. Cand `es, and T. Hashimoto, “Synthetic continued pretraining,” 2024

    2024

  35. [42]

    Metamath: Bootstrap your own mathematical questions for large language models,

    L. Yu, W. Jiang, H. Shi, J. Yu, et al., “Metamath: Bootstrap your own mathematical questions for large language models,” 2024

    2024

  36. [43]

    Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol- instruct,

    H. Luo, Q. Sun, C. Xu, P. Zhao, et al., “Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol- instruct,” 2025

    2025

  37. [44]

    Pp-ocrv2: Bag of tricks for ultra lightweight ocr system,

    Y . Du, C. Li, R. Guo, C. Cui, et al., “Pp-ocrv2: Bag of tricks for ultra lightweight ocr system,” 2021

    2021

  38. [45]

    Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system,

    C. Li, W. Liu, R. Guo, X. Yin, K. Jiang, et al., “Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system,” 2022

    2022

  39. [47]

    Uscl: Pretraining deep ultrasound image diagnosis model through video contrastive representation learning,

    Y . Chen, C. Zhang, L. Liu, C. Feng, C. Dong, Y . Luo, and X. Wan, “Uscl: Pretraining deep ultrasound image diagnosis model through video contrastive representation learning,” 2021

    2021

  40. [48]

    Pmc-clip: Contrastive language- image pre-training using biomedical documents,

    W. Lin, Z. Zhao, X. Zhang, et al., “Pmc-clip: Contrastive language- image pre-training using biomedical documents,” arXiv preprint arXiv:2303.07240, 2023

    2023 arXiv

  41. [49]

    s1: Simple test-time scaling,

    N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, et al., “s1: Simple test-time scaling,” 2025

    2025

  42. [50]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, et al., “Evaluating large language models trained on code,” 2021

    2021

  43. [51]

    Llamafactory: Unified efficient fine-tuning of 100+ language mod- els,

    Y . Zheng, R. Zhang, J. Zhang, Y . Ye, Z. Luo, Z. Feng, and Y . Ma, “Llamafactory: Unified efficient fine-tuning of 100+ language mod- els,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 3: System Demonstrations), (Bangkok,...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.