Pith. sign in

REVIEW 4 major objections 4 minor 30 references

${\mu}^2$Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for Radiology Report Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A differentiable tokenizer lets a 1B multimodal LLM outperform 7B–14B models at writing CT radiology reports.

desk verdict A plausible tokenizer upgrade for 3D CT report generation, but the SOTA claim is unsupported until baseline training and the GREEN circularity are addressed. read the letter →

arxiv 2507.00316 v2 pith:INWMM2SN submitted 2025-06-30 cs.LG cs.CLeess.IV

Siyou Li , Pengyao Qin , Huanan Wu , Dong Nie , Arun J. Thirunavukarasu , Juntao Yu , Le Zhang This is my paper
classification cs.LGcs.CLeess.IV
keywords radiologyreportgenerationmultimodallargelanguagemodel3DCTimagingdifferentiabletokenselectionmulti-scalepoolingdirectpreferenceoptimizationGREENmetricvisualquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a 1-billion-parameter multimodal LLM can generate clinically usable radiology reports from 3D CT scans more accurately than larger 7B–14B models, provided the CT volume is compressed by a carefully designed differentiable tokenizer. The central object is the µ2Tokenizer, an intermediate layer between a 3D vision transformer and the LLM that fuses question text with refined multi-scale visual tokens. The authors argue that three changes—relative positional encoding, soft differentiable token selection, and dynamic multi-scale pooling—preserve clinically relevant local detail and keep training feasible on four A40 GPUs. Report quality is measured and optimized with the GREEN score, a metric that compares clinical content rather than surface word overlap, using direct preference optimization after supervised fine-tuning. This matters because radiology report generation has been limited by information loss when CT scans are resized or cropped and by evaluation metrics that miss clinical meaning; the paper offers one mechanism aimed at both.

What carries the argument

The µ2Tokenizer is a differentiable multi-scale, multi-modal tokenizer placed between the 3D image encoder (ViT3D) and the LLM. It takes visual tokens $V \in \mathbb{R}^{T \times N_v \times E}$ and question tokens $Q \in \mathbb{R}^{N_q \times E}$, and emits compact visual tokens $V'$ by modifying the linear video tokenizer LinVT in three ways: relative positional encoding added within attention, soft differentiable token selection that computes attention weights $\alpha^{(r)}$ over all flattened visual tokens and forms each top token as a weighted sum, and dynamic multi-scale average pooling whose kernel weights are predicted per input. The mechanism's job is to compress a high-dimensional CT volume into a small token budget without losing the local anatomical structure that carries diagnostic information, while keeping the whole pipeline end-to-end trainable.

What would settle it

Fine-tune CT-CHAT-8B and LaMed-Llama-2-7B on the same CT-Rate, AMOS-MM, and AbdomenAtlas training splits used for µ2LLM, then score with the same GREEN protocol. If either baseline reaches or beats GREEN = 0.429 on CT-Rate, the paper's central efficiency claim fails; if they stay near 0.113, the claim survives.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a small, efficiently tokenized multimodal model can beat much larger baselines on CT report generation, with the largest margin on the clinical-semantic GREEN metric. On CT-Rate, µ2LLM-1B (SFT&DPO) reaches GREEN = 0.429 versus 0.113 for CT-CHAT-8B, and on AbdomenAtlas it reaches 0.346 versus 0.011 for LaMed-Phi-3-4B. The authors attribute the gain to the three tokenizer substructures: relative positional encoding inside attention, a differentiable soft top-k selection in which each selected token is a weighted sum over all visual tokens ($V^{(r)}_{top} = \sum_i \alpha_i^{(r)} V^{(i)}_{flat}$), and dynamic multi-scale pooling with learned kernel weights. They also show that DPO training against GREEN-scored preferences adds a consistent improvement, and that a five-stage prompt pipeline can convert routine CT reports into question-answer-reasoning triples for scalable supervision.

Load-bearing premise

The reported superiority rests on the assumption that the baselines were evaluated under the same conditions as µ2LLM (same training data, fine-tuning, and scoring protocol) and that the GREEN score reflects clinically meaningful report quality; the paper does not show the baselines were fine-tuned on the target datasets.

Editorial extensions

If this is right

  • A 1B model trained this way can surpass 7B–14B baselines on GREEN, which implies that careful tokenization and alignment can substitute for raw model scale in CT report generation.
  • Using GREEN as the DPO reward ties training directly to clinical content, so gains should show up as fewer clinically significant omissions rather than just better n-gram overlap.
  • The differentiable selection and dynamic pooling make the approach trainable on modest hardware, which lowers the barrier for specialized medical teams to fine-tune their own models.
  • The five-stage reasoning-synthesis pipeline turns existing free-text reports into (question, answer, reasoning) triples, providing a scalable supervision source for explainable medical VQA.
  • Relative positional encoding lets the model handle variable slice counts and resolutions, reducing the distortion caused by fixed resizing of CT volumes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to apply the same tokenizer to other 3D modalities such as MRI or PET, since none of the three modifications is CT-specific.
  • Inspection of the learned soft-selection weights $\alpha^{(r)}$ per question could yield an interpretability map showing which CT regions drive each diagnostic answer, an application the paper does not develop.
  • A controlled comparison that fine-tunes the strongest baselines on the same training splits would isolate how much of the advantage is the tokenizer versus supervised fine-tuning itself.
  • The reasoning-synthesis pipeline could be reused to generate explanation-linked supervision for other dense imaging tasks, such as tumor staging or post-treatment follow-up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes µ2Tokenizer, a differentiable multi-scale multi-modal tokenizer for CT-based radiology report generation. It integrates a 3D ViT encoder with an LLM via relative positional encoding, differentiable token selection, and dynamic multi-scale pooling. Training consists of SFT followed by DPO, where GREEN is used to rank preference pairs. A five-stage prompt-engineering pipeline converts CT reports into VQA-style supervisory data. Experiments on three datasets (AMOS-MM, CT-Rate, AbdomenAltas) report state-of-the-art results, including a 1B model outperforming 7B–14B baselines.

Significance. If the results hold, the work constitutes a meaningful advance toward efficient and clinically oriented RRG, particularly the architectural components (RPE, DTS, DMTP) and the proposed data synthesis pipeline. The paper also promises public code/datasets, which would aid reproducibility. However, the validity of the central SOTA claim is currently compromised by the uncontrolled baseline evaluation, the circular use of GREEN as both training reward and evaluation metric, and apparent numerical inconsistencies in the tables.

major comments (4)
  1. [§3 (Baselines and Evaluation Metrics) and Table 1] The paper does not specify whether any of the baseline models (LaMed-Phi-3-4B, LaMed-Llama-2-7B, CT-CHAT-8B, RadGPT-N, RadFM-14B) were fine-tuned on the target datasets with the same preprocessing (8×32×256×256, Min-Max Normalization), prompt templates, or decoding configuration. Without this information, the reported margins in Table 1 (e.g., GREEN 0.429 vs. 0.113 for CT-CHAT-8B on CT-Rate) cannot be attributed to the proposed method; they may reflect protocol differences. The authors must either retrain all baselines under identical conditions or state clearly that baseline numbers are from prior publications; in the latter case, the SOTA claim is not interpretable.
  2. [§2.3 and Table 1] GREEN is used to score model-generated reports to select preferred (yw) and dispreferred (yl) responses for DPO training, and the same GREEN metric is then used as the primary evaluation metric in Table 1. This circularity means the reported GREEN improvements (e.g., from 0.33 to 0.40 on AMOS-MM) may largely reflect reward overfitting. To support the clinical-quality claim, the authors should report evaluation on an independent metric (e.g., human expert ratings, or an LLM-based metric not involved in training) and discuss whether the gains generalize.
  3. [Table 1 (AMOS-MM row) and Table 2] The value METEOR=0.876 reported for µ2LLM-1B(SFT&DPO) on AMOS-MM in Table 1, and again in Table 2, is implausible relative to the neighboring METEOR values (approximately 0.25–0.36) and is numerically identical to the BERTScore column in Table 2. This suggests a column misalignment or a copy-paste error. The numerical results in Tables 1 and 2 need to be corrected and carefully re-verified before the performance claims can be assessed.
  4. [§3 (Datasets)] The abstract states that experiments were performed on 'four large CT image-report medical datasets,' but Section 3 describes only three datasets (AMOS-MM, CT-Rate, AbdomenAltas). In addition, the SFT stage in §2.3 references a 'CT-Reports dataset' that is never defined. The authors should clarify the exact data used for training versus evaluation, including dataset sizes, splits, and the role of the GPT-4o mini expansion; without this, the experiments are not reproducible.
minor comments (4)
  1. [§2.3] The DPO objective as written contains a typesetting error: the second term is missing the logarithm and should read β log(πθ(yl|x)/πref(yl|x)); as printed, the formula is not correct.
  2. [Figure 5] The question shown in Figure 5 contains a typo ('fingings' should be 'findings').
  3. [References] Some references are incomplete; for example, references [15] and [17] lack journal/volume/page information, making them difficult to locate.
  4. [General] The paper does not report variance or statistical significance for any of the main results; including error bars or multiple-seed runs would strengthen the comparison.

Circularity Check

1 steps flagged · score 6.0 of 10

The DPO component is trained against GREEN and then evaluated with GREEN, so the reported GREEN improvements are partly fitted; independent lexical metrics still support part of the SOTA claim.

  1. fitted input called prediction [Section 2.3 (Direct Preference Optimization with GREEN-Score) and Section 3 (Results Analysis, Table 1)]
    "To obtain the preference dataset, we use the trained SFT model to generate a large number of medical reports on the existing dataset, and then these medical reports are scored by GREEN against the ground truth. Finally, the scored reports are used in DPO [14] training, to guide the model generating preferred reports that have the highest GREEN score."

    The DPO loss is optimized on preference pairs whose win/lose labels are assigned by GREEN, so GREEN is the training reward. The paper then reports GREEN in Table 1 as the evidence that DPO improves performance ('the GREEN Score evaluation indicates a model capability improvement of 20% with DPO in GREEN Score') and uses that improvement to support the SOTA claim. The reported GREEN gain is therefore partly a measure of fitting the training reward rather than an independent assessment of clinical report quality. No held-out human evaluation or non-reward metric is reported to separate genuine quality improvement from reward overfitting.

full rationale

The architecture derivation itself is self-contained: the µ2Tokenizer equations define differentiable token selection, dynamic multi-scale pooling, and relative positional encoding, and the ablation is an empirical comparison rather than a circular reduction. The author-overlapping citations present (e.g., ref. [17] and possibly ref. [9]) are background or dataset citations and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The one genuine circularity concern is the DPO-GREEN loop: §2.3 scores DPO preference pairs with GREEN and optimizes the model to produce 'preferred reports that have the highest GREEN score', while §3 uses GREEN as the headline metric for the DPO improvement and for the SOTA claim. Those GREEN gains are partly a fitted quantity. I do not count the uncontrolled baseline protocol or the implausible METEOR=0.876 entry as circularity; those are correctness and external-validity concerns. Because the lexical metrics in Table 1 (ROUGE, METEOR, BERTScore) were not used as DPO rewards and also favor the proposed model, the broader SOTA claim retains independent content; hence partial rather than total circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims depend on a set of learned parameters (queries, positional embeddings, pooling weights) and on the assumption that the GREEN metric is a valid proxy for clinical quality. The paper does not provide independent evidence for the clinical validity of GREEN beyond citing prior work, and the use of GREEN as both reward and evaluation introduces a circular element.

free parameters (4)
  • Scale-specific learnable queries = 1024 queries, hidden size 768
    These queries are learned during training and are central to the token selection and pooling mechanism.
  • Dynamic multi-scale pooling weights = computed by a small MLP
    The weights w_s are learned via a MLP over pooled outputs, adapting pooling kernel contributions to the input.
  • Relative positional embedding parameters = learned per attention head
    These embeddings are added in the attention mechanism (A_ij = Q_i K_j / sqrt(d) + P_r(i-j)) and are fitted to the data.
  • DPO beta (β) = in range (0.1, 0.5)
    The temperature parameter for DPO is chosen by hand and not derived from data.
assumptions (4)
  • domain assumption CT volumes can be represented as frames and patches for the 3D ViT.
    The model divides each CT volume I ∈ R^{T×K×H×W} into T frames of K slices; this assumes the chosen frame/slice structure preserves clinically relevant image information.
  • domain assumption The GREEN metric accurately reflects clinical report quality.
    The paper uses GREEN both as the DPO reward and as the evaluation metric; if GREEN is not a valid measure of clinical correctness, the reported gains are meaningless.
  • domain assumption LLM-generated question-answer and reasoning data are clinically correct and useful.
    The prompt-engineering pipeline relies on an external LLM to generate supervisory data, and the paper assumes these generations are accurate enough for training.
  • ad hoc to paper The fixed preprocessing dimension of 8×32×256×256 is adequate for all CT volumes.
    The paper resizes all volumes to this shape, losing the original depth information; this may not generalize to arbitrary CT acquisitions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ${\mu}^2$Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for Radiology Report Generation." pith.science (2026). https://pith.science/paper/INWMM2SN

@misc{pith2026250700316,
  author       = {Pith},
  title        = {Pith review of: $\mu^2$Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for Radiology Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INWMM2SN}},
  note         = {Machine review of arXiv:2507.00316}
}
abstract

Automated radiology report generation (RRG) aims to produce detailed textual reports from clinical imaging, such as computed tomography (CT) scans, to improve the accuracy and efficiency of diagnosis and provision of management advice. RRG is complicated by two key challenges: (1) inherent complexity in extracting relevant information from imaging data under resource constraints, and (2) difficulty in objectively evaluating discrepancies between model-generated and expert-written reports. To address these challenges, we propose $\mu^2$LLM, a $\underline{\textbf{mu}}$ltiscale $\underline{\textbf{mu}}$ltimodal large language models for RRG tasks. The novel ${\mu}^2$Tokenizer, as an intermediate layer, integrates multi-modal features from the multiscale visual tokenizer and the text tokenizer, then enhances report generation quality through direct preference optimization (DPO), guided by GREEN-RedLlama. Experimental results on four large CT image-report medical datasets demonstrate that our method outperforms existing approaches, highlighting the potential of our fine-tuned $\mu^2$LLMs on limited data for RRG tasks. At the same time, for prompt engineering, we introduce a five-stage, LLM-driven pipeline that converts routine CT reports into paired visual-question-answer triples and citation-linked reasoning narratives, creating a scalable, high-quality supervisory corpus for explainable multimodal radiology LLM. All code, datasets, and models will be publicly available in our official repository. https://github.com/Siyou-Li/u2Tokenizer

Figures

Figures reproduced from arXiv: 2507.00316 by the authors.

Figure 1
Figure 1. Overview of our proposed µ 2LLM model that is centered with the µ 2Tokenizer layer for high quality RRG task. clinically significant errors using the GREEN model [12]—a specialized RRG metric that leverages large language model-based natural language understand￾ing. To enhance the quality of generated reports, we employ direct preference optimization (DPO) [14] to align model outputs with expert-validated clinical a… view at source ↗
Figure 2
Figure 2. The illustration of our proposed µ 2Tokenizer. The improvement is applied to steps of Token Selection, Multi-scale Pooling, and the Positional Encoding. attention mechanism. This ensures that relevant image information is efficiently passed to the LLM while reducing computational overhead. Report Generation: The processed image embeddings are then integrated with a text question to generate a radiology report. We ut… view at source ↗
Figure 3
Figure 3. Overview of training process with Direct Preference Optimization (DPO). [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The pipeline of our proposed CT Report Reasoning Synthesis [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: An example of the generated report from our [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 22 canonical work pages

  1. [1]

    Bai, F., Du, Y., Huang, T., Meng, M.Q.H., Zhao, B.: M3d: Advancing 3d medical image analysis with multi-modal large language models (2024) µ2Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer 11

    work page 2024

  2. [2]

    In: Goldstein, J., Lavie, A., Lin, C.Y., Voss, C

    Banerjee, S., Lavie, A.: METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Goldstein, J., Lavie, A., Lin, C.Y., Voss, C. (eds.) Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. pp. 65–72. Association for Computational Linguistics, An...

    work page 2005

  3. [3]

    arXiv preprint arXiv:2501.04678 (2025)

    Bassi, P.R., Yavuz, M.C., Wang, K., Chen, X., Li, W., Decherchi, S., Cavalli, A., Yang, Y., Yuille, A., Zhou, Z.: Radgpt: Constructing 3d image-text tumor datasets. arXiv preprint arXiv:2501.04678 (2025)

    arXiv 2025

  4. [4]

    Everlight: Radiology unlocked: The global radiologist report 2025

    work page 2025

  5. [5]

    arXiv preprint arXiv:2412.05185 (2024)

    Gao,L.,Zhong,Y.,Zeng,Y.,Tan,H.,Li,D.,Zhao,Z.:Linvt:Empoweryourimage- level large language model to understand videos. arXiv preprint arXiv:2412.05185 (2024)

    arXiv 2024

  6. [6]

    Hamamci, I.E., Er, S., Almas, F., Simsek, A.G., Esirgun, S.N., Dogan, I., Das- delen, M.F., Durugol, O.F., Wittmann, B., Amiranashvili, T., Simsar, E., Sim- sar, M., Erdemir, E.B., Alanbay, A., Sekuboyina, A., Lafci, B., Bluethgen, C., Ozdemir, M.K., Menze, B.: Developing generalist foundation models from a mul- timodal dataset for 3d computed tomography (2024)

    work page 2024

  7. [7]

    arXiv preprint arXiv:2403.17834 (2024)

    Hamamci, I.E., Er, S., Almas, F., Simsek, A.G., Esirgun, S.N., Dogan, I., Dasdelen, M.F., Wittmann, B., Simsar, E., Simsar, M., et al.: A foundation model utilizing chest ct volumes and radiology reports for supervised-level zero-shot detection of abnormalities. arXiv preprint arXiv:2403.17834 (2024)

    arXiv 2024

  8. [8]

    In: International conference on machine learning

    Jaegle, A., Gimeno, F., Brock, A., Vinyals, O., Zisserman, A., Carreira, J.: Per- ceiver: General perception with iterative attention. In: International conference on machine learning. pp. 4651–4664. PMLR (2021)

    work page 2021

Show all 30 references
  1. [9]

    Advances in Neural Information Processing Systems 35, 36722–36732 (2022)

    Ji, Y., Bai, H., Ge, C., Yang, J., Zhu, Y., Zhang, R., Li, Z., Zhanng, L., Ma, W., Wan, X., et al.: Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. Advances in Neural Information Processing Systems 35, 36722–36732 (2022)

    2022

  2. [10]

    In: Text Summarization Branches Out

    Lin, C.Y.: ROUGE: A package for automatic evaluation of summaries. In: Text Summarization Branches Out. pp. 74–81. Association for Computational Linguis- tics, Barcelona, Spain (Jul 2004)

    2004

  3. [11]

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning (2023)

    2023

  4. [12]

    In: Al-Onaizan, Y., Bansal, M., Chen, Y.N

    Ostmeier, S., Xu, J., Chen, Z., Varma, M., Blankemeier, L., Bluethgen, C., Md, A.E.M., Moseley, M., Langlotz, C., Chaudhari, A.S., Delbrouck, J.B.: GREEN: Generative radiology report evaluation and error notation. In: Al-Onaizan, Y., Bansal, M., Chen, Y.N. (eds.) Findings of t...

    2024 doi

  5. [13]

    In: Isabelle, P., Charniak, E., Lin, D

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Isabelle, P., Charniak, E., Lin, D. (eds.) Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. pp. 311–318. Association for C...

    2002

  6. [14]

    In: Thirty-seventh Conference on Neural Information Processing Systems (2023)

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Di- rect preference optimization: Your language model is secretly a reward model. In: Thirty-seventh Conference on Neural Information Processing Systems (2023)

    2023

  7. [15]

    https://doi.org/10.2214/AJR.16.16633, publisher: American Roent- gen Ray Society 12 Siyou Li et al

    Rosenkrantz, A.B.: Differences in perceptions among radiologists, referring physi- cians, and patients regarding language for incidental findings reporting208(1), 140–143. https://doi.org/10.2214/AJR.16.16633, publisher: American Roent- gen Ray Society 12 Siyou Li et al

    doi

  8. [16]

    Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position represen- tations (2018)

    2018

  9. [17]

    1038/s41591-023-02448-8

    Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., Ting, D.S.W.: Large language models in medicine29, 1930–1940.https://doi.org/10. 1038/s41591-023-02448-8

    1930

  10. [18]

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: Llama: Open and efficient foundation language models (2023)

    2023

  11. [19]

    Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: Towards generalist foundation model for radiology (2023)

    2023

  12. [20]

    CoRRabs/1904.09675 (2019)

    Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: Bertscore: Evaluating text generation with BERT. CoRRabs/1904.09675 (2019)

    2019 arXiv

  13. [21]

    Zhao, G., Zhao, Z., Gong, W., Li, F.: Radiology report generation with medical knowledge and multilevel image-report alignment: A new method and its verifica- tion. Artificial Intelligence in Medicine146, 102714 (2023) µ2Tokenizer: Differentiable Multi-Scale Multi-Modal Tokeni...

    2023

  14. [22]

    Take the following 3 examples for style of writing

  15. [23]

    You MUST NOT change any meaning of the original report, nor add or remove any information, not event correction

  16. [24]

    Give out the paraphrased report directly, without any other content

  17. [25]

    Here are some examples of CT reports: {SOME EXAMPLES OF DATASETS} The original report: ‘‘‘ {} ‘‘‘ A.2 CT Report Reasoning Synthesis: Prompt

    In English only. Here are some examples of CT reports: {SOME EXAMPLES OF DATASETS} The original report: ‘‘‘ {} ‘‘‘ A.2 CT Report Reasoning Synthesis: Prompt

  18. [26]

    *?\d\. ?([\^\n]*)

    Questions generation Use the following prompt to call LLM to generate a question list, and then use the regular expressionr".*?\d\. ?([\^\n]*)" to extract the questions separately: Here is a medical radiology report for a CT image. ‘‘‘ {report} ‘‘‘ Imagine you are assessing a ...

  19. [27]

    Thinking: ?([^\n]*)

    Answer&Thinking generationUse the following prompt to call LLM to generate the thinking process and answers, and extract the thought content with the regular expressionr"Thinking: ?([^\n]*)" and extract the answer content with the regular expressionr"Answer: ?([^\n]*)" : You a...

  20. [28]

    Yes". If the question is not proper for a radiology exam, or the answer is incorrect, return

    FilterUse the following prompt to call LLM to filter out incorrect questions or answers that were generated in the previous step: You are an expert in radiology. Now you are reviewing a some questions and answers made by another expert. You need to determine if the question is...

  21. [29]

    Refine Thinking Use the following prompt to call LLM to rewrite the thinking data generated in the previous step to make it more in line with VQA habits: Help me edit the narrative below: - If the narrative refers to a report, you change it as if you see it from the radiology ...

  22. [30]

    Now you are looking at a radiology image

    Report Thinking SynthesisUse the following prompt to call LLM to gen- erate the data that generates thoughts in the report process.Thinking_before is spliced using all the questions, thinking, and answers in the QA dataset: ou are a radiology medicine expert. Now you are looki...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.