Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Pith's one-line read Lightweight LoRA adapters trained on a modest synthetic Romanian VQA corpus can substantially improve open vision-language models' Romanian question answering and image captioning, while also cutting grammatical errors.

desk verdict A genuinely useful dataset and a credible captioning-transfer result, but the headline VQA numbers rest on a self-generated benchmark and the abstract overclaims. read the letter →

arxiv 2512.14926 v2 pith:L2SM7KLA submitted 2025-12-16 cs.CL cs.AIcs.LG

classification cs.CLcs.AIcs.LG
keywords Romanianvision-languagemodelsmultimodalinstructiontuningLoRAlow-resourcelanguagesvisualquestionansweringimagecaptioningsyntheticdata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a low-resource language like Romanian can be served by open vision-language models without expensive full retraining. It builds the first Romanian visual question answering corpus by translating Flickr30k captions and having a 70-billion-parameter text-only model generate QA pairs, then fine-tunes only small LoRA adapters (about 0.5% of parameters) inside three existing VLM families. The experiments show consistent gains for the two backbones with enough multilingual pretraining: the top model improves Romanian VQA by more than 6 BERTScore points and 14 ROUGE-L points over its base, transfers to captioning with no extra training, and produces answers with fewer grammatical errors. A sympathetic reader would care because the recipe is cheap, open, and suggests that democratizing multimodal AI for under-represented languages may be within reach of modest compute.

What carries the argument

The dataset is the backbone of the argument: Flickr30k-Ro, a native-speaker translation of Flickr30k (158,915 Romanian captions), extended into Flickr30k-RoQA with 31,783 question-answer pairs. The QA pairs are generated by a 70-billion-parameter open-weights language model conditioned only on the concatenated Romanian captions, deliberately avoiding any image input. Training uses LoRA: low-rank adapter matrices inserted into the linear layers of the language transformer while vision and base weights stay frozen, with rank 16. The QA generation pipeline is what makes large-scale Romanian instruction data cheap; the frozen-backbone LoRA tuner is what makes adapting existing models to that dat

What would settle it

Have native Romanian speakers rate whether a random sample of the 31,783 generated QA pairs is answerable from the images alone, and test the same LoRA recipe on QA pairs generated from shuffled or unrelated captions. If answerability is low, or if the model's gains persist under the shuffled-caption control, the central claim of improved visual understanding fails; the gains would then be attributable to better Romanian text generation, not multimodal grounding.

Watch

Extended reading notes

Core claim

On the paper's own terms: Romanian visual question answering can be improved substantially by a parameter-efficient instruction-tuning pipeline. A corpus built from human-verified Romanian translations of Flickr30k plus synthetic, text-only-generated QA pairs is sufficient to train LoRA adapters that, when inserted into the language branch of open VLMs, lift performance on Romanian VQA. The strongest result comes from the 7-billion-parameter Qwen model: 80.15% BERTScore F1 and 44.25% ROUGE-L on Romanian VQA, up 6.05 and 14.03 absolute points over the unadapted model, with BLEU rising 38.74 points. The same adapter, trained only on VQA, also improves Romanian image captioning by up to 2.61 BE

Load-bearing premise

The load-bearing premise is that the machine-generated, human-unverified question–answer pairs—produced by a text-only model from Romanian captions—are reliable enough as both training and test data; the paper's own limitations paragraph (Section 5) concedes these pairs 'may omit visually grounded reasoning patterns specific to Romanian usage.' If the pairs are noisy or trivially derivable from the captions, the measured quality gains may be capturing caption-matching rather

Editorial extensions

If this is right

  • Romanian gains a public, human-verified caption corpus and the first Romanian visual QA corpus with released adapters, giving future work a fixed point of comparison.
  • Because the same VQA-trained adapter also boosts captioning, the gains are not narrowly task-specific; instruction tuning on one multimodal task can improve a related generation task.
  • Grammatical-error reductions suggest the adapter affects linguistic form, not just semantic overlap, which matters for downstream user-facing applications.
  • The recipe (translate a standard corpus, synthesize QA from captions, tune LoRA on a single GPU) is directly portable to other low-resource languages with a strong multilingual base model.
  • The different outcomes across backbones imply that base-model language prior is the gating factor: parameter-efficient tuning works when the backbone already 'knows' the language, and cannot supply it from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: because QA pairs are generated from captions only and were never human-verified, the VQA benchmark may reward restating caption content more than visual reasoning; checking a random sample for image-answerability would clarify what the reported +6 BERTScore actually measures.
  • My inference: the captioning transfer might partly reflect a shared answer style (short, full-sentence Romanian) rather than improved visual understanding; a control that tunes LoRA on unpaired or shuffled QA pairs would test whether the visual modality is doing the work.
  • My inference: for low-resource languages without a multilingual base model, the paper's own LLaVA result suggests that vocabulary expansion or a modest amount of full fine-tuning may be needed before LoRA can help, so the recipe's scope depends on the base model's language coverage.
  • My inference: the WER reduction reported through an LLM-as-judge protocol should be validated against human annotators; the judge's own stylistic preferences could inflate or mask the grammatical improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Flickr30k-Ro, a Romanian translation of the Flickr30k captions, and Flickr30k-RoQA, a synthetic VQA dataset created by prompting LLaMA-3.3 70B with concatenated Romanian captions. Using LoRA on the VQA corpus, the authors fine-tune three open VLMs—LLaMA-3.2-11B-Vision, Qwen2-VL-7B-Instruct, and LLaVA-v1.6-Mistral-7B—with frozen vision encoders. They evaluate Romanian VQA and zero-shot image captioning using BLEU, ROUGE-L, and BERTScore, plus an LLM-as-judge grammatical error analysis. They report large gains for LLaMA-3.2 and Qwen2-VL (e.g., Qwen2-VL-RoVQA: 80.15 BERTScore F1, +6.05 on VQA; 73.08, +2.61 on captioning), while LLaVA-v1.6-RoVQA degrades on most metrics.

Significance. If the results hold, the paper would be a useful first Romanian multimodal resource and a demonstration that lightweight LoRA adaptation can improve Romanian capabilities of multilingual and English-centric VLMs. The released human-translated captions, synthetic QA corpus, and adapter weights are concrete contributions, and the pipeline is open and reproducible. However, the central VQA evidence is weakened by the fact that train and test QA pairs are generated by the same unverified synthetic generator from the same captions; the captioning results, which use human-translated references, provide independent but only partial support. The broad conclusion that 'our models show improved Romanian capabilities' is also contradicted by the LLaVA-v1.6 results. The paper's own limitations section (§5) concedes the synthetic QA may omit visually grounded reasoning patterns.

major comments (4)
  1. [§3.1.2 / §4.2 / §5] The VQA evaluation is circular: both training and test QA pairs are produced by the same LLaMA-3.3 70B generator from the same Flickr30k captions, with no human verification. Fine-tuning on this distribution can improve answers by matching the generator's lexical and syntactic preferences rather than by improving image-grounded reasoning. The paper itself concedes in §5 that the synthetic QA 'may omit visually grounded reasoning patterns specific to Romanian usage.' Because Tables 7's gains are the headline result, the authors should (i) manually validate a random sample of test QA pairs; (ii) evaluate on an external Romanian VQA set or a human-created held-out set; or (iii) at minimum, ablate training on captions alone to disentangle style matching from visual learning.
  2. [Tables 7–8, §4.2–§4.3] The abstract and conclusion claim that 'our models show improved Romanian capabilities,' but this is too strong. LLaVA-v1.6-RoVQA degrades relative to its base on BERTScore F1 and ROUGE-L in both VQA (60.33→55.13; 6.38→3.98) and captioning (58.27→53.61; 4.48→3.92); only BLEU rises. The paper acknowledges this in §4.2 but does not qualify the aggregate claim at the level of the abstract or conclusion. The contribution should be restated as applying to LLaMA-3.2 and Qwen2-VL, or the LLaVA failure case needs a more substantive analysis.
  3. [§4.2, Tables 7–8] All results come from a single run, with no error bars, confidence intervals, or significance tests. Some headline deltas are modest—e.g., Qwen2-VL captioning BERTScore +2.61 and ROUGE-L +6.27—and could plausibly be within run-to-run variance, especially given only 500 training steps and a single randomly selected QA pair per image. Please provide multiple seeds (at least three) with mean±std and a paired significance test, or justify why a single run is sufficient.
  4. [§4.4, Table 9] The grammatical error analysis is also confounded: the 'judge' model is LLaMA 3.3 70B, the same model family and configuration used to generate the VQA answers on which the adapters were trained. The judge may systematically favor outputs resembling its own style, and the paper's own caveat that WER conflates lexical paraphrase with grammar errors does not address this. An independent human evaluation, or at least a second judge from a different model family, is needed to support the fluency claims.
minor comments (5)
  1. [Abstract / front matter] The executive summary reports Qwen2-VL-RoVQA improvements of '+2.29% and +4.45%', while the full abstract and Tables 7–8 report '+6.05% and +2.61%'. Please harmonize the numbers.
  2. [§3.2] LLaMA-3.2 Vision is described as using a 'convolutional image encoder' with patch embeddings concatenated to text tokens, but §2.1 correctly describes the cross-attention image adapter. Please correct the architecture description.
  3. [§3.3] The text says 'Max steps 500 (≈1 epoch)', but with an effective batch size of 16, 500 steps correspond to 8,000 samples, not the 25,426-example training split. Please correct the epoch statement.
  4. [Tables 7–8 footnotes and §4.1] The phrase 'BLEU is shown in raw corpus units' is unclear; BLEU is a unitless score. Also, the single-reference caveat for BLEU/ROUGE-L is stated for VQA but applies equally to captioning.
  5. [Throughout] There are several typos and unfinished edits, including 'respetively' in §4.1, 'redsometimes very different vocabulary means too much tokenization' in Table 2, 'Table 75' in §4.2, and the PathChat reference listing 'others' as an author. Please clean these up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the self-generated VQA benchmark is a validity limitation, not a circular derivation.

full rationale

The paper's derivation chain is straightforward: construct translated Romanian captions, generate QA pairs with LLaMA-3.3 70B from those captions, LoRA-fine-tune three VLMs on 80% of the QA pairs, and evaluate on the held-out 20% plus on captioning. Every step is empirical; no equation is used both as input and output, and no fitted parameter is relabeled as a prediction. The closest candidate is the VQA evaluation: both training and test QA pairs are generated by the same LLaMA-3.3 70B from the same Romanian caption source (§3.1.2), so the test set is in-distribution and not independently human-verified. This is a real limitation, explicitly conceded in §5: 'the synthetic QA component relies on a LLaMA-3 generator (mostly considered monolingual) conditioned only on text, which may omit visually grounded reasoning patterns specific to Romanian usage.' It is also noted for the grammar protocol in §4.4 that 'relying on a single LLM as reference risks propagating its own stylistic preferences.' However, these concerns are about external validity and benchmark quality, not circularity: the held-out split is genuinely out-of-sample, the test answers are not produced by the LoRA adapters, and the model must generalize to unseen images and captions. The captioning results (§4.3) are measured against human-translated references and provide an independent check. There are no load-bearing self-citations, no uniqueness arguments imported from the authors' prior work, and no ansatz smuggled in via citation. The VQA claims should be interpreted cautiously because of the synthetic benchmark, but they do not reduce by construction to the training inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on dataset quality (human translation, synthetic QA correctness), the validity of the LLM-as-judge grammar protocol, and the assumption that LoRA suffices. These are domain assumptions rather than derived results; no formal proofs or independent benchmarks are provided for them.

free parameters (4)
  • LoRA rank (r) = 16
    Chosen a priori, not tuned; controls the capacity of the adapters (§3.3).
  • LoRA alpha (α) = 16
    Scaling factor chosen a priori (§3.3).
  • Learning rate = 2e-4
    Chosen a priori with linear schedule; not tuned on a validation set (§3.3).
  • Max training steps = 500 (≈1 epoch)
    Chosen a priori; stopping rule is not data-driven (§3.3).
assumptions (5)
  • domain assumption The human-translated Romanian captions are grammatically correct and faithful to the English originals.
    Central to both dataset quality and evaluation references; no independent inter-annotator agreement or validation is reported (§3.1.1).
  • domain assumption LLaMA-3 70B-generated QA pairs are correct and answerable from the corresponding images.
    The VQA training and test sets are produced automatically from captions; no human verification or visual grounding check is performed (§3.1.2).
  • domain assumption LLaMA-3 70B's corrections are a valid reference for measuring grammatical errors.
    The WER analysis relies entirely on this LLM-as-judge; the paper itself notes it may conflate grammar errors with paraphrases (§4.4).
  • domain assumption LoRA can approximate full fine-tuning for this adaptation.
    The method assumes low-rank updates are sufficient to transfer Romanian knowledge; not proven here, but consistent with prior work (§3.3).
  • domain assumption Flickr30k images and captions are representative enough to support Romanian multimodal instruction tuning.
    The dataset is derived from a single English photographic corpus; limitation noted in §5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models." pith.science (2026). https://pith.science/paper/L2SM7KLA

@misc{pith2026251214926,
  author       = {Pith},
  title        = {Pith review of: Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L2SM7KLA}},
  note         = {Machine review of arXiv:2512.14926}
}
read the original abstract

Focusing on low-resource languages is an essential step toward democratizing generative AI. In this work, we contribute to reducing the multimodal NLP resource gap for Romanian. We translate the widely known Flickr30K dataset into Romanian and further extend it for visual question answering by leveraging open-source LLMs. We demonstrate the usefulness of our datasets by fine-tuning open-source VLMs on Romanian visual question answering. We select VLMs from three widely used model families: LLaMA 3.2, LLaVA 1.6, and Qwen2. For fine-tuning, we employ the parameter-efficient LoRA method. Our models show improved Romanian capabilities in visual QA, as well as on tasks they were not trained on, such as Romanian image description generation. The seven-billion-parameter Qwen2-VL-RoVQA obtains top scores on both tasks, with improvements of +2.29% and +4.45% in BERTScore F1 on VQA and captioning, respectively, over its original version. Finally, the models show substantial reductions in grammatical errors compared to their original forms, indicating improvements not only in language understanding but also in Romanian fluency.

Figures

Figures reproduced from arXiv: 2512.14926 by the authors.

Figure 1
Figure 1. Overview of the dataset construction pipeline [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Token count distributions reported for the training and test splits. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. "\^{I}n\c{t}elegi Rom\^ane\c{s}te?'' A Recipe for Romanian Vision-Language Models

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    Translating English VLM corpora to Romanian and training adapted models produces VLMs that outperform same-sized and even next-larger non-adapted models on Romanian benchmarks.

Reference graph

Works this paper leans on

18 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [2]

    Lavy: Vietnamese multimodal large language model.arXiv preprint arXiv:2404.07922,

    Chi Tran and Huong Le Thanh. Lavy: Vietnamese multimodal large language model.arXiv preprint arXiv:2404.07922,

  2. [3]

    X-LLaV A: Optimizing bilingual large vision-language alignment

    DongJae Shin, HyeonSeok Lim, Inho Won, ChangSu Choi, Minjun Kim, SeungWoo Song, HanGyeol Yoo, SangMin Kim, and KyungTae Lim. X-LLaV A: Optimizing bilingual large vision-language alignment. In Kevin Duh, Helena Gomez, and Steven Bethard, editors,Findings of the Association for Computational Linguistics: NAACL 2024, pages 2463–2473, Mexico City, Mexico, Jun...

  3. [6]

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao

    URL https://llava-vl.github.io/blog/ 2024-01-30-llava-next/. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. LLaV A-med: Training a large language-and-vision assistant for biomedicine in one day. In Thirty-seventh Conference on Neural Information Processing Systems Datasets a...

  4. [7]

    Kangyu Zhu, Ziyuan Qin, Huahui Yi, Zekun Jiang, Qicheng Lao, Shaoting Zhang, and Kang Li

    URL https://openreview.net/forum?id=GSuP99u2kR. Kangyu Zhu, Ziyuan Qin, Huahui Yi, Zekun Jiang, Qicheng Lao, Shaoting Zhang, and Kang Li. Guiding medical vision-language models with diverse visual prompts: Framework design and comprehensive exploration of prompt variations. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings of the 2025 Confer...

  5. [9]

    Lu, Bowen Chen, Andrew Zhang, Richard J

    Ming Y . Lu, Bowen Chen, Andrew Zhang, Richard J. Chen, others, and Faisal Mahmood. A multimodal generative ai copilot for human pathology.Nature, 2024a. doi: 10.1038/s41586-024-07618-3. R. J. Chen, T. Ding, M. Y . Lu, et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30:850–862,

  6. [10]

    URL https://doi.org/10.1038/ s41591-024-02857-3

    doi: 10.1038/s41591-024-02857-3. URL https://doi.org/10.1038/ s41591-024-02857-3. Ming Y . Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, Georg Gerber, Anil V . Parwani, and Faisal Mahmood. A visual- language foundation model for computational pathology.Nature Medici...

  7. [11]

    doi: 10.18653/v1/2024.findings-emnlp.268

    Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-emnlp.268. URLhttps://aclanthology.org/2024.findings-emnlp.268/. Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing HONG, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong. G-LLaV A: Solving geometric problem with multi-modal large language model. ...

  8. [12]

    Dongjae Shin, HyeonSeok Lim, Inho Won, Changsu Choi, Minjun Kim, Seungwoo Song, Hangyeol Yoo, Sangmin Kim, and Kyungtae Lim

    URL https: //openreview.net/forum?id=px1674Wp3C. Dongjae Shin, HyeonSeok Lim, Inho Won, Changsu Choi, Minjun Kim, Seungwoo Song, Hangyeol Yoo, Sangmin Kim, and Kyungtae Lim. X-llava: Optimizing bilingual large vision-language alignment.arXiv preprint arXiv:2403.11399, 2024b. Junchen Li, Qing Yang, Bojian Jiang, Shaolin Zhu, and Qingxuan Sun. Lrm-llava: Ov...

Show all 18 references
  1. [13]

    Anwer, Tim Baldwin, Michael Felsberg, and Fahad S

    Hanoona Rasheed, Muhammad Maaz, Abdelrahman Shaker, Salman Khan, Hisham Cholakal, Rao M. Anwer, Tim Baldwin, Michael Felsberg, and Fahad S. Khan. Palo: A large multilingual multimodal language model. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer ...

  2. [14]

    Maya: An instruction finetuned multilingual multimodal model.arXiv preprint arXiv:2412.07112,

    Nahid Alam, Karthik Reddy Kanjula, Surya Guthikonda, Timothy Chung, Bala Krishna S Vegesna, Abhipsha Das, Anthony Susevski, Ryan Sze-Yin Chan, SM Uddin, Shayekh Bin Islam, et al. Maya: An instruction finetuned multilingual multimodal model.arXiv preprint arXiv:2412.07112,

  3. [15]

    Chitrarth: Bridging vision and language for a billion people

    Shaharukh Khan, Ayush Tarun, Abhinav Ravi, Ali Faraz, Praveen Kumar Pokala, Anagha Bhangare, Raja Kolla, Chandra Khatri, and Shubham Agarwal. Chitrarth: Bridging vision and language for a billion people. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and...

  4. [18]

    The question generation prompt is used to produce four candidate questions for each image during the dataset augmentation phase

    16 APREPRINT- DECEMBER18, 2025 A Prompts Used In Table 10, we present the Romanian prompts used in this paper and their English translations. The question generation prompt is used to produce four candidate questions for each image during the dataset augmentation phase. The <i...

  5. [2002]

    Rouge: A package for automatic evaluation of summaries

    Lin Chin-Yew. Rouge: A package for automatic evaluation of summaries. InProceedings of the Workshop on Text Summarization Branches Out, 2004,

  6. [2019]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langua...

  7. [2021]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E

    URLhttps://arxiv.org/abs/2103.00020. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March

  8. [2023]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee

    URLhttps://lmsys.org/blog/2023-03-30-vicuna/. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023b. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasonin...

  9. [2024]

    co-located with 23th International Conference of the Italian Association for Artificial Intelligence (AI* IA 2024), CEUR-WS. org,

  10. [2025]

    ISBN 979-8-89176-189-6

    Association for Computational Linguistics. ISBN 979-8-89176-189-6. doi: 10.18653/v1/2025.naacl-long.587. URL https://aclanthology. org/2025.naacl-long.587/. Mehmet Saygin Seyfioglu, Wisdom O. Ikezogwo, Fatemeh Ghezloo, Ranjay Krishna, and Linda Shapiro. Quilt-llava: Visual ins...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.