{"id":"e8a0e4c0-4e5c-41be-b2d5-e95be7844db5","arxiv_id":"2501.05122","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Systematic experiments show that training a vision-language model on 100 languages with only 25 to 50 percent non-English data yields strong multilingual gains, and synthetic OCR data is key for non-Latin scripts.","lead":"This paper tests how many languages and how much non-English data a vision-language model needs during training, finding that 100 languages with 25 to 50 percent non-English data greatly improves multilingual performance without hurting English. It also introduces a synthetic benchmark for reading text in images and trains Centurio, a 100-language model that outperforms open rivals on low-resource languages.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"xGQA is not demonstrably held out: Table 8 includes GQA (72k images) and VG (86k Visual Genome images) in the instruction mix, while Appendix C.4 builds xGQA from 300 Visual Genome images; image-level overlap would inflate the headline multilingual gains.","rationale":"The reader's CONDITIONAL verdict is already the right target. I agree with the reader's weakest assumption: xGQA is the most concrete place where training and test are not clearly separated. The concern is load-bearing because the central claim is an empirical statement about measured multilingual ability; if the measurement is inflated on one of the 13 tasks, the magnitude of the claimed gains is in question. It is not, however, a demonstrated falsification: Table 1a's 'tasks unaffected by language fidelity' and the Llama 3 replications suggest the no-curse trend may hold even if xGQA is removed, and other benchmarks such as MaXM, XM3600, and BabelImageNet are less obviously image-overlapping. The paper also gets credit for a broad, systematic design and an honest limitations section, but no code or data release and no machine-checked verification are offered, so the overlap check is the decisive next step. A secondary issue is that no repeated runs or error bars are reported, so all quantitative ranges should be read with caution; that is secondary to contamination for the central claim. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":84488,"tokens_out":12157,"duration_ms":133176,"concrete_test":"Check overlap directly: extract the 300 xGQA image IDs from the xGQA release and intersect them with the image IDs used for the GQA and VG rows of Table 8. If the intersection is nonzero, recompute the RQ1 and RQ2 tier averages with xGQA (and any other task whose images intersect training) excluded, then re-examine whether the L100-vs-English and 25-50% non-English conclusions survive. If the trends survive, the contamination is bounded; if they weaken or invert, the headline claim must be revised or the model must be retrained with a truly held-out xGQA split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires evaluation on truly held-out examples, and the paper's own appendices undermine this for xGQA. Appendix C.4 states that xGQA translates the GQA balanced test-dev set, giving 9,666 questions in eight languages over 300 unique Visual Genome images. Table 8 shows the instruction-tuning mix contains GQA (72k images) and, separately, VG (86k images from Visual Genome), both machine-translated into the training languages. Even if the GQA train/test split is image-disjoint, the VG portion is not a GQA split; it draws from the same Visual Genome image pool as xGQA, so the exact xGQA images are plausibly in the training set. Because xGQA is included in the RQ1/RQ2 tier averages and in the final Centurio comparison, a nontrivial overlap would let a model answer from image memorization and task-format familiarity rather than from newly learned multilingual ability. This makes the quantitative headline ('greatly improving multilingual performance') less secure, although other tasks in the suite may still support the qualitative trend.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a systematic empirical study of multilingual training-data composition for large vision-language models (LVLMs). Using machine-translated versions of English instruction-tuning and pre-training data, the authors vary (1) the number of training languages, (2) the English-to-multilingual ratio in instruction tuning, (3) the same ratio in pre-training, and (4) the amount of synthetic multilingual OCR data. They evaluate across 13 tasks and 43 languages (plus CVQA for 56 languages total), report tier-wise averages, and conclude that training with up to 100 languages causes little English degradation, that 25-50% non-English data is a practical sweet spot, and that multilingual synthetic OCR data helps text-in-image understanding, especially for Latin scripts. They introduce a new benchmark, SMPQA, and train Centurio (two variants) using the derived recipe, obtaining strong multilingual results against open-weight LVLMs.","tokens_in":84738,"tokens_out":6248,"duration_ms":65401,"significance":"If the main claims hold, the paper would provide unusually actionable guidance for a practical question in multilingual LVLM training: how many languages and what data mix should be used, and at what cost. The study is broad (two LLM backbones, 100 languages, 13-14 tasks), the training pipeline is controlled via translation, and the authors are transparent about limitations. The introduction of SMPQA as a targeted OCR diagnostic is a useful community asset. However, the quantitative headline is weakened by a training-evaluation overlap for xGQA, by the lack of repeated runs or uncertainty estimates for the central 'optimal mix' comparisons, and by selecting the final recipe on the same evaluation suite used to demonstrate Centurio's state-of-the-art results. The qualitative direction of the findings is likely robust, but the specific 25-50% recommendation and the claimed 'greatly improved multilingual performance' need stronger support.","major_comments":[{"comment":"xGQA is not demonstrably held out: the instruction-tuning mix in Table 8 includes GQA (72k images) and VG (86k Visual Genome images), while Appendix C.4 states that xGQA consists of 9,666 manually translated questions over 300 unique Visual Genome images taken from the GQA balanced test-dev set. Even if the GQA train/test split is image-disjoint, the VG portion draws from the same Visual Genome image pool as xGQA, so the exact evaluation images are plausibly seen during training. Because xGQA contributes to the RQ1/RQ2 tier averages (Tables 1-2) and to the final Centurio comparison (Table 5), a nontrivial overlap would inflate the apparent multilingual gains from in-language instruction tuning. Please quantify the image-level overlap, exclude overlapping images, or report whether the conclusions are unchanged when xGQA is removed from the averages.","section":"§2.2-2.3, Table 8, Appendix C.4"},{"comment":"Each configuration is trained once, without multiple seeds or error bars, and the differences among the ratios used to select the 'optimal' 50% English setting are small (e.g., Table 2: T3 scores are 29.9, 30.7, 30.7 for 25%, 50%, 75%; T4 scores are 27.9, 28.9, 28.4; T5 scores are 33.0, 34.4, 34.6). The claim that 25-50% is a robust optimum is therefore not statistically supported. In addition, the same evaluation suite used in §2 to choose E_IT=50 and E_PT=50 is then reused in §3 to benchmark Centurio, creating a selection-on-test effect. Please provide repeated runs or confidence intervals, and validate the recipe on a benchmark not used in the analysis (CVQA is a useful step, but it is only one additional task).","section":"§2.3-2.4, Tables 2-3; §3.1-3.2"},{"comment":"The abstract's recommendation of 'as little as 25-50% non-English data' is not directly supported by the reported tables. In Table 2 the lower-resource tiers T1 and T2 perform best at 25% English, while T5 and English perform best at 75-90% English, and Table 3 shows E_PT=1% and E_PT=50% give nearly identical results for most tiers. The 25-50% sweet spot is thus a judgment call across tiers and training stages rather than a quantitative finding. Please either revise the claim to state the tier-dependent pattern precisely or provide an explicit averaging criterion that justifies the selected operating point.","section":"Abstract and §2.3-2.4"}],"minor_comments":[{"comment":"The text says performance peaks 'between 25% and 75%', while the abstract says '25-50%'; these should be reconciled, ideally with the tier-dependent numbers in the main text.","section":"§2.3, Table 2 caption"},{"comment":"There is a typo: 'Qwen achievs' should be 'Qwen achieves'.","section":"Appendix D.3"},{"comment":"The row for Design2Code lists '0k' as the dataset size, which appears to be a typo; please clarify the actual number of samples.","section":"Table 9"},{"comment":"The label 'A VG.' appears in several table captions and column headers; this should be formatted as 'Avg.' for readability.","section":"Table 5 caption and several appendix tables"},{"comment":"The phrase '56 language tasks' should read '56 languages' (or 'language-task combinations') for accuracy.","section":"§5, conclusion"},{"comment":"There is a missing space in 'E = 50as a robust value'; it should read 'E = 50 as a robust value'.","section":"§2.3, last sentence"}],"recommendation":"major_revision","confidential_remarks":"The xGQA training/evaluation overlap (VG images in the instruction mix vs. xGQA images from Visual Genome) is the most serious threat to the headline quantitative claim. If the authors can demonstrate that no overlap exists, or rerun the key tables with xGQA excluded, the central message is likely salvageable and valuable. The absence of repeated runs and the selection-on-test issue are also important but can be addressed with additional experiments or careful reframing. I would be willing to re-review a revised version that directly tackles these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The practical takeaway: this is one of the few papers that actually varies the number of training languages and the English/non-English split under a fixed budget, and it does so in a controlled way. The finding that 100 languages with 25-50% non-English data works well, without an English collapse, is plausible and useful. The Llama 3 repeats and the detailed ablations make it sturdier than most multilingual VLM papers. I'd trust the qualitative trends.\n\nOn the credit side: the setup is transparent -- 13 tasks, 43 languages, tiered analysis; the SMPQA benchmark is deterministic and isolates OCR from language understanding; and the limitations section is honest about MT noise, the synthetic-only OCR data, and the limited image resolution.\n\nThe soft spots are real but not fatal, mostly.\n\n1. xGQA leakage. Table 8 shows GQA (72k images) and VG (86k images) in the instruction mix. Appendix C.4 says xGQA is built from 300 Visual Genome images. Even if the GQA train/test split is image-disjoint, the VG training portion plausibly contains the same images. Since xGQA is in the RQ1/RQ2 averages and the final Centurio comparison, part of the reported multilingual gain may come from image memorization rather than newly learned ability. That needs fixing, either by dropping xGQA from the averaged numbers or by verifying image-disjointness.\n\n2. No error bars, single run per config. Annoying but common at this compute scale.\n\n3. The 50% English value is selected from the same experiments used to support the recommendation. Minor circularity, but the plateau across 25-75% English makes the recipe robust to that choice.\n\n4. SMPQA is brand-new, with no external validation. Still, the non-Latin script failures without OCR training are dramatic and line up with MTVQA results, so I'm not worried about the direction of that finding.\n\nAlso, code and data are not released. For a recipe paper, that is a real cost: the value is in the exact mixes and splits, and the community cannot reproduce them.\n\nSummary: this deserves a serious referee, not a desk reject. The xGQA overlap is the main issue to push on; the rest holds up as honest, careful experimentation. I would read it, I would cite the recipe, and I would want the released data.","headline":"A useful, mostly solid empirical recipe for multilingual LVLM training, but the xGQA training/evaluation overlap makes the headline numbers provisional.","tokens_in":85314,"tokens_out":2461,"would_cite":true,"duration_ms":26912,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A vision-language model can be trained on 100 languages simultaneously, with as little as 25–50% of the data non-English, without degrading English performance.","keywords":["multilingual vision-language models","training data composition","curse of multilinguality","machine-translated training data","language fidelity","multilingual OCR","low-resource languages","synthetic OCR data"],"falsifier":"Re-run the key English-only versus L100 comparison on xGQA after removing GQA from the instruction-tuning mix, or on a never-released multilingual VQA test set; if the reported multilingual gains shrink toward zero, the central in-language training effect is largely an artifact of test-train overlap.","tokens_in":84284,"feed_emoji":"🌐","tokens_out":14031,"duration_ms":120076,"temperature":0.7,"pith_summary":"The paper claims that the familiar \"curse of multilinguality\" — the trade-off where each added language costs performance in the others — does not appear in vision-language models at practical training budgets. By machine-translating identical English instruction and caption data into up to 99 other languages, the authors show that scaling from 7 to 100 training languages greatly improves non-English performance and language fidelity while leaving English and previously covered languages essentially untouched. The optimal mix is surprisingly English-heavy: 25–50% non-English data captures nearly all of the multilingual gain, and larger non-English shares can even hurt. For text inside images, only explicit synthetic OCR data in the target script moves performance, mainly for Latin scripts. If these results hold, a mostly-English training pipeline with a translated minority is enough to serve 100 languages, which matters because it puts massively multilingual models within reach of ordinary budgets.","feed_headline":"Training on 100 languages needs as little as 25–50% non-English data","feed_subtitle":"Scaling from 7 to 100 languages lifts low-resource performance with almost no cost to English.","key_machinery":"The machinery is a controlled data-mix experiment: identical high-quality English data (the LLaVA-Next instruction mix and ShareGPT4v dense captions) is machine-translated via NLLB into 99 languages, and models are trained under a fixed budget in the standard two-stage LLaVA protocol — caption pre-training, then instruction tuning — with LoRA on a multilingual LLM, changing only the language count and the English percentage. This isolates language composition as the single driver under study. The second mechanism is synthetic OCR: Synthdog-generated plot-style images with text sampled from each language's Wikipedia, probed by a new benchmark, SMPQA (Synthetic Multilingual Plot Question Answering), which splits reading text from an image and grounding prompt text to image text into separately scored tasks.","core_discovery":"The paper's central claim is that the multilingual behavior of a vision-language model is set almost entirely by which languages are in the training data, and hardly at all by how much data each language gets. In fixed-budget controlled runs, the authors machine-translate the identical English instruction and caption datasets into up to 99 other languages and vary only (a) the number of training languages, from 7 to 100, and (b) the English share of the budget, from 1% to 90%. Scaling to 100 languages sharply improves performance for the newly added languages — especially language fidelity, the ability to answer in the language asked — while costing previously covered languages and English only a negligible tax. The effective mixing ratio is English-heavy: 25–50% non-English data captures most of the multilingual gain, and larger non-English shares can even degrade some tasks. For text rendered inside images, translated prompts alone do nothing; only synthetic OCR data in the target script, added in pre-training and instruction tuning with the image encoder unfrozen, moves performance, and mostly for Latin scripts. Assembling these rules into a final recipe yields Centurio, a 100-language model that matches leading open vision-language models on English and high-resource languages and surpasses them on lower-resource ones across 14 tasks and 56 languages.","pith_inferences":["Editorial inference: because GQA appears in the training mix while xGQA reuses GQA's images and translated questions, part of the measured multilingual gain on xGQA may be test-train overlap; a contamination-free re-run could leave the paper's other benchmarks carrying more of the argument.","Editorial inference: the persistent Latin-versus-non-Latin gap suggests the vision encoder, not language data volume, bounds text-in-image ability; a directly testable consequence is that an OCR-pretrained, script-aware encoder should close the gap faster than more synthetic data.","Editorial inference: the 25–50% sweet spot was found at one architecture scale with 384px inputs, so the ratio may shift for much larger models, native-resolution encoders, or much bigger OCR budgets rather than transferring unchanged.","Editorial inference: if the ratio transfers, the cheapest route to massively multilingual assistants is machine-translating a minority share of an existing English-only pipeline — a cost argument that follows from the paper's data but is not its headline."],"forward_implications":["A 100-language vision-language model is affordable: keep roughly half the training data in English and machine-translate the rest, and low-resource languages improve sharply while English performance is retained.","The curse of multilinguality does not bite at these budgets: growing the language set from 7 to 100 costs previously covered languages almost nothing, so coverage decisions can be made optimistically.","More multilingual data is not automatically better: 25–50% non-English data saturates the gains in both pre-training and instruction tuning, and larger non-English shares can degrade some tasks.","Multilingual text-in-image ability requires explicit synthetic OCR data in the target script plus an unfrozen image encoder; Latin-script languages respond quickly while non-Latin scripts remain far behind.","Following the recipe end-to-end produces Centurio, which matches leading open multilingual vision-language models on English and high-resource languages and outperforms them on low-resource languages across 14 tasks and 56 languages."],"supporting_citations":[{"why":"Defines the \"curse of multilinguality\" in cross-lingual models that this paper tests against and reports as absent in vision-language models.","marker":"(Conneau et al., 2020)"},{"why":"Provides the second statement of the curse and modular pre-training as prior mitigation, framing the RQ1 comparison.","marker":"(Pfeiffer et al., 2022b)"},{"why":"Supplies NLLB, the open machine-translation model that produces all 99 non-English training copies used in the controlled mixes.","marker":"(Costa-jussà et al., 2022)"},{"why":"Supplies the five-tier resourceness taxonomy used to group training and evaluation languages throughout the study.","marker":"(Joshi et al., 2020)"},{"why":"Provides the LLaVA visual instruction tuning architecture and two-stage training protocol that all experiments build on.","marker":"(Liu et al., 2023b)"},{"why":"Provides the LLaVA-Next instruction mix that is translated and re-balanced for the language-ratio experiments.","marker":"(Liu et al., 2024)"},{"why":"Provides the ShareGPT4v dense captions used as the pre-training corpus in the RQ3 experiments.","marker":"(Chen et al., 2024b)"},{"why":"Provides Synthdog, the synthetic text-in-image generator used to create multilingual OCR training data for RQ4.","marker":"(Kim et al., 2022)"},{"why":"Concurrent Pangea result that 20–80% English data works well, corroborating the paper's 25–50% sweet spot.","marker":"(Yue et al., 2024a)"}],"fun_headline_variants":["100 languages for LVLMs: 25-50% non-English data suffices","Scaling LVLMs to 100 languages costs little English performance","Number of languages beats data ratio in multilingual LVLMs","Centurio: 100 languages with minimal non-English training data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes the test benchmarks were not seen in training, and that assumption fails for xGQA, whose images and translated questions come from GQA, a dataset the model is trained on, so part of the reported multilingual gain may be memorization rather than genuine in-language transfer.","fun_headline_variants_meta":{"raw":{"variants":["100 languages for LVLMs: 25-50% non-English data suffices","Scaling LVLMs to 100 languages costs little English performance","Number of languages beats data ratio in multilingual LVLMs","Centurio: 100 languages with minimal non-English training data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1675,"prompt_tokens":1087,"completion_tokens":588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":703,"completion_tokens_details":{"reasoning_tokens":511}},"tokens_in":703,"tokens_out":588,"duration_ms":6007,"temperature":1.0,"reasoning_tokens":511,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:18:13.026237+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the key English-only versus L100 comparison on xGQA after removing GQA from the instruction-tuning mix, or on a never-released multilingual VQA test set; if the reported multilingual gains shrink toward zero, the central in-language training effect is largely an artifact of test-train overlap.","supporting_citations":[],"review_version":1}