{"id":"48b67667-83bf-4f23-b3cb-334fb42d3bcb","arxiv_id":"2508.12774","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"The authors present SALAMANDRATA, improved multilingual translation models built by continual pre-training on parallel data and supervised fine-tuning, with the 7B variant submitted to WMT25.","lead":"The paper introduces SALAMANDRATA, a family of two machine translation models (2B and 7B parameters) trained for 38 European languages and additional non-European languages for the WMT25 shared task. The 7B variant is the team's submission, and both models are released on Hugging Face.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified; central claim currently unverified because abstract lacks evaluation evidence.","rationale":"The reader's weakest_assumption focuses on the reliability of the training recipe, especially possible catastrophic forgetting during vocabulary adaptation and continual pre-training. That is a plausible concern, but without the full text it is not the most load-bearing issue identifiable. The more direct issue is that the abstract asserts strong performance without presenting any evaluation metrics, baselines, or comparisons. Since the full text was unavailable, I cannot assess internal consistency or technical correctness. The reader's UNVERDICTED verdict with low confidence is therefore the appropriate outcome. I agree with the reader in spirit but would shift the emphasis from the training-recipe assumption to the complete absence of reported evaluation evidence in the abstract, which makes the central claim unverifiable rather than false.","tokens_in":649,"tokens_out":2003,"duration_ms":25600,"concrete_test":"Obtain the full paper and reproduce the main evaluation table: compare SALAMANDRATA 7B's chrF/BLEU/COMET against the stated baselines and WMT25 official results for all 38+ languages. If the reported gains are within noise or only appear on a few languages, the 'strong performance' claim should be weakened. Additionally, verify the dev/test split used for the COMET-KIWI re-ranking to ensure no test-set leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that SALAMANDRATA 7B achieves strong performance for WMT25 across 38 European and additional non-European languages. What would have to be true is that the described continual pre-training on parallel data plus supervised fine-tuning and the two decoding strategies actually produce translation quality that is competitive on the official test sets, without catastrophic forgetting or mismatch from vocabulary adaptation. The abstract asserts this ('strong performance', 'carefully designed') but provides no evaluation numbers, baselines, or language-specific results. Because only the abstract was available, there is no internal inconsistency to point to; the claim is simply unsupported on the available evidence. In good faith, I cannot identify a specific technical flaw from the abstract alone. The load-bearing concern is epistemic: acceptance of the central claim depends on results that the abstract does not report.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript, available only as an abstract, announces the SALAMANDRATA family of multilingual translation models (2B and 7B) from the Barcelona Supercomputing Center for the WMT25 General Machine Translation shared task. The proposed recipe is two-stage: continual pre-training on parallel data followed by supervised fine-tuning on high-quality instructions, with a vocabulary adaptation step for additional non-European languages. The submission is based on the 7B variant, and decoding uses Minimum Bayes Risk decoding and COMET/COMET-KIWI-based tuned re-ranking. The abstract claims 'strong performance' across 38 European languages and additional non-European languages, and states that both 2B and 7B models, plus a newer SALAMANDRATA-V2, are publicly released on Hugging Face.","tokens_in":854,"tokens_out":2048,"duration_ms":26390,"significance":"If the claimed performance is substantiated in the full text, this work would be a useful open-source contribution to multilingual machine translation, particularly for European languages and the WMT shared task. The practical strengths that are visible from the abstract include the public release of models, the use of two quality-aware decoding strategies, and a clearly stated two-stage training recipe. However, because the abstract contains no quantitative evaluation results, no baselines, and no language-level breakdown, the central empirical claim is currently unverified. The significance of the work cannot be properly assessed without the evaluation section; the potential contribution is real but conditional on the missing evidence.","major_comments":[{"comment":"The central claim 'strong performance' is asserted without any supporting evaluation numbers, such as BLEU or COMET scores, and without comparison to a baseline system. For a shared-task system description, official test-set results are the load-bearing evidence. Please add a results section with at least aggregate scores over the task languages and, if space permits, per-language or per-language-family results, along with the evaluation protocol (test sets, metrics, number of systems compared).","section":"Abstract"},{"comment":"The training recipe is described as 'carefully designed to optimize performance across all translation directions,' but no evidence is provided for this claim. Critical details such as the composition and size of the parallel data, the vocabulary adaptation procedure, the fine-tuning instruction data, and the hyperparameters are absent from the abstract. If these are present in the full text, please make them explicit; if they are not, include a description of the design and, ideally, an ablation or at least a discussion of why the design was chosen.","section":"Abstract"},{"comment":"The abstract does not address the risk of catastrophic forgetting during continual pre-training and fine-tuning, which is a common concern when adapting a multilingual model to new languages and tasks. Since the claim is that performance is strong 'across all translation directions,' the manuscript should report whether performance on previously supported languages or general capabilities was maintained. A simple evaluation before and after adaptation would strengthen the claim.","section":"Abstract"}],"minor_comments":[{"comment":"The phrase 'Tuned Re-ranking using COMET and COMET-KIWI respectively' is grammatically unclear. The sentence lists two decoding strategies but the 'respectively' suggests a pairing that is not fully specified. Please rephrase, e.g., 'Minimum Bayes Risk Decoding and tuned re-ranking with COMET/COMET-KIWI.'","section":"Abstract"},{"comment":"The abstract introduces 'SALAMANDRATA-V2' without explaining its relationship to the 2B and 7B models. If this model is part of the submission, clarify its role; otherwise, it may be a distraction in an already dense abstract.","section":"Abstract"},{"comment":"Minor typographical/presentation issue: the Hugging Face reference is followed by '1' without a visible footnote marker in the abstract text. Please ensure footnote formatting is consistent.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The review is based solely on the abstract, as no full text was provided. The main concern is not a technical error but the absence of evaluation evidence for the central claim. If the full manuscript contains the relevant results, a revision that adds them to the abstract or makes them more prominent should be sufficient. The editor may wish to obtain the full text before making a final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a systems paper for WMT25, and the abstract is exactly what you'd expect—a description of a training recipe and a claim of 'strong performance' with zero numbers. If you're deciding whether to look at the full version, the interest hinges on whether the actual test results are reported there. The release of the models on Hugging Face is real value, assuming the checkpoints exist and are usable.\n\nWhat's actually new: the SALAMANDRATA checkpoints, the vocabulary adaptation for non-European languages, and the use of MBR and COMET-KIWI reranking for this specific shared task. That's incremental over SALAMANDRA, but legitimate incremental work. The paper does a good job, in the abstract, of stating the pipeline in plain terms: continual pre-training on parallel data, then SFT, then decoding strategies. No mystery.\n\nThe soft spots are proportionate to the fact that we only have the abstract. The central sentence—'strong performance'—is unsupported. No BLEU, no COMET, no baselines, no language-by-language breakdown. For a paper claiming to optimize across 38 languages, you need to see the per-language scores and ideally comparisons to the prior SALAMANDRA and to other WMT submissions. The 'carefully designed' second phase is an assertion, not a demonstration. There's also no discussion of catastrophic forgetting after vocabulary adaptation, which is a real risk when you widen the tokenizer.\n\nI want to be clear: none of this is a technical flaw I can point to, because the abstract doesn't contain enough substance to have a flaw. It's an epistemic issue. Either the full paper reports numbers on the WMT25 test sets, in which case this is a useful system description, or it doesn't, in which case the paper is a model release note with an unsupported performance claim.\n\nWho'd get value: anyone building multilingual MT systems for European languages, especially lower-resource ones, and anyone tracking the BSC model family. If you're not in MT, this won't change your day.\n\nRecommendation: if you're the editor, send it to review only if the full text actually contains evaluation results. If the full paper matches the abstract alone—claims without scores—desk reject. But given the open release and the shared task context, I'd give it a fair chance with a referee who can check the numbers. My own verdict is not positive on the abstract, but the artifact could be solid.","headline":"Abstract-only WMT25 system paper that promises open 7B/2B MT models for 38 European languages; the recipe is sensible but the central performance claim is entirely unevidenced in the abstract.","tokens_in":1260,"tokens_out":1733,"would_cite":false,"duration_ms":18885,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-step training yields a 38-language translator from the SALAMANDRA base","keywords":["SALAMANDRATA","machine translation","continual pre-training","supervised fine-tuning","WMT25","multilingual LLM","Minimum Bayes Risk","COMET reranking"],"falsifier":"Run a head-to-head ablation on the released 7B model: train a control model with the same supervised fine-tuning data but without the parallel-data continual pre-training, and evaluate both on the same held-out translation benchmark across the 38 European languages. If the control matches or exceeds the full recipe on BLEU/COMET, the paper's central claim about the recipe's contribution collapses.","tokens_in":643,"feed_emoji":"🌍","tokens_out":6343,"duration_ms":57525,"temperature":0.7,"pith_summary":"This paper presents SALAMANDRATA, a family of 2B and 7B language models built from the SALAMANDRA base and trained specifically for machine translation. The core proposal is a two-stage recipe: first continual pre-training on parallel data, then supervised fine-tuning on high-quality instruction data. The authors submit the 7B variant to the WMT25 General Machine Translation shared task, with the vocabulary extended to cover additional non-European languages and decoding improved by Minimum Bayes Risk and COMET-based reranking. The paper claims this combination achieves strong performance across 38 European languages and the extra languages of the task. The models are released publicly.","feed_headline":"SALAMANDRATA 7B: two training passes give a 38-language translator","feed_subtitle":"Parallel-data pretraining, instruction fine-tuning, and COMET reranking drive the WMT25 translation submission.","key_machinery":"The load-bearing mechanism is the two-stage training recipe: (1) continual pre-training on parallel data to realign the model toward translation, and (2) supervised fine-tuning on high-quality instructions to teach it to follow translation prompts. A third component, vocabulary adaptation, extends the tokenizer to cover the additional non-European languages before the second round of training. The decoding side adds quality-aware selection: Minimum Bayes Risk Decoding and tuned reranking with COMET/COMET-KIWI.","core_discovery":"The central claim is that a general-purpose multilingual LLM can be turned into a strong translation model by two successive training passes: continual pre-training on parallel sentence pairs, followed by supervised fine-tuning on high-quality translation instructions. Applied to the SALAMANDRA 7B model, after adapting its vocabulary to the non-European languages required by the WMT25 task, this recipe yields competitive translation quality across 38 European and additional non-European languages. The authors also introduce a second released variant, SALAMANDRATA-V2, and report that quality-aware decoding strategies—Minimum Bayes Risk Decoding and tuned reranking with COMET and COMET-KIWI—ar","pith_inferences":["The recipe is not translation-specific: continual pre-training on task-parallel data followed by instruction fine-tuning is a general blueprint for specializing a base LLM, so the same two-stage approach could plausibly be applied to other structured-generation tasks (e.g., summarization, code translation).","A controlled ablation that removes the parallel-data continual pre-training (keeping only the SFT step) would isolate the contribution of each stage; the paper's abstract does not report such an ablation, so the recipe's individual step contributions remain untested here.","Because the abstract reports no human evaluation, the 'strong performance' claim rests on the shared task's automatic metrics; a human study would test whether the quality holds up for real-world use.","The SALAMANDRATA-V2 model is mentioned but not described; comparing it to the base SALAMANDRATA would show how much the added training phase changes behavior."],"forward_implications":["If the recipe is effective, then a large general-purpose LLM can be specialized into a strong translator with continual pre-training on parallel data, rather than needing a dedicated translation model trained from scratch.","The vocabulary-extension step shows how to add new languages to an existing tokenizer without full retraining.","The two scales (2B and 7B) suggest the recipe is robust to model size, at least across that range.","The public release enables direct evaluation and reuse of the models by other researchers and practitioners.","The use of MBR and COMET/COMET-KIWI reranking at decoding time suggests that the same models can be improved further by better inference-time search."],"supporting_citations":[{"why":"Supplies the base SALAMANDRA model and pretrained weights that SALAMANDRATA extends with continual pre-training and fine-tuning.","marker":"(Gonzalez-Agirre et al., 2025)"}],"fun_headline_variants":["Two-step recipe turns SALAMANDRA into 38-language translator","From SALAMANDRA to SALAMANDRATA: two training passes for 38 languages","SALAMANDRATA: two-phase training for 38-language translation","SALAMANDRATA 7B: parallel pretraining plus instruction tuning for WMT25"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the two-stage recipe—parallel-data continual pre-training then instruction fine-tuning—improves translation for every target language without damaging the base model's existing abilities; the abstract asserts this but gives no comparative evidence.","fun_headline_variants_meta":{"raw":{"variants":["Two-step recipe turns SALAMANDRA into 38-language translator","From SALAMANDRA to SALAMANDRATA: two training passes for 38 languages","SALAMANDRATA: two-phase training for 38-language translation","SALAMANDRATA 7B: parallel pretraining plus instruction tuning for WMT25"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001153,"raw_usage":{"total_tokens":4627,"prompt_tokens":770,"completion_tokens":3857,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":514,"completion_tokens_details":{"reasoning_tokens":3767}},"tokens_in":514,"tokens_out":3857,"duration_ms":32480,"temperature":1.0,"reasoning_tokens":3767,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:17:11.654146+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a head-to-head ablation on the released 7B model: train a control model with the same supervised fine-tuning data but without the parallel-data continual pre-training, and evaluate both on the same held-out translation benchmark across the 38 European languages. If the control matches or exceeds the full recipe on BLEU/COMET, the paper's central claim about the recipe's contribution collapses.","supporting_citations":[],"review_version":1}