{"id":"64687eb7-5282-47e1-b305-81251280a913","arxiv_id":"2607.22300","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Zero-data merging of Arabic and Persian biomedical LoRA adapters comes within 1.4–3.5 CHrF++ points of supervised adaptation for Dari; Pashto and Sorani Kurdish stay below usable quality.","lead":"This paper tests whether translation models trained on Arabic and Persian medical text can be reused for Dari, Pashto, Sorani Kurdish, and Urdu without large target-language datasets. It finds that combining the two pivot adapters by simple weight-averaging comes close to supervised fine-tuning for Dari, while Pashto and Sorani Kurdish remain far below usable quality.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline numbers rest on single-seed runs; the 1.36–3.5 CHrF++ gaps may be within noise.","rationale":"The paper is a systematic, honest, and clearly written empirical study. The qualitative finding — that adapter merging gives a cost-free boost for closely related languages, and that 500-sentence adaptation helps Dari and Urdu more than Pashto and Sorani Kurdish — is plausible and consistent with prior cross-lingual transfer literature. The authors also candidly list limitations including metric adequacy and confounding factors. However, the central claim is a quantitative one: merging comes within 1.36–3.5 CHrF++ of supervised adaptation. The single-seed, no-variance design means these numbers are point estimates with no error bars; in low-resource MT, gaps of a few CHrF++ points are frequently within run-to-run variability, especially when fine-tuning with only 500 sentences. The dev-to-test flip of which adapter initialization is best for Dari, and the drop of AR-initialized adaptation from 41.01 to 36.95, strongly suggest high variance. This concern is more load-bearing than the reader's weakest_assumption about CHrF++ fidelity: even a semantically perfect metric cannot make a noise-level difference reliable. The reader's concern about metric validity and the 500-sentence redirect is legitimate but secondary; the numbers need to be stable before we can ask what they mean. I therefore agree with the CONDITIONAL verdict but would base the condition on repeated-seed reporting and ideally code/data release, rather than solely on metric validity. My proposed concrete test — multiple seeds with confidence intervals — would directly settle whether the reported gaps are real.","tokens_in":18741,"tokens_out":9719,"duration_ms":83727,"concrete_test":"Re-run the key experiments (Dari adaptation initialized from the Arabic adapter, Dari adaptation initialized from the Persian adapter, and simple-average adapter merging) with at least 5 different random seeds, including different random samples of the 500 FLORES-200 sentences for adaptation. Compute mean CHrF++ and 95% bootstrap confidence intervals for the difference between merging and each supervised-adaptation baseline on both the dev and test sets of TICO-19. If the confidence interval for the gap includes 0 or has width greater than, say, 5 CHrF++, the central claim that merging approximates supervised adaptation is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All central quantitative claims — the 3.5-point dev-set gap and 1.36-point test-set gap between zero-data adapter merging and supervised adaptation for Dari, the near-pivot 41.01 for Dari, and the meaningful 28.88 for Urdu — come from single runs with a fixed seed (B.1: 'A fixed random seed of 3407 is used throughout') and no variance or significance testing. The paper's headline contribution is that merging 'reaches within 3.5 CHrF++ of supervised adaptation for Dari'; this is essentially a claim about the size of a difference. If that difference is within run-to-run noise, the claim is unsupported. The instability of the comparison across dev and test sets reinforces this concern: on the dev set, the best Dari adaptation is AR-initialized (41.01); on the test set, it is FA-initialized (38.52), while AR-initialized adaptation on the test set (36.95) is actually below the merge score (37.16). Such a large shift in the identity and value of the best adaptation is consistent with high variance. Additionally, per-target α values for merging are selected on the dev set, which can overstate the zero-data claim for languages other than Dari, though for Dari the headline uses simple averaging and thus avoids that particular confound. Without repeated seeds and confidence intervals, the reported 1.36–3.5 CHrF++ gaps are not interpretable as robust evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies cross-lingual biomedical MT from English into four low-resource Arabic-script languages (Dari, Pashto, Sorani Kurdish, Urdu) using Arabic and Persian as pivots. The authors train LoRA adapters on 25k biomedical sentence pairs for en→ar and en→fa, then compare three transfer strategies: few-shot in-context learning, minimal supervised adaptation on 500 general-domain FLORES-200 sentences, and zero-data merging of the two pivot adapters via simple/weighted averaging, TIES, and DARE. Evaluation is on the held-out TICO-19 dev/test sets with CHrF++. The central claims are that supervised adaptation reaches near-pivot quality for Dari (41.01 dev, 36.95–38.52 test) and meaningful quality for Urdu (28.88), while zero-data adapter merging comes within 3.5 CHrF++ (dev) and 1.36 CHrF++ (test) of supervised adaptation for Dari, and that Pashto/Sorani remain far from usable. A secondary finding is an apparent model inversion where Llama 3.2 3B transfers better than Gemma 2 2B despite weaker pivot performance.","tokens_in":19008,"tokens_out":10180,"duration_ms":81117,"significance":"If the results are robust, the paper makes a useful practical contribution: it demonstrates a computationally cheap, zero-target-data route to biomedical MT for a closely related language cluster, and it does so with a clean evaluation protocol that keeps TICO-19 entirely out of training and uses an external benchmark for both model selection and final testing. The comparison of adapter merging against joint multilingual training and supervised adaptation is informative, and the candid limitations section correctly identifies most residual confounds. The main risk is that all headline differences—especially the 1.36–3.5 CHrF++ margins that define the merging claim—are based on a single training run per configuration, so the paper's quantitative contribution is not yet statistically grounded. The work is within the scope of the journal and, after the variance and reporting issues are addressed, could be an accept.","major_comments":[{"comment":"All headline quantitative claims rest on a single run with a fixed seed (B.1: 'A fixed random seed of 3407 is used throughout'). The abstract's 'within 3.5 CHrF++' (dev) and §5.6's 'within 1.36 points' (test) are statements about the size of a difference, but no variance estimates, confidence intervals, or significance tests are provided. The instability of the comparison across splits reinforces the concern: on dev, the best Dari adaptation is AR-initialized (41.01) and on test it is FA-initialized (38.52), while AR-initialized test adaptation (36.95) falls below the simple-average merge score (37.16). This pattern is consistent with high run-to-run variability. I request multiple seeds (at least 3–5) for the headline configurations, with means and bootstrap CIs or paired tests, so the 1.36–3.5 CHrF++ margins are interpretable.","section":"§B.1, Tables 2–3, §5.5–5.6"},{"comment":"The caption says 'Stage 4 uses Llama 3.2 3B; all other stages use Gemma 2 2B.' If 'Stage 4' refers to adaptation fine-tuning (§B.4), then the test-set adaptation scores in Table 3 (AR 36.95, FA 38.52) are Llama, while the merge rows are Gemma. The sentence in §5.6 'within 1.36 points of supervised adaptation' would then compare a Gemma merge against a Llama adaptation, which is not an apples-to-apples comparison. If all adaptation and merge rows in Table 3 are actually Gemma, the caption must be corrected and the notion of 'stage' defined. Please clarify the base model for every row/column in Tables 2–3 and ensure that any merge-vs-adaptation comparison holds the base model fixed.","section":"Table 3 caption, §5.6"}],"minor_comments":[{"comment":"Table 2 appears to report development-set scores but the caption never says 'dev'; Table 3 is the test set but only §5.6 identifies it as such. Add explicit 'development' / 'test' labels to the captions and in the running text.","section":"Tables 2–3"},{"comment":"The model-inversion claim cites specific numbers (Pashto 16.70 vs. 6.28; Urdu 22.26 vs. 9.96) that do not all appear in Table 2, which shows only the Llama AR+FA few-shot condition. Report the full few-shot matrix for both base models so the claimed inversion is verifiable.","section":"§5.3"},{"comment":"The sentence 'Table 1 shows the full comparison' should presumably refer to Table 2; Table 1 is the linguistic-difficulty table. Please correct the cross-reference.","section":"§5.5"},{"comment":"Text normalization is applied to Arabic, Persian, and Dari but not to Pashto, Sorani Kurdish, or Urdu, because no standardized tools exist. Table 4 draws cross-language conclusions from CHrF++ scores that are therefore not strictly comparable across languages. Please add a sensitivity analysis or an explicit caveat that CHrF++ differences across languages may partly reflect normalization asymmetry.","section":"§4.2.2, §4.3"},{"comment":"Minor wording issues: the citation '(Talwar and Laasri, 2025)' is inserted mid-sentence after 'pivot languages' in an awkward place; and §B.3 has a typo, 'monolingual dapters'.","section":"§1 and §B.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is promising and the core idea is reasonable, but the single-seed evidence is the main risk. The Table 3 caption ambiguity also needs careful resolution because it affects the validity of the test-set headline number. I would be willing to accept after the authors provide multi-seed results (or otherwise justify why single-seed differences of 1–3 CHrF++ are stable) and clarify the base-model assignment in Tables 2–3. I do not think the lack of human evaluation is by itself a blocker given the paper's metric-based scope and the authors' explicit limitations, though it would strengthen a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is a clear, systematic empirical study of cross-lingual transfer for biomedical MT into Dari, Pashto, Sorani Kurdish, and Urdu, using Arabic and Persian as pivots. The genuinely new part is the zero-data LoRA adapter-merging comparison against few-shot ICL and 500-sentence supervised adaptation. The evaluation design is solid: TICO-19 held out for evaluation, separate dev/test splits, clean separation from training data, and a defensible metric for morphologically rich languages. The authors also deserve real credit for reporting that Pashto and Sorani remain unusable and for a Limitations section that concedes the main confounds — metric adequacy, dataset characteristics, and the difficulty of separating linguistic distance from shared script and pretraining exposure. That honesty matches what the results actually show.\n\nThe central qualitative findings hold up. Supervised adaptation with roughly 500 general-domain sentences gets near-pivot quality for Dari and meaningful gains for Urdu; merging comes close to adaptation for Dari at zero data cost; and transfer quality tracks linguistic proximity. The model-inversion finding is interesting, but the comparison is confounded (Gemma single-adapter versus Llama combined-adapter few-shot), so I would treat it as a hypothesis, not a result.\n\nThe softest spot is the precision of the headline numbers. Everything is one run per configuration with a fixed seed, no confidence intervals, no significance testing. The dev/test instability in which adapter gives the best Dari result suggests real variance, so the reported 1.36–3.5 CHrF++ gaps between merging and adaptation could be within noise. That matters because the headline contribution is essentially a claim about the size of a difference. The per-target alpha selection on the dev set is a milder issue; the Dari headline uses simple averaging, so the main zero-data claim avoids that particular confound. The lack of code release also makes it harder to treat the numbers as reproducible.\n\nNone of this invalidates the paper. The large margins over zero-shot baselines and the consistent ordering across dev and test are reassuring. But the authors need multiple seeds and ideally significance testing before their specific gaps are cited by others. A careful reviewer should push for that, plus a controlled model comparison for the inversion claim and code release.\n\nWho this is for: people working on low-resource MT, adapter merging, or health NLP for Arabic-script languages. It deserves a serious referee — the empirical setup is careful and the question is relevant — but it needs revision, not acceptance as is.\n\nRecommendation: send to peer review, with a request for variance reporting and code.","headline":"A systematic, honestly-written empirical study of adapter merging for biomedical MT into four Arabic-script low-resource languages; the headline gaps are plausible but rest on single-seed runs, so the quantitative claims need variance reporting before they hold weight.","tokens_in":19590,"tokens_out":1888,"would_cite":true,"duration_ms":17023,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Zero-data LoRA adapter merging of Arabic and Persian biomedical adapters reaches within 3.5 CHrF++ points of supervised adaptation for Dari on the dev set and within 1.36 points on the test set, while supervised adaptation with just 500 sen","keywords":["biomedical machine translation","cross-lingual transfer","LoRA adapter merging","low-resource languages","Arabic script","Dari","zero-data adaptation","CHrF++ evaluation"],"falsifier":"Have a qualified translator review 200 Dari sentences produced by the merged adapter and by the supervised-adaptation model. If the merged outputs carry a clinically significant error rate no better than the zero-shot baseline, despite the 1.36-point CHrF++ gap, the claim that zero-data merging approaches supervised quality for clinical use collapses.","tokens_in":18544,"feed_emoji":"🩺","tokens_out":4403,"duration_ms":34401,"temperature":0.7,"pith_summary":"Biomedical translation is a lives-at-stake task, yet four Arabic-script languages—Dari, Pashto, Sorani Kurdish, and Urdu—have almost no healthcare-domain parallel data. The paper tries to establish that knowledge encoded in Arabic and Persian biomedical LoRA adapters can be transferred to these targets without any target-language biomedical data. Its central finding is that zero-data adapter merging comes within 3.5 CHrF++ points of supervised adaptation for Dari on the dev set and within 1.36 on the test set, while supervised adaptation with just 500 general-domain sentences matches pivot-level quality for Dari and gives meaningful Urdu gains. Sympathetic readers should care because, if true, this offers a nearly free path to initial biomedical MT for closely related low-resource languages, and it sharply exposes where transfer stops working (Pashto, Sorani).","feed_headline":"Zero-data trick nearly matches supervised biomedical MT for Dari","feed_subtitle":"Arabic and Persian adapters, merged in weight space, translate Dari within 1.4 CHrF++ of supervised—no target data.","key_machinery":"LoRA adapters: rank-16 low-rank matrix updates on a frozen decoder-only LLM, so the only learned parameters per language are small adapter matrices; because both adapters share the same frozen base, their differences from the base form task vectors that live in the same parameter space. Four merging functions (simple/weighted averaging, TIES-Merging, DARE) combine the Arabic and Persian task vectors to produce a zero-data multilingual adapter; the merging function does the work of redirecting biomedical knowledge to target languages.","core_discovery":"The paper demonstrates that a merged LoRA adapter, formed by tensor arithmetic over the Arabic and Persian task vectors from a shared frozen base model, yields Dari translations within 3.5 CHrF++ (dev) and 1.36 (test) of supervised adaptation—without any Dari training data—and that fine-tuning either pivot adapter on only 500 FLORES-200 sentences produces near-pivot quality for Dari (41.01) and meaningful quality for Urdu (28.88). It further finds a model inversion: the weaker pivot model transfers better than the stronger one, suggesting a trade-off between pivot specialisation and cross-lingual generalisation.","pith_inferences":["Editorial inference: the same merging recipe could be tested on other script-sharing families (e.g., Hindi–Urdu, or Spanish–Portuguese biomedical adapters) to see whether the close-relative result generalises.","Editorial inference: because the adaptation data is general-domain, replacing it with even 200 in-domain biomedical sentences may push Pashto and Sorani over the usability threshold; this is a direct testable extension.","Editorial inference: the CHrF++ metric rewards surface similarity, so the headline 1.36-point gap may not reflect clinical adequacy; a small human review of Dari outputs is the natural next check."],"forward_implications":["Zero-data adapter merging is a viable starting point for closely related low-resource languages when no target-language data exists.","500 sentences of general-domain data are enough to reach near-pivot quality for Dari, so minimal supervision can replace large in-domain corpora for close relatives.","Pivot choice becomes irrelevant once target-language supervision is available; the Arabic and Persian adapters converge within 1.0 CHrF++ after adaptation.","Pashto and Sorani Kurdish cannot be brought to clinically usable quality by cross-lingual transfer alone; in-domain data or a closer pivot is required.","The model inversion result implies that choosing the best pivot model for the pivot language is not the same as choosing the best model for transfer."],"fun_headline_variants":["Merged LoRA adapters translate Dari without a single training sentence","Zero-data adapter merge brings Dari MT within 1.4 CHrF++ of supervised","Weaker pivot transfers better: LoRA merge beats stronger for Dari","Biomedical Dari MT without data: LoRA merge close to supervised","Zero-data Dari MT: merged LoRA adapters nearly match supervised tuning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central premise is that 500 general-domain sentences can redirect biomedical knowledge already stored in the pivot adapters, and that CHrF++ gains reflect translation adequacy in a medical setting; if either fails, the headline numbers overstate deployable quality.","fun_headline_variants_meta":{"raw":{"variants":["Merged LoRA adapters translate Dari without a single training sentence","Zero-data adapter merge brings Dari MT within 1.4 CHrF++ of supervised","Weaker pivot transfers better: LoRA merge beats stronger for Dari","Biomedical Dari MT without data: LoRA merge close to supervised","Zero-data Dari MT: merged LoRA adapters nearly match supervised tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00053,"raw_usage":{"total_tokens":2410,"prompt_tokens":781,"completion_tokens":1629,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":1530}},"tokens_in":525,"tokens_out":1629,"duration_ms":11188,"temperature":1.0,"reasoning_tokens":1530,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T05:11:13.285360+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have a qualified translator review 200 Dari sentences produced by the merged adapter and by the supervised-adaptation model. If the merged outputs carry a clinically significant error rate no better than the zero-shot baseline, despite the 1.36-point CHrF++ gap, the claim that zero-data merging approaches supervised quality for clinical use collapses.","supporting_citations":[],"review_version":1}