{"id":"373e844e-477c-4ded-9904-4f181ab3cd09","arxiv_id":"2508.19481","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Dictionary-guided reinforcement learning improves Spanish-to-Wayuunaiki translation BLEU from 0.93 to 4.45 on the AmericasNLP 2025 test set.","lead":"This paper trains small language models to consult a Spanish-Wayuunaiki dictionary while translating, using supervised fine-tuning plus reinforcement learning with BLEU as the reward. The approach raises BLEU scores by a few points over earlier systems on a shared-task test set, a real but modest gain for a severely low-resource language.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline BLEU gains rest on single-run differences of ~0.3 points; without repeated runs or significance tests, the claimed RL and dictionary improvements may be within run-to-run noise.","rationale":"The reader's weakest_assumption is that the headline effects rest on single-run experiments without variance estimates. My stress-test identifies the same load-bearing concern: the primary quantitative claims (SFT+RL vs. SFT, tool vs. no-tool) are based on one run per configuration, with differences as small as 0.34 BLEU. This directly undermines the central claim because the paper's own appendix admits only one successful run was considered. I considered other potential concerns—such as the dictionary coverage being limited to 10.4% of unique test words, possible overfitting to BLEU, or the novelty overstatements—but none is as load-bearing as the lack of repeated runs, because even if the method is conceptually sound, the reported effect sizes cannot be separated from noise. The reader's verdict of CONDITIONAL is appropriate: the request for repeated runs, significance tests, and released code is exactly what would settle this concern. I therefore recommend no change to the verdict, while emphasizing the concrete experimental check needed.","tokens_in":14271,"tokens_out":4181,"duration_ms":43471,"concrete_test":"Re-run the Qwen2.5-0.5B SFT and SFT+RL configurations with at least 5 random seeds each (varying LoRA initialization and RL seed), evaluate every run on the same 503-sentence test set using identical BLEU computation (ideally sacrebleu with the same tokenization), and report mean, standard deviation, and a paired bootstrap or Wilcoxon signed-rank test at the sentence level for SFT vs. SFT+RL and tool vs. no-tool. Also fix and document the evaluation decoding procedure (e.g., greedy with seed) and repeat. If the 0.34 BLEU gap falls inside the bootstrap confidence interval or fails to reach p<0.05, the claimed 11% and 18% relative gains are not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that adding GRPO reinforcement learning and an external dictionary tool improves Spanish-to-Wayuunaiki translation, as measured by BLEU, beyond supervised fine-tuning alone and beyond prior work. The quantitative support is Table 1/Figure 2: SFT 3.08 vs. SFT+RL 3.42 BLEU (an 11% relative gain) and a roughly 6% tool-vs-no-tool gain. Appendix A3 states: \"Only one successful run was considered for each experiment.\" For a 503-sentence test set, sentence-level average BLEU has high variance, and a 0.34 BLEU difference can easily arise from LoRA initialization seeds, RL sampling stochasticity, or evaluation decoding settings. The paper reports no error bars, no repeated runs, and no significance test for the primary comparisons. The only reported t-tests compare model outputs against \"dictionary-only\" BLEU (Section 4.1), which addresses a different question and does not establish that SFT+RL beats SFT. Consequently, the central causal claim that RL and dictionary access produce the observed gains is not yet distinguished from noise. This is a measurement and reproducibility concern rather than an internal inconsistency; the method may still be effective, but the evidence as presented is insufficient to support the headline percentages.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a tool-augmented translation pipeline for Spanish-to-Wayuunaiki, combining supervised fine-tuning (SFT) on parallel data with synthetic dictionary-use demonstrations, followed by GRPO reinforcement learning (RL) in which sentence-level BLEU against references serves as the reward. The base model is Qwen2.5-0.5B-Instruct, with additional experiments on Llama-3.2-1B, Qwen2.5-7B, and an NLLB checkpoint. The authors report that SFT+RL with dictionary access improves average BLEU from 3.08 (SFT) to 3.42 for Qwen-0.5B, that dictionary access adds roughly 6% relative BLEU, that Qwen-7B+SFT+RL reaches 4.45 BLEU, and that BLEU as a reward outperforms a character-level reward. The evaluation uses the 503-sentence AmericasNLP 2025 shared task test set. The paper also analyzes dictionary lookup success rates and shows that trained models outperform dictionary-only retrieval by a statistically significant margin.","tokens_in":14556,"tokens_out":4379,"duration_ms":41549,"significance":"If the reported gains are robust, the paper makes a useful empirical contribution: it is one of the first demonstrations of RL-based tool use for low-resource machine translation, it uses an official shared-task test set rather than a self-constructed one, and it includes ablations across model architectures, model sizes, reward types, and RL durations. The release of code and data is also valuable for reproducibility. However, the central quantitative claims currently rest on single-run comparisons with small BLEU differences, and BLEU is used both as the RL reward and as the evaluation metric. These issues do not invalidate the approach, but they make the headline improvements provisional until the measurement uncertainty is quantified.","major_comments":[{"comment":"The central claim that SFT+RL and dictionary access improve translation quality rests on very small BLEU differences measured from single runs. Appendix A3 states that 'Only one successful run was considered for each experiment,' and Tables 1-3 report no error bars or repeated seeds. The headline comparison, SFT 3.08 vs. SFT+RL 3.42 BLEU, is a 0.34-point (11% relative) difference, and the tool-vs-no-tool comparison is about 6% relative. On a 503-sentence test set, sentence-level BLEU has high variance, and LoRA initialization, RL sampling, or decoding settings could plausibly shift scores by this amount. The paired t-tests in Section 4.1 compare model outputs against dictionary-only retrieval, not SFT vs. SFT+RL, so they do not address whether the RL gain is significant. I would need repeated runs with different seeds, bootstrap confidence intervals over test sentences, or paired significance tests for the specific configuration comparisons that support the abstract's '18% relative gain' and RL-improvement claims.","section":"Appendix A3, Table 1, Figure 2"},{"comment":"BLEU is used both as the RL reward signal and as the evaluation metric. Because the policy is explicitly optimized against sentence-level BLEU, the reported test-set BLEU gains partly reflect better optimization of the training objective rather than independent evidence of improved translation quality. The paper's conclusion that the method 'improves translation quality' would be strengthened by reporting complementary automatic metrics (e.g., chrF, COMET) or at least a qualitative analysis of outputs. The authors acknowledge the lack of native-speaker evaluation in the Limitations section, but complementary metrics are feasible even without a native speaker and would help establish that the gains are not an artifact of reward-evaluation coupling.","section":"Section 3.2, Section 4"},{"comment":"The abstract's claim of 'up to +3.37 BLEU improvement over previous work' is not traceable to a specific comparison in the results. Section 4 states that the prior NLLB-based system achieved 0.93 BLEU on the curated test set, while the best model here (Qwen-7B+SFT+RL) reaches 4.45 BLEU, which would be a +3.52 difference, not +3.37. If the +3.37 refers to a different baseline or a different model configuration, that comparison should be explicitly identified; if it is derived from Table 2 or Figure 2, the exact numbers and the name of the prior system should be stated. As written, the headline quantitative claim is unverifiable from the tables.","section":"Abstract, Section 4"}],"minor_comments":[{"comment":"The expansion of GRPO is inconsistent: the abstract says 'Guided Reward Policy Optimization,' while Section 2 calls it 'Generalized Reinforcement Policy Optimization,' and the standard name is 'Group Relative Policy Optimization.' Please standardize the terminology.","section":"Abstract, Section 2"},{"comment":"The CharacTer reward is cited to Morris et al. (2004) on WER/MER, but CharacTer is a character-level translation edit rate metric introduced in a different paper (Wang et al., 2016, 'CharacTer: Translation Edit Rate on Character Level'). The current reference does not support the metric used.","section":"Table 3, References"},{"comment":"The text says the SFT model's mean BLEU 'reaches 3.07,' but Table 1 reports 3.08 for the same configuration. Please align the numbers.","section":"Section 4.1"},{"comment":"The statements that 'RL methods have yet to be explored in the context of machine translation' and that this is 'the first to apply RL to adapt LLMs in the translation context' are contradicted by the paper's own citation of Zhan et al. (2024) and by broader prior work on RL for MT. Please soften these overclaims.","section":"Section 2"},{"comment":"The caption says the dictionary tool provides an 'estimated 6% relative gain,' but Figure 2 does not show error bars or confidence intervals. Given the single-run setup, the caption should describe this as an observed difference in one run rather than an estimate with implied precision.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest and clearly written, and the experimental design is mostly sound apart from the single-run issue. The main risk is that the small reported gains may not replicate under different seeds, so I have made the addition of repeated runs or significance testing a load-bearing requirement rather than a stylistic suggestion. I would also encourage the editor to ask the authors to correct the GRPO terminology and the CharacTer citation before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuine new combination — GRPO RL plus an external bilingual dictionary tool for low-resource MT — and the paper is honest about its limits. The central result, though, rests on single-run BLEU differences of about 0.3 points, and there is no significance testing or variance estimate for the comparisons that matter. That is a real weakness, but it is a fixable measurement problem, not a broken method.\n\nWhat is actually new: the specific setup — an instruction-tuned LLM (Qwen2.5-0.5B, Llama 3.2-1B, Qwen-7B) that learns during GRPO when to query a filtered 29k-entry Spanish–Wayuunaiki dictionary, with BLEU as reward. The related work is appropriately cited: Search-R1, ReTool, SWiRL do RL tool-use for other tasks, Zhang et al. do RL for NMT without tools, and Briva-Iglesias does multi-agent MT without RL. The paper's 'first' claims overshoot slightly, but the combination is new.\n\nWhat it does well: the ablations are thoughtful. They test reward type (BLEU vs CharacTer), training duration (400 vs 1400 steps), model family and size, and tool access on/off. The tool-usage analysis (calls, success rate, dictionary coverage) is a nice addition. The authors also openly list limitations: no native-speaker evaluation, limited compute, and the dictionary's low coverage (10.4% of test words appear; 16.3% of those have a matching translation). That last point, honestly reported, actually tempers the headline — if the dictionary helps at all despite such sparse coverage, that is mildly interesting, but it also means the 'tool gain' is almost certainly not from retrieving correct answers.\n\nSoft spots, in order of severity. First, the single-run problem. Appendix A3 says 'Only one successful run was considered for each experiment.' The headline 18% relative gain (3.08 to 3.42 BLEU) and the ~6% tool gain are within the range where LoRA seeds or RL sampling could easily flip the ranking. The only significance tests they report compare model output against 'dictionary-only' BLEU, which is not the comparison that supports the central claim. Second, BLEU is used as both the RL reward and the evaluation metric. That is standard but worth flagging; the reported improvement is optimized against the same objective. Third, the paper says code is 'available at RLTranslator' but no URL or repository identifier appears — that should be fixed. None of this is a load-bearing flaw: the method is plausible, the paper is transparent, and the errors are correctable in revision.\n\nWho it's for: anyone working on low-resource MT, RL with tool use, or Indigenous-language NLP. It deserves a serious referee — conditional accept with requests for repeated runs, significance testing, and a real code link. My own verdict: promising but not proven yet.","headline":"A genuinely new RL-plus-dictionary recipe for low-resource MT, with thoughtful ablations but effect sizes that sit on single runs; deserves review, not a desk reject.","tokens_in":15100,"tokens_out":3386,"would_cite":true,"duration_ms":25939,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small LLM that can look words up in a bilingual dictionary and learns when to do so via RL translates Spanish-Wayuunaiki better than supervised fine-tuning alone.","keywords":["low-resource machine translation","Wayuunaiki","dictionary-guided translation","GRPO","reinforcement learning","tool-augmented LLM","Spanish-Wayuunaiki","BLEU"],"falsifier":"Re-running the Qwen-0.5B SFT and SFT+RL configurations several times with different random seeds and checking whether the dictionary-enabled RL model beats SFT alone in every run would settle whether the 3.42 versus 3.08 BLEU difference is systematic.","tokens_in":14071,"feed_emoji":"📖","tokens_out":8319,"duration_ms":65161,"temperature":0.7,"pith_summary":"Wayuunaiki, an Arawakan language spoken by about 420,000 people in Colombia and Venezuela, has very little parallel text for machine translation, so the paper asks whether an external bilingual dictionary can make up for missing data. The authors' answer is to treat translation as a tool-augmented decision problem: an instruction-tuned language model is first fine-tuned on parallel sentences and synthetic dictionary-call examples, then trained with GRPO reinforcement learning using sentence-level BLEU as the reward. On the AmericasNLP 2025 Spanish-Wayuunaiki test set, their best 0.5B-parameter model reaches 3.42 BLEU, an 11% gain over supervised fine-tuning alone and an 18% relative gain over a supervised baseline without dictionary access, while a 7B variant reaches 4.45 BLEU. The paper presents this as the first application of RL-based tool use to low-resource machine translation, and argues the recipe generalizes to any language pair with a dictionary.","feed_headline":"Dictionary plus RL lifts low-resource translation by 18 percent","feed_subtitle":"Small models learn when to query a bilingual dictionary, beating supervised baselines on the AmericasNLP 2025 test set.","key_machinery":"The load-bearing mechanism is the dictionary tool-augmented generation loop. During decoding the model can emit structured calls of the form <spa to wayuu>word</spa to wayuu>, the environment returns the dictionary's first five matches inside <matches> tags, and the final translation must appear inside <answer> tags. SFT first teaches the model this format using synthetic demonstrations where 0–4 Spanish words are randomly queried, and then GRPO samples 8 candidate translations per prompt, masks tool outputs from the policy loss, and rewards final answers with sentence-level BLEU. This two-stage design lets RL spend its budget on refining when and how to query rather than learning the tool from scratch.","core_discovery":"The central discovery is that the dictionary tool and the RL stage combine to produce a model that uses the lexicon aggressively and effectively. The Qwen2.5-0.5B-Instruct model trained with SFT plus 1400 GRPO steps calls the dictionary in every answer, averaging 3.94 of 4 allowed lookups, with a 95% success rate, and improves BLEU from 3.08 (SFT only) to 3.42. The same pipeline lifts LLaMA-3.2-1B from 3.15 to 3.48 and Qwen2.5-7B to 4.45 BLEU, while the non-instruction-tuned NLLB model, which cannot use the tool, stays at 0.93 BLEU through RL. The paper also reports that the dictionary contributes roughly 6% relative BLEU at both stages, that BLEU as reward works while a character-level edit reward degrades performance by 10.4%, and that the trained model's output is statistically significantly better than picking the dictionary's best suggestion, showing the model combines lexical matches with learned grammar rather than copying.","pith_inferences":["A natural next experiment this paper does not run is to vary dictionary coverage deliberately, such as subsampling the 29,000 entries and measuring BLEU gain per lookup, which would separate the value of the tool from the value of RL in a way the current ablations do not.","The dictionary's low match rate (only 16.3% of entries that cover a test word supply the reference translation) suggests the method may work even better with a morphological analyzer for Wayuunaiki's agglutinative forms; that is my extrapolation, not a claim in the paper.","Because the test set is a single book excerpt with one reference, the 18% relative gain could shrink or grow on more varied contemporary text; a community-sourced evaluation with multiple references and native-speaker judgment would be the test of whether the BLEU gains are usable translations."],"forward_implications":["The recipe transfers across instruction-tuned architectures: applying the full pipeline to LLaMA-3.2-1B and Qwen2.5-7B improves their BLEU scores, whereas RL alone does not help the non-tool NLLB model.","Expected gains on other language pairs are method-dependent: for agglutinative languages like Wayuunaiki the dictionary helps modestly, and the authors argue non-agglutinative languages could benefit more because words translate more independently.","Training duration and reward signal matter: the 0.5B model improves from 3.16 after 400 RL steps to 3.42 after 1400 steps, while the CharacTer reward degrades quality even at 1400 steps.","A dictionary usable during training does not require high coverage: only 10.4% of unique Spanish test words appear as entries, and the model still gains, suggesting RL can squeeze value from sparse lexical resources.","The best configurations use the dictionary on nearly every example, which in deployment would mean predictable tool-call load: up to four lookups per sentence, with most returning valid matches."],"supporting_citations":[{"why":"Supplies the GRPO algorithm and the reinforcement-learning setup used for the RL stage.","marker":"[4]"},{"why":"Is the external bilingual Spanish-Wayuunaiki dictionary that models query during generation.","marker":"[1]"},{"why":"Provides the Spanish-Wayuunaiki parallel corpus used for training and for the NLLB checkpoint.","marker":"[22]"},{"why":"Defines the AmericasNLP 2025 test set and the previous-work baseline the paper compares against.","marker":"[3]"},{"why":"Supplies the Qwen2.5 instruction-tuned base model family used in the main experiments.","marker":"[23]"},{"why":"Defines BLEU, the metric used both as the RL reward signal and as the evaluation measure.","marker":"[21]"},{"why":"Is the tool-use RL work whose tool-output masking the paper adopts for its dictionary calls.","marker":"[13]"},{"why":"Reports the prior Wayuunaiki translation system, the source of the 10.54 and 0.93 BLEU comparisons.","marker":"[25]"}],"fun_headline_variants":["Dictionary tool + RL boost low-resource translation by 18%","RL teaches models to use dictionary for rare-language translation","Small LLMs gain 18% BLEU with dictionary-guided RL","Tool-augmented LLMs beat baselines in low-resource translation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's comparisons rest on a single successful training run per configuration, so the central assumption is that the reported BLEU gaps—especially the 11% RL improvement—are bigger than run-to-run variation.","fun_headline_variants_meta":{"raw":{"variants":["Dictionary tool + RL boost low-resource translation by 18%","RL teaches models to use dictionary for rare-language translation","Small LLMs gain 18% BLEU with dictionary-guided RL","Tool-augmented LLMs beat baselines in low-resource translation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000638,"raw_usage":{"total_tokens":2996,"prompt_tokens":1058,"completion_tokens":1938,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":674,"completion_tokens_details":{"reasoning_tokens":1864}},"tokens_in":674,"tokens_out":1938,"duration_ms":12443,"temperature":1.0,"reasoning_tokens":1864,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:52:27.721220+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-running the Qwen-0.5B SFT and SFT+RL configurations several times with different random seeds and checking whether the dictionary-enabled RL model beats SFT alone in every run would settle whether the 3.42 versus 3.08 BLEU difference is systematic.","supporting_citations":[{"cited_title":"Osf spanish-wayuunaki,","cited_arxiv_id":null,"evidence_quote":"Is the external bilingual Spanish-Wayuunaiki dictionary that models query during generation."},{"cited_title":"Translation systems for low-resource colombian indige- nous languages, a first step towards cultural preserva- tion","cited_arxiv_id":null,"evidence_quote":"Provides the Spanish-Wayuunaiki parallel corpus used for training and for the NLLB checkpoint."},{"cited_title":"Ortega, Luis Chiruzzo, Arturo On- cevay, Shruti Rijhwani, Katharina Von Der Wense, and Manuel Mager","cited_arxiv_id":null,"evidence_quote":"Defines the AmericasNLP 2025 test set and the previous-work baseline the paper compares against."},{"cited_title":"Mart´ ınez, Juan C","cited_arxiv_id":null,"evidence_quote":"Reports the prior Wayuunaiki translation system, the source of the 10.54 and 0.93 BLEU comparisons."}],"review_version":2}