{"id":"2df1431f-61b4-4eb1-82a1-9b155f971f24","arxiv_id":"2607.02966","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Teacher-regularized on-policy RL with a reverse-KL anchor on student prefixes improves language consistency and evidence-grounded correctness for English-evidence cross-lingual RAG, acting as a safety net against large language-drift collapses.","lead":"TR-RAG trains small multilingual RAG generators by combining task rewards with reverse-KL guidance from a frozen teacher only on the prefixes the student itself visits. The recipe raises language adherence and evidence use under English-only retrieval and stops reward-only RL from collapsing into English or code-switched answers.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified beyond the reader's already-flagged reward-proxy caveat; the safety-net claim is supported by the reported tables.","rationale":"The paper's strongest claim is empirical and multi-faceted: TR-RAG wins the composite and the reverse-KL anchor prevents large in-domain LC collapses while still helping OOD grounding. Tables 1, 9–11 and the MKQA stress tests (Norwegian/Khmer) supply direct numerical support; ablations (Table 2) show the combination beats RL-only and OPD-only. The reader's identified weakest assumption (reward decomposition + fixed β) is correctly the softest link, yet the paper already mitigates it with cross-family judges, judge-independent metrics, and the dramatic non-judge LC collapse of Naive-RL. Single-seed reporting remains the cleanest remaining uncertainty; the proposed multi-seed check would settle it without requiring new theory. Because that uncertainty is already priced into the CONDITIONAL verdict, no adjustment is warranted.","tokens_in":46394,"tokens_out":522,"duration_ms":5783,"concrete_test":"Re-run the Hotpot-ENKB5 Qwen-4B Naive-RL vs TR-RAG comparison for three independent seeds and report mean±std of Indonesian LC and ALL-AVG composite; if the ~27 pp LC gap shrinks below ~10 pp or loses statistical significance, the safety-net claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (best composite + teacher reverse-KL as safety net against ~27 pp LC collapses) is internally consistent with the multi-benchmark, multi-backbone tables and ablations. The reader's weakest assumption (three-component reward + fixed β=0.02 as a faithful proxy) is the softest point, but it is already partially stress-tested: cross-family Llama judge preserves rankings and widens the TR-RAG margin (Table 7 / §D.2.4), judge-independent LC and char-3-gram still favor TR-RAG, and the Hotpot Qwen-4B Indonesian LC collapse of Naive-RL (28.27 % vs 55.60 % TR-RAG, Table 11) is a concrete, non-judge failure mode. Residual single-seed reporting and translationese are acknowledged limitations, not hidden contradictions. No stronger load-bearing flaw overturns the claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper studies English-evidence cross-lingual RAG, where queries and answers are non-English but retrieved passages remain English. It attributes generation failures (language drift and brittle evidence use) to prefix mismatch under fixed-trajectory supervision and high-variance credit assignment under sequence-level rewards. TR-RAG couples GRPO-style reward optimization with a length-normalized reverse-KL anchor from a frozen teacher queried only on student-visited prefixes (Eqs. 9–11), using a three-component reward of language consistency, character 3-gram recall, and an LLM-judge score (Eq. 3). Across BioASQ-ENKB5, Hotpot-ENKB5, and native MKQA, with Llama-3B and Qwen-4B students, TR-RAG reports the best composite of language adherence and evidence-grounded correctness, and the teacher anchor is claimed to act as a safety net against large in-domain LC collapses (up to ~27 pp) that reward-only RL can suffer while still improving grounding on distant OOD languages.","tokens_in":46736,"tokens_out":1130,"duration_ms":10897,"significance":"If the results hold, the paper offers a practical, generation-centric post-training recipe for a recurring industrial setting (English-only knowledge bases with multilingual users). Strengths include a clear problem formulation, a clean on-policy distillation design that avoids teacher rollouts, multi-benchmark and multi-backbone evaluation, full ablations of RL-only / OPD-only / reward components (Table 2), β sensitivity, weak-teacher controls, cross-family judges, and qualitative cases. The safety-net claim is concrete and falsifiable (e.g., Hotpot Qwen-4B Indonesian LC 28.27% for Naive-RL vs. 55.60% for TR-RAG in Table 11). The work is a solid systems contribution rather than a foundational theoretical advance, but it is well-scoped and useful for the community.","major_comments":[{"comment":"The central evaluation relies on a composite that includes an LLM-judge which is also a training reward component (§3.2, Eq. 3; Metrics in §4.1). Although the paper reports cross-family Llama judges (Table 7 / §D.2.4) and judge-independent LC and char-3-gram signals, the main tables still lead with the same-family Qwen judge for ENKB-RAG-5. For the composite claim to be fully load-bearing, the main results should foreground the judge-independent metrics and the cross-family judge as primary, not only as appendix checks, so that the ranking is not partly driven by the training reward itself.","section":null},{"comment":"The ~27 pp safety-net claim is supported most strongly by a single setting (Hotpot-ENKB5, Qwen-4B, Indonesian LC in Table 11: Naive-RL 28.27% vs. TR-RAG 55.60%). Elsewhere LC is often near-saturated or subject to acknowledged LID artifacts (§D.5). The paper should either (i) quantify how often and under what conditions reward-only RL collapses across all backbone–dataset pairs, or (ii) reframe the safety-net claim as conditional on settings where unconstrained RL is unstable, rather than as a general property of the method.","section":null},{"comment":"ENKB-RAG-5 is constructed by machine-translating questions and short answers while leaving English evidence untouched (§4.1, Appendix D.1). The authors acknowledge translationese and residual term drift as limitations, and MKQA is a helpful native check, but the main training and most of the reported gains still rest on the translated construction. A clearer quantification of how much of the composite gain survives when both training and evaluation are restricted to native multilingual queries (or a stronger statement that the method is intended primarily for the controlled English-evidence construction) would strengthen the external-validity claim.","section":null}],"minor_comments":[{"comment":"Single-seed reporting is acknowledged (§D.4.5) but should be stated more prominently in the main experimental section; variance over a few seeds would help readers assess the magnitude of the composite gaps.","section":null},{"comment":"The Hotpot LC plateau and fastText LID artifacts (§D.5) make the LC column hard to interpret for several languages; consider scoring against the known query language ℓ(q) as the primary LC metric in the main tables.","section":null},{"comment":"Notation: M vs. K for number of passages vs. rollouts is explained in the figure caption but could be stated once in §2 for clarity.","section":null},{"comment":"Appendix B is thorough; a short pointer in the main text to which lemmas are used for the reverse-KL estimator and length-normalization would help readers who skip the appendix.","section":null},{"comment":"Figures 3–7 and 17–21 are useful; ensure axis labels and β units are consistent across the main text and appendix versions.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The paper is a competent systems contribution with careful ablations and a clear industrial motivation. Novelty relative to concurrent multilingual RAG-RL work (LcRL, CroSearch-R1) is adequately scoped to generation under English-only evidence. Fit for a solid CL/ML venue is good after the evaluation-presentation revisions above; I would not escalate to major revision unless the authors cannot re-center the judge-independent and cross-family evidence."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The punchline is simple: they take existing pieces (GRPO-style RL, GKD-style on-policy reverse KL) and show that the joint objective actually fixes a recurring production failure—language drift and brittle evidence use when the query is non-English but the passages are English. The safety-net claim is the part worth remembering: reward-only RL can drop language consistency by ~27 pp below the base model (Hotpot Qwen-4B Indonesian), while the frozen teacher on student prefixes prevents that and still helps grounding on distant OOD languages where pure RL stalls at the base ceiling.\n\nWhat is new is not either ingredient alone. It is the combination plus a three-part reward (fastText LID, char 3-gram recall, LLM judge) built for this regime, and the multi-benchmark evidence that the anchor is insurance rather than pure capability transfer. They do the work: two student–teacher families, BioASQ/Hotpot ENKB constructions plus native MKQA, full ablations (RL-only, OPD-only, reward drops), β sweeps, weak-teacher controls, cross-family judges, and qualitative cases. Char 3-gram is judge-independent and sometimes beats the 70B teacher; the composite ranking holds under a Llama judge. Appendix theory is standard KL-regularized RL, not oversold. Citations to GKD, GRPO, LcRL, CroSearch-R1 are honest about what is concurrent and what is different (generation-side, English-only evidence).\n\nSoft spots are real but proportionate. Single-seed reporting is the main empirical gap. Translationese on ENKB is acknowledged and partly checked with MKQA. The reward + fixed β=0.02 is a proxy; they stress-test it with cross-family judges and judge-independent metrics, and the Indonesian LC collapse is not a judge artifact. Hotpot LC is partly saturated by LID noise—they flag it. None of that overturns the central tables.\n\nThis is for people who ship multilingual RAG or who care about stabilizing noisy sequence rewards with on-policy distillation. It deserves a serious referee. I would engage with it and expect revision on seeds and a cleaner LC metric, not a rewrite of the claim.","headline":"Clean joint recipe for a real industrial failure mode: on-policy reverse-KL as a safety net against reward-only language collapse under English evidence.","tokens_in":47313,"tokens_out":552,"would_cite":true,"duration_ms":7881,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"TR-RAG couples on-policy rewards with a frozen-teacher reverse-KL on student prefixes, improving language adherence and evidence use when answers must be non-English but evidence is English.","keywords":["cross-lingual RAG","English-evidence generation","on-policy distillation","teacher-regularized RL","language drift","prefix mismatch","character n-gram reward"],"falsifier":"On a held-out native multilingual English-evidence RAG set scored by humans, if reward-only RL matched or beat TR-RAG on both language consistency and human-rated faithfulness without large language collapses, the safety-net claim for the teacher anchor would be falsified.","tokens_in":47314,"feed_emoji":"🌐","tokens_out":1057,"duration_ms":16752,"temperature":0.7,"pith_summary":"When users query in many languages but the knowledge base stays English, generators often drift into English or code-switching and use the passages unreliably. The paper traces this to two training problems: errors depend on the prefixes the model actually visits, so fixed-trajectory supervision never corrects them, and sequence-level rewards give noisy credit. TR-RAG answers both by letting a compact student sample answers on-policy while a stronger frozen teacher is queried only on those same prefixes, supplying a reverse-KL anchor that is optimized jointly with a three-part reward for language consistency, character 3-gram recall, and judge-scored groundedness. Across biomedical, multi-hop, and native multilingual benchmarks the method raises the joint score of language adherence and evidence-grounded correctness; the teacher anchor prevents the large language collapses pure reward RL can suffer and still lifts grounding on distant languages where reward-only training stalls. Anyone building cross-border assistants or global support systems cares because small models can stay fluent and faithful without non-English knowledge bases.","feed_headline":"Teacher anchor stops language collapse in cross-lingual RAG","feed_subtitle":"A reverse-KL safety net on student prefixes keeps small models fluent and evidence-faithful under English-only passages.","key_machinery":"The prefix-wise reverse-KL teacher anchor: the student samples on-policy answers; a stronger frozen teacher is queried only on those student-visited prefixes (no teacher rollouts) and supplies the length-normalized student-to-teacher reverse KL, which is combined with a three-component sequence reward (language consistency, character 3-gram recall, LLM-judge groundedness) inside a single GRPO-style objective.","core_discovery":"TR-RAG attains the best composite of language adherence and evidence-grounded correctness across BioASQ-ENKB5, Hotpot-ENKB5 and MKQA with two compact backbones. The frozen-teacher reverse-KL anchor on student-visited prefixes acts as a safety net: on in-domain languages it prevents large language-consistency collapses (up to about 27 percentage points) that reward-only RL can suffer by drifting below the base model, while on distant out-of-distribution languages, where reward-only RL stalls at the base ceiling, it still improves evidence grounding; on character 3-gram recall the compact student sometimes surpasses its 70B teacher.","pith_inferences":["The same student-visited-prefix reverse-KL recipe may stabilize other early-error cascade settings such as code-switching, mixed-language evidence, or multimodal RAG.","If language consistency stays weak on extreme scripts, richer script-aware rewards could raise the OOD ceiling without changing the anchor.","Systems that already hold a large same-family teacher can treat the anchor as a cheap safety layer rather than a full capability-transfer pipeline.","The insurance-premium framing suggests a general pattern: accept a small judge-score tax in safe regimes to buy protection against catastrophic language drift."],"forward_implications":["Compact generators can be post-trained for English-evidence cross-lingual RAG without non-English knowledge bases or teacher rollouts.","Pure reward-only RL is unsafe for language control: the teacher anchor is required to avoid large in-domain language-consistency collapses.","On distant low-resource languages the same anchor still lifts evidence grounding even when language consistency is capacity-bottlenecked at the base model.","Character 3-gram recall is a dense, tokenization-robust signal that can let a small student exceed a much larger teacher on lexical content transfer.","Combining on-policy distillation with task rewards strictly dominates either alone on the composite metric."],"fun_headline_variants":["TR-RAG teacher anchor blocks language collapse in English-evidence RAG","Reverse-KL on student prefixes stops fluency drift under English passages","Teacher-regularized RL keeps compact models fluent and evidence-faithful","On-policy reverse-KL net prevents 27-point language collapses in RAG","Student with frozen teacher lifts grounding without dropping language adherence"],"cache_read_input_tokens":30848,"weakest_assumption_plain":"The method assumes that a fixed mix of language-ID, character-overlap and a single LLM-judge score, plus a modest reverse-KL weight, is a faithful enough proxy for true multilingual generation quality that optimizing it yields better real answers.","fun_headline_variants_meta":{"raw":{"variants":["TR-RAG teacher anchor blocks language collapse in English-evidence RAG","Reverse-KL on student prefixes stops fluency drift under English passages","Teacher-regularized RL keeps compact models fluent and evidence-faithful","On-policy reverse-KL net prevents 27-point language collapses in RAG","Student with frozen teacher lifts grounding without dropping language adherence"]},"model":"grok-4.5","effort":"low","cost_usd":0.00467,"raw_usage":{"total_tokens":1427,"prompt_tokens":973,"num_sources_used":0,"completion_tokens":93,"cost_in_usd_ticks":46700000,"prompt_tokens_details":{"text_tokens":973,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":361,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":973,"tokens_out":93,"duration_ms":3282,"temperature":1.0,"reasoning_tokens":361,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T05:46:14.666712+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a held-out native multilingual English-evidence RAG set scored by humans, if reward-only RL matched or beat TR-RAG on both language consistency and human-rated faithfulness without large language collapses, the safety-net claim for the teacher anchor would be falsified.","supporting_citations":[],"review_version":1}