{"id":"5be06e7a-febf-4207-8db9-5b98798d34c8","arxiv_id":"2509.06650","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage RAG training pipeline, MoL continual pre-training plus GRPO reinforcement learning with single-passage training and multi-passage inference, reports state-of-the-art retrieval recall on NFCORPUS and SCIFACT using small Qwen3 models.","lead":"MoLER is a retrieval method that first continues training a language model on domain text, then uses reinforcement learning to make the model generate better search queries and answer-like passages, so smaller models retrieve documents more reliably. The paper reports that a 1.7B model trained this way beats a 32B model used with an existing query augmentation pipeline on two retrieval benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The log-linear MMLF scalability preservation after MSLF training is asserted from unfitted small-N curves; if the slope is within noise, the paper's efficiency/scalability contribution is unsupported.","rationale":"Reader's weakest_assumption is the same one I would flag: the efficiency/scalability contribution rests on the transfer of MMLF scaling to MSLF-trained models. Table 3 gives consistent positive deltas, so I do not see an internal contradiction that falsifies the recall gains; the problem is that the paper's most distinctive claim goes beyond 'recall improves.' The missing significance tests are real and should be fixed, but the scalability claim is more load-bearing because it is what separates MoLER from simply doing RL with MMLF. It is also readily testable from the released code. The appropriate verdict remains CONDITIONAL: not reject, because the concern is addressable; not accept, because the current evidence is visual and underpowered. I therefore mark the verdict unchanged.","tokens_in":24309,"tokens_out":10489,"duration_ms":112737,"concrete_test":"Using the released repository, run MMLF evaluation with Qwen3-0.6B in four conditions: base, +MoL, +MoL+Dr.GRPO(MSLF-trained), and +MoL+Dr.GRPO(MMLF-trained), on NFCORPUS and SCIFACT, for expansion counts n=3,5,7,9,11 and at least 5 decoding seeds per condition. Fit Recall(n)=a + b·ln(n) by least squares; report b with 95% confidence intervals. The preservation claim holds only if the MSLF-trained MMLF slope is significantly positive and its CI overlaps the base MMLF slope CI. If the CI includes 0 or the slope is below the base slope, the scalability-preservation claim fails and the verdict should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's distinctive contribution is the claim (Section 1; Section 4.2.3; Figures 4-6) that MMLF's scalability — 'a log-linear relationship between retrieval performance and the number of generated queries' — is preserved at inference even when RL training uses the cheaper MSLF strategy. This is the load-bearing assumption: without it, MoLER is only 'RL fine-tuning with MMLF,' and the MSLF design loses its stated efficiency justification. The evidence is a visual assertion: curves at a handful of expansion counts (n=3,5,7,9,11) with no fitted equation, no R², no confidence intervals, no repeated-seed variance. Figure 5 compares MMLF with MSLF after MoLER training, but 'preserved' requires a before/after comparison (base, +MoL, +MoL+Dr.GRPO) for MMLF; Figure 4 provides base vs MoLER, again without quantifying slopes. If the post-training slope is flat, or the base and trained slopes are statistically indistinguishable from noise, the title's 'Scalable Retrieval' and the central efficiency claim are unsupported, even though Table 3 might still show improved recall at a fixed expansion count.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MoLER, a two-stage method for improving coarse-ranking recall in RAG. In stage one, continuous pre-training (CPT) applies a Mixture of Losses (MoL) dual-loss objective: cross-entropy on a domain corpus and KL divergence on a general corpus, with a 1:1 ratio imported from reference [2]. In stage two, GRPO or Dr.GRPO reinforcement learning optimizes query and pseudo-passage generation, using the Multi-query Single-passage Late Fusion (MSLF) strategy during training to reduce LLM interactions, and Multi-query Multi-passage Late Fusion (MMLF) at inference for effectiveness. Experiments on NFCORPUS and SCIFACT report that Qwen3-1.7B+MoL+GRPO outperforms all baselines, including Qwen3-32B+MMLF, and that MMLF retains a log-linear relationship between retrieval performance and number of generated queries even after MSLF-based training. The paper interprets these results as demonstrating efficient training and scalable retrieval for domain-adaptive RAG.","tokens_in":24681,"tokens_out":5554,"duration_ms":61654,"significance":"If the empirical claims hold, the paper makes a practically valuable contribution: a 1.7B parameter model with domain CPT and RL-tuned query generation surpassing a 32B model on standard BEIR subsets, together with a training/inference asymmetry (MSLF/MMLF) that promises reduced RL cost without losing query-expansion scalability. The manuscript's strengths include an explicit two-stage design, a clear ablation structure (Tables 4-7), and a public code repository. However, the two headline claims are not yet established: the 'significantly outperforming' statement rests on single-run point estimates with no error bars or significance tests, and the central scalability-preservation claim is asserted from visual inspection of small-N curves without fitted models, confidence intervals, or repeated-seed variance. The contribution is plausible and interesting, but the current evidence is insufficient for the strength of the claims made.","major_comments":[{"comment":"The load-bearing claim that MMLF's log-linear scalability is 'preserved' after MSLF training is not quantitatively supported. The text in §4.2.3 infers 'approximately linear growth' and 'near-linear relationships' from five expansion counts (n=3,5,7,9,11) by eye, with no fitted log-linear or power-law model, no R², no confidence intervals, and no repeated-seed variance. Furthermore, 'preserved' requires a before/after comparison for MMLF (base, +MoL, +MoL+RL). Figure 5 compares MMLF vs MSLF only after training, and Figure 4 compares base vs MoLER; neither quantifies the slope or its uncertainty. Without this, the title's 'Scalable Retrieval' and the efficiency justification of MSLF are unsupported. Please fit the scaling law, provide CIs, and report variance.","section":"§4.2.3, Figures 4–6"},{"comment":"The words 'significantly outperforming' and 'statistically significant improvements' are used without any supporting statistical evidence. Table 3 shows single point estimates with no error bars, no repeated runs, and no significance tests. The margins over the closest baseline are tiny: e.g., NFCORPUS Recall@1k 61.42 vs 60.87 (+0.55 pp), and SCIFACT nDCG@10 62.59 vs 62.55 (+0.04 pp). These are within plausible decoding noise for LLM-based query expansion. Please add per-instance bootstrap or paired tests, or soften the claims.","section":"Abstract, §1, §4.2.1, Table 3"},{"comment":"The claim that MSLF 'reduces computational overhead' is only supported structurally (reducing LLM interactions from n+1 to 2 per rollout). No wall-clock time, FLOPs, GPU-hours, or comparative training convergence data are reported. Since 'Efficient Training' is part of the paper's title and contribution, the efficiency benefit needs empirical demonstration. Without it, the reader cannot assess whether the reduced interaction count translates into practical savings after the full RL pipeline.","section":"§4.1.4, §3.3"},{"comment":"The 'optimal 1:1 corpus ratio' for MoL is imported from the self-cited reference [2] and treated as a fixed hyperparameter. Table 7 abandons only the ratio (comparing CE vs MoL at fixed 4 epochs), not the ratio itself, so the optimality of 1:1 for these Qwen3 models and NFCORPUS/SCIFACT is not established. This is a correctness-risk concern: if the 1:1 ratio is not optimal for the new setting, the MoL advantage could be under- or over-stated. Please add a sensitivity analysis or explicitly frame the ratio as a transferred hyperparameter.","section":"§3.2, Table 7"}],"minor_comments":[{"comment":"Several typos reduce readability: 'enhencement', 'agunent', 'ambigous', 'nothinking' (in Table 4 caption, inconsistent with 'nonthinking' elsewhere), and reference [30] title 'Quary Expansion'. Please proofread.","section":"Throughout"},{"comment":"The caption refers to 'logarithmic scaling subplots (shown as insets)', but in the provided PDF the inset text and axis labels are illegible. Ensure insets are large enough and axes are clearly labeled in the revised version.","section":"Figure 4"},{"comment":"References [10] and [11] appear to be the same MILL paper with duplicated entries. Please consolidate.","section":"References"},{"comment":"'Unless specified' should be capitalized and punctuated as a sentence ('Unless otherwise specified, MMLF is used...').","section":"§4.1.5"},{"comment":"'an average improvement of 0.49% over its closest competitor' is ambiguous: please specify which metrics and across which datasets this average is computed.","section":"§4.2.1"}],"recommendation":"major_revision","confidential_remarks":"The direction of the work is interesting and the experimental design (MSLF for training, MMLF for inference) is elegant, but the current evidence falls short of the paper's strong claims. The scalability-preservation claim is the central efficiency rationale and needs quantitative support; the significance claims need error bars or tests; and the efficiency claim needs actual training-time measurements. These are addressable within the manuscript's scope, so I recommend major revision rather than rejection. I would also encourage the editor to ask for a direct comparison of training wall-clock time, not just interaction counts."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this paper's real contribution is the MSLF training / MMLF inference split. It consolidates n generated queries into one synthetic passage during RL rollouts, cutting interaction counts from n+1 to 2, then restores per-query passage generation at inference. That is a genuine, if modest, efficiency trick, and it is new relative to the MMLF, MoL, and DeepRetrieval work they cite. The rest of the pipeline is assembled from their own MoL continual pre-training [2] and MMLF [14], plus standard GRPO, all properly cited.\n\nThe headline empirical claim is that Qwen3-1.7B+MoL+GRPO beats Qwen3-32B+MMLF by about 0.5 recall points on NFCORPUS and 0.4 nDCG on SCIFACT. That direction is consistent across Table 3, and the evaluation is on external BEIR test sets, so the result is not circular. But the margins are under one recall point, there are no error bars, no significance tests, and no repeated-seed variance. The prose says \"significantly outperforming\" and \"statistically significant improvements\"; the data do not support that language. This is the biggest issue.\n\nThe second soft spot is the scaling-law claim. The paper's distinctive efficiency story rests on MMLF's log-linear scalability being preserved after MSLF training. The evidence is eyeballed curves at n=3,5,7,9,11 with no fitted power law, no R², no confidence intervals. The stress-test note is right to target this. A flat post-training slope would gut the \"scalable retrieval\" part of the title. However, I would not call it a load-bearing flaw in the sense of making the main table wrong. The fixed-n results stand on their own; it is the generalization to variable n that is unsupported.\n\nAlso worth noting: DeepRetrieval [12] is discussed in related work but absent from Table 3. Since it is the closest RL-for-retrieval baseline, omitting it weakens the claim of SOTA.\n\nOn the positive side, the ablations are reasonably informative. Table 5 shows MMLF beats MSLF at inference and that RL with MSLF helps MMLF. Table 7 shows MoL beats CE under matched epochs. The code link is provided. Those are real evidence.\n\nWho is this for? Practitioners building domain-specific RAG on small models, and researchers working on RL-tuned query generation. It is not a field re-organizer. It deserves a serious referee; with a few added baselines, error bars, and a properly fitted scaling analysis, it could become a solid systems paper. I would send it to review, but I would not accept it as-is.","headline":"A modest but real training-efficiency trick (MSLF) wrapped in a pipeline of existing components, with headline gains that are plausible but sloppily evidenced—worth refereeing, not desk-rejecting.","tokens_in":25124,"tokens_out":2286,"would_cite":true,"duration_ms":23606,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims MoLER—domain continual pre-training plus RL-tuned query and passage generation—lets a 1.7B model beat a 32B model on coarse-ranking retrieval benchmarks while keeping training cheap via MSLF and inference scalable via MMLF.","keywords":["retrieval-augmented generation","coarse ranking","query expansion","continual pre-training","mixture of losses","GRPO","late fusion","domain adaptation"],"falsifier":"Fix the embedding model, train the same base model with and without MSLF, then evaluate MMLF inference at expansion counts 1, 2, 4, 8, 16, and 32 on NFCORPUS and SCIFACT; fit the recall-versus-log-count slope with confidence intervals. The scalability-transfer claim stands only if the post-MSLF slope is positive and statistically indistinguishable from the base model's; if the curve flattens past the training count of 3, the central efficiency claim fails.","tokens_in":24265,"feed_emoji":"🔎","tokens_out":6353,"duration_ms":73208,"temperature":0.7,"pith_summary":"MoLER is a two-stage recipe for making a small language model retrieve well in specialized domains. First, continual pre-training uses a mixture of losses—cross-entropy on domain text, KL divergence on general text—so the model gains domain knowledge without losing general ability. Second, reinforcement learning with GRPO tunes the model to generate multiple query variants and a synthetic answer passage, with document recall as the reward. The paper reports that on the NFCORPUS and SCIFACT benchmarks, a 1.7B-parameter model with MoLER outperforms a 32B model using prompt-only multi-query expansion. The central efficiency claim is that training with a single fused passage preserves the inference-time scalability of generating one passage per query and fusing the rankings.","feed_headline":"A 1.7B model beats a 32B model at RAG retrieval","feed_subtitle":"Domain-aware pretraining plus RL query generation lifts recall on biomedical and scientific retrieval benchmarks.","key_machinery":"Three building blocks carry the argument: MoL (Mixture of Losses), a continual pre-training objective that applies cross-entropy loss to domain corpora and KL-divergence loss to general corpora; GRPO (Group Relative Policy Optimization), an RL algorithm whose reward is the recall of the fused retrieval result; and the MSLF/MMLF pair, where MSLF generates one passage from many queries during training and MMLF generates one passage per query during inference, with reciprocal rank fusion combining the ranked lists. MoL supplies domain knowledge, GRPO tunes generation toward recall, MSLF keeps RL cheap, and MMLF keeps inference scalable.","core_discovery":"On its own terms, the paper's discovery is that retrieval performance in a specialized domain can be optimized end-to-end by treating query generation and passage generation as a reinforcement-learned policy rewarded by recall, provided the model first absorbs domain knowledge through a dual-loss continual pre-training. The key named mechanism is the pair MSLF/MMLF: during RL training, the policy generates several sub-queries but collapses them into one synthetic passage before retrieval, cutting policy rollouts from n+1 model interactions to 2; at inference, the same policy generates a separate passage per sub-query and merges the ranked lists with reciprocal rank fusion. The paper reports","pith_inferences":["A direct consequence the paper leaves implicit: the log-linear recall-versus-query-count curve is a budget rule—deployers could pick the number of generated queries from latency or cost targets rather than the fixed n=3 used in training.","The same train-cheap/infer-expensive split could transfer to other retrieval-oriented generation tasks, such as reranking or entity linking, by using a single fused rollout as the RL reward surrogate and deploying fully diverse rollouts only at inference.","Because the paper fixes one embedding model, a natural test is whether the gains transfer across embedding spaces; if they shrink or vanish, much of the benefit may lie in the retriever rather than the trained generation policy.","Applying the same two-stage pipeline to the 32B baseline would show whether MoL+GRPO gains stack with scale or saturate; the current comparison leaves that question open."],"forward_implications":["RL-tuned query and passage generation can lift coarse-ranking recall on specialized corpora beyond prompt-only augmentation with a much larger model.","Training with MSLF keeps RL rollouts cheap—two model interactions instead of n+1—while preserving the inference-time benefit of MMLF.","Recall improves approximately logarithmically with the number of generated queries, so more expansions at inference buy predictable but diminishing returns.","MoL continual pre-training reaches higher and more stable RL rewards than CE-only domain training in the reported convergence curves.","Nonthinking mode gives nearly the same retrieval performance as thinking mode, so deployment can skip chain-of-thought latency."],"supporting_citations":[{"why":"Supplies the MoL dual-loss continual pre-training method the paper adopts.","marker":"[2]"},{"why":"Defines MMLF, the multi-query multi-passage late-fusion baseline and inference strategy MoLER builds on.","marker":"[14]"},{"why":"Introduces GRPO, the reinforcement learning algorithm used in the post-training stage.","marker":"[6]"},{"why":"Motivates RL-trained query generation and rewriting for retrieval, the approach MoLER extends.","marker":"[12]"},{"why":"Provides Query2doc, a pseudo-passage query expansion baseline compared against MoLER.","marker":"[25]"},{"why":"Supplies reciprocal rank fusion, the ranking combination rule used in both MSLF and MMLF.","marker":"[4]"},{"why":"Provides the BEIR benchmark datasets NFCORPUS and SCIFACT used for evaluation.","marker":"[24]"},{"why":"Defines the Qwen3 base models and decoding hyperparameters used in all experiments.","marker":"[31]"},{"why":"Specifies the embedding model used for similarity scoring in retrieval and reward computation.","marker":"[19]"},{"why":"Introduces Dr.GRPO, the bias-corrected GRPO variant used in the main comparison and ablations.","marker":"[16]"}],"fun_headline_variants":["RL-tuned query generation boosts domain RAG recall","MoLER: Efficient RL training for scalable domain retrieval","Multi-query RL with single-passage training, full fusion at inference","Domain-aware pretraining plus RL yields SOTA retrieval in specialized domains","Late fusion cuts RL compute, keeps recall high in RAG"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The efficiency claim rests on the assumption that training with one fused passage preserves the retrieval gains from generating many separate passages at inference time—a trend read from a few data points, not established by a fitted relationship or significance test.","fun_headline_variants_meta":{"raw":{"variants":["RL-tuned query generation boosts domain RAG recall","MoLER: Efficient RL training for scalable domain retrieval","Multi-query RL with single-passage training, full fusion at inference","Domain-aware pretraining plus RL yields SOTA retrieval in specialized domains","Late fusion cuts RL compute, keeps recall high in RAG"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1238,"prompt_tokens":724,"completion_tokens":514,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":429}},"tokens_in":468,"tokens_out":514,"duration_ms":5884,"temperature":1.0,"reasoning_tokens":429,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T23:18:12.551373+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fix the embedding model, train the same base model with and without MSLF, then evaluate MMLF inference at expansion counts 1, 2, 4, 8, 16, and 32 on NFCORPUS and SCIFACT; fit the recall-versus-log-count slope with confidence intervals. The scalability-transfer claim stands only if the post-MSLF slope is positive and statistically indistinguishable from the base model's; if the curve flattens past the training count of 3, the central efficiency claim fails.","supporting_citations":[{"cited_title":"2022.New and improved embedding model","cited_arxiv_id":null,"evidence_quote":"Specifies the embedding model used for similarity scoring in retrieval and reward computation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Dr.GRPO, the bias-corrected GRPO variant used in the main comparison and ablations."}],"review_version":1}