{"id":"c30ced0c-5d1b-4a3c-b52f-b4a4695dea7e","arxiv_id":"2505.20285","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A pre-training task called RAMP, where models practice searching to fill masked text spans, improves downstream agentic open-domain QA performance across Qwen and LLaMA models.","lead":"MaskSearch trains language models to fill in blanked-out words in Wikipedia sentences by searching the web, then fine-tunes them on question-answering tasks. The method improves small open-source models on multi-hop search-and-reasoning benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RAMP's gains may reflect extra data/compute rather than the mask-prediction task: no same-scale control pre-training is reported, so the central transfer claim is not yet established.","rationale":"I agree with the reader's weakest assumption. The paper's headline claim is causal: Section 5.1 states that MASKSEARCH, which incorporates RAMP as the pre-training stage, significantly enhances model performance, and the load-bearing result is that RAMP pre-training plus HotpotQA post-training beats HotpotQA-only training. But the comparison is asymmetric in data and compute. The RAMP stage consumes a 10M-example, 14B-token corpus synthesized through multi-agent generation and self-evolve distillation, while the baselines see only 58K HotpotQA trajectories. Without a same-scale control objective, the improvement cannot be attributed to the RAMP task design itself. This is the single most load-bearing concern because the paper's contribution and its claimed universality rest on the transferable value of the mask-prediction task, not on the generic value of more training data. The concern is not an accusation of error in the reported numbers; rather, it is a missing experiment. The rest of the paper is internally coherent: the appendix gives concrete prompts, reward templates, and training hyperparameters, and the self-reported result that 7B SFT gains are limited by self-evolved data diversity is consistent with the method description. I also note the absence of error bars or multiple seeds, which compounds the attribution problem because single-run differences of a few recall points could be noise. Given the reader already returned a CONDITIONAL verdict on exactly this gap, no change to the verdict is needed; the condition should be the controlled pre-training comparison described above.","tokens_in":22921,"tokens_out":3162,"duration_ms":34636,"concrete_test":"Run a controlled pre-training comparison on Qwen2.5-1.5B with a fixed compute budget equal to the RAMP SFT stage: same 10M paragraphs, same global batch size, same 2 epochs, same learning rate (Appendix A). Condition A: RAMP trajectories as in the paper. Condition B: the same 10M paragraphs with salient spans left unmasked, trained with standard next-token prediction (or, to control tool practice, the same search-augmented trajectories with masks removed so the model imitates the same think/search/answer format but not fill-in-the-blank). Condition C: Distilled Search-R1, 58K HotpotQA only. Then run identical HotpotQA SFT and evaluate token-level recall on the dev split with at least 3 seeds. If Condition B matches Condition A within noise, the mask-prediction task is not the causal ingredient and the framework's central claim fails; if A clearly beats B, the confound is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RAMP pre-training, not merely additional tokens or additional practice with a search tool, causes the 3-8 point recall gains on HotpotQA and larger out-of-domain gains. The evidence in Table 2 compares MASKSEARCH (10M-example RAMP corpus, 14B tokens, Section 3.3) followed by 58K HotpotQA SFT against Distilled Search-R1 and Search-R1, which train only on the 58K HotpotQA trajectories (Section 4). No baseline pre-trains the same 10M corpus, with the same trajectory-generation pipeline and compute budget, under an alternative objective. Therefore the design of RAMP - salient-span masking plus search-and-fill - is confounded with at least three ingredients: (a) roughly 14B extra training tokens, (b) extra exposure to the retriever/search tool, and (c) the self-evolve distillation data-generation process. The scaling plot (Figure 2) shows that more RAMP steps help, but it does not identify which ingredient helps; it is equally consistent with generic continued pretraining on domain text or generic tool practice. Section 5.2's own note that 7B SFT gains are limited by self-evolved data diversity reinforces that data itself, not just task structure, is doing work. This is a missing control, not an internal inconsistency; the reported numbers may be correct and still not support the label 'universal pre-training framework.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MaskSearch, a two-stage training framework for LLM-based search agents. In the first stage, models are trained on a Retrieval-Augmented Mask Prediction (RAMP) task: given a paragraph with salient spans masked, the model must use a search tool and multi-step reasoning to fill them. The RAMP corpus is constructed from Wikipedia with agent-based trajectory generation (planner/rewriter/observer) followed by iterative self-evolve distillation, yielding 10M CoT examples (14B tokens). In the second stage, models are trained on downstream HotpotQA via SFT or RL (DAPO with a hybrid reward), and evaluated on HotpotQA plus out-of-domain multi-hop QA benchmarks (FanoutQA, Musique, 2WikiMultiHopQA, Bamboogle, FreshQA). The central claim is that RAMP pre-training produces stable recall gains over single-stage training on HotpotQA alone, with larger out-of-domain gains, across Qwen and LLaMA model families.","tokens_in":23161,"tokens_out":5529,"duration_ms":56924,"significance":"If the causal attribution holds, MaskSearch would be a useful scalable objective for pre-training agentic retrieval-and-reasoning behavior, and the paper provides a plausible and well-motivated recipe. The strengths are the breadth of the empirical sweep (six model sizes across two families, six datasets), the explicit scaling analysis in Figure 2, the curriculum-learning analysis, and the release of code. The gains are consistent across model families, which makes the phenomenon credible. However, the paper's central transfer claim currently rests on a comparison that confounds task design with additional training data and compute, so the significance is conditional on an added control experiment.","major_comments":[{"comment":"The comparison in Table 2 does not isolate RAMP's task design. MaskSearch pre-trains on a 10M-example / 14B-token RAMP corpus (Section 3.3) before the 58K HotpotQA SFT, whereas Distilled Search-R1 and Search-R1 receive only the 58K HotpotQA trajectories described in Section 4. No baseline pre-trains the same corpus at the same scale with an alternative objective, so the reported 3-8 point gains could be explained by the extra training tokens, the extra exposure to the search tool, or the self-evolve data-generation pipeline rather than by masked-span prediction. Figure 2's scaling curve is equally consistent with generic continued pre-training. The admitted need for deeper analysis in Section H does not substitute for this control. Please add a same-scale control that continues pre-training on the same Wikipedia corpus with a standard language-modeling objective, and ideally a second control that trains on the same corpus with masks removed while keeping the search-tool interaction.","section":"Section 4 (Baselines); Section 3.3; Table 2"},{"comment":"All headline results are single-run token-level recall scores on development splits (Table 1 marks several datasets with a dagger to indicate that the dev split is used as the test set), with no error bars, multiple seeds, or significance tests. The claimed improvements are 3-8 points on the in-domain task and vary across datasets, and the authors themselves show in Section 6.2 that the token-level recall metric can be gamed by longer answers. To support the wording 'significantly enhances,' the paper should report variance over at least a few seeds or a paired significance test, especially for the smaller improvements.","section":"Section 5.1; Table 1; Table 2"},{"comment":"The search tool used for RAMP pre-training, downstream training, and evaluation is never concretely specified: no API or retriever name, result count, snippet length, or number of retrieval tokens is provided beyond the prompt template in Appendix E. Because the paper's central claim is that the model learns to use a search tool, this omission blocks replication and makes it difficult to determine how much of the observed gain is specific to the RAMP task versus to the particular retrieval environment. Please document the retrieval backend, the number and length of retrieved results, and any query-rewriting or retrieval-token limits used in both stages.","section":"Section 3.3; Appendix E; Experimental setup"}],"minor_comments":[{"comment":"In the sentence reporting the Bamboogle gain, 'Qwen2.15-1.5B' should be 'Qwen2.5-1.5B'.","section":"Section 5.1"},{"comment":"The sentence 'Model-based reward offer significant improvements of 34.71 and 19.48 over the other two reward methods' is ungrammatical and the numbers are not tied to any table or axis in Figure 5; please state what quantities these numbers refer to or remove them.","section":"Section 6.2"},{"comment":"The table header 'w/ RAMP w/o RAMP CL Mix' is ambiguous; it is unclear which columns correspond to curriculum learning and mixed training under each condition. Also, the text says CL outperforms Mix on Qwen2.5-7B and LLaMA-3.2-1B, but the table shows the opposite direction for Qwen2.5-1.5B and LLaMA-3.1-8B, so the sentence 'indicating that the approach can generalize across different model architectures' is too strong.","section":"Table 3"},{"comment":"The PPL-based masking experiment uses a 40K RAMP dataset with 6K HotpotQA trajectories, whereas the main results use the 10M RAMP corpus with 58K HotpotQA trajectories; the comparison in Figure 4(b) should state explicitly that these are smaller-scale settings and that the two settings are not directly comparable.","section":"Section 6.1"},{"comment":"The text says that results at training step 0 align precisely with the Direct SFT baseline, but no Direct SFT baseline is plotted or labeled in Figure 2; please add the baseline curve or state where it appears.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the missing same-scale control. If the authors add a continued-pretraining baseline on the same corpus and, ideally, a no-mask search-tool control, and also report variance or significance for the main comparison, the paper could become acceptable. I do not see citation or scope issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere’s my take: the paper is a solid empirical recipe — pre-train on masked Wikipedia spans with search-fill (RAMP), then post-train on HotpotQA — and it shows consistent recall gains across two model families and several multi-hop QA benchmarks. The engineering is careful, the appendix gives the prompts and hyperparameters, and the code is promised. That is real value, especially for practitioners working on agentic RAG.\n\nWhat’s genuinely new is the RAMP objective itself and the two-stage SFT/RL recipe around it, plus the curriculum over mask counts and a self-evolving teacher that scales the trajectory set to 10M examples. The reward-design discussion is honest: the model-based reward beats token-recall because the latter is hackable, and the scaling plots show that more RAMP steps help. I also credit the paper for acknowledging in Section 5.2 and the Limitations appendix that self-evolved data diversity caps 7B gains.\n\nThe soft spots are proportionate but real. The main one is causal attribution. The best configuration consumes 10M RAMP examples (14B tokens) before HotpotQA SFT, while the Search-R1 and Distilled Search-R1 baselines train only on the 58K HotpotQA trajectories. That means the reported 3–8 point gains could just as easily come from extra training tokens, additional exposure to the search tool, or the self-evolution data pipeline. Figure 2 shows scaling with RAMP steps, but it does not isolate RAMP’s task structure from generic continued pre-training or tool practice. This is a missing control, not an internal contradiction; the numbers may be right, but the title’s “universal pre-training framework” is not supported by the current evidence.\n\nSecondary issues: results are single-run token-level recall on dev splits with no error bars; some tables have formatting inconsistencies; and the comparison is limited to open-domain QA, so “universal” overstates the scope. The paper’s own limitation notes mention the single-tool setting and the lack of theory, but they do not flag the control problem, which is the first thing a referee would ask about.\n\nWho should read it: anyone building training pipelines for search agents, and anyone who wants a worked example of a data-generation scaffold (multi-agent startup plus self-evolve). I’d bring it to a reading group to debate evaluation methodology. A serious referee should engage — the work is substantial and reproducible in spirit — but the revision should include a same-scale alternative-objective pre-training baseline and ideally repeated runs. Without that, the transfer claim stays conditional.\n\nRecommendation: send to review, but expect heavy revision on the attribution question.","headline":"A genuine, well-engineered pre-training recipe for agentic search, but the central causal claim is undercut by a missing same-scale control.","tokens_in":23769,"tokens_out":3340,"would_cite":true,"duration_ms":33098,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pre-training a language model to fill masked spans by web search transfers to open-domain QA and beats single-stage training across model sizes.","keywords":["retrieval-augmented generation","agentic search","mask prediction pretraining","multi-hop question answering","reinforcement learning","curriculum learning","self-evolving distillation","supervised fine-tuning"],"falsifier":"Fine-tune the same base model on a same-size 10M corpus of control trajectories that include the same searches but require no mask-filling (for example, copying the retrieved passage), then fine-tune on HotpotQA; if downstream recall matches the RAMP-pretrained model, the specific mask-filling claim is refuted.","tokens_in":22674,"feed_emoji":"🔎","tokens_out":6408,"duration_ms":58854,"temperature":0.7,"pith_summary":"MaskSearch proposes that a language model's agentic search ability can be pre-trained as a self-supervised task: given a paragraph with several spans masked out, the model must query a search tool, read the results, and reconstruct the missing spans. The paper claims that this Retrieval-Augmented Mask Prediction (RAMP) task transfers to open-domain multi-hop question answering, so that pre-training on a 10M-example corpus of such tasks before fine-tuning on HotpotQA improves token-level recall by roughly 3 to 8 points over single-stage training on HotpotQA alone. The gains appear on both in-domain and out-of-domain QA benchmarks, across seven model sizes in the Qwen and LLaMA families, and with either supervised fine-tuning or reinforcement learning in the pre-training stage. The paper also reports that reinforcement learning on RAMP raises the performance ceiling after downstream training, and that a curriculum ordered by number of masks helps.","feed_headline":"Search-and-fill pretraining lifts QA recall 3-8 points","feed_subtitle":"A two-stage recipe, search to fill masked facts then fine-tune on QA, beats single-stage training on six benchmarks.","key_machinery":"The load-bearing object is the RAMP instance: a Wikipedia paragraph, 1 to 4 salient spans replaced by a [mask] token, and a search tool the model may call multiple times before emitting the reconstructed paragraph. Salient spans are chosen to require world knowledge rather than local context. Training examples are chain-of-thought trajectories from a multi-agent system (planner, rewriter, observer) filtered by LLM-as-Judge, then scaled by iterative self-evolve distillation with the growing model as teacher. SFT optimizes the trajectory directly, while RL uses DAPO with a reward that is half format compliance and half answer correctness, with the paper adopting a model-based answer judge after finding token-recall rewards invite reward hacking. Curriculum learning sorts examples by number of masks, from 1 to 4, to increase difficulty gradually.","core_discovery":"The central claim is that a universal, infinitely scalable search-agent pre-training task exists, and that RAMP is one. RAMP turns any paragraph into a training example by masking salient spans (entities, dates, ontologies, numbers) and requiring the model to plan, issue search queries, read retrieved passages, and output the complete restored text. Trained this way, the model internalizes a general decompose-search-observe-reason capability rather than a dataset-specific pattern. The paper demonstrates the transfer by first pre-training on RAMP, using up to 10M examples generated through multi-agent synthesis followed by self-evolving distillation, and then post-training on HotpotQA, reporting consistent recall improvements over the same models trained only on HotpotQA, with larger out-of-domain gains on FanoutQA, Musique, 2WikiMultiHopQA, Bamboogle, and FreshQA. It further claims that applying RL to RAMP via DAPO with a hybrid format-and-answer reward yields even higher downstream upper bounds, and that curriculum learning over mask counts strengthens the effect.","pith_inferences":["If the transfer is real, any task expressible as 'fill missing spans by searching' (entity resolution, fact-checking, data wrangling) could inherit the same pre-training benefit without new labeled data, since RAMP data is generated from raw text alone.","The paper does not test whether the gains come from the mask-filling objective itself or from sheer additional search-tool practice; a same-scale control pre-training on search trajectories without masks would isolate the mechanism.","The multi-tool extension the authors list as future work is a natural next test: masking spans that require dates, numbers, code, or images would turn RAMP into a general agentic pre-training task beyond retrieval.","Because the 10M dataset is generated by Qwen-family teachers and still transfers to LLaMA models, the task may enable cross-family pre-training data sharing at scale."],"forward_implications":["RAMP pre-training before HotpotQA post-training raises average token recall by several points across all six tested QA datasets and all tested model sizes, so the benefit is not tied to one architecture or benchmark.","Smaller models pre-trained with RAMP can reach recall levels of larger models that only received downstream training, suggesting pre-training can partially substitute for model scale.","Reinforcement learning on the RAMP task, not just supervised learning, improves later agentic QA performance, indicating the pre-training signal works for both training paradigms.","Curriculum learning ordered by number of masks outperforms mixing all difficulties, so example difficulty ordering matters for the pre-training stage.","The 10M-example scaling experiment shows performance continues to rise with more RAMP data for small models, supporting the scalability of the task."],"supporting_citations":[{"why":"Supplies the downstream task and the 58K CoT post-training set on which all compared methods are evaluated.","marker":"[52]"},{"why":"Provides the RL-with-search-engine paradigm and the Distilled Search-R1 and Search-R1 baselines that MaskSearch must beat.","marker":"[13]"},{"why":"Establishes retrieval-augmented language model pre-training, the line of work that RAMP extends to agentic search.","marker":"[11]"},{"why":"Supplies the salient span masking strategy used to select knowledge-demanding spans for RAMP.","marker":"[51]"},{"why":"The DAPO reinforcement learning algorithm used in the RL variant of RAMP pre-training.","marker":"[15]"},{"why":"The Qwen2.5 base models and the Qwen-Turbo extractor used to build RAMP data.","marker":"[1]"},{"why":"The LLaMA model family used to demonstrate that RAMP transfer holds across architectures.","marker":"[3]"},{"why":"An out-of-domain benchmark where the largest relative gains are reported.","marker":"[56]"}],"fun_headline_variants":["Search-mask pretraining boosts QA recall across domains","MaskSearch: universal pretraining for agentic search","RAMP: fill masks to make LLMs better search agents","Search-and-fill pretraining generalizes to QA","Masking spans teaches agents to search and reason"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attribution of the gains to the RAMP task design assumes that the effect is not just extra training data and extra search-tool practice, because the pre-training corpus has 10M examples while the compared baselines see only 58K HotpotQA trajectories.","fun_headline_variants_meta":{"raw":{"variants":["Search-mask pretraining boosts QA recall across domains","MaskSearch: universal pretraining for agentic search","RAMP: fill masks to make LLMs better search agents","Search-and-fill pretraining generalizes to QA","Masking spans teaches agents to search and reason"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000359,"raw_usage":{"total_tokens":2008,"prompt_tokens":1072,"completion_tokens":936,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":688,"completion_tokens_details":{"reasoning_tokens":859}},"tokens_in":688,"tokens_out":936,"duration_ms":7754,"temperature":1.0,"reasoning_tokens":859,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:55:30.671206+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune the same base model on a same-size 10M corpus of control trajectories that include the same searches but require no mask-filling (for example, copying the retrieved passage), then fine-tune on HotpotQA; if downstream recall matches the RAMP-pretrained model, the specific mask-filling claim is refuted.","supporting_citations":[{"cited_title":"Salient Span Masking for Temporal Understanding","cited_arxiv_id":"2303.12860","evidence_quote":"Supplies the salient span masking strategy used to select knowledge-demanding spans for RAMP."}],"review_version":1}