Pith. sign in

REVIEW 2 major objections 4 minor 66 references

Deep Research Pretraining via Predictive Navigation

T0 review · 2 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Paper citations and web links can pretrain deep-research navigation offline, so a model fine-tuned on a quarter of the expert trajectories beats the full-data model.

desk verdict A genuinely careful paper on offline pretraining for deep-research agents; the sample-efficiency gain looks real, but the hidden-oracle queries leave the central causal claim not fully closed. read the letter →

arxiv 2608.00432 v1 pith:3DT7UV2J submitted 2026-08-01 cs.CL

Jiang Zhou , Zhiyuan Fan , Xing Wu , Tinghao Yu , Feng Zhang , Lilin Wang This is my paper · ORCID
classification cs.CL
keywords deepresearchpretrainingpredictivenavigationofflinetrajectorysynthesiscitationgraphshyperlinksampleefficiencycontinualevidence-conditioned
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep research agents are usually trained on expensive, environment-grounded trajectories requiring repeated retrieval, document inspection, and report evaluation. This paper claims the same navigation skills can be acquired offline from evidence structures that already exist in document collections: citation edges in scholarly papers and hyperlinks in Wikipedia. The proposed method, Deep Research Pretraining (DRP), turns a citation-bearing or hyperlinked passage into a proxy research question, recovers the documents that support it, and renders a search–open–write trajectory without a live retrieval environment or any executed policy rollout. Continually pretraining a 14-billion-parameter base model on one billion tokens of such trajectories makes it consistently outperform its base model at one-sixteenth and one-quarter of the downstream expert-trajectory budget on the DeepResearch Bench evaluation, and with one quarter of the data the Wikipedia-based variant beats the no-DRP full-data model by 1.0 point. If the finding holds, expensive expert trajectories for research agents could be substantially replaced by cheap, static evidence graphs.

What carries the argument

The central object is predictive navigation: conditioned on the research objective and the evidence observed so far, the model learns to predict what to search for, which returned documents to open or ignore, and how to synthesize the gathered evidence. The machinery that produces this supervision is offline reverse synthesis from an evidence graph — starting from a citation-bearing or hyperlinked passage as the target report, the constructor recovers the supporting evidence set, draws topologically nearby but unselected documents as candidate alternatives, and builds the trajectory backwards into interleaved search–open–write steps. Search observations are rendered as titles and snippets for the candidate set, open observations as source text, and the final write as the normalized report, all without a live retrieval environment or an executed policy rollout. Training uses a masked causal language-model loss that puts weight only on assistant reasoning, tool calls and arguments, and the final report, masking out user objectives and tool observations; the ablations show the masking is load-bearing, since removing it drops the evaluation score by 8.1 points and collapses downstream tool use.

What would settle it

Generate the same DRP trajectories with every query and rationale produced by a generator that never sees the target report — the offline analogue of blind search — while keeping evidence partitioning, candidate sets, rendering budgets, and downstream fine-tuning identical. If the low-data gain over the base model collapses toward the level of the paper's raw-document control, then oracle knowledge of which documents are evidence, rather than an acquired navigation skill, is what the current numbers measure. A cheaper companion check: count how often DRP's synthesized queries reproduce the titles or key phrases of the target evidence documents; near-verbatim overlap would indicate the supervision leaks the answer into the question.

Watch

Extended reading notes

Core claim

The paper's central claim is that the decisions a deep-research agent must make — what to search for, which documents to inspect, and how to synthesize evidence into a report — can be taught by a predictive-navigation objective trained entirely offline on naturally occurring evidence graphs. Given a target passage that cites or links to documents, DRP infers a proxy research objective, partitions the supporting evidence into rounds, fills each round's candidate set with the evidence plus topologically related but unselected alternatives, and renders the interleaved search–open–write trajectory with rationales conditioned only on the history so far. The model is continually pretrained on these trajectories with the language-model loss masked to assistant reasoning, tool calls, and the final report, so it learns evidence-conditioned decisions rather than imitation of environment responses. Across five independently sampled subsets at one-sixteenth and one-quarter of the 13,062 expert trajectories, both the citation-graph variant and the hyperlink variant beat matched no-DRP models on DeepResearch Bench; the hyperlink variant at one-quarter data exceeds the no-DRP full-data checkpoint by 1.0 point with its 95% interval above zero, and the low-data gains transfer to ResearchQA, WebWalkerQA, and SimpleQA while surviving 80 steps of downstream agentic reinforcement learning. Source-matched raw-text and evidence-mismatch controls are used to argue that the benefit comes from evidence-conditioned navigation rather than domain exposure or agent-format imitation.

Load-bearing premise

The load-bearing premise is that supervision written by an oracle that already knows which documents are the evidence still teaches a policy that transfers to live search, where the model has to discover for itself which documents matter.

Editorial extensions

If this is right

  • With one quarter of the expert trajectories, a hyperlink-pretrained model beats the no-DRP full-data baseline by 1.0 point on DeepResearch Bench with its 95% interval above zero, so the same report quality can be reached with roughly 75% fewer expert demonstrations.
  • The low-data gains transfer beyond long-form report evaluation to ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy, and the advantage over the base model persists through 80 steps of downstream agentic reinforcement learning from matched checkpoints.
  • Because construction renders supervision from static corpora and graph neighborhoods, no search engine, browser, or executed policy rollout is needed to generate training data; the paper's cost accounting estimates at least a 51.8% reduction in standardized data-construction cost for the one-quarter-SFT recipe.
  • The same 1B-token DRP stage helps on two structurally different evidence graphs and at two model scales (a 14B dense model and a 30B-A3B model), and replacing half of a general midtraining budget with DRP tokens preserves aggregate general capability while improving downstream agent initialization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct consequence of the paper's evidence-mismatch control that the authors do not spell out: DRP's value should degrade as the evidence graph becomes noisier or less topically structured, so rewiring hyperlink or citation edges into random co-occurrence graphs would be a cheap test of whether the stated mechanism rather than raw text exposure carries the gain.
  • The offline oracle leaves a deployment gap the current benchmarks cannot close: during construction the generator knows which documents are evidence, while at inference the agent must discover that itself, so a blind-search evaluation with queries generated without access to the target report would separate genuine navigation skill from question-side leakage.
  • The unmasked-observation failure (predicting tool outputs costs 8.1 points and cuts downstream tool calls from 5.7 to 0.6) suggests a general design rule for offline agent pretraining — supervise decisions and synthesis, never the environment's responses — that could transfer to other agent-pretraining pipelines the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper introduces Deep Research Pretraining (DRP), an offline method that converts citation networks and Wikipedia hyperlink structures into synthetic 'predictive navigation' trajectories (search-open-write) for continual pretraining, with the goal of improving the sample efficiency of subsequent supervised fine-tuning on deep-research agent trajectories. The authors continually pretrain Qwen3-14B-Base (and Qwen3-30B-A3B-Base) on 1B tokens of DRP data, then fine-tune on 1/16, 1/4, or full DR Tulu SFT data, evaluating on DeepResearch Bench (DRB) and transfer benchmarks. Across five matched SFT subsets, DRP-Paper and DRP-Web outperform direct fine-tuning at low data budgets; DRP-Web with 1/4 data also surpasses the fixed full-data Base checkpoint on DRB. The paper includes source-matched raw-CPT controls, an evidence-mismatch ablation, a repeated-evaluation stability check, and cross-scale results, and argues that the gains stem from evidence-conditioned navigation rather than domain exposure or format imitation.

Significance. The experimental design is a genuine strength: independent SFT subsets with paired t-intervals, paired-bootstrap question intervals, raw-domain CPT controls, an evidence-mismatch ablation, a repeated-evaluation stability check, and cross-scale transfer are all included. If the sample-efficiency gain is robust, DRP offers a cheap, scalable source of proxy navigation supervision for deep-research agents, and the paper's careful uncertainty reporting is exemplary. The main weakness is that the causal attribution to 'evidence-conditioned navigation' is not fully isolated from oracle-informed query synthesis: the ablations hold the queries fixed and vary only observations or domain text, so they cannot rule out the possibility that part of the gain comes from learning to imitate gold-targeting search patterns. This is a load-bearing concern for the paper's central claim, but it is addressable with an additional ablation or a suitably tempered claim.

major comments (2)
  1. [Section 2.2 and Algorithm A1] The oracle leakage in query synthesis is not controlled. In Algorithm A1, each query q_i is generated by SynthesizeQuery(r, P_i, h), which receives the target report r and the gold evidence partition P_i; the filtering rule on line 21 removes only rationales that reveal the hidden construction process or unrevealed evidence, not the oracle signal embedded in the queries themselves. The raw-CPT control (Section 4.1) and the evidence-mismatch control (Section 4.2) both preserve these oracle-conditioned queries, so neither can distinguish a transferable evidence-conditioned navigation policy from a policy that has memorized gold-targeting query patterns. Because the abstract and conclusion attribute the gains to 'evidence-conditioned navigation,' this gap is load-bearing for the paper's main mechanistic claim. I request an ablation that re-synthesizes or perturbs queries without access to the gold report or evidence (e.g., conditioning query generation only on the history up to that point), or, failing that, a clear softening of the attribution claim to 'DRP improves trajectory-SFT sample efficiency' without the strong mechanistic conclusion.
  2. [Appendix A.7 / primary benchmark] Contamination is checked only for ResearchQA, not for the primary DeepResearch Bench evaluation. The DRP-Paper corpus is derived from arXiv source papers, and DRB is the headline benchmark; without an equivalent k-gram overlap analysis for the 50 DRB questions (and ideally their reference evidence pages), the possibility of benchmark contamination inflating the DRP-Paper DRB gains remains open. The transfer results and the DRP-Web improvements mitigate this concern, but the lack of a direct check for the main benchmark is a gap in an otherwise rigorous experimental report.
minor comments (4)
  1. [Section 5 (Related Work)] The sentence 'These literature motivates both our pre-SFT intervention and our source-matched raw-CPT controls' contains a subject-verb agreement error; it should read 'This literature motivates'.
  2. [Section 4.5] The claim that 'Longer context therefore provides no monotonic benefit in this experiment' is based on a single fixed SFT subset and a single pointing of the downstream budget; the sentence should explicitly say 'in this single-setting comparison' to avoid overgeneralization.
  3. [Table A3 / Section B.3] The repeated-evaluation stability check is reported for only one full-SFT checkpoint, which is acknowledged; the text could add one sentence noting that a similar check at a low-data budget would strengthen the main sample-efficiency intervals, which currently mix subset variation with evaluation noise.
  4. [Section A.1 / Algorithm A1] In Algorithm A1, the return of (τ, m) and the definition of m in step 22 are consistent, but the pseudocode for VerifyCite(y) = union of P_i is a step that is asserted rather than checked with a fallback; adding a brief note on how failures of this verification are handled (e.g., resampling or discarding) would improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the DRP gains are externally measured empirical results, and the only same-group citation (WRAP++ hyperlink motifs) is an implementation detail that does not load-bear the central claim.

full rationale

The paper's central claim is an empirical sample-efficiency measurement: continual pretraining on reverse-synthesized search–open–write trajectories improves downstream trajectory-SFT on DeepResearch Bench, with transfers to ResearchQA, WebWalkerQA, and SimpleQA and persistence through RL. None of these quantities is fitted to the target benchmark; the DRP objective in Eq. (7) is a standard masked language-model loss over synthetic trajectories, and the reported deltas are paired score differences computed from evaluation data that are not used to construct the DRP supervision. The construction in Section 2.2 uses the hidden target report as an oracle when generating supervision, and the paper explicitly filters rationales that reveal the hidden construction process or unrevealed evidence; this is a possible distribution-shift or leakage limitation in training-data synthesis, not a circular reduction, because the benchmark scores are not used to define or fit the DRP trajectories. The only same-group citation is WRAP++ (Zhou et al., 2026), used in Appendix A.1 to choose dual-link and co-mention hyperlink motifs for candidate alternatives; four authors overlap with the present paper, but this choice is an implementation detail and no uniqueness or forced-choice argument rests on it. The raw-domain and evidence-mismatch controls are empirical attribution checks rather than definitions. Hence no equation or fitted parameter reduces the claimed gain to its own input; the score of 2 reflects the minor non-load-bearing self-citation, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No invented physical or conceptual entities beyond the training objective itself. The free parameters are design choices and the response-length budget tuned on the target benchmark. The axioms are standard empirical assumptions for this type of LLM post-training study; the most fragile is the realism of oracle-generated synthetic supervision.

free parameters (3)
  • Search/open response-length budget (mid) = 400 tokens search; 40,000 tokens open
    Chosen as canonical after an ablation on a fixed 1/16 SFT subset and same DRB evaluation (Table 2); mid outperformed short and long budgets.
  • Search candidate set size = 10 candidates per observation
    Each search observation contains 10 candidates: all evidence documents for that round plus alternatives. The size is fixed by design and was not swept.
  • Continual-pretraining token budget = 1B tokens
    Fixed budget for all matched comparisons; the choice of 1B affects the magnitude of the effect, but it is a budget constraint rather than a tuned optimum.
assumptions (4)
  • domain assumption The GPT-5.5 RACE evaluator on the 50-question English DRB subset is a valid measure of deep-research report quality.
    The primary claim of sample-efficiency improvement rests on this evaluator; the small question count is addressed with paired bootstrap intervals.
  • domain assumption The 13,062 DR Tulu GPT-5-generated trajectories are representative expert demonstrations for deep-research agent SFT.
    Downstream SFT uses these trajectories as the ground-truth supervision whose sample efficiency DRP aims to improve.
  • domain assumption Qwen3.6-35B-A3B-FP8 generates faithful proxy research objectives, queries, and rationales that resemble genuine researcher navigation.
    The synthetic supervision is produced by this generator; if these traces are unrealistic, the learned navigation may not transfer to live environments.
  • domain assumption Citation and hyperlink graph neighborhoods supply plausible but unselected alternatives that make the selection contrast informative.
    The related-alternative candidates are drawn from graph topology; if these are trivially distinguishable from evidence, the model may learn a superficial selection signal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Research Pretraining via Predictive Navigation." pith.science (2026). https://pith.science/paper/3DT7UV2J

@misc{pith2026260800432,
  author       = {Pith},
  title        = {Pith review of: Deep Research Pretraining via Predictive Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3DT7UV2J}},
  note         = {Machine review of arXiv:2608.00432}
}
read the original abstract

Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, and report evaluation. We introduce Deep Research Pretraining (DRP), an offline framework that derives predictive navigation supervision from naturally occurring evidence structures. Given a citation-bearing or hyperlinked passage, DRP constructs a proxy research objective, recovers linked evidence and graph-related alternatives, and converts them into search-open-write trajectories. This teaches models what to search for, which documents to inspect, and how to synthesize evidence, without a live retrieval environment or executed policy rollout. We instantiate DRP on scholarly citation graphs (DRP-Paper) and Wikipedia hyperlinks (DRP-Web), continually pretrain separate Qwen3-14B-Base models on 1B tokens, and fine-tune them on controlled fractions of 13K agent trajectories. Across five independently sampled subsets at each low-data budget, both variants consistently outperform matched no-DRP models on DeepResearch Bench. With one quarter of the SFT data, DRP-Web even surpasses a fixed no-DRP full-data checkpoint, with gains transferring to ResearchQA, WebWalkerQA, and SimpleQA. Starting from matched low-data SFT checkpoints, the DRP-Web advantage also persists through subsequent agentic RL. Source-matched and evidence-mismatch controls indicate that these improvements arise from evidence-conditioned navigation rather than domain exposure or agent-format imitation. DRP thus provides a promising complementary approach to trajectory-based agent training.

Figures

Figures reproduced from arXiv: 2608.00432 by the authors.

Figure 1
Figure 1. DRP overview. Starting from a target report and its natural evidence graph, DRP infers a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. DeepResearch Bench (English) sample efficiency with Qwen3-14B. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. RACE dimension scores. Low-data bars average five independently sampled, matched SFT [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Transfer over Base on 500 examples per benchmark. Markers show paired metric differ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Agentic RL from matched 1/16-SFT checkpoints. Both initializations follow the same ARPO deep-search setting. Scores are the macro-average of ResearchQA coverage, WebWalkerQA accuracy, and SimpleQA accuracy. −15 −10 −5 0 5 10 15 Paired Δ RACE vs. Base (%) DRP-Web Raw-We…
Figure 6
Figure 6. Figure 6: Raw-domain CPT controls on DeepResearch Bench (English; Qwen3-14B; 1B-token CPT; [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Evidence-alignment control on DeepResearch Bench (English; Qwen3-14B; 1B-token [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 32 canonical work pages

  1. [1]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =

  2. [2]

    2019 , eprint=

    CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge , author=. 2019 , eprint=

    2019

  3. [3]

    2023 , eprint=

    FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning , author=. 2023 , eprint=

    2023

  4. [4]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  5. [5]

    2025 , eprint=

    Olmo 3 , author=. 2025 , eprint=

    2025

  6. [6]

    2025 , publisher =

    Guilherme Penedo , title =. 2025 , publisher =

    2025

  7. [7]

    2024 , eprint=

    Measuring short-form factuality in large language models , author=. 2024 , eprint=

    2024

  8. [8]

    Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering

    Zhou, Jiawei and Li, Xiaoguang and Shang, Lifeng and Luo, Lan and Zhan, Ke and Hu, Enrui and Zhang, Xinyu and Jiang, Hao and Cao, Zhao and Yu, Fan and Jiang, Xin and Liu, Qun and Chen, Lei. Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering. Proceedings of the 60th Annual Meeting of the Association for Computational Lin...

    doi:10.18653/v1/2022.acl-long.493 2022

Show all 66 references
  1. [9]

    The Synthetic Data Playbook: Generating Trillions of the Finest Tokens , author=

  2. [10]

    Second Conference on Language Modeling , year=

    Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models , author=. Second Conference on Language Modeling , year=

  3. [11]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  4. [12]

    2024 , eprint=

    Phi-4 Technical Report , author=. 2024 , eprint=

    2024

  5. [13]

    arXiv preprint arXiv:2602.15763 , year=

    GLM-5: from Vibe Coding to Agentic Engineering , author=. arXiv preprint arXiv:2602.15763 , year=

    arXiv

  6. [14]

    Nemotron- CC : Transforming C ommon C rawl into a Refined Long-Horizon Pretraining Dataset

    Su, Dan and Kong, Kezhi and Lin, Ying and Jennings, Joseph and Norick, Brandon and Kliegl, Markus and Patwary, Mostofa and Shoeybi, Mohammad and Catanzaro, Bryan. Nemotron- CC : Transforming C ommon C rawl into a Refined Long-Horizon Pretraining Dataset. Proceedings of the 63r...

    2025 doi

  7. [15]

    2024 , eprint=

    Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling , author=. 2024 , eprint=

    2024

  8. [16]

    arXiv preprint arXiv:1803.05457 , year=

    Think you have solved question answering? try arc, the ai2 reasoning challenge , author=. arXiv preprint arXiv:1803.05457 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2407.03120 , year=

    Are We Done with MMLU? , author=. arXiv preprint arXiv:2407.03120 , year=

    arXiv

  10. [18]

    arXiv preprint arXiv:1806.03822 , year=

    Know what you don't know: Unanswerable questions for SQuAD , author=. arXiv preprint arXiv:1806.03822 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:1903.00161 , year=

    DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs , author=. arXiv preprint arXiv:1903.00161 , year=

    1903 arXiv

  12. [20]

    arXiv preprint arXiv:1809.02789 , year=

    Can a suit of armor conduct electricity? a new dataset for open book question answering , author=. arXiv preprint arXiv:1809.02789 , year=

    arXiv

  13. [21]

    arXiv preprint arXiv:2109.07958 , year=

    TruthfulQA: Measuring how models mimic human falsehoods , author=. arXiv preprint arXiv:2109.07958 , year=

    arXiv

  14. [22]

    Communications of the ACM , volume=

    Winogrande: An adversarial winograd schema challenge at scale , author=. Communications of the ACM , volume=. 2021 , publisher=

    2021

  15. [23]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Piqa: Reasoning about physical commonsense in natural language , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  16. [24]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    HellaSwag: Can a Machine Really Finish Your Sentence? , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

  17. [25]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  18. [26]

    Compositional Semantic Parsing on Semi-Structured Tables , author=. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages=

  19. [27]

    arXiv preprint arXiv:1705.03551 , year=

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension , author=. arXiv preprint arXiv:1705.03551 , year=

    arXiv

  20. [28]

    arXiv preprint arXiv:2508.10975 , year=

    Beyondweb: Lessons from scaling synthetic data for trillion-scale pretraining , author=. arXiv preprint arXiv:2508.10975 , year=

    arXiv

  21. [29]

    2025 , eprint =

    Du, Mingxuan and Xu, Benfeng and Zhu, Chiwei and Wang, Xiaorui and Mao, Zhendong , title =. 2025 , eprint =

    2025

  22. [30]

    Shao, Rulin and Asai, Akari and Shen, Shannon Zejiang and Ivison, Hamish and Kishore, Varsha and Zhuo, Jingming and Zhao, Xinran and Park, Molly and Finlayson, Samuel G. and Sontag, David and Murray, Tyler and Min, Sewon and Dasigi, Pradeep and Soldaini, Luca and Brahman, Faez...

    2025

  23. [31]

    The Fourteenth International Conference on Learning Representations , year =

    Dong, Guanting and Mao, Hangyu and Ma, Kai and Bao, Licheng and Chen, Yifei and Wang, Zhongyuan and Chen, Zhongxia and Du, Jiazhen and Wang, Huiyang and Zhang, Fuzheng and Zhou, Guorui and Zhu, Yutao and Wen, Ji-Rong and Dou, Zhicheng , title =. The Fourteenth International Co...

  24. [32]

    2025 , eprint =

    Jin, Bowen and Zeng, Hansi and Yue, Zhenrui and Wang, Dong and Zamani, Hamed and Han, Jiawei , title =. 2025 , eprint =

    2025

  25. [33]

    2025 , eprint =

    Zheng, Yuxiang and Fu, Dayuan and Hu, Xiangkun and Cai, Xiaojie and Ye, Lyumanshan and Lu, Pengrui and Liu, Pengfei , title =. 2025 , eprint =

    2025

  26. [34]

    2025 , eprint =

    Sun, Hao and Qiao, Zile and Guo, Jiayan and Fan, Xuanbo and Hou, Yingyan and Jiang, Yong and Xie, Pengjun and Huang, Fei and Zhang, Yan , title =. 2025 , eprint =

    2025

  27. [35]

    2026 , eprint =

    Li, Zhuofeng and Jiang, Dongfu and Ma, Xueguang and Zhang, Haoxiang and Nie, Ping and Zhang, Yuyu and Zou, Kai and Xie, Jianwen and Zhang, Yu and Chen, Wenhu , title =. 2026 , eprint =

    2026

  28. [36]

    2025 , eprint =

    Su, Liangcai and Zhang, Zhen and Li, Guangyu and Chen, Zhuo and Wang, Chenxi and Song, Maojia and Wang, Xinyu and Li, Kuan and Wu, Jialong and Chen, Xuanzhong and Qiao, Zile and Zhang, Zhongwang and Yin, Huifeng and Cai, Shihao and Fang, Runnan and Tao, Zhengwei and Yin, Wenbi...

    2025

  29. [37]

    2025 , eprint =

    Li, Zijian and Guan, Xin and Zhang, Bo and Huang, Shen and Zhou, Houquan and Lai, Shaopeng and Yan, Ming and Jiang, Yong and Xie, Pengjun and Huang, Fei and Zhang, Jun and Zhou, Jingren , title =. 2025 , eprint =

    2025

  30. [38]

    2025 , eprint =

    Yang, An and Li, Anfeng and others , title =. 2025 , eprint =

    2025

  31. [39]

    2512.13961 , archivePrefix =

    OLMo 3 , year =. 2512.13961 , archivePrefix =

    arXiv

  32. [40]

    2026 , eprint =

    Xie, Jian and Lin, Tianhe and Wang, Zilu and Ning, Yuting and Yao, Yuekun and Xue, Tianci and Zhang, Zhehao and Li, Zhongyang and Zhang, Kai and Wu, Yufan and Chen, Shijie and Gou, Boyu and Han, Mingzhe and Wang, Yifei and Lee, Vint and Wei, Xinpeng and Wang, Xiangjun and Su, ...

    2026

  33. [41]

    How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data , year =

    Niklaus, Joel and Yamaguchi, Atsuki and. How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data , year =. 2604.13977 , archivePrefix =

    arXiv

  34. [42]

    2025 , eprint =

    Wu, Jialong and Yin, Wenbiao and Jiang, Yong and Wang, Zhenglin and Xi, Zekun and Fang, Runnan and Zhang, Linhai and He, Yulan and Zhou, Deyu and Xie, Pengjun and Huang, Fei , title =. 2025 , eprint =

    2025

  35. [43]

    2024 , eprint =

    Wei, Jason and Nguyen, Karina and Chung, Hyung Won and Jiao, Yunxin Joy and Papay, Spencer and Glaese, Amelia and Schulman, John and Fedus, William , title =. 2024 , eprint =

    2024

  36. [44]

    2021 , eprint =

    Nakano, Reiichiro and Hilton, Jacob and Balaji, Suchir and Wu, Jeff and Ouyang, Long and Kim, Christina and Hesse, Christopher and Jain, Shantanu and Kosaraju, Vineet and Saunders, William and Jiang, Xu and Cobbe, Karl and Eloundou, Tyna and Krueger, Gretchen and Button, Kevin...

    2021

  37. [45]

    International Conference on Learning Representations , year =

    Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , title =. International Conference on Learning Representations , year =

  38. [46]

    Don't Stop Pretraining: Adapt Language Models to Domains and Tasks , booktitle =

    Gururangan, Suchin and Marasovi. Don't Stop Pretraining: Adapt Language Models to Domains and Tasks , booktitle =. 2020 , pages =

    2020

  39. [47]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , year =

    Lee, Kenton and Chang, Ming-Wei and Toutanova, Kristina , title =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , year =

  40. [48]

    unarXive 2024: A Large-Scale Scientific Corpus for Citation-Aware Retrieval and Generation , year =

    Besrour, Ines and F. unarXive 2024: A Large-Scale Scientific Corpus for Citation-Aware Retrieval and Generation , year =. doi:10.63317/2nqzwzhq3j3t , booktitle =

    2024 doi

  41. [49]

    2025 , howpublished =

    Penedo, Guilherme , title =. 2025 , howpublished =

    2025

  42. [50]

    International Conference on Learning Representations , year =

    Loshchilov, Ilya and Hutter, Frank , title =. International Conference on Learning Representations , year =

  43. [51]

    , title =

    Efron, Bradley and Tibshirani, Robert J. , title =

  44. [52]

    2018 , eprint =

    Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , title =. 2018 , eprint =

    2018

  45. [53]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , year =

    Mihaylov, Todor and Clark, Peter and Khot, Tushar and Sabharwal, Ashish , title =. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , year =

    2018

  46. [54]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , year =

    Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin , title =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , year =

  47. [55]

    Proceedings of the AAAI Conference on Artificial Intelligence , year =

    Bisk, Yonatan and Zellers, Rowan and Le Bras, Ronan and Gao, Jianfeng and Choi, Yejin , title =. Proceedings of the AAAI Conference on Artificial Intelligence , year =

  48. [56]

    2021 , eprint =

    Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and Chen, Mark and Jun, Heewoo and Kaiser, Lukasz and Plappert, Matthias and Tworek, Jerry and Hilton, Jacob and Nakano, Reiichiro and Hesse, Christopher and Schulman, John , title =. 2021 , eprint =

    2021

  49. [57]

    Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    Talmor, Alon and Herzig, Jonathan and Lourie, Nicholas and Berant, Jonathan , title =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    2019

  50. [58]

    Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics , year =

    Gema, Aryo Pradipta and others , title =. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics , year =

    2025

  51. [59]

    Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    Dua, Dheeru and Wang, Yizhong and Dasigi, Pradeep and Stanovsky, Gabriel and Singh, Sameer and Gardner, Matt , title =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    2019

  52. [60]

    and Salakhutdinov, Ruslan and Manning, Christopher D

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William W. and Salakhutdinov, Ruslan and Manning, Christopher D. , title =. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , year =

    2018

  53. [61]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , year =

    Joshi, Mandar and Choi, Eunsol and Weld, Daniel and Zettlemoyer, Luke , title =. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , year =

  54. [62]

    Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics , year =

    Pasupat, Panupong and Liang, Percy , title =. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics , year =

  55. [63]

    Proceedings of the 28th International Conference on Computational Linguistics , year =

    Ho, Xanh and Nguyen, Anh-Khoa Duong and Sugawara, Saku and Aizawa, Akiko , title =. Proceedings of the 28th International Conference on Computational Linguistics , year =

  56. [64]

    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics , year =

    Rajpurkar, Pranav and Jia, Robin and Liang, Percy , title =. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics , year =

  57. [65]

    Transactions of the Association for Computational Linguistics , volume =

    Trivedi, Harsh and Balasubramanian, Niranjan and Khot, Tushar and Sabharwal, Ashish , title =. Transactions of the Association for Computational Linguistics , volume =. 2022 , doi =

    2022

  58. [66]

    2604.06829 , archivePrefix =

    Jiang Zhou and Yunhao Wang and Xing Wu and Tinghao Yu and Feng Zhang , year =. 2604.06829 , archivePrefix =

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.