REVIEW 4 major objections 6 minor 7 cited by
Scientific taste can be learned from community feedback, and a trained 30B judge outperforms frontier LLMs at predicting which papers will earn more citations.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 18:09 UTC pith:A4TIKN42
load-bearing objection A large, well-run citation-preference training study whose judge results are credible and transferable, but whose ideation claims rest on LLM judges predicting the same citation construct used as reward. the 4 major comments →
AI Can Learn Scientific Taste
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that 'scientific taste' is a learnable objective rather than an ineffable human trait. It formalizes taste as two measurable capabilities: judging which research idea has higher potential impact, defined through expected future citations, and proposing ideas likely to accumulate impact. The proposed paradigm, Reinforcement Learning from Community Feedback (RLCF), converts citation inequalities into preference labels, trains a generative reward model with reinforcement learning to predict those labels, and then uses that judge to train an ideation model via comparison-based reinforcement learning. Across model sizes, judgment accuracy rises consistently; the 30B j
What carries the argument
The load-bearing mechanism is RLCF, instantiated in three stages. SciJudgeBench is a dataset of 696,758 matched paper pairs where the label is which abstract attracted more citations. Scientific Judge is a generative reward model trained with GRPO on binary preference-prediction correctness; this is the component that turns raw citation counts into a reusable 'taste' signal. Scientific Thinker is a policy trained by Comparison-Based GRPO, where candidate ideas are ranked against group-mates by the judge in a round-robin tournament and receive win-rate rewards. The matched-pair construction matters most: it removes field and time confounds so that the preference signal reflects community judg
Load-bearing premise
The load-bearing premise — acknowledged in the paper's own limitations — is that citation count, once matched by field and time, is a valid proxy for 'potential impact'; if citations measure popularity or momentum rather than scientific value, the trained judge and thinker have learned citation popularity, not taste.
What would settle it
A concrete check: assemble pairs where the lower-cited paper later becomes recognized as the more intellectually important work despite slow early citations, and ask Scientific Judge to rank them. The paper itself concedes such high-potential, delayed-impact papers exist. If accuracy on those pairs is at chance while accuracy on uniformly popular papers stays high, the learned signal is popularity rather than taste.
If this is right
- Scientific judgment scales with data and model size: accuracy improves log-linearly with more training pairs and with larger backbones, so the approach should continue improving as research corpora and models grow.
- The learned taste is transferable: a judge trained on citation preferences from one period can predict future-year preferences, generalize to fields it never trained on, and align with peer-review scores.
- A citation-trained judge works as an effective reward model for open-ended ideation, substantially improving proposed research ideas without degrading general language-model capabilities.
- This provides a pathway for AI scientists to be trained on what is worth doing, not just how to do it, by treating community feedback as a learnable supervision signal.
Where Pith is reading between the lines
- A testable extension not in the paper: train the judge on citation trajectories split by early versus delayed impact, and check whether it recognizes 'sleeping beauties' — papers with low early citations that later become highly influential. If it fails on those pairs, the model is tracking popularity rather than underlying scientific value.
- The same RLCF recipe could be ported to other community signals, such as dataset downloads, code reuse, or replication attempts, testing whether taste learned from one community metric transfers to another.
- If citation-preference optimization enters real research pipelines, it may entrench existing popularity dynamics and field biases; the strongest use is as a filter to support human decisions rather than as an autonomous evaluator.
- Prospective validation would settle the ideation claim: execute or publish a sample of the Thinker's proposals and track their actual citation and impact trajectories against judge predictions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Reinforcement Learning from Community Feedback (RLCF), a pipeline that converts citation counts into field- and time-matched pairwise preference labels (SciJudgeBench, 696,758 pairs), trains a generative reward model (Scientific Judge) with GRPO, and then trains a policy (Scientific Thinker) using that judge as a comparison-based reward. The narrow empirical claim is that Scientific Judge predicts which paper in a matched pair will receive more citations, that this capability scales with model and data size, and that it transfers to future years, unseen fields, and peer-review preferences. The broader claim is that this constitutes learning 'scientific taste'—the capacity to judge and propose research ideas with high potential impact—and that Scientific Thinker generates ideas with higher potential impact than strong baselines. The paper reports consistent gains across eight open backbones and several proprietary LLM baselines, with in-domain judge accuracy up to 83.7% and Thinker win rates of 76.5% and 81.5% against base policies.
Significance. If the narrow claim is accepted, this is a useful and reproducible contribution: a large public benchmark for citation-preference prediction, a scalable RL training recipe, and a broad set of transfer results across time, field, and evaluation metric. The consistency of the gains across model families and sizes, the position-swap evaluation protocol, and the released code repository are strengths. However, the paper's central advertised claim—that AI can learn scientific taste—is not established. The formal definition in Sec. 2.1 equates potential impact with expected citations, so the judge experiments show learning of citation-preference patterns, not taste in any broader sense. More importantly, the ideation half of the claim is evaluated entirely by LLM judges predicting the same citation construct used as the training reward, with no human-expert, prospective, or realized-impact anchor. The paper's own Limitations section concedes this. The ideation results are therefore not load-bearing evidence for 'learning scientific taste' without additional external validation or a substantial reframing of the claim.
major comments (4)
- [Sec. 2.1 (Eq. 4) vs. Sec. 5.1 / Appendix C.2] ThinkerCap is formally defined as the expected actual citation impact of proposed ideas, but the evaluation in Sec. 5.1 and Appendix C.2 measures win rates assigned by GPT-5.2, GLM-5, and Gemini 3 Pro asked to predict which idea 'has a higher citation count.' This is the same construct used as the training reward (Eq. 7). The paper's Limitations section states that 'our ideation evaluation mainly relies on strong LLM evaluators... may not fully reflect their potential impact.' That concession applies directly to the central claim. The 84.4% agreement of the LLM-judge majority on SciJudgeBench (Appendix C.2) validates the judges on real paper pairs, not on unvalidated generated ideas. An external anchor—human expert ratings, prospective citation tracking, implementation of a sample of ideas, or at least a held-out realized-impact test on proposed ideas—is needed before the ideation result
- [Sec. 3.3, Eq. (7) and Sec. 5.2] The Thinker is optimized to win round-robin comparisons made by SciJudge, a model trained on citation labels. The evaluation then asks three LLMs to predict citations for the generated ideas. Because the evaluator and the reward model share the same objective and are trained on the same type of signal, the reported 81.5%/83.0% win rates may reflect reward hacking—e.g., producing abstracts containing stylistic markers such as 'framework' or 'fundamental bottleneck' that sway citation-predicting LLMs—rather than genuine potential impact. The paper does not test this alternative explanation. Control analyses, such as ablating the reward model, comparing against an independent human panel, or testing whether the generated ideas actually attract citations when posted or implemented, are necessary to distinguish reward hacking from taste.
- [Sec. 2.1, Eq. (1)] The formal definition sets I(p) to cumulative expected citations and then defines scientific taste entirely in terms of this quantity. Consequently, the statement 'AI can learn scientific taste' is, by construction, equivalent to 'AI can learn to predict citation impact.' This is circular for the broad claim: the judge's training labels, the judge's evaluation, the Thinker's reward, and the Thinker's evaluation all reduce to the same citation-prediction objective. The narrow claim—that models can learn transferable citation-preference patterns—is supported by the experiments, but the paper should either provide an independent operationalization of scientific value or explicitly restrict the title, abstract, and conclusions to 'citation-based community preference.' The current framing overstates what the evidence shows.
- [Appendix C.2 / Sec. 5.2] The ideation evaluation uses only 200 in-domain and 200 out-of-domain seed papers, and reports aggregate win rates without confidence intervals, inter-judge agreement, or per-seed variance. Given that some of the field-wise judge test sets are very small (e.g., Math has 9 pairs in Table 4), the stability of the reported differences is unclear. The paper should report seed-level bootstrap or Bayesian intervals, and the per-model vote breakdown for majority-win rates. This is especially important because the central ideation claim rests entirely on a small LLM-judged sample.
minor comments (6)
- [Abstract vs. Table 3] The abstract states that SciJudge-30B outperforms 'GPT-5.4 Thinking,' but Table 3 and the results text report 'GPT-5.2-Thinking.' Please reconcile the model names.
- [Table 1 caption] Typo: 'distributional priming—ahat' should read '—a hat' or '—a'. Also, the table caption says 'Scientific Thinker' but the figure is Figure 1; renumber if necessary.
- [Fig. 3] The x-axis is labeled 'Training Progress' with ticks 0 to 1, but no definition or correspondence to steps/epochs is given. Please state what the axis represents, e.g., fraction of training tokens or steps.
- [Tables 4, 5, and 7] Some cells are based on very small samples (e.g., 9 Math pairs in Table 4, 160 bioRxiv pairs in Table 7). Please add sample sizes and confidence intervals or explicitly caution against over-interpreting field-wise differences.
- [Appendix B.5] The position-swap consistency metric requires correct predictions under both orderings, which penalizes models that are correct but position-sensitive. This is a reasonable robustness choice, but it should be reported alongside the non-consistency accuracy so readers can separate position bias from correctness.
- [Appendix A.3] The ICLR test set evaluation prompt asks 'which paper is more likely to be accepted,' which differs from the training prompt ('which paper has a higher citation count'). The text acknowledges this, but the main body (Sec. 4.1, Table 6) describes it simply as 'peer-review preference.' Please make the prompt change explicit in the main text, as it affects interpretation of the cross-metric transfer result.
Circularity Check
Scientific taste is stipulated as citation-based impact, and the ideation half is trained and evaluated with citation-preference models, so the core learning claim partly reduces to citation-preference prediction.
specific steps
-
self definitional
[Sec 2.1 (Eq. 1), Sec 3.1, Sec 4.1]
"Here, we use the term to refer to the capacity to judge and propose research ideas with high potential impact. To make this notion precise, we provide a layered formal definition. Potential Impact. ... Citations are the most common way to measure the impact of scientific research [16,17]. ... The cumulative expected impact of paper p is defined as: I(p)=lim_{N→∞} ∑_{t=1}^{N} E[c_t(p)], (1)"
The paper's formal definition makes 'scientific taste' equivalent to predicting citation-based impact: Eq. 1 defines potential impact I(p) as cumulative expected citations, and Sec 3.1 constructs all training labels from citation counts. Therefore the empirical statement 'AI can learn scientific taste' is, by the paper's own equations, the statement 'AI can learn to predict citation-count comparisons.' The conclusion is entailed by the stipulative definition rather than independently tested; the judge experiments, while valid for citation prediction, cannot confirm the broader construct labeled 'scientific taste.'
-
fitted input called prediction
[Sec 3.3 (Eq. 7), Sec 5.1, Appendix C.1/C.2]
"using Scientific Judge as the reward model, we train a policy model via reinforcement learning called Scientific Thinker. ... The comparison-based reward for o_i is the research idea's win rate within the group: r_i = 1/(G-1) ∑_{j≠i} s(o_i,o_j), (7) ... Based on the titles and abstracts of the following two papers A and B, determine which paper has a higher citation count. Suppose the two papers are published at the same time."
The Thinker's training reward (Eq. 7) is SciJudge's pairwise citation-preference judgement, and SciJudge is itself trained on citation-count labels (Sec 3.2). The ideation evaluation uses different LLMs but the identical construct: the prompt asks which idea 'has a higher citation count,' and Appendix C.2 validates the evaluator by its 84.4% agreement with SciJudgeBench citation labels. Thus the reported win rates measure how well the Thinker satisfies citation-preference predictors—the same construct optimized during training. The paper's own limitation concedes that LLM evaluation 'may not fully reflect their potential impact,' so no independent anchor (human experts, implementation, realized citations) supports the ideation claim.
full rationale
The judgement half is a genuine empirical contribution: SciJudge is trained on 696,758 citation pairs and evaluated on held-out, future-year, cross-field, and ICLR peer-review test sets; those results are not circular because the labels are external and the model must generalize. The circularity is concentrated at the level of interpretation and the ideation loop. First, the paper stipulates that potential impact equals cumulative citations (Eq. 1), so 'learning scientific taste' and 'learning citation-preference prediction' are the same statement by construction; the broad title claim cannot be independently confirmed by the judge experiments. Second, the Thinker is trained with SciJudge as reward (Eq. 7) and evaluated by LLM judges prompted to predict citation counts, with the evaluator itself benchmarked against citation labels (Appendix C.2). The ideation win rates therefore show optimization and measurement of the same citation-preference proxy, not realized scientific impact. The paper's limitations section explicitly acknowledges this. Self-citations are present but not load-bearing; no uniqueness theorem or ansatz is smuggled via self-citation. Overall, because the central ideation claim reduces to in-loop proxy optimization and the taste construct is defined as that proxy, the score is 6.
Axiom & Free-Parameter Ledger
free parameters (4)
- Training pair thresholds (absolute/relative citation difference) =
abs ≥ 8 and relative ≥ 30% for training; abs ≥ 32 and relative ≥ 50% for main test
- Temporal OOD threshold δ_s =
max(12, 0.7 × (q99 − q1)) within each subcategory
- ICLR metric-OOD retention rule =
top/bottom 10% by review rating, capped at 75 papers per side
- GRPO hyperparameters =
lr 8e-7 (judge) / 5e-7 (thinker), KL β 0.03 / 0.001, G=8, temperature 1.0
axioms (4)
- domain assumption Citation count is a valid proxy for long-term scientific impact / scientific taste
- domain assumption Field- and time-matched pairwise citations remove confounders sufficiently to reveal 'taste'
- domain assumption LLM judges can predict which of two contemporaneous ideas would receive more citations, and this is a valid measure of ideation capability
- domain assumption Comparison-based GRPO on a citation-trained judge reward does not reward-hack and faithfully improves ideation
read the original abstract
Scientific discovery depends on expert judgement and foresight, which we call scientific taste: the ability to judge and propose research ideas with potential for long-term scientific impact. Whether AI can learn this ability remains an open question. Here we provide evidence that artificial intelligence can learn judgement and ideation. We introduce Reinforcement Learning from Community Feedback (RLCF), a training paradigm that uses large-scale signals from scientific community as supervision. We first train Scientific Judge on field- and time-matched pairs of high- vs. low-citation papers to judge ideas. We then train a Scientific Thinker, to propose research ideas with high potential impact. Experiments show that the 30B Scientific Judge variant outperforms strong LLM baselines (e.g., GPT-5.4 Thinking), while Scientific Judge generalizes across future-year papers, unseen fields, and other community metrics. Furthermore, Scientific Thinker proposes research ideas with higher potential impact than baselines. These results suggest that AI can learn scientific taste, marking an important step towards AI systems that could help accelerate scientific discovery.
Forward citations
Cited by 7 Pith papers
-
GIANTS: Generative Insight Anticipation from Scientific Literature
GIANTS-4B, trained with RL on a new 17k-example benchmark of parent-to-child paper insights, achieves 34% relative improvement over gemini-3-pro in LM-judge similarity and is rated higher-impact by a citation predictor.
-
Measuring the Gap Between Human and LLM Research Ideas
LLM-generated research ideas cluster more around bridge-like opportunities and synthesis methods than the broader distribution seen in human papers.
-
SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?
SoundnessBench shows frontier LLMs exhibit pervasive optimism bias when rating the soundness of ML research proposals, frequently calling low-soundness ideas sound under standard prompts.
-
GraphReview: Scientific Paper Evaluation via LLM-Based Graph Message Passing
GraphReview models paper evaluation as LLM-driven message passing on a semantic paper graph that links intrinsic quality, contemporaneous papers, and prior work, then applies Personalized PageRank for ranking and revi...
-
FAME: Forecasting Academic Impact via Continuous-Time Manifold Evolution
FAME models scientific topic trajectories in continuous time to forecast paper impact more accurately than LLMs by aligning manuscripts with field momentum in a dynamic latent space.
-
ForeSci: Evaluating LLM Agents for Forward-Looking AI Research Judgment
ForeSci is a temporally controlled benchmark with 500 tasks for assessing LLM agents on forward-looking AI research judgments in four domains using cutoff-aligned knowledge bases.
-
ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
ARIS is a three-layer open-source system that uses cross-model adversarial collaboration plus claim-auditing pipelines to make LLM-driven research workflows more reliable.
Reference graph
Works this paper leans on
-
[1]
Terri and g&d: celebrating 50 years of enlightened scientific judgment.Genes&Development, 37(1-2): 6–8, 2023
Robert Tjian. Terri and g&d: celebrating 50 years of enlightened scientific judgment.Genes&Development, 37(1-2): 6–8, 2023
2023
-
[2]
Mitchison
Timothy J. Mitchison. A question of taste.Molecular Biology of the Cell, 24:3278 – 3280, 2013. URLhttps: //api.semanticscholar.org/CorpusID:264271579
2013
-
[3]
Introducing deep research
OpenAI. Introducing deep research. https://openai.com/index/introducing-deep-research/, 2025. Accessed: 2025-02-02
2025
-
[4]
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025
Pith/arXiv arXiv 2025
-
[5]
Deepresearcher: Scalingdeepresearchviareinforcementlearninginreal-worldenvironments
YuxiangZheng,DayuanFu,XiangkunHu,XiaojieCai,LyumanshanYe,PengruiLu,andPengfeiLiu. Deepresearcher: Scalingdeepresearchviareinforcementlearninginreal-worldenvironments. In Proceedingsofthe2025Conference on EmpiricalMethods inNaturalLanguageProcessing, pages 414–431, 2025
2025
-
[6]
Wispaper: Your ai scholar search engine, 2026
Li Ju, Jun Zhao, Mingxu Chai, Ziyu Shen, Xiangyang Wang, Yage Geng, Chunchun Ma, Hao Peng, Guangbin Li, Tao Li, Chengyong Liao, Fu Wang, Xiaolong Wang, Junshen Chen, Rui Gong, Shijia Liang, Feiyan Li, Ming Zhang, Kexin Tan, Junjie Ye, Zhiheng Xi, Shihan Dou, Tao Gui, Yuankai Ying, Yang Shi, Yue Zhang, and Qi Zhang. Wispaper: Your ai scholar search engine, ...
Pith/arXiv arXiv 2026
-
[7]
Codex, 2025
OpenAI. Codex, 2025. URLhttps://openai.com/codex/
2025
-
[8]
Claude code, 2025
Anthropic. Claude code, 2025. URLhttps://www.anthropic.com/claude-code
2025
-
[9]
Introducing fars, 2026
Analemma. Introducing fars, 2026. URLhttps://analemma.ai/blog/introducing-fars/. 13
2026
-
[10]
Agent laboratory: Using llm agents as research assistants, 2025
Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Michael Moor, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants, 2025. URLhttps: //arxiv.org/abs/2501.04227
Pith/arXiv arXiv 2025
-
[11]
The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search, 2025
Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist-v2: Workshop-level automated scientific discovery via agentic tree search, 2025. URL https://arxiv.org/abs/2504.08066
Pith/arXiv arXiv 2025
-
[12]
Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers, 2024
Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers, 2024. URLhttps://arxiv.org/abs/2409.04109
Pith/arXiv arXiv 2024
-
[13]
The ideation-execution gap: Execution outcomes of llm-generated versus human research ideas, 2025
Chenglei Si, Tatsunori Hashimoto, and Diyi Yang. The ideation-execution gap: Execution outcomes of llm-generated versus human research ideas, 2025. URLhttps://arxiv.org/abs/2506.20803
Pith/arXiv arXiv 2025
-
[14]
Of the Standard of Taste (1757), pages 145–154
David Hume. Of the Standard of Taste (1757), pages 145–154. SUNY Press, 2026. ISBN 9798855805482. doi: doi:10.1515/9798855805482-019. URLhttps://doi.org/10.1515/9798855805482-019
-
[15]
Art and Its Significance: An Anthology of Aesthetic Theory,Third Edition
Immanuel Kant. Art and Its Significance: An Anthology of Aesthetic Theory,Third Edition. State University of New York Press, 1994. ISBN 9780791418529. URLhttp://www.jstor.org/stable/jj.18254729
1994
-
[16]
Quantifying long-term scientific impact.Science, 342 (6154):127–132, 2013
Dashun Wang, Chaoming Song, and Albert-László Barabási. Quantifying long-term scientific impact.Science, 342 (6154):127–132, 2013
2013
-
[17]
Science of science.Science, 359(6379):eaao0185, 2018
SantoFortunato,CarlTBergstrom,KatyBörner,JamesAEvans,DirkHelbing,StašaMilojević,AlexanderMPetersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, et al. Science of science.Science, 359(6379):eaao0185, 2018
2018
-
[18]
Worldpm: Scaling human preference modeling, 2025
Binghai Wang, Runji Lin, Keming Lu, Le Yu, Zhenru Zhang, Fei Huang, Chujie Zheng, Kai Dang, Yang Fan, Xingzhang Ren, An Yang, Binyuan Hui, Dayiheng Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Bowen Yu, Jingren Zhou, and Junyang Lin. Worldpm: Scaling human preference modeling, 2025. URLhttps: //arxiv.org/abs/2505.10527
Pith/arXiv arXiv 2025
-
[19]
Training language models to follow instructions with human feedback, 2022
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback, 2022
2022
-
[20]
NisanStiennon,LongOuyang,JeffWu,DanielM.Ziegler,RyanLowe,ChelseaVoss,AlecRadford,DarioAmodei,and PaulChristiano.Learningtosummarizefromhumanfeedback,2022.URL https://arxiv.org/abs/2009.01325
Pith/arXiv arXiv 2022
-
[21]
Smith, and Hannaneh Hajishirzi
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language modeling, 2024. URLhttps://arxiv.org/abs/2403.13787
Pith/arXiv arXiv 2024
-
[22]
Rmb: Comprehensively benchmarking reward models in llm alignment, 2025
Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. Rmb: Comprehensively benchmarking reward models in llm alignment, 2025. URLhttps://arxiv.org/abs/2410.09893
Pith/arXiv arXiv 2025
-
[23]
Jiaxin Guo, Zewen Chi, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. Reward reasoning model. arXiv preprintarXiv:2505.14674, 2025
Pith/arXiv arXiv 2025
-
[24]
Generative reward models.arXivpreprintarXiv:2410.12832, 2024
Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models.arXivpreprintarXiv:2410.12832, 2024
Pith/arXiv arXiv 2024
-
[25]
Generative verifiers: Reward modeling as next-token prediction.arXiv preprintarXiv:2408.15240, 2024
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction.arXiv preprintarXiv:2408.15240, 2024
Pith/arXiv arXiv 2024
-
[26]
Inference-time scaling for generalist reward modeling.arXivpreprintarXiv:2504.02495, 2025
Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling.arXivpreprintarXiv:2504.02495, 2025
arXiv 2025
-
[27]
Rm-r1: Reward modeling as reasoning.arXivpreprintarXiv:2505.02387, 2025
Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. Rm-r1: Reward modeling as reasoning.arXivpreprintarXiv:2505.02387, 2025
arXiv 2025
-
[28]
Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation.arXiv preprintarXiv:2503.05236, 2025. 14
Pith/arXiv arXiv 2025
-
[29]
Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Unified multimodal chain-of-thought reward model through reinforcement fine-tuning.arXivpreprintarXiv:2505.03318, 2025
arXiv 2025
-
[30]
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models,
-
[31]
Yibin Wang, Zhimin Li, Yuhang Zang, Yujie Zhou, Jiazi Bu, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Pref-grpo: Pairwise preference reward-based grpo for stable text-to-image reinforcement learning.arXivpreprint arXiv:2508.20751, 2025
Pith/arXiv arXiv 2025
-
[32]
The invisible leash: Why rlvr may or may not escape its origin.arXivpreprintarXiv:2507.14843, 2025
Fang Wu, Weihao Xuan, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, and Yejin Choi. The invisible leash: Why rlvr may or may not escape its origin.arXivpreprintarXiv:2507.14843, 2025
arXiv 2025
-
[33]
Group sequence policy optimization, 2025
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy optimization, 2025. URL https: //arxiv.org/abs/2507.18071
Pith/arXiv arXiv 2025
-
[34]
Mingzhe Li, Xin Lu, and Yanyan Zhao. Self-foveate: Enhancing diversity and difficulty of synthesized instructions from unsupervised text via multi-level foveation, 2026. URLhttps://arxiv.org/abs/2507.23440
arXiv 2026
-
[35]
Ming Zhang, Jiabao Zhuang, Wenqing Jing, Kexin Tan, Ziyu Kong, Jingyi Deng, Yujiong Shen, Yuhang Zhao, Ning Luo, Renzhe Zheng, Jiahui Lin, Mingqi Wu, Long Ma, Shihan Dou, Tao Gui, Qi Zhang, and Xuanjing Huang. Can deep research agents retrieve and organize? evaluating the synthesis gap with expert taxonomies, 2026. URL https://arxiv.org/abs/2601.12369
Pith/arXiv arXiv 2026
-
[36]
Mle-bench: Evaluating machine learning agents on machine learning engineering
Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. Mle-bench: Evaluating machine learning agents on machine learning engineering. arXivpreprintarXiv:2410.07095, 2024
Pith/arXiv arXiv 2024
-
[37]
The ai scientist: Towards fully automated open-ended scientific discovery, 2024
Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery, 2024. URLhttps://arxiv.org/abs/2408.06292
Pith/arXiv arXiv 2024
-
[38]
Ai4research: A survey of artificial intelligence for scientific research, 2025
Qiguang Chen, Mingda Yang, Libo Qin, Jinhao Liu, Zheng Yan, Jiannan Guan, Dengyun Peng, Yiyan Ji, Hanjing Li, Mengkang Hu, Yimeng Zhang, Yihao Liang, Yuhang Zhou, Jiaqi Wang, Zhi Chen, and Wanxiang Che. Ai4research: A survey of artificial intelligence for scientific research, 2025. URLhttps://arxiv.org/abs/2507.01903
Pith/arXiv arXiv 2025
-
[39]
Yixuan Weng, Minjun Zhu, Qiujie Xie, Qiyao Sun, Zhen Lin, Sifan Liu, and Yue Zhang. Deepscientist: Advancing frontier-pushing scientific findings progressively.arXivpreprintarXiv:2509.26603, 2025
arXiv 2025
-
[40]
Innovatorbench: Evaluatingagents’abilitytoconductinnovativellmresearch
Yunze Wu, Dayuan Fu, Weiye Si, Zhen Huang, Mohan Jiang, Keyu Li, Shijie Xia, Jie Sun, Tianze Xu, Xiangkun Hu, etal. Innovatorbench: Evaluatingagents’abilitytoconductinnovativellmresearch. arXivpreprintarXiv:2510.27598, 2025
arXiv 2025
-
[41]
Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers, 2024
Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can LLMs generate novel research ideas? a large-scale human study with 100+ NLP researchers, 2024
2024
-
[42]
Agentreview: Exploring peer review dynamics with llm agents
YiqiaoJin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. Agentreview: Exploring peer review dynamics with llm agents. InProceedings of the 2024 Conference on Empirical Methods in Natural LanguageProcessing, pages 1208–1226, 2024
2024
-
[43]
Marg: Multi-agent review generation for scientific papers
Mike D’Arcy, Tom Hope, Larry Birnbaum, and Doug Downey. Marg: Multi-agent review generation for scientific papers. arXivpreprintarXiv:2401.04259, 2024
Pith/arXiv arXiv 2024
-
[44]
aixiv: A next-generation open access ecosystem for scientific discovery generated by ai scientists
Pengsong Zhang, Xiang Hu, Guowei Huang, Yang Qi, Heng Zhang, Xiuxu Li, Jiaxing Song, Jiabin Luo, Yijiang Li, Shuo Yin, et al. aixiv: A next-generation open access ecosystem for scientific discovery generated by ai scientists. arXiv preprintarXiv:2508.15126, 2025
arXiv 2025
-
[45]
Can large language models provide useful feedback on research papers? a large-scale empirical analysis.NEJMAI, 1(8):AIoa2400196, 2024
Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Scott Smith, Yian Yin, et al. Can large language models provide useful feedback on research papers? a large-scale empirical analysis.NEJMAI, 1(8):AIoa2400196, 2024. 15
2024
-
[46]
Nitya Thakkar, Mert Yuksekgonul, Jake Silberg, Animesh Garg, Nanyun Peng, Fei Sha, Rose Yu, Carl Vondrick, and James Zou. Can llm feedback enhance review quality? a randomized study of 20k reviews at iclr 2025.arXiv preprint arXiv:2504.09737, 2025
Pith/arXiv arXiv 2025
-
[47]
Towards an ai co-scientist.arXivpreprintarXiv:2502.18864, 2025
Juraj Gottweis, Wei-Hung Weng, Alexander Daryin, Tao Tu, Anil Palepu, Petar Sirkovic, Artiom Myaskovsky, Felix Weissenberger, Keran Rong, Ryutaro Tanno, et al. Towards an ai co-scientist.arXivpreprintarXiv:2502.18864, 2025
Pith/arXiv arXiv 2025
-
[48]
Deepreview: Improving llm-based paper review with human-likedeepthinkingprocess
Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. Deepreview: Improving llm-based paper review with human-likedeepthinkingprocess. In Proceedingsofthe63rdAnnualMeetingoftheAssociationforComputational Linguistics(Volume1: Long Papers), pages 29330–29355, 2025
2025
-
[49]
Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. Cycleresearcher: Improving automated research via automated review.arXivpreprintarXiv:2411.00816, 2024
Pith/arXiv arXiv 2024
-
[50]
Opennovelty: An llm-powered agentic system for verifiable scholarly novelty assessment, 2026
Ming Zhang, Kexin Tan, Yueyuan Huang, Yujiong Shen, Chunchun Ma, Li Ju, Xinran Zhang, Yuhui Wang, Wenqing Jing, Jingyi Deng, Huayu Sha, Binze Hu, Jingqi Tong, Changhao Jiang, Yage Geng, Yuankai Ying, Yue Zhang, Zhangyue Yin, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, and Xuanjing Huang. Opennovelty: An llm-powered agentic system for verifiable scholarly n...
arXiv 2026
-
[51]
Training a helpful and harmless assistant with reinforcement learning from human feedback,
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, ...
-
[52]
DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2025
DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2025
2025
-
[53]
Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D
NathanLambert,JacobMorrison,ValentinaPyatkin,ShengyiHuang,HamishIvison,FaezeBrahman,LesterJamesV. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishirzi. Tulu 3: Push...
Pith/arXiv arXiv 2025
-
[54]
Game-rl: Synthesizing multimodal verifiable game data to boost vlms’ general reasoning, 2025
Jingqi Tong, Jixin Tang, Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, Chaoran Tao, Zhiyuan Guo, Jizhou Yu, Tianhao Cheng, Zhiheng Xi, Changhao Jiang, Zhangyue Yin, Yining Zheng, Weifeng Ge, Guanhua Chen, Tao Gui, Xipeng Qiu, Qi Zhang, and Xuanjing Huang. Game-rl: Synthesizing multimodal verifiable game data ...
arXiv 2025
-
[55]
Exploring the compositional deficiency of large language models in mathematical reasoning, 2024
Jun Zhao, Jingqi Tong, Yurong Mou, Ming Zhang, Qi Zhang, and Xuanjing Huang. Exploring the compositional deficiency of large language models in mathematical reasoning, 2024. URLhttps://arxiv.org/abs/2405. 06680
2024
-
[56]
From words to worth: Newborn article impact prediction with llm
Penghai Zhao, Qinghua Xing, Kairan Dou, Jinyu Tian, Ying Tai, Jian Yang, Ming-Ming Cheng, and Xiang Li. From words to worth: Newborn article impact prediction with llm. InProceedings ofthe AAAI Conferenceon Artificial Intelligence, volume 39, pages 1183–1191, 2025
2025
-
[57]
Naipv2: Debiased pairwise learning for efficient paper quality estimation, 2025
Penghai Zhao, Jinyu Tian, Qinghua Xing, Xin Zhang, Zheng Li, Jianjun Qian, Ming-Ming Cheng, and Xiang Li. Naipv2: Debiased pairwise learning for efficient paper quality estimation, 2025. URLhttps://arxiv.org/abs/ 2509.25179
arXiv 2025
-
[58]
Arenarl: Scaling rl for open-ended agents via tournament-based relative ranking, 2026
Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, Shihang Wang, Qiuchen Wang, Yinfeng Huang, Haonan Zhang, Rongxiang Zhu, Pengyong Wang, Ailin Ren, Xin Li, Pengjun Xie, Jiawei Liu, Ning Guo, Jingren Zhou, and Zheng-Jun Zha. Arenarl: Scaling rl for open-ended agents via tournament-based relative ranking, 2026. URL https://arxiv.org/abs/2601.06487
Pith/arXiv arXiv 2026
-
[59]
Qwen2.5 technical report, 2025
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...
Pith/arXiv arXiv 2025
-
[60]
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...
Pith/arXiv arXiv 2025
-
[61]
The llama 3 herd of models, 2024
AaronGrattafiori, AbhimanyuDubey, AbhinavJauhri, AbhinavPandey, AbhishekKadian, AhmadAl-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783
Pith/arXiv arXiv 2024
-
[62]
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URLhttps://arxiv.org/abs/2306.05685
Pith/arXiv arXiv 2023
-
[63]
Swift:a scalable lightweight infrastructure for fine-tuning, 2024
Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. Swift:a scalable lightweight infrastructure for fine-tuning, 2024. URLhttps://arxiv.org/abs/2408.05517
Pith/arXiv arXiv 2024
-
[64]
Jie Zhu, Qian Chen, Huaixia Dou, Junhui Li, Lifan Guo, Feng Chen, and Chi Zhang. Dianjin-r1: Evaluating and enhancing financial reasoning in large language models.arXivpreprintarXiv:2504.15716, 2025
Pith/arXiv arXiv 2025
-
[65]
rstar2-agent: Agentic reasoning technical report.arXivpreprintarXiv:2508.20722, 2025
Ning Shang, Yifei Liu, Yi Zhu, Li Lyna Zhang, Weijiang Xu, Xinyu Guan, Buze Zhang, Bingcheng Dong, Xudong Zhou, Bowen Zhang, et al. rstar2-agent: Agentic reasoning technical report.arXivpreprintarXiv:2508.20722, 2025
Pith/arXiv arXiv 2025
-
[66]
Tek Raj Chhetri, Yibei Chen, Puja Trivedi, Dorota Jarecka, Saif Haobsh, Patrick Ray, Lydia Ng, and Satrajit S. Ghosh. Structsense: A task-agnostic agentic framework for structured information extraction with human-in-the-loop evaluation and benchmarking, 2025. URLhttps://arxiv.org/abs/2507.03674
Pith/arXiv arXiv 2025
-
[67]
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. URL https://arxiv.org/abs/2507.06261
Pith/arXiv arXiv 2025
-
[68]
Kimi k1.5: Scaling reinforcement learning with llms, 2025
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, et al. Kimi k1.5: Scaling reinforcement learning with llms, 2025. URLhtt...
Pith/arXiv arXiv 2025
-
[69]
URL https://arxiv.org/abs/2501.13712
MarkChevallier,FilipSmola,RichardSchmoetten,andJacquesD.Fleuriot.Formallyverifiedneurosymbolictrajectory learningviatensor-basedlineartemporallogiconfinitetraces,2025. URL https://arxiv.org/abs/2501.13712
Pith/arXiv arXiv 2025
-
[70]
The logic of graph neural networks, 2022
Martin Grohe. The logic of graph neural networks, 2022. URLhttps://arxiv.org/abs/2104.14624
Pith/arXiv arXiv 2022
-
[71]
Corpus based amharic sentiment lexicon generation
Girma Neshir Alemneh, Andreas Rauber, and Solomon Atnafu. Corpus based amharic sentiment lexicon generation. In Proceedings ofthe FourthWideningNaturalLanguageProcessingWorkshop, pages 1–3, 2020
2020
-
[72]
Erratum: Orientation dynamics of asymmetric rotors using random phase wave functions [phys
Shimshon Kallush and Sharly Fleischer. Erratum: Orientation dynamics of asymmetric rotors using random phase wave functions [phys. rev. a 91, 063420 (2015)].Physical Review A, 92(4), October 2015. ISSN 1094-1622. doi: 10.1103/physreva.92.049901. URLhttp://dx.doi.org/10.1103/PhysRevA.92.049901
-
[73]
Jarrod R McClean, Jonathan Romero, Ryan Babbush, and Alán Aspuru-Guzik. The theory of variational hybrid quantum-classical algorithms.NewJournal ofPhysics, 18(2):023023, February 2016. ISSN 1367-2630. doi: 10.1088/ 1367-2630/18/2/023023. URLhttp://dx.doi.org/10.1088/1367-2630/18/2/023023. 17
-
[74]
Identifying boosted objects with n-subjettiness.Journal of High EnergyPhysics, 2011(3), March 2011
Jesse Thaler and Ken Van Tilburg. Identifying boosted objects with n-subjettiness.Journal of High EnergyPhysics, 2011(3), March 2011. ISSN 1029-8479. doi: 10.1007/jhep03(2011)015. URL http://dx.doi.org/10.1007/ JHEP03(2011)015
-
[75]
One-side forward-backward asymmetry at the lhc.PhysicalReview D, 83(1), January 2011
You-kai Wang, Bo Xiao, and Shou-hua Zhu. One-side forward-backward asymmetry at the lhc.PhysicalReview D, 83(1), January 2011. ISSN 1550-2368. doi: 10.1103/physrevd.83.015002. URLhttp://dx.doi.org/10.1103/ PhysRevD.83.015002
-
[76]
Pu-net: Point cloud upsampling network, 2018
Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Pu-net: Point cloud upsampling network, 2018. URLhttps://arxiv.org/abs/1801.06761
Pith/arXiv arXiv 2018
-
[77]
Open3d: A modern library for 3d data processing, 2018
Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3d: A modern library for 3d data processing, 2018. URL https://arxiv.org/abs/1801.09847
Pith/arXiv arXiv 2018
-
[78]
Opencil: Benchmarking out-of-distribution detection in class-incremental learning, 2024
WenjunMiao,GuansongPang,Trong-TungNguyen,RuohangFang,JinZheng,andXiaoBai. Opencil: Benchmarking out-of-distribution detection in class-incremental learning, 2024. URLhttps://arxiv.org/abs/2407.06045
Pith/arXiv arXiv 2024
-
[79]
Yolov11: An overview of the key architectural enhancements, 2024
Rahima Khanam and Muhammad Hussain. Yolov11: An overview of the key architectural enhancements, 2024. URLhttps://arxiv.org/abs/2410.17725
Pith/arXiv arXiv 2024
-
[80]
Jukebox: A generative model for music, 2020
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A generative model for music, 2020. URLhttps://arxiv.org/abs/2005.00341
Pith/arXiv arXiv 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.