REVIEW 2 major objections 167 references
Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces
T0 review · 2 major / 0 minor · reviewed 2026-06-27 · grok-4.3
Pith's one-line read Reasoning in large output spaces proceeds via broad shortlisting followed by fine-grained evaluation over the narrowed set.
desk verdict The paper frames large-output reasoning as shortlisting plus fine-grained steps, isolates those phases, and builds a distillation method that beats standard baselines across datasets. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The two-phase reasoning process of broad candidate shortlisting followed by fine-grained reasoning over the shortlist.
What would settle it
A demonstration that the proposed mechanistic distillation does not outperform standard distillation on the tested datasets would falsify the central claim.
Extended reading notes
Core claim
Reasoning is a two-phase process of broad shortlisting of candidates followed by fine-grained reasoning over the resulting set. These steps can be isolated and are complementary. This characterization supports a mechanistic distillation strategy that consistently outperforms standard distillation.
Load-bearing premise
The shortlisting and fine-grained reasoning phases can be reliably isolated from each other in a manner that directly yields a superior distillation procedure.
Editorial extensions
If this is right
- Shortlisting reduces the candidate pool from millions to a tractable size before detailed evaluation occurs.
- The two phases are complementary, so both must function for strong overall performance.
- Isolating the phases permits targeted knowledge transfer during distillation.
- The resulting distillation method improves performance consistently across multiple datasets.
Reading between the lines
- The two-phase separation may apply to other prediction tasks with very large label spaces.
- Models could potentially be trained or prompted to make the shortlisting step more explicit.
- Disrupting one phase independently might produce predictable drops in accuracy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that modern reasoning models achieve strong zero-shot performance on multi-label tasks with very large candidate sets via a two-phase process of broad shortlisting followed by fine-grained reasoning over the shortlist. It asserts that these phases can be isolated and shown to be complementary across datasets, and that a distillation procedure derived from this characterization consistently outperforms standard distillation.
Significance. If the two-phase characterization is valid and the isolation procedure is shown to be causal rather than post-hoc, the work could provide a useful mechanistic lens on how transformers manage large output spaces and yield practically better distillation recipes for such tasks. The absence of any equations, algorithms, datasets, or quantitative results in the manuscript prevents assessment of whether these benefits are realized or attributable to the proposed framing.
major comments (2)
- [Abstract] Abstract: the manuscript asserts that 'evidence across a range of datasets' exists for isolability and complementarity and that the resulting distillation 'consistently outperforms standard distillation,' yet supplies no methods section, no experimental protocol, no tables of results, and no description of baselines or controls. This renders the central empirical claims unverifiable from the text.
- [Abstract] Abstract: the claim that the phases 'can be isolated' is load-bearing for the distillation contribution, but no procedure, loss function, or intervention for performing the isolation is described, making it impossible to determine whether the reported gains are due to the mechanistic insight or to ancillary implementation choices.
Simulated Author's Rebuttal
We thank the referee for their comments. We agree that the current manuscript text consists only of a high-level abstract and lacks the detailed methods, protocols, results, and isolation procedure needed to substantiate the claims. We will revise the manuscript accordingly to address these issues.
read point-by-point responses
-
Referee: [Abstract] Abstract: the manuscript asserts that 'evidence across a range of datasets' exists for isolability and complementarity and that the resulting distillation 'consistently outperforms standard distillation,' yet supplies no methods section, no experimental protocol, no tables of results, and no description of baselines or controls. This renders the central empirical claims unverifiable from the text.
Authors: We accept this observation. The provided manuscript is limited to the abstract summarizing the claims without supporting details. In the revised version we will add a methods section, full experimental protocol, descriptions of datasets and baselines, and tables of quantitative results to make the claims verifiable. revision: yes
-
Referee: [Abstract] Abstract: the claim that the phases 'can be isolated' is load-bearing for the distillation contribution, but no procedure, loss function, or intervention for performing the isolation is described, making it impossible to determine whether the reported gains are due to the mechanistic insight or to ancillary implementation choices.
Authors: We agree that the isolation procedure must be explicitly described for the contribution to be assessable. The revised manuscript will include a detailed account of the isolation method, including any loss functions or interventions used, to clarify its role in the distillation strategy. revision: yes
Circularity Check
No significant circularity detected
full rationale
The provided abstract and description characterize reasoning as a two-phase shortlisting plus fine-grained process, claim empirical isolation across datasets, and derive a distillation strategy from that characterization. No equations, fitted parameters renamed as predictions, self-citations, or ansatzes are supplied that would allow any load-bearing step to reduce to its own inputs by construction. The derivation therefore remains self-contained and relies on external empirical evidence rather than internal redefinition or self-referential fitting.
Assumptions & free parameters
Cite this review
Pith. "Pith review of Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces." pith.science (2026). https://pith.science/paper/VYPOSO4A
@misc{pith2026260606840,
author = {Pith},
title = {Pith review of: Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces},
year = {2026},
howpublished = {\url{https://pith.science/paper/VYPOSO4A}},
note = {Machine review of arXiv:2606.06840}
}
read the original abstract
Modern reasoning models offer surprisingly strong zero-shot performance on challenging multi-label tasks that require selecting a small set of relevant options from hundreds of thousands to millions of candidate labels. We investigate how they achieve this mechanistically. We characterize reasoning as a two-phase process: A broad "shortlisting" of candidates followed by fine-grained reasoning over the resulting set. We provide evidence across a range of datasets that these steps can be isolated and are complementary. Using this characterization, we develop a mechanistic distillation strategy that consistently outperforms standard distillation.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
RIP , author=
Kurtosis as peakedness, 1905--2014. RIP , author=. The American Statistician , volume=. 2014 , publisher=
1905
-
[2]
2025 , eprint =
Eliciting Latent Predictions from Transformers with the Tuned Lens , author =. 2025 , eprint =
2025
-
[3]
2024 , journal =
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet , author =. 2024 , journal =
2024
-
[4]
arXiv preprint arXiv:2509.25002 , year=
Circuit Distillation , author=. arXiv preprint arXiv:2509.25002 , year=
-
[5]
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=
-
[6]
arXiv e-prints , pages=
Test-time computing: from system-1 thinking to system-2 thinking , author=. arXiv e-prints , pages=. 2025 , url=
2025
-
[7]
Introducing OpenAI o3 and o4-mini , year =
-
[8]
Introducing GPT-5.2 , year =
Show all 167 references
-
[9]
2023 , howpublished =
Kamradt, Greg , title =. 2023 , howpublished =
2023
-
[10]
and Dahiya, K
Bhatia, K. and Dahiya, K. and Jain, H. and Kar, P. and Mittal, A. and Prabhu, Y. and Varma, M. , title =
-
[11]
Scientific data , volume=
MIMIC-IV, a freely accessible electronic health record dataset , author=. Scientific data , volume=. 2023 , publisher=
2023
-
[12]
Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=
Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=
-
[13]
Journal of Machine learning research , volume=
Statistical comparisons of classifiers over multiple data sets , author=. Journal of Machine learning research , volume=
-
[14]
Investigating Mysteries of C o T -Augmented Distillation
Wadhwa, Somin and Amir, Silvio and Wallace, Byron C. Investigating Mysteries of C o T -Augmented Distillation. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.349
2024 doi
-
[15]
2025 , url =
WHO , title =. 2025 , url =
2025
-
[16]
Advances in Neural Information Processing Systems , volume=
Towards semi-structured automatic ICD coding via tree-based contrastive learning , author=. Advances in Neural Information Processing Systems , volume=
-
[17]
arXiv preprint arXiv:2509.20317 , year=
SIM-CoT: Supervised Implicit Chain-of-Thought , author=. arXiv preprint arXiv:2509.20317 , year=
-
[18]
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
Softcot: Soft chain-of-thought for efficient reasoning with llms , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
-
[19]
arXiv preprint arXiv:2311.01460 , year=
Implicit chain of thought reasoning via knowledge distillation , author=. arXiv preprint arXiv:2311.01460 , year=
-
[20]
arXiv e-prints , pages=
Cothink: Token-efficient reasoning via instruct models guiding reasoning models , author=. arXiv e-prints , pages=
-
[21]
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=
Codi: Compressing chain-of-thought into continuous space via self-distillation , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=
2025
-
[22]
arXiv preprint arXiv:2412.13171 , year=
Compressed chain of thought: Efficient reasoning through dense representations , author=. arXiv preprint arXiv:2412.13171 , year=
-
[23]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
C3ot: Generating shorter chain-of-thought without compromising effectiveness , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[24]
arXiv preprint arXiv:2405.16064 , year=
Keypoint-based progressive chain-of-thought distillation for llms , author=. arXiv preprint arXiv:2405.16064 , year=
-
[25]
Advances in Neural Information Processing Systems , volume=
Iteration head: A mechanistic study of chain-of-thought , author=. Advances in Neural Information Processing Systems , volume=
-
[27]
doi:10.48550/ARXIV.2510.24940 , abstract =
He, Yinhan and Zheng, Wendy and Zhu, Yaochen and Zheng, Zaiyi and Su, Lin and Vasudevan, Sriram and Guo, Qi and Hong, Liangjie and Li, Jundong , year =. doi:10.48550/ARXIV.2510.24940 , abstract =
-
[28]
Chen, Xiaoshu and Zhou, Sihang and Liang, Ke and Sun, Xiaoyu and Liu, Xinwang , editor =. Skip-. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.emnlp-main.610 , abstract =
2025 doi
-
[29]
Yan, JianZhi and Liu, Le and Pan, Youcheng and Chen, Shiwei and Xiang, Yang and Tang, Buzhou , editor =. Towards. Findings of the. 2025 , pages =. doi:10.18653/v1/2025.findings-emnlp.413 , abstract =
2025 doi
-
[30]
Zhuang, Xianwei and Zhu, Zhihong and Wang, Zhichang and Cheng, Xuxin and Zou, Yuexian , year =
-
[31]
arXiv.org , author =
Probing to. arXiv.org , author =
-
[32]
and Aslam, Javed A
Roy, Debjyoti Saha and Wallace, Byron C. and Aslam, Javed A. , month = dec, year =. Don't. doi:10.48550/arXiv.2410.23066 , abstract =
- [33]
-
[34]
Kim, Jaehoon and Seo, Kwangwook and Lee, Dongha , month = sep, year =. In. doi:10.48550/arXiv.2509.22230 , abstract =
- [35]
-
[36]
Ramesh, Suhas Kamasetty and Sengupta, Ayan and Chakraborty, Tanmoy , month = aug, year =. On the. doi:10.48550/arXiv.2505.15442 , abstract =
-
[37]
Knowledge
Fang, Luyang and Yu, Xiaowei and Cai, Jiazhang and Chen, Yongkai and Wu, Shushan and Liu, Zhengliang and Yang, Zhenyuan and Lu, Haoran and Gong, Xilin and Liu, Yufang and Ma, Terry and Ruan, Wei and Abbasi, Ali and Zhang, Jing and Wang, Tao and Latif, Ehsan and You, Weihang an...
-
[38]
Belinkov, Yonatan , month = mar, year =. Probing. Computational Linguistics , publisher =. doi:10.1162/coli_a_00422 , abstract =
-
[39]
Mixture of
Fu, Tianyu and Huang, Haofeng and Ning, Xuefei and Zhang, Genghan and Chen, Boju and Wu, Tianqi and Wang, Hongyi and Huang, Zixiao and Li, Shiyao and Yan, Shengen and Dai, Guohao and Yang, Huazhong and Wang, Yu , month = nov, year =. Mixture of. doi:10.48550/arXiv.2406.14909 ,...
-
[40]
doi:10.48550/arXiv.2407.15891 , abstract =
Tang, Hanlin and Lin, Yang and Lin, Jing and Han, Qingsen and Hong, Shikuan and Yao, Yiwu and Wang, Gongyi , month = jul, year =. doi:10.48550/arXiv.2407.15891 , abstract =
-
[41]
Attention
Zheng, Zifan and Wang, Yezhaohui and Huang, Yuxin and Song, Shichao and Yang, Mingchuan and Tang, Bo and Xiong, Feiyu and Li, Zhiyu , month = dec, year =. Attention. doi:10.48550/arXiv.2409.03752 , abstract =
-
[42]
Retrieval
Wu, Wenhao and Wang, Yizhong and Xiao, Guangxuan and Peng, Hao and Fu, Yao , month = apr, year =. Retrieval. doi:10.48550/arXiv.2404.15574 , abstract =
-
[43]
arXiv.org , author =
A. arXiv.org , author =
-
[44]
arXiv.org , author =
Eliciting. arXiv.org , author =
-
[45]
Attribution
Syed, Aaquib and Rager, Can and Conmy, Arthur , editor =. Attribution. Proceedings of the 7th. 2024 , pages =. doi:10.18653/v1/2024.blackboxnlp-1.25 , abstract =
2024 doi
-
[46]
arXiv.org , author =
-
[47]
Iteration
Cabannes, Vivien and Arnal, Charles and Bouaziz, Wassim and Yang, Alice and Charton, Francois and Kempe, Julia , month = oct, year =. Iteration. doi:10.48550/arXiv.2406.02128 , abstract =
- [48]
- [49]
-
[50]
Shu, Dong and Wu, Xuansheng and Zhao, Haiyan and Rai, Daking and Yao, Ziyu and Liu, Ninghao and Du, Mengnan , month = sep, year =. A. doi:10.48550/arXiv.2503.05613 , abstract =
-
[51]
and Tutubalina, Elena and Oseledets, Ivan , month = aug, year =
Galichin, Andrey and Dontsov, Alexey and Druzhinina, Polina and Razzhigaev, Anton and Rogov, Oleg Y. and Tutubalina, Elena and Oseledets, Ivan , month = aug, year =. I. doi:10.48550/arXiv.2503.18878 , abstract =
-
[52]
Distill , author =
Multimodal. Distill , author =. 2021 , pages =. doi:10.23915/distill.00030 , number =
2021 doi
-
[53]
Distill , author =
High/. Distill , author =. 2021 , pages =. doi:10.23915/distill.00024.005 , number =
2021 doi
-
[54]
Distill , author =
Feature. Distill , author =. 2017 , pages =. doi:10.23915/distill.00007 , number =
2017 doi
-
[55]
Distill , author =
Zoom. Distill , author =. 2020 , pages =. doi:10.23915/distill.00024.001 , number =
2020 doi
-
[56]
Scaling and evaluating sparse autoencoders , url =
Gao, Leo and Tour, Tom Dupré la and Tillman, Henk and Goh, Gabriel and Troll, Rajan and Radford, Alec and Sutskever, Ilya and Leike, Jan and Wu, Jeffrey , month = jun, year =. Scaling and evaluating sparse autoencoders , url =. doi:10.48550/arXiv.2406.04093 , abstract =
- [57]
- [58]
- [59]
-
[60]
Nanda, Neel , month = jul, year =. An
-
[61]
Stolfo, Alessandro and Belinkov, Yonatan and Sachan, Mrinmaya , month = oct, year =. A. doi:10.48550/arXiv.2305.15054 , abstract =
- [62]
-
[63]
Ren, Zirui and Liu, Ziming , month = jan, year =. Are. doi:10.48550/arXiv.2601.10679 , abstract =
- [64]
-
[65]
and Rossi, Ryan A
Basu, Samyadeep and Morariu, Vlad I. and Rossi, Ryan A. and Zhao, Nanxuan and Wang, Zichao and Feizi, Soheil and Manjunatha, Varun , month = aug, year =. On
-
[66]
Du, Hongzhe and Li, Weikai and Cai, Min and Saraipour, Karim and Zhang, Zimin and Lakkaraju, Himabindu and Sun, Yizhou and Zhang, Shichang , month = nov, year =. How. doi:10.48550/arXiv.2504.02904 , abstract =
-
[67]
Hanna, Michael and Pezzelle, Sandro and Belinkov, Yonatan , month = jul, year =. Have. doi:10.48550/arXiv.2403.17806 , abstract =
-
[68]
Yan, Jianzhi and Liu, Le and Pan, Youcheng and Chen, Shiwei and Xiang, Yang and Tang, Buzhou , month = sep, year =. Towards. doi:10.48550/arXiv.2509.23574 , abstract =
-
[69]
Distilling the
Chen, Wei-Rui and Kothapalli, Vignesh and Fatahibaarzi, Ata and Sang, Hejian and Tang, Shao and Song, Qingquan and Wang, Zhipeng and Abdul-Mageed, Muhammad , month = jan, year =. Distilling the. doi:10.48550/arXiv.2512.21002 , abstract =
-
[70]
, month = nov, year =
Tian, Yijun and Han, Yikun and Chen, Xiusi and Wang, Wei and Chawla, Nitesh V. , month = nov, year =. Beyond. doi:10.48550/arXiv.2402.04616 , abstract =
-
[71]
Dai, Chengwei and Li, Kun and Zhou, Wei and Hu, Songlin , month = may, year =. Beyond. doi:10.48550/arXiv.2405.19737 , abstract =
- [72]
-
[73]
Li, Chenglin and Chen, Qianglong and Li, Liangyue and Wang, Caiyu and Li, Yicheng and Chen, Zulong and Zhang, Yin , month = feb, year =. Mixed. doi:10.48550/arXiv.2312.10730 , abstract =
-
[74]
doi:10.48550/arXiv.2310.14747 , abstract =
Chen, Hongzhan and Wu, Siyue and Quan, Xiaojun and Wang, Rui and Yan, Ming and Zhang, Ji , month = dec, year =. doi:10.48550/arXiv.2310.14747 , abstract =
-
[75]
Chen, Qiguang and Du, Yantao and Li, Ziniu and Liu, Jinhao and Duan, Songyao and Guo, Jiarui and Liu, Minghao and Liu, Jiaheng and Yang, Tong and Zhang, Ge and Qin, Libo and Che, Wanxiang and Huang, Wenhao , month = jan, year =. The. doi:10.48550/arXiv.2601.06002 , abstract =
-
[76]
and Zaharia, Matei and Gonzalez, Joseph E
Li, Dacheng and Cao, Shiyi and Griggs, Tyler and Liu, Shu and Mo, Xiangxi and Tang, Eric and Hegde, Sumanth and Hakhamaneshi, Kourosh and Patil, Shishir G. and Zaharia, Matei and Gonzalez, Joseph E. and Stoica, Ion , month = feb, year =. doi:10.48550/arXiv.2502.07374 , abstract =
- [77]
-
[78]
Keypoint-based
Feng, Kaituo and Li, Changsheng and Zhang, Xiaolu and Zhou, Jun and Yuan, Ye and Wang, Guoren , month = may, year =. Keypoint-based. doi:10.48550/arXiv.2405.16064 , abstract =
-
[79]
Chen, Xiao and Zhou, Sihang and Liang, Ke and Sun, Xiaoyu and Liu, Xinwang , month = may, year =. Skip-. doi:10.48550/arXiv.2505.18642 , abstract =
-
[80]
Michaud, Eric J. and Liao, Isaac and Lad, Vedang and Liu, Ziming and Mudide, Anish and Loughridge, Chloe and Guo, Zifan Carl and Kheirkhah, Tara Rezaei and Vukelić, Mateja and Tegmark, Max , month = feb, year =. Opening the. doi:10.48550/arXiv.2402.05110 , abstract =
-
[81]
doi:10.48550/arXiv.2402.04678 , abstract =
Chuang, Yu-Neng and Wang, Guanchu and Chang, Chia-Yuan and Tang, Ruixiang and Zhong, Shaochen and Yang, Fan and Du, Mengnan and Cai, Xuanting and Braverman, Vladimir and Hu, Xia , month = oct, year =. doi:10.48550/arXiv.2402.04678 , abstract =
-
[82]
doi:10.48550/arXiv.2409.10504 , abstract =
Wu, John and Wu, David and Sun, Jimeng , month = mar, year =. doi:10.48550/arXiv.2409.10504 , abstract =
-
[83]
Lin, Zihao and Basu, Samyadeep and Beigi, Mohammad and Manjunatha, Varun and Rossi, Ryan A. and Wang, Zichao and Zhou, Yufan and Balasubramanian, Sriram and Zarei, Arman and Rezaei, Keivan and Shen, Ying and Yao, Barry Menglong and Xu, Zhiyang and Liu, Qin and Zhang, Yuxiang a...
-
[84]
and Perez, Ethan , month = jul, year =
Lanham, Tamera and Chen, Anna and Radhakrishnan, Ansh and Steiner, Benoit and Denison, Carson and Hernandez, Danny and Li, Dustin and Durmus, Esin and Hubinger, Evan and Kernion, Jackson and Lukošiūtė, Kamilė and Nguyen, Karina and Cheng, Newton and Joseph, Nicholas and Schief...
-
[85]
Progress measures for grokking via mechanistic interpretability , url =
Nanda, Neel and Chan, Lawrence and Lieberum, Tom and Smith, Jess and Steinhardt, Jacob , month = oct, year =. Progress measures for grokking via mechanistic interpretability , url =. doi:10.48550/arXiv.2301.05217 , abstract =
-
[86]
Geiger, Atticus and Ibeling, Duligur and Zur, Amir and Chaudhary, Maheep and Chauhan, Sonakshi and Huang, Jing and Arora, Aryaman and Wu, Zhengxuan and Goodman, Noah and Potts, Christopher and Icard, Thomas , month = may, year =. Causal. doi:10.48550/arXiv.2301.04709 , abstract =
- [87]
-
[88]
How does
Chen, Xi and Plaat, Aske and Stein, Niki van , month = jul, year =. How does. doi:10.48550/arXiv.2507.22928 , abstract =
-
[89]
How to think step-by-step:
Dutta, Subhabrata and Singh, Joykirat and Chakrabarti, Soumen and Chakraborty, Tanmoy , month = may, year =. How to think step-by-step:. doi:10.48550/arXiv.2402.18312 , abstract =
-
[90]
Consistent algorithms for multi-label classification with macro-at-\ k\ metrics , url =
Schultheis, Erik and Kotłowski, Wojciech and Wydmuch, Marek and Babbar, Rohit and Borman, Strom and Dembczyński, Krzysztof , month = jun, year =. Consistent algorithms for multi-label classification with macro-at-\ k\ metrics , url =. doi:10.48550/arXiv.2401.16594 , abstract =
-
[91]
Consistent algorithms for multi-label classification with macro-at-k metrics , abstract =
Schultheis, Erik and Wydmuch, Marek and Kotłowski, Wojciech and Babbar, Rohit and Borman, Strom and Dembczyński, Krzysztof , year =. Consistent algorithms for multi-label classification with macro-at-k metrics , abstract =
-
[92]
ACM Conferences , doi =
Gandalf:. ACM Conferences , doi =. doi:10.1145/3637528.3672063 , language =
-
[93]
Zhang, Jinbin and Ullah, Nasib and Babbar, Rohit , editor =. Large. Proceedings of the 31st. 2025 , pages =
2025
-
[94]
Ullah, Nasib and Schultheis, Erik and Zhang, Jinbin and Babbar, Rohit , month = jul, year =. How. Proceedings of the 31st. doi:10.1145/3690624.3709333 , abstract =
-
[95]
doi:10.48550/arXiv.2405.03714 , abstract =
Kharbanda, Siddhant and Gupta, Devaansh and K, Gururaj and Malhotra, Pankaj and Singh, Amit and Hsieh, Cho-Jui and Babbar, Rohit , month = mar, year =. doi:10.48550/arXiv.2405.03714 , abstract =
- [96]
-
[97]
and Hashimoto, Tatsunori and Vinyals, Oriol and Liang, Percy and Dean, Jeff and Fedus, William , month = oct, year =
Wei, Jason and Tay, Yi and Bommasani, Rishi and Raffel, Colin and Zoph, Barret and Borgeaud, Sebastian and Yogatama, Dani and Bosma, Maarten and Zhou, Denny and Metzler, Donald and Chi, Ed H. and Hashimoto, Tatsunori and Vinyals, Oriol and Liang, Percy and Dean, Jeff and Fedus...
-
[98]
Emergent
Berti, Leonardo and Giorgi, Flavio and Kasneci, Gjergji , month = mar, year =. Emergent. doi:10.48550/arXiv.2503.05788 , abstract =
- [99]
-
[100]
Guan, Melody Y. and Joglekar, Manas and Wallace, Eric and Jain, Saachi and Barak, Boaz and Helyar, Alec and Dias, Rachel and Vallone, Andrea and Ren, Hongyu and Wei, Jason and Chung, Hyung Won and Toyer, Sam and Heidecke, Johannes and Beutel, Alex and Glaese, Amelia , month = ...
-
[101]
doi:10.48550/arXiv.2504.07128 , abstract =
Marjanović, Sara Vera and Patel, Arkil and Adlakha, Vaibhav and Aghajohari, Milad and BehnamGhader, Parishad and Bhatia, Mehar and Khandelwal, Aditi and Kraft, Austin and Krojer, Benno and Lù, Xing Han and Meade, Nicholas and Shin, Dongchan and Kazemnejad, Amirhossein and Kama...
-
[102]
doi:10.48550/arXiv.2406.10149 , abstract =
Kuratov, Yuri and Bulatov, Aydar and Anokhin, Petr and Rodkin, Ivan and Sorokin, Dmitry and Sorokin, Artyom and Burtsev, Mikhail , month = nov, year =. doi:10.48550/arXiv.2406.10149 , abstract =
-
[103]
Reasoning on
Wang, Yidong , month = apr, year =. Reasoning on. doi:10.48550/arXiv.2504.04150 , abstract =
-
[104]
Prototypical
Dahiya, Kunal and Ortego, Diego and Jimenez-Cabello, David , editor =. Prototypical. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.naacl-long.537 , abstract =
2025 doi
-
[105]
and Bhojanapalli, Srinadh and Jain, Prateek and Dhillon, Inderjit , month = mar, year =
Gupta, Nilesh and Khatri, Devvrit and Rawat, Ankit S. and Bhojanapalli, Srinadh and Jain, Prateek and Dhillon, Inderjit , month = mar, year =. Dual-. doi:10.48550/arXiv.2310.10636 , abstract =
-
[106]
Findings of the
Zhu, Yaxin and Zamani, Hamed , editor =. Findings of the. 2024 , pages =. doi:10.18653/v1/2024.findings-naacl.134 , abstract =
2024 doi
-
[107]
Tabatabaei, Seyed Amin and Fancher, Sarah and Parsons, Michael and Askari, Arian , editor =. Can. Proceedings of the 31st. 2025 , pages =
2025
-
[108]
Tabatabaei, Seyed Amin and Fancher, Sarah and Parsons, Michael and Askari, Arian , month = dec, year =. Can. doi:10.48550/arXiv.2412.05137 , abstract =
-
[109]
Prototypical
Dahiya, Kunal and Ortego, Diego and Jiménez, David , month = oct, year =. Prototypical. doi:10.48550/arXiv.2410.20401 , abstract =
-
[110]
doi:10.48550/arXiv.2510.11168 , abstract =
Zhang, Jinbin and Ullah, Nasib and Schultheis, Erik and Babbar, Rohit , month = oct, year =. doi:10.48550/arXiv.2510.11168 , abstract =
-
[111]
doi:10.48550/arXiv.2502.20238 , abstract =
Chen, Guizhen and Xu, Weiwen and Zhang, Hao and Chan, Hou Pong and Liu, Chaoqun and Bing, Lidong and Zhao, Deli and Luu, Anh Tuan and Rong, Yu , month = jun, year =. doi:10.48550/arXiv.2502.20238 , abstract =
-
[112]
Wang, Zhenhailong and Xu, Haiyang and Wang, Junyang and Zhang, Xi and Yan, Ming and Zhang, Ji and Huang, Fei and Ji, Heng , month = jan, year =. Mobile-. doi:10.48550/arXiv.2501.11733 , abstract =
- [113]
- [114]
-
[115]
Benchmarking
Chen, Hanjie and Fang, Zhouxiang and Singla, Yash and Dredze, Mark , editor =. Benchmarking. Proceedings of the 2025. 2025 , pages =. doi:10.18653/v1/2025.naacl-long.182 , abstract =
2025 doi
-
[116]
doi:10.48550/arXiv.2502.16614 , abstract =
Zhang, Alexander and Dong, Marcus and Liu, Jiaheng and Zhang, Wei and Wang, Yejie and Yang, Jian and Zhang, Ge and Liu, Tianyu and Peng, Zhongyuan and Tan, Yingshui and Zhang, Yuanxing and Wang, Zhexu and Wang, Weixun and He, Yancheng and Deng, Ken and Zhou, Wangchunshu and Hu...
- [117]
-
[118]
doi:10.48550/arXiv.2406.19314 , abstract =
White, Colin and Dooley, Samuel and Roberts, Manley and Pal, Arka and Feuer, Ben and Jain, Siddhartha and Shwartz-Ziv, Ravid and Jain, Neel and Saifullah, Khalid and Dey, Sreemanti and Shubh-Agrawal and Sandha, Sandeep Singh and Naidu, Siddartha and Hegde, Chinmay and LeCun, Y...
-
[119]
doi:10.48550/arXiv.2502.06453 , abstract =
Huang, Kaixuan and Guo, Jiacheng and Li, Zihao and Ji, Xiang and Ge, Jiawei and Li, Wenzhe and Guo, Yingqing and Cai, Tianle and Yuan, Hui and Wang, Runzhe and Wu, Yue and Yin, Ming and Tang, Shange and Huang, Yangsibo and Jin, Chi and Chen, Xinyun and Zhang, Chiyuan and Wang,...
-
[120]
, month = aug, year =
Rein, David and Hou, Betty Li and Stickland, Asa Cooper and Petty, Jackson and Pang, Richard Yuanzhe and Dirani, Julien and Michael, Julian and Bowman, Samuel R. , month = aug, year =
-
[121]
doi:10.48550/arXiv.2404.01230 , abstract =
Zhang, Yadong and Mao, Shaoguang and Ge, Tao and Wang, Xun and Wynter, Adrian de and Xia, Yan and Wu, Wenshan and Song, Ting and Lan, Man and Wei, Furu , month = apr, year =. doi:10.48550/arXiv.2404.01230 , abstract =
-
[122]
Reasoning
Besta, Maciej and Barth, Julia and Schreiber, Eric and Kubicek, Ales and Catarino, Afonso and Gerstenberger, Robert and Nyczyk, Piotr and Iff, Patrick and Li, Yueling and Houliston, Sam and Sternal, Tomasz and Copik, Marcin and Kwaśniewski, Grzegorz and Müller, Jürgen and Flis...
-
[123]
Automatic
Zhang, Zhuosheng and Zhang, Aston and Li, Mu and Smola, Alex , month = feb, year =. Automatic. The
-
[124]
Zhang, Yifan and Yuan, Yang and Yao, Andrew Chi-Chih , month = dec, year =. Meta. doi:10.48550/arXiv.2311.11482 , abstract =
- [125]
-
[126]
Shao, Hao and Qian, Shengju and Xiao, Han and Song, Guanglu and Zong, Zhuofan and Wang, Letian and Liu, Yu and Li, Hongsheng , month = nov, year =. Visual. doi:10.48550/arXiv.2403.16999 , abstract =
-
[127]
Shaikh, Omar and Zhang, Hongxin and Held, William and Bernstein, Michael and Yang, Diyi , editor =. On. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.244 , abstract =
2023 doi
-
[128]
Wang, Boshi and Min, Sewon and Deng, Xiang and Shen, Jiaming and Wu, You and Zettlemoyer, Luke and Sun, Huan , editor =. Towards. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.153 , abstract =
2023 doi
-
[129]
Reasoning with
Qiao, Shuofei and Ou, Yixin and Zhang, Ningyu and Chen, Xiang and Yao, Yunzhi and Deng, Shumin and Tan, Chuanqi and Huang, Fei and Chen, Huajun , editor =. Reasoning with. Proceedings of the 61st. 2023 , pages =. doi:10.18653/v1/2023.acl-long.294 , abstract =
2023 doi
-
[130]
Huang, Jie and Chang, Kevin Chen-Chuan , editor =. Towards. Findings of the. 2023 , pages =. doi:10.18653/v1/2023.findings-acl.67 , abstract =
2023 doi
-
[131]
OpenAI , month = jan, year =
-
[132]
Learning to reason with
OpenAI , month = sep, year =. Learning to reason with
-
[133]
2025 , note =
Nature , author =. 2025 , note =. doi:10.1038/s41586-025-09422-z , abstract =
2025 doi
- [134]
- [135]
-
[136]
Proceedings of the AAAI Conference on Artificial Intelligence , author =
Hierarchical. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2025 , pages =. doi:10.1609/aaai.v39i24.34803 , abstract =
2025 doi
-
[137]
and Kumar, Sricharan and Zhang, Jiaxin , editor =
Cui, Wendi and Li, Zhuohang and Lopez, Damien and Das, Kamalika and Malin, Bradley A. and Kumar, Sricharan and Zhang, Jiaxin , editor =. Divide-. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.emnlp-industry.25 , abstract =
2024 doi
-
[138]
, month = nov, year =
Ortego, Diego and Rodríguez, Marlon and Almagro, Mario and Dahiya, Kunal and Jiménez, David and SanMiguel, Juan C. , month = nov, year =. Large. doi:10.48550/arXiv.2511.13189 , abstract =
-
[139]
Langner, Mikołaj and Eliasz, Jan and Rudnicka, Ewa and Kocoń, Jan , month = nov, year =. Divide,. doi:10.48550/arXiv.2511.03830 , abstract =
-
[140]
Vandemoortele, Nathan and Steenwinckel, Bram and Ongenae, Femke and Hoecke, Sofie Van , month = nov, year =. From. doi:10.48550/arXiv.2502.08436 , abstract =
-
[141]
Behera, Adarsh Prasad and Champati, Jaya Prakash and Morabito, Roberto and Tarkoma, Sasu and Gross, James , month = jun, year =. Towards. doi:10.48550/arXiv.2506.06579 , abstract =
-
[142]
Entropy-guided sequence weighting for efficient exploration in
Vanlioglu, Abdullah , month = mar, year =. Entropy-guided sequence weighting for efficient exploration in. doi:10.48550/arXiv.2503.22456 , abstract =
- [143]
-
[144]
Divide and
Mao, Yanxu and Liu, Peipei and Cui, Tiehan and Yan, Zhaoteng and Liu, Congying and You, Datao , month = may, year =. Divide and. doi:10.48550/arXiv.2412.16555 , abstract =
-
[145]
Xu, Zhen and Zhu, Shang and Wang, Jue and Wang, Junlin and Athiwaratkun, Ben and Wang, Chi and Zou, James and Zhang, Ce , month = jun, year =. When. doi:10.48550/arXiv.2506.16411 , abstract =
-
[146]
Enhancing
Xiong, Zhen and Cai, Yujun and Hooi, Bryan and Peng, Nanyun and Li, Zhecheng and Wang, Yiwei , month = mar, year =. Enhancing. doi:10.48550/arXiv.2502.08180 , abstract =
-
[147]
Zhang, Yizhou and Du, Lun and Cao, Defu and Fu, Qiang and Liu, Yan , month = jul, year =. An. doi:10.48550/arXiv.2402.05359 , abstract =
-
[148]
Expert Systems with Applications , author =
An entropy-driven method for llm dataset evaluation and optimization , volume =. Expert Systems with Applications , author =. 2026 , keywords =. doi:10.1016/j.eswa.2025.130293 , abstract =
2026 doi
- [149]
-
[150]
D'Oosterlinck, Karel and Khattab, Omar and Remy, François and Demeester, Thomas and Develder, Chris and Potts, Christopher , month = jan, year =. In-. doi:10.48550/arXiv.2401.12178 , abstract =
-
[151]
Findings of the
Zhou, Chuang and Dong, Junnan and Huang, Xiao and Liu, Zirui and Zhou, Kaixiong and Xu, Zhaozhuo , editor =. Findings of the. 2024 , pages =. doi:10.18653/v1/2024.findings-emnlp.226 , abstract =
2024 doi
-
[152]
Amazon Science , author =
Using generative. Amazon Science , author =
-
[153]
Ma, Marcus and Chochlakis, Georgios and Pandiyan, Niyantha Maruthu and Thomason, Jesse and Narayanan, Shrikanth , year =. Large. Proceedings of the 2025. doi:10.18653/v1/2025.emnlp-main.126 , abstract =
2025 doi
-
[154]
and Yoon, Seunghyun and Schütze, Hinrich , month = jul, year =
Modarressi, Ali and Deilamsalehy, Hanieh and Dernoncourt, Franck and Bui, Trung and Rossi, Ryan A. and Yoon, Seunghyun and Schütze, Hinrich , month = jul, year =. doi:10.48550/arXiv.2502.05167 , abstract =
-
[155]
and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy , month = nov, year =
Liu, Nelson F. and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy , month = nov, year =. Lost in the. doi:10.48550/arXiv.2307.03172 , abstract =
- [156]
-
[157]
Distilling
Shridhar, Kumar and Stolfo, Alessandro and Sachan, Mrinmaya , month = may, year =. Distilling. doi:10.48550/arXiv.2212.00193 , abstract =
-
[158]
doi:10.48550/arXiv.2305.01879 , abstract =
Wang, Peifeng and Wang, Zhengyang and Li, Zheng and Gao, Yifan and Yin, Bing and Ren, Xiang , month = aug, year =. doi:10.48550/arXiv.2305.01879 , abstract =
-
[159]
Unveiling the
Chen, Xinghao and Sun, Zhijing and Guo, Wenjin and Zhang, Miaoran and Chen, Yanjun and Sun, Yirong and Su, Hui and Pan, Yijie and Klakow, Dietrich and Li, Wenjie and Shen, Xiaoyu , month = may, year =. Unveiling the. doi:10.48550/arXiv.2502.18001 , abstract =
-
[160]
Speculative
Wang, Zilong and Wang, Zifeng and Le, Long and Zheng, Huaixiu Steven and Mishra, Swaroop and Perot, Vincent and Zhang, Yuwei and Mattapalli, Anush and Taly, Ankur and Shang, Jingbo and Lee, Chen-Yu and Pfister, Tomas , month = feb, year =. Speculative. doi:10.48550/arXiv.2407....
- [161]
-
[162]
Lyu, Yuanjie and Wang, Chengyu and Huang, Jun and Xu, Tong , month = oct, year =. From. doi:10.48550/arXiv.2509.14257 , abstract =
-
[163]
Recall with
Ma, Junyu and Fang, Tianqing and Zhang, Zhisong and Zhang, Hongming and Mi, Haitao and Yu, Dong , month = jun, year =. Recall with. doi:10.48550/arXiv.2505.03320 , abstract =
-
[164]
Distilling
Kang, Minki and Jeong, Jongwon and Lee, Seanie and Cho, Jaewoong and Hwang, Sung Ju , month = nov, year =. Distilling. doi:10.48550/arXiv.2505.17612 , abstract =
- [165]
-
[166]
Reasoning with
Qiao, Shuofei and Ou, Yixin and Zhang, Ningyu and Chen, Xiang and Yao, Yunzhi and Deng, Shumin and Tan, Chuanqi and Huang, Fei and Chen, Huajun , month = sep, year =. Reasoning with. doi:10.48550/arXiv.2212.09597 , abstract =
-
[167]
Navigate through
Chu, Zheng and Chen, Jingchang and Chen, Qianglong and Yu, Weijiang and He, Tao and Wang, Haotian and Peng, Weihua and Liu, Ming and Qin, Bing and Liu, Ting , month = jun, year =. Navigate through. doi:10.48550/arXiv.2309.15402 , abstract =
- [168]
Reviewed June 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.