Pith. sign in

REVIEW 3 major objections 4 minor 42 references

Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that instruction-data selection should be configured dynamically from a user's natural-language intent and the candidate pool, and it reports that its four-stage agent pipeline beats static baselines in 16 of 18…

desk verdict A useful, clearly-written agentic data-selection tool whose headline win counts are undercut by a benchmark-influenced LR choice and single-seed reporting. read the letter →

arxiv 2608.10579 v1 pith:UUHESACC submitted 2026-08-11 cs.AI

Fanqi Zhou , Qiaosheng Chen , Zixian Huang , Gong Cheng This is my paper · ORCID
classification cs.AI
keywords instructiondataselectionagenticpipelinelargelanguagemodelssupervisedfine-tuningnatural-languageintentcurationLLM-as-judgeinformativenessmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DataMaster is an agentic system that turns a natural-language instruction into a full instruction-data selection strategy. Its claim is that selection cannot be a static, one-size-fits-all metric because the right subset depends jointly on the user's goal and the heterogeneous composition of the pool. The system implements this with four cascaded agents that handle domain matching, characteristic filtering, informativeness scoring, and quality judging, all configured at run time from the instruction and a tool-assisted inspection of the data. In experiments on three 7B-8B models, 10K-sample subsets chosen this way beat static baselines in 16 of 18 single-domain settings and beat training on the full uncurated pool in 12 of 18, with per-task API costs below $5. If the claim holds, developers can replace manual heuristic design with plain-language curation requests.

What carries the argument

The load-bearing machinery is a four-agent cascade, each stage reading the same user instruction and the data it receives. The Domain Agent embeds and clusters the pool, asks an LLM to label each cluster, and keeps only clusters matching the instruction; the Characteristic Agent inspects the filtered pool through a sandboxed data-analysis tool and emits deterministic filtering rules such as length bounds, refusal-phrase blacklists, and noise caps; the Informativeness Agent computes four target-model metrics (negative log-likelihood, token entropy, semantic drift, and MeanDiff, the magnitude of parameter change caused by one gradient step on a sample) and weights them according to the instruction; the Quality Agent synthesizes a task-specific rubric and applies it in batched listwise LLM judging to pick the final subset. The same frozen LLM configures all stages, while the target model is used only for the Stage-3 metrics, and the whole design keeps the selection objective conditioned on the user's request at every step instead of collapsing it into one proxy.

What would settle it

Take one pool and two contrasting user instructions, run DataMaster on each, and compute the overlap of the two selected 10K subsets; if the subsets are nearly identical, or if a single fixed high-quality subset outscores both on both target benchmarks, then intent-conditioned dynamic selection is not what drives the reported gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that instruction-data selection should be configured dynamically from a user's natural-language intent and from the actual contents of the candidate pool, not by a fixed metric. It argues that the same pool should be curated differently for different goals because the optimal subset depends on the target task, the data distribution, and the target model together. The paper instantiates this as DataMaster, a four-agent cascade that composes domain matching, characteristic rule filtering, informativeness scoring against the target model, and LLM-judged quality at run time. Across six single-domain pools and two multi-domain pools, with three 7B-8B target models, the 10K subsets produced this way beat static selectors in 16 of 18 single-domain settings, beat the strongest static multi-dimensional baseline in 13 of 18, and beat full-pool training in 12 of 18 settings while using 2-41 times less data.

Load-bearing premise

The load-bearing premise is that the LLM's own quality rubrics and metric weights are trustworthy proxies for what actually improves downstream fine-tuning; if that judge has systematic taste biases, the selected subset can be skewed even though the final benchmark scores are measured independently.

Editorial extensions

If this is right

  • A developer can curate a task-specific fine-tuning set by writing a sentence in natural language, removing the need to hand-craft per-dataset heuristics.
  • A compact 10K subset selected for the stated intent can outperform training on the entire uncurated pool in most tested settings, so compute and data budgets can shrink without sacrificing benchmark scores.
  • Dynamic, agentic selection is a workable replacement for static metric-based selection in both single-domain and multi-domain pools, including target-domain extraction from mixed pools.
  • The first-stage domain filter can be reused as a preprocessing module for existing static selectors, improving them, which supports the value of separating domain selection from the rest of the pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the selected subset is defined relative to an instruction, benchmark comparisons of selection methods may need to be reported per intent; a method's 'best subset' is not a property of the pool alone.
  • A direct test of the self-judging design would swap the stage-four LLM judge for an independent model or human scores; if gains persist, intent-conditioned rubric generation, not judge identity, is what matters.
  • The emitted rule policies and metric weights could be recorded as reusable curation recipes, letting users transfer a strategy across pools or audit what the agent actually optimized.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DataMaster, an agentic instruction-data-selection system that interprets a natural-language selection instruction and composes a four-stage cascade of domain, characteristic, informativeness, and quality filters to curate a 10K subset from an instruction pool. Each stage is configured by the same frozen LLM (DeepSeek-V4-Flash), with tool-assisted pool inspection, target-model metrics, and an LLM-generated quality rubric. Across three 7B–8B target models and eight source pools, the reported experiments claim that DataMaster outperforms six static baselines in 16 of 18 single-domain settings, outperforms Claude Code in 14 of 18 settings, and beats full-pool training in 12 of 18 settings, with additional multi-domain gains.

Significance. If the empirical claims hold, this is a useful and timely contribution: it replaces per-task handcrafted selection heuristics with intent-conditioned, agentic orchestration, and it ships a transparent protocol, per-benchmark tables, prompt templates, and public code. The central idea is clearly novel relative to static selection metrics, and the low API cost (about US$4–5 per selection task) makes the system practical. The main obstacle is the fairness and stability of the headline comparison: the shared learning rate is selected using the reported benchmark scores themselves, and all results come from a single seed with no variance quantification. Because the headline claims are count-based and several margins are below one point, the comparison protocol must be repaired before the counts can be accepted. The same-LLM rubric-generation/judging coupling is a secondary internal-validity concern but is not a circular derivation, since the final numbers come from external benchmarks.

major comments (3)
  1. [Appendix I, 'Fine-tuning protocol' and 'Runs per setting'] The shared learning rate is selected per setting by running every method at all three retained rates and picking the rate with the highest setting-level average across all methods. Because DataMaster has the highest scores in most settings, it disproportionately influences the average that selects the rate, so the evaluation benchmarks serve simultaneously as the learning-rate selection criterion and as the outcome measure, with no held-out split. The paper's central claims are count-based (16/18 over static baselines, 12/18 over Full Pool, 14/18 over Claude Code), and several margins are sub-1-point (e.g., Table 2, OpenHermes-2.5 Code: 41.11 vs. 41.07). This protocol can therefore materially inflate DataMaster's win count. Please rerun the comparison with multi-seed variance and with a learning-rate choice that is independent of the reported test scores, such as a per-model fixed LR or an LR selected on a development split, and report whether the win counts survive.
  2. [Appendix I, 'Runs per setting'] The paper explicitly states that 'multi-seed variance is not quantified' and that each setting is reported on a single seed (42). Given that the headline claims are counts of wins and losses and that multiple reported margins are well under one point, the absence of variance estimates leaves the 16/18 and 12/18 claims statistically unsupported. At a minimum, report 3–5 seeds for DataMaster and the strongest baselines in the settings with small margins, together with confidence intervals or per-seed ranges, and state whether the rank order is stable across seeds.
  3. [Section 4, 'Stage 4: Quality Agent' (Eqs. 12–14); Appendix G.4] The same backbone LLM both generates the quality rubric (Eq. 13) and applies it as the judge (Eq. 14), and no analysis shows that these LLM-judge scores predict downstream fine-tuning quality. Because the final benchmark numbers are external, this is not a circular derivation, but it is an unvalidated internal proxy: if the judge has systematic taste biases, the selected subset is corrupted even though the reported accuracies remain independent. I ask for a sensitivity analysis, for example comparing against a fixed generic rubric, or a correlation of LLM-judge scores with per-sample downstream utility; if that is infeasible, the limitation should be stated explicitly and its potential impact on the quality-stage contribution discussed.
minor comments (4)
  1. [Title/Front matter] The full-text title reads 'LetDataMasterInterpret Your Intent' without spaces; this should be corrected to 'Let DataMaster Interpret Your Intent'.
  2. [Table 1 and Appendix C.1] The column layout of Table 1 is difficult to read because the column headers are not clearly separated from the row labels and the Full Pool column is interleaved with the selection methods; a clearer table design with explicit column headers would improve reproducibility.
  3. [Appendix H, Select2Reason] Select2Reason is a reimplementation because the original implementation is not public; please release this reimplementation in the public repository and note the exact version used so that the baseline is fully reproducible.
  4. [Section 2 and Appendix K] The related-work section cites agentic curation systems that optimize through downstream feedback, but no such system is included as a baseline; a direct comparison, or a clear statement of why those systems are out of scope, would strengthen the positioning of the feedback-free design.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DataMaster's effectiveness claims are supported by external benchmark evaluations, not by a self-referential derivation.

full rationale

The paper's central claims are empirical comparisons: DataMaster selects a 10K subset via a four-stage cascade (Eqs. 1-14) and the resulting fine-tuned models are evaluated on standard benchmarks in Tables 1-2. The selection scores—weighted informativeness in Stage 3 (Eqs. 9-10) and rubric-based quality in Stage 4 (Eqs. 11-14)—are generated from the user instruction and tool-assisted observations of the data pool; they are never defined in terms of the downstream benchmark accuracies. Thus no benchmark result is equivalent to a selection input by construction. The same LLM both writes and applies the Stage 4 rubric, and Appendix I selects a shared learning rate using the reported benchmark averages; these are evaluation-fairness and stability concerns rather than logical circles, and neither reduces a prediction to a fitted parameter. The only apparent self-citation is Huang et al. (2026) in the Related Work, used to note that LLMs can generate semantic domain labels for clustering; it is not load-bearing for DataMaster's design or claims. No equation equates a fitted parameter with a prediction, and no self-cited uniqueness theorem is invoked to force the design. Verdict: no significant circularity.

Assumptions & free parameters 11 free parameters · 8 assumptions · 0 invented entities

DataMaster is an empirical engineering system. It introduces no new physical or mathematical entities; its free parameters are hand-set hyperparameters and prompt-engineered instructions, and its axioms are domain assumptions about LLM judgment, clustering coherence, and evaluation protocol. The heaviest implicit assumptions are that LLM-generated rubrics and metric weights proxy downstream SFT quality and that the single-seed shared-LR protocol faithfully ranks methods.

free parameters (11)
  • selection_budget_k = 10,000
    All methods select 10K subsets; reported gains are for this budget and no sensitivity to budget is tested.
  • clustering_switch_threshold_nc = 50,000
    Stage 1 switches from HDBSCAN to KMeans above 50K samples; no sensitivity analysis.
  • kmeans_clusters_kc = 20
    Number of KMeans clusters used in Stage 1 for large pools.
  • cluster_label_samples_ns = 8
    Number of random samples per cluster shown to the LLM in the labeling prompt Plab.
  • stage3_internal_topk_kstar = not reported (k* > k)
    Intermediate keep-count after the Informativeness stage; controls how many candidates reach Stage 4 but is not disclosed.
  • stage4_batch_size_b = 10 samples per batch, 5 batches per round
    Batched LLM-judge scoring; JSON parse failures empty a batch with no retry, so this affects which samples are judged.
  • characteristic_default_thresholds = Lmin=40, Lmax=4096, Lx=5, Ly=15, rho_min=0.2, tau_n=0.15, tau_e=0.30
    Hand-set default filtering rules in Algorithm 2; the agent may override, but no sensitivity analysis is given.
  • llm_decoding_temperature = 0.01
    Near-greedy decoding for all agent configuration and judging calls.
  • shared_lr_protocol = chosen per setting from {1e-6, 5e-6, 1e-5} to maximize average benchmark score across all methods
    The learning rate is selected using the evaluation benchmarks themselves; this protocol affects all reported numbers.
  • random_seed = 42
    Single seed for sampling, training order, NumPy/PyTorch/CUDA, and KMeans; no multi-seed statistics.
  • mean_diff_update_lr_and_layer = 1e-5, last-layer up_proj only
    Single-step gradient update for MeanDiff is restricted to one layer to save compute, a deviation from ResoFilter that changes Stage 3 scores.
assumptions (8)
  • domain assumption Embedding-based clustering yields semantically coherent clusters, so labels from 8 samples per cluster represent the cluster.
    Stage 1 relies on HDBSCAN/KMeans plus LLM labeling (Eqs. 4-5); if clusters are incoherent, cluster selection is meaningless.
  • domain assumption LLM tool-assisted observations are read-only and sufficient to configure rules, weights, and rubrics.
    Stages 2-4 condition decisions on Inspect() observations (Eq. 6) without any guarantee that these statistics capture what matters.
  • domain assumption LLM-generated quality rubrics and metric weights are valid proxies for downstream SFT performance.
    Stage 3 weights (Eq. 10) and Stage 4 judge scores (Eqs. 12-14) drive selection, but no validation shows these scores predict held-out accuracy.
  • domain assumption The fixed four-metric informativeness set (NLL, entropy, drift, MeanDiff) spans the relevant selection dimensions.
    Stage 3 can only weight these four metrics; any useful signal outside them is ignored.
  • domain assumption The natural-language instructions in Tables 8 and 9 are representative of real user intents and carry enough signal.
    The whole method is evaluated on these author-written instructions; real-world instructions may be vaguer or noisier.
  • domain assumption The shared learning-rate selection and single-seed final-epoch scores give a fair comparison across methods.
    Appendix I fixes one LR per setting from three candidates using benchmark averages, and reports only seed 42; this assumes no method-specific LR sensitivity.
  • domain assumption Overriding evaluator chat templates fixes a tokenizer artifact without changing model behavior.
    Appendix G.5 changes the tokenizer chat_template to a raw prompt; the assumption is that this removes leakage rather than distorting evaluation.
  • domain assumption Near-greedy decoding of the configurator LLM is stable enough that one run represents the agent's behavior.
    All agent calls use temperature 0.01, but hosted LLM non-determinism is acknowledged and not measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent." pith.science (2026). https://pith.science/paper/UUHESACC

@misc{pith2026260810579,
  author       = {Pith},
  title        = {Pith review of: Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UUHESACC}},
  note         = {Machine review of arXiv:2608.10579}
}
read the original abstract

Although existing instruction data selection methods have introduced various metrics, the inherent complexity of real-world datasets makes it impractical for any single metric to generalize across all scenarios. Developers are thus often forced to manually inspect data and craft heuristic rules for each new application---a tedious and error-prone process. In this paper, we propose a paradigm shift from manual configuration to automated orchestration via the Instruction Data Selection Agent (DataMaster), which interprets user intent and autonomously composes optimal selection strategies. By allowing users to specify data needs through natural language descriptions, DataMaster simplifies data curation and removes the burden of manual strategy design. Extensive experiments across the math, medical, and code domains show that DataMaster outperforms static baselines in most settings and surpasses full-pool training in a substantial number of cases. The implementation of DataMaster and the scripts needed to reproduce the reported pipeline are publicly available at https://github.com/nju-websoft/DataMaster.

Figures

Figures reproduced from arXiv: 2608.10579 by the authors.

Figure 1
Figure 1. Static selection versus intent-conditioned selection. (a) Conventional methods apply fixed selection criteria independently [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Stage 1 ablation results on OpenHermes-2.5 using [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Ablation results for Stages 2 to 4 of DataMaster. data characteristics, model-relative informativeness, and qual￾ity. This decomposition is effective because these criteria play complementary roles, allowing the method to maintain instruction conditioning throughout selection without col￾lapsing the objective into a single fixed proxy. Across three target models and eight source data pools, DataMaster achieves stron… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 14 canonical work pages

  1. [1]

    From Quantity to Quality: Boosting

    Ming Li and Yong Zhang and Zhitao Li and Jiuhai Chen and Lichang Chen and Ning Cheng and Jianzong Wang and Tianyi Zhou and Jing Xiao , editor =. From Quantity to Quality: Boosting. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers),. 2024 , ...

    doi:10.18653/v1/2024.naacl-long.421 2024

  2. [2]

    Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning , booktitle =

    Ming Li and Yong Zhang and Shwai He and Zhitao Li and Hongyu Zhao and Jianzong Wang and Ning Cheng and Tianyi Zhou , editor =. Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning , booktitle =. 2024 , url =. doi:10.18653/V1/2024.ACL-LONG.769 , timestamp =

    doi:10.18653/v1/2024.acl-long.769 2024

  3. [3]

    Findings of the Association for Computational Linguistics,

    Yicheng Chen and Yining Li and Kai Hu and Zerun Ma and Haochen Ye and Kai Chen , editor =. Findings of the Association for Computational Linguistics,. 2025 , url =. doi:10.18653/V1/2025.FINDINGS-ACL.515 , timestamp =

    work page doi:10.18653/v1/2025.findings-acl.515 2025

  4. [4]

    Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning , booktitle =

    Cehao Yang and Xueyuan Lin and Xiaojun Wu and Chengjin Xu and Xuhui Jiang and Honghao Liu and Hui Xiong and Jian Guo , editor =. Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning , booktitle =. 2026 , url =

    work page 2026

  5. [5]

    Wong and Dongfang Li and Ziyi Wang and Baotian Hu and Min Zhang , editor =

    Liangxin Liu and Xuebo Liu and Derek F. Wong and Dongfang Li and Ziyi Wang and Baotian Hu and Min Zhang , editor =. SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection , booktitle =. 2024 , url =

    work page 2024

  6. [6]

    The Twelfth International Conference on Learning Representations,

    Lichang Chen and Shiyang Li and Jun Yan and Hai Wang and Kalpa Gunaratna and Vikas Yadav and Zheng Tang and Vijay Srinivasan and Tianyi Zhou and Heng Huang and Hongxia Jin , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    work page 2024

  7. [7]

    The Twelfth International Conference on Learning Representations,

    Wei Liu and Weihao Zeng and Keqing He and Yong Jiang and Junxian He , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    work page 2024

  8. [8]

    The Twelfth International Conference on Learning Representations,

    Keming Lu and Hongyi Yuan and Zheng Yuan and Runji Lin and Junyang Lin and Chuanqi Tan and Chang Zhou and Jingren Zhou , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    work page 2024

Show all 42 references
  1. [9]

    Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation , booktitle =

    Yuan Ge and Yilun Liu and Chi Hu and Weibin Meng and Shimin Tao and Xiaofeng Zhao and Mahong Xia and Zhang Li and Boxing Chen and Hao Yang and Bei Li and Tong Xiao and JingBo Zhu , editor =. Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-align...

    2024 doi

  2. [10]

    CoRR , volume =

    Qianlong Du and Chengqing Zong and Jiajun Zhang , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2311.15653 , eprinttype =. 2311.15653 , timestamp =

    2023 doi arXiv

  3. [11]

    CoRR , volume =

    Yihan Cao and Yanbin Kang and Lichao Sun , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2307.06290 , eprinttype =. 2307.06290 , timestamp =

    2023 doi arXiv

  4. [12]

    Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence,

    Jia Zhang and Chen. Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence,. 2025 , url =. doi:10.24963/IJCAI.2025/928 , timestamp =

    2025 doi

  5. [13]

    Chunting Zhou and Pengfei Liu and Puxin Xu and Srinivasan Iyer and Jiao Sun and Yuning Mao and Xuezhe Ma and Avia Efrat and Ping Yu and Lili Yu and Susan Zhang and Gargi Ghosh and Mike Lewis and Luke Zettlemoyer and Omer Levy , editor =. Advances in Neural Information Processi...

    2023

  6. [14]

    ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis , booktitle =

    Zeao Tu and Xiangdi Meng and Yu He and Zihan Yao and Tianyu Qi and Jun Liu and Ming Li , editor =. ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis , booktitle =. 2025 , url =. doi:10.18653/V1/2025.FINDINGS-N...

    2025 doi

  7. [15]

    CoRR , volume =

    Junying Chen and Zhenyang Cai and Ke Ji and Xidong Wang and Wanlong Liu and Rongsheng Wang and Jianye Hou and Benyou Wang , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2412.18925 , eprinttype =. 2412.18925 , timestamp =

    2024 doi arXiv

  8. [16]

    UltraMedical: Building Specialized Generalists in Biomedicine , booktitle =

    Kaiyan Zhang and Sihang Zeng and Ermo Hua and Ning Ding and Zhang. UltraMedical: Building Specialized Generalists in Biomedicine , booktitle =. 2024 , url =

    2024

  9. [17]

    SelfCodeAlign: Self-Alignment for Code Generation , booktitle =

    Yuxiang Wei and Federico Cassano and Jiawei Liu and Yifeng Ding and Naman Jain and Zachary Mueller and Harm de Vries and Leandro von Werra and Arjun Guha and Lingming Zhang , editor =. SelfCodeAlign: Self-Alignment for Code Generation , booktitle =. 2024 , url =

    2024

  10. [18]

    Miranda and Alisa Liu and Nouha Dziri and Shane Lyu and Yuling Gu and Saumya Malik and Victoria Graf and Jena D

    Nathan Lambert and Jacob Morrison and Valentina Pyatkin and Shengyi Huang and Hamish Ivison and Faeze Brahman and Lester James V. Miranda and Alisa Liu and Nouha Dziri and Shane Lyu and Yuling Gu and Saumya Malik and Victoria Graf and Jena D. Hwang and Jiangjiang Yang and Rona...

    2024 doi arXiv

  11. [19]

    CoRR , volume =

    An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and...

    2024 doi arXiv

  12. [20]

    CoRR , volume =

    Llama Team , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2407.21783 , eprinttype =. 2407.21783 , timestamp =

    2024 doi arXiv

  13. [21]

    Allyson Ettinger and Amanda Bertsch and Bailey Kuehl and David Graham and David Heineman and Dirk Groeneveld and Faeze Brahman and Finbarr Timbers and Hamish Ivison and Jacob Morrison and Jake Poznanski and Kyle Lo and Luca Soldaini and Matt Jordan and Mayee F. Chen and Michae...

    2025 doi arXiv

  14. [22]

    CoRR , volume =

    Karl Cobbe and Vineet Kosaraju and Mohammad Bavarian and Mark Chen and Heewoo Jun and Lukasz Kaiser and Matthias Plappert and Jerry Tworek and Jacob Hilton and Reiichiro Nakano and Christopher Hesse and John Schulman , title =. CoRR , volume =. 2021 , url =. 2110.14168 , timestamp =

    2021 arXiv

  15. [23]

    Measuring Mathematical Problem Solving With the

    Dan Hendrycks and Collin Burns and Saurav Kadavath and Akul Arora and Steven Basart and Eric Tang and Dawn Song and Jacob Steinhardt , editor =. Measuring Mathematical Problem Solving With the. Proceedings of the Neural Information Processing Systems Track on Datasets and Benc...

    2021

  16. [24]

    Ramasesh and Ambrose Slone and Cem Anil and Imanol Schlag and Theo Gutman

    Aitor Lewkowycz and Anders Andreassen and David Dohan and Ethan Dyer and Henryk Michalewski and Vinay V. Ramasesh and Ambrose Slone and Cem Anil and Imanol Schlag and Theo Gutman. Solving Quantitative Reasoning Problems with Language Models , booktitle =. 2022 , url =

    2022

  17. [25]

    Evaluating Large Language Models Trained on Code , journal =

    Mark Chen and Jerry Tworek and Heewoo Jun and Qiming Yuan and Henrique Pond. Evaluating Large Language Models Trained on Code , journal =. 2021 , url =. 2107.03374 , timestamp =

    2021 arXiv

  18. [26]

    LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code , booktitle =

    Naman Jain and King Han and Alex Gu and Wen. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code , booktitle =. 2025 , url =

    2025

  19. [27]

    BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions , booktitle =

    Terry Yue Zhuo and Minh Chien Vu and Jenny Chim and Han Hu and Wenhao Yu and Ratnadira Widyasari and Imam Nur Bani Yusuf and Haolan Zhan and Junda He and Indraneil Paul and Simon Brunner and Chen Gong and James Hoang and Armel Randy Zebaze and Xiaoheng Hong and Wen. BigCodeBen...

    2025

  20. [28]

    What Disease does this Patient Have?

    Di Jin and Eileen Pan and Nassim Oufattole and Wei. What Disease does this Patient Have?. CoRR , volume =. 2020 , url =. 2009.13081 , timestamp =

    2020 arXiv

  21. [29]

    9th International Conference on Learning Representations,

    Dan Hendrycks and Collin Burns and Steven Basart and Andy Zou and Mantas Mazeika and Dawn Song and Jacob Steinhardt , title =. 9th International Conference on Learning Representations,. 2021 , url =

    2021

  22. [30]

    Bowman , title =

    David Rein and Betty Li Hou and Asa Cooper Stickland and Jackson Petty and Richard Yuanzhe Pang and Julien Dirani and Julian Michael and Samuel R. Bowman , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2311.12022 , eprinttype =. 2311.12022 , timestamp =

    2023 doi arXiv

  23. [31]

    M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation , booktitle =

    Jianlyu Chen and Shitao Xiao and Peitian Zhang and Kun Luo and Defu Lian and Zheng Liu , editor =. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation , booktitle =. 2024 , url =. doi:10.18653/V1/2024.FINDI...

    2024 doi

  24. [32]

    Balappanawar and Weicong Chen and Shashank Kambhatla and Srinivasan Iyengar and Shivkumar Kalyanaraman and Ponnurangam Kumaraguru and Vipin Chaudhary , title =

    Debargha Ganguly and Sumit Kumar and Ishwar B. Balappanawar and Weicong Chen and Shashank Kambhatla and Srinivasan Iyengar and Shivkumar Kalyanaraman and Ponnurangam Kumaraguru and Vipin Chaudhary , title =. 2025 , url =. doi:10.1109/BIGDATA66926.2025.11401579 , timestamp =

    2025

  25. [33]

    CoRR , volume =

    Qizheng Li and Yifei Zhang and Xiao Yang and Xu Yang and Zhuo Wang and Weiqing Liu and Jiang Bian , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2603.01712 , eprinttype =. 2603.01712 , timestamp =

    2026 doi arXiv

  26. [34]

    CoRR , volume =

    Yujie Luo and Xiangyuan Ru and Jingsheng Zheng and Jingjing Wang and Yuqi Zhu and Jintian Zhang and Runnan Fang and Kewei Xu and Ye Liu and Zheng Wei and Jiang Bian and Zang Li and Shumin Deng , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2605.30407 , eprinttype...

    2026 doi arXiv

  27. [35]

    Ma and Frederic Sala and Dawn Song and Ruoxi Jia , title =

    Feiyang Kang and Hanze Li and Adam Nguyen and Mahavir Dabas and Jiaqi W. Ma and Frederic Sala and Dawn Song and Ruoxi Jia , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.04261 , eprinttype =. 2606.04261 , timestamp =

    2026 doi arXiv

  28. [36]

    2026 , publisher =

    Shaojie Dai and Xin Liu and Yue Yu , title =. 2026 , publisher =. doi:10.1109/TASLPRO.2025.3642562 , url =

    2026

  29. [37]

    Take the essence and discard the dross:

    Ziche Liu and Rui Ke and Yajiao Liu and Feng Jiang and Haizhou Li , editor =. Take the essence and discard the dross:. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,. 2025...

    2025 doi

  30. [38]

    Data Selection for Language Models via Importance Resampling , booktitle =

    Sang Michael Xie and Shibani Santurkar and Tengyu Ma and Percy Liang , editor =. Data Selection for Language Models via Importance Resampling , booktitle =. 2023 , url =

    2023

  31. [39]

    Zifan Liu and Amin Karbasi and Theodoros Rekatsinas , editor =. Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 , year =

    2024

  32. [40]

    CoRR , volume =

    Zixian Huang and Kaichen Yang and Xu Huang and Feiyang Hao and Qiming Ge and Bowen Li and He Du and Kai Chen and Qipeng Guo , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2604.14164 , eprinttype =. 2604.14164 , timestamp =

    2026 doi arXiv

  33. [41]

    DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , journal =

    DeepSeek. DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , journal =. 2026 , url =. doi:10.48550/ARXIV.2606.19348 , eprinttype =. 2606.19348 , timestamp =

    2026 doi

  34. [42]

    ZeRO: memory optimizations toward training trillion parameter models , booktitle =

    Samyam Rajbhandari and Jeff Rasley and Olatunji Ruwase and Yuxiong He , editor =. ZeRO: memory optimizations toward training trillion parameter models , booktitle =. 2020 , url =. doi:10.1109/SC41405.2020.00024 , timestamp =

    2020 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.