Pith. sign in

REVIEW 3 major objections 4 minor 68 references

IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Per-domain data reweighting guided by second-order influence lifts multi-capability SFT scores by roughly 7% over uniform mixing, reaching its best balance in two iterations.

desk verdict A reasonable domain-level influence-function idea, undercut by training on the evaluation benchmarks; the reported 7% gain is contaminated. read the letter →

arxiv 2505.12762 v1 pith:DULOFREY submitted 2025-05-19 cs.AI

Chenlin Ming , Chendi Qu , Mengzhang Cai , Qizhi Pei , Zhuoshi Pan , Yu Li , Xiaoming Duan , Lijun Wu
show 1 more author
Conghui He
This is my paper · ORCID
classification cs.AI
keywords datamixtureoptimizationsupervisedfine-tuningmulti-capabilityalignmentinfluencefunctionbi-levelK-FACreweightingLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the composition of a multi-domain supervised fine-tuning dataset—how many examples each capability contributes—can be optimized by a short, gradient-guided iterative procedure instead of uniform mixing or hand-tuning. The procedure, called IDEAL, measures the influence of each domain's data volume on a small reference loss, then upsamples or downsamples domains in the direction that most reduces that loss, repeating for two rounds. In experiments across mathematics, coding, reasoning, and instruction-following, the reweighted mixtures beat uniform data blending by roughly 7% on average, and the paper argues the gain comes from the mixture distribution rather than from adding data. The practical point is that SFT data preparation currently relies on trial and error, and the paper proposes a principled, cheaply computable update to replace that guesswork.

What carries the argument

The load-bearing object is the influence of a per-domain volume parameter $\beta$ on the reference loss, $\alpha_j = -(\partial L(D_{\mathrm{ref}},\theta^*)/\partial \theta^*)^\top \nabla^2 L(D_{\mathrm{tr}},\theta^*)^{-1} \nabla L(D_j,\theta^*)$, derived by differentiating the first-order stationarity condition of the weighted training objective through the implicit function theorem. The update rule $\beta = -\gamma \odot \alpha$ converts this influence into concrete upsample and downsample decisions, with a dynamic scaling vector capping the step size so that each round makes a moderate, stable adjustment. A K-FAC Kronecker-factorized, eigendecomposed approximation of the Hessian is what makes the inverse-Hessian-vector product tractable at the scale of an 8-billion-parameter model. Together these pieces turn dataset composition into a quantity that can be optimized by gradient descent rather than guessed.

What would settle it

Take a small model and the same four domains, and train it to genuine convergence instead of one epoch: compute the exact influence of Eq. (5) and the K-FAC approximation, then compare both with the best mixture found by exhaustive search over domain shares. If the exact influence points away from the empirical optimum, the lemma's optimality premise is the failure point; if the directions agree but two IDEAL updates still do not beat uniform mixing, the step size or stopping rule deserves the blame.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that the optimal SFT data mixture is a differentiable object: the best per-domain data volume is found by differentiating the trained model's reference loss with respect to a volume-controlling parameter $\beta$, evaluated at the optimum of the weighted training objective. Lemma 1 gives the derivative of the optimal parameters with respect to each $\beta_j$ as $-H^{-1}\nabla L(D_j,\theta^*)$, and substituting this into the chain rule yields the influence of each domain's share on the reference loss. A K-FAC block-diagonal approximation makes the inverse Hessian computable for an 8-billion-parameter model, and a scaling vector turns the influence into a concrete update $D_i \leftarrow (1+\beta_i)D_i$. Two such updates suffice in the reported settings: IDEAL reaches an average of 57.9–59.2 across the four benchmarks versus 54.8–55.4 for joint SFT, and the controlled comparison that equalizes data volume shows the advantage comes from the distribution rather than the amount of data.

Load-bearing premise

The influence calculation assumes the fine-tuned model has reached the optimum of its weighted training objective, but the experiments train for only one or three epochs—far short of that—so the gradient that drives each reweighting step may not be the true influence of a domain's data share on the reference loss.

Editorial extensions

If this is right

  • Uniform or hand-set data mixtures are a costly default: the influence-guided update raises average multi-task scores by about 7% over uniform blending with no new data and no hyperparameter sweeps.
  • Two iterations of reweighting are sufficient in the tested settings, and the method is stable enough that the second-round model substantially improves coding performance without sacrificing the other domains.
  • Data volume is not the lever: the extended experiment shows that the full 66k-sample joint SFT does not beat the balanced 25k-sample mixture, and equalizing volume across reweighting baselines leaves IDEAL ahead.
  • The mechanism carries over to a five-domain, eight-benchmark setup with still-positive average gains (about 2.1%), indicating the effect is not an artifact of the four-domain configuration.
  • Because each pass only needs the trained model, a reference set, and K-FAC Hessian computations, the procedure can replace manual reweighting in routine SFT pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the update direction from Lemma 1 is roughly correct even when training stops far short of convergence (as the results hint, though the derivation assumes optimality), the same two-step influence update should transfer to other mixture-tuning settings where volumes are currently hand-set, such as preference-data blending for alignment or continual fine-tuning.
  • The Hessian approximation is replaceable: a damped diagonal or low-rank curvature estimate would probe whether the update direction, rather than the K-FAC precision, does the work, and would make the method cheaper still.
  • A small-scale replica where a model can be trained to genuine convergence could separate the lemma from the approximations: compute the exact inverse-Hessian influence and check whether it points toward the empirically best mixture found by exhaustive search.
  • The 'optimal in two iterations' result is tied to the step-size cap $m=0.15$; running more iterations or an adaptive $m$ would reveal whether two is a genuine fixed point or just the first two safe steps.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes IDEAL, an iterative framework that adjusts per-domain data volumes in multi-capability supervised fine-tuning (SFT). A domain-level weight beta controls upsampling or downsampling of each domain's training data, and an influence-function-style derivation (Lemma 1) is used to compute the gradient of a reference loss L(Dref, theta*) with respect to beta. The Hessian is approximated with K-FAC, and the resulting beta updates are applied over two iterations. Experiments on Llama3.1-8B cover mathematics, coding, reasoning, instruction-following, and (in an extended setting) trustworthiness, claiming an average improvement of about 7% over uniform data mixing and about 2.1% in the extended five-domain setting.

Significance. If the central claim were valid, IDEAL would be a useful contribution: a principled, gradient-guided alternative to heuristic data mixing for SFT, with a clearly stated objective (minimizing reference loss) and a scalable Hessian approximation. The paper includes a formal derivation, an explicit limitation appendix, and sensitivity analyses, and it reports repeated runs. However, the significance is undermined by two load-bearing problems. First, the training data for mathematics and reasoning are generated from the official GSM8K and BBH datasets, which are also the evaluation benchmarks, and the reference dataset Dref is never specified or shown to be disjoint from the training/evaluation data; the reported gains may therefore reflect test-set optimization rather than generalization. Second, Lemma 1's derivation assumes the model is trained to optimality, while the experiments use 1 or 3 epochs and Appendix C concedes the resulting sub-optimality; the computed influence gradient is thus not the true influence of beta on the reference loss. These issues are not local presentation defects: they affect the validity of the main empirical and theoretical claims.

major comments (3)
  1. [Appendix A, Section 4.1, Algorithm 1] The evaluation protocol is compromised by training/evaluation overlap. Appendix A states that the mathematics training set is built by starting with 'the official GSM8K dataset' and using GPT-4 to generate chain-of-thought solutions, and that the reasoning training set is built by using 'the official BBH dataset as a foundation' with GPT-4-regenerated answers. Section 4.1 lists GSM8K and BBH as the evaluation benchmarks for mathematics and reasoning. The paper does not report a train/test split for these datasets, nor does it describe any deduplication between the training set, Dref, and the test sets. Section 3.1 introduces Dref as a 'small, independent reference dataset' that is 'strictly excluded from the training process,' but its source is never given and its disjointness from GSM8K, BBH, IFEval, and the derived training sets is never established. Since Eq. (2) optimizes beta by minimizing L(Dref, theta*), and the reported improvements in Table 1 are computed on those same benchmarks, the claimed ~7% improvement cannot be interpreted as evidence of generalization. This is a load-bearing flaw in the central empirical claim.
  2. [Lemma 1, Appendix B.1, Appendix C, Section 4.2] The derivation of the influence gradient in Eq. (5) relies on Eq. (10), which assumes that theta* is a local optimum of the weighted training objective in Eq. (1), so that the implicit function theorem can be applied. The experimental protocol, however, trains all models for exactly 1 or 3 epochs (Section 4.2), which is far from convergence, and the number of epochs is a fixed experimental choice, not a convergence criterion. Appendix C explicitly concedes that 'models are typically not trained to reach the globally optimal parameters' and that the gradients and Hessian-related calculations 'may not accurately represent the true behavior of the model at its optimal state.' Consequently, the quantity computed in Eq. (5) is not the true derivative of L(Dref, theta*) with respect to beta, and the updates in Algorithm 1 are not justified by Lemma 1 as written. This is not a minor approximation: the beta update rule is the core mechanism of the method, and its validity under the actual training regime is not established.
  3. [Section 5.1, Eq. (9), Introduction] The method's key hyperparameter m is selected on the same evaluation benchmarks used to report the headline improvement. Section 5.1 compares m in {0.1, 0.15, 0.3} using the D1(IDEAL) setting of Section 4.2, whose performance is measured on GSM8K, HumanEval, BBH, and IFEval -- the same benchmarks that produce the ~7% claim in the abstract. Choosing m=0.15 because it 'achieves the highest average performance' is a form of test-set model selection. The introduction states that the framework 'operates without costly hyperparameter sweeps,' but the method has several free parameters (m, sigma, the number of iterations T, the K-FAC layer-selection threshold, and the damping lambda), and at least m is tuned on the evaluation benchmarks. This compounds the circularity identified above and further weakens the claim that the reported gains are attributable to the optimized data distribution rather than to selection on the test set.
minor comments (4)
  1. [Algorithm 1, Line 2] Line 2 says 'Train M0 on Dtr,t until optimal,' but the experiments in Section 4.2 train for a fixed 1 or 3 epochs; the pseudocode should state the actual stopping condition used in the experiments, or the experiments should be reconciled with the convergence assumption in Lemma 1.
  2. [Table 1, 'Specific' rows] The 'Specific' rows report averages that include zero-size datasets for the other three domains, so the Overall column for those rows is an average over the single nonzero domain's score and three zero scores; this presentation makes the single-task baselines look much weaker than they are and should be clarified with a footnote or a different aggregation.
  3. [Appendix B.2] The statement that a nonconvex problem 'can be locally approximated by a strictly convex optimization problem' near a local minimum is handwaving; a strict local minimum only guarantees positive semidefiniteness of the Hessian, not invertibility, and the damping argument is introduced without tying it to the actual loss used in training.
  4. [Eq. (8)] The definition of Lambda_ii as the variance of the projected pseudo-gradient is not clearly connected to the eigendecompositions in Eq. (7); please spell out how Lambda relates to the eigenvalues of X_l and Delta_l, since the current notation is ambiguous.

Circularity Check

2 steps flagged · score 7.0 of 10

Reported gains are fitted to the evaluation benchmarks: math and reasoning training sets are GPT-4-regenerated from the official GSM8K and BBH test sets, Dref is never shown disjoint, and m is selected on the same benchmarks.

  1. fitted input called prediction [Section 3.1 (Eq. 2); Section 4.1 and Appendix A (Dataset Preparation)]
    "We employ a small, independent reference dataset Dref ... our ultimate objective is to minimize the reference loss: L(Dref, θ∗). ... Mathematics: We start with the official GSM8K dataset and leverage the GPT-4 to generate the corresponding chain-of-thought (CoT) solutions. ... Reasoning: we utilize the official BBH dataset as a foundation and employ GPT-4 to regenerate the corresponding answers."

    Eq. (2) defines the optimized objective as L(Dref, θ*), and Section 4.2 reports average scores on GSM8K, BBH, HumanEval, and IFEval. Appendix A builds the mathematics and reasoning training sets from the official GSM8K and BBH datasets, which are exactly two of the four reported evaluation benchmarks. The paper never specifies Dref's source and never proves that Dref is disjoint from these training sets. Minimizing L(Dref) therefore tunes β to the same benchmark family that is later reported as the method's performance; the 'approximately 7% improvement' is a value of the fitted objective, not an independent prediction of generalization.

  2. fitted input called prediction [Section 5.1, Figure 2]
    "we carry out experiments on three different settings for the choice of m∈{0.1,0.15,0.3}. Results on D1(IDEAL) in Sec. 4.2 after training 1 epoch are shown in Figure 2. ... Based on our experiments, we recommend m=0.15 as the optimal value."

    The main-result configuration m=0.15 is selected by comparing performance on the same four benchmarks (GSM8K, BBH, HumanEval, IFEval) that constitute the reported main result. The reported 'approximately 7%' gain is therefore a post-selection maximum over the tested hyperparameter values, not an out-of-sample prediction from a fixed, fully specified algorithm.

full rationale

The optimization objective in Eq. (2) is L(Dref, θ*), and the paper's central empirical claim is an average score over GSM8K, BBH, HumanEval, and IFEval. Appendix A reveals that the mathematics and reasoning training sets are GPT-4-regenerated versions of the official GSM8K and BBH datasets, which are exactly two of the four reported benchmarks; the paper never specifies Dref's source nor demonstrates disjointness from these training sets. Consequently the beta trajectory that produces the roughly 7% improvement is fitted to the evaluation family, making the headline result an optimization artifact rather than an independent test of generalization. This is the load-bearing circularity. Separately, Section 5.1 selects m=0.15 by comparing performance on the same reported benchmarks, so the final number is a post-selection maximum. The mathematical derivation in Lemma 1 and Eq. (5) is not itself circular, but Appendix C admits that in practice models are not trained to optimality, so the influence calculation is approximate; that is a correctness risk rather than a circularity. The authors' self-citations (e.g., [33,45,48]) appear only as contextual references and are not load-bearing. Because the empirical validation reduces to optimizing the reported metric, the circularity score is 7.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

Free parameters: m (0.15), sigma (0.5), iteration count (2), damping lambda (unset), K-FAC layer count (unset). Axioms are the optimality assumption, Hessian invertibility, K-FAC fidelity, random-sampling unbiasedness, and the repeat-data equivalence; the first is contradicted by the experimental schedule. No new entities are postulated.

free parameters (5)
  • m (gradient scale bound) = 0.15
    Eq. (9): gamma = m / maxabs(alpha), beta = -gamma * alpha. Tuned in Section 5.1 on the same evaluation benchmarks; controls the magnitude of per-iteration data volume changes.
  • sigma (sampling factor) = 0.5
    Section 4.1: sample factor sigma = 0.5 for accelerated Hessian computation; no sensitivity study is provided.
  • number of iterations T = 2
    Algorithm 1 stops at 2 iterations in all main results; the text claims 'optimal balance in 2 iterations' but no formal stopping criterion is defined.
  • K-FAC layer selection threshold = unspecified
    Section 3.3: 'important' MLP layers are chosen by lower variance, but no threshold or number of layers is given.
  • damping lambda = unspecified
    Appendix B.2 introduces lambda > 0 to make the Hessian invertible; no value or usage in experiments is reported.
assumptions (6)
  • domain assumption Empirical risk L is twice differentiable with respect to theta, and the beta domain S is compact and convex.
    Assumption 1, Section 3.2; standard for cross-entropy losses but never verified for the actual model.
  • ad hoc to paper The model is trained to optimality (theta* is a local minimum) so that Eq. (10) and the implicit function theorem apply.
    Proof of Lemma 1, Appendix B.1; contradicted by 1-3 epoch training, acknowledged in Appendix C.
  • standard math The Hessian is invertible, ensured by local strict convexity or by an added damping term.
    Appendix B.2; the damping lambda is never given a value.
  • domain assumption The K-FAC block-diagonal approximation captures the Hessian well enough for influence computation.
    Section 3.3 Eq. (6); errors acknowledged in Appendix C.
  • standard math Random sampling from the training set provides unbiased estimates of the Hessian factors.
    Appendix C invokes the law of large numbers; finite-sample bias remains.
  • domain assumption Repeating training data up to four times is approximately as effective as adding new data.
    Section 3.2, cited from Muennighoff et al. [43]; used to justify upsampling and downsampling as a proxy for volume changes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment." pith.science (2026). https://pith.science/paper/DULOFREY

@misc{pith2026250512762,
  author       = {Pith},
  title        = {Pith review of: IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DULOFREY}},
  note         = {Machine review of arXiv:2505.12762}
}
read the original abstract

Large Language Models (LLMs) have achieved impressive performance through Supervised Fine-tuning (SFT) on diverse instructional datasets. When training on multiple capabilities simultaneously, the mixture training dataset, governed by volumes of data from different domains, is a critical factor that directly impacts the final model's performance. Unlike many studies that focus on enhancing the quality of training datasets through data selection methods, few works explore the intricate relationship between the compositional quantity of mixture training datasets and the emergent capabilities of LLMs. Given the availability of a high-quality multi-domain training dataset, understanding the impact of data from each domain on the model's overall capabilities is crucial for preparing SFT data and training a well-balanced model that performs effectively across diverse domains. In this work, we introduce IDEAL, an innovative data equilibrium adaptation framework designed to effectively optimize volumes of data from different domains within mixture SFT datasets, thereby enhancing the model's alignment and performance across multiple capabilities. IDEAL employs a gradient-based approach to iteratively refine the training data distribution, dynamically adjusting the volumes of domain-specific data based on their impact on downstream task performance. By leveraging this adaptive mechanism, IDEAL ensures a balanced dataset composition, enabling the model to achieve robust generalization and consistent proficiency across diverse tasks. Experiments across different capabilities demonstrate that IDEAL outperforms conventional uniform data allocation strategies, achieving a comprehensive improvement of approximately 7% in multi-task evaluation scores.

Figures

Figures reproduced from arXiv: 2505.12762 by the authors.

Figure 1
Figure 1. IDEAL adjusts data mixture proportions to optimize model performance, leading to a de￾crease in loss. Extensive experiments validate IDEAL’s effec￾tiveness across diverse capability combinations, including reasoning, mathematics, code gen￾eration, and instruction-following. On multi￾ple benchmarks, IDEAL outperforms uniform data blending by 7% on average and even sur￾passes single-task SFT baselines after 2 itera￾ti… view at source ↗
Figure 2
Figure 2. Models’ performance on different domains with varying [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 20 canonical work pages

  1. [1]

    Agarwal, K

    I. Agarwal, K. Killamsetty, and L. e. a. Popa. Delift: Data efficient language model instruction fine tuning.arXiv preprint arXiv:2411.04425, 2024

    arXiv 2024

  2. [2]

    Albalak, Y

    A. Albalak, Y . Elazar, and S. M. e. a. Xie. A survey on data selection for language models. arXiv preprint arXiv:2402.16827, 2024

    arXiv 2024

  3. [3]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. https://www-cdn.anthropic. com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf, 2023. Accessed 18 May 2024

    work page 2023

  4. [5]

    Austin, A

    J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    arXiv 2021

  5. [6]

    Bengio, J

    Y . Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. InProceedings of the 26th annual international conference on machine learning, pages 41–48, 2009

    2009

  6. [7]

    M. Chen, J. Tworek, and H. J. et al. Evaluating large language models trained on code. 2021

    work page 2021

  7. [8]

    X. Chen, Z. Wang, and D. e. a. Sow. Take the bull by the horns: Hard sample-reweighted continual training improves llm generalization.arXiv preprint arXiv:2402.14270, 2024

    arXiv 2024

  8. [9]

    H. W. Chung, N. Constant, and X. e. a. Garcia. Unimax: Fairer and more effective language sampling for large-scale multilingual pretraining.arXiv preprint arXiv:2304.09151, 2023

    arXiv 2023

Show all 68 references
  1. [10]

    Clark, I

    P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  2. [11]

    Cobbe, V

    K. Cobbe, V . Kosaraju, and M. e. a. Bavarian. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  3. [12]

    Contributors

    O. Contributors. Opencompass: A universal evaluation platform for foundation models. https: //github.com/open-compass/opencompass, 2023

    2023

  4. [13]

    Dehaerne, B

    E. Dehaerne, B. Dey, and S. e. a. Halder. Code generation using machine learning: A systematic review.Ieee Access, 10:82434–82455, 2022

    2022

  5. [14]

    G. Dong, H. Yuan, and K. e. a. Lu. How abilities in large language models are affected by supervised fine-tuning data composition.arXiv preprint arXiv:2310.05492, 2023

    2023 arXiv

  6. [15]

    Engstrom, A

    L. Engstrom, A. Feldmann, and A. Madry. Dsdm: Model-aware dataset selection with datamod- els.arXiv preprint arXiv:2401.12926, 2024

    2024 arXiv

  7. [16]

    S. Fan, M. Pagliardini, and M. Jaggi. Doge: Domain reweighting with generalization estimation. arXiv preprint arXiv:2310.15393, 2023

    2023 arXiv

  8. [17]

    Z. Fan, S. Du, S. Hu, P. Wang, L. Shen, Y . Zhang, D. Tao, and Y . Wang. Combatting dimensional collapse in llm pre-training data via diversified file selection.arXiv preprint arXiv:2504.20644, 2025

    2025 arXiv

  9. [18]

    Franceschelli and M

    G. Franceschelli and M. Musolesi. On the creativity of large language models.AI & SOCIETY, pages 1–11, 2024

    2024

  10. [19]

    Fulay, W

    S. Fulay, W. Brannon, S. Mohanty, C. Overney, E. Poole-Dayan, D. Roy, and J. Kabbara. On the relationship between truth and political bias in language models.arXiv preprint arXiv:2409.05283, 2024

    2024 arXiv

  11. [20]

    T. Ge, X. Chan, and X. e. a. Wang. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv:2406.20094, 2024. 11

    2024 arXiv

  12. [21]

    Gómez-Rodríguez and P

    C. Gómez-Rodríguez and P. Williams. A confederacy of models: A comprehensive evaluation of llms on creative writing.arXiv preprint arXiv:2310.08433, 2023

    2023 arXiv

  13. [22]

    Grattafiori, A

    A. Grattafiori, A. Dubey, and A. J. et al. The llama 3 herd of models, 2024. URL https: //arxiv.org/abs/2407.21783

    2024 arXiv

  14. [23]

    Grosse, J

    R. Grosse, J. Bae, and C. e. a. Anil. Studying large language model generalization with influence functions.arXiv preprint arXiv:2308.03296, 2023

    2023 arXiv

  15. [24]

    Y . Gu, X. Han, Z. Liu, and M. Huang. Ppt: Pre-trained prompt tuning for few-shot learning. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8410–8423, 2022

    2022

  16. [25]

    Y . Gu, L. Dong, H. Wang, Y . Hao, Q. Dong, F. Wei, and M. Huang. Data selection via optimal control for language models.arXiv preprint arXiv:2410.07064, 2024

    2024 arXiv

  17. [26]

    D. Guo, D. Yang, and H. e. a. Zhang. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  18. [27]

    F. R. Hampel. The influence curve and its role in robust estimation.Journal of the american statistical association, 69(346):383–393, 1974

    1974

  19. [28]

    Hendrycks, C

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Stein- hardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  20. [29]

    L. Hu, Z. Liu, and Z. e. a. Zhao. A survey of knowledge enhanced pre-trained language models. IEEE Transactions on Knowledge and Data Engineering, 2023

    2023

  21. [30]

    Ilyas, S

    A. Ilyas, S. M. Park, and L. e. a. Engstrom. Datamodels: Predicting predictions from training data.arXiv preprint arXiv:2202.00622, 2022

    2022 arXiv

  22. [31]

    F. Kang, H. A. Just, and Y . e. a. Sun. Get more for less: Principled data selection for warming up fine-tuning in llms.arXiv preprint arXiv:2405.02774, 2024

    2024 arXiv

  23. [32]

    P. W. Koh and P. Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR, 2017

    2017

  24. [33]

    Y . Li, Q. Pei, M. Sun, H. Lin, C. Ming, X. Gao, J. Wu, C. He, and L. Wu. Cipherbank: Exploring the boundary of llm reasoning capabilities through cryptography challenges.arXiv preprint arXiv:2504.19093, 2025

    2025 arXiv

  25. [34]

    S. Lin, J. Hilton, and O. Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021

    2021 arXiv

  26. [35]

    L. Liu, X. Liu, D. F. Wong, D. Li, Z. Wang, B. Hu, and M. Zhang. Selectit: Selective instruction tuning for llms via uncertainty-aware self-reflection.Advances in Neural Information Processing Systems, 37:97800–97825, 2024

    2024

  27. [36]

    Q. Liu, X. Zheng, and N. M. et al. Regmix: Data mixture as regression for language model pre-training, 2025. URLhttps://arxiv.org/abs/2407.01492

    2025 arXiv

  28. [37]

    W. Liu, W. Zeng, and K. e. a. He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning.arXiv preprint arXiv:2312.15685, 2023

    2023 arXiv

  29. [38]

    K. Lu, H. Yuan, and Z. e. a. Yuan. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  30. [39]

    T. Q. Luong, X. Zhang, and Z. e. a. Jie. Reft: Reasoning with reinforced fine-tuning.arXiv preprint arXiv:2401.08967, 2024

    2024 arXiv

  31. [40]

    Martens and R

    J. Martens and R. Grosse. Optimizing neural networks with kronecker-factored approximate curvature. InInternational conference on machine learning, pages 2408–2417. PMLR, 2015. 12

    2015

  32. [41]

    Mecklenburg, Y

    N. Mecklenburg, Y . Lin, and X. e. a. Li. Injecting new knowledge into large language models via supervised fine-tuning.arXiv preprint arXiv:2404.00213, 2024

    2024 arXiv

  33. [42]

    Mekala, A

    D. Mekala, A. Nguyen, and J. Shang. Smaller language models are capable of selecting instruction-tuning training data for larger language models.arXiv preprint arXiv:2402.10430, 2024

    2024 arXiv

  34. [43]

    Muennighoff, A

    N. Muennighoff, A. Rush, and B. e. a. Barak. Scaling data-constrained language models. Advances in Neural Information Processing Systems, 36:50358–50376, 2023

    2023

  35. [44]

    Achiam, and S

    OpenAI, J. Achiam, and S. A. et al. Gpt-4 technical report, 2024. URL https://arxiv.org/ abs/2303.08774

    2024 arXiv

  36. [45]

    Z. Pan, Y . Li, H. Lin, Q. Pei, Z. Tang, W. Wu, C. Ming, H. V . Zhao, C. He, and L. Wu. Lemma: Learning from errors for mathematical advancement in llms.arXiv preprint arXiv:2503.17439, 2025

    2025 arXiv

  37. [46]

    S. M. Park, K. Georgiev, and A. e. a. Ilyas. Trak: Attributing model behavior at scale.arXiv preprint arXiv:2303.14186, 2023

    2023 arXiv

  38. [47]

    Parmar, S

    J. Parmar, S. Prabhumoye, and J. e. a. Jennings. Data, data everywhere: A guide for pretraining dataset construction.arXiv preprint arXiv:2407.06380, 2024

    2024 arXiv

  39. [48]

    Q. Pei, L. Wu, Z. Pan, Y . Li, H. Lin, C. Ming, X. Gao, C. He, and R. Yan. Mathfusion: Enhancing mathematic problem-solving of llm through instruction fusion.arXiv preprint arXiv:2503.16212, 2025

    2025 arXiv

  40. [49]

    L. S. Pontryagin.Mathematical theory of optimal processes. Routledge, 2018

    2018

  41. [50]

    Pruthi, F

    G. Pruthi, F. Liu, and S. e. a. Kale. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33:19920–19930, 2020

    2020

  42. [51]

    Y . Shao, L. Li, and Z. e. a. Fei. Balanced data sampling for language model training with clustering.arXiv preprint arXiv:2402.14526, 2024

    2024 arXiv

  43. [52]

    M. Shen. Rethinking data selection for supervised fine-tuning.arXiv preprint arXiv:2402.06094, 2024

    2024 arXiv

  44. [53]

    Z. Shen, T. Tao, and L. M. et al. Slimpajama-dc: Understanding data combinations for llm training, 2024. URLhttps://arxiv.org/abs/2309.10818

    2024 arXiv

  45. [54]

    C. Si, Y . Zhang, and Z. e. a. Yang. Design2code: How far are we from automating front-end engineering?arXiv e-prints, pages arXiv–2403, 2024

    2024

  46. [55]

    Suzgun, N

    M. Suzgun, N. Scales, and N. e. a. Schärli. Challenging big-bench tasks and whether chain-of- thought can solve them.arXiv preprint arXiv:2210.09261, 2022

    2022 arXiv

  47. [56]

    Touvron, T

    H. Touvron, T. Lavril, and G. e. a. Izacard. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  48. [57]

    Tunstall, E

    L. Tunstall, E. Beeching, N. Lambert, N. Rajani, S. Huang, K. Rasul, A. Bartolome, A. M. Rush, and T. Wolf. The Alignment Handbook. URL https://github.com/huggingface/ alignment-handbook

  49. [58]

    Y . Ueno, K. Osawa, and Y . e. a. Tsuji. Rich information is affordable: A systematic performance analysis of second-order optimization using k-fac. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2145–2153, 2020

    2020

  50. [59]

    T. Wang, J. Chen, and Q. e. a. Jia. Weaver: Foundation models for creative writing.arXiv preprint arXiv:2401.17268, 2024

    2024 arXiv

  51. [60]

    Y . Wang, W. Zhong, and L. e. a. Li. Aligning large language models with human: A survey. arXiv preprint arXiv:2307.12966, 2023. 13

    2023 arXiv

  52. [61]

    M. Xia, S. Malladi, and S. e. a. Gururangan. Less: Selecting influential data for targeted instruction tuning.arXiv preprint arXiv:2402.04333, 2024

    2024 arXiv

  53. [62]

    S. M. Xie, H. Pham, X. Dong, N. Du, H. Liu, Y . Lu, P. S. Liang, Q. V . Le, T. Ma, and A. W. Yu. Doremi: Optimizing data mixtures speeds up language model pretraining.Advances in Neural Information Processing Systems, 36, 2024

    2024

  54. [63]

    C. Xu, Q. Sun, and K. e. a. Zheng. Wizardlm: Empowering large language models to follow complex instructions.arXiv preprint arXiv:2304.12244, 2023

    2023 arXiv

  55. [64]

    J. Ye, P. Liu, T. Sun, Y . Zhou, J. Zhan, and X. Qiu. Data mixing laws: Optimizing data mixtures by predicting language modeling performance.arXiv preprint arXiv:2403.16952, 2024

    2024 arXiv

  56. [65]

    Z. Yu, S. Das, and C. Xiong. Mates: Model-aware data selection for efficient pretraining with data influence models.arXiv preprint arXiv:2406.06046, 2024

    2024 arXiv

  57. [66]

    Zhang, H

    C. Zhang, H. Zhong, and K. e. a. Zhang. Harnessing diversity for important data selection in pretraining large language models.arXiv preprint arXiv:2409.16986, 2024

    2024 arXiv

  58. [67]

    Zhang, Y

    H. Zhang, Y . Wu, and D. e. a. Li. Balancing speciality and versatility: a coarse to fine framework for supervised fine-tuning large language model.arXiv preprint arXiv:2404.10306, 2024

    2024 arXiv

  59. [68]

    J. Zhou, T. Lu, and S. M. et al. Instruction-following evaluation for large language models,

  60. [2023]

    14 A Dataset and Training Information Reasoning:We initially select BigBench Hard (BBH) [55] as the benchmark to evaluate the reasoning capabilities of our model

    URLhttps://arxiv.org/abs/2311.07911. 14 A Dataset and Training Information Reasoning:We initially select BigBench Hard (BBH) [55] as the benchmark to evaluate the reasoning capabilities of our model. BBH is a widely recognized benchmark designed to test a model’s ability to ha...

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.