Pith. sign in

REVIEW 3 major objections 175 references

A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction

T0 review · 3 major / 0 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pre-hoc LLM fine-tuning performance prediction faces a lower bound on how fast its uncertainty can decrease.

desk verdict New risk decomposition and lower bound on optimization variance decay for pre-hoc fine-tuning prediction, but the bound only applies if the stochastic estimation model matches actual LLM dynamics. read the letter →

arxiv 2606.17649 v1 pith:JT55P42D submitted 2026-06-16 cs.LG cs.AI

Yuxiang Luo , Chen Wang , Nan Tang This is my paper
classification cs.LGcs.AI
keywords pre-hocpredictionriskdecompositionoptimizationvariancephasediagramLLMfine-tuningperformancestochasticestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper models the task of forecasting an LLM's post-fine-tuning performance without running the fine-tuning as a stochastic estimation problem that respects information limits. It splits the total prediction risk into an intrinsic limit fixed by data-model compatibility and a reducible component called optimization variance. A proof shows that the variance term cannot decay faster than a specific rate no matter which predictor is chosen, which directly caps how quickly uncertainty about fine-tuning outcomes can shrink. This rate limit produces a phase diagram that places tasks into Static-Sufficient, Dynamic-Critical, or Noise-Dominant categories and yields a budget-optimal probing rule. The framing matters because fine-tuning large models is costly, so knowing these hard speed limits helps decide when pre-hoc checks are worth the investment.

What carries the argument

Risk decomposition into intrinsic limit and optimization variance, with the proven lower bound on the variance decay rate.

What would settle it

An experiment in which optimization variance for a concrete fine-tuning predictor decays faster than the derived lower bound on any benchmark would falsify the central claim.

Watch

Extended reading notes

Core claim

We formulate pre-hoc performance prediction as a stochastic estimation problem under information constraints, decomposing prediction risk into an intrinsic limit set by static data-model compatibility and a reducible optimization variance. We prove that optimization variance admits a necessary lower bound on its decay rate, implying fundamental constraints on how quickly uncertainty dissipates regardless of the predictor used. From these dynamics we derive a budget-optimal probing principle and a predictability phase diagram that organizes tasks into Static-Sufficient, Dynamic-Critical, and Noise-Dominant regimes.

Load-bearing premise

The pre-hoc performance prediction problem can be accurately formulated as a stochastic estimation problem under information constraints.

Editorial extensions

If this is right

  • Uncertainty dissipates at a bounded rate independent of the choice of predictor.
  • Tasks fall into one of three predictability regimes according to the phase diagram.
  • A budget-optimal probing strategy follows directly from the derived dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same decomposition could be applied to decide whether pre-hoc checks are economical for other expensive training procedures.
  • The phase diagram offers a way to pre-screen tasks before committing compute to any predictor.
  • Real-world validation on the reported benchmarks indicates the three regimes are observable rather than purely theoretical.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The manuscript presents a risk decomposition framework for pre-hoc fine-tuning prediction of large language models. It formulates the problem as a stochastic estimation task under information constraints, decomposes the risk into an intrinsic limit and reducible optimization variance, proves a lower bound on the decay rate of the optimization variance, derives a budget-optimal probing principle, and introduces a predictability phase diagram with three regimes: Static-Sufficient, Dynamic-Critical, and Noise-Dominant. The theoretical findings are supported by experiments on synthetic and real-world benchmarks.

Significance. If the central claims hold, this work would be significant for providing theoretical foundations for pre-hoc performance prediction in LLM fine-tuning, potentially leading to more efficient use of computational resources. The proof of a necessary lower bound on variance decay and the phase diagram could influence how practitioners approach probing strategies. The experimental validation adds practical value, though its strength depends on the fidelity of the model to real fine-tuning dynamics.

major comments (3)
  1. [Abstract] Abstract: The proof that optimization variance admits a necessary lower bound on its decay rate is load-bearing for the central claim of fundamental constraints independent of the predictor. However, this bound is derived from the stochastic estimation formulation under information constraints; it is unclear if this bound is a genuine necessary limit or if it is induced by the specific modeling assumptions about information constraints.
  2. [Abstract] Abstract: The predictability phase diagram with regimes Static-Sufficient, Dynamic-Critical, and Noise-Dominant is derived from the decay-rate dynamics. The manuscript should explicitly show how these regimes are defined and distinguished, including any thresholds or conditions based on the lower bound, to allow assessment of whether they organize tasks in a non-trivial way.
  3. [Abstract] Abstract: The experiments are claimed to validate the theoretical regimes and the efficiency of the probing strategy. Without details on the specific benchmarks, how the regimes were identified, or error analysis, it is difficult to evaluate if the results support the claims or if confounding factors in the data affect the validation.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments on the abstract. We address each point below with clarifications on the theoretical claims and indicate where revisions will strengthen the presentation.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The proof that optimization variance admits a necessary lower bound on its decay rate is load-bearing for the central claim of fundamental constraints independent of the predictor. However, this bound is derived from the stochastic estimation formulation under information constraints; it is unclear if this bound is a genuine necessary limit or if it is induced by the specific modeling assumptions about information constraints.

    Authors: The stochastic estimation formulation under information constraints directly models the pre-hoc setting, where predictors have restricted access to fine-tuning dynamics. The lower bound is a necessary consequence of these constraints and is independent of the predictor; it is not an artifact but follows from the information-theoretic limits in the problem formulation. We will revise the abstract to explicitly note that the bound holds under this modeling framework. revision: partial

  2. Referee: [Abstract] Abstract: The predictability phase diagram with regimes Static-Sufficient, Dynamic-Critical, and Noise-Dominant is derived from the decay-rate dynamics. The manuscript should explicitly show how these regimes are defined and distinguished, including any thresholds or conditions based on the lower bound, to allow assessment of whether they organize tasks in a non-trivial way.

    Authors: The regimes are distinguished by the relationship between the optimization variance decay rate and the intrinsic limit, using the proven lower bound: Static-Sufficient when decay exceeds the bound sufficiently to reach the limit, Dynamic-Critical when the bound governs the transition, and Noise-Dominant when decay is constrained below effective reduction. We will add explicit definitions, thresholds, and conditions based on the lower bound to the phase diagram section. revision: yes

  3. Referee: [Abstract] Abstract: The experiments are claimed to validate the theoretical regimes and the efficiency of the probing strategy. Without details on the specific benchmarks, how the regimes were identified, or error analysis, it is difficult to evaluate if the results support the claims or if confounding factors in the data affect the validation.

    Authors: The full manuscript details the synthetic benchmarks (controlled variance parameters) and real-world NLP tasks, regime identification via empirical decay plots against the theoretical bound, and error analysis with repeated runs and confidence intervals. The abstract is concise, but we will add a brief validation summary. The experiments isolate the modeled effects and support the claims. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation is a standard mathematical consequence of the stochastic model

full rationale

The paper's central result is a mathematical proof of a lower bound on optimization variance decay, obtained after explicitly formulating the problem as stochastic estimation under information constraints and decomposing risk into intrinsic limit plus reducible variance. This is a first-principles derivation internal to the chosen model rather than a tautology, fitted parameter renamed as prediction, or load-bearing self-citation. No equations or steps in the provided text reduce the bound to its inputs by construction; the bound follows from the information constraints assumed in the formulation. The modeling choice itself is an assumption whose validity is separate from circularity.

Assumptions & free parameters 0 free parameters · 1 assumptions · 1 invented entities

The central claim rests primarily on the domain assumption that pre-hoc prediction is a stochastic estimation problem under information constraints; no free parameters or invented entities with independent evidence are described in the abstract.

assumptions (1)
  • domain assumption Pre-hoc fine-tuning performance prediction can be formulated as a stochastic estimation problem under information constraints.
    This modeling choice is the starting point for the risk decomposition and the proof of the variance decay bound.
invented entities (1)
  • Predictability phase diagram with regimes Static-Sufficient, Dynamic-Critical, Noise-Dominant
    purpose: Organizes tasks according to the dynamics of prediction risk.
    Introduced based on the derived lower bound and budget-optimal probing principle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction." pith.science (2026). https://pith.science/paper/JT55P42D

@misc{pith2026260617649,
  author       = {Pith},
  title        = {Pith review of: A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JT55P42D}},
  note         = {Machine review of arXiv:2606.17649}
}
read the original abstract

The high cost of fine-tuning LLMs poses a significant economic barrier; pre-hoc performance prediction offers a critical solution to substantially reduce this expense. However, the theoretical limits of pre-hoc performance prediction remain unexplored. We formulate it as a stochastic estimation problem under information constraints, decomposing prediction risk into two components: an intrinsic limit (static data-model compatibility) and a reducible optimization variance. We prove that optimization variance admits a necessary lower bound on its decay rate, implying fundamental constraints on how quickly uncertainty dissipates, regardless of the predictor used. Based on these dynamics, we derive a budget-optimal probing principle and introduce a predictability phase diagram that organizes tasks into three distinct regimes: Static-Sufficient, Dynamic-Critical, and Noise-Dominant. Extensive experiments on synthetic and real-world benchmarks validate these theoretical regimes and demonstrate the efficiency of our probing strategy.

Figures

Figures reproduced from arXiv: 2606.17649 by the authors.

Figure 1
Figure 1. Perspective shift. We move from treating pre-hoc pre￾diction as a black-box regression problem (Left) to a structural decomposition of risk into intrinsic limits and optimization vari￾ance (Right). This motivates the problem of pre-hoc fine-tuning predic￾tion: estimating the final fine-tuning performance of a fine-tuning task before executing full training. More concretely, given a pretrained model, a dataset, and a… view at source ↗
Figure 2
Figure 2. Population-level uncertainty decay across regimes. The figure shows normalized observable uncertainty as a function of probing depth c in log-log scale, aggregated by regime. Solid curves represent regime-wise means across tasks, with shaded regions indicating bootstrap confidence intervals. Vertical dashed lines denote the stable probing interval used for fitting. reflects that probing is largely unnecessary, as ou… view at source ↗
Figure 4
Figure 4. Regime-dependent marginal gain of probing. The figure plots the normalized marginal reduction in observable uncertainty ∆(c) as a function of probing depth c for different regimes. All curves exhibit diminishing returns, but at markedly different rates. These patterns empirically validate the regime-aware stop￾ping principles derived in Theorem 6.1. They also sug￾gest that the practical value of probing lies in iden… view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Observable uncertainty decay and effective intrinsic floors across regimes. Mean Ub(c) with confidence intervals is shown as a function of probing depth. Noise-Dominant regimes plateau at higher levels, indicating larger intrinsic ambiguity. 0 1 2 3 4 5 0 20 40 60 80 1…
Figure 6
Figure 6. Figure 6: Distribution of estimated uncertainty decay rates across regimes. Dynamic-Critical tasks exhibit slower decay, while Static￾Sufficient tasks show rapid contraction. Regime I (Static-Sufficient) Regime II (Dynamic-Critical) Regime III (Noise-Dominant) 0.0000 0.0002 0.00…
Figure 7
Figure 7. Figure 7: Ablation of information sources under a fixed probing budget. Dynamic signals are informative only in Dynamic-Critical regimes. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_7.png]
Figure 8
Figure 8. Figure 8: Localization of hard cases on the empirical phase diagram. Hard-to-predict tasks cluster in Noise-Dominant regimes or near regime boundaries. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

175 extracted references · 19 canonical work pages

  1. [1]

    Langley , title =

    P. Langley , title =. Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , address =. 2000 , pages =

    2000

  2. [2]

    T. M. Mitchell. The Need for Biases in Learning Generalizations. 1980

    1980

  3. [3]

    M. J. Kearns , title =

  4. [4]

    Xiaotian Lin and Yanlin Qi and Yizhang Zhu and Themis Palpanas and Chengliang Chai and Nan Tang and Yuyu Luo , title =. Proc. 2025 , url =

    2025

  5. [5]

    Machine Learning: An Artificial Intelligence Approach, Vol. I. 1983

    1983

  6. [6]

    R. O. Duda and P. E. Hart and D. G. Stork. Pattern Classification. 2000

    2000

  7. [7]

    Suppressed for Anonymity , author=

  8. [8]

    Newell and P

    A. Newell and P. S. Rosenbloom. Mechanisms of Skill Acquisition and the Law of Practice. Cognitive Skills and Their Acquisition. 1981

    1981

Show all 175 references
  1. [9]

    A. L. Samuel. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959

    1959

  2. [10]

    Scaling Learning Algorithms Towards

    Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards

  3. [11]

    and Osindero, Simon and Teh, Yee Whye , journal =

    Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume =

  4. [12]

    2016 , publisher=

    Deep learning , author=. 2016 , publisher=

    2016

  5. [13]

    2025 , eprint=

    COSMOS: Predictable and Cost-Effective Adaptation of LLMs , author=. 2025 , eprint=

    2025

  6. [14]

    2024 , eprint=

    Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models , author=. 2024 , eprint=

    2024

  7. [15]

    2025 , eprint=

    LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection , author=. 2025 , eprint=

    2025

  8. [16]

    2017 , eprint=

    A Unified Approach to Interpreting Model Predictions , author=. 2017 , eprint=

    2017

  9. [17]

    2020 , eprint=

    Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples , author=. 2020 , eprint=

    2020

  10. [18]

    2021 , eprint=

    Dataset2Vec: Learning Dataset Meta-Features , author=. 2021 , eprint=

    2021

  11. [19]

    2022 , eprint=

    Model Zoos: A Dataset of Diverse Populations of Neural Network Models , author=. 2022 , eprint=

    2022

  12. [20]

    Proceedings of the 24th International Conference on Artificial Intelligence , pages =

    Domhan, Tobias and Springenberg, Jost Tobias and Hutter, Frank , title =. Proceedings of the 24th International Conference on Artificial Intelligence , pages =. 2015 , isbn =

    2015

  13. [21]

    2020 , eprint=

    Scaling Laws for Neural Language Models , author=. 2020 , eprint=

    2020

  14. [22]

    2024 , eprint=

    ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models , author=. 2024 , eprint=

    2024

  15. [23]

    2020 , eprint=

    Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics , author=. 2020 , eprint=

    2020

  16. [24]

    2019 , eprint=

    Data Shapley: Equitable Valuation of Data for Machine Learning , author=. 2019 , eprint=

    2019

  17. [25]

    2025 , eprint=

    Refine-n-Judge: Curating High-Quality Preference Chains for LLM-Fine-Tuning , author=. 2025 , eprint=

    2025

  18. [26]

    2018 , eprint=

    The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards , author=. 2018 , eprint=

    2018

  19. [27]

    2021 , eprint=

    MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers , author=. 2021 , eprint=

    2021

  20. [28]

    2023 , eprint=

    GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation , author=. 2023 , eprint=

    2023

  21. [29]

    2021 , eprint=

    On the Importance of Gradients for Detecting Distributional Shifts in the Wild , author=. 2021 , eprint=

    2021

  22. [30]

    2018 , eprint=

    GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks , author=. 2018 , eprint=

    2018

  23. [31]

    Louis , year=

    Shuofeng Zhang and Isaac Reid and Guillermo Valle-Perez and Ard A. Louis , year=

  24. [32]

    2024 , url=

    Gradient norm as a powerful proxy to out-of-distribution error estimation , author=. 2024 , url=

    2024

  25. [33]

    2023 , eprint=

    Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization , author=. 2023 , eprint=

    2023

  26. [34]

    2021 , eprint=

    Self-Validation: Early Stopping for Single-Instance Deep Generative Priors , author=. 2021 , eprint=

    2021

  27. [35]

    2025 , eprint=

    Autoencoder-Based Framework to Capture Vocabulary Quality in NLP , author=. 2025 , eprint=

    2025

  28. [36]

    2025 , eprint=

    Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality , author=. 2025 , eprint=

    2025

  29. [37]

    2025 , eprint=

    Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models , author=. 2025 , eprint=

    2025

  30. [38]

    Model Explainability using SHAP Values for LightGBM Predictions , year=

    Bugaj, Michal and Wrobel, Krzysztof and Iwaniec, Joanna , booktitle=. Model Explainability using SHAP Values for LightGBM Predictions , year=

  31. [39]

    Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =

    Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan , title =. Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =. 2017 , isbn =

    2017

  32. [40]

    2025 , issue_date =

    Wang, Ying and Wang, Pengxin and Tansey, Kevin and Liu, Junming and Delaney, Bethany and Quan, Wenting , title =. 2025 , issue_date =. doi:10.1016/j.compag.2024.109758 , journal =

    2025 doi

  33. [41]

    Garcia and Carlos Soares and Joaquin Vanschoren and André C.P.L.F

    Adriano Rivolli and Luís P.F. Garcia and Carlos Soares and Joaquin Vanschoren and André C.P.L.F. Meta-features for meta-learning , journal =. 2022 , issn =. doi:https://doi.org/10.1016/j.knosys.2021.108101 , url =

    2022 doi

  34. [42]

    2022 , eprint=

    Predicting Fine-Tuning Performance with Probing , author=. 2022 , eprint=

    2022

  35. [43]

    2024 , eprint=

    Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need , author=. 2024 , eprint=

    2024

  36. [44]

    2020 , eprint=

    The Break-Even Point on Optimization Trajectories of Deep Neural Networks , author=. 2020 , eprint=

    2020

  37. [45]

    2019 , eprint=

    Visualizing and Understanding the Effectiveness of BERT , author=. 2019 , eprint=

    2019

  38. [46]

    2023 , eprint=

    Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks , author=. 2023 , eprint=

    2023

  39. [47]

    International Conference on Learning Representations (ICLR) , year =

    Measuring Massive Multitask Language Understanding , author =. International Conference on Learning Representations (ICLR) , year =

  40. [48]

    arXiv preprint arXiv:2412.15115 , year =

    Qwen2.5 Technical Report , author =. arXiv preprint arXiv:2412.15115 , year =. doi:10.48550/arXiv.2412.15115 , url =

    doi arXiv

  41. [49]

    2025 , url =

    Qwen2.5-7B-Instruct , author =. 2025 , url =

    2025

  42. [50]

    2025 , eprint=

    A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules , author=. 2025 , eprint=

    2025

  43. [51]

    2020 , eprint=

    Don't Stop Pretraining: Adapt Language Models to Domains and Tasks , author=. 2020 , eprint=

    2020

  44. [52]

    Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research

    Mu, Yida and Jin, Mali and Song, Xingyi and Aletras, Nikolaos. Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:...

    2024 doi

  45. [53]

    2025 , eprint=

    Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric , author=. 2025 , eprint=

    2025

  46. [54]

    2023 , eprint=

    The Vendi Score: A Diversity Evaluation Metric for Machine Learning , author=. 2023 , eprint=

    2023

  47. [55]

    2018 , eprint=

    Visualizing the Loss Landscape of Neural Nets , author=. 2018 , eprint=

    2018

  48. [56]

    2020 , eprint=

    Adversarial Weight Perturbation Helps Robust Generalization , author=. 2020 , eprint=

    2020

  49. [57]

    Mathematics , VOLUME =

    Liang, Hailun and Zheng, Haowen and Wang, Hao and He, Liu and Lin, Haoyi and Liang, Yanyan , TITLE =. Mathematics , VOLUME =. 2025 , NUMBER =

    2025

  50. [58]

    Technometrics , volume =

    Ridge Regression: Biased Estimation for Nonorthogonal Problems , author =. Technometrics , volume =. 1970 , publisher =

    1970

  51. [59]

    Advances in Neural Information Processing Systems , volume =

    Support Vector Regression Machines , author =. Advances in Neural Information Processing Systems , volume =. 1997 , publisher =

    1997

  52. [60]

    Statistics and Computing , volume =

    A Tutorial on Support Vector Regression , author =. Statistics and Computing , volume =. 2004 , doi =

    2004

  53. [61]

    Machine Learning , volume =

    Random Forests , author =. Machine Learning , volume =. 2001 , doi =

    2001

  54. [62]

    Nature , volume =

    Learning Representations by Back-Propagating Errors , author =. Nature , volume =. 1986 , doi =

    1986

  55. [63]

    Neural Networks , volume =

    Multilayer Feedforward Networks Are Universal Approximators , author =. Neural Networks , volume =. 1989 , doi =

    1989

  56. [64]

    Journal of Machine Learning Research , volume =

    Scikit-learn: Machine Learning in Python , author =. Journal of Machine Learning Research , volume =. 2011 , url =

    2011

  57. [65]

    arXiv preprint arXiv:1706.10239 , year=

    Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes , author=. arXiv preprint arXiv:1706.10239 , year=

    arXiv

  58. [66]

    arXiv preprint arXiv:2412.13573 , year=

    Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes , author=. arXiv preprint arXiv:2412.13573 , year=

  59. [67]

    2019 , eprint=

    Characterizing classification datasets: a study of meta-features for meta-learning , author=. 2019 , eprint=

    2019

  60. [68]

    2024 , eprint=

    Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets , author=. 2024 , eprint=

    2024

  61. [69]

    2023 , eprint=

    Advances and Challenges in Meta-Learning: A Technical Review , author=. 2023 , eprint=

    2023

  62. [70]

    2018 , eprint=

    Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization , author=. 2018 , eprint=

    2018

  63. [71]

    ROUGE : A Package for Automatic Evaluation of Summaries

    Lin, Chin-Yew. ROUGE : A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out. 2004

    2004

  64. [72]

    2018 , eprint=

    A Survey of Machine Learning for Big Code and Naturalness , author=. 2018 , eprint=

    2018

  65. [73]

    , title =

    Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J. , title =. J. Mach. Learn. Res. , month = jan, articleno =. 2020 , issue_date =

    2020

  66. [74]

    2022 , eprint=

    Deduplicating Training Data Makes Language Models Better , author=. 2022 , eprint=

    2022

  67. [75]

    2023 , eprint=

    Extracting Training Data from Diffusion Models , author=. 2023 , eprint=

    2023

  68. [76]

    2022 , eprint=

    Training Compute-Optimal Large Language Models , author=. 2022 , eprint=

    2022

  69. [77]

    2019 , eprint=

    Deep Anomaly Detection with Outlier Exposure , author=. 2019 , eprint=

    2019

  70. [78]

    2022 , eprint=

    Confident Learning: Estimating Uncertainty in Dataset Labels , author=. 2022 , eprint=

    2022

  71. [79]

    2021 , eprint=

    Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus , author=. 2021 , eprint=

    2021

  72. [80]

    2020 , eprint=

    The Curious Case of Neural Text Degeneration , author=. 2020 , eprint=

    2020

  73. [81]

    A Qualitative Comparison of C o QA , SQ u AD 2.0 and Q u AC

    Yatskar, Mark. A Qualitative Comparison of C o QA , SQ u AD 2.0 and Q u AC. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019. doi:10.18653/v1/N19-1241

    2019 doi

  74. [82]

    2016 , eprint=

    Exploring the Limits of Language Modeling , author=. 2016 , eprint=

    2016

  75. [83]

    2018 , eprint=

    Universal Sentence Encoder , author=. 2018 , eprint=

    2018

  76. [84]

    2020 , eprint=

    ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators , author=. 2020 , eprint=

    2020

  77. [85]

    2016 , eprint=

    A Diversity-Promoting Objective Function for Neural Conversation Models , author=. 2016 , eprint=

    2016

  78. [86]

    2020 , eprint=

    Fine-Tuning Language Models from Human Preferences , author=. 2020 , eprint=

    2020

  79. [87]

    2020 , eprint=

    Unsupervised Domain Clusters in Pretrained Language Models , author=. 2020 , eprint=

    2020

  80. [88]

    Domain Adaptation via Pseudo In-Domain Data Selection

    Axelrod, Amittai and He, Xiaodong and Gao, Jianfeng. Domain Adaptation via Pseudo In-Domain Data Selection. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 2011

    2011

  81. [89]

    Survey of Hallucination in Natural Language Generation , volume=

    Ji, Ziwei and Lee, Nayeon and Frieske, Rita and Yu, Tiezheng and Su, Dan and Xu, Yan and Ishii, Etsuko and Bang, Ye Jin and Madotto, Andrea and Fung, Pascale , year=. Survey of Hallucination in Natural Language Generation , volume=. ACM Computing Surveys , publisher=. doi:10.1...

    doi

  82. [90]

    2020 , eprint=

    Reducing Quantity Hallucinations in Abstractive Summarization , author=. 2020 , eprint=

    2020

  83. [91]

    2017 , eprint=

    Deep Learning Scaling is Predictable, Empirically , author=. 2017 , eprint=

    2017

  84. [92]

    2017 , eprint=

    A Closer Look at Memorization in Deep Networks , author=. 2017 , eprint=

    2017

  85. [93]

    2022 , eprint=

    A Survey of Learning Curves with Bad Behavior: or How More Data Need Not Lead to Better Performance , author=. 2022 , eprint=

    2022

  86. [94]

    2013 , eprint=

    On the difficulty of training Recurrent Neural Networks , author=. 2013 , eprint=

    2013

  87. [95]

    Proceedings of the 30th International Conference on Machine Learning , pages =

    On the importance of initialization and momentum in deep learning , author =. Proceedings of the 30th International Conference on Machine Learning , pages =. 2013 , editor =

    2013

  88. [96]

    2019 , eprint=

    Towards Understanding Generalization in Gradient-Based Meta-Learning , author=. 2019 , eprint=

    2019

  89. [97]

    2021 , eprint=

    GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training , author=. 2021 , eprint=

    2021

  90. [98]

    2017 , eprint=

    Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , author=. 2017 , eprint=

    2017

  91. [99]

    2019 , eprint=

    The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks , author=. 2019 , eprint=

    2019

  92. [100]

    2021 , eprint=

    Rigging the Lottery: Making All Tickets Winners , author=. 2021 , eprint=

    2021

  93. [101]

    2020 , eprint=

    Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers , author=. 2020 , eprint=

    2020

  94. [102]

    2017 , eprint=

    SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability , author=. 2017 , eprint=

    2017

  95. [103]

    2017 , eprint=

    On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima , author=. 2017 , eprint=

    2017

  96. [104]

    2018 , eprint=

    Overcoming catastrophic forgetting problem by weight consolidation and long-term memory , author=. 2018 , eprint=

    2018

  97. [105]

    Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =

    Deep Sparse Rectifier Neural Networks , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =. 2011 , editor =

    2011

  98. [106]

    Networks of spiking neurons: The third generation of neural network models , journal =

    Wolfgang Maass , keywords =. Networks of spiking neurons: The third generation of neural network models , journal =. 1997 , issn =. doi:https://doi.org/10.1016/S0893-6080(97)00011-7 , url =

    1997 doi

  99. [107]

    Predicting Fine-tuned Performance on Larger Datasets Before Creating Them

    Kuramoto, Toshiki and Suzuki, Jun. Predicting Fine-tuned Performance on Larger Datasets Before Creating Them. Proceedings of the 31st International Conference on Computational Linguistics: Industry Track. 2025

    2025

  100. [108]

    2023 , eprint=

    SemDeDup: Data-efficient learning at web-scale through semantic deduplication , author=. 2023 , eprint=

    2023

  101. [109]

    2023 , eprint=

    Deep Learning on a Data Diet: Finding Important Examples Early in Training , author=. 2023 , eprint=

    2023

  102. [110]

    Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , pages =

    Curriculum Learning by Optimizing Learning Dynamics , author =. Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , pages =. 2021 , editor =

    2021

  103. [111]

    2022 , eprint=

    Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognition , author=. 2022 , eprint=

    2022

  104. [112]

    2020 , eprint=

    Neural Tangent Kernel: Convergence and Generalization in Neural Networks , author=. 2020 , eprint=

    2020

  105. [114]

    T ruthful QA : Measuring How Models Mimic Human Falsehoods

    Lin, Stephanie and Hilton, Jacob and Evans, Owain. T ruthful QA : Measuring How Models Mimic Human Falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.229

    2022 doi

  106. [115]

    2024 , eprint=

    Selecting Large Language Model to Fine-tune via Rectified Scaling Law , author=. 2024 , eprint=

    2024

  107. [116]

    2025 , eprint=

    Instruction Tuning for Large Language Models: A Survey , author=. 2025 , eprint=

    2025

  108. [117]

    2024 , eprint=

    Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey , author=. 2024 , eprint=

    2024

  109. [118]

    2025 , eprint=

    Can Pre-training Indicators Reliably Predict Fine-tuning Outcomes of LLMs? , author=. 2025 , eprint=

    2025

  110. [119]

    Submitted to The Fourteenth International Conference on Learning Representations , year=

    TuneAhead: Predicting Fine-tuning Performance Before Training Begins , author=. Submitted to The Fourteenth International Conference on Learning Representations , year=

  111. [120]

    2025 , eprint=

    An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning , author=. 2025 , eprint=

    2025

  112. [121]

    2024 , eprint=

    Revisiting Catastrophic Forgetting in Large Language Model Tuning , author=. 2024 , eprint=

    2024

  113. [122]

    Reconciling modern machine-learning practice and the classical bias–variance trade-off , volume=

    Belkin, Mikhail and Hsu, Daniel and Ma, Siyuan and Mandal, Soumik , year=. Reconciling modern machine-learning practice and the classical bias–variance trade-off , volume=. Proceedings of the National Academy of Sciences , publisher=. doi:10.1073/pnas.1903070116 , number=

    doi

  114. [123]

    2021 , eprint=

    On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines , author=. 2021 , eprint=

    2021

  115. [124]

    2025 , eprint=

    Uncertainty Profiles for LLMs: Uncertainty Source Decomposition and Adaptive Model-Metric Selection , author=. 2025 , eprint=

    2025

  116. [125]

    Uncertainty Quantification and Decomposition for LLM-based Recommendation , url=

    Kweon, Wonbin and Jang, Sanghwan and Kang, SeongKu and Yu, Hwanjo , year=. Uncertainty Quantification and Decomposition for LLM-based Recommendation , url=. doi:10.1145/3696410.3714601 , booktitle=

    doi

  117. [126]

    2017 , eprint=

    What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , author=. 2017 , eprint=

    2017

  118. [127]

    Deep learning in neural networks: An overview , volume=

    Schmidhuber, Jürgen , year=. Deep learning in neural networks: An overview , volume=. doi:10.1016/j.neunet.2014.09.003 , journal=

    2014 doi

  119. [128]

    2025 , eprint=

    Fine-Grained Uncertainty Decomposition in Large Language Models: A Spectral Approach , author=. 2025 , eprint=

    2025

  120. [129]

    2025 , eprint=

    Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey , author=. 2025 , eprint=

    2025

  121. [130]

    2025 , eprint=

    Phase Transitions in Large Language Models and the O(N) Model , author=. 2025 , eprint=

    2025

  122. [131]

    2025 , eprint=

    Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency , author=. 2025 , eprint=

    2025

  123. [132]

    2024 , eprint=

    An Empirical Study of Scaling Laws for Transfer , author=. 2024 , eprint=

    2024

  124. [133]

    International Conference on Learning Representations , year=

    Learning Curve Prediction with Bayesian Neural Networks , author=. International Conference on Learning Representations , year=

  125. [134]

    2026 , eprint=

    Scaling Laws for Downstream Task Performance of Large Language Models , author=. 2026 , eprint=

    2026

  126. [135]

    2025 , eprint=

    Zero-Shot Performance Prediction for Probabilistic Scaling Laws , author=. 2025 , eprint=

    2025

  127. [136]

    2025 , eprint=

    A Generalized Information Bottleneck Theory of Deep Learning , author=. 2025 , eprint=

    2025

  128. [137]

    2019 , eprint=

    Critical Learning Periods in Deep Neural Networks , author=. 2019 , eprint=

    2019

  129. [138]

    International Conference on Learning Representations , year=

    On the Information Bottleneck Theory of Deep Learning , author=. International Conference on Learning Representations , year=

  130. [139]

    2023 , eprint=

    Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning , author=. 2023 , eprint=

    2023

  131. [140]

    2024 , eprint=

    Understanding the Performance and Estimating the Cost of LLM Fine-Tuning , author=. 2024 , eprint=

    2024

  132. [141]

    2025 , eprint=

    Steering LLM Thinking with Budget Guidance , author=. 2025 , eprint=

    2025

  133. [142]

    2024 , eprint=

    Evaluating language models as risk scores , author=. 2024 , eprint=

    2024

  134. [143]

    2023 , eprint=

    It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Risk , author=. 2023 , eprint=

    2023

  135. [144]

    2019 , eprint=

    Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings , author=. 2019 , eprint=

    2019

  136. [145]

    2026 , eprint=

    Measuring Intent Comprehension in LLMs , author=. 2026 , eprint=

    2026

  137. [146]

    On Linear Stochastic Approximation: Fine-grained

    Mou, Wenlong and Li, Chris Junchi and Wainwright, Martin J and Bartlett, Peter L and Jordan, Michael I , booktitle =. On Linear Stochastic Approximation: Fine-grained. 2020 , editor =

    2020

  138. [147]

    2018 , eprint=

    Stochastic Gradient Descent as Approximate Bayesian Inference , author=. 2018 , eprint=

    2018

  139. [148]

    2022 , eprint=

    Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets , author=. 2022 , eprint=

    2022

  140. [149]

    2025 , eprint=

    Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs , author=. 2025 , eprint=

    2025

  141. [150]

    2025 , eprint=

    Power-Law Decay Loss for Large Language Model Finetuning: A Theory Perspective , author=. 2025 , eprint=

    2025

  142. [151]

    2025 , eprint=

    Mitigating Fine-tuning Risks in LLMs via Safety-Aware Probing Optimization , author=. 2025 , eprint=

    2025

  143. [152]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  144. [153]

    CoRR , volume =

    Llama Team , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2407.21783 , eprinttype =. 2407.21783 , timestamp =

    2024 doi arXiv

  145. [154]

    2023 , eprint=

    Mistral 7B , author=. 2023 , eprint=

    2023

  146. [155]

    2019 , eprint=

    GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , author=. 2019 , eprint=

    2019

  147. [156]

    2021 , eprint=

    Training Verifiers to Solve Math Word Problems , author=. 2021 , eprint=

    2021

  148. [157]

    SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization , url=

    Gliwa, Bogdan and Mochol, Iwona and Biesek, Maciej and Wawer, Aleksander , year=. SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization , url=. doi:10.18653/v1/d19-5409 , booktitle=

    doi

  149. [158]

    2019 , eprint=

    Decoupled Weight Decay Regularization , author=. 2019 , eprint=

    2019

  150. [159]

    2021 , eprint=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. 2021 , eprint=

    2021

  151. [160]

    Transformers: State-of-the-Art Natural Language Processing

    Wolf, Thomas and Debut, Lysandre and Sanh, Victor and Chaumond, Julien and Delangue, Clement and Moi, Anthony and Cistac, Pierric and Rault, Tim and Louf, Remi and Funtowicz, Morgan and Davison, Joe and Shleifer, Sam and von Platen, Patrick and Ma, Clara and Jernite, Yacine an...

    2020 doi

  152. [161]

    Meihao Fan and Xiaoyue Han and Ju Fan and Chengliang Chai and Nan Tang and Guoliang Li and Xiaoyong Du , title =. 40th

  153. [162]

    Changlun Li and Chenyu Yang and Yuyu Luo and Ju Fan and Nan Tang , title =. Proc

  154. [163]

    Meihao Fan and Ju Fan and Nan Tang and Lei Cao and Guoliang Li and Xiaoyong Du , title =. Proc

  155. [164]

    Cafarella and Tim Kraska , title =

    Zui Chen and Lei Cao and Sam Madden and Ju Fan and Nan Tang and Zihui Gu and Zeyuan Shang and Chunwei Liu and Michael J. Cafarella and Tim Kraska , title =. CoRR , volume =

  156. [165]

    Yuhao Deng and Chengliang Chai and Lei Cao and Nan Tang and Jiayi Wang and Ju Fan and Ye Yuan and Guoren Wang , title =. Proc

  157. [166]

    Chengliang Chai and Kaisen Jin and Nan Tang and Ju Fan and Dongjing Miao and Jiayi Wang and Yuyu Luo and Guoliang Li and Ye Yuan and Guoren Wang , title =

  158. [167]

    Chengliang Chai and Jiabin Liu and Nan Tang and Ju Fan and Dongjing Miao and Jiayi Wang and Yuyu Luo and Guoliang Li , title =. Proc

  159. [168]

    Chengliang Chai and Jiabin Liu and Nan Tang and Guoliang Li and Yuyu Luo , title =. Proc

  160. [169]

    Yizhang Zhu and Shiyin Du and Boyan Li and Yuyu Luo and Nan Tang , title =. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 , year =

    2024

  161. [170]

    Proceedings of the 31st

    Xinyu Liu and Shuyu Shen and Boyan Li and Nan Tang and Yuyu Luo , title =. Proceedings of the 31st

  162. [171]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,

    Teng Lin and Yuyu Luo and Honglin Zhang and Jicheng Zhang and Chunlin Liu and Kaishun Wu and Nan Tang , title =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,

    2025

  163. [172]

    Xiao Yang and Kai Sun and Hao Xin and Yushi Sun and Nikita Bhalla and Xiangsen Chen and Sajal Choudhary and Rongze Daniel Gui and Ziran Will Jiang and Ziyu Jiang and Lingkun Kong and Brian Moran and Jiaqi Wang and Yifan Xu and An Yan and Chenyu Yang and Eting Yuan and Hanwen Z...

    2024

  164. [173]

    Findings of the Association for Computational Linguistics:

    Yifan Wu and Lutao Yan and Leixian Shen and Yunhai Wang and Nan Tang and Yuyu Luo , title =. Findings of the Association for Computational Linguistics:

  165. [174]

    Chengliang Chai and Kaisen Jin and Nan Tang and Ju Fan and Lianpeng Qiao and Yuping Wang and Yuyu Luo and Ye Yuan and Guoren Wang , title =. 40th. 2024 , url =. doi:10.1109/ICDE60146.2024.00278 , timestamp =

    2024 doi

  166. [175]

    Chenyu Yang and Yuyu Luo and Chuanxuan Cui and Ju Fan and Chengliang Chai and Nan Tang , title =. Proc. 2025 , url =. doi:10.14778/3748191.3748200 , timestamp =

    2025 doi

  167. [176]

    Data Quality Problems beyond Consistency and Deduplication , booktitle =

    Wenfei Fan and Floris Geerts and Shuai Ma and Nan Tang and Wenyuan Yu , editor =. Data Quality Problems beyond Consistency and Deduplication , booktitle =. 2013 , url =. doi:10.1007/978-3-642-41660-6\_12 , timestamp =

    2013 doi

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.