Pith. sign in

REVIEW 2 major objections 1 minor 123 references

TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins

T0 review · 2 major / 1 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TUNEAHEAD predicts LLM fine-tuning performance from dataset features and a short probe run.

desk verdict TuneAhead shows a workable probe-based predictor for fine-tuning outcomes on one model family, but the fixed-probe design leaves the hyperparameter transfer claim under-tested. read the letter →

arxiv 2606.17660 v1 pith:3YMCGVB5 submitted 2026-06-16 cs.LG cs.AI

Yuxiang Luo , Haonan Long , Chen Wang , Qiqi Duan , Xiaotian Lin , Yanwei Xu , Yuyu Luo , Weikai Yang
show 1 more author
Nan Tang
This is my paper · ORCID
classification cs.LGcs.AI
keywords fine-tuningperformancepredictionlargelanguagemodelsmeta-featurevectorsprobefeaturesestimationSHAPattributions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that final fine-tuning outcomes can be estimated before any full training begins. It does so by turning each possible run into a meta-feature vector built from fixed dataset statistics plus measurements collected during one brief standardized probe. A predictor then turns that vector into a performance number, and the whole system is checked against more than 1300 actual fine-tuning experiments on Qwen2.5-7B-Instruct. A sympathetic reader would care because fine-tuning is costly and can sometimes make models worse; early accurate forecasts could let practitioners avoid many wasted runs. The same features also supply explanations for why any given prediction is high or low.

What carries the argument

The meta-feature vector that combines static dataset descriptors with dynamic features from a short standardized probe and feeds them to a predictor for performance estimates.

What would settle it

Applying the same predictor to a fresh collection of fine-tuning runs on different models or datasets and obtaining RMSE well above 1.47 percentage points would show the method does not generalize.

Watch

Extended reading notes

Core claim

TUNEAHEAD encodes each candidate run as a meta-feature vector that combines static dataset descriptors with dynamic probe features from a short standardized probe. A predictor maps these features to performance estimates. Across 1,300+ fine-tuning runs on Qwen2.5-7B-Instruct, TUNEAHEAD consistently outperforms strong baselines such as Early-Stop Extrapolation and ProxyLM. On a held-out test set of 370 runs, TUNEAHEAD achieves an RMSE of 1.47 percentage points and places 95.1% of predictions within +3/-3 percentage points of the true score.

Load-bearing premise

Static dataset descriptors together with measurements from one short probe already contain enough information to forecast final performance no matter how the data quality or hyperparameters vary.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper introduces TUNEAHEAD, a lightweight framework for pre-hoc prediction of LLM fine-tuning performance. Each candidate run is encoded as a meta-feature vector combining static dataset descriptors with dynamic features extracted from a short standardized probe; a meta-predictor then maps these features to an estimated final performance score, accompanied by SHAP-based attributions. On more than 1,300 fine-tuning runs of Qwen2.5-7B-Instruct the method outperforms baselines such as Early-Stop Extrapolation and ProxyLM; on a held-out test set of 370 runs it reports RMSE = 1.47 percentage points and places 95.1 % of predictions inside a ±3 percentage-point band.

Significance. If the reported accuracy generalizes, the approach could materially reduce wasted compute by enabling early go/no-go screening of fine-tuning configurations. The evaluation scale (>1,300 runs) and the explicitly non-circular feature construction (dataset descriptors plus an independent short probe) are concrete strengths. The provision of SHAP diagnostics adds practical interpretability that is often absent from pure black-box predictors.

major comments (2)
  1. [Abstract] Abstract: the claim that predictions succeed 'across varied … hyperparameter choices without requiring the full training trajectory' is load-bearing, yet the probe is described as 'standardized' (fixed settings). If the dynamic features (early loss, gradient norms, etc.) are collected under a single fixed learning rate/batch size/optimizer, they cannot reflect hyperparameter-specific convergence behavior; the low RMSE on the held-out set would then be consistent with learning correlations inside the probe regime rather than true transfer across HP regimes.
  2. [Evaluation] Evaluation section (implied by the 370-run held-out test set): no information is given on how the train/test split was constructed with respect to hyperparameter diversity or dataset similarity. Without explicit stratification or leakage controls, the 1.47 RMSE and 95.1 % within-band statistic cannot be taken as evidence of generalization across the very hyperparameter variations the central claim asserts.
minor comments (1)
  1. [Abstract] Abstract contains a typographic artifact ('na"ive'); standard spelling is 'naive'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. The comments highlight important aspects of our claims and evaluation design. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim that predictions succeed 'across varied … hyperparameter choices without requiring the full training trajectory' is load-bearing, yet the probe is described as 'standardized' (fixed settings). If the dynamic features (early loss, gradient norms, etc.) are collected under a single fixed learning rate/batch size/optimizer, they cannot reflect hyperparameter-specific convergence behavior; the low RMSE on the held-out set would then be consistent with learning correlations inside the probe regime rather than true transfer across HP regimes.

    Authors: We appreciate the referee pointing out this distinction. The probe is run under fixed, standardized settings to ensure consistent and low-cost feature extraction that does not depend on the target run's hyperparameters. The meta-predictor is trained across a collection of runs that themselves use varied hyperparameters, so it learns to associate the resulting (dataset + fixed-probe) feature vectors with the final performance achieved under those specific hyperparameters. We agree, however, that the features themselves do not encode hyperparameter-specific dynamics. To prevent any overstatement of the generalization claim, we will revise the abstract to clarify that predictions are made for the hyperparameter configurations observed in the training data, using static descriptors plus a fixed probe, rather than implying direct transfer to arbitrary unseen hyperparameter regimes. revision: yes

  2. Referee: [Evaluation] Evaluation section (implied by the 370-run held-out test set): no information is given on how the train/test split was constructed with respect to hyperparameter diversity or dataset similarity. Without explicit stratification or leakage controls, the 1.47 RMSE and 95.1 % within-band statistic cannot be taken as evidence of generalization across the very hyperparameter variations the central claim asserts.

    Authors: The referee is correct that the manuscript currently lacks explicit details on the train/test split procedure with respect to hyperparameter diversity and dataset similarity. We will add a dedicated paragraph in the evaluation section describing how the 370-run held-out set was constructed, including any stratification by key hyperparameters (learning rate, batch size, etc.) and controls for dataset overlap or similarity to ensure the reported metrics reflect generalization across the hyperparameter variations present in the data. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: predictions use independent short-probe features

full rationale

The method constructs meta-feature vectors from static dataset descriptors plus dynamic signals extracted from a short standardized probe run; a separate predictor then maps these to performance estimates. No equation or step in the described pipeline reduces the output to a quantity defined by the full training trajectory itself. Evaluation on a held-out set of 370 runs is performed against actual full-run scores, preserving independence. No self-citation load-bearing steps or fitted-input-as-prediction patterns are present in the provided description.

Assumptions & free parameters 1 free parameters · 1 assumptions · 0 invented entities

The framework rests on the domain assumption that short probes capture predictive signal about full runs and that the chosen meta-features generalize; no free parameters or invented entities are explicitly introduced in the abstract.

free parameters (1)
  • meta-predictor parameters
    The performance predictor is trained on the collected fine-tuning runs, so its internal weights are fitted to data.
assumptions (1)
  • domain assumption Short standardized probe features correlate sufficiently with full fine-tuning outcomes
    Invoked when the method uses probe data to stand in for complete training trajectories.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins." pith.science (2026). https://pith.science/paper/3YMCGVB5

@misc{pith2026260617660,
  author       = {Pith},
  title        = {Pith review of: TuneAhead: Predicting Fine-tuning Performance Before Full Training Begins},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3YMCGVB5}},
  note         = {Machine review of arXiv:2606.17660}
}
read the original abstract

Fine-tuning large language models (LLMs) is compute-intensive and error-prone: model performance depends sensitively on data quality and hyperparameter choices, and na\"ive runs can even degrade model performance. This raises a practical question:can we predict fine-tuning performance before committing to a full training run? We present TUNEAHEAD, a lightweight framework for pre-hoc prediction of fine-tuning performance. TUNEAHEAD encodes each candidate run as a meta-feature vector that combines static dataset descriptors with dynamic probe features from a short standardized probe. A predictor maps these features to performance estimates, while SHAP-based attributions provide interpretable diagnostics that reveal which specific features drive the prediction. Across 1,300+ fine-tuning runs on Qwen2.5-7B-Instruct, TUNEAHEAD consistently outperforms strong baselines such as Early-Stop Extrapolation and ProxyLM. On a held-out test set of 370 runs, TUNEAHEAD achieves an RMSE of 1.47 percentage points and places 95.1% of predictions within +3/-3 percentage points of the true score. These accurate continuous predictions support practical go/no-go screening policies that can reduce unnecessary full fine-tuning while retaining most promising runs.

Figures

Figures reproduced from arXiv: 2606.17660 by the authors.

Figure 1
Figure 1. Predicting fine-tuning performance: (A) Without TUNEAHEAD: failed runs are only identified after training, wasting computational resources and time. (B) With TUNEAHEAD: low-cost features predict performance in advance, enabling go/no-go decisions and diagnosis for the failure cases. (A) Without TuneAhead 1 2 3 4 5 6 7 8 9 10 Total compute (no prediction) — 30h (B) With TuneAhead 1 2 4 7 9 Overhead + compute (with Tu… view at source ↗
Figure 2
Figure 2. Compute time for 10 runs without TUNEAHEAD (A) vs. with TUNEAHEAD (B). With TUNEAHEAD (see [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. TUNEAHEAD Overview. Stage 1 (Meta-dataset curation) builds meta-feature vectors Vi,j by combining static features with dynamic features. Stage 2 (Predictive & Diagnostic Modeling) maps Vi,j to performance predictions and uses SHAP for diagnostics. tuning configuration. We therefore run a standardized 100- step probe for each dataset–hyperparameter pair (Di , Hj ). The resulting dynamic features should be interpreted… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Predicted vs True accuracy across methods. The diagonal line (y=x) indicates a perfect prediction [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: (a) SHAP summary plot ranking the global importance of meta-features for predicting fine-tuning success; (b) SHAP waterfall plot for a representative failure case (the model correctly predicted low performance). contribution of TUNEAHEAD is its ability to provide diag￾…
Figure 6
Figure 6. Figure 6: Effect of probe length on prediction accuracy, stability, and time cost. (a) Accuracy at 2pp steadily improves with longer probe runs but exhibits diminishing returns beyond 100 steps. (b) RMSE decreases sharply in the early stage and stabilizes after 100 steps. (c) Av…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

123 extracted references · 14 canonical work pages

  1. [1]

    Scaling Learning Algorithms Towards

    Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards

  2. [2]

    and Osindero, Simon and Teh, Yee Whye , journal =

    Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume =

  3. [3]

    2016 , publisher=

    Deep learning , author=. 2016 , publisher=

    2016

  4. [4]

    2025 , eprint=

    COSMOS: Predictable and Cost-Effective Adaptation of LLMs , author=. 2025 , eprint=

    2025

  5. [5]

    2024 , eprint=

    Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models , author=. 2024 , eprint=

    2024

  6. [6]

    2025 , eprint=

    LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection , author=. 2025 , eprint=

    2025

  7. [7]

    2017 , eprint=

    A Unified Approach to Interpreting Model Predictions , author=. 2017 , eprint=

    2017

  8. [8]

    2020 , eprint=

    Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples , author=. 2020 , eprint=

    2020

Show all 123 references
  1. [9]

    2021 , eprint=

    Dataset2Vec: Learning Dataset Meta-Features , author=. 2021 , eprint=

    2021

  2. [10]

    2022 , eprint=

    Model Zoos: A Dataset of Diverse Populations of Neural Network Models , author=. 2022 , eprint=

    2022

  3. [11]

    Proceedings of the 24th International Conference on Artificial Intelligence , pages =

    Domhan, Tobias and Springenberg, Jost Tobias and Hutter, Frank , title =. Proceedings of the 24th International Conference on Artificial Intelligence , pages =. 2015 , isbn =

    2015

  4. [12]

    2020 , eprint=

    Scaling Laws for Neural Language Models , author=. 2020 , eprint=

    2020

  5. [13]

    2024 , eprint=

    ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models , author=. 2024 , eprint=

    2024

  6. [14]

    2020 , eprint=

    Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics , author=. 2020 , eprint=

    2020

  7. [15]

    2019 , eprint=

    Data Shapley: Equitable Valuation of Data for Machine Learning , author=. 2019 , eprint=

    2019

  8. [16]

    2025 , eprint=

    Refine-n-Judge: Curating High-Quality Preference Chains for LLM-Fine-Tuning , author=. 2025 , eprint=

    2025

  9. [17]

    2018 , eprint=

    The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards , author=. 2018 , eprint=

    2018

  10. [18]

    2021 , eprint=

    MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers , author=. 2021 , eprint=

    2021

  11. [19]

    2023 , eprint=

    GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation , author=. 2023 , eprint=

    2023

  12. [20]

    2021 , eprint=

    On the Importance of Gradients for Detecting Distributional Shifts in the Wild , author=. 2021 , eprint=

    2021

  13. [21]

    2018 , eprint=

    GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks , author=. 2018 , eprint=

    2018

  14. [22]

    Louis , year=

    Shuofeng Zhang and Isaac Reid and Guillermo Valle-Perez and Ard A. Louis , year=

  15. [23]

    2024 , url=

    Gradient norm as a powerful proxy to out-of-distribution error estimation , author=. 2024 , url=

    2024

  16. [24]

    2023 , eprint=

    Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization , author=. 2023 , eprint=

    2023

  17. [25]

    2021 , eprint=

    Self-Validation: Early Stopping for Single-Instance Deep Generative Priors , author=. 2021 , eprint=

    2021

  18. [26]

    2025 , eprint=

    Autoencoder-Based Framework to Capture Vocabulary Quality in NLP , author=. 2025 , eprint=

    2025

  19. [27]

    2025 , eprint=

    Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality , author=. 2025 , eprint=

    2025

  20. [28]

    2025 , eprint=

    Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models , author=. 2025 , eprint=

    2025

  21. [29]

    Model Explainability using SHAP Values for LightGBM Predictions , year=

    Bugaj, Michal and Wrobel, Krzysztof and Iwaniec, Joanna , booktitle=. Model Explainability using SHAP Values for LightGBM Predictions , year=

  22. [30]

    Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =

    Ke, Guolin and Meng, Qi and Finley, Thomas and Wang, Taifeng and Chen, Wei and Ma, Weidong and Ye, Qiwei and Liu, Tie-Yan , title =. Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =. 2017 , isbn =

    2017

  23. [31]

    2025 , issue_date =

    Wang, Ying and Wang, Pengxin and Tansey, Kevin and Liu, Junming and Delaney, Bethany and Quan, Wenting , title =. 2025 , issue_date =. doi:10.1016/j.compag.2024.109758 , journal =

    2025 doi

  24. [32]

    Garcia and Carlos Soares and Joaquin Vanschoren and André C.P.L.F

    Adriano Rivolli and Luís P.F. Garcia and Carlos Soares and Joaquin Vanschoren and André C.P.L.F. Meta-features for meta-learning , journal =. 2022 , issn =. doi:https://doi.org/10.1016/j.knosys.2021.108101 , url =

    2022 doi

  25. [33]

    2022 , eprint=

    Predicting Fine-Tuning Performance with Probing , author=. 2022 , eprint=

    2022

  26. [34]

    2024 , eprint=

    Predictable Emergent Abilities of LLMs: Proxy Tasks Are All You Need , author=. 2024 , eprint=

    2024

  27. [35]

    2020 , eprint=

    The Break-Even Point on Optimization Trajectories of Deep Neural Networks , author=. 2020 , eprint=

    2020

  28. [36]

    2019 , eprint=

    Visualizing and Understanding the Effectiveness of BERT , author=. 2019 , eprint=

    2019

  29. [37]

    2023 , eprint=

    Efficient Bayesian Learning Curve Extrapolation using Prior-Data Fitted Networks , author=. 2023 , eprint=

    2023

  30. [38]

    International Conference on Learning Representations (ICLR) , year =

    Measuring Massive Multitask Language Understanding , author =. International Conference on Learning Representations (ICLR) , year =

  31. [39]

    arXiv preprint arXiv:2412.15115 , year =

    Qwen2.5 Technical Report , author =. arXiv preprint arXiv:2412.15115 , year =. doi:10.48550/arXiv.2412.15115 , url =

    doi arXiv

  32. [40]

    2025 , url =

    Qwen2.5-7B-Instruct , author =. 2025 , url =

    2025

  33. [41]

    2025 , eprint=

    A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules , author=. 2025 , eprint=

    2025

  34. [42]

    2020 , eprint=

    Don't Stop Pretraining: Adapt Language Models to Domains and Tasks , author=. 2020 , eprint=

    2020

  35. [43]

    Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research

    Mu, Yida and Jin, Mali and Song, Xingyi and Aletras, Nikolaos. Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:...

    2024 doi

  36. [44]

    2025 , eprint=

    Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric , author=. 2025 , eprint=

    2025

  37. [45]

    2023 , eprint=

    The Vendi Score: A Diversity Evaluation Metric for Machine Learning , author=. 2023 , eprint=

    2023

  38. [46]

    2018 , eprint=

    Visualizing the Loss Landscape of Neural Nets , author=. 2018 , eprint=

    2018

  39. [47]

    2020 , eprint=

    Adversarial Weight Perturbation Helps Robust Generalization , author=. 2020 , eprint=

    2020

  40. [48]

    Mathematics , VOLUME =

    Liang, Hailun and Zheng, Haowen and Wang, Hao and He, Liu and Lin, Haoyi and Liang, Yanyan , TITLE =. Mathematics , VOLUME =. 2025 , NUMBER =

    2025

  41. [49]

    Technometrics , volume =

    Ridge Regression: Biased Estimation for Nonorthogonal Problems , author =. Technometrics , volume =. 1970 , publisher =

    1970

  42. [50]

    Advances in Neural Information Processing Systems , volume =

    Support Vector Regression Machines , author =. Advances in Neural Information Processing Systems , volume =. 1997 , publisher =

    1997

  43. [51]

    Statistics and Computing , volume =

    A Tutorial on Support Vector Regression , author =. Statistics and Computing , volume =. 2004 , doi =

    2004

  44. [52]

    Machine Learning , volume =

    Random Forests , author =. Machine Learning , volume =. 2001 , doi =

    2001

  45. [53]

    Nature , volume =

    Learning Representations by Back-Propagating Errors , author =. Nature , volume =. 1986 , doi =

    1986

  46. [54]

    Neural Networks , volume =

    Multilayer Feedforward Networks Are Universal Approximators , author =. Neural Networks , volume =. 1989 , doi =

    1989

  47. [55]

    Journal of Machine Learning Research , volume =

    Scikit-learn: Machine Learning in Python , author =. Journal of Machine Learning Research , volume =. 2011 , url =

    2011

  48. [56]

    arXiv preprint arXiv:1706.10239 , year=

    Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes , author=. arXiv preprint arXiv:1706.10239 , year=

    arXiv

  49. [57]

    arXiv preprint arXiv:2412.13573 , year=

    Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes , author=. arXiv preprint arXiv:2412.13573 , year=

  50. [58]

    2019 , eprint=

    Characterizing classification datasets: a study of meta-features for meta-learning , author=. 2019 , eprint=

    2019

  51. [59]

    2024 , eprint=

    Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets , author=. 2024 , eprint=

    2024

  52. [60]

    2023 , eprint=

    Advances and Challenges in Meta-Learning: A Technical Review , author=. 2023 , eprint=

    2023

  53. [61]

    2018 , eprint=

    Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization , author=. 2018 , eprint=

    2018

  54. [62]

    ROUGE : A Package for Automatic Evaluation of Summaries

    Lin, Chin-Yew. ROUGE : A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out. 2004

    2004

  55. [63]

    2018 , eprint=

    A Survey of Machine Learning for Big Code and Naturalness , author=. 2018 , eprint=

    2018

  56. [64]

    , title =

    Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J. , title =. J. Mach. Learn. Res. , month = jan, articleno =. 2020 , issue_date =

    2020

  57. [65]

    2022 , eprint=

    Deduplicating Training Data Makes Language Models Better , author=. 2022 , eprint=

    2022

  58. [66]

    2023 , eprint=

    Extracting Training Data from Diffusion Models , author=. 2023 , eprint=

    2023

  59. [67]

    2022 , eprint=

    Training Compute-Optimal Large Language Models , author=. 2022 , eprint=

    2022

  60. [68]

    2019 , eprint=

    Deep Anomaly Detection with Outlier Exposure , author=. 2019 , eprint=

    2019

  61. [69]

    2022 , eprint=

    Confident Learning: Estimating Uncertainty in Dataset Labels , author=. 2022 , eprint=

    2022

  62. [70]

    2021 , eprint=

    Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus , author=. 2021 , eprint=

    2021

  63. [71]

    2020 , eprint=

    The Curious Case of Neural Text Degeneration , author=. 2020 , eprint=

    2020

  64. [72]

    A Qualitative Comparison of C o QA , SQ u AD 2.0 and Q u AC

    Yatskar, Mark. A Qualitative Comparison of C o QA , SQ u AD 2.0 and Q u AC. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019. doi:10.18653/v1/N19-1241

    2019 doi

  65. [73]

    2016 , eprint=

    Exploring the Limits of Language Modeling , author=. 2016 , eprint=

    2016

  66. [74]

    2018 , eprint=

    Universal Sentence Encoder , author=. 2018 , eprint=

    2018

  67. [75]

    2020 , eprint=

    ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators , author=. 2020 , eprint=

    2020

  68. [76]

    2016 , eprint=

    A Diversity-Promoting Objective Function for Neural Conversation Models , author=. 2016 , eprint=

    2016

  69. [77]

    2020 , eprint=

    Fine-Tuning Language Models from Human Preferences , author=. 2020 , eprint=

    2020

  70. [78]

    2020 , eprint=

    Unsupervised Domain Clusters in Pretrained Language Models , author=. 2020 , eprint=

    2020

  71. [79]

    Domain Adaptation via Pseudo In-Domain Data Selection

    Axelrod, Amittai and He, Xiaodong and Gao, Jianfeng. Domain Adaptation via Pseudo In-Domain Data Selection. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 2011

    2011

  72. [80]

    Survey of Hallucination in Natural Language Generation , volume=

    Ji, Ziwei and Lee, Nayeon and Frieske, Rita and Yu, Tiezheng and Su, Dan and Xu, Yan and Ishii, Etsuko and Bang, Ye Jin and Madotto, Andrea and Fung, Pascale , year=. Survey of Hallucination in Natural Language Generation , volume=. ACM Computing Surveys , publisher=. doi:10.1...

    doi

  73. [81]

    2020 , eprint=

    Reducing Quantity Hallucinations in Abstractive Summarization , author=. 2020 , eprint=

    2020

  74. [82]

    2017 , eprint=

    Deep Learning Scaling is Predictable, Empirically , author=. 2017 , eprint=

    2017

  75. [83]

    2017 , eprint=

    A Closer Look at Memorization in Deep Networks , author=. 2017 , eprint=

    2017

  76. [84]

    2022 , eprint=

    A Survey of Learning Curves with Bad Behavior: or How More Data Need Not Lead to Better Performance , author=. 2022 , eprint=

    2022

  77. [85]

    2013 , eprint=

    On the difficulty of training Recurrent Neural Networks , author=. 2013 , eprint=

    2013

  78. [86]

    Proceedings of the 30th International Conference on Machine Learning , pages =

    On the importance of initialization and momentum in deep learning , author =. Proceedings of the 30th International Conference on Machine Learning , pages =. 2013 , editor =

    2013

  79. [87]

    2019 , eprint=

    Towards Understanding Generalization in Gradient-Based Meta-Learning , author=. 2019 , eprint=

    2019

  80. [88]

    2021 , eprint=

    GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training , author=. 2021 , eprint=

    2021

  81. [89]

    2017 , eprint=

    Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , author=. 2017 , eprint=

    2017

  82. [90]

    2019 , eprint=

    The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks , author=. 2019 , eprint=

    2019

  83. [91]

    2021 , eprint=

    Rigging the Lottery: Making All Tickets Winners , author=. 2021 , eprint=

    2021

  84. [92]

    2020 , eprint=

    Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers , author=. 2020 , eprint=

    2020

  85. [93]

    2017 , eprint=

    SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability , author=. 2017 , eprint=

    2017

  86. [94]

    2017 , eprint=

    On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima , author=. 2017 , eprint=

    2017

  87. [95]

    2018 , eprint=

    Overcoming catastrophic forgetting problem by weight consolidation and long-term memory , author=. 2018 , eprint=

    2018

  88. [96]

    Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =

    Deep Sparse Rectifier Neural Networks , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =. 2011 , editor =

    2011

  89. [97]

    Networks of spiking neurons: The third generation of neural network models , journal =

    Wolfgang Maass , keywords =. Networks of spiking neurons: The third generation of neural network models , journal =. 1997 , issn =. doi:https://doi.org/10.1016/S0893-6080(97)00011-7 , url =

    1997 doi

  90. [98]

    Predicting Fine-tuned Performance on Larger Datasets Before Creating Them

    Kuramoto, Toshiki and Suzuki, Jun. Predicting Fine-tuned Performance on Larger Datasets Before Creating Them. Proceedings of the 31st International Conference on Computational Linguistics: Industry Track. 2025

    2025

  91. [99]

    2023 , eprint=

    SemDeDup: Data-efficient learning at web-scale through semantic deduplication , author=. 2023 , eprint=

    2023

  92. [100]

    2023 , eprint=

    Deep Learning on a Data Diet: Finding Important Examples Early in Training , author=. 2023 , eprint=

    2023

  93. [101]

    Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , pages =

    Curriculum Learning by Optimizing Learning Dynamics , author =. Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , pages =. 2021 , editor =

    2021

  94. [102]

    2022 , eprint=

    Sentence-Select: Large-Scale Language Model Data Selection for Rare-Word Speech Recognition , author=. 2022 , eprint=

    2022

  95. [103]

    2020 , eprint=

    Neural Tangent Kernel: Convergence and Generalization in Neural Networks , author=. 2020 , eprint=

    2020

  96. [104]

    SAMS um Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization

    Gliwa, Bogdan and Mochol, Iwona and Biesek, Maciej and Wawer, Aleksander. SAMS um Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization. Proceedings of the 2nd Workshop on New Frontiers in Summarization. 2019. doi:10.18653/v1/D19-5409

    2019 doi

  97. [105]

    T ruthful QA : Measuring How Models Mimic Human Falsehoods

    Lin, Stephanie and Hilton, Jacob and Evans, Owain. T ruthful QA : Measuring How Models Mimic Human Falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.229

    2022 doi

  98. [106]

    2024 , eprint=

    Selecting Large Language Model to Fine-tune via Rectified Scaling Law , author=. 2024 , eprint=

    2024

  99. [107]

    Chengliang Chai and Kaisen Jin and Nan Tang and Ju Fan and Lianpeng Qiao and Yuping Wang and Yuyu Luo and Ye Yuan and Guoren Wang , title =. 40th. 2024 , url =. doi:10.1109/ICDE60146.2024.00278 , timestamp =

    2024 doi

  100. [108]

    Chenyu Yang and Yuyu Luo and Chuanxuan Cui and Ju Fan and Chengliang Chai and Nan Tang , title =. Proc. 2025 , url =. doi:10.14778/3748191.3748200 , timestamp =

    2025 doi

  101. [109]

    Data Quality Problems beyond Consistency and Deduplication , booktitle =

    Wenfei Fan and Floris Geerts and Shuai Ma and Nan Tang and Wenyuan Yu , editor =. Data Quality Problems beyond Consistency and Deduplication , booktitle =. 2013 , url =. doi:10.1007/978-3-642-41660-6\_12 , timestamp =

    2013 doi

  102. [110]

    Chengliang Chai and Jiabin Liu and Nan Tang and Ju Fan and Dongjing Miao and Jiayi Wang and Yuyu Luo and Guoliang Li , title =. Proc

  103. [111]

    Chengliang Chai and Jiabin Liu and Nan Tang and Guoliang Li and Yuyu Luo , title =. Proc

  104. [112]

    Chengliang Chai and Kaisen Jin and Nan Tang and Ju Fan and Dongjing Miao and Jiayi Wang and Yuyu Luo and Guoliang Li and Ye Yuan and Guoren Wang , title =

  105. [113]

    Yuhao Deng and Chengliang Chai and Lei Cao and Nan Tang and Jiayi Wang and Ju Fan and Ye Yuan and Guoren Wang , title =. Proc

  106. [114]

    Xiaotian Lin and Yanlin Qi and Yizhang Zhu and Themis Palpanas and Chengliang Chai and Nan Tang and Yuyu Luo , title =. Proc. 2025 , url =

    2025

  107. [115]

    Meihao Fan and Xiaoyue Han and Ju Fan and Chengliang Chai and Nan Tang and Guoliang Li and Xiaoyong Du , title =. 40th

  108. [116]

    Changlun Li and Chenyu Yang and Yuyu Luo and Ju Fan and Nan Tang , title =. Proc

  109. [117]

    Meihao Fan and Ju Fan and Nan Tang and Lei Cao and Guoliang Li and Xiaoyong Du , title =. Proc

  110. [118]

    Cafarella and Tim Kraska , title =

    Zui Chen and Lei Cao and Sam Madden and Ju Fan and Nan Tang and Zihui Gu and Zeyuan Shang and Chunwei Liu and Michael J. Cafarella and Tim Kraska , title =. CoRR , volume =

  111. [119]

    Yizhang Zhu and Shiyin Du and Boyan Li and Yuyu Luo and Nan Tang , title =. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 , year =

    2024

  112. [120]

    Proceedings of the 31st

    Xinyu Liu and Shuyu Shen and Boyan Li and Nan Tang and Yuyu Luo , title =. Proceedings of the 31st

  113. [121]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,

    Teng Lin and Yuyu Luo and Honglin Zhang and Jicheng Zhang and Chunlin Liu and Kaishun Wu and Nan Tang , title =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,

    2025

  114. [122]

    Xiao Yang and Kai Sun and Hao Xin and Yushi Sun and Nikita Bhalla and Xiangsen Chen and Sajal Choudhary and Rongze Daniel Gui and Ziran Will Jiang and Ziyu Jiang and Lingkun Kong and Brian Moran and Jiaqi Wang and Yifan Xu and An Yan and Chenyu Yang and Eting Yuan and Hanwen Z...

    2024

  115. [123]

    Findings of the Association for Computational Linguistics:

    Yifan Wu and Lutao Yan and Leixian Shen and Yunhai Wang and Nan Tang and Yuyu Luo , title =. Findings of the Association for Computational Linguistics:

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.