From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning

Haoyu Li; Kun Liu; Xuhong Li; Yiming Dong

REVIEW 4 major objections 7 minor 59 references

From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that controlling the variety of mid-frequency tokens in model responses is a more effective lever for fine-tuned LLM quality than instruction-level diversity controls, with maximum token diversity producing the best models.

desk verdict A useful taxonomy and a plausible response-diversity effect, but the effect is entangled with sequence length and the single-run design leaves the headline claim under-supported. read the letter →

arxiv 2505.24768 v1 pith:RLNMVC3E submitted 2025-05-30 cs.CL

Haoyu Li , Xuhong Li , Yiming Dong , Kun Liu This is my paper

classification cs.CL

keywords datasetdiversitysupervisedfine-tuningtoken-levelmid-bandtokensresponseconstructioninformationentropyinstructiontuning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Supervised fine-tuning (SFT) makes a language model follow instructions, and dataset diversity is usually controlled on the instruction side by choosing more topics or more intent tags. This paper tries to establish that the response side and the finest grain matter more: the variety of mid-frequency tokens in the generated answers. Starting from 117K instruction-response pairs with responses regenerated for quality control, the authors build fixed-size datasets (10K, 20K, 30K) at six diversity-control settings spanning macro, meso, and micro levels on both instructions and responses, then fine-tune models on each. They report that increasing response token diversity gives a stronger and more consistent performance gain than any instruction-side strategy, and that maximum token diversity yields the best models overall. If this is right, constructing high-performance SFT datasets should treat the token distribution of responses as a primary design axis, not just a by-product of instruction selection.

What carries the argument

The central object is the 'mid band' of the token frequency spectrum: tokens whose corpus counts lie between 10 and 500, which a manual inspection identifies as the carriers of semantic content, between high-frequency function words and low-frequency names, loanwords, and typos. Two algorithms make diversity controllable at this grain: inverse greedy pruning removes samples that contribute the fewest unique mid-band tokens until a target token-type count is reached, and token-aware sampling then selects samples to reach the fixed dataset size while keeping all target token types covered. The connecting mechanism is token-level exposure during autoregressive training: because every output token is a supervised training signal, a flatter and broader token distribution in responses is proposed to reduce overfitting and improve generalization.

What would settle it

A decisive test is to build two fixed-size response datasets matched on instruction set, response length distribution, topic mixture, and per-sample quality, differing only in mid-band token coverage, and check whether the higher-coverage set still wins; if the performance gap vanishes once those covariates are matched, the mid-band diversity claim is not causal. A second check is threshold invariance: shifting the band boundaries (for example to 5–1000) should preserve the effect if the band is the true carrier of semantic diversity.

Watch

Extended reading notes

Core claim

The paper's central claim is that the microscopic strategy—controlling how many distinct mid-band tokens appear in the responses of an SFT dataset—is the strongest diversity lever available, stronger than macroscopic topic-semantics diversity and stronger than mesoscopic tag diversity, on both the instruction and response components. Evidence comes from fixed-size datasets with diversity scaled from minimal to maximal under each strategy; the response-token strategy has the steepest positive slope between diversity percentage and benchmark score across three dataset sizes and across three model families in ablations, and at 100% diversity it reaches the highest scores in the comparison. The authors' explanation is direct: language models are supervised on response tokens and decode token by token, so broadening the token distribution in responses mitigates overfitting and makes the trained model generalize across varied tokens. They also show that the effect is not explained by response length, that it requires the model's own tokenizer rather than word segmentation, and that response information entropy—the flattest token distribution—correlates most with performance.

Load-bearing premise

The load-bearing assumption is that the 'mid band' of tokens—those appearing between 10 and 500 times in the corpus—carries most of the semantic meaning, so that increasing their variety is a clean intervention on diversity; if that threshold is arbitrary, or if the pruning and sampling also change response length, topic mix, or response quality, then the observed performance gains could come from a confound rather than from token diversity itself.

Editorial extensions

If this is right

Fixed-size SFT datasets can be sorted by marginal mid-band token coverage, and curators who select for that coverage should expect higher downstream performance than curators who select for topic or tag variety alone.
A smaller but token-diverse dataset can outperform a larger but less diverse one, so response token diversity is a concrete way to train stronger models under limited data or compute budgets.
Instruction-side diversity remains useful, but its achievable score range is narrower; topic and tag diversity should be treated as complements rather than substitutes for response-token diversity.
Tokenizer choice is part of the method: diversity measured with word segmentation does not produce the same gain, so the target model's own tokenizer should be used when constructing the training set.
Maximizing mid-band token coverage is a safe default: across the tested sizes and model families, the highest diversity level consistently gives the best or near-best performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the paper's own correlation data point to response information entropy as an even more direct objective than mid-band coverage; optimizing full-vocabulary entropy rather than mid-band token types might further improve performance and deserves a head-to-head test.
Editorial inference: the 10-to-500 threshold is likely corpus- and tokenizer-dependent; if the mechanism is causal, the correct band should scale with corpus size and vocabulary, and a principled rule for setting it would make the method portable to other data regimes.
Editorial inference: because response-token diversity and instruction diversity appear to act independently, a combined strategy that controls both axes at once may produce compounding gains, which the paper leaves as future work.
Editorial inference: the strongest practical test is whether token-diverse response selection also improves performance when responses come from the original, lower-quality data sources rather than regenerated ones; the paper reports an ablation with original responses, but a systematic comparison across quality levels would settle the scope.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: the paper gives you a useful taxonomy and a plausible direction, but the headline claim—that response-level token diversity beats instruction-level strategies—is not as clean as the writing suggests, because sequence length is a serious confound and no point has an error bar.

What is actually new: the macro/meso/micro framing applied to both sides of the SFT pair is a genuinely convenient organizing scheme, and the token-level response control algorithm (inverse greedy pruning plus token-aware sampling) is concrete and usable. Comparing six strategies in one fixed pipeline is helpful for anyone building SFT datasets. The authors also tried to address the obvious length confound with an ablation, and they add alternative judge models and benchmarks in the appendix. That is real effort.

Where it falls short. First, the length issue is front and center: their own Figure 4 gives sequence length a 0.73 correlation with score, higher than any diversity metric. The microscopic selection procedure inevitably favors longer responses, since long texts contain more of the rare mid-band tokens the algorithm chases. The Section 4.2 length-control ablation restricts to roughly 500 tokens, but the description and Figure 6(a) subplot do not show that length distributions are matched across diversity levels; it's at best a partial control. Second, everything is single-run. The Bayesian regression shading in Figure 3 is fitted to one observation per condition, so the slope differences in Table 2 have no uncertainty attached. Third, the judge is Llama-3.1-70B-Nemotron, the same model family that generated the training responses; the two alternative judges are tested on one condition only, which mitigates the concern but does not eliminate it. The 10–500 mid-band threshold comes from manual inspection and is not stress-tested. The information-entropy confirmation is somewhat circular since the controlled quantity is a subset of what entropy measures—minor, because the core performance-diversity correlation is empirical. The authors themselves, in Appendix F, admit the entropy-diversity relationship needs more work.

Who this is for: dataset builders and anyone active in SFT data selection. It deserves a serious referee, but the referee should require multiple seeds, significance tests on the slopes, a rigorous matched-length control, and judge evaluation on all main conditions with an outside judge. With those, this could be a solid contribution.

Referee Report

4 major / 7 minor

Summary. The paper investigates dataset diversity in supervised fine-tuning (SFT) of large language models, organizing diversity-control strategies into three granularities—macroscopic (semantic clustering), mesoscopic (tagging), and microscopic (token-level)—and applying them to both the instruction and response components of SFT data. It proposes a new microscopic response-diversity method based on the variety of mid-frequency tokens, constructs fixed-size datasets (10K, 20K, 30K) at diversity scales from minimal to maximal, fine-tunes Llama-2-7B (plus Llama-3-8B and Llama-2-70B in ablations), and evaluates with an LLM judge. The central empirical claim is that the microscopic response strategy shows a stronger correlation between diversity and model performance and achieves the best performance at maximum diversity compared with all other strategies. The paper also analyzes posterior diversity metrics and reports ablations on model family, tokenizer, length control, and additional benchmarks.

Significance. If the central finding is upheld, the paper provides a practical, token-level criterion for SFT data curation that goes beyond instruction-level diversity and aligns with the supervised-signal nature of SFT. The strengths of the paper are its systematic taxonomy, the breadth of the comparison (three strategies, two components, three dataset sizes), the inclusion of multiple diversity metrics, and the additional robustness checks via alternative judges, model families, tokenizers, and non-LLM benchmarks (MMLU, LiveBench). The algorithmic descriptions in Algorithms 1-2 are sufficiently detailed to be reimplemented, though no code is provided. The main risks are that the headline result may be confounded with response length, that the experiments lack repeated-seed uncertainty quantification, and that the token-frequency-band definition rests on a manual and unvalidated choice.

major comments (4)

[§3.2, §4.2, Figure 4, Figure 6(a)]
[§3.1, Table 2]
[§2.2, Figure 2]
[§2.3, Appendix B, Table 4]

minor comments (7)

[§3.1.2]
[Table 2]
[Appendix A]
[§4.2]
[§3.2, Appendix E]
[Figure 4]
[§2.3, Appendix H]

Circularity Check

1 steps flagged · score 2.0 of 10

One mild self-referential validation step (IE as confirmation of the entropy-like micro objective); the central empirical claim is not circular.

self definitional [Section 3.2 (Tests of Diversity Metrics), paragraph on information entropy; related to Section 2.2 and Appendix E.]
"The information entropy metric relates to the core mechanism of our microscopic strategy, with the slight difference that the strategy optimizes the entropy over the set of mid-band tokens and the metric computes over the whole vocabulary set. Moreover, the strong correlation with information entropy implies the effectiveness of the microscopic strategy."

The micro strategy's diversity scale is defined as the proportion of unique important (mid-band) token types, and Algorithms 1-2 are designed to maximize coverage of those token types. Information entropy over the token distribution is the same variety measure at whole-vocabulary granularity. The paper uses the IE-performance correlation as evidence that the micro strategy is effective ('implies the effectiveness of the microscopic strategy'), but since the strategy is constructed to increase this very entropy, a positive IE correlation is expected by construction and cannot serve as independent confirmation. The performance scores are externally measured, so the main diversity-performance claim is empirical; only this confirmatory metric is partially self-referential.

full rationale

The central claim (response-micro diversity has stronger correlation and best max-diversity performance) rests on measured fine-tuning scores across constructed datasets, not on a derivation from the strategy's definition. The x-axis diversity scale is indeed the controlled token-type coverage, but the y-axis scores come from external pairwise evaluations; slopes in Table 2 are empirical fits, not predictions forced by construction. No load-bearing self-citation: the in-house Nemotron judge ([43], co-authored by Y. Dong) is used for evaluation and response generation, but its agreement with GPT-4 Turbo is checked (8% reversal) and alternative judges reproduce the trend, so it does not by itself determine the conclusion. The only circular element is Section 3.2's use of information entropy as evidence of effectiveness: IE is essentially the whole-vocabulary version of the entropy over mid-band tokens that the micro strategy optimizes, so that correlation is partly self-referential. This does not affect the independent empirical comparison across strategies. The sequence-length confounding concern is a validity/confound issue, not a circularity one. Hence score 2.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several hand-chosen thresholds (token frequency bands, clustering parameters) and on the assumption that responses generated by one model can be scored fairly by a judge from the same model family. The paper does not report values for alpha and batch size in the sampling algorithm, and does not release code or constructed data.

free parameters (5)

Mid-band token frequency thresholds = high: >500, mid: 10-500, low: <10
Manually chosen after inspecting the frequency distribution in Figure 2; directly determines which tokens are counted as 'important' and thus defines the diversity scale in all micro experiments.
UMAP neighbors and components = neighbors=15, components=5
Used for macroscopic embedding clustering; changes cluster structure, but likely not central to the main response-token claim.
HDBSCAN parameters for macro and meso strategies = min_cluster_size=20; eps=0.15, min_samples=2
Clustering hyperparameters for macro and meso strategies; influence the number of clusters or tags in those strategies.
Trade-off alpha and batch size B in Algorithm 2 = not specified
The token-aware sampling algorithm requires these values, but they are not reported, impeding exact replication.
Length control threshold = approximately 500 tokens
In the length-control ablation, responses are constrained to about 500 tokens; this choice may affect the conclusion that length is an incidental covariate.

assumptions (5)

domain assumption SFT training signal is in the response tokens
The paper's motivation that 'LLMs are conventionally supervised using responses as the primary training signals' (Section 2.2).
ad hoc to paper Mid-band tokens carry most of the semantic meaning
Stated as a result of manual inspection in Section 2.2; no independent evidence provided.
ad hoc to paper Nemotron-generated responses are of sufficiently high and uniform quality
The paper replaces all original responses with Llama-3.1-70B-Nemotron generations to 'alleviate the quality variance' (Section 2.3); this is a strong assumption because the evaluation judge is also Nemotron.
domain assumption The pairwise scoring with a reference model gives meaningful performance differences
Used throughout evaluations; the paper reports an 8% reversal rate against GPT-4 Turbo.
domain assumption Uniform sampling across clusters, tags, or token types preserves usefulness
The dataset construction in Section 2.3 samples to achieve uniform distribution across target types.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning." pith.science (2026). https://pith.science/paper/RLNMVC3E

@misc{pith2026250524768,
  author       = {Pith},
  title        = {Pith review of: From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RLNMVC3E}},
  note         = {Machine review of arXiv:2505.24768}
}

read the original abstract

Dataset diversity plays a pivotal role for the successful training of many machine learning models, particularly in the supervised fine-tuning (SFT) stage of large language model (LLM) development. Despite increasing recognition of its importance, systematic analyses of dataset diversity still remain underexplored. To address this gap, this work presents a systematic taxonomy of existing diversity-control strategies, which primarily focus on the instruction component, operating at either macroscopic (entire instruction semantics) or mesoscopic levels (instruction units), and furthermore introduces a novel analysis of microscopic diversity within the response component, specifically analyzing the statistical distribution of tokens in SFT training samples. In the experimental evaluation, we construct fixed-size datasets (e.g., 10,000 samples each) from a corpus of 117,000 open-source SFT samples, incorporating six distinct diversity-control strategies spanning macro-, meso-, and microscopic levels applied to both instructions and responses. We then fine-tune LLMs on these datasets to assess the six diversity-control strategies. Results reveal that while macroscopic and mesoscopic strategies lead to higher performance with increasing diversity, the microscopic strategy in responses exhibits both a stronger correlation between model performance and the degree of diversity and superior performance with maximum diversity across all strategies. These findings offer actionable insights for constructing high-performance SFT datasets.

Figures

Figures reproduced from arXiv: 2505.24768 by the authors.

**Figure 2.** Distribution for tokens in the SFT dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The relationship between the diversity percentage and model performance for instructions [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Pearson correlation coefficients of multiple di [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Ablation study on microscopic diversity: comparing different models (Llama-2-70B and Llama-3-8B). The abscissa indicates the microscopic level diversity percentage from the response perspective in the case of 10K dataset. As evidenced in [PITH_FULL_IMAGE:figures/ful…

**Figure 6.** Figure 6: Ablation study on microscopic diversity: Figure (a) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Token distributions from the response perspective under different microscopic diversity [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

**Figure 8.** Figure 8: The relationship between the diversity percentage and model performance for instructions [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

**Figure 9.** Figure 9: Pearson correlation coefficients of multiple diversity parameters and model performance [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 38 canonical work pages

[1]

Abusamra

H. Abusamra. A comparative study of feature selection and classification methods for gene expression data of glioma.Procedia Computer Science, 23:5–14, 2013

work page 2013
[2]

Achiam, S

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

arXiv 2023
[3]

Arora, Y

S. Arora, Y . Liang, and T. Ma. A simple but tough-to-beat baseline for sentence embeddings. InProceedings of the International Conference on Learning Representations, 2017

work page 2017
[4]

A. L. Berger, S. A. Della Pietra, and V . J. Della Pietra. A maximum entropy approach to natural language processing.Computational Linguistics, 22(1):39–71, 1996

work page 1996
[5]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901, 2020

1901
[6]

Bubeck, V

S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y . T. Lee, Y . Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4.arXiv preprint arXiv:2303.12712, 2023

arXiv 2023
[7]

Bukharin, S

A. Bukharin, S. Li, Z. Wang, J. Yang, B. Yin, X. Li, C. Zhang, T. Zhao, and H. Jiang. Data diversity matters for robust instruction tuning. InFindings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3411–3425, 2024

work page 2024
[8]

H. Chen, A. Waheed, X. Li, Y . Wang, J. Wang, B. Raj, and M. I. Abdin. On the diversity of synthetic data and its impact on training large language models.arXiv preprint arXiv:2410.15226, 2024

arXiv 2024

Show all 59 references

[9]

Conover, M

M. Conover, M. Hayes, A. Mathur, J. Xie, J. Wan, S. Shah, A. Ghodsi, P. Wendell, M. Zaharia, and R. Xin. Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023

2023
[10]

Dettmers, A

T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2023

2023
[11]

Devlin, M.-W

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,...

2019
[12]

Q. Du, C. Zong, and J. Zhang. Mods: Model-oriented data selection for instruction tuning.arXiv preprint arXiv:2311.15653, 2023

2023 arXiv
[13]

Dubey, A

A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

2024 arXiv
[14]

Ester, H.-P

M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. InProceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, volume 96, pages 226–231, 1996

1996
[15]

T. Ge, X. Chan, X. Wang, D. Yu, H. Mi, and D. Yu. Scaling synthetic data creation with 1,000,000,000 personas.arXiv preprint arXiv:2406.20094, 2024

2024 arXiv
[16]

Y . Ge, Y . Liu, C. Hu, W. Meng, S. Tao, X. Zhao, M. Xia, Z. Li, B. Chen, H. Yang, B. Li, T. Xiao, and J. Zhu. Clustering and ranking: Diversity-preserved instruction selection through expert-aligned quality estimation. InProceedings of the 2024 Conference on Empirical Methods...

2024
[17]

Grootendorst

M. Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure.arXiv preprint arXiv:2203.05794, 2022. 10

2022 arXiv
[18]

Hendrycks, C

D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring mas- sive multitask language understanding. InProceedings of the International Conference on Learning Representations, 2021

2021
[19]

E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. LoRA: Low- rank adaptation of large language models. InProceedings of the International Conference on Learning Representations, 2022

2022
[20]

Kaplan, S

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

2001 arXiv
[21]

Land and M

S. Land and M. Bartolo. Fishing for magikarp: Automatically detecting under-trained tokens in large language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11631–11646, 2024

2024
[22]

J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan. A diversity-promoting objective function for neural conversation models. InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages...

2016
[23]

Li, W.-L

T. Li, W.-L. Chiang, E. Frick, L. Dunlap, T. Wu, B. Zhu, J. E. Gonzalez, and I. Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline.arXiv preprint arXiv:2406.11939, 2024

2024 arXiv
[24]

X. Li, T. Zhang, Y . Dubois, R. Taori, I. Gulrajani, C. Guestrin, P. Liang, and T. B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_ eval, 2023

2023
[25]

Z. Li, C. Chen, T. Xu, Z. Qin, J. Xiao, R. Sun, and Z.-Q. Luo. Entropic distribution matching for supervised fine-tuning of LLMs: Less overfitting and better diversity. InProceedings of the 38th Annual Conference on Neural Information Processing Systems Workshop on Fine-Tuning...

2024
[26]

C.-Y . Lin. ROUGE: A package for automatic evaluation of summaries. InProceedings of Text Summariza- tion Branches Out, pages 74–81, July 2004

2004
[27]

Z. Lin, Z. Gou, Y . Gong, X. Liu, yelong shen, R. Xu, C. Lin, Y . Yang, J. Jiao, N. Duan, and W. Chen. Not all tokens are what you need for pretraining. InProceedings of the thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

2024
[28]

W. Liu, W. Zeng, K. He, Y . Jiang, and J. He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. InProceedings of the International Conference on Learning Representations, 2024

2024
[29]

Z. Liu, A. Karbasi, and T. Rekatsinas. TSDS: Data selection for task-specific model finetuning. In Proceedings of the Annual Conference on Neural Information Processing Systems, 2024

2024
[30]

K. Lu, H. Yuan, Z. Yuan, R. Lin, J. Lin, C. Tan, C. Zhou, and J. Zhou. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. InProceedings of the International Conference on Learning Representations, 2023

2023
[31]

Madsen, N

A. Madsen, N. Meade, V . Adlakha, and S. Reddy. Evaluating the faithfulness of importance measures in nlp by recursively masking allegedly important tokens and retraining. InFindings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1731–1751, 2022

2022
[32]

McInnes, J

L. McInnes, J. Healy, and S. Astels. hdbscan: Hierarchical density based clustering.Journal of Open Source Software, 2(11):205, 2017

2017
[33]

McInnes, J

L. McInnes, J. Healy, N. Saul, and L. Großberger. Umap: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29):861, 2018

2018
[34]

Mikolov, I

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality.Advances in Neural Information Processing Systems, 26, 2013

2013
[35]

Ouyang, J

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35, 2022

2022
[36]

Padmakumar and H

V . Padmakumar and H. He. Does writing with language models reduce content diversity? InProceedings of the International Conference on Learning Representations, 2024. 11

2024
[37]

Shaib, J

C. Shaib, J. Barrow, J. Sun, A. F. Siu, B. C. Wallace, and A. Nenkova. Standardizing the measurement of text diversity: A tool and a comparative analysis of scores.arXiv preprint arXiv:2403.00553, 2024

2024
[38]

F. Song, B. Yu, H. Lang, H. Yu, F. Huang, H. Wang, and Y . Li. Scaling data diversity for fine-tuning language models in human alignment. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, pages 14358–14369, 2024

2024
[39]

Touvron, L

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

2023 arXiv
[40]

Vaswani, N

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need.Advances in Neural Information Processing Systems, 30, 2017

2017
[41]

P. Wang, Y . Shen, Z. Guo, M. Stallone, Y . Kim, P. Golland, and R. Panda. Diversity measurement and subset selection for instruction tuning datasets.arXiv preprint arXiv:2402.02318, 2024

2024 arXiv
[42]

Y . Wang, Y . Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 13484–13508, 2023

2023
[43]

Z. Wang, A. Bukharin, O. Delalleau, D. Egert, G. Shen, J. Zeng, O. Kuchaiev, and Y . Dong. Helpsteer2- preference: Complementing ratings with preferences.arXiv preprint arXiv:2410.01257, 2024

2024 arXiv
[44]

J. Wei, Y . Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al. Emergent abilities of large language models.Transactions on Machine Learning Research, 2022

2022
[45]

White, S

C. White, S. Dooley, M. Roberts, A. Pal, B. Feuer, S. Jain, R. Shwartz-Ziv, N. Jain, K. Saifullah, S. Dey, Shubh-Agrawal, S. S. Sandha, S. V . Naidu, C. Hegde, Y . LeCun, T. Goldstein, W. Neiswanger, and M. Goldblum. Livebench: A challenging, contamination-free LLM benchmark. ...

2025
[46]

W. F. Wiggins and A. S. Tejani. On the opportunities and risks of foundation models for natural language processing in radiology.Radiology: Artificial Intelligence, 4(4):e220119, 2022

2022
[47]

A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

2024 arXiv
[48]

Zhang, S

Y . Zhang, S. Sun, M. Galley, Y .-C. Chen, C. Brockett, X. Gao, J. Gao, J. Liu, and B. Dolan. DIALOGPT : Large-scale generative pre-training for conversational response generation. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Syste...

2020
[49]

D. Zhao, J. Andrews, O. Papakyriakopoulos, and A. Xiang. Position: Measure dataset diversity, don’t just claim it. InProceedings of the 41st International Conference on Machine Learning, pages 60644–60673, 2024

2024
[50]

H. Zhao, M. Andriushchenko, F. Croce, and N. Flammarion. Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning. InProceedings of the 41st International Conference on Machine Learning, 2024

2024
[51]

Zheng, R

Y . Zheng, R. Zhang, J. Zhang, Y . Ye, Z. Luo, Z. Feng, and Y . Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2024

2024
[52]

Zhong, L

Q. Zhong, L. Ding, J. Liu, X. Liu, M. Zhang, B. Du, and D. Tao. Revisiting token dropping strategy in efficient bert pretraining. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 10391–10405, 2023

2023
[53]

C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y . Mao, X. Ma, A. Efrat, P. Yu, L. Yu, et al. Lima: Less is more for alignment.Advances in Neural Information Processing Systems, 36, 2023

2023
[54]

important tokens

Y . Zhu, S. Lu, L. Zheng, J. Guo, W. Zhang, J. Wang, and Y . Yu. Texygen: A benchmarking platform for text generation models. InProceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1097–1100, 2018. 12 A Collection...

2018
[55]

Assistant A is significantly better: [[A»B]]
[56]

Assistant A is slightly better: [[A>B]]
[57]

Tie, relatively the same: [[A=B]]
[58]

Assistant B is slightly better: [[B>A]]
[59]

Spelling and Grammar Check

Assistant B is significantly better: [[B»A]] Example output: {My final verdict is tie: [[A=B]]} prompt_template: ["<|User Prompt|> {question_1} <|The Start of Assistant A’s Answer|> {answer_1} <|The End of Assistant A’s Answer|> <|The Start of Assistant B’s Answer|> {answer_2}...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Abusamra

H. Abusamra. A comparative study of feature selection and classification methods for gene expression data of glioma.Procedia Computer Science, 23:5–14, 2013

work page 2013

[2] [2]

Achiam, S

J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

arXiv 2023

[3] [3]

Arora, Y

S. Arora, Y . Liang, and T. Ma. A simple but tough-to-beat baseline for sentence embeddings. InProceedings of the International Conference on Learning Representations, 2017

work page 2017

[4] [4]

A. L. Berger, S. A. Della Pietra, and V . J. Della Pietra. A maximum entropy approach to natural language processing.Computational Linguistics, 22(1):39–71, 1996

work page 1996

[5] [5]

Brown, B

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901, 2020

1901

[6] [6]

Bubeck, V

S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y . T. Lee, Y . Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4.arXiv preprint arXiv:2303.12712, 2023

arXiv 2023

[7] [7]

Bukharin, S

A. Bukharin, S. Li, Z. Wang, J. Yang, B. Yin, X. Li, C. Zhang, T. Zhao, and H. Jiang. Data diversity matters for robust instruction tuning. InFindings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3411–3425, 2024

work page 2024

[8] [8]

H. Chen, A. Waheed, X. Li, Y . Wang, J. Wang, B. Raj, and M. I. Abdin. On the diversity of synthetic data and its impact on training large language models.arXiv preprint arXiv:2410.15226, 2024

arXiv 2024