Pith. sign in

REVIEW 4 major objections 5 minor 49 references

SDO claims that grouping training samples by prompt-embedding similarity into coherent mini-batches, then rebalancing exposure each epoch, accelerates LLM post-training across SFT, DPO, and GRPO while preserving coverage.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

SDO accelerates LLM post-training by forming coherent KNN-based mini-batches and dynamically rebalancing sample exposure across epochs.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection A genuine but thinly validated data-organization idea; the main ablation conflates repetition with locality, so the mechanism is not yet established, but the method is worth a serious referee. the 4 major comments →

arxiv 2607.27273 v1 pith:FWUQPLDD submitted 2026-07-29 cs.LG

SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

Jinliang Gao , Ning Yang , Hai Wang , Baili Xiao , Pin Lyu This is my paper
classification cs.LG
keywords LLM post-trainingdata organizationmini-batch constructionKNN batchingexposure balancinggradient conflictconvergence accelerationcoverage preservation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that how data is organized into mini-batches, not just which samples are selected or in what order they are shown, is an underexploited lever for cutting LLM post-training cost. It introduces SDO, a plug-in data organizer that groups prompts into semantically coherent batches using nearest neighbors in a frozen embedding space, and then rebalances which samples keep appearing in later epochs via a per-sample exposure counter. The paper claims this two-level scheme accelerates convergence across supervised fine-tuning, direct preference optimization, and GRPO reinforcement learning, with the biggest gains in the early-to-mid phase, while also producing more balanced accuracy across question types. The justification ties representation-space locality to reduced intra-batch gradient conflict through a theorem and an identity connecting batch gradient norm to gradient disagreement. A sympathetic reader would care because the method changes only the data pipeline, leaves the loss and schedule untouched, adds negligible runtime, and never permanently discards samples.

Core claim

SDO's central claim is that the composition of mini-batches, and the frequency with which individual samples are revisited across epochs, are trainable data-flow variables that materially affect LLM post-training efficiency. Concretely, SDO builds locality-aware mini-batches by traversing a K-nearest-neighbor graph over frozen prompt embeddings, so that samples within a batch are close in representation space; after each epoch, it keeps all under-exposed samples and re-samples over-exposed ones with probability inversely proportional to their exposure, so no sample is permanently dropped. Across SFT, DPO, and GRPO experiments, SDO converges faster than uniform shuffling at matched checkpoint

What carries the argument

Locality-aware batching via KNN traversal over frozen prompt embeddings, paired with an exposure ledger. KNN defines a neighborhood graph over the active data pool; a visitation mask selects anchor samples and pulls overlapping neighborhoods into mini-batches, giving each batch a small representation diameter. The exposure ledger records how many times each sample has participated; after each epoch, a monotonically rising threshold splits the full dataset into a fully retained cold set and a hot set resampled with probability inversely proportional to exposure. The theoretical core is an identity relating the batch gradient norm to intra-batch gradient conflict, plus Theorem 1, which shows t

Load-bearing premise

The load-bearing premise is the paper's Assumption 1, that prompts close in the frozen external embedding space also have similar gradients for the current model; if a given encoder's embeddings do not track task-relevant gradient structure, KNN grouping gives no coherence benefit and the reported acceleration must be explained by something else.

What would settle it

Run SDO on the same three training setups but replace the prompt embeddings with random vectors before KNN construction, or use embeddings from an encoder deliberately chosen to be semantically irrelevant; if convergence still beats uniform shuffling, the gradient-coherence mechanism is not the explanation. Alternatively, directly measure the average batch gradient norm in SDO versus shuffled batches: Theorem 1 predicts a norm increase proportional to cluster dispersion, so a measurement showing no such increase would refute the stated mechanism.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Post-training speedups are achievable without changing the loss, training schedule, or dataset: SDO acts only on data flow, so it can be bolted onto existing pipelines.
  • The exposure ledger provides a generic mechanism for preventing locality-based grouping from over-concentrating on dense regions, directly addressing coverage collapse.
  • Because gains concentrate in the early-to-mid phase, data organization matters most when gradients are least stable; later training is comparatively insensitive to batch composition.
  • Since SDO uses frozen external embeddings, it extends beyond SFT to preference optimization and reinforcement-learning post-training objectives.
  • No sample is permanently excluded, so data that becomes useful only later in training remains recoverable, unlike hard-filtering selection methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The acceleration mechanism rests on a representation-gradient alignment that the paper measures only for one encoder and one model; a natural extension is to test whether the gain persists when embeddings come from the model being trained rather than a frozen general-purpose encoder, since the Lipschitz link is model-dependent.
  • If reduced gradient conflict is the true cause, SDO should combine additively with batch-size or gradient-accumulation schedules: larger coherent batches may compound the gradient-norm preservation effect.
  • The exposure ledger could be reused as a diagnostic: per-sample exposure statistics reveal which regions the model has already mastered, suggesting an exposure-driven early-stopping rule or a difficulty curriculum.
  • Theorem 1 predicts a quantitative gradient-norm gap proportional to cluster dispersion; measuring that gap across the three paradigms would separate the locality mechanism from any side effect of the resampling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SDO (Structure-Aware Data Organization), a data-organizing framework for LLM post-training. Within each epoch, SDO constructs mini-batches by traversing a K-nearest-neighbor graph over frozen prompt embeddings, appending each anchor sample together with its K neighbors to a batch buffer. Across epochs, an exposure ledger records cumulative per-sample participation, and the active data pool is reconstructed by retaining under-exposed samples and inverse-exposure resampling from an over-exposed hot set. The authors evaluate SDO on SFT, DPO, and GRPO with a 4B model, reporting faster convergence, improved per-cluster accuracy balance on GSM8K, and more coherent gradients relative to uniform shuffling. A theoretical section derives an identity linking intra-batch gradient conflict to batch gradient magnitude and states a conflict-reduction theorem under a Lipschitz smoothness assumption on gradients in embedding space.

Significance. If the central mechanism is correct, SDO is a low-cost, plug-and-play data-organization method that is orthogonal to loss design and applicable across alignment paradigms. The paper provides a concrete algorithm with pseudo-code, three-seed main experiments with consistent trends, and a plausible theoretical link between representation-space locality and gradient conflict. The method preserves samples (no permanent filtering) and shows empirically improved balance metrics. However, the current evidence has a load-bearing confound: the main ablation does not isolate semantic locality from the repeated-sampling structure inherent in the KNN traversal. The theoretical support relies on a strong untested assumption and the associated proofs are deferred to a missing supplement. These issues must be addressed before the paper's mechanistic claims can be accepted.

major comments (4)
  1. [Ablation Studies (Table 4; Algorithm 1)] The 'SDO w/o locality' variant replaces KNN with uniform shuffling, but Algorithm 1 (lines 5–7) appends {x_i} ∪ N_K(i) for each anchor, so overlapping neighborhoods cause samples to appear multiple times within an epoch. Uniform shuffling removes both semantic locality and this repetition/overlap structure. The comparison therefore conflates the two. This is not merely theoretical: 'SDO w/o exposure' (which retains the KNN overlap while disabling pool reconstruction) already improves 3k accuracy to 78.62 vs. baseline 77.18, while 'SDO w/o locality' gives 78.01, so the overlap/repetition structure alone may account for much of the early-to-mid gain. Please add a control with random neighbors (e.g., each anchor's K neighbors drawn uniformly at random from the pool) that preserves the same anchor order, overlap multiplicity, exposure ledger, and pool reconstruction. Without such a control,
  2. [Theoretical Justification (Assumption 1, Figure 5)] Assumption 1 posits L-Lipschitz smoothness of per-sample gradients in the frozen embedding space: ||g_i(θ) − g_j(θ)|| ≤ L ||z_i − z_j||. This is essentially the hypothesis that the paper sets out to test. The only empirical support cited is the positive embedding–gradient correlation in Figure 5, but because SDO constructs batches by selecting embedding neighbors, the pairs used in the correlation are those selected by the method; the correlation is therefore at least partly a selection artifact. Please report the same correlation on the baseline's random pairs and on the proposed random-neighbor control, to separate genuine coupling of embedding and gradient spaces from the effect of grouping. Additionally, Theorem 1 only shows that locality-aware batching increases the expected batch-gradient norm; the paper does not demonstrate that this norm increase translates into faster convergenc
  3. [Gradient Behavior Analysis (Figure 4)] The reported mean pairwise gradient coherence is inconsistent between text and figure. The text states 'the baseline exhibits sparse, scattered coherence (mean 0.17), while SDO displays broad high-coherence regions (mean 0.29)', but Figure 4's caption gives 'Mean coherence = 0.017' and 'Mean coherence = 0.029'. This factor-of-10 discrepancy is too large to ignore, especially because gradient coherence is presented as key evidence for the mechanism. The definition of coherence (including how gradients are pooled and normalized) and the correct numerical values must be stated exactly and made consistent across text, captions, and any supplement.
  4. [Technical Supplement / Reproducibility] All proofs of Lemma 1, Theorem 1, Proposition 1, the gradient collection details, the KNN approximation gap analysis, and all experimental hyperparameters (batch size, LR, steps, DPO pair handling) are deferred to a 'Technical Supplement' that is not included in the manuscript. Without this material, the theoretical claims cannot be verified and the experiments cannot be reproduced. If the supplement is available, please include it or provide it as an appendix; otherwise, the paper needs at least the theorem proofs and hyperparameter tables inline.
minor comments (5)
  1. [Theoretical Justification (Proposition 1)] Proposition 1 is nearly tautological: (i) follows directly from Eq. (4) and (ii) from Eqs. (3) and (5). It is a useful observation that no sample is permanently excluded, but calling it a 'coverage preservation guarantee' overstates the result—it does not guarantee any particular level of balance, only that under-exposed samples are retained.
  2. [Tables 3–5] These tables report single-seed results (seed 617). Please state this in each caption or in the main text, and if possible add standard errors or a small multi-seed check for the main ablation rows.
  3. [Experiments (Paired t-tests)] The text states 'Paired t-tests confirm significant mid-training improvements (p<0.05, Cohen's d>2)' but gives no t-statistics, degrees of freedom, or the exact checkpoints tested. These should be reported or referenced to a specific supplement table.
  4. [Related Work and References] There are several formatting/name issues: 'hattami' appears with non-standard capitalization, 'zembed-1-embedding' is not identified with a standard model name or reference, and some references (e.g., 'Croitoru et al. 2026') lack page numbers or venue details. These should be cleaned up.
  5. [Figure 5] Figure 5 is mentioned only in a single sentence and its axes or quantities are not defined in the text. Please clarify what is being plotted (e.g., pairwise embedding cosine similarity vs. pairwise gradient cosine similarity, pooled over which steps) and how the smoothing is applied.

Circularity Check

0 steps flagged

No significant circularity: the convergence claim is empirical and independent; the theoretical proposition is a definitional property, and the single self-citation is not load-bearing.

full rationale

The paper's central claim is an empirical convergence acceleration of SDO over uniform-shuffling baselines across SFT, DPO, and GRPO. This comparison is self-contained and does not reduce to the method's definitions. The theoretical justification (Theorem 1, Eqs. 9-12) is an algebraic identity: it derives the expected intra-batch gradient conflict from the definitions of Γ(B), Φ(θ), and the mixture model. No fitted parameters are renamed as predictions, and the paper explicitly notes that a larger gradient norm does not by itself guarantee faster convergence, so the theory is presented as a mechanism rather than a circular proof of the empirical results. Proposition 1 is indeed a direct restatement of the algorithm's construction: p_i is defined as 1/u_i in Eq. (4) and D_cold is defined by u_i < τ_e in Eq. (3), so monotonicity and retention of cold samples are true by definition. However, the paper uses this only as a design guarantee, not as an independent empirical prediction. The gradient-coherence diagnostics are labeled 'suggestive rather than conclusive' and follow from Assumption 1 together with the KNN batching construction; they are manipulation checks rather than independent evidence. The one self-citation (Yang et al. 2026) appears only in a related-work list about token-level DPO and is not load-bearing. The 'SDO w/o locality' ablation confound—where uniform shuffling removes both semantic locality and the overlapping-neighborhood repetition structure—is a genuine experimental validity concern, but it is not a definitional circularity: the reported gain is not equivalent to the inputs by construction. Overall, the paper's derivation chain does not exhibit significant circularity; the score reflects only the minor non-load-bearing self-citation and the mildly self-referential diagnostic framing.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The central claims rest on three hyperparameters selected by offline tuning and on the assumption that frozen embeddings encode gradient-relevant structure. No new physical or conceptual entities are introduced beyond the exposure ledger, which is a counter rather than an entity.

free parameters (3)
  • K (neighborhood size) = 4 (default; 2 and 8 tested)
    K trades off batch purity and exploration; K=4 gives best 3k accuracy in sensitivity analysis; tuned on GSM8K GRPO.
  • Δτ (exposure threshold increment) = 2 (default; 1 and 3 tested)
    Controls how fast the cold/hot boundary rises; Δτ=2 chosen as stable middle ground in ablation.
  • r (hot-set retention ratio) = 0.2 default; 0.1 used for diagnostics
    Proportion of hot-set samples retained each epoch; r=0.1 gives better balance but was not used in main experiments, suggesting a deliberate bias toward stability over the reported advantage.
axioms (3)
  • domain assumption Assumption 1 (Local Gradient Smoothness): ||g_i(θ) − g_j(θ)|| ≤ L ||z_i − z_j|| for any pair.
    Links embedding proximity to gradient similarity; the validity is untested and the paper concedes it depends on the encoder capturing task-relevant semantics.
  • domain assumption Data Mixture Model with G latent semantic clusters aligned to embedding space.
    Used in Theorem 1 to define cluster-pure batching; in practice KNN approximates clusters but the alignment is assumed, not measured.
  • standard math The identity ||ĝ||² = ā − (|B|−1)/(2|B|) Γ(B) and variance decomposition Φ(θ) = Σ_{g<h} π_g π_h ||μ_g − μ_h||².
    These are algebraic facts used in the proof; presumed correct.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of SDO: Structure-Aware Data Organization for Efficient LLM Post-Training." pith.science (2026). https://pith.science/paper/FWUQPLDD

@misc{pith2026260727273,
  author       = {Pith},
  title        = {Pith review of: SDO: Structure-Aware Data Organization for Efficient LLM Post-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FWUQPLDD}},
  note         = {Machine review of arXiv:2607.27273}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules. However, data organization itself is usually treated as a static preprocessing step: embedding-based grouping methods construct fixed partitions before training and cannot adapt to the evolving sample exposure during optimization. As a result, all samples receive similar exposure despite their different optimization needs, leading to redundant updates for some samples while leaving others under-optimized. To address this problem, we propose SDO (Structure-Aware Data Organization), a plug-and-play data organization framework with an exposure-driven feedback mechanism that organizes mini-batch composition and sample exposure according to representation-space structure. SDO operates epoch by epoch on frozen external embeddings, avoiding model warm-up training overhead: within each epoch, locality-aware batching forms coherent mini-batches via KNN neighborhood traversal; across epochs, exposure-balanced scheduling records per-sample participation and reduces the sampling probability of over-exposed samples to preserve long-term coverage. Across SFT, DPO, and GRPO, SDO accelerates convergence, with the largest gains observed in the early-to-mid phase, producing more coherent gradients and more balanced accuracy across question types without permanently excluding training samples.

Figures

Figures reproduced from arXiv: 2607.27273 by Baili Xiao, Hai Wang, Jinliang Gao, Ning Yang, Pin Lyu.

Figure 1
Figure 1. Figure 1: Data-centric optimization paradigms vs. SDO. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SDO. The framework consists of three stages: (1) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Convergence curves across three post-training paradigms. GRPO/GSM8K reports validation accuracy, while DPO and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Pairwise gradient coherence over the first 3,000 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Co-evolution of embedding and gradient similarity. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 9 linked inside Pith

  1. [1]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  2. [2]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  3. [3]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2001.08361 , year=

    Scaling laws for neural language models , author=. arXiv preprint arXiv:2001.08361 , year=

    Pith/arXiv arXiv 2001

  5. [5]

    Proceedings of the 2023 SIAM international conference on data mining (SDM) , pages=

    Data-centric ai: Perspectives and challenges , author=. Proceedings of the 2023 SIAM international conference on data mining (SDM) , pages=. 2023 , organization=

    2023

  6. [6]

    International conference on machine learning , pages=

    Not all samples are created equal: Deep learning with importance sampling , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  7. [7]

    Advances in neural information processing systems , volume=

    Deep learning on a data diet: Finding important examples early in training , author=. Advances in neural information processing systems , volume=

  8. [8]

    International Conference on Learning Representations , volume=

    Alpagasus: Training a better alpaca with fewer data , author=. International Conference on Learning Representations , volume=

  9. [9]

    arXiv preprint arXiv:2402.04333 , year=

    Less: Selecting influential data for targeted instruction tuning , author=. arXiv preprint arXiv:2402.04333 , year=

    Pith/arXiv arXiv

  10. [10]

    Proceedings of the 26th annual international conference on machine learning , pages=

    Curriculum learning , author=. Proceedings of the 26th annual international conference on machine learning , pages=

  11. [11]

    International Conference on Machine Learning , year=

    Token-level Direct Preference Optimization , author=. International Conference on Machine Learning , year=

  12. [12]

    International Conference on Learning Representations , year=

    Token-Importance Guided Direct Preference Optimization , author=. International Conference on Learning Representations , year=

  13. [13]

    International Conference on Learning Representations , year=

    A General Theoretical Paradigm to Understand Learning from Human Preferences , author=. International Conference on Learning Representations , year=

  14. [14]

    International Conference on Machine Learning , year=

    KTO: Model Alignment as Prospect Theoretic Optimization , author=. International Conference on Machine Learning , year=

  15. [15]

    arXiv preprint arXiv:2503.14476 , year=

    DAPO: An Open-Source LLM Reinforcement Learning System , author=. arXiv preprint arXiv:2503.14476 , year=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2507.18071 , year=

    GSPO: Group Sequence Policy Optimization , author=. arXiv preprint arXiv:2507.18071 , year=

    Pith/arXiv arXiv

  17. [17]

    Forty-third International Conference on Machine Learning , year=

    Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief , author=. Forty-third International Conference on Machine Learning , year=

  18. [18]

    International Journal of Computer Vision , volume=

    Curriculum learning: A survey , author=. International Journal of Computer Vision , volume=. 2022 , publisher=

    2022

  19. [19]

    Contrastive post-training large language models on data curriculum , author=

  20. [20]

    arXiv preprint arXiv:2310.02527 , year=

    Citing: Large language models create curriculum for instruction tuning , author=. arXiv preprint arXiv:2310.02527 , year=

    Pith/arXiv arXiv

  21. [21]

    International Conference on Learning Representations , volume=

    DELIFT: Data efficient language model instruction fine-tuning , author=. International Conference on Learning Representations , volume=

  22. [22]

    Advances in Neural Information Processing Systems , volume=

    Deepvideo-r1: Video reinforcement fine-tuning via difficulty-aware regressive grpo , author=. Advances in Neural Information Processing Systems , volume=

  23. [23]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    Pith/arXiv arXiv

  24. [24]

    International Conference on Learning Representations , volume=

    What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning , author=. International Conference on Learning Representations , volume=

  25. [25]

    NEURIPS 2023 , year=

    Xie, Sang Michael and Santurkar, Shibani and Ma, Tengyu and Liang, Percy , title=. NEURIPS 2023 , year=

    2023

  26. [26]

    IEEE transactions on neural networks and learning systems , year=

    Matiisen, Tambet and Oliver, Avital and Cohen, Taco and Schulman, John , title=. IEEE transactions on neural networks and learning systems , year=

  27. [27]

    ICLR 2024 , year=

    Xu, Canwen and Rosset, Corby and Corro, Luciano Del and Mahajan, Shweti and McAuley, Julian and Neville, Jennifer and Awadallah, Ahmed Hassan and Rao, Nikhil , title=. ICLR 2024 , year=

    2024

  28. [28]

    ICLR 2024 , year=

    hattami, Amine El and Chapados, Nicolas and Pal, Christopher , title=. ICLR 2024 , year=

    2024

  29. [29]

    ICLR 2024 , year=

    Feng, Tao and Wang, Zifeng and Sun, Jimeng , title=. ICLR 2024 , year=

    2024

  30. [30]

    CoRR , year=

    Croitoru, Florinel-Alin and Hondru, Vlad and Ionescu, Radu Tudor and Sebe, Nicu and Shah, Mubarak , title=. CoRR , year=

  31. [31]

    Nips , year=

    Johnson, Rie and Zhang, Tong , title=. Nips , year=

  32. [32]

    Mathematical Programming , year=

    Schmidt, Mark and Roux, Nicolas Le and Bach, Francis , title=. Mathematical Programming , year=

  33. [33]

    ICLR 2020 , year=

    Khandelwal, Urvashi and Levy, Omer and Jurafsky, Dan and Zettlemoyer, Luke and Lewis, Mike , title=. ICLR 2020 , year=

    2020

  34. [34]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2310.01377 , year =

    UltraFeedback: Boosting Language Models with High-quality Feedback , author =. arXiv preprint arXiv:2310.01377 , year =

    Pith/arXiv arXiv

  36. [36]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Beyond Neural Scaling Laws: Beating Power Law Scaling via Data Pruning , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  37. [37]

    International Conference on Learning Representations (ICLR) , year =

    Selection via Proxy: Efficient Data Selection for Deep Learning , author =. International Conference on Learning Representations (ICLR) , year =

  38. [38]

    SIAM Review , volume =

    Optimization Methods for Large-Scale Machine Learning , author =. SIAM Review , volume =

  39. [39]

    International Conference on Learning Representations (ICLR) , year =

    Don't Decay the Learning Rate, Increase the Batch Size , author =. International Conference on Learning Representations (ICLR) , year =

  40. [40]

    International Conference on Learning Representations (ICLR) , year =

    On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima , author =. International Conference on Learning Representations (ICLR) , year =

  41. [41]

    2004 , isbn =

    Introductory Lectures on Convex Optimization: A Basic Course , author =. 2004 , isbn =

    2004

  42. [42]

    arXiv preprint arXiv:2509.12541 , year =

    zELO: ELO-inspired Training Method for Rerankers and Embedding Models , author =. arXiv preprint arXiv:2509.12541 , year =

    arXiv

  43. [43]

    Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages =

    Chiang, Wei-Lin and Liu, Xuanqing and Si, Si and Li, Yang and Bengio, Samy and Hsieh, Cho-Jui , title =. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages =. 2019 , isbn =. doi:10.1145/3292500.3330925 , abstract =

    arXiv 2019

  44. [44]

    Proceedings of the 43rd International Conference on Machine Learning (ICML) , year =

    Group-wise Data Ordering: Enhancing Instruction Tuning of Large Language Models via Embedding Proximity , author =. Proceedings of the 43rd International Conference on Machine Learning (ICML) , year =

  45. [45]

    Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

    Diversified Mini-Batch Selection for Stochastic Gradient Descent , author =. Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

  46. [46]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    Demystifying Data Organization for Enhanced LLM Training , author =. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL) , year =

  47. [47]

    arXiv preprint arXiv:2510.01135 , year =

    Prompt Curriculum Learning for Efficient LLM Post-Training , author =. arXiv preprint arXiv:2510.01135 , year =

    arXiv

  48. [48]

    arXiv preprint , year =

    Data Organization Matters in Multimodal Instruction Tuning: A Controlled Study of Capability Trade-offs , author =. arXiv preprint , year =

  49. [49]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    A Survey on Efficient Large Language Model Training: From Data-centric Perspectives , author =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.