Pith. sign in

REVIEW 4 major objections 7 minor 108 references

Instance-dependent Early Stopping

T0 review · 4 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an instance is mastered when the second-order difference of its loss stays near zero, and that skipping such instances in backpropagation cuts computation by 10-50% with no accuracy loss.

desk verdict Useful empirical paper on adaptive data pruning, but the 'mastered' criterion actually flags locally linear loss decreases, not plateaus, so the central story needs rework. read the letter →

arxiv 2502.07547 v1 pith:DPYSJLRX submitted 2025-02-11 cs.LG

Suqin Yuan , Runqi Lin , Lei Feng , Bo Han , Tongliang Liu This is my paper · ORCID
classification cs.LG
keywords instance-dependentearlystoppingsecond-orderlossdifferencedynamicdatapruningbackpropagationsavingssampleselectiontrainingefficiencytransferlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that conventional early stopping is too coarse: it halts all training at once, while many instances are already learned long before the whole dataset is. Its proposal, Instance-dependent Early Stopping (IES), treats each training example as its own stopping problem, declaring an instance mastered when the second-order difference of its loss over three consecutive epochs stays below a single threshold. Once mastered, the instance stops participating in backpropagation, while still being scored by forward passes. On CIFAR and ImageNet-1k benchmarks, the method reports 10-55% fewer backpropagation instances, about 1.2-1.4x wall-clock speedups, and test accuracy that is maintained or slightly improved; downstream transfer also improves by around 1.5 points on average. If correct, this turns early stopping from a global training switch into a schedule-free per-sample pruning rule.

What carries the argument

The core object is the IES stopping rule built on the second-order loss difference. It defines $\Delta^2 L_i(w(t))$, a forward-pass-only estimator of whether instance $i$'s loss has flattened over the last three epochs, and compares its absolute value to a single threshold $\delta$. The algorithm keeps a dynamic active set $D^{(t)} = D^{(0)} \setminus M^{(t)}$ of not-yet-mastered instances, recomputed each epoch from forward passes, and runs backpropagation only on that set; removed instances can re-enter if their second-order difference grows again, which the paper argues prevents catastrophic forgetting.

What would settle it

On a long-tailed or noisy dataset, compute for every instance whether $|\Delta^2 L_i(w(t))| < \delta$ holds at the same time that the model's accuracy on that instance stops improving; if many mastered instances later show loss increases and re-enter training, or if savings collapse toward zero on clean data when the unified threshold is used, the criterion is not tracking stable instance-level mastery.

Watch

Extended reading notes

Core claim

The central claim is that instance-level mastery can be read off the curvature of the loss trajectory: with $\Delta^2 L_i(w(t)) = L_i(w(t)) - 2L_i(w(t-1)) + L_i(w(t-2))$, an instance is mastered when $|\Delta^2 L_i(w(t))| < \delta$ for a single $\delta$ shared across all instances. The paper argues this criterion is better than using raw loss because optimal loss levels differ across instances (due to noise, class imbalance, ambiguity), while second-order differences concentrate near zero for learned instances late in training, allowing one universal threshold. Excluding mastered instances from backprop concentrates updates on not-yet-learned examples with larger gradient norms, lowering training loss and Hessian sharpness faster, which the paper presents as the mechanism behind maintained or improved generalization and transferability.

Load-bearing premise

The load-bearing premise is that a single scalar threshold $\delta$ on the second-order difference of an instance's loss marks it as mastered across all samples, datasets, optimizers, and training stages; the paper's own Limitation section concedes that a theoretical analysis of why this criterion works is still open.

Editorial extensions

If this is right

  • IES reduces backpropagation instances by 10-55% on CIFAR and ImageNet-1k with test accuracy matching full-data training, giving about 1.2-1.4x wall-clock speedups.
  • Models pretrained with IES transfer better: fine-tuning ImageNet-1k features on CIFAR-10, CIFAR-100, and Caltech-101 beats the full-data baseline by up to roughly 2.5 points after one epoch.
  • IES is schedule-free: it needs no preset removal rate or curriculum, and the fraction of dropped samples adapts automatically as training proceeds.
  • At matched 2x and 3x speedup targets, IES compares favorably with InfoBatch, importance sampling, and curriculum-learning baselines on CIFAR-10 and CIFAR-100.
  • If a 1% test-accuracy drop is tolerated, savings grow to as much as 80% on CIFAR-10 and about 40% on ImageNet-1k.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The unified-threshold claim is the fragile part; the paper's own Limitation section concedes that a theoretical analysis of the second-order criterion is open, so a natural test is whether one $\delta$ transfers across datasets with very different per-instance difficulty distributions.
  • Under typical label-noise settings the paper itself reports zero savings (Appendix G), which means the benefit is concentrated in clean or mildly noisy regimes; a noise-robust version of the mastery criterion would be a direct extension.
  • Because the criterion uses only per-sample losses already computed during the forward pass, IES is a candidate drop-in for large-scale training pipelines that log loss values, with no extra gradient computation.
  • The reversibility of removals (samples re-enter when their second-order difference spikes) connects IES to continual-learning and forgetting analyses beyond what the paper's appendix tracks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Instance-dependent Early Stopping (IES), a dynamic per-sample training method. IES computes the second-order difference of each sample's loss over three consecutive epochs, labels an instance as 'mastered' when that difference falls below a fixed threshold delta (Eq. (3)), and removes mastered instances from backpropagation while still evaluating forward passes on the full dataset. The authors report a 10-50% reduction in backpropagated mini-batch instances across CIFAR, ImageNet-1k, object detection, and segmentation, with test accuracy essentially unchanged and small improvements in transfer learning; they attribute the speedup to focusing parameter updates on not-yet-mastered instances with larger gradient norms.

Significance. If the empirical results hold, IES is a simple and broadly applicable training accelerator with little or no accuracy cost on common vision benchmarks. The paper's strengths are the wide evaluation grid (multiple architectures, optimizers, learning-rate schedulers, datasets, and high-level tasks), the robustness study over delta, the public implementation, and the explicit limitation statement. The central conceptual claim is nonetheless fragile: the criterion in Eq. (3) is not a plateau detector, since it vanishes for any linearly changing loss. The contribution is therefore better characterized as an empirically driven pruning heuristic than as a principled mastery criterion, and the manuscript should be revised so that the claims match the actual criterion.

major comments (4)
  1. [Sec. 3.1, Eq. (3)] The criterion in Eq. (3) does not implement the plateau detector described in Sec. 3.1. Equation (2) defines Delta^2 L_i = L_i(t) - 2 L_i(t-1) + L_i(t-2), so |Delta^2 L_i| < delta is a bound on the change of the per-epoch loss decrement, not on the size of that decrement. A sample whose loss decreases by the same amount in three consecutive epochs has Delta^2 L_i = 0 and will be marked as mastered even though its loss is still changing substantially. This contradicts the motivation that a mastered instance's loss is 'flat and insensitive to parameter updates.' The failure mode can occur in steady-learning phases and under exponential LR decay, and Appendix H's re-inclusion statistics (Table 13: average 13.14 re-inclusions) are consistent with the criterion firing on samples that are still being learned. Please either change the criterion to one that directly measures flatness (e.g., a bound on |Delta L_i| with a slope check) or explicitly reframe IES as a loss-trajectory heuristic and add an experiment measuring how many removed samples still have large first-order loss changes at the moment of removal.
  2. [Alg. 1, line 5] The definition of the mastered set is ambiguous. Line 5 of Algorithm 1 writes M(t) = { i in D(0) : |Delta^2 L_i(w(t'))| < delta } with an undefined t', while Sec. 3.1 speaks of the 'sum of the absolute values' over k epochs and Eq. (3) has no summation or window parameter. This ambiguity affects the re-inclusion dynamics and the reported Mini-batch Saved numbers: whether the condition is evaluated on the latest epoch only or as an accumulated statistic changes which samples are removed and when. Please state the exact condition used in all experiments and make Eq. (3), Algorithm 1, and the text mutually consistent.
  3. [Table 2] The ImageNet-1k results in Table 2 come from a single run. The claim of 'lossless acceleration' at this scale rests on differences such as 68.8% vs. 69.0% and 74.3% vs. 74.1%, which are within typical run-to-run variation for ImageNet training. Without multiple seeds or error bars, the paper cannot support the strong statement that IES maintains accuracy on ImageNet. Please report repeated runs for at least a subset of the Table 2 configurations, or temper the claim to say that IES achieves comparable accuracy in single-run comparisons.
  4. [Sec. 3.3, 'Larger Gradient Norms'] The mechanism claim in Sec. 3.3 is partly circular. Removing samples whose losses change slowly will mechanically increase the average gradient norm of the retained mini-batch, so Figure 4's observation that IES 'focuses on instances with larger gradient norms' restates the selection rule rather than explains faster convergence. The paper should provide a direct test of the mechanism, for example by comparing IES against a control that removes the same fraction of samples at random but reweights the retained gradients to match the IES gradient-norm distribution, or by reporting full-set training loss at equal wall-clock time rather than only the retained-set trajectory.
minor comments (7)
  1. [Sec. 4.1] The phrase 'unified threshold' is used for a fixed delta within a dataset, but the base delta differs by three orders of magnitude between CIFAR (1e-3) and ImageNet (1). Please clarify that the unification is across instances within a dataset and explain how the scale of the losses determines the appropriate delta.
  2. [Fig. 3 and Appendix C] The coefficient-of-variation argument is weak because the CV is unstable for quantities whose mean approaches zero; the lower CV of second-order differences may reflect the scale of the values rather than a more consistent learning status. The empirical robustness to delta in Figure 5 is more convincing and should be the primary justification for a unified threshold.
  3. [Eq. (1)] Equation (1) states the mastered condition without an absolute value, whereas Eq. (3) and Algorithm 1 use |Delta^2 L_i(w(t))|. Since second-order differences can be negative, the absolute value should appear consistently in the formal definition.
  4. [Table 4 and Appendix F] At the target 2x and 3x speedups, IES loses 0.6/2.1 percentage points (2x) and 1.1/4.0 percentage points (3x) on CIFAR-10/CIFAR-100 relative to the full-data baseline. Sentences such as 'while maintaining model performance' should therefore be qualified to reflect the magnitude of the degradation in these high-speedup comparisons.
  5. [Appendix I] The fairness discussion is internally inconsistent: it first states that IES 'may introduce or amplify certain biases' and then concludes that 'the overall fairness is slightly improved.' These statements should be reconciled, and the paper should not claim an improvement on the basis of a 0.007 difference in DPD.
  6. [Sec. 2 vs. Sec. 4.1] The Related Work section states that IES eliminates the need for 'pre-set training schedules or removal rates,' yet Sec. 4.1 follows Qin et al. (2023) in annealing to full data for the last 10% of ImageNet epochs. Please clarify how this preset schedule is compatible with that claim.
  7. [Throughout] There are minor typographical errors, including 'an unified' (Abstract and Section 1) and 'sharpnes' (Section 3.3), that should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: IES's savings and accuracy claims are empirical comparisons against baselines, and the second-order-difference criterion is an input definition rather than a result derived from the target claims.

full rationale

Walking the claimed derivation chain, none of the load-bearing steps reduces to the paper's own inputs. The mastered criterion in Sec. 3.1-3.2 is a stipulated operational definition (Eq. 3), not a fitted parameter or a prediction derived from the test-accuracy results; the claim that it supports a unified threshold is supported by coefficient-of-variation measurements in Fig. 3 and Appendix C, which are independent empirical observations. The headline results (10-50% backpropagation savings with comparable or slightly better accuracy, Tables 1-3) are obtained by running Algorithm 1 against a no-removal baseline; the savings are emergent outcomes, not enforced by tuning delta to a target removal rate. The 'larger gradient norms' and 'faster sharpness reduction' discussion in Sec. 3.3 is a post-hoc mechanism explanation rather than a derivation of the accuracy results, and even if the gradient-norm statement is near-tautological, it does not make the reported comparisons circular. Self-citations (e.g., Yuan et al. 2024, Lin et al. 2024a;b;c) appear only as background related work, not as load-bearing justifications, and no uniqueness theorem or ansatz is imported from prior work by the same authors. The Limitation section explicitly concedes that a comprehensive theoretical analysis of the criterion's superiority remains an open question, which is inconsistent with a circular derivation. The skeptic's concern that the second-order difference can vanish for a linearly decreasing loss is a correctness or robustness issue about whether the criterion matches the 'plateau' motivation, not a circularity issue.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The ledger contains one hand-set hyperparameter, delta, and three domain assumptions about loss-flatness as a mastery signal. The paper introduces no new physical or mathematical entities.

free parameters (1)
  • delta (mastered threshold) = 1e-3 for CIFAR, 1 for ImageNet
    Hand-set per dataset in Section 4.1; robustness across four orders of magnitude is shown in Figure 5, but the criterion and savings rate depend on this value.
assumptions (3)
  • domain assumption Small second-order difference of loss implies the instance is mastered and can be removed from backpropagation.
    Sec 3.1 Eq (3): instances with small |Delta^2 L| are declared mastered; this is the core operational definition and is not proven.
  • domain assumption Removed instances stay mastered without repeated backpropagation.
    Figure 2 shows counts of mastered instances are similar with and without backprop, but this is an empirical observation on CIFAR-10, not a general guarantee.
  • domain assumption The full forward pass over the entire dataset is acceptable overhead, so savings come mainly from skipping backpropagation.
    Algorithm 1 always forward-passes the full dataset; the wall-clock speedup depends on backprop dominating forward cost, stated in Sec 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Instance-dependent Early Stopping." pith.science (2026). https://pith.science/paper/DPYSJLRX

@misc{pith2026250207547,
  author       = {Pith},
  title        = {Pith review of: Instance-dependent Early Stopping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DPYSJLRX}},
  note         = {Machine review of arXiv:2502.07547}
}
read the original abstract

In machine learning practice, early stopping has been widely used to regularize models and can save computational costs by halting the training process when the model's performance on a validation set stops improving. However, conventional early stopping applies the same stopping criterion to all instances without considering their individual learning statuses, which leads to redundant computations on instances that are already well-learned. To further improve the efficiency, we propose an Instance-dependent Early Stopping (IES) method that adapts the early stopping mechanism from the entire training set to the instance level, based on the core principle that once the model has mastered an instance, the training on it should stop. IES considers an instance as mastered if the second-order differences of its loss value remain within a small range around zero. This offers a more consistent measure of an instance's learning status compared with directly using the loss value, and thus allows for a unified threshold to determine when an instance can be excluded from further backpropagation. We show that excluding mastered instances from backpropagation can increase the gradient norms, thereby accelerating the decrease of the training loss and speeding up the training process. Extensive experiments on benchmarks demonstrate that IES method can reduce backpropagation instances by 10%-50% while maintaining or even slightly improving the test accuracy and transfer learning performance of a model.

Figures

Figures reproduced from arXiv: 2502.07547 by the authors.

Figure 2
Figure 2. The curves show the number of instances that meet the corresponding mastered criteria (N [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Coefficient of variation (CV) of different orders of loss differences during training. By quantifying the rate of change in the loss for each instance around the current parameters w (t) , the second-order difference effectively captures the stability of the loss function, regardless of the specific value of Li(w ∗ ). This property allows for using a unified threshold δ across all instances, greatly simplifying the … view at source ↗
Figure 4
Figure 4. Comparison of model performance metrics between the IES method and the baseline method [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Comparison of the proposed IES method of different IES criteria (loss, 1st, 2nd, and 3rd [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Coefficient of variation (CV) of different orders of loss differences during training. Using [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Comparison of the wall-time between IES method and InfoBatch method [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

108 extracted references · 47 canonical work pages

  1. [1]

    Variance reduction in sgd by distributed importance sampling

    Guillaume Alain, Alex Lamb, Chinnadhurai Sankar, Aaron Courville, and Yoshua Bengio. Variance reduction in sgd by distributed importance sampling. arXiv preprint arXiv:1511.06481, 2015

    arXiv 2015

  2. [2]

    Towards understanding sharpness-aware minimization

    Maksym Andriushchenko and Nicolas Flammarion. Towards understanding sharpness-aware minimization. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th ICML, volume 162 of Proceedings of Machine Learning Research, pp.\ 639--668. PMLR, 17--23 Jul 2022

    2022

  3. [3]

    A closer look at memorization in deep networks

    Devansh Arpit, Stanis aw Jastrz e bski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In ICML, pp.\ 233--242. PMLR, 2017

    2017

  4. [4]

    Reconciling modern machine-learning practice and the classical bias--variance trade-off

    Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias--variance trade-off. Proceedings of the National Academy of Sciences, 116 0 (32): 0 15849--15854, 2019

    2019

  5. [5]

    Curriculum learning

    Yoshua Bengio, J \'e r \^o me Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual ICML, pp.\ 41--48, 2009

    2009

  6. [6]

    The power of uniform sampling for coresets

    Vladimir Braverman, Vincent Cohen-Addad, H-C Shaofeng Jiang, Robert Krauthgamer, Chris Schwiegelshohn, Mads Bech Toftrup, and Xuan Wu. The power of uniform sampling for coresets. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS), pp.\ 462--473. IEEE, 2022

    2022

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurIPS, 33: 0 1877--1901, 2020

    1901

  8. [8]

    Learning imbalanced datasets with label-distribution-aware margin loss

    Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. NeurIPS, 32, 2019

    2019

Show all 108 references
  1. [9]

    Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping

    Rich Caruana, Steve Lawrence, and C Giles. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. NeurIPS, 13, 2000

    2000

  2. [10]

    Active bias: Training more accurate neural networks by emphasizing high variance samples

    Haw-Shiuan Chang, Erik Learned-Miller, and Andrew McCallum. Active bias: Training more accurate neural networks by emphasizing high variance samples. NeurIPS, 30, 2017

    2017

  3. [11]

    Rethinking atrous convolution for semantic image segmentation

    Liang-Chieh Chen. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017

    2017 arXiv

  4. [12]

    Importance sampling for minibatches

    Dominik Csiba and Peter Richt \'a rik. Importance sampling for minibatches. Journal of Machine Learning Research, 19 0 (27): 0 1--21, 2018

    2018

  5. [13]

    Class-balanced loss based on effective number of samples

    Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In CVPR, pp.\ 9268--9277, 2019

    2019

  6. [14]

    Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

    Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. NeurIPS, 27, 2014

    2014

  7. [15]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

  8. [16]

    Sharp minima can generalize for deep nets

    Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In ICML, pp.\ 1019--1028. PMLR, 2017

    2017

  9. [17]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL V isual O bject C lasses C hallenge 2007 (VOC2007) R esults. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html, a

    2007

  10. [18]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL V isual O bject C lasses C hallenge 2012 (VOC2012) R esults. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html, b

    2012

  11. [19]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020

    2010 arXiv

  12. [20]

    Cascast: Skillful high-resolution precipitation nowcasting via cascaded modelling

    Junchao Gong, Lei Bai, Peng Ye, Wanghan Xu, Na Liu, Jianhua Dai, Xiaokang Yang, and Wanli Ouyang. Cascast: Skillful high-resolution precipitation nowcasting via cascaded modelling. arXiv preprint arXiv:2402.04290, 2024

    2024 arXiv

  13. [21]

    Deep learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016

    2016

  14. [22]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In ICML, pp.\ 1321--1330. PMLR, 2017

    2017

  15. [23]

    On the power of curriculum learning in training deep networks

    Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep networks. In ICML, pp.\ 2535--2544. PMLR, 2019

    2019

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  17. [25]

    Large-scale dataset pruning with dynamic uncertainty

    Muyang He, Shuo Yang, Tiejun Huang, and Bo Zhao. Large-scale dataset pruning with dynamic uncertainty. arXiv preprint arXiv:2306.05175, 2023

    2023 arXiv

  18. [26]

    Deep learning scaling is predictable, empirically

    Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017

    2017 arXiv

  19. [27]

    Flat minima

    Sepp Hochreiter and J \"u rgen Schmidhuber. Flat minima. Neural computation, 9 0 (1): 0 1--42, 1997

    1997

  20. [28]

    Ridge regression: Biased estimation for nonorthogonal problems

    Arthur E Hoerl and Robert W Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12 0 (1): 0 55--67, 1970

    1970

  21. [29]

    Improving non-transferable representation learning by harnessing content and style

    Ziming Hong, Zhenyi Wang, Li Shen, Yu Yao, Zhuo Huang, Shiming Chen, Chuanwu Yang, Mingming Gong, and Tongliang Liu. Improving non-transferable representation learning by harnessing content and style. In ICLR, 2024

    2024

  22. [30]

    Densely connected convolutional networks

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 4700--4708, 2017

    2017

  23. [31]

    Epsilon-coresets for clustering (with outliers) in doubling metrics

    Lingxiao Huang, Shaofeng H-C Jiang, Jian Li, and Xuan Wu. Epsilon-coresets for clustering (with outliers) in doubling metrics. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pp.\ 814--825. IEEE, 2018

    2018

  24. [32]

    Harnessing out-of-distribution examples via augmenting content and style

    Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing out-of-distribution examples via augmenting content and style. arXiv preprint arXiv:2207.03162, 2022

    2022 arXiv

  25. [33]

    Robust generalization against photon-limited corruptions via worst-case sharpness minimization

    Zhuo Huang, Miaoxi Zhu, Xiaobo Xia, Li Shen, Jun Yu, Chen Gong, Bo Han, Bo Du, and Tongliang Liu. Robust generalization against photon-limited corruptions via worst-case sharpness minimization. In CVPR, pp.\ 16175--16185, 2023

    2023

  26. [34]

    Winning prize comes from losing tickets: Improve invariant learning by exploring variant parameters for out-of-distribution generalization

    Zhuo Huang, Muyang Li, Li Shen, Jun Yu, Chen Gong, Bo Han, and Tongliang Liu. Winning prize comes from losing tickets: Improve invariant learning by exploring variant parameters for out-of-distribution generalization. International Journal of Computer Vision, pp.\ 1--19, 2024

    2024

  27. [35]

    Coresets for scalable bayesian logistic regression

    Jonathan Huggins, Trevor Campbell, and Tamara Broderick. Coresets for scalable bayesian logistic regression. NeurIPS, 29, 2016

    2016

  28. [36]

    Do we need zero training loss after achieving zero training error? arXiv preprint arXiv:2002.08709, 2020

    Takashi Ishida, Ikko Yamane, Tomoya Sakai, Gang Niu, and Masashi Sugiyama. Do we need zero training loss after achieving zero training error? arXiv preprint arXiv:2002.08709, 2020

    2002 arXiv

  29. [37]

    Accelerating deep learning by focusing on the biggest losers

    Angela H Jiang, Daniel L-K Wong, Giulio Zhou, David G Andersen, Jeffrey Dean, Gregory R Ganger, Gauri Joshi, Michael Kaminksy, Michael Kozuch, Zachary C Lipton, et al. Accelerating deep learning by focusing on the biggest losers. arXiv preprint arXiv:1910.00762, 2019

    1910 arXiv

  30. [38]

    Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels

    Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In ICML, pp.\ 2304--2313. PMLR, 2018

    2018

  31. [39]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  32. [40]

    Biased importance sampling for deep neural network training

    Angelos Katharopoulos and Fran c ois Fleuret. Biased importance sampling for deep neural network training. arXiv preprint arXiv:1706.00043, 2017

    2017 arXiv

  33. [41]

    Not all samples are created equal: Deep learning with importance sampling

    Angelos Katharopoulos and Fran c ois Fleuret. Not all samples are created equal: Deep learning with importance sampling. In ICML, pp.\ 2525--2534. PMLR, 2018

    2018

  34. [42]

    On large-batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836, 2016

    2016 arXiv

  35. [43]

    Uniform convergence of rank-weighted learning

    Justin Khim, Liu Leqi, Adarsh Prasad, and Pradeep Ravikumar. Uniform convergence of rank-weighted learning. In ICML, pp.\ 5254--5263. PMLR, 2020

    2020

  36. [44]

    Grad-match: Gradient matching based data subset selection for efficient deep model training

    Krishnateja Killamsetty, Sivasubramanian Durga, Ganesh Ramakrishnan, Abir De, and Rishabh Iyer. Grad-match: Gradient matching based data subset selection for efficient deep model training. In ICML, pp.\ 5464--5474. PMLR, 2021 a

    2021

  37. [45]

    Glister: Generalization based data subset selection for efficient and robust learning

    Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Generalization based data subset selection for efficient and robust learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 8110--8118, 2021 b

    2021

  38. [46]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  39. [47]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  40. [48]

    Self-paced learning for latent variable models

    M Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. NeurIPS, 23, 2010

    2010

  41. [49]

    Caltech 101, 2022

    Fei-Fei Li, Marco Andreeto, Marc'Aurelio Ranzato, and Pietro Perona. Caltech 101, 2022

    2022

  42. [50]

    Towards realistic model selection for semi-supervised learning

    Muyang Li, Xiaobo Xia, Runze Wu, Fengming Huang, Jun Yu, Bo Han, and Tongliang Liu. Towards realistic model selection for semi-supervised learning. In ICML, 2023 a

    2023

  43. [51]

    Stochastic modified equations and adaptive stochastic gradient algorithms

    Qianxiao Li, Cheng Tai, and E Weinan. Stochastic modified equations and adaptive stochastic gradient algorithms. In ICML, pp.\ 2101--2110. PMLR, 2017

    2017

  44. [52]

    Loftq: Lora-fine-tuning-aware quantization for large language models

    Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao. Loftq: Lora-fine-tuning-aware quantization for large language models. arXiv preprint arXiv:2310.08659, 2023 b

    2023 arXiv

  45. [53]

    Enhancing the reliability of out-of-distribution image detection in neural networks

    Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690, 2017

    2017 arXiv

  46. [54]

    On the over-memorization during natural, robust and catastrophic overfitting

    Runqi Lin, Chaojian Yu, Bo Han, and Tongliang Liu. On the over-memorization during natural, robust and catastrophic overfitting. In ICLR, 2024 a

    2024

  47. [55]

    Layer-aware analysis of catastrophic overfitting: Revealing the pseudo-robust shortcut dependency

    Runqi Lin, Chaojian Yu, Bo Han, Hang Su, and Tongliang Liu. Layer-aware analysis of catastrophic overfitting: Revealing the pseudo-robust shortcut dependency. arXiv preprint arXiv:2405.16262, 2024 b

    2024 arXiv

  48. [56]

    Eliminating catastrophic overfitting via abnormal adversarial examples regularization

    Runqi Lin, Chaojian Yu, and Tongliang Liu. Eliminating catastrophic overfitting via abnormal adversarial examples regularization. NeurIPS, 36, 2024 c

    2024

  49. [57]

    Cs-isolate: Extracting hard confident examples by content and style isolation

    Yexiong Lin, Yu Yao, Xiaolong Shi, Mingming Gong, Xu Shen, Dong Xu, and Tongliang Liu. Cs-isolate: Extracting hard confident examples by content and style isolation. NeurIPS, 36, 2023

    2023

  50. [58]

    Learning the latent causal structure for modeling label noise

    Yexiong Lin, Yu Yao, and Tongliang Liu. Learning the latent causal structure for modeling label noise. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 d

    2024

  51. [59]

    Online batch selection for faster training of neural networks

    Ilya Loshchilov and Frank Hutter. Online batch selection for faster training of neural networks. arXiv preprint arXiv:1511.06343, 2015

    2015 arXiv

  52. [60]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  53. [61]

    o ren Mindermann, Jan M Brauner, Muhammed T Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt H \

    S \"o ren Mindermann, Jan M Brauner, Muhammed T Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt H \"o ltgen, Aidan N Gomez, Adrien Morisot, Sebastian Farquhar, et al. Prioritized training on points that are learnable, worth learning, and not yet learnt. In ICML, pp...

    2022

  54. [62]

    o sung. ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift f \

    RV Mises and Hilda Pollaczek-Geiringer. Praktische verfahren der gleichungsaufl \"o sung. ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift f \"u r Angewandte Mathematik und Mechanik , 9 0 (1): 0 58--77, 1929

    1929

  55. [63]

    Deep double descent: Where bigger models and more data hurt

    Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021 0 (12): 0 124003, 2021

    2021

  56. [64]

    Exploring generalization in deep learning

    Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. NeurIPS, 30, 2017

    2017

  57. [65]

    Deep learning on a data diet: Finding important examples early in training

    Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on a data diet: Finding important examples early in training. NeurIPS, 34: 0 20596--20607, 2021

    2021

  58. [66]

    Some methods of speeding up the convergence of iteration methods

    Boris T Polyak. Some methods of speeding up the convergence of iteration methods. Ussr computational mathematics and mathematical physics, 4 0 (5): 0 1--17, 1964

    1964

  59. [67]

    Early stopping-but when? In Neural Networks: Tricks of the trade, pp.\ 55--69

    Lutz Prechelt. Early stopping-but when? In Neural Networks: Tricks of the trade, pp.\ 55--69. Springer, 2002

    2002

  60. [68]

    Infobatch: Lossless training speed up by unbiased dynamic data pruning

    Ziheng Qin, Kai Wang, Zangwei Zheng, Jianyang Gu, Xiangyu Peng, Zhaopan Xu, Daquan Zhou, Lei Shang, Baigui Sun, Xuansong Xie, et al. Infobatch: Lossless training speed up by unbiased dynamic data pruning. arXiv preprint arXiv:2303.04947, 2023

    2023 arXiv

  61. [69]

    Accelerating deep learning with dynamic data pruning

    Ravi S Raju, Kyle Daruwalla, and Mikko Lipasti. Accelerating deep learning with dynamic data pruning. arXiv preprint arXiv:2111.12621, 2021

    2021 arXiv

  62. [70]

    Early stopping and non-parametric regression: an optimal data-dependent stopping rule

    Garvesh Raskutti, Martin J Wainwright, and Bin Yu. Early stopping and non-parametric regression: an optimal data-dependent stopping rule. The Journal of Machine Learning Research, 15 0 (1): 0 335--366, 2014

    2014

  63. [71]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren. Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015

    2015 arXiv

  64. [72]

    Overfitting in adversarially robust deep learning

    Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In ICML, pp.\ 8093--8104. PMLR, 2020

    2020

  65. [73]

    A stochastic approximation method

    Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pp.\ 400--407, 1951

    1951

  66. [74]

    An investigation of why overparameterization exacerbates spurious correlations

    Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterization exacerbates spurious correlations. In ICML, pp.\ 8346--8356. PMLR, 2020

    2020

  67. [75]

    Data parameters: A new family of parameters for learning a differentiable curriculum

    Shreyas Saxena, Oncel Tuzel, and Dennis DeCoste. Data parameters: A new family of parameters for learning a differentiable curriculum. NeurIPS, 32, 2019

    2019

  68. [76]

    Prioritized experience replay

    Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015

    2015 arXiv

  69. [77]

    Diversity-aware batch active learning for dependency parsing

    Tianze Shi, Adrian Benton, Igor Malioutov, and Ozan Irsoy. Diversity-aware batch active learning for dependency parsing. arXiv preprint arXiv:2104.13936, 2021

    2021 arXiv

  70. [78]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  71. [79]

    Beyond neural scaling laws: beating power law scaling via data pruning

    Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. NeurIPS, 35: 0 19523--19536, 2022

    2022

  72. [80]

    Regression shrinkage and selection via the lasso

    Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58 0 (1): 0 267--288, 1996

    1996

  73. [81]

    An empirical study of example forgetting during deep neural network learning

    Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159, 2018

    2018 arXiv

  74. [82]

    Kakurenbo: Adaptively hiding samples in deep neural network training

    Thao Nguyen Truong, Balazs Gerofi, Edgar Josafat Martinez-Noriega, Fran c ois Trahay, and Mohamed Wahib. Kakurenbo: Adaptively hiding samples in deep neural network training. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  75. [83]

    Normalized flat minima: Exploring scale invariant definition of flat minima for neural networks using pac-bayesian analysis

    Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Normalized flat minima: Exploring scale invariant definition of flat minima for neural networks using pac-bayesian analysis. In ICML, pp.\ 9636--9647. PMLR, 2020

    2020

  76. [84]

    Optimizing data usage via differentiable rewards

    Xinyi Wang, Hieu Pham, Paul Michel, Antonios Anastasopoulos, Jaime Carbonell, and Graham Neubig. Optimizing data usage via differentiable rewards. In ICML, pp.\ 9983--9995. PMLR, 2020

    2020

  77. [85]

    Computation-efficient deep learning for computer vision: A survey

    Yulin Wang, Yizeng Han, Chaofei Wang, Shiji Song, Qi Tian, and Gao Huang. Computation-efficient deep learning for computer vision: A survey. Cybernetics and Intelligence, 2024 a

    2024

  78. [86]

    Efficienttrain++: Generalized curriculum learning for efficient visual backbone training

    Yulin Wang, Yang Yue, Rui Lu, Yizeng Han, Shiji Song, and Gao Huang. Efficienttrain++: Generalized curriculum learning for efficient visual backbone training. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 b

    2024

  79. [87]

    Minimal effort back propagation for convolutional neural networks

    Bingzhen Wei, Xu Sun, Xuancheng Ren, and Jingjing Xu. Minimal effort back propagation for convolutional neural networks. arXiv preprint arXiv:1709.05804, 2017

    2017 arXiv

  80. [88]

    Self-filtering: A noise-aware sample selection for label noise with confidence penalization

    Qi Wei, Haoliang Sun, Xiankai Lu, and Yilong Yin. Self-filtering: A noise-aware sample selection for label noise with confidence penalization. In European Conference on Computer Vision, pp.\ 516--532. Springer, 2022

    2022

  81. [89]

    Curriculum learning by transfer learning: Theory and experiments with deep networks

    Daphna Weinshall, Gad Cohen, and Dan Amir. Curriculum learning by transfer learning: Theory and experiments with deep networks. In ICML, pp.\ 5238--5246. PMLR, 2018

    2018

  82. [90]

    Benign overfitting in classification: Provably counter label noise with larger models

    Kaiyue Wen, Jiaye Teng, and Jingzhao Zhang. Benign overfitting in classification: Provably counter label noise with larger models. arXiv preprint arXiv:2206.00501, 2022

    2022 arXiv

  83. [91]

    Sharpness minimization algorithms do not only minimize sharpness to achieve better generalization

    Kaiyue Wen, Zhiyuan Li, and Tengyu Ma. Sharpness minimization algorithms do not only minimize sharpness to achieve better generalization. NeurIPS, 36, 2024

    2024

  84. [92]

    When do curricula work?, 2021

    Xiaoxia Wu, Ethan Dyer, and Behnam Neyshabur. When do curricula work?, 2021

    2021

  85. [93]

    Mitigating label noise on graphs via topological sample selection

    Yuhao Wu, Jiangchao Yao, Xiaobo Xia, Jun Yu, Ruxin Wang, Bo Han, and Tongliang Liu. Mitigating label noise on graphs via topological sample selection. In ICLR, 2024

    2024

  86. [94]

    Robust early-learning: Hindering the memorization of noisy labels

    Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nannan Wang, Zongyuan Ge, and Yi Chang. Robust early-learning: Hindering the memorization of noisy labels. In ICLR, 2020 a

    2020

  87. [95]

    Part-dependent label noise: Towards instance-dependent label noise

    Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. NeurIPS, 33: 0 7597--7610, 2020 b

    2020

  88. [96]

    Moderate coreset: A universal method of data selection for real-world data-efficient deep learning

    Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In ICLR, 2022

    2022

  89. [97]

    Refined coreset selection: Towards minimal coreset size under model performance constraints

    Xiaobo Xia, Jiale Liu, Shaokun Zhang, Qingyun Wu, Hongxin Wei, and Tongliang Liu. Refined coreset selection: Towards minimal coreset size under model performance constraints. In Forty-first International Conference on Machine Learning, 2024

    2024

  90. [98]

    Rethinking bias-variance trade-off for generalization of neural networks

    Zitong Yang, Yaodong Yu, Chong You, Jacob Steinhardt, and Yi Ma. Rethinking bias-variance trade-off for generalization of neural networks. In ICML, pp.\ 10767--10777. PMLR, 2020

    2020

  91. [99]

    Dual t: Reducing estimation error for transition matrix in label-noise learning

    Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. Dual t: Reducing estimation error for transition matrix in label-noise learning. Advances in neural information processing systems, 33: 0 7260--7271, 2020

    2020

  92. [100]

    Instance-dependent label-noise learning under a structural causal model

    Yu Yao, Tongliang Liu, Mingming Gong, Bo Han, Gang Niu, and Kun Zhang. Instance-dependent label-noise learning under a structural causal model. Advances in Neural Information Processing Systems, 34: 0 4409--4420, 2021

    2021

  93. [101]

    Late stopping: Avoiding confidently learning from mislabeled examples

    Suqin Yuan, Lei Feng, and Tongliang Liu. Late stopping: Avoiding confidently learning from mislabeled examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 16079--16088, 2023

    2023

  94. [102]

    Early stopping against label noise without validation data

    Suqin Yuan, Lei Feng, and Tongliang Liu. Early stopping against label noise without validation data. In ICLR, 2024

    2024

  95. [103]

    Understanding deep learning (still) requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64 0 (3): 0 107--115, 2021

    2021

  96. [104]

    Curriculum learning by dynamic instance hardness

    Tianyi Zhou, Shengjie Wang, and Jeffrey Bilmes. Curriculum learning by dynamic instance hardness. NeurIPS, 33: 0 8602--8613, 2020

    2020

  97. [105]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  98. [106]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  99. [107]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  100. [108]

    " 3 머 Xqp/ n2:,#,HVRJ tIm649i&9Yz B G!

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.