Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Class-wise Autoencoders Measure Classification Difficulty And Detect Label Mistakes

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Class-wise reconstruction-error ratios on frozen features measure classification difficulty and catch label mistakes without training a classifier.

desk verdict Cheap, plausible dataset-difficulty measure with solid threshold-free AUROC evidence on hard datasets, but the headline F1 mislabel claim rests on thresholds tuned on the same benchmarks and needs held-out validation. read the letter →

arxiv 2412.02596 v1 pith:EUBETGGP submitted 2024-12-03 cs.LG cs.CV

Jacob Marks , Brent A. Griffin , Jason J. Corso This is my paper · ORCID
classification cs.LGcs.CV
keywords reconstructionerrorratiosclassificationdifficultymislabeldetectionlabelnoiseclass-wiseautoencodersfoundationmodelfeaturesdatasetanalysisdata-centricAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single family of cheap quantities—reconstruction error ratios (RERs) from tiny autoencoders trained per class on frozen foundation-model features—can characterize a classification dataset at three levels at once: sample, class, and whole dataset. Across 19 visual datasets, the average RER correlates with the error rate of state-of-the-art classifiers, and the paper shows the difficulty decomposes into a finite-sample-size part and a part due to Bayes error and decision-boundary complexity. Reinterpreting each sample's RER as a label-mistakenness score, the same pipeline detects corrupted labels at state-of-the-art F1 and AUROC levels for symmetric and asymmetric noise on hard datasets, using a threshold formula and a noise-rate estimator rather than training a classifier. If correct, the payoff is a data-curation tool that measures how hard a dataset is and which samples to reannotate, in seconds to minutes on CPU, from embeddings that already exist.

What carries the argument

The machinery is the class-wise reconstructor pair and the ratio built from it. Each class $c$ gets a shallow encoder-decoder trained on that class's frozen features with a UMAP graph-layout regularization term, so that reconstruction error $\Delta_c(x)$ measures how well $x$ fits class $c$'s learned manifold. The sample-level score is $\chi(x_c) = \Delta_c(x_c)/\min_{c'\neq c}\Delta_{c'}(x_c)$; values above 1 mean some other class's reconstructor represents the sample better than its own label does. The dataset-level determinant is the expectation $\chi = \mathbb{E}[\chi]$, and two auxiliary ratios, $\chi_0 = \mathbb{E}[\Delta_{\tilde{c}}(x_{\tilde{c}})/\Delta_{\text{rand}}]$ and $\chi_{\text{rand}} = \mathbb{E}[\Delta_{\text{best}}/\Delta_{\text{rand}}]$, feed a noise-rate estimator $\eta \approx (\chi_0 - \chi_{\text{rand}})/(1-\chi_{\text{rand}})$. For binary mislabel prediction, a hand-set threshold ansatz $\chi^* = \gamma_4 \chi^{-\gamma_5}/(1+\gamma_6 \eta_0)$, with $\gamma_4 = 1.01$, $\gamma_5 = 1.5$, $\gamma_6 = 13.8$, converts the per-sample score into clean/dirty predictions. The same per-sample scores also convert into mistake probabilities via Bayes' rule with kernel density estimates.

What would settle it

Run the exact pipeline on a held-out benchmark suite not among the 19: if $\chi$ does not rank datasets by their actual state-of-the-art error rates (Pearson correlation near zero), the difficulty claim fails; and if per-sample $\chi$ gives chance-level AUROC for detecting corrupted labels at 20% symmetric noise, the mislabel claim fails.

Watch

Extended reading notes

Core claim

The paper's central object is the reconstruction error ratio $\chi(x_c) = \Delta_c(x_c)/\min_{c'\neq c}\Delta_{c'}(x_c)$, where $\Delta_c(x_c)$ is the reconstruction error of a 'reconstructor'—a shallow autoencoder with a UMAP graph-layout loss term—trained on the frozen CLIP or DINOv2 features of class $c$, evaluated on a sample labeled $c$. The denominator is the best reconstruction the sample gets from any other class's reconstructor. Averaged over a dataset, $\chi = \mathbb{E}[\chi]$ serves as a dataset-difficulty determinant: across 19 visual datasets spanning four domains, $\log(1-\text{Accuracy})$ of state-of-the-art models correlates with $\chi$ with Pearson $\rho = 0.639$ ($\rho = 0.780$ after removing Oxford 102 Flowers), and the same ratio, evaluated per sample, ranks how mistaken a label is. The paper argues that RERs decompose difficulty into a finite-sample-size term, extrapolated by fitting $\chi_n = \chi_\infty (n^{\gamma_0}+\gamma_1)/(n^{\gamma_0}+\gamma_2)$, and a term capturing Bayes error and decision-boundary complexity, and that per-sample $\chi$ with a threshold ansatz $\chi^* = \gamma_4 \chi^{-\gamma_5}/(1+\gamma_6 \eta_0)$ yields state-of-the-art mislabel detection under symmetric and asymmetric noise on hard datasets.

Load-bearing premise

The method assumes that a sample is, on average, reconstructed better by its own class's autoencoder than by any other class's, and the headline mislabel-detection scores additionally assume a fixed threshold formula whose constants were tuned to the same benchmark datasets.

Editorial extensions

If this is right

  • Dataset difficulty becomes measurable without training a classifier: after one embedding pass, computing $\chi$ takes seconds to minutes on CPU, so any team with frozen features can rank datasets by expected error rate.
  • Because $\chi_n$ scales predictably with samples per class, the framework yields concrete estimates of how much accuracy is lost by pruning data or gained by collecting more annotations.
  • Per-sample RERs rank label mistakes, and with the threshold ansatz they match or beat Confident Learning and zero-shot baselines on symmetric and asymmetric noise on hard datasets, at realistic noise rates up to 30%.
  • The scores also convert into mistake probabilities, which allow budgeted reannotation decisions rather than only a clean/dirty binary flag.
  • The approach transfers across feature backbones (CLIP and DINOv2) and to out-of-domain medical datasets, suggesting the framework is not tied to a particular embedding model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the per-sample RER is a distance-based ratio in feature space, it likely approximates a classification-margin signal; formalizing that connection could let RERs inherit margin theory and predict which samples will move under distribution shift.
  • The same pipeline should apply to text, audio, and time-series by swapping the foundation model, offering a uniform data-curation check across modalities rather than only vision.
  • The threshold-free AUROC evidence is the part of the mislabel claim that does not depend on the hand-set constants; a practical deployment could use RER rankings plus a small annotated validation set to pick a threshold, rather than the fixed ansatz.
  • Pairing RER mistakenness scores with a classifier-based method like Confident Learning could combine the class-decomposed manifold signal with inter-class boundary information, potentially covering the confidence-based noise regime where RERs are weaker.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Reconstruction Error Ratios (RERs): for each class, a shallow autoencoder (reconstructor) is trained on foundation-model features, and the ratio chi(x_c) = Delta_c(x_c)/min_{c' != c} Delta_{c'}(x_c) is used as a sample-level difficulty score. The authors claim that the dataset-level average chi correlates with state-of-the-art classification error rate across 19 visual datasets, that chi_n follows a universal finite-size scaling law (Eq. 5) enabling a first-time decomposition of difficulty into finite-size and Bayes-error/boundary components, and that sample-level chi serves as a label-mistakenness score achieving state-of-the-art feature-based mislabel detection under symmetric and asymmetric noise. The paper reports both F1 results using a hand-set threshold ansatz (Eq. 9) and threshold-free AUROC results (Figs. 13-14), and includes experiments on 19 datasets, an out-of-domain medical benchmark, and robustness checks across feature backbones.

Significance. If the central claims hold, RERs would be a valuable practical tool: one cheap pass on frozen foundation-model features yields dataset-level difficulty and sample-level mislabel scores without training a classifier. The paper has clear strengths: the method is efficient and interpretable; the code is publicly released; the dataset-level difficulty claim is grounded in an external criterion (SOTA error rates); and the threshold-free AUROC comparisons provide evidence that the ranking signal is not entirely an artifact of the tuned threshold. The mislabel-detection results on hard datasets under symmetric and asymmetric noise are a useful contribution even if the F1-level SOTA claim is not yet fully supported. However, the current evidence for the headline F1 claim depends on constants that are tuned on the same benchmark datasets, and the correlation with SOTA error is moderate (rho = 0.639, R^2 about 0.41) rather than 'strong' as stated in the abstract.

major comments (4)
  1. [§5, Eq. (9)] The F1-level SOTA mislabel detection claim in Section 5 and Fig. 6 is not yet supported, because the threshold ansatz (9) uses constants gamma4=1.01, gamma5=1.5, gamma6=13.8 that the paper states are 'close to optimal' on the same benchmark datasets. The optimal threshold varies across datasets and noise types (Appendix C.1), so the F1 curves evaluate a tuned threshold rather than a parameter-free score; no held-out validation is reported. Please specify how these constants were selected, apply them to untouched datasets or noise configurations, or present the threshold-free AUROC results (Figs. 13-14) as the primary mislabel-detection evidence.
  2. [§4.3, Appendix B.2, Eq. (8)] The noise estimator eta about (chi0 - chirand)/(1 - chirand) used by the threshold ansatz relies on the approximation Delta_tilde_c(x_c) approximately min_c Delta_c(x_c) (Appendix B.2), which is justified only for small symmetric noise. Figure 9 shows substantial systematic bias even on the training datasets, e.g., DeepWeeds has intercept 0.207, and Eq. (9) depends directly on this eta. Please quantify the sensitivity of the F1 scores to the noise-estimation bias and test the eta estimator under asymmetric and confidence-based noise beyond the low-noise symmetric regime.
  3. [§4.2, Eq. (5)] The claimed decomposition of classification difficulty into finite-size and Bayes-error/boundary contributions rests on the fitted rational function (5) with a universal exponent gamma0=1.808 and per-dataset constants, and the extrapolated chi_infinity is then interpreted as the difficulty at infinite data. No independent validation of chi_infinity against a second estimator of Bayes error is provided, so the statement that this is the 'first time such a separation has been explicitly possible' (Section 4.2) goes beyond the evidence. Please add an out-of-sample check of the extrapolation, for example by fitting Eq. (5) on n <= n0 and comparing predictions to measured chi_n at n > n0, or by comparing chi_infinity to an independent estimate of irreducible error.
  4. [§4.1, Fig. 3] The central difficulty claim is supported by a Pearson correlation of rho=0.639 (R^2 about 0.41) between log-error rate and chi across 19 datasets; after excluding Oxford 102 Flowers the correlation rises to rho=0.780, but the exclusion is post hoc. Calling this 'strong' in the Abstract and Conclusion overstates a moderate relationship, and the text's appeal to top-5 accuracy for large-class datasets (ImageNet, SUN397, Places205) is a plausible explanation rather than a correction. Please report the correlation with confidence intervals, include alternative rank correlations, and state the post hoc exclusion clearly in the Abstract.
minor comments (4)
  1. [Appendix C.1, Eq. (22)] The threshold definition says 'maximizes our F1 score' but writes chi* = arg min_{chi_thresh} F1(chi_thresh); this should be arg max.
  2. [Appendix C.2, Fig. 14] The definition of hard datasets is inconsistent: the text says 'easy (SOTA accuracy > 0.95) and hard (SOTA accuracy > 0.95)' while the surrounding discussion and Fig. 14 use SOTA accuracy < 0.95 for hard; please correct the typo.
  3. [§A.3.2] The sentence 'we compare our reconstruction error-based method to two three alternatives' contains a typo and should read 'to three alternatives'.
  4. [§2.1] The claim that RERs 'are defined similarly to' the silhouette-score-based difficulty scores of Scheidegger et al. is vague; a concrete comparison of the two formulas in one sentence would help the reader assess novelty.

Circularity Check

2 steps flagged · score 4.0 of 10

Partial circularity: the finite-size difficulty decomposition is a fitted extrapolation restated as a separation, and the F1-level mislabel claim uses a threshold chosen 'close to optimal' on the same benchmarks; AUROC and external-SOTA correlations remain independent.

  1. self definitional [Section 4.2, Eq. (5) and following text (contribution 3)]
    "Empirically, we find that for all datasets the data fit well to rational functions of the form: χn = χ∞ (n^γ0 + γ1)/(n^γ0 + γ2), where γ0 = 1.808 is fixed for all datasets. ... Given χ for the dataset as is, and an estimate for χ∞, we can estimate the contribution to classification difficulty arising from the finite size of the dataset as χ∞ − χ."

    The 'finite sample size' component is not measured against any independent quantity; it is defined as the difference between an extrapolated fitted value (χ∞ from the rational ansatz) and the observed χ. The remaining difficulty is then attributed to Bayes error and decision-boundary complexity. Thus the claimed 'decomposition' is a restatement of the fit rather than a separation validated by an external signal. The paper presents this as 'the first time such a separation has been explicitly possible,' but the separation holds by construction of the fitting ansatz.

  2. fitted input called prediction [Section 5, Eq. (9) and caption of Fig. 6]
    "In practice, we find that this ansatz with the values γ4 = 1.01, γ5 = 1.5, γ6 = 13.8 is close to optimal for symmetric and asymmetric noise outside of fine-grained classification scenarios. ... RER thresholds are selected using ansatz (9)."

    The F1-score comparisons in Fig. 6 are the support for the abstract's claim of SOTA mislabel detection, but the binary predictions use threshold Eq. (9) whose constants were selected to be 'close to optimal' on the same benchmark datasets, and whose η input comes from the noise estimator Eq. (8) derived from the same noisy data. The F1 curves therefore evaluate an in-sample-calibrated operating point rather than a parameter-free prediction. The paper's threshold-free AUROC analysis remains independent, so this is partial rather than total circularity.

full rationale

The paper's strongest independent evidence is threshold-free: dataset-level RER difficulty correlates with external SOTA accuracy (Fig. 3, ρ = 0.639-0.780), and mislabel-detection AUROC (Figs. 13-14) does not depend on the threshold ansatz. These parts are self-contained and are not circular. No load-bearing self-citation chain is present; the method is code-released and benchmarked against external methods and external accuracy tables. However, two claims are partially circular. First, the finite-size decomposition in Sec. 4.2 defines the finite sample size contribution as χ∞ − χ, where χ∞ is a fitted parameter of a rational ansatz; the claimed separation into finite-size and Bayes/boundary components is thus a restatement of the fit rather than an independently validated decomposition. Second, the F1-level SOTA mislabel-detection claim depends on threshold Eq. (9) with constants that the paper states are 'close to optimal' on the same benchmark datasets, so the F1 curves in Fig. 6 measure a tuned operating point. Because the central ranking claims survive threshold-free evaluation, the overall circularity is partial rather than total, warranting a score of 4.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The framework relies on several hand-set parameters and ad hoc functional forms. The finite-size scaling law, the threshold ansatz, and the noise estimator all introduce fitted or assumed structure that the central claims depend on. The invented entities are paper-specific metrics and terminology rather than new physical objects.

free parameters (4)
  • gamma_0 = 1.808
    Global exponent in the finite-sample scaling ansatz Eq. (5), fixed across all 8 fitted datasets; no derivation or uncertainty given.
  • gamma_1, gamma_2 (per dataset) = e.g., CIFAR-10: 19755.34, 19875.82
    Per-dataset parameters of the rational scaling ansatz Eq. (5), fitted to observed chi_n curves for 8 datasets (Table 2).
  • gamma_4, gamma_5, gamma_6 = 1.01, 1.5, 13.8
    Constants in the mislabel threshold ansatz Eq. (9), chosen as 'close to optimal' on the benchmark datasets; no held-out validation.
  • Autoencoder hyperparameters (spread, min_dist, n_neighbors, loss weight, etc.) = spread=25, min_dist=24, n_neighbors=40, loss weight=20
    Chosen via ablations (Fig. 7); authors claim results stabilize for large values, but spread/min_dist are explicitly found to matter.
assumptions (5)
  • domain assumption In-class reconstructor yields lower average reconstruction error than out-of-class reconstructors for every class and dataset.
    Stated in Section 3 as the core premise of the RER definition; verified only empirically on the 19 studied datasets.
  • ad hoc to paper The finite-size scaling of chi_n follows the rational function chi_n = chi_inf * (n^gamma0 + gamma1)/(n^gamma0 + gamma2) with a universal exponent gamma0 = 1.808.
    No derivation; used to extrapolate chi_inf and support the claimed decomposition of difficulty (Eq. 5, Section 4.2).
  • domain assumption For small symmetric noise, the noisy-class reconstructor for the true class yields the minimum reconstruction error (Delta_tilde_c(x_c) approx min Delta(x_c)).
    Used in the noise-rate estimator derivation in Appendix B.2; the resulting estimate is biased on some datasets (e.g., DeepWeeds offset 0.207 in Fig. 9).
  • domain assumption SOTA accuracy from PapersWithCode, as of September 2024, is a valid external measure of dataset classification difficulty.
    Used as the ground truth for the correlation in Section 4.1; the authors exclude Oxford 102 Flowers and give post-hoc explanations for several datasets.
  • ad hoc to paper The threshold ansatz chi_hat* = gamma4 * chi^(-gamma5) / (1 + gamma6 * eta_0) accurately tracks the optimal F1 threshold.
    Used to produce F1-based mislabel detection results (Eq. 9); constants are hand-picked, and the paper notes it overshoots for confidence-based noise.
invented entities (2)
  • Reconstructor
    purpose: Name for a class-wise autoencoder trained on foundation model features; central object of the RER framework.
    A new term for an existing technique (autoencoder); no falsifiable handle outside the paper.
  • Confidence-weighted F1-score (F_W^1) and NCFD
    purpose: New evaluation metrics to show the usefulness of RER-derived mistake probabilities.
    Paper-defined metrics that combine prediction confidence with the F1 score; no external validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Class-wise Autoencoders Measure Classification Difficulty And Detect Label Mistakes." pith.science (2026). https://pith.science/paper/EUBETGGP

@misc{pith2026241202596,
  author       = {Pith},
  title        = {Pith review of: Class-wise Autoencoders Measure Classification Difficulty And Detect Label Mistakes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUBETGGP}},
  note         = {Machine review of arXiv:2412.02596}
}
read the original abstract

We introduce a new framework for analyzing classification datasets based on the ratios of reconstruction errors between autoencoders trained on individual classes. This analysis framework enables efficient characterization of datasets on the sample, class, and entire dataset levels. We define reconstruction error ratios (RERs) that probe classification difficulty and allow its decomposition into (1) finite sample size and (2) Bayes error and decision-boundary complexity. Through systematic study across 19 popular visual datasets, we find that our RER-based dataset difficulty probe strongly correlates with error rate for state-of-the-art (SOTA) classification models. By interpreting sample-level classification difficulty as a label mistakenness score, we further find that RERs achieve SOTA performance on mislabel detection tasks on hard datasets under symmetric and asymmetric label noise. Our code is publicly available at https://github.com/voxel51/reconstruction-error-ratios.

Figures

Figures reproduced from arXiv: 2412.02596 by the authors.

Figure 1
Figure 1. Reconstruction error distributions for in-class and out-of-class samples shown for the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Visualization of χ for the easiest (left) and hardest (right) samples in CIFAR-10, using CLIP ViT-B/32 features used to train class reconstructors. Images generated using the Fiftyone library (Moore & Corso, 2020). 4 RERS AND CLASSIFICATION DIFFICULTY 4.1 RERS AS DATASET DETERMINANTS Now we turn our attention to a specific reconstruction error ratio. Let χ(x c ) = ∆c (x c ) minc ′̸=c ∆c ′ (xc) , (3) be the ratio of … view at source ↗
Figure 3
Figure 3. Scatterplot of SOTA classification error rate (plotted on a logarithmic scale) for [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Dependence of dataset difficulty measure [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Relationship between χ using CLIP ViT-B/32 features and symmetric, asymmetric, and confidence-based label noise for five exemplary datasets. Each point in the plot is generated by averaging over three random noise initializations. denote the average reconstruction erro…
Figure 6
Figure 6. Figure 6: F1-scores for Zero-Shot, Confident Learning, and RER-based mislabel detection methods on four exemplary datasets. RER thresholds are selected using ansatz (9). All three methods are compared using the same CLIP ViT-B/32 features. Each point represents an average over t…
Figure 7
Figure 7. Figure 7: Hyperparameter sweeps for parametric reconstruction loss weight (left) and [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Finite-size scaling behavior of χ0 and χrand using CLIP ViT-B/32 features and fitting to equations of the form Eq. (5) with exponent 1.808. B.2 ESTIMATING THE NOISE RATE IN THE DATASET Let x c˜ denote that sample x has been assigned noisy label c˜, which may or not be …
Figure 9
Figure 9. Figure 9: Estimated noise in the dataset versus noise added for 9 exemplary datasets. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Correlations among χ values generated for all 18 non-ImageNet datasets considered when training autoencoders on various features. Within model families, (CLIP ViT-B/32 ↔ CLIP ViT-L/14 and DINOv2-B ↔ DINOv2-S) there is strong positive correlation. CLIP and DINOv2- styl…
Figure 11
Figure 11. Figure 11: Alignment in ordering of χ(x) across multiple CLIP and DINOv2 models. Spearman Rank Correlation (first and second plots) characterizes the overall quality of the alignment, whereas nDCG (third and fourth plots) more heavily weights the top portion of the ranking. C AD…
Figure 12
Figure 12. Figure 12: Scatterplot of best classification error rate from MedMNISTv2 paper Yang et al. (2023) [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: AUROC scores for binary mislabel detection tasks on four datasets. [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Average AUROC across all 5 hard classification datasets (which have SOTA classification accuracy < 0.95). 0.80 0.85 0.90 0.95 1.00 Symmetric CIFAR-10 Fit Fraction 0.01 0.05 0.1 0.25 0.5 0.75 1 CIFAR-100 0.10 0.15 0.20 0.25 0.30 Noise Fraction 0.80 0.85 0.90 0.95 1.00 …
Figure 15
Figure 15. Figure 15: AUROC scores for CIFAR-10 and CIFAR-100 with reconstructors fitted using a fraction [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: Likelihood of label mistakes as a function of reconstruction error ratio [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]
Figure 17
Figure 17. Figure 17: Comparison of standard F1-score and confidence-weighted F1-score for three exemplary datasets across three varieties of label noise. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_17.png]
Figure 18
Figure 18. Figure 18: NCFD scores for mistakenness probabilities arising from reconstruction error ratios, [PITH_FULL_IMAGE:figures/full_fig_p029_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 18 canonical work pages

  1. [1]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  2. [2]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  3. [3]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page

  4. [4]

    Food-101 -- mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 -- mining discriminative components with random forests. In European Conference on Computer Vision, 2014

    2014

  5. [5]

    Bourlard and Y

    H. Bourlard and Y. Kamp. Auto-association by multilayer perceptrons and singular value decomposition. Biological Cybernetics, 59 0 (4): 0 291--294, 1988. doi:10.1007/BF00332918. URL https://doi.org/10.1007/BF00332918

    doi:10.1007/bf00332918 1988

  6. [6]

    Remote sensing image scene classification: Benchmark and state of the art

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105 0 (10): 0 1865--1883, Oct 2017. ISSN 1558-2256. doi:10.1109/jproc.2017.2675998. URL http://dx.doi.org/10.1109/JPROC.2017.2675998

    arXiv 2017

  7. [7]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition ( CVPR ) , 2014

    work page 2014

  8. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 248--255, 2009 b . doi:10.1109/CVPR.2009.5206848

    arXiv 2009

Show all 50 references
  1. [10]

    Understanding dataset difficulty with v-usable information

    Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with v-usable information. In International Conference on Machine Learning, 2021. URL https://api.semanticscholar.org/CorpusID:250340652

    2021

  2. [11]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Pattern Recognition Workshop, 2004

    2004

  3. [12]

    Caltech-256 object category dataset

    Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. California Institute of Technology, 2007

    2007

  4. [13]

    Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. In IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, pp.\ 204--207. IEEE, 2018

    2018

  5. [14]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019

    2019

  6. [15]

    Autoencoders, minimum description length and helmholtz free energy

    Geoffrey E Hinton and Richard Zemel. Autoencoders, minimum description length and helmholtz free energy. In J. Cowan, G. Tesauro, and J. Alspector (eds.), Advances in Neural Information Processing Systems, volume 6. Morgan-Kaufmann, 1993. URL https://proceedings.neurips.cc/pap...

    1993

  7. [16]

    Tin Kam Ho and M. Basu. Complexity measures of supervised classification problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24 0 (3): 0 289--300, 2002. doi:10.1109/34.990132

    2002 doi

  8. [17]

    Papadopoulos, and Vittorio Ferrari

    Radu Tudor Ionescu, Bogdan Alexe, Marius Leordeanu, Marius Popescu, Dim P. Papadopoulos, and Vittorio Ferrari. How hard can it be? estimating the difficulty of visual search in an image. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 2157--2166...

    2016 doi

  9. [18]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.08361, 2020. URL https://arxiv.org/abs/2001.08361

    2001 arXiv

  10. [19]

    Novel dataset for fine-grained image categorization

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, June 2011

    2011

  11. [20]

    Auto-encoding variational bayes, 2022

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022. URL https://arxiv.org/abs/1312.6114

    2022 arXiv

  12. [21]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

    2009

  13. [22]

    Mnist handwritten digit database

    Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010

    2010

  14. [23]

    Oxford University

    S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. Technical report, "Oxford University", 2013

    2013

  15. [24]

    Umap: Uniform manifold approximation and projection

    Leland McInnes, John Healy, Nathaniel Saul, and Lukas Gro berger. Umap: Uniform manifold approximation and projection. J. Open Source Softw., 3: 0 861, 2018. URL https://api.semanticscholar.org/CorpusID:53244226

    2018

  16. [25]

    B. E. Moore and J. J. Corso. Fiftyone. GitHub. Note: https://github.com/voxel51/fiftyone, 2020

    2020

  17. [26]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, Dec 2008

    2008

  18. [27]

    Confident learning: Estimating uncertainty in dataset labels

    Curtis Northcutt, Lu Jiang, and Isaac Chuang. Confident learning: Estimating uncertainty in dataset labels. J. Artif. Int. Res., 70: 0 1373–1411, May 2021. ISSN 1076-9757. doi:10.1613/jair.1.12125. URL https://doi.org/10.1613/jair.1.12125

    2021 doi

  19. [28]

    Konovalov, Bronson Philippa, Peter Ridd, Jake C

    Alex Olsen, Dmitry A. Konovalov, Bronson Philippa, Peter Ridd, Jake C. Wood, Jamie Johns, Wesley Banks, Benjamin Girgenti, Owen Kenny, James Whinney, Brendan Calvert, Mostafa Rahimi Azghadi , and Ronald D. White. DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Lear...

    2019 doi

  20. [29]

    Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Micha...

    2024

  21. [30]

    Elenberg, and Kilian Q

    Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, and Kilian Q. Weinberger. Identifying mislabeled data using the area under the margin ranking. In Advances in Neural Information Processing Systems, 2020. URL https://proceedings.neurips.cc/paper/2020/file/c6102b3727b2a7d8b1bb6981...

    2020

  22. [31]

    Estimating training data influence by tracing gradient descent

    Garima Pruthi, Frederick Liu, Mukund Sundararajan, and Satyen Kale. Estimating training data influence by tracing gradient descent. In Advances in Neural Information Processing Systems (NeurIPS), 2020. URL https://api.semanticscholar.org/CorpusID:211204970

    2020

  23. [32]

    Quattoni and A

    A. Quattoni and A. Torralba. Recognizing indoor scenes. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 413--420. IEEE, 2009. doi:10.1109/CVPRW.2009.5206537. URL https://doi.org/10.1109/CVPRW.2009.5206537. Indoor Scene Recognition Dataset available at ...

    2009

  24. [33]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018

    2018

  25. [34]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https://ar...

    2021 arXiv

  26. [35]

    Real-time flying object detection with yolov8, 2024

    Dillon Reis, Jordan Kupec, Jacqueline Hong, and Ahmad Daoudi. Real-time flying object detection with yolov8, 2024. URL https://arxiv.org/abs/2305.09972

    2024 arXiv

  27. [36]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \" o rn Ommer. High-resolution image synthesis with latent diffusion models. CoRR, abs/2112.10752, 2021. URL https://arxiv.org/abs/2112.10752

    2021 arXiv

  28. [37]

    Rumelhart, Geoffrey E

    David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors. Nature, 323 0 (6088): 0 533--536, 1986. doi:10.1038/323533a0. URL https://doi.org/10.1038/323533a0

    1986 doi

  29. [38]

    Parametric umap embeddings for representation and semisupervised learning

    Tim Sainburg, Leland McInnes, and Timothy Q Gentner. Parametric umap embeddings for representation and semisupervised learning. Neural Computation, 33 0 (11): 0 2881--2907, 2021

    2021

  30. [39]

    Efficient image dataset classification difficulty estimation for predicting deep-learning accuracy

    Florian Scheidegger, Roxana Istrate, Giovanni Mariani, Luca Benini, Costas Bekas, and Cristiano Malossi. Efficient image dataset classification difficulty estimation for predicting deep-learning accuracy. The Visual Computer, 37 0 (6): 0 1593--1610, 2021. ISSN 1432-2315. doi:1...

    2021 doi

  31. [41]

    Maya Srikanth, Jeremy Irvin, Brian Wesley Hill, Felipe Godoy, Ishan Sabane, and Andrew Y. Ng. An empirical study of automated mislabel detection in real world vision datasets, 2023 b . URL https://arxiv.org/abs/2312.02200

    2023 arXiv

  32. [42]

    Extracting and composing robust features with denoising autoencoders

    Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pp.\ 1096–1103, New York, NY, USA, 2008. Associ...

    2008

  33. [43]

    Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol

    Pascal Vincent, H. Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11: 0 3371--3408, 2010. URL https://api.semanti...

    2010

  34. [44]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011

    2011

  35. [45]

    Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

    Han Xiao , Kashif Rasul , and Roland Vollgraf . Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms . arXiv e-prints, art. arXiv:1708.07747, August 2017. doi:10.48550/arXiv.1708.07747

    2017 doi arXiv

  36. [46]

    Xiao , J

    J. Xiao , J. Hays , K. A. Ehinger , A. Oliva , and A. Torralba . Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.\ 3485--3492, June 2010. doi:10.1109/CVPR.2010.5539970

    2010

  37. [47]

    Medmnist v2 - a large-scale lightweight benchmark for 2d and 3d biomedical image classification

    Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2 - a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10 0 (1), January 2023. ISSN 2052-4463. doi:10.1038/s415...

    2023 doi

  38. [48]

    DIME : An information-theoretic difficulty measure for AI datasets

    Peiliang Zhang, Huan Wang, Nikhil Naik, Caiming Xiong, and richard socher. DIME : An information-theoretic difficulty measure for AI datasets. In NeurIPS 2020 Workshop: Deep Learning through Information Geometry, 2020. URL https://openreview.net/forum?id=kvqPFy0hbF

    2020

  39. [49]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 40, pp.\ 1452--1464. IEEE, 2017. doi:10.1109/TPAMI.2017.2723009. U...

    2017

  40. [50]

    Detecting corrupted labels without training a model to predict

    Zhaowei Zhu, Zihao Dong, and Yang Liu. Detecting corrupted labels without training a model to predict. In International Conference on Machine Learning, 2021. URL https://api.semanticscholar.org/CorpusID:246431058

    2021

  41. [51]

    Unmasking and improving data credibility: A study with datasets for training harmless language models

    Zhaowei Zhu, Jialu Wang, Hao Cheng, and Yang Liu. Unmasking and improving data credibility: A study with datasets for training harmless language models. arXiv preprint arXiv:2311.11202, 2023

    2023 arXiv

  42. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.