Pith. sign in

REVIEW 4 major objections 5 minor 46 references

CUAL: Continual Uncertainty-aware Active Learner

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CUAL claims that when old and new classes co-occur in an unlabeled stream, the most informative samples to label are the ambiguous ones, not the most confidently novel, and that confident pseudo-labels can stretch a tiny labeling budget.

desk verdict Solid and genuinely new continual active learning recipe, but the pseudo-labeling branch that drives much of the gain is validated only indirectly—worth refereeing, not desk-rejecting. read the letter →

arxiv 2412.09701 v1 pith:YGJFCKNJ submitted 2024-12-12 cs.LG cs.AI

Amanda Rios , Ibrahima Ndiour , Parual Datta , Jerry Sydir , Omesh Tickoo , Nilesh Ahuja This is my paper · ORCID
classification cs.LGcs.AI
keywords continuallearningactivenovelclassdiscoveryuncertaintyestimationpseudo-labelingout-of-distributiondetectionfeaturereconstructionerrorexperiencereplay
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a realistic operational problem: a deployed classifier receives a stream of unlabeled images that mix old, previously learned classes with entirely new classes, and it has only a tiny labeling budget to keep learning. CUAL, the proposed method, actively asks a human to label the ambiguous samples—the ones that sit near the boundary between novel and familiar—rather than the most confidently novel ones, and simultaneously pseudo-labels the small fraction of samples it is most certain are new. The paper reports that this combination beats the usual continual-learning baselines (experience replay with entropy sampling, replay with pseudo-labeling, and the semi-supervised continual learner CCIC) across four datasets and two frozen feature backbones, at labeling budgets of 0.5–2.5%. The central claim is that ambiguity, not novelty confidence, is the right criterion for spending a scarce labeling budget when old and new classes co-occur.

What carries the argument

The load-bearing object is the uncertainty score $$S_i(u)=\frac{\min_{j\in\mathcal{C}_{\text{old}}}\mathrm{FRE}$_j^{0}$(u)}{\mathrm{FRE}_{m}^{i-1}(u)},\quad m=A^s_{cl}(u,i-1,t)\in\mathcal{C}^t_{\text{new}},$$ where $\mathrm{FRE}_m$ is a per-class feature reconstruction error: the $\ell^2$ distance between a deep feature and its reconstruction after projection into a class-specific PCA subspace and back. FRE is built for each old class at task 0 and for each discovered novel class from the actively labeled samples of the previous inner-loop iteration. The score separates unlabeled samples into confidently novel (high), confidently old (low), and ambiguous (near the inner-loop threshold $T_{\text{inner}}$); ambiguity is formalized as $1/\lVert S_i(u)-T_{\text{inner}}\rVert^2$, so active queries select the samples closest to the old/new decision boundary, while the top $\alpha=20\%$ of high scores are pseudo-labeled. The short-term head $A^s_{cl}$ maps features to novel-class labels and routes the FRE selection; the long-term head is updated at task end by experience replay.

What would settle it

Track the precision of CUAL's pseudo-labels on each task's novel-class pool. If the top $\alpha=20\%$ of highest-score samples are frequently mislabeled (for example, more than a small fraction are actually old-class outliers), or if training the same experience-replay procedure with corrupted pseudo-labels at that rate reproduces CUAL's accuracy drop, the uncertainty-scoring premise is falsified. A sharp test is to run CUAL with $\alpha$ varied from 0% to 50% and check whether accuracy peaks near the claimed confident region or degrades immediately.

Watch

Extended reading notes

Core claim

The paper's central claim is that when an unlabeled continual-learning stream mixes old and new classes, the most valuable labels come from samples whose uncertainty score falls in the middle band: far enough from the old classes to look novel but close enough to the predicted novel class to be uncertain. CUAL computes, for each sample and each old class, a per-class feature reconstruction error (FRE), then forms a score $S_i(u)$ as the ratio of the minimum old-class error to the error under the novel class the pseudo-label head predicts. High ratios mark confidently novel samples, low ratios mark familiar samples, and samples near the threshold are ambiguous. The labeling budget is spent on the ambiguous band, while the top $\alpha=20\%$ of confidently novel samples receive pseudo-labels and both sets are used to retrain the heads and update the long-term classifier via experience replay. Across ImageNet21K-OOD, Places365, Eurosat, and Cifar100-superclasses, with frozen ResNet50 and ViT-S/16 features, the paper reports average accuracies such as 86.6% versus 76.1% for entropy-based experience replay on ImageNet21K-OOD; ablations show that removing pseudo-labeling, or querying the most novel or random samples, drops performance substantially, and that an eight-fold larger labeling budget without pseudo-labeling still trails the full method.

Load-bearing premise

The method assumes that the most confident fifth of the samples it flags as novel are genuinely from a novel class, even though the classifier assigning those pseudo-labels was trained on only a tiny set of human-labeled examples; if those pseudo-labels are wrong, training on them can degrade performance rather than help.

Editorial extensions

If this is right

  • If the results hold, continual learners can operate in mixed old/new unlabeled streams with tiny labeling budgets (0.5–2.5%) instead of assuming the oracle condition that only novel classes appear.
  • Spending the labeling budget on ambiguous samples—those near the old/new boundary—matters more than selecting the most novel-looking samples; ablations show AL-Top and AL-Rand lag CUAL by 7–11 points on ImageNet21K-OOD.
  • Pseudo-labeling the most confident novel samples at $\alpha=20\%$ adds more value than multiplying the active labeling budget by eight without pseudo-labeling (83.6 vs 86.6 on ImageNet21K-OOD).
  • The advantage is reproducible across different frozen feature extractors (ResNet50 and ViT-S/16), indicating the method does not depend on one backbone's geometry.
  • Semi-supervised continual baselines that assume old and new classes do not co-occur (CCIC) fall well behind when that assumption is dropped.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication left implicit is that the uncertainty scoring function does not depend on FRE being the only possible metric; the paper states CUAL is agnostic to the elemental uncertainty measure, so substituting a different per-class distance (for instance a nearest-classifier distance or a calibrated softmax score) would be a direct test of the method's generality.
  • The reported scaling with class increments (90.7% for one new class per task down to 83.6% for seven on ImageNet21K-OOD) suggests the confident-novel set becomes noisier as more classes arrive at once; a stress experiment with many novel classes per task and a fixed small budget would reveal whether the pseudo-labeling branch needs an early correction.
  • Because all experiments mix old and new samples at a fixed 2:1 ratio, a natural next test is to vary the novelty rate, especially making novel classes rare; in that regime the ambiguity band may contain mostly old-class samples, and CUAL's advantage over random labeling could narrow.
  • The paper explicitly leaves distribution shift of old classes for future work, so the current method should not be read as handling appearance changes in previously learned classes inside the unlabeled stream.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CUAL, a continual active learning method for a setting where unlabeled post-deployment data contains both unseen old-class samples and samples from novel classes, under a tiny labeling budget. CUAL builds a score S_i(u) that combines a feature-reconstruction-error (FRE) distance to old classes in the numerator and to a pseudo-predicted novel class in the denominator. Samples are partitioned into confident-novel, confident-old, and ambiguous; the method actively labels the ambiguous ones and pseudo-labels the top α=20% most confident novel samples, updating a short-term novel-class head and the long-term classifier via experience replay. Experiments compare CUAL with ER, PseudoER, and CCIC on four reported datasets (Imagenet21K-OOD, Places365, Eurosat, Cifar100-superclasses) and two frozen backbones, with ablations and budget sweeps, reporting accuracy advantages such as 86.6 vs. 76.1 on Im21K with ResNet50.

Significance. If the reported results are reliable, CUAL addresses a realistic and under-explored problem: continual active learning without the oracle assumption that new and old classes do not co-occur. The paper is among the first to propose ambiguity-based querying in continual AL, and the combination of active labeling and confidence-gated pseudo-labeling is a sensible mechanism. Strengths include the clear problem formulation, the use of a published uncertainty metric (FRE) with prior external benchmarking, ablations isolating the active-labeling and pseudo-labeling branches, and experiments across two backbones and multiple datasets. The main weaknesses that constrain the significance are the absence of any error bars or significance tests and the lack of any direct measurement or control for pseudo-label correctness, which is the mechanism behind the largest ablation gap.

major comments (4)
  1. [Section 3.1, Figure 2] No error bars or statistical significance tests are reported for any of the central comparisons. All claims that CUAL outperforms ER, PseudoER, and CCIC rest on single point estimates per dataset and backbone. Given that the margins in the left table of Figure 2 range from about 1.7 points (Eurosat ViT) to 11.7 points (Im21K R50), it is unknown whether the smallest gaps are within run-to-run noise. The authors should report results over multiple seeds (at least 3, preferably 5) with standard deviations, and where feasible a paired significance test, for the main tables and the ablations in Figure 2.
  2. [Section 2.2.2, Eq. (2), Appendix 4.1.1] The pseudo-labeling branch is the load-bearing component of the reported gains, but its reliability is never measured. The only-AL ablation in Figure 2 (right) shows a large drop relative to the default CUAL (e.g., Im21K R50: 86.6 vs. 79.1; Eurosat R50: 95.8 vs. 74.8), which indicates that the advantage largely depends on the correctness of the top-α pseudo-labels selected via Eq. (2) with α=20%. However, the paper reports no pseudo-label accuracy or precision, and it does not include a control experiment that replaces pseudo-labels with ground-truth novel-class labels. Since Section 2.2.2 itself warns that wrong pseudo-labels can cause severe degradation, the central claim requires at least (i) a measurement of pseudo-label precision on the selected α=20% samples, and (ii) an oracle-pseudo-label experiment to establish the ceiling of the pseudo-labeling branch.
  3. [Section 3.1, Section 3.2] There is a mismatch between the listed datasets and the reported experiments. The text states 'We test on 4 diverse datasets' but then enumerates five: Imagenet21K-OOD, Places365-OOD, Eurosat, iNaturalist-Plants-20 (Plants), and Cifar100-superclasses. All result tables (Figure 1 and Figure 2) report only four datasets, and iNaturalist-Plants-20 does not appear in any table or figure. Either report the Plants results in the main paper or in the appendix, or correct the dataset list in the text to match the experiments actually presented.
  4. [Section 4.2.3] The description of how the ER and PseudoER baselines select 'ambiguous' samples is unclear and potentially not a fair comparison. The text says that, similar to CUAL, the baselines actively label ambiguous samples 'according to the same formula as outlined in appendix 4.1.2', but that formula is defined specifically in terms of CUAL's S_i(u) score, which uses FRE and the pseudo-labeling head. For entropy-based ER and PseudoER, it is not clear how this formula is instantiated. Please specify the exact ambiguity score used for each baseline, and clarify whether the baselines have access to the same T_inner and PCA machinery; if they do not, the comparison may be giving CUAL an advantage in query selection that is not a property of the proposed method alone.
minor comments (5)
  1. [Section 3.1] The phrase 'We test on 4 diverse datasets' should be revised to either 'five' or the list shortened to four, since the mismatch with the enumerated dataset list is confusing.
  2. [Equation (2)] In Eq. (2), the notation F RE_{m}^{i-1}(u) should be defined more precisely: the superscript i-1 indexes the PCA transform from the previous inner-loop iteration, while the subscript m is the pseudo-label predicted by A_s^cl(u; i-1, t). Make the dependence of the transform T_m^{t,i-1} explicit to avoid confusion.
  3. [Figure 2 caption] The left table in Figure 2 is referred to as 'Default CUAL and baselines' but the rows include 'Oracle', 'CUAL', 'ER-Ent', 'ER-Rand', etc. The caption would be clearer if it stated that the table reports average accuracy over all tasks for each dataset and backbone.
  4. [Section 4.1.1] The thresholds T_inner and α are described as 'not highly sensitive', but no quantitative evidence or sensitivity analysis is provided. A short table or figure showing performance for a few values of α (e.g., 10%, 20%, 30%) and for T_inner at 1, 2, and 3 standard deviations would substantiate this claim.
  5. [Section 4.2.1] There is a minor grammar issue: 'the latter Acl is same for all baselines' should be 'the latter Acl is the same for all baselines'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CUAL is an empirical system whose uncertainty component comes from independently published work, and no reported number is forced by construction.

full rationale

CUAL's central claim is an empirical performance comparison, not a derivation from first principles. The uncertainty score in Eq. (2) is built from FRE (Eq. 3), a metric introduced in the authors' prior work [16] and validated in a separate published paper [17]; those sources are externally evaluated and their stated assumptions do not include CUAL's target accuracy figures. The selection thresholds (Tinner at 2 standard deviations, pseudo-label cutoff alpha = 20%) are fixed hyperparameters rather than quantities fitted to the reported test accuracies, so no reported number is forced by construction. The pseudo-labeling branch (Section 2.2.2, Eq. 4 with weight gamma = 0.25) is an empirical design choice whose benefit is supported by the only-AL ablation; the absence of measured pseudo-label precision is a correctness or robustness weakness, not circularity, because the paper does not define its success metric in terms of those pseudo-labels. The self-citations to [16,17] are component-level and independently benchmarked, so they do not form a load-bearing self-citation chain. Overall, no equation in the paper reduces to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a chain of imported and hand-set assumptions: the reliability of FRE as a distance metric, the sufficiency of frozen features, the informativeness of the ambiguity definition, the representativeness of the holdout, and the correctness of confident pseudo-labels. None of these is derived in this paper, so they are tracked here as free parameters and axioms.

free parameters (4)
  • Pseudo-label selection ratio alpha = 20%
    Set in Appendix 4.1.1; controls the fraction of predicted-novel samples that receive pseudo-labels, directly determining the amount of synthetic training signal.
  • Novelty threshold Tinner = 2 standard deviations above in-distribution validation mean
    Set in Appendix 4.1.1; defines the old/new boundary used for ambiguity scoring, stopping, and confidence filtering.
  • ER loss weights beta, gamma, theta = 0.25, 0.25, 0.5
    Manually chosen in Appendix 4.1.4 to balance active-labeled, pseudo-labeled, and replay loss terms; affects the final classifier update.
  • Per-class PCA subspace dimension = not specified
    FRE requires a PCA transform per class (Appendix 4.1.3), but the number of principal components is not stated anywhere in the paper, leaving an essential tuning knob underspecified.
assumptions (5)
  • domain assumption FRE, computed as per-class PCA reconstruction error, reliably measures sample-to-class distance in the frozen feature space.
    The entire uncertainty score Si(u) in Eq. 2 is built from FRE; if FRE fails to rank old-vs-novel distances correctly, both active queries and pseudo-labels are misdirected. The paper imports this from refs [16,17] without re-deriving it.
  • domain assumption Frozen foundation-model features are sufficient to separate all classes in the four test datasets without backbone fine-tuning.
    All methods, including the Oracle, use a frozen ResNet50-SwAV or ViT-S16-DINO extractor (Section 3.1, Appendix 4.2.1); if these features cannot discriminate the classes, the results would not transfer to a real deployable system.
  • ad hoc to paper The ambiguity score, inverse squared distance to Tinner, identifies samples whose labels will most improve the short-term learner.
    Defined in Appendix 4.1.2 without a derivation or external validation; the AL-Top and AL-Rand ablations provide the only empirical support, and no statistical test is given.
  • domain assumption The old-class holdout set (0.35% of each dataset) is representative of the previously learned classes used to form Uold(t).
    Section 3.1 builds each task's unlabeled pool from this holdout; if the holdout is not representative, the old/new mix is artificial and the benchmark's conclusions may not generalize.
  • ad hoc to paper The top alpha% of samples ranked by Si(u) have correct novel-class pseudo-labels, despite being produced by a head trained on a tiny labeled set.
    Section 2.2.2 selects 'the topmost most confident alpha percent of samples to pseudo-label' and Appendix 4.1.1 sets alpha=20%; no pseudo-label accuracy is reported, and the authors acknowledge wrong pseudo-labels can severely degrade performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CUAL: Continual Uncertainty-aware Active Learner." pith.science (2026). https://pith.science/paper/YGJFCKNJ

@misc{pith2026241209701,
  author       = {Pith},
  title        = {Pith review of: CUAL: Continual Uncertainty-aware Active Learner},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YGJFCKNJ}},
  note         = {Machine review of arXiv:2412.09701}
}
read the original abstract

AI deployed in many real-world use cases should be capable of adapting to novelties encountered after deployment. Here, we consider a challenging, under-explored and realistic continual adaptation problem: a deployed AI agent is continuously provided with unlabeled data that may contain not only unseen samples of known classes but also samples from novel (unknown) classes. In such a challenging setting, it has only a tiny labeling budget to query the most informative samples to help it continuously learn. We present a comprehensive solution to this complex problem with our model "CUAL" (Continual Uncertainty-aware Active Learner). CUAL leverages an uncertainty estimation algorithm to prioritize active labeling of ambiguous (uncertain) predicted novel class samples while also simultaneously pseudo-labeling the most certain predictions of each class. Evaluations across multiple datasets, ablations, settings and backbones (e.g. ViT foundation model) demonstrate our method's effectiveness. We will release our code upon acceptance.

Figures

Figures reproduced from arXiv: 2412.09701 by the authors.

Figure 1
Figure 1. (Left A.1-C.1) Continual classification accuracy over continual tasks. The number of novel classes introduced per task is in parenthesis. CUAL over-performs other methods in this challenging setting. The Oracle (gray) is fully supervised ER.(Right A.2-C.2) Results varying the AL budget [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Continual Classification Results, averaged over all tasks. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results for Cifar100; (Left D.1) Continual classification accuracy over continual tasks. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 19 canonical work pages

  1. [1]

    Lifelong Learning Without a Task Oracle

    Amanda Rios and Laurent Itti. “Lifelong Learning Without a Task Oracle”. In: 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) . IEEE. 2020, pp. 255–263

    2020

  2. [2]

    Continual lifelong learning with neural networks: A review

    German I Parisi et al. “Continual lifelong learning with neural networks: A review”. In:Neural Networks 113 (2019), pp. 54–71

    2019

  3. [3]

    Rethinking experience replay: a bag of tricks for continual learning

    Pietro Buzzega et al. “Rethinking experience replay: a bag of tricks for continual learning”. In: 2020 25th International Conference on Pattern Recognition (ICPR). IEEE. 2021, pp. 2180– 2187

    2020

  4. [4]

    A soft nearest-neighbor framework for continual semi-supervised learn- ing

    Zhiqi Kang et al. “A soft nearest-neighbor framework for continual semi-supervised learn- ing”. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, pp. 11868–11877

    2023

  5. [5]

    Continual semi-supervised learning through contrastive interpolation consistency

    Matteo Boschini et al. “Continual semi-supervised learning through contrastive interpolation consistency”. In: Pattern Recognition Letters162 (Oct. 2022), pp. 9–14. ISSN : 0167-8655. DOI: 10.1016/j.patrec.2022.08.006 . URL: http://dx.doi.org/10.1016/j.patrec. 2022.08.006

    doi:10.1016/j.patrec.2022.08.006 2022

  6. [6]

    Beyond Supervised Continual Learning: a Review

    Benedikt Bagus, Alexander Gepperth, and Timothée Lesort. Beyond Supervised Continual Learning: a Review. 2022. arXiv: 2208.14307 [cs.LG]

    arXiv 2022

  7. [7]

    A survey of deep active learning

    Pengzhen Ren et al. “A survey of deep active learning”. In: ACM computing surveys (CSUR) 54.9 (2021), pp. 1–40

    work page 2021

  8. [8]

    Learning loss for active learning

    Donggeun Yoo and In So Kweon. “Learning loss for active learning”. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, pp. 93–102

    2019

Show all 46 references
  1. [9]

    Active Learning for Convolutional Neural Networks: A Core-Set Approach

    Ozan Sener and Silvio Savarese. “Active Learning for Convolutional Neural Networks: A Core-Set Approach”. In: International Conference on Learning Representations. 2018

    2018

  2. [10]

    Deep bayesian active learning with image data

    Yarin Gal, Riashat Islam, and Zoubin Ghahramani. “Deep bayesian active learning with image data”. In: International Conference on Machine Learning. PMLR. 2017, pp. 1183–1192

    2017

  3. [11]

    How to measure uncertainty in uncertainty sampling for active learning

    Vu-Linh Nguyen, Mohammad Hossein Shaker, and Eyke Hüllermeier. “How to measure uncertainty in uncertainty sampling for active learning”. In: Machine Learning 111.1 (2022), pp. 89–122

    2022

  4. [12]

    Few-shot continual active learning by a robot

    Ali Ayub and Carter Fendley. “Few-shot continual active learning by a robot”. In: Advances in Neural Information Processing Systems 35 (2022), pp. 30612–30624

    2022

  5. [13]

    Active Continual Learning: On Balancing Knowledge Retention and Learnability

    Thuy-Trang Vu et al. “Active Continual Learning: On Balancing Knowledge Retention and Learnability”. In: arXiv preprint arXiv:2305.03923 (2023)

    2023 arXiv

  6. [14]

    Online active continual learning for robotic lifelong object recognition

    Xiangli Nie et al. “Online active continual learning for robotic lifelong object recognition”. In: IEEE Transactions on Neural Networks and Learning Systems (2023)

    2023

  7. [15]

    Most-surely vs. least-surely uncertain

    Manali Sharma and Mustafa Bilgic. “Most-surely vs. least-surely uncertain”. In: 2013 IEEE 13th International Conference on Data Mining. IEEE. 2013, pp. 667–676

    2013

  8. [16]

    Out-Of-Distribution Detection With Subspace Techniques And Probabilistic Modeling Of Features

    Ibrahima Ndiour, Nilesh A Ahuja, and Omesh Tickoo. “Out-Of-Distribution Detection With Subspace Techniques And Probabilistic Modeling Of Features”. In: arXiv preprint arXiv:2012.04250 (2020)

    2020 arXiv

  9. [17]

    incdfm: Incremental deep feature modeling for continual novelty detec- tion

    Amanda Rios et al. “incdfm: Incremental deep feature modeling for continual novelty detec- tion”. In: European Conference on Computer Vision. Springer. 2022, pp. 588–604

    2022

  10. [18]

    Experience replay for continual learning

    David Rolnick et al. “Experience replay for continual learning”. In: Advances in Neural Information Processing Systems 32 (2019)

    2019

  11. [19]

    Deep residual learning for image recognition

    Kaiming He et al. “Deep residual learning for image recognition”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778

    2016

  12. [20]

    Unsupervised learning of visual features by contrasting cluster as- signments

    Mathilde Caron et al. “Unsupervised learning of visual features by contrasting cluster as- signments”. In: Advances in Neural Information Processing Systems 33 (2020), pp. 9912– 9924. 5

    2020

  13. [21]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy Alexey. “An image is worth 16x16 words: Transformers for image recognition at scale”. In: arXiv preprint arXiv: 2010.11929 (2020)

    2020 arXiv

  14. [22]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron et al. “Emerging properties in self-supervised vision transformers”. In: Pro- ceedings of the IEEE/CVF international conference on computer vision . 2021, pp. 9650– 9660

    2021

  15. [23]

    ImageNet-21K Pretraining for the Masses

    Tal Ridnik et al. ImageNet-21K Pretraining for the Masses . 2021. arXiv: 2104 . 10972 [cs.CV]

    2021

  16. [24]

    Places: A 10 million Image Database for Scene Recognition

    Bolei Zhou et al. “Places: A 10 million Image Database for Scene Recognition”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence(2017)

    2017

  17. [25]

    EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification

    Patrick Helber et al. EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. 2019. arXiv: 1709.00029 [cs.CV]

    2019 arXiv

  18. [26]

    The iNaturalist Species Classification and Detection Dataset

    Grant Van Horn et al. The iNaturalist Species Classification and Detection Dataset . 2018. arXiv: 1707.06642 [cs.CV]

    2018 arXiv

  19. [27]

    Learning Multiple Layers of Features from Tiny Images

    Alex Krizhevsky. “Learning Multiple Layers of Features from Tiny Images”. In: University of Toronto(May 2012)

    2012

  20. [28]

    Mixmatch: A holistic approach to semi-supervised learning

    David Berthelot et al. “Mixmatch: A holistic approach to semi-supervised learning”. In: Advances in neural information processing systems 32 (2019)

    2019

  21. [29]

    Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks

    Dong-Hyun Lee. “Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks”. In:ICML 2013 Workshop : Challenges in Representation Learning (WREPL) (July 2013)

    2013

  22. [30]

    Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data

    Yen-Chang Hsu et al. “Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, pp. 10951–10960

    2020

  23. [31]

    Closed-loop memory GAN for continual learning

    Amanda Rios and Laurent Itti. “Closed-loop memory GAN for continual learning”. In: arXiv preprint arXiv:1811.01146 (2018)

    2018 arXiv

  24. [32]

    Head2toe: Utilizing intermediate representations for better transfer learning

    Utku Evci et al. “Head2toe: Utilizing intermediate representations for better transfer learning”. In: International Conference on Machine Learning. PMLR. 2022, pp. 6009–6033

    2022

  25. [33]

    The dynamics of perceptual learning: an incremental reweighting model

    Alexander A Petrov, Barbara Anne Dosher, and Zhong-Lin Lu. “The dynamics of perceptual learning: an incremental reweighting model.” In: Psychological review 112.4 (2005), p. 715

    2005

  26. [34]

    A baseline for few-shot image classification

    Guneet S Dhillon et al. “A baseline for few-shot image classification”. In: arXiv preprint arXiv:1909.02729 (2019)

    2019 arXiv

  27. [35]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. “Adam: A method for stochastic optimization”. In: arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  28. [36]

    ImageNet Large Scale Visual Recognition Challenge

    Olga Russakovsky et al. “ImageNet Large Scale Visual Recognition Challenge”. English (US). In: International Journal of Computer Vision 115.3 (Dec. 2015). Publisher Copyright: © 2015, Springer Science+Business Media New York., pp. 211–252. ISSN : 0920-5691. DOI: 10.1007/s11263...

    2015 doi

  29. [37]

    Mos: Towards scaling out-of-distribution detection for large semantic space

    Rui Huang and Yixuan Li. “Mos: Towards scaling out-of-distribution detection for large semantic space”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, pp. 8710–8719

    2021

  30. [38]

    Delving into out-of-distribution detection with vision-language representa- tions

    Yifei Ming et al. “Delving into out-of-distribution detection with vision-language representa- tions”. In: Advances in neural information processing systems 35 (2022), pp. 35087–35102

    2022

  31. [39]

    React: Out-of-distribution detection with rectified activations

    Yiyou Sun, Chuan Guo, and Yixuan Li. “React: Out-of-distribution detection with rectified activations”. In: Advances in Neural Information Processing Systems 34 (2021), pp. 144–157. 4 Appendix 4.1 CUAL Methodology Details 4.1.1 Thresholds for Stopping the AL inner-loop and Pse...

    2021

  32. [40]

    For ViTs16 we tried several extraction points, e.g

    pretrained on Imagenet1K via DINO [22]. For ViTs16 we tried several extraction points, e.g. head, last norm later, different transformer block outputs with different pool factors (e.g. 2,4). Best results were obtained with Block 9 features with pooling range of 2, yielding dee...

  33. [41]

    We use a random set of 500 samples from each of the 50 classes

    Imagenet21K-OOD (Im21K-OOD) [23]: We curated a subset of Imagenet21K containing the top-most populous 50 classes and that do not overlap with the classes present in Imagenet1K. We use a random set of 500 samples from each of the 50 classes. Because Imagenet21K is a superset of...

  34. [42]

    environment

    Places365-OOD (Places) [24, 37]: is a subset of Places365 also originally curated by [37] to contain 51 "environment" categories orthogonal to Imagenet1K, containing a total of 9822 images. It has also been used as a test OOD dataset in [37, 38, 39] with respect to Imagenet1K

  35. [43]

    Eurosat [25]: An RGB dataset of 10 classes and 27K images of Sentinel-2 satellite images, which is also orthogonal to Imagenet1K

  36. [44]

    This totals 20 labels (super) and 50K images

    Cifar100-Superclasses (Cifar100) [27]: We use the super-label granularity of Cifar100 dataset. This totals 20 labels (super) and 50K images. While Cifar100 is not orthogonal to Imagenet1K, we decided to showcase its results since it is a widespread dataset in CL. 8 4.2.3 Basel...

  37. [45]

    to only use actively labeled samples (as embeddings) for replay; (3) We also adapt PseudoER

  38. [46]

    In both ER and PseudoER, we utilize the cumulative classification entropy as an uncertainty score to actively-label and Pseudo-Label (PseudoER)

    to continual AL by further incorporating pseudo-labeling for high confidence unlabeled samples in addition to actively labeled samples to be used for training. In both ER and PseudoER, we utilize the cumulative classification entropy as an uncertainty score to actively-label a...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.