Pith. sign in

REVIEW 3 major objections 6 minor 73 references

Open Set Label Shift with Test Time Out-of-Distribution Reference

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that open set label shift can be estimated and corrected using only a source ID classifier, an ID/OOD score, and a reference OOD set, without retraining the source network.

desk verdict Genuinely useful reduction of open-set label shift to K+1 closed-set label shift with EM and concentration bounds, but the pseudo-OOD reference used in all experiments sits outside the proved guarantees. read the letter →

arxiv 2505.05868 v1 pith:ATSJ6COV submitted 2025-05-09 cs.LG

Changkun Ye , Russell Tsuchida , Lars Petersson , Nick Barnes This is my paper · ORCID
classification cs.LG
keywords opensetlabelshiftestimationout-of-distributiondetectionEMalgorithmmaximumlikelihoodtest-timeadaptationclassifiercorrectionpseudo-OODreference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that when a target test set contains an extra out-of-distribution class, the target class proportions and the fraction of target data that is in-distribution can be recovered from unlabeled target data, a source classifier, an ID/OOD score, and a reference OOD set. The method rewrites the open-set likelihood as a closed-set label-shift likelihood with K+1 classes, so that standard EM machinery applies to the reparameterized target distribution. A three-stage procedure first estimates the source ID ratio, then runs an EM algorithm for the target ID class probabilities and ID ratio, then corrects the target ID ratio when the OOD score is imperfect. If the claims hold, the source classifier can be reweighted to the target domain without retraining or fine-tuning.

What carries the argument

The carrying object is the augmented K+1-class classifier \tilde f(x)_i = h(x)f(x)_i for each ID class and \tilde f(x)_{K+1} = 1-h(x) for the OOD class, paired with the reparameterized source and target label vectors \tilde c = [\rho_s c_1,\dots,\rho_s c_K,1-\rho_s] and \tilde\pi = [\rho_t\pi_1,\dots,\rho_t\pi_K,1-\rho_t]. Writing the target negative log likelihood with these objects turns the open-set likelihood into exactly the closed-set label shift likelihood, so the EM updates of the closed-set problem apply unchanged; the ID data ratio appears as the sum of the first K entries of \tilde\pi. A second mechanism, the ratio correction in Eq. (14), subtracts the offset \hat\mu_0/(\hat\mu_1-\hat\mu_0) to debias the estimated target ID ratio when the OOD score is not a true posterior but still separates ID from OOD on average and behaves uniformly across ID classes.

What would settle it

Take a fixed source dataset and a held-out true OOD set, compute the mean OOD score on the true OOD set and on the pseudo-OOD set of Eq. (15) with the paper's chosen \gamma and T, and check whether the difference exceeds the Hoeffding-scale bound. Alternatively, run the full estimator twice on the same unlabeled target set, once with a genuine OOD reference and once with the pseudo-OOD reference; if the estimated \pi and \rho_t differ by more than the stated concentration bounds, the pseudo-OOD relaxation is invalid.

Watch

Extended reading notes

Core claim

The central claim is that open set label shift reduces to an augmented closed-set label shift problem. By combining a calibrated ID classifier f and an ID/OOD classifier h into the augmented classifier \tilde f with entries h(x)f(x)_i for ID classes and 1-h(x) for the OOD class, the target negative log likelihood becomes identical in form to the K+1-class closed-set label shift likelihood. Maximum likelihood in the reparameterized variable \tilde\pi = [\rho_t\pi_1,\dots,\rho_t\pi_K,1-\rho_t] therefore yields both the target ID label distribution and the target ID data ratio, and the corrected classifier in Eq. (17) approximates p_t(y|x). Theorems 4.1 and 4.4 provide concentration bounds for the source ID ratio estimator and for a corrected target ID ratio estimator under a relaxed assumption on the OOD score.

Load-bearing premise

The whole estimator chain stands on the assumption that the pseudo-OOD reference set, made by adding Gaussian noise to source images and rescaling the OOD score by T, has the same expected score under the OOD classifier as the true out-of-distribution distribution; if that single expectation is off, both the source ID ratio and the corrected target ID ratio become biased and the theorem bounds no longer apply.

Editorial extensions

If this is right

  • Target label proportions and the ID/OOD mix can be estimated from unlabeled target data plus a reference OOD set, removing the need for target labels.
  • The corrected classifier requires no retraining or fine-tuning, so adapting a frozen or expensive source model to a shifting target becomes cheap.
  • Even an imperfect OOD score can be used, as long as it responds differently to ID versus OOD data and uniformly across ID classes, after applying the linear correction of Theorem 4.4.
  • The Hoeffding-based concentration bounds make sample-size requirements explicit: estimation error decays at the usual rate \sqrt{\log(1/\delta)/n} in the smallest of the involved dataset sizes.
  • Because the open-set objective is the closed-set objective in disguise, existing closed-set label-shift estimation and correction software can be reused for the open-set case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The pseudo-OOD construction suggests the method may not need a physically realistic OOD reference at all: any set whose average score under h matches the true OOD average would work, making the actual requirement a calibration condition rather than a distributional match.
  • If the OOD score's expected response is not exactly equal across ID classes, the correction in Theorem 4.4 inherits a bias proportional to those class-average differences; measuring those differences on a source validation set would predict when the method degrades.
  • Because the objective is convex in the reparameterized target vector, adding Dirichlet and Beta priors gives a route to uncertainty estimates for the target proportions and ID ratio without new optimization machinery.
  • The estimator chain could be applied sequentially to arriving target batches, repeatedly correcting the ID ratio as new unlabeled data appear, which would amount to an online open-set label shift estimator.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies Open Set Label Shift (OSLS), where the target label distribution gains an extra out-of-distribution (OOD) class. The authors propose a three-stage estimator: (1) estimate the source ID data ratio ρ_s using an OOD classifier h and an OOD reference dataset; (2) reformulate the target likelihood as a K+1-class closed-set label shift problem and apply an EM algorithm to estimate target ID class proportions π and the target ID ratio ρ_t; and (3) correct ρ_t for imperfect OOD classifiers under a relaxed assumption. They provide concentration inequalities for the source-ratio estimator and the corrected target-ratio estimator, prove EM convergence for the MLE, and construct an adapted classifier in Eq. (17). Experiments on CIFAR-10/100 and ImageNet-200 with five OOD detectors show consistent improvements over closed-set label-shift baselines across many shift configurations. The paper releases code and reports detailed ablations, including sensitivity to the pseudo-OOD mixing coefficient γ.

Significance. The likelihood factorization in Lemma 4.2 is elegant: it reduces OSLS to a K+1-class closed-set label shift problem, making existing CSLS machinery applicable. The EM derivation is sound under Assumptions 3.2 and 3.3, and Theorems 4.1 and 4.4 provide genuine finite-sample concentration bounds for the setting where the OOD reference dataset contains true OOD samples. The paper also demonstrates good empirical performance across a wide range of datasets, shift types, and OOD detectors, and it releases publicly available code. However, the actual experimental pipeline replaces the ground-truth OOD reference with pseudo-OOD samples and an ad hoc rescaling factor T, which is outside the scope of the proved theorems; this gap materially weakens the theoretical support for the reported experimental results.

major comments (3)
  1. [§4.5, Eqs. (15)–(16); §5 experiments] The pseudo-OOD reference dataset D_o^γ = {(1-γ)x_i + γ ε} and the rescaling factor T are not covered by The theoretical guarantees. Theorems 4.1 and 4.4 require the reference dataset to consist of i.i.d. samples from p(x|y=K+1), the true source/target OOD conditional distribution. For D_o^γ, E_{D_o^γ}[h] is not generally equal to µ0 = E_{X_s|B_s=0}[h(X)], and the statement in §4.5 that the T rescaling makes the expectations match is not supported by any theorem or lemma. Because ˆµ0 enters the source ratio estimator (2) and the target ratio correction (13)–(14), a biased ˆµ0 propagates through ρ_s, π, and ρ_t, so the concentration bounds in Theorems 4.1 and 4.4 do not apply to the algorithm actually evaluated in Section 5. The Remark in §4.1 acknowledges the relaxation, but no bias bound, consistency result, or even a formal condition on T is provided. This gap affects the central claim that sampling errors of the estimators are quantified. I recommend either adding a rigorous analysis of the pseudo-OOD bias (e.g., bounding |E_{D_o^γ}[h]/T - µ0| in terms of γ and properties of h) or repositioning the pseudo-OOD variant as a heuristic and validating the theoretical claims on experiments that use a true OOD reference dataset.
  2. [§4.3 and Appendix C.5, Proposition C.2] The claim that the MAP-OLS EM algorithm converges to the global MAP estimate πMAP is not established. Proposition C.2 proves that the posterior is strictly convex in π for fixed ρ_t and strictly convex in ρ_t for fixed π, but the objective is not shown to be jointly convex in (π, ρ_t). For a non-jointly-convex objective, standard EM theory only guarantees convergence to a stationary point or a local maximum, not the global maximum. Since the experimental results use the MAP variant (as stated in §5.2), the convergence claim in Theorem 4.3 and Proposition C.2 should be weakened to local convergence, or a joint convexity argument (or an additional condition) should be supplied.
  3. [§5.2, Table 4] The comparison omits Garg et al. 14, which is the only existing method specifically designed for Open Set Label Shift. The authors justify this by noting that Garg et al. requires retraining and does not report the estimation error metric (w - ŵ)^2/K. While this is a reasonable practical concern, the absence of any comparison against the existing OSLS method means that the claim of 'superior performance' is not demonstrated against the state of the art in OSLS, only against closed-set methods and a uniform baseline. At the very least, the paper should report results on a setting where a runnable implementation of Garg et al. can be compared, or clearly state that no OSLS baseline currently reports this metric and discuss the expected trade-off.
minor comments (6)
  1. [Definition 3.1 and §4.1] Definition 3.1 lists the given inputs as D_s, D_t, f, and h, but does not include the OOD reference dataset D_o, which Section 4.1 introduces as required. This inconsistency should be resolved, either by adding D_o to the problem definition or by explicitly stating that D_o is an auxiliary input that may be replaced by pseudo-OOD data.
  2. [Appendix A] There are minor typos: 'Tource domain distribution' should be 'Target domain distribution', and 'subsset' should be 'subset'.
  3. [Appendix B.3] The word 'pipline' should be 'pipeline'.
  4. [§4.5 and Appendix D.3] The hyperparameters γ and T appear in Algorithm 2 without any selection procedure or sensitivity analysis. Appendix E.2 ablates γ but not T; please add a sensitivity study for T or provide a principled way to choose it.
  5. [Appendix F, Tables 15–17] Some entries for the Ash OOD detector on CIFAR-10 show very large standard deviations (e.g., Table 15, Ash Near r=1.0: 0.280±0.086 and later values above 1.0), which suggests instability. This should be mentioned in the discussion of the results.
  6. [Appendix C.3, Eq. (36)] The chain of equalities deriving ps(y=K+1|x) = 1-h(x) is hard to follow and contains an apparent typo in the denominator (ps(b=0) appears twice). A cleaner derivation would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the estimators are algebraic consequences of the stated probabilistic assumptions, and the pseudo-OOD relaxation is an acknowledged gap in guarantees rather than a circular reduction.

full rationale

The derivation chain is self-contained under Assumptions 3.2 and 3.3. Theorem 4.1 derives rho_s = mu0/(1 - mu1 + mu0) from the law of total expectation ps(b=1)=E[h(X)]; mu0 and mu1 are independent expectations over OOD and ID source data, and the concentration bound is a standard Hoeffding argument. Lemma 4.2 and Theorem 4.3 construct the target likelihood by substituting the assumptions into the OSLS likelihood; convexity and EM convergence are reduced to the externally established MLLS convergence result of Alexandari et al. [1], not to the authors' own work. The self-citations to MAPLS [62] concern optional MAP priors and experimental initialization/recipe, which are not load-bearing for the MLE claim. The correction formula (14) and classifier correction (17) are the standard CSLS correction identities applied to the K+1 augmented classes, so they do not smuggle the target quantities in as inputs. Section 4.5 is the only caveat: the pseudo-OOD set D_gamma and the ad hoc rescale T are introduced with the explicit condition that the expectation of h over D_gamma equals the expectation over the true OOD distribution, but no theorem extends the guarantees of Theorems 4.1 and 4.4 to this relaxation. That is an unverified assumption and a robustness limitation, not circularity, because D_gamma and T are not defined in terms of the target label proportions pi or rho_t that the paper predicts. No fitted target-data parameter is renamed as a prediction, and no load-bearing argument reduces to a self-citation chain.

Assumptions & free parameters 4 free parameters · 6 assumptions · 1 invented entities

Central method rests on three calibration-style distributional assumptions, one ad hoc pseudo-OOD expectation assumption, and two hand-tuned hyperparameters; no invented physical entities beyond the pseudo-OOD dataset.

free parameters (4)
  • gamma (pseudo-OOD mixing coefficient) = 0.2 for CIFAR10 and ImageNet-200, 0.1 for CIFAR100
    Hand-chosen in Eq. (15) to generate pseudo-OOD samples; affects mu0 and all downstream estimates; no theoretical selection criterion.
  • T (pseudo-OOD reweight factor) = 2 for all datasets
    Ad hoc rescaling of mu0 in Eq. (16) to compensate for pseudo-OOD samples being closer to ID than true OOD; no sensitivity analysis or theoretical grounding.
  • ASH percentile = Searched in [65, 70, 75, 80, 85, 90, 95]
    OOD detector hyperparameter selected on a source validation subset; inherited from OpenOOD but affects the h used by the central estimator.
  • ReAct percentile = Searched in [85, 90, 95, 99]
    OOD detector hyperparameter selected on a source validation subset; affects the h used by the central estimator.
assumptions (6)
  • domain assumption Label shift invariance: ps(x|y=i)=pt(x|y=i) for all classes including the OOD class (Assumption 3.2).
    Core problem assumption; without it the target likelihood cannot be written in terms of the source classifier and OOD detector.
  • domain assumption ID classifier calibration: ps(y|x,b=1)=f(x) (Assumption 3.3A).
    Empirically checked only via ECE on source validation; assumed for all x, including target data.
  • domain assumption OOD classifier calibration: ps(b=1|x)=h(x) (Assumption 3.3B).
    Load-bearing for Eq. (5) and the EM stage; no direct empirical validation; later corrected only for rho_t, not for pi.
  • domain assumption Condition (11): E[h'(x)|Ys=i] is equal across ID classes and different for the OOD class (Theorem 4.4).
    Needed to replace target expectations with source estimates; justified heuristically rather than verified directly.
  • ad hoc to paper Pseudo-OOD reference expectation matches true OOD expectation after T rescale.
    Introduced in Section 4.5; no proof that E_{Do_gamma}[h] equals E_{true OOD}[h], and the concentration bounds do not cover the pseudo-OOD case.
  • standard math Standard concentration inequalities and MLE/MAP EM convergence from closed-set label shift literature.
    Theorems 4.1 and 4.4 rely on Hoeffding; Theorem 4.3 relies on MLE invariance and MLLS convergence.
invented entities (1)
  • Pseudo-OOD reference dataset D_o^gamma
    purpose: Provides a test-time reference for estimating mu0 without true OOD samples.
    Generated as Gaussian-noise mixtures of source ID samples; must behave like OOD under h, but no independent validation beyond a hand-tuned reweight factor T.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Open Set Label Shift with Test Time Out-of-Distribution Reference." pith.science (2026). https://pith.science/paper/ATSJ6COV

@misc{pith2026250505868,
  author       = {Pith},
  title        = {Pith review of: Open Set Label Shift with Test Time Out-of-Distribution Reference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ATSJ6COV}},
  note         = {Machine review of arXiv:2505.05868}
}
read the original abstract

Open set label shift (OSLS) occurs when label distributions change from a source to a target distribution, and the target distribution has an additional out-of-distribution (OOD) class. In this work, we build estimators for both source and target open set label distributions using a source domain in-distribution (ID) classifier and an ID/OOD classifier. With reasonable assumptions on the ID/OOD classifier, the estimators are assembled into a sequence of three stages: 1) an estimate of the source label distribution of the OOD class, 2) an EM algorithm for Maximum Likelihood estimates (MLE) of the target label distribution, and 3) an estimate of the target label distribution of OOD class under relaxed assumptions on the OOD classifier. The sampling errors of estimates in 1) and 3) are quantified with a concentration inequality. The estimation result allows us to correct the ID classifier trained on the source distribution to the target distribution without retraining. Experiments on a variety of open set label shift settings demonstrate the effectiveness of our model. Our code is available at https://github.com/ChangkunYe/OpenSetLabelShift.

Figures

Figures reproduced from arXiv: 2505.05868 by the authors.

Figure 1
Figure 1. Open Set Label Shift (OSLS) set up, where source and target domain have different label distributions p(y) but iden￾tical conditional distribution of data given label p(x|y). OSLS extends the Closed Set Label Shift (CSLS) with an extra Out-of￾Distribution (OOD) class on the target domain. data; 3) correction: adapt a classifier trained on the source domain to a target domain. The label shift problem has been widely … view at source ↗
Figure 2
Figure 2. Structure of our proposed Open Set Label Shift estimation and correction method. The target ID label distribution proba￾bilities pt(y = ·) = π and ID data ratio pt(b = 1) = ρt are estimated through three steps: 1) retrieve source ID data ratio ρs (Sec. 4.2), 2) estimate target ID data ratio ρt and target ID label distribution π via an EM algorithm under Assumption 3.2,3.3 (Sec. 4.3) and 3) correct the target ID data… view at source ↗
Figure 3
Figure 3. Graphical model of the Open Set Label Shift set￾ting and our assumptions. Xs, Xt are data for the source and target domain, Ys, Yt are the corresponding categorical-valued la￾bels and Bs, Bt are binary values representing ID/OOD data. c, π are source and target domain label distribution class probabilities. Source domain data Xs is observed with ground truth ID data in D s and reference OOD data in D o . p(x|y) are … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Estimation result comparison of ρˆ ∗ t by our model (Solid lines), ρˆt by our model but without ρt correction (§ 4.4) (Dashed lines) based on different OOD classifiers and the Ground truth ρt (Black, Solid line), on CIFAR10/100 dataset with Dirichlet shift and Near OOD…
Figure 5
Figure 5. Figure 5: Estimation result comparison of ρˆ ∗ t by our model (Solid lines), ρˆt by our model but without ρt correction (Sec. 4.4) (Dashed lines) based on different OOD classifiers and the Ground truth ρt (Black, Solid line), on CIFAR10/100 dataset with Dirichlet shift and Near …
Figure 6
Figure 6. Figure 6: Estimation result comparison of ρˆ ∗ t by our model (Solid lines), ρˆt by our model but without ρt correction (Sec. 4.4) (Dashed lines) based on different OOD classifiers and the Ground truth ρt (Black, Solid line), on CIFAR10/100 dataset with LT10 shift (“F” for Forwa…
Figure 7
Figure 7. Figure 7: Estimation result comparison of ρˆ ∗ t by our model (Solid lines), ρˆt by our model but without ρt correction (Sec. 4.4) (Dashed lines) based on different OOD classifiers and the Ground truth ρt (Black, Solid line), on the CIFAR10/100 dataset with LT50 shift (“F” for F…
Figure 8
Figure 8. Figure 8: Estimation result comparison of ρˆ ∗ t by our model (Solid lines), ρˆt by our model but without ρt correction (Sec. 4.4) (Dashed lines) based on different OOD classifiers and the Ground truth ρt (Black, Solid line), on the CIFAR10/100 dataset with LT100 shift (“F” for …
Figure 9
Figure 9. Figure 9: Estimation result comparison of ρˆ ∗ t by our model (Solid lines), ρˆt by our model but without ρt correction (Sec. 4.4) (Dashed lines) based on different OOD classifiers and the Ground truth ρt (Black, Solid line), on the ImageNet-200 dataset with LT10/LT100 shift (“F…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 60 canonical work pages

  1. [1]

    Maximum likelihood with bias-corrected calibration is hard- to-beat at label shift adaptation

    Amr Alexandari, Anshul Kundaje, and Avanti Shrikumar. Maximum likelihood with bias-corrected calibration is hard- to-beat at label shift adaptation. In International Conference on Machine Learning, pages 222–232. PMLR, 2020. 1, 2, 5, 6, 14, 20, 29

    work page 2020

  2. [2]

    Regularized learning for domain adaptation under label shifts

    Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, and An- imashree Anandkumar. Regularized learning for domain adaptation under label shifts. In International Conference on Learning Representations, 2018. 1, 2, 7, 29

    work page 2018

  3. [3]

    On- line label shift: Optimal dynamic regret meets practical algo- rithms

    Dheeraj Baby, Saurabh Garg, Tzu-Ching Yen, Sivaraman Balakrishnan, Zachary Lipton, and Yu-Xiang Wang. On- line label shift: Optimal dynamic regret meets practical algo- rithms. Advances in Neural Information Processing Systems, 36, 2024. 2

    work page 2024

  4. [4]

    Towards open set deep networks

    Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1563–1572,

  5. [5]

    In or out? fixing imagenet out-of-distribution detection eval- uation

    Julian Bitterwolf, Maximilian Mueller, and Matthias Hein. In or out? fixing imagenet out-of-distribution detection eval- uation. In ICML, 2023. 6

    work page 2023

  6. [6]

    Adversarial reciprocal points learning for open set recognition

    Guangyao Chen, Peixi Peng, Xiangqian Wang, and Yonghong Tian. Adversarial reciprocal points learning for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8065–8081, 2021. 2, 15

    work page 2021

  7. [7]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE conference on com- puter vision and pattern recognition , pages 248–255, 2009. 6

    work page 2009

  8. [8]

    Extremely simple activation shaping for out- of-distribution detection

    Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out- of-distribution detection. arXiv preprint arXiv:2209.09858,

    arXiv

Show all 73 references
  1. [9]

    Learning classifiers from only positive and unlabeled data

    Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 213–220, 2008. 2

    2008

  2. [10]

    Open set domain adaptation: Theoretical bound and algorithm

    Zhen Fang, Jie Lu, Feng Liu, Junyu Xuan, and Guangquan Zhang. Open set domain adaptation: Theoretical bound and algorithm. IEEE transactions on neural networks and learn- ing systems, 32(10):4309–4322, 2020. 2, 15

    2020

  3. [11]

    Is out-of-distribution detection learnable? Advances in Neural Information Processing Systems , 35: 37199–37213, 2022

    Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? Advances in Neural Information Processing Systems , 35: 37199–37213, 2022. 2, 15

    2022

  4. [12]

    A unified view of label shift estimation.Ad- vances in Neural Information Processing Systems, 33:3290– 3300, 2020

    Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, and Zachary Lipton. A unified view of label shift estimation.Ad- vances in Neural Information Processing Systems, 33:3290– 3300, 2020. 2, 7, 14

    2020

  5. [13]

    Leveraging un- labeled data to predict out-of-distribution performance

    Saurabh Garg, Sivaraman Balakrishnan, Zachary Chase Lip- ton, Behnam Neyshabur, and Hanie Sedghi. Leveraging un- labeled data to predict out-of-distribution performance. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021. 2

    2021

  6. [14]

    Domain adaptation under open set label shift

    Saurabh Garg, Sivaraman Balakrishnan, and Zachary Lip- ton. Domain adaptation under open set label shift. Advances in Neural Information Processing Systems, 35:22531–22546,

  7. [15]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014. 2

    2014

  8. [16]

    Explaining and harnessing adversarial examples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 15

    2014 arXiv

  9. [17]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR,

  10. [18]

    Ltf: A label transformation framework for correcting label shift

    Jiaxian Guo, Mingming Gong, Tongliang Liu, Kun Zhang, and Dacheng Tao. Ltf: A label transformation framework for correcting label shift. In International Conference on Machine Learning, pages 3843–3853. PMLR, 2020. 2

    2020

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6

    2016

  12. [20]

    Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the prob- lem

    Matthias Hein, Maksym Andriushchenko, and Julian Bitter- wolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the prob- lem. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 4...

    2019

  13. [21]

    A baseline for detect- ing misclassified and out-of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detect- ing misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Repre- sentations, 2022. 15

    2022

  14. [23]

    Scaling out-of-distribution detection for real- world settings

    Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joe Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real- world settings. arXiv preprint arXiv:1911.11132, 2019. 6

    1911 arXiv

  15. [24]

    Using self-supervised learning can improve model robustness and uncertainty

    Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. Advances in neural in- formation processing systems, 32, 2019. 3, 15

    2019

  16. [25]

    Generalized odin: Detecting out-of-distribution image with- out learning from out-of-distribution data

    Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image with- out learning from out-of-distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10951–10960, 2020. 3, 4, 15

    2020

  17. [26]

    Mos: Towards scaling out-of- distribution detection for large semantic space

    Rui Huang and Yixuan Li. Mos: Towards scaling out-of- distribution detection for large semantic space. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8710–8719, 2021. 6

    2021

  18. [27]

    Opengan: Open-set recog- nition via open data generation

    Shu Kong and Deva Ramanan. Opengan: Open-set recog- nition via open data generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 813–822, 2021. 3, 15

    2021

  19. [28]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 6

    2009

  20. [29]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. Advances in neural information processing systems , 25, 2012. 6

    2012

  21. [30]

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural infor- mation processing systems, 31, 2018. 3, 15

    2018

  22. [31]

    Enhanc- ing the reliability of out-of-distribution image detection in neural networks

    Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhanc- ing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690, 2017. 3, 4, 7, 15

    2017 arXiv

  23. [32]

    De- tecting and correcting for label shift with black box predic- tors

    Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. De- tecting and correcting for label shift with black box predic- tors. In International conference on machine learning, pages 3122–3130. PMLR, 2018. 1, 2, 6, 7, 29

    2018

  24. [33]

    Separate to adapt: Open set domain adaptation via progressive separation

    Hong Liu, Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Qiang Yang. Separate to adapt: Open set domain adaptation via progressive separation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2927–2936, 2019. 3, 15

    2019

  25. [34]

    Model calibration in dense classification with adaptive label perturbation

    Jiawei Liu, Changkun Ye, Shan Wang, Ruikai Cui, Jing Zhang, Kaihao Zhang, and Nick Barnes. Model calibration in dense classification with adaptive label perturbation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1173–1184, 2023. 7

    2023

  26. [35]

    Self- calibrating vicinal risk minimisation for model calibration

    Jiawei Liu, Changkun Ye, Ruikai Cui, and Nick Barnes. Self- calibrating vicinal risk minimisation for model calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3335–3345, 2024. 7

    2024

  27. [36]

    Energy-based out-of-distribution detection

    Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464–21475,

  28. [37]

    Mini- max optimal approaches to the label shift problem in non- parametric settings

    Subha Maity, Yuekai Sun, and Moulinath Banerjee. Mini- max optimal approaches to the label shift problem in non- parametric settings. Journal of Machine Learning Research, 23(346):1–45, 2022. 2

    2022

  29. [38]

    Towards neural net- works that provably know when they don’t know

    Alexander Meinke and Matthias Hein. Towards neural net- works that provably know when they don’t know. In Inter- national Conference on Learning Representations, 2019. 3, 15

    2019

  30. [39]

    Provably adversarially robust detection of out-of-distribution data (almost) for free

    Alexander Meinke, Julian Bitterwolf, and Matthias Hein. Provably adversarially robust detection of out-of-distribution data (almost) for free. Advances in Neural Information Pro- cessing Systems, 35:30167–30180, 2022. 7

    2022

  31. [40]

    Accuracy on the line: on the strong correlation between out-of-distribution and in- distribution generalization

    John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of-distribution and in- distribution generalization. In International Confere...

    2021

  32. [41]

    Machine learning: a probabilistic perspec- tive

    Kevin P Murphy. Machine learning: a probabilistic perspec- tive. MIT press, 2012. 5, 20

    2012

  33. [42]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learn- ing, page 4. Granada, 2011. 6

    2011

  34. [43]

    Open set domain adaptation

    Pau Panareda Busto and Juergen Gall. Open set domain adaptation. In Proceedings of the IEEE international con- ference on computer vision, pages 754–763, 2017. 3, 15

    2017

  35. [44]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017. 6

    2017

  36. [45]

    Handling new class in online label shift

    Yu-Yang Qian, Yong Bai, Zhen-Yu Zhang, Peng Zhao, and Zhi-Hua Zhou. Handling new class in online label shift. In 2023 IEEE International Conference on Data Mining (ICDM), pages 1283–1288. IEEE, 2023. 2

    2023

  37. [46]

    Discriminant analysis and statistical pat- tern recognition, 1993

    William S Rayens. Discriminant analysis and statistical pat- tern recognition, 1993. 2

    1993

  38. [47]

    Adjusting the outputs of a classifier to new a priori probabil- ities: a simple procedure

    Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the outputs of a classifier to new a priori probabil- ities: a simple procedure. Neural computation, 14(1):21–41,

  39. [48]

    Open set domain adaptation by backpropa- gation

    Kuniaki Saito, Shohei Yamamoto, Yoshitaka Ushiku, and Tatsuya Harada. Open set domain adaptation by backpropa- gation. In Proceedings of the European conference on com- puter vision (ECCV), pages 153–168, 2018. 3, 15

    2018

  40. [49]

    Detecting out-of-distribution examples with gram matrices

    Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution examples with gram matrices. In Interna- tional Conference on Machine Learning , pages 8491–8501. PMLR, 2020. 15

    2020

  41. [50]

    React: Out-of- distribution detection with rectified activations

    Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of- distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34:144–157, 2021. 7, 15, 27

    2021

  42. [51]

    Out- of-distribution detection with deep nearest neighbors

    Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out- of-distribution detection with deep nearest neighbors. In In- ternational Conference on Machine Learning, pages 20827– 20840. PMLR, 2022. 7, 15, 27

    2022

  43. [52]

    Csi: Novelty detection via contrastive learning on dis- tributionally shifted instances

    Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on dis- tributionally shifted instances. Advances in neural informa- tion processing systems, 33:11839–11852, 2020. 3, 15

    2020

  44. [53]

    Elsa: Effi- cient label shift adaptation through the lens of semiparamet- ric models

    Qinglong Tian, Xin Zhang, and Jiwei Zhao. Elsa: Effi- cient label shift adaptation through the lens of semiparamet- ric models. In International Conference on Machine Learn- ing, pages 34120–34142. PMLR, 2023. 2

    2023

  45. [54]

    Open-set recognition: A good closed-set classifier is all you need

    Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisser- man. Open-set recognition: A good closed-set classifier is all you need. In International Conference on Learning Rep- resentations, 2021. 3, 15

    2021

  46. [55]

    High-dimensional probability: An intro- duction with applications in data science

    Roman Vershynin. High-dimensional probability: An intro- duction with applications in data science . Cambridge uni- versity press, 2018. 4, 16

    2018

  47. [56]

    Vim: Out-of-distribution with virtual-logit matching

    Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual-logit matching. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4921–4930, 2022. 6

    2022

  48. [57]

    Vim: Out-of-distribution with virtual-logit matching

    Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual-logit matching. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4921–4930, 2022. 3, 15

    2022

  49. [58]

    Learning to augment distributions for out-of-distribution detection

    Qizhou Wang, Zhen Fang, Yonggang Zhang, Feng Liu, Yix- uan Li, and Bo Han. Learning to augment distributions for out-of-distribution detection. Advances in neural informa- tion processing systems, 36:73274–73286, 2023. 3, 15

    2023

  50. [59]

    Online adaptation to label distribution shift

    Ruihan Wu, Chuan Guo, Yi Su, and Kilian Q Weinberger. Online adaptation to label distribution shift. Advances in Neural Information Processing Systems , 34:11340–11351,

  51. [60]

    Towards cal- ibrated model for long-tailed visual recognition from prior perspective

    Zhengzhuo Xu, Zenghao Chai, and Chun Yuan. Towards cal- ibrated model for long-tailed visual recognition from prior perspective. Advances in Neural Information Processing Systems, 34, 2021. 7

    2021

  52. [61]

    Openood: Benchmarking generalized out-of-distribution detection

    Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neu- ral Information Processing Systems, 35:32598–32611, 2022. 27

    2022

  53. [62]

    Label shift estimation for class-imbalance problem: A bayesian approach

    Changkun Ye, Russell Tsuchida, Lars Petersson, and Nick Barnes. Label shift estimation for class-imbalance problem: A bayesian approach. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), pages 1073–1082, 2024. 1, 2, 3, 6, 7, 8, 14, 2...

    2024

  54. [63]

    Coping with label shift via distributionally robust optimisation

    Jingzhao Zhang, Aditya Krishna Menon, Andreas Veit, Sri- nadh Bhojanapalli, Sanjiv Kumar, and Suvrit Sra. Coping with label shift via distributionally robust optimisation. InIn- ternational Conference on Learning Representations (ICLR),

  55. [64]

    Openood v1.5: Enhanced benchmark for out-of-distribution detection

    Jingyang Zhang, Jingkang Yang, Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xue- feng Du, Kaiyang Zhou, Wayne Zhang, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. Openood v1.5: Enhanced benchmark for out-of-distribution detection. arXiv preprint arXiv:2306.0...

    2023 arXiv

  56. [65]

    Learning from a complementary-label source domain: theory and algorithms

    Yiyang Zhang, Feng Liu, Zhen Fang, Bo Yuan, Guangquan Zhang, and Jie Lu. Learning from a complementary-label source domain: theory and algorithms. IEEE Transactions on Neural Networks and Learning Systems , 33(12):7667– 7681, 2021. 3, 15

    2021

  57. [66]

    Active learning under label shift

    Eric Zhao, Anqi Liu, Animashree Anandkumar, and Yisong Yue. Active learning under label shift. In International Con- ference on Artificial Intelligence and Statistics, pages 3412–

  58. [67]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 2017. 6 Contents A . List of Symbols 13 B . Related Works 14 B.1. MLLS . ....

    2017

  59. [69]

    Proof of Lemma 4.2 (See page 5) Lemma 4.2

    (31) C.3. Proof of Lemma 4.2 (See page 5) Lemma 4.2. Under Assumption 3.2, 3.3, givenDt, the negative log likelihood− logL(π,ρt;Dt) can be written as: − logL(π,ρt;Dt) =− Nt X i=1 log   K+1X j=1 ˜πj ˜cj ˜f(xi)j   +C, (5) whereC does not depend on either π orρt and ˜f(x)i :=...

  60. [70]

    Identify the latent variable, derive the complete posterior

  61. [71]

    Construct the Q(π,ρt|π(m),ρ (m) t ) and obtain E-Step

  62. [72]

    Step 1: As discussed in the main paper (Eq

    Optimize Q(π,ρt|π(m),ρ (m) t ) w.r.t π,ρt and obtain M-Step. Step 1: As discussed in the main paper (Eq. (5)), we can construct the latent variable ˜Ys ∼ Cat(K + 1, ˜c) and ˜Yt ∼ Cat(K + 1, ˜π). With ˜Yt as latent variable, let ˜Y = {˜yt i}N i=1 with ˜yt i ∈ Y∪{K + 1}, the com...

  63. [73]

    (65) Option 1 (11): Under Assumption 3.2, the condition (11) holds implies: EXs|Ys=i[h′(x)] = EXt|Yt=j[h′(x)] for all i,j ∈Y , (66) then according Eq. (1) we have: µt 1 = EXt|Bt=1[h′(x)] = KX i=1 EXt|Yt=i[pt(y =i|b = 1)·h′(x)] = KX i=1 EXt|Yt=i[πi·h′(x)] = KX i=1 πi· EXt|Yt=1[...

  64. [74]

    Forward/Backward

    (69) whereµ′ 0 := EXs|Bs=0[h′(x)] are defined in the same way asµ1,µ 0 defined in Theorem 4.1 but substituteh ash′. The expectations can be approximated by ˆµ′ 1, ˆµ′ 1 with source domain ID dataset Ds and OOD dataset Do (Eq. (3)). Moreover, EXt[h(x)] =ρ can be estimated with ...

    2024

  65. [2002]

    1, 2, 5, 7, 14, 20, 29

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.