Boosting Test Performance with Importance Sampling--a Subpopulation Perspective

Hongyu Shen; Zhizhen Zhao

REVIEW 4 major objections 6 minor 15 references

Boosting Test Performance with Importance Sampling--a Subpopulation Perspective

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that one importance-sampling weight, derived from the test distribution, directly maximizes test performance under subpopulation shift, and that a closed-form version depending only on attribute-label agreement makes a…

desk verdict The DBA framework is a genuinely useful way to frame subpopulation shift, but the printed Theorem 1 weight is algebraically wrong, so the paper's central theoretical claim is not established as written. read the letter →

arxiv 2412.13003 v1 pith:TVAMZXQF submitted 2024-12-17 cs.LG stat.ML

Hongyu Shen , Zhizhen Zhao This is my paper · ORCID

classification cs.LGstat.ML

keywords importancesamplingsubpopulationshiftspuriouscorrelationworst-groupaccuracydatasetbiasanalysisreweightinggrouprobustnessdistribution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that subpopulation shift — where training and test sets differ in the mix of groups defined by a hidden spurious attribute — can be repaired by a single importance-sampling weight, provided the weight is built from the test distribution rather than from class balance. It formalizes the test objective as a weighted version of the training objective and proves a closed-form weight that depends on only one unknown quantity: the probability that the spurious attribute agrees with the label for a given input ($p(s=y|y,x,I_{tr})$). If correct, this yields a universal training algorithm that improves both average and worst-group test accuracy, and it explains why existing methods trade one for the other: they implicitly optimize for a test mix they assume, not the one they face. The paper also places several existing methods, including logit adjustment, mixup-style augmentation, and class reweighting, under the same framework as special cases.

What carries the argument

The load-bearing identity is importance sampling over the joint $(x,y)$ distribution: $\mathbb{E}_{p(x,y|I_{te})}[\log q(y|x)] = \mathbb{E}_{p(x,y|I_{tr})}[g(x,y,I_{tr},I_{te}) \log q(y|x)]$ with $g(x,y,I_{tr},I_{te}) = p(x,y|I_{te})/p(x,y|I_{tr})$. The paper's contribution is to decompose this general weight under specific structural assumptions — a shared data generator, a test set whose spurious attribute is uniform given the label, and a binary majority/minority split of the training set — into the closed form of Eq. (5), which reduces the whole problem to estimating one scalar function $p(s=y|y,x,I_{tr})$.

What would settle it

Run a synthetic experiment with known generative distributions: compute the exact optimal weight $p(x,y|I_{te})/p(x,y|I_{tr})$ numerically and compare it against Eq. (5) evaluated with the same known probabilities; any mismatch invalidates the theorem's algebra. Alternatively, on CivilComments — where test groups are not uniform — re-estimate the weight using the actual group proportions and see whether the accuracy improves beyond what the uniform-assumption formula delivers.

Watch

Extended reading notes

Core claim

The paper's central claim is Theorem 1: under five stated assumptions, maximizing the weighted training objective with the weight function $g(x,y,I_{tr},I_{te})$ defined in Eq. (5) directly maximizes the test likelihood. The weight is the reciprocal of the test-to-train joint density ratio, and the theorem reduces it to a closed form involving only $p(s=y|y,x,I_{tr})$, the probability that a sample's spurious attribute matches its label. The paper then offers three estimators for this single unknown term — direct estimation when the attribute is known, a two-model disagreement measure when a similar validation set exists, and the model's own confidence when it does not — and reports that the resulting algorithm beats empirical risk minimization and benchmark methods on three datasets on both average and worst-group accuracy at once.

Load-bearing premise

The formula depends on the test set having a uniformly distributed spurious attribute within each label (Assumption 3, $p(s|y,I_{te})=1/L$), together with a training set whose non-representative majority perfectly pairs attribute with label; if either condition fails, the closed-form weight is not the test-to-train density ratio.

Editorial extensions

If this is right

Training with the closed-form weight is consistent for test performance under the stated assumptions, so no separate worst-group regularization term is needed.
Existing methods that lose average accuracy do so because their implicit assumed test mix (for example, uniform groups) differs from the real test set; correcting the weight restores average accuracy.
Without group labels, the two-model disagreement estimator gives a working recipe whenever a validation set similar to the training set is available.
Because the weight is always the joint density ratio, the same framework extends to covariate and joint shifts beyond the subpopulation setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct practical check follows: on a new dataset, estimate the test group proportions; if they deviate strongly from uniform, Assumption 3 is violated and the closed form should be replaced by a direct estimate of $p(s|y,I_{te})$.
The binary $m_0/m_1$ split could be generalized to a continuous mixture; the same derivation would likely produce a weight depending on the full distribution of attribute-label agreement.
The two-model disagreement estimator (Eq. 6) could be calibrated against the attribute-known estimator on a fully labelled dataset, turning the method into a testable self-consistency check.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The high-level idea here is good: treat subpopulation shift as an importance-sampling problem, state the assumptions explicitly, and use that language to connect existing methods. That framing is worth something. The paper also makes an honest attempt to explain why existing methods trade average accuracy for worst-group accuracy, and the empirical results on average accuracy are strong on ColorMNIST and Waterbirds. I want to give credit for that.

The problem is Theorem 1. The stress-test note is right: a direct derivation under Assumptions 1–5 gives a different expression from Eq. (5). In a simple binary case the printed formula yields g^{-1}=4.5 while the correct importance ratio is 1.5. That is not a cosmetic difference; it changes which samples get up-weighted. The proof in Appendix B.3 contains the same algebraic misstep, so it is not a typo. Further, the prose in Sec. 3.2 says g decreases as p(s=y|y,x) increases, but Eq. (5) does the opposite. The central claim that Eq. (4)+(5) maximizes test performance is unsupported as written.

A separate, smaller weakness is that the unknown term p(s=y|y,x) is estimated with heuristics (Eqs. 6–7) that are not derived from Theorem 1, and the hyperparameters (p(m0)=0.85, tau=1) are tuned across benchmarks. The assumptions are also strong: uniform test attributes and a binary majority/minority structure. The paper does not quantify the error when these fail, and on CivilComments the test group proportions are not uniform.

The empirical claims are also overstated. The abstract promises state-of-the-art on both average and worst-group accuracy, but on CivilComments the known-s variant gets 43.33 worst-group, below ReSample (62.17), ReWeight (58.27), and GroupDRO (56.51). Average accuracy is genuinely good, and the method beats ERM consistently, which is interesting. But the headline claim needs to be scaled back.

Who is this for? People working on subpopulation shift who want a unified perspective and a careful statement of assumptions. The reinterpretations in Sec. 4 are thought-provoking even if the theorem is broken. But as printed, the paper cannot be used as a reliable method.

Recommendation: this deserves serious peer review, not a desk reject, because the framework is valuable and the error may be correctable. But the referee should ask for a corrected derivation of Theorem 1 and a re-run of the experiments with the correct weight. As it stands, it should not be accepted.

Referee Report

4 major / 6 minor

Summary. This paper proposes a 'DBA' framework based on importance sampling for subpopulation shift. It claims that under a set of explicit assumptions (Assumptions 1–5), a single reweighting function g(x,y,Itr,Ite) in Eq. (5) makes training on Dtr equivalent to optimizing test likelihood on Dte. The paper then proposes three estimators for the unknown term p(s=y|y,x,Itr), interprets existing subpopulation methods within this framework, and reports state-of-the-art average and worst-group accuracies on ColorMNIST, Waterbirds, and CivilComments. The central theoretical result is Theorem 1; all practical methods and experiments depend on Eq. (5).

Significance. The contribution is potentially valuable: it provides an explicit assumptions-based importance-sampling perspective, a unified view of several existing subpopulation methods, and a single reweighting algorithm with published code. However, the main theorem's weight formula as printed is algebraically incorrect, so the central theoretical claim is not established in the submitted version. Because the experiments use the incorrect formula, they cannot currently serve as evidence for the theorem. The conceptual framework is coherent enough that the error appears repairable, but the correction requires re-deriving the weight, restating the assumptions, and regenerating the experimental results.

major comments (4)

[Theorem 1, Eq. (5), Appendix B.3] The printed weight in Eq. (5) does not match the derivation in Appendix B.3 or a direct calculation. Let A=p(x|y,s=y,Itr), B=\sum_{s\neq y}p(x|y,s,Itr), S=A+B, and r=p(s=y|y,x,Itr). Under Assumptions 2–5 (with the additional condition p(s|y,m0,Itr)=1/L noted in Major 2), the correct expression is g^{-1}=p(m0|Itr)+[p(m1|Itr)L p(y|m1,Itr)/p(y|Itr)]/[1+K(1-r)/r], where K=[p(m0|Itr)p(y|Itr)/L+p(m1|Itr)p(y|m1,Itr)]/[p(m0|Itr)p(y|Itr)/L]. Eq. (5) instead has p(m1|Itr)L/[p(y|Itr)p(y|m1,Itr)] in the leading numerator and K with p(y|m1,Itr) instead of p(m1|Itr)p(y|m1,Itr). These are not equivalent even under Assumption 4, which gives p(y|m1,Itr)=p(y|Itr). For example, with L=2, p(m0|Itr)=p(m1|Itr)=0.5, p(y|Itr)=0.5, and r=1, Eq. (5) gives g^{-1}=4.5, while the correct importance ratio is g^{-1}=1.5. Thus Theorem 1 as stated is false, and Algorithm 1, which uses Eq. (5), does not optimize the test log-likelihood stated in Claim 2. The theorem, the proof, and the experiments based on Eq. (5) need to be corrected and rerun.
[Assumptions 4 and 5, Eq. (20)] The proof of Theorem 1 uses p(s|y,m0,Itr)=1/L in Eq. (20), writing p(y,s|Itr)=p(m0|Itr)p(y|Itr)/L + p(m1|Itr)p(y|m1,Itr)1{y=s}. This equality is not a consequence of Assumptions 4 and 5 as formally stated. Assumption 4 only fixes p(y|m0,Itr)=p(y|Ite), and Assumption 5 only fixes the m1 component. The uniform conditional p(s|y,m0,Itr)=1/L must be stated as an explicit assumption or derived from the informal definition of m0 as sharing statistics with Dte together with Assumption 3. As written, the hypotheses of Theorem 1 do not imply the formula, so the theorem is not self-contained.
[Section 3.2, Eqs. (6)–(7), Section 5.1] The practical estimators of p(s=y|y,x,Itr) in Eqs. (6) and (7) are heuristic and are not shown to be consistent for the quantity that Theorem 1 requires. They are not derived from the optimality claim, and the hyperparameters \tau and p(m0|Itr) are set empirically (e.g., p(m0|Itr)=0.85 'across datasets'). Without a consistency or error analysis, the empirical success of DBCM does not establish the 'single estimator is enough' claim. Moreover, Assumption 3 (uniform p(s|y,Ite)) is known to fail on CivilComments, where test group proportions are not uniform, and the paper does not quantify the error introduced by this violation.
[Section 5, Tables 1 and 2] The experimental section is not a valid test of Theorem 1 as stated because Algorithm 1 uses the incorrect Eq. (5). In addition, treating p(m0|Itr) as a hyperparameter chosen as 0.85 on the basis of performance across benchmarks makes the comparison to methods whose hyperparameters are selected on validation accuracy not apples-to-apples. The claims of state-of-the-art performance and the explanation of degraded average accuracy should be revisited after correcting the weight formula.

minor comments (6)

[Abstract] The abstract contains a typo: 'important sampling' should be 'importance sampling'.
[Section 3.1, Assumption 4] The text after Assumption 4 contains a typo: 'subpupolation' should be 'subpopulation'.
[Appendix C, Assumption 3 discussion] In the discussion of Assumption 3, 'the label distribution x does not need to be uniformly distributed' should refer to the label y, not the data x.
[Equations (6)–(7)] Writing \hat p \propto \exp(...)^{-1} is unnecessarily ambiguous; it would be clearer to write \exp(-|...|/\tau) with an explicit normalizing constant.
[Section 5.2] The claim that 'DBCM is the only model that consistently outperforms ERM' should be qualified as referring to average accuracy; on CivilComments the worst-group accuracy of DBCM is below ERM in both Table 1 (43.33 vs 48.17) and Table 2 (43.41 vs 48.17).
[Theorem 2 statement] The theorem statement has a typo: 'Assumption 1, 2, 3 6, and 7' should be 'Assumptions 1, 2, 3, 6, and 7'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the theorem's weight is derived from explicit assumptions, and the empirical heuristics are approximations rather than fitted predictions.

full rationale

The paper's central derivation is a self-contained importance-sampling argument. Claim 2 defines g(x,y,Itr,Ite) as the likelihood ratio p(x,y|Ite)/p(x,y|Itr), so the equality in Eq. (4) is an identity; this is a starting point, not a disguised fit. Theorem 1 then derives a closed form for g under Assumptions 1–5, with the explicit caveat that p(s=y|y,x,Itr) is the only unknown term. The empirical estimators in Eqs. (6) and (7) are heuristic proxies for that term and hyperparameters such as p(m0|Itr) and tau are tuned, but they are not fitted to the test objective and do not make the theorem's conclusion true by construction. There are no load-bearing self-citations, no imported uniqueness theorem, and no known result merely renamed as unification. The algebraic inconsistency between printed Eq. (5) and the proof in Appendix B.3 flagged by the skeptic is a correctness concern, not a circularity concern: it means the theorem may be unproven as written, but it does not show that the claimed prediction is equivalent to its input. Accordingly, no specific circular reduction can be quoted, and the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The central derivation relies on five explicit assumptions plus the invented latent split m. The free parameters p(m0) and tau are tuned or hand-set, and the structural assumptions (especially uniform s in test and perfect spurious correlation in the majority group) are strong and unverified for the benchmarks used.

free parameters (2)

p(m0|Itr) / p(m1|Itr) = 0.85 (unknown composition), 0.005 or 0.02 for ColorMNIST
The assumed fraction of the minority group m0. Treated as known for ColorMNIST and as a hyperparameter otherwise; the paper states 0.85 'performed well across datasets'.
tau (temperature) = 1
Temperature in the estimators Eq (6) and Eq (7); fixed to 1 in all experiments without a search.

assumptions (6)

domain assumption Assumption 1: Supptr(x,y) superset Suppva(x,y) superset Suppte(x,y)
Support inclusion for train/val/test needed for well-defined importance weights; invoked in Claims 1 and 2 and Theorem 1.
domain assumption Assumption 2: p(x|y,s,Itr)=p(x|y,s,Ite)
Identical conditional generator of x given y and s across datasets; typical covariate shift assumption.
domain assumption Assumption 3: p(s|y,Ite)=1/L
Uniform spurious attribute s given label y in the test set; needed for the closed form in Eq (5).
domain assumption Assumption 4: p(y|Ite)=p(y|m0,Itr)=p(y|Itr)
Label distribution unchanged between train and test, and minority group m0 has the same label distribution as the whole train set.
ad hoc to paper Assumption 5: p(s|y,m1,Itr)=1{y=s}
Majority group m1 has attribute perfectly equal to label; a strong characterization of spurious correlation that may not hold exactly in real data.
ad hoc to paper Existence of binary m with m0 statistics matching test
The paper introduces a latent binary split m0/m1 with the property that m0 statistics match test; this is not observed and is assumed to exist.

invented entities (1)

Binary group indicator m (m0/m1)
purpose: Latent variable that splits training data into a minority group matching test statistics and a majority group with spurious correlation, enabling the closed-form weight in Theorem 1.
No direct measurement or falsifiable prediction outside the paper; it is a modeling construct to justify the weight function.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Boosting Test Performance with Importance Sampling--a Subpopulation Perspective." pith.science (2026). https://pith.science/paper/TVAMZXQF

@misc{pith2026241213003,
  author       = {Pith},
  title        = {Pith review of: Boosting Test Performance with Importance Sampling--a Subpopulation Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TVAMZXQF}},
  note         = {Machine review of arXiv:2412.13003}
}

read the original abstract

Despite empirical risk minimization (ERM) is widely applied in the machine learning community, its performance is limited on data with spurious correlation or subpopulation that is introduced by hidden attributes. Existing literature proposed techniques to maximize group-balanced or worst-group accuracy when such correlation presents, yet, at the cost of lower average accuracy. In addition, many existing works conduct surveys on different subpopulation methods without revealing the inherent connection between these methods, which could hinder the technology advancement in this area. In this paper, we identify important sampling as a simple yet powerful tool for solving the subpopulation problem. On the theory side, we provide a new systematic formulation of the subpopulation problem and explicitly identify the assumptions that are not clearly stated in the existing works. This helps to uncover the cause of the dropped average accuracy. We provide the first theoretical discussion on the connections of existing methods, revealing the core components that make them different. On the application side, we demonstrate a single estimator is enough to solve the subpopulation problem. In particular, we introduce the estimator in both attribute-known and -unknown scenarios in the subpopulation setup, offering flexibility in practical use cases. And empirically, we achieve state-of-the-art performance on commonly used benchmark datasets.

Figures

Figures reproduced from arXiv: 2412.13003 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 13 canonical work pages

[1]

Assumption 5: This assumption characterizes the nature of the subpopulation shift itself by assuming that the shift is caused by a spurious variable influencing Y

for training and testing datasets, this assumption is not restrictive, as we can subsample the training set to approximate the distribution of labels in the test set. Assumption 5: This assumption characterizes the nature of the subpopulation shift itself by assuming that the shift is caused by a spurious variable influencing Y . Such a scenario is freque...

work page 2023
[2]

In Companion proceedings of the 2019 world wide web conference, 491–500

Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, 491–500. Byrd, J.; and Lipton, Z

work page 2019
[4]

In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9,

Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9,

work page 2019
[5]

Assumption 1: This assumption requires inclusion relationships on the supports of the training, testing, and validation sets, ensuring a well-defined weight function for importance sampling—a fundamental assumption required for this approach. Assumption 2: this assumption states that the data generation process for x is identical in both the training and ...

work page 2007
[6]

arXiv preprint arXiv:2403.01874

A Survey on Evaluation of Out-of-Distribution Generalization. arXiv preprint arXiv:2403.01874. Zhang, H.; Cissé, M.; Dauphin, Y . N.; and Lopez-Paz, D

arXiv
[8]

(2023) is the first and possibly the only survey paper that provides a comprehensive experimental study on existing subpopulation methods

To the best of our knowledge, Yang et al. (2023) is the first and possibly the only survey paper that provides a comprehensive experimental study on existing subpopulation methods. The paper decomposes the distribution of y|x to take into account the effect of attributes (i.e. spurious features) via Bayes’ theorem, where y is the random variable for the l...

work page 2023
[9]

for details). In particular, the paper categorizes 12 different datasets into 4 classes that have different correlations between the labels and the attributes, and benchmarks 20 popular subpopulation methods with these datasets, to characterize the performance of the methods given different attribute setups. Nonetheless, the statistical quantification of ...

work page 2024
[10]

We investigate all the papers it cites and all papers that cite this one to guarantee the coverage and scope as much as possible

to identify other existing survey work as this paper is the latest work that conducts comprehensive experimental analysis on existing subpopulation methods. We investigate all the papers it cites and all papers that cite this one to guarantee the coverage and scope as much as possible. data samples, to reduce the effect of the spurious backgrounds. They d...

work page 2024

Show all 15 references

[14]

All models are trained with a single 16 GB NVIDIA V100 GPU for 3 independent runs3

(learning rate = 0.0001) for the language dataset. All models are trained with a single 16 GB NVIDIA V100 GPU for 3 independent runs3. We follow the convention and report the mean and the standard deviation on both the average accuracy and the worst group accuracy. F Discussio...

2023
[15]

In comparison, since DBCM imposes weaker assumptions, it is reasonable to observe improved performance

However, since benchmarking datasets Dte do not necessarily share the identical class-balance setup (see Table 2 in (LaBonte, Muthukumar, and Kumar 2024)), the objective with which the existing methods optimize may introduce misspecification between the true testing data and t...

2024
[2019]

arXiv preprint arXiv:1907.02893

Invariant risk minimization. arXiv preprint arXiv:1907.02893. Asgari, S.; Khani, A.; Khani, F.; Gholami, A.; Tran, L.; Mahdavi Amiri, A.; and Hamarneh, G

1907 arXiv
[2021]

In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7,

Long-tail learning via logit adjustment. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7,

2021
[2022]

Tsirigotis et al

assume that training data are conditionally identical given different subpopulation conditionals. Tsirigotis et al. (2024) and Menon et al. (2021) requires a uniformity assumption on the label y in the training data. Interestingly, we also discover that all methods can be view...

2024
[2023]

arXiv:arXiv:2312.16243

Mixture Data for Training Cannot Ensure Out-of- distribution Generalization. arXiv:arXiv:2312.16243. Appendix A Supplementary Related Work A.1 Importance Sampling DBA interprets the distributional shift as the mismatch of the weight function from the importance sampling perspe...

2009 arXiv
[2024]

In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27,

Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27,

2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Assumption 5: This assumption characterizes the nature of the subpopulation shift itself by assuming that the shift is caused by a spurious variable influencing Y

for training and testing datasets, this assumption is not restrictive, as we can subsample the training set to approximate the distribution of labels in the test set. Assumption 5: This assumption characterizes the nature of the subpopulation shift itself by assuming that the shift is caused by a spurious variable influencing Y . Such a scenario is freque...

work page 2023

[2] [2]

In Companion proceedings of the 2019 world wide web conference, 491–500

Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, 491–500. Byrd, J.; and Lipton, Z

work page 2019

[3] [4]

In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9,

Decoupled Weight Decay Regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9,

work page 2019

[4] [5]

Assumption 1: This assumption requires inclusion relationships on the supports of the training, testing, and validation sets, ensuring a well-defined weight function for importance sampling—a fundamental assumption required for this approach. Assumption 2: this assumption states that the data generation process for x is identical in both the training and ...

work page 2007

[5] [6]

arXiv preprint arXiv:2403.01874

A Survey on Evaluation of Out-of-Distribution Generalization. arXiv preprint arXiv:2403.01874. Zhang, H.; Cissé, M.; Dauphin, Y . N.; and Lopez-Paz, D

arXiv

[6] [8]

(2023) is the first and possibly the only survey paper that provides a comprehensive experimental study on existing subpopulation methods

To the best of our knowledge, Yang et al. (2023) is the first and possibly the only survey paper that provides a comprehensive experimental study on existing subpopulation methods. The paper decomposes the distribution of y|x to take into account the effect of attributes (i.e. spurious features) via Bayes’ theorem, where y is the random variable for the l...

work page 2023

[7] [9]

for details). In particular, the paper categorizes 12 different datasets into 4 classes that have different correlations between the labels and the attributes, and benchmarks 20 popular subpopulation methods with these datasets, to characterize the performance of the methods given different attribute setups. Nonetheless, the statistical quantification of ...

work page 2024

[8] [10]

We investigate all the papers it cites and all papers that cite this one to guarantee the coverage and scope as much as possible

to identify other existing survey work as this paper is the latest work that conducts comprehensive experimental analysis on existing subpopulation methods. We investigate all the papers it cites and all papers that cite this one to guarantee the coverage and scope as much as possible. data samples, to reduce the effect of the spurious backgrounds. They d...

work page 2024