Pith. sign in

REVIEW 4 major objections 6 minor 26 references

The paper claims that downweighting heavily biased features during subsampling lets debiasing keep more survey responses without significantly hurting downstream task performance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 19:43 UTC pith:UAF4APEW

load-bearing objection Useful incremental extension with a real retention advantage, but the MMD-based distribution-alignment claim is circular and the Algorithm 1 labels contradict the text. the 4 major comments →

arxiv 2603.01013 v1 pith:UAF4APEW submitted 2026-03-01 cs.LG

Feature-Weighted Maximum Representative Subsampling

Tony Hauptmann , Stefan Kramer This is my paper
classification cs.LG
keywords feature weightingdebiasingmaximum representative subsamplingPU learningsample weightsdomain classifiersoftmin temperaturesurvey bias
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that when a survey is biased in only a few features, you can debias it by downweighting those features instead of discarding many rows. It introduces feature-weighted MRS (FW-MRS), which turns a domain classifier's feature importances into weights via a temperature-controlled softmin, then uses those weights inside the MRS iterative removal process. The paper reports that FW-MRS retains more instances than MRS on several datasets while downstream AUROC shows no statistically significant difference. Applied to a real-world social-science survey, lower temperatures keep more samples and reduce distribution mismatch, though extremely low temperatures concentrate weight on a single feature.

Core claim

FW-MRS extends MRS by computing feature weights from a domain classifier trained to separate the representative set from the biased set. Features that strongly separate the two sets are considered highly biased and receive low weights through the softmin transformation with temperature t. These weights then influence both the PU-learning classifier that decides which samples to drop and the downstream classifier. The paper's experimental claim is that this feature downweighting reduces the number of samples that must be removed while keeping downstream classification performance statistically indistinguishable from MRS, and that feature weights provide an interpretable per-feature measure of

What carries the argument

The central object is the feature-weight transformation: softmin of domain-classifier feature importances with temperature t, producing weights w_f that downweight the most biased features. This transformation controls the trade-off between aggressively removing non-representative samples and aggressively suppressing biased features. The mechanism sits inside the PU-learning loop of MRS, where the domain classifier is trained with both sample weights and feature weights, and the stopping criterion is a mean AUROC at or below 0.5. The temperature parameter is the main dial: lower temperatures give more peaked feature weights, fewer dropped samples, and lower MMD, but risk over-concentrating w

Load-bearing premise

The paper's prose and pseudocode disagree about which dataset is treated as the positive class in the PU classifier—Section 3 says the representative set R is positive, while Algorithm 1 trains with N as positive—so a reader cannot determine which samples are actually removed and whether the experimental results reflect the described method.

What would settle it

Rerun the debiasing experiments with a logging step that records, for each dropped sample, its predicted probability of belonging to N versus R. If the implementation follows Algorithm 1, the dropped samples will be those most confidently in N, yielding a retained set that still resembles N and contradicts the stated goal of aligning N to R; if the implementation follows Section 3, dropped samples will be those most confidently in R, matching the paper's narrative. This distinction can be tested on a synthetic dataset where only one feature is biased.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Surveys with a few highly biased features can be debiased while discarding fewer responses, preserving statistical power.
  • Feature weights offer an interpretable per-feature bias score, which could help researchers identify which survey questions drive the bias.
  • The temperature parameter provides a continuous lever between aggressive sample removal and aggressive feature downweighting, enabling a targeted trade-off.
  • Since downstream classification performance shows no statistically significant drop, FW-MRS could replace MRS in settings where retaining samples matters.
  • The method generalizes the MRS framework to handle non-uniform bias across features, not just uniform bias.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One clear internal contradiction: the pseudocode in Algorithm 1 trains the PU classifier with positive class = N_train and negative class = R_train, while Section 3 states R is the positive set P and N is the unlabeled set U. If the code follows the pseudocode, the algorithm may be dropping samples that are most confidently part of N, which is the opposite of the intended removal of non-representa
  • The feature-weighting idea could be applied to other weight-based debiasing methods such as propensity-score reweighting or kernel mean matching, not only to MRS, potentially reducing sample loss in those settings too.
  • The temperature could be tuned automatically toward a compound objective that balances sample retention and distribution alignment, rather than optimizing downstream AUROC alone.
  • If the pseudocode version is actually what runs in the experiments, the observed 'no significant difference' in downstream AUROC might be an artifact of removing the wrong samples, and the real behavior of the described FW-MRS could be better or worse than reported.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Feature-Weighted Maximum Representative Subsampling (FW-MRS), an extension of MRS that computes feature weights from a domain classifier via a softmin transformation and uses them both during the iterative removal of samples and in the downstream classifier. Two variants are described (random forest and linear SVM). The method is evaluated on eight tabular datasets with artificially introduced bias and on a real-world survey (GBS) against MRS, KMM, and PSA. The main empirical claims are that FW-MRS retains more samples than MRS and that downstream AUROC is not statistically significantly worse, with supplementary MMD results used to argue for improved distribution alignment.

Significance. If the claims hold, FW-MRS offers a practical tool for debiasing surveys when only a subset of features is biased, with a better sample-retention/downstream-performance trade-off than MRS. The paper ships open-source code and employs a corrected repeated k-fold cross-validation t-test with Benjamini–Hochberg correction, which are notable strengths. However, the distribution-alignment evidence is compromised by circularity: the reported MMD is computed on a kernel that uses the very feature weights produced by FW-MRS, so the improved MMD may largely reflect downweighting of biased features rather than genuine alignment. The downstream AUROC null result is the most robust non-circular evidence, but it is power-limited and does not by itself demonstrate that bias has been removed. The algorithm pseudocode also has a label inversion relative to the text, undermining reproducibility.

major comments (4)
  1. [Section 3 and Algorithm 1 (line 10)] The text states that MRS treats the representative dataset R as the positive set P and the non-representative dataset N as the unlabeled set U. However, Algorithm 1 line 10 explicitly uses 'positive class = N_train, negative class = R_train'. The removal direction is only well-defined if the reader knows which class probability is stored in p. If p is the probability of N, then argmax p removes the most N-like samples, which matches the method's stated intent; if p is the probability of R, argmax p would remove representative-like samples. The prose and pseudocode must be reconciled, and the text must state explicitly which score is stored and which side is dropped.
  2. [Eqs. (2)–(3), Table 2, Table 5, Section 4.4] The MMD is computed with feature weights w_f that are exactly the output of FW-MRS. Because FW-MRS deliberately downweights the features where N and R differ most (Figure 5 shows near-total concentration on Sex at low temperature), the reported MMD decreases may reflect the kernel de-emphasizing the biased features rather than actual alignment of the retained subsample. This circularity undermines the distribution-alignment claims in Section 4.4 and Supplementary C. Please report MMD on the original feature scale (w_f = 1) or on a held-out subset of features, and show whether the improvement persists.
  3. [Algorithm 1, lines 15–16 (stopping criterion)] The AUROC stopping criterion uses the same feature-weighted classifier. If the feature weights suppress the most discriminative features, the classifier may appear unable to separate N and R (AUROC ≤ 0.5) and stop early even though the unweighted distributions remain substantially different on those features. This compounds the circularity: the early stopping may be an artifact of the weighting rather than a sign of achieved alignment. Please analyze the per-iteration AUROC on unweighted features or otherwise show that the retained subsample is aligned in the original feature space.
  4. [Section 4.1 (hyperparameter optimization)] The hyperparameters, including temperature, are optimized on N using downstream AUROC as the model selection criterion. This optimizes for downstream task performance, not for retention or alignment. The retention advantage in Figure 4 may therefore be a by-product of the chosen temperature and other hyperparameters rather than a rigorous property of the method. Please report the distribution of selected temperatures across datasets and runs, and ideally provide results when hyperparameters are optimized for retention or a compound alignment objective, as the Discussion suggests is possible.
minor comments (6)
  1. [Figure 2 legend] The temperature value '0.0050' appears twice in the legend; one should likely be '0.05'.
  2. [Section 4.2] The sentence 'We performed the same procedure for MRS to work as a baseline' should read 'to serve as a baseline.'
  3. [Algorithm 1] The while-condition 'while R differs from N' is informal; it should reference the AUROC stopping test. Also, lines 12–13 should explicitly define which class is treated as positive in the AUROC computation.
  4. [Tables 1 and 5] The layout of the 'Unbiased' row/column is inconsistent (e.g., 'Rank' row in Table 1 has a dangling '/ 0'). Please clarify that 'Unbiased' is a reference, not a debiasing method.
  5. [Appendix B] The formula for the number of distinct datasets is truncated ('allows [equation] distinct datasets'). Please complete the equation and its explanation.
  6. [Table 1 and significance tests] Since the paper claims no statistically significant differences, it would be helpful to include an effect-size measure or a power analysis to clarify what differences the test could detect.

Circularity Check

2 steps flagged

Distribution-alignment evidence is circular: the reported MMD uses FW-MRS's own feature weights, so improved MMD is partly by construction; the downstream AUROC results are independent and keep the paper from full circularity.

specific steps
  1. self definitional [Section 4.4 / Table 2; Appendix C; Eqs. (2)-(3)]
    "MMD(X,Y,w_X,w_Y,w_f) = [ ... ] (2) where w_X and w_Y are the sample weights, w_f are the feature weights, and k is the weighted radial basis function kernel: k(x,y,w_f)=exp(−∑_j w_f^{(j)}(x_j−y_j)^2/(2σ^2)). (3)"

    The MMD reported as evidence of 'distribution alignment' is a function of w_f, the very feature-weight vector FW-MRS outputs via softmin of domain-classifier importances (Eq. 1). Because softmin assigns near-zero weights to the features that separate N from R, the kernel measures distance only in a subspace from which the biased signal has been deliberately removed. In the GBS experiment, lowering t concentrates weight on Sex (Fig. 5) and MMD drops to 0.032 vs MRS's 0.153 (Table 2), while the biased features (Employment Status, School Degree, Occupational Group) are downweighted almost to zero. The MMD comparison is therefore not on a common metric: FW-MRS's improvement is built into the kernel, not evidence that the retained sample matches R on the originally biased features.

  2. fitted input called prediction [Algorithm 1, lines 4-16; Section 3]
    "If the AUROC falls below 0.5, indicating the classifier is not better than random guessing, FW-MRS terminates, assuming sufficient distribution alignment. ... classifier←train_pu_classifier(positive class=N_train, negative class=R_train, w_s, w_f)"

    The stopping condition that determines how many samples are retained and declares 'sufficient distribution alignment' is evaluated by a domain classifier trained with the same w_f that was designed to reduce the influence of the most biased features. The early stop (and hence 'retains more instances') is induced by the feature weighting itself: the classifier can no longer see the biased signal. This makes the retention/alignment result partly a consequence of the algorithm's own construction rather than an independent validation. The downstream AUROC comparison, being an external test-set benchmark, remains the only non-circular guard.

full rationale

FW-MRS is a legitimate algorithmic extension of MRS, and the downstream-task evaluation (test-set AUROC, corrected repeated k-fold t-test, Table 1) is external and independent: temperature and hyperparameters are optimized on N, and the null result is an honest, non-circular finding. The feature weights themselves are not fitted to the downstream labels, and the paper does not invoke a self-citation chain or a uniqueness theorem to force its method. The circularity is concentrated in the distribution-alignment evidence. Equations (2)-(3) define the MMD with feature weights w_f that are exactly FW-MRS's output; comparing FW-MRS's weighted MMD to MRS's unweighted MMD (or using the method's own weights to measure its own success) makes the reported MMD improvement largely by construction. The Algorithm 1 label inconsistency (positive/negative classes reversed relative to Section 3) is a real correctness issue in the text but is not itself circularity. Overall, because the central debiasing claim leans on the circular MMD and stopping-criterion evidence, while the downstream performance claim remains independent, a score of 6 is appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The central claim rests on the PU learning setup (same population for N and R), the validity of SHAP importances as bias indicators, the AUROC stopping rule, and the synthetic bias simulation. No new physical or mathematical entities are introduced; the only fitted quantities are hyperparameters (temperature, RF leaf fraction, SVM C, MMD sigma) and the per-iteration drop count d.

free parameters (5)
  • temperature t = grid-searched in [0.001, 0.5] per dataset
    Controls the peakedness of the softmin feature weights; optimized for downstream AUROC (Section 4.1).
  • minimum weight fraction per leaf (RF) = grid [0.025, 0.01, 0.001, 0.0]
    Hyperparameter of the feature-weighted random forest in FW-MRSRF (Section 4.1).
  • SVM regularization C = grid [1e-2, ..., 1e2]
    Hyperparameter of the linear SVM in FW-MRSSVM (Section 4.1).
  • samples dropped per iteration d = 1 for small datasets, 5 for larger ones
    Chosen by dataset size to manage runtime; affects convergence (Section 4.1).
  • MMD kernel width sigma = mean pairwise distance in the combined dataset
    Heuristic choice for the RBF kernel in MMD computation (Section 4.1).
axioms (6)
  • domain assumption R and N are drawn from the same population and share relevant variables
    MRS requires representative dataset R from the same population; stated in Section 3 and used throughout.
  • domain assumption SHAP feature importance from the domain classifier reflects the degree of bias in each feature
    FW-MRS derives feature weights from these importances; Section 3, 'Features that strongly differentiate between the two classes, non-representative (N) and representative (R), are considered highly biased.'
  • domain assumption AUROC <= 0.5 indicates that the distributions are aligned
    Used as the stopping criterion in Algorithm 1 (Line 15-16).
  • domain assumption Undersampling the positive class to 10% is a valid proxy for real selection bias
    Used in all synthetic experiments; Section 4.1: 'Artificial bias is introduced by undersampling the positive class in N, retaining only 10% of its original positive samples.'
  • standard math Kernel MMD with heuristic sigma measures distribution distance
    Equations (2)-(3); standard RKHS distance metric.
  • standard math Corrected repeated k-fold CV t-test is valid for comparing algorithm performance
    Used for significance testing; Section 4.3.

pith-pipeline@v1.3.0-alltime-deepseek · 13646 in / 11536 out tokens · 105052 ms · 2026-08-02T19:43:47.686693+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Feature-Weighted Maximum Representative Subsampling." pith.science (2026). https://pith.science/paper/UAF4APEW

@misc{pith2026260301013,
  author       = {Pith},
  title        = {Pith review of: Feature-Weighted Maximum Representative Subsampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UAF4APEW}},
  note         = {Machine review of arXiv:2603.01013}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In the social sciences, it is often necessary to debias studies and surveys before valid conclusions can be drawn. Debiasing algorithms enable the computational removal of bias using sample weights. However, an issue arises when only a subset of features is highly biased, while the rest is already representative. Algorithms need to strongly alter the sample distribution to manage a few highly biased features, which can in turn introduce bias into already representative variables. To address this issue, we developed a method that uses feature weights to minimize the impact of highly biased features on the computation of sample weights. Our algorithm is based on Maximum Representative Subsampling (MRS), which debiases datasets by aligning a non-representative sample with a representative one through iterative removal of elements to create a representative subsample. The new algorithm, named feature-weighted MRS (FW-MRS), decreases the emphasis on highly biased features, allowing it to retain more instances for downstream tasks. The feature weights are derived from the feature importance of a domain classifier trained to differentiate between the representative and non-representative datasets. We validated FW-MRS using eight tabular datasets, each of which we artificially biased. Biased features can be important for downstream tasks, and focusing less on them could lead to a decline in generalization. For this reason, we assessed the generalization performance of FW-MRS on downstream tasks and found no statistically significant differences. Additionally, FW-MRS was applied to a real-world dataset from the social sciences. The source code is available at https://github.com/kramerlab/FeatureWeightDebiasing.

Figures

Figures reproduced from arXiv: 2603.01013 by Stefan Kramer, Tony Hauptmann.

Figure 1
Figure 1. Figure 1: Schema for feature-weighted maximum representative sampling (FW-MRS). Two surveys stem from the same population: one is biased and includes the variable under investigation, while the other is representative but does not include it. FW-MRS mitigates bias by comparing the distributions of the biased and representative datasets using a classifier that leverages auxiliary information from the representative d… view at source ↗
Figure 2
Figure 2. Figure 2: Validation AUROC vs. number of dropped samples: All hyperparameters were fixed except for the temperature, which was varied. Each point denotes the mean AUROC across runs, with ellipses indicating the standard deviation. Circles correspond to FW-MRSRF and the triangle represents MRS. point at which most biased features are used infrequently, resulting in only marginal reductions in dropped samples. At this… view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Relative dropped samples over 50 iterations with 10 times repeated 5-fold cross-validation [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Feature weights used in FW-MRSRF debiasing of GBS with auxiliary information of Allensbach [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of mean AUROC of MRS vs. FW-MRSRF with different temperatures on 50 debiasing runs of GBS with auxiliary information of Allensbach. 15/16 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: illustrates the feature importance of the unweighted domain classifier for GBS and Allensbach. It reveals that the most important and, hence, most biased features are associated with subjects’ education and employment. This finding confirms the bias towards an overrepresentation of highly educated individuals. 0.00 0.02 0.04 0.06 0.08 0.10 Feature Importance Employment Status School Degree Occupational Gro… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 8 canonical work pages

  1. [1]

    & Mare, R

    Winship, C. & Mare, R. D. Models for Sample Selection Bias.Annu. Rev. Sociol.18, 327–350, DOI: 10.1146/annurev.so.1 8.080192.001551 (1992)

    arXiv 1992

  2. [2]

    T., Sakshaug, J

    West, B. T., Sakshaug, J. W. & Aurelien, G. A. S. How Big of a Problem is Analytic Error in Secondary Analyses of Survey Data?PLOS ONE11, e0158120, DOI: 10.1371/journal.pone.0158120 (2016)

    doi:10.1371/journal.pone.0158120 2016

  3. [3]

    Smith, L. H. Selection Mechanisms and Their Consequences: Understanding and Addressing Selection Bias.Curr. Epidemiol. Reports7, 179–189, DOI: 10.1007/s40471-020-00241-6 (2020)

    work page doi:10.1007/s40471-020-00241-6 2020

  4. [4]

    & Cusson, A

    Infante-Rivard, C. & Cusson, A. Reflection on modern methods: Selection bias—a review of recent developments.Int. J. Epidemiol.47, 1714–1722, DOI: 10.1093/ije/dyy138 (2018)

    work page doi:10.1093/ije/dyy138 2018

  5. [5]

    R., Barber, S

    Keeble, C., Law, G. R., Barber, S. & Baxter, P. D. Choosing a Method to Reduce Selection Bias: A Tool for Researchers. Open J. Epidemiol.5, 155–162, DOI: 10.4236/ojepi.2015.53020 (2015)

    arXiv 2015

  6. [6]

    & Kramer, S

    Hauptmann, T., Fellenz, S., Nathan, L., Tüscher, O. & Kramer, S. Discriminative machine learning for maximal representative subsampling.Sci. Reports13, 20925, DOI: 10.1038/s41598-023-48177-3 (2023). 10/16

    work page doi:10.1038/s41598-023-48177-3 2023

  7. [7]

    & Zheng, L

    Huang, J., Zhou, J. & Zheng, L. Support Vector Machine Classification Algorithm Based on Relief-F Feature Weighting. In2020 International Conference on Computer Engineering and Application (ICCEA), 547–553, DOI: 10.1109/ICCEA5 0009.2020.00121 (IEEE, Guangzhou, China, 2020)

    arXiv 2020

  8. [8]

    & Schapire, R

    Dudík, M., Phillips, S. & Schapire, R. E. Correcting sample selection bias in maximum entropy density estimation. In Advances in Neural Information Processing Systems, vol. 18 (MIT Press, 2005)

    2005

  9. [9]

    & Smola, A

    Huang, J., Gretton, A., Borgwardt, K., Schölkopf, B. & Smola, A. Correcting Sample Selection Bias by Unlabeled Data. InAdvances in Neural Information Processing Systems, vol. 19 (MIT Press, 2006)

    2006

  10. [10]

    Rosenbaum, P. R. The Central Role of the Propensity Score in Observational Studies for Causal Effects. In Rubin, D. B. (ed.)Matched Sampling for Causal Effects, 170–184, DOI: 10.1017/CBO9780511810725.016 (Cambridge University Press, Cambridge, 2006)

    work page doi:10.1017/cbo9780511810725.016 2006

  11. [11]

    Rueda, M. d. M., Pasadas-del-Amo, S., Rodríguez, B. C., Castro-Martín, L. & Ferri-García, R. Enhancing estimation methods for integrating probability and nonprobability survey samples with machine-learning techniques. An application to a Survey on the impact of the COVID-19 pandemic in Spain.Biom. J.65, 2200035, DOI: 10.1002/bimj.202200035 (2023)

    work page doi:10.1002/bimj.202200035 2023

  12. [12]

    1505.07818

    Ganin, Y .et al.Domain-Adversarial Training of Neural Networks, DOI: 10.48550/arXiv.1505.07818 (2016). 1505.07818

    Pith/arXiv arXiv doi:10.48550/arxiv.1505.07818 2016

  13. [13]

    & Portillo, E

    Niño-Adan, I., Manjarres, D., Landa-Torres, I. & Portillo, E. Feature weighting methods: A review.Expert. Syst. with Appl.184, 115424, DOI: 10.1016/j.eswa.2021.115424 (2021)

    arXiv 2021

  14. [14]

    & Lee, Y .-S

    Amaratunga, D., Cabrera, J. & Lee, Y .-S. Enriched random forests.Bioinforma. (Oxford, England)24, 2010–2014, DOI: 10.1093/bioinformatics/btn356 (2008)

    work page doi:10.1093/bioinformatics/btn356 2010

  15. [15]

    & Davis, J

    Bekker, J. & Davis, J. Learning from positive and unlabeled data: A survey.Mach. Learn.109, 719–760, DOI: 10.1007/s10994-020-05877-5 (2020)

    doi:10.1007/s10994-020-05877-5 2020

  16. [16]

    M.et al.From local explanations to global understanding with explainable AI for trees.Nat

    Lundberg, S. M.et al.From local explanations to global understanding with explainable AI for trees.Nat. Mach. Intell.2, 56–67, DOI: 10.1038/s42256-019-0138-9 (2020)

    doi:10.1038/s42256-019-0138-9 2020

  17. [17]

    & Bloebaum, P

    Janzing, D., Minorics, L. & Bloebaum, P. Feature relevance quantification in explainable AI: A causal problem. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, 2907–2916 (PMLR, 2020)

    2020

  18. [18]

    & Lee, S.-I

    Lundberg, S. & Lee, S.-I. A Unified Approach to Interpreting Model Predictions, DOI: 10.48550/arXiv.1705.07874 (2017). 1705.07874

    Pith/arXiv arXiv doi:10.48550/arxiv.1705.07874 2017

  19. [19]

    & Schmidt, L

    Ding, F., Hardt, M., Miller, J. & Schmidt, L. Retiring Adult: New Datasets for Fair Machine Learning. InAdvances in Neural Information Processing Systems, vol. 34, 6478–6490 (Curran Associates, Inc., 2021). 20.Hofmann, H. Statlog (German Credit Data), DOI: 10.24432/C5NC77 (1994). 21.Kahn, M. Diabetes, DOI: 10.24432/C5T59G (0). 22.S. Moro, P. R. Bank Marke...

    doi:10.24432/c5nc77 2021

  20. [23]

    & Street, W

    Wolberg, W., Mangasarian, O., Street, N. & Street, W. Breast cancer wisconsin (diagnostic). UCI Machine Learning Repository, DOI: 10.24432/C5DW2B (1995)

    doi:10.24432/c5dw2b 1995

  21. [24]

    & Thomas-Agnan, C.Reproducing Kernel Hilbert Spaces in Probability and Statistics(Springer Science & Business Media, Berlin Heidelberg, 2011)

    Berlinet, A. & Thomas-Agnan, C.Reproducing Kernel Hilbert Spaces in Probability and Statistics(Springer Science & Business Media, Berlin Heidelberg, 2011)

    2011

  22. [25]

    M., Rasch, M

    Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B. & Smola, A. A Kernel Two-Sample Test.J. Mach. Learn. Res. 13, 723–773 (2012)

    2012

  23. [26]

    & Couper, M

    Schonlau, M. & Couper, M. P. Options for Conducting Web Surveys.Stat. Sci.32, 279–292, DOI: 10.1214/16-STS597 (2017)

    work page doi:10.1214/16-sts597 2017

  24. [27]

    Bouckaert, R. R. & Frank, E. Evaluating the Replicability of Significance Tests for Comparing Learning Algorithms. In Kanade, T.et al.(eds.)Advances in Knowledge Discovery and Data Mining, vol. 3056, 3–12, DOI: 10.1007/978-3-540-2 4775-3_3 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2004)

    work page doi:10.1007/978-3-540-2 2004

  25. [28]

    & Wolpert, D

    Kohavi, R. & Wolpert, D. Bias plus variance decomposition for zero-one loss functions. InProceedings of the Thirteenth International Conference on International Conference on Machine Learning, ICML’96, 275–283 (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1996)

    1996

  26. [29]

    A Unified Bias-Variance Decomposition for Zero-One and Squared Loss

    Domingos, P. A Unified Bias-Variance Decomposition for Zero-One and Squared Loss. InProceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, 564–569 (AAAI Press, 2000). 11/16 Author Contributions Tony Hauptmann implemented methods and experiments and wrote t...

    arXiv 2000