Pith. sign in

REVIEW 3 major objections 5 minor 47 references

CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CaliMatch argues that overconfidence, not the unseen-class mismatch itself, breaks semi-supervised learning, and that calibrating the classifier and OOD detector during training fixes it, beating seven methods on five image benchmarks.

desk verdict Adaptive calibration scheme for safe SSL, but the empirical comparison may be confounded by CaliMatch's online use of a validation set that baselines do not get. read the letter →

arxiv 2508.00922 v1 pith:UORNZOTW submitted 2025-07-30 cs.LG

Jinsoo Bae , Seoung Bum Kim , Hyungrok Do This is my paper
classification cs.LG
keywords safesemi-supervisedlearningout-of-distributiondetectionconfidencecalibrationadaptivelabelsmoothingtemperaturescalingpseudo-labelingdistributionmismatchoverconfidence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CaliMatch argues that in safe semi-supervised learning — the setting where the unlabeled pool contains classes absent from the labeled set — the main saboteur is overconfidence, not the mismatch itself: deep networks produce wrong pseudo-labels and missed out-of-distribution samples while sounding certain, and every threshold-based selection rule trusts those numbers. The proposed fix is to calibrate both the multiclass classifier and the one-versus-rest OOD detector during training, using label smoothing whose strength adapts per sample to the model's measured accuracy in that confidence bin, together with learnable logit temperatures. On CIFAR-10, CIFAR-100, SVHN, TinyImageNet, and ImageNet, the method reports the best accuracy and calibration among eight compared methods at unseen-class ratios of $\kappa = 30\%$ and $\kappa = 60\%$. The paper also gives a formal argument that calibrated confidence lowers the probability of selecting bad samples and pulls the training gradient toward the fully supervised ideal. If the claim is right, calibration is not a post-hoc nicety but a load-bearing component of any threshold-based SSL method.

What carries the argument

The carrying mechanism is the pairing of adaptive label smoothing with learnable logit scaling on both heads. The unit interval of model confidence is split into $M = 30$ equal bins; for each bin the model's accuracy on a held-out validation set is computed, and that per-bin accuracy becomes the reference confidence ($\gamma$ for the multiclass classifier, $\delta$ for the OOD detector) that defines the smoothed label targets for any sample whose confidence lands in that bin. The scaled probabilities $p_k^s(x) = \mathrm{softmax}_k(f_\theta(x)/T_M)$ and $q_k^s(x) = \sigma_k(g_\theta(x)/T_O)$ are trained against those instance-specific targets with $T_M$ and $T_O$ initialized at 1.5 and learned by gradient descent, so the model is pushed to be exactly as confident as it is accurate. These calibrated probabilities form the seen-class score $s_i^u = \sum_{k\in\mathcal{Y}} p_k^s(x_i^u)\,q_k^s(x_i^u)$ and the OOD score $u_i^u = 1 - s_i^u$; an unlabeled sample is consumed by the FixMatch consistency loss only if $s_i^u > \tau_1$ and $\max_k p_k^s(x_i^u) > \tau_2$.

What would settle it

On a benchmark where the true labels of all unlabeled in-distribution samples are known, record throughout training the per-batch gradient distance between the FixMatch surrogate loss and the ideal cross-entropy loss on the correctly pseudo-labeled ID subset; the paper's Theorem 1 predicts this distance shrinks as calibration error drops, so if accuracy gains appear while that distance stays flat, the claimed mechanism would be falsified. A second check: shrink the labeled validation holdout (for example from 10% to 2%) and see whether the bin-accuracy references stop tracking true per-bin accuracy and whether CaliMatch's advantage over OpenMatch collapses.

Watch

Extended reading notes

Core claim

CaliMatch's central claim is that safe SSL fails less from the label-distribution mismatch itself than from overconfident confidence estimates: deep networks assign high probability to wrong pseudo-labels and to unseen-class samples, and the two heads a safe-SSL model depends on — the multiclass classifier $f_\theta$ and the one-versus-rest OOD detector $g_\theta$ — are both miscalibrated. The paper proposes to calibrate both heads during training with adaptive label smoothing, where the smoothing degree for each sample is set by the validation accuracy of the confidence bin the sample falls into, plus learnable temperature parameters $T_M$ and $T_O$ (initialized at 1.5) that rescale the logits and are optimized by gradient descent. A seen-class score $s_i^u = \sum_{k \in \mathcal{Y}} p_k^s(x_i^u)\, q_k^s(x_i^u)$ and a calibrated max-confidence $c_i^u$ gate which unlabeled samples enter the FixMatch consistency loss, and the OOD score is $u_i^u = 1 - s_i^u$. Lemma 1 bounds the probability of selecting an OOD or mislabeled sample by $1 - \min\{\tau_1, \tau_2\} + \eta$ where $\eta$ is the calibration error, and Theorem 1 bounds the gradient gap between the surrogate FixMatch loss and the ideal loss on clean in-distribution data by a term proportional to $|B_t^u|\varepsilon$, so better calibration provably steers optimization toward the supervised ideal. Across the five benchmarks the method reports top or near-top accuracy and the lowest expected calibration error, including 63.04% top-1 on ImageNet at $\kappa = 60\%$.

Load-bearing premise

CaliMatch's per-bin accuracy references are computed on a held-out labeled validation set (10% of the labeled training split on the four small benchmarks), so the calibration targets and smoothing degrees are only as trustworthy as that validation set's representativeness of the labeled distribution.

Editorial extensions

If this is right

  • Selecting pseudo-labels on calibrated confidence $c_i^u$ instead of raw max-softmax reduces the number of wrong labels that enter consistency regularization, which the paper's learning curves identify as the main accuracy driver.
  • Calibrating the OOD detector shrinks the fraction of unseen-class samples among high-confidence selections, so fewer out-of-distribution instances are absorbed into training and the OOD detection ECE drops accordingly.
  • The ablations indicate the two calibrations play complementary roles: multiclass calibration produces most of the classification gain, OOD calibration most of the OOD-detection calibration gain, and the shared encoder benefits from both.
  • The gains persist when the unseen-class ratio rises from 30% to 60% and on ImageNet at 60% mismatch, where CaliMatch reaches 63.04% top-1 accuracy and beats OpenMatch by 1.24 percentage points, so the mechanism is not limited to small-scale datasets.
  • CaliMatch stays ahead of OpenMatch across a grid of the two thresholds $\tau_1$ and $\tau_2$, so the improvement is not an artifact of a lucky threshold choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: the same bin-accuracy smoothing losses could be attached to FixMatch itself — without any OOD head — and should measurably reduce its vulnerability to unseen classes in the unlabeled pool.
  • The fragile input is the held-out validation estimate: with very small labeled budgets the per-bin accuracy references become noisy, so one natural variant is to estimate bin accuracy from the training labels themselves or from a cross-fit of the validation split; the paper leaves this unexplored.
  • The learnable temperatures can be read as an online, train-time analogue of temperature scaling, which suggests tracking whether $T_M$ and $T_O$ converge to values that minimize validation ECE — a diagnostic the paper does not report.
  • Because the seen-class score multiplies classifier and OOD probabilities, each head caps the other: if either is miscalibrated the combined score inherits the worse head, consistent with the paper's finding that on TinyImageNet calibrating only the OOD detector gives the lowest OOD-detection ECE.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. CaliMatch proposes a safe semi-supervised learning method that calibrates both the multiclass classifier and the one-versus-rest OOD detector during training. It uses adaptive label smoothing, where the smoothing degree is set by binning validation-set predictions and computing per-bin accuracies, and learnable temperature parameters for both heads. The method is built on FixMatch and uses the calibrated scores for pseudo-label selection and OOD rejection. Experiments on CIFAR-10, CIFAR-100, SVHN, TinyImageNet, and ImageNet report accuracy improvements over existing safe SSL methods, together with improved expected calibration error (ECE). The paper also includes a theoretical analysis (Lemma 1 and Theorem 1 in Supplementary Section S-3) intended to justify why calibration of both heads helps safe SSL.

Significance. If the reported gains are genuine, calibrating both the multiclass classifier and the OOD detector is a valuable and potentially generalizable contribution to safe SSL: the method is simple, the ablations are extensive, and the authors provide source code. The paper also makes a useful empirical point that overconfidence in both heads degrades pseudo-label quality and OOD rejection. However, the central empirical claim is confounded by the training-time use of a held-out labeled validation set, and the theoretical justification is tautological; these issues must be resolved before the contribution can be assessed. The breadth of datasets (including ImageNet) and the inclusion of learning-curve and threshold analyses are strengths, but they do not compensate for the unfair comparison induced by extra validation-label access.

major comments (3)
  1. [Algorithm 1 and Table S-2] The method's calibration targets Γ and Δ are computed at every epoch from a held-out labeled validation set, which is 10% of the training dataset (Table S-2). These targets enter the training loss through Eqs. (4) and (5). This constitutes training-time use of labels that the comparison methods do not receive: the baselines train only on the labeled training set, whereas CaliMatch additionally uses validation labels to shape its per-bin accuracy estimates and therefore its loss. The paper does not state whether the baselines also train on the full labeled split or on the remaining 90%, but in either case CaliMatch has access to strictly more labeled information. The reported gains (e.g., CIFAR-10 with κ=60%: 87.62 vs 86.19; Table S-8: ResNet-50 accuracy 72.73 vs 58.72) could therefore reflect this extra supervision rather than the calibration mechanism. The ablation in Table S-6 does not control for this, because the OpenMatch variants are not given the same online validation-based target computation. Please add an experiment in which every method is given the same number of labeled examples for training (with CaliMatch's validation set drawn from that budget) and report the resulting accuracies, or, alternatively, compute Γ and Δ on a separate validation set that is not used by any method and show the gains persist. This is the load-bearing issue: the main claim of improved accuracy is confounded by unequal access to validation labels.
  2. [Supplementary Section S-3.2, Lemma 1 and Theorem 1] The theoretical justification is circular. Lemma 1 defines ε as the probability of the bad event (a selected sample being OOD or incorrectly pseudo-labeled) and then concludes that the probability of the bad event is at most ε; the bound ε ≤ 1 - min{τ1,τ2} + η follows directly from the calibration assumption that accuracy equals confidence, not from any property of the proposed adaptive smoothing or temperature scaling. Moreover, the condition min{τ1,τ2} ≥ 1-η is inconsistent with the experimental setting τ1 = 0.5 (main text, Section 4) and small calibration error η. Theorem 1 relies on Lemma 1 and therefore inherits its circularity. Since the abstract and contributions claim a theoretical justification for improving calibration of both heads, this claim is not supported. I recommend either rewriting the theory to make a non-tautological statement (e.g., a bound that explicitly depends on the proposed losses' effect on calibration error) or removing the theoretical claim from the abstract and contributions.
  3. [Section 3.2 and Table S-6] The claim that adaptive label smoothing 'eliminates the need to manually tune the smoothing degree' is overstated. The method still requires choosing the number of bins M, the thresholds τ1 and τ2, the coefficients λO, λOCal, and λS, and the initial temperatures TM and TO. The sensitivity analysis in Table S-5 covers only λO and λOCal; there is no sensitivity analysis for M or for the initialization of the temperatures. Furthermore, Table S-6 compares CaliMatch against OpenMatch with fixed label smoothing, mixup, MbLS, and RankMixup, but those baselines are not given the same validation-based adaptive targets, so the comparison conflates the validation-access effect with the calibration mechanism. Please provide an ablation that varies M and the temperature initialization, and add a control where the alternative calibration methods receive the same kind of validation-based adaptation (if feasible) or at least explicitly state the training-label budget for each row of Table S-6.
minor comments (5)
  1. [Table S-4] In the TinyImageNet row, the CaliMatch F1 value is printed as '0.691 0.001' without parentheses around the standard deviation; please format consistently with the other entries.
  2. [Algorithm 1] The line 'Update Γ and ∆ using validation dataset at every epoch' appears after the parameter update, but it is unclear whether the update happens before or after the epoch's training iterations; please specify the order explicitly, especially because the calibration loss in Eqs. (4) and (5) uses Γ and ∆ from the previous epoch or from the current epoch.
  3. [Section 3.2] The initial values of the scaling parameters TM and TO are set to 1.5 without any justification or sensitivity analysis; please provide a brief explanation or a small study showing the effect of this choice.
  4. [Equation (5)] The OOD calibration loss in Eq. (5) contains a min over k of the second term; the notation is confusing because the min is applied to a sum over k of per-class terms. Please clarify whether the min is inside the sum or over the whole second term, and specify the exact operation that defines the OOD smoothing target.
  5. [Section 4.2 and Figure 2] The reliability diagrams in Figure 2 are described with blue and red bars, but the figure appears in grayscale; please ensure the figure is reproducible in color or add patterns to distinguish the bars.

Circularity Check

1 steps flagged · score 4.0 of 10

Theoretical justification is partly tautological: Lemma 1 defines ε as the bad-event probability and then 'concludes' that good selection occurs with probability at least 1−ε; the main empirical accuracy claims remain independent.

  1. self definitional [Supplementary Section S-3.2, Lemma 1 (and its use in Theorem 1)]
    "Here, we define ε as follows: ε = P((x_i^u ∈ B_t^u) ∧ ((y_i^u ∉ Y) ∨ (y_i^u ≠ ŷ_i^u))), which represents the probability that a sample x_i^u is either OOD or incorrectly pseudo-labeled. ... P((x_i^u ∈ B_t^u) ⇒ (y_i^u ∈ Y) ∧ (y_i^u = ŷ_i^u)) ≥ 1 − ε. Moreover, as model calibration improves (i.e., as calibration error η decreases), the probability of incorrect selection ε decreases."

    The lemma defines ε as exactly the probability of the bad event (a selected unlabeled sample is OOD or is assigned an incorrect pseudo-label). Its main conclusion, that a selected sample is good with probability at least 1−ε, is therefore a restatement of the definition and carries no independent content. The causal claim that ε decreases as the calibration error η decreases is asserted rather than derived; the proof only establishes an upper bound on the already-defined quantity. Theorem 1 then takes ε as a small parameter to conclude that better calibration aligns surrogate and ideal gradients, so the advertised theoretical justification reduces to bounding a quantity by itself plus an unproved monotonicity assertion. The empirical benchmark results, however, do not rely on this lemma.

full rationale

Most of CaliMatch's derivation is self-contained. The adaptive label-smoothing targets γ and δ are per-bin validation accuracies, and Equations (4) and (5) train confidences to match those measured accuracies; this is a self-consistency calibration objective rather than a fit of the final test accuracy, so the reported accuracy comparisons in Tables 1 and 2 are not forced by construction. No load-bearing self-citation or imported uniqueness argument appears: the method uses standard FixMatch and OpenMatch building blocks, and reference [1] by the same authors is only cited as one of several safe-SSL methods. The one exhibitable circular step is the supplementary theory: Lemma 1 defines ε as the bad-event probability and then 'proves' that the good-event probability is at least 1−ε, which is true by definition, while the needed implication 'calibration error η decreases ⇒ ε decreases' is not derived. This makes the theoretical justification partly tautological, but the central empirical claims stand independently of that lemma. The online use of a 10% validation split to update Γ and Δ at every epoch (Table S-2) is a potential experimental confound relative to baselines that do not receive those labels, but that is a fairness/correctness concern rather than a circular dependency, so it does not increase the circularity score under the stated rules.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No new entities are introduced. The method relies on existing models and thresholds. Its main external assumptions are the availability of a labeled validation set and the validity of the empirical accuracy targets, which are core to the adaptive smoothing.

assumptions (4)
  • domain assumption The labeled validation set (10% of the labeled training split) accurately reflects the class distribution and accuracy of the training data.
    Used to compute per-bin accuracy targets γ and δ for adaptive label smoothing.
  • domain assumption The OOD detector's OvR outputs are trustworthy enough before warm-up for the loss to improve OOD detection.
    The losses for OOD training and soft consistency are used without a robust alternative.
  • domain assumption The calibration model L = ∑(p_k q_k) captures the joint seen-class likelihood for thresholding.
    The seen score treats the product of classifier and OOD probabilities as the likelihood of belonging to a seen class.
  • domain assumption The proof of Lemma 1 assumes the model is 'well-calibrated' in a bin sense and then uses the error bound η in an algebraic step.
    The proof contains a step (ε ≤ 1 - min{τ1, τ2} + η) that is not algebraically consistent with the expansion (1 - τ1 + η) + (1 - τ2 + η).

how reviews work

0 comments
Cite this review

Pith. "Pith review of CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning." pith.science (2026). https://pith.science/paper/UORNZOTW

@misc{pith2026250800922,
  author       = {Pith},
  title        = {Pith review of: CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UORNZOTW}},
  note         = {Machine review of arXiv:2508.00922}
}
read the original abstract

Semi-supervised learning (SSL) uses unlabeled data to improve the performance of machine learning models when labeled data is scarce. However, its real-world applications often face the label distribution mismatch problem, in which the unlabeled dataset includes instances whose ground-truth labels are absent from the labeled training dataset. Recent studies, referred to as safe SSL, have addressed this issue by using both classification and out-of-distribution (OOD) detection. However, the existing methods may suffer from overconfidence in deep neural networks, leading to increased SSL errors because of high confidence in incorrect pseudo-labels or OOD detection. To address this, we propose a novel method, CaliMatch, which calibrates both the classifier and the OOD detector to foster safe SSL. CaliMatch presents adaptive label smoothing and temperature scaling, which eliminates the need to manually tune the smoothing degree for effective calibration. We give a theoretical justification for why improving the calibration of both the classifier and the OOD detector is crucial in safe SSL. Extensive evaluations on CIFAR-10, CIFAR-100, SVHN, TinyImageNet, and ImageNet demonstrate that CaliMatch outperforms the existing methods in safe SSL tasks.

Figures

Figures reproduced from arXiv: 2508.00922 by the authors.

Figure 1
Figure 1. Learning curves averaged over five runs on CIFAR-100 for CaliMatch and other SSL methods. The shaded region indicates [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Reliability diagrams of OpenMatch and CaliMatch for unseen-label data detection on four datasets. The blue and red bars [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Performance variations in CaliMatch and OpenMatch with threshold [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 42 canonical work pages

  1. [1]

    Safe semi- supervised learning using a bayesian neural network

    Jinsoo Bae, Minjung Lee, and Seoung Bum Kim. Safe semi- supervised learning using a bayesian neural network. Infor- mation Sciences, 612:453–464, 2022. 1

    work page 2022

  2. [2]

    Mixmatch: A holistic approach to semi-supervised learning

    David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, 32, 2019. 2

    work page 2019

  3. [3]

    Cubuk, Alex Ku- rakin, Kihyuk Sohn, Han Zhang, and Colin Raffel

    David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Ku- rakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remix- match: Semi-supervised learning with distribution alignment and augmentation anchoring. In International Conference on Learning Representations, 2020. 2

    work page 2020

  4. [4]

    Semi-supervised learning under class distribution mismatch

    Y anbei Chen, Xiatian Zhu, Wei Li, and Shaogang Gong. Semi-supervised learning under class distribution mismatch. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 3569–3576, 2020. 1

    work page 2020

  5. [5]

    Boosting semi- supervised learning by exploiting all unlabeled data

    Y uhao Chen, Xin Tan, Borui Zhao, Zhaowei Chen, Ren- jie Song, Jiajun Liang, and Xuequan Lu. Boosting semi- supervised learning by exploiting all unlabeled data. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7548–7557, 2023. 1

    work page 2023

  6. [6]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 5

    2009

  7. [7]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Y u Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321–1330. PMLR,

    work page

  8. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

Show all 47 references
  1. [9]

    Safe-student for safe deep semi-supervised learn- ing with unseen-class unlabeled data

    Rundong He, Zhongyi Han, Xiankai Lu, and Yilong Yin. Safe-student for safe deep semi-supervised learn- ing with unseen-class unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14585–14594, 2022. 1, 2, 5

    2022

  2. [10]

    Semi-supervised learning with deep gen- erative models

    Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep gen- erative models. Advances in Neural Information Processing Systems, 27, 2014. 1

    2014

  3. [11]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 5

    2009

  4. [12]

    Tiny imagenet visual recognition challenge

    Y a Le and Xuan Y ang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. 5

    2015

  5. [13]

    Pseudo-label: The simple and effi- cient semi-supervised learning method for deep neural net- works

    Dong-Hyun Lee et al. Pseudo-label: The simple and effi- cient semi-supervised learning method for deep neural net- works. In Workshop on Challenges in Representation Learn- ing, ICML, page 896. Atlanta, 2013. 2

    2013

  6. [14]

    Iomatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization

    Zekun Li, Lei Qi, Yinghuan Shi, and Y ang Gao. Iomatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15870–15879, 2023. 1, 2, 5

    2023

  7. [15]

    Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the re- liability of out-of-distribution image detection in neural net- works. In International Conference on Learning Represen- tations, 2018. 2

    2018

  8. [16]

    The devil is in the margin: Margin-based label smooth- ing for network calibration

    Bingyuan Liu, Ismail Ben Ayed, Adrian Galdran, and Jose Dolz. The devil is in the margin: Margin-based label smooth- ing for network calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 80–88, 2022. 1

    2022

  9. [17]

    Revisiting the calibration of modern neu- ral networks

    Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neu- ral networks. Advances in Neural Information Processing Systems, 34:15682–15694, 2021. 1

    2021

  10. [18]

    When does label smoothing help? Advances in Neural In- formation Processing Systems, 32, 2019

    Rafael M ¨uller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Advances in Neural In- formation Processing Systems, 32, 2019. 2

    2019

  11. [19]

    Obtaining well-calibrated probabilities using bayesian binning

    Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well-calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence , 2015. 5

    2015

  12. [20]

    Reading digits in natural images with unsupervised feature learning

    Y uval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learn- ing, page 7. Granada, Spain, 2011. 5

    2011

  13. [21]

    Predicting good probabilities with supervised learning

    Alexandru Niculescu-Mizil and Rich Caruana. Predicting good probabilities with supervised learning. In International Conference on Learning Representations , pages 625–632,

  14. [22]

    Rankmixup: Ranking-based mixup training for net- work calibration

    Jongyoun Noh, Hyekang Park, Junghyup Lee, and Bumsub Ham. Rankmixup: Ranking-based mixup training for net- work calibration. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 1358–1368, 2023. 1

    2023

  15. [23]

    Realistic evaluation of deep semi-supervised learning algorithms

    Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Do- gus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in Neural In- formation Processing Systems, 31, 2018. 5

    2018

  16. [24]

    Open- match: Open-set semi-supervised learning with open-set consistency regularization

    Kuniaki Saito, Donghyun Kim, and Kate Saenko. Open- match: Open-set semi-supervised learning with open-set consistency regularization. Advances in Neural Information Processing Systems, 34:25956–25967, 2021. 1, 3, 4, 5

    2021

  17. [25]

    Flexible distribu- tion alignment: Towards long-tailed semi-supervised learn- 9 ing with proper calibration

    Emanuel Sanchez Aimar, Nathaniel Helgesen, Y onghao Xu, Marco Kuhlmann, and Michael Felsberg. Flexible distribu- tion alignment: Towards long-tailed semi-supervised learn- 9 ing with proper calibration. In European Conference on Computer Vision, pages 307–327. Springer, 2024. 2, 5

    2024

  18. [26]

    Fixmatch: Simpli- fying semi-supervised learning with consistency and confi- dence

    Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simpli- fying semi-supervised learning with consistency and confi- dence. Advances in Neural Information Processing Systems , ...

    2020

  19. [27]

    On mixup train- ing: Improved calibration and predictive uncertainty for deep neural networks

    Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup train- ing: Improved calibration and predictive uncertainty for deep neural networks. Advances in Neural Information Process- ing Systems, 32, 2019. 2

    2019

  20. [28]

    Post-hoc uncer- tainty calibration for domain drift scenarios

    Christian Tomani, Sebastian Gruber, Muhammed Ebrar Er- dem, Daniel Cremers, and Florian Buettner. Post-hoc uncer- tainty calibration for domain drift scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 10124–10132, 2021. 1

    2021

  21. [29]

    Scomatch: Alleviating overtrusting in open-set semi-supervised learning

    Zerun Wang, Liuyu Xiang, Lang Huang, Jiafeng Mao, Ling Xiao, and Toshihiko Y amasaki. Scomatch: Alleviating overtrusting in open-set semi-supervised learning. In Eu- ropean Conference on Computer Vision , pages 217–233. Springer, 2024. 2, 5

    2024

  22. [30]

    Towards realistic long-tailed semi- supervised learning: Consistency is all you need

    Tong Wei and Kai Gan. Towards realistic long-tailed semi- supervised learning: Consistency is all you need. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3469–3478, 2023. 1, 2

    2023

  23. [31]

    Vime: Extending the success of self-and semi-supervised learning to tabular domain

    Jinsung Y oon, Y ao Zhang, James Jordon, and Mihaela V an der Schaar. Vime: Extending the success of self-and semi-supervised learning to tabular domain. Advances in Neural Information Processing Systems , 33:11033–11043, 2020. 1

    2020

  24. [32]

    Multi- task curriculum framework for open-set semi-supervised learning

    Qing Y u, Daiki Ikami, Go Irie, and Kiyoharu Aizawa. Multi- task curriculum framework for open-set semi-supervised learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XII 16, pages 438–454. Springer, 2020. 1, 2, 5

    2020

  25. [33]

    Wide residual net- works

    Sergey Zagoruyko and Nikos Komodakis. Wide residual net- works. Proceedings of the British Machine Vision Confer- ence, pages 87.1–87.12, 2016. 5 10 CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning Supplementary Material S-1. Detailed Experiment Setu...

    2016

  26. [34]

    On the con- vergence of sgd with biased gradients

    Ahmad Ajalloeian and Sebastian U Stich. On the con- vergence of sgd with biased gradients. arXiv preprint arXiv:2008.00051, 2020. 9

    2008 arXiv

  27. [35]

    Randaugment: Practical automated data augmen- tation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmen- tation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020. 1

    2020

  28. [36]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1

    2016

  29. [37]

    Densely connected convolutional net- works

    Gao Huang, Zhuang Liu, Laurens V an Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4700–4708, 2017. 6

    2017

  30. [38]

    Iomatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization

    Zekun Li, Lei Qi, Yinghuan Shi, and Y ang Gao. Iomatch: Simplifying open-set semi-supervised learning with joint inliers and outliers utilization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15870–15879, 2023. 1

    2023

  31. [39]

    The devil is in the margin: Margin-based label smooth- ing for network calibration

    Bingyuan Liu, Ismail Ben Ayed, Adrian Galdran, and Jose Dolz. The devil is in the margin: Margin-based label smooth- ing for network calibration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 80–88, 2022. 2

    2022

  32. [40]

    Rankmixup: Ranking-based mixup training for net- work calibration

    Jongyoun Noh, Hyekang Park, Junghyup Lee, and Bumsub Ham. Rankmixup: Ranking-based mixup training for net- work calibration. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 1358–1368, 2023. 2

    2023

  33. [41]

    Realistic evaluation of deep semi-supervised learning algorithms

    Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Do- gus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in Neural In- formation Processing Systems, 31, 2018. 1

    2018

  34. [42]

    Open- match: Open-set semi-supervised learning with open-set consistency regularization

    Kuniaki Saito, Donghyun Kim, and Kate Saenko. Open- match: Open-set semi-supervised learning with open-set consistency regularization. Advances in Neural Information Processing Systems, 34:25956–25967, 2021. 1

    2021

  35. [43]

    Flexible distribu- tion alignment: Towards long-tailed semi-supervised learn- ing with proper calibration

    Emanuel Sanchez Aimar, Nathaniel Helgesen, Y onghao Xu, Marco Kuhlmann, and Michael Felsberg. Flexible distribu- tion alignment: Towards long-tailed semi-supervised learn- ing with proper calibration. In European Conference on Computer Vision, pages 307–327. Springer, 2024. 1

    2024

  36. [44]

    Fixmatch: Simpli- fying semi-supervised learning with consistency and confi- dence

    Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simpli- fying semi-supervised learning with consistency and confi- dence. Advances in Neural Information Processing Systems , ...

    2020

  37. [45]

    Scomatch: Alleviating overtrusting in open-set semi-supervised learning

    Zerun Wang, Liuyu Xiang, Lang Huang, Jiafeng Mao, Ling Xiao, and Toshihiko Y amasaki. Scomatch: Alleviating overtrusting in open-set semi-supervised learning. In Eu- ropean Conference on Computer Vision , pages 217–233. Springer, 2024. 1

    2024

  38. [46]

    Multi- task curriculum framework for open-set semi-supervised learning

    Qing Y u, Daiki Ikami, Go Irie, and Kiyoharu Aizawa. Multi- task curriculum framework for open-set semi-supervised learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XII 16, pages 438–454. Springer, 2020. 1

    2020

  39. [47]

    Wide residual net- works

    Sergey Zagoruyko and Nikos Komodakis. Wide residual net- works. Proceedings of the British Machine Vision Confer- ence, pages 87.1–87.12, 2016. 3 11

    2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.