{"id":"b6cc9f4a-e788-4cc5-9fcc-0545fd4ac918","arxiv_id":"2506.00467","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"SST uses per-class thresholds updated once per cycle to pick pseudo-labels, reporting 84.9% ImageNet top-1 accuracy with 10% labeled data on a huge ViT.","lead":"Semi-supervised learning trains a model on a few labeled images plus many unlabeled ones; this paper proposes a thresholding rule that decides which model-made labels to trust and updates the rule only a few times. The authors report 84.9% ImageNet top-1 accuracy using 10% labeled data, matching a fully supervised model that used all labels.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The '10% labels beats 100% fully supervised' claim is confounded by pretraining: Semi-SST uses MAE/DINO-initialized ViT-Huge, while DeiT-III is trained from scratch, so the comparison does not isolate SAT.","rationale":"The paper does provide useful internal evidence: Figure 3 shows SAT against fixed-threshold variants, Table 5 reports multiple runs with standard deviations on classical benchmarks, and a Friedman test is included. Those results suggest the framework may have real empirical value. However, the strongest advertised claim, that 10% labeled data beats 100% fully supervised DeiT-III, is precisely the part that is not supported by a controlled experiment. All transformer results use DINO/MAE pretrained weights, while the DeiT-III baseline is a from-scratch supervised recipe, so the comparison conflates self-supervised pretraining and training pipeline with the proposed SAT mechanism. The controlled gains over Semi-ViT are small (0.6-0.7 points on ViT-Huge) and lack error bars, making the headline SOTA claim fragile. The reader's weakest assumption about pseudo-label quality is real, but Figure 3 actually shows the method continuing to improve despite declining selected-pseudo-label accuracy, so that concern is less decisive. The uncontrolled fully supervised comparison is the most load-bearing because if it lands, the abstract's marquee result is not valid as stated. A matched fully supervised run with the same pretrained checkpoint and recipe would settle it directly.","tokens_in":25435,"tokens_out":7206,"duration_ms":74295,"concrete_test":"Train the same MAE-pretrained ViT-Huge checkpoint fully supervised on 100% ImageNet-1K using the paper's own training recipe (Table 10 supervised settings, mixup/cutmix, 50 epochs), and compare its Top-1 accuracy to Semi-SST's 84.9% on 10% labels. If this fully supervised run also reaches or exceeds 84.9%, the paper's central 'tenfold annotation saving' claim is an artifact of pretraining and recipe, not of SAT; if it falls clearly below 84.9%, the claim survives this particular confound.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim in the abstract and in Section 4.4 is that Semi-SST-ViT-Huge reaches 84.9% with 10% ImageNet-1K labels, surpassing the 84.8% of fully supervised DeiT-III-ViT-Huge and yielding a tenfold annotation saving. This comparison is not controlled. Section 4.1 states that all transformer experiments start from DINO or MAE pretrained weights ('self-supervised pretraining is optional in our methods'), whereas the DeiT-III baseline is a from-scratch supervised training recipe. The 0.1-point margin could therefore be entirely due to MAE initialization and the SST/Semi-ViT training recipe (mixup, cutmix, EMA teacher) rather than to SAT's adaptive thresholds. The paper's own controlled comparison in Table 3 shows Semi-SST-ViT-Huge exceeds Semi-ViT-ViT-Huge by only 0.7/0.6 points on 1%/10% labels, with no reported error bars for ImageNet, so the headline 'best results' rests on a small, uncontrolled difference. To support the tenfold-reduction claim, the authors must compare against a fully supervised model trained on 100% labels with the same pretrained initialization and the same augmentation/optimizer recipe.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Self-training with Self-adaptive Thresholding (SST), a semi-supervised learning framework with two variants: Super-SST, which repeatedly selects high-confidence pseudo-labels via class-specific thresholds and retrains on the expanded labeled set, and Semi-SST, which augments the offline pipeline with an EMA-teacher online pseudo-labeling component. The core contribution is the Self-Adaptive Thresholding (SAT) mechanism, which computes class-specific thresholds once per cycle from the model's own confidence distribution using a cutoff C and a scaling factor S. Experiments are reported on ImageNet-1K, CIFAR-10/100, SVHN, STL-10, Food-101, iNaturalist, and Clothing-1M, with claims of state-of-the-art performance, including 80.7%/84.9% Top-1 accuracy on ImageNet-1K with 1%/10% labels and a claim that 10%-label Semi-SST-ViT-Huge surpasses fully-supervised DeiT-III-ViT-Huge on 100% labels.","tokens_in":25773,"tokens_out":6266,"duration_ms":51605,"significance":"The paper makes a useful methodological contribution in simplifying adaptive thresholding: updating class-specific thresholds once per training cycle from a converged model is an elegant way to avoid the per-iteration overhead of FlexMatch and FreeMatch, and the pseudo-code and architecture-agnostic framework make the method easy to reproduce. The controlled comparisons in Table 3, where the same pretrained backbone is used for baseline and proposed method, show consistent gains on CIFAR-100, Food-101, iNaturalist, and ImageNet-1K, which is evidence that SST can improve over Semi-ViT in a matched setting. The efficiency claims (3 vs. over 1,000,000 threshold updates) are plausible and well demonstrated in Table 7. However, the paper's headline claims — 'best results on competitive ImageNet-1K SSL benchmarks' and 'tenfold reduction in human annotation costs' — are not supported by the evidence as presented, because the key comparisons are uncontrolled, the hyperparameters C and S are tuned on the same benchmark used for the state-of-the-art claim, and the multi-benchmark comparison in Table 5 is confounded by architecture differences.","major_comments":[{"comment":"The claim that Semi-SST-ViT-Huge achieves 84.9% with 10% labels and surpasses fully-supervised DeiT-III's 84.8% is an uncontrolled comparison, since Section 4.1 states that all transformer experiments start from DINO or MAE pretrained weights, while DeiT-III is a from-scratch training recipe; the controlled comparison in Table 3 shows only a 0.6-point gain over Semi-ViT-ViT-Huge with no error bars. To support the tenfold-annotation-saving claim, the authors must either train a fully supervised baseline with the same pretrained initialization and augmentation recipe, or substantially soften the claim.","section":"Section 4.4, Table 6, Abstract"},{"comment":"The cutoff C and scaling factor S are selected by ablation on the same ImageNet-1K benchmark (1% labeled data) and then used to produce the headline 1%/10% results, with no held-out validation split mentioned; this selection on the target benchmark can inflate the reported gains. Please report the chosen hyperparameters' performance on a validation split or demonstrate that the results are stable across a range of C and S on all datasets, not just ImageNet-1K.","section":"Section 4.7, Figure 2"},{"comment":"The Friedman test and mean-error-rate ranking are confounded by the architecture mismatch: the proposed methods use a DINO-pretrained ViT-Small while all baselines are WRN/ResNet models trained from scratch. The SVHN 40-label results (Super-SST 29.41%, Semi-SST 23.18%) are orders of magnitude worse than the baselines (e.g., FixMatch 3.37%), and the paper's own limitation section acknowledges this; as a result, the 'best overall' claim is not supported by a fair comparison.","section":"Table 5 and Section 4.3"},{"comment":"The paper treats SAT-selected pseudo-labels as 'equally reliable as human labels' when forming the combined training set, yet Figure 3 shows the accuracy of selected pseudo-labels declining from 89.5% to 77.7% across cycles; the manuscript provides no analysis of how error accumulation is controlled, and Section 5.1's promise of a 'detailed theoretical analysis' is not fulfilled anywhere in the paper. Please either provide such analysis or rephrase the confirmation-bias claims to be explicitly empirical.","section":"Section 3, Eq. (9) and Figure 3"}],"minor_comments":[{"comment":"The introduction states that on CIFAR-100 with 400/2500/10000 labels, FreeMatch attains a mean error rate of 28.71% using about 240 GPU hours, but Table 5 shows FreeMatch's CIFAR-100 mean is approximately 29.55%, and Table 8 reports FreeMatch's overall mean error rate as 12.51%; these numbers should be reconciled.","section":"Section 1, Table 8"},{"comment":"The claim of 'Detailed theoretical analysis' is not backed by any theoretical section; either remove the phrase or add the analysis.","section":"Section 5.1"},{"comment":"The pseudo-code for Semi-SST is incomplete: it shows model.train(combined_labeled_data) but does not show the teacher model, weak/strong augmentations, or the online pseudo-labeling loss; please align Algorithm 1 with the description in Section 3 and Figure 1b.","section":"Algorithm 1, Section 3"},{"comment":"The hyperparameters for Super-SST and Semi-SST differ in multiple respects (weight decay, drop path, layer-wise decay rate, batch size), so the relative improvement of Semi-SST over Super-SST in Table 3 is not a clean ablation of the online pseudo-labeling component.","section":"Table 10, Section 4.2"},{"comment":"The axis labels in Figure 2 are garbled (e.g., '6 0.36 6.36 8.66 9.57 0.07 0.37 0.4T'), likely a rendering artifact; please regenerate the figures with correct labels.","section":"Figures 2 and 3"}],"recommendation":"major_revision","confidential_remarks":"The paper would be substantially strengthened by recalibrating the abstract and conclusions to the controlled comparisons. In particular, the DeiT-III comparison in Table 6 should be replaced or explicitly labeled as not apples-to-apples. The SVHN 40-label failures, while acknowledged in the limitation section, are severe enough that they should be discussed in the abstract or conclusion. The authors might also consider including error bars for the ImageNet results and a validation-based hyperparameter selection to preempt reviewer concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SST is a simple, sensible variant of adaptive-threshold pseudo-labeling that updates class-specific thresholds once per self-training cycle rather than every iteration. That per-cycle update plus the cutoff-average-scaling formula is genuinely not in FlexMatch or FreeMatch. The paper does a lot of empirical work, and the efficiency gains over FreeMatch look real (a handful of threshold updates vs a million). The method also consistently improves over Semi-ViT on several datasets, which is credible.\n\nBut the headline claim in the abstract and Section 4.4, that Semi-SST-ViT-Huge at 84.9% with 10% labels beats fully-supervised DeiT-III at 84.8% with 100% labels, is not a controlled comparison. All transformer experiments start from DINO or MAE pretrained weights, whereas DeiT-III is trained from scratch. The 0.1-point margin could easily come from initialization or the EMA-teacher, mixup, and cutmix recipe rather than SAT. The paper's own controlled comparison in Table 3 shows only 0.6 to 0.7 points over Semi-ViT, with no error bars on ImageNet. So the 'tenfold annotation reduction' claim outruns the evidence.\n\nOther soft spots: C and S are tuned on ImageNet-1K, the same benchmark used for the SOTA claim; Table 5's architecture mixture (ViT-Small vs WRN) makes the Friedman ranking hard to interpret; and SVHN with 40 labels gives 23-29% error, which the authors honestly flag. Figure 3 also shows pseudo-label accuracy declining from 89.5% to 77.7% across cycles, which cuts against the 'high-quality' language.\n\nNone of this kills the paper. The core method is coherent, the efficiency argument is meaningful, and the empirical scope is broad. But the public framing oversells the ImageNet result. A serious referee should ask for error bars on large-scale numbers, a fair fully-supervised baseline with the same pretrained initialization, and code or at least detailed hyperparameter and selection logs.\n\nWho this is for: SSL practitioners looking for a cheap way to do adaptive-threshold self-training. It deserves a real peer review with revisions, not a desk reject.","headline":"Per-cycle adaptive thresholding is a clean efficiency idea, but the '10% labels beats 100% labels' headline rests on an uncontrolled pretrained-vs-from-scratch comparison.","tokens_in":26258,"tokens_out":3433,"would_cite":false,"duration_ms":28637,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Self-adaptive pseudo-label thresholds let a ViT-Huge trained on 10% of ImageNet's labels reach 84.9% Top-1 accuracy, beating the fully supervised 84.8% that uses all the labels.","keywords":["semi-supervised learning","self-training","pseudo-labeling","self-adaptive thresholding","confirmation bias","ImageNet-1K","vision transformer","EMA-teacher"],"falsifier":"Freeze the class thresholds at the values SAT derives from the first converged checkpoint and run the rest of the Super-SST loop on ImageNet-1K with 1% labels without re-estimating them; if this frozen-threshold run reaches the same 70.4% Top-1 accuracy as the full algorithm, the cycle-by-cycle re-estimation that is the paper's core mechanism is not what carries the result, while a clear shortfall would confirm that tracking learning progress per cycle does real work. A companion check follows from the paper's own numbers: accepted-pseudo-label accuracy slides from 89.5% to 77.7% over cycles, so extending the run until that accuracy falls below the accuracy of the original labeled set should, by the paper's account, eventually produce a peak followed by a decline.","tokens_in":25271,"feed_emoji":"🎯","tokens_out":15029,"duration_ms":123978,"temperature":0.7,"pith_summary":"The paper proposes SST, a semi-supervised learning framework built around a rule for deciding which automatically generated pseudo-labels to trust. That rule, Self-Adaptive Thresholding (SAT), derives a separate acceptance threshold for each class from the model's own confidence statistics and re-derives it only once per training cycle, when the model has converged. The paper's claim is that this selects pseudo-labels that are both accurate enough and plentiful enough that retraining on them rivals training on human labels, cutting annotation cost roughly tenfold. Support for the claim is the reported 84.9% Top-1 accuracy on ImageNet-1K with 10% labeled data and 80.7% with 1%, where the fully supervised DeiT-III-ViT-Huge achieves 84.8% with 100%.","feed_headline":"84.9% top-1 with 10% of ImageNet labels, matching full supervision","feed_subtitle":"Re-estimating class-wise confidence thresholds just a few times lets unlabeled data replace nine of ten human labels.","key_machinery":"The load-bearing object is SAT, Self-Adaptive Thresholding, the mechanism that turns the model's current confidence statistics into a per-class acceptance threshold. For each class it sorts the predicted probabilities over the unlabeled pool, removes every probability at or below a cutoff $C$ (0.5 by default), averages the remaining probabilities, and multiplies by a scaling factor $S$ (about 0.8-0.9) to obtain that class's threshold $\\tau_j = S\\,\\bar{p}_j$; a pseudo-label is accepted when its confidence exceeds the threshold of its predicted class. SAT sits inside a cyclic self-training loop: predict with the converged model, apply SAT, retrain on the original labeled data plus the accepted pseudo-labels, repeat. In Semi-SST the same thresholds also gate the online pseudo-labels produced by an EMA-Teacher. The defining design choice is that thresholds are updated only a handful of times per run (three updates on ImageNet-1K), whereas competing adaptive methods such as FlexMatch and FreeMatch update every iteration, which the paper argues is computationally wasteful and feeds confirmation bias because their thresholds are estimated from models still in training.","core_discovery":"The central claim is that the quantity-quality trade-off in pseudo-labeling is best managed by class-specific thresholds computed from the model's own confidence distribution and refreshed only when the model has converged, not at every training step. After each cycle, SAT sorts the softmax probabilities for each class, discards probabilities at or below a cutoff $C$, averages the survivors, and scales that average by a factor $S$ to set the class threshold $\\tau_j = S\\,\\bar{p}_j$; an unlabeled sample is admitted to the training set when its top predicted probability exceeds the threshold of its predicted class. Classes the model has actually learned well therefore contribute many pseudo-labels, while uncertain classes contribute few, and because thresholds are recomputed from a well-trained model rather than a mid-training one, the paper argues that inaccurate pseudo-labels and the confirmation bias they reinforce are suppressed. The framework comes in two versions, Super-SST (offline SAT selection plus plain supervised training on the combined set) and Semi-SST (the same offline loop plus an EMA-Teacher, a teacher model updated as an exponential moving average of the student, that generates online pseudo-labels also gated by SAT), and the paper reports leading results with both: Semi-SST-ViT-Huge achieves 80.7% and 84.9% Top-1 accuracy on ImageNet-1K with 1% and 10% labeled data, against 84.8% for the fully supervised DeiT-III-ViT-Huge.","pith_inferences":["A testable extension the paper does not pursue: on very small labeled sets, which the paper's own limitation section flags with weak results on SVHN using only 40 labels, per-class confidence statistics are noisy, so a variance-aware correction that widens thresholds when a class has few confident predictions could extend SAT's gains into ultra-low-label regimes.","The paper's Figure 3 shows accepted-pseudo-label accuracy declining from 89.5% to 77.7% across cycles, so the method's ceiling is plausibly set by how long model improvement outruns the degradation of its own output; imposing a floor so that accepted pseudo-labels never drop below the accuracy of the original labeled set is a natural, untested variant.","The classical-benchmark comparison (Table 5) rests on a DINO-pretrained ViT-Small, whereas the prior methods it is ranked against typically use Wide ResNets, a mismatch the paper itself acknowledges; a same-architecture head-to-head would be the cleanest confirmation that SAT, rather than the backbone, drives the gains.","Because SAT needs only per-class confidence statistics and a converged checkpoint, the same machinery should transfer to object detection, segmentation, and NLP self-training, which the paper names as future work rather than demonstrated results."],"forward_implications":["Annotation cost falls by roughly tenfold for image classification: a ViT-Huge trained on 10% of ImageNet labels reaches 84.9% Top-1, slightly above the 84.8% of the same model family trained on all labels.","Thresholding becomes nearly free: Super-SST needs about three threshold updates per run versus over one million for FlexMatch and FreeMatch, while reporting higher accuracy on the 100K-label ImageNet protocol (64.42% versus 58.15% and 59.43%).","Gains concentrate where labels are scarcest: relative improvements over supervised and semi-supervised baselines are 0.7%-20.7% at 1% labels and 0.2%-12.1% at 10%, across ImageNet-1K, CIFAR-100, Food-101, and iNaturalist.","The gains survive distillation and imbalance: a ViT-Small distilled from a Super-SST ViT-Huge teacher reaches 80.3% Top-1 at 10% labels, above the fully supervised DeiT-ViT-Small's 79.8%, and Super-SST reaches 75.69% on Clothing-1M with 5% clean labels against a 78.99% fully supervised upper bound."],"supporting_citations":[{"why":"Semi-ViT supplies the EMA-Teacher architecture, the training protocol, and the pretrained checkpoints SST builds on, and is the principal semi-supervised baseline.","marker":"[35]"},{"why":"Noisy Student establishes the iterative self-training-with-pseudo-labels paradigm at ImageNet scale that Super-SST extends with adaptive thresholds.","marker":"[27]"},{"why":"FlexMatch is the curriculum-thresholding competitor whose per-iteration threshold updates SAT is explicitly contrasted with in cost and accuracy.","marker":"[31]"},{"why":"FreeMatch is the self-adaptive-thresholding competitor that supplies the comparison protocol and the mean error rates Super-SST and Semi-SST improve upon.","marker":"[36]"},{"why":"FixMatch is the fixed-threshold pseudo-labeling baseline used as the efficiency and accuracy reference in the ImageNet-1K comparison.","marker":"[6]"},{"why":"Arazo et al. supplies the confirmation-bias account of pseudo-labeling errors that motivates designing thresholds to exclude low-confidence predictions.","marker":"[25]"},{"why":"Pseudo-Label provides the original hard pseudo-label self-training technique that SAT's selection step replaces with class-specific thresholds.","marker":"[22]"},{"why":"DeiT-III supplies the fully supervised ViT-Huge result (84.8% Top-1) against which the paper's 84.9%-at-10%-labels headline claim is measured.","marker":"[74]"}],"fun_headline_variants":["Class-wise thresholds adapt: SSL hits 84.9% with 10% labels","Adaptive pseudo-label thresholds match full supervision","SST: Self-adaptive thresholds cut labeled data need tenfold","Self-adaptive thresholding boosts semi-supervised learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that any unlabeled image whose top predicted score clears its class threshold is as trustworthy as a human label and can be retrained on with plain cross-entropy, even though the paper's own measurements show the accuracy of accepted pseudo-labels falling from 89.5% to 77.7% across training cycles.","fun_headline_variants_meta":{"raw":{"variants":["Class-wise thresholds adapt: SSL hits 84.9% with 10% labels","Adaptive pseudo-label thresholds match full supervision","SST: Self-adaptive thresholds cut labeled data need tenfold","Self-adaptive thresholding boosts semi-supervised learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000446,"raw_usage":{"total_tokens":2372,"prompt_tokens":1184,"completion_tokens":1188,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":800,"completion_tokens_details":{"reasoning_tokens":1126}},"tokens_in":800,"tokens_out":1188,"duration_ms":8917,"temperature":1.0,"reasoning_tokens":1126,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:04:23.274886+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Freeze the class thresholds at the values SAT derives from the first converged checkpoint and run the rest of the Super-SST loop on ImageNet-1K with 1% labels without re-estimating them; if this frozen-threshold run reaches the same 70.4% Top-1 accuracy as the full algorithm, the cycle-by-cycle re-estimation that is the paper's core mechanism is not what carries the result, while a clear shortfall would confirm that tracking learning progress per cycle does real work. A companion check follows from the paper's own numbers: accepted-pseudo-label accuracy slides from 89.5% to 77.7% over cycles, so extending the run until that accuracy falls below the accuracy of the original labeled set should, by the paper's account, eventually produce a peak followed by a decline.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Semi-ViT supplies the EMA-Teacher architecture, the training protocol, and the pretrained checkpoints SST builds on, and is the principal semi-supervised baseline."},{"cited_title":"Xie, M.-T","cited_arxiv_id":null,"evidence_quote":"Noisy Student establishes the iterative self-training-with-pseudo-labels paradigm at ImageNet scale that Super-SST extends with adaptive thresholds."},{"cited_title":"Zhang, Y","cited_arxiv_id":null,"evidence_quote":"FlexMatch is the curriculum-thresholding competitor whose per-iteration threshold updates SAT is explicitly contrasted with in cost and accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FreeMatch is the self-adaptive-thresholding competitor that supplies the comparison protocol and the mean error rates Super-SST and Semi-SST improve upon."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FixMatch is the fixed-threshold pseudo-labeling baseline used as the efficiency and accuracy reference in the ImageNet-1K comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Pseudo-Label provides the original hard pseudo-label self-training technique that SAT's selection step replaces with class-specific thresholds."},{"cited_title":"Touvron, M","cited_arxiv_id":null,"evidence_quote":"DeiT-III supplies the fully supervised ViT-Huge result (84.8% Top-1) against which the paper's 84.9%-at-10%-labels headline claim is measured."}],"review_version":1}