{"id":"7218982f-af6f-4b76-8372-5a883a69aeb5","arxiv_id":"2507.22075","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A prototype- and neighbor-aware pseudo-labeling framework that reports improved unsupervised CLIP adaptation accuracy on 11 image classification benchmarks.","lead":"ALPHA is a new learning scheme that adapts CLIP to unlabeled image datasets by trusting images that sit close to their class prototype and to neighboring images. On 11 benchmarks it reports the best average accuracy among the tested methods, though the gains over the previous DPA method are a few points and no code is released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PICS filter (Eq. 15) is self-referential and can confirm biased pseudo-labels; the paper's own limitation admits informative samples are discarded, so the mechanism behind SOTA gains is unproven.","rationale":"I focused on the reader's weakest assumption and found it is indeed load-bearing. The proposed PICS filter is the first of two key components, and the paper's own ablation (Table 2) shows that removing PICS reduces average accuracy (ALPHA w/o PICS 74.94 vs ALPHA-CS 78.41), so the central claim depends on PICS actually selecting cleaner pseudo-labels. However, the criterion is computed from structures derived from the very pseudo-labels it is meant to validate. This circularity is not merely conceptual: if a class is systematically misclassified, both the prototype mu_c and the cross-class set O are wrong, and a wrong label can satisfy Eq. 15. The paper's stated limitation in the Conclusion is an admission that the geometric assumption fails in some regimes. The absence of a direct measurement of filtered-set precision leaves the mechanism unsupported. I also considered the internal contradiction about discarding the text encoder while using it in Eq. 17; this is a reproducibility flaw but can be resolved by precomputing text embeddings, so I did not treat it as the primary concern. The lack of error bars and possible test-set hyperparameter choice are secondary. A precision/recall diagnostic on PICS-selected samples would settle whether the filter is doing what the central claim requires. If the diagnostic shows no improvement in precision, the paper's explanation of its own gains is wrong, even if the average accuracy numbers are reproducible. Therefore I recommend keeping the CONDITIONAL verdict: accept only if the authors provide this diagnostic (or equivalent evidence) and release code to resolve the implementation ambiguity.","tokens_in":15685,"tokens_out":7498,"duration_ms":80109,"concrete_test":"Use ground-truth labels (available in benchmarks) to measure the precision of the PICS-selected clean set at epochs 1, 2, and 3 on EuroSAT and DTD with CLIP ViT-B/32, following Eqs. 7-15. Compare against (a) precision of all zero-shot pseudo-labels and (b) a FixMatch confidence>0.95 filter. If precision(PICS) is not significantly above precision(all) in early epochs, Eq. 15 is not selecting more accurate pseudo-labels and the SOTA gain cannot be attributed to PICS. Also compute recall: if PICS discards a large fraction of correct samples, the 'more accurate pseudo-labels' claim is incomplete. This test is diagnostic only; labels are not needed at deployment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PICS identifies clean pseudo-labels via in-class compactness versus cross-class separation. But Eq. 15's criterion phi(x_i) > psi(x_i) is computed entirely from pseudo-label-derived structures: phi uses prototypes mu_c from Eq. 7, built from pseudo-labels and confidence scores; psi uses cross-class sets O from Eqs. 10-13, also built from pseudo-labels. If zero-shot CLIP is biased (as the paper argues in Sec. 1), the prototypes and cross-class samples are biased in the same direction, so a confidently wrong pseudo-label can satisfy the inequality. The feedback loop in Eq. 22, which inserts NALR-refined labels into the memory bank, can then propagate early filtering errors into later prototypes. The authors concede in the Conclusion that PICS may filter out informative samples in datasets with high intra-class variability; Table 2 indeed shows ALPHA-CS w/o NALR slightly drops on OxfordPets and Caltech101. Crucially, Figure 4 reports pseudo-label accuracy on the full training set, not on the PICS-selected subset, so the paper never demonstrates that Eq. 15's selected samples are actually cleaner than the raw pseudo-labels. Without that evidence, the method's gains could come from NALR or the fairness regularizer rather than from the claimed filtering mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ALPHA, an unsupervised adaptation framework for CLIP that combines two pseudo-labeling components: PICS, which filters pseudo-labels by comparing in-class feature compactness with cross-class feature separation, and NALR, which refines noisy pseudo-labels using LLM-generated class descriptions and neighborhood consistency, together with an adaptive weighting scheme. The method is evaluated on 11 image classification datasets with ViT-B/32 and ViT-B/16 backbones, and is compared against zero-shot CLIP, CuPL, UPL, POUF, LaFTer, and DPA. The authors report consistent gains across three variants of the cross-class set construction (CS, RS, FS), achieving an average accuracy of 74.86% with ALPHA-CS on ViT-B/32, which is 1.42 points above DPA. Additional experiments cover component ablations, scalability, transductive settings, parameter-efficient fine-tuning strategies, robustness to injected noise, and hyperparameter sensitivity.","tokens_in":15925,"tokens_out":4536,"duration_ms":45244,"significance":"If the reported results are reliable, ALPHA would be a useful empirical contribution to unsupervised CLIP adaptation, with a filtering criterion that is more geometric than fixed confidence thresholds and a refinement module that exploits both visual and textual neighborhood structure. The paper has several strengths: it evaluates three internally consistent variants across 11 diverse datasets, includes a transductive comparison, reports scalability and noise-robustness experiments, and is unusually candid about the PICS limitation concerning high intra-class variability. However, the central SOTA claim is currently supported only by single-run accuracy numbers, and the paper does not directly demonstrate that the PICS-selected samples are actually cleaner than the raw pseudo-labels. Given that the average margin over DPA is small and per-dataset results are mixed, the missing statistical validation and the lack of direct evidence for the filtering mechanism are load-bearing issues that need to be addressed before the empirical claims can be fully accepted.","major_comments":[{"comment":"The SOTA claim rests on single-run accuracies without error bars, random seeds, or significance tests. The average margin over DPA is 1.42 points, and on several datasets ALPHA-CS is below DPA (Caltech101 94.94 vs 95.94; EuroSAT 78.18 vs 79.94; CUB 56.54 vs 56.70). With no variance estimates, these differences are not distinguishable from training stochasticity. Please report mean and standard deviation over at least three seeds for the main comparison, and provide a paired test or per-dataset effect sizes to support the claim that ALPHA outperforms DPA and other baselines.","section":"Section 4.1, Table 1"},{"comment":"The PICS clean/noisy decision is self-referential: phi(x_i) is computed with prototypes mu_c from Eq. (7), which are built from pseudo-labels and confidence scores, and psi(x_i) is computed with cross-class sets O from Eqs. (10)-(13), which are also built from pseudo-labels. If zero-shot CLIP is biased, a confidently wrong pseudo-label can satisfy phi(x_i) > psi(x_i), so the filter can confirm the same bias it is meant to remove. The paper does not provide direct evidence that the selected subset is cleaner: Figure 4 reports pseudo-label accuracy on the full training set, not on the PICS-selected subset. Please report the accuracy of accepted versus rejected samples during training, or compare the PICS score with oracle label correctness on a held-out split, so that the filter's precision is established.","section":"Section 3.3, Eq. (15)"},{"comment":"The component ablation is run on only 6 of the 11 datasets, excluding StanfordCars, Food101, CIFAR-100, CUB-200-2011, and RESISC45. These excluded datasets include the largest gains over DPA (StanfordCars +3.98, RESISC45 +4.60) and one negative result (CUB -0.16), so the averaged ablation conclusions do not necessarily transfer to the full benchmark. Please extend the ablation to all datasets, or at least to the excluded fine-grained and large-scale datasets, and report per-dataset component contributions so the reader can verify that PICS and NALR help in the same regimes where the final method is claimed to be SOTA.","section":"Section 4.2, Table 2"},{"comment":"The paper concedes that PICS can discard informative samples in datasets with high intra-class variability, and Table 2 shows drops for ALPHA-CS w/o NALR on OxfordPets (89.94 vs ALPHA-B 90.30) and for ALPHA-FS w/o NALR on Caltech101 (93.42 vs 93.48). Since PICS is the main novelty, this trade-off needs quantitative characterization: for example, report how many samples are filtered per epoch and the accuracy of the filtered-out samples, to show that the loss of informative samples is outweighed by noise reduction. Without this, the attribution of the final gains to PICS rather than to NALR or the fairness regularizer remains unproven.","section":"Section 4.2.1 and Conclusion"}],"minor_comments":[{"comment":"The FS variant is described as 'Confusion-based top-k Selection' in the text but as 'Fusion-based top-k Selection' in the Table 2 caption; please harmonize the terminology.","section":"Section 3.3.2 and Table 2 caption"},{"comment":"Both Eq. (4) and Eq. (15) define the self-training loss Lst, with Eq. (15) adding the PICS indicator. Please use a distinct symbol or explicitly state that Eq. (15) replaces Eq. (4), to avoid confusion in Section 4.2.1 where ALPHA-B is said to use 'the loss function from Eq. 4 + Lreg'.","section":"Equations (4) and (15)"},{"comment":"The text says the text encoder is discarded after initializing the text prototypes from CuPL descriptions, but Eq. (5) and the losses use textual prototypes Z. Please clarify whether Z is a learnable parameter vector after initialization, and whether the text encoder is used anywhere after the first prototype computation.","section":"Section 4, Implementation details"},{"comment":"The t-SNE visualization is not described in enough detail; please specify which embeddings are plotted, how the visual and textual prototypes are obtained, and how the in-class and cross-class metrics are represented in the figure.","section":"Figure 1"},{"comment":"The scalability experiment is performed only on Flowers and does not describe how the 20%, 40%, 60%, and 80% subsets are sampled. Please state the sampling strategy and whether results are averaged over multiple subset draws.","section":"Section 4.2.2 and Figure 5"},{"comment":"The caption says circle radius indicates the number of trainable parameters, but the axes are training time and accuracy; please clarify whether the parameter count is encoded only by the circle size and define the unit for the radius.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an honest empirical contribution with a clear structure, and the authors explicitly acknowledge the main weakness of PICS. The main risk is evaluation rigor: all results are single-run, the margin over DPA is small, and the key filtering mechanism is never directly validated on the selected subset. I would require the additional experiments described in the major comments before publication, but I do not see a fundamental flaw that would force rejection. The paper fits the scope of the journal, though the reproducibility section could be strengthened by releasing code and seeds."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a sensible continuation of the authors' DPA line. The combination of prototype-based clean/noisy selection (PICS), LLM-refined neighbor-aware labels (NALR), and an adaptive weight is new, and the ablations in Table 2 make a coherent case that each piece contributes. On 11 datasets the best variant (ALPHA-CS) averages 74.86, which is +1.42 over DPA. That is the whole story: a useful but modest improvement.\n\nWhat the paper does well: the three cross-class set strategies (CS, RS, FS) are a nice robustness check; the transductive/inductive comparison with ReCLIP* is a fair extra; the noisy-data experiment with CIFAR-100 injected into CIFAR-10 is relevant; and the authors are explicit about the limitation of PICS on high intra-class variability datasets. The writing is clear.\n\nWhere it is soft. The reported numbers are single-run. With a 1.4-point average gain, no error bars or seeds, some of the per-dataset wins and losses may just be noise. The PICS filter is self-referential: both phi and psi are computed in CLIP's embedding space from prototypes built from CLIP's own pseudo-labels. The paper's own limitation statement concedes this can discard informative samples, and Table 2 shows small drops on OxfordPets and Caltech101 when PICS is used without NALR. Figure 4 reports pseudo-label accuracy on the full training set, not on the subset PICS selects, so the mechanism is not actually demonstrated. Relatedly, the implementation details say the text encoder is discarded post-initialization, but Eq. 17 uses Et(T) to score image-text pairs; that is an internal contradiction that needs to be resolved. Hyperparameters k and kn are tuned on DTD and Flowers, which are also evaluation datasets, with no label-free validation protocol; the sensitivity analysis suggests the method is not extremely sensitive, so this is a moderate concern rather than fatal. No code is released.\n\nThe central claim—that the combination works and generalizes across datasets—is credible, but the evidence for the specific filtering mechanism is weaker than the paper suggests. This is a solid workshop- or conference-level paper that a serious referee could push to add multi-seed runs, a direct precision/recall analysis of the PICS-selected subset, and a fix to the text-encoder contradiction.\n\nVerdict: give it a serious peer review, ask for major revision. It is not a breakthrough but it is a legitimate, honest contribution.","headline":"A credible, incremental extension of the authors' DPA line with a 1.4-point average gain, but the core filtering mechanism is self-referential and the empirical case would benefit from multi-seed runs.","tokens_in":16478,"tokens_out":2800,"would_cite":false,"duration_ms":30495,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ALPHA filters and refines CLIP's own pseudo-labels using the geometry of the embedding space, and reports state-of-the-art accuracy on 11 unlabeled benchmarks.","keywords":["unsupervised adaptation","pseudo-labeling","CLIP","prototype consistency","neighborhood consistency","self-training","vision-language models","label noise"],"falsifier":"Freeze a pretrained CLIP, compute the in-class and cross-class scores for every sample of a high-intra-class-variability dataset with ground-truth labels available, and measure the precision of the rule against those labels; if the retained set's precision is no higher than the pseudo-label accuracy of unfiltered predictions, the PICS filter is not doing the work attributed to it. A complementary check on the same dataset is to ablate the adaptive weight by setting all values to one and verifying whether accuracy drops, since NALR's weighting is credited with preventing mismatched text descriptions from hurting training.","tokens_in":15455,"feed_emoji":"🏷️","tokens_out":6023,"duration_ms":60020,"temperature":0.7,"pith_summary":"Unsupervised adaptation of CLIP usually generates pseudo-labels from the model's own zero-shot predictions, and those labels are noisy, especially under domain shift, so self-training accumulates errors. ALPHA replaces fixed confidence thresholds with two geometric checks: PICS keeps a pseudo-label only if the image's embedding is closer to its assigned class prototype than to a set of confident samples from other classes, and NALR re-labels the rejected images using LLM-generated class descriptions, weighted by how locally consistent each image-text pair is. On 11 benchmarks the confidence-based variant, ALPHA-CS, is reported to reach 74.86% average accuracy, 1.42 points above the previous state-of-the-art method and more than 10 points above zero-shot CLIP, while fine-tuning only layer-normalization parameters and text prototypes. If correct, the paper shows that pseudo-label noise in label-free CLIP adaptation can be managed without external supervision by exploiting the geometry of the embedding space itself.","feed_headline":"Geometry check refines CLIP's pseudo-labels, topping prior adaptation","feed_subtitle":"Prototype and neighborhood checks replace confidence thresholds, curbing label noise with no ground truth.","key_machinery":"The machinery is a pair of complementary filters built from cosine similarities. PICS computes per-class prototypes via a memory bank of features, pseudo-labels, and confidence scores, then scores each sample by the difference between in-class compactness and cross-class separation; only samples with a positive difference supervise the main self-training loss. NALR handles the rejected samples by retrieving the most similar LLM-generated class description for each image, then re-weighting that assignment by the sigmoid of the difference between the image's similarity to its chosen description and the average similarity of its nearest image-text neighbors. A memory-bank feedback loop merges both branches so refined labels feed the next epoch's prototypes.","core_discovery":"ALPHA's central claim is that a pseudo-label is trustworthy exactly when the sample is more similar to its own class prototype than to confident samples of other classes, and that samples failing this test should not be discarded but refined through neighbor-aware weighting of LLM-generated text descriptions. The paper argues that this two-branch design, filter with PICS and refine with NALR, prevents error accumulation better than either branch alone, and that its adaptive weight assigns low influence to image-text pairs whose similarity is below that of their neighbors, suppressing mismatched descriptions. Results across Caltech101, DTD, EuroSAT, Flowers, OxfordPets, UCF101, StanfordCars, Food101, CIFAR100, CUB-200-2011, and RESISC45 support the claim, with all three cross-class selection strategies outperforming prior unsupervised adaptation methods.","pith_inferences":["An implication the authors leave implicit: PICS's geometric criterion could serve as a label-free calibration proxy for any self-training pipeline whose confidence scores are miscalibrated, not just for CLIP.","Beyond the paper's classification setting, the filter-and-refine split suggests a general recipe for noisy pseudo-labels: instead of discarding low-confidence samples, re-route them through a richer source of signal, here LLM text descriptions, with local agreement as the trust signal.","A testable extension the paper does not pursue is making the cross-class set size k adaptive per class based on prototype overlap, since the sensitivity analysis shows k matters more on some datasets than others.","Because the fairness regularizer already pushes predictions toward balance, a stress test for the method would be a severely class-imbalanced target dataset, where the prototype statistics and the regularizer may interact in ways the current benchmarks do not expose."],"forward_implications":["PICS-style filtering yields more clean samples than FixMatch-style confidence thresholding from the first epochs, reducing the early overfitting that limits self-training.","All three ALPHA variants beat zero-shot CLIP by about 10 points on average and outperform prior unsupervised adaptation methods on the 11-dataset suite, with the largest gains on texture, satellite, and action-recognition datasets.","The method transfers to a ViT-B/16 backbone and to the transductive setting, outperforming ReCLIP and DPA in both regimes.","Under label-irrelevant noise injected into the training set, ALPHA-CS stays ahead of DPA up to 70% noise and ALPHA-FS remains most stable at higher noise levels.","Layer-normalization-only fine-tuning suffices to match or beat heavier parameter-efficient methods such as LoRA and K-Adaptation within the ALPHA framework."],"supporting_citations":[{"why":"Supplies the base CLIP model with visual and textual encoders that ALPHA adapts and whose zero-shot predictions generate the initial pseudo-labels.","marker":"[1]"},{"why":"DPA is the primary state-of-the-art baseline that ALPHA must beat, with an average accuracy of 73.44% on the 11 datasets.","marker":"[34]"},{"why":"CuPL provides the LLM-generated class descriptions used to initialize text prototypes and to form the candidate labels for NALR refinement.","marker":"[21]"},{"why":"ReCLIP is the transductive baseline and also motivates layer-normalization-only fine-tuning and cross-modal self-training for label reliability.","marker":"[7]"},{"why":"FixMatch supplies the fixed-confidence-threshold filtering approach that PICS is directly compared against in the clean-sample and accuracy plots.","marker":"[5]"},{"why":"LaFTer is a baseline that uses LLM-generated descriptions but lacks explicit pseudo-label filtering, illustrating the need for the PICS component.","marker":"[6]"},{"why":"POUF is a prompt-oriented unsupervised fine-tuning baseline whose results ALPHA must exceed on the shared benchmark suite.","marker":"[3]"},{"why":"Provides the fairness regularization loss that ALPHA adopts to reduce confirmation bias and promote balanced predictions during self-training.","marker":"[18]"}],"fun_headline_variants":["Prototype and neighbor checks sharpen CLIP pseudo-labels","Adaptive pseudo-labeling beats prior CLIP adaptation methods","Filter with prototypes, refine with neighbors for CLIP","No-threshold pseudo-labeling for unsupervised CLIP adaptation","Prototype consistency and neighbor refinement top CLIP tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, in CLIP's embedding space, the inequality that in-class similarity exceeds cross-class similarity reliably marks a pseudo-label as correct; on datasets with high intra-class variability the authors concede this filter can discard informative samples, and if the premise fails early in training the refined set is not actually clean and error accumulation is not prevented.","fun_headline_variants_meta":{"raw":{"variants":["Prototype and neighbor checks sharpen CLIP pseudo-labels","Adaptive pseudo-labeling beats prior CLIP adaptation methods","Filter with prototypes, refine with neighbors for CLIP","No-threshold pseudo-labeling for unsupervised CLIP adaptation","Prototype consistency and neighbor refinement top CLIP tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2684,"prompt_tokens":882,"completion_tokens":1802,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":1723}},"tokens_in":498,"tokens_out":1802,"duration_ms":14107,"temperature":1.0,"reasoning_tokens":1723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:59:46.805369+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Freeze a pretrained CLIP, compute the in-class and cross-class scores for every sample of a high-intra-class-variability dataset with ground-truth labels available, and measure the precision of the rule against those labels; if the retained set's precision is no higher than the pseudo-label accuracy of unfiltered predictions, the PICS filter is not doing the work attributed to it. A complementary check on the same dataset is to ablate the adaptive weight by setting all values to one and verifying whether accuracy drops, since NALR's weighting is credited with preventing mismatched text descriptions from hurting training.","supporting_citations":[{"cited_title":"Radford et al","cited_arxiv_id":null,"evidence_quote":"Supplies the base CLIP model with visual and textual encoders that ALPHA adapts and whose zero-shot predictions generate the initial pseudo-labels."},{"cited_title":"Ali et al","cited_arxiv_id":null,"evidence_quote":"DPA is the primary state-of-the-art baseline that ALPHA must beat, with an average accuracy of 73.44% on the 11 datasets."},{"cited_title":"Pratt et al","cited_arxiv_id":null,"evidence_quote":"CuPL provides the LLM-generated class descriptions used to initialize text prototypes and to form the candidate labels for NALR refinement."},{"cited_title":"Hu et al","cited_arxiv_id":null,"evidence_quote":"ReCLIP is the transductive baseline and also motivates layer-normalization-only fine-tuning and cross-modal self-training for label reliability."},{"cited_title":"Sohn et al","cited_arxiv_id":null,"evidence_quote":"FixMatch supplies the fixed-confidence-threshold filtering approach that PICS is directly compared against in the clean-sample and accuracy plots."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LaFTer is a baseline that uses LLM-generated descriptions but lacks explicit pseudo-label filtering, illustrating the need for the PICS component."},{"cited_title":"Tanwisuth et al","cited_arxiv_id":null,"evidence_quote":"POUF is a prompt-oriented unsupervised fine-tuning baseline whose results ALPHA must exceed on the shared benchmark suite."},{"cited_title":"Li et al","cited_arxiv_id":null,"evidence_quote":"Provides the fairness regularization loss that ALPHA adopts to reduce confirmation bias and promote balanced predictions during self-training."}],"review_version":1}