{"id":"65f4d989-271c-4459-b8eb-89ec236c81a9","arxiv_id":"2507.09500","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"ReTA improves CLIP's test-time adaptation by reweighting sample selection with prediction consistency and adapting class decision boundaries via Gaussian text embeddings.","lead":"This paper presents ReTA, a test-time adaptation method for vision-language models like CLIP that improves accuracy on shifted images without using labels. It combines a consistency check that keeps only trustworthy samples in the model's memory cache with a Gaussian-based calibration step that makes classification boundaries more flexible.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim may depend on per-dataset validation-accuracy tuning (App. B), not on the unlabeled TTA protocol; margins over DPE/BoostAdapter are only ~1-2 pp.","rationale":"The reader's weakest assumption was that the stability-consistency score reliably identifies correct pseudo-labels. That is a legitimate concern, but the paper's own ablations show CER contributes only about half a point on both benchmarks, so even a partially unreliable consistency signal does not by itself threaten the headline SOTA claim. The evaluation-protocol issue is more load-bearing: Appendix B explicitly describes selecting eta by maximizing validation accuracy and alpha/beta as dataset-specific. If this means target-domain validation labels are used to choose hyperparameters, ReTA is not operating under the unlabeled TTA setting it claims. The reported advantages over DPE and BoostAdapter are 1-2 pp, which is exactly the size of improvement that per-dataset tuning on validation labels can inject. I therefore keep the CONDITIONAL verdict, but the binding condition should be a label-free, fixed-hyperparameter re-evaluation rather than (only) the consistency-score assumption. This is a concrete, checkable protocol issue, not a disagreement with consensus or an ad hominem point.","tokens_in":21990,"tokens_out":5110,"duration_ms":62176,"concrete_test":"Run the released code on the 15 benchmark datasets with a single fixed hyperparameter configuration chosen without any target-domain labels—e.g., eta=0.4, alpha=1.0, beta=5.0 (or the values used by TDA/DPE) for all datasets—and compare the resulting averages with Tables 1-2. If ReTA no longer beats DPE/BoostAdapter by the reported margins, or if the gap shrinks below the 1 pp level, the SOTA claim depends on label-guided validation tuning rather than on the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that ReTA is the current best cache-based TTA method. The most load-bearing threat to that claim is not the consistency score but the evaluation protocol. Appendix B states that the affine cache parameters alpha, beta and the final balance weight eta are 'dataset-specific' and that eta is selected by 'maximizing the average validation accuracy.' If that validation accuracy is computed on labeled target-domain validation splits, then ReTA's numbers are not produced under the unlabeled test-time adaptation protocol the paper claims, and the comparison against baselines (whose reported numbers use their published, typically fixed hyperparameters) is not apples-to-apples. The margins over the strongest baselines are small: 1.97 pp over DPE on Cross-Datasets and 1.10 pp over BoostAdapter on NDS (ViT-B/16). A per-dataset label-guided selection of three hyperparameters can plausibly account for margins of this size. The consistency-score concern is real but less decisive: Table 3 shows CER alone contributes only 0.49-0.62 pp, and the aggregative evidence (cache accuracy curves, ablations) suggests the consistency assumption is not what carries the headline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReTA, a cache-based test-time adaptation method for CLIP-style vision-language models. The method has two components: Consistency-aware Entropy Reweighting (CER), which uses a stability-consistency score across adjacent text embeddings to reweight entropy when updating a test-time cache; and Diversity-driven Distribution Calibration (DDC), which models class-wise text embeddings as multivariate Gaussians with residual learning to obtain more flexible decision boundaries. The final prediction combines cache logits with Gaussian-calibrated logits. The authors evaluate on Cross-Datasets generalization and natural distribution shifts (ImageNet and four variants) with CLIP-RN50 and CLIP-ViT-B/16, and report state-of-the-art averages (e.g., 71.37% on Cross-Datasets with ViT-B/16 versus 69.40% for DPE, and 67.29% on natural shifts versus 66.19% for BoostAdapter). They also report ablations, ECE, runtime, and experiments with EVA-02 and CoCa.","tokens_in":22270,"tokens_out":5255,"duration_ms":60461,"significance":"If the results hold, ReTA is a practically useful contribution to cache-based TTA: it avoids encoder backpropagation, improves cache quality under distribution shift, and provides better calibration than prior cache-based methods. The paper's strengths include a broad evaluation across two backbones and two additional model families, a public code release, runtime comparisons, ECE measurements, and careful ablations of the proposed components. The main caveat is that the headline SOTA claim currently rests on an evaluation-protocol issue: Appendix B discloses per-dataset selection of alpha, beta, and eta using validation accuracy, which conflicts with the unlabeled TTA setting and makes the comparison with fixed-parameter baselines not apples-to-apples. The consistency-correctness assumption is a further risk, though the ablation evidence suggests it is not the sole driver of the gains.","major_comments":[{"comment":"Appendix B states that the affine cache parameters alpha and beta are dataset-specific and that the balance weight eta is selected by 'maximizing the average validation accuracy.' This is a load-bearing protocol issue: the paper advertises unlabeled test-time adaptation, but a label-guided search over at least three hyperparameters per dataset can plausibly account for the small reported margins (1.97 pp over DPE on Cross-Datasets and 1.10 pp over BoostAdapter on natural shifts with ViT-B/16). The authors should state clearly whether the validation split is labeled, report results with a single fixed set of hyperparameters across all datasets, or, if tuning is retained, demonstrate that the method is insensitive to the choice and that all baselines were tuned under the same protocol.","section":"Appendix B"},{"comment":"The paper states that all experiments were run with three random seeds and that the average is reported, but no standard deviations or per-seed results appear in Tables 1-6. Since the SOTA margins are only one to two percentage points, the absence of variance estimates leaves it unclear whether the ranking over DPE and BoostAdapter is statistically stable. Please report per-seed results or at least standard deviations for the main tables.","section":"Section 4.1"},{"comment":"The stability-consistency score w (Eq. 13) gates both cache prioritization (Eq. 14) and the residual updates in DDC (Algorithm 1, line 10). The method therefore assumes that prediction consistency across adjacent text embeddings is positively correlated with pseudo-label correctness under distribution shift. The supporting evidence is indirect: aggregate cache-accuracy curves (Figure 1), a few qualitative cache examples (Figure 7), and ablations of the score formulation (Tables 8-10). Please add a quantitative analysis of precision at w=1 versus w>1, ideally with a per-dataset breakdown on ImageNet-A and other hard OOD sets, to show that confidently wrong samples do not contaminate the cache or the DDC updates.","section":"Section 3.2 and Algorithm 1"}],"minor_comments":[{"comment":"References [3] and [4] are the same work, and references [26], [27], and [28] are also the same work; please deduplicate them.","section":"References"},{"comment":"Tables 8 and 9 are identical; one of them appears to be a duplicate and should be removed.","section":"Appendix C"},{"comment":"The BoostAdapter average accuracy is reported as 66.19 in Table 2 but 66.34 in Table 7; please reconcile these numbers and state which setup each refers to.","section":"Tables 2 and 7"},{"comment":"The caption does not specify the dataset, backbone, or number of test samples for the cache-accuracy curves shown in Figure 1; please provide this information so the reader can judge how representative the example is.","section":"Figure 1"},{"comment":"In Algorithm 1, line 7, the condition max(H'(Fcache)) is ambiguous until the cache is full; please specify whether the comparison is made per class and how the first SZ samples per class are admitted.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible and well-scoped contribution, and the code release, multi-backbone evaluation, and runtime analysis are strengths. The main risk is that the SOTA claim rests on validation-accuracy-tuned hyperparameters, which conflicts with the unlabeled TTA setting. If the authors can provide a fixed-hyperparameter variant or a convincing sensitivity analysis, the paper would be acceptable; otherwise the claims should be substantially scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plainly: this is a solid empirical paper. The method is a synthesis of existing pieces—prompt-ensemble committee voting, ProDA's Gaussian text modeling, DPE's residual prototype updates, and TPT's augmentation entropy—with a consistency-aware entropy reweighting that gates cache insertion. That combination is new in the TTA literature, and the authors show it yields consistent 1–2 point average gains over DPE and BoostAdapter on two standard benchmarks, across CLIP-RN50/ViT-B/16 and two additional backbones. The ablations are thorough: both CER and DDC contribute, DDC more, and the SVD projection and progressive binning choices are tested, not just asserted. I also appreciate the ECE and runtime tables, and the fact that they reproduced BoostAdapter's ImageNet numbers.\n\nNow the soft spots. The load-bearing issue is not the consistency assumption, which the reader worried about. Table 3 shows CER alone buys only 0.49–0.62 pp, and Figure 1's cache-accuracy curves give indirect support. The real question is the evaluation protocol. Appendix B states that alpha, beta, and eta are 'dataset-specific' and that eta is picked by maximizing average validation accuracy. If that validation accuracy uses labeled target-domain splits, then the method is not being evaluated under the unlabeled TTA protocol it claims, and the 1.1–2.0 pp margins over baselines with published fixed hyperparameters are not apples-to-apples. That concern is substantial because the margins are small. The paper needs to say plainly what the validation set is, how the baselines were configured, and ideally re-run the comparison with baselines tuned under the same per-dataset validation rule.\n\nLesser issues: no error bars despite three seeds; some hyperparameters (gamma, delta, M, tau_c, cache size) appear dataset-specific but are described vaguely. Also, the DDC component is essentially ProDA's Gaussian prompt modeling applied to TTA with DPE's residual updates; the novelty claim should be toned down.\n\nWho is this for? People working on efficient TTA for CLIP-like models. It is a useful, well-engineered integration with strong empirical coverage. It does not open a new line of research, but it raises the bar for cache-based methods.\n\nRecommendation: send it to peer review. A serious referee can push on the protocol question and the error bars, but the paper is coherent, reproducible in spirit, and likely to be a reference point. Accept with revision if the validation question is resolved.","headline":"Solid integrated TTA method; the real caveat is the per-dataset validation tuning behind the SOTA claim, not the consistency score.","tokens_in":22839,"tokens_out":2309,"would_cite":true,"duration_ms":26652,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ReTA outperforms prior cache-based test-time adaptation methods by reweighting entropy with a consistency vote and by reshaping decision boundaries as evolving Gaussians, achieving the best reported average accuracy on both benchmarks.","keywords":["test-time adaptation","vision-language models","CLIP","cache-based adaptation","entropy reweighting","prediction consistency","distribution calibration","out-of-distribution generalization"],"falsifier":"Run ReTA on ImageNet-A, where CLIP is overconfident on many natural adversarial images, and measure the accuracy of the samples admitted to the cache with w = 1; if that cache accuracy is no better than chance while ReTA still improves, the gains come from the Gaussian boundary rather than the consistency signal. A cleaner experiment is a synthetic shift engineered so CLIP is consistently confident and wrong — for instance heavy blur combined with color negation — and check whether ReTA then falls below an entropy-only cache baseline.","tokens_in":21779,"feed_emoji":"🖼️","tokens_out":11176,"duration_ms":107253,"temperature":0.7,"pith_summary":"This paper claims that cache-based test-time adaptation of vision-language models fails for two reasons: entropy-based selection admits confidently wrong samples into the cache, and fixed text-embedding decision boundaries cannot accommodate large visual shifts. ReTA attacks both problems, using a stability-consistency score computed by a vote across adjacent class-specific text embeddings to reweight entropy during cache updates, and modeling each class's text embeddings as a multivariate Gaussian that evolves through residual learning to calibrate decision boundaries. On the standard benchmarks, ReTA reports average top-1 accuracy of 71.37% across ten cross-dataset tasks and 67.29% on ImageNet plus its four shifted variants with CLIP-ViT-B/16, ahead of the previous best cache-based methods. If the results hold, consistency voting over neighboring text embeddings, rather than raw confidence, is a reliable signal for keeping test-time caches clean.","feed_headline":"Consistency votes and Gaussian boundaries lift test-time accuracy","feed_subtitle":"ReTA averages 71.37% across 10 cross-datasets and 67.29% on ImageNet and its shifted variants.","key_machinery":"The central object is the stability-consistency score w (Eq. 13), which reweights prediction entropy during cache updates. To build it, the method sorts the hand-crafted prompts of each class by cumulative intra-class cosine similarity, progressively bins them into M adjacent class-specific text embeddings, projects image features into the text subspace via SVD to reduce the modality gap, and treats the M embeddings as a semantic voting committee whose pseudo-labels yield the score; w = 1 + log(RS) is minimal when all committee members agree and the majority vote matches the original prediction, and grows with disagreement or instability. The complementary mechanism, Diversity-driven Distribution Calibration (DDC), models each class's adjacent text embeddings as an approximate multivariate Gaussian N(μ, Σ), updates them through residual learning and progressive averaging, and uses the Gaussian means as a refined classifier with a surrogate loss that exploits the covariance terms. The score gates what enters the cache and what is allowed to update the residuals, while DDC reshapes the decision boundary.","core_discovery":"ReTA's central claim is that both failure modes of cache-based test-time adaptation can be addressed without labels or backbone updates. For the cache, instead of ranking samples by raw prediction entropy, ReTA computes a stability-consistency score w = 1 + log(RS) from agreement among pseudo-labels produced by M adjacent class-specific text embeddings and agreement of the majority vote with the original prediction; this score multiplies the entropy, so unstable or inconsistent samples are preferentially evicted and only samples with w = 1 drive residual updates. For the decision boundary, the adjacent text embeddings of each class are treated as an approximate multivariate Gaussian, whose means are evolved with learnable residuals and progressive averaging, and the resulting Gaussian logits are combined with cache logits and CLIP logits for the final prediction. The paper reports that this combination consistently outperforms state-of-the-art cache-based methods such as DPE and BoostAdapter on both cross-dataset generalization and natural distribution shift benchmarks.","pith_inferences":["The same consistency-voting signal could serve as a pseudo-label filter in other self-training and source-free adaptation settings, where confident-but-wrong samples cause the same error accumulation that ReTA targets.","Because DDC contributes more than CER in the ablations, a plausible reading is that the residually updated text embeddings carry most of the adaptation; a frozen-cache variant of ReTA would isolate how much of the gain is due to the cache alone.","The committee is built from a fixed set of hand-crafted prompts plus CuPL, so the score's reliability is coupled to prompt quality; perturbing or shrinking that prompt set is a direct stress test of the consistency assumption.","Since per-sample cost is dominated by 64 augmented views and the SVD projection, a batched or adaptive-view variant of the consistency assessment could cut inference time while preserving the reliability signal."],"forward_implications":["On Cross-Datasets with CLIP-ViT-B/16, ReTA reports 71.37% average accuracy versus 69.40% for DPE, the previous best cache-based method, and 67.29% versus 66.19% for BoostAdapter on ImageNet and its four shifted variants.","Cache quality stays high throughout testing under ReTA, while earlier methods' caches fill with misclassified low-entropy samples as adaptation proceeds (Figure 1 and the qualitative cache comparisons in Figure 7).","The Gaussian decision boundary carries more of the gain than cache refinement: ablations attribute larger improvements to DDC than to CER, and removing the Gaussian logits flips 6.18% of predictions.","The gains are not specific to CLIP-ViT-B/16: ReTA also improves over DPE and TDA on EVA-02-B-16 and CoCa-ViT-B-32, and it beats RLCF when the entropy objective is swapped for CLIP reward feedback, with a lower average Expected Calibration Error than TDA and DPE."],"supporting_citations":[{"why":"CLIP provides the frozen vision and text encoders and the shared embedding space that ReTA adapts at test time.","marker":"[42]"},{"why":"TDA establishes the dynamic entropy-ranked cache paradigm and the cache-based prediction formula that ReTA inherits and modifies.","marker":"[25]"},{"why":"DPE supplies the strongest cache-based baseline, the per-class mean visual prototypes, and the residual learning that DDC extends to Gaussian distributions.","marker":"[60]"},{"why":"BoostAdapter is the strongest baseline on ImageNet variants and the regional bootstrapping cache method ReTA must beat.","marker":"[62]"},{"why":"ProDA contributes the Gaussian prompt-distribution modeling and the surrogate loss that DDC adapts for its calibration objective.","marker":"[33]"},{"why":"Tip-Adapter introduces the key-value cache construction that the cache-based test-time prediction in ReTA builds upon.","marker":"[61]"},{"why":"TPT defines the test-time adaptation benchmark protocol, including AugMix augmentation and the 64-view setup, and is a main prompt-tuning baseline.","marker":"[46]"},{"why":"The Zero method is cited as evidence that entropy is an unreliable confidence measure under test-time shifts, motivating the consistency-based reweighting.","marker":"[11]"},{"why":"DMN provides a complementary dual-memory cache baseline compared across both benchmarks.","marker":"[63]"}],"fun_headline_variants":["ReTA: consistency-weighted cache and Gaussian boundaries","Reliable test-time adaptation with consistency and Gaussian models","Consistency-aware entropy and Gaussian boundaries improve TTA","Test-time adaptation fixed with consistency and Gaussian calibration","Cache-based TTA gets reliability via consistency and Gaussian fits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the stability-consistency score w really marks which test samples carry correct pseudo-labels, because both the cache and the Gaussian calibration are updated only from samples the score trusts; if agreement across the adjacent text embeddings stops correlating with correctness under a particular shift, the method's own training signal becomes polluted.","fun_headline_variants_meta":{"raw":{"variants":["ReTA: consistency-weighted cache and Gaussian boundaries","Reliable test-time adaptation with consistency and Gaussian models","Consistency-aware entropy and Gaussian boundaries improve TTA","Test-time adaptation fixed with consistency and Gaussian calibration","Cache-based TTA gets reliability via consistency and Gaussian fits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000768,"raw_usage":{"total_tokens":3448,"prompt_tokens":1033,"completion_tokens":2415,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":2340}},"tokens_in":649,"tokens_out":2415,"duration_ms":19388,"temperature":1.0,"reasoning_tokens":2340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:54:58.473001+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ReTA on ImageNet-A, where CLIP is overconfident on many natural adversarial images, and measure the accuracy of the samples admitted to the cache with w = 1; if that cache accuracy is no better than chance while ReTA still improves, the gains come from the Gaussian boundary rather than the consistency signal. A cleaner experiment is a synthetic shift engineered so CLIP is consistently confident and wrong — for instance heavy blur combined with color negation — and check whether ReTA then falls below an entropy-only cache baseline.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CLIP provides the frozen vision and text encoders and the shared embedding space that ReTA adapts at test time."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TDA establishes the dynamic entropy-ranked cache paradigm and the cache-based prediction formula that ReTA inherits and modifies."},{"cited_title":"Sycara, and Yaqi Xie","cited_arxiv_id":null,"evidence_quote":"DPE supplies the strongest cache-based baseline, the per-class mean visual prototypes, and the residual learning that DDC extends to Gaussian distributions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BoostAdapter is the strongest baseline on ImageNet variants and the regional bootstrapping cache method ReTA must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ProDA contributes the Gaussian prompt-distribution modeling and the surrogate loss that DDC adapts for its calibration objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Tip-Adapter introduces the key-value cache construction that the cache-based test-time prediction in ReTA builds upon."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TPT defines the test-time adaptation benchmark protocol, including AugMix augmentation and the 64-view setup, and is a main prompt-tuning baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Zero method is cited as evidence that entropy is an unreliable confidence measure under test-time shifts, motivating the consistency-based reweighting."}],"review_version":1}