{"id":"5043c400-a506-47b1-b0d1-ee1c857baa8b","arxiv_id":"2607.03900","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Adaptive self-ensembling of weak and strong views yields more reliable pseudo-labels for test-time prompt tuning of CLIP and unifies training with inference.","lead":"The paper reinterprets classic Test-Time Prompt Tuning (TPT) for CLIP as reverse-cross-entropy learning from self-generated pseudo-labels, then introduces Self-Ensembling (SE) that adaptively up-weights the original test image over strong augmentations. The resulting USE framework applies the same SE rule at both optimization and inference, and SE alone works as a cheap training-free adapter that lifts many existing TTA methods.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The manuscript cleanly rewrites TPT’s marginal-entropy objective as reverse cross-entropy against a stop-gradient pseudo-label (Eqs. 4–5), replaces the indiscriminate average with an entropy-rank-weighted self-ensemble (Eqs. 6–7), and re-uses that same ensemble at inference (Eq. 9). All supporting tables (1–10), ablations (Table 6), plug-in experiments (Table 7), hyper-parameter sweeps (Fig. 3, Table 8, Appendix D) and efficiency numbers are internally consistent and obtained under a shared public benchmark. The only modelling choice that could be questioned—the fixed linear map from δ to β—is already stress-tested by the authors and shown not to be critical. Because the concrete verification proposed above is expected to leave the headline numbers essentially unchanged, the reader’s ACCEPT verdict with high confidence requires no adjustment.","tokens_in":25971,"tokens_out":579,"duration_ms":5462,"concrete_test":"Re-run the full USE pipeline of Algorithm 1 on the ViT-B/16 FGVC suite while forcing β=0.5 for every sample (i.e., disable the entropy-rank adaptation). If average accuracy remains within 0.3 pp of the adaptive result reported in Table 2 (65.35 %), the adaptive component is non-essential and the claim still holds; a drop larger than 1 pp would elevate the reader’s weakest assumption to a genuine load-bearing concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader’s weakest assumption (that relative entropy rank δ of the weak view is a reliable proxy for semantic trustworthiness, with fixed γ=0.4) is the natural soft spot, yet the paper already supplies direct empirical counter-evidence that the assumption is not load-bearing for the central claim. Fig. 2 shows that even a naïve 0.5/0.5 weak–strong average already stabilizes accuracy across FGVC datasets; Tables 6–8 and Fig. 3 further demonstrate that the adaptive linear rescaling of β improves over both uniform averaging and the extreme fixed values β∈{0,0.5,1}, while non-linear alternatives (square-root, squared) give only marginal further gains. The same SE module, when plugged into four independent TTA methods (Table 7), yields consistent lifts without any re-tuning of γ. Consequently the central claim—that the RCE reinterpretation plus the identical SE ensemble at both stages produces better pseudo-labels and final predictions—rests on reproducible empirical regularities rather than on an untested modelling assumption. No internal inconsistency or missing control that would overturn the reported gains is present.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper reinterprets the marginal-entropy objective of Test-Time Prompt Tuning (TPT) as reverse cross-entropy (RCE) against a stop-gradient average of low-entropy AugMix views (Eqs. 4–5), thereby casting TPT as implicit self-training with a pseudo-label. Building on this view it introduces Self-Ensembling (SE): an adaptive linear combination of the weak (original) view and the selected strong views whose weight β is obtained from the relative entropy rank δ of the weak view (Eqs. 6–7, γ fixed at 0.4). The same SE rule is reused at inference (Eq. 9), yielding the Unified Self-Ensembling framework (USE). SE alone also functions as an optimization-free TTA method and as a drop-in module. Experiments on ImageNet and four OOD variants plus ten fine-grained datasets, two backbones, CoOp initialization, multi-seed averages, component ablations (Table 6), plug-in results (Table 7), efficiency measurements and a skip technique demonstrate consistent gains over both optimization-free and optimization-based baselines.","tokens_in":26279,"tokens_out":729,"duration_ms":18710,"significance":"If the reported gains hold, the work supplies a clean, low-overhead improvement to a still-strong baseline (TPT) together with a reusable, training-free module that lifts several independent TTA methods. Strengths that raise the contribution above a pure empirical tweak include (i) the algebraic re-derivation of marginal entropy as RCE (verified analytically and by near-identical accuracy tables in Appendix B), (ii) the explicit train–inference consistency principle, (iii) public code, multi-seed reporting, thorough ablations of every design choice (including non-linear rescalings of β and the skip ratio), and (iv) efficiency numbers showing that SE costs roughly the same as 64 CLIP forwards while USE is cheaper than most gradient-based competitors. These elements make the paper a useful reference point for future episodic TTA work on vision–language models.","major_comments":[],"minor_comments":[{"comment":"Footnote 1 contains a typographical run-on (“weseparatelyensure”); similar missing spaces appear around “SEalone” and a few other compound terms. A global proof-read would remove them.","section":null},{"comment":"Figure 2 caption and the surrounding paragraph in §3.2 refer to four inference strategies; the legend labels are clear, yet the y-axis scale (“accuracy gains over pw”) would be easier to read if the zero line were drawn more prominently.","section":null},{"comment":"In Algorithm 1 the line that recomputes pw and ps after the parameter update is slightly ambiguous about whether the same selected set S is reused; a one-sentence clarification would help reproducibility.","section":null},{"comment":"Appendix D and Figure 7 show robustness to γ, but the main text never states the precise range of β that results from γ = 0.4; adding “[0.3, 0.7]” once in §3.2 would make the design choice immediately transparent.","section":null},{"comment":"Table 10 reports wall-clock time and peak memory; stating the hardware (GPU model) used for these measurements would allow readers to contextualize the absolute numbers.","section":null}],"recommendation":"accept","confidential_remarks":"Solid, carefully executed incremental contribution that cleanly improves a hard-to-beat baseline. No novelty or citation concerns; suitable for the journal."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that this is a careful engineering paper on CLIP test-time prompt tuning, not a conceptual breakthrough. They rewrite TPT’s marginal-entropy objective as reverse cross-entropy against a stop-gradient average prediction (Eqs. 4–5), then replace that average with an entropy-rank-weighted mix of the original image and selected AugMix views (SE, Eqs. 6–7), and reuse the identical mix at inference (USE). That train–inference consistency is the actual contribution.\n\nWhat they do well: the algebra is correct and they verify it with near-identical accuracy tables (Appendix B). The experiments are thorough—15 datasets, two backbones, CoOp init, multi-seed, full ablations (Table 6), plug-in gains on four other methods (Table 7), efficiency numbers, skip trick, and code promised. SE alone already beats or matches the optimization-free baselines and lifts existing optimizers without retuning γ. Fig. 2 and the fixed-β sweeps show that even a naïve 0.5/0.5 weak–strong average helps; the adaptive rule is a modest refinement on top of that regularity. Citation pattern is fair; they treat TPT as the hard baseline it is.\n\nSoft spots are real but small. The relative-entropy-rank proxy for “trustworthiness” of the weak view is an assumption, yet Tables 6–8 and Fig. 3 show the method is not brittle to it—uniform averaging and non-linear rescalings still work, and γ=0.4 is fixed once. Gains are typically 0.5–1.5 points; significance is therefore local to the VLM-TTA subfield. Overhead of 64 views remains, and they note the obvious failure mode when the original image is badly corrupted. Nothing load-bearing is broken.\n\nThis is for people already running TPT-style pipelines who want a cheap, reproducible lift and a cleaner story about what the entropy objective is doing. It deserves a serious referee; the evidence is solid enough that the incremental nature should be judged on the numbers, not desk-rejected. I would engage with it if I were working on CLIP adaptation; I would not reorganize a research agenda around it.","headline":"Clean, well-executed systems paper that re-derives TPT as RCE self-training and adds a simple adaptive weak-view ensemble used consistently at train and test; gains are real but incremental.","tokens_in":26860,"tokens_out":571,"would_cite":true,"duration_ms":6021,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Test-time prompt tuning for CLIP is really self-training on a better pseudo-label: weight the original image more than its strong augmentations, then use the same ensemble at inference.","keywords":["test-time adaptation","vision-language models","CLIP","prompt tuning","self-ensembling","pseudo-labels","test-time augmentation"],"falsifier":"On a held-out suite where the original image is systematically more corrupted than its AugMix views, measure whether SE still improves over uniform averaging or weak-only prediction; a consistent drop would falsify the rank-based weighting premise.","tokens_in":26890,"feed_emoji":"🖼️","tokens_out":640,"duration_ms":5065,"temperature":0.7,"pith_summary":"Classic Test-Time Prompt Tuning (TPT) improves CLIP by generating many AugMix views of a test image, keeping the low-entropy ones, and minimizing the entropy of their average prediction. This paper shows that the objective is mathematically equivalent to reverse cross-entropy against a stop-gradient version of that average, so TPT is already performing a form of self-training on a self-generated pseudo-label. The quality of that pseudo-label is therefore the bottleneck. The authors replace the uniform average with a self-ensembling (SE) rule that first ranks the original (weak) view by entropy relative to the strong views and then adaptively up-weights it. The identical weighted ensemble is reused after the prompt update to produce the final prediction, yielding a single consistent pipeline called USE. SE alone already functions as a cheap, optimization-free adapter and can be dropped into existing TTA methods as a plug-in that raises their accuracy. Experiments on ImageNet distribution shifts and ten fine-grained datasets confirm consistent gains for both SE and USE over strong baselines.","feed_headline":"Weight the original image more and CLIP adapts better at test time","feed_subtitle":"Self-ensembling turns TPT into consistent self-training and lifts accuracy without extra labels","key_machinery":"Self-ensembling (SE): the relative entropy rank δ of the weak view is linearly rescaled into a weight β ∈ [0.3,0.7] that blends the weak prediction with the average of selected strong views; the identical blend is used both to form the training target and to produce the final output.","core_discovery":"TPT’s marginal-entropy loss is reverse cross-entropy against a stop-gradient average prediction; replacing that average by an entropy-rank-weighted self-ensemble of the original image and selected strong augmentations, and applying the same ensemble at inference, produces more reliable pseudo-labels and higher accuracy than either stage acting alone.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Self-ensembling weights original image for reliable TPT pseudo-labels","Unify TPT optimization and inference with entropy-rank self-ensemble","Emphasize test image over augmentations to lift CLIP test-time accuracy","Self-ensemble turns TPT into consistent self-training without labels","Same self-ensembling strategy at train and inference beats TPT alone"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The relative entropy rank of the single original image among its strong AugMix views is a trustworthy proxy for how much semantic weight that original image deserves.","fun_headline_variants_meta":{"raw":{"variants":["Self-ensembling weights original image for reliable TPT pseudo-labels","Unify TPT optimization and inference with entropy-rank self-ensemble","Emphasize test image over augmentations to lift CLIP test-time accuracy","Self-ensemble turns TPT into consistent self-training without labels","Same self-ensembling strategy at train and inference beats TPT alone"]},"model":"grok-4.5","effort":"low","cost_usd":0.005242,"raw_usage":{"total_tokens":1451,"prompt_tokens":770,"num_sources_used":0,"completion_tokens":97,"cost_in_usd_ticks":52420000,"prompt_tokens_details":{"text_tokens":770,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":584,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":770,"tokens_out":97,"duration_ms":5362,"temperature":1.0,"reasoning_tokens":584,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T23:09:16.054403+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a held-out suite where the original image is systematically more corrupted than its AugMix views, measure whether SE still improves over uniform averaging or weak-only prediction; a consistent drop would falsify the rank-based weighting premise.","supporting_citations":[],"review_version":1}