{"id":"06d75c41-2e1a-445f-8af4-6a72e2832186","arxiv_id":"2505.02325","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A training-free test-time adaptation method shifts CLIP embeddings of 3D objects toward confident target samples, with LLM-generated text cues, and reports large gains on four open-set retrieval benchmarks.","lead":"This paper presents a way to make an off-the-shelf image-text model (CLIP) retrieve 3D objects it has never seen, by adjusting its features during testing. The method claims to beat both trained and zero-shot alternatives on four standard benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pseudo-label accuracy is unvalidated and the threshold alpha is tuned on test benchmarks, so the reported gains may reflect test-set selection rather than a general adaptation principle.","rationale":"The reader's weakest_assumption correctly identifies pseudo-label selection as unvalidated and notes that alpha is tuned on the test benchmarks. I agree that this is the load-bearing assumption, and I also flag the gradient mismatch between Eq. (7) and Eq. (8) as a correctness issue. The paper has strong empirical results and clear ablations, but without a validation split and pseudo-label accuracy diagnostics, the central claim is plausible but not fully established. Hence CONDITIONAL, not REJECT.","tokens_in":16898,"tokens_out":1700,"duration_ms":19228,"concrete_test":"Re-run the OS-ABO-core and OS-MN40-core experiments with a validation protocol: use one benchmark (e.g., OS-ABO-core) to tune alpha, lambda, tau_i, and the learning rate schedule, then evaluate on the remaining three benchmarks with those fixed hyperparameters. Compare the resulting mAP to the reported test-tuned numbers. In addition, instrument the method to log pseudo-label precision (fraction of selected pairs that share the ground-truth category) and the percentage of targets passing the alpha=0.6 threshold, and measure how these evolve over the 2000 iterations. If the validation-tuned results drop materially (e.g., more than 3 mAP points on OS-MN40-core) or if pseudo-label precision is below roughly 70%, the reported gains are not established as a general test-time adaptation effect.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that TeDA's test-time distribution alignment, driven by confident query-target pseudo-labels, generalizes to unseen 3D categories and beats trained methods. The load-bearing assumption is that the initial CLIP nearest-neighbor structure is accurate enough that pseudo-labels selected above threshold alpha are correct. However, the paper never measures pseudo-label accuracy, the fraction of samples above alpha, or how performance depends on alpha on a validation split. Eq. (6) converts soft labels to one-hot pseudo-labels, and Eqs. (8)-(9) pull query features toward the corresponding targets. If the initial nearest-neighbor structure is wrong for an unseen category, the self-boosting update can lock in and amplify errors. The risk is heightened because alpha, tau_i, the learning rate schedule, and lambda are selected on the four test benchmarks themselves: Table 4 tunes alpha on OS-ABO-core, Figure 5 tunes lambda on OS-MN40-core and OS-ABO-core, and Figure 6 tunes tau_i on OS-MN40-core, all with no held-out validation split. Since the headline comparisons in Table 1 use those same test datasets, the improvement over HGM2R could include selection bias. Additionally, the gradient in Eq. (8) is not the gradient of the stated KL objective in Eq. (7) under the paper's definitions, and the paper does not reconcile this mismatch, so the optimization is not fully specified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"TeDA proposes a training-free, test-time adaptation method for zero-shot 3D object retrieval. Given a query set and a target set of 3D objects, the method renders each object into multi-view images, extracts CLIP image features, generates an InternVL textual description per object, and fuses visual and textual features by weighted addition followed by tanh normalization. It then iteratively refines the query embeddings by minimizing a KL divergence between a softmax similarity distribution (between targets and learnable query embeddings) and a pseudo-label distribution obtained by thresholding the softmax similarities computed with the original query embeddings. The authors evaluate TeDA on four open-set 3D object retrieval benchmarks (OS-ESB-core, OS-NTU-core, OS-MN40-core, OS-ABO-core) and report large mAP/NDCG improvements over both training-required baselines such as HGM2R and training-free baselines such as ULIP, Uni3D, and MV-CLIP. They also include ablations over the fusion weight, pseudo-label threshold, temperature, number of views, fusion scheme, and normalization, plus an extension to depth-map inputs on Objaverse-LVIS.","tokens_in":17226,"tokens_out":3696,"duration_ms":41157,"significance":"If the reported results survive proper validation, TeDA is a conceptually simple and appealing alternative to trained 3D retrieval models: it requires no 3D training data, no fine-tuning of CLIP, and only adapts extracted embeddings at test time. The paper is, to my knowledge, the first to frame this task as test-time distribution alignment for 3D retrieval, and the empirical gains over strong baselines are large (e.g., 65.45 vs. 51.74 mAP over HGM2R on OS-ESB-core with OpenCLIP ViT-L/14). The work also contributes a practical multi-view-plus-text representation for open-set 3D categories. However, the significance is tempered by two load-bearing issues: the stated gradient update does not match the claimed KL objective, and key hyperparameters are tuned directly on the test benchmarks, both of which undermine the current evidence for the method's general effectiveness.","major_comments":[{"comment":"The update rule in Eq. (8) is not the gradient of the KL divergence in Eq. (7). With P = softmax(XQ^T / tau_i) and P' fixed, the correct gradient of sum_i KL(P'_i || P_i) with respect to Q is (1/tau_i) * (P - P')^T X, not X^T (XQ^T/tau_i - P'). The latter omits the softmax over the logits. This is not a notational difference: the two expressions behave differently (e.g., the raw logits are unbounded and the update direction is not the natural gradient of the stated objective). The authors should either correct Eq. (8) to include the softmax, or explicitly state that Eq. (8) is an approximate or alternative update and reconcile it with the claim that the method minimizes KL divergence.","section":"Section 3.3, Eq. (8)"},{"comment":"Key hyperparameters are selected on the same test benchmarks that produce the headline results in Table 1, without a validation split. Specifically, alpha=0.6 is chosen by maximizing mAP on OS-ABO-core (Table 4), lambda is tuned on OS-MN40-core and OS-ABO-core (Figure 5), and tau_i is tuned on OS-MN40-core (Figure 6), and all of these are then used for the Table 1 results on those datasets. This test-set selection can inflate the reported margins relative to baselines that use fixed or previously validated hyperparameters, and it limits the confidence that the improvement generalizes. The authors should report results using hyperparameters chosen on a held-out validation set or through nested cross-validation, and provide error bars over multiple runs.","section":"Section 4.3, Tables 4 and Figures 5-6"},{"comment":"The pseudo-label accuracy is never measured, despite being load-bearing for the adaptation. The method selects query-target pairs with the highest initial CLIP similarity, converts them to one-hot targets, and then pulls query features toward those targets. If the initial nearest-neighbor structure is wrong for some unseen categories, the iterative update can amplify errors. The paper should report, at minimum, the precision of the selected pseudo-labels (e.g., the fraction of query-target pairs above alpha that share the same ground-truth category), the fraction of samples above the threshold, and the effect of alpha on a validation split. Without this, the reported gains may partly reflect a self-confirming loop on easy pairs rather than a robust adaptation principle.","section":"Section 3.3, Eq. (6) and Section 4.3"}],"minor_comments":[{"comment":"The title contains a typo: 'Lanuage' should be 'Language'.","section":"Title and Abstract"},{"comment":"The caption contains 'pretraind' and should be 'pretrained'.","section":"Figure 2 caption"},{"comment":"There are inconsistent abbreviations: 'Uin3D' in Table 1 should be 'Uni3D', and 'InforNCE' in Figure 3 should be 'InfoNCE'.","section":"Table 1 and Figure 3"},{"comment":"The table title says 'ZS-Objaverse-Core' but the text and the surrounding description refer to 'Objaverse-LVIS'. Please make the dataset name consistent.","section":"Section 4.4, Table 7"},{"comment":"Reference [55] is titled as a multimodal emotion recognition method, yet it is used in the context of 3D object retrieval auto-encoder methods. This is mismatched; please verify the citation or replace it with the correct source.","section":"References"},{"comment":"The two softmax distributions P' and P use different temperatures, tau_t and tau_i, but the roles of the two temperatures are only explained later in the ablation section. Please state their roles when the equation is introduced.","section":"Section 3.3, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and reports very large improvements, but the two load-bearing concerns (incorrect gradient equation and test-set hyperparameter tuning) are serious enough that the current form cannot be accepted. The gradient issue is easy to fix in a revision; the evaluation issue requires additional experiments with proper validation splits. I would also encourage the authors to provide pseudo-label quality statistics and error bars. The work's scope fits ICMR well."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TeDA is a clever, well-engineered application of test-time adaptation to open-set 3D object retrieval. The reported gains over strong baselines are large and consistent, but two issues - a mis-specified gradient and test-set hyperparameter tuning - keep me from fully trusting the margins.\n\nWhat's genuinely new and good: the query-target pair formulation for pseudo-label self-training in retrieval, combined with InternVL-generated captions, appears to be new. The experiments are broad: four open-set benchmarks, 13 baselines, ablations for each module, and a depth-map extension. The improvements are substantial - 65.45 mAP on OS-ESB-core versus 51.74 for HGM2R - and hold across both CLIP and OpenCLIP backbones. Code is promised. This is a real contribution.\n\nSoft spots, in order of seriousness. First, Eq. (8) does not match Eq. (7). If P is the softmax of XQ^T/tau_i, then the gradient of KL(P'||P) with respect to Q is X^T(P - P')/tau_i, not X^T(XQ^T/tau_i - P'). The update omits the softmax probability term. It may be a typo, and the implemented code might do the right thing, but the paper as written does not specify the actual objective that is optimized. Referees should ask for the corrected derivation or the code.\n\nSecond, hyperparameters are selected directly on the test benchmarks: alpha on OS-ABO-core (Table 4), lambda on OS-MN40-core and OS-ABO-core (Figure 5), tau_i on OS-MN40-core (Figure 6). No held-out validation split. That introduces selection bias into the headline comparisons. No error bars either, so we cannot tell whether a few points of mAP are meaningful.\n\nThe stress-test concern about pseudo-label accuracy is valid. The paper never measures the fraction of pseudo-labels that are correct, nor how alpha affects label accuracy - only the final mAP. For a self-boosting method, that is exactly the failure mode to watch: if the initial nearest-neighbor structure is wrong, the updates can amplify errors. The paper does not address this.\n\nOn circularity: I don't think this is a fatal issue. It is transductive self-training on the test set, no labels involved, and evaluation is on fixed benchmarks. That is legitimate for a TTA method, though it does mean the adaptation is performed on the evaluation set itself.\n\nWho it's for: researchers in 3D retrieval, zero-shot transfer, and test-time adaptation. The paper is clearly written and the experiments are thorough. It deserves a serious referee, not a desk reject, but the referee should require a corrected derivation and a validation protocol. Without those, the quantitative claims should be treated as provisional.","headline":"TeDA is a clever test-time adaptation for 3D retrieval with large reported gains, but the KL gradient is mis-derived and key hyperparameters are tuned on the test benchmarks, so the margins are not yet fully trustworthy.","tokens_in":17712,"tokens_out":4295,"would_cite":false,"duration_ms":44696,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TeDA adapts CLIP at test time with pseudo-labeled multi-view pairs and InternVL text cues, beating training-required and training-free methods on four 3D retrieval benchmarks.","keywords":["3D object retrieval","zero-shot retrieval","test-time adaptation","vision-language models","CLIP","multi-view images","pseudo-labeling","distribution alignment"],"falsifier":"Take a benchmark with known category labels and run TeDA while logging the pseudo-labels selected at each iteration; if the precision of the selected pseudo-labels against ground-truth matches is at or below chance for unseen categories, or if the reported mAP gains vanish when the same pipeline is run with pseudo-labels replaced by random labels, then the gains are not caused by correct distribution alignment.","tokens_in":16721,"feed_emoji":"🧊","tokens_out":9652,"duration_ms":99059,"temperature":0.7,"pith_summary":"The paper introduces TeDA, a test-time adaptation method that lets a frozen CLIP model retrieve 3D objects from categories it has never seen, without any 3D training data. TeDA renders each 3D object into multi-view images, pools CLIP's view features, and then iteratively shifts query features toward confident target neighbors using a KL-divergence objective. It also feeds the views to InternVL to produce a one-sentence object description and adds the CLIP text embedding to the visual one. On four open-set 3D object retrieval benchmarks, TeDA reports higher mAP than all compared training-required methods (e.g., HGM2R) and all training-free methods (e.g., MV-CLIP, Uni3D), with OpenCLIP ViT-L/14 reaching 65.45 mAP on OS-ESB-core and 67.93 on OS-NTU-core. If the result holds, it suggests that expensive 3D-specific pretraining may be replaceable by lightweight adaptation of off-the-shelf 2D vision-language models.","feed_headline":"Test-time CLIP alignment beats trained 3D retrieval models","feed_subtitle":"TeDA adapts a frozen CLIP and adds LLM captions, topping four open-set 3D benchmarks without any 3D training.","key_machinery":"The load-bearing object is the iterative distribution-alignment loop over the query feature matrix $Q$. Starting from CLIP's pooled multi-view embeddings, TeDA computes a softmax similarity matrix between targets $X$ and queries $Q$ (Eq. 5), thresholds the highest entry per target row to make one-hot pseudo-labels $P'$ (Eq. 6), and then takes gradient steps on the KL divergence between the soft retrieval distribution and $P'$ (Eqs. 7-9), which has the closed-form update $\\Delta Q = X^T(X \\cdot Q^T / \\tau_i - P')$. The text side is a second channel: InternVL descriptions are embedded by CLIP's text encoder and fused by $\\tanh(g + \\lambda f_{\\text{text}})$, exploiting CLIP's aligned image-text space. The mechanism's job is to contract query features toward the nearest reliable targets so that the final retrieval scores $R = Q \\cdot X^T$ become sharper on unseen classes.","core_discovery":"TeDA's central claim is that the distribution gap between CLIP's 2D natural-image training and 3D multi-view test data can be largely closed at test time, without retraining, by treating each query object as its own class and pushing its embedding toward high-confidence target instances. The refinement is driven by pseudo-labels: a target is assigned to a query when the softmax similarity exceeds a threshold, and the query features are updated by gradient descent on the KL divergence between the pseudo-label distribution and the current retrieval distribution. A second claim is that complementary text descriptions generated by InternVL and embedded in CLIP space add stable semantic cues that further help unseen categories. Together these mechanisms let TeDA surpass not only zero-shot baselines but also methods that train on the benchmark data, and the same recipe transfers to depth-map projections on Objaverse-LVIS.","pith_inferences":["Our inference: the pseudo-label threshold $\\alpha$ acts as a self-trust knob; a principled way to set it without test labels would be to compare the distribution of max-softmax scores before and after adaptation and choose a point that stabilizes—an auto-calibration step the paper does not explore.","Our inference: if the initial CLIP nearest-neighbor structure is systematically wrong for a broad unseen category, the iterative updates will amplify that error; one testable guard would be to restrict updates to query-target pairs whose similarity is high in both visual and text channels, or to add a small entropy regularizer on the pseudo-label distribution.","Our inference: the method's reliance on InternVL captions suggests a failure mode when the LLM misnames an object class; prompting for shape-only descriptions (the paper's Q2) already degrades results, so caption accuracy is likely a first-order factor in the text-fusion gain.","Our inference: TeDA's formulation is agnostic to the feature source; applying the same KL pseudo-label loop to features from other multimodal encoders (e.g., a stronger open CLIP variant) would test whether the alignment mechanism, rather than the specific features, drives the reported gains."],"forward_implications":["On the four open-set benchmarks, a training-free CLIP adaptation outperforms every compared training-required method, including HGM2R, so task-specific 3D training is not required for top retrieval accuracy on these splits.","Adding InternVL-generated text features improves retrieval on all four benchmarks, with the largest gain (8.43 mAP on OS-ABO-core) on datasets dominated by common, nameable objects.","The framework extends beyond gray multi-view renders to depth maps: on Objaverse-LVIS it reaches 20.35 mAP, surpassing ULIP-2's 18.15 without point-cloud pretraining.","Because the CLIP backbone stays frozen and only test-time embeddings are optimized, future improvement in the frozen encoder should translate directly into better TeDA retrieval."],"supporting_citations":[{"why":"Supplies the four open-set benchmarks, the 24-view rendering scheme, and the strongest training-required baseline (HGM2R) that TeDA is compared against and surpasses.","marker":"[9]"},{"why":"Provides the frozen CLIP image and text encoders whose aligned feature space TeDA adapts.","marker":"[31]"},{"why":"InternVL generates the one-sentence object descriptions that TeDA embeds and fuses with visual features.","marker":"[2]"},{"why":"OpenCLIP supplies the ViT-L/14 variant used to report TeDA's best results.","marker":"[17]"},{"why":"FixMatch is the source of the high-confidence thresholding idea behind the one-hot pseudo-label construction in Eq. (6).","marker":"[33]"},{"why":"Non-parametric instance discrimination grounds the design choice to treat each query as a unique instance class.","marker":"[43]"},{"why":"MV-CLIP is the strongest training-free baseline for zero-shot 3D retrieval; TeDA is compared against it (including its ground-truth-label variant) and reports higher mAP.","marker":"[34]"},{"why":"Uni3D is a large-scale pretrained training-free baseline that TeDA claims to outperform without 3D pretraining.","marker":"[57]"},{"why":"ULIP-2 is a training-free CLIP-distilled point-cloud baseline used for comparison, including on the Objaverse-LVIS depth-map extension.","marker":"[49]"},{"why":"OpenShape is a training-free 3D representation baseline that TeDA surpasses in the open-set benchmarks.","marker":"[24]"}],"fun_headline_variants":["TeDA: test-time CLIP tuning tops trained 3D retrieval","No 3D training: TeDA aligns CLIP at test time for retrieval","TeDA adapts frozen CLIP to beat trained 3D retrieval","Zero-shot 3D retrieval: test-time CLIP alignment wins","TeDA: CLIP tunes itself for 3D retrieval, no training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole adaptation hinges on the initial CLIP similarity being a trustworthy guide: TeDA assumes the highest-scoring query-target pairs it converts into one-hot pseudo-labels are genuine matches, and if that nearest-neighbor structure is wrong for an unseen category, the iterative updates drag query features toward incorrect targets.","fun_headline_variants_meta":{"raw":{"variants":["TeDA: test-time CLIP tuning tops trained 3D retrieval","No 3D training: TeDA aligns CLIP at test time for retrieval","TeDA adapts frozen CLIP to beat trained 3D retrieval","Zero-shot 3D retrieval: test-time CLIP alignment wins","TeDA: CLIP tunes itself for 3D retrieval, no training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001356,"raw_usage":{"total_tokens":5546,"prompt_tokens":1027,"completion_tokens":4519,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":4435}},"tokens_in":643,"tokens_out":4519,"duration_ms":35064,"temperature":1.0,"reasoning_tokens":4435,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:55:13.413196+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a benchmark with known category labels and run TeDA while logging the pseudo-labels selected at each iteration; if the precision of the selected pseudo-labels against ground-truth matches is at or below chance for unseen categories, or if the reported mAP gains vanish when the same pipeline is run with pseudo-labels replaced by random labels, then the gains are not caused by correct distribution alignment.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the four open-set benchmarks, the 24-view rendering scheme, and the strongest training-required baseline (HGM2R) that TeDA is compared against and surpasses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"InternVL generates the one-sentence object descriptions that TeDA embeds and fuses with visual features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OpenCLIP supplies the ViT-L/14 variant used to report TeDA's best results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FixMatch is the source of the high-confidence thresholding idea behind the one-hot pseudo-label construction in Eq. (6)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Uni3D is a large-scale pretrained training-free baseline that TeDA claims to outperform without 3D pretraining."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ULIP-2 is a training-free CLIP-distilled point-cloud baseline used for comparison, including on the Objaverse-LVIS depth-map extension."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OpenShape is a training-free 3D representation baseline that TeDA surpasses in the open-set benchmarks."}],"review_version":1}