{"id":"39ed79a5-35b2-48a3-91bd-04babc71d227","arxiv_id":"2507.00566","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"PGFA improves zero-shot skeleton action recognition via end-to-end contrastive training and test-time prototype-based reclassification, reporting large gains on NTU-60, NTU-120, and PKU-MMD.","lead":"This paper proposes PGFA, a method for zero-shot skeleton-based action recognition that trains skeleton and text features together and then reclassifies test samples using prototypes computed from pseudo-labeled test data. It reports large gains over prior methods on three benchmarks, but its results rely on test-set hyperparameter tuning and a transductive protocol.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains over SMIE are confounded by test-set transduction and per-dataset α tuned to maximize test accuracy.","rationale":"The central claim is that the prototype-guided text feature alignment strategy yields large absolute gains (22.96%, 12.53%, 18.54%) over SMIE. The most load-bearing condition for this claim is that the comparison isolates the proposed strategy from the transductive use of the test set and from per-dataset hyperparameter tuning. The paper's Table V already shows that the prototype step, applied to an SMIE-like pretrained-and-fixed encoder, adds 13-16 points, indicating that the transductive prototype mechanism is responsible for a large share of the improvement. Because the reported final numbers are obtained with α values chosen per dataset to maximize accuracy (Fig. 6), and because SMIE is evaluated inductively with no test-time adaptation, the headline comparison is not apples-to-apples. The reader's weakest assumption highlighted pseudo-label noise and the gap in Theorem 1; the concern here is closely related but focuses on the causal attribution of the gains to the specific proposed method. The concrete test of augmenting SMIE with the same prototype self-training would settle whether the gains are a generic transduction effect. If the augmented SMIE baseline matches PGFA, then the novelty of the alignment strategy and the magnitude of the claimed improvements are substantially weakened. The paper does disclose the transductive requirement in the Future Work section and provides code, so the concern is addressable rather than fatal; the verdict remains conditional pending a fair baseline and a fixed or properly validated α.","tokens_in":31850,"tokens_out":7150,"duration_ms":84634,"concrete_test":"Run a fair transductive baseline: take the SMIE-pretrained encoder and first-pass text features, apply PGFA's prototype construction (Eqs. 8-11) with the same entropy filter and the same per-dataset α (0.9, 0.4, 1.0), and reclassify the test set. If SMIE+prototype closes most of the gap to PGFA (e.g., reaches over 90% of PGFA's accuracy), then the claimed superiority over SMIE is primarily due to transduction, not to the proposed components. In the same run, fix α=0.5 for all datasets and also select α by cross-validation on seen classes only; if the reported numbers change by more than a few points, the test-set-tuning of α is inflating the results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline gains over SMIE conflate the proposed end-to-end training with a transductive test-time prototype step. In Eqs. (8)-(11), the entire unlabeled test set is used to build class prototypes from entropy-filtered pseudo-labels, and the same test samples are reclassified; SMIE and other baselines are inductive, using only static text features. Section IV-C sets α to 0.9, 0.4, and 1.0 for NTU-60, NTU-120, and PKU-MMD, respectively, i.e., per-dataset values selected from Fig. 6 to maximize test accuracy, so the reported 22.96%, 12.53%, and 18.54% improvements include test-set-tuned transduction. Table V shows the prototype step alone adds 13-16 points even under the SMIE-like Pretraining&Fixed framework, so most of the advantage may come from generic transductive self-training rather than PGFA's alignment or end-to-end training. Theorem 1 (supplementary Eq. 1) assumes support samples are drawn from the true class, whereas Eq. (8) uses pseudo-labels, and it never compares prototype classification against text-feature classification.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PGFA, a paradigm for zero-shot skeleton-based action recognition consisting of two components: (i) an end-to-end cross-modal contrastive training framework that aligns skeleton features with fixed Sentence-BERT text features using a KL-divergence-based bidirectional contrastive loss, and (ii) a prototype-guided text feature alignment strategy applied at test time, in which pseudo-labels are generated from text-feature similarities, entropy filtering selects high-confidence samples per unseen class, per-class prototypes are computed, and the same test samples are reclassified against these prototypes. The authors report large absolute gains over the SMIE baseline on NTU-60, NTU-120, and PKU-MMD, provide a von Mises-Fisher consistency theorem, and include ablations over description types, frameworks, the tolerance margin alpha, pseudo-labeling strategies, one-shot settings, and cross-dataset transfer.","tokens_in":32097,"tokens_out":6031,"duration_ms":74646,"significance":"If the reported results are reproducible under a clean evaluation protocol, PGFA would be a substantial advance: it simplifies zero-shot skeleton action recognition by removing the separate cross-entropy pretraining stage, and it demonstrates that adapting unseen text features with test-time prototypes can substantially improve accuracy. The manuscript has clear strengths: the code is released, the end-to-end contrastive training idea is well motivated, the ablations in Tables III-VI isolate several design choices, and the theoretical statement, while limited, is not circular and is correctly identified as a consistency result under idealized assumptions. The main significance risk is that the headline comparisons are confounded by a transductive test-time procedure and by per-dataset selection of alpha on the test set, so the claimed 22.96%, 12.53%, and 18.54% improvements over SMIE are not yet established as an advantage of the proposed alignment mechanism over generic transductive self-training.","major_comments":[{"comment":"The tolerance margin alpha is selected per dataset by inspecting test accuracy curves and then reused to report the main results, with no held-out validation described. Specifically, alpha is set to 0.9, 0.4, and 1.0 for NTU-60, NTU-120, and PKU-MMD, respectively, and Fig. 6 marks the maxima of the test-set accuracy curves at exactly these values. This means the reported gains over SMIE partially reflect test-set tuning. Please provide a validation-based selection rule (e.g., a small labeled validation split or a fixed alpha across datasets and folds), or alternatively report accuracy as a function of alpha with confidence intervals and clarify which alpha values would be used in a deployment scenario.","section":"Section IV-C and Fig. 6"},{"comment":"The proposed testing strategy is transductive: the entire unlabeled test set is used to build support sets and prototypes, and the same test samples are reclassified using Eq. (11). All baselines in Table I, including SMIE, are inductive and use only static text features. Table V shows that the prototype step alone adds roughly 13-17 points on NTU-60 even under the Pretraining&Fixed framework (73.65 to 87.23) and roughly 10-13 points under the End-to-End framework, so a substantial part of the headline improvement may come from generic transductive self-training rather than from the proposed alignment or end-to-end training. The paper should explicitly state that this is transductive zero-shot inference, compare against transductive baselines such as T3A or self-training with the same entropy filtering on the same learned features, and report the decomposition of the SMIE gap accordingly.","section":"Section III-C, Eqs. (8)-(11), and Table V"},{"comment":"Theorem 1 assumes that the n support samples v_i^(k) are drawn from the true von Mises-Fisher distribution of class k and that the prototype is the normalized mean of these true-class samples. In contrast, Eq. (8) constructs support sets from pseudo-labels that contain errors, and Eq. (9) filters them by entropy. The theorem therefore does not establish that the entropy-filtered pseudo-label prototypes improve classification over the original text features, nor does it analyze how pseudo-label noise propagates into the prototype. It is a standard law-of-large-numbers consistency statement under noiseless labels. Please qualify the theoretical claim accordingly, or extend the analysis to a noise model for pseudo-labels.","section":"Supplementary Materials, Theorem 1 and Eq. (1)"},{"comment":"The problem formulation in Section III-A defines the testing set as Du = {(x_u_i, t_u_i)} but does not state that inference is performed in a transductive batch mode using all unlabeled test skeletons. Algorithm 2 explicitly loops over the entire test set before computing prototypes, and the future-work paragraph in Section V acknowledges that this is unsuitable for online scenarios. The paper should state the transductive assumption in the problem formulation, because the validity of the comparison with inductive baselines depends on this assumption being explicit and on the baselines being adapted or clearly labeled as not having access to the unlabeled test distribution.","section":"Section III-A and Algorithm 2"}],"minor_comments":[{"comment":"In the paragraph on Complete Description, 'we follow the approach of SIME [32]' should read 'SMIE'.","section":"Section III-B"},{"comment":"The text above Eq. (9) says 'we create a final support set Zk for each seen class k', but this should refer to unseen classes; the notation and surrounding discussion are about unseen classes.","section":"Section III-C, Eq. (9)"},{"comment":"The table captions contain the typo 'COMPARSION' and should be 'COMPARISON'.","section":"Table I and Table II captions"},{"comment":"In Setting I, the average accuracy over the three class splits is reported without standard deviations; please report the per-fold results or standard deviations, especially because the alpha values are selected on the aggregate test accuracy.","section":"Section IV-D"},{"comment":"The weighted pseudo-labeling formula in the ablation study sums over all test samples Du, whereas the default prototype in Eq. (10) is restricted to the per-class support set Zk; please clarify whether the weighted variant also uses entropy filtering or whether it is a deliberately different aggregation over the full test set.","section":"Section IV-E, weighted pseudo-labeling formula"},{"comment":"The notation P(v|class k) in Eq. (12) denotes the von Mises-Fisher density value rather than a calibrated class posterior; please clarify this to avoid the impression that the theorem establishes a probabilistic guarantee about classification probabilities.","section":"Theorem 1 statement"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to be of interest to the TIP audience, and the core training idea is simple and plausible. However, the evaluation protocol needs substantial rework before publication: the alpha selection on test data and the absence of transductive baselines are load-bearing for the headline claims. I would encourage the editor to seek a revision with a clean validation protocol and explicit transductive comparisons rather than rejecting outright, because the presented ablation structure suggests the authors have the infrastructure to provide those controls."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: real method, inflated headline numbers. The end-to-end contrastive training is a genuine, if modest, advance over the two-stage baselines in this niche, and the test-time prototype step is a clean idea. But the reported gains over SMIE are mostly the gain from transduction plus test-set-tuned α, not from the alignment strategy per se.\n\nWhat is actually new: applying CLIP-style contrastive alignment to skeleton ZSL, and using entropy-filtered pseudo-labels to build class prototypes from the unlabeled test set and replace text features at inference. The paper is well-ablated: description types, LLM choices, pseudo-label strategies, α sensitivity, computational cost, plus one-shot and cross-dataset extensions. Code is public. Theorem 1 is not circular; it is a consistency statement for von Mises–Fisher data, though it assumes the support samples come from the true class and never compares prototype classification directly with text-feature classification. The end-to-end training itself buys a respectable 5–7 points over the Pretraining&Fixed baseline.\n\nSoft spots, in proportion. First, the big gains are transductive. Table V shows the prototype step alone adds 13–16 points even under the SMIE-style Pretraining&Fixed framework, while the end-to-end training adds only 5–7. The paper never says the testing strategy is transductive, never states it in the problem formulation, and never compares against transductive baselines. So the 22.96%, 12.53%, and 18.54% improvements over SMIE conflate transduction with the proposed alignment. The authors do flag in Future Work that the method requires all test samples in advance, so they know it; the evaluation protocol just doesn't own it.\n\nSecond, α is set to 0.9, 0.4, and 1.0 for the three datasets, and those values are peak-picked from Fig. 6, which plots test accuracy against α. No held-out validation is described. The authors' robustness argument (α≥0.1 gives over 95% of peak) blunts the damage, but the principle is still a test-set-tuned hyperparameter, and it interacts with the transduction.\n\nThird, Theorem 1 is too weak to carry the weight the paper puts on it. It assumes samples are drawn from the true class, while Eq. (8) uses pseudo-labels. It never argues why prototypes beat text features; it only says that under the vMF model, nearest-center classification is asymptotically consistent. That is fine as a sanity check, but it does not support the effectiveness claim.\n\nWho it is for: researchers in zero-shot skeleton action recognition and test-time prototype adaptation. The method is practical and clearly described; the empirical claims need re-evaluation. A serious referee should see this paper, but the revision must either disclose and benchmark the transductive protocol or provide an inductive variant, and α should be chosen on a validation split. I would not cite the headline numbers as they stand, but the code and the end-to-end training framework are worth knowing about.","headline":"A solid method paper with real contributions, but the headline gains are inflated by undisclosed test-time transduction and per-dataset α tuned on the test set.","tokens_in":32570,"tokens_out":3589,"would_cite":false,"duration_ms":48140,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Zero-shot skeleton action recognition improves when unseen-class text features are replaced by prototypes built from confident test samples.","keywords":["zero-shot action recognition","skeleton-based action recognition","prototype-guided feature alignment","cross-modal contrastive learning","pseudo-label entropy filtering","text-skeleton alignment","distribution discrepancy","prototypical learning"],"falsifier":"Corrupt the pseudo-labels used to build the prototype support set, for example by randomly flipping an increasing fraction of labels to other unseen classes, and measure the final accuracy; if prototype reclassification does not degrade smoothly with this noise, or if a near-chance initial classifier still produces large gains, then the mechanism attributed to prototype alignment is not doing the claimed work.","tokens_in":31684,"feed_emoji":"🦴","tokens_out":7980,"duration_ms":80741,"temperature":0.7,"pith_summary":"The paper argues that zero-shot skeleton-based action recognition suffers from two correctable problems: skeleton features trained by a frozen cross-entropy pretrained encoder are not compact enough, and the text features of unseen classes are misaligned with skeleton features at test time. It proposes an end-to-end cross-modal contrastive training framework for the first problem, and a prototype-guided text feature alignment strategy for the second, in which each unseen class is represented by a prototype computed from the highest-confidence pseudo-labeled test skeletons. On NTU-60, NTU-120, and PKU-MMD, the full method reaches 93.17%, 71.38%, and 87.80%, which are absolute gains of 22.96, 12.53, and 18.54 points over the strongest prior baseline. The central point is that adjusting the class representations at test time, not just improving training, is what drives large zero-shot gains.","feed_headline":"Prototype features lift zero-shot skeleton action accuracy 23 points","feed_subtitle":"Unseen actions are classified by high-confidence test-skeleton prototypes instead of static text features, closing the alignment gap.","key_machinery":"The central mechanism is the prototype-guided text feature alignment strategy (Eqs. 8-11): for each unseen class k, build a normalized support set S_k from test skeleton features whose pseudo-label is k, filter by prediction entropy to a final support set Z_k, and define the class prototype c_{u,k} as the normalized centroid of Z_k, using it in place of the text feature w_{u,k} in the softmax similarity classifier. The supporting training-side mechanism is an end-to-end KL-divergence contrastive loss over a batch, where every sample in a class is a positive match for that class's text feature; this is what produces the intra-class compactness on which the entropy filter and prototype centroids rely.","core_discovery":"The central claim is that alignment bias between skeleton features and unseen-class text features can be reduced by substituting prototype features for text features at test time. Given the unlabeled test set, the method first pseudo-labels every test skeleton by cosine similarity to the original unseen-class text features, then for each class retains only the pseudo-labeled skeletons whose prediction entropy falls below a per-class threshold, and sets the class prototype to the normalized centroid of those skeletons; if the filtered set is empty, the original text feature is retained as a fallback. Final predictions are made by cosine similarity to these prototypes. Theorem 1 states that if normalized skeleton features follow von Mises–Fisher distributions with a common concentration parameter, then as the support set grows, assigning a sample to the prototype with highest cosine similarity is equivalent to assigning it to the class with highest class-conditional probability. The paper also claims that training the skeleton encoder and projection layer end-to-end with a bidirectional KL-divergence contrastive loss, instead of pretraining with cross-entropy and freezing the encoder, produces more compact skeleton features, and that these two changes together explain the reported improvements over previous methods.","pith_inferences":["Beyond the paper, the same prototype-substitution mechanism should transfer to zero-shot image or video classification, where unseen-class text features are also known to sit off-center relative to visual features.","The paper's Theorem 1 assumes clean pseudo-labels, so a natural extension is to bound final accuracy as a function of initial label noise; the entropy filter softens but does not remove that dependence.","Because the method needs all test samples before reclassifying, a practical follow-up is an online prototype bank that updates as samples arrive; comparing its accuracy with the batch version would quantify the transductive advantage.","A testable implication of the paper's reasoning is that iterative refinement, re-pseudo-labeling and re-centering prototypes several times, should further close the alignment gap whenever the initial text classifier is better than chance."],"forward_implications":["If the end-to-end training claim is correct, zero-shot skeleton recognition can skip cross-entropy pretraining entirely, saving a training stage and reducing parameters (1.00M versus 2.38M for the compared baseline).","If the prototype replacement claim is correct, any static text feature can be improved by test-time adaptation whenever the unlabeled test set is available as a whole.","The entropy filter is robust across a wide range of the tolerance margin alpha, so the strategy does not require precise hyperparameter tuning to produce most of its gain.","The same framework extends directly to one-shot recognition, where a single labeled skeleton per unseen class replaces the text feature, giving competitive results on NTU-60, NTU-120, and PKU-MMD.","Complete and skeleton-focused LLM-generated action descriptions consistently beat bare class names, so text quality and test-time prototype alignment compound rather than substitute for each other."],"supporting_citations":[{"why":"Supplies the SMIE baseline, the three-fold class-split evaluation setting, and the complete-description text inputs that PGFA is compared against.","marker":"[32]"},{"why":"Provides the bidirectional softmax similarity and contrastive training scheme that PGFA adapts into a KL-divergence loss for skeleton-text alignment.","marker":"[35]"},{"why":"Supplies the prototypical learning idea of representing a class by the centroid of its feature support set.","marker":"[39]"},{"why":"Provides the test-time pseudo-prototype adjustment idea that the prototype-guided text feature alignment strategy extends to unseen classes.","marker":"[43]"},{"why":"Offers prototype-based alignment used in domain adaptation, motivating the use of prototypes to correct distribution shift.","marker":"[42]"},{"why":"Supplies the fixed Sentence-BERT text encoder used to extract the 768-dimensional text features for both training and testing.","marker":"[59]"},{"why":"Supplies the von Mises-Fisher distribution assumption on which Theorem 1's proof rests.","marker":"[75]"},{"why":"Supplies the ST-GCN skeleton encoder used in the main experiments and in the comparison with SMIE.","marker":"[17]"}],"fun_headline_variants":["Prototype-guided alignment beats static text features in zero-shot skeleton actions","Test-time prototypes close skeleton-text gap, lift zero-shot accuracy up to 23%","Prototype features reduce test-time alignment bias in zero-shot skeleton action","Prototype-guided alignment yields 23% gain in zero-shot skeleton action"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the unlabeled test set is available all at once and that the initial text-based pseudo-labels are accurate enough that entropy filtering leaves prototypes that are closer to the true skeleton class centers than the original text features are.","fun_headline_variants_meta":{"raw":{"variants":["Prototype-guided alignment beats static text features in zero-shot skeleton actions","Test-time prototypes close skeleton-text gap, lift zero-shot accuracy up to 23%","Prototype features reduce test-time alignment bias in zero-shot skeleton action","Prototype-guided alignment yields 23% gain in zero-shot skeleton action"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000995,"raw_usage":{"total_tokens":4261,"prompt_tokens":1041,"completion_tokens":3220,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":3140}},"tokens_in":657,"tokens_out":3220,"duration_ms":25345,"temperature":1.0,"reasoning_tokens":3140,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:12:29.698602+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Corrupt the pseudo-labels used to build the prototype support set, for example by randomly flipping an increasing fraction of labels to other unseen classes, and measure the final accuracy; if prototype reclassification does not degrade smoothly with this noise, or if a near-chance initial classifier still produces large gains, then the mechanism attributed to prototype alignment is not doing the claimed work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the bidirectional softmax similarity and contrastive training scheme that PGFA adapts into a KL-divergence loss for skeleton-text alignment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the prototypical learning idea of representing a class by the centroid of its feature support set."},{"cited_title":"Rebuffi, A","cited_arxiv_id":null,"evidence_quote":"Provides the test-time pseudo-prototype adjustment idea that the prototype-guided text feature alignment strategy extends to unseen classes."},{"cited_title":"Huang, B","cited_arxiv_id":null,"evidence_quote":"Offers prototype-based alignment used in domain adaptation, motivating the use of prototypes to correct distribution shift."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the fixed Sentence-BERT text encoder used to extract the 768-dimensional text features for both training and testing."},{"cited_title":"Noor and I","cited_arxiv_id":null,"evidence_quote":"Supplies the ST-GCN skeleton encoder used in the main experiments and in the comparison with SMIE."}],"review_version":1}