{"id":"ff718599-2921-400d-8d5e-be1dd5b741ae","arxiv_id":"2504.16640","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A pseudo-labeling, pose-based Transformer gives accuracy close to or slightly better than supervised training on WLASL-100, but the reported improvements are small, noisy, and not statistically tested.","lead":"This paper applies a standard pseudo-labeling trick to train a sign language recognition model using pose data and very few labeled videos. The authors report that their semi-supervised model sometimes beats a fully supervised baseline, but the gains are small and inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The pseudo-label selection rule is ambiguous between Algorithm 1 and Section 3.3, and the duplicated rows in Table 2 prevent the central outperformance claim from being verified.","rationale":"The reader's weakest assumption was that high-confidence pseudo-labels are reliable and that the labeled subset is random and class-balanced. My stress-test confirms this is the right weak point, and sharpens it: the paper contains two incompatible descriptions of how pseudo-labels are selected, and the reported numbers contain anomalies that suggest either a typo, a data-selection artifact, or an unstated policy. The duplicated 1% and 5% rows across every class count, plus the non-monotonic 5-class accuracy curve, cannot be dismissed as ordinary training noise. These issues do not by themselves falsify the idea that pseudo-labeling can help pose-based SLR, so a rejection would be too strong. However, the lack of a reproducible algorithm and the anomalous table entries mean the current evidence cannot support the strong contribution claim. A conditional acceptance with a request for clarification, code, and reproducibility experiments is the correct outcome, which matches the reader's verdict.","tokens_in":9848,"tokens_out":3422,"duration_ms":33028,"concrete_test":"Implement both selection policies on WLASL-100 with the paper's pose preprocessing and normalization, using the same train/validation/test split and several random labeled subsets. Rerun the 5-class, 40-class, and 100-class conditions at 1%, 5%, 10%, and 75% labeled data with at least three seeds. If the per-class rule is needed to reproduce Table 2, or if the 5-class SSL accuracy at 10% is not robustly higher than at 1% and 5% under the global-max rule, the central outperformance claim is not supported. Also request the authors' exact table logs to verify whether the identical 1% and 5% rows are duplicated entries.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim, that SSLR significantly outperforms fully supervised learning with the same labeled data, depends entirely on the pseudo-labeling procedure producing labels that help rather than hurt. Section 3.3 states that for each class the highest-confidence unlabeled sample is selected, while Algorithm 1 lines 5-7 select a single global maximum. These are different policies: global maximum selection repeatedly picks easy samples from the same classes, generating imbalanced pseudo-labels; per-class selection covers more classes and changes measured accuracy. Which policy produced Tables 1-2 is not stated. The labeled subset is also unspecified, since the paper does not describe whether it is random, class-balanced, or seed-controlled. Table 2 shows the 1% and 5% rows identical for every class count, and the 5-class SSL result drops from 60% at 1% and 5% labeled data to 50% at 10%, while FSL jumps from 25% to 65%. This is precisely the non-monotonic signature of self-reinforced label error or of a data-selection artifact. The paper's own future-work sentence concedes that uncertainty-thresholding and class-balanced pseudo-labeling are needed, which is a direct admission that the current rule lacks those safeguards. Because the headline claim is supported only by these tables, with no code, seeds, or error bars, a reader cannot tell whether the claimed advantage is a property of the method or of an unstated implementation choice.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SSLR, a semi-supervised pseudo-labeling method for isolated sign language recognition. Skeletal pose keypoints from the WLASL-100 dataset are fed into a SPOTER-style Transformer, which is first trained on a small labeled subset, then used to assign pseudo-labels to unlabeled samples; the highest-confidence pseudo-labeled samples are iteratively added to the training set. The authors compare this method against a fully supervised baseline trained on the same amount of labeled data, varying both the percentage of labeled data and the number of sign classes, and report accuracy tables, an ablation study of normalization and augmentation, and a figure showing the effect of training cycles. The abstract claims that SSL outperforms supervised learning with less labeled data in many cases, while the contribution list makes the stronger claim that SSLR significantly outperforms fully supervised methods even with the same amount of labeled data.","tokens_in":10057,"tokens_out":5032,"duration_ms":47850,"significance":"If the strong claim were substantiated, the paper would be practically useful for low-resource sign language recognition, because pseudo-labeling with a pose Transformer that beats supervised training from the same labeled set would directly address annotation scarcity. The evaluation on a held-out test set is a genuine strength and avoids the circularity of evaluating on the pseudo-labeled data itself. However, the claimed advantage is not established: the contribution bullet is contradicted by several rows of the paper's own tables, there are no error bars or significance tests, the pseudo-label selection rule is ambiguous between the text and Algorithm 1, and the reported tables contain duplicated rows that prevent verification. The method is a standard application of pseudo-labeling to pose-based SLR rather than a new algorithmic idea, so the contribution rests entirely on the empirical evidence, which is currently not reliable enough to support the headline claim.","major_comments":[{"comment":"The pseudo-label selection rule is described inconsistently. Section 3.3 states that \"for each class, the pseudo-labeled sample with the highest confidence is selected,\" while Algorithm 1 line 5 performs a single global maximum over all classes. These are different policies: global maximum selection tends to repeatedly select easy samples from the most confident classes, creating imbalanced pseudo-labels, whereas per-class selection spreads labels across classes. The authors must state which policy generated Tables 1 and 2, and justify the choice; without this clarification the reported accuracies cannot be interpreted.","section":"Section 3.3 / Algorithm 1"},{"comment":"The rows labeled 1% and 5% in Table 2 are identical for every class count (5, 20, 40, 60, 80, 100), and the same duplication appears for 100 classes in Table 1. This is either a copy-and-paste error or an unexplained coincidence, and it makes the table unreliable as evidence. Additionally, no error bars, repeated-seed results, or significance tests are reported, so the word \"significantly\" in the contribution list is not supported by the data. The authors should correct the table and provide either confidence intervals or a clear statement of the number of runs and their variance.","section":"Table 2 / Section 4"},{"comment":"The contribution bullet claims that \"SSLR significantly outperforms fully supervised methods, even with the same amount of labeled data,\" but the paper's own results contradict this. For example, with 100 classes at 25% labeled data FSL achieves 22.5% while SSL achieves 21.3%; with 5 classes at 10% labeled data FSL achieves 65.0% while SSL achieves 50.0%; and with 20 classes at 50% labeled data FSL achieves 60.0% while SSL achieves 53.8%. Across Table 2, SSL is worse than FSL in a substantial fraction of configurations. The central empirical claim must be either substantially softened to \"competitive in some cases\" or supported with a statistically valid analysis that explains why certain configurations favor SSL and others do not.","section":"Section 1 / Tables 1 and 2"},{"comment":"The construction of the labeled subset L is not described. The paper does not state whether L is chosen randomly, whether it is class-balanced, or whether seeds are controlled, and it does not specify how the split is performed for the different class counts. This matters because the pseudo-labeling method is sensitive to the composition of the initial labeled set, and the non-monotonic result in Table 2 for 5 classes (SSL drops from 60% at 1% and 5% labeled data to 50% at 10%) suggests either label-error reinforcement or an artifact of how L was selected. The authors should specify the exact sampling procedure and, ideally, report results over multiple random splits.","section":"Section 3.3 / Section 4 (Dataset split)"}],"minor_comments":[{"comment":"There is a typo in the results paragraph: \"the FFL model\" should be \"the FSL model.\"","section":"Section 4"},{"comment":"The terms \"unannotated\" and \"unlabeled\" are used interchangeably; the paper should pick one term and use it consistently.","section":"Throughout"},{"comment":"Line 5, \"P = max(P)\", is underspecified: the authors should indicate whether the maximum is taken over all samples, over each class, or over the probability vector of a single sample, and should name the function as argmax when the selected sample is returned.","section":"Algorithm 1"},{"comment":"The training configuration is not reported in enough detail: \"Stochastic Gradient Descent optimizer\" is not specific, and no learning rate, batch size, number of epochs, or stopping criterion for Algorithm 1's \"until convergence or maximum iterations\" are given.","section":"Section 3.2 / Section 3.3"},{"comment":"The caption of Table 3 says \"Ablation study results with normalization and different augmentation techniques,\" but the first row removes all techniques; the caption should clarify that the first row is the no-preprocessing baseline.","section":"Section 4.1 / Table 3"},{"comment":"The sentence \"the SSL model consistently outperformed the FSL, especially when trained with 75% of the labeled data\" is not consistent with Table 2, since at 75% labeled data SSL is worse than FSL for 60 classes (49.7 vs. 50.9) and for 100 classes the margin is 0.3 percentage points in favor of SSL (48.4 vs. 48.1).","section":"Section 4"},{"comment":"The text says the performance converges early for small labeled-data percentages and continues to improve for larger percentages, but no numerical convergence criterion or quantitative support is given, so the claim is difficult to assess from the figure alone.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a straightforward application of pseudo-labeling to pose-based sign language recognition; its contribution rests almost entirely on the empirical claim that it beats supervised learning with the same labeled data. As submitted, that claim is contradicted by the paper's own tables, and the duplicated rows plus the ambiguous selection rule make the experiments unverifiable. I think a major revision is appropriate rather than rejection, because the central problems (softening the claims, clarifying the algorithm, rerunning with proper statistics) are fixable within the scope of the paper. However, if the authors cannot resolve the table inconsistencies or provide credible evidence for the same-labeled-data advantage, the contribution becomes too incremental for this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a straightforward pseudo-labeling baseline applied to a pose-based Transformer for isolated sign language recognition, and it is a useful incremental data point. The headline claim in the contribution bullet—that SSLR 'significantly outperforms' fully supervised methods with the same labeled data—is not supported by the numbers. The abstract's weaker claim is technically true, but barely.\n\nWhat is genuinely useful: the paper targets a real problem (annotation scarcity in sign language), uses pose inputs that are cheap and signer-relative, and includes an ablation showing that normalization improves test accuracy from 46.2% to 58.5% on a 20-class subset. That ablation is the most informative piece in the paper. The idea of applying pseudo-labeling to a Transformer backbone is not new, but the specific combination with pose normalization on WLASL-100 with varying labeled fractions is not in the cited literature.\n\nThe soft spots are in the evaluation and reporting. Across 100 classes, SSL beats FSL at 10%, 50%, and 75% labeled data with margins of 1.6, 1.5, and 0.3 points; at 25% FSL wins by 1.2. No error bars or significance tests are given, so 'significantly' is unjustified. Table 2 has identical 1% and 5% rows for every class count, which looks like a copy/paste artifact, and the 5-class SSL accuracy drops from 60% at 1%/5% to 50% at 10% while FSL jumps from 25% to 65%—a non-monotonic pattern that suggests pseudo-label noise or a data-selection quirk. There is also a real inconsistency between Algorithm 1, which selects the single global maximum confidence sample, and Section 3.3, which describes selecting the highest-confidence sample per class. These rules produce different label distributions and can easily explain the observed differences. No code, seeds, hyperparameters, or description of how labeled subsets were chosen are provided. The authors themselves say in the conclusion that uncertainty thresholding and class-balanced pseudo-labeling are needed, which is a fair admission that the current rule is fragile.\n\nThis paper is not a reject out of hand. The question is worth asking, and the ablation is a useful sanity check. But the current evidence does not support the central claim. A serious referee should ask for the actual selection rule, corrected tables, variability estimates, and a toned-down contribution statement. I would send it to review, expecting a substantially revised version to come back.","headline":"A reasonable pseudo-labeling baseline for pose-based sign language recognition whose reported gains don't support the 'significantly outperforms' claim; worth a careful referee, not a desk reject.","tokens_in":10675,"tokens_out":3630,"would_cite":false,"duration_ms":29163,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes SSLR, a semi-supervised sign language recognition method that pseudo-labels unlabeled pose clips with a Transformer, and reports it matching or beating fully supervised training on WLASL-100 across most labeled-data…","keywords":["semi-supervised learning","isolated sign language recognition","pseudo-labeling","pose-based recognition","Transformer","WLASL-100","skeletal joint points","low-resource recognition"],"falsifier":"In the 5-class, 10%-labeled row of Table 2, SSL drops from 60% at 5% labeled to 50% at 10% while FSL rises from 25% to 65%; re-running that single cell with several random labeled subsets and logging the true labels of each added pseudo-label would show whether the loop is injecting systematically wrong examples. The claim also fails if an audit of the training loop shows that Algorithm 1's single global maximum, not Section 3.3's one-per-class selection, is what is actually executed and that this choice changes which clips are added.","tokens_in":9534,"feed_emoji":"🤟","tokens_out":8673,"duration_ms":74811,"temperature":0.7,"pith_summary":"The paper sets out to show that isolated sign language recognition does not need a fully annotated corpus: a pose Transformer trained on a small labeled subset can confidently label unlabeled clips, add them to its own training set, and iteratively improve. On the WLASL-100 benchmark the proposed SSLR procedure matches or beats a fully supervised model trained on the same labeled portions in most settings, with the clearest wins at 10%, 50%, and 75% labeled data. The significance is practical: annotated sign data is expensive and scarce, so a loop that converts an unlabeled archive into training signal could lower the cost of building sign recognizers. At the smallest labeled fractions both methods sit near the floor, and one 5-class condition shows a non-monotonic drop, so the advantage is not uniform.","feed_headline":"Pseudo-labeling beats fully supervised sign-language recognition","feed_subtitle":"On WLASL-100, the pseudo-label loop matches or beats supervised training at 10%, 50%, and 75% labeled data.","key_machinery":"The engine is an iterative pseudo-labeling loop wrapped around a pose Transformer. The backbone, taken from SPOTER, encodes 54 body-joint landmarks per frame as a 108-dimensional vector, adds positional encoding, and uses six encoder and six decoder blocks with nine-head self-attention; a single learned \"class query\" is decoded and passed through softmax to produce the sign label. Pseudo-labeling is the learning mechanism: after each round, the model labels unlabeled clips and the highest-confidence predictions are moved into the labeled set and used to retrain, repeating until all clips are consumed. The supporting machinery is pose preprocessing: normalizing joint coordinates into the signing space and applying Gaussian noise, in-plane and arm-level rotation, and shearing, which the ablation credits with the largest portion of the accuracy gain.","core_discovery":"SSLR's central claim is that self-training with pseudo-labels turns the same pose Transformer into a better recognizer than supervised training on the same labeled set. The loop starts from labeled clips L, trains the Transformer, predicts softmax scores for unlabeled clips U, moves high-confidence predictions into L, retrains, and repeats until U is empty. On WLASL-100 the authors report SSL accuracies of 7.0, 7.0, 9.7, 21.3, 36.8, and 48.4 for 1%, 5%, 10%, 25%, 50%, and 75% labeled data, against FSL's 7.0, 7.0, 8.1, 22.5, 35.3, and 48.1; SSL wins at 10%, 50%, and 75% and matches at the floor. The authors state the contribution as SSLR significantly outperforming fully supervised methods even with the same amount of labeled data.","pith_inferences":["If the strong version holds, the same confidence-selection loop is a plausible recipe for other pose-based action recognition tasks where a large unlabeled archive exists alongside a small labeled seed; a fair comparison with standard self-training and confidence-threshold baselines on the same backbone would place the gain, and the paper does not report such baselines.","The paper describes two different selection rules: Section 3.3 says the highest-confidence sample is chosen per class, while Algorithm 1 selects a single global maximum; fixing this ambiguity matters because per-class selection preserves rare signs, whereas global selection would bias the added pseudo-labels toward frequent classes.","A direct test of the mechanism is to log whether early-round pseudo-labels are correct; the 5-class, 10%-labeled drop (SSL 50 vs FSL 65) is exactly the signature of injected wrong labels, so re-running that cell with random labeled subsets would reveal whether the drop is real or a selection artifact."],"forward_implications":["At 100 classes, SSLR surpasses FSL with 10%, 50%, and 75% labeled data (9.7 vs 8.1, 36.8 vs 35.3, and 48.4 vs 48.1), so the pseudo-label loop gains advantage exactly where labeled data is moderately scarce.","Across 5, 20, 40, 60, 80, and 100 classes at 75% labeled data, SSL matches or exceeds FSL at every class count and opens its largest gap at 40 classes (47.9 vs 42.9).","On 5 classes, SSL reaches 60% accuracy from just 1% and 5% labeled data while FSL stays at 25%, indicating the loop can bootstrap in very low label regimes.","The ablation attributes a jump from 46.2% to 63.1% test accuracy to normalization with signing-space projection plus augmentation, so the pseudo-labeling claim depends on those preprocessing steps."],"supporting_citations":[{"why":"Supplies the WLASL-100 dataset, its 100 sign classes, and the official train/validation/test split used for every reported comparison.","marker":"[23]"},{"why":"Provides the SPOTER pose Transformer architecture that serves as the backbone for both SSL and FSL baselines, plus the rotation augmentation style.","marker":"[11]"},{"why":"Provides the signing-space normalization that projects pose coordinates relative to the signer, which the ablation shows is the largest single accuracy contributor.","marker":"[6]"},{"why":"Cited as the semi-supervised training paradigm in which a model's high-confidence predictions on unlabeled samples are used to expand the labeled set.","marker":"[34]"}],"fun_headline_variants":["Pseudo-labeling wins with less labeled sign data","Semi-supervised pseudo-labels boost sign recognition accuracy","Pseudo-label loop outperforms supervised training on WLASL","SSL sign recognition: beats supervision with less data","Self-training with pseudo-labels improves sign language AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the model's highest-confidence predictions on unlabeled clips are usually correct, and that the initial labeled subset is representative enough that the first model has something reliable to teach.","fun_headline_variants_meta":{"raw":{"variants":["Pseudo-labeling wins with less labeled sign data","Semi-supervised pseudo-labels boost sign recognition accuracy","Pseudo-label loop outperforms supervised training on WLASL","SSL sign recognition: beats supervision with less data","Self-training with pseudo-labels improves sign language AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2602,"prompt_tokens":927,"completion_tokens":1675,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":1597}},"tokens_in":543,"tokens_out":1675,"duration_ms":15689,"temperature":1.0,"reasoning_tokens":1597,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:58:06.700954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the 5-class, 10%-labeled row of Table 2, SSL drops from 60% at 5% labeled to 50% at 10% while FSL rises from 25% to 65%; re-running that single cell with several random labeled subsets and logging the true labels of each added pseudo-label would show whether the loop is injecting systematically wrong examples. The claim also fails if an audit of the training loop shows that Algorithm 1's single global maximum, not Section 3.3's one-per-class selection, is what is actually executed and that this choice changes which clips are added.","supporting_citations":[{"cited_title":"Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison","cited_arxiv_id":null,"evidence_quote":"Supplies the WLASL-100 dataset, its 100 sign classes, and the official train/validation/test split used for every reported comparison."},{"cited_title":"Sign Pose-based Trans- former for Word-level Sign Language Recognition","cited_arxiv_id":null,"evidence_quote":"Provides the SPOTER pose Transformer architecture that serves as the backbone for both SSL and FSL baselines, plus the rotation augmentation style."},{"cited_title":"The use of signing space in a shared sign language of Australia","cited_arxiv_id":null,"evidence_quote":"Provides the signing-space normalization that projects pose coordinates relative to the signer, which the ablation shows is the largest single accuracy contributor."},{"cited_title":"Semi-supervised action recognition with temporal con- trastive learning","cited_arxiv_id":null,"evidence_quote":"Cited as the semi-supervised training paradigm in which a model's high-confidence predictions on unlabeled samples are used to expand the labeled set."}],"review_version":1}