{"id":"808400f7-552e-4d7a-87d1-b517d0aba1ba","arxiv_id":"2506.21592","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"SignBart is a BART-style transformer that encodes skeleton x and y coordinates separately and reaches 96.04% on LSA-64, but its claim of outperforming larger models is not supported by its own comparison table.","lead":"A BART-style model that processes skeleton x and y coordinates separately reaches 96.04% accuracy on the LSA-64 sign language dataset with under 750,000 parameters. The paper's headline claim that it beats all larger models conflicts with its own table, where SPOTER scores 100%.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central 'significantly outperforming' claim is contradicted by the paper's own Tables 2 and 4: three >1M-parameter baselines beat SignBart on LSA-64, and NLA-SLR beats it on WLASL-100/300.","rationale":"The reader's REJECT verdict is sound. The reader's weakest assumption was the unspecified train/test split protocol, which is a real reproducibility problem. I agree with that concern, but the more direct, load-bearing issue is internal: the central superiority claim is contradicted by the paper's own Tables 2 and 4. Even granting every reported number, the abstract's assertion that SignBart 'significantly outperforms previous models with over one million parameters' cannot be true because three baselines with over one million parameters—SPOTER, HWGATE, and SL-GCN—report higher LSA-64 accuracy. Similarly, 'superior generalization' on WLASL is inconsistent with SignBart trailing NLA-SLR on WLASL-100 and WLASL-300. This is not a disagreement with external consensus; it is an internal inconsistency between the paper's claims and its evidence. Therefore REJECT remains appropriate, and no verdict change is needed. I would only strengthen the reader's rationale by citing this internal contradiction alongside the split-protocol issue.","tokens_in":10110,"tokens_out":4952,"duration_ms":48489,"concrete_test":"Recompute the comparisons from the paper's own tables: list every baseline in Tables 2 and 4 with parameter count over one million, and count how many exceed SignBart's 96.04% on LSA-64 and how many exceed SignBart's WLASL-100/300 scores. If SPOTER, HWGATE, and SL-GCN are all higher on LSA-64, the abstract's 'significantly outperforming' is contradicted by the paper's data; the verdict should remain REJECT unless the claims are revised to the narrower 'competitive with fewer parameters.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SignBart 'significantly outperforms previous models with over one million parameters' (Abstract) and shows 'superior generalization' on WLASL. This is falsified by the paper's own reported numbers. Table 4 lists SPOTER at 100%, HWGATE at 98.59%, and SL-GCN at 98.13%, all with >1M parameters, all above SignBart's 96.04%. The paragraph in §4.3 quietly narrows the comparison to ST-GCN (92.81%) and 3DGCN (94.84%), but the abstract and conclusion do not carry that qualification. On WLASL (Table 2), SignBart scores 78.00% and 78.50% on WLASL-100 and WLASL-300, while NLA-SLR scores 93.08% and 87.33%; 'superior generalization' holds only for WLASL-1000/2000, not for the full set of subsets. If the claim is meant as parameter-efficiency rather than accuracy, the paper never defines it that way, and the stated 'outperforms' is inaccurate. This is a claim-level inconsistency independent of the split-protocol issue: even accepting all reported numbers, the headline does not follow.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SignBart, a skeleton-based encoder-decoder transformer for isolated sign language recognition. The encoder processes x-coordinates and the decoder processes y-coordinates, with cross-attention linking the two streams; inputs are MediaPipe keypoints normalized per body part, and the model is evaluated on LSA-64, WLASL, and ASL-Citizen. The abstract claims 96.04% accuracy on LSA-64 with 749,888 parameters, 'significantly outperforming previous models with over one million parameters,' and superior generalization on WLASL and ASL-Citizen. Ablation studies examine the effects of coordinate projection, normalization, and skeleton components.","tokens_in":10452,"tokens_out":7365,"duration_ms":71800,"significance":"The proposed architecture is an interesting lightweight alternative in skeleton-based ISLR: separating x and y coordinate streams and reconnecting them through cross-attention is a non-standard design, and the paper reports results on three datasets together with ablations that isolate useful components. If the claims were fully supported, the parameter-efficiency result would be a useful contribution. However, the central comparative claims are contradicted by the paper's own tables, and the evaluation protocol is not specified, so the significance cannot currently be assessed at face value. The paper also provides no code or data release, which limits reproducibility and verification.","major_comments":[{"comment":"The claim that SignBart 'significantly outperforms previous models with over one million parameters' is contradicted by the reported numbers. In Table 4, SPOTER (100%, 5,918,848 parameters), HWGATE (98.59%, 10,758,354), and SL-GCN (98.13%, 4,872,306) all exceed SignBart's 96.04% on LSA-64. In Table 2, NLA-SLR outperforms SignBart on WLASL-100 (93.08% vs. 78.00%) and WLASL-300 (87.33% vs. 78.50%). The abstract and conclusion therefore need to be reworded to claim competitive accuracy at a lower parameter count rather than superior accuracy; alternatively, the empirical comparisons must be redone under a common protocol.","section":"Abstract; §4.3; Tables 2 and 4"},{"comment":"No train/validation/test split protocol is stated for any dataset. LSA-64, WLASL, and ASL-Citizen all have multiple signers and multiple class subsets, so the comparisons in Tables 2–6 are only meaningful if SignBart uses the same splits as the cited baselines. The paper must specify the exact split (for example, signer-independent vs. random), the subsetting procedure, and the number of runs, and should report mean and standard deviation over multiple seeds.","section":"§4.2–§4.3; Tables 2–6"},{"comment":"The text states that the model achieves a 5.73% increase on WLASL-300 and a 9.69% increase on WLASL-2000. Table 2 shows the WLASL-300 accuracy is 78.50% for SignBart versus 87.33% for NLA-SLR, a decrease rather than an increase; the 5.73% improvement corresponds to WLASL-1000 (81.45% vs. 75.72%). This numerical error weakens the 'superior generalization' claim and should be corrected.","section":"§4.3; Table 2"},{"comment":"The ASL-Citizen comparison is incomplete: SignBart's Rec@5 is reported as '-', so the comparison against I3D and ST-GCN on Rec@5 cannot be evaluated. Additionally, the statement that there has been no benchmark comparison on ASL-Citizen should be checked against the evaluation protocol used in the original ASL-Citizen paper before asserting that this is the first such comparison.","section":"§4.3; Table 6"},{"comment":"The conceptual motivation is presented as a 'coordinate theory' in which x and y are independent but interdependent, but the cited reference [34] concerns motor coordination and does not support this specific claim. The separation of x and y streams should be framed explicitly as a design hypothesis with references or preliminary evidence, rather than as an established theory.","section":"§3.1; §3.2"}],"minor_comments":[{"comment":"The sentence 'With separate encoding, the encoder will encode the x coordinates while the decoder will encode the y coordinates' appears twice in the same paragraph; one occurrence should be removed.","section":"§3.2"},{"comment":"The LSA-64 dataset is cited as [9] in the table captions, but [9] is a modified LSTM paper; the correct reference for LSA-64 is [35].","section":"Tables 4, 7, 8"},{"comment":"The paper states that only 6 of the 33 MediaPipe body keypoints are used, but does not specify which 6 keypoints are retained; this is needed for reproducibility.","section":"§3.3"},{"comment":"The model is called 'BART-based,' but it does not use BART's denoising pretraining objective; using the name 'BART' without qualification may mislead readers about the relationship to the original BART model.","section":"§3.2; Title"},{"comment":"Ablation results are reported as single accuracy numbers without error bars, confidence intervals, or significance tests, so the relative importance of the ablated components is not statistically established.","section":"Tables 7–9"},{"comment":"No code, training configuration, or data availability statement is provided; given the number of unspecified preprocessing details, this substantially limits reproducibility.","section":"General"}],"recommendation":"reject","confidential_remarks":"The paper's advertised central claim is directly contradicted by its own reported results, and the evaluation protocol is not described well enough to verify any of the numerical comparisons. Even setting aside the missing split details, the abstract and conclusion overstate the findings on LSA-64 and WLASL. In my view, the manuscript would require a fundamental reframing and a full experimental protocol before it could be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely new: splitting skeleton x and y coordinate streams across an encoder-decoder with cross-attention isn't in the cited SPOTER, SignBERT, or GCN work, and the WLASL-1000/2000 results (81.45% and 68.00%) are notably higher than NLA-SLR, which is the strong point here. The ablations are also useful—projection, normalization, and multi-component skeleton input clearly matter, and the parameter counts are honestly reported. That part deserves credit.\n\nThe soft spot is the one the stress-test flags, and it's real. The abstract says the model 'significantly outperforms previous models with over one million parameters,' but Table 4 on LSA-64 shows SPOTER at 100%, HWGATE at 98.59%, and SL-GCN at 98.13%, all with >1M params, beating SignBart's 96.04%. The paper's own text quietly narrows the comparison to ST-GCN and 3DGCN, but the abstract and conclusion never carry that qualification. That's not a split-protocol issue; it's a claim internally contradicted by the reported numbers. On WLASL, the 'superior generalization' claim only holds for the 1000/2000 subsets, not the 100/300 sets where NLA-SLR is well ahead. The reader's weakest assumption about unspecified train/test splits is also valid and serious—without knowing the split protocol, even the honest numbers aren't verifiable. No code, no data release, so the results are just assertions.\n\nThe 'coordinate theory' is the other weak point: it's presented as if it's a well-established framework, but the citation is to a motor-control paper that has nothing to do with skeleton coordinate separation, and the authors don't actually test the theory against an alternative (e.g., encoding xy jointly). That's a conceptual gap, not a fatal one, but it should be addressed.\n\nIs this paper for a serious referee? I'd say yes, conditionally. The architecture is worth a look and the large-subset WLASL numbers are intriguing enough that a reviewer might ferret out whether the splits explain the gap. But as written, I'd recommend major revision: fix the abstract and conclusions, provide exact split protocols for all datasets, compare against the baselines on equal footing, and at minimum release the keypoint data or pipeline. If the authors can't or won't do that, the strong claims should be withdrawn. For now, I'd treat the results as unverified but the idea as promising.","headline":"Interesting architecture idea, but the headline accuracy claim collapses against the paper's own tables; worth a look if revised honestly.","tokens_in":10959,"tokens_out":1953,"would_cite":false,"duration_ms":20514,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Isolated sign language recognition can be made accurate and lightweight by encoding a skeleton's x and y coordinates separately in a BART-style encoder-decoder, then reconnecting the streams with cross-attention; the paper reports 96.04%…","keywords":["sign language recognition","skeleton sequences","coordinate theory","model complexity","encoder-decoder","cross-attention","LSA-64","WLASL"],"falsifier":"A reader could settle it by training SignBart with the exact keypoint extraction and normalization pipeline on the official LSA-64 split used by SPOTER and on NLA-SLR's WLASL splits; if the 96.04% figure cannot be reproduced, or if the margins over ST-GCN and NLA-SLR reverse under those splits, the central comparison fails.","tokens_in":9922,"feed_emoji":"🤟","tokens_out":15128,"duration_ms":131723,"temperature":0.7,"pith_summary":"SignBart sets out to break the usual trade-off between accuracy and model size in isolated sign language recognition. Instead of feeding each skeleton keypoint as an inseparable (x, y) pair, the model encodes the x coordinates in a bidirectional encoder and the y coordinates in a causal decoder, then uses cross-attention so the y stream can query the encoded x stream. The paper reports 96.04% top-1 accuracy on the 64-word LSA-64 dataset with 749,888 parameters; on WLASL-1000 and WLASL-2000 it reports 81.45% and 68.00%, above NLA-SLR's 75.72% and 58.31%; and on the 2731-gloss ASL-Citizen set it reports 75.22% Rec@1, above I3D and ST-GCN. Ablations attribute most of the gain to the linear coordinate projection, per-part bounding-box normalization, and combining body with both hands. The authors note the method has not been tested on continuous sign recognition and that its three attention mechanisms may add computational cost on keypoint-heavy data.","feed_headline":"Skeleton x/y split reaches 96% sign-language accuracy","feed_subtitle":"A 749,888-parameter encoder-decoder splits skeleton x and y, then cross-attention reunites them for classification.","key_machinery":"The load-bearing object is the coordinate-split BART-style encoder-decoder. BART is a transformer sequence-to-sequence architecture whose encoder reads bidirectionally and whose decoder reads causally. Here an input skeleton $I \\in \\mathbb{R}^{T \\times K \\times 2}$ is split into $x_{\\text{coord}}$ and $y_{\\text{coord}}$, each projected to the model dimension by its own linear layer. The encoder runs bidirectional self-attention on the x stream; the decoder runs self-causal attention on the y stream with a mask $M(i,j)=1$ when $i \\ge j$, then cross-attention in which queries come from the y attention map while keys and values come from the encoder's x attention map. A second load-bearing piece is preprocessing: the body, left hand, and right hand are each normalized inside their own bounding box with a 5% margin, and the ablations show that the projection step, the three-box normalization, and the combination of all three skeleton components each add large accuracy gains.","core_discovery":"On the paper's own terms, the discovery is that the x and y coordinates of skeleton keypoints carry separable information, and that encoding them separately instead of as fused (x, y) pairs makes a small transformer more accurate. The architecture gives x to the encoder's bidirectional self-attention and y to the decoder's self-causal attention; cross-attention then lets each y-based query attend to x-based keys and values, so the independence never becomes a disconnection. The authors report that this scheme reaches 96.04% top-1 accuracy on LSA-64 with 749,888 parameters, higher than the ST-GCN and 3DGCN baselines listed for that dataset, though below SPOTER's reported 100%, and that accuracy degrades less than previous methods as WLASL grows from 100 to 2000 glosses. On the 2731-gloss ASL-Citizen version the model reaches 75.22% Rec@1, which is reported as better than I3D and ST-GCN from the dataset's original paper.","pith_inferences":["Because the paper never states the train/test split protocol, the cross-dataset comparisons should be read as provisional until SignBart is run on the official SPOTER and NLA-SLR splits; this is an editorial caution, not a claim in the paper.","The coordinate-split design is a general mechanism that could be tried on other skeleton-based tasks such as action recognition or gesture control; if the mechanism is what carries the gain, separating x and y before cross-attention should help wherever horizontal and vertical motion carry different meaning, but the paper does not test this.","The large right-hand versus left-hand accuracy gap (70.20% versus 23.02%) suggests the model may be exploiting signer handedness rather than learning symmetric language content; flipping hand keypoints as augmentation would be a direct test, and the paper does not report it.","The WLASL pattern (below NLA-SLR on 100 and 300 glosses, above on 1000 and 2000 glosses) suggests the model's advantage is scalability rather than small-vocabulary discrimination; a matched-split comparison would be needed to confirm, which the paper omits."],"forward_implications":["On LSA-64, SignBart reports 96.04% top-1 accuracy with 749,888 parameters, which exceeds the ST-GCN (92.81%) and 3DGCN (94.84%) baselines in the same table while using far fewer parameters; SPOTER still reports the highest number at 100% with 5,918,848 parameters.","On WLASL, the model's top-1 accuracy stays between 68.00% and 81.45% as the vocabulary grows from 100 to 2000 glosses, and its 81.45% on WLASL-1000 and 68.00% on WLASL-2000 exceed NLA-SLR's 75.72% and 58.31% despite NLA-SLR using both RGB and skeleton inputs.","On ASL-Citizen-2731, SignBart's 75.22% Rec@1 exceeds the I3D (63.10%) and ST-GCN (59.52%) baselines from the original dataset paper.","The ablations show that removing the coordinate projection drops LSA-64 accuracy from 96.04% to 62.08%, that normalization adds 13.54 points over no normalization, and that the three-box normalization outperforms one-box and two-box variants.","The component ablation shows body-only keypoints reach 86.97%, right-hand-only 70.20%, and left-hand-only 23.02%, so the combined body-plus-two-hands input is essential to the reported 96.04%."],"supporting_citations":[{"why":"Supplies the BART encoder-decoder structure with bidirectional encoding and causal decoding that SignBart adapts to skeleton coordinates.","marker":"[20]"},{"why":"Introduces SPOTER, the transformer baseline that reaches 100% on LSA-64 with class-query cross-attention; it is the main design contrast and comparison point.","marker":"[13]"},{"why":"Presents NLA-SLR, the RGB-plus-skeleton state-of-the-art baseline on WLASL that SignBart claims to beat on the 1000- and 2000-gloss subsets.","marker":"[37]"},{"why":"Defines the LSA-64 dataset, the primary benchmark where SignBart reports 96.04% accuracy.","marker":"[35]"},{"why":"Defines the WLASL dataset and its subsets, used to test generalization across vocabulary sizes.","marker":"[36]"},{"why":"Introduces ASL-Citizen and reports the I3D and ST-GCN baselines that SignBart compares against on the 2731-gloss version.","marker":"[28]"},{"why":"Supplies the HWGATE 98.59% LSA-64 result and the reported ST-GCN 92.81% result used in the comparison table.","marker":"[40]"},{"why":"Presents 3DGCN, the 94.84% LSA-64 graph-convolutional baseline that SignBart claims to outperform.","marker":"[41]"},{"why":"Presents SignBERT, a BERT-based skeleton transformer baseline whose WLASL results appear in the comparison table.","marker":"[12]"},{"why":"Presents SL-GCN, a 98.13% LSA-64 skeleton-graph baseline with over one million parameters, used to illustrate the complexity of prior high-accuracy models.","marker":"[14]"}],"fun_headline_variants":["Splitting skeleton x,y boosts sign AI to 96%","Tiny transformer signs 96% by separating x and y","Sign language AI: 749K params, 96% via x/y split","Cross-attention reunites split skeleton for 96% sign recognition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that every comparison uses the same train/test splits as the cited baselines, because the paper never specifies the split protocol for LSA-64, WLASL, or ASL-Citizen; if SignBart was evaluated under an easier split, the headline accuracy and generalization claims lose their basis.","fun_headline_variants_meta":{"raw":{"variants":["Splitting skeleton x,y boosts sign AI to 96%","Tiny transformer signs 96% by separating x and y","Sign language AI: 749K params, 96% via x/y split","Cross-attention reunites split skeleton for 96% sign recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000678,"raw_usage":{"total_tokens":3097,"prompt_tokens":977,"completion_tokens":2120,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":2043}},"tokens_in":593,"tokens_out":2120,"duration_ms":13394,"temperature":1.0,"reasoning_tokens":2043,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:40:57.930505+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle it by training SignBart with the exact keypoint extraction and normalization pipeline on the official LSA-64 split used by SPOTER and on NLA-SLR's WLASL splits; if the 96.04% figure cannot be reproduced, or if the margins over ST-GCN and NLA-SLR reverse under those splits, the central comparison fails.","supporting_citations":[{"cited_title":"A survey on skeleton-basedactivityrecognitionusinggraphconvolutionalnetworks(gcn),","cited_arxiv_id":null,"evidence_quote":"Supplies the BART encoder-decoder structure with bidirectional encoding and causal decoding that SignBart adapts to skeleton coordinates."},{"cited_title":"Sign pose-based transformer for word-level sign language recognition,","cited_arxiv_id":null,"evidence_quote":"Introduces SPOTER, the transformer baseline that reaches 100% on LSA-64 with class-query cross-attention; it is the main design contrast and comparison point."},{"cited_title":"Lsa64: A dataset of argentinian sign language,","cited_arxiv_id":null,"evidence_quote":"Presents NLA-SLR, the RGB-plus-skeleton state-of-the-art baseline on WLASL that SignBart claims to beat on the 1000- and 2000-gloss subsets."},{"cited_title":"Signbert: Pre-training of hand-model-aware representation for sign language recognition,","cited_arxiv_id":null,"evidence_quote":"Presents SignBERT, a BERT-based skeleton transformer baseline whose WLASL results appear in the comparison table."},{"cited_title":"Skeleton Aware Multi-modal Sign Language Recognition","cited_arxiv_id":"2103.08833","evidence_quote":"Presents SL-GCN, a 98.13% LSA-64 skeleton-graph baseline with over one million parameters, used to illustrate the complexity of prior high-accuracy models."}],"review_version":1}