{"id":"83266997-2f47-4fb3-9185-820a640f0699","arxiv_id":"2607.11215","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Adaptive segmentation plus shared-base residual VQ yields discrete Q-units that let one LLM translate ASL, CSL, and DGS into English, Chinese, and German with SOTA native and strong non-native scores.","lead":"Q-BridgeNet turns continuous sign poses into discrete shared-plus-private tokens (Q-units) and fine-tunes one multilingual LLM for many-to-many sign-to-text translation. It reports SOTA native results on three sign languages and usable non-native transfer, which matters for accessibility across language communities.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Non-native SOTA rests on GPT-translated training targets and references; the Table 2 control does not fully isolate caption artifacts from true cross-lingual sign modeling.","rationale":"The reader correctly isolates the weakest link: GPT-5/4.1 captions serve dual roles as training targets and non-native ground truth, so the many-to-many generalization claim is only partially controlled by the post-hoc translation baseline in Table 2. Native-pair SOTA remains comparatively solid (original references, competitive ablations on shared vs private vs residual codebooks and iterative segmentation). No stronger internal inconsistency appears in the residual VQ or adaptive segmentation design; the concern is evaluation validity for the distinctive claim, not method incoherence. Therefore the CONDITIONAL verdict stands: accept-shaped if native results and code hold, contingent on cleaner non-native evaluation. Agreement with the reader is full on the load-bearing assumption; no adjustment beyond reinforcing that contingency is warranted.","tokens_in":14435,"tokens_out":594,"duration_ms":6902,"concrete_test":"Hold the frozen RQ-VAE and Q-unit tokenizer fixed. Retrain only the LLM stage using (a) original native captions only, or (b) an independent MT system (e.g., NLLB/mBART) for the two non-native targets, then re-evaluate non-native directions against a small human-translated reference set (or against the independent MT references). If non-native BLEU-4 drops by >15% relative to Table 1 multilingual rows while native scores stay within ~1 BLEU of the reported numbers, the non-native claim is largely caption-driven.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper’s distinctive claim is not only native SOTA but “strong generalization to non-native pairs” under a unified many-to-many model (abstract; §4.3; Table 1 rows ASL-ZH/DE, CSL-EN/DE, DGS-EN/ZH). Both the many-to-many training targets and the non-native evaluation references are produced by GPT-5 (and GPT-4.1) from the original native captions (§3.4; §4.1 Evaluation Metrics). The only control (“GPT-5 Translation” in Table 2) first runs monolingual SLT then post-hoc translates the prediction; it does not retrain under independent non-GPT multilingual supervision, nor does it evaluate against human non-native references. If GPT captions systematically simplify, align style, or reduce lexical diversity toward the decoder’s prior, measured non-native BLEU/ROUGE can rise without the shared–private Q-units having learned transferable sign semantics. Native SOTA is less affected (original GT), but the central many-to-many contribution is load-bearing on this assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"Q-BridgeNet proposes a unified many-to-many sign language translation framework that represents continuous pose streams as discrete Q-units obtained by alternating adaptive variable-length segmentation with residual vector quantization. A shared base codebook is intended to capture language-agnostic signing primitives, while language-specific residual codebooks refine ASL/CSL/DGS variation; a multilingual LLM (Qwen3-1.7B with LoRA) is then fine-tuned to map Q-unit sequences to spoken text in German, English, and Chinese. Multilingual supervision is created by translating native captions with GPT-5. On PHOENIX14T, How2Sign, and CSL-Daily the method reports state-of-the-art native BLEU/ROUGE and competitive non-native scores (Table 1), with ablations of codebook sharing, residual stacking, iterative segmentation, and LLM scale (Table 2) and supporting t-SNE/gloss-frequency analyses of the discrete space.","tokens_in":14809,"tokens_out":1141,"duration_ms":10801,"significance":"If the claims hold, the work would be a useful step toward practical multilingual SLT: a single model that both improves native pairs and transfers across sign–spoken combinations without per-pair re-tokenization. The shared–private residual design and the joint segmentation–quantization procedure are concrete technical contributions relative to fixed-length single-language VQ and continuous multilingual baselines such as Uni-Sign. Public code, systematic ablations (Table 2), and distributional analyses of Q-units (Figs. 4–5) strengthen reproducibility and interpretability. The main scientific stake is whether the non-native gains reflect transferable sign semantics rather than properties of machine-translated captions used for both training and evaluation.","major_comments":[{"comment":"§3.4 and §4.1 Evaluation Metrics: non-native training targets and non-native evaluation references are both produced by GPT-5 (or GPT-4.1) from the same native captions. The central claim of “strong generalization to non-native pairs” (abstract, §4.3, Table 1 rows ASL-ZH/DE, CSL-EN/DE, DGS-EN/ZH) therefore rests on machine-translated references. The Table 2 “GPT-5 Translation” control (monolingual SLT then post-hoc translate) is lower and helpful, but does not retrain under independent non-GPT multilingual supervision nor evaluate against human non-native references. Without at least a human-validated subset or an independent MT system for references, measured non-native BLEU/ROUGE may partly reflect caption style/simplification rather than cross-lingual sign modeling. This is load-bearing for the many-to-many contribution even though native SOTA (original GT) is less affected.","section":null},{"comment":"§3.3, Eqs. (7)–(13): the joint objective J(S,V) and the block-coordinate alternation are presented as yielding “semantically coherent” variable-length units, yet the only quantitative evidence is reconstruction-driven segmentation plus downstream BLEU. There is no direct boundary-quality metric (e.g., agreement with gloss boundaries where available, or held-out reconstruction under fixed vs. adaptive segments). Given that adaptive segmentation is a core claimed advantage over fixed-length VQ, a load-bearing validation of segment quality—or a clearer statement that semantic coherence is only inferred from SLT gains—would strengthen the central representation claim.","section":null}],"minor_comments":[{"comment":"Table 1: “Avg.” rows for the multilingual model are not defined (mean over three targets? weighted?). Clarify how averages are computed and whether they are comparable across datasets of different sizes.","section":null},{"comment":"§3.2: residual depth R, exact codebook sizes (1024 shared / 512 private), and the factor-binning of residual codebooks are stated in Implementation Details but not fully formalized in the equations; a short explicit definition of language-conditioned Q^r_i would help.","section":null},{"comment":"Fig. 3 qualitative examples: bold/strike markup is useful, but adding the corresponding Q-unit sequence length or segment count would make the link to adaptive segmentation more concrete.","section":null},{"comment":"Related Work: Uni-Sign and MGSLT are discussed; a short explicit contrast on discrete vs. continuous interfaces and on whether those works also use MT-augmented targets would sharpen novelty.","section":null},{"comment":"Typos/spacing: “SL T” in Fig. 1 caption; occasional missing spaces after periods in the abstract and §1; “factor-binned” could be defined once for non-VQ readers.","section":null}],"recommendation":"major_revision","confidential_remarks":"The native-pair SOTA and ablations look solid enough for a solid venue if the non-native evaluation is tightened (human subset or independent MT references). The GPT-loop issue is the main reason I chose major_revision rather than minor_revision; if the authors can only partially address it, the paper remains publishable with a clearly scoped claim that non-native numbers are MT-referenced. Scope fit for a general CL/ML venue is good; novelty relative to Uni-Sign is real but incremental."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The thing worth knowing is that this is a coherent systems paper that actually ships a working many-to-many SLT stack, not just another monolingual LLM wrapper. Adaptive variable-length segmentation co-optimized with residual VQ (shared base codebook + language-private residuals) into discrete Q-units, then a multilingual LLM fine-tuned in that space, is a clean combination. Prior work already had fixed-length VQ for signs, gloss-free SLT, and multilingual models like Uni-Sign/MGSLT; the joint segmentation–quantization loop and the shared–private residual split are the real design moves.\n\nNative results look solid. On How2Sign, CSL-Daily, and PHOENIX14T they beat the usual baselines on BLEU/ROUGE, and the ablations (uni-codebook, non-shared, non-residual, single-pass, smaller LLM) move in the right direction. t-SNE and gloss-frequency comparisons are secondary but consistent with the claim that the discrete space is more structured than flat VQ. Code is promised; pose-only input is standard for this literature.\n\nThe soft spot is exactly the stress-test point, and it is real but not fatal. Non-native training targets and non-native references both come from GPT-5/4.1 translations of the native captions. The Table 2 “monolingual then post-hoc GPT translate” control is lower, which helps, but it does not retrain under independent multilingual supervision or evaluate against human non-native references. So the “strong generalization to non-native pairs” claim is partly entangled with caption artifacts. Native SOTA is less affected. No error bars, free parameters everywhere (codebook sizes, β/γ, segment length, alternation rounds), and no formal verification—normal for this venue, but keep expectations calibrated.\n\nThis is for people building multilingual SLT or discrete sign representations, not for general multimodal theory. Citation pattern is appropriate; math is standard RQ-VAE + DP-style segmentation, not deep theory. I would send it to peer review: the native gains and design are enough to deserve referee time, with a clear ask for cleaner non-native evaluation. Worth engaging if you work in the area; cite the native numbers and the Q-unit construction, treat non-native claims with the GPT caveat.","headline":"Solid native SOTA via adaptive shared–private residual Q-units; the many-to-many story is real but partly GPT-caption-dependent.","tokens_in":15444,"tokens_out":579,"would_cite":true,"duration_ms":6062,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A single model can translate American, Chinese and German sign languages into English, Chinese and German by turning continuous signing into shared discrete Q-units that a multilingual language model reads.","keywords":["sign language translation","multilingual modeling","vector quantization","residual codebooks","adaptive segmentation","large language models","cross-lingual transfer"],"falsifier":"Retrain the identical pipeline using only human-authored multilingual captions (or evaluate non-native outputs against a held-out set of human translations) and check whether native and non-native BLEU/ROUGE still exceed the strongest prior baselines by the same margins reported in Table 1.","tokens_in":15289,"feed_emoji":"🤟","tokens_out":842,"duration_ms":17972,"temperature":0.7,"pith_summary":"Most sign-language translation systems only handle one sign language paired with its local spoken language. This paper argues that many-to-many translation becomes practical once continuous pose streams are turned into discrete Q-units: variable-length segments that are quantized with a shared base codebook for language-agnostic meaning and language-specific residual codebooks for local signing style. A multilingual language model is then fine-tuned to map those Q-units into any of the target spoken languages. On the three standard datasets How2Sign, CSL-Daily and PHOENIX14T the method reports state-of-the-art native-pair scores and strong transfer to non-native pairs. If the claim holds, people using different sign languages could communicate through one system instead of a patchwork of pair-specific models.","feed_headline":"One model maps three sign languages to three spoken languages","feed_subtitle":"Shared discrete Q-units cut cross-language interference and beat prior native scores","key_machinery":"Discrete Q-units: adaptive variable-length segmentation of pose sequences, jointly refined with multi-codebook residual vector quantization (shared base codebook plus language-specific residual codebooks), after which a multilingual LLM is fine-tuned to operate directly in the resulting Q-unit token space.","core_discovery":"Residual vector quantization of adaptively segmented pose streams—using one shared base codebook for cross-lingual semantic primitives and private residual codebooks for language-specific detail—produces discrete Q-units that reduce cross-lingual interference while remaining expressive enough for a fine-tuned multilingual LLM to reach state-of-the-art native sign-to-text scores and strong non-native generalization on PHOENIX14T, How2Sign and CSL-Daily.","pith_inferences":["The alternating segmentation–quantization loop may transfer to other continuous multimodal streams whose natural temporal boundaries are ambiguous.","If Q-units truly encode reusable semantic primitives, retrieval-augmented or few-shot cross-lingual SLT could become cheaper to scale.","Stronger human-translated multilingual corpora would be a direct next test of whether the reported non-native gains are limited by caption quality."],"forward_implications":["A single many-to-many SLT model can replace separate monolingual systems for ASL–English, CSL–Chinese and DGS–German.","Non-native pairs (for example ASL to Chinese) become usable without re-learning sign units for each spoken language.","The same discrete Q-unit inventory can later serve as the bridge for the reverse task of sign language production.","Shared–private residual quantization can limit negative transfer when additional sign languages are added to the model."],"fun_headline_variants":["Shared base plus residual Q-units unify multilingual SLT","Q-BridgeNet quantizes poses to cut sign-language interference","Discrete Q-units let one LLM handle native and non-native SLT","Adaptive residual VQ bridges three sign languages to spoken text","One quantized bridge model beats native scores across three SLT sets"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The method treats machine translations of the original captions as both training targets and evaluation references for non-native pairs; if those translations systematically simplify or style-match the decoder, measured cross-lingual gains partly reflect caption artifacts rather than true sign understanding.","fun_headline_variants_meta":{"raw":{"variants":["Shared base plus residual Q-units unify multilingual SLT","Q-BridgeNet quantizes poses to cut sign-language interference","Discrete Q-units let one LLM handle native and non-native SLT","Adaptive residual VQ bridges three sign languages to spoken text","One quantized bridge model beats native scores across three SLT sets"]},"model":"grok-4.5","effort":"low","cost_usd":0.003848,"raw_usage":{"total_tokens":1254,"prompt_tokens":823,"num_sources_used":0,"completion_tokens":71,"cost_in_usd_ticks":38480000,"prompt_tokens_details":{"text_tokens":823,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":360,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":823,"tokens_out":71,"duration_ms":4016,"temperature":1.0,"reasoning_tokens":360,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T06:13:37.342166+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Retrain the identical pipeline using only human-authored multilingual captions (or evaluate non-native outputs against a held-out set of human translations) and check whether native and non-native BLEU/ROUGE still exceed the strongest prior baselines by the same margins reported in Table 1.","supporting_citations":[],"review_version":1}