{"id":"03e7c6d4-975a-4ae0-82fe-da3523a96cb6","arxiv_id":"2501.12183","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DexChar adds UNK-mediated character perturbations and noisy discriminator augmentation to produce semantic-preserving adversarial examples for subword NMT.","lead":"This paper extends a reinforcement-learning attack on neural machine translation so it can also use character-level edits, not just token substitutions. The method, called DexChar, generates semantic-preserving typos that can break translation even when token-based attacks fail.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on the discriminator's reward tracking true semantics; augmenting D with DexChar's own edits makes reward hacking a live risk that current PA/human checks do not rule out.","rationale":"The reader's weakest assumption already identifies the discriminator's semantic reward as the load-bearing point, and my reading of Sections 3.1-3.2 and Figure 1 agrees. The strongest empirical claims, such as MD 51.905 with PA 0.91 in Table 5, depend on D rejecting semantically destructive perturbations while accepting meaning-preserving ones. Because D's positive training data is augmented with the very DexChar edits the policy produces, a surface-level shortcut is plausible and would explain the observed MD levels with high PA. The GPT-3.5 PA and 100-sample human validation are useful but do not break the circularity, since they evaluate the same distribution of DexChar-style edits rather than a deliberately challenging set of near-miss non-equivalent perturbations. The missing ablation and missing D validation curves make it impossible to separate the effect of the UNK-based policy from the effect of noisy augmentation. I therefore keep the reader's CONDITIONAL verdict: the mechanism is plausible and internally consistent, but the semantic-constraint claim needs an independent human-labeled probe before acceptance. If the proposed probe shows that D or GPT cannot separate human-equivalent from human-non-equivalent pairs, the verdict should move toward REJECT or UNVERDICTED; if the probe passes, the conditional can be lifted.","tokens_in":10220,"tokens_out":5474,"duration_ms":62193,"concrete_test":"Build a held-out set of 300 candidate perturbations in the Table 5 en-de shared-vocab setting: 150 DexChar-generated adversarial inputs and 150 random Algorithm 2 character perturbations. For each, collect three independent human labels of whether the perturbed source is semantically equivalent to the original source and reference. Then compute D's acceptance reward and GPT-3.5 PA separately for human-equivalent and human-non-equivalent pairs, stratified by edit type. If D or GPT assigns high reward/PA to human-non-equivalent pairs (for example, more than 10% false acceptance), the semantic-preservation claim is not established; if they cleanly separate, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 augments the discriminator's positive training data with the same Algorithm 1/2 character perturbations that the DexChar policy produces, and Figure 1 shows D's acceptance probability is the survival reward in the RL loop. This creates a circularity: D is supposed to enforce semantic preservation, but its positive examples are drawn from the same edit family the agent is trained to emit. If D learns surface cues of DexChar-style edits (UNK or low-frequency subword status, repeated characters, homoglyph substitutions) rather than semantic equivalence to the reference, the policy can exploit D and produce large BLEURT drops with inputs that are not actually meaning-preserving. The reported PA uses GPT-3.5 on the same perturbation distribution, and the 100-pair human 'acc' numbers validate GPT's decisions rather than independently checking all generated examples; they therefore cannot rule out systematic reward hacking. The paper also reports no D validation curves (rho) and no ablation separating the UNK action from the noisy augmentation, so we cannot attribute the high MD/PA results to the claimed mechanism rather than to D being trained to approve DexChar's own outputs.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DexChar, an extension of the RL-attacker framework for generating adversarial examples against NMT. DexChar adds an 'UNK' token as a substitution candidate, backed by a deterministic character perturbation generator (Algorithms 1 and 2), and augments the semantic discriminator's positive training data with DexChar-style perturbations. Experiments on WMT14 en-de and CWMT17 en-zh across three embedding settings report BLEURT-based metric degradation, degradation per edit, and GPT-3.5-based pairing accuracy, together with a fine-tuning robustness analysis.","tokens_in":10464,"tokens_out":10085,"duration_ms":96480,"significance":"If validated, the contribution would be practically relevant: it offers a way to generate character-level adversarial examples for subword-based NMT systems, a scenario where token-substitution attacks are ineffective. The algorithmic specification is concrete and the experimental coverage across language pairs and embedding regimes is broad. The use of an external LLM semantic oracle with human validation, and the fine-tuning analysis, are positive elements. However, the lack of variance estimates, ablations, and a direct comparison to character-level attack baselines currently limits confidence in the claimed results.","major_comments":[{"comment":"The discriminator D is trained with positive samples generated by the same DexChar perturbation family (Algorithms 1 and 2) that the policy is optimized to emit. Because D operates on the tokenized input and subword tokenization maps the low-frequency character combinations to a single UNK token, D cannot explicitly represent which character-level edit was applied; the augmentation conditions D to accept UNK tokens as semantically preserved. This makes the survival reward (Figure 1, step 3) susceptible to reward hacking, so the high PA values in Tables 4 and 5 do not by themselves establish semantic preservation. The external GPT-3.5 PA is an independent check, but the human validation (acc) covers only 100 sampled pairs per experiment and validates the oracle's decisions on that sample rather than the full generated set. The paper does not report validation curves for D (the ρ values used for dynamic augmentation) or ablate the augmentation, so the reward signal's fidelity to true semantic equivalence remains unverified.","section":"Section 3.2, Figure 1, Section 4.1"},{"comment":"All reported MD, DPE, PA, and acc values come from a single pass without variance estimates or significance tests. RL training is stochastic, and the GPT-3.5 oracle is sampled, so point estimates cannot establish that Ours significantly outperforms RL-attacker, particularly in settings where the gap is small (e.g., en-zh shared-vocab MD 27.2 vs 21.32) or where PA differences are within a few points. Repeated runs with confidence intervals or statistical tests are needed to support the central empirical claims.","section":"Tables 4 and 5"},{"comment":"The two contributions—the DexChar UNK action and the noisy augmentation of D—are never ablated independently. Without comparing (i) DexChar without augmentation, (ii) augmentation without the UNK entry, and (iii) the full method, the paper cannot attribute the observed MD/DPE and PA improvements to the proposed mechanism. This is load-bearing because the augmentation directly affects the reward model, and the circularity concern from the preceding comment cannot be resolved without such an ablation.","section":"Section 3.1 and Section 3.2"},{"comment":"No existing character-level adversarial attack is used as a baseline. RNI is a random noise sanity check, while GS and RL-attacker are substitution-based and are the direct predecessor. To support the claim that the method is 'compatible with the scenario where baseline adversaries fail' and to position the work against prior art, the authors should compare against at least one character-level attack method (e.g., HotFlip-style attacks or a character-level variant of an existing attack) on the same target NMT models.","section":"Section 4.1 and Section 4.2"}],"minor_comments":[{"comment":"The 'UNK setting' is not clearly an experimental setting; the sentence 'We follow the baseline transformer training setting to truncate the top 30k vocabulary...' describes how the vocabulary is constructed. Please clarify whether this is a distinct scenario or a general setting applied to all experiments.","section":"Section 4.1"},{"comment":"The captions of Figures 2 and 3 appear to be copied from Figure 1 and do not describe the overhead and POS preference results. Please provide correct captions.","section":"Figures 2 and 3"},{"comment":"The baseline is named 'RL-base' in Table 4 and 'RL-attacker' in Table 5; please standardize the naming.","section":"Tables 4 and 5"},{"comment":"The system prompt contains the typo 'knowledgable'; it should be 'knowledgeable'.","section":"Table 3"},{"comment":"The reference to the '863 standard' for POS tags is only a URL; please add a proper bibliographic reference.","section":"Section 5.2"},{"comment":"The pseudocode has minor issues: 'return bywfinal' should be 'return wfinal', and the variable names `wfinal` and `wtmp` are difficult to read. Please polish the pseudocode.","section":"Algorithms 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant robustness problem and the algorithmic idea is interesting after the UNK entry and the augmentation mechanism. The central risk is that the discriminator's reward signal may be circular, and the current empirical evidence does not rule out reward hacking. The revision should add ablations (UNK action vs. augmentation), variance estimates, and a character-level attack baseline, and should report D's validation performance against human semantic judgments on DexChar-style edits. If these points are addressed, the contribution could be publishable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Your reader's conditional verdict is right, and I'd go slightly further: this is a useful, clearly-written extension of the RL-attacker with a plausible mechanism, but the evaluation is not yet tight enough to fully pin down the semantic-preservation claim.\n\nThe genuinely new pieces are the UNK-as-entry for character-level edits and the dynamic augmentation of the discriminator with the same perturbation family. That second piece is double-edged: it solves the false-negative problem, but it raises exactly the circularity concern the stress-test note flags. D is trained to accept DexChar-style edits; if it learns surface cues, the policy can exploit it. The paper's counters are the GPT-3.5 PA and a 100-pair human check. These help, but they are not decisive: the human sample is small and only validates GPT's judgments on a subsample, and the PA is measured on the same perturbation distribution the policy was trained to produce. A clean ablation separating the UNK action from the augmentation, plus a few D validation curves, would have settled this. The absence of variance estimates, significance tests, and released code then makes it hard to judge the size and stability of the reported gains.\n\nWhat the paper does well: the tokenization failure is clearly diagnosed in Table 2, the POS analysis in Figure 3 gives useful qualitative insight into where character-level attacks land, and the fine-tuning experiments show a concrete practical use. The improvements over RL-attacker are consistent across Tables 4 and 5, which gives me confidence the effect is real even if the magnitude is uncertain.\n\nI don't see a load-bearing flaw. The circularity is a genuine worry, but it is not a refutation; it is a request for more evidence. The lack of a direct comparison to existing character-level attack baselines (e.g., HotFlip-style) is a gap, not an error.\n\nWho this is for: people working on adversarial robustness of NMT, especially those who want a cheap way to generate character-level perturbations for testing and fine-tuning. It deserves a serious referee: the idea is simple, the problem is real, and the paper does enough to justify the work. I'd send it to review with a request for ablations, variance, code, and a stronger external semantic check.\n\nRecommendation: do not desk-reject. Send to peer review; expect revision.","headline":"A well-explained extension of RL-attacker to character-level perturbations, with consistent gains but an evaluation that needs stronger evidence on semantic preservation.","tokens_in":10976,"tokens_out":2724,"would_cite":false,"duration_ms":28139,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that adding an 'unknown' token to a reinforcement-learning adversarial policy, plus noisy augmentation of its semantic check, lets character-level typos degrade machine translation while preserving meaning.","keywords":["neural machine translation","adversarial examples","reinforcement learning","character-level perturbation","unknown token","semantic discriminator","noisy data augmentation","adversarial fine-tuning"],"falsifier":"Have independent human judges score a large sample (for example, 1,000 or more pairs per setting) of DexChar adversarial inputs against the original annotation; if true pairing accuracy falls substantially below the reported ~0.9 while translation degradation stays high, the discriminator's reward is not tracking semantics.","tokens_in":10012,"feed_emoji":"🔤","tokens_out":16638,"duration_ms":146463,"temperature":0.7,"pith_summary":"This paper extends a reinforcement-learning attack on neural machine translation—previously centered on replacing whole tokens with similar ones—by adding the unknown (UNK) token as a substitution candidate. When the agent picks UNK, a deterministic routine swaps, inserts, or substitutes characters until the tokenizer treats the word as unknown, turning arbitrary typos into single actions the policy can learn. The authors also retrain the semantic-matching discriminator with positive examples noised by the same character edits, so edits that change how the model splits text but not what it means are no longer rejected. As a result, on shared-vocabulary English-German the method raises translation-quality degradation to 51.9 from the RL baseline's 29.9 while keeping pairing accuracy near 0.9, at roughly the RL attack's low overhead. This would give a cheap way to test and harden translation systems against realistic typo-style noise.","feed_headline":"UNK token enables typo-style attacks on translation models","feed_subtitle":"DexChar policy keeps meaning while quality drops 51.9 vs 29.9 for the baseline attack.","key_machinery":"The load-bearing object is the UNK token used as an action interface, paired with the perturbation functions of Algorithms 1 and 2. UNK is added to all substitution candidates, so the RL agent can select 'perturb this token at character level' as one discrete action; the algorithms then search left-to-right through swap, insertion, and substitution (with keyboard-vicinity and homoglyph dictionaries; deletion is excluded) for a low-frequency variant the target vocabulary treats as unknown. The supporting mechanism is noisy data augmentation of the discriminator: positive semantic-matching examples are randomly perturbed with the same DexChar edits at a confidence-dependent rate $\\xi = \\rho - \\bar{\\rho}$ and retokenized, so $D$ stops issuing false negatives for character edits that preserve meaning. This pair—an entry point for arbitrary perturbation granularity and a reward signal that tolerates it—carries the whole argument.","core_discovery":"The central claim is that the barrier to character-level adversarial attacks on NMT is not the search loss but the policy's entry point and its semantic reward. DexChar adds UNK to every substitution candidate; choosing it triggers a left-to-right character perturbation—swapping adjacent characters, inserting a repeated character, or substituting from keyboard/homoglyph dictionaries—until the tokenizer treats the word as unknown, with a second algorithm chaining actions if one edit fails. This maps arbitrarily many low-frequency character edits onto one discrete RL action, so the policy can learn to use them without a handcrafted, tokenization-dependent loss. To keep those edits semantically safe, the discriminator $D$ is trained with positive pairs noised by random DexChar-style edits at probability $\\xi = \\rho - \\bar{\\rho}$, where $\\rho$ is $D$'s current validation confidence, so it learns to accept meaning-preserving character perturbations even when tokenization shifts. The paper reports that this combination yields high translation degradation and high pairing accuracy where substitution-only baselines stall, and that adversarial fine-tuning with the generated examples improves robustness to mixed character/subword noise.","pith_inferences":["The UNK-as-action trick should transfer to other subword-tokenized generation tasks such as summarization, dialogue, or code generation, because any low-frequency character edit changes tokenization in the same way and can be reached through one discrete action.","A testable extension would replace the learned discriminator reward with a fixed semantic-similarity metric, isolating whether DexChar's gains come from the UNK action space or from the noisy augmentation.","The confidence-gated augmentation schedule $\\xi = \\rho - \\bar{\\rho}$ could be reused as a general curriculum for training auxiliary classifiers that must tolerate tokenization shifts.","The method doubles as an automated typo-data generator for robustness fine-tuning of any encoder-decoder model, not only translation."],"forward_implications":["In shared-vocabulary settings where substitution-only adversaries stall, character-level RL attacks recover high degradation: en-de shared-vocab translation-quality degradation rises to 51.9 from the RL baseline's 29.9.","The generated adversarial examples stay meaning-preserving: LLM-based pairing accuracy remains around 0.9 across language pairs and embedding settings.","The extension adds negligible overhead, keeping the RL paradigm about 200 times faster than gradient search.","Fine-tuning a target NMT on DexChar-generated pairs improves robustness to character noise and mixed character/subword attacks more than fine-tuning on RL-attacker examples.","The method applies across base, shared-vocabulary, and pretrained-embedding NMT settings, so it does not depend on one tokenizer."],"supporting_citations":[{"why":"It provides the RL-attacker framework—agent-environment loop, semantic discriminator, and episodic reward from NMT degradation—that DexChar extends.","marker":"[29]"},{"why":"It supplies the keyboard-vicinity and homoglyph character-substitution candidates used by DexChar's substitution action.","marker":"[7]"},{"why":"It contributes the character-level perturbation strategies (swap, insert, substitute) and the observation that tokenization changes invalidate substitution-based attacks.","marker":"[6]"},{"why":"It defines adversarial examples as annotation-preserving perturbations that trigger system degradation, the criterion DexChar optimizes.","marker":"[9]"},{"why":"It provides the gradient-search baseline and the metric-degradation evaluation protocol used in the experiments.","marker":"[14]"},{"why":"It supplies BLEURT, the learned metric the paper uses to compute translation-quality degradation.","marker":"[21]"},{"why":"It supplies the pretrained multilingual vocabulary and embedding setting (bart_emb) used as one test scenario.","marker":"[13]"},{"why":"It supplies the homophone substitution candidates used in Algorithm 2 for Chinese character perturbations.","marker":"[28]"}],"fun_headline_variants":["UNK token unlocks character-level attacks on NMT","DexChar: character adversarial attacks via unknown token","Unknown token enables typo-driven translation attacks","Character perturbations via UNK: new NMT attack policy","Using UNK to extend adversarial policies to character edits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's success depends on the semantic check's reward genuinely tracking meaning preservation after noisy augmentation, rather than merely accepting the character-perturbation style it was trained on.","fun_headline_variants_meta":{"raw":{"variants":["UNK token unlocks character-level attacks on NMT","DexChar: character adversarial attacks via unknown token","Unknown token enables typo-driven translation attacks","Character perturbations via UNK: new NMT attack policy","Using UNK to extend adversarial policies to character edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1304,"prompt_tokens":877,"completion_tokens":427,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":352}},"tokens_in":493,"tokens_out":427,"duration_ms":4723,"temperature":1.0,"reasoning_tokens":352,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:24:54.270285+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent human judges score a large sample (for example, 1,000 or more pairs per setting) of DexChar adversarial inputs against the original annotation; if true pairing accuracy falls substantially below the reported ~0.9 while translation degradation stays high, the discriminator's reward is not tracking semantics.","supporting_citations":[{"cited_title":"In: Proc","cited_arxiv_id":null,"evidence_quote":"It provides the RL-attacker framework—agent-environment loop, semantic discriminator, and episodic reward from NMT degradation—that DexChar extends."},{"cited_title":"In: Proceedings of ACL (2020)","cited_arxiv_id":null,"evidence_quote":"It supplies BLEURT, the learned metric the paper uses to compute translation-quality degradation."},{"cited_title":"TACL (2020)","cited_arxiv_id":null,"evidence_quote":"It supplies the pretrained multilingual vocabulary and embedding setting (bart_emb) used as one test scenario."},{"cited_title":"Journal of Electronics and In- formation Technology (2023)","cited_arxiv_id":null,"evidence_quote":"It supplies the homophone substitution candidates used in Algorithm 2 for Chinese character perturbations."}],"review_version":1}