{"id":"cc4679b9-f46c-48dd-b1af-ddea86d4970b","arxiv_id":"2505.12686","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RoVo perturbs BARK codec embeddings instead of raw audio, and reports that speech-synthesis clones of protected voices are rejected by speaker verification 70+ percentage points more often, with better robustness to speech enhancement than Antifake.","lead":"RoVo is a voice-protection method that adds adversarial noise to the internal neural-codec embedding of an audio clip, then reconstructs protected speech, instead of adding noise directly to the waveform.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The post-enhancement DSR gain partly comes from severe audio distortion (acknowledged in §6.2), and RoVo's pre-enhancement MOS is only 2.09–2.91, so 'robust under enhancement' and 'preserves naturalness' are not separated from quality destruction.","rationale":"The reader's weakest-assumption analysis identifies exactly the concern that is most load-bearing for the central claim: high DSR after speech enhancement is partly explained by the authors' own observation that enhancement severely distorts RoVo-protected audio, so the metric cannot distinguish successful perturbation persistence from simple quality destruction. The abstract explicitly promises both robustness to enhancement and preservation of naturalness, and the paper's Table 4 MOS values (2.09–2.91 before enhancement, and generally lower after) show that the naturalness commitment is already weak at the point of attack. The threat model in Section 3 also requires naturalness and usability as user objectives, so a defense that works only by ruining the audio does not satisfy the stated goal. I do not see this as a reason to reject: the narrow claim that embedding-level perturbations raise DSR is supported by Table 2, and the transferability results are coherent. The concern is empirical and testable, not an internal contradiction, and the authors were transparent about the distortion mechanism. Therefore the reader's CONDITIONAL verdict is appropriate, and no verdict change is needed; a quality-conditioned recomputation of the enhancement tables would settle whether the stronger robustness claim survives.","tokens_in":19327,"tokens_out":4206,"duration_ms":48132,"concrete_test":"Recompute Table 3 while conditioning on enhancement output quality: for every enhanced sample, compute word error rate (WER) with a fixed ASR model and only retain samples whose WER is within a pre-specified margin of that sample's pre-enhancement WER (e.g., no more than 10 percentage points worse). If RoVo's post-enhancement DSR advantage over Antifake collapses or approaches the unprotected baseline in this quality-filtered subset, the robustness claim is an artifact of quality destruction rather than perturbation persistence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RoVo's central robustness claim — that embedding-level perturbations survive speech-enhancement post-processing better than signal-level defenses — is not actually separable from the paper's own admission that enhancement severely distorts RoVo-protected audio. In §6.2 the authors state that when enhancement increased DSR (e.g., YourTTS/Resemblyzer/DeepFilterNet +25.6 points, AVC/Resemblyzer/Spectral Masking +7.2 points in Table 3), this was because enhancement 'severely distorted the protected audio itself,' so verification models rejected the result as different from the original speaker. That makes DSR a compound of two effects: perturbation persistence and sheer audio destruction. Since Table 4 reports pre-enhancement MOS of only 2.09±0.27 to 2.91±0.67, with further MOS drops after enhancement, the quality cost is already large before any secondary attack. The abstract's claim that RoVo is 'robust even under strong speech enhancement' and 'preserves naturalness and usability' therefore rests on a metric that counts destroyed audio as successful defense. A secondary internal inconsistency is worth noting: Algorithm 1 optimizes a speaker encoder G over codec embeddings e_perturb, while Eq. (1) defines the same loss over audio signals; this needs clarification, but the DSR/quality conflation is the load-bearing issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RoVo, a proactive defense against unauthorized speech synthesis. Instead of adding perturbations at the waveform level, RoVo uses a neural audio codec (BARK) to encode speech, applies PGD-based adversarial perturbations to the resulting embeddings under a PerC-AL loss that alternates between a speaker-identity target loss and an SNR-preservation loss, and then decodes the perturbed embeddings into protected audio. The authors evaluate RoVo against SV2TTS, YourTTS, AVC, and black-box Tortoise TTS, using ECAPA-TDNN, Resemblyzer, and Microsoft Azure speaker verification to compute Defense Success Rate (DSR), and they measure quality with NISQA MOS and a user study. They report large DSR gains over unprotected speech, claim superior robustness to speech-enhancement post-processing relative to the signal-level Antifake baseline, and conclude that RoVo preserves naturalness and usability.","tokens_in":19686,"tokens_out":6028,"duration_ms":59784,"significance":"The core idea of perturbing a learned audio-codec embedding rather than the raw waveform is a plausible and timely direction for making proactive voice defenses survive enhancement-based post-processing. The paper has notable strengths: evaluation uses three speaker-verification systems, including a commercial API; black-box transfer to Tortoise TTS is tested; an ensemble black-box setting under enhancement is included; and sound samples are publicly provided. However, the central robustness claim is currently confounded: the paper itself states that speech enhancement sometimes raises DSR because it severely distorts the protected audio, and the reported pre-enhancement MOS values are low. If the authors can separate perturbation persistence from quality collapse and align the optimization description with the algorithm, the work would be a useful contribution to the proactive-defense literature.","major_comments":[{"comment":"The robustness claim is not separable from quality destruction. In §6.2 the authors state that when enhancement raised DSR (e.g., +25.6 points for YourTTS/Resemblyzer/DeepFilterNet and +7.2 points for AVC/Resemblyzer/Spectral Masking), the enhancement models \"severely distorted the protected audio itself\" rather than leaving RoVo's perturbation intact. Because DSR counts any verification rejection as a success, a sample that is unintelligible after enhancement is scored identically to a sample whose speaker embedding remains perturbed. The abstract's statement that RoVo \"remained robust even under strong speech enhancement\" therefore requires a decomposition of post-enhancement DSR into cases where the perturbation survived and cases where the enhancement destroyed the signal. I recommend reporting DSR conditioned on a minimum post-enhancement quality threshold (e.g., MOS or STOI above a floor), or reporting joint distributions of DSR and quality for the same enhanced samples.","section":"§6.2, Table 3"},{"comment":"The objective is specified inconsistently. Eq. (1) defines L_identity as D(g(x_U + δx_U), g(x_T)) over audio signals, with g a speaker embedding encoder, while Algorithm 1 computes L_identity = ||G(e_perturb) - G(e_target)||_2 on embeddings and updates e_perturb before the codec decoder reconstructs audio. If the adversarial loss is actually evaluated on embeddings, Eq. (1) is not the optimized objective; if it is evaluated on reconstructed audio, the decoder and the inverse mapping from audio to embeddings must appear in the optimization loop. This distinction is central to the paper's \"embedding-level perturbation\" claim and needs to be stated precisely, including how gradients flow back to e_perturb.","section":"§4.3, Eq. (1) and Algorithm 1"},{"comment":"The evidence does not support the naturalness claim. Table 4 reports pre-enhancement NISQA MOS of 2.09±0.27 to 2.91±0.67 for RoVo, whereas the text reports that BARK reconstruction without perturbations achieves MOS 4.53, so RoVo itself introduces a large quality drop. The user study in §6.4 additionally shows that only about 50% of participants rated RoVo-defended speech as \"Very Similar\" or \"Somewhat Similar\" to the original, and about 80% rated it \"Somewhat Different\" or \"Very Different\" after enhancement. Neither the abstract's \"preserves naturalness and usability\" nor the §7 conclusion that RoVo \"maintains acceptable audio quality and naturalness\" follows from these numbers. Please compare original, BARK-reconstructed, RoVo, and enhanced RoVo under the same protocol, with confidence intervals and a stated acceptability threshold.","section":"Table 4, §6.4, §7"},{"comment":"The claim that RoVo \"consistently maintained higher defense performance\" than Antifake after enhancement is not supported by the full table. For example, with Spectral Masking and ECAPA-TDNN on SV2TTS, Antifake reaches 86.3% while RoVo reaches 77.1%, and with MP-SENet_VB and ECAPA-TDNN on SV2TTS, Antifake reaches 84.0% while RoVo reaches 81.5%. The paper should either qualify the claim or provide a systematic comparison (e.g., average rank or matched-pair differences across all configuration cells) rather than relying on selected examples.","section":"Table 3"}],"minor_comments":[{"comment":"The label \"Enhancment\" is misspelled; it should be \"Enhancement\".","section":"Figure 5"},{"comment":"DSR values are reported without error bars or significance tests. Given the 1,090 protected samples and 5,045 synthesized samples per condition, bootstrapped confidence intervals would substantially strengthen the quantitative claims.","section":"Tables 2, 3, 5, 6"},{"comment":"The speaker-verification thresholds are described only as \"set using the VCTK dataset.\" Please state the threshold criterion (e.g., equal error rate), report the resulting false-accept and false-reject rates, and clarify whether the same thresholds are used for all synthesis and enhancement conditions.","section":"§5.4"},{"comment":"Reference [1] is a Prolific OAuth callback URL that contains a live authorization code and state parameter. This looks like a leaked secret and should be replaced with the canonical Prolific URL.","section":"Reference [1]"},{"comment":"The description of Table 5 says \"the second row represents...\" when the relevant information is in the column headers; please rephrase to describe the table structure accurately.","section":"§6.3"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the external evaluation is a genuine strength, but the robustness and quality claims require the decomposition described in the major comments. I do not see a basis for rejection if the authors can separate perturbation persistence from enhancement-induced distortion and can align the optimization description with Algorithm 1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhiyuan, quick take on 2505.12686 (RoVo). The genuinely new thing is moving adversarial perturbation into neural-codec embedding space: encode with BARK, PGD on embeddings, decode. That is distinct from Antifake's signal-level attack and from prior anonymization work, and the experiments are broad enough (four synthesis models, three enhancement models, three verification systems, including Azure) that the narrow claim—embedding-level perturbations raise DSR—is credible. Table 2 is the backbone and it holds up.\n\nWhat the paper does well: it checks black-box transfer to Tortoise, tests ensemble optimization, and is honest in §6.2 that some post-enhancement DSR gains come from severe distortion of the protected audio itself. That admission is more than most papers offer, but it cuts against the headline \"robust under enhancement\" claim more than the authors seem to realize.\n\nSoft spots, in order of size. First, DSR after enhancement is a compound of perturbation persistence and plain quality destruction; Table 4 shows MOS around 2–3, and the user study shows 80% of listeners find enhanced defended audio different. So claiming \"preserves naturalness\" while counting destroyed audio as defense success is not separated. They need a distortion-aware decomposition—report DSR conditional on intelligibility or perceptual quality, or measure perturbation-removal success separately. Second, thresholds tau_identity/tau_SNR and hyperparameters are not disclosed, so the PerC-AL switching is not reproducible. Third, Algorithm 1 optimizes over embeddings e_perturb while Eq. (1) writes the loss over audio signals x_U + delta; likely just notation, but it matters for a paper whose whole point is embedding-level. Fourth, no code release; given the claim rests on a specific BARK pipeline, that is a real gap.\n\nThe citation pattern is fine—Antifake, PerC-AL, speaker anonymization via codecs are all cited. No invented entities. I don't think the circularity burden is high; the target loss is from Antifake but evaluation uses independent verification systems.\n\nWho this is for: security researchers working on proactive voice protection; people building voice-sharing platforms. It deserves a serious referee slot—conditional accept, not desk reject—because the core axis is new and the negative results are honestly reported. My recommendation: send to peer review, but the authors need to decompose DSR, release thresholds/code, and soften the naturalness claim.","headline":"RoVo's embedding-level perturbation axis is new and the core DSR claim holds, but the robustness-to-enhancement headline is inflated by counting quality destruction as defense success.","tokens_in":20145,"tokens_out":1495,"would_cite":true,"duration_ms":16388,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RoVo hides adversarial perturbations in neural-codec embedding vectors, making cloned speech fail speaker verification at high rates even after speech enhancement.","keywords":["voice protection","speech synthesis defense","embedding-level adversarial perturbation","neural audio codec","speaker verification","speech enhancement","proactive defense","voice spoofing"],"falsifier":"Filter the post-enhancement test set to samples that a speech recognizer transcribes correctly and that keep a fixed MOS floor, then recompute RoVo's DSR on that subset; if the filtered DSR falls to roughly the level of a signal-level defense, the claimed robustness is largely an artifact of destroyed audio.","tokens_in":19075,"feed_emoji":"🛡️","tokens_out":10028,"duration_ms":99435,"temperature":0.7,"pith_summary":"RoVo is a proposed defense against unauthorized voice cloning: it alters a person's audio so that when a speech-synthesis model tries to clone that voice, the resulting fake speech no longer matches the victim's identity in a speaker-verification check. The paper's core move is to inject the protective distortion into the embedding vectors of a neural audio codec rather than into the raw waveform, which makes the defense harder for speech-enhancement tools to strip away. In tests against four speech-synthesis models and three verification systems, RoVo raises the defense success rate by more than 70% relative to unprotected speech, reaching 99.5% against a commercial verification API. This matters because current reactive defenses only detect fakes after they exist, while RoVo aims to prevent a posted voice clip from ever being cloned convincingly.","feed_headline":"Codec-level noise blocks cloned voices even after audio cleanup","feed_subtitle":"RoVo's distortion survives cleanup tools that strip signal-level noise, so posted voice clips stay protected.","key_machinery":"RoVo's central object is the embedding-level adversarial perturbation: a budget-limited change applied not to the waveform but to the vector representation produced by a neural audio codec (EnCodec and SoundStream) inside the BARK codec-language-model backbone, with the protected clip reconstructed by decoding the perturbed embedding. The perturbation is optimized by PGD under a PerC-AL (Perceptual Alternating Loss) controller that first minimizes target-based embedding distance to a wrong speaker and then switches to an SNR constraint to keep the distortion small. This machinery does the work of making the defense survive enhancement: the perturbation lives in the same representation as genuine speech structure, so removing it is indistinguishable from destroying the speech.","core_discovery":"The paper claims that perturbing the high-dimensional embedding representation of speech—inside a neural audio codec language model—protects a voice from cloning while surviving speech-enhancement post-processing. Because the perturbation is entangled with the codec's representation of the entire acoustic signal, an enhancement model cannot remove it without damaging the speech itself, whereas signal-level perturbations such as Antifake's sit on the waveform and are easily filtered out. The optimization alternates between a target-based loss that pushes the protected audio's speaker embedding toward a different speaker and an SNR loss that caps distortion, under a PGD loop. Experiments report high DSR across SV2TTS, YourTTS, AdaptVC, and black-box Tortoise, small DSR drops after spectral masking, DeepFilterNet, and MP-SENet enhancement, and better transfer than Antifake. The paper also observes that some post-enhancement DSR gains come from enhancement severely distorting the protected audio rather than from perturbation persistence, an observation that frames how the robustness numbers should be interpreted.","pith_inferences":["The reported MOS values (roughly 1.97–2.91 before enhancement) imply that protected audio is noticeably less natural than the original; 'preserves naturalness' should be read as preserving intelligibility rather than perceptual transparency.","Because the DSR metric counts any verification failure as success, a sharper evaluation is to compute DSR only on enhanced samples that remain intelligible; that would separate perturbation persistence from quality collapse.","An adaptive attacker could train a speech-enhancement model on RoVo-protected audio and try to learn the perturbation's signature; the paper tests only non-adaptive enhancement, so this is an open arms race.","The same embedding-level mechanism could be combined with watermarking or synthetic-speech detection to cover post-hoc attribution, but RoVo does not explore that layering."],"forward_implications":["Voice samples shared on social media or customer-service channels could be pre-protected, so cloning attempts yield fakes that fail speaker verification.","Because the defense survives standard enhancement tools, an attacker cannot trivially undo it by running a denoiser before synthesis.","The transfer results suggest a defense tuned on a few known synthesizers can partially protect against a held-out black-box synthesizer.","The 99.5% DSR on a commercial verification API suggests the method can plug into existing fraud-prevention pipelines without requiring the platform to change its verification model."],"supporting_citations":[{"why":"Supplies Antifake, the signal-level defense baseline, and the target-based speaker-embedding loss that RoVo adapts and compares against.","marker":"[50]"},{"why":"Provides the Projected Gradient Descent optimization loop that constructs the embedding perturbations.","marker":"[30]"},{"why":"EnCodec, one of the neural audio codecs whose high-dimensional embeddings form the perturbation surface.","marker":"[17]"},{"why":"SoundStream, the other neural audio codec whose embeddings RoVo perturbs and decodes into protected speech.","marker":"[52]"},{"why":"BARK, the neural codec language model used as RoVo's backbone for encoding and reconstructing speech.","marker":"[5]"},{"why":"Shows that neural audio codec language model embeddings carry speaker identity, the premise RoVo exploits for defense.","marker":"[35]"},{"why":"PerC-AL, the alternating loss strategy RoVo imports to balance speaker-embedding distortion against audio quality.","marker":"[54]"},{"why":"SV2TTS, one of the four speech-synthesis attack models used as a white-box test bed for DSR.","marker":"[24, 25]"},{"why":"YourTTS, another white-box attack model used to evaluate RoVo's defense success.","marker":"[13]"},{"why":"Commercial speaker-verification API that produced RoVo's headline 99.5% DSR result.","marker":"[31]"}],"fun_headline_variants":["Embedding-level noise blocks voice cloning even after audio cleanup","Perturbing speech embeddings thwarts voice cloning and survives cleanup","RoVo: Embedding attack foils voice cloning despite enhancement","Voice protection that hides in codec embeddings, immune to cleanup","99% defense: RoVo's embedding noise stops cloning post-enhancement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a high Defense Success Rate—synthesized audio failing speaker verification—signals genuine protection, yet the paper's own observation that speech enhancement sometimes raises DSR by severely distorting the protected audio means defense success and quality destruction are not cleanly separated.","fun_headline_variants_meta":{"raw":{"variants":["Embedding-level noise blocks voice cloning even after audio cleanup","Perturbing speech embeddings thwarts voice cloning and survives cleanup","RoVo: Embedding attack foils voice cloning despite enhancement","Voice protection that hides in codec embeddings, immune to cleanup","99% defense: RoVo's embedding noise stops cloning post-enhancement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000486,"raw_usage":{"total_tokens":2412,"prompt_tokens":974,"completion_tokens":1438,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":1358}},"tokens_in":590,"tokens_out":1438,"duration_ms":12113,"temperature":1.0,"reasoning_tokens":1358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:28:19.433255+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Filter the post-enhancement test set to samples that a speech recognizer transcribes correctly and that keep a fixed MOS floor, then recompute RoVo's DSR on that subset; if the filtered DSR falls to roughly the level of a signal-level defense, the claimed robustness is largely an artifact of destroyed audio.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Antifake, the signal-level defense baseline, and the target-based speaker-embedding loss that RoVo adapts and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"BARK, the neural codec language model used as RoVo's backbone for encoding and reconstructing speech."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that neural audio codec language model embeddings carry speaker identity, the premise RoVo exploits for defense."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PerC-AL, the alternating loss strategy RoVo imports to balance speaker-embedding distortion against audio quality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"YourTTS, another white-box attack model used to evaluate RoVo's defense success."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Commercial speaker-verification API that produced RoVo's headline 99.5% DSR result."}],"review_version":1}