Pith. sign in

REVIEW 5 cited by

UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.16083 v1 pith:CUJFI7WE submitted 2023-06-28 cs.SD eess.AS

Heeseung Kim , Sungwon Kim , Jiheum Yeom , Sungroh Yoon This is my paper · ORCID
classification cs.SDeess.AS
keywords speechunitmodelunitspeechdatasynthesistasksdecoder
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

We propose UnitSpeech, a speaker-adaptive speech synthesis method that fine-tunes a diffusion-based text-to-speech (TTS) model using minimal untranscribed data. To achieve this, we use the self-supervised unit representation as a pseudo transcript and integrate the unit encoder into the pre-trained TTS model. We train the unit encoder to provide speech content to the diffusion-based decoder and then fine-tune the decoder for speaker adaptation to the reference speaker using a single $<$unit, speech$>$ pair. UnitSpeech performs speech synthesis tasks such as TTS and voice conversion (VC) in a personalized manner without requiring model re-training for each task. UnitSpeech achieves comparable and superior results on personalized TTS and any-to-any VC tasks compared to previous baselines. Our model also shows widespread adaptive performance on real-world data and other tasks that use a unit sequence as input.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Entropy-based Coarse and Compressed Semantic Speech Representation Learning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    Predictive entropy from a token-level speech language model finds merge boundaries, producing compressed semantic tokens that keep ASR and translation accuracy at 15 Hz while lowering latency.

  2. Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model

    cs.SD 2025-02 conditional novelty 6.0 of 10

    PFlow-VC performs expressive voice conversion by conditioning a flow-matching Mel-spectrogram decoder on discrete speaker-normalized pitch tokens and a target speaker prompt, improving emotion style transfer.

  3. Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching

    eess.AS 2025-06 conditional novelty 5.0 of 10

    R-VC performs zero-shot voice conversion in two sampling steps while transferring the target speaker's rhythm, matching or exceeding prior systems in naturalness and intelligibility.

  4. Optimizing Speech Multi-View Feature Fusion through Conditional Computation

    eess.AS 2025-01 conditional novelty 5.0 of 10

    A gradient-sensitive gating network plus multi-stage dropout fuses FBanks and HuBERT features, matching BLEU while cutting MuST-C training epochs by roughly 1.24x.

  5. A2TTS: TTS for Low Resource Indian Languages

    cs.SD 2025-07 conditional novelty 4.0 of 10

    A2TTS adds a reference-audio cross-attention duration predictor to a Grad-TTS and UnitSpeech style diffusion TTS, improving speaker similarity scores in seven Indian languages.

Pith tools