FlexiSLM is the first spoken language model supporting dynamic and controllable frame rates on speech input and output, outperforming fixed-rate 7B models at high quality and enabling faster inference at lower rates like 6.25 Hz.
hub
https://arxiv.org/abs/2312.05187
27 Pith papers cite this work. Polarity classification is still indexing.
27
Pith papers citing it
hub tools
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
Gumbel-BEARD automates Whisper layer selection with Gumbel-Softmax and BEST-RQ for self-supervised domain adaptation, matching fully supervised performance on 10h vs 133h data and setting new SOTA WERs on MyST and OGI datasets.
AffectCodec is an emotion-guided neural speech codec that preserves emotional cues during quantization while maintaining semantic fidelity and prosodic naturalness.
Multilingual ASR models show 39.7-297% zero-shot WER on Pashto public data, Whisper models output correct script in under 0.8% of cases, and fine-tuned models degrade to 32.5-59% WER on out-of-domain sets.
Appropriateness of TTS varies independently across domains while naturalness scores penalize stylized speech and reward spontaneity.
Next-Turn introduces time-to-next-speech-onset prediction for duration-aware streaming endpoint detection, reporting a 25.9% improvement in accuracy within 320 ms.
Self-guidance adds a lightweight feature-mapping loss to align decoder manifolds in VQ-VAE speech codecs, raising reconstruction metrics and allowing 4x codebook reduction with no fidelity loss.
HybridCodec unifies SSL distillation and dual-stream design in a neural audio codec for improved semantic specialization, competitive reconstruction, and faster inference.
Audio-Interaction unifies offline and online audio tasks into one streaming model via the SoundFlow framework and a new 2.6M-item streaming corpus, enabling real-time instruction following and proactive responses.
COMPASS is a new reproducible benchmarking framework for S2ST that deploys 46 metrics on 1248 configurations, shows single-metric rankings mislead, reduces to 10 metrics per direction, and finds domain-specific metrics better match human judgments than standalone MOS predictors.
MindVoice disentangles neural-to-speech reconstruction into semantic and acoustic pathways using pretrained priors, then fuses them with speech generation models to produce intelligible output from non-invasive recordings.
PoDAR disentangles audio signal power from semantic content in latents using power augmentation and consistency objectives, yielding 2x faster convergence and gains of 0.055 speaker similarity and 0.22 UTMOS when applied to Stable Audio VAE with F5-TTS.
Emotion embedding similarities are unsuitable for zero-shot evaluation of emotional expressiveness in speech generation due to confounding by non-emotional acoustic features.
DM-ASR reformulates multi-speaker ASR as multi-turn dialogue generation conditioned on diarization results, achieving competitive benchmark performance with relatively small models and limited data.
MoVE uses specialized LoRA expert adapters and a soft router to translate non-verbal vocalizations in S2ST, reproducing them in 76% of cases versus at most 14% for baselines while scoring highest on naturalness and emotional fidelity.
KM-Speaker introduces a keypoint-conditioned generative framework for speech-driven 3D facial animation offering global style guidance and frame-level temporal control via disentangled lip and upper-face dynamics.
OscillaTTS adds an adaptive oscillatory nonlinearity with linear bypass to diffusion TTS decoders for improved modeling of abrupt prosodic dynamics, reporting gains on LJSpeech and Emotional Speech Dataset.
Zero-VC applies speaker anonymization as a perturbation to achieve strictly causal zero-lookahead streaming voice conversion by balancing timbre leakage against prosodic utility.
BrainWorld is a structural-prior-conditioned generative model that produces stable whole-brain 4D fMRI trajectories up to 400 frames, augments downstream tasks, and learns transferable multimodal representations across 22 datasets.
Proxy-Anchor metric learning on Wav2Vec2-BERT embeddings with architecture merging achieves 99.76% closed-set accuracy and 2.04% FPR@95 OOD detection on MLAAD v9, doubling prior OOD accuracy on v5 splits.
OpenWER improves cross-lingual WER robustness via language-specific normalization, compound word detection, and token-based Levenshtein alignment, reporting up to 25% absolute reductions across 52 languages.
Fine-tuned Wav2Vec2 and HuBERT models recognize click consonants more accurately than non-clicks in G|ui and West !Xoon data.
Empirical study of unit vocoders finds cluster size controls intelligibility via phonetic discriminability and speaker conditioning is required to avoid collapse in multilingual multi-speaker settings.
Demographics-agnostic training with augmentation and distillation reduces predictive disparity in wake-up word detection by 40-84% across demographic groups.
citing papers explorer
-
KM-Speaker: Keypoint-Based Style Control for High-Quality Speech-Driven 3D Facial Animation and Dialogue Localization
KM-Speaker introduces a keypoint-conditioned generative framework for speech-driven 3D facial animation offering global style guidance and frame-level temporal control via disentangled lip and upper-face dynamics.
-
BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics
BrainWorld is a structural-prior-conditioned generative model that produces stable whole-brain 4D fMRI trajectories up to 400 frames, augments downstream tasks, and learns transferable multimodal representations across 22 datasets.