AccLock extracts user-specific features from in-ear ballistocardiogram signals via a disentanglement model and Siamese network to achieve average FAR of 3.13% and FRR of 2.99% in tests with 33 participants.
hub Mixed citations
Conformer: Convolution-augmented Transformer for Speech Recognition
Mixed citation behavior. Most common role is method (60%).
31
Pith papers citing it
385
external citations · Pith
Method
60% of classified citations
abstract
Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. In this work, we achieve the best of both worlds by studying how to combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. To this regard, we propose the convolution-augmented transformer for speech recognition, named Conformer. Conformer significantly outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies. On the widely used LibriSpeech benchmark, our model achieves WER of 2.1%/4.3% without using a language model and 1.9%/3.9% with an external language model on test/testother. We also observe competitive performance of 2.7%/6.3% with a small model of only 10M parameters.
hub tools
citation-role summary
method 3
background 2
citation-polarity summary
representative citing papers
NodePFN pre-trains on synthetic graphs with controllable homophily and causal feature-label models to achieve 71.27 average accuracy on 23 node classification benchmarks without graph-specific training.
Source-aware MT metrics adapted to speech translation via ASR transcripts or back-translations as audio proxies, plus a new cross-lingual re-segmentation algorithm, improve correlation with human judgments over reference-only baselines.
FlashAttention-3 achieves 1.5-2x speedup on H100 GPUs for attention, reaching 740 TFLOPs/s (75% utilization) in FP16 and near 1.2 PFLOPs/s in FP8 while cutting numerical error by 2.6x versus baseline FP8 attention.
TRADE augments multimodal Speech LLMs with a transducer branch for streaming ASR, reporting 6.71% WER offline and 8.40% streaming on the Open ASR Leaderboard from one checkpoint.
M2S-AVSR introduces multi-view self-supervised visual encoding and modality-aware fusion for AVSR, releases the AISHELL8-RealScene dataset, and reports relative gains up to 29.4% on LRS3 under perturbations plus new SOTA on MISP2021.
Defines executable boundary contracts for sound event traces using an STL-embeddable Boolean fragment plus interval and duration clauses, then evaluates them on speech and soundscape data where they disagree with standard scores.
SynIB is an information-theoretic objective that adds a penalty for unimodal confidence to standard task loss, improving accuracy on synergy-dependent examples by up to 7.8% across synthetic XOR tasks and five real-world multimodal benchmarks.
A multi-stage training method for LLM-based ASR uses new entropy allocation metrics to achieve competitive benchmark performance with 2.3B parameters while mitigating hallucinations via better encoder-LLM decoupling.
MALEFA reaches 90% accuracy and 0.007% false alarm rate on AMI for zero-shot KWS via cross-attention and multi-granularity contrastive learning while running efficiently on constrained hardware.
A reinforcement learning attacker manipulates client sensor observations in federated learning to induce repetitive server memory updates, achieving around 70% repeated update rate and enabling remote Rowhammer bit flips on an automatic speech recognition model.
DGSNA dynamically generates scene-specific noise via prompt-driven language models and text-to-audio diffusion, then mixes it with speech to improve recognition and keyword spotting robustness by up to 11.32%.
Scaling an autoregressive Transformer to 20B parameters for text-to-image generation using image token sequences achieves new SOTA zero-shot FID of 7.23 and fine-tuned FID of 3.22 on MS-COCO.
Graph-regularized losses grounded in Russell's circumplex raise best-case EEG emotion accuracy by up to 5.42% and cut non-adjacent misclassifications by up to 39% across transformer, hybrid, and GNN backbones.
UniVoice is a conditional flow matching model with a Diffusion Transformer backbone that unifies TTS and SVS via modality-specific encoders and a null melody token for speech, achieving 5.26% speech PER and 16.22% singing PER.
FastTurn fuses streaming CTC semantics with acoustic cues for lower-latency, more accurate turn detection in full-duplex dialogue and releases a real-dialogue test set.
Iterative pseudo-labeling on 22.4k hours of unlabeled code-switching audio reduces Mix Error Rate on SEAME devman to 12.88% and devsge to 18.89%.
A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-level RAG for hotword customization.
TinyGiantALM, a compact 1.5B audio-language model with instruction-aware refinement, achieves 46.4% zero-shot accuracy on MMAR and outperforms models up to 8x larger in mixed-modality tasks.
FOAM adaptively controls damping and update frequency in Shampoo based on staleness-oriented error approximation to cut wall-clock time while preserving convergence.
ConTrans is a multi-scale encoder fusing convolutional inductive biases with transformer self-attention for improved local-global features in zero-shot temporal action localization.
MedASR is an open-source 105M-parameter ASR model achieving 58% relative WER reduction versus Whisper Large-v3 on medical dictation.
A multichannel energy-based noisy-segment rejection step combined with an MFCC-Conformer classifier improves CAD detection accuracy to 78.4% on 297 subjects, a 4.1% gain over baseline training without rejection.
A survey that classifies non-intrusive ASR refinement methods into five categories, reviews domain adaptation and evaluation datasets, proposes standardized metrics, and identifies future research directions.
citing papers explorer
-
AccLock: Unlocking Identity with Heartbeat Using In-Ear Accelerometers
AccLock extracts user-specific features from in-ear ballistocardiogram signals via a disentanglement model and Siamese network to achieve average FAR of 3.13% and FRR of 2.99% in tests with 33 participants.
-
Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors
NodePFN pre-trains on synthetic graphs with controllable homophily and causal feature-label models to achieve 71.27 average accuracy on 23 node classification benchmarks without graph-specific training.
-
How to Evaluate Speech Translation with Source-Aware Neural MT Metrics
Source-aware MT metrics adapted to speech translation via ASR transcripts or back-translations as audio proxies, plus a new cross-lingual re-segmentation algorithm, improve correlation with human judgments over reference-only baselines.
-
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
FlashAttention-3 achieves 1.5-2x speedup on H100 GPUs for attention, reaching 740 TFLOPs/s (75% utilization) in FP16 and near 1.2 PFLOPs/s in FP8 while cutting numerical error by 2.6x versus baseline FP8 attention.
-
TRADE: Transducer-Augmented Decoder for Speech LLM
TRADE augments multimodal Speech LLMs with a transducer branch for streaming ASR, reporting 6.71% WER offline and 8.40% streaming on the Open ASR Leaderboard from one checkpoint.
-
M2S-AVSR: Modality-aware Multi-view Self-supervised Representation for Robust Audio-Visual Speech Recognition
M2S-AVSR introduces multi-view self-supervised visual encoding and modality-aware fusion for AVSR, releases the AISHELL8-RealScene dataset, and reports relative gains up to 29.4% on LRS3 under perturbations plus new SOTA on MISP2021.
-
Executable Boundary Contracts for Sound Event Traces
Defines executable boundary contracts for sound event traces using an STL-embeddable Boolean fragment plus interval and duration clauses, then evaluates them on speech and soundscape data where they disagree with standard scores.
-
SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning
SynIB is an information-theoretic objective that adds a penalty for unimodal confidence to standard task loss, improving accuracy on synergy-dependent examples by up to 7.8% across synthetic XOR tasks and five real-world multimodal benchmarks.
-
Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
A multi-stage training method for LLM-based ASR uses new entropy allocation metrics to achieve competitive benchmark performance with 2.3B parameters while mitigating hallucinations via better encoder-LLM decoupling.
-
MALEFA: Multi-grAnularity Learning and Effective False Alarm Suppression for Zero-shot Keyword Spotting
MALEFA reaches 90% accuracy and 0.007% false alarm rate on AMI for zero-shot KWS via cross-attention and multi-granularity contrastive learning while running efficiently on constrained hardware.
-
Remote Rowhammer Attack using Adversarial Observations on Federated Learning Clients
A reinforcement learning attacker manipulates client sensor observations in federated learning to induce repetitive server memory updates, achieving around 70% repeated update rate and enabling remote Rowhammer bit flips on an automatic speech recognition model.
-
DGSNA: Dynamic Generative Scene-based Noise Addition method
DGSNA dynamically generates scene-specific noise via prompt-driven language models and text-to-audio diffusion, then mixes it with speech to improve recognition and keyword spotting robustness by up to 11.32%.
-
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
Scaling an autoregressive Transformer to 20B parameters for text-to-image generation using image token sequences achieves new SOTA zero-shot FID of 7.23 and fine-tuned FID of 3.22 on MS-COCO.
-
Graph-Regularized Deep Learning for EEG-Based Emotion Recognition with Psychologically-Grounded Label Structure
Graph-regularized losses grounded in Russell's circumplex raise best-case EEG emotion accuracy by up to 5.42% and cut non-adjacent misclassifications by up to 39% across transformer, hybrid, and GNN backbones.
-
UniVoice: A Unified Model for Speech and Singing Voice Generation
UniVoice is a conditional flow matching model with a Diffusion Transformer backbone that unifies TTS and SVS via modality-specific encoders and a null melody token for speech, achieving 5.26% speech PER and 16.22% singing PER.
-
FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection
FastTurn fuses streaming CTC semantics with acoustic cues for lower-latency, more accurate turn detection in full-duplex dialogue and releases a real-dialogue test set.
-
Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR
Iterative pseudo-labeling on 22.4k hours of unlabeled code-switching audio reduces Mix Error Rate on SEAME devman to 12.88% and devsge to 18.89%.
-
NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-level RAG for hotword customization.
-
TinyGiantALM: A Compact Audio-Language Model for Intent-Aware Reasoning under Resource Constraints
TinyGiantALM, a compact 1.5B audio-language model with instruction-aware refinement, achieves 46.4% zero-shot accuracy on MMAR and outperforms models up to 8x larger in mixed-modality tasks.
-
FOAM: Frequency and Operator Error-Based Adaptive Damping Method for Reducing Staleness-Oriented Error for Shampoo
FOAM adaptively controls damping and update frequency in Shampoo based on staleness-oriented error approximation to cut wall-clock time while preserving convergence.
-
ConTrans: Learning Text-enhanced Local-global Temporal Representations for Zero-shot Temporal Action Localization
ConTrans is a multi-scale encoder fusing convolutional inductive biases with transformer self-attention for improved local-global features in zero-shot temporal action localization.
-
MedASR: An Open-Source Model for High-Accuracy Medical Dictation
MedASR is an open-source 105M-parameter ASR model achieving 58% relative WER reduction versus Whisper Large-v3 on medical dictation.
-
Noise-Robust Contrastive Learning with an MFCC-Conformer For Coronary Artery Disease Detection
A multichannel energy-based noisy-segment rejection step combined with an MFCC-Conformer classifier improves CAD detection accuracy to 78.4% on 297 subjects, a 4.1% gain over baseline training without rejection.
-
Non-Intrusive Automatic Speech Recognition Refinement: A Survey
A survey that classifies non-intrusive ASR refinement methods into five categories, reviews domain adaptation and evaluation datasets, proposes standardized metrics, and identifies future research directions.
-
From Objectives to Applications: Aligning Architectural Biases in Audio Self-Supervised Learning
A survey that organizes audio SSL into five objective paradigms, relates their demands to architectural biases, and interprets downstream applications as tests of generalization.
-
Responsible ASR: Overcoming Challenges of Foundational Models in Narrow-Band and Low-Resource Settings
Evaluation of open-source and commercial ASR models on narrow-band Hindi and Indian English shows poor zero-shot results and inconsistent fine-tuning benefits tied to pretraining exposure.
-
Data-Efficient On-Policy Distillation for Automatic Speech Recognition
On-policy distillation from a Qwen-ASR teacher improves a 0.6B Ark-ASR model over supervised fine-tuning and a same-scale baseline on four of five ASR benchmarks using 100k hours of speech.
-
ResAF-Net: An Anchor-Free Attention-Based Network for Tree Detection and Agricultural Mapping in Palestine
ResAF-Net detects trees in satellite imagery for Palestinian agriculture, reaching 82% recall and 63% mAP@0.5 on the MillionTrees validation set and deployed in a web GIS.
-
Music Audio-Visual Question Answering Requires Specialized Multimodal Designs
Survey of Music AVQA finds specialized input processing, dedicated spatial-temporal designs, and music-specific modeling are critical for strong performance.
-
Spatial Speech Perception Systems: A Survey of Sound Source Localization, Directional Enhancement, and Speech Recognition
A survey of spatial speech perception systems covering sound source localization, directional enhancement, and automatic speech recognition methods and their integration.
-
Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
A tutorial synthesizing foundations, recent models such as PALO and Maya, and low-cost methods for tri-modal multilingual AI in resource-constrained settings.