NeMo: a toolkit for building AI applications using Neural Modules
read the original abstract
NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition. NeMo is built around neural modules, conceptual blocks of neural networks that take typed inputs and produce typed outputs. Such modules typically represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system. The toolkit comes with extendable collections of pre-built modules for automatic speech recognition and natural language processing. Furthermore, NeMo provides built-in support for distributed training and mixed precision on latest NVIDIA GPUs. NeMo is open-source https://github.com/NVIDIA/NeMo
This paper has not been read by Pith yet.
Forward citations
Cited by 22 Pith papers
-
Online Dynamic Batching with Formal Guarantees for LLM Training
ODB is an online batching system for distributed LLM training that forms batches post-preprocessing, provides formal deadlock-free guarantees via the Distributed Group Alignment Problem, and reports 1.58-3.78x through...
-
Cross-Talk Speech Reduction, by Separation, for Separation
Proposes cross-talk reduction task with CTRnet and pseudo-label far-field separation (PuLSS) to train on real close-talk/far-field pairs, achieving SOTA ASR on CHiME-6 and outperforming guided source separation.
-
The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning
FLAIR enables spoken dialogue AI to conduct continuous latent reasoning while perceiving speech through recursive latent embeddings and an ELBO-based finetuning objective.
-
Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs
Cascaded systems remain the most reliable for speech translation overall, but recent SpeechLLMs match or outperform them in many conditions while standalone speech models lag.
-
Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier
Plug-in losses approximate EDL training objectives at the Dirichlet mean with decaying error as evidence grows, including softmax under a specific mapping, and match classical EDL performance on Google Speech Commands.
-
Real-Time Streamable Generative Speech Restoration with Flow Matching
A streaming flow-matching model performs real-time generative speech restoration across multiple tasks at 48 ms latency with state-of-the-art quality.
-
Improving Speech Recognition of Named Entities in Classroom Speech with LLM Revision and Phonetic-Semantic Context
An LLM-based revision method with phonetic-semantic context reduces named entity word error rate by up to 30% relative on a new 45-hour MIT classroom speech dataset.
-
Audio--Image Alignment as a Continued-Pretraining Stage Improves Low-Resource ASR
Audio-image representation alignment as a continued-pretraining stage improves low-resource ASR performance without requiring transcription data.
-
Scaling Conversational Hungarian ASR: The BEA-Dialogue+ Corpus
BEA-Dialogue+ expands the prior 85-hour Hungarian dialogue corpus to 200 hours via relaxed splits and demonstrates that SOT fine-tuning improves Whisper and FastConformer performance on word and character error metrics.
-
Orbax: Distributed Checkpointing with JAX
Orbax is a JAX-native distributed checkpointing library claiming up to 3.5x faster saves and 2x faster loads than PyTorch alternatives.
-
Orbax: Distributed Checkpointing with JAX
Orbax supplies a modular JAX-native checkpointing library that claims up to 3.5× faster saves and 2× faster loads than comparable PyTorch solutions.
-
Mind the Gap: Impact of Synthetic Conversational Data on Multi-Talker ASR and Speaker Diarization
Task-dependent simulation strategies for synthetic conversational data allow synthetic-only training to approach real-data baselines for multi-talker ASR and diarization, with mixing yielding further gains.
-
Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models
Newer LLM backbones in VLMs do not always improve performance; gains are task-dependent, with VQA models solving different questions due to better confidence calibration and stable representations.
-
Exploring Speech Foundation Models for Speaker Diarization Across Lifespan
Adult-trained speech foundation models lose diarization accuracy on child and older-adult speech, but multi-age joint training and adaptation improve robustness across the lifespan.
-
NVILA: Efficient Frontier Visual Language Models
NVILA improves on VILA with a scale-then-compress visual token strategy and full-lifecycle efficiency optimizations, matching or exceeding leading VLMs on image and video benchmarks while reducing training cost 1.9-5....
-
SE-AGCNet: An End-to-End Framework for Joint Speech Enhancement and Loudness Control in Meeting Scenarios
SE-AGCNet jointly optimizes speech enhancement and loudness control in an end-to-end neural network for meeting scenarios, achieving target loudness while improving speech quality and ASR accuracy over separate-module...
-
Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR
Multilingual encoder warm-start gives a data-scale-limited WER advantage in cross-lingual streaming ASR that decays with target data volume and is independent of latency tier.
-
IRAF: Interference-Resilient Adaptive Fusion for Noise-Robust End-to-End Full-Duplex Spoken Dialogue Systems
IRAF introduces an adaptive fusion module that uses a predicted scalar reliability gate to reduce the impact of interfering speakers on user audio representations in end-to-end full-duplex spoken dialogue systems, wit...
-
Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026
KIT's IWSLT submission uses segment concatenation, LLM label generation and cross-lingual translation to create >1M long-form training instances and shows that likelihood re-ranking harms semantic tasks unless combine...
-
Exploring Speech Foundation Models for Speaker Diarization Across Lifespan
Cross-lifespan evaluation shows adult-trained speech foundation models degrade on child and older-adult data, with joint multi-age training and targeted adaptation improving robustness especially using Whisper encoder.
-
The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning
FLAIR enables simultaneous latent reasoning during speech input in full-duplex dialogue models via recursive latent embeddings and an ELBO-based training objective without added latency.
-
BUT System Description for CHiME-9 MCoRec Challenge
BUT's CHiME-9 MCoRec system conditions Parakeet-v2 ASR on AV-HuBERT visuals for 33.7% WER and uses Qwen3.5 LLM for hierarchical clustering to reach 0.97 F1, beating the baseline by 16.2% WER and 0.15 F1 on the develop...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.