Pith. sign in

REVIEW 4 cited by

SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.09424 v1 pith:Z3ZJ6F7L submitted 2023-10-13 cs.CL cs.HCcs.SDeess.AS

Zhehuai Chen , He Huang , Andrei Andrusenko , Oleksii Hrinchuk , Krishna C. Puvvada , Jason Li , Subhankar Ghosh , Jagadeesh Balam
show 1 more author
Boris Ginsburg
This is my paper
classification cs.CLcs.HCcs.SDeess.AS
keywords speechin-contextlearningsalmmodelcapabilitieslanguageproposed
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present a novel Speech Augmented Language Model (SALM) with {\em multitask} and {\em in-context} learning capabilities. SALM comprises a frozen text LLM, a audio encoder, a modality adapter module, and LoRA layers to accommodate speech input and associated task instructions. The unified SALM not only achieves performance on par with task-specific Conformer baselines for Automatic Speech Recognition (ASR) and Speech Translation (AST), but also exhibits zero-shot in-context learning capabilities, demonstrated through keyword-boosting task for ASR and AST. Moreover, {\em speech supervised in-context training} is proposed to bridge the gap between LLM training and downstream speech tasks, which further boosts the in-context learning ability of speech-to-text models. Proposed model is open-sourced via NeMo toolkit.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Phonemes vs. Projectors: An Investigation of Speech-Language Interfaces for LLM-based ASR

    eess.AS 2026-04 unverdicted novelty 7.0 of 10

    Phoneme-based interfaces match or surpass projector-based ones for LLM ASR, especially in low-resource languages, and a BPE-phoneme hybrid offers additional improvements.

  2. TRADE: Transducer-Augmented Decoder for Speech LLM

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    TRADE augments multimodal Speech LLMs with a transducer branch for streaming ASR, reporting 6.71% WER offline and 8.40% streaming on the Open ASR Leaderboard from one checkpoint.

  3. Safety-Oriented Evaluation of Language Understanding Systems for Air Traffic Control

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    A consequence-aware evaluation framework applied to LLMs in ATC finds peak Risk Score of only 0.69 despite high macro-F1, with errors concentrated in high-impact entities.

  4. Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accents

    cs.CL 2026-05 unverdicted novelty 4.0 of 10

    Audio language models are benchmarked on five semantic and paralinguistic reasoning tasks to reveal limitations in handling spoken audio evidence, accent variation, and domain shifts.

Pith tools