Pith. sign in

REVIEW 4 cited by

Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.08967 v2 pith:Y32ZSMFP submitted 2025-06-10 cs.SD cs.CLeess.AS

Ailin Huang , Bingxin Li , Bruce Wang , Boyong Wu , Chao Yan , Chengli Feng , Heng Wang , Hongyu Zhou
show 68 more authors
Hongyuan Wang Jingbei Li Jianjian Sun Joanna Wang Mingrui Chen Peng Liu Ruihang Miao Shilei Jiang Tian Fei Wang You Xi Chen Xuerui Yang Yechang Huang Yuxiang Zhang Zheng Ge Zheng Gong Zhewei Huang Zixin Zhang Bin Wang Bo Li Buyun Ma Changxin Miao Changyi Wan Chen Xu Dapeng Shi Dingyuan Hu Enle Liu Guanzhe Huang Gulin Yan Hanpeng Hu Haonan Jia Jiahao Gong Jiaoren Wu Jie Wu Jie Yang Junzhe Lin Kaixiang Li Lei Xia Longlong Gu Ming Li Nie Hao Ranchen Ming Shaoliang Pang Siqi Liu Song Yuan Tiancheng Cao Wen Li Wenqing He Xu Zhao Xuelin Zhang Yanbo Yu Yinmin Zhong Yu Zhou Yuanwei Liang Yuanwei Lu Yuxiang Yang Zidong Yang Zili Zhang Binxing Jiao Heung-Yeung Shum Jiansheng Chen Jing Li Xiangyu Zhang Xinhao Zhang Yibo Zhu Daxin Jiang Shuchang Zhou Chen Hu
This is my paper
classification cs.SDcs.CLeess.AS
keywords audioend-to-endlalmsmodelspeechstep-audio-aqaaaqaafully
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Audio-Language Models (LALMs) have significantly advanced intelligent human-computer interaction, yet their reliance on text-based outputs limits their ability to generate natural speech responses directly, hindering seamless audio interactions. To address this, we introduce Step-Audio-AQAA, a fully end-to-end LALM designed for Audio Query-Audio Answer (AQAA) tasks. The model integrates a dual-codebook audio tokenizer for linguistic and semantic feature extraction, a 130-billion-parameter backbone LLM and a neural vocoder for high-fidelity speech synthesis. Our post-training approach employs interleaved token-output of text and audio to enhance semantic coherence and combines Direct Preference Optimization (DPO) with model merge to improve performance. Evaluations on the StepEval-Audio-360 benchmark demonstrate that Step-Audio-AQAA excels especially in speech control, outperforming the state-of-art LALMs in key areas. This work contributes a promising solution for end-to-end LALMs and highlights the critical role of token-based vocoder in enhancing overall performance for AQAA tasks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    VITA-QinYu is the first expressive end-to-end spoken language model supporting role-playing and singing alongside conversation, trained on 15.8K hours of data and outperforming prior models on expressiveness and conve...

  2. Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints

    cs.SD 2026-04 unverdicted novelty 6.0 of 10

    Rule-generated preference data aligned via sequential DPO and KTO reduces musical constraint violations and improves coherence in lyric-to-melody generation over baselines.

  3. Step-Audio 2 Technical Report

    cs.CL 2025-07 unverdicted novelty 6.0 of 10

    Step-Audio 2 integrates a latent audio encoder, reasoning-centric reinforcement learning, and discrete audio token generation into language modeling to deliver state-of-the-art performance on audio understanding and c...

  4. WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training

    cs.AI 2026-04 unverdicted novelty 5.0 of 10

    WavAlign introduces an adaptive hybrid post-training recipe that makes reinforcement learning practical for spoken dialogue models by separating semantic preference updates from acoustic anchoring and regulating their...

Pith tools