REVIEW 7 cited by
LLaSM: Large Language and Speech Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Multi-modal large language models have garnered significant interest recently. Though, most of the works focus on vision-language multi-modal models providing strong capabilities in following vision-and-language instructions. However, we claim that speech is also an important modality through which humans interact with the world. Hence, it is crucial for a general-purpose assistant to be able to follow multi-modal speech-and-language instructions. In this work, we propose Large Language and Speech Model (LLaSM). LLaSM is an end-to-end trained large multi-modal speech-language model with cross-modal conversational abilities, capable of following speech-and-language instructions. Our early experiments show that LLaSM demonstrates a more convenient and natural way for humans to interact with artificial intelligence. Specifically, we also release a large Speech Instruction Following dataset LLaSM-Audio-Instructions. Code and demo are available at https://github.com/LinkSoul-AI/LLaSM and https://huggingface.co/spaces/LinkSoul/LLaSM. The LLaSM-Audio-Instructions dataset is available at https://huggingface.co/datasets/LinkSoul/LLaSM-Audio-Instructions.
Forward citations
Cited by 7 Pith papers
-
A Synonymous Variational Perspective on the Rate-Distortion-Perception Tradeoff
Synset-based reconstruction and synonymous variational inference are claimed to derive the distributional divergence in RDP and unify it with classical rate-distortion theory.
-
MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Mesh
MeshLLM improves LLM-based 3D mesh understanding and generation through primitive decomposition, a 1500k+ sample dataset, and topology-focused training strategies.
-
Unlocking Speech Instruction Data Potential with Query Rewriting
A multi-LLM rewriting and multi-agent validation pipeline makes text-to-speech synthesized speech instruction data far more usable and improves downstream speech instruction following.
-
EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
EdgeLoRA combines automatic adapter routing, LRU caching with a memory pool, and grouped LoRA batching to serve thousands of LoRA adapters on edge devices with up to 4x higher throughput than llama.cpp.
-
Enhancing Generalization of Speech Large Language Models with Multi-Task Behavior Imitation and Speech-Text Interleaving
Multi-task behavior imitation with speech-text interleaving improves speech LLM generalization on prompts and zero-shot tasks using only paired speech and transcripts.
-
Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data
Listen, Perceive, Express (LPE) uses two-stage ASR/SER training plus chain-of-thought prompting to let a frozen LLM generate empathetic responses from speech without question-answer fine-tuning data.
-
Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges
A broad but error-prone survey of LLM and MLLM architectures, training methods, benchmarks, and challenges.
Discussion (0). Continue with ORCID to comment.