Pith. sign in

REVIEW 4 cited by

SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1910.11559 v4 pith:4UXSHCUT submitted 2019-10-25 cs.CL cs.SDeess.AS

Yung-Sung Chuang , Chi-Liang Liu , Hung-Yi Lee , Lin-shan Lee This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords modelend-to-endspokentasksansweringlanguagequestionspeechbert
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

While various end-to-end models for spoken language understanding tasks have been explored recently, this paper is probably the first known attempt to challenge the very difficult task of end-to-end spoken question answering (SQA). Learning from the very successful BERT model for various text processing tasks, here we proposed an audio-and-text jointly learned SpeechBERT model. This model outperformed the conventional approach of cascading ASR with the following text question answering (TQA) model on datasets including ASR errors in answer spans, because the end-to-end model was shown to be able to extract information out of audio data before ASR produced errors. When ensembling the proposed end-to-end model with the cascade architecture, even better performance was achieved. In addition to the potential of end-to-end SQA, the SpeechBERT can also be considered for many other spoken language understanding tasks just as BERT for many text processing tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Spoken question answering for visual queries

    eess.AS 2025-05 conditional novelty 6.0 of 10

    A LLaVA-style model with an added Whisper speech encoder answers spoken questions about images, trained on TTS-synthesized speech and reaching near the text-input baseline.

  2. Reasoning-Based Approach with Chain-of-Thought for Alzheimer's Detection Using Speech and Large Language Models

    cs.AI 2025-06 conditional novelty 4.0 of 10

    Adding chain-of-thought prompts with hand-picked picture cues to an LLM classifier improves Alzheimer's detection accuracy on ADReSS from 75% to 83.3% with ASR transcripts and to 87.5% with manual transcripts, though ...

  3. Amplifying Emotional Signals: Data-Efficient Deep Learning for Robust Speech Emotion Recognition

    eess.AS 2025-08 reject novelty 3.0 of 10

    A pretrained ResNet34 with augmentation reaches 66.7% accuracy on a combined RAVDESS/SAVEE emotion set, but only on a validation split, so the claimed new benchmark is unverified.

  4. DeepEmoNet: Building Machine Learning Models for Automatic Emotion Recognition in Human Speeches

    eess.AS 2025-08 conditional novelty 2.0 of 10

    A ResNet34 pretrained on ImageNet and fine-tuned on log-mel spectrograms, with data augmentation, classifies eight speech emotions at 66.7% accuracy and F1 0.631 on the pooled RAVDESS/SAVEE validation set.

Pith tools