Pith. sign in

REVIEW 4 cited by

Semantic-WER: A Unified Metric for the Evaluation of ASR Transcript for End Usability

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2106.02016 v2 pith:Q3KA24K4 submitted 2021-06-03 cs.CL cs.SDeess.AS

Somnath Roy This is my paper
classification cs.CLcs.SDeess.AS
keywords metricsystemsdownstreamevaluationsemantic-werswerachievedadvances
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advances in supervised, semi-supervised and self-supervised deep learning algorithms have shown significant improvement in the performance of automatic speech recognition(ASR) systems. The state-of-the-art systems have achieved a word error rate (WER) less than 5%. However, in the past, researchers have argued the non-suitability of the WER metric for the evaluation of ASR systems for downstream tasks such as spoken language understanding (SLU) and information retrieval. The reason is that the WER works at the surface level and does not include any syntactic and semantic knowledge.The current work proposes Semantic-WER (SWER), a metric to evaluate the ASR transcripts for downstream applications in general. The SWER can be easily customized for any down-stream task.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    Agentic ASR adds closed-loop semantic correction to ASR and introduces S²ER, an LLM judge for meaning-level errors, showing larger gains on semantic than token metrics across multilingual benchmarks.

  2. Evaluation of Automatic Speech Recognition Using Generative Large Language Models

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    Decoder-based LLMs reach 92-94% agreement with humans when choosing correct ASR hypotheses, beating WER at 63% and outperforming encoder-based semantic metrics.

  3. Interactive ASR: Towards Human-Like Interaction and Semantic Coherence Evaluation for Agentic Speech Recognition

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    The authors introduce LLM-based semantic judgment and an agentic interaction loop that improves semantic fidelity and enables iterative corrections in automatic speech recognition beyond traditional WER.

  4. Evaluation of Automatic Speech Recognition Using Generative Large Language Models

    cs.CL 2026-04 conditional novelty 4.0 of 10

    Decoder-based LLMs achieve 92-94% agreement with human annotators for ASR hypothesis selection on HATS, substantially outperforming WER (63%) and embedding-based semantic metrics.

Pith tools