Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Add a difference vector to BERT; Chinese query correction hits 89.6 F1

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

CTD uses BERT token representations plus a composition layer to correct Chinese spelling errors in ASR queries, but the reported gains lack matched baselines and released data.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Plausible ESIM-style composition layer, but the SIGHAN comparison is uninterpretable because the baselines were not given the same MLM fine-tuning. the 3 major comments →

arxiv 2509.04393 v1 pith:CWXYTMQP submitted 2025-09-04 cs.SD cs.CL

Contextualized Token Discrimination for Speech Search Query Correction

Junyu Lu , Di Jiang , Mengze Hong , Victor Junqiu Wei , Qintian Guo , Zhiyang Su This is my paper
classification cs.SD cs.CL
keywords Chinese spelling correctionspeech searchASR error correctioncontextualized token discriminationBERT fine-tuningcomposition layerSIGHANAAM benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a lightweight layer on top of BERT, a pretrained transformer language model, can fix misrecognized words in Chinese speech-search queries, and that the right test for that fix is a benchmark built from real ASR errors. The central move is to give the corrector three views of each token: its original embedding, its context-aware BERT representation, and the difference between the two, so the model can flag tokens that clash with their surroundings. With this setup the method reaches 89.6% F1 on the standard SIGHAN spelling-correction set and 52.2% F1 on a newly constructed AAM set, outperforming BERT and Soft-Masked BERT. The paper also reports that most of the gain comes from an initial masked-language-model fine-tuning step on domain text, a result it calls surprising; the added composition layer then gives a smaller, consistent boost.

Core claim

CTD's core claim is that an erroneous token in an ASR query is detectable by the mismatch between what the token looks like on its own and what BERT thinks it means in context. The paper builds a composition layer that concatenates the input token vector, the contextualized vector from BERT's last layer, and their difference, then feeds that triple to a classifier which selects the correct token from the vocabulary. Because the difference vector makes the mismatch explicit, the classifier can learn to replace exactly the tokens that are out of place while leaving correct tokens alone. The authors report 89.6% F1 on SIGHAN and 52.2% on AAM, and they attribute the largest share of this to doma

What carries the argument

The load-bearing mechanism is the composition layer. For each token i it forms c_i = [v_C_i; v_I_i; v_C_i - v_I_i], concatenating the contextualized representation from BERT's last layer, the original input token representation, and the difference between the two, and sends this triple through a linear classifier. The difference vector is the key: when a token is misrecognized, its contextualized vector points away from its input vector, giving the classifier a direct, explicit signal of the mismatch. Training uses a loss computed only on error tokens, plus a random sample of correct tokens in a 1:5 ratio, so the model learns to keep correct tokens as well as replace wrong ones.

Load-bearing premise

The headline improvement over BERT and Soft-Masked BERT rests on those baselines being trained and scored under exactly the same protocol—including the extra domain-language-model fine-tuning and per-character F1—which the paper does not describe for the baselines; the authors themselves note that the base model's 20-point jump was surprising.

What would settle it

Re-run the published BERT and Soft-Masked BERT baselines using the same domain-language-model fine-tuning, the same token-level loss, and the same per-character F1 scoring on the same SIGHAN and AAM splits; if fine-tuned BERT alone reaches 89.6% and 52.2% F1, then CTD's composition layer contributes nothing beyond its base model.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • With the reported numbers, CTD reaches 89.6% F1 on SIGHAN and 52.2% F1 on the new AAM benchmark, beating BERT (64.9% / 33.2%) and Soft-Masked BERT (66.4% / 33.7%) by a wide margin on both.
  • The biggest part of the gain comes from an initial masked-language-model fine-tuning step on domain text; the base model alone reaches 89.3% on SIGHAN, which the paper calls surprising.
  • The composition layer adds a smaller, consistent boost on both datasets (89.3 to 89.6 and 50.7 to 52.2 F1), so the layer's value is real but modest compared with domain fine-tuning.
  • The method can repair homophone and grammar/logic errors in ASR transcripts, as the examples show, but it still fails on errors that require world knowledge.
  • Because AAM excludes insertion/deletion and stopword errors, the claims are about substitution errors only, not full ASR error correction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not say this, but if the baselines did not receive the same masked-language-model fine-tuning, the roughly 20-point gap likely comes mostly from that fine-tuning rather than from CTD's composition layer.
  • The same input-context-difference composition should transfer to other token-level rewriting tasks, such as grammatical error correction, OCR post-processing, or dialogue repair, where one token is locally inconsistent with its context.
  • A natural extension is to add an alignment or length-prediction head so CTD can handle the insertion and deletion errors the AAM benchmark currently filters out.
  • The paper's own limitations note the reliance on domain-specific training data, so few-shot or zero-shot variants would be the next test of generality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Contextualized Token Discrimination (CTD), a BERT-based method for Chinese speech-search query spelling correction. CTD augments BERT token representations with a composition layer that concatenates the input token embedding, the contextualized embedding from BERT's last layer, and their difference, then applies a linear classifier over the Chinese vocabulary. The method is evaluated on the SIGHAN benchmark and on a newly constructed AAM benchmark built from public Mandarin speech corpora (Aidatatang, AISHELL-1, MagicData) using a Conformer ASR model. Table 1 reports F1 of 89.6% on SIGHAN and 52.2% on AAM, exceeding the cited BERT and Soft-Masked BERT baselines by a large margin, while the proposed CTD improves over the authors' own base model by only 0.3 F1 points on SIGHAN.

Significance. If the reported gains were reliable, CTD would be a simple and practical extension to BERT for ASR query correction, and the AAM dataset would be a useful text-pair benchmark derived from public audio corpora. The architecture is described clearly enough to re-implement, and the use of public speech resources for benchmark construction is a positive step. However, the central empirical claim rests on a single table without variance estimates and on a baseline comparison that mixes a possible training-protocol change with the proposed method. The benchmark's filtering choices further limit the generality of the ASR-correction claims. The contribution is therefore plausible but not yet established at the level needed for publication.

major comments (3)
  1. [Table 1 / §4.1] The 20+ point F1 improvement of the authors' Base Model (89.3) over the reported BERT (64.9) and Soft-Masked BERT (66.4) is explicitly attributed by the authors to an added MLM fine-tuning stage on the training set. The manuscript does not state whether the BERT and Soft-Masked BERT numbers were obtained under the same protocol (same MLM fine-tuning on the same data, same token sampling, same early stopping, same sentence-level F1 computation), nor does it report re-running those baselines. Since the Base Model already reaches 89.3 without the CTD composition layer, the comparison confounds the proposed architecture with training-protocol differences. The 0.3-point CTD-vs-BaseModel increment is small relative to this confound. The claimed superiority over prior BERT-based methods is therefore unsupported unless the baselines are re-run under identical conditions or exact matching citatio
  2. [§4.2 / Limitations] The AAM benchmark explicitly filters to same-length erroneous/ground-truth pairs and removes misrecognized stopwords, which excludes insertion and deletion errors as well as stopword substitutions. As the authors acknowledge in the Limitations section, these are common in ASR transcription. Consequently the reported 52.2% F1 is not a realistic end-to-end ASR-correction figure; it measures a restricted substitution-only, content-word task. The paper nevertheless frames AAM as 'comprehensive' and 'realistic'. The central claim about speech query correction needs to be scoped to this filtered benchmark, or the benchmark needs to include a version that does not filter these error types.
  3. [Table 1 / §4.2] All results are reported as single numbers with no standard deviations, confidence intervals, or significance tests. The advantage of CTD over the authors' Base Model is 0.3 F1 on SIGHAN and 1.5 F1 on AAM. Without multiple seeds or a paired bootstrap, it is not possible to determine whether this small increment is meaningful. At minimum, report mean and std over at least three training runs, and ideally paired significance tests for the CTD-vs-BaseModel comparison.
minor comments (6)
  1. [§4.2] The sentence 'our well-trained methods achieve comparable results to both the previous BERT benchmark and Soft-Masked BERT' is misleading; Table 1 shows substantially higher F1. The text should say 'superior' or 'higher'.
  2. [Figure 2] The caption describes 'the left picture' and 'the right side', but the exact cosine-similarity computation and token mapping are not defined. Please specify how the matrices in the figure are produced, and make the figure legible in grayscale.
  3. [§3.2] Equation (1) uses a semicolon in the concatenation notation without defining it. Also, the set definitions '∀Ti ≠ T'i ∈ Λ' and '∃Ti = T'i ∈ Λ' are mathematically awkward; define Λ1, Λ2, and Λ with standard set notation.
  4. [§4.1] The statement 'we only fine-tuned the MLM using the training set' is confusing because a classification head is also trained. Please separate the two stages: (i) MLM fine-tuning on correct sentences, (ii) supervised training of the correction head, and state which stage is meant in each sentence.
  5. [§4.2] The AAM dataset is said to contain 6965 pairs, but the 'Generated Pairs' column in Table 3 sums to 6965 while the training/validation/test split sums to 6965 as well. Please clarify how the 6965 pairs are partitioned and whether the corpus-specific numbers include all splits.
  6. [References] Several references are formatted inconsistently (e.g., 'CHEN CHEN' in the reference list) and a number of recent citations to the authors' own work appear tangential to the technical content. A pass to tighten the reference list would help.

Circularity Check

0 steps flagged

No significant circularity: CTD is an empirical supervised method evaluated on external benchmarks; the unmatched-baseline issue is an evidence concern, not a circular derivation.

full rationale

The derivation chain is a standard supervised classification setup: BERT contextual representations, a composition layer using concatenation and difference vectors (Eq. 1), a softmax classifier (Eq. 2), and a cross-entropy loss restricted to tokens where the erroneous query differs from the ground truth plus sampled correct tokens (Eq. 3). The target token labels are defined by string comparison between the erroneous query and the ground-truth sentence, not by the model's own outputs, so the training objective is not self-definitional. Evaluation on SIGHAN uses an external public benchmark, and the AAM benchmark is constructed from ASR hypotheses paired with ground-truth transcripts; both are independent of the model's predicted outputs. The paper's many self-citations (e.g., Hong et al. 2025a-d, Wei et al. 2024a-b, Jiang et al. 2013-2021) appear in related work and as motivation/benchmark references, not as load-bearing justification of the CTD mechanism or its uniqueness. The most serious concern is in Section 4.1: the authors report a 'surprising' 20+ point gain over cited BERT/Soft-Masked BERT numbers after adding MLM fine-tuning on the training set, and they do not describe re-running those baselines under the same protocol. That is an unmatched-baseline/evidence problem for the superiority claim, not a circularity, because the superiority is not true by construction and would be resolved by re-running the baselines. The acknowledged same-length and stopword filtering of AAM similarly weakens external validity but does not make the evaluation circular. Therefore no circular step is identified.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim depends only on standard pretrained models, cited observations, and hand-set hyperparameters; no new theoretical entities are introduced.

free parameters (4)
  • training token sampling ratio Λ1:Λ2 = 1:5
    Ratio of error tokens to randomly sampled correct tokens used in the training objective (Section 3.2, Eq. 3). Hand-set, not tuned against a validation metric.
  • peak learning rate = 5e-4
    Peak learning rate in the Transformer schedule, with warmup over first 5% of steps (Section 4.1). Hand-set.
  • batch size = 512
    Mini-batch size for all experiments (Section 4.1). Hand-set.
  • number of epochs for AAM = 25
    Training epochs for the AAM task (Section 4.2). Hand-set.
axioms (4)
  • domain assumption BERT-base-chinese pretrained weights provide useful contextualized token representations for Chinese spelling correction.
    The method relies on this pretrained model as the backbone, and the paper fine-tunes it. This is standard practice.
  • domain assumption Upper layers of contextualized models produce context-specific representations (Ethayarajh, 2019).
    The method's motivation rests on this observation, cited in the introduction, that erroneous tokens will have inconsistent representations.
  • domain assumption The ESIM heuristic matching with a difference vector captures token-level compatibility (Chen et al., 2017).
    The composition layer is borrowed from ESIM and assumed to transfer to this task.
  • domain assumption The ground-truth transcripts of the AAM corpora (Aidatatang, AISHELL-1, MagicData) are correct and the Conformer models' CER values in Table 3 are accurate.
    The AAM benchmark is built on these assumptions; wrong transcripts or inaccurate ASR would distort the evaluation.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Contextualized Token Discrimination for Speech Search Query Correction." pith.science (2026). https://pith.science/paper/CWXYTMQP

@misc{pith2026250904393,
  author       = {Pith},
  title        = {Pith review of: Contextualized Token Discrimination for Speech Search Query Correction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CWXYTMQP}},
  note         = {Machine review of arXiv:2509.04393}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Query spelling correction is an important function of modern search engines since it effectively helps users express their intentions clearly. With the growing popularity of speech search driven by Automated Speech Recognition (ASR) systems, this paper introduces a novel method named Contextualized Token Discrimination (CTD) to conduct effective speech query correction. In CTD, we first employ BERT to generate token-level contextualized representations and then construct a composition layer to enhance semantic information. Finally, we produce the correct query according to the aggregated token representation, correcting the incorrect tokens by comparing the original token representations and the contextualized representations. Extensive experiments demonstrate the superior performance of our proposed method across all metrics, and we further present a new benchmark dataset with erroneous ASR transcriptions to offer comprehensive evaluations for audio query correction.

Figures

Figures reproduced from arXiv: 2509.04393 by Di Jiang, Junyu Lu, Mengze Hong, Qintian Guo, Victor Junqiu Wei, Zhiyang Su.

Figure 1
Figure 1. Figure 1: Overview: ASR and spelling correction. To enhance the user experience with accurate in￾tent transcription, it is fundamental to enhance the performance of the ASR model (Hong and Jiang, 2025). Typically, the ASR model first generates multiple transcription hypotheses from the audio, then selects the hypothesis with the fewest errors using a language model (LM). Finally, spelling cor￾rection is applied to r… view at source ↗
Figure 2
Figure 2. Figure 2: The left picture shows the cosine similarity [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Our proposed Contextualized Token Discrimination (CTD) method consists of two main components: [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Loss of our CTD on the AAM data To demonstrate the effectiveness of our CTD method, we present several cases in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 50 canonical work pages · 5 internal anchors

  1. [1]

    C Anantaram, Amit Sangroya, Mrinal Rawat, and Aishwarya Chhabra. 2018. Repairing asr output by artificial development and ontology based learning. In IJCAI, pages 5799--5801

    work page 2018

  2. [2]

    Yuya Asano, Sabit Hassan, Paras Sharma, Anthony B Sicilia, Katherine Atwell, Diane Litman, and Malihe Alikhani. 2025. Contextual asr error handling with llms augmentation for goal-oriented conversational ai. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 374--386

    work page 2025

  3. [3]

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, 33

    work page 2020

  4. [4]

    Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng. 2017. Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline. In 2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA), pages 1--5. IEEE

    work page 2017

  5. [5]

    CHEN CHEN, Yuchen Hu, Chao-Han Huck Yang, Sabato Marco Siniscalchi, Pin-Yu Chen, and EngSiong Chng. 2023. https://openreview.net/forum?id=cAjZ3tMye6 Hyporadise: An open baseline for generative speech recognition with large language models . In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    work page 2023

  6. [6]

    Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. https://doi.org/10.18653/v1/p17-1152 Enhanced lstm for natural language inference . Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    doi:10.18653/v1/p17-1152 2017

  7. [7]

    Zhiyu Chen, Mohamed Trabelsi, Jeff Heflin, Yinan Xu, and Brian D. Davison. 2020. https://doi.org/10.1145/3397271.3401044 Table search using a deep contextualized language model . In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20, page 589–598, New York, NY, USA. Association for Co...

    arXiv 2020

  8. [8]

    Horia Cucu, Andi Buzo, Laurent Besacier, and Corneliu Burileanu. 2013. Statistical error correction methods for domain-specific asr systems. In Statistical Language and Speech Processing, pages 83--92, Berlin, Heidelberg. Springer Berlin Heidelberg

    work page 2013

  9. [9]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171--4186

    2019

  10. [10]

    Huizhong Duan and Bo-June Hsu. 2011. Online spelling correction for query completion. In Proceedings of the 20th international conference on World wide web, pages 117--126

    work page 2011

  11. [11]

    Luis Fernando D’Haro and Rafael E Banchs. 2016. Automatic correction of asr outputs by using machine translation. proceedings Interspeech 2016, pages 3469--3473

    work page 2016

  12. [12]

    Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 55--65

    work page 2019

  13. [13]

    Peizhu Gong, Jin Liu, Yurong Xie, Minjie Liu, and Xiliang Zhang. 2023. Enhancing context representations with part-of-speech information and neighboring signals for question classification. Complex & Intelligent Systems, 9(6):6191--6209

    work page 2023

  14. [14]

    Priya Goyal, Piotr Doll \'a r, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677

    Pith/arXiv arXiv 2017

  15. [15]

    Alex Graves. 2012. https://arxiv.org/abs/1211.3711 Sequence transduction with recurrent neural networks . CoRR, abs/1211.3711

    Pith/arXiv arXiv 2012

  16. [16]

    Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. 2020. https://arxiv.org/abs/2005.08100 Conformer: Convolution-augmented transformer for speech recognition . Preprint, arXiv:2005.08100

    Pith/arXiv arXiv 2020

  17. [17]

    Ryan P Hafen and Michael J Henry. 2012. Speech information retrieval: a review. Multimedia systems, 18(6):499--518

    work page 2012

  18. [18]

    Matthias Hagen, Martin Potthast, Marcel Gohsen, Anja Rathgeber, and Benno Stein. 2017. A large-scale query spelling correction corpus. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1261--1264

    work page 2017

  19. [19]

    Sebastian Hofst \"a tter, Navid Rekabsaz, Mihai Lupu, Carsten Eickhoff, and Allan Hanbury. 2019. Enriching word embeddings for patent retrieval with global context. In European Conference on Information Retrieval, pages 810--818. Springer

    work page 2019

  20. [20]

    Mengze Hong and Di Jiang. 2025. https://arxiv.org/abs/2506.07149 Technical report: A practical guide to kaldi asr optimization . Preprint, arXiv:2506.07149

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    Mengze Hong, Wailing Ng, Chen Jason Zhang, and Di Jiang. 2025 a . Qualbench: Benchmarking chinese LLM s with localized professional qualifications for vertical domain evaluation. In The 2025 Conference on Empirical Methods in Natural Language Processing

    work page 2025

  22. [22]

    Mengze Hong, Wailing Ng, Chen Jason Zhang, Yuanfeng SONG, and Di Jiang. 2025 b . Dial-in LLM : Human-aligned LLM -in-the-loop intent clustering for customer service dialogues. In The 2025 Conference on Empirical Methods in Natural Language Processing

    work page 2025

  23. [23]

    Mengze Hong, Wailing Ng, Chen Jason Zhang, Yifei Wang, Yuanfeng Song, and Di Jiang. 2025 c . Llm-in-the-loop: Replicating human insight with llms for better machine learning applications. Authorea Preprints

    work page 2025

  24. [24]

    Mengze Hong, Chen Jason Zhang, Lingxiao Yang, Yuanfeng SONG, and Di Jiang. 2025 d . https://proceedings.mlr.press/v260/hong25a.html InfantCryNet : A data-driven framework for intelligent analysis of infant cries . In Proceedings of the 16th Asian Conference on Machine Learning, volume 260 of Proceedings of Machine Learning Research, pages 845--857. PMLR

    work page 2025

  25. [25]

    Di Jiang, Kenneth Wai-Ting Leung, Lingxiao Yang, and Wilfred Ng. 2015. Teii: Topic enhanced inverted index for top-k document retrieval. Knowledge-Based Systems, 89:346--358

    work page 2015

  26. [26]

    Di Jiang, Yuanfeng Song, Rongzhong Lian, Siqi Bao, Jinhua Peng, Huang He, Hua Wu, Chen Zhang, and Lei Chen. 2021. https://doi.org/10.1007/978-3-030-73200-4_36 Familia: A configurable topic modeling framework for industrial text engineering . In Database Systems for Advanced Applications: 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11...

    work page doi:10.1007/978-3-030-73200-4_36 2021

  27. [27]

    Di Jiang, Yongxin Tong, and Yuanfeng Song. 2016. https://doi.org/10.1145/2956235 Cross-lingual topic discovery from multilingual search engine query log . ACM Trans. Inf. Syst., 35(2)

    work page doi:10.1145/2956235 2016

  28. [28]

    Di Jiang, Jan Vosecky, Kenneth Wai-Ting Leung, and Wilfred Ng. 2013. https://doi.org/10.1145/2452376.2452420 Panorama: a semantic-aware application search framework . In Proceedings of the 16th International Conference on Extending Database Technology, EDBT '13, page 371–382, New York, NY, USA. Association for Computing Machinery

    arXiv 2013

  29. [29]

    Taegyun Kwon, Dasaem Jeong, and Juhan Nam. 2017. Audio-to-score alignment of piano music using rnn-based automatic music transcription. arXiv preprint arXiv:1711.04480

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  30. [30]

    Yichong Leng, Xu Tan, Rui Wang, Linchen Zhu, Jin Xu, Wenjie Liu, Linquan Liu, Xiang-Yang Li, Tao Qin, Edward Lin, and Tie-Yan Liu. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.367 F ast C orrect 2: Fast error correction on multiple candidates for automatic speech recognition . In Findings of the Association for Computational Linguistics: EMNLP 20...

    work page doi:10.18653/v1/2021.findings-emnlp.367 2021

  31. [31]

    Chenda Li, Jing Shi, Wangyou Zhang, Aswin Shanmugam Subramanian, Xuankai Chang, Naoyuki Kamo, Moto Hira, Tomoki Hayashi, Christoph Boeddeker, Zhuo Chen, and Shinji Watanabe. 2021. ESPnet-SE : End-to-end speech enhancement and separation toolkit designed for ASR integration. In Proceedings of IEEE Spoken Language Technology Workshop (SLT), pages 785--792. IEEE

    work page 2021

  32. [32]

    Yinghui Li, Shang Qin, Haojing Huang, Yangning Li, Libo Qin, Xuming Hu, Wenhao Jiang, Hai-Tao Zheng, and Philip S Yu. 2024. Rethinking the roles of large language models in chinese grammatical error correction. arXiv preprint arXiv:2402.11420

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  33. [33]

    Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li, Yunbo Cao, and Hai-Tao Zheng. 2022. https://doi.org/10.18653/v1/2022.findings-acl.252 The past mistake is the future wisdom: Error-driven contrastive probability optimization for C hinese spell checking . In Findings of the Association for Computational Lingui...

    work page doi:10.18653/v1/2022.findings-acl.252 2022

  34. [34]

    Zihong Liang, Xiaojun Quan, and Qifan Wang. 2023. Disentangled phonetic representation for chinese spelling correction. In The 61st Annual Meeting Of The Association For Computational Linguistics

    work page 2023

  35. [35]

    Junwei Liao, Sefik Eskimez, Liyang Lu, Yu Shi, Ming Gong, Linjun Shou, Hong Qu, and Michael Zeng. 2023. Improving readability for automatic speech recognition transcription. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(5):1--23

    work page 2023

  36. [36]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    Pith/arXiv arXiv 2017

  37. [37]

    Anirudh Mani, Shruti Palaskar, Nimshi Venkat Meripo, Sandeep Konam, and Florian Metze. 2020. Asr error correction and domain adaptation using machine translation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6344--6348. IEEE

    work page 2020

  38. [38]

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 a . https://arxiv.org/abs/1301.3781 Efficient estimation of word representations in vector space . Preprint, arXiv:1301.3781

    Pith/arXiv arXiv 2013

  39. [39]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 b . https://arxiv.org/abs/1310.4546 Distributed representations of words and phrases and their compositionality . Preprint, arXiv:1310.4546

    Pith/arXiv arXiv 2013

  40. [40]

    Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365

    Pith/arXiv arXiv 2018

  41. [41]

    Golan Pundak, Tara N Sainath, Rohit Prabhavalkar, Anjuli Kannan, and Ding Zhao. 2018. Deep context: end-to-end contextual speech recognition. In 2018 IEEE spoken language technology workshop (SLT), pages 418--425. IEEE

    2018

  42. [42]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  43. [43]

    Kanishka Rao, Hasim Sak, and Rohit Prabhavalkar. 2018. https://arxiv.org/abs/1801.00841 Exploring architectures, data and units for streaming end-to-end speech recognition with rnn-transducer . CoRR, abs/1801.00841

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  44. [44]

    Dhruva Sahrawat, Debanjan Mahata, Haimin Zhang, Mayank Kulkarni, Agniv Sharma, Rakesh Gosangi, Amanda Stent, Yaman Kumar, Rajiv Ratn Shah, and Roger Zimmermann. 2020. Keyphrase extraction as sequence labeling using contextualized embeddings. Advances in Information Retrieval, 12036:328

    work page 2020

  45. [45]

    Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862

    Pith/arXiv arXiv 2019

  46. [46]

    Yuanfeng Song, Di Jiang, Xuefang Zhao, Qian Xu, Raymond Chi-Wing Wong, Lixin Fan, and Qiang Yang. 2021. https://doi.org/10.1145/3474085.3481542 L2rs: A learning-to-rescore mechanism for hybrid speech recognition . In Proceedings of the 29th ACM International Conference on Multimedia, MM '21, page 1157–1166, New York, NY, USA. Association for Computing Machinery

    work page arXiv 2021

  47. [47]

    End-to-end asr: from supervised to semi-supervised learning with modern architectures

    Gabriel Synnaeve, Qiantong Xu, Jacob Kahn, Tatiana Likhomanenko, Edouard Grave, Vineel Pratap, Anuroop Sriram, Vitaliy Liptchinsky, and Ronan Collobert. End-to-end asr: from supervised to semi-supervised learning with modern architectures. In ICML 2020 Workshop on Self-supervision in Audio and Speech

    work page 2020

  48. [48]

    Bruno Taill \'e , Vincent Guigue, and Patrick Gallinari. 2020. Contextualized embeddings in named-entity recognition: An empirical study on generalization. Advances in Information Retrieval, 12036:383

    work page 2020

  49. [49]

    Tomohiro Tanaka, Ryo Masumura, Hirokazu Masataki, and Yushi Aono. 2018. Neural error corrective language models for automatic speech recognition. In INTERSPEECH, pages 401--405

    work page 2018

  50. [50]

    Chenming Tang, Xiuyu Wu, and Yunfang Wu. 2023. Are pre-trained language models useful for model ensemble in chinese grammatical error correction? In The 61st Annual Meeting Of The Association For Computational Linguistics

    work page 2023

  51. [51]

    Raphael Tang, Ferhan Ture, and Jimmy Lin. 2019. Yelling at your TV : An analysis of speech recognition errors and subsequent user behavior on entertainment systems. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 853--856

    work page 2019

  52. [52]

    Ghazaleh H Torbati, Andrew Yates, and Gerhard Weikum. 2021. You get what you chat: Using conversations to personalize search-based recommendations. In European Conference on Information Retrieval, pages 207--223. Springer

    work page 2021

  53. [53]

    Yuen-Hsien Tseng, Lung-Hao Lee, Li-Ping Chang, and Hsin-Hsi Chen. 2015. Introduction to sighan 2015 bake-off for chinese spelling check. In Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing, pages 32--37

    work page 2015

  54. [54]

    Takuma Udagawa, Masayuki Suzuki, Masayasu Muraoka, and Gakuto Kurata. 2024. Robust asr error correction with conservative data filtering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 256--266

    work page 2024

  55. [55]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, and I. Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998--6008

    work page 2017

  56. [56]

    Dingmin Wang, Yan Song, Jing Li, Jialong Han, and Haisong Zhang. 2018. A hybrid approach to automatic corpus generation for chinese spelling check. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2517--2527

    work page 2018

  57. [57]

    Yixuan Wang, Baoxin Wang, Yijun Liu, Dayong Wu, and Wanxiang Che. 2024. Lm-combiner: A contextual rewriting model for chinese grammatical error correction. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10675--10685

    work page 2024

  58. [58]

    Victor Junqiu Wei, Weicheng Wang, Di Jiang, Yuanfeng Song, and Lu Wang. 2024 a . Asr-ec benchmark: Evaluating large language models on chinese asr error correction. arXiv preprint arXiv:2412.03075

    Pith/arXiv arXiv 2024

  59. [59]

    Victor Junqiu Wei, Weicheng Wang, Di Jiang, Conghui Tan, and Rongzhong Lian. 2024 b . Acoustic model optimization over multiple data sources: Merging and valuation. arXiv preprint arXiv:2410.15620

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  60. [60]

    Hongqiu Wu, Shaohua Zhang, Yuchen Zhang, and Hai Zhao. 2023. Rethinking masked language modeling for chinese spelling correction. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10743--10756

    work page 2023

  61. [61]

    Xueyang Wu, Rongzhong Lian, Di Jiang, Yuanfeng Song, Weiwei Zhao, Qian Xu, and Qiang Yang. 2022. A phonetic-semantic pre-training model for robust speech recognition. CAAI Artificial Intelligence Research, 1(1):1--7

    work page 2022

  62. [62]

    Jinhua Xiong, Qiao Zhang, Shuiyuan Zhang, Jianpeng Hou, and Xueqi Cheng. 2015. Hanspeller: a unified framework for chinese spelling correction. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 20, Number 1, June 2015-Special Issue on Chinese as a Foreign Language

    work page 2015

  63. [63]

    Junjie Yu and Zhenghua Li. 2014. Chinese spelling error detection and correction based on language model, pronunciation, and shape. In Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, pages 220--223

    work page 2014

  64. [64]

    Liang-Chih Yu, Lung-Hao Lee, Yuen-Hsien Tseng, and Hsin-Hsi Chen. 2014. Overview of sighan 2014 bake-off for chinese spelling check. In Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, pages 126--132

    work page 2014

  65. [65]

    Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020 a . Spelling error correction with soft-masked bert. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 882--890

    work page 2020

  66. [66]

    Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V

    Yu Zhang, James Qin, Daniel S. Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V. Le, and Yonghui Wu. 2020 b . https://arxiv.org/abs/2010.10504 Pushing the limits of semi-supervised learning for automatic speech recognition . Preprint, arXiv:2010.10504

    Pith/arXiv arXiv 2020

  67. [67]

    Yutao Zhu, Jian - Yun Nie, Kun Zhou, Pan Du, and Zhicheng Dou. 2021. https://doi.org/10.1007/978-3-030-72113-8\_50 Content selection network for document-grounded retrieval-based chatbots . In Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I , volume 12656 ...

    doi:10.1007/978-3-030-72113-8 2021

  68. [68]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  69. [69]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.