{"id":"4e68da13-c73f-4f38-8c81-04ebfa899e97","arxiv_id":"2508.14048","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"RAG-Boost adds on-the-fly retrieval of audio-text pairs and domain terms to an LLM-based speech recognizer, fusing retrieved text with live hypotheses to reduce recognition errors and improve responses.","lead":"This system paper describes RAG-Boost, a module that searches a database of audio-text pairs and domain terms during speech recognition and feeds what it finds back into an LLM-based recognizer to correct errors. It was built as a Task I entry for the MLC-SLM Challenge, which asks models to produce spoken responses to speech input.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported ASR gain depends on the vector store being both coverage-sufficient and disjoint from the evaluation set; without a documented leakage check, the improvement could be lookup rather than fusion.","rationale":"The reader's weakest assumption identifies essentially the same condition: the store must contain the correct text for the words being decoded, and it must not have been built from evaluation references. My reading agrees and sharpens the second part: because retrieval returns whole audio-text pairs, any inclusion of test-set material in the index would trivially explain the reported improvement. This is a correctness risk, not a stylistic or novelty objection. The paper may be perfectly honest; the concern is structural and empirical. The full-text corruption prevents me from checking whether the authors already performed a disjointness check, but the abstract and readable fragments do not report one. I therefore recommend no change to the reader's UNVERDICTED verdict: the central empirical claim remains unverified until the store provenance and non-overlap with the test set are established. I do not see a basis for ACCEPT, CONDITIONAL, or REJECT, because the evidence needed to adjudicate is unreadable in the available extraction.","tokens_in":17828,"tokens_out":2684,"duration_ms":34447,"concrete_test":"Rebuild the vector store using only the challenge training and development splits, then repeat the Task I evaluation and report WER relative to the baseline. In addition, compute exact and near-duplicate overlap between test reference transcripts and all indexed text entries; if any overlap exists, rerun with those entries removed. If the improvement over baseline largely disappears under this disjoint-store condition, the central claim reduces to memorization rather than RAG-based error correction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract states that each partial hypothesis queries a vector store of audio-text pairs and domain terms and that retrieved results are fused with live hypotheses to fix recognition errors. For this to support the central claim, two conditions must hold: (i) the store contains the correct text, or a close paraphrase, for the words being decoded, and (ii) the store was built without including reference transcripts or audio from the evaluation utterances. The second condition is the more dangerous: if a test utterance's audio-text pair is in the index, retrieval can return the exact reference and the 'correction' is a lookup rather than a learned fusion effect. The abstract does not say what data produced the store, and the readable fragments do not document any leakage check. The full-text extraction is too badly corrupted to verify the experimental setup, so this is the most load-bearing open question about the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RAG-Boost, a system submitted to the MLC-SLM Challenge (task I), which augments an LLM-based automatic speech recognition (ASR) baseline with a retrieval-augmented generation (RAG) module. As described in the abstract, each partial ASR hypothesis is used to query a vector store of audio-text pairs and domain terms; the retrieved results are then fused with the live ASR hypotheses to fix recognition errors, and the fused hypotheses are passed to the LLM to produce improved responses. The central claim is that this retrieval-augmented pipeline reduces ASR recognition errors and improves downstream LLM responses relative to the challenge baseline.","tokens_in":1278,"tokens_out":1537,"duration_ms":37760,"significance":"If the central claim is supported by sound experiments, the paper would make a useful contribution to domain-specific ASR error correction and to the emerging area of coupling retrieval with LLM-based speech processing. The idea of using a vector store of audio-text pairs and domain terms for on-the-fly correction is plausible and relevant to the MLC-SLM Challenge setting, and the paper explicitly situates the contribution within a concrete benchmark. However, the present manuscript does not allow an assessment of whether the claim holds: the experimental tables are unreadable, no quantitative result is attributable to any system variant, and the provenance of the retrieval store is not documented. These are load-bearing issues that must be fixed before the contribution can be evaluated.","major_comments":[{"comment":"The experimental tables are presented as garbled rows of numbers that cannot be attributed to system conditions or metrics. For example, the rows labeled with system names are not aligned with the numeric columns, and values around 21-25 cannot be identified as WER figures or otherwise. As a result, the central claim of improved recognition accuracy and improved LLM responses is not supported by any readable quantitative evidence. The authors must provide clean tables with explicit row and column labels, the baseline and RAG-Boost numbers, the number of utterances, and, where appropriate, confidence intervals or significance tests.","section":"Section 4 (experimental results, including Tables 1 and 2)"},{"comment":"The vector store is described as containing 'audio-text pairs and domain terms', but the manuscript does not state what data produced these pairs, nor whether the evaluation set's reference transcripts or audio were included. If a test utterance's audio-text pair is present in the index, retrieval can return the exact reference, making the reported 'correction' a lookup rather than a genuine fusion effect. The authors need to specify the store's provenance, report an overlap analysis between the store and the evaluation set, and describe any exclusion or leakage-check procedure. This is essential for interpreting the experimental results.","section":"Abstract and Section 2 (vector store construction)"},{"comment":"The fusion mechanism between the retrieved results and the live ASR hypothesis is not described precisely enough to be reproduced. The equations in Section 3 are unreadable in the rendered text, and the free parameters (fusion weights, retrieval top-k, similarity threshold, embedding model, and store construction details) are not reported. The authors should give a complete algorithmic description, including the fusion formula, the prompt template used for the LLM, and the hyperparameter values, so that the method can be reimplemented and the contribution isolated.","section":"Section 3 (fusion and LLM prompting)"},{"comment":"No readable ablation separates the contributions of retrieval, fusion, and the LLM. In particular, the paper does not compare RAG-Boost against a simpler contextual-biasing or n-best rescoring baseline, which would be necessary to show that the gains come from the RAG mechanism rather than from merely exposing the LLM to extra text. The authors should include ablations with and without retrieval, with and without fusion, and with a non-retrieval baseline.","section":"Section 4 (ablations)"}],"minor_comments":[{"comment":"The rendered text contains many unintelligible passages, including corrupted equations and section headings. The authors should ensure that the LaTeX/PDF source compiles correctly and that all tables and equations are legible in the submitted version.","section":"General presentation"},{"comment":"The abstract states that the system yields 'improved responses' but gives no quantitative target or metric (e.g., WER or downstream task accuracy). A brief statement of the evaluation metric and the main result would make the contribution clearer.","section":"Abstract"},{"comment":"The paper does not report the size of the vector store, the number of evaluation utterances, or the coverage rate of the store with respect to the evaluation set. These statistics are important for understanding the conditions under which the method can help.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and relevant to the challenge, but the manuscript as submitted is not verifiable because the experimental section is unreadable and the store provenance is undocumented. I would encourage the editor to invite a revision that provides clean tables, a leakage check, and full method details. If the authors cannot provide these, the paper would not be publishable in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible, modest engineering contribution—RAG for an LLM-based ASR challenge baseline—but I can't honestly assess the reported gains because the text I got is unreadable mojibake. The abstract is coherent, and the design sits cleanly inside the existing RAG-for-ASR and contextual-biasing program. The one genuinely new twist is querying the vector store with each partial hypothesis on the fly rather than after a final transcript; that is a real refinement, not a first-principle novelty.\n\nGood things: the system is simple, the challenge framing gives it a concrete comparison point, and the idea of fusing retrieved audio-text pairs and domain terms with live hypotheses is a reasonable way to fix domain vocabulary before the LLM generates responses. The authors are not overclaiming in the abstract: they say 'improved responses,' which is the right unit given task I.\n\nSoft spots, in order of how soft they actually are. First, the supplied full text is corrupted, so the tables (which look like WER numbers in the 21–25 range) cannot be attributed to any system or condition. That is not the authors' fault, but it means my confidence in the central empirical claim is low. Second, the vector store is built from audio-text pairs and domain terms from the same challenge domain, and the abstract doesn't say whether evaluation utterances or their transcripts were excluded. If a test utterance's reference is in the store, retrieval can return the exact answer and the improvement is a lookup. That is the load-bearing open question, and the paper needs an explicit leakage check or a detailed store-construction description. Third, the body embeds an arXiv header for a different identifier than the submission under review; likely an extraction artifact, but worth checking against the real PDF. Fourth, novelty is modest: RAG-for-ASR is established, and the contribution is the on-the-fly querying and the challenge-specific integration.\n\nWho gets value: participants in MLC-SLM-type benchmarks and people building LLM-based ASR pipelines with domain vocabularies. This is not a methods paper for the broader ASR community. The citation pattern and related-work discussion could not be checked from the corrupted text.\n\nRecommendation: treat the unreadable text as a reason to ask for the actual PDF, not to desk-reject. If the leakage check is clean and the ablation separates retrieval from fusion, the paper is a solid challenge-system report. I'd send it to review. My own verdict on the supplied evidence is 'unable to verify,' not 'wrong.'","headline":"A modest, plausible RAG-for-ASR challenge system whose reported numbers I can't verify from the supplied text, but the idea is coherent and deserves a referee's look at the actual PDF.","tokens_in":18521,"tokens_out":2914,"would_cite":false,"duration_ms":33277,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that on-the-fly retrieval from a vector store of audio-text pairs and domain terms repairs LLM-based speech-recognition errors before the LLM responds.","keywords":["retrieval-augmented generation","LLM-based speech recognition","automatic speech recognition","vector store","audio-text pairs","partial hypotheses","domain terms","speech error correction"],"falsifier":"Build a held-out set whose reference transcripts are deliberately absent from the vector store and run the system against the baseline on that set; if recognition accuracy does not improve, the reported gain is store coverage rather than fusion.","tokens_in":17509,"feed_emoji":"🎙️","tokens_out":9508,"duration_ms":96837,"temperature":0.7,"pith_summary":"The paper introduces RAG-Boost, a system that adds a retrieval-augmented generation module to an existing LLM-based automatic speech recognition baseline. As the recognizer produces each partial hypothesis, the system queries a vector store built from audio-text pairs and domain terms, retrieves matches, and fuses them with the live hypothesis to correct recognition errors. The corrected hypothesis is then passed to the LLM, so the language model sees a transcript closer to what was actually said rather than the raw errorful output. The claim is that this on-the-fly retrieval both lowers recognition errors and improves the LLM's downstream responses, without retraining the recognizer. A reader would care because the approach offers a plug-in route to domain-specific accuracy for LLM-based speech recognition.","feed_headline":"A lookup store repairs LLM speech errors before the LLM answers","feed_subtitle":"Audio-text and domain-term lookups correct partial hypotheses so transcripts and LLM answers improve.","key_machinery":"The central object is a prebuilt vector store of audio-text pairs and domain terms, queried in real time by each partial ASR hypothesis. A retrieval-augmented generation (RAG) module embeds the partial hypothesis, pulls the nearest matches from the store, and fuses them with the live hypothesis before the LLM decodes. The argument rests on this loop: coverage in the store determines what can be repaired, retrieval decides which candidates are seen, and fusion decides which candidate corrects the hypothesis.","core_discovery":"On its own terms, the paper's central claim is that an LLM-based ASR system can be improved by making retrieval part of the decoding loop rather than a post-processing step. Each partial ASR hypothesis queries a vector store of audio-text pairs and domain terms; the retrieved results are fused with the live hypothesis to fix recognition errors, and the corrected hypothesis is then passed to the LLM. The paper claims this fused input yields improved responses, meaning both the transcript and the LLM's downstream answer are better than the baseline's. The mechanism is coverage-driven: the store supplies candidate spellings the recognizer missed, and fusion selects among them before the language model sees the text.","pith_inferences":["The approach should transfer most readily to closed-vocabulary or domain-constrained speech such as commands, proper nouns, and jargon, where a compact store can cover most utterances; open-domain speech would require a much larger store to see the same benefit.","A testable next step is to delete from the store all entries matching a held-out set's reference transcripts and measure how much of the recognition gain survives, which would separate the contribution of fusion from the contribution of lookup.","The same partial-hypothesis retrieval-fusion pattern could be applied to other LLM-based perception tasks, such as speech translation or keyword spotting, whenever a vector store can supply the correct surface form that the noisy front end misses."],"forward_implications":["At inference time, an existing LLM-based ASR baseline can be enhanced by a retrieval-fusion module rather than by fine-tuning the recognizer.","Because the fused hypothesis is what reaches the LLM, transcript corrections should propagate to downstream answers, improving LLM responses as well as recognition accuracy.","Retrieval acts on partial hypotheses, so the correction happens before the utterance is finished, matching the paper's on-the-fly design.","The repair ceiling is set by the store: only errors whose correct text is represented among the audio-text pairs and domain terms can be fixed."],"supporting_citations":[],"fun_headline_variants":["RAG fixes ASR errors before the LLM speaks","Retrieval-augmented speech recognition improves LLM answers","Lookup store corrects ASR hypotheses mid-decoding","RAG in the loop yields better LLM speech output"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The vector store must already contain the correct text, or a close paraphrase, for the words being decoded, so retrieval can only repair errors whose correct form is in the index.","fun_headline_variants_meta":{"raw":{"variants":["RAG fixes ASR errors before the LLM speaks","Retrieval-augmented speech recognition improves LLM answers","Lookup store corrects ASR hypotheses mid-decoding","RAG in the loop yields better LLM speech output"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000882,"raw_usage":{"total_tokens":3717,"prompt_tokens":757,"completion_tokens":2960,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":373,"completion_tokens_details":{"reasoning_tokens":2892}},"tokens_in":373,"tokens_out":2960,"duration_ms":20311,"temperature":1.0,"reasoning_tokens":2892,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T04:13:08.958292+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a held-out set whose reference transcripts are deliberately absent from the vector store and run the system against the baseline on that set; if recognition accuracy does not improve, the reported gain is store coverage rather than fusion.","supporting_citations":[],"review_version":1}