{"id":"7942eea7-d00a-430b-8142-4ed3f5a9a4e7","arxiv_id":"2607.05058","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":2,"one_line_summary":"Self-built lecture glossaries from first-pass ASR raise technical-term recall across five backbones while holding or lowering CER on a new Mandarin AI/ML lecture benchmark.","lead":"A two-pass ASR method builds a glossary of technical terms from its own first-pass transcript and feeds it back as context, lifting term recall on Mandarin AI/ML lectures while holding or cutting CER. Term-centric metrics show that ordinary character error rate hides the failures that matter most for lecture content.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The paper’s strongest claim is an empirical multi-model result, not a universal generalization. Tables 3–5, the frequency-vs-random ablation, and the purity statistics already test the load-bearing mechanism inside the constructed benchmark. The single-domain/single-instructor scope and residual coverage gap for never-seen surface forms are real but are openly acknowledged and do not falsify the reported deltas. The reader’s CONDITIONAL verdict already correctly reflects those caveats; no stronger objection is warranted.","tokens_in":9542,"tokens_out":366,"duration_ms":3593,"concrete_test":"Once the promised dataset and term metadata are released, recompute Table 3 first-pass-glossary rows after replacing the rule-based EXTRACTTERMS with a simple frequency-ranked English-token filter (no acronym/model/code patterns); if term-recall gains remain ≥ +5 pp and CER non-worse on at least four of the five backbones, the claim is robust to extractor choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that a reference-free first-pass glossary raises term recall on all five backbones while holding or lowering CER—is directly supported by Table 3, the k-sweep and selection ablations in §6.1, purity numbers (90–95 %), and the hybrid result in Table 4. The reader’s weakest assumption (burstiness + first-pass recovery under a single-instructor AI/ML distribution and a rule-based extractor) is already stated as a limitation in §7 and does not undermine the measured gains inside the reported setting. No internal inconsistency, missing control, or untested premise that would reverse the headline numbers was found.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper argues that CER understates technical-term failures in Mandarin AI/ML lectures that code-switch English terms, and that lecture-level context recovered from a model’s own first-pass output can improve term recognition without external references. It releases a term-rich 5.01 h benchmark (8,888 term occurrences, 1,030 unique terms), defines term recall/precision/F1/TermER over canonical keys, and proposes ASR-GLOSSARY: extract a frequency-ranked top-k glossary from segment-only hypotheses and re-decode with that glossary as prompt/context. Across five frozen ASR backbones the first-pass glossary raises term recall while holding or lowering CER; on Breeze-ASR-25 recall rises from 52.50% to 60.13% with lower CER, and a hybrid with a small external list reaches 62.05% recall and 82.73% precision. Ablations on k, selection criterion, term category, purity, beam search, and a third pass support the design choices.","tokens_in":9753,"tokens_out":1142,"duration_ms":8546,"significance":"If the result holds, the work supplies a practical, reference-free decoding recipe for long-form technical ASR and a term-centric evaluation suite that CER alone cannot replace. Strengths include multi-backbone consistency (Table 3), paired-bootstrap significance, glossary-size and selection ablations (§6.1), per-category gains (Table 5), first-pass purity numbers (90–95%), and an oracle/hybrid comparison that cleanly separates deployable from non-deployable gains (Table 4). The method is model-agnostic at decoding time and does not require fine-tuning. The single-instructor, single-domain limitation is stated, so the contribution is best read as a strong existence proof plus reusable metrics and benchmark rather than a universal claim.","major_comments":[{"comment":"§3 and §7: The rule-based extractor (four hand-defined term kinds + NFKC/lowercasing/light stemming) is load-bearing for both the benchmark labels and the first-pass glossary. The paper reports purity of the extracted glossary against reference terms but does not report extractor precision/recall or inter-annotator agreement on a held-out sample of segments. Without that, it is hard to know how much of the remaining oracle gap (55–65% of oracle-only terms never appear in first-pass hypotheses) is acoustic failure versus extractor coverage failure. A short human audit or learned-extractor comparison would make the term-centric numbers more trustworthy.","section":"§3 Benchmark; §7 Conclusion"},{"comment":"Table 3 and §5: Combined context (title + prev-ASR + glossary) raises CER sharply on whisper-l-v3-turbo and Qwen3-ASR-0.6B while the short first-pass glossary does not. The paper attributes this to length and copying, which is plausible, but does not quantify how often the model copies unspoken glossary or prev-ASR text into the hypothesis (beyond the guard-sentence ablation on Qwen). A short copy-rate or hallucination analysis would strengthen the claim that the glossary is the robust default and that longer free-form context is model-dependent.","section":"Table 3; §5 Method; §6 Experiments"}],"minor_comments":[{"comment":"Table 2 reports CER on the full 16.92 h set while Table 3 rescores baselines on the 5.01 h term-rich subset; the text notes the difference, but a single footnote or parenthetical in Table 2 would prevent readers from treating the two CER columns as directly comparable.","section":"Table 2 vs Table 3"},{"comment":"Fig. 1 caption and body correctly highlight OpenClaw as lecture-specific; adding the lecture ID or a note that the top-12 are aggregated across 15 lectures would make the burstiness claim easier to inspect.","section":"Fig. 1"},{"comment":"Algorithm 1 uses EXTRACTTERMS without specifying whether the same rule-based pipeline as the reference extractor is applied to hypotheses; a one-sentence clarification would remove ambiguity.","section":"Algorithm 1"},{"comment":"The hybrid external list is described as “course-level” and “ranked by lecture-title match”; stating the list size and construction procedure more explicitly (or releasing it with the benchmark) would aid reproducibility.","section":"§5 Context variants; Table 4"},{"comment":"Minor wording: “TermER” is introduced as “term error rate” but the abbreviation is not expanded in the abstract; consistent expansion on first use would help.","section":"Abstract; §4"}],"recommendation":"minor_revision","confidential_remarks":"The central empirical claim is solid inside the reported setting; the main risk is over-generalization beyond single-instructor AI/ML lectures. Fit for a speech/ASR venue is good. No circularity or citation-pattern concerns. I would accept after the extractor-audit and copy-rate clarifications (or a clear statement that they are left to future work)."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The punchline is simple: CER hides the failures that matter in technical lectures, and a short glossary mined from the model’s own first-pass output fixes a useful chunk of them. Across five frozen backbones the first-pass glossary raises term recall and holds or lowers CER; on Breeze-ASR-25 it moves recall from 52.5 % to 60.1 % while cutting CER, and a hybrid with a tiny external list reaches 62 %. That is the real result.\n\nWhat is new is the combination, not any single piece. Contextual biasing and Whisper-style prompts already exist. Building the bias list by frequency-ranking terms extracted from the lecture’s own first-pass hypotheses, then measuring success with term recall/precision/F1/TermER on a new 5-hour term-rich Mandarin AI/ML set (8 888 occurrences), is the contribution. The ablations are clean: k=30 is the sweet spot for CER, frequency ranking beats first-occurrence or random, purity of the self-built lists sits at 90–95 %, and the gains hold under beam search. Tables 3–5 and the bootstrap tests support the headline numbers. The authors also flag the remaining coverage gap (55–65 % of oracle-only terms never appear in any first pass) and the single-instructor/single-domain limit, so they are not overselling.\n\nSoft spots are real but proportional. Data and code are still “will be released,” the term extractor is rule-based, and everything rests on the burstiness of one AI/ML lecture series. Those are ordinary empirical caveats, not load-bearing cracks. The method is reference-free, the oracle is clearly labeled non-deployable, and CER and term metrics move together rather than trade off.\n\nThis is for people who ship lecture ASR or who care about evaluation that actually tracks technical content. It deserves a serious referee; the experiments are thorough enough that the remaining questions (cross-domain, multi-speaker, learned extractor) are natural revision items, not desk-reject reasons. I would engage with it.","headline":"Solid practical ASR paper: self-built lecture glossary lifts term recall on five models without hurting CER, plus a useful term-centric benchmark.","tokens_in":10308,"tokens_out":522,"would_cite":true,"duration_ms":4424,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A self-built glossary from a first ASR pass recovers technical terms that CER misses in Mandarin AI lectures.","keywords":["speech recognition","code-switching","contextual biasing","technical terms","Mandarin","lecture ASR","term-centric metrics"],"falsifier":"On a multi-speaker or cross-domain lecture set where terms are not bursty, re-run the same two-pass protocol and check whether term-recall gains disappear or CER rises.","tokens_in":10464,"feed_emoji":"🎤","tokens_out":582,"duration_ms":5223,"temperature":0.7,"pith_summary":"Technical lectures in Mandarin repeatedly insert English terms that carry the meaning of the talk, yet those terms are short enough that ordinary character error rate can look fine while the key words are wrong. The paper shows that the lecture itself already contains the needed vocabulary: run ordinary segment-by-segment recognition once, rank the technical terms that appear most often in those hypotheses, and feed that short list back as context for a second pass. Across five frozen ASR systems the self-built glossary raises term recall and never worsens overall CER; a hybrid that also injects a few course-level terms reaches still higher recall and precision. The accompanying term-rich benchmark and term-centric metrics make the improvement visible, because CER alone hides the failures that matter to a student.","feed_headline":"Self-built glossary lifts term recall in Mandarin lecture ASR","feed_subtitle":"First-pass terms fed back as context raise recall on five models without raising CER","key_machinery":"ASR-GLOSSARY: a two-pass, reference-free procedure that extracts canonical technical terms from first-pass hypotheses, ranks them by lecture-wide frequency, keeps the top-k surface forms (k=30), and re-decodes every segment with that short glossary as context.","core_discovery":"Lecture-level context recovered from a model’s own first-pass hypotheses is a practical, reference-free signal for technical-term recognition. A frequency-ranked top-30 glossary extracted that way lifts term recall on every one of five ASR backbones while holding or lowering CER; on the strongest backbone the gain is from 52.50 % to 60.13 % recall with lower CER, and a hybrid that adds a small external list reaches 62.05 % recall and 82.73 % precision.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Self-built first-pass glossary lifts term recall on five ASR models","Model-extracted terms raise technical recall while holding CER","Two-pass decoding recovers lecture context for better term recognition","Frequency-ranked glossary from own hypotheses boosts term recall","Hybrid glossary reaches 62% term recall with lower CER on Breeze-ASR-25"],"cache_read_input_tokens":128,"weakest_assumption_plain":"Technical terms in a lecture must repeat often enough, and appear correctly often enough in the first-pass output, that a short frequency-ranked list supplies usable context.","fun_headline_variants_meta":{"raw":{"variants":["Self-built first-pass glossary lifts term recall on five ASR models","Model-extracted terms raise technical recall while holding CER","Two-pass decoding recovers lecture context for better term recognition","Frequency-ranked glossary from own hypotheses boosts term recall","Hybrid glossary reaches 62% term recall with lower CER on Breeze-ASR-25"]},"model":"grok-4.5","effort":"low","cost_usd":0.00478,"raw_usage":{"total_tokens":1384,"prompt_tokens":785,"num_sources_used":0,"completion_tokens":92,"cost_in_usd_ticks":47800000,"prompt_tokens_details":{"text_tokens":785,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":507,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":785,"tokens_out":92,"duration_ms":4337,"temperature":1.0,"reasoning_tokens":507,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T09:26:22.465489+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a multi-speaker or cross-domain lecture set where terms are not bursty, re-run the same two-pass protocol and check whether term-recall gains disappear or CER rises.","supporting_citations":[],"review_version":1}