Scalable Offline ASR for Command-Style Dictation in Courtrooms

Kavya Manohar; Kriti Anandan; Kumarmanas Nethil; Vaibhav Mishra

REVIEW 4 major objections 5 minor 6 references

Scalable Offline ASR for Command-Style Dictation in Courtrooms

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multiplexing VAD-segmented audio through a shared Whisper inference pipeline cuts p90 latency for courtroom dictation by up to 26 percent versus sequential batch processing.

desk verdict A plausible system paper whose central latency claim is confounded—the baseline likely processes whole audio files while the proposed system uses VAD segments—so the 26% gain may not come from multiplexing at all. read the letter →

arxiv 2507.01021 v1 pith:QGAEVFHL submitted 2025-06-07 eess.AS cs.CLcs.SD

Kumarmanas Nethil , Vaibhav Mishra , Kriti Anandan , Kavya Manohar This is my paper

classification eess.AScs.CLcs.SD

keywords automaticspeechrecognitionvoiceactivitydetectioncommand-styledictationmultiplexedbatchinferenceWhispercourtroomtranscriptionp90latencyCTCASR

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that command-style dictation, meaning short discrete spoken segments such as courtroom instructions, can be transcribed at low latency without giving every user a dedicated real-time recognizer. The system splits each audio stream into segments with voice activity detection, pools segments from all active users into one queue, and transcribes batches of segments in parallel on a single GPU. On 100 courtroom recordings, p90 latency (the time by which 90 percent of transcriptions finish) was consistently lower than sequential batch processing, with the largest gain at high concurrency: for 105-120 second clips with 20 concurrent users, latency fell from about 13.5 seconds to about 10 seconds, a 26 percent improvement. The system is designed to work with Whisper and with CTC-based speech recognition models, and the paper reports deployment in roughly 15 percent of India's courtrooms. A sympathetic reader would care because the result suggests that cheap offline ASR infrastructure can serve many simultaneous users at interactive speeds.

What carries the argument

The mechanism is a multiplexing queue driven by voice activity detection. Silero VAD splits each user's audio into 3-30 second segments with 300ms silence padding, chosen to match natural command boundaries; those segments are pushed into a centralized priority queue that dynamically batches them for one GPU inference call. The load-bearing identity is the claim that a VAD segment is a self-contained linguistic unit, which lets an autoregressive decoder like Whisper process many segments in parallel by encoding a batch of feature tensors and generating transcriptions with shared prompt tokens and timestamp generation disabled. The same independence property is what makes CTC-based ASR models compatible with the system.

What would settle it

Run the same set of courtroom recordings through two pipelines: transcribe each voice-activity-detection segment in isolation (the multiplexed mode) and transcribe the full audio with surrounding context (the sequential baseline), then compare word error rates on each recording. If segment-wise transcripts are systematically worse, especially for commands that cross segment boundaries, the reported latency gain is buying speed at the expense of transcription quality.

Watch

Extended reading notes

Core claim

The central claim is a transformation of the batching problem: by treating every VAD-detected segment as a self-contained unit, the system removes the sequential dependency that normally forces one audio file to finish before the next starts. Segments from many users are merged into a single encoder pass and a single generation call, so GPU capacity idle during one user's silence can be filled with another user's speech. The paper reports p90 latency improvements at every concurrency level tested (5, 10, and 20 users), growing with load: 14 percent for 105-120 second clips at C=10, and 26 percent at C=20, where multiplexing holds latency near 10 seconds while sequential processing reaches 13.5 seconds. It further claims the same multiplexing works for CTC-based ASR architectures, not only autoregressive Whisper models.

Load-bearing premise

The load-bearing premise is that each detected speech segment is a complete, self-contained command that needs no surrounding audio to be transcribed correctly; if that fails, parallel transcription buys speed by sacrificing accuracy.

Editorial extensions

If this is right

At concurrency 20, the longest audio clips gain most (26 percent), so adding users widens the multiplexing advantage rather than eroding it.
Because the pipeline works without per-user GPUs or streaming decoders, one offline ASR server can serve interactive dictation to many users at once.
The gain should grow for audio longer than 120 seconds, since overlapping computation and reduced queuing compound on longer segments.
CTC-based ASR models that decode segments independently can use the same multiplexing layer, so the latency benefit is not tied to Whisper specifically.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper never measures word error rate, so its strongest unstated assumption is that segment-wise decoding is as accurate as full-context decoding; comparing WER on identical recordings would show whether the latency gains are usable or partly illusory.
The same segmentation-plus-multiplexing pattern should transfer to other short-utterance dictation domains, such as medical transcription or dispatch logging, where commands are discrete and context-independent.
Dynamic batching in the priority queue could be tuned with deadline or fairness policies to prevent one long segment from delaying many short commands; the paper lists smarter batching as future work, so this is a natural next step.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing you should know: the reported latency win is not demonstrated by the evaluation as written. Section 3.1 defines the baseline as "each audio is processed completely before starting the next," with no mention of VAD segmentation, while the proposed system splits audio into 3–30 s segments. Whisper decodes a 105–120 s file far more slowly than several short segments, so the 26% gap at C=20 could be entirely due to segmentation rather than multiplexing. The paper offers no ablation that holds segmentation constant. The stress-test note is right; I checked the text and the natural reading is that the baseline processes whole files.

What is genuinely useful: the problem is real, the architecture is sensible, and the multiplexing-queue-over-segments design is not present in the two cited references. The p90 measurements across concurrency levels 5/10/20 are a new empirical datapoint, and the paper states its central assumption explicitly—that VAD segments are self-contained linguistic units. That is honest and testable.

The soft spots are significant. There is no WER evaluation, so we cannot verify the self-contained-segment assumption; if it fails, the latency gain buys unusable transcripts. There are no error bars or statistical tests. The abstract's claim that this is "open-source" is not backed by released code or data in the preprint. The 15% courtroom deployment is self-reported with no external evidence. These are fixable, but they are exactly what a reviewer would need.

The reader's conditional verdict is fair, maybe generous. The paper's core claim is unproven, but not because of any fundamental incoherence—the system could work, and the measurements, once the confound is removed, might still show an advantage. This is a demo-style system paper, not a scientific breakthrough.

Recommendation: send it to peer review only if the venue is a demo/system track and reviewers will insist on a corrected comparison. As a full paper, I would not accept the latency conclusion in its current form. The right next step for the authors is an ablation: same VAD segments, sequential versus multiplexed scheduling, plus WER on the segments. If the gap persists, the result is real and worth publishing.

Referee Report

4 major / 5 minor

Summary. The paper proposes an open-source ASR framework for command-style dictation in courtroom settings, combining Silero VAD segmentation with multiplexed, batched Whisper inference across concurrent users. The central claim is that this multiplexed approach reduces p90 latency compared with sequential processing, with the largest reported gain at concurrency C=20 for 105–120 s audio (13.5 s vs. 10 s, a 26% improvement). The abstract and introduction also state that the system is deployed in approximately 15% of India's courtrooms, and the paper closes with plans for a live demonstration at Interspeech.

Significance. If the latency claims survive controlled evaluation, the paper would describe a practical and potentially valuable system: it is open-source, model-agnostic in the sense of supporting CTC-based architectures, and targets a real multi-user workload on a modest T4 GPU. The paper states that the latency measurements are empirical comparisons rather than derived from fitted parameters, and the code snippet in Listing 1 illustrates a concrete batching mechanism. The significance is currently limited, however, because the latency comparison is confounded by an undefined sequential baseline, no transcription-quality metric is reported, and no uncertainty quantification is provided; these gaps prevent the reader from attributing the measured gains to multiplexing or from judging whether the output remains usable.

major comments (4)

[Section 3.1, Section 3.2, Figure 2] The sequential baseline is not specified as using the same VAD segmentation as the multiplexed system. The text says only that the baseline processes "each audio... completely before starting the next," so the baseline may feed entire 5 s–5 min files into Whisper while the multiplexed system decodes 3–30 s VAD segments. If so, the p90 gap at 105–120 s and C=20 (13.5 s vs. 10 s) could reflect the benefit of shorter, padded inputs rather than the benefit of multiplexing. Please rerun the baseline on the identical VAD-segmented units, processed one by one (or file-by-file with the same batching policy within a file), and report the multiplexing-only gain with segmentation held constant.
[Section 2.3 and Section 3.2] The load-bearing assumption that "properly segmented commands don't require inter-segment context" is never evaluated. The paper reports only latency, with no word error rate or other accuracy metric on the 100 courtroom recordings. Because the parallel decoding strategy depends on each VAD segment being a self-contained linguistic unit, the manuscript must show that multiplexed segment-wise transcription does not degrade quality relative to whole-file transcription. Add a WER comparison or human judgment on a representative subset.
[Section 3.1 and Section 3.2] The p90 latency results are presented as point estimates without error bars, confidence intervals, or statistical tests. With 100 recordings, three concurrency levels, and multiple duration bins, it is unclear whether the reported 0.5–1.0 s differences at C=5 and the 26% difference at C=20 are reliable and reproducible. Please report the variance across runs or bootstrap confidence intervals for each configuration, and state how many independent trials were performed.
[Abstract and Section 1] The claim of deployment in "approximately 15% of India's courtrooms" is unsupported and under-specified. No evidence, definition of the denominator, deployment period, or usage statistics are given, and the claim is used in the introduction as evidence of real-world effectiveness. Either provide verifiable deployment/usage metrics or remove or clearly qualify the claim so that it does not function as an unsubstantiated proof of system viability.

minor comments (5)

[Abstract and Section 2.1] The text contains spacing/capitalization inconsistencies such as "V oice Activity Detection" and "V AD"; please copyedit for consistency.
[Section 3.2] The sentence "The most significant benefits appears at high concurrency" has a subject-verb agreement error; it should read "The most significant benefit appears...".
[Listing 1] The code snippet uses `model.model.generate`, `prompt_tokens`, and `no_ts_token` without defining these variables in the text; please clarify or point to the open-source repository so the snippet is reproducible.
[Section 2.1 and Section 3.1] Silero VAD and faster-whisper are used but not cited; please add references. Also, the paper says the corpus is 100 recordings of 5 s–5 min but the evaluation covers audio durations of 15–120 s; clarify how the recordings were binned or subsampled.
[Figure 2] The figure caption should state explicitly that lower p90 latency is better, define the units, and, if possible, include error bars; the current text references Figure 2 but the figure itself is not shown in the submitted version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning found; latency evaluation is empirical and the multiplexing claim does not reduce to its inputs.

full rationale

The paper's central claim is an empirical latency comparison between a multiplexed VAD-based pipeline and a sequential baseline, measured on 100 courtroom recordings at three concurrency levels. No parameter is fitted to the reported outcomes and no equation is derived from a target result. The closest thing to a definitional premise is Section 2.3's assertion that VAD-segmented commands are self-contained units; this is an unvalidated assumption about transcription quality, not a circular derivation, and the paper makes no WER claim from it. The deployment statement ('deployed in around 15% of India's courtrooms') is self-reported but is used as a deployment credential, not as evidence that makes the latency comparison true; it is therefore not load-bearing circularity. Concerns that the sequential baseline may not apply the same VAD segmentation, so the 26% gain may be confounded, are a correctness/experimental-design issue, not a circularity issue. No self-citation chain or renamed known result is present. Accordingly, the derivation is self-contained and no circular step is identified.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The system introduces no new physical or model entities; the ledger captures the hand-tuned VAD parameters and the key domain assumptions required for the latency claim. All parameters are chosen manually rather than fitted, so the circularity burden is low.

free parameters (2)

VAD segment duration range = 3-30 seconds
Chosen by hand in Section 2.1 to align with natural speech boundaries; no data-driven optimization is reported.
VAD silence padding = 300 ms
Also chosen by hand in Section 2.1; affects segment boundaries and batch composition.

assumptions (4)

domain assumption VAD-segmented dictation commands are self-contained linguistic units that require no inter-segment context for transcription.
Stated in Section 2.3 as the enabler for parallel decoding; if false, parallel processing would degrade accuracy.
domain assumption Silero VAD accurately segments courtroom dictation audio.
The whole approach relies on clean segmentation; no segmentation accuracy evaluation is presented.
domain assumption GPU compute, not I/O or scheduling overhead, is the primary bottleneck.
The multiplexing benefit assumes that batching more segments improves throughput; no profiling evidence is given.
domain assumption The sequential baseline processes each recording without inter-file batching and is a fair comparator.
The comparison in Section 3.2 contrasts multiplexing with sequential processing; if the baseline also batched segments within a file, the improvement might shrink.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Scalable Offline ASR for Command-Style Dictation in Courtrooms." pith.science (2026). https://pith.science/paper/QGAEVFHL

@misc{pith2026250701021,
  author       = {Pith},
  title        = {Pith review of: Scalable Offline ASR for Command-Style Dictation in Courtrooms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QGAEVFHL}},
  note         = {Machine review of arXiv:2507.01021}
}

read the original abstract

We propose an open-source framework for Command-style dictation that addresses the gap between resource-intensive Online systems and high-latency Batch processing. Our approach uses Voice Activity Detection (VAD) to segment audio and transcribes these segments in parallel using Whisper models, enabling efficient multiplexing across audios. Unlike proprietary systems like SuperWhisper, this framework is also compatible with most ASR architectures, including widely used CTC-based models. Our multiplexing technique maximizes compute utilization in real-world settings, as demonstrated by its deployment in around 15% of India's courtrooms. Evaluations on live data show consistent latency reduction as user concurrency increases, compared to sequential batch processing. The live demonstration will showcase our open-sourced implementation and allow attendees to interact with it in real-time.

Figures

Figures reproduced from arXiv: 2507.01021 by the authors.

**Figure 1.** System architecture widely used CTC-based models. Deployed in approximately 15% of India’s courtrooms, this system demonstrates effectiveness in real-world, missioncritical environments. Evaluations show it maintains transcription quality and acceptable latency even as concurrent user load increases - validating its viability for command-style dictation at scale. The remainder of this paper details our system arch… view at source ↗

**Figure 2.** p90 Latency: Sequential vs Multiplexed inference across audio durations and user concurrencies The most significant benefits appears at high concurrency (C=20). For longer audio clips (105–120s), sequential processing latency increases dramatically to approximately 13.5s, while multiplexed processing maintains a lower latency of around 10s—a 26% improvement. This demonstrates the scalability of our approach in rea… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

6 extracted references · 6 canonical work pages

[1]

In judicial settings particularly, this modality enables real-time documentation of proceedings with- out disrupting courtroom dynamics

Introduction Command-style dictation represents a critical workflow in en- vironments where users need to dictate discrete segments of text with minimal latency. In judicial settings particularly, this modality enables real-time documentation of proceedings with- out disrupting courtroom dynamics. Traditional ASR approaches are poorly suited for this task...

work page
[2]

Scalable Offline ASR for Command-Style Dictation in Courtrooms

Approach We implemented a distributed architecture where V AD and ASR components operate independently. As shown in Figure 1, our system identifies discrete speech segments within each audio file, enabling parallel processing across multiple users. 2.1. V AD Optimization We configure Silero V AD to produce segments between 3-30 seconds with 300ms silence ...

work page Pith review arXiv 2025
[3]

We compare this against the baseline of sequential processing, where each audio is pro- cessed completely before starting the next

Efficiency of Multiplexing Our multiplexing approach transforms ASR economics in multi-user environments by dynamically batching speech seg- ments across concurrent users. We compare this against the baseline of sequential processing, where each audio is pro- cessed completely before starting the next. 3.1. Methodology We evaluated performance using 100 a...

work page
[4]

Conclusion We present an open-source ASR framework tailored for command-style dictation, combining V AD-based segmentation with parallel, multiplexed inference to achieve low-latency tran- scription at scale. By decoupling speech detection from decod- ing, the system avoids the trade-offs of traditional batch and on- line ASR approaches, enabling efficien...

work page
[5]

Whisperx: Time- accurate speech transcription of long-form audio,

M. Bain, J. Huh, T. Han, and A. Zisserman, “Whisperx: Time- accurate speech transcription of long-form audio,”INTERSPEECH 2023, 2023

work page 2023
[6]

Turning whisper into real- time transcription system,

D. Mach ´aˇcek, R. Dabre, and O. Bojar, “Turning whisper into real- time transcription system,” inProceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations , S. Saha and H. Sujaini, Eds. Bali, Indonesia: As...

work page 2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

In judicial settings particularly, this modality enables real-time documentation of proceedings with- out disrupting courtroom dynamics

Introduction Command-style dictation represents a critical workflow in en- vironments where users need to dictate discrete segments of text with minimal latency. In judicial settings particularly, this modality enables real-time documentation of proceedings with- out disrupting courtroom dynamics. Traditional ASR approaches are poorly suited for this task...

work page

[2] [2]

Scalable Offline ASR for Command-Style Dictation in Courtrooms

Approach We implemented a distributed architecture where V AD and ASR components operate independently. As shown in Figure 1, our system identifies discrete speech segments within each audio file, enabling parallel processing across multiple users. 2.1. V AD Optimization We configure Silero V AD to produce segments between 3-30 seconds with 300ms silence ...

work page Pith review arXiv 2025

[3] [3]

We compare this against the baseline of sequential processing, where each audio is pro- cessed completely before starting the next

Efficiency of Multiplexing Our multiplexing approach transforms ASR economics in multi-user environments by dynamically batching speech seg- ments across concurrent users. We compare this against the baseline of sequential processing, where each audio is pro- cessed completely before starting the next. 3.1. Methodology We evaluated performance using 100 a...

work page

[4] [4]

Conclusion We present an open-source ASR framework tailored for command-style dictation, combining V AD-based segmentation with parallel, multiplexed inference to achieve low-latency tran- scription at scale. By decoupling speech detection from decod- ing, the system avoids the trade-offs of traditional batch and on- line ASR approaches, enabling efficien...

work page

[5] [5]

Whisperx: Time- accurate speech transcription of long-form audio,

M. Bain, J. Huh, T. Han, and A. Zisserman, “Whisperx: Time- accurate speech transcription of long-form audio,”INTERSPEECH 2023, 2023

work page 2023

[6] [6]

Turning whisper into real- time transcription system,

D. Mach ´aˇcek, R. Dabre, and O. Bojar, “Turning whisper into real- time transcription system,” inProceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations , S. Saha and H. Sujaini, Eds. Bali, Indonesia: As...

work page 2023