Pith. sign in

REVIEW 3 major objections 6 minor 47 references

FormBharo: Designing and Evaluating a Voice Agent for Conversational Form Filling in Rural India

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FormBharo pairs large language models with a deterministic rule-based validation layer to fill health enrollment forms over phone calls in Hindi, and its benchmark shows end-to-end form completion—not component accuracy—decides which…

desk verdict A careful, honest evaluation with a useful new benchmark; the core findings hold, but the simulation and 8 kHz mismatch mean the numbers should be read as a floor, not a guarantee. read the letter →

arxiv 2608.06027 v1 pith:JEQXYYQN submitted 2026-08-06 cs.CL cs.AIcs.HC

Aman Dalmia , Sanskriti Midha , Jigar Doshi This is my paper · ORCID
classification cs.CLcs.AIcs.HC
keywords voiceagentsconversationalformfillingHindispeechrecognitionLLMextractionrule-basedvalidationend-to-endcompletionlow-literacyusersParetomodelselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FormBharo is a phone-call voice agent that fills a structured health-enrollment form by asking one question at a time in Hindi, for women who may not be able to read or write. The paper argues that a hybrid design—large language models for understanding and phrasing, plus deterministic rule-based validation and flow control for everything else—can complete these forms accurately under the tight latency and cost limits of a real NGO pilot. Its central empirical claim is that end-to-end form completion, not per-turn extraction accuracy, is the right measure: real-speech transcription errors can cut form completion by up to about 41 percentage points, while the rule layer recovers many extraction errors and lets smaller, cheaper models match or beat frontier models. If right, this means automated spoken enrollment for social-benefit programs is within reach for low-resource settings, and it gives the field a benchmark, FormVoiceAgentBench, to test such agents on noisy Hindi audio.

What carries the argument

The load-bearing mechanism is the deterministic rule-based layer that sits between the EXTRACT and REPLY LLMs. It validates each extracted value against field-specific guards (ten-digit phone numbers, dates in the past, option indices), normalizes values such as strings returned for numeric fields, enforces retry limits, skips optional fields the caller declines, evaluates conditional branches, and selects the next question. This layer is what lets smaller, cheaper models reach the same form-completion accuracy as frontier models, because it absorbs type and format errors that would otherwise corrupt the form state. The benchmark itself—FormVoiceAgentBench, pairing human-recorded Hindi audio with 3,760 multi-turn conversation tests across 960 simulated calls—is the measuring instrument, and the key metric is form-completion accuracy: the fraction of final form fields captured correctly at the end of the call.

What would settle it

Collect a corpus of live or role-played full clinic calls containing wrong, partial, and self-corrected answers with combined acoustic degradations (noise plus distant microphone), run the same STT-extract-reply pipeline, and compare form-completion accuracy and model rankings against FormVoiceAgentBench; a large drop or a ranking flip would falsify the claim that the measured behavior transfers.

Watch

Extended reading notes

Core claim

The paper's central claim is that conversational form-filling over a phone call for low-literacy users is feasible with a pipeline that uses LLMs only where they add value: EXTRACT turns each transcribed answer into structured form values, and REPLY phrases the next question, while a deterministic rule-based layer validates values, handles retries and branching, and decides the next question. On FormVoiceAgentBench—3,760 unit tests and 960 simulated calls built from 380 human recordings of Hindi speech under noise, microphone-distance, and pace variations—the authors find that feeding LLMs real speech transcripts instead of clean reference transcripts drops form completion by up to about 41 points, a much larger drop than per-turn extraction accuracy suggests. The rule-based layer recovers many of those errors: Gemini 3 Flash scores 95.96 percent extraction accuracy but 100 percent form completion on reference transcripts because a type mismatch is normalized before storage. Consequently component performance does not predict end-to-end performance; GPT-5.5 leads extraction at 99.8 percent but ranks lower on form completion. No single model wins on accuracy, cost, and latency, so the authors select a deployable configuration by Pareto filtering and weighted-sum scalarization, landing on Scribe v2 for transcription, Gemini 3.5 Flash for extraction, and GPT-5.4-mini for reply generation. To the authors' knowledge, this is the first conversational voice agent piloted to fill an enrollment form for this population.

Load-bearing premise

The load-bearing premise is that scripted, well-formed answers stitched into calls with one acoustic variation per call faithfully represent real clinic conversations, and that reference transcripts generated by GPT-5.5—itself one of the evaluated models—are a neutral test input.

Editorial extensions

If this is right

  • Deployers can choose a cheap configuration that clears the completion bar: with the rule layer in place, Gemini 3.5 Flash as EXTRACT and GPT-5.4-mini as REPLY reach 92.50 percent and 96.70 percent accuracy respectively on Scribe v2 transcripts.
  • Real-speech robustness, not reference-transcript accuracy, determines deployed quality, since transcription errors can cut form completion by up to about 41 points; agents should be tuned and tested on error-prone transcripts from the chosen STT model.
  • Per-turn leaderboards can mislead model selection: GPT-5.5's 99.8 percent extraction accuracy does not translate into top form completion, so end-to-end evaluation should be the standard for voice-agent pipelines.
  • The hybrid architecture makes the system tolerant of one class of small-model errors: Gemini 3 Flash's type mismatches (numeric fields returned as strings) are normalized by the rule layer, yielding 100 percent form completion despite 95.96 percent extraction accuracy.
  • Deployment is an explicit multi-objective choice: Pareto filtering plus weighted-sum scalarization turns accuracy, latency, and cost priorities into a concrete model pick, giving a template for constrained deployments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Live-call speech is the obvious stress test: the paper's own limitations section concedes real callers give wrong, partial, or self-corrected values, and distant-microphone-plus-noise combinations were never recorded, so a natural extension is to measure whether the ~41-point gap and the model rankings survive a corpus of full, naturalistic calls.
  • The 'errors both propagate and cancel' finding likely generalizes beyond speech: any multi-stage LLM pipeline may reward choosing models by end-to-end task success rather than by component benchmarks.
  • Adding cheap deterministic cross-field consistency checks (e.g., date-of-birth versus reported gestational age) could recover additional real-speech errors the current guards miss, since the rule layer already absorbs format and type errors.
  • Because the pilot form has only 12 fields with mostly fixed order, the rule layer's rescue effect may shrink on longer or more open-ended documents; testing the same hybrid on a larger form would bound the approach's generality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents FormBharo, a hybrid voice agent for conversational form filling over phone calls, combining LLMs for extraction and reply generation with deterministic rule-based validation and flow control. It introduces FormVoiceAgentBench, a Hindi benchmark with 960 simulated calls assembled from 380 human recordings, covering 3,760 unit tests across transcription, extraction, reply generation, and end-to-end form completion. Experiments across five STT models and 11 LLMs show that component-level accuracy does not predict end-to-end form completion, that a rule-based layer recovers many extraction errors so cheaper models can match or surpass frontier models, and that no single model dominates accuracy, latency, and cost simultaneously; a weighted-sum scalarization is used to select a deployable configuration.

Significance. The benchmark and the hybrid architecture are a useful contribution to voice-agent evaluation in low-resource settings, and the release of human-recorded Hindi audio with multi-turn tests fills a gap. The paper's central finding - that per-turn extraction accuracy can mislead about end-to-end form completion - is important and well illustrated (e.g., GPT-5.5 leading extraction but not form completion). The authors are transparent about limitations in Section 7 and provide confidence intervals and judge calibration details. However, the transfer of the reported numbers to the actual telephony deployment is not yet established, and the integration-test design does not exercise branch-level error propagation.

major comments (3)
  1. [Appendix E / Section 1] The benchmark audio is stored and processed as 16 kHz mono 16-bit PCM (Appendix E), while the deployed system receives 8 kHz µ-law audio from Exotel over a WebSocket (Appendix E). Section 1 explicitly lists telephone-channel distortion as a core challenge, and the paper's headline result is the ~41-point form-completion drop when models receive error-prone real-speech transcripts (Section 5.4, Table 5). Band-limiting to 8 kHz and µ-law companding change ASR error rates and error types, particularly for Hindi retroflex/aspirated contrasts and fricatives, so the measured gaps and the ranking of EXTRACT models under Scribe v2 transcripts are conditional on a more forgiving channel than the one in the pilot. The authors should re-run the STT and downstream evaluations on audio downsampled and companded to the 8 kHz µ-law codec, or explicitly qualify all deployment claims as conditional on 16 kHz.
  2. [Section 4.2 / Section 2] The integration tests chain extractions by writing each turn's extracted values into the form state, but the conversation history and the next user turn are fixed by the simulated user's scripted path (Section 3.4). Consequently, an extraction error that would change a branching decision (e.g., misclassifying the pregnancy answer) does not alter the subsequent questions in the evaluation, even though Section 2 motivates the design by the risk that 'an incorrectly captured field can send the agent down the wrong path.' The end-to-end form-completion numbers therefore do not capture branch-level error propagation and may overestimate accuracy in real calls. The authors should either make the next question depend on the predicted form state in the integration loop, or clearly state in Section 7 that branch misrouting is not evaluated.
  3. [Section 5.5, Tables 11 and 13] The final EXTRACT selection depends on a 0.51-percentage-point difference in form-completion accuracy between Gemini 3.5 Flash (92.50±0.53) and Claude Sonnet 4.6 (93.01±0.50) on Scribe v2 transcripts. These 95% confidence intervals overlap, and since each configuration was run only once (Appendix E), item-level intervals do not account for run-level nondeterminism. The weighted-sum scalarization then selects different models for w_a in [0.50,0.61] versus [0.62,0.90], and the paper does not state the actual w_a used for deployment. The selection is therefore not statistically grounded. Please report repeated-run variability or a significance test, and specify the chosen weights.
minor comments (6)
  1. [Section 1 / Abstract] The term 'piloted' is used to claim novelty ('first conversational voice agent piloted'), but no live pilot results are reported; the evaluation is entirely offline simulation. Please either temper the claim or clarify that the pilot is ongoing and this paper reports the pre-pilot benchmark.
  2. [Section 5.5] The deployment-constraint sentence ('discard models that fail our deployment constraints: p95 latency below 5s and form-completion accuracy above 90%') could be rephrased to specify which direction passes; as written it is ambiguous whether latency must be below or above 5s.
  3. [Section 3.2] The statement that each form field has 'three representations: the value, the desired form entry and the ground truth' is confusing because 'desired form entry' and 'ground truth' appear to be the same thing; clarify the intended distinction.
  4. [Appendix B.3] The LLM judges were calibrated against human labels on only 50 test cases, and one judge (GPT-5.4-mini) is also an evaluated model; the small calibration set is acknowledged, but the report should also discuss the potential for judge bias toward outputs of the same model family.
  5. [Section 4.1] LLM-WER is referenced to an external blog post; the manuscript would benefit from a self-contained definition of the metric, including how the LLM's semantic-equivalence classification is obtained and how inconsistent LLM judgments are handled.
  6. [Table 3] Costs are given as totals for transcribing the benchmark; it would be helpful to also report cost per audio hour for comparability.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: FormBharo's results are empirical benchmark measurements; the same-family generator/judge overlaps and the self-citation to Calibrate are not load-bearing.

full rationale

The paper's central claims are measured, not derived: form-completion accuracy is computed by executing the pipeline on fixed transcripts and ground-truth values, and the roughly 41-point drop is an empirical difference between reference and STT transcripts (Tables 5 and 6). No equation or selection step defines its result in terms of its input. The only author-overlap elements are (i) GPT-5.5 generating reference transcripts while also being an evaluated EXTRACT model, and (ii) GPT-5.4-mini serving as a calibrated LLM judge and as a candidate model. These create same-family bias risks, but they do not make any finding true by construction; in particular, the headline result is that GPT-5.5 does not win end-to-end, contrary to the direction its self-generated transcripts would bias. The judges were calibrated against human labels on a held-out 50-test batch (Appendix B.3), so the evaluation is externally anchored. The self-citation to Calibrate (Dalmia and Doshi 2025) is a software-harness reference, not a load-bearing scientific premise. Section 7's stated limitations (scripted answers, single acoustic variation, five simulated users) and the 16 kHz benchmark versus 8 kHz Exotel codec mismatch are validity concerns about transfer to deployment, not circularity. Therefore no circular steps are identified; the score of 2 reflects only the minor non-load-bearing self-citation.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or theoretical entities are introduced. The benchmark is a dataset artifact, not a postulated entity, and its independent use depends on the promised public release, which is not yet available.

free parameters (1)
  • weighted-sum scalarization weights (w_a, w_l, w_c) = w_a swept 0.5-0.9, w_l=0.1, w_c=1-w_a-w_l
    Chosen to reflect deployment priorities, not fitted to data; the selected EXTRACT model changes at w_a=0.62, so the headline configuration depends on this arbitrary priority choice.
assumptions (4)
  • ad hoc to paper Reference transcripts generated by GPT-5.5 are representative of real caller speech
    Section 3.2: the value and transcript are generated sequentially by GPT-5.5, which is also evaluated as an EXTRACT candidate, so the test input may favor that model family.
  • domain assumption LLM judges calibrated on 100 examples remain accurate across all 3,760 tests
    Appendix B.3: calibration used 50 tests for iteration and 50 held-out tests; the authors acknowledge the calibration set is small.
  • domain assumption Offline chaining of per-turn extraction outputs captures live call behavior
    Section 4.2: integration tests replay recorded turns and write extracted values into the form without exercising the live VAD, TTS, barge-in, and telephony loop.
  • ad hoc to paper A single acoustic variation per call matches deployment conditions
    Section 7: the authors state combinations such as distant microphone in a noisy room remain untested, so the measured acoustic robustness may be optimistic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FormBharo: Designing and Evaluating a Voice Agent for Conversational Form Filling in Rural India." pith.science (2026). https://pith.science/paper/JEQXYYQN

@misc{pith2026260806027,
  author       = {Pith},
  title        = {Pith review of: FormBharo: Designing and Evaluating a Voice Agent for Conversational Form Filling in Rural India},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JEQXYYQN}},
  note         = {Machine review of arXiv:2608.06027}
}
read the original abstract

In India, almost every social benefit starts with a form, yet the people who need these benefits most are often unable to read or write. Reaching them requires a spoken conversation. Today that work falls to frontline health workers who enroll beneficiaries one at a time, a poor use of stretched capacity. We built FormBharo ("fill the form" in Hindi), a voice agent that fills a structured form over a phone call under tight latency and cost budgets by pairing Large Language Models (LLMs) with deterministic, rule-based validation and flow control. It is being piloted with ARMMAN, an NGO running large-scale maternal and child mobile-health programs in India, to enroll low-income, Hindi-speaking mothers in antenatal and postnatal care. To our knowledge, it is the first voice agent piloted to fill an enrollment form for this population. We openly release FormVoiceAgentBench, a benchmark pairing human-recorded Hindi audio with 3,760 multi-turn conversation tests across 960 simulated calls, to evaluate our agent's components (transcription, extraction, reply generation) and end-to-end form completion under real acoustic variations. Form completion drops by up to ~41 points when LLMs receive error-prone real-speech transcripts instead of reference ones. The rule-based controls recover many turn-level extraction errors, helping smaller, cheaper models match or surpass frontier models on form completion. Component performance does not predict end-to-end performance: GPT-5.5 leads turn-level extraction accuracy on reference transcripts (99.8%) but ranks lower on form completion. Since errors both propagate and cancel across the pipeline, the optimal model choice of models emerges only through end-to-end evaluation. Finally, no single model is best across accuracy, cost, and latency at once, so we use a Pareto-based weighted-sum scalarization to select a deployable configuration balancing the three.

Figures

Figures reproduced from arXiv: 2608.06027 by the authors.

Figure 1
Figure 1. FormBharo’s architecture. EXTRACT extracts the form values from the transcribed text. The rule-based layer validates the extracted values, updates the form state, skips inactive branches, and selects the next question to ask. REPLY phrases the next question which is spoken back to the user through a TTS model. The “Anganwadi name” is the public clinic’s name. them, updates the form state and picks the next question … view at source ↗
Figure 2
Figure 2. Cost–quality–latency trade-off among EXTRACT models satisfying the deployment constraints. The logarith￾mic x-axis shows cost per turn, the y-axis shows form￾completion accuracy using Scribe v2 transcripts, and marker area encodes p95 latency per turn (bigger is slower). All six models are Pareto-optimal across the three objectives. The model selected for deployment is highlighted. 5.6 Response Generation REPLY has … view at source ↗
Figure 3
Figure 3. Call flow of the enrollment form used to build [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Cost–quality–latency trade-off among REPLY models satisfying the deployment constraints. The logarith￾mic x-axis shows cost per turn, the y-axis shows response accuracy using Scribe v2 transcripts, and marker area en￾codes p95 latency per turn (bigger is slower). Both …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 44 canonical work pages

  1. [1]

    ARMMAN . 2008. ARMMAN --- Advancing Reduction in Mortality and Morbidity of Mothers, Children and Neonates. https://armman.org

    work page 2008

  2. [2]

    Bashingwa, J. J. H.; Mohan, D.; Chamberlain, S.; Arora, S.; Mendiratta, J.; Rahul, S.; Chauhan, V.; Scott, K.; Shah, N.; Ummer, O.; Ved, R.; Mulder, N.; and LeFevre, A. E. 2021. Assessing exposure to Kilkari : a big data analysis of a large maternal mobile messaging service across 13 states in India . BMJ Global Health, 6(Suppl 4): e005213

    work page 2021

  3. [3]

    K.; Kumar, P.; Kumar, S.; Kolladath, A

    Bhanushali, A.; Bridgman, G.; G, D.; Ghosh, P. K.; Kumar, P.; Kumar, S.; Kolladath, A. R.; Ravi, N.; Seth, A.; Seth, A.; Singh, A.; Sukhadia, V. N.; Umesh, S.; Udupa, S.; and Durga Prasad, L. V. S. V. 2022. Gram Vaani ASR Challenge on Spontaneous Telephone Speech Recordings in Regional Variations of Hindi. In Proceedings of Interspeech 2022, 3548--3552. ISCA

    work page 2022

  4. [4]

    Bhogale, K.; Dhir, M.; Walecha, A.; Kaur, M.; Chhabra, V.; Pareek, A.; Sidh, H.; Manik, M.; Jain, S.; Singh, B.; Singh, U.; Javed, T.; Banga, S.; and Khapra, M. M. 2026. Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India. arXiv:2604.19151

    arXiv 2026

  5. [5]

    EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents

    Bogavelli, T.; Gauthier Melan c on, G.; Stankiewicz, K.; Bamgbose, O.; Riols, F.; Nguyen, H. H.; Mehndiratta, R.; Brin, L. D.; Marinier, J.; Subramani, H.; Madamala, A.; Nemala, S. K.; and Sunkara, S. 2026. EVA-Bench : A New End-to-end Framework for Evaluating Voice Agents. arXiv:2605.13841

    work page Pith review arXiv 2026

  6. [6]

    T.; and Li, H

    Chen, Y.; Yue, X.; Zhang, C.; Gao, X.; Tan, R. T.; and Li, H. 2026. VoiceBench : Benchmarking LLM -Based Voice Assistants. Transactions of the Association for Computational Linguistics, 14: 378--398

    work page 2026

  7. [7]

    L.; and Landay, J

    Cuadra, A.; Breuch, J.; Estrada, S.; Ihim, D.; Hung, I.; Askaryar, D.; Hassanien, M.; Fessele, K. L.; and Landay, J. A. 2024. Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data Entry. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(2): 1--39

    work page 2024

  8. [8]

    Dalmia, A.; and Doshi, J. 2025. Calibrate: An Open-Source Evaluation Platform for AI Agents. https://calibrate.artpark.ai/. ARTPARK, Indian Institute of Science. Code: https://github.com/artpark-sahai-org/calibrate

    work page 2025

Show all 47 references
  1. [9]

    M.; Khare, S.; Unni, V.; Vyas, S.; Rajpuria, A.; Yarra, C.; Mittal, A

    Diwan, A.; Vaideeswaran, R.; Shah, S.; Singh, A.; Srinivasa Raghavan, K. M.; Khare, S.; Unni, V.; Vyas, S.; Rajpuria, A.; Yarra, C.; Mittal, A. R.; Ghosh, P. K.; Jyothi, P.; Bali, K.; Seshadri, V.; Sitaram, S.; Bharadwaj, S.; Nanavati, J.; Nanavati, R.; and Sankaranarayanan, K...

    2021

  2. [10]

    Dr \`e ze, J.; and Khera, R. 2017. Recent Social Security Initiatives in India. World Development, 98: 555--572

    2017

  3. [11]

    Exotel . 2026. Exotel: Cloud Telephony and Contact Center Platform. https://exotel.com. Accessed: 2026-08-01

    2026

  4. [12]

    Ganesan, K.; Bamdev, P.; B, J.; Venugopal, A.; and Tushar, A. 2021. N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conf...

    2021

  5. [13]

    Google Cloud . 2026. Chirp 3: HD Voices --- Text-to-Speech. https://cloud.google.com/text-to-speech/docs/chirp3-hd. Accessed: 2026-08-01

    2026

  6. [14]

    Indus Action . 2026. Administrative Burden in India 's Welfare System: Examining the Learning, Compliance and Psychological Costs Faced by Vulnerable Citizens in Accessing Social Protection Programs. https://indusaction.org/case-study/. Accessed: 2026-07-23

    2026

  7. [15]

    International Institute for Population Sciences (IIPS) ; and ICF . 2021. National Family Health Survey ( NFHS -5), 2019--21: India Report. Technical Report FR375, International Institute for Population Sciences, Mumbai

    2021

  8. [16]

    Jacqmin, L.; Druart, L.; Est \`e ve, Y.; Favre, B.; M Rojas, L.; and Vielzeuf, V. 2023. OLISIA : a Cascade System for Spoken Dialogue State Tracking. In Proceedings of the Eleventh Dialog System Technology Challenge, 95--104. Prague, Czech Republic: Association for Computation...

    2023

  9. [17]

    Jain, D.; Shukla, H.; Rajeev, G.; Kulkarni, A.; Khatri, C.; and Agarwal, S. 2025. VoiceAgentBench : Are Voice Assistants Ready for Agentic Tasks? arXiv:2510.07978

    2025

  10. [18]

    Javed, T.; Bhogale, K.; Raman, A.; Kumar, P.; Kunchukuttan, A.; and Khapra, M. M. 2023. IndicSUPERB : A Speech Processing Universal Performance Benchmark for Indian Languages. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11): 12942--12950

    2023

  11. [19]

    Javed, T.; Nawale, J.; George, E.; Joshi, S.; Bhogale, K.; Mehendale, D.; Sethi, I.; Ananthanarayanan, A.; Faquih, H.; Palit, P.; Ravishankar, S.; Sukumaran, S.; Panchagnula, T.; Murali, S.; Gandhi, K.; R, A.; M, M.; Vaijayanthi, C.; Karunganni, K.; Kumar, P.; and Khapra, M. 2...

    2024

  12. [20]

    Jo, J.; Zhang, H.; Cai, J.; and Goyal, N. 2025. AI Trust Reshaping Administrative Burdens: Understanding Trust-Burden Dynamics in LLM -Assisted Benefits Systems. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT), 1172--1183. New Yo...

    2025

  13. [21]

    I.; Javed, T.; Bhogale, K.; Narasimhan, N.; and Khapra, M

    Joshi, S.; George, E. I.; Javed, T.; Bhogale, K.; Narasimhan, N.; and Khapra, M. M. 2025. Recognizing Every Voice: Towards Inclusive ASR for Rural Bhojpuri Women. In Proceedings of Interspeech 2025, 4243--4247. ISCA

    2025

  14. [22]

    R.; Jakasania, A.; and Raut, A

    Khandre, R. R.; Jakasania, A.; and Raut, A. 2023. ``We are working for seven days a week'': Time motion study of accredited social health activists from central India . Medical Journal Armed Forces India, 79(Suppl 1): S142--S149

    2023

  15. [23]

    Kothari, A.; Vossler, P.; Digitale, J.; Forouzannia, M.; Rosenberg, E.; Lee, M.; Bryant, J.; Molina, M.; Marks, J.; Zier, L.; and Feng, J. 2026. When the Domain Expert Has No Time and the LLM Developer Has No Clinical Expertise: Real-World Lessons from LLM Co-Design in a Safet...

    2026

  16. [24]

    Kunchukuttan, A. 2020. The IndicNLP Library. https://github.com/anoopkunchukuttan/indic_nlp_library. Accessed: 2026-08-01

    2020

  17. [25]

    E.; Shah, N.; Scott, K.; Chamberlain, S.; Ummer, O.; Bashingwa, J

    LeFevre, A. E.; Shah, N.; Scott, K.; Chamberlain, S.; Ummer, O.; Bashingwa, J. J. H.; Chakraborty, A.; Godfrey, A.; Dutt, P.; Ved, R.; and Mohan, D. 2022. The impact of a direct to beneficiary mobile communication program on reproductive and child health outcomes: a randomised...

    2022

  18. [26]

    T.; and Arora, J

    Marler, R. T.; and Arora, J. S. 2010. The weighted sum method for multi-objective optimization: new insights. Structural and Multidisciplinary Optimization, 41(6): 853--862

    2010

  19. [27]

    Mohammadi, S.; Paldhe, M.; Chhabra, A.; Son, Y.; and Seshagiri, V. 2026. L ing V ar B ench: Benchmarking LLM s on Entity Recognitions and Linguistic Verbalization Patterns in Phone-Call Transcripts. In Proceedings of the 19th Conference of the E uropean Chapter of the A ssocia...

    2026

  20. [28]

    Mukherjee, R. 2026. What exactly is an AI voice agent? And why does it matter in enterprise communication? https://www.techradar.com/pro/what-exactly-is-an-ai-voice-agent. Accessed: 2026-07-24

    2026

  21. [29]

    Mukherjee, S.; Sanz Ausin, M.; Aggarwal, K.; Datta, D.; Puri, S.; Jin, W.; Laud, T.; Manjunath, N.; Ding, J.; Paudel, B.; Schellenberger, J.; Huo, Z. F.; Shen, W.; Shirazian, N.; Potter, N.; Perkari, S.; Filippova, D.; Morozov, A.; Mease, A.; Muppalla, V.; Shakir, G.; Miller, ...

    2026

  22. [30]

    U.; Shahin, M.; and Ahmed, B

    Mustafa, M.; Shahnawaz, A.; Ammara, U.; Abrar, M.; Ahtisham, B.; Qureshi, F. U.; Shahin, M.; and Ahmed, B. 2026. Awaaz-e-Sehat: A Mobile Voice-based AI System for EMR Generation and Clinical Decision Support in Low-resource Maternal Healthcare. Proceedings of the ACM on Intera...

    2026

  23. [31]

    D.; Ramani, S.; Albert, S.; and Closser, S

    Nongrum, M.; Dhaliwal, B.; Na, Y.; Jamir, T.; Shekhawat, S.; Rao, K. D.; Ramani, S.; Albert, S.; and Closser, S. 2025. Disconnected data: mHealth data systems and challenges for primary health care workers in India . SSM - Health Systems, 5: 100124

    2025

  24. [32]

    OpenAI . 2026. GPT-5.5 System Card. https://openai.com/index/gpt-5-5-system-card/. Accessed: 2026-07-23

    2026

  25. [33]

    OpenRouter . 2026. OpenRouter : A Unified API for Large Language Models. https://openrouter.ai. Accessed: 2026-07-31

    2026

  26. [34]

    Patel, N.; Chittamuru, D.; Jain, A.; Dave, P.; and Parikh, T. S. 2010. Avaaj Otalo: A Field Study of an Interactive Voice Forum for Small Farmers in Rural India. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '10, 733--742. ACM

    2010

  27. [35]

    Pipecat AI . 2026 a . Pipecat: Open Source Framework for Voice and Multimodal Conversational AI . https://github.com/pipecat-ai/pipecat. Accessed: 2026-07-31

    2026

  28. [36]

    Pipecat AI . 2026 b . stt-benchmark: Benchmarking Speech-to-Text with Semantic WER and TTFS Latency. https://github.com/pipecat-ai/stt-benchmark. Accessed: 2026-07-23

    2026

  29. [37]

    Poole-Dayan, E.; Roy, D.; and Kabbara, J. 2026. LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 39116--39124

    2026

  30. [38]

    K.; Bhat, P

    Pulikodan, S.; Singh, A.; Basu, A.; Desai, N.; J, P. K.; Bhat, P. D.; Dharmaraju, R.; Gupta, R.; Udupa, S.; Kumar, S.; Sharma, S.; Sanka, V.; Tewari, D.; Dhand, H.; Kamat, A.; Singh, S.; Vashishth, S.; Talukdar, P.; Acharya, R.; and Ghosh, P. K. 2026. VAANI : Capturing the Lan...

    2026 arXiv

  31. [39]

    Sarvam AI . 2026. Evaluating Indian Language ASR . https://www.sarvam.ai/blogs/evaluating-indian-language-asr. Accessed: 2026-06-29

    2026

  32. [40]

    Sherwani, J.; Palijo, S.; Mirza, S.; Ahmed, T.; Ali, N.; and Rosenfeld, R. 2009. Speech vs. Touch-tone: Telephony Interfaces for Information Access by Low Literate Users. In Proceedings of the 3rd International Conference on Information and Communication Technologies and Devel...

    2009

  33. [41]

    Si, S.; Ma, W.; Gao, H.; Wu, Y.; Lin, T.-E.; Dai, Y.; Li, H.; Yan, R.; Huang, F.; and Li, Y. 2023. SpokenWOZ : A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Datasets and Ben...

    2023

  34. [42]

    Silero Team . 2024. Silero VAD : Pre-Trained Enterprise-Grade Voice Activity Detector. https://github.com/snakers4/silero-vad. Accessed: 2026-07-31

    2024

  35. [43]

    Sun, G.; Feng, S.; Jiang, D.; Zhang, C.; Gasic, M.; and Woodland, P. 2024. Speech-based Slot Filling using Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2024, 6351--6362. Bangkok, Thailand: Association for Computational Linguistics

    2024

  36. [44]

    Vaessen, N. 2024. jiwer : Evaluate Automatic Speech Recognition Systems. https://github.com/jitsi/jiwer. Accessed: 2026-08-01

    2024

  37. [45]

    W.; and Kharrufa, A

    Watson, C.; Parnaby, A. W.; and Kharrufa, A. 2024. Precarious Experiences: Citizens' Frustrations, Anxieties and Burdens of an Online Welfare Benefit System. arXiv:2405.08515

    2024 arXiv

  38. [46]

    Yoon, J.; Hwang, S.; Ran, H.; Bang, J.-U.; and Kim, K.-E. 2023. Adapting Text-based Dialogue State Tracker for Spoken Dialogues. In Proceedings of the Eleventh Dialog System Technology Challenge, 81--88. Prague, Czech Republic: Association for Computational Linguistics

    2023

  39. [47]

    P.; Zhang, H.; Gonzalez, J

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Zhang, H.; Gonzalez, J. E.; and Stoica, I. 2023. Judging LLM -as-a-Judge with MT -Bench and Chatbot Arena. In Advances in Neural Information Processing Systems (NeurIPS)

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.