Pith. sign in

REVIEW 3 major objections 8 minor 32 references

Mitigating Noisy Inputs for Question Answering

T0 review · 3 major / 8 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Spoken-question answering improves when the model reads ASR confidence and forces a guess instead of an unknown token.

desk verdict The synthetic corpus and forced-decoding trick are real contributions, but the human-transfer claim rests on a tiny sample with a possible train/test overlap that the paper never rules out. read the letter →

arxiv 1908.02914 v1 pith:44XXVUDO submitted 2019-08-08 cs.CL

Denis Peskov , Joe Barrow , Pedro Rodriguez , Graham Neubig , Jordan Boyd-Graber This is my paper · ORCID
classification cs.CL
keywords questionansweringautomaticspeechrecognitionASRconfidenceforceddecodingspokendeepaveragingnetworksynthetictrainingdatanoisytext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper addresses what happens when a question-answering system receives spoken questions through an automatic speech recognizer, which corrupts precisely the low-frequency words that carry the answer. Because large human speech corpora for QA are impractical to record, the authors synthesize over 500,000 noisy sentences by reading Quizbowl and Jeopardy! questions with text-to-speech and decoding them with an ASR system. They claim that a neural model can recover part of the accuracy lost to noise by weighting word embeddings with the recognizer's per-word confidence and by forcing the recognizer to output the closest known word instead of a generic unknown token. On human-recorded questions, the combination improves accuracy over the noisy-text neural baseline at the sentence level, though a retrieval baseline stays stronger on full multi-sentence Quizbowl questions.

What carries the argument

The first mechanism is forced decoding: the paper removes the unknown-token symbol from the ASR system's finite state transducer, so the recognizer must emit the acoustically closest known word instead of <unk>, converting a useless marker into an embeddable (if wrong) word. The second is a confidence-augmented Deep Averaging Network, which replaces the plain average of word embeddings with a confidence-weighted average $r^{**} = \frac{1}{N}\sum_i E[w_i]\, f(c_i)$, where $f$ is a learned affine function $f(c) = W^{(c)}c + b^{(c)}$ of the recognizer's per-word confidence $c_i$; the resulting vector feeds the standard hidden layers and softmax classifier. Together they let the downstream model treat uncertainty as a feature rather than a failure.

What would settle it

Record a larger set of human speakers reading the same Quizbowl and Jeopardy! questions, compare per-word ASR error types (substitutions, deletions, insertions) between the synthetic and human audio, and measure the forced-decoding-plus-confidence model's accuracy on both: if human speech produces systematically different error types or the accuracy gain disappears on the human set, the synthetic noise proxy is not reliable.

Watch

Extended reading notes

Core claim

The central claim, stated in the paper's terms, is that a neural question-answering model which uses the ASR system's word-level confidence outputs and systematically forces decoding of words rather than unknowns improves QA accuracy on Quizbowl and Jeopardy! questions over a neural baseline trained and evaluated on the same noisy text. In the reported results, confidence weighting and forced decoding each help in some settings, and the combined model reaches the highest neural accuracy on the sentence-level and short-question settings; a strong retrieval baseline remains more accurate on full multi-sentence Quizbowl questions. The paper also reports that models trained on a synthetic corpus of text-to-speech audio transferred to human-recorded questions, based on BLEU and WER distribution overlap on a small human sample.

Load-bearing premise

The load-bearing premise is that ASR errors on synthetic text-to-speech audio match ASR errors on real human speech well enough that models trained on the synthetic corpus transfer to human-recorded questions, a premise checked only indirectly through distribution overlap on a small human sample.

Editorial extensions

If this is right

  • If the gains transfer, any embedding-averaging or bag-of-words downstream model can cheaply consume ASR uncertainty by adding a learned confidence scalar and forcing the recognizer to guess at unknown words, without retraining the ASR system.
  • The synthetic TTS-to-ASR pipeline supplies a scalable recipe for training supervised models for spoken input on tasks where no large human audio corpus exists.
  • The paper's results imply that for one-sentence or first-sentence questions, neural models that use confidence and forced decoding can outperform an IR baseline, so voice assistants handling short queries may benefit most directly.
  • Because the combination helps at the first-sentence level of Quizbowl, it could give a computer buzzer-accuracy an edge in the early, hard portion of a question, where humans often wait for easier clues.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment would replace the single affine confidence function $f(c)$ with a small neural network per word, or feed phone-level confidences; the paper's ablation only shows that one learned affine transform helps, not that it is optimal.
  • The forced-decoding trick should transfer to recognizers that output a special token instead of <unk>; any system that replaces unknown words with the nearest known alternative and weights by confidence could see similar gains on other ASR-downstream tasks such as semantic parsing.
  • The Appendix's human-sample check uses only 50 Quizbowl questions and one Jeopardy episode, so the synthetic-to-human transfer claim would be much stronger if re-run on a larger corpus; the paper's own numbers suggest human WER is higher, so real-world gains could be smaller.
  • One could test whether the improvements are driven by high-confidence words being up-weighted or low-confidence words being down-weighted by ablating the learned $f$: clamping $f$ to the identity and comparing against the affine version would isolate the mechanism.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper addresses the problem of question answering (QA) systems receiving noisy inputs from automatic speech recognition (ASR). The authors construct a large synthetic corpus by passing roughly 96,000 Quizbowl questions and over 200,000 Jeopardy questions through Google Text-to-Speech and then through a Kaldi ASR system, producing noisy transcriptions with word-level confidence scores. They propose two mitigations: forced decoding, which removes the <unk> token from the ASR finite state transducer so that every word is decoded as a real word, and a confidence-augmented Deep Averaging Network (DAN) that learns to reweight word embeddings by a learned function of ASR confidence. They evaluate these methods against IR and DAN baselines on synthetic test sets and on small human speech corpora: 50 recorded Quizbowl questions (five unique questions across ten speakers) and one manually parsed Jeopardy episode. The paper reports that the combination of forced decoding and confidence features improves accuracy in several conditions, particularly for first-sentence Quizbowl accuracy, and claims that these methods improve QA accuracy on Quizbowl and Jeopardy questions.

Significance. If the results hold, the paper offers a practical, low-cost recipe for making QA systems more robust to ASR noise using only off-the-shelf TTS and ASR, which is relevant for virtual assistants and spoken trivia systems. The authors release code and data, and the paper includes negative results from noisy-channel and lattice-RNN attempts, which is a transparency strength. The synthetic corpus itself may be a useful resource. However, the central human-transfer claim is currently supported by a very small human evaluation and the paper does not establish that the human test questions are disjoint from the training corpus; in addition, some headline claims are not consistently supported by the tabulated numbers. These issues are fixable but require substantive revision.

major comments (3)
  1. [§4.1, Table 2] The human Quizbowl evaluation does not establish transfer to unseen spoken questions. The paper never states that the five source questions used in the human recordings are disjoint from the 96,000-question Quizbowl corpus used to generate the synthetic training set, and the clean-text rows of Table 2 show 1.000 accuracy on the human Quizbowl End set versus 0.540 on the synthetic End set, which is consistent with training overlap. The human Jeopardy evidence is one manually parsed episode, so the reported 0.033 versus 0.017 for FD+Conf versus DAN corresponds to about one question, and no confidence intervals or significance tests are reported for any human row. Because the conclusion claims improvement on Quizbowl and Jeopardy questions without restricting to synthetic data, these rows are load-bearing; the authors should either confirm disjointness and add appropriate statistical measures or restrict the human-data claim.
  2. [Abstract and §5 vs. Table 2] The unqualified claim that confidence integration and forced decoding improve the accuracy of QA is not consistently supported by Table 2. For example, the Confidence model alone underperforms the DAN baseline on synthetic Jeopardy (0.095 vs. 0.097), forced decoding alone underperforms the DAN at synthetic Quizbowl Start (0.032 vs. 0.035), and several human rows are ties (e.g., Quizbowl Human End 0.440 for DAN, FD, and FD+Conf). The Table 2 caption's statement that combining the two methods leads to a further joint improvement is also contradicted by the synthetic Quizbowl End row, where Confidence alone (0.374) is higher than FD+Conf (0.371). The claims should be qualified to the specific conditions (e.g., first-sentence accuracy, particular datasets) where the improvements actually hold.
  3. [Appendix A, §4.1] The validation of synthetic-to-human transfer is weaker than claimed. Figures 2 and 3 provide only a visual BLEU/WER distributional overlap on the same 50 human Quizbowl questions and one Jeopardy episode, with no statistical comparison (e.g., a two-sample test or effect size) and no direct evaluation of whether task accuracy on synthetic data predicts accuracy on human data under matched methods. The sentence in Appendix A that 'our results are comparable in accuracy to our synthetic data' is not backed by any quantitative comparison in the paper; the human rows of Table 2 are not directly compared against the synthetic rows with the same method and a test of difference. At minimum, the authors should report the sample size, the test statistic, and a confidence interval for the human rows, and temper the transfer conclusion accordingly.
minor comments (8)
  1. [§1] The phrase 'it easier and cheaper' is missing 'is'; it should read 'it is easier and cheaper'.
  2. [§3.2] There is a doubled word in 'knows when when a word is not recognized'; one 'when' should be removed.
  3. [§3.2] 'commerical' should be 'commercial'.
  4. [§B.2] The phrase 'cannot not reconstruct alternate paths' contains a double negative; it should be 'cannot reconstruct alternate paths'.
  5. [§3.3, Eq. (3)] The notation r** is not defined; the paper should introduce it explicitly or use a cleaner symbol such as r_conf.
  6. [§4] There is a formatting artifact 'textscir' in the sentence about IR methods; it should read 'IR'.
  7. [Figures 2 and 3] The figures are described as color-coded by speaker, but the text-only rendering does not make the distinction clear; please ensure the figures are legible in grayscale and provide a colorblind-safe palette.
  8. [Table 2] The caption and body would benefit from stating the number of test instances for each condition (especially human rows) so that the reader can judge the resolution of the reported differences.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical comparison on held-out synthetic and human speech, with no target quantity defined in terms of the prediction.

full rationale

The paper makes no formal derivation; its claims are empirical. The proposed methods are defined independently of the evaluation metric: forced decoding (Section 3.2) removes <unk> from the Kaldi finite state transducer, and the confidence-augmented DAN (Section 3.3, Eqs. 2-3) uses ASR word confidences as input features to a learned classifier. Model parameters, including the learned transform f(c), are fit on the synthetic noisy corpus and evaluated on held-out synthetic and human speech in Table 2; there is no step in which the test accuracy or final claim is fed back into the method. The synthetic-data validity concern is checked in Appendix A via BLEU/WER distributional overlap against human speech, which is an empirical validation rather than an assumption of the conclusion. Self-citations to the Quizbowl dataset [2] and DAN [18] supply background and baselines, and the DAN baseline is reproduced in Table 2 rather than imported wholesale; hence they are not load-bearing for the claimed improvement. Concerns about the size or disjointness of the human evaluation set would bear on evidence quality, not on circularity. I find no step in which a target quantity is defined in terms of the prediction, or a fitted parameter is relabeled as a prediction.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the validity of the synthetic corpus and on the usefulness of Kaldi confidences; both are empirically motivated but not formally proven. No new physical or mathematical entities are introduced.

free parameters (2)
  • DAN architecture hyperparameters
    Hidden layer size 1000, number of layers, dropout and batch-norm schedules, and optimizer settings are selected by hyperparameter sweeps (Section 3.3). Exact values are not listed in the paper, so a re-implementation must tune them.
  • Confidence transformation weights W(c), b(c)
    Equation (2) defines a learned linear mapping from raw ASR confidence to a scalar weight; these parameters are trained on the synthetic corpus. They are fitted parameters, but they are optimized on the training set in a standard supervised way, not hand-set to force results.
assumptions (3)
  • domain assumption Synthetic TTS speech is a sufficient proxy for human speech for ASR noise transfer.
    Used in Section 2 to justify the main training pipeline. Appendix A compares BLEU/WER distributions with a small human set, but the assumption is not proven; if TTS errors are systematically different, the trained models may not transfer.
  • domain assumption Kaldi word-level confidence scores are informative for transcription quality.
    Section 3.3 builds the confidence model on the assumption that low-confidence words are more likely to be errors. The paper does not analyze calibration; the network learns the weighting, so a weaker version of this assumption may suffice.
  • domain assumption Answers form a closed set with each answer label seen in training.
    The DAN classifier predicts among answer labels; Section 4 notes that some test answers do not appear in training, making perfect accuracy impossible. The task is thus framed as closed-set classification, which limits generalization to novel answers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mitigating Noisy Inputs for Question Answering." pith.science (2026). https://pith.science/paper/44XXVUDO

@misc{pith2026190802914,
  author       = {Pith},
  title        = {Pith review of: Mitigating Noisy Inputs for Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/44XXVUDO}},
  note         = {Machine review of arXiv:1908.02914}
}
read the original abstract

Natural language processing systems are often downstream of unreliable inputs: machine translation, optical character recognition, or speech recognition. For instance, virtual assistants can only answer your questions after understanding your speech. We investigate and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks. Integrating confidences into the model and forced decoding of unknown words are empirically shown to improve the accuracy of downstream neural QA systems. We create and train models on a synthetic corpus of over 500,000 noisy sentences and evaluate on two human corpora from Quizbowl and Jeopardy! competitions.

Figures

Figures reproduced from arXiv: 1908.02914 by the authors.

Figure 1
Figure 1. ASR errors on QA data: original spoken words (top of box) are garbled (bottom). While many words become into “noise”— frequent words or the unknown token—consistent errors (e.g., “claren￾don” to “clarintin”) can help downstream systems. Additionally, words reduced to <unk> (e.g., “kermit”) can be useful through forced decod￾ing into the closest incorrect word (e.g., “hermit” or even “car”). interpreted by a generic … view at source ↗
Figure 2
Figure 2. A comparison of BLEU score distributions across human speakers (color-coded) to our artificial method, visualized by the step line. The distributions of BLEU scores are similar, with human data being slightly lower, justifying our weak supervision training approach. A. Further Data Analysis One potential concern with the synthetically-generated dataset is that ASR systems might be either better or worse at recognizi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 31 canonical work pages

  1. [1]

    Mitigating Noisy Inputs for Question Answering

    Introduction Progress on question answering (QA) has claimed human- level accuracy. However, most factoid QA models are trained and evaluated on clean text input, which becomes noisy when questions are spoken due to Automatic Speech Recognition ( ASR) errors. This consideration is disre- garded in trivia match-ups between machines and humans: IBM Watson [...

    work page Pith review arXiv 2019

  2. [2]

    cyclohexane

    Spoken question answering datasets Neural networks require a large training corpus, but record- ing hundreds of thousands of questions is not feasible. Crowd-sourcing with the required quality control (speak- ers who say “cyclohexane” correctly) is expensive. As an alternative, we generate a data-set with Google Text- to-Speech on 96,000 factoid questions...

    work page

  3. [3]

    Louis Vampas

    Mitigating noise This section discusses two approaches to mitigating the ef- fects of missing and corrupted information caused by ASR systems. The first approach—forced decoding—exploits systematic errors to arrive at the correct answer. The sec- ond uses confidence information from the ASR system to down-weight the influence of low-confidence terms. Both app...

    work page

  4. [4]

    novel”, “character

    Results Achieving 100% accuracy on this dataset is not a realistic goal, as not all test questions are answerable (specifically, some answers do not occur in the training data and hence cannot be learned by a machine learning system). Base- lines for the DAN (Table 2) establish realistic goals: a DAN trained and evaluated on the same train and dev set, onl...

    work page

  5. [5]

    Introducing ASR into a QA pipeline corrupts the data

    Conclusion Question answering, like many NLP tasks are impaired by noisy inputs. Introducing ASR into a QA pipeline corrupts the data. A neural model that uses the ASR system’s confi- dence outputs and systematic forced decoding of words rather than unknowns improves QA accuracy on Quizbowl and Jeopardy! questions. Our methods are task agnostic and can be ...

    work page

  6. [6]

    The views expressed in this paper are our own

    Acknowledgments This work was supported by NSF Grants IIS-1748663 and IIS-1748642. The views expressed in this paper are our own. We thank the reviewers, the Quizbowl and Kaldi communities, and Yogarshi Vyas for their help

    work page

  7. [7]

    Build Watson: an overview of DeepQA for the Jeopardy! challenge,

    D. A. Ferrucci, “Build Watson: an overview of DeepQA for the Jeopardy! challenge,” in 19th International Confer- ence on Parallel Architecture and Compilation Techniques, 2010, pp. 1–2

    work page 2010

  8. [8]

    Boyd-Graber, S

    J. Boyd-Graber, S. Feng, and P. Rodriguez, Human- Computer Question Answering: The Case for Quizbowl . Springer Verlag, 2018

    work page 2018

Show all 32 references
  1. [9]

    Adversarial examples for evaluating reading comprehension systems,

    R. Jia and P. Liang, “Adversarial examples for evaluating reading comprehension systems,” inProceedings of Empir- ical Methods in Natural Language Processing, 2017, pp. 2021–2031

    2017

  2. [10]

    Qme!: A speech-based question-answering system on mobile devices,

    T. Mishra and S. Bangalore, “Qme!: A speech-based question-answering system on mobile devices,” inHuman Language Technologies:, 2010, pp. 55–63

    2010

  3. [11]

    Building effective question answering characters,

    A. Leuski, R. Patel, D. Traum, and B. Kennedy, “Building effective question answering characters,” inProceedings of the Annual SIGDIAL Meeting on Discourse and Dialogue, 2009, pp. 18–27

    2009

  4. [12]

    Neural lattice-to-sequence models for uncertain inputs,

    M. Sperber, G. Neubig, J. Niehues, and A. Waibel, “Neural lattice-to-sequence models for uncertain inputs,” in Pro- ceedings of the Association for Computational Linguistics, 2017

    2017

  5. [13]

    Jhu aspire system: Robust lvcsr with tdnns, ivector adaptation and rnn-lms,

    V . Peddinti, G. Chen, V . Manohar, T. Ko, D. Povey, and S. Khudanpur, “Jhu aspire system: Robust lvcsr with tdnns, ivector adaptation and rnn-lms,” in Automatic Speech Recognition and Understanding (ASRU), IEEE Workshop on, 2015, pp. 539–546

    2015

  6. [14]

    The fisher corpus: a resource for the next generations of speech-to-text,

    C. Cieri, D. Miller, and K. Walker, “The fisher corpus: a resource for the next generations of speech-to-text,” in Proceedings of the Language Resources and Evaluation Conference, 2004

    2004

  7. [15]

    Mtnt: A testbed for machine translation of noisy text,

    P. Michel and G. Neubig, “Mtnt: A testbed for machine translation of noisy text,” in Proceedings of Empirical Methods in Natural Language Processing, 2018

    2018

  8. [16]

    Synthetic and natural noise both break neural machine translation,

    Y . Belinkov and Y . Bisk, “Synthetic and natural noise both break neural machine translation,” in Proceedings of the International Conference on Learning Representations , 2017

    2017

  9. [17]

    Exploring speech enhancement with generative adversarial networks for ro- bust speech recognition,

    C. Donahue, B. Li, and R. Prabhavalkar, “Exploring speech enhancement with generative adversarial networks for ro- bust speech recognition,” in IEEE International Confer- ence on Acoustics, Speech and Signal Processing , 2018, pp. 5024–5028

    2018

  10. [18]

    Odsqa: Open-domain spoken question answering dataset,

    C.-H. Lee, S.-M. Wang, H.-C. Chang, and H.-Y . Lee, “Odsqa: Open-domain spoken question answering dataset,” in 2018 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2018, pp. 949–956

    2018

  11. [19]

    Studio Ousia’s quiz bowl question answering system,

    I. Yamada, R. Tamaki, H. Shindo, and Y . Takefuji, “Studio Ousia’s quiz bowl question answering system,” in NIPS Competition: Building Intelligent Systems, 2018, pp. 181– 194

    2018

  12. [20]

    Searchqa: A new Q&A dataset augmented with context from a search engine,

    M. Dunn, L. Sagun, M. Higgins, V . U. Güney, V . Cirik, and K. Cho, “Searchqa: A new Q&A dataset augmented with context from a search engine,”CoRR, vol. abs/1704.05179, 2017

    2017 arXiv

  13. [21]

    Using tf-idf to determine word relevance in document queries,

    J. Ramos, “Using tf-idf to determine word relevance in document queries,” in Proceedings of the International Conference of Machine Learning, 2003

    2003

  14. [22]

    The probabilistic rele- vance framework: Bm25 and beyond,

    S. Robertson, H. Zaragoza et al., “The probabilistic rele- vance framework: Bm25 and beyond,” Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009

    2009

  15. [23]

    The Kaldi speech recognition toolkit,

    D. Povey, A. Ghoshal, G. Boulianne, N. Goel, M. Hanne- mann, Y . Qian, P. Schwarz, and G. Stemmer, “The Kaldi speech recognition toolkit,” in IEEE Workshop on Auto- matic Speech Recognition and Understanding, 2011

    2011

  16. [24]

    Deep unordered composition rivals syntactic methods for text classification,

    M. Iyyer, V . Manjunatha, J. Boyd-Graber, and H. Daumé III, “Deep unordered composition rivals syntactic methods for text classification,” in Proceedings of the Association for Computational Linguistics, 2015

    2015

  17. [25]

    Automatic differentiation in pytorch,

    A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” inConference on Neural Information Processing Systems: Autodiff Work- shop: The Future of Gradient-based Machine Learning Sof...

    2017

  18. [26]

    Unsupervised training of acoustic models for large vocabulary continuous speech recogni- tion,

    F. Wessel and H. Ney, “Unsupervised training of acoustic models for large vocabulary continuous speech recogni- tion,” IEEE Transactions on Speech and Audio Processing, vol. 13, no. 1, pp. 23–31, 2004

    2004

  19. [27]

    Unsupervised feature learning for audio classification using convolutional deep belief networks,

    H. Lee, P. Pham, Y . Largman, and A. Y . Ng, “Unsupervised feature learning for audio classification using convolutional deep belief networks,” in Proceedings of Advances in Neu- ral Information Processing Systems, 2009, pp. 1096–1104

    2009

  20. [28]

    BLEU: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: a method for automatic evaluation of machine translation,” in Proceedings of the Association for Computational Lin- guistics, 2002, pp. 311–318

    2002

  21. [29]

    A mathematical theory of communication,

    C. E. Shannon, “A mathematical theory of communication,” Bell system technical journal, vol. 27, no. 3, pp. 379–423, 1948. Figure 2: A comparison of BLEU score distributions across human speakers (color-coded) to our artificial method, visualized by the step line. The distribut...

    1948

  22. [30]

    A “lattice” LSTM that consumes the full lattice by linearizing the graphs with a topological sort and feeding it through a normal LSTM

  23. [31]

    This network only sees the word vectors when consuming the lattice structure

    A lattice LSTM without confidences. This network only sees the word vectors when consuming the lattice structure

  24. [32]

    The confidences are concatenated to the word vector inputs

    A lattice LSTM with confidences integrated as fea- tures. The confidences are concatenated to the word vector inputs. This sequence demonstrates the gain from each part of the model. The first tests the benefit of additional data. The second tests the benefit of the structure of th...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.