Pith. sign in

REVIEW 3 major objections 6 minor 22 references

Multi-turn conversation collapses vision-language model accuracy on French financial documents to about 50 percent, regardless of model size.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:07 UTC pith:YEOQJXOR

load-bearing objection The benchmark is a useful contribution; the error-propagation headline overclaims. the 3 major comments →

arxiv 2602.10384 v4 pith:YEOQJXOR submitted 2026-02-11 cs.CL

When Tables Go Crazy: Evaluating Multimodal Models on French Financial Documents

Virginie Mouilleron , Th\'eo Lasnier , Anna Mosolova , Djam\'e Seddah This is my paper
classification cs.CL
keywords financial documentsvision-language modelsmultimodal benchmarkFrench NLPdocument understandingchart interpretationmulti-turn dialogueerror propagation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to test whether current open-weight vision-language models can be trusted on real French financial documents—prospectuses, key information documents, and PRIIPs—where a wrong percentage is not a small error. It builds a 1,204-question benchmark covering text extraction, table reading, chart interpretation, and multi-turn conversation, and evaluates six models from 8B to 124B parameters. Its central finding is a split: models look deployable on well-scoped extraction (roughly 85–90% on text and tables), but they are poor at reading charts (34–62%), and they collapse in multi-turn dialogue, landing near 50% accuracy regardless of model size. The paper argues the conversational collapse is caused by error propagation, not missing reasoning capacity, because when models are given correct previous answers they score much higher, with a clear scaling benefit. A sympathetic reader would care because interactive financial analysis is where automation would be most useful, and the benchmark provides a concrete way to measure whether fixes actually work.

Core claim

A current open-weight vision-language model can pull a fee or a duration out of a French table with high accuracy, but it cannot carry a 5–10 turn conversation about the same document. The paper's evidence: the best text/table accuracy reaches 85–90%, chart accuracy tops out at 61.7%, and in the conversational condition where the model's own previous answers are fed back, all six models land between 46.2% and 58.5%, with the 8B model beating the 124B model by only 6 points. In the matched gold-context condition—same questions, same image, but correct previous answers in the history—scores rise to 63.1–86.2% and scale cleanly with size. The paper reads this as direct evidence that the bottlen

What carries the argument

The load-bearing instrument is the benchmark's matched conversational pair. The same multi-turn questions are run twice: once with the model's own previous answers placed in the conversation history (Conversational Standard) and once with gold answers in their place (Conversational Gold). Because the only difference is the content of the history, the score gap between the two conditions is a direct measure of how much early errors contaminate later turns. The scoring system is an LLM-as-judge protocol: three open-weight judge models independently label each response correct/incorrect, and a majority vote decides. The judge rubric explicitly allows numerical equivalences (e.g., 20% = 0.2) to

Load-bearing premise

The headline claim rests on the majority votes of three open-weight LLM judges reading French financial answers; if those judges are biased—no human evaluation or calibration is reported—then the specific numbers, including the ~50% conversational floor, are not established.

What would settle it

Score a stratified sample of 200–300 responses (50+ per task family) with a human French-financial expert, then compare against the LLM-majority labels; if agreement on the conversational subset is low, or if rotating the judge models (e.g., replacing all judges from the evaluated families) moves the conversational numbers outside the 46–59% band, the scaling-independent-collapse claim is refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Single-turn extraction from French regulatory documents is close to practical for the best tested models (85–90% on text and tables); the limiting factor for productization is charts and conversations, not basic OCR-style reading.
  • The gold-vs-standard gap implies that equipping a system with a verification or self-correction loop, rather than a larger base model, is the most promising route to usable multi-turn financial analysis.
  • Any deployment of conversational VLM assistants in finance should treat early answers as state that can poison later ones; resetting or re-grounding context is an architectural requirement, not a convenience.
  • Chart understanding deserves separate evaluation and training: it is a distinct failure mode with a 20–30 point drop relative to table tasks across all model sizes.
  • Benchmark results from single-turn tasks are not predictive of multi-turn reliability; evaluation suites for high-stakes domains should include the gold-vs-model context contrast used here.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's judge pool draws two of its three members from the same model families as the evaluated systems; until a human-scored subset or judge-family swap is checked, the exact ordering and even the sharpness of the conversational collapse are not fully separated from judge self-preference.
  • A testable extension: add an explicit 'second chance' turn where the model can revise its previous answer after seeing the next question; the gold-context result predicts a substantial accuracy recovery, which would confirm the state-contamination explanation in a practical intervention.
  • The same error-propagation mechanism should appear in any multi-step multimodal chain—not just finance—so the benchmark design could be ported to legal, medical, or insurance document workflows by swapping the document corpus and re-running the gold-vs-model contrast.
  • Because generation favored locally salient facts, the observed 85–90% text/table performance may be an upper bound for real holistic document understanding; a follow-up benchmark weighted toward cross-section synthesis would likely find larger gaps.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces MULTIMODALFINANCEEVAL (referred to in the abstract as "Scribe Finance"), a benchmark of 1,204 expert-validated question–answer pairs for French financial document understanding, covering text extraction, table comprehension, chart interpretation, and multi-turn conversation. Six open-weight VLMs (8B–124B) are evaluated with an LLM-as-judge protocol. The reported results show strong performance on text and table tasks (roughly 85–90%), weak chart interpretation (34–62%), and a striking drop in the conversational condition where models must use their own prior answers rather than gold answers (46–59%). The paper's central claim is that this drop demonstrates error propagation across dialogue turns and that conversational accuracy converges to ~50% regardless of model scale.

Significance. If the conversational finding holds, it is an important negative result: it would indicate that current open-weight VLMs are not reliable for interactive multi-step financial document analysis, and that scaling alone does not mitigate error accumulation. The benchmark itself is a substantial contribution: it is larger than prior French financial QA resources, covers multiple modalities, includes a reasoned semi-automatic construction pipeline with expert revision, and the authors transparently document generation biases and limitations. The paper also ships reproducible prompts and a clear evaluation template. However, the headline claim about error propagation is not directly established by the reported evidence, and the evaluation protocol has validity risks that could affect all absolute scores.

major comments (3)
  1. [§5, Table 4, Appendix 11.2.3] The paper's headline claim — "early mistakes propagate across turns, driving accuracy down to roughly 50% regardless of model size" — is inferred from the aggregate comparison between Conversational Gold and Conversational Standard conditions. This comparison shows that replacing model completions with gold answers improves overall accuracy, but it does not demonstrate that a model's own error at turn t causes an error at turn t+1. No turn-level analysis is provided (e.g., accuracy at turn k conditioned on correctness at turn k−1, or the distribution of first-error positions). Alternative explanations remain plausible, such as models being worse at processing longer multi-turn contexts even when prior answers are correct, or a general difficulty with the specific MCQ reasoning task. Additionally, only 65 conversational items exist (Table 2); for proportions near 0.5 the standard error is
  2. [§4 Evaluation Protocol] All reported accuracies rely on majority-vote judgments from three open-weight LLMs. No human evaluation or judge calibration is reported, and two of the three judges (Qwen3-32B, Gemma-3-27B) belong to the same model families as the evaluated VLMs (Qwen3-VL and Gemma-3). The citation of Chen et al. (2025) does not directly address this exact overlap; that study's self-preference results are not demonstrated for this judge/answer configuration. Since the absolute scores and model rankings could be affected by judge self-preference, the paper should validate the judge protocol on a human-annotated subsample (e.g., 100 items), report per-judge accuracy and inter-judge agreement, and disclose the possible impact on the reported rankings. The Gold-vs-Standard contrast is less sensitive to this bias, but the overall numbers in Table 4 and the abstract's "~50%" claim inherit the concern.
  3. [§1 and §2.2, Table 1] The contribution claim "the first multimodal benchmark for French financial document understanding" conflicts with the cited FAMMA (Xue et al., 2025), which is a multilingual multimodal financial benchmark and, by the paper's own Table 1, includes French content (9%). If the intended claim is "the first French-only" or "the first focused on regulatory prospectuses", it should be restated precisely. Otherwise the novelty assertion is overstated. Relatedly, the abstract introduces the benchmark as "Scribe Finance" while the body consistently uses "MULTIMODALFINANCEEVAL"; this naming inconsistency should be corrected.
minor comments (6)
  1. [Abstract and §1] The benchmark name is inconsistent: the abstract says "Scribe Finance", the body says "MULTIMODALFINANCEEVAL", and footnote 1 links to an anonymous repository named "squad_finance". Unify the name throughout.
  2. [Table 4] The column header "Conv." is ambiguous. Rename to "Conv. Standard" (or "Conv. Model") to clearly distinguish it from "Conv. Gold".
  3. [§2.2, References] The Faysse et al. (2025) citation is associated with a dataset of 210 retrieval/table questions, but the reference points to the ColPali document-retrieval paper. If a different work is intended, update the citation; if the ColPali paper indeed contains this dataset, clarify the connection.
  4. [Table 4] The "Avg" column appears to be an unweighted mean of task accuracies, even though task sizes differ greatly (e.g., 501 text items vs 65 conversational items). State whether the average is weighted by number of items; otherwise the aggregate scores may be misleading.
  5. [§3, Dataset Construction] The dataset is described as "expert-validated", but only a single annotator is mentioned and no inter-annotator reliability or independent audit is reported. A brief discussion of this choice would be helpful, especially since the benchmark's validity depends on annotation quality.
  6. [§5 and §6] The paper does not discuss the possibility that models have seen some of these public financial documents during pretraining. A contamination check or an explicit acknowledgment of this risk would strengthen the benchmark's conclusions.

Circularity Check

0 steps flagged

No significant circularity: the central results are empirical measurements, and the Gold-vs-Standard contrast is a controlled comparison rather than a definitional identity.

full rationale

The paper makes no derivation claim in which an output is defined in terms of the quantity it purports to predict. The central conversational-collapse claim is an observed accuracy gap between two conditions, Conversational Gold and Conversational Standard, which differ only in whether the dialogue history contains expert gold answers or model-generated previous answers. The gap is not forced by construction: nothing in the prompt templates or dataset design guarantees that the standard condition will yield ~50% accuracy, and the gold-condition scores (63.1-86.2%) show the same models can answer similar questions when given correct context. The LLM-as-judge protocol introduces a possible self-preference bias because two of the three judges (Qwen3-32B, Gemma-3-27B) come from the same model families as evaluated VLMs; however, this is an evaluation-validity concern, not circularity: the judge prompt compares model predictions against expert-validated gold answers, and the paper explicitly acknowledges the risk and cites an external study (Chen et al. 2025) rather than its own prior results. The statistical weaknesses the reviewers note—65 conversational items, no per-turn conditional analysis, and an aggregate rather than turn-level contrast—bear on whether the 'error propagation regardless of scale' conclusion is established, but they do not make the claim equivalent to its inputs by construction. No load-bearing self-citation, imported uniqueness theorem, ansatz smuggled through citation, or renaming of a known result appears in the paper.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claims rest on empirical measurements, not derivation. The most consequential assumptions are the validity of the LLM-as-judge scoring, the quality of a single-expert gold annotation, the representativeness of the semi-automatically generated questions, and the assumption that the conversational Gold-vs-Standard comparison isolates error propagation.

axioms (4)
  • domain assumption Majority-vote LLM-as-judge scoring is a valid and unbiased measure of correctness for all 1,204 answers.
    Used in §4 to produce every accuracy in Table 4; no human evaluation is reported, and two judges (Qwen3-32B, Gemma-3-27B) overlap with evaluated model families.
  • domain assumption The single financial expert's validation and rewriting of all questions yields correct gold answers.
    §3.2 states one expert reviewed and rewrote or removed 75% of the questions; no inter-annotator agreement is reported.
  • domain assumption GPT-4o/Gemini-2.0-generated questions, after expert revision, are representative of real financial document understanding tasks.
    The paper itself notes generation biases toward locally salient facts and limited lexical diversity (§6.2, §8), so representativeness is not automatically guaranteed.
  • domain assumption The Conv Gold vs Conv Standard comparison isolates the effect of error propagation.
    Used to infer that degradation is due to error accumulation rather than reasoning capacity; alternative confounds (e.g., different judge behavior or task difficulty) are not fully controlled.

pith-pipeline@v1.3.0-alltime-deepseek · 15399 in / 12203 out tokens · 121290 ms · 2026-08-03T01:07:10.417496+00:00 · methodology

0 comments
read the original abstract

Vision-language models (VLMs) perform well on many document understanding tasks, yet their reliability in specialized, non-English domains remains underexplored. This gap is especially critical in finance, where documents mix dense regulatory text, numerical tables, and visual charts, and where extraction errors can have real-world consequences. We introduce Scribe Finance, the first multimodal benchmark for evaluating French financial document understanding. The dataset contains 1,204 expert-validated questions spanning text extraction, table comprehension, chart interpretation, and multi-turn conversational reasoning, drawn from real investment prospectuses, KIDs, and PRIIPs. We evaluate six open-weight VLMs (8B-124B parameters) using an LLM-as-judge protocol. While models achieve strong performance on text and table tasks (85-90% accuracy), they struggle with chart interpretation (34-62%). Most notably, multi-turn dialogue reveals a sharp failure mode: early mistakes propagate across turns, driving accuracy down to roughly 50% regardless of model size. These results show that current VLMs are effective for well-defined extraction tasks but remain brittle in interactive, multi-step financial analysis. Scribe Finance offers a challenging benchmark to measure and drive progress in this high-stakes setting.

Figures

Figures reproduced from arXiv: 2602.10384 by Anna Mosolova, Djam\'e Seddah, Th\'eo Lasnier, Virginie Mouilleron.

Figure 1
Figure 1. Figure 1: Overview of the MULTIMODAL FINANCE EVAL benchmark construction and evaluation pipeline. French financial documents (prospectuses, KIDs, PRIIPs) are collected from asset management compa￾nies, then processed to generate question-answer pairs spanning text, tables, and charts. Six Vision￾Language Models are evaluated on these tasks, with responses assessed using a majority-vote LLM-as￾judge protocol. dataset… view at source ↗
Figure 2
Figure 2. Figure 2: Model accuracy on image-based question subcategories. Performance remains strong on table [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Model accuracy on text-based question subcategories by context length. All models achieve [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example table from a financial document. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: Chart interpretation example based on [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: This example demonstrates how ques￾tions require maintaining context across turns and performing numerical reasoning based on tabular financial data. QUELS SONT LES RISQUES ET QU’EST-CE QUE CELA POURRAIT ME RAPPORTER ? (SUITE) Scénarios de performance Les scénarios présentés illustrent la performance de votre investissement au cours des 5 prochaines années en supposant que vous investissiez 10 000,00 $. Vo… view at source ↗
Figure 10
Figure 10. Figure 10: Prompt template for Image-based tasks. User: Context: {context} Question: {question} Answer the question concisely based on the context provided. Don’t include any explanations. Assistant: Answer: [model completion] [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Prompt template for Text-based task. cluded in the conversation history. In the Conv. Gold setting, ground-truth answers replace model completions. 11.2.4. LLM-as-judge Evaluation Template [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: LLM-as-judge evaluation template. Five examples covering numerical equivalence and multiple-choice formats are included in the full prompt [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    Jiang, Kartik Khandelwal, Timothée Lacroix, Guillaume Lample, Diego Las Casas, Thibaut Lavril, and 23 others

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, Soham Ghosh, Amélie Héliou, Paul Jacob, Albert Q. Jiang, Kartik Khandelwal, Timothée Lacroix, Guillaume Lample, Diego Las Casas, Thibaut Lavril, and 23 others. 2024. https://arxiv.org/ab...

    Pith/arXiv arXiv 2024

  2. [2]

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. 2022 a . https://doi.org/10.48550/arXiv.2109.00122 FinQA : A Dataset of Numerical Reasoning over Financial Data . arXiv preprint. ArXiv:2109.00122 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2109.00122 2022

  3. [3]

    Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. 2022 b . https://doi.org/10.48550/arXiv.2210.03849 ConvFinQA : Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering . arXiv preprint. ArXiv:2210.03849 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.03849 2022

  4. [4]

    Yang Deng, Wenqiang Lei, Wenxuan Zhang, Wai Lam, and Tat-Seng Chua. 2023. https://doi.org/10.48550/arXiv.2210.08817 PACIFIC : Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance . arXiv preprint. ArXiv:2210.08817 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.08817 2023

  5. [5]

    Martin d ' Hoffschmidt, Wacim Belblidia, Quentin Heinrich, Tom Brendl \'e , and Maxime Vidal. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.107 FQ u AD : F rench question answering dataset . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1193--1208, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.findings-emnlp.107 2020

  6. [6]

    European Commission . 2014. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX Official Journal of the European Union

    2014

  7. [7]

    European Union . 2014. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX Official Journal of the European Union

    2014

  8. [8]

    G20 . 2009. https://www.g20.utoronto.ca/analysis/hse/2009-london-compliance.pdf Leaders' statement: The global plan for recovery and reform . G20 London Summit, April 2009

    2009

  9. [9]

    Gemma Team , Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, and 197 others. 2025. https://arxiv.org/abs/2503.1978...

    Pith/arXiv arXiv 2025

  10. [10]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. https://arxiv.org/abs/2407.21783 The llama 3...

    Pith/arXiv arXiv 2024

  11. [11]

    Rachel Keraron, Guillaume Lancrenon, Mathilde Bras, Frédéric Allary, Gilles Moyse, Thomas Scialom, Edmundo-Pavel Soriano-Morales, and Jacopo Staiano. 2020. https://doi.org/10.48550/arXiv.2007.00968 Project PIAF : Building a Native French Question - Answering Dataset . arXiv preprint. ArXiv:2007.00968 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2007.00968 2020

  12. [12]

    Yanis Labrak, Adrien Bazoge, Richard Dufour, Mickael Rouvier, Emmanuel Morin, Béatrice Daille, and Pierre-Antoine Gourraud. 2023. https://doi.org/10.48550/arXiv.2304.04280 FrenchMedMCQA : A French Multiple - Choice Question Answering Dataset for Medical domain . arXiv preprint. ArXiv:2304.04280 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2304.04280 2023

  13. [13]

    Jiahua Liu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2019. https://doi.org/10.18653/v1/P19-1227 XQA : A Cross -lingual Open -domain Question Answering Dataset . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 2358--2368, Florence, Italy. Association for Computational Linguistics

    work page doi:10.18653/v1/p19-1227 2019

  14. [14]

    Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. https://doi.org/10.48550/arXiv.2007.15207 MKQA : A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering . arXiv preprint. ArXiv:2007.15207 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2007.15207 2021

  15. [15]

    Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020. https://doi.org/10.48550/arXiv.2010.00133 CrowS - Pairs : A Challenge Dataset for Measuring Social Biases in Masked Language Models . arXiv preprint. ArXiv:2010.00133 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2010.00133 2020

  16. [16]

    Aurélie Névéol, Yoann Dupont, Julien Bezançon, and Karën Fort. 2022. French CrowS - Pairs : Extension à une langue autre que l’anglais d’un corpus de mesure des biais sociétaux dans les modèles de langue masqués. Actes de la 29e Conférence sur le Traitement Automatique des Langues Naturelles

    2022

  17. [17]

    Qwen Team . 2025. https://arxiv.org/abs/2505.09388 Qwen3 technical report . Preprint, arXiv:2505.09388

    Pith/arXiv arXiv 2025

  18. [18]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2016

  19. [19]

    Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023. https://doi.org/10.1162/tacl_a_00595 MIRACL : A Multilingual Retrieval Dataset Covering 18 Diverse Languages . Transactions of the Association for Computational Linguistics, 11:1114--1131

    doi:10.1162/tacl_a_00595 2023

  20. [20]

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. https://doi.org/10.48550/arXiv.2105.07624 TAT - QA : A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance . arXiv preprint. ArXiv:2105.07624 [cs]

    Pith/arXiv arXiv doi:10.48550/arxiv.2105.07624 2021

  21. [21]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  22. [22]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...