Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StructText shows LLMs can generate accurate reports that automated extractors struggle to read back into tables.

desk verdict A useful, reproducible benchmark-generation stack with a plausible but not fully proven main claim about faithful text resisting extraction. read the letter →

arxiv 2507.21340 v1 pith:XINJFHBU submitted 2025-07-28 cs.CL cs.AIcs.DBcs.IR

Satyananda Kashyap , Sola Shirai , Nandana Mihindukulasooriya , Horst Samulowitz This is my paper
classification cs.CLcs.AIcs.DBcs.IR
keywords syntheticbenchmarkgenerationtable-to-texttext-to-tableextractionkey-valueLLM-as-judgefactualityevaluationnumericaccuracyplan-then-execute
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

StructText is an automated pipeline for turning existing tabular data into natural-language reports and then benchmarking systems that extract the table back out. The paper's central claim is that this pipeline produces reports that preserve the source row's numeric and temporal facts with precision and recall above 0.9, yet are hard for an LLM-based extractor to parse: column-identification F1 lands at 0.455 on SEC filings and 0.413 on WikiDB tables, with value-extraction precision/recall below 0.26/0.14. If true, the framework makes benchmark construction scalable to any domain and exposes a gap between generation fidelity and information accessibility. The authors use the generated corpus of 71,539 examples across 49 datasets to argue that current LLMs write factually strong, hallucination-light reports whose narrative organization hides the semantic relations needed for automated extraction.

What carries the argument

The load-bearing mechanism is the plan-then-execute generation loop plus a multi-dimensional validation stack. In planning, the model inspects ten sample rows and proposes one to five report types, each naming a coherent group of columns; in execution, it generates one report per row from exactly those columns, with the row's cell values as the only grounding. The evaluation stack combines LLM-as-judge 5-point rubrics for factuality, hallucination, and coherence, with a sentence-level claims self-consistency check, and objective parser-based checks for numeric and temporal fidelity using normalized values with a 0.1% relative error tolerance. A baseline extractor then recovers schema and values from the reports, and predictions are matched to ground-truth columns with a bipartite assignment maximizing similarity before precision, recall, and F1 are computed. These pieces together make the claimed fidelity-versus-extractability gap measurable.

What would settle it

Swap the baseline extractor for a stronger open-weights LLM on the same released reports and recompute value and column F1; if F1 rises above about 0.8 while numeric fidelity remains above 0.9, the paper's central 'accurate but not extractable' gap is an artifact of the baseline rather than a property of the text.

Watch

Extended reading notes

Core claim

The core discovery, stated on the paper's own terms, is that the two-stage plan-then-execute generator, where an LLM first chooses column groupings and then writes grounded prose from only those columns, yields text that scores 4.58/4.56 on factuality and 4.90/4.55 on hallucination avoidance on a 5-point scale, with numeric and temporal precision and recall above 0.9, while the same text defeats the paper's baseline text-to-table extractor. The gap is quantified by extraction results: column F1 of 0.455 on SEC and 0.413 on WikiDB, and value-extraction precision/recall of 0.257/0.110 and 0.179/0.137. The authors read this as evidence that LLMs can generate accurate text but embed numeric and temporal values in narratives whose attributions to source columns are not recoverable by current extraction approaches. They position StructText as a reusable benchmark generator and release the datasets, judge prompts, filtering tools, and baseline extractor.

Load-bearing premise

The load-bearing premise is that Llama-3.3-70B-Instruct, when given the authors' 5-point rubrics, produces valid measures of factuality, hallucination, and coherence; if those judge scores do not track human judgment, the headline finding that LLMs generate accurate but poorly organized reports loses its evidentiary base.

Editorial extensions

If this is right

  • Any tabular dataset can be turned into a domain-specific text-to-table benchmark without manual annotation, so benchmark size and coverage are no longer limited by human effort.
  • Generation quality and extraction difficulty are separable axes: numeric and temporal fidelity approaching 0.9 does not imply that a reader can recover the structured record.
  • The published baseline extraction numbers give later systems a concrete reference point; beating column F1 of 0.455/0.413 and value precision below 0.26 means closing the accessibility gap.
  • Organizations evaluating LLM pipelines for report generation should measure downstream extractability in addition to factual accuracy, because the two can diverge sharply.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark is used as a training or selection signal, optimizing for extractability may pull generators toward more templated, explicitly attributed sentences, which could in turn lower the very coherence scores the paper reports as weak.
  • The 0.1% numeric tolerance and the parser's set-based comparison ignore semantic roles; a value that appears in the text but attached to the wrong column would still count as correct, so the near-ceiling numeric precision may be an upper bound on true fidelity.
  • The filtering sweep on temporal precision removes the hardest reports; the retained corpus may be easier for future extractors, so downstream results on the released filtered data must be compared against the same filtering settings.
  • Plan-then-execute generation suggests a natural control experiment: if the planning step is bypassed and columns are passed in a fixed order, extraction F1 should change measurably if narrative structure is what hurts accessibility.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes StructText, a two-stage pipeline that converts existing tabular data into natural-language reports (a 'plan-then-execute' generation using Qwen2.5-72B-Instruct), evaluates the generated text with LLM-based judges (Llama-3.3-70B-Instruct) on factuality, hallucination, and coherence, validates numeric and temporal information with parsers, optionally filters low-quality reports, and provides a baseline text-to-table extractor. Experiments on SEC filings and a WikiDB subset report high numeric/temporal fidelity (precision/recall mostly above 0.9) but low baseline extraction performance (column F1 around 0.41-0.46, value precision/recall below 0.26/0.14), which the authors interpret as a gap between generation fidelity and information accessibility. The framework and artifacts are released for community use.

Significance. If the reported pattern is robust, StructText would be a useful contribution: it addresses a real need for scalable, multi-domain benchmarks for key-value/text-to-table extraction, and the plan-then-execute design plus the release of code and data are concrete strengths. It is also commendable that the generation model and the evaluation judge are distinct (Qwen2.5-72B for generation, Llama-3.3-70B for evaluation), avoiding direct self-scoring. However, the paper's central empirical claim is not yet fully established: the two pillars supporting it—occurrence-based numeric/temporal validation and a single Levenshtein-matched extraction baseline—do not separately measure information preservation and information accessibility as cleanly as the text suggests. The missing validation of the LLM judge and the absence of error bars or significance tests further weaken the quantitative conclusions.

major comments (4)
  1. [Section 3.2, Table 3] The numeric/temporal validation explicitly checks only whether normalized values occur in the generated text, not whether they are attributed to the correct columns ('we do not specifically check whether the numeric values are expressing the semantics in the text accurately, but only check whether the numeric values occur'). Consequently, precision/recall above 0.9 does not establish that values are attached to the right fields: a report that moves revenue figures to the net-income position could pass the occurrence check. Since the headline claim is that the generated text preserves source information with high fidelity, the evaluation must also verify value-to-attribute binding, for example by judging extracted key-value pairs against the ground-truth row or by a human annotation sample. Without this, Table 3 overstates what the validation actually measures.
  2. [Sections 4.2 and 6.3, Table 4] The low extraction F1 that motivates the 'information accessibility' finding is produced by a single in-house baseline whose schema identification and column mapping depend on normalized Levenshtein distance. Natural paraphrases such as 'cash and cash equivalents' versus 'cash_equivalents', or 'fiscal year 2024' versus a 2024-12-31 cell, are penalized even when the extraction is semantically correct. Section 4.1 acknowledges null-valued columns but no handling for omitted nulls is described, so nulls may be counted as false negatives. The reported gap could therefore measure brittleness of the baseline and the matcher rather than the difficulty of extracting information from the narratives. Please add at least one stronger extractor, a templated-text control condition, or a human extraction study on a sample; without such a control, the main finding is not separable from evaluation artifacts.
  3. [Sections 3.2 and 5.2] The LLM-as-judge scores for factuality, hallucination, and coherence are used as evidence for the qualitative findings (strong factual accuracy, near-zero hallucination, weaker coherence), but no human agreement study, no correlation with existing judge benchmarks, and no error analysis are reported. The rubrics and self-consistency checks are reasonable safeguards, yet they do not establish that the judge's scores correspond to valid human judgments. Please report inter-annotator agreement (human versus LLM) on a sample, or at least provide judge rationales with failure cases, so readers can assess whether the near-ceiling factuality/hallucination numbers and the coherence gap are properties of the generated text or artifacts of the judge.
  4. [Sections 3.3, 5.3, 6.4] The quality-filtering step and the numeric/temporal conclusions are presented without any variance or significance information. The paper reports macro averages over datasets, but no error bars, confidence intervals, or significance tests accompany the key comparisons (e.g., SEC versus WikiDB differences in Table 4). Several conclusions, such as 'temporal precision emerged as the weakest component', depend on small numeric differences (0.818 vs 0.849 precision in Table 3), and it is unclear whether these differences are stable. Please report per-dataset variability or otherwise quantify the uncertainty of the headline metrics.
minor comments (5)
  1. [Abstract, Section 6, Section 7.2] The abstract reports 71,539 examples across 49 datasets, while Section 6 says the evaluation was on 50 datasets; Section 7.2 then refers to 'a representative subset of 50 tables' from WikiDBs, although Section 5.1 states 1,000 tables. Please reconcile these numbers.
  2. [Table 4] The value-extraction F1 cells are left as '–' even though the text says the table shows precision, recall, and F1. Please either report the F1 values or explicitly explain why they are omitted.
  3. [Section 6.4] The sentence 'Temporal precision emerged as the weakest component (see Table 4)' appears to reference the wrong table; temporal precision is reported in Table 3, not Table 4.
  4. [References [35] and [36]] References [35] and [36] both cite the same MT-Bench paper with overlapping author lists; please deduplicate or clearly differentiate the two entries.
  5. [Figure 5] The example in Figure 5 is presented as a parsing challenge, but it also illustrates a limitation of the validation metric: 'fiscal year 2024' is valid temporal information that is absent from the ground-truth columns, so the false positive is an annotation mismatch rather than an error in the generated text. This should be acknowledged when interpreting temporal precision scores.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark pipeline is externally grounded in source tables and independent models; the flagged limitations are validity concerns, not circular reasoning.

full rationale

I found no load-bearing circular steps. The generation model (Qwen2.5-72B) is distinct from the evaluation judge (Llama-3.3-70B), the source tables are external ground truth, and the extraction baseline is an independent pipeline, so the reported low extraction F1 is not mathematically forced by the generation inputs. The numeric/temporal validation explicitly checks only value occurrence rather than semantic column attachment, as stated in Section 3.2: 'we do not specifically check whether the numeric values are expressing the semantics in the text accurately, but only check whether the numeric values occur.' This limits the strength of the fidelity claim but does not make the measured scores equal to the inputs by construction. The quality-filtering sweep over tau is disclosed as a sweep over 1.0 to 0.70, and the resulting retention curve is reported as an empirical trade-off, not as a prediction from fitted parameters. The paper contains no self-citations used as load-bearing evidence, no imported uniqueness theorem, and no ansatz smuggled in via prior work by the same authors. The main threats to the central claim, such as the baseline's normalized Levenshtein matching penalizing paraphrases or the absence of human judge validation, are external-validity and measurement concerns rather than circular derivations, so the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The pipeline introduces no entities, just a workflow. The true cost of the method is in its modeling assumptions: single-row passages, fixed per-type schemas, the unvalidated LLM judge, and parser-defined fidelity. These assumptions are disclosed by the authors but not stress-tested.

free parameters (4)
  • number of planning rows sampled = 10
    Chosen as a balance between computational efficiency and coverage; the paper states ablations on sample size remain future work (Section 3.1).
  • report type count limit = 1 to 5
    The planning prompt caps output at one to five report types to enforce analytical grouping rather than column enumeration (Section 3.1).
  • numeric relative error tolerance = 0.1%
    Used to treat rounded values as correct in numeric validation (Section 3.2).
  • quality filtering threshold tau = swept 1.0 to 0.70 in 0.05 steps
    Reports below the threshold on the weakest validation dimension (temporal precision) are dropped; the choice is user-set and affects the released benchmark's difficulty (Section 3.3).
assumptions (4)
  • domain assumption Each generated report corresponds to exactly one table row.
    Section 3.1: the authors assume one row per passage, arguing multi-row passages would be 'unreasonably challenging' for extraction and evaluation; this shapes the entire benchmark and excludes a common real-world scenario.
  • domain assumption A fixed, consistent column set can be extracted from every report of a given report type.
    Section 4.1: the baseline extraction and evaluation assume a consistent set of columns per report type, which simplifies evaluation but limits the benchmark's coverage of variable-schema documents.
  • domain assumption Llama-3.3-70B-Instruct's rubric-based judgments are valid measurements of factuality, hallucination, and coherence.
    Sections 3.2 and 5.2: no human agreement study, correlation with existing benchmarks, or error analysis is provided; the headline qualitative findings rest on this assumption.
  • domain assumption CoreNLP/SUTime extractions correctly delimit the ground-truth numeric and temporal values.
    Sections 3.2 and Figure 5: the validation treats parser output as ground truth, and the figure shows a case where the parser's 'fiscal year 2024' is counted as a false positive because it is not a ground-truth cell, illustrating this measurement choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation." pith.science (2026). https://pith.science/paper/XINJFHBU

@misc{pith2026250721340,
  author       = {Pith},
  title        = {Pith review of: StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XINJFHBU}},
  note         = {Machine review of arXiv:2507.21340}
}
read the original abstract

Extracting structured information from text, such as key-value pairs that could augment tabular data, is quite useful in many enterprise use cases. Although large language models (LLMs) have enabled numerous automated pipelines for converting natural language into structured formats, there is still a lack of benchmarks for evaluating their extraction quality, especially in specific domains or focused documents specific to a given organization. Building such benchmarks by manual annotations is labour-intensive and limits the size and scalability of the benchmarks. In this work, we present StructText, an end-to-end framework for automatically generating high-fidelity benchmarks for key-value extraction from text using existing tabular data. It uses available tabular data as structured ground truth, and follows a two-stage ``plan-then-execute'' pipeline to synthetically generate corresponding natural-language text. To ensure alignment between text and structured source, we introduce a multi-dimensional evaluation strategy that combines (a) LLM-based judgments on factuality, hallucination, and coherence and (b) objective extraction metrics measuring numeric and temporal accuracy. We evaluated the proposed method on 71,539 examples across 49 datasets. Results reveal that while LLMs achieve strong factual accuracy and avoid hallucination, they struggle with narrative coherence in producing extractable text. Notably, models presume numerical and temporal information with high fidelity yet this information becomes embedded in narratives that resist automated extraction. We release a framework, including datasets, evaluation tools, and baseline extraction systems, to support continued research.

Figures

Figures reproduced from arXiv: 2507.21340 by the authors.

Figure 1
Figure 1. LLM planning phase output showing natural group [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example of LLM table-to-text transformation for [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. SEC dataset extraction pipeline showing the programmatic generation of structured financial data from EDGAR [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Impact of filtering on the SEC dataset, showing the percentage of total data remaining after applying filtering with [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Parsing Challenges: The temporal metrics reveal a [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DTBench: A Synthetic Benchmark for Document-to-Table Extraction

    cs.DB 2026-02 conditional novelty 6.0 of 10

    A new synthetic benchmark shows LLMs doing document-to-table extraction are far weaker on indirect cells requiring reasoning, faithfulness, or conflict resolution than on direct text copying.

  2. Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning

    cs.AI 2025-09 conditional novelty 6.0 of 10

    An LLM agent using retrieval and summary uncertainty as training rewards and inference filters produces more factual, useful multi-omics summaries and better downstream survival predictions.

Reference graph

Works this paper leans on

38 extracted references · 21 canonical work pages · cited by 2 Pith papers

  1. [1]

    Hamed Babaei Giglou, Jennifer D’Souza, and Sören Auer. 2025. LLMs4Synthesis: Leveraging Large Language Models for Scientific Synthesis . Association for Com- puting Machinery, New York, NY, USA. https://doi.org/10.1145/3677389.3702565

    work page arXiv 2025

  2. [2]

    Alberto Berenguer, Adriana Morejón, David Tomás, and Jose-Norberto Mazón

    work page

  3. [3]

    Anjali Chaudhary. 2025. Factuality in LLMs: Key Metrics, Challenges & Improve- ment Strategies. Turing. https://www.turing.com/resources/llm-factuality-guide Accessed: 2025-05-28

    work page 2025

  4. [4]

    Zhiyu Chen, Wenhu Chen, Hanwen Zha, Xiyou Zhou, Yunkai Zhang, Sairam Sundaresan, and William Yang Wang. 2020. Logic2Text: High-Fidelity Natural Language Generation from Logical Forms. In Findings of the Association for Computational Linguistics: EMNLP 2020 , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 209...

    work page doi:10.18653/v1/2020.findings-emnlp.190 2020

  5. [5]

    Sergio De La Fe Jr, CPA Charles Hoffman, XBRL Solutions, Inc Elmer Huh, Morgan Stanley Dean Witter, and Elmer Huh. 2000. XBRL Taxonomy: Financial Reporting for Commercial and Industrial Companies, US GAAP

    work page 2000

  6. [6]

    Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, and Yangqiu Song. 2024. Text-Tuple-Table: Towards Informa- tion Integration in Text-to-Table Generation via Global Tuple Extraction. arXiv:2404.14215 [cs.CL] https://arxiv.org/abs/2404.14215

    arXiv 2024

  7. [7]

    Aaron Grattafiori et al. 2024. The Llama 3 Herd of Models. https://arxiv.org/ abs/2407.21783

    arXiv 2024

  8. [8]

    Jenny Rose Finkel, Trond Grenager, and Christopher D Manning. 2005. Incor- porating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd annual meeting of the association for compu- tational linguistics (ACL’05). 363–370

    work page 2005

Show all 38 references
  1. [9]

    Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez- Beltrachini. 2017. The WebNLG Challenge: Generating Text from RDF Data. In Proceedings of the 10th International Conference on Natural Language Gen- eration, Jose M. Alonso, Alberto Bugarín, and Ehud Reiter ...

    2017 doi

  2. [10]

    Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpek- tor. 2023. TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models. In Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing , Houda Bouamor, J...

    2023 doi

  3. [11]

    Hardian Health. 2023. Regulatory Approval for Medical LLMs. https://www. hardianhealth.com/insights/regulatory-approval-for-medical-llms. Accessed: 2025-05-28

    2023

  4. [12]

    Alon Jacovi, Andrew Wang, Chris Alberti, Connie Tao, Jon Lipovetz, Kate Ol- szewska, Lukas Haas, Michelle Liu, Nate Keating, Adam Bloniarz, et al . 2025. The FACTS Grounding Leaderboard: Benchmarking LLMs’ Ability to Ground Responses to Long-Form Input. arXiv preprint arXiv:25...

    2025 arXiv

  5. [13]

    Martin Josifoski, Marija Sakota, Maxime Peyrard, and Robert West. 2023. Exploit- ing Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , Ho...

    2023 doi

  6. [14]

    Zdeněk Kasner and Ondrej Dusek. 2024. Beyond Traditional Benchmarks: Ana- lyzing Behaviors of Open LLMs on Data-to-Text Generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers) , Lun-Wei Ku, Andre Martins,...

    2024 doi

  7. [15]

    Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav San- thanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2023. DSPy: Compiling Declarative Language Model Calls into Se...

    2023 arXiv

  8. [16]

    Rémi Lebret, David Grangier, and Michael Auli. 2016. Neural Text Generation from Structured Data with Application to the Biography Domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , Jian Su, Kevin Duh, and Xavier Carreras (Eds.)...

    2016 doi

  9. [17]

    Vladimir I Levenshtein et al. 1966. Binary codes capable of correcting deletions, insertions, and reversals. InSoviet physics doklady, Vol. 10. Soviet Union, 707–710

    1966

  10. [18]

    Wei Li, Wenhao Wu, Moye Chen, Jiachen Liu, Xinyan Xiao, and Hua Wu. 2022. Faithfulness in natural language generation: A systematic survey of analysis, evaluation and optimization methods. arXiv preprint arXiv:2203.05227 (2022)

    2022 arXiv

  11. [19]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013/

    2004

  12. [20]

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment. In Proceedings of EMNLP 2023

    2023

  13. [21]

    Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstratio...

    2014

  14. [22]

    Oshri Naparstek, Ophir Azulai, Inbar Shapira, Elad Amrani, Yevgeny Yaroker, Yevgeny Burshtein, Roi Pony, Nadav Rubinstein, Foad Abo Dahood, Orit Prince, Idan Friedman, Christoph Auer, Nikolaos Livathinos, Maksym Lysak, Ahmed Nassar, Peter W. J. Staar, and Udi Barzelay. 2024. K...

    2024

  15. [23]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , Pierre Isabelle, Eugene Charniak, and Dekang Lin (Eds...

    2002

  16. [24]

    Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A Controlled Table-To-Text Generation Dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Bonnie We...

    2020 doi

  17. [25]

    Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. [n.d.]. CORD: A Consolidated Receipt Dataset for Post- OCR Parsing. ([n. d.])

  18. [26]

    Laurent Perron and Vincent Furnon. [n.d.]. OR-Tools. Google. https://developers. google.com/optimization/

  19. [27]

    Cai, Michael Terry, and Minsuk Kahng

    Crystal Qian, Michael Xieyang Liu, Emily Reif, Grady Simon, Nada Hussein, Nathan Clement, James Wexler, Carrie J. Cai, Michael Terry, and Minsuk Kahng

  20. [28]

    Securities and Exchange Commission (SEC). 2025. SEC Filings. https://www. sec.gov/

    2025

  21. [29]

    arXiv:2412.16089 [cs.HC] https://arxiv.org/abs/2412.16089

    The Evolution of LLM Adoption in Industry Data Curation Practices. arXiv:2412.16089 [cs.HC] https://arxiv.org/abs/2412.16089

    arXiv

  22. [30]

    Liane Vogel, Jan-Micha Bodensohn, and Carsten Binnig. 2024. WikiDBs: A Large-Scale Corpus Of Relational Databases From Wikidata. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https://openreview.net/forum?id=abXaOcvujs

    2024

  23. [31]

    Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https://qwenlm. github.io/blog/qwen2.5/

    2024

  24. [32]

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. BloombergGPT: A Large Language Model for Finance. arXiv:2303.17564 [cs.LG] https://arxiv.org/abs/2303.17564

    2023 arXiv

  25. [33]

    Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM 57, 10 (2014), 78–85

    2014

  26. [34]

    Weinberger, and Yoav Artzi

    Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating Text Generation with BERT. In Interna- tional Conference on Learning Representations . https://openreview.net/forum? id=SkeHuCVFDr

    2020

  27. [35]

    Xueqing Wu, Jiacheng Zhang, and Hang Li. 2022. Text-to-Table: A New Way of Information Extraction. arXiv:2109.02707 [cs.CL] https://arxiv.org/abs/2109. 02707

    2022 arXiv

  28. [36]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT- bench and Chatbot Arena. In Proceedings of the 37th Interna...

    2023

  29. [37]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In NeurIPS 2023. 10

    2023

  30. [2024]

    Applied Sciences 14, 6 (2024)

    Leveraging Large Language Models for Sensor Data Retrieval. Applied Sciences 14, 6 (2024). https://doi.org/10.3390/app14062506

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.