Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and Retrieval

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Prompt design changes LLM performance on tabular EHR extraction and retrieval by up to 26.79%.

desk verdict Useful new EHR benchmark datasets and credible qualitative findings, but the headline effect sizes are inflated by test-set selection and by averaging relative gains across metrics. read the letter →

arxiv 2501.09384 v1 pith:2LV3UAOS submitted 2025-01-16 cs.CL cs.IR

Jesus Lovon (IRIT-IRIS) , Martin Mouysset (IRIT-IRIS) , Jo Oleiwan (IRIT-IRIS) , Jose G. Moreno (IRIT-IRIS) , Christine Damase-Michel , Lynda Tamine (IRIT-IRIS) This is my paper · ORCID
classification cs.CLcs.IR
keywords largelanguagemodelselectronichealthrecordstabulardatainformationextractionretrievalin-contextlearningpromptengineeringMIMICSQL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the way an EHR table is turned into text determines how well large language models can extract a single patient's answer from it and retrieve which patients match a query. Using the MIMICSQL patients, it compares Llama2 and Meditron across serialization formats, feature-selection strategies, guided versus non-guided instructions, and in-context demonstrations. It claims that combining all features (or their averaged values) with an LLM self-generated table description improves task performance by up to 26.79% over naive random feature choices, and that query-matched demonstrations improve extraction by 5.95%. It also claims retrieval is consistently harder than extraction for both models. The practical payoff is a set of prompt-design guidelines for building LLM-based health search tools.

What carries the argument

The load-bearing object is the prompt template `\textless Instruction [Demonstration] Context \textgreater` with `Context = $\phi(T_i) + q$`, where `$\phi$` is a serialization function and `$T_i$` is the patient's EHR table restricted to a feature subset. The paper compares three serializations (plain text template `txt`, HTML-tag-separated `xsep`, and LLM self-generated `sgen`) crossed with four feature selections (all, all averaged, random, random averaged). The second mechanism is the demonstration retriever `$\sigma$`, which chooses in-context examples by patient similarity or query similarity; it is the component that makes ICL help extraction. Serialization plus feature selection carries most of the observed performance spread; instruction type carries little.

What would settle it

Run the same MIMICask and MIMICsearch experiments but select the prompt configuration on a held-out validation split and evaluate only on untouched test patients; if the best-configuration advantage over the naive baseline shrinks below the reported 26.79% and 5.95% or disappears, the claim of large prompt-design gains is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that a prompt's context construction—specifically the combination of which medical features are kept and how the table is serialized—is the dominant factor in LLM performance on tabular EHR tasks, more than instruction phrasing. With the best settings, Llama2 and Meditron improve by 26.79% and 21.53% respectively over naive random feature selection; the best context uses all features (or averaged longitudinal values) and often an LLM self-generated description. In-context learning helps extraction when demonstrations are selected by query similarity (+5.95% BERTScore), but does not help retrieval, which performs best zero-shot. The authors read this as evidence that LLMs can extract fine-grained patient data from well-serialized tables but still struggle to match coarse patient-level queries to the right tables.

Load-bearing premise

The headline gains assume that picking the best settings from Table 2 on the test set is a valid way to measure those settings' benefit; without a separate validation split, the reported improvements may be inflated.

Editorial extensions

If this is right

  • Prompt context, not instruction wording, is the main lever: feature selection plus serialization should be tuned before trying to improve instructions.
  • Using all available features or their averaged values beats random feature subsets for both models, so defaulting to all features is a low-cost improvement.
  • Query-similar demonstrations should be the default ICL choice for extraction; patient-similar or random demonstrations offer no consistent benefit and can hurt.
  • Retrieval tasks should be treated separately: zero-shot prompting outperforms ICL for retrieval, and LLM pointwise rankers underperform BM25 and MonoT5 on the studied data.
  • MIMICask and MIMICsearch provide reusable benchmarks for future EHR extraction and retrieval evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A replication that selects prompt configurations on a held-out validation split rather than the test set would clarify how much of the reported 26.79% and 5.95% gains are genuine versus selection-dependent.
  • The serialization-and-feature-selection interaction may transfer to other structured clinical tables, such as pharmacy or imaging metadata, but the paper only studies MIMIC-derived tables, so that transfer is untested.
  • The extraction-versus-retrieval gap suggests EHR retrieval may need task-specific decoding or summary-based candidate generation rather than pointwise relevance judgments; the paper does not explore that alternative.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The manuscript presents an empirical study of how prompt design affects the ability of two open 7B LLMs (Llama2 and Meditron) to perform patient data extraction and retrieval from tabular electronic health records. The authors introduce two new datasets derived from MIMICSQL, MIMICask and MIMICsearch, and evaluate combinations of feature selection (all, all aggregated, random, random aggregated), serialization (template, X-separate, self-generated), instruction type (guided vs non-guided), and in-context demonstration selection (patient-based, query-based, random) using metrics ROUGE-1 and BERTScore for extraction and MAP and Recall@100 for retrieval. The paper reports that feature selection and serialization improve performance by up to 26.79%, that query-based demonstrations improve extraction by 5.95%, that guided instructions have minimal impact, and that retrieval is more difficult than extraction. It concludes with practical guidelines for prompting LLMs on EHR data.

Significance. If the quantitative findings were supported by a sound evaluation protocol, this would be a useful contribution to the growing literature on LLMs and tabular EHR data. The paper provides two new benchmark datasets, releases code, compares against several baselines including fine-tuned models and TREQS, and addresses a practical question that is relevant to health-search applications. However, the headline improvement magnitudes are currently threatened by a configuration-selection protocol that uses the test set, and by an aggregate metric that is dominated by a single low-baseline component. The qualitative direction of the findings is plausible and the datasets and code are valuable, but the central quantitative claims need to be re-established with a valid selection procedure and with uncertainty quantification.

major comments (5)
  1. [Section 4.1, Table 2] The optimal settings are selected from the test evaluations themselves: Section 4.1 states 'we chose the best settings from Table 2', and Sections 4.2 and 4.3 then fix 'best-performing settings' from the same tables before reporting final comparisons. Consequently, the headline gains of 26.79% and 5.95% are maxima over many unpaired configurations rather than unbiased estimates of an expected effect; they are likely inflated by selection bias. Please either hold out a validation split for configuration selection and report test results only for the final configuration, or use a selection-aware evaluation with appropriate correction for multiple comparisons.
  2. [Tables 2-4] All results appear to be single runs with no variance estimates or significance tests. Several comparisons are extremely small, e.g., Meditron retrieval MAP 10.34 vs 10.30 in Table 2 and BERTScore differences near 0.1 in Table 4, so summary claims such as '5 out of 8 metrics' and '9 out of 16 metrics' may reflect noise. Please report multiple seeds or bootstrap confidence intervals, and account for the number of configurations compared when claiming that one setting is optimal.
  3. [Table 2 and abstract] The Delta aggregate in Table 2 is an average of per-metric relative changes. For the Llama 'all/txt' row versus the 'rnd/txt' baseline, the relative gains are approximately +8.9% BERTScore, +80.0% ROUGE-1, +6.7% MAP, and +11.7% Recall@100, which average to the reported 26.79%. The headline is therefore dominated by a single low-baseline metric. Please report per-metric deltas alongside the aggregate and justify the aggregation, because the current abstract implies a more uniform across-the-board improvement than the data show.
  4. [Section 3.2, Table 2] The random feature-selection strategy keeps only 60% of the features (Section 3.2, footnote 3), so the 'all' versus 'rnd' and 'allavg' versus 'rndavg' comparisons vary both the feature subset and the number of features. The guideline 'Context is improved when using all available EHR features' is therefore not a clean feature-selection result. A comparison with matched feature counts, or with a systematic feature-selection method, is needed to attribute the gain to selection rather than to the sheer quantity of information.
  5. [Section 4.1, Section 6] The claim that retrieval is more difficult than extraction is not directly established by the presented comparisons, because the two tasks are evaluated with different metrics (Bscore/R-1 vs MAP/R@100) and the cross-task argument relies on relative improvements over task-specific baselines. Please reframe this as a claim about LLM performance relative to strong task-specific baselines, or provide a common evaluation protocol that makes task difficulty commensurable.
minor comments (5)
  1. [Table 1] Table 1 appears misaligned: the column labeled '# k/n' shows values 34 and 557, which do not equal the ratios k/n for MIMICask (5414/100) and MIMICsearch (19970/4000). Please correct the table or the column label.
  2. [Section 5, Table 5] The sentence 'explored LLMs perform better than all baselines, with the best score for Lma* with Bscore=62.44, except BART_ft' is self-contradictory; Table 5 shows that Lma*_ft reaches the best Bscore (84.79). Please reword to distinguish zero-shot LLMs, fine-tuned LLMs, and fine-tuned pre-trained baselines.
  3. [Section 5, Table 5] The statement that in retrieval 'studied LLMs only outperform MonoBERT' is imprecise: Med* beats MonoBERT on MAP but not on Recall@100, and Lma* does not beat it on either metric. Please report the per-metric comparisons accurately.
  4. [Section 3.3, Table 1] The text says 'We used the MIMICsearch small dataset for our study', but the experimental setup does not explicitly state that the reported retrieval results use the 250-query test set rather than the 1101-query set. Please state this clearly in the setup.
  5. [Section 3.2, Table 4] The formal definition of the demonstration retrieval function sigma says it provides examples for both extraction and retrieval, but the patient-based function sigma_p is only applicable to extraction. Please clarify the domain of each retrieval function.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the study is empirical and self-contained, with one minor self-citation that is not load-bearing.

full rationale

The paper does not contain a derivation chain that reduces to its own inputs. The central claims are empirical measurements of LLM performance on EHR extraction and retrieval using the MIMICSQL-derived datasets. The 26.79% and 5.95% improvement figures are computed from test-set evaluations in Tables 2 and 4, not from a fitted parameter or a self-referential definition; selecting the best-performing configuration from the same test table is a test-set selection-bias concern, not a circularity. There is no equation in which an output is defined in terms of the claimed result, and no fitted parameter is renamed as a prediction. The only self-citation is reference [16], used in Section 3.4 to justify the choice of Llama2 and Meditron as backbone models previously applied to patient-related tasks; this is a design choice and is not load-bearing for the paper's conclusions. The paper's claims are anchored to external data, external baselines (T5, BART, BM25, MonoBERT, MonoT5, TREQS), and publicly available toolkits. The limitation statement in Section 6 acknowledges that the exploration was restricted to optimal settings, which is a transparent caveat rather than evidence of circularity. No circular step can be quoted or exhibited, so the steps list is empty.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No mathematically fitted constants appear in the paper. The central claims rest on dataset-construction assumptions, evaluation metric choices, and experimental settings rather than on derived equations.

free parameters (1)
  • Fraction of features kept in random feature selection = 60%
    Section 3.2 states 'For both random approaches, we kept 60% of the total features'. This hand-chosen fraction affects the random baselines against which improvements are measured.
assumptions (3)
  • domain assumption MIMICSQL gold SQL queries provide correct ground truth for the derived extraction and retrieval datasets
    Section 3.3 builds MIMICask and MIMICsearch by converting golden SQL queries into natural language and executing them to create labels. Errors in the original SQL or in the SQL-to-NL conversion propagate to all metrics.
  • domain assumption ROUGE-1 and BERTScore are valid proxies for clinical answer quality
    Metrics are defined in Section 3.4. Lexical or embedding similarity may not reflect clinical correctness, e.g., an answer with the right words but wrong values could score well.
  • domain assumption The Dragon+ dense encoder used for demonstration selection is task-agnostic and does not leak the target answer into the prompt
    Section 3.4 uses an off-the-shelf unsupervised retriever to select examples based on representation similarity. If the encoder surface-forms the query and patient text, selection may introduce task-specific biases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and Retrieval." pith.science (2026). https://pith.science/paper/2LV3UAOS

@misc{pith2026250109384,
  author       = {Pith},
  title        = {Pith review of: Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2LV3UAOS}},
  note         = {Machine review of arXiv:2501.09384}
}
read the original abstract

Electronic Health Record (EHR) tables pose unique challenges among which is the presence of hidden contextual dependencies between medical features with a high level of data dimensionality and sparsity. This study presents the first investigation into the abilities of LLMs to comprehend EHRs for patient data extraction and retrieval. We conduct extensive experiments using the MIMICSQL dataset to explore the impact of the prompt structure, instruction, context, and demonstration, of two backbone LLMs, Llama2 and Meditron, based on task performance. Through quantitative and qualitative analyses, our findings show that optimal feature selection and serialization methods can enhance task performance by up to 26.79% compared to naive approaches. Similarly, in-context learning setups with relevant example selection improve data extraction performance by 5.95%. Based on our study findings, we propose guidelines that we believe would help the design of LLM-based models to support health search.

Figures

Figures reproduced from arXiv: 2501.09384 by the authors.

Figure 1
Figure 1. Illustration of the prompts used for the extraction and retrieval tasks, including Guided vs. Non-Guided instructions, and patient with txt (left) serializations. To synergically consider EHR table structure and content, we jointly explore a table structure serialization strategy ϕ with table feature selection of a subset of medical features F pi ⊆ F as follows: – All (all): all features and associated longitudinal … view at source ↗
Figure 2
Figure 2. (top) Example of random and query-based demonstrations in an ICL setup for extraction. (bottom) Example of ICL and zeroshot setup for retrieval. Highlighted the features (and values) referenced in demonstration and input. 5 Comparative evaluation and guidelines To better support our final guidelines, we compare in [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PatientDx: Merging Large Language Models for Protecting Data-Privacy in Healthcare

    cs.CL 2025-04 reject novelty 5.0 of 10

    PatientDx merges a math-specialized LLM with a medical or instruct LLM via SLerp and reports mortality-prediction gains on MIMIC-IV, but the merging weight is tuned on the test set, undermining the claimed improvement.

Reference graph

Works this paper leans on

38 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Chen, N., Shou, L., Gong, M., Pei, J., You, C., Chang, J., Jiang, D., Li, J.: Bridgethegapbetweenlanguagemodelsandtabularunderstanding.arXivpreprint arXiv:2302.09302 (2023)

    arXiv 2023

  2. [2]

    In: Findings of EACL (2023)

    Chen, W.: Large language models are few(1)-shot table reasoners. In: Findings of EACL (2023)

    work page 2023

  3. [3]

    Deng, X., Bashlovkina, V., Han, F., Baumgartner, S., Bendersky, M.: What do llms knowaboutfinancialmarkets?acasestudyonredditmarketsentimentanalysis.In: Companion Proceedings of the ACM Web Conference 2023. p. 107–110. WWW ’23 Companion, Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3543873.3587324

    arXiv 2023

  4. [4]

    ACM SIGMOD Record51(1), 33–40 (2022)

    Deng, X., Sun, H., Lees, A., Wu, Y., Yu, C.: Turl: Table understanding through representation learning. ACM SIGMOD Record51(1), 33–40 (2022)

    work page 2022

  5. [5]

    arXiv preprint arXiv:2402.17944 (2024)

    Fang, X., Xu, W., Tan, F.A., Zhang, J., Hu, Z., Qi, Y., Nickleach, S., Socolinsky, D., Sengamedu, S., Faloutsos, C.: Large language models on tabular data–a survey. arXiv preprint arXiv:2402.17944 (2024)

    arXiv 2024

  6. [6]

    In: Scott, D., Bel, N., Zong, C

    Gong, H., Sun, Y., Feng, X., Qin, B., Bi, W., Liu, X., Liu, T.: TableGPT: Few-shot table-to-text generation with table structure reconstruction and content matching. In: Scott, D., Bel, N., Zong, C. (eds.) Proceedings of the 28th Inter- national Conference on Computational Linguistics. pp. 1978–1988. International Committee on Computational Linguistics, B...

    doi:10.18653/v1/2020.coling-main.179 1978

  7. [7]

    In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A

    Haug, T., Ganea, O.E., Grnarova, P.: Neural multi-step reasoning for question answering on semi-structured tables. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) Advances in Information Retrieval. pp. 611–617. Springer In- ternational Publishing, Cham (2018)

    work page 2018

  8. [8]

    In: AISTATG

    Hegselmann, S., Buendia, A., Lang, H., Agrawal, M., Jiang, X., Sontag, D.A.: Tabllm: Few-shot classification of tabular data with large language models. In: AISTATG. vol. abs/2210.10723 (2022)

    arXiv 2022

Show all 38 references
  1. [9]

    In: European Conference on Information Retrieval

    Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., Zhao, W.X.: Large language models are zero-shot rankers for recommender systems. In: European Conference on Information Retrieval. pp. 364–381 (2024)

    2024

  2. [10]

    Sci Data3 (2016)

    Johnson, A., T., Pollard, T., Shen, L., al.: Mimic-iii, a freely accessible critical care database. Sci Data3 (2016)

    2016

  3. [11]

    In: Proceedings of the 44th Annual International ACM SI- GIR Conference on Research and Development in Information Retrieval (SIGIR 2021)

    Lin, J., Ma, X., Lin, S.C., Yang, J.H., Pradeep, R., Nogueira, R.: Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. In: Proceedings of the 44th Annual International ACM SI- GIR Conference on Research and Developm...

    2021

  4. [12]

    In: Findings of the Association for Computational Linguistics: EMNLP 2023

    Lin, S.C., Asai, A., Li, M., Oguz, B., Lin, J., Mehdad, Y., Yih, W.t., Chen, X.: How to train your dragon: Diverse augmentation towards generalizable dense retrieval. In: Findings of the Association for Computational Linguistics: EMNLP 2023. pp. 6385–6400 (2023)

    2023

  5. [13]

    In: The Twelfth International Conference on Learning Representations (2023)

    Lin, X.V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al.: Ra-dit: Retrieval-augmented dual instruc- tion tuning. In: The Twelfth International Conference on Learning Representations (2023)

    2023

  6. [14]

    In: Wang, M., Zitouni, I

    Liu, S.C., Wang, S., Chang, T., Lin, W., Hsiung, C.W., Hsieh, Y.C., Cheng, Y.P., Luo,S.H.,Zhang,J.:JarviX:ALLMnocodeplatformfortabulardataanalysisand Evaluating LLM Abilities to Understand Tabular EHR 15 optimization. In: Wang, M., Zitouni, I. (eds.) Proceedings of the 2023 Co...

    2023

  7. [15]

    In: AAAI Conference on Artificial Intelligence (2017), https://api.semanticscholar.org/CorpusID:7672408

    Liu, T., Wang, K., Sha, L., Chang, B., Sui, Z.: Table-to-text generation by structure-aware seq2seq learning. In: AAAI Conference on Artificial Intelligence (2017), https://api.semanticscholar.org/CorpusID:7672408

    2017

  8. [16]

    In: Demner-Fushman, D., Ananiadou, S., Thompson, P., Ondov, B

    Lovon-Melgarejo, J., Ben-Haddi, T., Di Scala, J., Moreno, J.G., Tamine, L.: Revis- iting the MIMIC-IV benchmark: Experiments using language models for electronic health records. In: Demner-Fushman, D., Ananiadou, S., Thompson, P., Ondov, B. (eds.) Proceedings of the First Work...

    2024

  9. [17]

    In: Korhonen, A., Traum, D., Màrquez, L

    Puduppully, R., Dong, L., Lapata, M.: Data-to-text generation with entity mod- eling. In: Korhonen, A., Traum, D., Màrquez, L. (eds.) Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 2023–2035. Association for Computational Linguisti...

    2023 doi

  10. [18]

    In: Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Pro- ceedings, Part I 42

    Rebuffel, C., Soulier, L., Scoutheeten, G., Gallinari, P.: A hierarchical model for data-to-text generation. In: Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Pro- ceedings, Part I 42. pp. 65–80. Spr...

    2020

  11. [19]

    In: Carpuat, M., de Marneffe, M.C., Meza Ruiz, I.V

    Rubin, O., Herzig, J., Berant, J.: Learning to retrieve prompts for in-context learning. In: Carpuat, M., de Marneffe, M.C., Meza Ruiz, I.V. (eds.) Proceed- ings of the 2022 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Lang...

    2022

  12. [20]

    In: NeurIPS 2023 Second Table Representation Learning Workshop (2023)

    Sarkar, S., Lausen, L.: Testing the limits of unified sequence to sequence llm pre- training on diverse table data tasks. In: NeurIPS 2023 Second Table Representation Learning Workshop (2023)

    2023

  13. [21]

    In: Table Representation Learning Workshop at NeurIPS 2023 (December 2023)

    Singha, A., Cambronero, J., Gulwani, S., Le, V., Parnin, C.: Tabular representa- tion, noisy operators, and impacts on table structure understanding tasks in llms. In: Table Representation Learning Workshop at NeurIPS 2023 (December 2023)

    2023

  14. [22]

    arXiv preprint arXiv:2304.13188 (2023)

    Slack, D., Singh, S.: Tablet: Learning from instructions for tabular data. arXiv preprint arXiv:2304.13188 (2023)

    2023 arXiv

  15. [23]

    Journal of Biomedical Informatics113, 103637 (2021).https: //doi.org/https://doi.org/10.1016/j.jbi.2020.103637

    Steinberg, E., Jung, K., Fries, J.A., Corbin, C.K., Pfohl, S.R., Shah, N.H.: Lan- guage models are an effective representation learning technique for electronic health record data. Journal of Biomedical Informatics113, 103637 (2021).https: //doi.org/https://doi.org/10.1016/j.j...

    2021

  16. [24]

    In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining

    Sui, Y., Zhou, M., Zhou, M., Han, S., Zhang, D.: Table meets llm: Can large language models understand structured table data? a benchmark and empirical study. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining. p. 645–654. WSDM ’24, Associat...

    2024

  17. [25]

    In: Findings of the Association for Computational Lin- guistics: EMNLP 2024 (2024) 16 Author

    Sui, Y., Zou, J., Zhou, M., He, X., Du, L., Han, S., Zhang, D.: Tap4llm: Ta- ble provider on sampling, augmenting, and packing semi-structured data for large language model reasoning. In: Findings of the Association for Computational Lin- guistics: EMNLP 2024 (2024) 16 Author

    2024

  18. [26]

    In: Proceedings of the ACM Web Con- ference 2022

    Trabelsi, M., Chen, Z., Zhang, S., Davison, B.D., Heflin, J.: Strubert: Structure- aware bert for table search and matching. In: Proceedings of the ACM Web Con- ference 2022. p. 442–451. WWW ’22, Association for Computing Machinery, New York, NY, USA (2022).https://doi.org/10....

    2022

  19. [27]

    In: Proceedings of The Web Conference 2020

    Wang, P., Shi, T., Reddy, C.K.: Text-to-sql generation for question answering on electronic medical records. In: Proceedings of The Web Conference 2020. p. 350–361. WWW ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3366423.3380120

    2020

  20. [28]

    In: Rogers, A., Boyd-Graber, J., Okazaki, N

    Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N.A., Khashabi, D., Hajishirzi, H.: Self-instruct: Aligning language models with self-generated instructions. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual MeetingoftheAssociationforComputati...

    2023 doi

  21. [29]

    In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

    Wang, Z., Dong, H., Jia, R., Li, J., Fu, Z., Han, S., Zhang, D.: Tuta: Tree-based transformers for generally structured table pre-training. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. p. 1780–1790. KDD ’21, Association for Computing ...

    2021

  22. [30]

    ArXiv abs/2206.07682 (2022)

    Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., hsin Chi, E.H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., Fedus, W.: Emergent abilities of large language models. ArXiv abs/2206.07682 (2022)

    2022 arXiv

  23. [31]

    In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

    Yang, H., Zhang, Y., Xu, J., Lu, H., Heng, P.A., Lam, W.: Unveiling the gener- alization power of fine-tuned large language models. In: Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie...

    2024

  24. [32]

    In: Proceedings of the 46th International ACM SIGIR Con- ference on Research and Development in Information Retrieval

    Ye, Y., Hui, B., Yang, M., Li, B., Huang, F., Li, Y.: Large language mod- els are versatile decomposers: Decomposing evidence and questions for table- based reasoning. In: Proceedings of the 46th International ACM SIGIR Con- ference on Research and Development in Information R...

    2023

  25. [33]

    In: The Eleventh International Conference on Learning Representations (2023)

    Yu, W., Iter, D., Wang, S., Xu, Y., Ju, M., Sanyal, S., Zhu, C., Zeng, M., Jiang, M.: Generate rather than retrieve: Large language models are strong context gen- erators. In: The Eleventh International Conference on Learning Representations (2023)

    2023

  26. [34]

    In: Goldberg, Y., Kozareva, Z., Zhang, Y

    Zhang, Y., Feng, S., Tan, C.: Active example selection for in-context learning. In: Goldberg, Y., Kozareva, Z., Zhang, Y. (eds.) Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pp. 9134–9148. Association for Computational Linguistics, Ab...

    2022 doi

  27. [35]

    In: Conference on Empirical Methods in Natural Language Processing (2023)

    Zhao, B., Ji, C., Zhang, Y., He, W., Wang, Y., Wang, Q., Feng, R., Zhang, X.: Large language models are complex table parsers. In: Conference on Empirical Methods in Natural Language Processing (2023)

    2023

  28. [36]

    In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Zhuang, S., Zhuang, H., Koopman, B., Zuccon, G.: A setwise approach for effective and highly efficient zero-shot ranking with large language models. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 38–47 (2024)

    2024

  29. [630]

    Association for Computational Linguistics, Singapore (Dec 2023).https:// doi.org/10.18653/v1/2023.emnlp-industry.59

    2023 doi

  30. [2671]

    https://doi.org/10.18653/v1/2022.naacl-main.191

    Association for Computational Linguistics, Seattle, United States (Jul 2022). https://doi.org/10.18653/v1/2022.naacl-main.191

    2022 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.