Pith. sign in

REVIEW 3 major objections 6 minor 58 references

Document-Level Tabular Numerical Cross-Checking: A Coarse-to-Fine Approach

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two-stage AI checks cross-table numbers with 90% F1 in 41 seconds

desk verdict Potentially useful coarse-to-fine system for numerical consistency checking, but the headline margin over AutoCheck rests on a baseline from a different corpus and a test-set-tuned threshold. read the letter →

arxiv 2506.13328 v1 pith:KLUG7JQW submitted 2025-06-16 cs.CL

Chaoxu Pang , Yixuan Cao , Ganbin Zhou , Hongwei Li , Ping Luo This is my paper · ORCID
classification cs.CL
keywords numericalcross-checkingtabularfact-checkingdocument-levelverificationsemanticmatchinglargelanguagemodelscontrastivelearningfinancialdisclosuredocumentscoarse-to-fineretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CoFiTCheck, a coarse-to-fine system for checking that the same numerical fact is stated consistently across the tables of a disclosure document. It claims the system reaches roughly 90% F1 on auditor's reports, IPO prospectuses, and annual reports, beating the previous end-to-end method by about 10 points, and that it processes one document in 40.8 seconds on four RTX 4090 GPUs. The significance is that document-level numerical verification, previously dominated by either heuristic filtering that caps recall or full pairwise LLM inference that is too slow, becomes accurate and fast enough for routine financial-disclosure review.

What carries the argument

The load-bearing object is the coarse-to-fine cascade. It has three named mechanisms: Contextualized Instructional Parallel Encoding (CIPE), which jointly encodes all numerical mentions of a table in one LLM forward pass with attention masking; a decoupled InfoNCE objective, which separately trains non-isolated mentions to cluster with their semantic equivalents and isolated mentions to stay apart; and Cross-Table Numerical Alignment Pretraining (CNAP), which reorders tables in documents so that tables sharing equal numbers appear adjacent and then continues next-token pretraining. Together they make candidate pruning high-recall and cheap, and make the classifier specific to numerical semantics in financial documents.

What would settle it

Take the three new benchmarks (auditor's reports, IPO prospectuses, annual reports), retrain AutoCheck on the same training split, and run it on the same test split with the same set-level precision, recall, and F1 metrics. If its F1 comes out materially higher than the numbers quoted in Table II, the claimed roughly 10-point improvement over prior work is not established.

Watch

Extended reading notes

Core claim

The central claim is that treating numerical cross-checking as two sequential stages, embedding-based candidate filtering followed by discriminative classification with a specialized LLM, solves both the combinatorial explosion of candidate pairs and the need to understand multi-faceted numerical semantics. The embedding stage encodes every numerical mention in a table in a single forward pass and prunes pairs by embedding similarity; the classification stage then reads the full table context of each surviving pair and decides semantic equivalence, with values masked so the model cannot cheat by matching raw numbers. The paper reports F1 of 87.0% on auditor's reports, 90.3% on IPO prospectuses, and 90.8% on annual reports with a 7B classifier, and shows that its cross-table numerical alignment pretraining adds gains without requiring manual annotation.

Load-bearing premise

The reported gains over AutoCheck assume the baseline numbers quoted from AutoCheck's original paper were computed on the same training and test splits and the same three document benchmarks; if those numbers came from a different evaluation, the claimed margin is not apples-to-apples.

Editorial extensions

If this is right

  • At 7B scale the system reaches F1 of 87.0% on auditor's reports, 90.3% on IPO prospectuses, and 90.8% on annual reports, so the method works across three document genres.
  • A full document takes 40.8 seconds on four RTX 4090 GPUs, versus 166.6 seconds reported for the prior system and many hours of manual review, making routine deployment plausible.
  • Removing the embedding stage would inflate per-document processing to roughly 1.5 days with the 0.5B classifier or 12.9 days with the 7B classifier, so the recall-preserving filter is what makes the classifier affordable.
  • CNAP raises F1 without labeled data and lets the 3B classifier match the 7B classifier without pretraining on two of the three document types.
  • A 0.5B task-finetuned classifier beats the strongest general-purpose and reasoning-specialized LLMs tested in zero-shot classification, by up to roughly 14 F1 points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because CIPE and CNAP are not tied to Chinese financial language, the same two-stage recipe should transfer to tables in scientific papers, regulatory filings, and technical reports; that transfer is an extrapolation, since the paper only evaluates Chinese financial disclosures.
  • The embedding similarity threshold (chosen as 0.5 here) makes the system tunable: a deployment that fears missing an inconsistency can lower it at the cost of more classifier calls, and one that must bound latency can raise it, with the paper's recall-versus-candidate-pairs curves giving the trade-off.
  • CNAP's table-relevance graph, built from equality of numeric mentions, is a reusable weak-supervision signal: any downstream task that benefits from placing tables that share values near each other in the language-model context could adopt the same maximum-path ordering.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces CoFiTCheck, a two-stage coarse-to-fine framework for document-level tabular numerical cross-checking. Stage 1 uses an LLM-based embedding filter (CIPE with a decoupled InfoNCE objective) to prune candidate numerical mention pairs; Stage 2 uses a specialized discriminative LLM (ClsLLM) to classify the remaining pairs. A cross-table numerical alignment pretraining (CNAP) method is also proposed to improve ClsLLM without manual annotations. Evaluation on new Chinese IPO prospectuses, auditor's reports, and annual reports reports F1 scores around 87-91% with a 7B ClsLLM and an average processing time of 40.8 seconds per document, purportedly surpassing the previous AutoCheck system by about 10 F1 points.

Significance. If the results hold, this would be a meaningful advance: CoFiTCheck appears to be the first LLM-based coarse-to-fine system that makes document-level numerical consistency checking both accurate and practical. The CIPE parallel-encoding idea, the decoupled InfoNCE objective for isolated mentions, and the CNAP weak-supervision pretraining are all well-motivated and potentially reusable beyond this specific task. The paper also contributes three annotated Chinese financial-disclosure benchmarks and a detailed efficiency analysis. However, the headline superiority over AutoCheck is currently undermined by an unclear baseline comparison, and the embedding threshold is selected on the test set, so the reported F1 numbers are not clean held-out results. The technical core is defensible, but the empirical claims need substantial verification before the paper can be accepted.

major comments (3)
  1. [IV-A.4, Table II] Table II reports AutoCheck's P/R/F1 "as presented in the original paper [12]" and also states "We use the same training and test splits," but the three corpora in this paper are newly constructed in Section IV-A.1; AutoCheck's original KDD 2020 evaluation could not have used these splits. The AutoCheck row for annual reports is empty, so the claimed 11.6- and 9.0-point F1 improvements are at best comparisons against a different dataset and annotation pipeline, not controlled comparisons on the new benchmarks. Please re-run AutoCheck's released pipeline on the exact same training/test splits (or clearly identify which numbers were recomputed and how), and provide the missing annual-report numbers. Without this, the abstract's claim of "surpassing previous methods by around 10 points" is unsupported.
  2. [IV-A.3, IV-E] The embedding similarity threshold t=0.5 is fixed in Section IV-A.3 based on the analysis in Section IV-E, and Section IV-E explicitly states: "Using 126 test set documents as benchmark, we vary the embedding similarity threshold from 0.1 to 0.9." Thus t is selected on the test set, so the end-to-end F1 scores in Table II are not clean held-out numbers and are likely optimistic. The threshold (and any hyperparameters read off these curves) should be selected on the validation split or via nested cross-validation, with the test split reserved for final evaluation.
  3. [IV-A.1, IV-B, Fig. 6] CNAP pretrains on 11,635 annual reports "crawled from a stock exchange website," while the annual-report benchmark is one of the three evaluation corpora collected by the authors. The paper does not state that the test (or validation) annual reports were excluded from the pretraining corpus. If any overlap exists, the CNAP gains shown in Fig. 6 and Table II would be confounded by data leakage. Please state explicitly how the pretraining documents were deduplicated against the training, validation, and test sets, or re-run CNAP with all benchmark documents removed from pretraining.
minor comments (6)
  1. [IV-A.4] There are typos such as "heuristc" and "capabilites" in the baselines subsection; please proofread.
  2. [Table II] The 7B CoFiTCheck row presents values without spacing (e.g., "87.086.987.092.8"), making the table hard to read; use proper column alignment or additional vertical lines.
  3. [IV-C] The 1k-sample evaluation of SOTA LLMs is a single random draw with no reported seed or confidence intervals; please report the sampling procedure and variability (e.g., multiple draws or bootstrap intervals).
  4. [IV-D] The runtime comparison with AutoCheck (166.6 seconds) appears to come from the original paper and may have been measured on different hardware; please add a caveat or, ideally, measure AutoCheck on the same four-GPU setup.
  5. [IV-D] The rows for "Heuristic-based Filtering" and "Removing Stage 1" are estimates based on average processing times, not measured runs; the table should clearly mark these rows as estimates rather than only mentioning it in a footnote.
  6. [III-D, Eq. (8)] The relevance score R(t_i,t_j) counts equal numerical mentions; if a table contains repeated identical values that are not semantically equivalent cross-table pairs, this could overestimate relevance. Please clarify how repeated values are handled in the equality count.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity in the coarse-to-fine pipeline; the main circular-adjacent issue is a test-set-fitted similarity threshold, plus an apples-to-apples concern about the AutoCheck baseline.

  1. fitted input called prediction [Section IV-A.3 (Implementations), Section IV-E (Analysis of Embedding-Based Filtering), Eq. (4)]
    "Using 126 test set documents as benchmark, we vary the embedding similarity threshold from 0.1 to 0.9, measuring both the remaining candidate pairs per document and the recall. ... During inference, we set the embedding similarity threshold to 0.5, which provides an effective balance between recall and efficiency (see Section IV-E for detailed analysis)."

    The threshold t appears directly in Eq. (4), P_cand = {(i,j)|cos(e_i,e_j) > t, ...}, so it controls which candidate pairs survive to the ClsLLM classifier and therefore influences the final F1. The paper selects t=0.5 by inspecting recall and candidate-pair counts on the same 126 test documents that are then used to report the overall F1 in Table II. This makes the reported operating point a test-set-fitted hyperparameter, so the headline F1 is partly a fitted quantity rather than a purely out-of-sample prediction. This is a minor leakage: the model weights themselves are trained on the training split, and the threshold is only one scalar, so it does not make the whole derivation circular.

full rationale

The core derivation is self-contained. EmbLLM is trained with the decoupled InfoNCE objective using manually annotated semantic-equivalence positives (Eqs. 5-6), ClsLLM is trained with cross-entropy on the same labels (Eq. 7), and the prompts mask all numerical values, so the classifier cannot use value equality as a shortcut for semantic equivalence. CNAP uses cross-table numerical equality only as a pretraining objective (Eq. 8 and Algorithm 1), not as the evaluation label; the test labels are human-annotated semantic equivalence, so the pretraining signal is a proxy, not the target label itself. The AutoCheck baseline in Table II is reported 'as presented in the original paper [12]' with an ambiguous 'same training and test splits' claim; this is a comparison-validity risk because the original AutoCheck paper has overlapping authors and no annual-report F1, but it is not an input-output equivalence in CoFiTCheck's derivation. The only circular-adjacent issue is the embedding-similarity threshold: Section IV-E uses the 126 test documents to examine the recall/candidate-pair trade-off, and Section IV-A.3 fixes t=0.5 with reference to that analysis, so the threshold is fitted on the test set before the same test set is used for the reported F1. This is a mild evaluation-protocol flaw, not a constructed equivalence, and it does not affect the validity of the staged architecture itself.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework's empirical gains rest primarily on the quality of the private annotations, the reliability of equality-based weak supervision, and the comparability of the AutoCheck baseline. The free parameters, especially the embedding threshold, are fitted on data that includes the test benchmark.

free parameters (4)
  • embedding similarity threshold t = 0.5
    Selected by varying the threshold on the 126-document test set benchmark (Section IV-E) to balance recall and candidate-pair count. This is a data-fitted hyperparameter controlling which pairs reach the classifier.
  • decoupled InfoNCE weights alpha1, alpha2 = 0.75, 0.25
    Weights for the non-isolated and isolated loss terms, hand-chosen in Section IV-A.3 with no sensitivity analysis reported.
  • contrastive temperature tau = 0.15
    Set in Section IV-A.3; not justified by a hyperparameter search or by theory.
  • surrounding context length = 500 characters
    The context string in Section III-A limits surrounding text to 500 characters. This hand-chosen budget may exclude relevant context for mentions in long sections.
assumptions (4)
  • domain assumption Numerical equality between mentions is a useful and sufficiently precise weak supervision signal for semantic equivalence.
    CNAP's pretraining signal comes entirely from equal values across pairs of tables. If equal values frequently appear for unrelated facts, the pretraining teaches a wrong association. The paper provides no precision analysis of this signal.
  • domain assumption The manually annotated positive pairs in the three datasets are complete and correct.
    All F1 computations use this gold set. Annotation is done with a pipeline similar to prior work [12], but no inter-annotator agreement or adjudication is reported.
  • domain assumption The 8:1:1 document-level split prevents information leakage between training and test documents.
    It is assumed that no company or underlying report appears in both train and test; the paper does not check for duplicate documents or shared content across the split.
  • domain assumption Markdown linearization of tables preserves all information needed to judge semantic equivalence of numerical mentions.
    Complex layouts, merged cells, and visual structure may be lost in linearization; the paper does not analyze failure cases attributable to this representation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Document-Level Tabular Numerical Cross-Checking: A Coarse-to-Fine Approach." pith.science (2026). https://pith.science/paper/KLUG7JQW

@misc{pith2026250613328,
  author       = {Pith},
  title        = {Pith review of: Document-Level Tabular Numerical Cross-Checking: A Coarse-to-Fine Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KLUG7JQW}},
  note         = {Machine review of arXiv:2506.13328}
}
read the original abstract

Numerical consistency across tables in disclosure documents is critical for ensuring accuracy, maintaining credibility, and avoiding reputational and economic risks. Automated tabular numerical cross-checking presents two significant challenges: (C1) managing the combinatorial explosion of candidate instances at the document level and (C2) comprehending multi-faceted numerical semantics. Previous research typically depends on heuristic-based filtering or simplified context extraction, often struggling to balance performance and efficiency. Recently, large language models (LLMs) have demonstrated remarkable contextual understanding capabilities that helps address C2 at the instance level, yet they remain hampered by computational inefficiency (C1) and limited domain expertise. This paper introduces CoFiTCheck, a novel LLM-based coarse-to-fine framework that addresses these challenges through two sequential stages: embedding-based filtering and discriminative classification. The embedding-based filtering stage introduces an instructional parallel encoding method to efficiently represent all numerical mentions in a table with LLMs, as well as a decoupled InfoNCE objective to mitigate the isolated mention problem. The discriminative classification stage employs a specialized LLM for fine-grained analysis of the remaining candidate pairs. This stage is further enhanced by our crosstable numerical alignment pretraining paradigm, which leverages weak supervision from cross-table numerical equality relationships to enrich task-specific priors without requiring manual annotation. Comprehensive evaluation across three types of real-world disclosure documents demonstrates that CoFiTCheck significantly outperforms previous methods while maintaining practical efficiency.

Figures

Figures reproduced from arXiv: 2506.13328 by the authors.

Figure 1
Figure 1. Three tables from the BHP Annual Report 2024. Numerical mentions [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The CoFiTCheck Framework. CoFiTCheck operates in two stages. First, for each input document, embedding-based filtering efficiently narrows down [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The workflow of Cross-Table Numerical Alignment Pretraining (CNAP). CNAP first builds a table relevance graph from a collection of tables in the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: F1 scores of CoFiTCheck across three document types with varying ClsLLM sizes. CoFiTCheck w. CNAP generally outperforms CoFiTCheck, with [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparison of EmbLLM performance under different settings: (a) comparison between our decoupled InfoNCE objective and the standard InfoNCE [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Overall F1 scores of various pretraining strategies (1.5B parameters) [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 40 canonical work pages

  1. [12]

    Cracking tabular pre- sentation diversity for automatic cross-checking over numerical facts,

    H. Li, Q. Yang, Y . Cao, J. Yao, and P. Luo, “Cracking tabular pre- sentation diversity for automatic cross-checking over numerical facts,” Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020

    work page 2020

  2. [1]

    Fake news detection on social media: A data mining perspective,

    K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, “Fake news detection on social media: A data mining perspective,”ACM SIGKDD explorations newsletter, vol. 19, no. 1, pp. 22–36, 2017

    work page 2017

  3. [2]

    Sciclops: Detecting and con- textualizing scientific claims for assisting manual fact-checking,

    P. Smeros, C. Castillo, and K. Aberer, “Sciclops: Detecting and con- textualizing scientific claims for assisting manual fact-checking,” in Proceedings of the 30th ACM international conference on information & knowledge management, 2021, pp. 1692–1702

    work page 2021

  4. [3]

    Tabfact: A large-scale dataset for table-based fact verification,

    W. Chen, H. Wang, J. Chen, Y . Zhang, H. Wang, S. Li, X. Zhou, and W. Y . Wang, “Tabfact: A large-scale dataset for table-based fact verification,” inInternational Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia, April 2020

    work page 2020

  5. [4]

    FEVEROUS: Fact extraction and VERification over unstructured and structured in- formation,

    R. Aly, Z. Guo, M. S. Schlichtkrull, J. Thorne, A. Vlachos, C. Christodoulopoulos, O. Cocarascu, and A. Mittal, “FEVEROUS: Fact extraction and VERification over unstructured and structured in- formation,” inThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021

    work page 2021

  6. [5]

    Chain-of-table: Evolving tables in the reasoning chain for table understanding,

    Z. Wang, H. Zhang, C.-L. Li, J. M. Eisenschlos, V . Perot, Z. Wang, L. Miculicich, Y . Fujii, J. Shang, C.-Y . Leeet al., “Chain-of-table: Evolving tables in the reasoning chain for table understanding,” inICLR, 2024

    work page 2024

  7. [6]

    Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster,

    N. Hassan, F. Arslan, C. Li, and M. Tremayne, “Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster,” inProceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 2017, pp. 1803–1812

    work page 2017

  8. [7]

    Towards automated fact-checking of real-world claims: Exploring task formulation and assessment with llms,

    P. Sahitaj, I. Maab, J. Yamagishi, J. Kolanowski, S. M ¨oller, and V . Schmitt, “Towards automated fact-checking of real-world claims: Exploring task formulation and assessment with llms,”arXiv preprint arXiv:2502.08909, 2025

    arXiv 2025

Show all 58 references
  1. [8]

    Towards automatic numerical cross- checking: Extracting formulas from text,

    Y . Cao, H. Li, P. Luo, and J. Yao, “Towards automatic numerical cross- checking: Extracting formulas from text,” inProceedings of the 2018 World Wide Web Conference, 2018, pp. 1795–1804

    2018

  2. [9]

    [On- line]

    Wrong numbers: The risks of inaccurate financial statements. [On- line]. Available: https://www.rmmagazine.com/articles/article/2019/09/ 03/-Wrong-Numbers-The-Risks-of-Inaccurate-Financial-Statements-

    2019

  3. [10]

    A survey on automated fact-checking,

    Z. Guo, M. Schlichtkrull, and A. Vlachos, “A survey on automated fact-checking,”Transactions of the Association for Computational Lin- guistics, vol. 10, pp. 178–206, 2022

    2022

  4. [11]

    Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning,

    Y . Ye, B. Hui, M. Yang, B. Li, F. Huang, and Y . Li, “Large language models are versatile decomposers: Decomposing evidence and questions for table-based reasoning,” inProceedings of the 46th international ACM SIGIR conference on research and development in information retrie...

    2023

  5. [13]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” 2023. [Online]. Available: https: //api.semanticscholar.org/CorpusID:257532815

    2023

  6. [14]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2. 5 technical report,”arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  7. [15]

    Large language models and sentiment analysis in financial markets: A review, datasets, and case study,

    C. Liu, A. Arulappan, R. K. Naha, A. Mahanti, J. Kamruzzaman, and I.-H. Ra, “Large language models and sentiment analysis in financial markets: A review, datasets, and case study,”IEEE Access, vol. 12, pp. 134 041–134 061, 2024

    2024

  8. [16]

    Structgpt: A general framework for large language model to reason over structured data,

    J. Jiang, K. Zhou, Z. Dong, K. Ye, W. X. Zhao, and J.-R. Wen, “Structgpt: A general framework for large language model to reason over structured data,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 9237–9251

    2023

  9. [17]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeliet al., “On the opportunities and risks of foundation models,”arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  10. [18]

    Towards understanding factual knowledge of large language models,

    X. Hu, J. Chen, X. Li, Y . Guo, L. Wen, P. S. Yu, and Z. Guo, “Towards understanding factual knowledge of large language models,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  11. [19]

    Hello gpt-4o,

    OpenAI, “Hello gpt-4o,” 2024. [Online]. Available: https://openai.com/ index/hello-gpt-4o/

    2024

  12. [20]

    Introducing openai o1,

    “Introducing openai o1,” 2024. [Online]. Available: https://openai.com/ o1/

    2024

  13. [21]

    Qwen2.5: A party of foundation models,

    Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5/

    2024

  14. [22]

    Deepseek-v3 technical report,

    DeepSeek-AI, “Deepseek-v3 technical report,” 2024

    2024

  15. [23]

    Large language models as foundations for next-gen dense retrieval: A comprehensive empirical assessment,

    K. Luo, M. Qin, Z. Liu, S. Xiao, J. Zhao, and K. Liu, “Large language models as foundations for next-gen dense retrieval: A comprehensive empirical assessment,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 1354– 1365

    2024

  16. [24]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolog...

    2019

  17. [25]

    Large dual encoders are generalizable retrievers,

    J. Ni, C. Qu, J. Lu, Z. Dai, G. H. Abrego, J. Ma, V . Zhao, Y . Luan, K. Hall, M.-W. Changet al., “Large dual encoders are generalizable retrievers,” inProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 9844–9855

    2022

  18. [26]

    Llm2vec: Large language models are secretly powerful text encoders,

    P. BehnamGhader, V . Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, and S. Reddy, “Llm2vec: Large language models are secretly powerful text encoders,”arXiv preprint arXiv:2404.05961, 2024

    2024 arXiv

  19. [27]

    Promptreps: Prompting large language models to generate dense and sparse representations for zero-shot document retrieval,

    S. Zhuang, X. Ma, B. Koopman, J. Lin, and G. Zuccon, “Promptreps: Prompting large language models to generate dense and sparse representations for zero-shot document retrieval,”arXiv preprint arXiv:2404.18424, 2024

    2024 arXiv

  20. [28]

    Fine-tuning llama for multi-stage text retrieval,

    X. Ma, L. Wang, N. Yang, F. Wei, and J. Lin, “Fine-tuning llama for multi-stage text retrieval,” inProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 2421–2425

    2024

  21. [29]

    Generative representational instruction tuning,

    N. Muennighoff, S. Hongjin, L. Wang, N. Yang, F. Wei, T. Yu, A. Singh, and D. Kiela, “Generative representational instruction tuning,” inICLR 2024 Workshop: How Far Are We From AGI, 2024

    2024

  22. [30]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stoneet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2023 arXiv

  23. [31]

    Investigating table-to-text generation capabilities of large language models in real- world information seeking scenarios,

    Y . Zhao, H. Zhang, S. Si, L. Nan, X. Tang, and A. Cohan, “Investigating table-to-text generation capabilities of large language models in real- world information seeking scenarios,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Indu...

    2023

  24. [32]

    Uncovering limitations of large language models in information seeking from tables,

    C. Pang, Y . Cao, C. Yang, and P. Luo, “Uncovering limitations of large language models in information seeking from tables,” inFindings of the Association for Computational Linguistics ACL 2024, 2024, pp. 1388– 1409

    2024

  25. [33]

    Exploring the numerical reasoning capabilities of language models: A comprehensive analysis on tabular data,

    M. Akhtar, A. Shankarampeta, V . Gupta, A. Patil, O. Cocarascu, and E. Simperl, “Exploring the numerical reasoning capabilities of language models: A comprehensive analysis on tabular data,” inFindings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 15 ...

    2023

  26. [34]

    Introducing chatgpt,

    O. Team, “Introducing chatgpt,” November 2022. [Online]. Available: https://openai.com/index/chatgpt/

    2022

  27. [35]

    Tablellama: Towards open large generalist models for tables,

    T. Zhang, X. Yue, Y . Li, and H. Sun, “Tablellama: Towards open large generalist models for tables,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024, ...

    2024

  28. [36]

    Table meets llm: Can large language models understand structured table data? a benchmark and empirical study,

    Y . Sui, M. Zhou, M. Zhou, S. Han, and D. Zhang, “Table meets llm: Can large language models understand structured table data? a benchmark and empirical study,” inProceedings of the 17th ACM International Conference on Web Search and Data Mining, 2024, pp. 645–654

    2024

  29. [37]

    Multimodal graph causal embedding for multimedia-based recommendation,

    S. Li, F. Xue, K. Liu, D. Guo, and R. Hong, “Multimodal graph causal embedding for multimedia-based recommendation,”IEEE Transactions on Knowledge and Data Engineering, 2024

    2024

  30. [38]

    Graph diffusion-based representation learning for sequential recommendation,

    Z. Wang, Y . Zhu, C. Wang, X. Zhao, B. Li, J. Yu, and F. Tang, “Graph diffusion-based representation learning for sequential recommendation,” IEEE Transactions on Knowledge and Data Engineering, 2024

    2024

  31. [39]

    Dual variational graph reconstruction learning for social recommendation,

    Y . Zhang, Y . Zhang, Y . Zhao, S. Deng, and Y . Yang, “Dual variational graph reconstruction learning for social recommendation,”IEEE Trans- actions on Knowledge and Data Engineering, 2024

    2024

  32. [40]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Training language models to follow instructions with human feedback,”Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

    2022

  33. [41]

    Efficient and robust approxi- mate nearest neighbor search using hierarchical navigable small world graphs,

    Y . A. Malkov and D. A. Yashunin, “Efficient and robust approxi- mate nearest neighbor search using hierarchical navigable small world graphs,”IEEE transactions on pattern analysis and machine intelligence, vol. 42, no. 4, pp. 824–836, 2018

    2018

  34. [42]

    The faiss library,

    M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazar ´e, M. Lomeli, L. Hosseini, and H. J´egou, “The faiss library,”arXiv preprint arXiv:2401.08281, 2024

    2024 arXiv

  35. [43]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inInternational conference on machine learning. PmLR, 2020, pp. 1597–1607

    2020

  36. [44]

    Feature-aware contrastive learning with bidirectional transformers for sequential recommendation,

    H. Du, H. Yuan, P. Zhao, D. Wang, V . S. Sheng, Y . Liu, G. Liu, and L. Zhao, “Feature-aware contrastive learning with bidirectional transformers for sequential recommendation,”IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 12, pp. 8192–8205, 2023

    2023

  37. [45]

    Improving language understanding by generative pre-training,

    A. Radford, K. Narasimhan, T. Salimans, I. Sutskeveret al., “Improving language understanding by generative pre-training,” 2018

    2018

  38. [46]

    The traveling-salesman problem,

    M. M. Flood, “The traveling-salesman problem,”Operations research, vol. 4, no. 1, pp. 61–75, 1956

    1956

  39. [47]

    In-context pretraining: Language modeling beyond document boundaries,

    W. Shi, S. Min, M. Lomeli, C. Zhou, M. Li, X. V . Lin, N. A. Smith, L. Zettlemoyer, W. Yih, and M. Lewis, “In-context pretraining: Language modeling beyond document boundaries,” inThe Twelfth Inter- national Conference on Learning Representations, ICLR 2024, Vienna, Austria, M...

    2024

  40. [48]

    Transformers: State-of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State-of-the-art natura...

    2020

  41. [49]

    Zero: memory optimizations toward training trillion parameter models,

    S. Rajbhandari, J. Rasley, O. Ruwase, and Y . He, “Zero: memory optimizations toward training trillion parameter models,” inProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2020, Virtual Event / Atlanta, Georgia, ...

    2020

  42. [50]

    Efficient memory management for large language model serving with pagedattention,

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with pagedattention,”Proceedings of the 29th Symposium on Operating Systems Principles, 2023

    2023

  43. [51]

    Guideline learning for in- context information extraction,

    C. Pang, Y . Cao, Q. Ding, and P. Luo, “Guideline learning for in- context information extraction,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    2023

  44. [52]

    Breaking the bank with chatgpt: Few-shot text classification for finance,

    L. Loukas, I. Stogiannidis, P. Malakasiotis, and S. Vassos, “Breaking the bank with chatgpt: Few-shot text classification for finance,”CoRR, vol. abs/2308.14634, 2023

    2023 arXiv

  45. [53]

    Gpt-4o mini: advancing cost-efficient intelli- gence,

    OpenAI, “Gpt-4o mini: advancing cost-efficient intelli- gence,” 2024. [Online]. Available: https://openai.com/index/ gpt-4o-mini-advancing-cost-efficient-intelligence/

    2024

  46. [54]

    Openai o3-mini,

    “Openai o3-mini,” 2025. [Online]. Available: https://openai.com/index/ openai-o3-mini/

    2025

  47. [55]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    DeepSeek-AI, “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025

    2025

  48. [56]

    Unsupervised dense information retrieval with contrastive learning,

    G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave, “Unsupervised dense information retrieval with contrastive learning,”arXiv preprint arXiv:2112.09118, 2021

    2021 arXiv

  49. [57]

    The llama 3 herd of models,

    L. Team, “The llama 3 herd of models,” 2024

    2024

  50. [58]

    An integrated data processing framework for pretraining foundation models,

    Y . Sun, F. Wang, Y . Zhu, W. X. Zhao, and J. Mao, “An integrated data processing framework for pretraining foundation models,” inProceed- ings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, p. 2713–2718. Chaoxu Pangi...

    2012

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.