Pith. sign in

REVIEW 1 major objections 5 minor 38 references

This paper claims that modern text embedding models, which power semantic search and retrieval-augmented generation, do not reliably preserve numerical details in the text they encode: on a new 1,000-question financial benchmark, 13 models

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Across 13 embedding models and 18 numeric formats, retrieval accuracy on the new EmbedNum-1K benchmark averages 54%, just above chance, showing that embedding models largely fail to encode numeric detail.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The near-chance numeracy result is probably real, but the minimal-pair assumption and an impossible Adjusted R2 need fixing before I'd trust the fine-grained claims. the 1 major comments →

arxiv 2509.05691 v1 pith:4P6UIM7D submitted 2025-09-06 cs.CL cs.AI

Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models

Ningyuan Deng , Hanyu Duan , Yixuan Tang , Yi Yang This is my paper
classification cs.CL cs.AI
keywords text embedding modelsnumeracynumerical reasoningretrievalbenchmark evaluationfinancial NLP
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that modern text embedding models—the systems that turn sentences into vectors for search and retrieval—do not reliably preserve the numerical details of the text they encode. To test this, it introduces EmbedNum-1K, a set of 1,000 financial question-answer triples where the correct and incorrect answers are identical except for the number. Across 13 widely used embedding models, retrieval accuracy averages 0.54, barely above random guessing, and the best numeric format (decimal values like 0.8) reaches only 0.62 while four-digit integers fall to 0.50. The paper also finds that LLM-based embeddings outperform encoder-based ones by about five points, but that domain-specific financial fine-tuning does not improve numeracy, and that more context dilutes the numerical signal. If this holds, applications in finance and healthcare that rely on embeddings to compare numbers in text could be making errors at scale.

Core claim

The paper's central claim is that numeracy is a distinct capability that current text embedding models lack, and that this gap is masked by standard benchmarks. On EmbedNum-1K, a synthetic retrieval task constructed so that the correct answer and its distractor differ only in numeric value, the 13 evaluated models achieve an average accuracy of 0.54, with the original integer format at 0.519 and no format reaching 0.62. The authors interpret this as evidence that embeddings encode overall semantics while dropping the fine-grained numeric facts that determine whether an answer satisfies a question such as "Who owns over 15% of the company?" Additional experiments show the failure is not fixed

What carries the argument

EmbedNum-1K, a synthetic benchmark of 1,000 question-answer triples generated from financial text-to-SQL queries and augmented with seventeen numeric-format variants. For each triple, the correct answer (A+) and the incorrect answer (A−) are constructed to be identical in wording except for the number, so a model that retrieves A+ over A− must have encoded that a 20% stake satisfies "over 15%" while a 5% stake does not. The dataset isolates numeracy from semantics: cosine-similarity ranking on this task measures whether numeric detail is preserved in the embedding, and variations across formats, context length, and number length let the authors attribute failures to specific mechanisms such

Load-bearing premise

The benchmark assumes that each generated pair of candidate answers differs only in the numeric value, and the paper does not verify that this holds across all 1,000 triples; if the pairs differ in wording, specificity, or sentiment, the accuracy figures measure something other than numeracy.

What would settle it

Run a lexical-overlap baseline on EmbedNum-1K that scores each candidate answer by the length of the digit substring it shares with the question; if that baseline reaches or exceeds the reported 0.54 average accuracy, the claim that models are essentially at chance would be undermined because the task is solvable by surface pattern matching rather than numeracy.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Retrieval-augmented systems that rely on embedding similarity in financial or clinical domains will frequently return answers whose numeric detail is wrong, even when the overall meaning matches the query.
  • Scores on standard embedding benchmarks such as MTEB-style tasks should not be interpreted as evidence of numeric competence; a separate numeracy evaluation like EmbedNum-1K is necessary.
  • Adjusting tokenization so that numbers are kept intact, for instance avoiding comma-driven splits, may reduce some of the gap, but the paper's results on written-out numbers indicate this will not fully solve the problem.
  • Context-rich embeddings trade off semantic fidelity for fine detail: adding surrounding context weakens the numeric signal, so applications may need hybrid retrieval that explicitly checks numeric constraints outside the embedding comparison.
  • Domain-specific fine-tuning on financial text does not automatically confer numeracy, so future training should include explicit objectives that treat numbers as content-bearing tokens rather than ordinary words.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to add a token-overlap baseline: if a matcher that scores candidate answers by shared digit substrings with the question exceeds the reported 0.54 average on EmbedNum-1K, the 'barely above chance' headline would need qualification as a property of embedding models rather than of the task's surface difficulty.
  • The greater-than versus less-than performance asymmetry could be probed beyond embeddings: running the same 1,000 triples through an LLM in a direct question-answering setting would cheaply test whether the bias is a retrieval artifact or a deeper numerical-reasoning bias.
  • The significant-figure finding suggests a monotone numeracy difficulty curve; an extension would predict that all embedding models, regardless of architecture, will show the same ordering across formats, which could be checked with a larger format grid.
  • The probing result—encoder embeddings decode numbers more easily yet retrieve worse—implies that linear accessibility of a number in the embedding space is not sufficient for relational use; a possible test is to train a nonlinear probe or a probe that predicts the ordering relation rather than the magnitude.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper introduces EmbedNum-1K, a synthetic financial-domain dataset of 1,000 ⟨Q, A+, A−⟩ triples in which, by construction, the two answer candidates are claimed to differ only in their numeric values and the retrieval task is to choose the answer satisfying a greater-than/less-than condition. The authors evaluate 13 text embedding models (encoder-based, LLM-based, and commercial) on the original integer format and 17 numeric-format variants. The main empirical claim is that embedding models generally fail to capture numerical details: average retrieval accuracy is 0.54, only slightly above random, with LLM-based models outperforming encoder-based ones (0.56 vs. 0.51). Additional analyses examine format-dependent performance, greater-than/less-than asymmetry, written vs. digit number forms, token frequency, context effects, and probing-based predictability.

Significance. If the dataset construction is sound, the paper fills a real gap in embedding evaluation, because existing benchmarks such as MTEB and FinMTEB are largely insensitive to exact numeric content. The negative result across 13 diverse models, including recent LLM-based embedders, would be a useful caution for RAG and domain-specific applications. The paper also includes a deliberate design to vary numeric formats and to study context effects, which could inform future numeracy-aware embedding training. However, the central claim rests entirely on the assumption that A+ and A− differ only in numeric value and that lexical overlap does not already solve the task; neither is currently demonstrated. The probing analysis in §5.5 is also compromised by impossible Adjusted R² values.

major comments (1)
  1. [Table 3, §5.5] Table 3 reports Adjusted R² values greater than 1 (e.g., MiniLM-L6: 4.98, RoBERTa: 1.25, finance-embeddings: 1.99). Adjusted R² cannot exceed 1; such values indicate a computation or reporting error, likely in the regression or the adjusted-R² formula. This invalidates the quantitative ranking in §5.5 and the conclusion that encoder-based models yield higher Adjusted R² than LLM-based models. Recompute the probing results and report correct Adjusted R² values (or another metric). If the corrected ranking changes, revise the corresponding text and conclusion.
minor comments (5)
  1. [Section 2.2] The example 'Stock A roseby 2%' and 'Stock A fell by 2%' appears to contain a formatting typo ('Arose' should be 'A rose'), which slightly obscures the intended contrast.
  2. [Table 2 / §4.2] The claim that average accuracy is 'slightly above random guessing' is made without confidence intervals or significance tests. Given 1,000 samples, the standard error is about 1.6 percentage points under independence; adding per-model binomial CIs and a sign test or paired test across models would strengthen the interpretation.
  3. [§5.4, Figure 8] The 'context-reduced' version replaces rich financial questions with 'Which number is above 200?' and answers like '220' and '180'. This is a substantially different task (bare number comparison), so the comparison with the main dataset should be interpreted as an ablation of context length, not as evidence that the original dataset is easier. Consider adding a more controlled context-reduced variant that preserves the original sentence frame while dropping only the surrounding clauses.
  4. [§5.2, Figure 4] The written-form evaluation covers integers up to four digits (e.g., 'six thousand seven hundred and eighty-nine'), but it is unclear whether the written forms were generated automatically and whether they were checked for grammaticality. A short note on the generation and validation procedure would improve reproducibility.
  5. [Section 3, Step 4] The 17 variant formats are described in Table 1, but the mapping from each format to the exact transformation of numeric values is not formally defined. For example, the 'scaled down' and 'scaled up' rows preserve the significant digits but change the numerical value (e.g., 8 → 0.8). State explicitly how the numeric values are transformed and how the question threshold is adjusted (if at all) for these variants.

Circularity Check

0 steps flagged

No significant circularity: the benchmark's ground truth is external arithmetic and no fitted parameter is renamed as a prediction.

full rationale

EmbedNum-1K's correctness labels come from externally defined arithmetic conditions (e.g., SQL thresholds like 'pettm > 200'), and the reported accuracies are computed from third-party embedding models or commercial APIs; no parameter is fitted to the benchmark and no 'prediction' is derived from such a fit. The paper's self-citations (Fin-E5/FinMTEB, Duan et al., Liu et al.) are used as baselines, related work, or context, and the central claim—that retrieval accuracy on this controlled numeric task is only slightly above chance—does not depend on those citations. The unverified minimal-pair assumption in Step 3 is a genuine data-validity concern (the generated A+ and A− may differ in more than the number), but it is not a circularity: the paper does not define numeracy as retrieval accuracy, nor does it fit the observed accuracy into existence. A lexical-overlap baseline would strengthen the external interpretation of the 0.54 average, but its absence does not make the derivation circular. The probing analysis in Section 5.5 also uses an independent train/test split and is not used as evidence for the main retrieval claim. Overall, the derivation chain is self-contained with respect to circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper relies on standard assumptions about cosine similarity and the arithmetic validity of the benchmark. The most fragile assumption is that the LLM-generated pairs differ only numerically. No new physical or conceptual entities are introduced.

axioms (4)
  • domain assumption Cosine similarity is an appropriate measure of semantic closeness for the retrieval task.
    Used in Section 4.1 without justification; standard in embedding evaluation, but not the only possible similarity metric.
  • domain assumption The correctness of A+ over A- is determined purely by the numeric comparison (greater-than/less-than), decidable arithmetically.
    Section 3; this is the ground truth that makes the benchmark objective, but it depends on the pairs differing only in the number.
  • ad hoc to paper The LLM-generated candidate pairs (A+, A-) differ only in their numeric value, as stated in Section 3 Step 3.
    This is asserted without verification. If false, the benchmark measures something other than numeracy.
  • domain assumption Token l2-norm in the embedding lookup table correlates with training frequency, as cited from Li et al. 2020 and Yu et al. 2022.
    Invoked in Section 5.3 to stratify numbers into high/low frequency groups; the cited correlation is treated as a reliable proxy.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models." pith.science (2026). https://pith.science/paper/4P6UIM7D

@misc{pith2026250905691,
  author       = {Pith},
  title        = {Pith review of: Revealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4P6UIM7D}},
  note         = {Machine review of arXiv:2509.05691}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Text embedding models are widely used in natural language processing applications. However, their capability is often benchmarked on tasks that do not require understanding nuanced numerical information in text. As a result, it remains unclear whether current embedding models can precisely encode numerical content, such as numbers, into embeddings. This question is critical because embedding models are increasingly applied in domains where numbers matter, such as finance and healthcare. For example, Company X's market share grew by 2\% should be interpreted very differently from Company X's market share grew by 20\%, even though both indicate growth in market share. This study aims to examine whether text embedding models can capture such nuances. Using synthetic data in a financial context, we evaluate 13 widely used text embedding models and find that they generally struggle to capture numerical details accurately. Our further analyses provide deeper insights into embedding numeracy, informing future research to strengthen embedding model-based NLP systems with improved capacity for handling numerical content.

Figures

Figures reproduced from arXiv: 2509.05691 by Hanyu Duan, Ningyuan Deng, Yixuan Tang, Yi Yang.

Figure 9
Figure 9. Figure 9: Prompt used for question augmentation. Answer Generation Please generate a financial sentence that answers the following question: Format Requirements: The sentence must include only one specific number The description must be unambiguous. Only the sentence should be provided; no additional information is needed Example: Question: Which company's profit growth exceeded 15%? Answer: Tesla's profit in the se… view at source ↗
Figure 10
Figure 10. Figure 10: Prompt used for answer generation [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 16 canonical work pages · 1 internal anchor

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    DeepSeek-AI. 2024. http://arxiv.org/abs/2412.19437 Deepseek-v3 technical report

    Pith/arXiv arXiv 2024

  4. [4]

    Hanyu Duan, Yi Yang, and Kar Yan Tam. 2021. Learning numeracy: A simple yet effective number embedding approach using knowledge graph. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2597--2602

    work page 2021

  5. [5]

    Kenneth C. Enevoldsen, Isaac Chung, Imene Kerboua, M \'a rton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemi'nski, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystr m, Roman Sergeevich Solomatin, Omer Veysel cCaugatan, Akash Kund...

    arXiv 2025

  6. [6]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 6491--6501

    2024

  7. [7]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910. Association for Computational Linguistics

    work page 2021

  8. [8]

    Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, and Kewei Tu. 2020. Learning numeral embedding. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2586--2599

    work page 2020

  9. [9]

    Junseong Kim, Seolhwa Lee, Jihoon Kwon, Sangmo Gu, Yejin Kim, Minkyung Cho, Jy yong Sohn, and Chanyeol Choi. 2024. https://getlinq.com/blog/linq-embed-mistral/ Linq-embed-mistral:elevating text retrieval with improved gpt data through task-specific control and quality refinement . Linq AI Research Blog

    2024

  10. [10]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428

    Pith/arXiv arXiv 2024

  11. [11]

    Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.733 On the sentence embeddings from pre-trained language models . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9119--9130, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.733 2020

  12. [12]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    work page 2023

  13. [13]

    Jiaxin Liu, Yi Yang, and Kar Yan Tam. 2024. Beyond surface similarity: Detecting subtle semantic shifts in financial narratives. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2641--2652

    work page 2024

  14. [14]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://api.semanticscholar.org/CorpusID:198953378 Roberta: A robustly optimized bert pretraining approach . ArXiv, abs/1907.11692

    Pith/arXiv arXiv 2019

  15. [15]

    Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. 2024. https://www.salesforce.com/blog/sfr-embedding/ Sfr-embedding-mistral:enhance text retrieval with transfer learning . Salesforce AI Research Blog

    2024

  16. [16]

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

    Pith/arXiv arXiv 2013

  17. [17]

    Niklas Muennighoff. 2022. Sgpt: Gpt sentence embeddings for semantic search. arXiv preprint arXiv:2202.08904

    Pith/arXiv arXiv 2022

  18. [18]

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. https://doi.org/10.18653/v1/2023.eacl-main.148 MTEB : Massive text embedding benchmark . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014--2037, Dubrovnik, Croatia. Association for Computational Linguistics

    doi:10.18653/v1/2023.eacl-main.148 2023

  19. [19]

    Aakanksha Naik, Abhilasha Ravichander, Carolyn Rose, and Eduard Hovy. 2019. Exploring numeracy in word embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3374--3380

    work page 2019

  20. [20]

    OpenAI . 2025. Openai (august 25 version). https://api.openai.com/v1/embeddings

    work page 2025

  21. [21]

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

    2014

  22. [22]

    Jasivan Sivakumar and Nafise Sadat Moosavi. 2023. https://api.semanticscholar.org/CorpusID:258959201 Fermat: An alternative to accuracy for numerical reasoning . ArXiv, abs/2305.17491

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  23. [23]

    Jasivan Alex Sivakumar and Nafise Sadat Moosavi. 2025. How to leverage digit embeddings to represent numbers? In Proceedings of the 31st International Conference on Computational Linguistics, pages 7685--7697

    work page 2025

  24. [24]

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. https://api.semanticscholar.org/CorpusID:215827489 Mpnet: Masked and permuted pre-training for language understanding . ArXiv, abs/2004.09297

    Pith/arXiv arXiv 2020

  25. [25]

    Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika, and Lawrence Carin. 2020. Methods for numeracy-preserving word embeddings. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4742--4753

    work page 2020

  26. [26]

    Yixuan Tang and Yi Yang. 2025. https://api.semanticscholar.org/CorpusID:276409274 Finmteb: Finance massive text embedding benchmark . ArXiv, abs/2502.10990

    Pith/arXiv arXiv 2025

  27. [27]

    Nandan Thakur, Nils Reimers, Andreas Ruckl'e, Abhishek Srivastava, and Iryna Gurevych. 2021. https://api.semanticscholar.org/CorpusID:233296016 Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models . ArXiv, abs/2104.08663

    Pith/arXiv arXiv 2021

  28. [28]

    Avijit Thawani, Jay Pujara, and Filip Ilievski. 2021. Numeracy enhances the literacy of language models. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 6960--6967

    work page 2021

  29. [29]

    Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019. Do nlp models know numbers? probing numeracy in embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5307--5315

    work page 2019

  30. [30]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. https://api.semanticscholar.org/CorpusID:266693831 Improving text embeddings with large language models . ArXiv, abs/2401.00368

    Pith/arXiv arXiv 2023

  31. [31]

    Lionel Wong, Ayman Ali, Raymond M Xiong, Zejiang Shen, Yoon Kim, and Monica Agrawal. 2025. https://openreview.net/forum?id=LL39y0Tfxb Position: Retrieval-augmented systems can be dangerous medical communicators . In Forty-second International Conference on Machine Learning Position Paper Track

    work page 2025

  32. [32]

    Liyan Xu, Zhenlin Su, Mo Yu, Jiangnan Li, Fandong Meng, and Jie Zhou. 2025. Dense retrievers can fail on simple queries: Revealing the granularity dilemma of embeddings. arXiv preprint arXiv:2506.08592

    Pith/arXiv arXiv 2025

  33. [33]

    Antonio Jimeno Yepes, Yao You, Jan Milczek, Sebastian Laverde, and Renyu Li. 2024. Financial report chunking for effective retrieval augmented generation. arXiv preprint arXiv:2402.05131

    Pith/arXiv arXiv 2024

  34. [34]

    Sangwon Yu, Jongyoon Song, Heeseung Kim, Seongmin Lee, Woo-Jong Ryu, and Sungroh Yoon. 2022. https://doi.org/10.18653/v1/2022.acl-long.3 Rare tokens degenerate all tokens: Improving neural text generation via adaptive gradient gating for rare token embeddings . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol...

    work page doi:10.18653/v1/2022.acl-long.3 2022

  35. [35]

    Shaoyun Yu, Chanyuan Gu, Kexin Huang, and Ping Li. 2024. Predicting the next sentence (not word) in large language models: What model-brain alignment tells us about discourse comprehension. Science advances, 10(21):eadn7744

    work page 2024

  36. [36]

    Chao Zhang, Yuren Mao, Yijiang Fan, Yu Mi, Yunjun Gao, Lu Chen, Dongfang Lou, and Jinshu Lin. 2024. https://api.semanticscholar.org/CorpusID:267061057 Finsql: Model-agnostic llms-based text-to-sql framework for financial analysis . Companion of the 2024 International Conference on Management of Data

    work page 2024

  37. [37]

    Meishan Zhang, Xin Zhang, Xinping Zhao, Shouzheng Huang, Baotian Hu, and Min Zhang. 2025 a . On the role of pretrained language models in general-purpose text embeddings: A survey. arXiv preprint arXiv:2507.20783

    arXiv 2025

  38. [38]

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025 b . Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.