Pith. sign in

REVIEW 5 major objections 5 minor 34 references

DateLogicQA: Benchmarking Temporal Biases in Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DateLogicQA's human evaluation of twelve LLMs finds temporal reasoning failures split into two distinct biases: misreading the date and misreasoning about a correctly read date.

desk verdict A useful small benchmark idea undermined by an undefined core metric and a bias taxonomy inferred from final outputs rather than tokenizer evidence. read the letter →

arxiv 2412.13377 v2 pith:WH2RK4HE submitted 2024-12-17 cs.CL cs.AI

Gagan Bhatia , MingZe Tang , Cristina Mahanta , Madiha Kazi This is my paper · ORCID
classification cs.CLcs.AI
keywords DateLogicQAtemporalreasoningLLMbiastokenizationrepresentation-levellogical-levelSemanticIntegrityMetricformats
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DateLogicQA is a 190-question benchmark for temporal reasoning in large language models, covering seven date formats, past/present/future contexts, and four reasoning types. The paper claims that LLM temporal errors split into two distinct categories: Representation-Level Bias, where the model mis-encodes the date itself, and Logical-Level Bias, where the date is read correctly but the reasoning about it goes wrong. A human evaluation of twelve models, supported by a Semantic Integrity Metric for tokenization quality, finds both biases across models. The result matters because fixing each failure mode requires different remedies, from tokenization and pretraining data to post-training reasoning methods.

What carries the argument

The key mechanism is the four-colour human evaluation rubric combined with the Semantic Integrity Metric. The rubric labels each model response as incorrect, representation-level biased (wrong date, compensated reasoning), logical-level biased (right date, wrong reasoning), or correct. The Semantic Integrity Metric scores how faithfully a tokenizer preserves date components such as day, month, and year; the paper uses it to connect tokenization quality to performance. Together these tools attribute each error to either the encoding stage or the reasoning stage of the model.

What would settle it

Run each model's tokenizer on the benchmark's date strings and inspect token boundaries and embeddings for a sample of wrong-date answers. If a substantial fraction of wrong-date outputs come from inputs whose date tokens are correctly segmented and embedded, the Representation-Level Bias category is not supported, and the two bias types collapse into a single output-error category.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that LLMs exhibit two separable temporal biases rather than one undifferentiated error rate. Representation-Level Bias shows up when a model outputs a faulty date, for example 'March 273, 2040' for a Julian-format input, yet may still land on the right answer through compensating reasoning. Logical-Level Bias appears when the model reproduces the date tokens correctly but then misapplies arithmetic or world knowledge, as when a model computes October 2040 instead of July 1454 for a numerical date addition. The four-colour human rubric, which classifies each response as incorrect, representation-biased, logical-biased, or correct, is what separates these categories. Aggregate results show Julian calendar formats are hardest, numerical questions hardest, and future dates easier than historical or present dates.

Load-bearing premise

The paper assumes that a wrong date in the output proves the date was tokenized or embedded incorrectly, and a right date with a wrong answer proves reasoning failed; this attribution is inferred from final answers without direct observation of tokenizer internals.

Editorial extensions

If this is right

  • Improving tokenization and pretraining coverage for rare date formats should shrink Representation-Level Bias.
  • Post-training approaches such as Direct Preference Optimization and chain-of-thought prompting can be benchmarked specifically against Logical-Level Bias on DateLogicQA.
  • The format and context axes provide model developers with a checklist of which date notations and temporal periods need more training data.
  • Future dates being easier than historical or present ones suggests pretraining data coverage, not reasoning capacity, is the main bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The two-bias split implies that a single accuracy score hides which stage fails; two models with identical accuracy may need opposite fixes.
  • A direct test of the representation-level claim is to inspect tokenizer outputs for wrong-date answers: if date tokens are intact in those cases, the bias is not truly at the representation level.
  • The same rubric could transfer to other structured entities such as times, quantities, or identifiers to ask whether representation-level versus logical-level biases are a general phenomenon.
  • Because the benchmark is small and human-scored, the reported bias rates likely carry wide uncertainty; a larger validated version would make the comparisons between formats and models more precise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces DateLogicQA, a human-curated benchmark of 190 questions spanning seven date formats, three temporal contexts, and four reasoning types, and reports a human evaluation of 12 LLMs. The authors propose the Semantic Integrity Metric (SI) to assess tokenization quality and claim to identify two distinct temporal biases: Representation-Level Bias (faulty tokenization/embeddings of dates) and Logical-Level Bias (correct date tokens but flawed reasoning). The results are presented as accuracy breakdowns by date format, temporal context, and question type, followed by qualitative discussion of mitigation strategies. The central claim is that LLMs exhibit separable representation-level and logical-level temporal reasoning failures.

Significance. If the central claim were established, the paper would offer a useful diagnostic lens for temporal reasoning in LLMs, and the two-level bias taxonomy could guide targeted interventions in tokenization and post-training. The use of human evaluation with a reported Cohen's kappa of 0.80, the coverage of uncommon Julian and compact date formats, and the breadth of models tested are positive features. However, the paper's key analytical tool, the Semantic Integrity Metric, is never formally defined, and the bias categories are inferred from final answer text rather than from direct tokenization evidence. The benchmark itself is not released. As presented, the paper's main claims are not verifiable, and the proposed representation-versus-logic distinction is not supported by the reported methodology.

major comments (5)
  1. [Abstract; Section 4.1; Appendix Table 4; Figure 5] The Semantic Integrity Metric (SI) is never defined. The abstract and conclusion name it as the measure of tokenization quality, and Table 4 lists SI values (1.00, 0.66, 0.60, 0.45, 0.40) together with tokenized outputs, while Figure 5 plots an unnamed correlation between 'semantic integrity score' and token count. No formula, computational procedure, axis labels, or coefficient are provided anywhere. Because SI is the basis for the representation-level bias claim, this missing definition is a load-bearing gap that makes the central analysis impossible to evaluate or reproduce.
  2. [Section 4.1 (rubric definitions, Light Orange and Light Teal)] The assignment of responses to Representation-Level Bias versus Logical-Level Bias is made from the final answer text: Light Orange is defined as 'the model tokenizes dates inaccurately but reaches the correct answer through logical reasoning,' and Light Teal as 'the model tokenizes correctly but misapplies logic.' For API models, tokenization is not observable, so a wrong date string in the output could stem from decoding, sampling, prompting, or answer formatting rather than from suboptimal embeddings, while a correct date in the output does not verify that the model tokenized or interpreted the date correctly internally. The two bias categories are therefore not separable from final-answer content, and the headline claim of distinct representation-level and logical-level biases is not supported by the reported evidence.
  3. [Table 3, Factual row] The ground-truth answer for the question 'Which of the following famous people died on 23041616?' is listed as 'William Shakespeare OR Miguel de Cervantes.' This is an ambiguous label: both individuals died in April 1616 depending on the calendar convention, and the table shows models receiving credit for either B or D. A benchmark question with an inclusive ground truth of this kind cannot serve as a reliable gold label for measuring tokenization or reasoning quality, and it also compromises the human rubric labels derived from such answers.
  4. [Section 3; Section 5] The DateLogicQA dataset is not released or linked, and only five example questions are shown in Table 1. There is no stated availability plan, and the paper does not report the number of questions per date format, temporal context, or question type. This makes the percentage breakdowns in Section 5 uninterpretable in terms of sample sizes and prevents independent verification of the benchmark's composition and the reported model rankings.
  5. [Figure 5] The correlation plot between 'semantic integrity score' and token count is presented without a correlation coefficient, confidence interval, or axis descriptions. If the SI values are computed from the same tokenized outputs used by the human rubric in Section 4.1, the claimed relationship would be circular; because the metric is undefined, the reader cannot rule out this possibility.
minor comments (5)
  1. [Table 4 caption] The caption 'Generated by Spread-LaTeX' appears to be an artifact of the LaTeX table-generation tool rather than a meaningful scientific caption and should be replaced or removed.
  2. [Section 5.1] The sentence 'These results evidence two critical areas where LLMs shine and their struggles' is vague; the text should state on which formats, contexts, or question types the models specifically succeed or fail.
  3. [References] Several references are duplicated or appear under two different keys: Su et al. (2024a) and (2024b), Tan et al. (2023a) and (2023b), and Xiong et al. (2024a) and (2024b) cite the same works with different labels.
  4. [Section 5.1, human evaluation] The paper reports four annotators and Cohen's kappa of 0.80 but does not describe the annotation interface, the exact instructions given beyond the color rubric, or the procedure for resolving disagreements; additional detail would strengthen the reliability claim.
  5. [Section 4.1] The definitions of the four color categories are given only informally in the text; a formal decision tree or pseudocode describing how a response is assigned to Dark Orange, Light Orange, Light Teal, or Dark Teal would improve reproducibility.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: the two-bias taxonomy is defined by the annotation rubric and then reported as an empirical finding; the undefined Semantic Integrity Metric is a separate completeness gap.

  1. self definitional [Section 4.1 (Human-Led Temporal Bias Assessment) and Section 5.1 (Temporal Reasoning Analysis)]
    "Light Orange ( ) reflects Representation-Level Temporal Bias, where the model tokenizes dates inaccurately but reaches the correct answer through logical reasoning. Light Teal ( ) signifies Logical-Level Temporal Bias, where the model tokenizes correctly but misapplies logic due to misattributing events or calculation errors."

    The two headline biases are not derived from independent observations; they are the Light Orange and Light Teal cells of the annotation rubric. Section 5.1 then reports 'These results evidence two critical areas' using the counts of those cells. The existence and definition of Representation-Level Bias is identical to the rule an annotator applies when seeing a wrong date with correct reasoning, and Logical-Level Bias is identical to seeing a correct date with faulty reasoning. The causal mechanism named in the abstract ('arising from suboptimal embeddings') is imported into the label rather than measured: no tokenizer or embedding-level evidence is presented, and for API models no such evidence is obtainable.

full rationale

The paper contains no self-citations, imported uniqueness theorems, or fitted parameters renamed as predictions, so the usual load-bearing circularity patterns are absent. The one definitional reduction is the bias taxonomy: the 'identification' of Representation-Level and Logical-Level Bias restates the Light Orange/Light Teal rubric in Section 4.1 rather than testing a separable mechanism, and the abstract's causal attribution to 'suboptimal embeddings' is not backed by any tokenizer-level measurement. This makes the headline claim partially circular. The Semantic Integrity Metric is claimed in the abstract and conclusion and plotted in Figure 5 but has no formula or definition in the main text; the appendix's SI values (Table 4) cannot be verified. That is an omitted-definition and reproducibility problem, not a circular step. The dataset construction, human-evaluation protocol with reported Cohen's kappa of 0.80, and performance comparisons are independent contributions, so the circularity is partial rather than total.

Assumptions & free parameters 0 free parameters · 3 assumptions · 3 invented entities

The ledger shows three author-defined constructs with no independent definitions: the SI metric, and the two bias types. The central empirical analysis depends on these constructs, and the SI metric is effectively a black-box number.

assumptions (3)
  • domain assumption Human judges can correctly classify model outputs into the four rubric categories, including whether a wrong date implies faulty tokenization.
    Section 4.1 introduces the rubric and Section 5.1 reports kappa 0.80, but the path from output to tokenization inference is assumed.
  • ad hoc to paper The Semantic Integrity Metric measures tokenization quality as defined by the authors, even though its formula is never disclosed.
    Appendix Table 4 lists SI scores; Figure 5 plots them against token count, but no equation or algorithm is given.
  • standard math Cohen's kappa is an appropriate agreement measure for this ordinal rubric.
    Section 5.1 uses Cohen's kappa as the inter-annotator statistic.
invented entities (3)
  • Semantic Integrity Metric (SI)
    purpose: Quantifies tokenization quality of a date string by a model's tokenizer.
    Never defined in text; only tabulated values and a correlation plot are shown, so no independent verification is possible.
  • Representation-Level Temporal Bias
    purpose: Label for cases where the model outputs a wrong date but reaches the correct answer via reasoning.
    Defined behaviorally in Section 4.1, but the claim that the date was 'tokenized inaccurately' is inferred from output, not observed.
  • Logical-Level Temporal Bias
    purpose: Label for cases where the model outputs the correct date but applies wrong logic.
    Defined behaviorally in Section 4.1; whether the model 'tokenized correctly' is inferred from the inclusion of the right date in the output.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DateLogicQA: Benchmarking Temporal Biases in Large Language Models." pith.science (2026). https://pith.science/paper/WH2RK4HE

@misc{pith2026241213377,
  author       = {Pith},
  title        = {Pith review of: DateLogicQA: Benchmarking Temporal Biases in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WH2RK4HE}},
  note         = {Machine review of arXiv:2412.13377}
}
read the original abstract

This paper introduces DateLogicQA, a benchmark with 190 questions covering diverse date formats, temporal contexts, and reasoning types. We propose the Semantic Integrity Metric to assess tokenization quality and analyse two biases: Representation-Level Bias, affecting embeddings, and Logical-Level Bias, influencing reasoning outputs. Our findings provide a comprehensive evaluation of LLMs' capabilities and limitations in temporal reasoning, highlighting key challenges in handling temporal data accurately.

Figures

Figures reproduced from arXiv: 2412.13377 by the authors.

Figure 1
Figure 1. Examples of temporal biases in LLMs. Incorrect Response, Faulty Date but accurate rea￾soning indicating representation level temporal bias, Faulty reasoning but accurate date indicating logical level temporal bias, Correct response yield accurate outputs due to misaligned internal processing. Together, these biases highlight the challenges LLMs face in preserving the integrity and interpretability of temporal data a… view at source ↗
Figure 2
Figure 2. Human evaluation rubric torical periods (e.g., the 1600s), modern contexts (e.g., the 2000s), and hypothetical futures (e.g., the 2100s). For clarity, we categorised dates into past, present, and future, with some questions covering multiple dates to assess LLMs’ ability to manage temporal relationships across contexts. Rationale for Design The dataset prioritises models’ ability to interpret dates within broader na… view at source ↗
Figure 3
Figure 3. Results Visualisations Type of Q Question Answer Date Format Llama3-70B Qwen2.5-72B GPT-4o GPT-4-turbo CMDR+ Factual Which of the fol￾lowing famous people died on 23041616? Answer only with A, B, C, or D: A) Shah Jahan B) Miguel de Cervantes C) Princess Diana D) William Shakespeare William Shake￾speare OR Miguel de Cer￾vantes DDMMYYYY B) Miguel de Cervantes and D) William Shakespeare B, D B) Miguel de Cervantes D)Wi… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Each bar is segmented into four colors representing the quality of responses: Incorrect Response, Faulty Date but accurate reasoning in￾dicating representation level temporal bias, Faulty reasoning but accurate date indicating logical level tem￾poral bias, Correct resp…
Figure 5
Figure 5. Figure 5: Correlation plot between semantic integrity [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 3 canonical work pages

  1. [1]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matt...

    arXiv 2024

  2. [2]

    Mortensen, Noah A

    Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David R. Mortensen, Noah A. Smith, and Yulia Tsvetkov. 2023. http://arxiv.org/abs/2305.13707v1 Do all languages cost the same? tokenization in the era of commercial language models

    arXiv 2023

  3. [3]

    Marvin Alberts, Gianmarco Gabrieli, and Irina Espejo Morales. 2024. http://arxiv.org/abs/2410.19353v1 Interleaving text and number embeddings to solve mathemathics problems

    arXiv 2024

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    arXiv 2020

  5. [5]

    Cohere. 2024. https://docs.cohere.com/docs/command-r-plus#model-details Command r+ model documentation . Comprehensive documentation and model details for the Command R+ model

    work page 2024

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany...

    arXiv 2024

  7. [7]

    Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin, Karishma Malkan, Jinyeong Yim, John Palowitch, Sungyong Seo, Jonathan Halcrow, and Bryan Perozzi. 2024. http://arxiv.org/abs/2406.09170v1 Test of time: A benchmark for evaluating llms on temporal reasoning

    arXiv 2024

  8. [8]

    Juan Luis Gastaldi, John Terilla, Luca Malagutti, Brian DuSell, Tim Vieira, and Ryan Cotterell. 2024. http://arxiv.org/abs/2407.11606v3 The foundations of tokenization: Statistical and computational concerns

    arXiv 2024

Show all 34 references
  1. [9]

    Omer Goldman, Avi Caciularu, Matan Eyal, Kris Cao, Idan Szpektor, and Reut Tsarfaty. 2024. http://arxiv.org/abs/2403.06265v2 Unpacking tokenization: Evaluating text compression and its correlation with model performance

    2024 arXiv

  2. [10]

    Shuhao Gu, Mengdi Zhao, Bowen Zhang, Liangdong Wang, Jijie Li, and Guang Liu. 2024. http://arxiv.org/abs/2410.04335v1 Retok: Replacing tokenizer to enhance representation efficiency in large language model

    2024 arXiv

  3. [11]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  4. [12]

    Sindhu Kishore and Hangfeng He. 2024. http://arxiv.org/abs/2404.01453v1 Unveiling divergent inductive biases of llms on temporal data

    2024 arXiv

  5. [13]

    Jingyu Liu, Jiaen Lin, and Yong Liu. 2024. http://arxiv.org/abs/2410.02338 How much can rag help the reasoning of llm?

    2024 arXiv

  6. [14]

    Benjamin Minixhofer, Edoardo Maria Ponti, and Ivan Vulić. 2024. http://arxiv.org/abs/2405.07883v1 Zero-shot tokenizer transfer

    2024

  7. [15]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Pai...

    2024 arXiv

  8. [16]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2023 arXiv

  9. [17]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. http://arxiv.org/abs/2305.18290 Direct preference optimization: Your language model is secretly a reward model

    2024 arXiv

  10. [18]

    Nived Rajaraman, Jiantao Jiao, and Kannan Ramchandran. 2024. http://arxiv.org/abs/2404.08335v1 Toward a theory of tokenization in llms

    2024 arXiv

  11. [19]

    Schmidt, Varshini Reddy, Haoran Zhang, Alec Alameddine, Omri Uzan, Yuval Pinter, and Chris Tanner

    Craig W. Schmidt, Varshini Reddy, Haoran Zhang, Alec Alameddine, Omri Uzan, Yuval Pinter, and Chris Tanner. 2024. http://arxiv.org/abs/2402.18376v2 Tokenization is more than compression

    2024 arXiv

  12. [20]

    Singh and DJ Strouse

    Aaditya K. Singh and DJ Strouse. 2024. http://arxiv.org/abs/2402.14903v1 Tokenization counts: the impact of tokenization on arithmetic in frontier llms

    2024 arXiv

  13. [21]

    Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A

    Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Mu...

    2024 arXiv

  14. [23]

    Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, and Yu Cheng. 2024 b . http://arxiv.org/abs/2406.14192 Timo: Towards better temporal reasoning for language models

    2024 arXiv

  15. [24]

    Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023 a . http://arxiv.org/abs/2306.08952 Towards benchmarking and improving the temporal reasoning capability of large language models

    2023 arXiv

  16. [25]

    Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023 b . http://arxiv.org/abs/2311.09821v2 Towards robust temporal reasoning of large language models via a multi-hop qa dataset and pseudo-instruction tuning

    2023 arXiv

  17. [26]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  18. [27]

    Maurice Weber, Daniel Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, and Ce Zhang. 2024. h...

    2024 arXiv

  19. [28]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. http://arxiv.org/abs/2201.11903 Chain-of-thought prompting elicits reasoning in large language models

    2023 arXiv

  20. [30]

    Siheng Xiong, Ali Payani, Ramana Kompella, and Faramarz Fekri. 2024 b . http://arxiv.org/abs/2401.06853 Large language models can learn temporal reasoning

    2024 arXiv

  21. [31]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  22. [32]

    Xiang Zhang, Juntai Cao, and Chenyu You. 2024. http://arxiv.org/abs/2410.19730v2 Counting ability of large language models and impact of tokenization

    2024 arXiv

  23. [33]

    Bowen Zhao, Zander Brumbaugh, Yizhong Wang, Hannaneh Hajishirzi, and Noah A. Smith. 2024. http://arxiv.org/abs/2402.16797v2 Set the clock: Temporal alignment of pretrained language models

    2024 arXiv

  24. [34]

    Chenghao Zhu, Nuo Chen, Yufei Gao, Yunyi Zhang, Prayag Tiwari, and Benyou Wang. 2024. http://arxiv.org/abs/2405.08460v2 Is your llm outdated? evaluating llms at temporal generalization

    2024 arXiv

  25. [35]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  26. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.