Pith. sign in

REVIEW 3 major objections 5 minor 31 references

Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper presents TIDE, a benchmark of 3,050 expert-verified question-answer pairs over 644 official Bangladesh customs instruments issued between 1969 and 2025, and claims that even the strongest of nine large language models resolves…

desk verdict TIDE's dataset is a solid, expert-verified contribution to testing version resolution in evolving documents, but the Event Sorting scoring inconsistency makes the headline 68.5% and the ICL-vs-RAG comparison unauditable as printed. read the letter →

arxiv 2608.08512 v1 pith:MFVBCG4U submitted 2026-08-09 cs.AI

Mahbub E Sobhani , Md. Faiyaz Abdullah Sayeedi , Fahmid Hasan Chowdhury , Md Adnan Arefeen , Farig Sadeque , Md. Faizul Bari , Swakkhar Shatabda This is my paper · ORCID
classification cs.AI
keywords versionresolutionevolvingdocumentstemporalquestionansweringLLMbenchmarkcustomslawretrieval-augmentedgenerationreasoningknowledgecutoff
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that answering questions about laws, tax codes, and other documents that are amended over time requires a distinct skill, version resolution, which existing temporal-question benchmarks never test. TIDE is positioned as the first benchmark built for that skill: 3,050 question-answer pairs over authentic customs instruments where the correct answer changes with the queried date. Under one evaluation protocol across nine large language models, the best macro-averaged accuracy reaches only 68.5%, resolving the rule in force from an implied date works 59.7% of the time, and detecting that a supplied version does not govern the query reaches only 26.7%. The stakes are concrete because any deployment of language models in legal, tax, or compliance settings depends on knowing which version of a rule applies, and the paper shows that today's models cannot be trusted to do this even when the right documents are in front of them.

What carries the argument

The load-bearing object is the evolving concept thread: a cluster of dated provisions across multiple instruments that all speak about the same regulatory topic, stored as a time-ordered list of instruments and the values they set. Version resolution means mapping a query date to the value in force on that date, and every generated question is written against a full thread history rather than a snapshot of one document. Three supporting mechanisms carry the results: the entity-clustering and alias-judgement pipeline that assembles threads from 644 code-mixed bilingual documents; the three knowledge-access settings (parametric memory, gold context, and top-10 retrieval) that isolate where failure occurs; and the scoring protocol, a three-judge LLM council whose verdicts pass through a hard date gate that scores an answer zero whenever it commits to a wrong or contradictory date even if the meaning is correct. The date gate lets the paper separate correct meaning from correct time, which reveals that once grounding is supplied, the residual error flips from wrong substance with a right date to right substance with a wrong date.

What would settle it

Run the parametric probe with a model whose training-corpus membership can be independently verified, for example a model trained on a checked-in snapshot of the TIDE corpus, and see whether the version-sensitive scores stay in the reported near-zero range; if they rise, the low parametric scores reflect absent documents rather than failed version resolution. A second check: on the 29.2% of items that stayed wrong even with the gold document, add an explicit instruction to state which clause governs the queried date and see whether accuracy moves, since improvement would show the reported ceiling is a prompting artifact rather than a reasoning limit.

Watch

Extended reading notes

Core claim

The paper's central claim is that version resolution over evolving documents is a distinct and presently unsolved capability. In an evolving document, an amendment is itself an official text that names what it replaces and when it takes effect, so several versions of the same rule can be simultaneously correct, each for its own validity period; existing temporal QA datasets treat time only as an annotation and therefore never test this. TIDE operationalises the claim with 3,050 QA pairs over 644 official customs instruments, and the evaluation shows two things: reading the correct text is necessary but not sufficient, since even with the gold documents in the prompt the best model scores 68.49%, and models are systematically asymmetric, finding correct versions far more readily than rejecting incorrect ones, while tending to follow a confident parametric answer over the supplied authoritative text. The hardest failures sit exactly where deployment risk is highest, with a fluent wrong premise overriding correct evidence that appears in the same prompt.

Load-bearing premise

The load-bearing premise is that every evaluated model's training data actually contains the full 1969 to 2025 corpus, so low closed-book scores measure version-resolution failure rather than simply missing documents; the paper admits this cannot be verified because providers do not disclose training data, and two model cards list knowledge cutoffs earlier than the most recent benchmark documents.

Editorial extensions

If this is right

  • Any LLM deployment in legal, customs, tax, or compliance workflows that answers date-sensitive questions without a version check will produce superseded answers; the context-misalignment task shows a fluent wrong premise overrides correct evidence even when the authoritative text sits above it in the same prompt.
  • Retrieval completeness is the current bottleneck for amendment-wide reasoning: event-sorting accuracy for the best model collapses from 66.23% under gold context to 31.42% under a standard semantic retriever, so any retriever that cannot recall the full amendment history caps model performance.
  • The 29.2% of items that stay wrong even when the gold document is supplied define a reasoning ceiling that retrieval cannot address, so closing the gap requires better version-resolution behaviour rather than more context.
  • Detecting that a provision was altered (about 79% detection on average) is far easier than identifying what was altered (about 19% identification on average) without grounding, so verification tasks should be scored on identification rather than on a binary verdict alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If version resolution is a general skill rather than a corpus-specific one, the same eight-task design should transfer to income tax, VAT, medical guidelines, and software documentation; porting the pipeline to another jurisdiction or domain is a direct test of that generality that the paper does not run.
  • The monotonic-increase bias the paper observes, where models assume regulatory values only ever rise and therefore misorder or misdate downward amendments, suggests a broader prior in parametric knowledge; a small targeted probe of downward revisions across domains could quantify how widespread it is.
  • The date-gated scoring protocol could serve as a reusable evaluation standard for any time-sensitive QA, since it cleanly separates meaning errors from temporal errors; a lighter deployed variant might rely on the deterministic date gate alone without the three-judge council.
  • The reported 5.5% regression, where the gold document overturns a correct closed-book answer, implies that giving a model the authoritative source can actively harm accuracy when dual-calendar dates are misread, so document-grounded pipelines need an explicit disagreement mechanism between memory and evidence.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TIDE, a benchmark of 3,050 expert-verified QA pairs over 644 official Bangladesh customs instruments (1969–2025), across eight task types that are designed to test version resolution in evolving documents. The authors evaluate nine recent LLMs under three knowledge-access settings (parametric, in-context learning with gold documents, and retrieval-augmented generation), using a three-judge LLM council with a hard date gate for free-form answers. The central empirical claims are that the best macro-averaged accuracy is only 68.49% (GPT-5, ICL), that models find correct versions more readily than they reject incorrect ones, and that providing the correct text improves but does not solve version resolution. The paper also reports task-specific failures such as low Context Misalignment accuracy and a large ICL-vs-RAG gap on Event Sorting, which is attributed to retrieval incompleteness.

Significance. The dataset construction is a genuine contribution: the corpus is drawn from official legal documents, all QA pairs were verified by two subject-matter experts with high inter-expert agreement, the evaluation protocol is described in enough detail to be reproduced, and the deterministic scoring equation plus the date gate is a principled way to separate factual correctness from temporal correctness. If the reported results are sound, TIDE fills a real gap by testing formal amendment semantics rather than treating time as a mere annotation. The paper also ships the prompt templates, licensing details, and a reproducibility-oriented appendix, which are valuable for the community. However, the headline numbers are currently not auditable because of an internal contradiction about how Event Sorting is scored across settings, and the parametric setting relies on a cutoff assumption that is contradicted by the paper's own model table.

major comments (3)
  1. [Section 3.2 / Table 3 / Appendix A.10.2, Table 16] Section 3.2 and the Table 3 footnote state that Event Sorting is reported as exact-match accuracy in every setting, but Appendix A.10.2 (Table 16, footnote) states that 'Event sorting uses a graded ordering metric under ICL but exact match elsewhere, so this row is not on a common scale,' and Appendix A.11 excludes Event Sorting from the RAG ablation because it is 'under re-scoring.' Appendix A.10.1 further reports the same ICL–RAG event-sorting comparison as 0.57 versus 0.28 while also giving graded pairwise-order values of 0.81 versus 0.67, which is consistent only if the ICL number in Table 3 is not on the exact-match scale. Since Event Sorting is one of the eight tasks entering the headline macro average of 68.49% (Table 3), the 'unified protocol' claim and the ICL-vs-RAG comparison are not auditable from the published tables. Please re-score Event Sorting with a single metric across all three settings and recompute all affected macro averages and conclusions.
  2. [Section 3.1 / Section 3.3 / Table 8 / Limitations] The parametric setting is justified by the statement in Section 3.1 that models were selected 'whose cut-off dates cover the date of our most recent document.' This is the load-bearing assumption for interpreting the Parametric column of Table 3 as a measure of version-resolution failure rather than missing documents. Table 8, however, lists Gemini-3.5-Flash and Claude-Sonnet-4.5 with reliable knowledge cutoffs of January 2025, while Appendix A.4 contains benchmark items referencing instruments dated 2025-02-17 and 2025-06-03 (Temporal MCQ Example 1 and Event Sorting Example 1). The open-weight models are listed with undisclosed cutoffs, so their coverage cannot be confirmed. The Limitations section explicitly concedes that 'A low parametric score might indicate an absent document rather than a version resolution failure,' but this concession is not reflected in the main interpretation of the parametric scores. Please either restrict parametric analyses to items dated before every model's disclosed cutoff, report a cutoff-aware analysis, or reframe the parametric column as a model-specific lower bound that is not directly comparable across models.
  3. [Section 3.2 / Appendix A.6, Tables 10 and 11] The reliability of the LLM council is central to the free-form task scores, but the reported human agreement is not item-level agreement on the booleans that Equation (2) actually consumes. Section 3.2 says that two SMEs rescored the council's decisions and that 'their average agreement with the council is 94.21%,' while Appendix A.6 shows this value is the mean of two rubric-point totals (92.93 and 95.49 out of 100) from a rubric that awards points for verdict accuracy, reasoning, and clarity. A rubric-point total can be high even when the experts disagree with the council on a subset of the final verdicts, and those verdicts are the exact inputs to the deterministic scorer. Please report item-level agreement (for example, percent agreement or Cohen's kappa on the final verdict booleans) so that the validation statistic matches the decision-relevant quantity.
minor comments (5)
  1. [Abstract / Table 3, Relative-Time QA row] The abstract states that resolving a version from an implicit date reaches 59.7%, but Table 3 shows Claude-Sonnet-4.5 reaching 68.03% strict accuracy on Relative-Time QA under ICL; if 59.7% refers to a different model or aggregation, the abstract should say so.
  2. [Table 4 vs. Section 3.1 and Table 3] The model is called GPT-OSS-20B in Table 4 but GPT-OSS-120B in Section 3.1, Table 1, and Table 3; the name should be consistent.
  3. [Table 2] The table rows are visually compressed: for example, the Temporal MCQ and Event Sorting rows run together in the stimulus column ('4 options 136 128 41Event Sorting'), and the Event Sorting stimulus value '3.5 events' is unclear; use explicit column separators and state whether this is a mean, median, or range.
  4. [Appendix A.3] The appendix says pairs rated 1 are discarded and new pairs are written on the same thread theme, and that after repairs every retained pair meets the score-3 criterion; please state explicitly that the replacement pairs were also reviewed by both SMEs and whether the 94.89% figure is before or after these replacements.
  5. [Section 2.2 / Section 3.2 / Section 3.3] The paper would be easier to assess if it explicitly discussed the family-level overlap in which Gemini models are used for parsing, entity and concept extraction, QA generation, retrieval embeddings, as a council judge, and as a subject model; the SME verification mitigates the gold-data risk, but a short paragraph on the residual risk would be useful.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction in TIDE's construction or evaluation; the only self-citation is a reproduced rubric and is not load-bearing.

full rationale

TIDE's derivation chain is externally anchored rather than self-referential. The corpus is 644 official customs PDFs; parsing, entity extraction, and QA generation are performed by Gemini, but every generated QA pair is independently verified by two subject-matter experts (94.89% first-pass correct, Cohen's kappa=0.91) and carries clause-level provenance, so the gold answers are not defined by any evaluated model's outputs. Evaluation is deterministic for Temporal MCQ, perturbation verdict, and event ordering, and the free-form tasks are scored by an LLM council that emits only booleans converted by the deterministic Equation 2; the council's agreement with experts (94.21%) is reported in Appendix A.6. No parameter is fitted to a subset and then renamed as a prediction, and no task definition presupposes the conclusion it is used to support. The sole self-citation, E Sobhani et al. (2026), supplies a validation rubric that is reproduced in full in Table 10 and supported by in-paper SME scores, so it is not load-bearing. Two flagged concerns are auditability rather than circularity: Section 3.3's assumption that all models' pretraining covers the full corpus is explicitly acknowledged as unverifiable in the Limitations ('A low parametric score might indicate an absent document rather than a version resolution failure'), and Appendix A.10.2 states that 'Event sorting uses a graded ordering metric under ICL but exact match elsewhere, so this row is not on a common scale,' which undermines the headline 68.5% macro as a unified metric but does not reduce any result to its inputs. Score 2 reflects the minor non-load-bearing self-citation; the central claims remain independent of it.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The benchmark introduces no fitted physical parameters or new postulated entities. The free parameters listed are hand-chosen pipeline thresholds whose effects are partly ablated in Appendix A.11; the axioms are the human-verification and evaluation assumptions on which the gold labels and scores rest.

free parameters (3)
  • cosine similarity threshold for alias candidate selection = 0.90
    Section 2.4 sets the nearest-neighbour candidate threshold; it affects entity clustering and thread construction but is not fitted to outcomes.
  • retrieval over-fetch and target depth = 40 candidates then top-10 clauses
    Section 3.3 and Appendix A.11. Authors select top-10 as the most stable configuration; top-15 gives a marginally higher macro average for gemini-embedding-2, so RAG numbers are configuration-dependent.
  • chunk size for clauses over the token limit = 1,200 characters
    Section 3.3; used for 82 oversized clauses before parent-clause mapping, and it affects retrieval granularity.
assumptions (4)
  • domain assumption OCR and structured extraction by Gemini 3.1 Pro, corrected by 13 volunteers to a 0.07-0.08% word error rate, preserves the legal content of 644 PDFs.
    Section 2.2. If extraction loses or garbles legal meaning, the QA pairs and all evaluations built on them are compromised.
  • domain assumption Two customs subject-matter experts can correctly verify the 3,050 QA pairs, and their ratings provide valid gold labels.
    Section 2.4 and Appendix A.3. The benchmark's ground truth rests on this human verification, with 94.89% rated correct and Cohen's kappa 0.91.
  • domain assumption An LLM council of three judges, validated at 94.21% agreement with SMEs, is a reliable scorer for free-form temporal QA.
    Section 3.2 and Appendix A.6. All six free-form task scores depend on this assumption; disagreement with SMEs would shift every reported accuracy.
  • ad hoc to paper The nine evaluated models have parametric knowledge covering the full 1969-2025 corpus.
    Section 3.1 claims selection by cutoff coverage, but Table 8 lists January 2025 reliable cutoffs for Gemini and Claude while items reference mid-2025 instruments. This assumption is internally contradicted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding." pith.science (2026). https://pith.science/paper/MFVBCG4U

@misc{pith2026260808512,
  author       = {Pith},
  title        = {Pith review of: Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MFVBCG4U}},
  note         = {Machine review of arXiv:2608.08512}
}
read the original abstract

Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that states what it replaces and when it takes effect, and the earlier version stays correct for its validity period. The central challenge is therefore version resolution, that is, identifying the version in force on the queried date. Existing temporal QA datasets treat time only as an annotation, so version resolution stays untested. We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued between 1969 and 2025 by the Government of Bangladesh, covering eight task types over deeply code-mixed documents that are heterogeneous in layout and dated in two calendars. In addition, we evaluate nine recent LLMs under a single protocol across parametric, gold-context, and retrieval access, scored by a three-judge LLM council with a hard date gate separating correct meaning from correct time. The best macro-averaged accuracy is only 68.5%. Resolving a version from an implicit date reaches 59.7%, and detecting that the supplied version does not govern the query reaches only 26.7%. Models are more likely to find correct versions than to reject incorrect ones, and they tend to follow a confident parametric answer over the supplied authoritative text. All code and data are available at https://github.com/icsetepa44/TIDE

Figures

Figures reproduced from arXiv: 2608.08512 by the authors.

Figure 1
Figure 1. Comparison of traditional evolving knowledge [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of TIDE construction. From 644 official customs instruments, we build verified clauses and threads and generate 3,050 QA pairs across eight task types, evaluated under parametric, gold-context, and retrieval settings with a three-judge LLM council. Gemini 3.1 Pro for information extraction. Thir￾teen third-year Computer Science undergraduate volunteers verified OCR fidelity. Gemini omitted no words but occa… view at source ↗
Figure 3
Figure 3. Beyond the challenges of temporal reasoning, [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Partial accuracy per task type under the three settings. Each bar is the mean over the nine models, and [PITH_FULL_IMAGE:figures/full_fig_p033_4.png]
Figure 5
Figure 5. Figure 5: Macro partial accuracy per model under ICL, computed as the unweighted mean over the eight task [PITH_FULL_IMAGE:figures/full_fig_p033_5.png]
Figure 6
Figure 6. Figure 6: Perturbation detection under parametric knowledge. trast, the share of accurate clauses that are correctly left alone rises from 30.8% to 69.3% and 64.2%. The second gain is about two and a half times the first, therefore evidence mainly teaches the model when not to f…
Figure 8
Figure 8. Figure 8: Perturbation detection under ICL. A.11 RAG Ablation [PITH_FULL_IMAGE:figures/full_fig_p034_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 28 canonical work pages

  1. [1]

    2011-06-09— The baggage rules establish a 65kg primary allowance and a 35kg secondary allowance for clothing and personal items

    work page 2011

  2. [2]

    Google DeepMind

    Test of time: A benchmark for evaluating llms on temporal reasoning.Preprint, arXiv:2406.09170. Google DeepMind. 2026. Gemini 3.5 flash model card. Model card, Google DeepMind. Raphael Gruber, Abdelrahman Abdallah, Michael Fär- ber, and Adam Jatowt. 2025. ComplexTempQA: A 100m dataset for complex temporal question answer- ing. InProceedings of the 2025 Co...

    arXiv 2026

  3. [3]

    2016-06-02— The language regarding baggage composition is refined to specify that the 65kg al- lowance applies to total baggage brought by any means

    work page 2016

  4. [4]

    Example 2

    2025-06-03— The primary baggage allowance for passengers is reduced to 36kg. Example 2. Instruction:Arrange these events in chronological order (earliest first). Events (shuffled, dates removed): E1 The use of technical explanatory notes for machin- ery classification was mandated for duty purposes. E2 Large-scale industries were granted flexibility to ex...

    work page 2025

  5. [5]

    document_info

    2025-06-02— The legal basis for duty concessions was transitioned to the Customs Act. Perturbation Detection (ypert) Illustrates: each statement differs from the regulation in exactly one value, and the gold answer names and corrects that value. Example 1. 15 Statement:Under SRO 175/2024, pharmaceutical raw materials listed in TABLE-2 are granted a custom...

    work page 2012

  6. [6]

    2016-04-08— The baggage rules are updated to in- troduce a specific 40kg weight limit for passengers under twelve years of age

    work page 2016

  7. [9]

    2010-06-10— The government established a con- cessional duty threshold of one percent for indus- trial machinery

    work page 2010

  8. [10]

    2011-06-09— The concessional duty threshold was adjusted to three percent, with an expanded scope including specific components

    work page 2011

Show all 31 references
  1. [11]

    2016-06-02— Large-scale industries were granted flexibility to extend import timelines for machinery up to three years

    2016

  2. [12]

    2020-06-11— The use of technical explanatory notes for machinery classification was mandated for duty purposes

    2020

  3. [14]

    EXTRACT every page - capture ALL visible text in reading order

  4. [15]

    CLASSIFY each text block as METADATA or CLAUSE

  5. [16]

    MERGE clause text that continues across page boundaries into a single clause item

  6. [17]

    MERGE table rows that span multiple pages into a single table entry

  7. [18]

    metadata

    BUILD the JSON output with all items in document order. R1 OCR FIDELITY - Extract ALL text exactly as printed - every header, footer, page number, footnote, section number, and body text. - Bangla -> Bengali Unicode. English -> Latin script. Mixed -> keep both scripts exactly ...

  8. [19]

    Read the clause text below carefully

  9. [20]

    The clause may be in English, Bangla, or mixed Bangla-English - but ALL entity and concept values MUST be written in English regardless of the source language

  10. [21]

    Use the same form every time

    Be STRICT and CONSISTENT in entity naming - always use canonical, official English names. Use the same form every time

  11. [22]

    Be DESCRIPTIVE in concept formation - each concept must be a meaningful sentence (1-4 sentences) that clearly explains the legal provision or rule being applied to the entity

  12. [23]

    TABLES ARE ENTITY-RICH: When the clause text contains embedded table data (pipe-separated rows), treat the table content as equally important to the surrounding clause text. Tables in legal clauses often introduce distinct entities not mentioned in the main text - such as spec...

  13. [24]

    qa_pairs

    If no meaningful entities can be extracted (e.g., the text is a page number, blank, or purely navigational), return an empty entities list. A.8.3 Question–Answer Generation Prompt We used the prompt template in this section to guide Gemini 3.1 Pro in generating question– answe...

    1969

  14. [25]

    Do not use your own knowledge or opinion to second-guess it

    The GROUND-TRUTH ANSWER is authoritative and correct BY DEFINITION. Do not use your own knowledge or opinion to second-guess it

  15. [26]

    Decide ONLY whether the model answer commits to each of them; never introduce ground-truth dates of your own

    The ground-truth dates listed below were extracted programmatically from the ground-truth answer. Decide ONLY whether the model answer commits to each of them; never introduce ground-truth dates of your own

  16. [27]

    9 June 2011

    For each listed ground-truth date: matched = true iff the MODEL ANSWER commits to that date - in any language, script, or format (Bengali numerals, Bangla month names, "9 June 2011", "09/06/2011", spelled-out ordinals, or a relative expression that resolves to it) - supplying ...

    2011

  17. [28]

    Extra dates about other facts that do not conflict with the ground truth are NOT contradictions

    contradicting_dates_in_answer = true iff the model answer asserts a DIFFERENT date, at the required granularity, for a fact the ground truth dates - or gives multiple conflicting dates for the same fact without committing to one. Extra dates about other facts that do not confl...

  18. [29]

    raised to 15%

    meaning_correct is TRUE only if ALL of the following hold, judged against the ground-truth answer; otherwise FALSE. Binary - no partial credit. - The model answer entails the ground truth's key facts and contradicts none of them. - Correct legal instrument / entity / section (...

  19. [30]

    Documented conflict for this item: {conflict}

    If a judgment is genuinely borderline under the checklist, still commit to a boolean but set uncertain = true and say why in meaning_rationale (such items are routed to human review; they are never soft-scored). ===== TASK-SPECIFIC NOTES ===== (inserted after the header; only ...

  20. [31]

    {source_span}

    {canonical_date} - as written in the ground truth: "{source_span}" - required match: {granularity_description} ... (or, if none: "Ground-truth dates: none were extracted from the ground-truth answer. Return an empty date_matches list.") Model answer: {model_answer} (The Contex...

    2011

  21. [2000]

    The instruments are additionally mirrored on the Min- istry of Law portal,http://bdlaws.minlaw.gov.bd/. SCRIPT & LANGUAGE 2 Dual-calendar metadata 3 Code-switched legal prose MACRO LAYOUT 1 Institutional header 4 Heterogeneous layout TABLE MICRO-STRUCTURE 5 Sparse table struct...

    1969

  22. [2024]

    Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, and Bing Qin

    Dated data: Tracing knowledge cutoffs in large language models.Preprint, arXiv:2403.12958. Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, and Bing Qin. 2024. TimeBench: A comprehensive evaluation of tempo- ral reasoning abilities in large langu...

    2024 arXiv

  23. [2025]

    Siyue Zhang, Yuxiang Xue, Yiming Zhang, Xiaobao Wu, Anh Tuan Luu, and Chen Zhao

    Qwen3 technical report.arXiv preprint arXiv:2505.09388. Siyue Zhang, Yuxiang Xue, Yiming Zhang, Xiaobao Wu, Anh Tuan Luu, and Chen Zhao. 2025. MRAG: A modular retrieval framework for time-sensitive question answering. InFindings of the Association for Computational Linguistics...

    2023 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.