Pith. sign in

REVIEW 2 major objections 5 minor 49 references

Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative Framework

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that giving an instruction-tuned LLM a compact token derived from temporal knowledge graph structure boosts both its prediction accuracy and the quality of its explanations in explainable temporal reasoning.

desk verdict Solid prediction gains and a new benchmark, but the explanation evaluation is circular, so the explainable-reasoning claim needs a different yardstick. read the letter →

arxiv 2505.15245 v1 pith:N7BLDARJ submitted 2025-05-21 cs.CL cs.AI

Zihao Jiang , Ben Liu , Miao Peng , Wenjie Xu , Yao Xiao , Zhenyan Shan , Min Peng This is my paper · ORCID
classification cs.CLcs.AI
keywords explainabletemporalreasoningknowledgegraphlargelanguagemodelsstructure-textadaptersofttokeninstructiontuningETRbenchmarkLoRA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that large language models do explainable temporal reasoning better when they receive a structural token from a temporal knowledge graph in addition to plain text. The authors build the ETR benchmark, five datasets spanning minutes, days, and years, each with a query, a textualized reasoning chain, and a GPT-4o-written explanation. They then show that GETER, an instruction-tuned LLM with a soft graph token injected through a lightweight adapter, outperforms text-only LoRA tuning by 7 to 11 percent in F1 and produces explanations that score closer to the gold texts. If this is right, graph structure and text are complementary inputs for faithful temporal reasoning, and the gap can be bridged without retraining the LLM.

What carries the argument

The load-bearing object is the soft graph token produced by GETER. A frozen temporal encoder (RE-GCN) embeds entities and relations; the query and its reasoning chains are pooled by parameter-free message passing over concatenated subject-relation-object embeddings; a trainable projection matrix $W_p \in \mathbb{R}^{3d_s \times d_x}$ maps the pooled vector into the LLM embedding space; and the resulting token is prepended to the instruction-prompt tokens. LoRA adapts the LLM, which generates the target explanation text by maximizing the likelihood of the gold explanation given the graph token and text.

What would settle it

Have human annotators write explanations for a random sample of ETR queries, then score GETER and text-only LoRA against those human texts; if GETER's explanation advantage shrinks or reverses, the reported gains come from imitating GPT-4o style rather than from deeper temporal understanding.

Watch

Extended reading notes

Core claim

The central claim is that structural information encoded in a temporal knowledge graph, when mapped into an LLM embedding space as a soft graph token, improves both the correctness and the explainability of temporal predictions. On the ETR benchmark, GETER with Mistral raises overall F1 to 79.08 on ICEWS14, 72.02 on GDELT, and 81.80 on ICEWS05-15, beating LoRA-only tuning by 11.10 percent, 10.71 percent, and 7.54 percent respectively, and beats GPT-4o zero-shot BLEU-4 by 75 to 251 percent. The explanation text is generated by the LLM from the soft graph token plus instruction prompt, and the paper argues that this lets the model capture evolving relation patterns that text-only models miss.

Load-bearing premise

Everything about explanation quality rests on treating GPT-4o's revised explanations, written from prompts that already state the correct label, as ground truth; if those texts are not faithful reasoning, the BLEU, ROUGE, METEOR, and BERTScore improvements do not mean better reasoning.

Editorial extensions

If this is right

  • GETER with Mistral raises overall F1 over LoRA-only tuning by 11.10 percent on ICEWS14, 10.71 percent on GDELT, and 7.54 percent on ICEWS05-15.
  • Explanation outputs beat GPT-4o zero-shot on BLEU-4 by 75.28 percent, 251.31 percent, and 99.07 percent on the same three datasets.
  • Removing the structure-text adapter drops overall F1 by roughly 7 to 12 points across datasets, and removing reasoning chains text drops it further, so both structural and textual inputs are load-bearing.
  • GETER stays competitive when paired with CEN, CENET, or SiMFy instead of RE-GCN, so the framework does not depend on a single temporal encoder.
  • Randomly ordering the reasoning-chain text still yields competitive F1 (77.57 versus 79.08 for the best ordering on ICEWS14), suggesting the adapter couples structure and text robustly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If explanation quality is measured by agreement with GPT-4o texts, the 75 to 251 percent BLEU gains may partly reflect stylistic mimicry rather than reasoning fidelity; a human-judged faithfulness test would separate the two.
  • A direct testable extension is to swap the trained RE-GCN embeddings for random or untrained embeddings while keeping the adapter; if the F1 gain persists, the benefit comes from extra trainable parameters or token position rather than from graph structure.
  • The same soft-token recipe could transfer to other structured reasoning tasks, such as multi-hop question answering over knowledge graphs or event forecasting, whenever the input has a graph plus a textual rendering.
  • The neutral-sample construction depends on an NLI threshold of 0.7; varying that threshold could change the difficulty of the Unsure split, so reported neutral F1 is partly a property of the benchmark construction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces ETR, a benchmark for explainable temporal reasoning built from five temporal knowledge graph datasets, and GETER, a framework that prepends a soft graph token (derived from a frozen temporal encoder and a trained projection adapter) to LLM instruction-tuning inputs. The benchmark contains positive, negative, and neutral samples with GPT-4o-generated explanation text as gold output. GETER is evaluated against zero-shot LLMs, LoRA-tuned LLMs, and graph-based temporal reasoning baselines. The paper reports consistent F1 gains over LoRA-only tuning across five datasets, and improved BLEU/ROUGE/METEOR/BERTScore over GPT-4o and LoRA baselines, along with ablations supporting the contribution of the structural adapter and reasoning-chain text.

Significance. If the central claim were fully supported, the paper would make two useful contributions: a multi-granularity benchmark for temporal reasoning with explanation supervision, and a parameter-efficient method for injecting TKG structure into LLM generation. The prediction-side results are substantive and fairly consistent: GETER with Mistral improves overall F1 by roughly 7–11% over LoRA-only tuning on the three headline datasets, and the ablations in Table 4 show that both the structure-text adapter and the reasoning-chain text contribute. The code and dataset are claimed to be publicly available, and the benchmark construction addresses a genuine gap. However, the 'explainable' component is not established by the evaluation as presented. Explanation quality is scored by lexical/semantic similarity to GPT-4o-generated gold texts that also serve as training targets, and the human evaluation in Appendix A.5 rates only the gold explanations, not any model output.

major comments (2)
  1. [Sec. 3.2.2, Eq. (5), Sec. 5.2, Table 3] The explanation-quality evaluation is circular and does not measure reasoning quality. The gold explanations are constructed by prompting GPT-4o to revise a template that already asserts the correct label ('We predict that ... will happen ... please revise ... ensure that the prediction aligns with the reasoning steps'). These same GPT-4o texts are used as the supervision target in the instruction-tuning objective (Eq. 5, tokens Y_A) and as the references for BLEU-4, ROUGE-L, METEOR, and BERTScore in Table 3. Consequently, the explanation metrics report how closely a fine-tuned model reproduces GPT-4o's phrasing, not whether the explanation is faithful to the evidence, logically consistent, or even supportive of the emitted label. The human evaluation in Appendix A.5 scores only the benchmark's gold explanations (query, reasoning chains, and explanation text consistency), not the outputs of GETER, LoRA-tuned, or zero-shot models; therefore it cannot validate the relative explanation quality claimed in Section 5.2. The prediction F1 results in Table 2 are not affected by this flaw, but the paper's stated 'explainable temporal reasoning' contribution is not supported by the current evidence.
  2. [Sec. 5.3, Table 4] The ablation study reports only prediction F1 for the variants (GETER without the structure-text adapter, without reasoning-chain text, and without both). Since the paper's central claim concerns explainable reasoning, the ablation should also report explanation metrics (or at least a human faithfulness check) for these variants. Without such results, it is impossible to tell whether the graph token improves explanation quality independently of prediction accuracy, or whether its only measurable effect is on the classification head. This is load-bearing because the Introduction and Abstract state that GETER advances explainable temporal reasoning by integrating graph structure, whereas Table 4 demonstrates only a prediction-side benefit.
minor comments (5)
  1. [Abstract, Sec. 1] The abstract and introduction contain minor language issues, e.g., 'To address challenge' should be 'To address this challenge', and 'incoherence, unnatural flow, and insufficient logical consistency' is a run-on list. These do not affect the technical content but should be cleaned up.
  2. [Sec. 3.2.1, Eq. (1)] The notation in Equation (1), displayed as 'C(es, eo) ← l^ i=1 (Ei, Ri, Ei+1, Ti)', is garbled or nonstandard. The authors should define the composition or concatenation operation explicitly, and clarify what 'l ∈ {1, 2} denotes the path length' means for the case l=1 versus l=2.
  3. [Sec. 4.2, Eq. (4)] Equation (4) calls the graph representation computation 'parameter-free message passing', but the projection matrix Wp is a learned parameter. The word 'parameter-free' is misleading; the authors likely mean that the aggregation over the reasoning chain is non-parametric. Please rephrase.
  4. [Sec. 5.2, Tables 2–3] No standard deviations or confidence intervals are reported for the F1 and explanation metrics, despite the fact that the test sets are of modest size (e.g., 949–2150 instances). Reporting results from multiple seeds or at least statistical significance tests would strengthen the reproducibility claims.
  5. [Appendix A.5, Table 7] The human evaluation scores average around 2.8–2.9 on a 1–3 scale, which is high, but no inter-annotator agreement measure (e.g., Cohen's kappa or Krippendorff's alpha) is reported. Adding such a measure would make the benchmark quality assessment more convincing.

Circularity Check

2 steps flagged · score 6.0 of 10

Explanation-quality evaluation is circular: GPT-4o generates gold explanations from templates that already assert the label, and the same GPT-4o texts are both the training target and the evaluation reference for the lexical explanation metrics.

  1. fitted input called prediction [Section 3.2.2 (Explanation Generation); prompt in Appendix A.1]
    "Given the following text: "we predict that [es] [r] [eo] will happen on [tq]. Here are the reasoning steps: C(es,eo)." Please revise the provided text to ensure that the prediction aligns with the reasoning steps."

    The gold explanation is generated by GPT-4o from a template that already states the class label ('we predict that [es] [r] [eo] will happen on [tq]'), so the explanation is a rationalization of a pre-assigned answer rather than an independently derived reasoning trace. The negative and neutral prompts similarly bake in 'will not happen' and 'It is unsure'. Thus the benchmark defines a 'correct explanation' as text that supports a label chosen before any reasoning, making the reference answer itself the label in prose.

  2. fitted input called prediction [Section 4.3 Eq. (5); Section 5.1 Metrics; Table 3]
    "our optimization objective is to maximize the likelihood of generating the target explanation text YA ... For explanation, we employ BLEU (Papineni et al., 2002) (4-gram), ROUGE (Lin, 2004) (ROUGE-L), METEOR (Banerjee and Lavie, 2005), and BertScore (Zhang et al., 2020) to measure the similarity between model-generated explanations and the ground truth in the test set."

    The target YA in Eq. (5) is the GPT-4o-generated explanation, and the 'ground truth' for the explanation metrics in Table 3 is drawn from the same GPT-4o generation process. Consequently, BLEU-4, ROUGE-L, METEOR, and BERTScore measure how closely the fine-tuned model reproduces the GPT-4o reference distribution, not whether the explanation is logically valid or supports the predicted label. The paper's conclusion that GETER 'enhance[s] explainable temporal reasoning capabilities' from these lexical scores is therefore an imitation-score claim, not an independent measurement of explanation quality.

full rationale

The circularity is confined to the explanation-quality evaluation; the prediction-F1 results are not affected. The positive/negative/neutral labels in ETR come from TKG facts, entity-replacement counterfactuals, and NLI-based neutral relation replacement, respectively, and the held-out extrapolation split is standard, so the F1 gains in Tables 2, 4, and 13 are independent and credible. The structure-text adapter is also not circular: its contribution is established by ablations (Table 4) and by varying the temporal encoder (Figure 4), and the linear mapping is only motivated, not derived, by the cited works; no load-bearing self-citation or uniqueness argument forces the architecture. However, the explanation component of the central claim is, by construction, a measure of how well the model imitates the GPT-4o generator used to create the training target, because the gold explanations are GPT-4o rationalizations of pre-assigned labels, the training objective (Eq. 5) maximizes likelihood of those texts, and the evaluation references are the same generation process. The human evaluation in Appendix A.5 scores only the benchmark's gold explanations (average 2.7-2.9), never any model-generated explanation, so it does not break this loop. Overall, the 'explainable' claim is partially circular while the prediction claim stands, giving a partial-circularity score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the benchmark construction and the evaluation protocol. The key free parameters are the benchmark construction choices (window size, NLI threshold, path length) and the LoRA rank. The most important untested assumptions are that GPT-4o gold explanations are a valid ground truth, that lexical metrics capture explanation quality, and that the frozen temporal encoder does not leak test-time information.

free parameters (4)
  • window size w = 30
    Hand-chosen parameter that defines how many timesteps of history are included when extracting reasoning chains for the benchmark. It affects which facts are considered as evidence.
  • NLI threshold tau = 0.7
    Threshold for selecting semantically neutral relations when constructing neutral samples; chosen by hand and not swept.
  • reasoning chain path length l = 1 or 2
    BFS path length restriction; a design choice limiting the evidence graph to short paths.
  • LoRA rank r = 16
    Rank of LoRA adapters, chosen without sensitivity analysis.
assumptions (5)
  • domain assumption GPT-4o-generated explanations are a valid gold standard for explanation quality
    Section 3.2.2 uses GPT-4o to revise template explanations, and Section 5.2 measures model explanations against these texts. This assumes the teacher's text is the ground truth.
  • domain assumption Lexical similarity metrics (BLEU, ROUGE, METEOR, BERTScore) are adequate for evaluating explainable reasoning
    Used in Section 5.2 without validation that they correlate with human judgment of explanation correctness.
  • domain assumption Reasoning chains of length at most 2 extracted by BFS are sufficient evidence for temporal prediction
    A restriction in Equation 1 and Section 3.2.1; longer or absent paths could change the difficulty and gold labels.
  • domain assumption A frozen pretrained temporal encoder can be transferred to the benchmark without temporal leakage
    Section 4.1 uses RE-GCN as a frozen encoder; the paper does not state whether the encoder's training split is strictly before test timestamps, so future information may leak into the structural embeddings.
  • domain assumption Extrapolation condition (train before test) holds for all benchmark instances
    Section 3.1 states ttrain < ttest, but it is not verified whether the temporal encoder and LLM fine-tuning obey this split.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative Framework." pith.science (2026). https://pith.science/paper/N7BLDARJ

@misc{pith2026250515245,
  author       = {Pith},
  title        = {Pith review of: Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N7BLDARJ}},
  note         = {Machine review of arXiv:2505.15245}
}
read the original abstract

While large language models (LLMs) show great potential in temporal reasoning, most existing work focuses heavily on enhancing performance, often neglecting the explainable reasoning processes underlying the results. To address this gap, we introduce a comprehensive benchmark covering a wide range of temporal granularities, designed to systematically evaluate LLMs' capabilities in explainable temporal reasoning. Furthermore, our findings reveal that LLMs struggle to deliver convincing explanations when relying solely on textual information. To address challenge, we propose GETER, a novel structure-aware generative framework that integrates Graph structures with text for Explainable TEmporal Reasoning. Specifically, we first leverage temporal knowledge graphs to develop a temporal encoder that captures structural information for the query. Subsequently, we introduce a structure-text prefix adapter to map graph structure features into the text embedding space. Finally, LLMs generate explanation text by seamlessly integrating the soft graph token with instruction-tuning prompt tokens. Experimental results indicate that GETER achieves state-of-the-art performance while also demonstrating its effectiveness as well as strong generalization capabilities. Our dataset and code are available at https://github.com/carryTatum/GETER.

Figures

Figures reproduced from arXiv: 2505.15245 by the authors.

Figure 1
Figure 1. An illustration of existing temporal reasoning [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of generating ETR benchmark. Dataset Time Granularity Type |P os.| |Neg.| |Neu.| Total ICEWS14 1 day Train 5000 4800 4500 14300 Test 800 700 600 2100 ICEWS05-15 1 day Train 4500 4400 4200 13100 Test 720 680 660 2060 ICEWS18 1 day Train 4400 4200 4000 12600 Test 750 700 650 2100 GDELT 15 minutes Train 4800 4600 4400 13800 Test 800 700 650 2150 WIKI 1 year Train 2482 2504 2342 7328 Test 347 286 316 949 [… view at source ↗
Figure 3
Figure 3. The overall framework of GETER. To bridge the gap between graph and text, we leverage TKGs to train a temporal encoder that captures structural information. Subsequently, the query and reasoning chains are encoded into a soft graph token, which is mapped into the text embedding space through a lightweight adapter. Finally, the target explanation text is generated using the soft graph token and related instruction tu… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: MLP depth comparison on ICEWS14 and GDELT datasets in terms of overall F1 scores (%). Model Positive Negative Neutral Overall GETER (paths order) 77.45 75.73 85.15 79.08 descending order 80.53 76.00 86.34 80.68 ascending order 77.72 77.52 86.04 80.03 random order 75.02…
Figure 4
Figure 4. Figure 4: Comparison of GETER with different tempo [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Comparison of training time between GETER [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 32 canonical work pages

  1. [1]

    Satanjeev Banerjee and Alon Lavie. 2005. METEOR: an automatic metric for MT evaluation with improved correlation with human judgments. In IEEvaluation@ACL, pages 65--72. Association for Computational Linguistics

    work page 2005

  2. [2]

    Veronika Bogina, Tsvi Kuflik, Dietmar Jannach, M \' a ria Bielikov \' a , Michal Kompan, and Christoph Trattner. 2023. https://doi.org/10.1007/S11257-022-09335-W Considering temporal aspects in recommender systems: a survey . User Model. User Adapt. Interact., 33(1):81--119

    work page doi:10.1007/s11257-022-09335-w 2023

  3. [3]

    Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Haotian Wang, Ming Liu, and Bing Qin. 2024. Timebench: A comprehensive evaluation of temporal reasoning abilities in large language models. In ACL (1) , pages 1204--1228. Association for Computational Linguistics

    work page 2024

  4. [4]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aur \' e lien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozi \` e...

    arXiv 2024

  5. [5]

    Alberto Garc \' a - Dur \' a n, Sebastijan Dumancic, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proc. of EMNLP, pages 4816--4821

    work page 2018

  6. [6]

    Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. 2021. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In ICLR . OpenReview.net

    work page 2021

  7. [7]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. https://openreview.net/forum?id=sE7-XhLxHA Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    2023

  8. [8]

    Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi

    Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V. Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. https://doi.org/10.48550/ARXIV.2402.07630 G-retriever: Retrieval-augmented generation for textual graph understanding and question answering . CoRR, abs/2402.07630

    arXiv doi:10.48550/arxiv.2402.07630 2024

Show all 49 references
  1. [9]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representat...

    2022

  2. [10]

    Jie Huang and Kevin Chen - Chuan Chang. 2023. Towards reasoning in large language models: A survey. In ACL (Findings) , pages 1049--1065. Association for Computational Linguistics

    2023

  3. [11]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, L \' e lio Renard Lavaud, Lucile Saulnier, Marie...

    2024 arXiv

  4. [12]

    Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, Yaliang Li, and Ji - Rong Wen. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.228 Reasoninglm: Enabling structural subgraph reasoning in pre-trained language models for question answering over knowledge graph . In Proceedings of the 2...

    2023 doi

  5. [13]

    Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. 2020. Recurrent event network: Autoregressive structure inferenceover temporal knowledge graphs. In Proc. of EMNLP, pages 6669--6683

    2020

  6. [14]

    Julien Leblay and Melisachew Wudage Chekol. 2018. https://doi.org/10.1145/3184558.3191639 Deriving validity time in knowledge graph . In Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23-27, 2018 , pages 1771--1776. ACM

    2018

  7. [15]

    Dong - Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, and Jay Pujara. 2023. Temporal knowledge graph forecasting without knowledge using in-context learning. In EMNLP , pages 544--557. Association for Computational Linguistics

    2023

  8. [16]

    Zixuan Li, Saiping Guan, Xiaolong Jin, Weihua Peng, Yajuan Lyu, Yong Zhu, Long Bai, Wei Li, Jiafeng Guo, and Xueqi Cheng. 2022. Complex evolutional pattern learning for temporal knowledge graph reasoning. In ACL (2) , pages 290--296. Association for Computational Linguistics

    2022

  9. [17]

    Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. 2021. https://doi.org/10.1145/3404835.3462963 Temporal knowledge graph reasoning based on evolutional representation learning . In SIGIR '21: The 44th International ACM SIG...

    2021

  10. [18]

    Ruotong Liao, Xu Jia, Yangzhe Li, Yunpu Ma, and Volker Tresp. 2024. Gentkg: Generative forecasting on temporal knowledge graph with large language models. In NAACL-HLT (Findings) , pages 4303--4317. Association for Computational Linguistics

    2024

  11. [19]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  12. [20]

    Qika Lin, Jun Liu, Rui Mao, Fangzhi Xu, and Erik Cambria. 2023. TECHS: temporal logical graph networks for explainable extrapolation reasoning. In ACL (1) , pages 1281--1293. Association for Computational Linguistics

    2023

  13. [21]

    Ben Liu, Jihai Zhang, Fangquan Lin, Cheng Yang, and Min Peng. 2025 a . https://aclanthology.org/2025.coling-main.740/ Filter-then-generate: Large language models with structure-text adapter for knowledge graph completion . In Proceedings of the 31st International Conference on...

    2025

  14. [22]

    Ben Liu, Jihai Zhang, Fangquan Lin, Cheng Yang, Min Peng, and Wotao Yin. 2025 b . Symagent: A neural-symbolic self-learning agent framework for complex reasoning over knowledge graphs. In WWW , pages 98--108. ACM

    2025

  15. [23]

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. 2022 a . http://papers.nips.cc/paper\_files/paper/2022/hash/0cde695b83bd186c1fd456302888454c-Abstract-Conference.html Few-shot parameter-efficient fine-tuning is better and cheap...

    2022

  16. [24]

    Yushan Liu, Yunpu Ma, Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. 2022 b . Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In AAAI , pages 4120--4127. AAAI Press

    2022

  17. [25]

    Zhengtao Liu, Lei Tan, Mengfan Li, Yao Wan, Hai Jin, and Xuanhua Shi. 2023. Simfy: A simple yet effective approach for temporal knowledge graph reasoning. In EMNLP (Findings) , pages 3825--3836. Association for Computational Linguistics

    2023

  18. [26]

    Ruilin Luo, Tianle Gu, Haoling Li, Junzhe Li, Zicheng Lin, Jiayi Li, and Yujiu Yang. 2024. Chain of history: Learning and forecasting with llms for temporal knowledge graph completion. CoRR, abs/2401.06072

    2024 arXiv

  19. [27]

    Qiyao Ma, Xubin Ren, and Chao Huang. 2024. Xrec: Large language models for explainable recommendation. In EMNLP (Findings) , pages 391--402. Association for Computational Linguistics

    2024

  20. [28]

    Xin Mei, Libin Yang, Xiaoyan Cai, and Zuowei Jiang. 2022. An adaptive logical rule embedding model for inductive reasoning over temporal knowledge graphs. In EMNLP , pages 7304--7316. Association for Computational Linguistics

    2022

  21. [29]

    OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774

    2023 arXiv

  22. [30]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL , pages 311--318. ACL

    2002

  23. [31]

    Miao Peng, Nuo Chen, Zongrui Suo, and Jia Li. 2025. Rewarding graph reasoning process makes llms more generalized reasoners. CoRR, abs/2503.00845

    2025 arXiv

  24. [32]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. https://doi.org/10.1109/SC41405.2020.00024 Zero: memory optimizations toward training trillion parameter models . In Proceedings of the International Conference for High Performance Computing, Networking, ...

    2020 arXiv

  25. [33]

    Haohai Sun, Jialun Zhong, Yunpu Ma, Zhen Han, and Kun He. 2021. Timetraveler: Reinforcement learning for temporal knowledge graph forecasting. In EMNLP (1) , pages 8306--8319. Association for Computational Linguistics

    2021

  26. [34]

    Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2023. Towards benchmarking and improving the temporal reasoning capability of large language models. In ACL (1) , pages 14820--14835. Association for Computational Linguistics

    2023

  27. [35]

    Qingyu Tan, Hwee Tou Ng, and Lidong Bing. 2024. Towards robust temporal reasoning of large language models via a multi-hop QA dataset and pseudo-instruction tuning. In ACL (Findings) , pages 6272--6286. Association for Computational Linguistics

    2024

  28. [36]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In ICLR . OpenReview.net

    2023

  29. [37]

    Yuqing Wang and Yun Zhao. 2024. TRAM: benchmarking temporal reasoning for large language models. In ACL (Findings) , pages 6389--6415. Association for Computational Linguistics

    2024

  30. [38]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language mod...

    2022

  31. [39]

    Yifan Wei, Yisong Su, Huanhuan Ma, Xiaoyan Yu, Fangyu Lei, Yuanzhe Zhang, Jun Zhao, and Kang Liu. 2023. Menatqa: A new dataset for testing the temporal comprehension and reasoning abilities of large language models. In EMNLP (Findings) , pages 1434--1447. Association for Compu...

    2023

  32. [40]

    Weiqi Wu, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, and Hai Zhao. 2025. https://doi.org/10.48550/ARXIV.2501.00888 Unfolding the headline: Iterative self-questioning for news retrieval and timeline summarization . CoRR, abs/2501.00888

    2025 doi arXiv

  33. [41]

    Yuwei Xia, Ding Wang, Qiang Liu, Liang Wang, Shu Wu, and Xiaoyu Zhang. 2024. Chain-of-history reasoning for temporal knowledge graph forecasting. In ACL (Findings) , pages 16144--16159. Association for Computational Linguistics

    2024

  34. [42]

    Siheng Xiong, Ali Payani, Ramana Kompella, and Faramarz Fekri. 2024. Large language models can learn temporal reasoning. In ACL (1) , pages 10452--10470. Association for Computational Linguistics

    2024

  35. [43]

    Yi Xu, Junjie Ou, Hui Xu, and Luoyi Fu. 2023. Temporal knowledge graph reasoning with historical contrastive learning. In AAAI , pages 4765--4773. AAAI Press

    2023

  36. [44]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng...

    2024 arXiv

  37. [45]

    Wanqi Yang, Yanda Li, Meng Fang, and Ling Chen. 2024 b . Enhancing temporal sensitivity and reasoning for time-sensitive question answering. In EMNLP (Findings) , pages 14495--14508. Association for Computational Linguistics

    2024

  38. [46]

    Chenhan Yuan, Qianqian Xie, Jimin Huang, and Sophia Ananiadou. 2024. https://doi.org/10.1145/3589334.3645376 Back to the future: Towards explainable temporal reasoning with large language models . In Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17...

    2024

  39. [47]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with BERT . In ICLR . OpenReview.net

    2020

  40. [48]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  41. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.