TEQUILA: Temporal Question Answering over Knowledge Bases

Abdalghani Abujabal; Gerhard Weikum; Jannik Stroetgen; Rishiraj Saha Roy; Zhen Jia

REVIEW 4 major objections 4 minor 31 references

TEQUILA: Temporal Question Answering over Knowledge Bases

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TEQUILA's decomposition-and-reasoning pipeline lets existing KB-QA engines answer composite temporal questions, with significant F1 gains.

desk verdict A solid, reusable wrapper for temporal KB-QA with honest experiments, but the 'any engine' claim overreaches the tested syntactic coverage. read the letter →

arxiv 1908.03650 v4 pith:LGAGC4VU submitted 2019-08-09 cs.IR cs.CL

Zhen Jia , Abdalghani Abujabal , Rishiraj Saha Roy , Jannik Stroetgen , Gerhard Weikum This is my paper · ORCID

classification cs.IRcs.CL

keywords temporalquestionansweringdecompositionknowledgebasereasoningintervalconstraintsTempQuestionstimescopes

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TEQUILA is a plug-in pipeline that lets ordinary knowledge-base question answering (KB-QA) systems handle composite temporal questions, the kind that contain 'before', 'after', 'during', or dates. The paper argues that a composite temporal question can be split at the time-signal word into a non-temporal sub-question and a temporal one, answered separately, and then recombined by interval reasoning over the retrieved dates. Wrapped around two existing systems, AQQU and QUINT, TEQUILA raises F1 on the TempQuestions benchmark from 27.2 to 36.7 and from 30.0 to 34.0, respectively, and beats a state-of-the-art complex-QA baseline on the temporal slice of ComplexQuestions. The paper's point is that temporal competence can be added to any KB-QA engine without modifying the engine itself.

What carries the argument

The central mechanism is the lexico-syntactic decomposition rule set: four hand-written patterns that split a temporal question into a non-temporal sub-question and a temporal sub-question, borrowing the entity or relation from the first part when the second lacks it. Around this sit two supporting mechanisms: the mapping of the question's signal word to an Allen-style interval constraint (before becomes candidate start $\leq$ constraint end, during becomes an overlap test), and the retrieval of time scopes for candidate answers by comparing predicate-name embeddings, e.g., choosing team.joinDate as the temporal partner of team.player. These rules convert a linguistic problem into a set-theoretic intersection plus interval arithmetic.

What would settle it

Take a temporal question with two time constraints in one sentence, such as 'Which teams did Neymar play for between leaving Barcelona and joining PSG?', run it through TEQUILA, and check whether it can be answered. The four patterns in Table 1 are built for a single signal word, so a correct answer to such a question would contradict the paper's implicit scope; a failure would show the decomposition layer, not the reasoning, is the limiting factor.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that complex temporal questions over knowledge bases are answerable by a fixed four-stage recipe: detect temporal intent, decompose at a signal word into sub-questions, retrieve answers and time scopes from an off-the-shelf KB-QA engine, and combine them with interval-based constraints such as BEFORE, AFTER, and OVERLAP. The result is that two simple-question engines, AQQU and QUINT, become competitive on temporal questions, with statistically significant F1 improvements over their standalone versions, and the combined system exceeds a general complex-question baseline (Bao et al.) on the temporal slice of ComplexQuestions. Error analysis shows failures trace to wrong decomposition, wrong temporal predicate selection, or wrong constraint, not to the underlying engine's inability to answer simple sub-questions.

Load-bearing premise

Everything rests on the four hand-written decomposition templates being the right and complete way to split temporal questions, and on the part-of-speech labeling being accurate; if a question's surface form falls outside the templates or a word like 'landing' is misread as a noun, the split is wrong and the final answer is wrong.

Editorial extensions

If this is right

Any current simple-question KB-QA engine can be wrapped with TEQUILA and gain temporal competence, without retraining or changing its internal query generator.
Composite temporal QA becomes a modular pipeline: question decomposition and interval reasoning are separable from entity linking and relation mapping, each improvable independently.
Error analysis suggests the next bottleneck is robust decomposition and predicate selection, not the underlying engine's simple-question answering.
The approach extends to ordinal constraints ('first', 'last') by sorting candidate intervals rather than merely filtering them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the decomposition patterns were extended to handle two or more signal words in one question (e.g., 'between leaving Barcelona and joining PSG'), the same interval-reasoning stage would need only a conjunction of constraints; the paper does not explore this, but nothing in the design blocks it.
The predicate-similarity step currently chooses one temporal predicate from the KB by embedding cosine; a testable extension is to make this choice jointly with the decomposition, since a wrong split can invalidate even a good predicate match.
Because the pipeline is engine-agnostic, the same wrapper could be retrofitted to neural KB-QA models as a preprocessing and post-processing layer, which might give neural engines a cheaper path to temporal competence than training on temporal examples.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Bottom line: this is a competent engineering contribution, and the central empirical claim holds up better than the stress-test note suggests. TEQUILA is the first complete temporal-QA pipeline built as a wrapper around existing KB-QA engines, and the combination of question decomposition plus Allen-style interval constraint reasoning is genuinely new in this context. Credit where due: the code and data are public, they evaluate on two benchmarks, one of which (ComplexQuestions) is independent of the authors, and they report paired t-tests. The error analysis is also candid, listing decomposition failures as a real weakness rather than hiding them.

Now the soft spots. The Table 1 patterns all assume the signal word appears after the wh-phrase with the temporal constraint following it. Fronted or embedded temporal clauses, like 'Before joining PSG, which teams did Neymar play for?', are untested. That does not sink the paper, but it directly limits the abstract's 'any KB-QA engine' promise: the wrapper is demonstrated for one syntactic order, not for temporal questions in general. The TempQuestions benchmark is the authors' own, which raises selection bias concerns, but the independent ComplexQuestions results largely mitigate that worry. The absence of component-level ablations is a real gap, though not a fatal one; the overall F1 gains are consistent across two different base systems, which is decent evidence the pipeline itself helps.

The stress-test note is right to flag the decomposition completeness issue, but I would not call it load-bearing in a way that invalidates the paper. The rules are explicit and the failure modes are observable. This is a scoped method with honest reporting, not a disguised fit.

Who gets value from this: researchers working on complex KB-QA or temporal information access, and people building modular QA systems. It is a reasonable reading-group pick, but not a paradigm-shifter. My recommendation: send it to serious peer review. The right revision would add experiments on varied clause positions and at least one component ablation, or explicitly narrow the claimed scope to the tested pattern. The core result is worth publishing.

Would cite? Yes, if I were working on temporal QA.

Best, [You]

Referee Report

4 major / 4 minor

Summary. The paper presents TEQUILA, a rule-based preprocessing and post-processing pipeline that wraps existing KB-QA engines to answer temporal questions. Given a question, TEQUILA detects temporal intent, decomposes the question into a non-temporal sub-question and a temporal constraint using one of four lexico-syntactic patterns in Table 1, rewrites the parts into sub-questions for the underlying engine, retrieves time scopes for candidate answers via additional KB lookups and a word2vec-based predicate-similarity heuristic, and finally combines results using Allen-style interval constraints. The system is evaluated on the TempQuestions benchmark (1,271 questions) and on 341 temporal questions from ComplexQuestions, with AQQU and QUINT as underlying engines and Bao et al. as a complex-QA baseline. TEQUILA-enabled systems show statistically significant F1 improvements over the standalone simple-QA systems and outperform Bao et al. on the temporal slice.

Significance. The contribution is potentially useful: a component-level enabler that adds temporal reasoning to existing KB-QA engines without retraining them. The paper ships public code and data, uses no fitted parameters, and reports paired t-tests, which strengthens the credibility of the average-F1 comparisons. The rule-based design makes the method transparent and easily diagnosable. However, the significance of the headline claim—that the four hand-crafted decomposition patterns cover composite temporal questions generally—depends on coverage that is not demonstrated, and the evaluation relies mainly on a benchmark constructed by the same authors. If the open coverage and ablation gaps are closed, the paper would be a solid systems contribution.

major comments (4)

[Section 3.2, Table 1] The four decomposition patterns in Table 1 all assume the canonical surface order wh* t1...tk SIGNAL t{k+1}...tn, with the signal word following the wh-phrase and the temporal constraint at the end of the question. The paper provides no evidence for fronted or embedded temporal clauses, such as 'Before joining PSG, which teams did Neymar play for?' or 'Which teams, after Neymar left Barcelona, did he play for?'. The abstract's 'any KB-QA engine' claim is about engine independence, but the broader claim that TEQUILA decomposes composite temporal questions is not restricted to this one word order. Please add experiments on syntactic variants or explicitly restrict the stated scope to the canonical order.
[Section 4.1, Section 4.2, Table 3] The main benchmark, TempQuestions, was created by the same authors (reference [13]), and the four decomposition patterns were manually authored; the paper reports no component-level ablations and no coverage analysis. As a result, the observed F1 gains may reflect alignment between the hand-written rules and the benchmark's question distribution rather than a general capability. Please report the fraction of TempQuestions questions covered by each of the four cases in Table 1, add ablations that isolate the contribution of decomposition, time-scope retrieval, and interval reasoning, and evaluate on an external benchmark or on a held-out set with different syntactic constructions.
[Section 4.2, error analysis] The error analysis is purely qualitative. It lists three failure modes—incorrect decomposition (illustrated by 'landing' being tagged as a noun), failure to find the correct temporal predicate, and wrongly identified constraints or time scopes—but it does not quantify how often each occurs. Since the central claim is that the pipeline improves F1, the paper should provide a quantitative breakdown of error types, including the fraction of failures attributable to POS-tagging errors in decomposition. Without such a breakdown, it is difficult to judge whether the four manually crafted patterns or the downstream reasoning stage is the main bottleneck.
[Section 3.3] The temporal predicate selection heuristic—cosine similarity between averaged word2vec embeddings of predicate names—is a load-bearing component: if the wrong temporal predicate is chosen, the interval constraints in Table 2 are evaluated against irrelevant time scopes. The paper only mentions 'limitations of the similarity function' in the qualitative error analysis. Please evaluate this component directly, for example by reporting the accuracy of the chosen begin/end predicates on a sample of questions, or at least by reporting the correlation between predicate-selection errors and end-to-end failures.

minor comments (4)

[Title and abstract] The manuscript contains typographical artifacts, including 'Q_uestion' in the title and broken line breaks such as 'chal - lenges' and 'compo sed' in the abstract; please proofread the camera-ready version.
[Table 2] The mathematical notation in Table 2 is rendered with garbled Unicode symbols (e.g., '/u1D452/u1D45B/u1D451...'); please define the interval endpoints with clear symbols such as t_start and t_end so that the constraints are readable.
[Section 4.2] The sentence 'The improvement in F1-scores stems from a systematic boost in precision' is only partially accurate, since Table 3 shows aggregate recall decreases for both AQQU+TEQUILA (48.0 to 42.3) and QUINT+TEQUILA (52.8 to 44.6) on TempQuestions; please rephrase to acknowledge the precision-recall trade-off.
[Section 2] The definition of temporal questions includes cases where 'the answer type is temporal', but the detection patterns listed in Section 3.1 are only 'when, what date, in what year, which century'; it would be helpful to clarify whether forms such as 'how long' or 'since when' are also treated as temporal answer questions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the pipeline is rule-based, parameter-free, and validated on an external benchmark.

full rationale

The paper's derivation chain is empirical and modular: temporal intent detection (HeidelTime + small dictionaries), handcrafted decomposition rules in Table 1, calls to external KB-QA engines for sub-questions, and interval reasoning in Table 2. No equation defines a predicted quantity in terms of a fitted parameter; no parameter is fit to gold answers; and final answers are obtained by intersecting independently retrieved candidates and applying temporal constraints. The only potentially self-referential element is that the main benchmark, TempQuestions [13], is authored by the same group. However, this is a dataset, not a result that the paper's derivation presupposes; moreover, the paper also evaluates on the external ComplexQuestions temporal slice [4] and compares directly with Bao et al., providing independent evidence for the central claim. The self-citation therefore does not carry the derivation. The handcrafted decomposition rules could be incomplete (the paper's own error analysis admits cases like 'landing' being mislabeled in 'where did the pilgrims come from before landing in america?'), but incompleteness is a correctness or fragility concern, not circularity. Overall, no claimed prediction is equivalent by construction to its inputs.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

TEQUILA's central claim rests on several premises the paper does not prove: the completeness of its decomposition patterns, the robustness of underlying KB-QA engines to ungrammatical rewrites, and the reliability of the word2vec similarity heuristic for temporal predicate selection. In addition, it assumes standard temporal NLP tools (HeidelTime) work as expected and Allen's interval algebra captures the needed temporal semantics. There are no fitted free parameters; the method is deterministic given the dictionaries and pre-trained word embeddings.

assumptions (5)

ad hoc to paper The lexico-syntactic patterns in Table 1 cover the valid decompositions for temporal questions.
Section 3.2 states these patterns are designed from first principles, not learned or derived from data; the error analysis in Section 4.2 shows cases where the wrong pattern is triggered, confirming this is a manual encoding of linguistic behavior.
domain assumption KB-QA systems are robust to ungrammatical sub-question rewrites.
Section 3.2 explicitly asserts this to justify the rewrite step; if the underlying engines fail on such rewrites, the entire pipeline's accuracy collapses.
ad hoc to paper The temporal predicate for an answer can be identified by cosine similarity between averaged word2vec embeddings of predicate names.
Section 3.3 introduces this heuristic for retrieving time scopes; it is a manually chosen model with no theoretical justification and is listed as a source of errors in Section 4.2.
standard math Allen's interval algebra is sufficient to represent the temporal semantics of the questions.
Section 3.4 casts all results into intervals and applies BEFORE, AFTER, OVERLAP, etc.; this is a standard framework from Allen (1990).
domain assumption HeidelTime correctly extracts temporal expressions in the input questions.
Section 3.1 relies on HeidelTime for stage (i) detection; any extraction errors propagate to later stages.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TEQUILA: Temporal Question Answering over Knowledge Bases." pith.science (2026). https://pith.science/paper/LGAGC4VU

@misc{pith2026190803650,
  author       = {Pith},
  title        = {Pith review of: TEQUILA: Temporal Question Answering over Knowledge Bases},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LGAGC4VU}},
  note         = {Machine review of arXiv:1908.03650}
}

read the original abstract

Question answering over knowledge bases (KB-QA) poses challenges in handling complex questions that need to be decomposed into sub-questions. An important case, addressed here, is that of temporal questions, where cues for temporal relations need to be discovered and handled. We present TEQUILA, an enabler method for temporal QA that can run on top of any KB-QA engine. TEQUILA has four stages. It detects if a question has temporal intent. It decomposes and rewrites the question into non-temporal sub-questions and temporal constraints. Answers to sub-questions are then retrieved from the underlying KB-QA engine. Finally, TEQUILA uses constraint reasoning on temporal intervals to compute final answers to the full question. Comparisons against state-of-the-art baselines show the viability of our method.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 30 canonical work pages

[13]

Z. Jia, A. Abujabal, R. Saha Roy, J. Strötgen, and G. Weik um. 2018. Temp- Questions: A Benchmark for Temporal Question Answering. In HQA

work page 2018
[1]

Abdalghani Abujabal, Rishiraj Saha Roy, Mohamed Yahya, and Gerhard Weikum
[2]

Abujabal, M

A. Abujabal, M. Yahya, M. Riedewald, and G. Weikum. 2017. Automated template generation for question answering over knowledge graphs. I n WWW

work page 2017
[3]

J. F. Allen. 1990. Maintaining knowledge about temporal intervals. In Readings in qualitative reasoning about physical systems

work page 1990
[4]

J. Bao, N. Duan, Z. Yan, M. Zhou, and T. Zhao. 2016. Constra int-based question answering with knowledge graph. In COLING

work page 2016
[5]

J. Bao, N. Duan, M. Zhou, and T. Zhao. 2014. Knowledge-bas ed question answer- ing as machine translation. In ACL

work page 2014
[6]

Bast and E

H. Bast and E. Haussmann. 2015. More Accurate Question An swering on Free- base. In CIKM

work page 2015
[7]

Berant, A

J. Berant, A. Chou, R. Frostig, and P. Liang. 2013. Semant ic Parsing on Freebase from Question-Answer Pairs. In EMNLP

work page 2013

Show all 31 references

[8]

Cai and A

Q. Cai and A. Yates. 2013. Large-scale Semantic Parsing v ia Schema Matching and Lexicon Extension. In ACL

2013
[9]

Diefenbach, V

D. Diefenbach, V. Lopez, K. Singh, and P. Maret. 2017. Cor e techniques of ques- tion answering systems over knowledge bases: A survey. In Knowledge and In- formation systems

2017
[10]

Dietz and B

L. Dietz and B. Gamari. 2017. TREC CAR: A Data Set for Comp lex Answer Retrieval. In TREC

2017
[11]

D. A. Ferrucci et al. 2012. This is Watson. In IBM Journal of R&D

2012
[12]

Fader, L

A. Fader, L. Zettlemoyer, and O. Etzioni. 2014. Open que stion answering over curated and extracted knowledge bases. In KDD

2014
[14]

Erdal Kuzey, Vinay Setty, Jannik Strötgen, and Gerhard Weikum. 2016. As time goes by: Comprehensive tagging of textual phrases with temp oral scopes. In Proceedings of the 25th international conference on world wi de web. 915–925. TEQUILA: Temporal /Q_uestion Answering over Kn...

2016
[15]

Liang, M

P. Liang, M. I. Jordan, and D. Klein. 2011. Learning Depe ndency-Based Compo- sitional Semantics. In ACL

2011
[16]

Metzler, R

D. Metzler, R. Jones, F. Peng, and R. Zhang. 2009. Improv ing Search Relevance for Implicitly Temporal Queries. In SIGIR

2009
[17]

Alessandro Moschitti, Kateryna Tymoshenko, Panos Ale xopoulos, Andrew Walker, Massimo Nicosia, Guido Vetere, Alessandro Faraotti, Marco Monti, Jeﬀ Z Pan, Honghan Wu, et al. 2017. Question answering and knowled ge graphs. In Exploiting Linked Data and Knowledge Graphs in Large ...

2017
[18]

Reddy, M

S. Reddy, M. Lapata, and M. Steedman. 2014. Large-scale semantic parsing with- out question-answer pairs. In TACL

2014
[19]

Saquete, J

E. Saquete, J. L. Vicedo, P. Martínez-Barco, R. Muñoz, a nd H. Llorens. 2009. En- hancing QA Systems with Complex Temporal Question Processi ng Capabilities. J. Artif. Int. Res. (2009)

2009
[20]

Savenkov and E

D. Savenkov and E. Agichtein. 2016. When a Knowledge Bas e Is Not Enough: Question Answering over Knowledge Bases with External Text Data. In SIGIR

2016
[21]

A. Setzer. 2002. Temporal information in Newswire articles: An annotation scheme and corpus study . Ph.D. Dissertation. University of Sheﬃeld

2002
[22]

Strötgen and M

J. Strötgen and M. Gertz. 2010. HeidelTime: High Qualit y Rule-Based Extraction and Normalization of Temporal Expressions. In SemEval

2010
[23]

Unger, L

C. Unger, L. Bühmann, J. Lehmann, A. N. Ngomo, D. Gerber, and P. Cimiano
[24]

Wieting, M

J. Wieting, M. Bansal, K. Gimpel, and K. Livescu. 2016. T owards universal para- phrastic sentence embeddings. In ICLR

2016
[25]

K. Xu, S. Reddy, Y. Feng, S. Huang, and D. Zhao. 2016. Ques tion answering on freebase via relation extraction and textual evidence. ACL

2016
[26]

Yahya, K

M. Yahya, K. Berberich, S. Elbassuoni, M. Ramanath, V. T resp, and G. Weikum
[27]

W. Yih, M. Chang, X. He, and J. Gao. 2015. Semantic Parsin g via Staged Query Graph Generation: Question Answering with Knowledge Base. In ACL

2015
[28]

P. Yin, N. Duan, B. Kao, J. Bao, and M. Zhou. 2015. Answeri ng Questions with Complex Semantic Constraints on Open Knowledge Bases. In CIKM

2015
[29]

In EMNLP

Natural language questions for the web of data. In EMNLP
[2012]

Template-based question answering over RDF data. In WWW
[2018]

Never-ending learning for open-domain question answ ering over knowl- edge bases. In WWW. 1053–1062

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [13]

Z. Jia, A. Abujabal, R. Saha Roy, J. Strötgen, and G. Weik um. 2018. Temp- Questions: A Benchmark for Temporal Question Answering. In HQA

work page 2018

[2] [1]

Abdalghani Abujabal, Rishiraj Saha Roy, Mohamed Yahya, and Gerhard Weikum

[3] [2]

Abujabal, M

A. Abujabal, M. Yahya, M. Riedewald, and G. Weikum. 2017. Automated template generation for question answering over knowledge graphs. I n WWW

work page 2017

[4] [3]

J. F. Allen. 1990. Maintaining knowledge about temporal intervals. In Readings in qualitative reasoning about physical systems

work page 1990

[5] [4]

J. Bao, N. Duan, Z. Yan, M. Zhou, and T. Zhao. 2016. Constra int-based question answering with knowledge graph. In COLING

work page 2016

[6] [5]

J. Bao, N. Duan, M. Zhou, and T. Zhao. 2014. Knowledge-bas ed question answer- ing as machine translation. In ACL

work page 2014

[7] [6]

Bast and E

H. Bast and E. Haussmann. 2015. More Accurate Question An swering on Free- base. In CIKM

work page 2015

[8] [7]

Berant, A

J. Berant, A. Chou, R. Frostig, and P. Liang. 2013. Semant ic Parsing on Freebase from Question-Answer Pairs. In EMNLP

work page 2013