{"id":"88a0012b-7790-45ee-a56f-9118b0011eaa","arxiv_id":"1908.03650","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"TEQUILA is a four-stage plug-in that detects temporal questions, decomposes them into simpler sub-questions, answers those with existing KB-QA engines, and applies temporal interval reasoning to produce final answers.","lead":"This paper introduces TEQUILA, a system that helps question-answering programs handle time-based questions such as 'Which teams did Neymar play for before joining PSG?' It breaks such questions into simpler parts, answers each part separately, and combines the results with reasoning about time intervals. TEQUILA works as a plug-in on top of existing knowledge base question-answering engines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's four decomposition patterns cover only one syntactic order, leaving fronted or embedded temporal clauses untested; the general 'any KB-QA engine' enabler claim is therefore not supported.","rationale":"The reader's weakest assumption is that the four decomposition patterns in Table 1 are complete with respect to valid ways of splitting temporal questions. My stress-test narrows this to a concrete syntactic restriction: all patterns require the signal to occur after the wh-clause and the constraint to follow it, so questions with fronted or otherwise displaced temporal constituents are entirely unaddressed. This is a load-bearing limitation because the central claim is not merely that TEQUILA improves F1 on TempQuestions, but that it is a general enabler for temporal KB-QA. The paper's own error analysis confirms that decomposition errors are a real failure mode, and the absence of any component-level evaluation of decomposition (e.g., accuracy against gold splits) leaves the coverage question open. I do not think this concern warrants rejecting the paper; the reported gains on both TempQuestions and the ComplexQuestions temporal slice are positive evidence, and the public code allows replication. However, the generality claim in the abstract is stronger than the evidence, so the existing CONDITIONAL verdict (requiring external validation and component-level evaluation) remains appropriate. I mark agreement as partial because the reader emphasized completeness of the rules in general and POS mislabeling, whereas I emphasize the narrow syntactic coverage of the templates; both point to the same underlying fragility of the decomposition stage.","tokens_in":9096,"tokens_out":8832,"duration_ms":93737,"concrete_test":"Construct a held-out set of 100 temporal questions (sampled from an independent source such as TimeQuestions or from news-style queries) that deliberately include fronted temporal clauses, embedded clauses, and signals in non-final positions. Manually annotate the correct decomposition into a non-temporal sub-question and a temporal constraint. Run TEQUILA's stage 2 decomposition on these questions and measure the fraction of questions whose generated sub-questions match the gold decomposition, and also measure end-to-end F1 with AQQU and QUINT. If the decomposition accuracy falls below 60% or the F1 gain over the standalone baselines becomes non-significant or negative, the central enabler claim is not general.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that TEQUILA enables arbitrary KB-QA engines to answer composite temporal questions rests on the completeness of the hand-crafted decomposition rules in Table 1. Every listed pattern assumes the input has the form wh* t1...tk SIGNAL t{k+1}...tn, i.e., the signal word appears after the wh-phrase and the entire constraint follows it. Yet natural temporal questions frequently place the temporal clause in other positions, e.g., 'Before joining PSG, which teams did Neymar play for?' or 'Which teams, after Neymar left Barcelona, did he play for?' The paper reports no experiments with such word-order variants, and Section 4.2's error analysis already shows decomposition fragility ('landing' mislabeled as a noun in 'where did the pilgrims come from before landing in america?', triggering Case 3 instead of Case 1). Because the four rules are manually authored and were evaluated primarily on the authors' own TempQuestions benchmark, the reported F1 gains may reflect alignment between the benchmark's questions and these specific patterns rather than a general capability. A system that fails on a substantial class of syntactically plausible temporal questions does not substantiate the abstract's promise of a plug-in that works with 'any KB-QA engine'.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents TEQUILA, a rule-based preprocessing and post-processing pipeline that wraps existing KB-QA engines to answer temporal questions. Given a question, TEQUILA detects temporal intent, decomposes the question into a non-temporal sub-question and a temporal constraint using one of four lexico-syntactic patterns in Table 1, rewrites the parts into sub-questions for the underlying engine, retrieves time scopes for candidate answers via additional KB lookups and a word2vec-based predicate-similarity heuristic, and finally combines results using Allen-style interval constraints. The system is evaluated on the TempQuestions benchmark (1,271 questions) and on 341 temporal questions from ComplexQuestions, with AQQU and QUINT as underlying engines and Bao et al. as a complex-QA baseline. TEQUILA-enabled systems show statistically significant F1 improvements over the standalone simple-QA systems and outperform Bao et al. on the temporal slice.","tokens_in":9354,"tokens_out":5004,"duration_ms":54123,"significance":"The contribution is potentially useful: a component-level enabler that adds temporal reasoning to existing KB-QA engines without retraining them. The paper ships public code and data, uses no fitted parameters, and reports paired t-tests, which strengthens the credibility of the average-F1 comparisons. The rule-based design makes the method transparent and easily diagnosable. However, the significance of the headline claim—that the four hand-crafted decomposition patterns cover composite temporal questions generally—depends on coverage that is not demonstrated, and the evaluation relies mainly on a benchmark constructed by the same authors. If the open coverage and ablation gaps are closed, the paper would be a solid systems contribution.","major_comments":[{"comment":"The four decomposition patterns in Table 1 all assume the canonical surface order wh* t1...tk SIGNAL t{k+1}...tn, with the signal word following the wh-phrase and the temporal constraint at the end of the question. The paper provides no evidence for fronted or embedded temporal clauses, such as 'Before joining PSG, which teams did Neymar play for?' or 'Which teams, after Neymar left Barcelona, did he play for?'. The abstract's 'any KB-QA engine' claim is about engine independence, but the broader claim that TEQUILA decomposes composite temporal questions is not restricted to this one word order. Please add experiments on syntactic variants or explicitly restrict the stated scope to the canonical order.","section":"Section 3.2, Table 1"},{"comment":"The main benchmark, TempQuestions, was created by the same authors (reference [13]), and the four decomposition patterns were manually authored; the paper reports no component-level ablations and no coverage analysis. As a result, the observed F1 gains may reflect alignment between the hand-written rules and the benchmark's question distribution rather than a general capability. Please report the fraction of TempQuestions questions covered by each of the four cases in Table 1, add ablations that isolate the contribution of decomposition, time-scope retrieval, and interval reasoning, and evaluate on an external benchmark or on a held-out set with different syntactic constructions.","section":"Section 4.1, Section 4.2, Table 3"},{"comment":"The error analysis is purely qualitative. It lists three failure modes—incorrect decomposition (illustrated by 'landing' being tagged as a noun), failure to find the correct temporal predicate, and wrongly identified constraints or time scopes—but it does not quantify how often each occurs. Since the central claim is that the pipeline improves F1, the paper should provide a quantitative breakdown of error types, including the fraction of failures attributable to POS-tagging errors in decomposition. Without such a breakdown, it is difficult to judge whether the four manually crafted patterns or the downstream reasoning stage is the main bottleneck.","section":"Section 4.2, error analysis"},{"comment":"The temporal predicate selection heuristic—cosine similarity between averaged word2vec embeddings of predicate names—is a load-bearing component: if the wrong temporal predicate is chosen, the interval constraints in Table 2 are evaluated against irrelevant time scopes. The paper only mentions 'limitations of the similarity function' in the qualitative error analysis. Please evaluate this component directly, for example by reporting the accuracy of the chosen begin/end predicates on a sample of questions, or at least by reporting the correlation between predicate-selection errors and end-to-end failures.","section":"Section 3.3"}],"minor_comments":[{"comment":"The manuscript contains typographical artifacts, including 'Q_uestion' in the title and broken line breaks such as 'chal - lenges' and 'compo sed' in the abstract; please proofread the camera-ready version.","section":"Title and abstract"},{"comment":"The mathematical notation in Table 2 is rendered with garbled Unicode symbols (e.g., '/u1D452/u1D45B/u1D451...'); please define the interval endpoints with clear symbols such as t_start and t_end so that the constraints are readable.","section":"Table 2"},{"comment":"The sentence 'The improvement in F1-scores stems from a systematic boost in precision' is only partially accurate, since Table 3 shows aggregate recall decreases for both AQQU+TEQUILA (48.0 to 42.3) and QUINT+TEQUILA (52.8 to 44.6) on TempQuestions; please rephrase to acknowledge the precision-recall trade-off.","section":"Section 4.2"},{"comment":"The definition of temporal questions includes cases where 'the answer type is temporal', but the detection patterns listed in Section 3.1 are only 'when, what date, in what year, which century'; it would be helpful to clarify whether forms such as 'how long' or 'since when' are also treated as temporal answer questions.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a competently executed systems contribution with public code and data, and the paired-t-test evaluation gives reasonable support for the average F1 gains. My main reservation is that the headline generalizability claims outrun the evidence: the decomposition rules are tested almost exclusively on the authors' own benchmark and assume a single surface word order, and no component-level ablations isolate the source of the gains. I see these as fixable with additional experiments and a more carefully scoped set of claims, not as fatal flaws. I would encourage the editor to request a revision that adds syntactic-variant tests and a quantitative error breakdown."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBottom line: this is a competent engineering contribution, and the central empirical claim holds up better than the stress-test note suggests. TEQUILA is the first complete temporal-QA pipeline built as a wrapper around existing KB-QA engines, and the combination of question decomposition plus Allen-style interval constraint reasoning is genuinely new in this context. Credit where due: the code and data are public, they evaluate on two benchmarks, one of which (ComplexQuestions) is independent of the authors, and they report paired t-tests. The error analysis is also candid, listing decomposition failures as a real weakness rather than hiding them.\n\nNow the soft spots. The Table 1 patterns all assume the signal word appears after the wh-phrase with the temporal constraint following it. Fronted or embedded temporal clauses, like 'Before joining PSG, which teams did Neymar play for?', are untested. That does not sink the paper, but it directly limits the abstract's 'any KB-QA engine' promise: the wrapper is demonstrated for one syntactic order, not for temporal questions in general. The TempQuestions benchmark is the authors' own, which raises selection bias concerns, but the independent ComplexQuestions results largely mitigate that worry. The absence of component-level ablations is a real gap, though not a fatal one; the overall F1 gains are consistent across two different base systems, which is decent evidence the pipeline itself helps.\n\nThe stress-test note is right to flag the decomposition completeness issue, but I would not call it load-bearing in a way that invalidates the paper. The rules are explicit and the failure modes are observable. This is a scoped method with honest reporting, not a disguised fit.\n\nWho gets value from this: researchers working on complex KB-QA or temporal information access, and people building modular QA systems. It is a reasonable reading-group pick, but not a paradigm-shifter. My recommendation: send it to serious peer review. The right revision would add experiments on varied clause positions and at least one component ablation, or explicitly narrow the claimed scope to the tested pattern. The core result is worth publishing.\n\nWould cite? Yes, if I were working on temporal QA.\n\nBest,\n[You]","headline":"A solid, reusable wrapper for temporal KB-QA with honest experiments, but the 'any engine' claim overreaches the tested syntactic coverage.","tokens_in":9852,"tokens_out":1224,"would_cite":true,"duration_ms":15996,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TEQUILA's decomposition-and-reasoning pipeline lets existing KB-QA engines answer composite temporal questions, with significant F1 gains.","keywords":["temporal question answering","question decomposition","knowledge base question answering","temporal reasoning","interval constraints","TempQuestions","time scopes"],"falsifier":"Take a temporal question with two time constraints in one sentence, such as 'Which teams did Neymar play for between leaving Barcelona and joining PSG?', run it through TEQUILA, and check whether it can be answered. The four patterns in Table 1 are built for a single signal word, so a correct answer to such a question would contradict the paper's implicit scope; a failure would show the decomposition layer, not the reasoning, is the limiting factor.","tokens_in":8904,"feed_emoji":"⏳","tokens_out":5299,"duration_ms":47108,"temperature":0.7,"pith_summary":"TEQUILA is a plug-in pipeline that lets ordinary knowledge-base question answering (KB-QA) systems handle composite temporal questions, the kind that contain 'before', 'after', 'during', or dates. The paper argues that a composite temporal question can be split at the time-signal word into a non-temporal sub-question and a temporal one, answered separately, and then recombined by interval reasoning over the retrieved dates. Wrapped around two existing systems, AQQU and QUINT, TEQUILA raises F1 on the TempQuestions benchmark from 27.2 to 36.7 and from 30.0 to 34.0, respectively, and beats a state-of-the-art complex-QA baseline on the temporal slice of ComplexQuestions. The paper's point is that temporal competence can be added to any KB-QA engine without modifying the engine itself.","feed_headline":"A four-stage wrapper lets KB-QA engines handle time questions","feed_subtitle":"TEQUILA splits each question at the time cue, answers the pieces with the base engine, then combines results via interval reasoning.","key_machinery":"The central mechanism is the lexico-syntactic decomposition rule set: four hand-written patterns that split a temporal question into a non-temporal sub-question and a temporal sub-question, borrowing the entity or relation from the first part when the second lacks it. Around this sit two supporting mechanisms: the mapping of the question's signal word to an Allen-style interval constraint (before becomes candidate start $\\leq$ constraint end, during becomes an overlap test), and the retrieval of time scopes for candidate answers by comparing predicate-name embeddings, e.g., choosing team.joinDate as the temporal partner of team.player. These rules convert a linguistic problem into a set-theoretic intersection plus interval arithmetic.","core_discovery":"On the paper's own terms, the discovery is that complex temporal questions over knowledge bases are answerable by a fixed four-stage recipe: detect temporal intent, decompose at a signal word into sub-questions, retrieve answers and time scopes from an off-the-shelf KB-QA engine, and combine them with interval-based constraints such as BEFORE, AFTER, and OVERLAP. The result is that two simple-question engines, AQQU and QUINT, become competitive on temporal questions, with statistically significant F1 improvements over their standalone versions, and the combined system exceeds a general complex-question baseline (Bao et al.) on the temporal slice of ComplexQuestions. Error analysis shows failures trace to wrong decomposition, wrong temporal predicate selection, or wrong constraint, not to the underlying engine's inability to answer simple sub-questions.","pith_inferences":["If the decomposition patterns were extended to handle two or more signal words in one question (e.g., 'between leaving Barcelona and joining PSG'), the same interval-reasoning stage would need only a conjunction of constraints; the paper does not explore this, but nothing in the design blocks it.","The predicate-similarity step currently chooses one temporal predicate from the KB by embedding cosine; a testable extension is to make this choice jointly with the decomposition, since a wrong split can invalidate even a good predicate match.","Because the pipeline is engine-agnostic, the same wrapper could be retrofitted to neural KB-QA models as a preprocessing and post-processing layer, which might give neural engines a cheaper path to temporal competence than training on temporal examples."],"forward_implications":["Any current simple-question KB-QA engine can be wrapped with TEQUILA and gain temporal competence, without retraining or changing its internal query generator.","Composite temporal QA becomes a modular pipeline: question decomposition and interval reasoning are separable from entity linking and relation mapping, each improvable independently.","Error analysis suggests the next bottleneck is robust decomposition and predicate selection, not the underlying engine's simple-question answering.","The approach extends to ordinal constraints ('first', 'last') by sorting candidate intervals rather than merely filtering them."],"supporting_citations":[{"why":"Supplies the TempQuestions benchmark of 1,271 labeled temporal questions used for the main evaluation.","marker":"[13]"},{"why":"QUINT, one of the two underlying KB-QA engines that TEQUILA wraps.","marker":"[2]"},{"why":"AQQU, the other underlying KB-QA engine wrapped by TEQUILA.","marker":"[6]"},{"why":"Bao et al., the state-of-the-art complex-QA baseline that TEQUILA-enabled systems are compared against and beat on temporal questions.","marker":"[4]"},{"why":"Allen's interval relations that define the BEFORE, OVERLAP and other constraints used in the reasoning stage.","marker":"[3]"},{"why":"HeidelTime, used to tag and normalize temporal expressions when detecting temporal intent.","marker":"[22]"},{"why":"Supplies the dictionary of signal words and ordinals for detecting temporal cues.","marker":"[21]"},{"why":"The embedding model used to compute predicate-name similarity when selecting the temporal predicate for candidate time scopes.","marker":"[24]"}],"fun_headline_variants":["TEQUILA: a wrapper that makes KB-QA engines handle time","Decompose at time cue, retrieve, then reason intervals: TEQUILA","Temporal QA on any engine: TEQUILA's four-stage fix","Interval constraints combine sub-answers for temporal QA","TEQUILA improves two engines on temporal questions via decomposition"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the four hand-written decomposition templates being the right and complete way to split temporal questions, and on the part-of-speech labeling being accurate; if a question's surface form falls outside the templates or a word like 'landing' is misread as a noun, the split is wrong and the final answer is wrong.","fun_headline_variants_meta":{"raw":{"variants":["TEQUILA: a wrapper that makes KB-QA engines handle time","Decompose at time cue, retrieve, then reason intervals: TEQUILA","Temporal QA on any engine: TEQUILA's four-stage fix","Interval constraints combine sub-answers for temporal QA","TEQUILA improves two engines on temporal questions via decomposition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000248,"raw_usage":{"total_tokens":1497,"prompt_tokens":846,"completion_tokens":651,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":557}},"tokens_in":462,"tokens_out":651,"duration_ms":6820,"temperature":1.0,"reasoning_tokens":557,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:06:45.433215+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a temporal question with two time constraints in one sentence, such as 'Which teams did Neymar play for between leaving Barcelona and joining PSG?', run it through TEQUILA, and check whether it can be answered. The four patterns in Table 1 are built for a single signal word, so a correct answer to such a question would contradict the paper's implicit scope; a failure would show the decomposition layer, not the reasoning, is the limiting factor.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the TempQuestions benchmark of 1,271 labeled temporal questions used for the main evaluation."},{"cited_title":"Abujabal, M","cited_arxiv_id":null,"evidence_quote":"QUINT, one of the two underlying KB-QA engines that TEQUILA wraps."},{"cited_title":"Bast and E","cited_arxiv_id":null,"evidence_quote":"AQQU, the other underlying KB-QA engine wrapped by TEQUILA."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Bao et al., the state-of-the-art complex-QA baseline that TEQUILA-enabled systems are compared against and beat on temporal questions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Allen's interval relations that define the BEFORE, OVERLAP and other constraints used in the reasoning stage."},{"cited_title":"Strötgen and M","cited_arxiv_id":null,"evidence_quote":"HeidelTime, used to tag and normalize temporal expressions when detecting temporal intent."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the dictionary of signal words and ordinals for detecting temporal cues."},{"cited_title":"Wieting, M","cited_arxiv_id":null,"evidence_quote":"The embedding model used to compute predicate-name similarity when selecting the temporal predicate for candidate time scopes."}],"review_version":1}