Pith. sign in

REVIEW 4 major objections 3 minor 41 references

Wikipedia's text, tables, and knowledge graph disagree often, and a new pipeline can automatically detect and classify those disagreements.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:59 UTC pith:SHDYEGWY

load-bearing objection New task formalization and a genuinely honest limitations section, but the headline 61.4% inconsistency number mixes Text-to-SPARQL noise with real knowledge conflicts, so the central claim is softer than it looks. the 4 major comments →

arxiv 2607.25959 v2 pith:SHDYEGWY submitted 2026-07-28 cs.CL cs.AI

Detecting Knowledge Inconsistencies Across Text, Tables, and Knowledge Graphs

Fanfu Wei , Thibault Ehrhart , Rapha\"el Troncy This is my paper
classification cs.CL cs.AI
keywords knowledge inconsistencymodality-level inconsistency detectiontext-to-SPARQLknowledge graphstable question answeringWikidataWikipediaLLM reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that the same fact, looked up in Wikipedia text, a Wikipedia table, or the Wikidata knowledge graph, frequently yields different answers, and that these differences are not random noise but structured, classifiable knowledge inconsistencies. The authors propose a task called modality-level inconsistency detection and build a pipeline, Kontrast, that turns table-based questions into SPARQL queries against Wikidata, compares the returned graph answers with the table-grounded answers, and labels the mismatch according to a taxonomy of four inconsistency types: granularity differences, direct contradictions, temporal changes, and missing knowledge-graph structure. They assemble a 2,870-question benchmark from existing table-QA datasets and report that, with their best model, over 60 percent of cases where the graph returns a value are not fully aligned with the table answer. The paper's central, forward-looking claim is that text, tables, and knowledge graphs can be used to audit and correct one another at scale, rather than being treated as isolated sources.

Core claim

The paper's central discovery is that cross-modal knowledge inconsistencies between Wikipedia table-based evidence and Wikidata are common, measurable at scale, and can be automatically categorized. Using the Kontrast framework with Text-to-SPARQL translation, the authors find that a majority of value-bearing question-answer pairs show some mismatch between the table and the knowledge graph. They propose a taxonomy distinguishing 'same answer,' 'higher accuracy in KG than in table,' 'higher accuracy in table than in KG,' 'different answer,' 'temporal changes,' and three structural knowledge-graph gaps (missing edge, missing node, missing property or qualifier). They also report that the larg

What carries the argument

The load-bearing mechanism is the Text-to-SPARQL bridge: a zero-shot LLM-based query generator (GRASP) that turns each natural-language table-QA question into an executable SPARQL query against the Wikidata knowledge graph, running on a fixed 2025-05-10 dump for reproducibility. The resulting KG answer is compared with the table-based answer through a two-stage categorization pipeline: first rule-based matching with normalization and SBERT similarity (labeling clear same or granularity cases), then an LLM-as-a-judge with a 15-shot prompt that assigns the remaining cases to the taxonomy's labels. This pipeline converts the abstract idea of 'do two modalities agree?' into an operational, large

Load-bearing premise

The whole measurement depends on the assumption that an executable, value-bearing SPARQL query generated by GRASP faithfully encodes the information need expressed in the natural-language question, so that any mismatch between the KG answer and the table answer really reflects a property of the knowledge sources rather than a translation artifact — an assumption the paper itself concedes is often violated.

What would settle it

Run a manual human annotation of a random sample of, say, 200 value-bearing 'different answer' and structural-gap cases from the paper's Analysis Set, asking annotators to check the original Wikipedia table, the Wikidata item pages, and the raw SPARQL query, and count how many mismatches persist after correcting for query translation errors; if fewer than half of the non-'same' labels survive that correction, the claim that cross-modal inconsistencies are measurable at scale is substantially overstated.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If cross-modal inconsistencies are as common as reported, then any system that mixes Wikipedia tables with Wikidata — including retrieval-augmented generation and knowledge-grounded QA — must treat the two sources as complementary rather than redundant evidence.
  • The taxonomy gives knowledge-graph maintainers actionable signals: granularity differences indicate which source to enrich, temporal conflicts require snapshot-aware verification, and missing edges, nodes, properties, and qualifiers point to concrete completion and schema-refinement targets.
  • A reproducible benchmark of 2,870 table-QA questions with a fixed Wikidata dump allows future inconsistency-detection systems to be compared without confounds from live knowledge-graph updates.
  • The finding that larger reasoning models produce more reliable KG answers and fewer translation-induced mismatches suggests that improving Text-to-SPARQL quality directly improves the fidelity of inconsistency analysis.
  • Excluding the mixed 'different answer' bucket, roughly a fifth of value-bearing cases fall into interpretable categories (granularity differences and temporal shifts), meaning that the pipeline already isolates a meaningful, actionable subset without human review.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to run the same pipeline across multiple Wikipedia snapshots or Wikidata dumps over time, which could turn the 'temporal changes' category from a passive observation into a tool for detecting when the graph or the table lag behind reality.
  • The authors' own results suggest that many 'different answer' cases are translation errors, so a plausible next step is to build a self-consistency check that executes multiple paraphrased SPARQL variants of the same question and treats only disagreements that persist across variants as genuine knowledge conflicts.
  • One could apply the Kontrast approach in the reverse direction: use table-text-wikipedia agreements as a validation signal for KG completeness, effectively creating a continuous knowledge-audit loop rather than a one-shot comparison.
  • The reported sensitivity to question naturalness implies that extending the benchmark with questions written specifically to be answerable by both modalities — rather than inherited from table-only or text-only QA datasets — would provide a cleaner estimate of the true underlying inconsistency rate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper introduces modality-level inconsistency detection: given a question answerable from Wikipedia tables/text and from Wikidata, it compares the two answers and categorizes mismatches. It proposes a taxonomy (Same, Higher accuracy in KG/Table, Different answer, Temporal changes, Missing edge, Missing node, Missing property/qualifier) and builds Kontrast, a pipeline that translates questions to SPARQL with GRASP, executes them on a fixed Wikidata snapshot, and compares table-derived answers with KG answers using heuristics and an LLM judge. Experiments span 2,870 questions from six QA datasets and three Qwen3 backbones. The paper reports inconsistency rates of 61.4%–68.4% among value-bearing cases and concludes that cross-modal inconsistencies are measurable at scale and offer actionable signals for KG completion and knowledge reconciliation.

Significance. If the central claim were cleanly established, the paper would be a useful contribution: it operationalizes a largely unexplored comparison between Wikipedia tables/text and Wikidata, provides a reproducible snapshot and dataset, and its taxonomy of inconsistency types is intuitive and potentially actionable for KG auditing. The paper is also commendably candid about limitations. However, the headline quantitative claim is not currently supported because the main inconsistency rate is computed over a category that the paper itself says mixes genuine knowledge conflicts with Text-to-SPARQL translation errors and missing-qualifier fallbacks. The significance of the work therefore depends on whether this conflation can be corrected—for example, by estimating or filtering translation noise—and on whether the structural incompleteness categories are validated at all.

major comments (4)
  1. [Conclusion, §5.3, §5.6] The headline claim—"With Qwen3-235B-Thinking, 61.4% of value-bearing cases are not fully aligned with the table answer" (Conclusion)—is presented as evidence that cross-modal knowledge inconsistencies are measurable at scale. But Table 4's 'Different answer' category, which is the dominant contributor to the 61.4% figure, is defined in §5.3 as covering '(i) true cross-modal knowledge conflicts; (ii) Text-to-SPARQL translation noise where the model selects an incorrect entity or property; and (iii) missing property or qualifier cases.' §5.6 further states that 'automatic execution signals are insufficient to distinguish true KG incompleteness from semantic parsing errors.' Therefore the reported inconsistency rate conflates knowledge-conflict evidence with pipeline error. The claim that inconsistencies are measurable at scale is not established unless the noise contribution is estimated o
  2. [§5.5] The human evaluation only checks whether the assigned taxonomy label correctly describes the relation between the table answer and the KG answer. It does not verify whether the KG answer actually answers the original question. Consequently, the 100% label-accuracy result does not imply that the detected mismatches are genuine knowledge inconsistencies; it only implies that, conditional on the KG output being relevant, the label is reasonable. A validation of the Text-to-SPARQL fidelity for value-bearing cases—e.g., human assessment of whether the KG answer satisfies the question's information need—is needed before the inconsistency rates can be read as knowledge-level measurements.
  3. [§3, Future Work] The taxonomy was derived by manually inspecting the Wikipedia pages from which the test questions were constructed and comparing them with corresponding Wikidata items (§3). Because the same sources are then used to evaluate Kontrast, the taxonomy is in effect fit to the evaluation data; this risks overstating the generality of the proposed categories. More importantly, the structural incompleteness categories (Missing edge, Missing node, Missing property/qualifier) receive no automatic evaluation in the main results: Table 4 omits them from the distribution, and the future-work paragraph acknowledges that extending human annotation to these labels is still needed. As written, the paper does not demonstrate that these categories can be detected reliably at scale.
  4. [§5.3, Table 4] The paper defines the Analysis Set by excluding outputs with more than 10 rows and duplicate questions, and reports percentages 'among value-bearing cases.' It is not made clear how many of the excluded cases were value-bearing or how the row filter interacts with the inconsistency-rate estimate. Because the filter is applied after SPARQL execution and is not justified by any claim about knowledge conflicts (only about serialization limits), the reported 61.4% may be sensitive to this arbitrary threshold. At minimum, the authors should report the denominator change and provide a sensitivity analysis across row-limit choices.
minor comments (3)
  1. [Abstract/§5] The abstract and introduction refer to inconsistencies across 'text, tables, and KGs,' but the experimental setting compares table-based answers (from table-QA datasets) with KG answers. Textual evidence enters only as part of the table-based QA construction. Consider refining the wording to avoid overstating the modality coverage.
  2. [Table 3] The 'Analysis Set' row uses a different meaning of n (question–answer pairs rather than evaluated cases) and reports Exec. = 100.0 by construction. This is potentially confusing; a footnote clarifying the row's purpose would help.
  3. [§4.3] The Same-label alignment threshold of 0.95 and the 'row filter' are described as chosen through iterative human inspection and practical serialization constraints. These choices should be documented as tunable parameters, since the headline rates may depend on them.

Circularity Check

2 steps flagged

The 61.4% headline counts Text-to-SPARQL noise as knowledge inconsistency, and the taxonomy is derived from the same data it is used to measure.

specific steps
  1. self definitional [Section 3, Taxonomy of Knowledge Inconsistency, first paragraph]
    "We derived the taxonomy by inspecting the Wikipedia pages from which the table-based questions were constructed and manually comparing their answers with the corresponding Wikidata item pages. This process revealed recurring inconsistency patterns, which we refined by examining their causes and relating them to prior work [15]."

    The taxonomy is defined by inspecting the exact Wikipedia/Wikidata pages used to build the test questions. The paper then reports the distribution of those newly defined labels on those same questions (Table 4) and uses it to claim that inconsistencies are common and measurable. The prevalence numbers are therefore a restatement of the inspection that created the categories, not an independent first-principles measurement. Section 5.5 only checks that an annotator agrees with the pipeline's label assignment, so there is no external gold-standard label set to break the loop.

  2. fitted input called prediction [Section 4.3 (heuristic threshold), Section 5.3 (Different answer), Section 6 (Conclusion)]
    "This threshold was chosen through iterative human inspection of borderline cases, preserving clear matches while separating cases with missing or extra answers. ... Different answer is the largest category, covering (i) true cross-modal knowledge conflicts; (ii) Text-to-SPARQL translation noise where the model selects an incorrect entity or property; and (iii) missing property or qualifier cases (schema level), where the Text-to-SPARQL model falls back to the closest available KG entities. ... With Qwen3-235B-Thinking, 61.4% of value-bearing cases are not fully aligned with the table answer."

    The conclusion's central quantitative claim is the non-Same rate, and the dominant non-Same bucket is explicitly defined to include the system's own Text-to-SPARQL translation noise. The 61.4% is thus partly a measure of the translator's errors, not of knowledge inconsistency between Wikipedia and Wikidata. Section 5.6 concedes that automatic execution signals are insufficient to distinguish true KG incompleteness from semantic parsing errors. Because the comparison threshold was also tuned on the same instances, the headline claim that inconsistencies are measurable at scale is a fitted pipeline statistic rather than an independent measurement of the knowledge sources.

full rationale

No load-bearing self-citation chain or imported uniqueness theorem is present; the paper's self-citations (refs. [10] and [36]) are peripheral. The main tool and benchmark retain independent content. However, the headline conclusion is partly circular in two ways. First, the taxonomy is derived by manual inspection of the same Wikipedia pages and Wikidata items that generated the test questions, so reporting the distribution of those labels on the same data is not an independent discovery. Second, the 61.4% 'not fully aligned' figure counts the 'Different answer' category, which the paper itself defines to include Text-to-SPARQL translation noise and schema-level fallback behavior; Section 5.6 explicitly admits that execution signals cannot separate true KG incompleteness from semantic parsing errors. The human evaluation also validates only whether the label describes the relation between the two outputs, not whether the KG output actually answers the question. These issues make the central quantitative claim partially self-referential, but the system, taxonomy, and benchmark still have useful independent content, and the paper discloses the main limitations, so a moderate score of 4 is appropriate.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

This is an empirical benchmark paper, so the 'axioms' are domain assumptions about the reliability of table answers, SPARQL translation, taxonomy completeness, and LLM judging. Several of these assumptions are explicitly weakened by the authors themselves in Section 5.6.

free parameters (3)
  • Same-label alignment threshold = 0.95
    An instance is labeled 'Same' only when both precision and recall are at least 0.95 after normalized matching/SBERT; this threshold was chosen through iterative human inspection of borderline cases (Section 4.3), and it directly controls the headline Same vs inconsistent split.
  • Row filter for Analysis Set = 10 rows
    Outputs with more than 10 SPARQL rows are excluded because GRASP serializes only the first and last five rows (Section 5.2); this removes 3.3-8.7% of cases and shapes the analyzed distribution, yet it is a hand-chosen cutoff.
  • Judge prompt in-context examples = 15 (5 labels x 3 examples)
    The LLM-as-a-judge prompt includes three in-context examples per taxonomy label (Section 4.3). The particular examples are hand-selected from the same data and are not published in the paper, which makes the judge's behavior partly fit to the evaluation set.
axioms (4)
  • domain assumption The table-based answer in the source QA datasets is correct Wikipedia-derived evidence.
    Kontrast always compares the KG output against the table answer and labels mismatches relative to it (Sections 4.2, 4.3); if the table answer is wrong or under-specified, the resulting labels are wrong.
  • domain assumption A non-empty executable SPARQL query faithfully answers the natural-language question.
    The pipeline treats the KG answer as evidence for the same information need, but Section 5.6 concedes that wrong entity/property selection produces valid queries with spurious mismatches; the claim that inconsistencies are informative depends on this assumption holding often.
  • ad hoc to paper The taxonomy derived from manual inspection of the source pages is complete and applicable to all test instances.
    Section 3 says the taxonomy was derived by inspecting the Wikipedia pages used to construct the questions; this makes the taxonomy fit to the evaluation set rather than independently motivated, and the structural labels are not actually detected in experiments.
  • domain assumption LLM-as-judge labels are reliable for ambiguous cases not resolved by heuristics.
    The LLM judge's reliability is supported only by 50 self-selected cases checked by one annotator (Section 5.5); there is no inter-annotator agreement and no held-out gold label set.

pith-pipeline@v1.3.0-alltime-deepseek · 14821 in / 12695 out tokens · 118150 ms · 2026-08-01T00:59:07.715002+00:00 · methodology

0 comments
read the original abstract

Wikipedia and Wikidata are widely used for information access, LLM pre-training, and retrieval-augmented generation. Their knowledge is deeply connected but scattered across text, tables, and knowledge graphs. This raises a practical question: when these modalities disagree, how can we detect and explain the conflict? We study this problem as modality-level inconsistency detection. We first introduce a taxonomy of cross-modal knowledge inconsistencies, covering information granularity differences, direct conflicts, temporal changes, and KG incompleteness. We then present Kontrast, an automatic framework that uses Text-to-SPARQL and LLM reasoning to compare table-based answers with KG evidence and categorize the resulting inconsistencies. Experiments on various Table-QA datasets show that cross-modal inconsistencies are common and informative. They reveal not only true knowledge conflicts, but also missing KG structure and temporal mismatches while being limited by Text-to-SPARQL errors and noise. Our analysis shows that text, tables, and KGs can complement and correct one another through systematic comparison. Kontrast provides a practical tool for large-scale knowledge auditing and establishes a benchmark for future work on cross-modal knowledge consistency. Code and data are available at https://github.com/ECLADATTA/KONTRAST.

Figures

Figures reproduced from arXiv: 2607.25959 by Fanfu Wei, Rapha\"el Troncy, Thibault Ehrhart.

Figure 1
Figure 1. Figure 1: Workflow for detecting and categorizing modality-level inconsistencies by com￾paring table and KG answers using a knowledge inconsistency taxonomy. The question defines the information need, and the table-based answer represents the answer derived from Wikipedia tables and text. Text-to-SPARQL translation. A Text-to-SPARQL model translates a natural language question q into a SPARQL query over a KG. This s… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 6 canonical work pages

  1. [2]

    In: 3rd Workshop on NaturalLanguageGeneration,Evaluation,andMetrics.pp.97–110.Associationfor Computational Linguistics, Singapore (Dec 2023),https://aclanthology.org/ 2023.gem-1.9/

    Amouyal, S., Wolfson, T., Rubin, O., Yoran, O., Herzig, J., Berant, J.: QAMPARI: A benchmark for open-domain questions with many answers. In: 3rd Workshop on NaturalLanguageGeneration,Evaluation,andMetrics.pp.97–110.Associationfor Computational Linguistics, Singapore (Dec 2023),https://aclanthology.org/ 2023.gem-1.9/

    2023

  2. [3]

    In: Advances in Neural Information Processing Sys- tems (NeurIPS)

    Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Win- ter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford...

    1901

  3. [4]

    Cattan, A., Jacovi, A., Ram, O., Herzig, J., Aharoni, R., Goldshtein, S., Ofek, E., Szpektor, I., Caciularu, A.: DRAGged into conflicts: Detecting and addressing conflicting sources in search-augmented LLMs (2025),https://arxiv.org/abs/ 2506.08500

    Pith/arXiv arXiv 2025

  4. [5]

    In: 9th International Conference on Learning Rep- resentations (ICLR) (2021)

    Chen, W., Chang, M.W., Schlinger, E., Wang, W., Cohen, W.: Open question answering over tables and text. In: 9th International Conference on Learning Rep- resentations (ICLR) (2021)

    2021

  5. [6]

    In: Findings of the Association for Computational Linguistics: EMNLP

    Chen, W., Zha, H., Chen, Z., Xiong, W., Wang, H., Wang, W.Y.: HybridQA: A dataset of multi-hop question answering over tabular and textual data. In: Findings of the Association for Computational Linguistics: EMNLP. pp. 1026–1036. Associa- tion for Computational Linguistics (2020).https://doi.org/10.18653/v1/2020. findings-emnlp.91,https://aclanthology.org...

    doi:10.18653/v1/2020 2020

  6. [7]

    Cheng, Z., Dong, H., Wang, Z., Jia, R., Guo, J., Gao, Y., Han, S., Lou, J.G., Zhang, D.: HiTab: A hierarchical table dataset for question answering and natural languagegeneration.In:60thAnnualMeetingoftheAssociationforComputational Linguistics (ACL). pp. 1094–1110. Association for Computational Linguistics, Dublin, Ireland (May 2022).https://doi.org/10.18...

    doi:10.18653/v1/2022.acl-long.78 2022

  7. [8]

    In: Companion Proceedings of the ACM Web Confer- ence 2024

    Christmann, P., Saha Roy, R., Weikum, G.: CompMix: A benchmark for hetero- geneous question answering. In: Companion Proceedings of the ACM Web Confer- ence 2024. pp. 1091–1094. Association for Computing Machinery, New York, NY, USA(2024).https://doi.org/10.1145/3589335.3651444,https://doi.org/10. 1145/3589335.3651444

    arXiv 2024

  8. [9]

    In: 24th International Confer- ence on Conceptual Structures (ICCS)

    Djebri, A.E.A., Tettamanzi, A.G.B., Gandon, F.: Publishing uncertainty on the semantic web: Blurring the LOD bubbles. In: 24th International Confer- ence on Conceptual Structures (ICCS). pp. 42–56. Springer, Marburg, Germany (2019).https://doi.org/10.1007/978-3-030-23182-8_4,https://doi.org/10. 1007/978-3-030-23182-8_4

    work page doi:10.1007/978-3-030-23182-8_4 2019

  9. [10]

    In: International Conference on Language Resources and Evaluation (LREC)

    Ettaleb, M., Ehrhart, T., Aussenac-Gilles, N., Chabot, Y., Kamel, M., Moriceau, V., Troncy, R., Wei, F.: ReTaT: A Unified Benchmark for Relation Extraction across Text and Table. In: International Conference on Language Resources and Evaluation (LREC). Mallorca, Spain (2026)

    2026

  10. [11]

    In: Conference on Empirical Methods in Natural Language Process- ing (EMNLP)

    Feng, K., Zhang, S., Chen, B., Zhao, Y., Zhao, C.: SportReason: Evaluating retrieval-augmented reasoning across tables and text for sports question an- swering. In: Conference on Empirical Methods in Natural Language Process- ing (EMNLP). pp. 649–662. Association for Computational Linguistics, Suzhou, China (2025).https://doi.org/10.18653/v1/2025.emnlp-ma...

    work page doi:10.18653/v1/2025.emnlp-main.34 2025

  11. [12]

    Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., Prescod-Weinstein, C., Leahy, C.: The Pile: An 800gb dataset of diverse text for language modeling (2020),https://arxiv.org/ abs/2101.00027

    Pith/arXiv arXiv 2020

  12. [13]

    In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

    Herzig, J., Müller, T., Krichene, S., Eisenschlos, J.: Open domain question answer- ing over tables via dense retrieval. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 512–519. Association for Computational Linguistics (2021).https://doi.org/10. 18653/v1/2021.naacl-main.43,ht...

    2021

  13. [14]

    Hou, Y., Pascale, A., Carnerero-Cano, J., Tchrakian, T., Marinescu, R., Daly, E., Padhi, I., Sattigeri, P.: WikiContradict: A benchmark for evaluating LLMs on real- world knowledge conflicts from Wikipedia. In: Advances in Neural Information Processing Systems 37 (2024).https://doi.org/10.52202/079017-3481, https://proceedings.neurips.cc/paper_files/paper...

    work page doi:10.52202/079017-3481 2024

  14. [15]

    Transactions on Graph Data and Knowledge3(1), 3:1–3:48 (2025).https://doi.org/10.4230/TGDK.3.1.3,https: //drops.dagstuhl.de/entities/document/10.4230/TGDK.3.1.3

    Jarnac, L., Chabot, Y., Couceiro, M.: Uncertainty Management in the Con- struction of Knowledge Graphs: A Survey. Transactions on Graph Data and Knowledge3(1), 3:1–3:48 (2025).https://doi.org/10.4230/TGDK.3.1.3,https: //drops.dagstuhl.de/entities/document/10.4230/TGDK.3.1.3

    doi:10.4230/tgdk.3.1.3 2025

  15. [16]

    In: 13th Knowledge Capture Conference (K-CAP)

    Jarnac, L., et al.: Wikiconflict: A new dataset for conflicting data reconciliation in knowledge graph construction. In: 13th Knowledge Capture Conference (K-CAP). pp. 215–218. Association for Computing Machinery (2025).https://doi.org/10. 1145/3731443.3771371,https://doi.org/10.1145/3731443.3771371

    arXiv 2025

  16. [17]

    In: Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., Yih, W.t.: Dense passage retrieval for open-domain question answering. In: Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 6769–6781. Association for Computational Linguistics, Online (2020). 18 F. Wei et al. https://doi.org/10.18653/v1/2020.emnlp-main.5...

    doi:10.18653/v1/2020.emnlp-main.550 2020

  17. [18]

    Transactions of the Association for Computational Linguistics7, 452–466 (2019).https://doi.org/10.1162/tacl_ a_00276,https://aclanthology.org/Q19-1026/

    Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., Toutanova, K., Jones, L., Kelcey, M., Chang, M.W., Dai, A.M., Uszkoreit, J., Le, Q., Petrov, S.: Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics7,...

    doi:10.1162/tacl_ 2019

  18. [19]

    In: Advances in Neural Information Processing Systems

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems. vol. 33, pp. 9459–9474 (2020),https://proceedings.neurips.cc/paper/2020/hash/ 6b49323020...

    2020

  19. [20]

    In: Findings of the Association for Computational Linguistics: EMNLP

    Liu, S., Semnani, S., Triedman, H., Xu, J., Zhao, I.D., Lam, M.: SPINACH: SPARQL-based information navigation for challenging real-world questions. In: Findings of the Association for Computational Linguistics: EMNLP. pp. 15977–16001. Association for Computational Linguistics, Miami, Florida, USA (2024).https://doi.org/10.18653/v1/2024.findings-emnlp.938,...

    doi:10.18653/v1/2024.findings-emnlp.938 2024

  20. [21]

    In: Findings of the Association for Computational Linguistics: NAACL

    Liu, S., Ning, Q., Halder, K., Qi, Z., Xiao, W., Htut, P.M., Zhang, Y., Anna John, N., Min, B., Benajiba, Y., Roth, D.: Open domain question answering with con- flicting contexts. In: Findings of the Association for Computational Linguistics: NAACL. pp. 1838–1854. Association for Computational Linguistics, Albuquerque, New Mexico (2025).https://doi.org/10...

    doi:10.18653/v1/2025.findings-naacl.99 2025

  21. [22]

    In: Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Longpre, S., Perisetla, K., Chen, A., Ramesh, N., DuBois, C., Singh, S.: Entity- based knowledge conflicts in question answering. In: Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 7052–7063. Associa- tion for Computational Linguistics, Online and Punta Cana, Dominican Re- public (2021).https://doi.org/10.18653/v1/2021.emnlp-m...

    doi:10.18653/v1/2021.emnlp-main.565 2021

  22. [23]

    In: 2nd Work- shop on Uncertainty-Aware NLP

    Nachshoni, E., Cattan, A., Amar, S., Shapira, O., Dagan, I.: Consensus or conflict? fine-grained evaluation of conflicting answers in question-answering. In: 2nd Work- shop on Uncertainty-Aware NLP. Association for Computational Linguistics, Vi- enna, Austria (2025),https://aclanthology.org/2025.uncertainlp-main.13/

    2025

  23. [24]

    Transactions of the Association for Computa- tional Linguistics10, 35–49 (2022).https://doi.org/10.1162/tacl_a_00446, https://aclanthology.org/2022.tacl-1.3/

    Nan, L., Hsieh, C., Mao, Z., Lin, X.V., Verma, N., Zhang, R., Kryściński, W., Schoelkopf, H., Kong, R., Tang, X., Mutuma, M., Rosand, B., Trindade, I., Ban- daru, R., Cunningham, J., Xiong, C., Radev, D., Radev, D.: FeTaQA: Free- form table question answering. Transactions of the Association for Computa- tional Linguistics10, 35–49 (2022).https://doi.org/...

    doi:10.1162/tacl_a_00446 2022

  24. [25]

    In: Conference on Empiri- cal Methods in Natural Language Processing (EMNLP)

    Parikh, A., Wang, X., Gehrmann, S., Faruqui, M., Dhingra, B., Yang, D., Das, D.: ToTTo: A controlled table-to-text generation dataset. In: Conference on Empiri- cal Methods in Natural Language Processing (EMNLP). pp. 1173–1186. Associ- ation for Computational Linguistics, Online (2020).https://doi.org/10.18653/ v1/2020.emnlp-main.89,https://aclanthology.o...

    2020

  25. [26]

    In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing

    Pasupat, P., Liang, P.: Compositional semantic parsing on semi-structured tables. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. pp. 1470– Detecting Cross-Modal Knowledge Inconsistencies 19

  26. [27]

    In: International Semantic Web Conference (ISWC)

    Pérez, J., Arenas, M., Gutierrez, C.: Semantics and complexity of SPARQL. In: International Semantic Web Conference (ISWC). pp. 30–43. Springer (2006)

    2006

  27. [28]

    In: Conference on Empirical Methods in Natural Language Processing (EMNLP) (2019)

    Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert- networks. In: Conference on Empirical Methods in Natural Language Processing (EMNLP) (2019)

    2019

  28. [29]

    In: Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP)

    Semnani, S., Burapacheep, J., Khatua, A., Atchariyachanvanit, T., Wang, Z., Lam, M.: Detecting corpus-level knowledge inconsistencies in Wikipedia with large lan- guage models. In: Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP). pp. 34839–34866. Association for Computational Linguistics, Suzhou, China (2025).https://doi.org/10.18...

    work page doi:10.18653/v1/2025.emnlp-main.1765 2025

  29. [30]

    In: Semantic Web Evaluation Challenge

    Usbeck, R., Ngonga Ngomo, A.C., Haarmann, B., Krithara, A., Röder, M., Napoli- tano, G.: 7th open challenge on question answering over linked data (QALD-7). In: Semantic Web Evaluation Challenge. pp. 59–69. Springer (2017)

    2017

  30. [31]

    Semantic Web (2023)

    Usbeck, R., Yan, X., Perevalov, A., Jiang, L., Schulz, J., Kraft, A., Möller, C., Huang, J., Reineke, J., Ngonga Ngomo, A.C., Saleem, M., Both, A.: QALD-10 – the 10th challenge on question answering over linked data. Semantic Web (2023)

    2023

  31. [32]

    Com- munications of the ACM57(10), 78–85 (2014)

    Vrandečić, D., Krötzsch, M.: Wikidata: A free collaborative knowledgebase. Com- munications of the ACM57(10), 78–85 (2014)

    2014

  32. [33]

    In: 24th International Semantic Web Conference (ISWC)

    Walter, S., Bast, H.: GRASP: Generic reasoning and SPARQL generation across knowledge graphs. In: 24th International Semantic Web Conference (ISWC). pp. 271–289. Springer, Nara, Japan (2025).https://doi.org/10.1007/ 978-3-032-09527-5_15,https://doi.org/10.1007/978-3-032-09527-5_15

    work page doi:10.1007/978-3-032-09527-5_15 2025

  33. [34]

    Wan, A., Wallace, E., Klein, D.: What evidence do language models find con- vincing? In: 62nd Annual Meeting of the Association for Computational Linguis- tics (ACL). pp. 7663–7695. Association for Computational Linguistics, Bangkok, Thailand (2024).https://doi.org/10.18653/v1/2024.acl-long.403,https:// aclanthology.org/2024.acl-long.403/

    work page doi:10.18653/v1/2024.acl-long.403 2024

  34. [35]

    In: 2nd Conference on Language Modeling (2025), https://openreview.net/forum?id=z1MHB2m3V9

    Wang, H., Prasad, A., Stengel-Eskin, E., Bansal, M.: Retrieval-augmented gener- ation with conflicting evidence. In: 2nd Conference on Language Modeling (2025), https://openreview.net/forum?id=z1MHB2m3V9

    2025

  35. [36]

    In: International Workshop on Extraction from Triplet Text-Table-Knowledge Graph (TRIPLET) (2026)

    Wei, F., Ehrhart, T., Troncy, R.: From Rows to Narratives: Benchmarking Se- mantic Relatedness Across Tables and Paragraphs. In: International Workshop on Extraction from Triplet Text-Table-Knowledge Graph (TRIPLET) (2026)

    2026

  36. [37]

    Transactions of the Association for Com- putational Linguistics14, 23–46 (2026).https://doi.org/10.1162/tacl.a.64, https://aclanthology.org/2026.tacl-1.2/

    Wolfson, T., Trivedi, H., Geva, M., Goldberg, Y., Roth, D., Khot, T., Sabhar- wal, A., Tsarfaty, R.: MoNaCo: More natural and complex questions for rea- soning across dozens of documents. Transactions of the Association for Com- putational Linguistics14, 23–46 (2026).https://doi.org/10.1162/tacl.a.64, https://aclanthology.org/2026.tacl-1.2/

    doi:10.1162/tacl.a.64 2026

  37. [38]

    In: Conference on Empirical Methods in Natu- ral Language Processing (2024)

    Xu, R., Qi, Z., Guo, Z., Wang, C., Wang, H., Zhang, Y., Xu, W.: Knowledge conflicts for LLMs: A survey. In: Conference on Empirical Methods in Natu- ral Language Processing (2024). pp. 8541–8565. Association for Computational Linguistics, Miami, Florida, USA (2024).https://doi.org/10.18653/v1/2024. emnlp-main.486,https://aclanthology.org/2024.emnlp-main.486/

    doi:10.18653/v1/2024 2024

  38. [39]

    In: Conference on Empirical Methods in Natural Language Processing (EMNLP)

    Xu, S., Liu, S., Culhane, T., Pertseva, E., Wu, M.H., Semnani, S., Lam, M.: Fine- tuned LLMs know more, hallucinate less with few-shot sequence-to-sequence se- mantic parsing over Wikidata. In: Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 5778–5791. Association for Computational 20 F. Wei et al. Linguistics, Singapore (2023)...

    doi:10.18653/v1/2023.emnlp-main 2023

  39. [40]

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li, M., Xue, M., Li, M., Zhang, P., Wang, P., Zhu, Q...

    Pith/arXiv arXiv 2025

  40. [41]

    In: 53rd Annual Meet- ing of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing

    Yih, W.t., Chang, M.W., He, X., Gao, J.: Semantic parsing via staged query graph generation: Question answering with knowledge base. In: 53rd Annual Meet- ing of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. pp. 1321–1331. Association for Computational Linguistics, Beijing, China (...

    doi:10.3115/v1/ 2015

  41. [1480]

    Association for Computational Linguistics, Beijing, China (2015).https: //doi.org/10.3115/v1/P15-1142,https://aclanthology.org/P15-1142/

    doi:10.3115/v1/p15-1142 2015