Pith. sign in

REVIEW 4 major objections 5 minor 59 references

Graph Repairs with Large Language Models: An Empirical Study

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that LLMs can propose repairs that remove property-graph inconsistencies with moderate validity but low exact accuracy, making them useful as suggestion tools rather than autonomous repairers.

desk verdict A useful, reproducible empirical study undercut by a trivially satisfiable validity metric; the real result is that LLMs repair below a random baseline. read the letter →

arxiv 2507.03410 v1 pith:6CIDCHNE submitted 2025-07-04 cs.CL cs.DBcs.ET

Hrishikesh Terdalkar , Angela Bonifati , Andrea Mauri This is my paper
classification cs.CLcs.DBcs.ET
keywords largelanguagemodelspropertygraphsgraphrepairdenialconstraintspromptengineeringfew-shotlearningempiricalevaluationsyntheticmedicaldata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can take over the repair of property graphs—graphs whose nodes and edges carry labels and attributes—when the graphs violate integrity constraints. The authors inject known errors into a synthetic medical graph, encode each violation as text, prompt six open-source LLMs to emit structured repair operations, and score the output on format adherence, validity, and exact accuracy. They find that with a favorable prompt choice, models follow the requested output format 72–96% of the time and remove the inconsistency with 42–74% validity, but exact matches to the known correct repair peak at 38%. The paper's conclusion is that LLMs are not yet reliable enough for autonomous graph repair, although they could serve as suggestion tools inside a human-in-the-loop or hybrid pipeline.

What carries the argument

The load-bearing object is the graph denial constraint (GDC) violation triangle: a constraint $\phi = (Q[\bar{x}], X \to Y)$ such as 'a patient must not be treated with a medicine that contains an ingredient the patient is allergic to,' detected by a Cypher query whose every match is a violation. The matched subgraph is a triangle of three edges—TAKES_MEDICATION, HAS_INGREDIENT, and ALLERGIC_TO—so deleting any one of the three removes the inconsistency; the ground-truth repair is deletion of whichever edge the authors corrupted (the allergy edge if the ingredient is real, the ingredient edge if a wrong ingredient was introduced, with the medicine-taking edge never the correct fix). The pipeline feeds this triangle to LLMs under three encodings (node-edge list, human-authored template text, or LLM-generated description) and five few-shot example modes, then scores format adherence, validity, and exact accuracy.

What would settle it

Run the trivial baseline that always deletes one edge of the inconsistency triangle (for example the HAS_INGREDIENT edge) across the 165 injected violations and compute its validity score; if it matches or exceeds the reported 42–74% validity for LLMs, then validity is not evidence of repair capability and the study's positive result reduces to the at-most-38% accuracy numbers.

Watch

Extended reading notes

Core claim

On its own terms, the paper discovers that LLMs can read a property-graph inconsistency, follow structured instructions, and produce edits that eliminate the violation with moderate frequency, but they rarely reproduce the ground-truth repair. With favorable prompt choices, format adherence $F$ ranges from 72% to 96%, validity $V$ from 42% to 74%, and exact accuracy $A$ reaches at most 38%. The best-performing model in the study, deepseek-r1, reaches $A = 0.38$ in the template encoding while being the slowest and most verbose option; in several setups models achieve $V$ near 1.0 with $A$ at 0.0 because they delete one edge of the violation triangle rather than the specific edge that was corrupted. The authors take this as evidence that the current role for LLMs in graph repair is to suggest candidate repairs that a human or rule-based layer must verify.

Load-bearing premise

The evaluation assumes that any repair that removes the inconsistency counts as a meaningful 'valid' success, but every violation in the dataset is a triangle of three edges, so simply deleting one edge always succeeds—and the paper never runs that trivial baseline.

Editorial extensions

If this is right

  • In an interactive repair system, LLMs can draft candidate repairs and flag likely inconsistencies, but a human or rule-based filter must verify each edit before it is applied.
  • Repair quality depends heavily on prompt setup: template-based natural-language encodings with one or two worked output examples are the most reliable settings across models.
  • Model choice is a cost–quality trade-off: deepseek-r1 gives the best accuracy at roughly eleven seconds and about 479 output tokens per repair, while phi4 and gemma2 can reach comparable results at much lower cost in specific setups.
  • Format adherence is strong, so the bottleneck is not the models' ability to follow structured output syntax but their ability to select factually correct edits.
  • The main failure modes are excessive repair operations, indecision that offers multiple alternatives instead of one repair, and hallucinated updates to unrelated nodes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication the paper leaves implicit is that the ~38% accuracy ceiling comes from a small synthetic testbed with clean ground truth; in real-world graphs with many constraints and no labelled correct repair, exact-accuracy evaluation is even harder, so deployment would need a different yardstick.
  • Because validity is trivially attainable by deleting any one edge of the triangle, the informative signal in this study is exact accuracy, and future evaluations should compare LLMs against a trivial one-edge-deletion baseline.
  • The observed 'valid but wrong' repairs (for example deleting the TAKES_MEDICATION edge) suggest a testable extension: add a prompt or post-processing constraint that forbids deleting high-information edges, and measure whether accuracy rises without hurting validity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper reports an empirical study of six open-source LLMs (LLaMA 3.2, Mistral, Phi-4, Gemma 2, Qwen 2.5, and DeepSeek R1) applied to repairing inconsistencies in property graphs. Inconsistencies are instances of a graph denial constraint — a patient must not take a medication containing an ingredient to which they are allergic — injected into a Synthea-derived medical graph, yielding 165 violations. The authors vary the graph encoding (node-edge, template, LLM-generated), the few-shot example mode (none, 1-small, 2-small, 1-large, 2-mixed), and evaluate each model on format adherence (F), repair validity (V, elimination of the violation), repair accuracy (A, exact match to a known ground-truth repair), and computational cost. The headline findings are that models adhere to the requested output format with high success (72%-96%), produce 'valid' repairs with moderate success (42%-74%), but achieve low accuracy (best A=0.38, deepseek-r1 in template mode). The paper concludes that LLM-based graph repair is not yet ready for autonomous deployment and that human-in-the-loop or hybrid approaches are needed.

Significance. If the validity result were meaningful, this would be a useful empirical contribution to a relatively unexplored task: it maps a prompt-design space (three encodings times five example modes) across six open-weight models on a reproducible synthetic dataset, and it ships its code. The honest reporting of low accuracy and the failure taxonomy (eager generation, indecision, hallucination, Section 5.4) are genuinely useful for practitioners. The main problem is that the central positive claim — 'moderate success' in producing valid repairs — rests on a validity metric that is trivially satisfiable by deleting any one of the three edges of the inconsistency triangle, with no trivial baseline reported. Several table cells show V between 0.83 and 1.00 while A=0.00, and Section 5.4 explicitly describes the trivial TAKES-edge deletion as the dominant failure mode. Once V is discounted, the paper's positive finding reduces to format adherence, and its accuracy numbers are below what a random edge-deletion baseline would plausibly achieve.

major comments (4)
  1. [§4.3.1 / §3.1.1 / Figure 1] The validity metric V is trivially satisfiable, and no trivial baseline is reported. The denial-constraint pattern is a triangle (p)-[:TAKES]->(m)-[:HAS_INGREDIENT]->(i)-[:ALLERGIC_TO]->(p), and the paper's own Figure 1 caption acknowledges that 'the inconsistency may be repaired in several ways, including but not limited to, the removal of one of the three edges in red.' Because V=1 for any repair that eliminates the violation, a baseline that always deletes the TAKES_MEDICATION edge would score V=1.0. Section 5.4 reports precisely this behavior for phi4 ('suggesting the removal of an incorrect edge, thereby eliminating the information that a person takes a medication'), yet phi4 simultaneously achieves V=0.96 (Table 6, none row) and V=0.83 (Table 4, Template row) with A=0.00; mistral shows the same pattern (Table 4, Template row: V=0.98, A=0.00), as do gemma2 (V=1.00, A=0.00, Table 6, 2-small row) and qwen2.5 (V=0.99, A=0.00, Table 6, 1-large row). The Section 1 claim of 'moderate success (42%-74%)' in producing valid repairs is therefore forced by the definition of V rather than by model capability. The authors should recompute V excluding TAKES-edge deletions, report V for non-trivial repairs, and add delete-one-edge baselines (always-TAKES, always-ALLERGIC_TO, always-HAS_INGREDIENT, and random single-edge deletion) for comparison.
  2. [§4.1 / Tables 4 and 6] The accuracy results are not interpretable without baselines and without the ground-truth class balance. Section 4.1 states that the correct repair is always the deletion of one of two edges — ALLERGIC_TO when the ingredient is real, HAS_INGREDIENT when the ingredient was wrongly introduced — but the paper never reports the proportion of the 165 inconsistencies falling in each class. Under a balanced split, a random choice between the two candidate deletions would give A≈0.5, above the best reported A=0.38 (deepseek-r1, Template, Table 4); under an imbalanced split, a majority-class baseline could be higher still. The abstract's statement that 'LLMs have the potential to detect and correct errors' is thus unsupported by the current metrics. The authors should report the class balance and add rule-based baselines (e.g., always delete ALLERGIC_TO, always delete HAS_INGREDIENT) alongside the LLM results.
  3. [§5 / Tables 3-6 / Figure 3] No measure of variability is provided for any result. Each table cell is a single point value with no standard deviation, confidence interval, or statement of how many inconsistencies and how many independent runs underlie it. The captions say results are averaged across 15 runs, but the composition of these runs (3 encoding modes × 5 example modes?) and whether the 165 inconsistencies are constant across runs are never specified. Given that the differences the paper highlights (e.g., gemma2 A=0.32 at 1-small vs deepseek-r1 A=0.38 at 2-small, Table 6) are small relative to the expected sampling error of a 165-item test set (s.e. ≈ 0.038 for a 0.38 proportion), the qualitative conclusions about which model or prompt setup is best are not supported. The authors should report per-cell standard errors or confidence intervals, or at minimum the number of test items per cell.
  4. [Table 5 / §5.1] The reported 'Graph Encoding Performance' statistics coincide exactly with the repair-response statistics in Table 3 for every model (e.g., llama3.2: 151.0 tokens and 2.09 sec; deepseek-r1: 478.6 tokens and 11.40 sec appear in both tables). Since Table 5 is captioned as the statistics of the LLM-generated encoding of the inconsistency, it should contain different numbers than the repair-generation statistics. As published, either the table reports the wrong quantities or the encoding step was not separately measured, invalidating the computational-cost analysis for the M3 encoding. Please regenerate Table 5 or clarify what was actually measured.
minor comments (5)
  1. [§1] The format-adherence range '72% to 96%' does not match the reported F values across all setups; for instance, Table 6 (none row) reports F=0.11 for llama3.2 and F=0.18 for qwen2.5. Please qualify the claim with the setups over which the range holds.
  2. [§1] The cited validity range '42%-74%' is not directly derivable from any single table; please state explicitly over which models and averaging scheme this range is computed.
  3. [§4.3.1 / §5.4] Responses that contain the correct repair together with additional operations, and responses that offer multiple alternative repairs, are both counted as A=0, which conflates 'wrong repair' with 'indecision' — a distinction the paper itself draws in Section 5.4. Reporting these failure types separately would make the accuracy numbers more informative.
  4. [§6] The statement that deepseek-r1's correct repair 'did not utilize any real-world factual knowledge' is based on a single quoted response; a quantitative analysis over all correct repairs would be needed to support this claim.
  5. [§3.1.1 / Table 1 / M1 example] The label of the patient-medication edge is inconsistent across the manuscript: 'TAKES' in the Cypher query, 'TAKE_MEDICATION' in Table 1, and 'TAKES_MEDICATION' in the M1 example. Please unify the notation.

Circularity Check

1 steps flagged · score 4.0 of 10

Validity metric V is satisfied by any single edge deletion in the three-edge inconsistency triangle, so the reported 'moderate success' in V is an artifact of the definition; accuracy A is non-circular but the positive headline is overstated.

  1. self definitional [Section 4.3.1 (Repair Validity and Correctness), with Section 3.1.1 and Figure 1]
    "A repair is termed to be valid (V = 1) if the suggested repairs result in the elimination of the violation of the denial constraint, even if the repair might not match the ground-truth. ... The inconsistency may be repaired in several ways, including but not limited to, the removal of one of the three edges in red."

    The denial constraint is matched by a triangle: MATCH (p:Patient)-[rm:TAKES]->(m:Medication), (m)-[rc:CONTAINS]->(i:Ingredient), (p)-[ra:ALLERGIC_TO]->(i). Deleting any one of the three edges makes the pattern unmatchable, hence V=1 by the Section 4.3.1 definition. A trivial always-delete-one-edge baseline would therefore score V=1.0, but no such baseline is reported. The paper's own results show this: phi4 reaches V=0.98 with A=0.00 in the Template mode, and Section 5.4 says phi4 fails in 98.8% of cases by removing the TAKES edge, exactly the trivial repair. Thus the central 'moderate success' in validity (42%-74%) is substantially a consequence of the metric's definition plus the graph topology, not an independent measure of LLM repair capability.

full rationale

The paper is an empirical benchmark, not a derivation, and most reported numbers are direct measurements with no fitted parameters rediscovered as predictions. The accuracy metric A is grounded in a fixed injection-based ground truth where the correct repair is deletion of either the ALLERGIC_TO or the HAS_INGREDIENT edge, so A is not circular. No load-bearing self-citation or imported uniqueness theorem appears; self-citations to the authors' earlier interactive-repair work are contextual only. The identified issue is the validity metric V: the inconsistency subgraph is a three-edge triangle, and the Cypher query in Section 3.1.1 matches exactly that triangle, so removing any of the three edges eliminates the violation. Under the Section 4.3.1 definition, such a removal is 'valid' even when it is the wrong edge; phi4's V=0.98 with A=0.00 (Template row, Table 4) and the failure-case analysis in Section 5.4 make the vacuity concrete. Because the headline positive claim of 'moderate success' in validity is largely an artifact of this definition and no trivial baseline is reported, a partial circularity score is warranted. It is not higher because the paper also emphasizes low accuracy, and the accuracy and cost measurements remain meaningful and externally grounded.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the authors' synthetic benchmark and its ground-truth definition; no new entities are introduced. The hand-chosen injection probabilities and temperature are experimental controls, while the ground-truth rule is the key assumption.

free parameters (4)
  • p_wrong_ingredient = 0.15
    Probability of injecting a wrong ingredient into a medication; hand-chosen by the authors to create inconsistencies.
  • p_allergy = 0.05
    Probability of injecting an allergy to a real ingredient of the prescribed medication; hand-chosen.
  • p_wrong_allergy = 0.25
    Probability of injecting an allergy to an incorrect ingredient; hand-chosen.
  • temperature = 0.4
    LLM sampling temperature chosen to reduce response variance; affects all measurements.
assumptions (3)
  • standard math The graph denial constraint and property graph model from Fan and Lu (2017) apply without modification.
    Background formalism from [16] and [4] used to define inconsistencies in Section 3.
  • ad hoc to paper The injected error determines the ground truth: removing the allergy edge is correct when the ingredient is real, and removing the ingredient edge is correct when the ingredient is wrong.
    The correctness labels are defined by the authors' injection process in Section 4.1, not by an external gold standard.
  • domain assumption LLMs possess enough real-world pharmaceutical knowledge to distinguish real ingredients from injected wrong ingredients.
    The system prompt instructs models to use real-world knowledge (Section 3.1.2), and accuracy is scored against that expectation; the paper itself questions this in Section 6.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Graph Repairs with Large Language Models: An Empirical Study." pith.science (2026). https://pith.science/paper/6CIDCHNE

@misc{pith2026250703410,
  author       = {Pith},
  title        = {Pith review of: Graph Repairs with Large Language Models: An Empirical Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6CIDCHNE}},
  note         = {Machine review of arXiv:2507.03410}
}
read the original abstract

Property graphs are widely used in domains such as healthcare, finance, and social networks, but they often contain errors due to inconsistencies, missing data, or schema violations. Traditional rule-based and heuristic-driven graph repair methods are limited in their adaptability as they need to be tailored for each dataset. On the other hand, interactive human-in-the-loop approaches may become infeasible when dealing with large graphs, as the cost--both in terms of time and effort--of involving users becomes too high. Recent advancements in Large Language Models (LLMs) present new opportunities for automated graph repair by leveraging contextual reasoning and their access to real-world knowledge. We evaluate the effectiveness of six open-source LLMs in repairing property graphs. We assess repair quality, computational cost, and model-specific performance. Our experiments show that LLMs have the potential to detect and correct errors, with varying degrees of accuracy and efficiency. We discuss the strengths, limitations, and challenges of LLM-driven graph repair and outline future research directions for improving scalability and interpretability.

Figures

Figures reproduced from arXiv: 2507.03410 by the authors.

Figure 1
Figure 1. Inconsistency in a Property Graph: A patient is be [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Repairing Property Graph Inconsistencies with Large Language Models [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Repair Quality of LLMs Averaged Across 15 Runs [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Distribution of Repair Operations Across Models [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 41 canonical work pages

  1. [1]

    Cypher Query Language

    2021. Cypher Query Language. https://neo4j.com/developer/cypher/

    work page 2021

  2. [2]

    DeepSeek AI. 2024. DeepSeek-R1. https://deepseek-ai.com/product/deepseek-llm. Accessed via Ollama

    work page 2024

  3. [3]

    2023/2024

    Mistral AI. 2023/2024. Mistral AI Models. https://mistral.ai/. Accessed via Ollama

    work page 2023

  4. [4]

    Renzo Angles. 2018. The Property Graph Database Model. In AMW

    work page 2018

  5. [5]

    Abdallah Arioua and Angela Bonifati. 2018. User-guided repairing of inconsistent knowledge bases. In EDBT: Extending Database Technology. OpenProceedings. org, 133–144

    work page 2018

  6. [6]

    Leopoldo Bertossi. 2011. Database repairing and consistent query answering . Vol. 20. Morgan & Claypool Publishers

    work page 2011

  7. [7]

    Akansha Bhardwaj, Jie Yang, and Philippe Cudré-Mauroux. 2022. Human-in- the-loop rule discovery for micropost event detection. IEEE Transactions on Knowledge and Data Engineering 35, 8 (2022), 8100–8111

    work page 2022

  8. [8]

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

    arXiv 2021

Show all 59 references
  1. [9]

    Angela Bonifati, George H. L. Fletcher, Hannes Voigt, and Nikolay Yakovets. 2018. Querying Graphs. Morgan & Claypool Publishers

    2018

  2. [10]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    2020 arXiv

  3. [11]

    Yurong Cheng, Lei Chen, Ye Yuan, and Guoren Wang. 2018. Rule-based graph repairing: Semantic and efficient repairing methods. In2018 ieee 34th international conference on data engineering (icde) . IEEE, 773–784

    2018

  4. [12]

    2023/2024

    Alibaba Cloud. 2023/2024. Qwen 2.5. https://github.com/QwenLM/Qwen. Ac- cessed via Ollama

    2023

  5. [13]

    Isabelle Comyn-Wattiau and Jacky Akoka. 2017. Model driven reverse engi- neering of NoSQL property graph databases: The case of Neo4j. In 2017 IEEE International Conference on Big Data (Big Data) . IEEE, 453–458

    2017

  6. [14]

    Dong Dai, Robert B Ross, Philip Carns, Dries Kimpe, and Yong Chen. 2014. Using property graphs for rich metadata management in hpc systems. In 2014 9th parallel data storage workshop . IEEE, 7–12

    2014

  7. [15]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  8. [16]

    Wenfei Fan and Ping Lu. 2017. Dependencies for Graphs. In Proceedings of the 36th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (Chicago, Illinois, USA) (PODS ’17). Association for Computing Machinery, New York, NY, USA, 403–416. doi:10.1145/3034786.3056114

    2017

  9. [17]

    Wenfei Fan, Yinghui Wu, and Jingbo Xu. 2016. Functional Dependencies for Graphs. In Proceedings of the 2016 International Conference on Management of Data (San Francisco, California, USA) (SIGMOD ’16). Association for Computing Machinery, New York, NY, USA, 1843–1857. doi:10.1...

    2016

  10. [18]

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2023. Talk like a graph: Encoding graphs for large language models. arXiv preprint arXiv:2310.04560 (2023)

    2023 arXiv

  11. [19]

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2024. Talk like a Graph: Encoding Graphs for Large Language Models. In Proceedings of the International Conference on Learning Representations (ICLR) 2024 . International Conference on Learning Representations, Vienna, Austri...

    2024

  12. [20]

    Louie Giray. 2023. Prompt engineering with ChatGPT: a guide for academic writers. Annals of biomedical engineering 51, 12 (2023), 2629–2633

    2023

  13. [21]

    Google. 2024. Gemma 2. https://ai.google.dev/gemma/docs. Accessed via Ollama

    2024

  14. [22]

    Kan Hatakeyama-Sato, Naoki Yamane, Yasuhiko Igarashi, Yuta Nabae, and Teru- aki Hayakawa. 2023. Prompt engineering of GPT-4 for chemical research: what can/cannot be done? Science and Technology of Advanced Materials: Methods 3, 1 (2023), 2260300

    2023

  15. [23]

    Ziyue Hua, Wei Lin, Luyao Ren, Zongyang Li, Lu Zhang, Wenpin Jiao, and Tao Xie. 2023. GDsmith: Detecting bugs in Cypher graph database engines. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 163–174

    2023

  16. [24]

    Ilyas and Xu Chu

    Ihab F. Ilyas and Xu Chu. 2019. Data Cleaning . ACM Books, Vol. 28. ACM. doi:10.1145/3310205

    2019 doi

  17. [25]

    Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645 (2023)

    2023 arXiv

  18. [26]

    Paul Juillard, Angela Bonifati, and Andrea Mauri. 2024. Interactive Graph Repairs for Neighborhood Constraints. In Proceedings 27th International Conference on Extending Database Technology ( EDBT 2024 ) Paestum, Italy, March 25 - March 28 . OpenProceedings.org, 2:175–2:187

    2024

  19. [27]

    Paul Juillard, Angela Bonifati, and Andrea Mauri. 2024. Interactive graph repairs for neighborhood constraints. In International Conference on Extending Database Technology, Vol. 27. pp–175

    2024

  20. [28]

    Hoa Thi Le, Angela Bonifati, and Andrea Mauri. 2025. Graph Consistency Rule Mining with LLMs: an Exploratory Study. (2025)

    2025

  21. [29]

    Simon Liu, Wei Ma, Robin Moore, Vikraman Ganesan, and Stuart Nelson. 2005. RxNorm: prescription for electronic drug information exchange. IT professional 7, 5 (2005), 17–23

    2005

  22. [30]

    Michael Loster, Davide Mottin, Paolo Papotti, Jan Ehmüller, Benjamin Feld- mann, and Felix Naumann. 2021. Few-Shot Knowledge Validation Using Rules. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW ’21). Association for Computing Machinery, New York, NY, US...

    2021

  23. [31]

    Linhao Luo, Jiaxin Ju, Bo Xiong, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan

  24. [32]

    Ggaliwango Marvin, Nakayiza Hellen, Daudi Jjingo, and Joyce Nakatumba- Nabende. 2023. Prompt Engineering in Large Language Models. In International Conference on Data Intelligence and Cognitive Informatics . Springer, 387–402

    2023

  25. [33]

    Bertalan Meskó. 2023. Prompt engineering as an important emerging skill for medical professionals: tutorial. Journal of Medical Internet Research 25 (2023), e50638

    2023

  26. [34]

    Meta Platforms

    Inc. Meta Platforms. 2024. Llama 3. https://ai.meta.com/llama/. Accessed via Ollama

    2024

  27. [35]

    Microsoft. 2024. Phi-4. https://www.microsoft.com/en-us/research/blog/phi- 4-a-truly-multimodal-model-with-superior-reasoning-and-language- understanding/. Accessed via Ollama

    2024

  28. [36]

    Victoria Nebot and Rafael Berlanga. 2012. Finding association rules in semantic web data. Knowledge-Based Systems 25, 1 (2012), 51–62

    2012

  29. [37]

    Jasper Oosterman, Archana Nottamkandath, Chris Dijkshoorn, Alessandro Boz- zon, Geert-Jan Houben, and Lora Aroyo. 2014. Crowdsourcing knowledge- intensive tasks in cultural heritage. In Proceedings of the 2014 ACM conference on Web science. 267–268

    2014

  30. [38]

    Amedeo Pachera, Angela Bonifati, and Andrea Mauri. 2025. User-Centric Property Graph Repairs. Proceedings of the ACM on Management of Data 3, 1 (2025), 1–27

    2025

  31. [39]

    Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu

  32. [40]

    Yun Peng, Sen Lin, Qian Chen, Shaowei Wang, Lyu Xu, Xiaojun Ren, Yafei Li, and Jianliang Xu. 2024. ChatGraph: Chat with Your Graphs. In 2024 IEEE 40th International Conference on Data Engineering (ICDE) . IEEE, 5445–5448

    2024

  33. [41]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9

    2019

  34. [42]

    Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, and Chao Huang. 2024. A Survey of Large Language Models for Graphs. In KDD ’24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . ACM, 6616–6626. doi:10.1145/3637528.3671460 Published: 24 Au...

    2024

  35. [43]

    Siddhartha Sahu, Amine Mhedhbi, Semih Salihoglu, Jimmy Lin, and M Tamer Özsu. 2017. The ubiquity of large graphs and surprising challenges of graph processing. Proceedings of the VLDB Endowment 11, 4 (2017), 420–431

    2017

  36. [44]

    Siddhartha Sahu, Amine Mhedhbi, Semih Salihoglu, Jimmy Lin, and M Tamer Özsu. 2020. The ubiquity of large graphs and surprising challenges of graph processing: extended survey. The VLDB journal 29 (2020), 595–618

    2020

  37. [45]

    Ioannis Petros Samiotis, Sihang Qiu, Andrea Mauri, Cynthia CS Liem, Christoph Lofi, and Alessandro Bozzon. 2020. Microtask Crowdsourcing for Music Score Transcriptions: An Experiment with Error Detection.. In ISMIR. 901–907

    2020

  38. [46]

    Shaoxu Song, Boge Liu, Hong Cheng, Jeffrey Xu Yu, and Lei Chen. 2017. Graph repairing under neighborhood constraints. The VLDB Journal 26 (2017), 611–635

    2017

  39. [47]

    Hrishikesh Terdalkar. 2025. LLM-Graph-Repair. https://github.com/hrishikeshrt/ LLM-Graph-Repair. Accessed: 2025-05-01

    2025

  40. [48]

    David Tuck. 2022. A cancer graph: a lung cancer property graph database in Neo4j. BMC research notes 15, 1 (2022), 45

    2022

  41. [49]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  42. [50]

    Vincenzo Venerito, Devansh Lalwani, Sergio Del Vescovo, Florenzo Iannone, and Latika Gupta. 2024. Prompt engineering: The next big skill in rheumatology research. International Journal of Rheumatic Diseases 27, 5 (2024), e15157

    2024

  43. [51]

    Jason Walonoski, Mark Kramer, Joseph Nichols, Andre Quina, Chris Moesel, Dylan Hall, Carlton Duffett, Kudakwashe Dube, Thomas Gallagher, and Scott McLachlan. 2018. Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electron...

    2018

  44. [52]

    Jiaqi Wang, Enze Shi, Sigang Yu, Zihao Wu, Chong Ma, Haixing Dai, Qiushi Yang, Yanqing Kang, Jinru Wu, Huawen Hu, et al . 2023. Prompt engineering for healthcare: Methodologies and applications. arXiv preprint arXiv:2304.14670 (2023)

    2023 arXiv

  45. [53]

    Jim Webber. 2012. A programmatic introduction to neo4j. In Proceedings of the 3rd annual conference on Systems, programming, and applications: software for humanity. 217–218

    2012

  46. [54]

    Yangjian Wu and Gang Hu. 2023. Exploring prompt engineering with GPT language models for document-level machine translation: Insights and findings. In Proceedings of the Eighth Conference on Machine Translation . 166–169. GRADES-NDA ’25, June 22–27, 2025, Berlin, Germany Terda...

    2023

  47. [55]

    Fabian Yamaguchi, Nico Golde, Daniel Arp, and Konrad Rieck. 2014. Modeling and discovering vulnerabilities with code property graphs. In 2014 IEEE symposium on security and privacy . IEEE, 590–604

    2014

  48. [56]

    Linyao Yang, Hongyang Chen, Zhao Li, Xiao Ding, and Xindong Wu. 2023. Chatgpt is not enough: Enhancing large language models with knowledge graphs for fact-aware language modeling. arXiv preprint arXiv:2306.11489 (2023)

    2023 arXiv

  49. [57]

    Linyao Yang, Hongyang Chen, Zhao Li, Xiao Ding, and Xindong Wu. 2024. Give us the facts: Enhancing large language models with knowledge graphs for fact- aware language modeling. IEEE Transactions on Knowledge and Data Engineering (2024). Received 4 April 2025; accepted 2 May 2025

    2024

  50. [2023]

    arXiv preprint arXiv:2309.01538 (2023)

    Chatrule: Mining logical rules with large language models for knowledge graph reasoning. arXiv preprint arXiv:2309.01538 (2023)

    2023 arXiv

  51. [2024]

    IEEE Transactions on Knowledge and Data Engineering (2024)

    Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering (2024)

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.