Pith. sign in

REVIEW 4 major objections 6 minor 42 references

RELRaE: LLM-Based Relationship Extraction, Labelling, Refinement, and Evaluation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Refining simple rule-based relationship labels with a large language model produces more accurate ontology labels than either approach used alone.

desk verdict A sensible hybrid pipeline for XML-to-ontology labeling that is worth refereeing, but the evaluation leans too hard on an unvalidated similarity metric and needs multi-schema, statistically grounded support. read the letter →

arxiv 2507.03829 v1 pith:HTEVLQUG submitted 2025-07-04 cs.AI

George Hannah , Jacopo de Berardinis , Terry R. Payne , Valentina Tamma , Andrew Mitchell , Ellen Piercy , Ewan Johnson , Andrew Ng
show 2 more authors
Harry Rostron Boris Konev
This is my paper
classification cs.AI
keywords LLMXMLschemaontologyengineeringrelationshipextractionLLM-as-a-judgeknowledgegraphAnIMLprompt
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models can reliably upgrade the simple relationship labels that rules generate from XML schemas, producing labels closer to what a domain expert would write. The paper builds RELRaE, a four-stage pipeline that extracts hierarchical relationships from an XML schema, labels them with a rule-based module, refines those labels using an LLM with a few-shot prompt, and evaluates the results with a second LLM acting as a judge. On the AnIML analytical-chemistry schema, the refined labels consistently beat both the rule-based and LLM-only baselines by a wide margin on a Phrase-BERT cosine-similarity measure against a gold-standard set created by three experts. The findings suggest that LLMs can ease the knowledge-acquisition bottleneck in semi-automatic ontology engineering, and that LLM-as-a-judge evaluation is promising though imperfect.

What carries the argument

The central mechanism is the RELRaE pipeline, which combines a rule-based module (RuBREx) with two LLM stages. RuBREx extracts parent-child, grandparent-grandchild, element-attribute and boolean-type relationships from the XML schema, and generates initial labels using structural patterns such as 'has<r>', 'is<r>', and '<r> subclassOf <d>' (Table 1). The refinement LLM (GPT-4o) is then prompted, with role, domain, use-case and a few-shot set of generic and domain-specific examples, to accept or refine the rule-based label. A different LLM (Gemini-2.0-flash) finally evaluates the refined label on a five-point Likert scale, mapping scores to 'Yes/Likely' acceptance; labels not accepted revert to the RuBREx label. Cosine similarity between generated labels and the reference set, computed via Phrase-BERT embeddings, is the metric used to compare methods.

What would settle it

Ask a set of experts to judge, blind, whether each generated label correctly describes its domain-range pair without seeing the reference label. If the refined labels do not receive higher correctness ratings than the rule-based and LLM-only labels, the reported advantage is an artifact of the cosine-similarity measure.

Watch

Extended reading notes

Core claim

The central claim is that combining a rule-based labelling pass with an LLM refinement pass yields significantly better relationship labels than either pass alone. The paper's comparative evaluation on the AnIML schema shows that the refined labels achieve a mean Phrase-BERT cosine similarity of about 0.88 to the expert-generated reference labels, versus roughly 0.81 for both the rule-based RuBREx labels and the LLM-only labels, and that the refined approach also produces the highest share of labels meeting both a loose (≥0.6) and a strict (≥0.85) similarity threshold. The authors interpret this as evidence that the rule-based labels anchor the LLM to a consistent interpretation, while the LLM injects domain knowledge to adjust wording to match expert intuition. They also claim, on the basis of a separate comparison against a domain expert, that an LLM evaluator can serve as a useful proxy for judging label acceptability, especially when 'Possible' is treated as an acceptable score.

Load-bearing premise

The paper's central comparison assumes that how close a label is to an expert-written reference label, as measured by Phrase-BERT cosine similarity after thresholds chosen on a small sample, correctly reflects how good the label is.

Editorial extensions

If this is right

  • The RELRaE pipeline can be applied to other XML schemata beyond AnIML to produce skeleton ontologies with minimal human effort.
  • The hybrid approach reduces the number of labels a domain expert must hand-correct, since under a loose threshold about 95% of refined labels resemble the expert reference set.
  • The LLM-as-a-judge stage, accepting labels rated 'Possible' or above, can filter obviously wrong labels automatically, cutting the expert review workload.
  • If the refinement step's anchoring hypothesis is correct, similar pipelines could be built for other semi-structured formats such as JSON Schema or CSV by swapping the rule-based starter module.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The evaluation's reliance on Phrase-BERT cosine similarity may underweight semantically correct but lexically distant labels; a direct human-judged semantic-accuracy study could change the relative ranking between the methods.
  • The strict-threshold results (about 57.6% of refined labels above 0.85) suggest that a large fraction of labels still need human review; the pipeline's practical benefit may be in highlighting which labels to review, not in replacing review entirely.
  • The finding that the LLM-only baseline performs no better than the rule-based baseline, while the refined version clearly outperforms both, suggests that the rule-based label acts as a strong prior; one could test this by giving the LLM a random or deliberately poor initial label and checking whether refinement still helps.
  • The framework's modular design means the two LLMs can be swapped independently; the paper does not test model variation, so the reported gains may be specific to the GPT-4o/Gemini pair.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces RELRaE, a four-stage framework that converts an XML schema into a skeleton ontology by extracting hierarchical relationships, generating rule-based labels (RuBREx), refining those labels with an LLM (LLMR), and evaluating the refined labels with a different LLM acting as a judge (LLME). The authors evaluate three label-generation approaches—rule-based only, LLM-only, and the refined hybrid—on the AnIML analytical-chemistry schema, comparing generated labels against a gold-standard set produced by three domain experts using Phrase-BERT cosine similarity. They also compare LLME's Likert-scale acceptability judgments against a domain expert on a random sample of 60 labels per approach. The central reported finding is that the Refined approach consistently outperforms both baselines, with mean Phrase-BERT similarity around 0.88 versus about 0.81, and with roughly 95% of refined labels above a loose similarity threshold versus roughly 86-91% for baselines.

Significance. If the central claim holds, the paper is a useful applied contribution to semi-automatic ontology engineering: it demonstrates a concrete, reproducible pipeline that combines rule-based structure with LLM refinement, and it provides an empirical comparison grounded in an external human reference set rather than relying solely on intrinsic LLM judgment. The paper is also careful in several respects: experiments are reported across multiple temperature settings, the two LLM roles use different model families to reduce bias, the evaluation setup is described in enough detail to be replicable, and the authors openly acknowledge the subjectivity of label quality and the limitations of their ground truth. The main significance, however, is conditional on the validity of the semantic-similarity proxy used for the headline comparison; the current evidence does not fully support the strong claim that the refined pipeline 'significantly enhances label accuracy.'

major comments (4)
  1. [§5.1, Tables 6 and 7] The load-bearing quantitative claim—that the Refined approach 'consistently generates superior relationship labels'—rests entirely on Phrase-BERT cosine similarity between generated labels and the gold-standard labels. The paper itself acknowledges (end of §5.1) that a label can be penalized for lexical difference or rewarded for lexical similarity without representing the relationship correctly. No evidence is provided that Phrase-BERT cosine similarity correlates with human semantic quality for these short, domain-specific relationship labels, and no inter-annotator agreement is reported for the three domain experts who produced the gold standard. Without such validation, the 0.88 versus 0.81 gap could partly reflect lexical style or the anchoring of Refined labels to the rule-generated 'hasX' prefix rather than semantic accuracy. I recommend adding a human-evaluation subset or an inter-annotator agreement measure, and/or rephrasing the claim to 'lexical-semantic similarity to the reference labels' rather than 'accuracy.'
  2. [§5.1, thresholds and statistical inference] The acceptance thresholds (τ ≥ 0.6 and τ ≥ 0.85) were 'determined empirically prior to the main evaluation using a small sample set of the generated labels' from the same AnIML benchmark. This makes the percentages in Table 7 potentially optimistically biased, since the thresholds and the evaluation set are not independent. In addition, Tables 6 and 7 report no confidence intervals, standard deviations, or significance tests, so the claimed superiority of the Refined approach over RuBREx and LLM-only is not statistically established. I recommend reporting bootstrap confidence intervals or paired significance tests (e.g., Wilcoxon signed-rank across relationship pairs), and ideally evaluating threshold robustness on a held-out set or by sweeping τ and showing the ordering is stable.
  3. [§5.2, Table 8 and footnote 6] The RQ2 evaluation is under-specified in ways that affect the conclusion. The acceptance criterion of 'Likely' or 'Yes' is stated to be based on a preliminary study, yet the reported domain-expert agreement with LLME is only 41.7% for that criterion, rising to 93.3% when 'Possible' is included. This large gap suggests that the LLME's moderate-confidence responses may often mean 'not clearly wrong' rather than 'genuinely appropriate,' as the authors themselves speculate in §6. The random selection of 60 labels per approach is reported without a random seed or a description of the sampling distribution, and no chance-level baseline, Cohen's kappa, or per-label agreement breakdown is given. I recommend reporting the full Likert-score distribution, a chance baseline, and a chance-corrected agreement metric, and tempering the claim that LLME can successfully evaluate labels.
  4. [Abstract and §5.1] The abstract states that RELRaE 'significantly enhances label accuracy compared to other methods,' but the evaluation does not include a significance test, and the proxy used is similarity rather than accuracy. Given the acknowledged limitations of the Phrase-BERT proxy and the small, expert-produced reference set, 'significantly' is not supported. Please either provide appropriate statistical evidence or soften the claim throughout the paper.
minor comments (6)
  1. [§3.2] There is a typo: 'examples of the type of labels expected are given ... that cab improve the quality' should read 'can improve the quality.'
  2. [§6] The framework name is spelled inconsistently: 'ReLRAE' appears in §6 and the abstract uses 'RELRaE'; please standardize throughout.
  3. [§4.2] The sentence 'A few-shot approach was also used, by including the the examples listed in Table 5' contains a duplicated 'the.'
  4. [Tables 6 and 7] Several entries in Tables 6 and 7 are rendered as '...', which makes the tables incomplete in the manuscript text; all values should be filled in, and the temperature settings should be clearly aligned across rows.
  5. [Table 1] Pattern IDs jump from 7 to 9 and 10, with no row for ID 8; either renumber the patterns or explain the omission.
  6. [§5.2] The sentence 'LLME scores the confident with which it generates the relationships' should read 'the confidence with which it evaluates the relationships.'

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: central results rest on external human gold-standard labels and a separate domain-expert agreement check; disclosed post-hoc threshold choices are evaluation-design weaknesses, not construction-level circularity.

full rationale

The paper's central quantitative claim (Refined labels are superior, Tables 6-7) is an empirical comparison against an externally produced gold-standard reference set: 'An expert-led set of labels were independently generated (by three domain experts) for all of the relationships that had been extracted from the AnIML schema; resulting in a gold-standard reference set of labels' (Section 5.1). No stage of the pipeline fits the comparison metric or the gold labels, so the superiority result is not equivalent to an input by construction. The RQ2 claim is checked against a separate domain expert ('this expert was different to the ones generating the labels in the reference set'), yielding an external agreement rate. The only in-benchmark tuning is the choice of similarity thresholds: 'These thresholds were determined empirically prior to the main evaluation using a small sample set of the generated labels' (Section 5.1), and the acceptance criterion in footnote 6 was 'based on a preliminary study, and is reinforced through the results of the evaluation in Section 5.2.' This is post-hoc criterion selection that weakens the interpretability of the percentages in Tables 7-8, but it is disclosed and does not affect Table 6's threshold-free mean similarities or the expert-agreement check in Section 5.2. The single self-citation [15] merely supports the default 'has' label convention, a non-load-bearing design choice. The paper also acknowledges the gold-standard limitation ('based only on a limited number of experts'), which is a validity concern, not circularity. No equation or fitted parameter reduces the central claim to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or formal entities; it is a software pipeline built from existing models (GPT-4o, Gemini-2.0-flash) and standard RDF/RDFS constructs. The main assumptions are about evaluation validity and domain transferability.

free parameters (3)
  • Cosine similarity acceptance thresholds τ = 0.6 (loose), 0.85 (strict)
    Determined empirically on a small sample of generated labels before the main evaluation (Section 5.1); results shift substantially with the threshold (Table 7).
  • LLME Likert acceptance criterion = 4/5 (Likely/Yes) in pipeline; 3/4/5 in agreement analysis
    Chosen based on a preliminary study and reported as reinforced by the main evaluation (Section 3.3, footnote 6; Section 5.2).
  • Random selection of 60 labels per approach = 60 labels per method
    Selection procedure and seed are not specified in the paper, so the exact subset is not reproducible (Section 5.2).
assumptions (4)
  • domain assumption XML schema hierarchy encodes implicit relationships that can be labeled by structural patterns.
    Stages 1 and 2 extract relationships from hierarchical configuration and assign labels via patterns (Table 1).
  • domain assumption Phrase-BERT cosine similarity between generated and gold-standard labels is a valid measure of label correctness.
    Section 5.1 defines matches by cosine similarity thresholds and uses this to rank methods; the paper notes lexical similarity may not equal semantic correctness.
  • domain assumption Labels produced independently by three domain experts constitute a reliable gold standard.
    The reference set is used as ground truth in Section 5.1; the authors admit in Section 6 that the ground truth is based on a limited number of experts and is subjective.
  • domain assumption A separate LLM's Likert-scale verdict can proxy a domain expert's judgment of label suitability.
    Stage 4 and Section 5.2 compare LLME ratings to one expert's ratings on 60 labels per approach, with no inter-annotator baseline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RELRaE: LLM-Based Relationship Extraction, Labelling, Refinement, and Evaluation." pith.science (2026). https://pith.science/paper/HTEVLQUG

@misc{pith2026250703829,
  author       = {Pith},
  title        = {Pith review of: RELRaE: LLM-Based Relationship Extraction, Labelling, Refinement, and Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HTEVLQUG}},
  note         = {Machine review of arXiv:2507.03829}
}
read the original abstract

A large volume of XML data is produced in experiments carried out by robots in laboratories. In order to support the interoperability of data between labs, there is a motivation to translate the XML data into a knowledge graph. A key stage of this process is the enrichment of the XML schema to lay the foundation of an ontology schema. To achieve this, we present the RELRaE framework, a framework that employs large language models in different stages to extract and accurately label the relationships implicitly present in the XML schema. We investigate the capability of LLMs to accurately generate these labels and then evaluate them. Our work demonstrates that LLMs can be effectively used to support the generation of relationship labels in the context of lab automation, and that they can play a valuable role within semi-automatic ontology generation frameworks more generally.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 37 canonical work pages

  1. [1]

    Alharbi, V

    R. Alharbi, V. Tamma, F. Grasso, and T. R. Payne. The role of Generative AI in competency question retrofitting. In Extended Semantic Web Conference, ESWC2024, Hersonissos, Greece, 2024

    work page 2024

  2. [2]

    Alharbi, V

    R. Alharbi, V. Tamma, F. Grasso, and T. R. Payne. A review and comparison of competency question engineering approaches. In Knowledge Engineering and Knowledge Management, pages 271–290, Cham, 2025. Springer Nature Switzer- land

    work page 2025

  3. [3]

    Babaei Giglou, J

    H. Babaei Giglou, J. D’Souza, and S. Auer. Llms4ol: Large language models for ontology learning. In International Semantic Web Conference, pages 408–427. Springer, 2023

    work page 2023

  4. [4]

    Barile, C

    R. Barile, C. d’Amato, and N. Fanizzi. Lp-dixit: Evaluating explanations for link prediction on knowledge graphs using large language models. InTHE WEB CON- FERENCE 2025, 2025

    work page 2025

  5. [5]

    Bedini, C

    I. Bedini, C. Matheus, P. F. Patel-Schneider, A. Boran, and B. Nguyen. Transform- ing xml schema to owl using patterns. In2011 IEEE Fifth International Conference on Semantic Computing, pages 102–109. IEEE, 2011

    work page 2011

  6. [6]

    Bohring and S

    H. Bohring and S. Auer. Mapping XML to OWL ontologies. In K. P. Jantke, K. Fähnrich, and W. S. Wittig, editors,Marktplatz Internet: von E-Learning bis E-Payment, 13. Leipziger Informatik-Tage, LIT 2005, 21.-23. September 2005, Leipzig, volume P-72 ofLNI, pages 147–156. GI, 2005

    work page 2005

  7. [7]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, et al. Language models are few-shot learners. InProceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20. Curran Asso- ciates Inc., 2020

    work page 2020

  8. [8]

    S. M. Bsharat, A. Myrzakhan, and Z. Shen. Principled instructions are all you need for questioning llama-1/2, gpt-3.5/4.arXiv preprint arXiv:2312.16171, 3, 2023

    arXiv 2023

Show all 42 references
  1. [9]

    Csv on the web: Use cases and requirements

    CSV on the Web Working Group. Csv on the web: Use cases and requirements. W3c working group note, W3C, Feb. 2016. Accessed: May 14, 2025

    2016

  2. [10]

    Desmond, Z

    M. Desmond, Z. Ashktorab, Q. Pan, C. Dugan, and J. M. Johnson. Evalullm: Llm assisted evaluation of generative outputs. InCompanion Proceedings of the 29th International Conference on Intelligent User Interfaces, pages 30–32, 2024. 10 https://anonymous.4open.science/r/RELRaE L...

    2024

  3. [11]

    Faria, C

    D. Faria, C. Pesquita, E. Santos, M. Palmonari, I. F. Cruz, and F. M. Couto. The agreementmakerlight ontology matching system. InOn the Move to Meaningful In- ternet Systems: OTM 2013 Conferences: Confederated International Conferences: CoopIS, DOA-Trusted Cloud, and ODBASE 20...

    2013

  4. [12]

    E. A. Feigenbaum. The art of artificial intelligence: Themes and case studies of knowledge engineering. InProceedings of the 5th International Joint Conference on Artificial Intelligence (IJCAI), pages 1014–1029, Cambridge, MA, 1977. Morgan Kaufmann

    1977

  5. [13]

    Hacherouf, S

    M. Hacherouf, S. N. Bahloul, and C. Cruz. Transforming xml documents to owl ontologies: A survey.Journal of Information Science, 41(2):242–259, 2015

    2015

  6. [14]

    Hajjamy, L

    O. Hajjamy, L. Alaoui, and M. Bahaj. Xsd2owl2: Automatic mapping from xml schema into owl2 ontology.Journal of Theoretical and Applied Information Tech- nology, 95(8):1781–1796, 2017

    2017

  7. [15]

    Hannah, T

    G. Hannah, T. R. Payne, V. Tamma, A. Mitchell, E. Piercy, and B. Konev. To- wards a methodology for the semi-automatic generation of scientific knowledge graphs from xml documents. InIn: The 18th International Workshop on Ontology Matching, 2023-11-7 - 2023-11-7, Athens, Greece., 2023

    2023

  8. [16]

    Hannah, R

    G. Hannah, R. T. Sousa, I. Dasoulas, and C. d’Amato. On the legal implications of large language model answers: A prompt engineering approach and a view beyond by exploiting knowledge graphs.Journal of Web Semantics, 84:100843, 2025

    2025

  9. [17]

    Y. He, J. Chen, H. Dong, and I. Horrocks. Exploring large language models for ontology alignment, 2023

    2023

  10. [18]

    Hertling and H

    S. Hertling and H. Paulheim. Olala: Ontology matching with large language mod- els. In Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP ’23. ACM, Dec. 2023

    2023

  11. [19]

    Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung. Survey of hallucination in natural language generation.ACM computing surveys, 55(12):1–38, 2023

    2023

  12. [20]

    Jiménez-Ruiz and B

    E. Jiménez-Ruiz and B. Cuenca Grau. Logmap: Logic-based and scalable ontology matching. In International Semantic Web Conference, pages 273–288. Springer, 2011

    2011

  13. [21]

    H. Li, Q. Dong, J. Chen, H. Su, Y. Zhou, Q. Ai, Z. Ye, and Y. Liu. Llms-as- judges: a comprehensive survey on llm-based evaluation methods.arXiv preprint arXiv:2412.05579, 2024

    2024 arXiv

  14. [22]

    A. S. Lippolis, M. Ceriani, S. Zuppiroli, and A. G. Nuzzolese. Ontogenia: Ontology Generation with Metacognitive Prompting in Large Language Models. InPoster and demos track, Satellite proceedings of ESWC2024, 2024

    2024

  15. [23]

    Pre-train,prompt,and predict: A systematic survey of prompting methods in natural language processing

    P.Liu,W.Yuan,J.Fu,Z.Jiang,H.Hayashi,andG.Neubig. Pre-train,prompt,and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9), 2023

    2023

  16. [24]

    Y. Liu, G. Deng, Z. Xu, Y. Li, Y. Zheng, Y. Zhang, L. Zhao, T. Zhang, K. Wang, and Y. Liu. Jailbreaking chatgpt via prompt engineering: An empirical study. 2024

    2024

  17. [25]

    Marvin, N

    G. Marvin, N. Hellen, D. Jjingo, and J. Nakatumba-Nabende. Prompt engineering in large language models. In Proceedings of the Data Intelligence and Cognitive Informatics conference, pages 387–402. Springer Nature Singapore, 2024

    2024

  18. [26]

    Neches, R

    R. Neches, R. Fikes, T. Finin, T. Gruber, R. Patil, T. Senator, and W. R. Swartout. Enabling technology for knowledge sharing.AI Magazine, 12(3):36–56, 1991

    1991

  19. [27]

    N. F. Noy, D. L. McGuinness, et al. Ontology development 101: A guide to creating your first ontology, 2001. 18 G. Hannah et al

    2001

  20. [28]

    Qiang, W

    Z. Qiang, W. Wang, and K. Taylor. Agent-om: Leveraging llm agents for ontology matching, 2024

    2024

  21. [29]

    Gemini Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  22. [30]

    M. J. Saeedizade and E. Blomqvist. Navigating ontology development with large language models. InEuropean Semantic Web Conference, pages 143–161. Springer, 2024

    2024

  23. [31]

    J. F. Sequeda and D. P. Miranker. A pay-as-you-go methodology for ontology- based data access.IEEE Internet Computing, 21(2):92–96, 2017

    2017

  24. [32]

    Shankar, J

    S. Shankar, J. Zamfirescu-Pereira, B. Hartmann, A. Parameswaran, and I. Arawjo. Who validates the validators? aligning llm-assisted evaluation of llm outputs with human preferences. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, pages...

    2024

  25. [33]

    J. Shi, H. Dong, J. Chen, Z. Wu, and I. Horrocks. Taxonomy completion via implicit concept insertion. InProceedings of the ACM Web Conference 2024, pages 2159–2169, 2024

    2024

  26. [34]

    Z. Song, B. Yan, Y. Liu, M. Fang, M. Li, R. Yan, and X. Chen. Injecting domain- specific knowledge into large language models: a comprehensive survey. arXiv preprint arXiv:2502.10708, 2025

    2025 arXiv

  27. [35]

    Szymanski, N

    A. Szymanski, N. Ziems, H. A. Eicher-Miller, T. J.-J. Li, M. Jiang, and R. A. Metoyer. Limitations of the llm-as-a-judge approach for evaluating llm outputs in expert knowledge tasks. In Proceedings of the 30th International Conference on Intelligent User Interfaces, pages 952...

    2025

  28. [36]

    Tiddi and S

    I. Tiddi and S. Schlobach. Knowledge graphs as tools for explainable machine learning: A survey.Artificial Intelligence, 302:103627, 2022

    2022

  29. [37]

    G. Tom, S. P. Schmid, S. G. Baird, Y. Cao, K. Darvish, H. Hao, S. Lo, S. Pablo- García, E. M. Rajaonson, M. Skreta, et al. Self-driving laboratories for chemistry and materials science.Chemical Reviews, 124(16):9633–9732, 2024

    2024

  30. [38]

    Resource Description Framework (RDF) Model and Syntax Specification

    W3C. Resource Description Framework (RDF) Model and Syntax Specification. https://www.w3.org/TR/1999/REC-rdf-syntax-19990222/, 1999. [Accessed 11-03- 2025]

    1999

  31. [39]

    S. Wang, L. Thompson, and M. Iyyer. Phrase-bert: Improved phrase embeddings from bert with an application to corpus exploration. In Empirical Methods in Natural Language Processing, 2021

    2021

  32. [40]

    White, S

    C. White, S. Dooley, M. Roberts, A. Pal, B. Feuer, S. Jain, R. Shwartz-Ziv, N. Jain, K. Saifullah, S. Naidu, et al. Livebench: A challenging, contamination-free llm benchmark. arXiv preprint arXiv:2406.19314, 2024

    2024 arXiv

  33. [41]

    Zhang and Q

    F. Zhang and Q. Li. Constructing ontologies by mining deep semantics from xml schemas and xml instance documents.International Journal of Intelligent Systems, 37(1):661–698, 2022

    2022

  34. [2013]

    Springer, 2013

    Proceedings, pages 527–541. Springer, 2013

    2013

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.