Class Order Disorder in Wikidata and First Fixes

Ege Atacan Do\u{g}an; Peter F. Patel-Schneider

REVIEW 2 major objections 4 minor 19 references

Class Order Disorder in Wikidata and First Fixes

T0 review · 2 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A complete census of Wikidata's class-order violations finds millions of conflicts.

desk verdict Useful, current empirical survey of Wikidata class-order problems, but the 'complete enumeration' of split-order classes rests on an unverified query decomposition. read the letter →

arxiv 2411.15550 v1 pith:OCZWHR6A submitted 2024-11-23 cs.IR cs.AI

Peter F. Patel-Schneider , Ege Atacan Doğan This is my paper · ORCID

classification cs.IRcs.AI

keywords classorderWikidataSPARQLontologyqualityinstanceofsubclassmulti-levelmodelingknowledgegraph

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that the class-order problems long suspected in Wikidata can be fully enumerated with current query tools, and that the resulting counts are enormous: as of June 2024, 3,238 third-order classes, 2,447,483 second-order classes, and 14,062,244 first-order classes, with millions of overlapping memberships that violate the rule that a non-empty class has at most one fixed order. It also counts 120 instance loops and 6,379 split-order classes that cannot have a fixed order at all. The authors manually fixed a small number of these problems in Wikidata itself and showed that a handful of edits can cut some problem counts substantially, for example reducing the number of classes with minimum order four or more by roughly half. The wider point is that the Wikidata ontology's intended semantics for 'instance of' and 'subclass of' are not being enforced, and that better tooling and community processes could address this.

What carries the argument

The central object is class order, a hierarchy on Wikidata classes: non-classes have order zero, a first-order class has no classes as instances, a second-order class has only first-order classes as instances, and an nth-order class has only (n−1)th-order classes as instances. The paper exploits five built-in Wikidata classes—first-order class, second-order class, third-order class, fourth-order class, and fifth-order class—whose intended meaning fixes the order of their instances and subclasses, and uses SPARQL queries over the RDF dump to enumerate classes that are required to have a fixed order, classes whose order is bounded by instance chains, and split-order classes. The queries, run on a fast SPARQL engine, are the machinery that makes complete enumeration feasible.

What would settle it

Sample a few hundred of the reported split-order and multi-order classes and ask domain editors whether the conflicting statements are intentional. If a large fraction are defended as correct modeling (for example, a biological 'gene' class used both as a class of genes and as an instance of a gene-concept class), the paper's claim that these are problems to be fixed is weakened. Alternatively, a controlled experiment could take two identical Wikidata dumps, apply the paper's proposed editor warnings to one, and measure whether new class-order conflicts accumulate more slowly.

Watch

Extended reading notes

Core claim

The central discovery is that class order in Wikidata, defined by the rule that a first-order class has no classes as instances, a second-order class has only first-order classes as instances, and so on, can be checked exhaustively. Using SPARQL queries against the Wikidata RDF dump, the paper reports concrete counts: 3,238 third-order classes, 2,447,483 second-order classes, and 14,062,244 first-order classes, with 3,159 third-order classes also claimed to be second-order and 2,386,595 second-order classes also claimed to be first-order. It also identifies 120 instance loops and 6,379 split-order classes (classes having a subclass that is also a subclass of an instance of the class). A small set of manual fixes, such as removing a second-order parent from a large chemistry class and breaking 113 incorrect instance loops, reduced the count of classes with minimum order four from 2,854 to 1,924 and the count at order six from 1,413 to 722.

Load-bearing premise

The paper assumes that Wikidata editors intend 'instance of' and 'subclass of' to form a strict class-order hierarchy, so that a class cannot be both a first-order and a second-order class; if many editors legitimately use classes at multiple levels, the counted 'violations' are not necessarily errors.

Editorial extensions

If this is right

The reported counts give a baseline: any future edit or tooling change to Wikidata can be measured against the June 2024 numbers.
Because a few edits (removing one parent class, breaking 113 loops) cut some problem counts roughly in half, further targeted fixes could plausibly reduce the largest conflicts without a full rewrite of the ontology.
The 3,037 split-order classes caused by a single item each are candidates for manual or semi-automatic repair, while classes like gene and protein, each with hundreds of thousands of split-order entries, will require community-level modeling decisions.
If Wikidata adopted the proposed safeguards—warning editors when an edit creates an instance loop or class-order conflict, and upgrading the query service—the rate at which new order violations accumulate could drop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the class-order semantics are taken as the intended meaning, the count of 14 million first-order classes suggests that many items currently modeled as classes in Wikidata (e.g., lists and sequences) may not be genuine classes; the paper hints at this but does not push it.
The same enumeration technique could be applied to other collaborative knowledge bases or to later Wikidata dumps, turning the reported numbers into a periodic quality metric.
The large overlap between second- and first-order classes implies that many Wikidata queries that rely on the class hierarchy, such as retrieving all instances of a class, can silently return both individuals and classes, which would matter for downstream applications.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: this is a genuine empirical contribution—current (2024) counts of class-order violations in Wikidata, generalized anti-patterns, and a real attempt to fix some of them—but the paper's strongest claim, that QLever can completely enumerate split-order classes, is not fully supported by what's in the appendix. The central finding that Wikidata has millions of class-order problems is robust; the exact split-order number is less certain.

What's new: it updates Brasileiro/Dadalto's older counts with contemporary data, introduces instance loops and minimum-order chains, and reports before/after effects of manual fixes. The queries are published in the appendix, which is a real credit. The authors are transparent about the confounding effect of concurrent edits. The 120 instance-loops analysis and the fix effects are credible. The paper is honest about what is evidence and what is just 'suspect'.

Soft spots, in order of weight:

1. The split-order enumeration. The intended query has a possibly-empty path that QLever can't run. The authors split it into cases and subtract exclusions using `comm`, but give no proof—formal or otherwise—that the union of subcases minus exclusions equals the intended query. The raw result is 84M pairs, and after exclusions only 3.8M pairs and 6,379 classes remain. An error in any subcase or in the subtraction would change that headline number. This is a completeness claim, so the burden is on the authors to verify the decomposition. Independent reproduction or a machine-checkable equivalence would settle it.

2. Data availability. Archived dumps and query results are available only by contacting the first author. For a paper whose claims are counts, that's too weak. A public archive would make the numbers checkable.

3. The semantic premise. The paper assumes the intended meaning of P31/P279 includes a strict class-order hierarchy. That's a reasonable reading of Wikidata norms, and the authors acknowledge Wikidata has no formal semantics, but it means 'violations' here are violations of a community convention, not of logic. This is a caveat, not a fatal flaw.

This is a solid empirical paper for the Wikidata and KG-quality community. It deserves a serious referee, but I'd want the split-order decomposition either proven or reproduced before trusting 6,379. If that's fixed, it's worth citing and useful for anyone working on ontology cleaning in Wikidata.

Referee Report

2 major / 4 minor

Summary. The paper studies class-order violations in Wikidata as of June 2024. It defines fixed class order using the instance-of (P31) and subclass-of (P279) properties, counts classes whose order is directly determined by Wikidata's built-in fixed-order classes, computes minimum possible orders from instance chains, and enumerates split-order classes and instance loops. The authors report 3,238 third-order, 2,447,483 second-order, and 14,062,244 first-order classes with millions of order overlaps, 6,379 split-order classes, and 120 instance loops. They made a small number of manual edits to Wikidata, reran the main queries, and observed reductions in high-minimum-order classes. They conclude that QLever can completely enumerate certain class-order problems and offer suggestions for tooling and community processes.

Significance. If the counts and the completeness of the queries are correct, the paper provides the most extensive quantification to date of class-order problems in Wikidata, updating and generalizing earlier anti-pattern work. The authors are transparent: the queries are in the appendix, the use of QLever is a practical contribution, and the manual fixes demonstrate that some problems are tractable. The instance-chain queries provide evidence independent of Wikidata's own fixed-order classes. The main weaknesses are that the split-order enumeration lacks a proof of equivalence to the intended full query, and the instance-loop query is acknowledged to have missed loops with two instance links. These weaknesses bear directly on the 'complete enumeration' claim, so they need to be addressed before the numerical results can be fully relied upon.

major comments (2)

[Classes That Cannot Have Fixed Order (and Technical Appendix)] The paper's summary claims that QLever can 'completely enumerate' class-order problems, and the split-order count of 6,379 distinct classes is a headline result. The full split-order query in the Technical Appendix contains the possibly-empty path wdt:P279*/wdt:P31/wdt:P279*, which the authors state QLever cannot evaluate. They therefore split the query into self/AB/C/D subcases, run exclusion queries, and combine the results with comm. However, no proof is given that the union of the four subcases, after subtracting the exclusion sets, is exactly the answer of the original query. Because the exclusions remove about 80 million of the 84,158,269 raw pairs, leaving 3,821,300 pairs, an error in any subcase or in the comm-based subtraction would change the reported 6,379 classes and the ranked table in the 'Split-Order Classes' analysis. I would like to see either a formal containment/equivalence argument for the decomposition, a verification on a data graph small enough to be checked exhaustively, or an independent run on an engine that supports arbitrary-length paths.
[Instance Loops (and Technical Appendix)] The instance-loop query presented in the Technical Appendix only detects cycles of the form ?c wdt:P31 ?c and ?c wdt:P31/wdt:P279+ ?s with ?c = ?s. The paper later reports that 'there where instance loops that were not detected in the initial queries' and identifies loops with two instance links, such as legal abuse (Q6517445) being an instance of abuse of rights (Q12586068) and vice versa. The initial count of 120 items in instance loops is therefore not a complete enumeration of the kind of problem the paper says it is enumerating, and the before/after minimum-order counts after fixing loops are presented as reductions even though the query did not capture all loops. This needs to be reconciled with the summary's completeness claim: either supply queries covering pure P31/P31 cycles, or weaken the claim for this category.

minor comments (4)

[Resulting Counts] Because other editors may have changed Wikidata during the study, the before/after comparisons are not controlled. The authors acknowledge this, but the section should state explicitly that the reported reductions are not solely attributable to the described fixes.
[Technical Appendix] Reproducibility would be improved by depositing the archived dumps and result files with a persistent identifier rather than making them available 'by contacting the first author'.
[Split-Order Classes] There is a typo 'human (QA5)' that should be 'human (Q5)'.
[Technical Appendix] There are several typos and ungrammatical passages, including 'caes', 'sucessfully', 'minimim-order', and the sentence beginning 'It is useful to exclude items that have where the item has a superclass with...'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all load-bearing counts come from external Wikidata data and independent SPARQL queries; the one author self-citation is not load-bearing.

full rationale

The paper is an empirical audit rather than a derivation of a target result from its own premises. The class-order counts are obtained by direct SPARQL queries over Wikidata's built-in first-, second-, and third-order classes, and the overlap figures follow by set operations (comm) on those query results. The minimum-order bounds are computed by instance-chain queries that do not presuppose the directly determined orders. The split-order enumeration is a QLever-specific decomposition of the intended path query; although its equivalence to the original query is not formally proven and is therefore a completeness risk, it is not circular because the subqueries and exclusions are independent evaluations of the same intended relation, not fitted parameters or renamed outputs. The definitions of class order do depend on Wikidata's intended semantics for P31 and P279, but that is an external modeling assumption, not a reduction of the paper's conclusions to its inputs. The only author self-citation (Patel-Schneider 2019) appears in Related Work as one example of prior problem analyses and does not support any load-bearing claim. No prediction is statistically forced, and no uniqueness theorem is imported. The paper is self-contained against external benchmarks: all reported numbers are observable query results over the Wikidata dump.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no fitted parameters or new invented entities. It relies on Wikidata's own classes and properties, and on assumed intended semantics for P31 and P279. The main analytic construct, 'split-order class', is a definition derived from the existing ontology, not a postulated entity.

assumptions (4)

domain assumption The Wikidata properties 'instance of' (P31) and 'subclass of' (P279) are intended to enforce a class-order hierarchy: instances of a class are lower-order (or non-classes) and subclasses inherit the order of their superclasses.
The whole notion of 'class order violation' depends on this intended semantics, which is invoked in the definitions in 'Classes and Class Order' and in every SPARQL query.
domain assumption The universal fixed-order classes (first-order class Q104086571, second-order class Q24017414, third-order class Q24017465) are correctly linked in Wikidata and represent the intended hierarchy.
The direct determination of class order uses these classes as ground truth (Section 'Direct Determination of Class Order'); if they are misused, the derived orders are wrong. The paper itself later finds incorrect links to these classes.
domain assumption The SPARQL engine (QLever) and the RDF dump faithfully evaluate the property-path queries used to compute counts.
All counts come from executing these queries on a specific dump; engine bugs or query errors would change the numbers. The authors note they had to work around a QLever bug and memory constraints.
standard math No non-empty class can have more than one fixed order.
This follows from the definition of class order and is used to identify conflicts when classes appear in multiple order sets (Section 'Direct Determination of Class Order').

how reviews work

0 comments

Cite this review

Pith. "Pith review of Class Order Disorder in Wikidata and First Fixes." pith.science (2026). https://pith.science/paper/OCZWHR6A

@misc{pith2026241115550,
  author       = {Pith},
  title        = {Pith review of: Class Order Disorder in Wikidata and First Fixes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OCZWHR6A}},
  note         = {Machine review of arXiv:2411.15550}
}

read the original abstract

Wikidata has a large ontology with classes at several orders. The Wikidata ontology has long been known to have violations of class order and information related to class order that appears suspect. SPARQL queries were evaluated against Wikidata to determine the prevalence of several kinds of violations and suspect information and the results analyzed. Some changes were manually made to Wikidata to remove some of these results and the queries rerun, showing the effect of the changes. Suggestions are provided on how the problems uncovered might be addressed, either though better tooling or involvement of the Wikidata community.

Figures

Figures reproduced from arXiv: 2411.15550 by the authors.

**Figure 2.** Items per split-order class fauna of Asia (Q3509331) bad (Q4114379) bad (Q4243465) bad Central Plaza 1, Brisbane (Q5061672) bad Astrolabe of al-Sahlˆı (Q5710018) bad sounding rocket (Q579036) bad (Q6124703) bad Didachara mosque (Q66022902) bad long-acting reversible contraception (Q6671830) bad Estonian mythology (Q669000) bad Roman roads in Africa (Q7362325) bad Upper South Province (Q7898928) bad Fabio Valentini (… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

[1]

Abdulai, M. 2023. Wikidata: Ontology Issues Prioritization. www.wikidata.org/wiki/Wikidata:Ontology_issues_prioritization. Accessed 2024-07-23

work page 2023
[2]

Abdulai, M.; and Lacroix, L. 2023. Wikidata: Ontology Issues Prioritization. www.wikidata.org/wiki/Wikidata:Ontology_issues_prioritization. Accessed 2024-07-17

work page 2023
[3]

Bast, H.; and Buchhold, B. 2017. QLever : A Query Engine for Efficient SPARQL +Text Search. In CIKM '17: ACM Conference on Information and Knowledge Management. Singapore

work page 2017
[4]

Blazegraph. 2013. Welcome to Bladegraph. blazegraph.com. Accessed 2024-07-23

work page 2013
[5]

Brasileiro, F.; Almeida, J. P. A.; Carvalho, V. A.; and Guizzardi, G. 2016. Applying a Multi-Level Modeling Theory to Assess Taxonomic Hierarchies in W ikidata. In Wiki Workshop at WWW 2016: WWW '16 Companion for Proceedings of the 25th International Conference Companion on World Wide Web, 975--980

work page 2016
[6]

A.; Almeida, J

Dadalto, A. A.; Almeida, J. P. A.; Fonseca, C. M.; ; and Guizzardi, G. 2021. Type or Individual? E vidence of Large-Scale Conceptual Disarray in W ikidata. In 40th International Conference on Conceptual Modeling (ER 2021), 367--377

work page 2021
[7]

Kristen, M. 2020. COMO : A Lexicographical Data Structuring Game With a Purpose . Bachelor's thesis, Institut für Informatik, Ludwig-Maximilians-Universität München

work page 2020
[8]

Oceja, J.; and Sierra, A. O. 2018. Gamifiying Wikipedia? In 12th European Conference on Games Based Learning. Sophia Antipolis, France

work page 2018

Show all 19 references

[9]

Patel-Schneider, P. F. 2019. Barriers to Using W ikidata as a Knowledge Base. In WikidataCon 2019. Berlin. Accessed 2024-07-23

2019
[10]

Pintscher, L. 2023 a . Ontology Issues in W ikidata: Everything in neat and tidy boxes? Not quite! In WikidataCon 2023. Accessed 2024-07-23

2023
[11]

Pintscher, L. 2023 b . Wikidata Survey on Ontology Issues, Potential Solutions. www.wikidata.org/wiki/Wikidata_talk:Ontology_issues_prioritization#Overview_of_potential_solutions. Accessed 2024-07-17

2023
[12]

QuickStatments. 2022. Help:QuickStatements. www.wikidata.org/wiki/Help:QuickStatements. Accessed 2024-07-22

2022
[13]

Shenoy, K.; Ilievski, F.; Garijo, D.; Schwabe, D.; and Szekely, P. 2022. A Study of the Quality of W ikidata. Journal of Web Semantics, 72

2022
[14]

Shimizu, C.; Eells, A.; Gonzalez, S.; Zhou, L.; Hitzler, P.; Sheill, A.; Foley, C.; and Rehberger, D. 2024. Ontology Design Facilitating W ikibase Integration — and a worked example for historical data. Journal of Web Semantics, 82

2024
[15]

Turki, H.; Shafee, T.; Taieb, M. A. H.; Aouicha, M. B.; Vrandečić, D.; Das, D.; and Hamdi, H. 2019. Wikidata: A large-scale collaborative ontological medical database. Journal of Biomedical Informatics, 99

2019
[16]

Virtuoso. 2020. OpenLink Virtuoso Universal Server Documentation. docs.openlinksw.com/virtuoso/. Accessed 2024-07-23

2020
[17]

Vrande c i \'c , D.; and Kr \"o tzsch, M. 2014. Wikidata: A free collaborative knowledgebase. Communications of the ACM, 57(10): 78--85

2014
[18]

M.; Griffith, M.; Griffith, O

Waagmeester, A.; Stupp, G.; Burgstaller-Muehlbacher, S.; Good, B. M.; Griffith, M.; Griffith, O. L.; Hanspers, K.; Hermjakob, H.; Hudson, T. S.; Hybiske, K.; Keating, S. M.; Manske, M.; Mayers, M.; Mietchen, D.; Mitraka, E.; Pico, A. R.; Putman, T.; Riutta, A.; Queralt-Rosinac...

2020
[19]

Westerinen, A.; and Pintscher, L. 2023. Wikidata Challenges in the semantic web community. In Wikidata Data Modelling Days. Accessed 2024-07-23

2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Abdulai, M. 2023. Wikidata: Ontology Issues Prioritization. www.wikidata.org/wiki/Wikidata:Ontology_issues_prioritization. Accessed 2024-07-23

work page 2023

[2] [2]

Abdulai, M.; and Lacroix, L. 2023. Wikidata: Ontology Issues Prioritization. www.wikidata.org/wiki/Wikidata:Ontology_issues_prioritization. Accessed 2024-07-17

work page 2023

[3] [3]

Bast, H.; and Buchhold, B. 2017. QLever : A Query Engine for Efficient SPARQL +Text Search. In CIKM '17: ACM Conference on Information and Knowledge Management. Singapore

work page 2017

[4] [4]

Blazegraph. 2013. Welcome to Bladegraph. blazegraph.com. Accessed 2024-07-23

work page 2013

[5] [5]

Brasileiro, F.; Almeida, J. P. A.; Carvalho, V. A.; and Guizzardi, G. 2016. Applying a Multi-Level Modeling Theory to Assess Taxonomic Hierarchies in W ikidata. In Wiki Workshop at WWW 2016: WWW '16 Companion for Proceedings of the 25th International Conference Companion on World Wide Web, 975--980

work page 2016

[6] [6]

A.; Almeida, J

Dadalto, A. A.; Almeida, J. P. A.; Fonseca, C. M.; ; and Guizzardi, G. 2021. Type or Individual? E vidence of Large-Scale Conceptual Disarray in W ikidata. In 40th International Conference on Conceptual Modeling (ER 2021), 367--377

work page 2021

[7] [7]

Kristen, M. 2020. COMO : A Lexicographical Data Structuring Game With a Purpose . Bachelor's thesis, Institut für Informatik, Ludwig-Maximilians-Universität München

work page 2020

[8] [8]

Oceja, J.; and Sierra, A. O. 2018. Gamifiying Wikipedia? In 12th European Conference on Games Based Learning. Sophia Antipolis, France

work page 2018