{"id":"32fbcd1f-6a65-4b33-95ef-ba0e481af108","arxiv_id":"2501.04613","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A class-based partitioning heuristic for parallel knowledge graph embedding training, evaluated on FB15K and FB15K-237, shows modest and model-dependent gains over random partitioning.","lead":"This paper proposes sorting the triples of a knowledge graph by the class of their head entity, so that parallel training of embeddings also uses ontology information. It is a candidate way to make large-scale embeddings more semantically meaningful, though the reported gains are small, mixed across models, and not yet backed by the promised semantic evaluation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Semantic-information benefit is never measured: the only positive result is a single unseeded DistMult MRR gain, so the central claim is not supported by Table 2.","rationale":"The reader's weakest assumption concerns how overlapping entity embeddings are merged across partitions; that is a real implementation gap, but the even more fundamental issue is that the paper's central semantic claim has no direct evidence. The reader's rationale already notes the absence of entity typing, so my concern partially overlaps with the reader's. The method could be fully specified and still fail the semantic claim if entity typing does not improve on semantic embeddings. I therefore keep the verdict conditional: the method may be salvageable by adding the promised entity-typing evaluation and variance estimates, but the current evidence is insufficient. No code is released with the paper, and there is no machine-checked proof or independent reproduction, so Table 2 is the sole support for the central claim.","tokens_in":8301,"tokens_out":5784,"duration_ms":57669,"concrete_test":"Run each configuration (semantic vs random, TransE/DistMult/ComplEx, FB15K and FB15K-237) with at least 5 random seeds, reporting mean ± std, and add a centralized no-partition baseline. Then perform entity typing on the resulting embeddings using the Freebase type hierarchy at two granularities (e.g., Person versus Scientist) and compare F1 between semantic and random embeddings, as Section 5 specifies but does not execute. If the DistMult MRR advantage lies within one standard deviation and semantic embeddings do not achieve higher typing F1, the semantic-information claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section 4 claim that the partitioner 'includes more semantic information' in the learned embeddings, yet no semantic evaluation is reported. Section 5 states 'The obtained embeddings are then utilized for entity classification tasks' and Section 6/7 list entity classification as planned future work, but the paper contains no F1, precision, or recall numbers. The only evidence for the method is link prediction in Table 2, and that evidence is not robust: on FB15K-237, semantic partitioning beats random only for DistMult (+0.7 MRR, +0.7 Hits@10), is worse for ComplEx (-0.1 MRR, -0.5 Hits@10), and is essentially tied for TransE. On FB15K, DistMult gains +2.2 MRR/+1.3 Hits@10, but ComplEx loses 1.0 MRR/0.5 Hits@10 and TransE is identical. No error bars, seeds, or repeated runs are reported, so the DistMult gain could be partition-size imbalance or initialization noise rather than the semantic grouping; the paper's own Fig. 1 shows extreme class-frequency imbalance, so semantic and random partitions differ substantially in size. Because the asserted mechanism—richer semantic information—is never directly tested, the central claim that class-based partitioning improves large-scale KGE training is currently supported only by a fragile, model-dependent link-prediction delta.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semantic partitioning method for large-scale knowledge graph embedding training. Entities are assigned to their lowest ontology class, fact triplets are partitioned according to the class of the head entity, and the resulting partitions are trained in parallel with DGL-KE using TransE, DistMult, and ComplEx. The authors compare their method with random partitioning on FB15K and FB15K-237, reporting MRR and Hits@10 in Table 2. The abstract claims that the method incorporates ontology information, injects semantic information into embeddings, and performs well on popular benchmarks, but the paper also states that the current results are preliminary and that entity classification, which would test the semantic claim, is left to future work.","tokens_in":8547,"tokens_out":3807,"duration_ms":40108,"significance":"If the central claims were convincingly supported, the method would be an attractive low-cost preprocessing step: it is model-agnostic, uses existing ontology data, and fits naturally into parallel training. The paper's main strength is the simplicity of the proposed partitioning rule and the use of a standard, publicly available training library (DGL-KE). However, the evidence presented is not sufficient to establish the claimed benefits. The abstract's statement that the algorithm 'performs well on several popular benchmarks' is not supported by the mixed results in Table 2, and the paper's core assertion that the embeddings contain 'more semantic information' is never directly tested. The current contribution is therefore best viewed as an early-stage idea requiring substantial additional evaluation.","major_comments":[{"comment":"The central claim that the method 'performs well on several popular benchmarks' is not supported by the reported numbers. On FB15K-237, semantic partitioning improves over random partitioning only for DistMult (MRR 26.1 vs 25.4), while ComplEx is slightly worse (25.3 vs 25.4) and TransE gains only 0.1 MRR. On FB15K, TransE results are identical to random partitioning (69.2 MRR and 85.9 Hits@10), and ComplEx is worse by 1.0 MRR and 0.5 Hits@10. The only consistent positive result is DistMult, and no error bars, seeds, or repeated runs are reported. The paper's own Section 6 text acknowledges that the method is model-dependent and inferior to random partitioning for ComplEx, which contradicts the abstract's blanket claim of good performance.","section":"Section 6, Table 2"},{"comment":"The claimed mechanism, that class-based partitioning yields embeddings with more semantic information, is never directly evaluated. Section 5 states that 'the obtained embeddings are then utilized for entity classification tasks,' but no precision, recall, or F1 results appear anywhere in the paper; Sections 6 and 7 list entity classification as future work. Without any entity classification or other semantic evaluation, the assertion that the method 'preserves more semantic information' is an unsupported conjecture rather than a demonstrated result.","section":"Section 5, Section 6, Section 7"},{"comment":"The paper does not explain how embeddings of entities appearing in multiple partitions are updated, synchronized, or merged during parallel training. Section 5 says only that each fact triplet is classified based on the class of its head entity, but an entity can belong to several classes and appear in several partitions. Since the parallel implementation (DGL-KE) has its own consistency mechanisms, the observed differences in Table 2 cannot be attributed to semantic grouping without a description of how entity embeddings are reconciled across partitions and without verifying that the final vectors are consistent. The class-frequency imbalance shown in Figure 1 also means semantic and random partitions have very different sizes, so partition-size effects are a plausible confound.","section":"Section 5"},{"comment":"The paper lists subgraph selection as the first sub-problem and as a proposed contribution, but no experiment or evaluation of subgraph selection is provided. Section 2 defines sub-problem (1) as finding the subgraph of size p*s that maximizes prediction quality, and Section 4 claims a subgraph selection method, yet all reported results concern full-graph partitioning for link prediction. This is an untested part of the stated contribution and should be either evaluated or explicitly removed from the paper's claims.","section":"Section 2 and Section 4"}],"minor_comments":[{"comment":"The citations for the 'First Proposed' rows are incorrect: DistMult is cited as [4] and ComplEx is also cited as [4], but reference [4] is the ConvE paper. DistMult should cite Yang et al. [12] and ComplEx should cite Trouillon et al. [27].","section":"Table 2"},{"comment":"The 'First Proposed' values in Table 2 are taken from prior studies and may have been obtained with different training configurations than the DGL-KE-based random and semantic partitioning runs. A fair comparison should either use the same backbone and hyperparameters for all rows or clearly state that the first-proposed rows are literature reference points, not controlled comparisons.","section":"Section 6"},{"comment":"Figure 1, 'Freebase class frequency analysis,' has no visible axis labels or class names, and the text does not describe how class frequency was computed. For a reader to judge the claimed class imbalance, the figure should include labeled axes and a short description of the counting procedure.","section":"Figure 1"},{"comment":"The last paragraph of Section 5 says that the proposed partitioning is 'a preliminary, simple semantic partitioning method and requires further refinement.' This caveat is important, but the paper does not quantify how much refinement is needed or which aspects of the method are tentative.","section":"Section 5"},{"comment":"The sentence 'If we look at the preliminary link prediction results we obtained now, our method performs well in overall' is difficult to reconcile with the immediately preceding sentences reporting model-dependent and partly inferior results. Please either rephrase the conclusion to reflect the mixed evidence or provide statistical support for the claim.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an early-stage extended abstract rather than a completed study. The idea of ontology-guided partitioning is reasonable and worth pursuing, but the current evidence is too thin for the abstract's claims. I would recommend requiring (a) entity classification results or another direct test of semantic content, (b) repeated runs with error bars or significance tests, and (c) a clarification of how overlapping entity embeddings are handled in parallel training. If the authors cannot provide these, the claims should be scaled back to a preliminary, model-dependent observation. The paper may be more appropriate for a workshop venue than for a full journal publication, but the revision path is feasible within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this as a workshop report, not a finished research claim. The idea is genuinely simple: assign each entity to its lowest class, partition fact triples by the class of the head entity, then train standard KGE models (TransE, DistMult, ComplEx) in parallel. That exact combination is not in the cited literature, so it qualifies as a small new preprocessing heuristic. The paper also does something right: it reproduces a random-partitioning baseline and is openly honest that the gains are model-dependent.\n\nThat honesty makes the abstract curious. It says the method 'performs well on several popular benchmarks' and 'includes more semantic information', but neither claim is established. The only experimental results are link prediction in Table 2; entity classification is described as planned future work. On FB15K-237, only DistMult beats random (25.4 vs 26.1 MRR); ComplEx is worse and TransE is flat. On FB15K, DistMult gains 2.2 MRR, but ComplEx loses 1.0 and TransE is identical. There are no error bars, no seeds, no repeated runs, so the DistMult improvements could easily be variance. The paper's own Fig. 1 shows severe class-frequency imbalance, which makes the semantic and random partitions differ in size, another confound. Given that the stated mechanism is 'more semantic information in the embeddings', the absence of any entity-typing result is the central gap: the mechanism is asserted, not measured.\n\nTwo smaller soft spots. First, the paper never says how the rdf:type labels were obtained for FB15K entities, which matters because the whole method depends on class assignments. Second, entities appear in multiple partitions (a person is both Person and Writer), and the paper doesn't specify how overlapping embeddings are updated or reconciled across partitions. If that step is handled by DGL-KE's machinery, then the results tell us about DGL-KE, not about the semantic value of the partition.\n\nHats off for the limitations paragraph; the paper knows what it hasn't done. But the abstract and conclusion overstate the evidence.\n\nWho should read it: people tracking work on ontology-aware KGE preprocessing or parallel training might skim this as a negative data point and a useful baseline. Cite? Probably not, unless the exact lowest-class heuristic becomes relevant. For a workshop/short-paper venue with a 'reproducibility in progress' policy, it's acceptable. As a serious archival submission, I'd desk reject or send back for major revision because the load-bearing semantic-evaluation claim is missing. Should a serious editor spend referee time now? I'd say yes only if the venue explicitly welcomes position papers; otherwise no.","headline":"Simple ontology-based partitioning idea, honestly reported, but the semantic claim is untested and the link prediction evidence is mostly noise.","tokens_in":9090,"tokens_out":4008,"would_cite":false,"duration_ms":42448,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that partitioning knowledge-graph triples by the class of the head entity lets embeddings train in parallel without sacrificing link prediction quality.","keywords":["knowledge graph embeddings","semantic partitioning","ontology-aware partitioning","parallel training","link prediction","entity typing","Freebase","distributed embedding training"],"falsifier":"Run the same parallel training pipeline on the same datasets with semantically partitioned and randomly partitioned triplets, while also reporting how overlapping entity embeddings are merged; if random partitioning matches or beats semantic partitioning on both link prediction metrics and entity-typing F1 across class levels, then the class information in the cut is not what drives the results.","tokens_in":8048,"feed_emoji":"🧩","tokens_out":10423,"duration_ms":88348,"temperature":0.7,"pith_summary":"The paper claims that when a knowledge graph is too large to train embeddings on in a single pass, the right way to cut it into pieces is by the semantic class of the head entity rather than at random. This class-based partitioning keeps the scoring functions of existing embedding models untouched, so it can be dropped into most current methods, and it turns the class information of the ontology into an organizing principle for parallel training. On the Freebase-derived benchmarks FB15K and FB15K-237, using TransE, DistMult, and ComplEx, the preliminary results are generally comparable to random partitioning, with DistMult showing the clearest improvement. The paper presents this as a first step toward embedding training that is both scalable and semantically informative, with entity typing as the planned test of whether the extra semantic information actually helps.","feed_headline":"Class-based cuts train knowledge-graph embeddings without quality loss","feed_subtitle":"Class-aware splits match random baselines on link prediction while enabling parallel training.","key_machinery":"The machinery is ontology-based semantic partitioning. A knowledge graph is represented as $KG=(E,R,T)$ with entities, relations, and fact triplets; the ontology supplies class labels and rdf:type statements. Each entity is assigned to its lowest-level class to maximize the number of partitions, and every fact triplet is routed to the partition named by its head entity's class. This cut is what lets the method keep any existing model's scoring function while injecting class-level semantics into training, and it is what makes the workload parallelizable. The evaluation runs the partitioned training through a parallel knowledge-graph embedding library using TransE, DistMult, and ComplEx with the library's best hyperparameters on 64 CPUs.","core_discovery":"The paper's central claim is that ontology information should participate in large-scale knowledge graph embedding through the partition step, not through a new scoring function. Concretely, each entity is assigned to its most specific (lowest) class, and each fact triplet $\\langle h,r,t\\rangle$ is placed in the partition of its head entity's class, so every partition contains triples that share a semantic category. Training these partitions in parallel with off-the-shelf models then yields embeddings that are meant to carry more semantic information while matching the link prediction quality of a centralized or randomly partitioned run. The reported numbers bear this out unevenly: on FB15K-237, DistMult improves from MRR 25.4 to 26.1 and Hits@10 from 43.7 to 44.4, TransE is nearly unchanged, and ComplEx is slightly worse; on FB15K, DistMult again improves on both metrics, with TransE and ComplEx close to random partitioning. The paper states these are preliminary results and that the benefit is model-dependent.","pith_inferences":["A broader design principle is suggested: the partition boundary is itself an inductive bias, so the same idea could be tested with relation-based cuts, hierarchy-aware cuts, or cuts that respect subclass edges, not just head-entity class.","A decisive test the paper announces but does not run is entity typing F1 at several class levels; if low-level classes improve there while link prediction is flat, the semantic-information claim would be supported independently of ranking metrics.","The unresolved mechanism is how embeddings of entities shared across partitions stay consistent; until that synchronization is specified, some of the observed behavior may come from the parallel training infrastructure rather than from the semantic content of the cut.","The method could combine with ontology-embedding techniques that encode subclass structure, using the class partition to scale those techniques to graphs much larger than the centralized datasets where they are usually evaluated."],"forward_implications":["Very large knowledge graphs that do not fit on one machine could be trained in parallel by class-based partitions while keeping the scoring function of the chosen embedding model unchanged.","The method is a preprocessing layer: any translational, tensor-decomposition, or deep knowledge graph embedding model should be able to adopt it without retraining internals.","Partitioning by the most specific class increases the number of partitions and the parallelism, while concentrating triples with shared semantic categories in each partition.","If the semantic-information claim holds, downstream tasks such as entity typing and entity alignment should benefit even on benchmarks where link prediction is flat.","The best partition-and-negative-sampling configuration is likely model- and dataset-dependent, matching the paper's observation that DistMult gains while ComplEx loses."],"supporting_citations":[{"why":"defines the TransE translational scoring model used as one of the three testbed models.","marker":"[6]"},{"why":"introduces the Freebase knowledge graph, the source of the FB15K benchmark and the class-frequency analysis.","marker":"[10]"},{"why":"defines the DistMult bilinear scoring model, the case where semantic partitioning shows the clearest gains.","marker":"[12]"},{"why":"supplies evidence that standard embeddings do not capture knowledge graph semantics, motivating the use of ontology information.","marker":"[14]"},{"why":"compares partitioning and negative sampling techniques for parallel knowledge graph embedding training, motivating the claim that the partition choice matters.","marker":"[17]"},{"why":"introduces FB15K-237, the inverse-relation-free benchmark used to avoid test leakage.","marker":"[26]"},{"why":"defines the ComplEx complex-embedding model, the testbed where semantic partitioning performs slightly worse than random partitioning.","marker":"[27]"},{"why":"supplies the parallel knowledge graph embedding training and evaluation library on which both the random and semantic partitioning runs are implemented.","marker":"[36]"}],"fun_headline_variants":["Class-based cuts enable parallel KG embedding training","Semantic partitioning yields model-dependent gains in KG embeddings","Ontology-based partition lets you train KG embeddings in parallel","Class-based cuts speed up KG embedding training, with caveats"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that training on class-based partitions produces valid embeddings even though the same entity appears in many partitions, and the paper does not specify how the shared entity vectors are kept consistent across partitions.","fun_headline_variants_meta":{"raw":{"variants":["Class-based cuts enable parallel KG embedding training","Semantic partitioning yields model-dependent gains in KG embeddings","Ontology-based partition lets you train KG embeddings in parallel","Class-based cuts speed up KG embedding training, with caveats"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000702,"raw_usage":{"total_tokens":3130,"prompt_tokens":872,"completion_tokens":2258,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":2194}},"tokens_in":488,"tokens_out":2258,"duration_ms":17696,"temperature":1.0,"reasoning_tokens":2194,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:29:05.408420+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same parallel training pipeline on the same datasets with semantically partitioned and randomly partitioned triplets, while also reporting how overlapping entity embeddings are merged; if random partitioning matches or beats semantic partitioning on both link prediction metrics and entity-typing F1 across class levels, then the class information in the cut is not what drives the results.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the TransE translational scoring model used as one of the three testbed models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces the Freebase knowledge graph, the source of the FB15K benchmark and the class-frequency analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies evidence that standard embeddings do not capture knowledge graph semantics, motivating the use of ontology information."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"compares partitioning and negative sampling techniques for parallel knowledge graph embedding training, motivating the claim that the partition choice matters."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the ComplEx complex-embedding model, the testbed where semantic partitioning performs slightly worse than random partitioning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the parallel knowledge graph embedding training and evaluation library on which both the random and semantic partitioning runs are implemented."}],"review_version":1}