{"id":"7e1dcd28-8059-44be-afa4-7c1e74633222","arxiv_id":"2504.13631","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"VSNS, a two-step neighbor filter based on relation visualizability and CompGCN structural similarity, generates prompts that yield images better aligned with knowledge graphs.","lead":"This paper proposes an automated pipeline that turns plain knowledge graphs into multi-modal knowledge graphs by generating images for each entity from LLM-written prompts. The key idea, VSNS, selects which neighboring facts to include in the prompt by filtering relations for visualizability and neighbors for structural similarity.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The FID metric is computed per-image with min over three real images, not as a distribution distance; this undermines the quantitative evidence for the central claim of higher-quality images.","rationale":"The reader's weakest_assumption focused on the SNS module's implicit claim that structural similarity predicts visual usefulness (Eq. 4). That is a legitimate construct-validity concern about the mechanism, but the paper's central claim is comparative: VSNS produces higher-quality and more KG-relevant images than baselines. The human evaluation already shows only modest and sometimes negative gains in image quality, so the quantitative FID evidence is critical. The reported FID computation is inconsistent with the standard definition and likely degenerates to a per-image nearest-neighbor distance, which does not measure distributional fidelity. This directly threatens the 'higher-quality' component of the headline claim. The relevance component (CIE/CIKG) is better supported by human evaluation, so the paper could still be conditionally acceptable if the metric issue is addressed. Hence the reader's CONDITIONAL verdict remains appropriate, but the condition should explicitly include recomputing FID correctly. My concern differs from the reader's identified assumption, so agreement is partial.","tokens_in":10842,"tokens_out":5694,"duration_ms":54147,"concrete_test":"Recompute image quality on MKG-Y using the standard FID over the full set of generated images versus the full set of real images (or a proper per-image distance such as LPIPS or CLIP image-image cosine, clearly labeled as such), and compare Isvns against Is and Im on the same entity subsets. Also report results without the pairwise filtering and with error bars or significance tests. If the FID advantage of Isvns disappears or reverses under the standard computation, the quantitative support for the headline claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim that VSNS generates 'higher-quality images' is quantitatively supported mainly by FID results in Tables II, IV, and V. However, the FID calculation described in Section V-C does not follow the standard definition cited in Section V-A. Instead of computing the Fréchet distance between feature distributions of generated and real image sets, the authors state: 'For FID score calculation, we compare the generated image with each of the three real images separately and select the smallest FID value.' With a single generated image and a single real image, the covariance matrices are zero, so the 'FID' degenerates to a squared Euclidean distance between two feature vectors. The minimum over three real images rewards a generated image that matches any one reference rather than the reference distribution. This is not the distribution-level metric described in Heusel et al. (2017). Human evaluation in Table III shows that on MKG-Y, the image quality (IQ) of Isvns (2.68) is actually lower than that of Is (2.72), so the quality claim rests almost entirely on the questionable FID numbers. The relevance gains (CIE, CIKG) are supported by human evaluation, but the 'higher-quality' part of the central claim is not reliably established by the reported quantitative metrics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an automated pipeline for constructing multi-modal knowledge graphs from textual KGs by generating an image for each entity. The core contribution is VSNS, a two-stage neighbor-selection method: VNS filters relations whose triples receive positive ImageReward scores, and SNS ranks the remaining one-hop neighbors by cosine similarity of CompGCN embeddings, keeping neighbors above the mean. The selected neighbors are fed to an LLM to produce semantics-enriched prompts, which are then used by Stable Diffusion to generate entity images. The method is evaluated on MKG-Y and DB15K against two baselines: images from entity names alone (Is) and images from the longest-token neighbor (Im), using FID, CLIPscore, human ratings (IQ, CIE, CIKG), and downstream multi-modal knowledge graph completion with NATIVE.","tokens_in":11163,"tokens_out":4989,"duration_ms":45908,"significance":"If the claims were fully supported, the framework would provide a scalable way to generate contextually relevant images for KG entities without manual prompt engineering, and the downstream MMKGC results suggest that synthetic images can be useful for link prediction. The paper has strengths: it builds on external scoring models (ImageReward, CompGCN, CLIP) rather than tuning on the evaluation metrics; the human-evaluation protocol is described in detail; and the downstream reasoning task gives an application-oriented check. However, the central quality claim is not currently supported: the automatic metric labeled FID is not the standard Fréchet distance, and the human IQ results on MKG-Y contradict the 'higher-quality' part of the abstract. The relevance and KG-consistency improvements are more credible but would benefit from statistical testing. Methodologically, the SNS selection rule's core assumption that structural similarity implies visual prompt usefulness is not tested in isolation.","major_comments":[{"comment":"The reported 'FID' is not the Fréchet Inception Distance defined by Heusel et al. [23]. Section V-C states: 'For FID score calculation, we compare the generated image with each of the three real images separately and select the smallest FID value.' With one generated and one real image, covariance matrices cannot be estimated, so this quantity reduces to a per-image feature distance, not a distribution-level divergence. Taking the minimum over three references rewards matching any one real image and makes the numbers systematically lower. Since the abstract's 'higher-quality images' claim is supported mainly by the FID improvements in Tables II, IV, and V, this claim is not quantitatively established. Please either recompute a standard FID over the full generated and real image sets or relabel the metric and explicitly withdraw the distribution-level quality inference.","section":"V-C, Automatic Evaluation; Tables II, IV, V"},{"comment":"The human evaluation results do not support the abstract's blanket claim of 'higher-quality images that are more relevant' to the KG. On MKG-Y, IQ(Isvns)=2.68 is lower than IQ(Is)=2.72; on DB15K, CIE(Isvns)=2.27 is lower than CIE(Is)=2.31. The human data support only selective improvements, such as CIKG on both datasets and IQ on DB15K. The explanation of lower IQ for neighbor-based images (incomplete body parts, repetitive generation) is descriptive and does not address the comparison against Is. Please either restrict the quality claim to the settings where it is actually observed or provide evidence that the small observed differences are not within annotation noise.","section":"Table III, Human Evaluation"},{"comment":"The SNS module assumes that the cosine similarity between CompGCN entity embeddings and composed neighbor embeddings (Eq. 4) predicts how useful a neighbor is for visual prompt generation. This assumption is plausible but is not tested in isolation; Table V evaluates the full VSNS pipeline, so the observed improvement could stem from VNS filtering, the LLM prompt template, or the prompt length rather than from the structural ranking. Please provide a controlled comparison that holds relations and prompt template fixed and compares SNS-selected neighbors against random or longest-token neighbors of equal size, and report whether Snei correlates with human CIE/CIKG scores.","section":"IV-A2, Eqs. (3)-(5)"},{"comment":"The human evaluation uses small samples (50 entities on MKG-Y, 100 on DB15K) with three annotators, but no inter-annotator agreement, variance, or significance testing is reported. Several margins are tiny (e.g., MKG-Y CIE 2.27 vs 2.06; DB15K IQ 2.26 vs 2.06). Without confidence intervals or a paired test, the claimed advantages may be within annotation noise. Please report per-criterion score distributions, agreement statistics, and appropriate significance tests.","section":"V-C, Human Evaluation"}],"minor_comments":[{"comment":"The acronym is inconsistent: the paper defines VSNS but Section V-A and V-D repeatedly use 'SVNS'; please standardize the terminology.","section":"V-A, V-D"},{"comment":"The column header '#C Number' is unclear; define 'C' (presumably the number of compared entities) and clarify why the two baseline comparisons use different entity subsets, which prevents direct cross-row comparison of FID/CLIPscore values.","section":"Table II"},{"comment":"The VNS threshold µ=0.5 and the number of sampled triples per relation (ten) are free parameters; please include a sensitivity analysis or justify these choices.","section":"V-C, Implementation Details"},{"comment":"The automatic metrics are computed only on MKG-Y because many DB15K image URLs are unavailable; this limitation should be stated explicitly in the main experimental section rather than only in the implementation details.","section":"V-C, Automatic Evaluation"},{"comment":"The row label 'S+Isvns NATIVE' is redundant and confusing; also, the MMKGC gains over S+I are small (e.g., MRR 0.383 to 0.387 on MKG-Y) and significance is not reported, so the claim that generated images are 'comparable or even superior' to real images should be phrased more cautiously.","section":"Table VI"}],"recommendation":"major_revision","confidential_remarks":"The FID mislabeling is the main technical obstacle; I would ask the authors to provide the exact per-entity computation and, if possible, the feature vectors so a standard FID can be computed or ruled out. The human-evaluation sample sizes are small enough that a statistical check is warranted before publication. The use of the authors' own NATIVE model as a downstream tool is appropriate and does not raise circularity concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper to know about: Xu et al. propose a pipeline for auto-generating images for KG entities. They filter relations by visualizability using ImageReward (VNS), then select neighbors with CompGCN embeddings and cosine similarity (SNS), feed the chosen neighbors to an LLM to write a prompt, and generate the image with Stable Diffusion. VSNS is genuinely new relative to the cited baseline [5], which just picks longest-token neighbors, and the integration is clean. Credit where it is due: the two-module design makes sense, the evaluation includes human judgments and a downstream MMKGC task, and the authors honestly flag limitations (abstract entities, unvalidated QA).\n\nThe central claim—that VSNS yields \"higher-quality images\"—is not supported by their own evidence. The automatic FID numbers that drive that claim are not actual FID: the paper computes, for each generated image, the distance to the best of three real images, which collapses to a squared Euclidean distance between two feature vectors. That is not the Fréchet distance between distributions from Heusel et al. And their own human IQ scores on MKG-Y put Isvns at 2.68 versus 2.72 for the no-neighbor baseline. So the \"quality\" part of the abstract should be dropped or substantially softened. The relevance claims (CIE, CIKG) are backed by human ratings, and the DB15K MRR improvement (0.368 vs 0.355) is modest but directionally consistent.\n\nOther soft spots: the SNS assumption that structural embedding similarity tracks prompt usefulness is plausible but never tested in isolation—Table V only compares the full pipeline. No code or data are released, so independent replication is hard. The FID computation should be redone as a proper distributional statistic, and the paper should be revised to claim relevance improvement rather than unconditional quality improvement. These are addressable, not fatal.\n\nThis paper is for readers working on MMKG construction or on using generative models to enrich KGs. It deserves a serious referee, but the referee should insist on corrected metrics and a more careful claim. I would not cite the FID numbers, but the VSNS method and the downstream finding (synthetic images can substitute for real images in DB15K completion) are worth citing. Send it to peer review with a request for revision.","headline":"A useful pipeline for generating images from KG neighbors, but the headline 'higher-quality' claim rests on a mis-applied FID and is contradicted by the paper's own human ratings; the relevance and downstream gains are the real contribution.","tokens_in":11685,"tokens_out":1659,"would_cite":true,"duration_ms":16126,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Selecting an entity's neighbors by how visualizable their relation is and how structurally similar the neighbor is to the entity produces images that are more relevant to both the entity and its knowledge graph than images from…","keywords":["multi-modal knowledge graph","knowledge graph construction","neighbor selection","prompt engineering","text-to-image generation","image reward model","knowledge graph completion","graph embeddings"],"falsifier":"Take a set of entities with several one-hop neighbors and compare images generated using SNS-selected neighbors against images generated using the same number of neighbors chosen at random from the same relations; if FID, CLIPscore, and human entity-relevance ratings do not favor the SNS selection, the structural similarity criterion is not doing the work. A second check: find a relation where the highest-similarity neighbor is known to be visually irrelevant (for example, an abstract relation such as 'influences') and show that forcing that neighbor into the prompt lowers human entity-relevance scores.","tokens_in":10679,"feed_emoji":"🖼️","tokens_out":9768,"duration_ms":81463,"temperature":0.7,"pith_summary":"This paper argues that a knowledge graph can be turned into a multi-modal one automatically by generating an image for each entity from a prompt, but the quality of that image hinges on which neighboring triples are allowed into the prompt. It proposes a two-stage neighbor filter: first discard relations whose textual descriptions do not score well on an image-reward model, then keep only one-hop neighbors whose graph embeddings are most similar to the entity's embedding. A language model turns the surviving triples into a short descriptive prompt, and a diffusion model renders the image. On two multi-modal knowledge graph datasets the method beats generating from the entity name alone and from the longest-token neighbor baseline on FID, CLIPscore, and human ratings of entity and graph relevance, and the generated images support link prediction roughly as well as real images.","feed_headline":"Knowledge-graph images improve when neighbors are chosen by structure","feed_subtitle":"A two-stage neighbor filter beats name-only and longest-token baselines on FID, CLIPscore, and human ratings.","key_machinery":"The load-bearing object is the two-stage selection score inside VSNS. Stage one, VNS, defines a relation-level visualizability score $r_{\\mathrm{vis}}$ as the fraction of sampled triples of that relation whose text-to-image pair is scored above zero by ImageReward, a model trained on human preferences that scores how well an image matches a text prompt, and keeps only relations with $r_{\\mathrm{vis}}$ above a threshold $\\mu$. Stage two, SNS, embeds the graph with CompGCN, a graph convolutional network that learns entity and relation embeddings, composes a neighbor embedding $e_{(r,t)}=\\phi_{\\mathrm{CompGCN}}(e_r,e_t)$, and keeps neighbor $t$ of head $h$ when the cosine similarity $\\mathrm{sim}(e_h,e_{(r,t)})$ is at least the average similarity over all one-hop neighbors of $h$ through that relation. This score is what turns raw graph structure into a prompt-worthy subset, and it is the component whose removal in the ablation degrades FID and CLIPscore.","core_discovery":"The paper's central claim is that neighbor selection, not prompt length or entity-name-only prompting, is what determines whether a generated image is faithful to an entity and its knowledge graph context. The proposed VSNS method first computes a visualizability score per relation by sampling triples, generating an image from each with a diffusion model, and using an image-reward model to decide whether that relation's semantics can be rendered; relations below a threshold are dropped. For surviving relations, CompGCN embeddings of the head entity and the composed neighbor representation are compared by cosine similarity, and only neighbors at or above the average similarity for that relation are kept. The selected triples are fed to a language model with a fixed instruction to produce a one-sentence semantics-enriched prompt, and Stable Diffusion generates the final image. Quantitative results on two datasets show lower FID and higher CLIPscore than both baselines, human evaluators rank the VSNS images above both baselines on entity relevance and knowledge-graph relevance, and on the DB15K link-prediction test the generated images yield higher MRR and Hits@1 than real images when added to structural features.","pith_inferences":["A direct test the paper does not run: compare SNS against random neighbors sampled from the same relations at the same count. If random selection matches SNS on FID and human relevance, the structural ranking is not the cause of the gain; this would also isolate how much of the improvement comes from VNS's relation filtering alone.","The relation-level visualizability threshold is global ($\\mu=0.5$); per-domain thresholds might preserve relations that are easy to visualize for some entity types but not others.","The same selection pipeline could be reused for other modalities, such as selecting textual or audio neighbors for entity video generation, wherever a structural-similarity score can be computed.","A boundary the authors name is abstract entities: applying the pipeline to emotions or events likely requires additional cues, such as temporal or sentiment information, before the structural-similarity criterion becomes meaningful."],"forward_implications":["Knowledge graphs with no existing images can be enriched automatically: entity name, graph neighbors, a language model, and a diffusion model are all that is needed, so the pipeline scales to the size of the graph.","Because the generated images improve FID and CLIPscore over name-only and longest-token prompts, the neighbor filter is the effective ingredient, not the prompt length.","Generated images can stand in for real images in multi-modal knowledge graph completion: on the DB15K link-prediction experiment, structural features plus generated images beat structural features plus real images on MRR and Hits@1.","Filtering hard-to-visualize relations first (VNS) mainly helps entities with few relation types, while pruning neighbors by structure (SNS) mainly helps entities with many neighbors, so the two modules are complementary.","The framework targets attribute-style multi-modal knowledge graphs, meaning each entity receives an image attribute; entity-style multimodal graphs would need different construction logic."],"supporting_citations":[{"why":"Supplies the longest-token neighbor baseline and motivates the claim that triple content, not just triple count, drives image quality.","marker":"[5]"},{"why":"Supplies the DB15K multi-modal knowledge graph dataset used for evaluation.","marker":"[12]"},{"why":"Gives the ImageReward model that scores text-image alignment in the VNS visualizability filter.","marker":"[19]"},{"why":"Supplies CompGCN embeddings and the composition operator that SNS uses to rank one-hop neighbors.","marker":"[20]"},{"why":"Supplies the underlying knowledge-base data from which the MKG-Y evaluation set is derived.","marker":"[21]"},{"why":"Defines FID, the automated metric used to compare generated images with real image distributions.","marker":"[23]"},{"why":"Defines CLIPscore, the automated metric used to measure semantic alignment between generated and real images.","marker":"[24]"},{"why":"Provides the CLIP encoders used for CLIPscore and image feature extraction.","marker":"[25]"},{"why":"Provides the NATIVE method used to test whether generated images improve multi-modal knowledge graph completion.","marker":"[26]"}],"fun_headline_variants":["Neighbor choice, not prompt length, makes KG images faithful","Two-step filter picks better images for knowledge graphs","Visualizable neighbors beat name-only prompts for KG images","Selecting neighbors structurally improves generated KG images","Why some KG images look right: It's the neighbor filter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that carries the method is that a neighbor's structural similarity to an entity, measured by CompGCN embedding cosine similarity, tracks how much that neighbor will help a text-to-image model draw the entity correctly; if structural similarity and visual usefulness diverge, the selection step can discard informative neighbors and keep visually misleading ones.","fun_headline_variants_meta":{"raw":{"variants":["Neighbor choice, not prompt length, makes KG images faithful","Two-step filter picks better images for knowledge graphs","Visualizable neighbors beat name-only prompts for KG images","Selecting neighbors structurally improves generated KG images","Why some KG images look right: It's the neighbor filter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000587,"raw_usage":{"total_tokens":2767,"prompt_tokens":968,"completion_tokens":1799,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":1722}},"tokens_in":584,"tokens_out":1799,"duration_ms":12251,"temperature":1.0,"reasoning_tokens":1722,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:02:41.082927+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of entities with several one-hop neighbors and compare images generated using SNS-selected neighbors against images generated using the same number of neighbors chosen at random from the same relations; if FID, CLIPscore, and human entity-relevance ratings do not favor the SNS selection, the structural similarity criterion is not doing the work. A second check: find a relation where the highest-similarity neighbor is known to be visually irrelevant (for example, an abstract relation such as 'influences') and show that forcing that neighbor into the prompt lowers human entity-relevance scores.","supporting_citations":[{"cited_title":"Draw me like my triples: Leverag- ing generative AI for wikidata image completion,","cited_arxiv_id":null,"evidence_quote":"Supplies the longest-token neighbor baseline and motivates the claim that triple content, not just triple count, drives image quality."},{"cited_title":"MMKG: multi-modal knowledge graphs,","cited_arxiv_id":null,"evidence_quote":"Supplies the DB15K multi-modal knowledge graph dataset used for evaluation."},{"cited_title":"Imagereward: Learning and evaluating human preferences for text-to- image generation,","cited_arxiv_id":null,"evidence_quote":"Gives the ImageReward model that scores text-image alignment in the VNS visualizability filter."},{"cited_title":"Composition- based multi-relational graph convolutional networks,","cited_arxiv_id":null,"evidence_quote":"Supplies CompGCN embeddings and the composition operator that SNS uses to rank one-hop neighbors."},{"cited_title":"Wikidata: a free collaborative knowl- edgebase,","cited_arxiv_id":null,"evidence_quote":"Supplies the underlying knowledge-base data from which the MKG-Y evaluation set is derived."},{"cited_title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium,","cited_arxiv_id":null,"evidence_quote":"Defines FID, the automated metric used to compare generated images with real image distributions."},{"cited_title":"Clipscore: A reference-free evaluation metric for image captioning,","cited_arxiv_id":null,"evidence_quote":"Defines CLIPscore, the automated metric used to measure semantic alignment between generated and real images."},{"cited_title":"Learning transferable visual models from natural language supervi- sion,","cited_arxiv_id":null,"evidence_quote":"Provides the CLIP encoders used for CLIPscore and image feature extraction."},{"cited_title":"Native: Multi-modal knowledge graph completion in the wild,","cited_arxiv_id":null,"evidence_quote":"Provides the NATIVE method used to test whether generated images improve multi-modal knowledge graph completion."}],"review_version":1}