{"id":"bd59365a-da68-4774-b8c3-ff3425d4a4cb","arxiv_id":"2412.09940","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A proposed pipeline for graph-based predictive queries, illustrated with MovieLens and heart-disease data, but without quantitative support.","lead":"This report proposes a step-by-step pipeline that converts graph data into vector embeddings and then uses similarity searches to answer prediction-style questions. It illustrates the steps on a movie-rating example and a heart-disease dataset, but offers no quantitative validation of the approach.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that FastRP 50/100 dimensional embeddings are superior rests entirely on visual separation in t-SNE plots, with no measured classification accuracy, so the central claim is unsupported as stated.","rationale":"The reader's weakest assumption correctly identifies the reliance on visual inspection of t-SNE plots. My stress test confirms this and sharpens it: the central claim is not merely under-measured because plots are inspected instead of metrics; it is unsupported because no classification result of any kind is reported for the Heart Disease dataset. The manuscript's step 7 promises 'Prediction Results,' but none are given. The MovieLens tables in Chapter 3 do contain numeric prediction errors, but they compare GraphSAGE and Node2Vec, not FastRP, and they are not framed as a rigorous evaluation with aggregate error statistics. The additional inconsistency in the text's qualitative judgments across figures reinforces that no objective criterion was used. A simple classification benchmark on the existing embeddings would settle the question directly, since the embeddings are claimed to be stored as node properties and the required labels are the target field. This concern does not change the reader's verdict: the paper's central empirical claim remains unsupported, so rejection is appropriate.","tokens_in":14282,"tokens_out":3633,"duration_ms":43258,"concrete_test":"Re-run the heart-disease experiment with an actual classification benchmark: for each of the nine projection/embedding/dimension combinations in Table 4.1, take the stored Person-node embeddings, split by the target field, and evaluate a k-nearest-neighbor classifier (or logistic regression on the embedding coordinates) under repeated stratified cross-validation, reporting mean accuracy and AUC with standard deviations. If FastRP-50 and FastRP-100 fail to beat Node2Vec and GraphSAGE at the same dimensions by more than the cross-validation noise, the headline claim is refuted; if they do, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4's only evidence for 'superior results' in the Heart Disease experiment is the t-SNE scatterplots of Figures 4.6 and 4.7. No classifier is trained or evaluated on the Person-node embeddings; there is no train/test split, accuracy, F1, AUC, or comparison against a baseline. t-SNE is a stochastic, nonlinear projection that optimizes a 2-D layout; visually separated clusters in this projection do not imply that a classifier can separate the classes in the original 100-D embedding, and apparent overlap can be a projection artifact. The report itself never defines a metric for 'more discriminative.' This matters because the pipeline's stated step 7 is 'Prediction Results,' but no numeric prediction results are reported for the health dataset. The only numeric tables are for MovieLens rating prediction (Chapter 3) and concern GraphSAGE vs Node2Vec, not FastRP. The text also makes inconsistent visual judgments: GraphSAGE is called best in Figure 4.4, Node2Vec best in Figure 4.5, and FastRP best in Figure 4.7, underscoring the absence of any objective criterion. The claim might be true, but the provided evidence cannot support it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a 'predictive query-based pipeline' for graph databases. It describes graph embedding generation with Node2Vec, GraphSAGE, and FastRP, the storage of embeddings as node properties in Neo4j, dimensionality-reduction visualization, and similarity queries. Two applications are presented: MovieLens rating prediction (Chapter 3) and Heart Disease classification (Chapter 4). The stated conclusions are that GraphSAGE embeddings give more accurate rating predictions than Node2Vec, and that FastRP embeddings at dimensions 50 and 100 are superior for distinguishing healthy and sick patients.","tokens_in":14522,"tokens_out":4822,"duration_ms":45678,"significance":"The pipeline idea is pragmatic, and the use of Cypher queries over stored embeddings is a practical starting point for practitioners. However, the empirical validation is entirely qualitative: the Heart Disease conclusion rests on visual inspection of t-SNE scatterplots, with no classifier or accuracy measure, and the MovieLens comparison uses 19 hand-picked examples without statistical analysis. The manuscript is transparent that it is a first step, but the central claims are not supported by the evidence presented. A quantitative evaluation and a formal definition of the pipeline steps would be needed for the paper to be publishable.","major_comments":[{"comment":"The claim that FastRP embeddings with 50 and 100 dimensions 'yielded superior results for inferring the health status of individuals' is unsupported. No classifier is trained or evaluated, there is no held-out test set, and no accuracy, F1, or AUC is reported. The evidence is solely visual separation in 2-D t-SNE projections, which are stochastic and can introduce artifacts. This is load-bearing because the pipeline's step 7 is 'Prediction Results,' yet no numeric prediction result appears for the Heart Disease dataset.","section":"Sec. 4, Figs. 4.6–4.7"},{"comment":"The visual comparisons are internally inconsistent. The text states that GraphSAGE embeddings are 'more discriminative' in Figure 4.4, that Node2Vec embeddings are 'more discriminative' in Figure 4.5, and then that FastRP embeddings are most representative in Figure 4.7. With no objective classification metric, these judgments are post hoc and cannot support the conclusion that any particular embedding method is best.","section":"Sec. 4, Figs. 4.3–4.5"},{"comment":"The comparison of GraphSAGE versus Node2Vec for rating prediction is based on 19 (user, movie) pairs, with no error bars, no aggregate error measure such as MAE or RMSE, and no significance test. The text's conclusion that GraphSAGE is 'more accurate' rests on a count of absolute differences at least 1, which is not a validated evaluation protocol. This is insufficient evidence for the comparative claim.","section":"Ch. 3, Tables 3.1 and 3.2"}],"minor_comments":[{"comment":"The movie title 'The hawshank Redemption' should be 'The Shawshank Redemption'.","section":"Tables 3.1 and 3.2"},{"comment":"The sentence 'As depicted in Figure 4.3, the GraphSAGE embeddings demonstrate superior discriminative results' refers to the 50-dimension strict-extended projection, but Figure 4.3 shows the 10-dimension full projection; the cross-reference is misleading.","section":"Sec. 4, para. after Fig. 4.3"},{"comment":"There are numerous grammatical errors and typos, including 'a interesting', 'a important', 'tunning', 'the is no information', 'Multidimention Scaling', and 'infer' used where 'infer' or 'derive' is intended; a careful editing pass is needed.","section":"Throughout"},{"comment":"The reproducibility of the experiments would be improved by reporting the exact data split, embedding hyperparameters (walk length, number of walks, epochs, random seeds), and the version/date of the Kaggle dataset [Kol23].","section":"Sec. 4, data description"},{"comment":"The statement 'This work is only a first step' is consistent with a technical report, but it conflicts with the strength of the earlier claim that the pipeline 'demonstrated' FastRP's superiority; the conclusions should be reworded to match the preliminary nature of the evidence.","section":"Sec. 4.1"}],"recommendation":"reject","confidential_remarks":"This manuscript is closer to a postdoc progress report than a research paper. The core empirical claims are unsupported, and the main health application has no quantitative evaluation. The author acknowledges this in Sec. 4 ('There is no information about the quality of the embeddings') and Sec. 4.1 ('only a first step'). The practical idea of storing multiple embeddings as node properties and querying them in Cypher is a useful starting point, but in its current form the paper does not meet the standards of a peer-reviewed venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a postdoc report, not a research paper. It describes a pipeline for answering predictive queries on a graph database: build a Neo4j graph, generate Node2Vec/GraphSAGE/FastRP embeddings, store them as node properties, run KNN similarity, and visualize with t-SNE/Isomap/MDS. That part is coherent and even useful as a tutorial of what Neo4j GDS can do. The MovieLens rating-prediction chapter gives a concrete 21-row table with mean differences, so that part is at least testable, even if it is only five users and five movies.\n\nThe soft spot is the paper's actual conclusion. The claim that FastRP embeddings at 50 and 100 dimensions are “superior” for distinguishing healthy from sick patients rests entirely on eyeballing the t-SNE scatterplots in Figures 4.6 and 4.7. There is no classifier trained, no accuracy or F1, no train/test split, no baseline comparison, and no numeric prediction result for the heart disease data even though step 7 of the pipeline is called “Prediction Results.” To make it worse, the narrative says GraphSAGE is best in Figure 4.4, Node2Vec best in Figure 4.5, and FastRP best in Figure 4.7, which shows that the “best” choice is being selected after seeing the plots. That is post hoc fitting, not evaluation. The stress-test note is right on target.\n\nIn fairness, the paper itself flags that this is a first step and an “upcoming” method. There is no deception, just a large gap between what is claimed and what is measured. The author also says the embedding dataset is a contribution, but no code or data is released, so even that part is not currently reproducible.\n\nWho gets value from this? A practitioner who wants a compact example of how to wire graph embeddings into Neo4j and run similarity queries. It does not deserve full peer review at a research venue as a standalone contribution, because the central empirical claim is unsupported. It might merit a short workshop paper if the author adds a real classifier evaluation and releases the code and data. I would not cite it in my own work, but I might show it to a student as an example of how not to conclude anything from t-SNE alone.\n\nRecommendation: desk reject for a research venue, but point the author toward a workshop or a revised version with quantitative results. It is honest work, just not a finished scientific contribution yet.","headline":"A well-intentioned postdoc report on a graph embedding pipeline, but the headline claim about FastRP rests on eyeballing t-SNE plots rather than any measured accuracy.","tokens_in":15015,"tokens_out":1324,"would_cite":false,"duration_ms":16984,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a query-driven pipeline for graph data, applied to a heart-disease dataset, shows FastRP embeddings at 50 and 100 dimensions separate healthy from sick patients better than Node2Vec or GraphSAGE embeddings.","keywords":["graph embedding","predictive queries","graph database","pipeline","node classification","FastRP","Node2Vec","GraphSAGE"],"falsifier":"For the same Heart Disease graph, compute a standard classification metric, such as 10-fold cross-validated F1 or accuracy of a k-nearest-neighbor or logistic-regression classifier, on the FastRP 50/100 embeddings against the Node2Vec and GraphSAGE embeddings. If FastRP 50/100 does not outperform the other embeddings (or a baseline using the raw patient attributes), the paper's central claim is refuted.","tokens_in":14056,"feed_emoji":"🫀","tokens_out":6964,"duration_ms":65316,"temperature":0.7,"pith_summary":"The paper proposes a step-by-step pipeline that turns a raw dataset into a graph database, generates embeddings with Node2Vec, GraphSAGE, and FastRP at several dimensions and projection types, and uses dimensionality-reduced views to judge which configuration best answers a predictive query. Applied to a heart-disease dataset, the pipeline concludes that FastRP embeddings with 50 and 100 dimensions produce the clearest separation between healthy and sick patients. A reader would care because the paper offers a reusable recipe for choosing an embedding configuration for graph-based predictive queries, a choice that is usually made by trial and error. The demonstration is qualitative: the conclusion is drawn from 2D t-SNE plots rather than measured accuracy.","feed_headline":"FastRP at 50/100 dims best for heart-disease graph queries","feed_subtitle":"A query-driven pipeline picks FastRP to separate healthy and sick patients, but the verdict rests on visual t-SNE plots.","key_machinery":"The central object is the predictive query-based pipeline itself: a sequence of seven steps (data definition and cleaning, query prediction, graph model definition and import, graph projection, embedding generation, data visualization, and prediction results). Within it, three projection types (full, strict, and strict-extended) and three embedding methods (Node2Vec, GraphSAGE, and FastRP) at dimensions 10, 50, and 100 create a configuration space; embeddings are stored as node properties in a graph database and compared through dimensionality reduction (t-SNE, Isomap, MDS, Spectral Embedding) and KNN similarity. The load-bearing mechanism is the strict-extended projection—a node with its directly connected edges plus at least one additional node or edge—which the paper reports produces the clearest separation of Person nodes when combined with FastRP.","core_discovery":"The report's central claim is that, for the Heart Disease graph, the strict-extended graph projection combined with FastRP embeddings at 50 and 100 dimensions yields the most discriminative representation of Person nodes, grouping healthy and sick patients into visibly separate clusters. The paper further claims that projection type matters more than dimension in some settings, and that in a separate MovieLens rating-prediction example GraphSAGE embeddings give closer predictions than Node2Vec. The overarching discovery is a pipeline that makes these comparisons systematic and repeatable, spanning data cleaning, query definition, graph import, projection, embedding generation, visualization, and prediction results.","pith_inferences":["The paper's comparison is visual only; computing classification accuracy or silhouette scores on the original embeddings would confirm or overturn the FastRP claim.","The stated goal of a quality index for whether a query 'makes sense' on a graph is not implemented; turning that idea into a concrete index is the natural next step.","The FastRP advantage may be an artifact of t-SNE's nonlinear projection; measuring separation in the original high-dimensional space (for example, with k-NN accuracy) would settle whether the embeddings themselves are more discriminative."],"forward_implications":["FastRP embeddings at 50 and 100 dimensions are more discriminative for inferring heart-disease status than the other embedding configurations tested.","The strict-extended graph projection is the most promising of the three projection types for the Person-node classification task.","GraphSAGE embeddings predicted movie ratings more accurately than Node2Vec embeddings in the MovieLens example, showing that the best embedding depends on the query.","The pipeline can be applied to other datasets and query types, such as life-expectancy and mental-health prediction, without changing the core steps."],"supporting_citations":[{"why":"Supplies the FastRP embedding method that the paper reports as superior for heart-disease classification.","marker":"[CST+19]"},{"why":"Supplies the Node2Vec embedding method used as a comparison baseline.","marker":"[GB16]"},{"why":"Supplies the GraphSAGE embedding method used as a comparison baseline.","marker":"[HYL17a]"},{"why":"Provides the Heart Disease dataset on which the pipeline is executed.","marker":"[Kol23]"},{"why":"Provides the t-SNE dimensionality-reduction method whose visual output grounds the paper's conclusion.","marker":"[vdMH08]"},{"why":"Provides the Cypher query language used to retrieve embeddings and express similarity queries.","marker":"[FGG+18]"},{"why":"Provides the KNN algorithm used to compute node similarity from embeddings.","marker":"[Kra13]"}],"fun_headline_variants":["Query pipeline picks FastRP for heart-disease graph separation","FastRP best in systematic graph pipeline for patient clusters","Graph pipeline: FastRP at 50/100 dims separates healthy and sick","Pipeline reveals FastRP as optimal for heart-disease queries","Embedding pipeline tops FastRP for heart graph discrimination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on the assumption that separation visible in 2D t-SNE plots of the embeddings reflects genuinely better classification of healthy versus sick patients; the report does not measure predictive accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Query pipeline picks FastRP for heart-disease graph separation","FastRP best in systematic graph pipeline for patient clusters","Graph pipeline: FastRP at 50/100 dims separates healthy and sick","Pipeline reveals FastRP as optimal for heart-disease queries","Embedding pipeline tops FastRP for heart graph discrimination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000281,"raw_usage":{"total_tokens":1594,"prompt_tokens":805,"completion_tokens":789,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":421,"completion_tokens_details":{"reasoning_tokens":705}},"tokens_in":421,"tokens_out":789,"duration_ms":7985,"temperature":1.0,"reasoning_tokens":705,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:32:11.955315+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For the same Heart Disease graph, compute a standard classification metric, such as 10-fold cross-validated F1 or accuracy of a k-nearest-neighbor or logistic-regression classifier, on the FastRP 50/100 embeddings against the Node2Vec and GraphSAGE embeddings. If FastRP 50/100 does not outperform the other embeddings (or a baseline using the raw patient attributes), the paper's central claim is refuted.","supporting_citations":[],"review_version":1}