{"id":"db83060c-04b6-4e1e-bdc5-743d13f0e984","arxiv_id":"2506.02482","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A modified GraphSAGE for link prediction on the Amazon co-purchasing graph is reported to outperform a random forest baseline, but the evaluation never tests the stated problem of recommending isolated new products.","lead":"This paper applies a modified GraphSAGE model to the Amazon co-purchasing network to recommend new products by predicting co-purchase links. It compares against a random forest baseline and reports better top-k accuracy, but the evaluation does not actually test the claimed scenario of recommending truly new, isolated items.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Isolated-node claim is untested: training and evaluation are restricted to connected nodes, and the Section 3.5.2 neighbor-sampling procedure cannot be executed for degree-0 nodes.","rationale":"Read in good faith, the paper is a course-project-scale study with a clear pipeline, a public dataset, a baseline ablation, and available code. It correctly identifies the new-item problem and acknowledges the presence of isolated nodes. However, the central claim depends entirely on the isolated-node regime, and every experimental component avoids that regime. The training construction uses 1-degree nodes and their neighbors; the evaluation uses a BFS subgraph of the largest connected component; and the method description for isolated nodes relies on sampling neighbors, which is impossible for degree-0 nodes. This is not merely a missing baseline or a stylistic weakness; it means the abstract's conclusion about newly listed and unseen products is not entailed by the reported experiments. The reader's weakest assumption names the same mismatch, and I agree with the REJECT verdict. I would not manufacture an additional objection; the evaluation mismatch is sufficient. The concrete test above would settle whether the concern lands: a simulated cold-start evaluation on nodes whose edges are removed, or the originally isolated nodes with temporal ground truth.","tokens_in":4512,"tokens_out":2957,"duration_ms":29157,"concrete_test":"Construct a held-out set of 10,000 nodes from the largest connected component, delete all incident edges of these nodes before computing GraphSAGE embeddings, and then attempt to recover the deleted edges by ranking candidate neighbors from the rest of the graph; report top-k accuracy against the original edges. If the modified GraphSAGE cannot execute its neighbor-sampling step for these degree-0 held-out nodes, or if top-k accuracy falls to the random-choice level, the claim that the method recommends truly isolated new products is not supported. A secondary check is to run the same protocol on the 159,575 originally isolated nodes, using any future co-purchase edges if a temporal split can be derived from the dataset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a modified GraphSAGE can recommend relevant links for newly listed, isolated products. That claim requires the method to work when a target node has no neighbors, and the paper never tests this regime. Section 2.2 deliberately removes all 159,575 isolated nodes by extracting the largest connected component, and Section 4.3 evaluates on a BFS subgraph of that component, so every test node has degree at least 1. Section 3.3 trains on \"1-degree nodes and their neighbors,\" a connected proxy, not on isolated nodes. Moreover, the proposed fix in Section 3.5.2 (\"we sample from the target node's neighbors and use these as a proxy for aggregation\") is undefined for an isolated node, which by definition has no neighbors. Consequently, the reported top-k advantage of Modified-GraphSAGE over Random Forest (0.0187 vs 0.0125 at top-5) is evidence only for connected nodes, not for the new-item problem the abstract promises. The near-random absolute accuracies further weaken the inference, but the decisive defect is the mismatch between the stated problem and the evaluation setup.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a GraphSAGE-based inductive link prediction method for recommending newly listed products in the Amazon co-purchasing network. The authors build an undirected graph from the SNAP Amazon metadata, extract the largest connected component, generate node features (title embeddings, group one-hots, category vectors, degree, clustering coefficient), train a baseline Random Forest and a modified GraphSAGE model on positive/negative pairs involving degree-1 nodes, and evaluate top-k accuracy on a BFS-sampled connected subgraph. The abstract claims the method outperforms baselines and offers a solution for recommending new, isolated products.","tokens_in":4911,"tokens_out":4032,"duration_ms":38401,"significance":"If the central claim were supported, the approach would offer a scalable, inductive link-prediction method for new-item recommendation in e-commerce, which is a practically relevant problem. The manuscript's strengths are its detailed data-preprocessing pipeline, an ablation study, and a promise of code availability. However, the evaluation does not test the isolated-node regime that is the stated motivation, and the proposed neighbor-sampling mechanism is undefined for degree-0 nodes. The significance as claimed is therefore not established by the experiments; the results support only a weak claim about connected nodes in the same graph component.","major_comments":[{"comment":"The evaluation never tests the stated problem. Section 2.2 explicitly discards all 159,575 isolated nodes by extracting the largest connected component, and Section 4.3 samples test nodes via BFS from that component, so every test node has degree at least 1. The reported top-k results (e.g., top-5 accuracy 0.0187 for GraphSAGE vs. 0.0125 for Random Forest in Figure 4) therefore support only connected-node predictions, not the abstract's claim of recommending newly listed, isolated products. This is a load-bearing mismatch between the problem statement and the experimental design.","section":"Sections 2.2 and 4.3"},{"comment":"The proposed modification for isolated nodes is undefined. The text states, \"we sample from the target node's neighbors and use these as a proxy for aggregation,\" but an isolated node has no neighbors by definition, so this operation cannot be executed for the nodes the paper claims to handle. Section 3.3 trains on 1-degree nodes as surrogates, yet 1-degree nodes have a neighbor and are not isolated; no argument establishes that the two regimes are equivalent, and the connected test set cannot verify it.","section":"Section 3.5.2"},{"comment":"The absolute performance is close to random. At top-5, the random baseline is 0.0063, Random Forest is 0.0125, and Modified GraphSAGE is 0.0187 (Figure 4). Even if restricted to connected nodes, these numbers represent only about 1-2 percent correct recommendations, and no confidence intervals or significance tests are reported, so the claim of \"outperforming baseline algorithms\" is statistically unsubstantiated.","section":"Section 4.3"}],"minor_comments":[{"comment":"\"Associativity coefficient\" should be \"assortativity coefficient.\"","section":"Section 2.2"},{"comment":"\"Distirbution\" should be \"Distribution.\"","section":"Figure 2 caption"},{"comment":"\"reconmendation\" should be \"recommendation,\" and the phrase \"in almost all k's\" is awkward; use \"values of k.\"","section":"Section 4.3"},{"comment":"\"ScentenceTransformer\" should be \"SentenceTransformer.\"","section":"Resources"},{"comment":"The GitHub URL in the full text contains spaces (\"final-project-l-minghao z-catherine z-nathan.git\") and is not a single clean link; the abstract version without spaces should be used consistently.","section":"Abstract and full text"},{"comment":"The discussion claims the model generalizes to unseen nodes without retraining, but no temporal or held-out-node experiment is described; the BFS subgraph is sampled from the same LCC used in training, so the meaning of \"unseen\" is not clearly established.","section":"Section 4.4"}],"recommendation":"reject","confidential_remarks":"The manuscript reads as a course project report rather than a research paper. The central claim about isolated-node recommendation is not supported by the experimental design, and the proposed method is undefined for the target regime. The mismatch between the stated problem and the evaluation is fundamental and would require a completely new experimental setup and likely new data to address; a revision within the current scope does not appear feasible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the headline claim is untested. The paper says it solves the new-item cold-start problem for isolated products, but it removes all isolated nodes from the data, trains on 1-degree nodes, and evaluates on a BFS subgraph of the largest connected component where every node has degree at least 1. The proposed fix in Section 3.5.2, sampling from the target node's neighbors, is undefined for a node with no neighbors. So the central evaluation never faces the problem the abstract promises.\n\nCredit where due: the network construction and basic statistics (power law, modularity, community structure) are competently done for a course project. The ablation study on the random forest baseline is a reasonable exercise, and the authors are honest enough to report top-k numbers that are barely above random. Code and data references are included, which is more than many arXiv postings do.\n\nWhat's new: very little. Applying GraphSAGE to link prediction is standard. The 'modification' is a vaguely described tweak, not a new algorithm or theoretical insight. The near-random absolute accuracy (top-5 0.0187 vs 0.0125 random forest vs 0.0063 random) further weakens any claim of practical value.\n\nThe load-bearing flaw is the mismatch between problem and evaluation. Section 2.2 deliberately drops 159,575 isolated nodes. Section 3.3 uses 1-degree nodes as a proxy. Section 4.3 tests on a connected subgraph. Nothing tests the claimed ability to recommend truly new isolated products. If the paper were reframed as 'link prediction for low-degree nodes,' the evaluation might be relevant, but even then the gains are marginal and there are no error bars or statistical tests.\n\nThis looks like a well-executed class project, not a research paper. It deserves to be a poster at a course showcase, not peer review in a journal. A serious editor should desk reject it, mainly because the central claim is not tested. If the authors are willing to redo the evaluation on actual isolated nodes (perhaps with a cold-start simulation), the idea could be worth a short paper, but as it stands, I wouldn't spend referee time on it.","headline":"Despite competent network analysis, the paper never tests its central claim of recommending isolated new products, because the evaluation only uses connected nodes and the proposed fix is undefined for degree-0 nodes.","tokens_in":5283,"tokens_out":2589,"would_cite":false,"duration_ms":24651,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A modified GraphSAGE model predicts co-purchasing links better than a Random Forest baseline and random choice, offering a scalable, inductive way to rank new product recommendations.","keywords":["co-purchasing network","link prediction","GraphSAGE","inductive recommendation","new item problem","graph embeddings","category similarity"],"falsifier":"Take a product with no co-purchase edges, run the modified GraphSAGE link predictor, and compare its top-k accuracy against a random baseline; if accuracy does not exceed the baseline, the claimed ability to recommend truly new products fails.","tokens_in":4326,"feed_emoji":"🛒","tokens_out":3781,"duration_ms":33737,"temperature":0.7,"pith_summary":"The paper tries to show that recommending new products can be framed as link prediction on a co-purchase graph, and that a modified GraphSAGE model does this better than a Random Forest baseline. The authors build a graph where products are nodes and co-purchase relations are edges, train on pairs of one-degree nodes and their neighbors, and measure top-k accuracy. If correct, the approach would give e-commerce platforms a scalable, inductive way to rank candidate new products as they are added, using product features and local graph structure rather than past user behavior.","feed_headline":"Co-purchase graph model improves new-product link prediction","feed_subtitle":"A modified GraphSAGE ranks candidate products for new items, beating random forests on the largest connected component.","key_machinery":"The central object is a modified GraphSAGE model for link prediction. Standard GraphSAGE aggregates features from a node's neighbors; the modification instead treats the combined features of a node pair as the embedding, samples only one-hop neighbors of the target node, and uses those as a proxy for aggregation. The one-hop rule encodes the assumption that only direct co-purchase relationships, not transitive ones, are relevant for future co-purchases.","core_discovery":"The paper's central claim is that a modified GraphSAGE model for link prediction can outperform a Random Forest baseline and random choice at ranking co-purchasing links in an e-commerce product graph, and that because the model is inductive it can generalize to products not seen during training. The modification consists of treating the feature vector of a node pair as the embedding, sampling one-hop neighbors of the target node as a proxy for neighborhood aggregation, and deliberately avoiding two-hop sampling because co-purchase is not transitive. The authors construct a co-purchase graph from the dataset, use one-degree nodes as ground truth for link prediction, and evaluate with top-k accuracy on a breadth-first-sampled subgraph of the largest connected component.","pith_inferences":["Because the evaluation subgraph is sampled via BFS from the largest connected component, every test node has at least one neighbor; a truly cold-start product with degree zero would receive no neighborhood signal, so the 'new product' claim is as yet unsupported.","The method is essentially a pairwise classifier over handcrafted and structural features; the GraphSAGE name refers to the one-hop neighborhood sampling, not to a learned aggregation over multi-hop structure.","A natural testable extension is to use temporally split data, where edges after a cutoff date are predicted from earlier edges, to simulate true new products and verify the inductive claim."],"forward_implications":["If the approach is correct, new products can be ranked against existing items by predicted co-purchase probability, narrowing the candidate list for manual review.","The one-hop sampling assumption implies that co-purchase is a first-order relationship, so indirect connections should not be used for prediction.","The model can be updated in real time as new purchase edges appear, without full retraining.","The ablation results indicate that category similarity, degree, and clustering coefficient carry most of the predictive signal, while title embeddings contribute little."],"supporting_citations":[{"why":"Supplies the co-purchasing network dataset with product metadata, the source of nodes, features, and edges.","marker":"[1]"},{"why":"Provides the GraphSAGE inductive learning method that the paper modifies for one-hop link prediction.","marker":"[2]"},{"why":"Provides the pre-trained language model used to generate title embeddings, one of the node features.","marker":"[3]"},{"why":"Defines the Random Forest classifier used as the baseline model.","marker":"[4]"}],"fun_headline_variants":["Modified GraphSAGE outperforms random forest in co-purchase link prediction","Inductive GraphSAGE predicts new-product co-purchases better than baseline","GraphSAGE variant ranks new-item co-purchase links, beating random forest","Link prediction for new products via modified GraphSAGE","Co-purchase link prediction improved by inductive GraphSAGE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The technique for isolated nodes assumes a new product has neighbors to sample from, but a truly new product has none, and the experiments never test that case.","fun_headline_variants_meta":{"raw":{"variants":["Modified GraphSAGE outperforms random forest in co-purchase link prediction","Inductive GraphSAGE predicts new-product co-purchases better than baseline","GraphSAGE variant ranks new-item co-purchase links, beating random forest","Link prediction for new products via modified GraphSAGE","Co-purchase link prediction improved by inductive GraphSAGE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000537,"raw_usage":{"total_tokens":2537,"prompt_tokens":860,"completion_tokens":1677,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":1586}},"tokens_in":476,"tokens_out":1677,"duration_ms":11614,"temperature":1.0,"reasoning_tokens":1586,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:22:15.881119+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a product with no co-purchase edges, run the modified GraphSAGE link predictor, and compare its top-k accuracy against a random baseline; if accuracy does not exceed the baseline, the claimed ability to recommend truly new products fails.","supporting_citations":[{"cited_title":"The dynamics of viral marketing,","cited_arxiv_id":null,"evidence_quote":"Supplies the co-purchasing network dataset with product metadata, the source of nodes, features, and edges."},{"cited_title":"Sentence-bert: Sentence embeddings using siamese bert-networks,","cited_arxiv_id":null,"evidence_quote":"Provides the pre-trained language model used to generate title embeddings, one of the node features."},{"cited_title":"Random forests,","cited_arxiv_id":null,"evidence_quote":"Defines the Random Forest classifier used as the baseline model."}],"review_version":1}