{"id":"9b2ecb4d-e922-489c-8aa6-74b1b8e3d6f2","arxiv_id":"2505.19020","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding a t-SNE-based hierarchical item clustering step to graph contrastive learning yields small accuracy gains on three benchmark recommendation datasets.","lead":"HGCL is a recommendation method that learns user and item embeddings with graph contrastive learning, groups similar items into categories using t-SNE, and then fine-tunes with both the original and clustered graphs. It reports slightly better recall and NDCG than six baselines on three public datasets, but the gains are small and the paper gives no error bars.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed contribution of hierarchical item structures is not established: clustering uses stochastic t-SNE with an unspecified polar origin, so the reported gains over XSimGCL may come from arbitrary partitions rather than meaningful item hierarchies.","rationale":"The reader's weakest assumption is on target. I narrowed the focus to the validity and reproducibility of the clustering, because every novel component of the method (hierarchical item structures) is mediated by Sec. 4.2's sectors. The paper does not state how the polar origin is chosen; t-SNE is random; no seed is given; no measure of cluster quality or stability is reported. Without this, the only evidence that the hierarchy helps is Table 2's small, unreplicated margins. Figure 5 only shows that fine-tuning separates positives from negatives, which any additional capacity could do. The control I propose would directly separate 'meaningful clusters improve recommendations' from 'arbitrary cluster smoothing improves this model.' I kept the verdict at the reader's CONDITIONAL: the issues are fixable with added experiments and a precise clustering definition, but as written the core claim is not established.","tokens_in":11372,"tokens_out":5822,"duration_ms":54036,"concrete_test":"Run HGCL on Yelp2018 with the reported hyperparameters (ρ=8, θ=4, perplexity=30). (1) Center the t-SNE output by subtracting the mean before dividing into radial/angular sectors; if Recall@20 changes by more than about 0.001 (the reported margin over XSimGCL is 0.001), the method is sensitive to the unspecified origin. (2) Replace the t-SNE sectors with a random permutation of item-to-cluster assignments preserving cluster sizes; if Recall@20 still exceeds XSimGCL's 0.0726, the gain does not require meaningful item hierarchies. Repeating both over 3 random seeds would also quantify t-SNE variance. If either condition holds, the paper should withdraw the attribution claim or add validated, deterministic clustering.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central result—that the gap over XSimGCL substantiates the contribution of hierarchical item structures—depends entirely on the item clusters built in Sec. 4.2. Two properties of that construction are neither specified nor validated. First, the polar origin used to divide the t-SNE plane into sectors is never defined in Sec. 4.2 or Fig. 1; a translation of the t-SNE coordinates changes every cluster boundary. Second, t-SNE (Eq. 6) is stochastic: different runs change cluster assignments, and no seed or averaging is reported. The only place clusters enter the model is Eq. 7, where each item's score is augmented by its cluster embedding, so if the clusters are effectively arbitrary the extra power of HGCL over XSimGCL comes from added parameters and the extra user-cluster graph, not from meaningful item similarities. The sensitivity study (Fig. 3) varies ρ and θ but never varies the t-SNE run or the origin, so it cannot address this. Consequently, HGCL's mechanism is under-specified (not reproducible as written) and its headline attribution to hierarchy is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HGCL, a graph contrastive learning method for user-item recommendation that extends XSimGCL. The method first pre-trains user/item representations with cross-layer contrastive learning, then projects item embeddings to two dimensions with t-SNE and partitions the plane into radial and angular sectors to form item clusters. These clusters are used to build a user-clustered item graph, and the model is fine-tuned jointly on the original and clustered graphs, with the final score for a user-item pair augmented by the item's cluster embedding (Eq. 7). Experiments on Yelp2018, Amazon-Kindle, and Alibaba-iFashion report Recall@20 and NDCG@20 for seven methods and claim that HGCL outperforms all baselines, including XSimGCL, thereby demonstrating the value of hierarchical item structures.","tokens_in":11659,"tokens_out":4415,"duration_ms":43095,"significance":"If the empirical claims were robust, the paper would make a modest but useful contribution: a simple, model-agnostic way to inject coarse item-cluster information into an existing GCL recommender, with a clear two-stage pipeline and sensitivity analyses for the cluster-division and perplexity hyperparameters. The paper also openly acknowledges that scalability to large industrial datasets is not demonstrated. However, the central quantitative claim is supported only by point estimates with no variance information, and the cluster construction is under-specified because the polar origin is not defined and the t-SNE randomness is not controlled. The reported gains over XSimGCL are small, and the paper does not provide a validation protocol for the dataset-specific hyperparameters. No code is released, so reproducibility currently depends on a written specification that is incomplete. The idea is plausible but the evidence is not yet sufficient to establish the headline attribution of the gains to hierarchical item structures.","major_comments":[{"comment":"The radial/angular sector clustering is not a well-defined function of the data because the polar origin used to divide the t-SNE plane is never specified. Translating or rotating E_tsne changes every cluster boundary, and the paper does not state how the origin is selected or whether the coordinates are aligned before sectoring. Since the cluster memberships w_jk enter the final score in Eq. (7), the method is not reproducible as written; please specify the origin (e.g., the centroid of E_tsne) and any normalization or alignment step.","section":"Sec. 4.2, Fig. 1"},{"comment":"The t-SNE projection is stochastic, but no random seed, number of runs, or aggregation across runs is reported. The sensitivity study in Fig. 3 varies rho and theta but never varies the t-SNE run, so it cannot rule out the possibility that the reported gains over XSimGCL are artifacts of one particular t-SNE draw. Please report the mean and standard deviation over several t-SNE runs, or fix and disclose the seed and verify that the qualitative conclusions are stable across runs.","section":"Sec. 4.2, Eq. (6); Sec. 5.3, Fig. 3"},{"comment":"The central claim of superior performance and the attribution of that performance to hierarchical item structures rest on point estimates without error bars or significance tests. The improvements over XSimGCL are small, for example Yelp2018 Recall@20 0.0736 vs 0.0726, Amazon-Kindle 0.2117 vs 0.2061, and Alibaba-iFashion 0.1194 vs 0.1189, with similarly small NDCG differences. The connecting-strength analysis in Fig. 5 is descriptive only and reports no uncertainty. Please provide multiple seeds, standard deviations, and, where appropriate, paired significance tests to support the Q2 conclusion.","section":"Sec. 5.2, Table 2; Sec. 5.3, Fig. 5"},{"comment":"The hyperparameters rho, theta, and perplexity are tuned per dataset, and the values in Table 1 appear to be the ones that maximize test Recall@20 according to Figs. 3 and 4, but no validation split or model-selection protocol is described. If test performance is used to choose these hyperparameters, the reported results are selected on the test set and the gains relative to baselines are inflated. Please specify a validation procedure, or report results for fixed default settings in addition to the tuned settings.","section":"Sec. 5.3, Figs. 3-4; Table 1"}],"minor_comments":[{"comment":"The formula for p_xy is not the standard t-SNE conditional probability with perplexity; the notation 'k,l within the neighborhood' is vague, and the global normalization shown does not match the usual definition. Please align Eq. (6) with the standard t-SNE formulation or explain the simplified version used here.","section":"Sec. 4.2, Eq. (6)"},{"comment":"The term 'hierarchical item structures' suggests multiple resolutions of item similarity, but the construction adds only a single level of item clusters. The abstract's claim of structures 'across varying resolutions' is stronger than what is actually implemented; consider clarifying that the hierarchy has two levels.","section":"Sec. 4.2"},{"comment":"There is a typo: 'betwen' should be 'between'.","section":"Sec. 5.1.2"},{"comment":"The word 'persevering' appears where 'preserving' is intended.","section":"Sec. 4.2"},{"comment":"The footnote states that BUIR and DNN+SSL results are sourced from [38]; please state explicitly whether those numbers were obtained under the same data splits, evaluation protocols, and hyperparameter settings as the other baselines.","section":"Table 2"},{"comment":"If multiple runs are performed after addressing the stochasticity concerns, the sensitivity figures would be much more informative with error bars or shaded confidence bands.","section":"Figs. 3-4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal and the idea is worth pursuing, but the empirical case is not yet sufficient. The under-specified t-SNE-based clustering and the absence of variance reporting make the central attribution to hierarchical structures unverifiable as written. I would ask the authors to add the robustness and validation-protocol analyses described in the major comments before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou can skip this one if you're short on time, but it's worth a glance if you care about graph contrastive recommenders. HGCL takes XSimGCL and adds a second, coarser graph: item embeddings are run through t-SNE, then divided into radial/angular sectors around some origin, and users are connected to the sectors their items fall into. That user-cluster graph is then used for fine-tuning alongside the original user-item graph. The idea is neat—there's nothing quite like it in the GCL-for-recommendation literature—and the paper is honest about being an extension rather than a new framework.\n\nWhat's solid: the experiments use standard datasets (Yelp, Kindle, Alibaba), the baselines are strong, and the method is a clean deterministic pipeline after the t-SNE step. The learning curves and the connection-strength densities in Figure 5 give some descriptive support that the fine-tuning changes the score distribution in a sensible direction.\n\nThe problems are twofold, and they both land on the clustering step. First, the polar origin is never specified. t-SNE gives 2D coordinates; dividing the plane into sectors depends entirely on where the origin sits, and any translation changes every cluster. Second, t-SNE is stochastic. No seed or averaging is reported, so the clusters—and hence the user-cluster graph—may change from run to run. The sensitivity study varies rho and theta but never the t-SNE run or the origin, so it cannot address these questions. If the clusters are effectively arbitrary, the gain over XSimGCL may just come from extra parameters and an extra graph, not from meaningful item hierarchies.\n\nThe reported gains are real but small: roughly 0.1–0.6% absolute in Recall@20 and NDCG@20, with no error bars or significance tests. The hyperparameters rho, theta, and perplexity are tuned per dataset, and the tuning looks test-driven. That's a lot of fitting for a small signal.\n\nNone of this is fatal. The paper is well-written, the related work is appropriate, and the authors acknowledge the lack of large-scale industrial validation. It deserves a serious referee, but the referee should ask for major revisions: specify or remove the origin dependence, report multiple t-SNE runs with seeds, add variance estimates, and ideally release code. As is, I'd treat the hierarchy claim as unproven.\n\nFor you: if you're working on GCL, it's a useful pointer to a simple way to add coarse structure, but I wouldn't cite the current version in a paper. I'd bring it up in reading group only as an example of a plausible mechanism that lacks robustness validation.","headline":"An incremental GCL extension with a plausible but under-validated clustering mechanism; the gains over XSimGCL are real but tiny, and the paper needs robustness analysis before the hierarchy claim can be trusted.","tokens_in":12181,"tokens_out":2307,"would_cite":false,"duration_ms":20900,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A cluster layer on top of the item graph improves graph contrastive recommendation.","keywords":["hierarchical graph contrastive learning","graph neural networks","recommendation systems","contrastive learning","item clustering","t-SNE","user-item bipartite graph","representation learning"],"falsifier":"Run HGCL on Yelp2018 with multiple t-SNE random seeds and with different choices of the polar origin (coordinate origin, centroid, median), keeping $\\rho$ and $\\theta$ fixed. If the Recall@20 advantage over XSimGCL (0.0736 versus 0.0726) disappears or changes sign under any of those variations, the claim that hierarchical item structure causes the gain is not supported. A complementary check is a permutation test: assign items to sectors at random with the same cluster sizes and same number of clusters; if random sectors match HGCL's gain, the specific t-SNE-derived clusters are not doing the work.","tokens_in":11203,"feed_emoji":"📈","tokens_out":6588,"duration_ms":58877,"temperature":0.7,"pith_summary":"Hierarchical item structure, discovered from learned embeddings rather than item metadata, is the missing signal this paper adds to graph contrastive recommendation. HGCL pre-trains a GCL model on the user-item graph, projects the item embeddings into two dimensions with t-SNE, and divides that plane into $\\rho$ radial and $\\theta$ angular sectors; each sector becomes a cluster node in a second user-to-cluster graph. Fine-tuning on both graphs lets cluster-level collaborative signal enter the final item score, and the paper reports the highest Recall@20 and NDCG@20 among seven models on Yelp2018, Amazon-Kindle, and Alibaba-iFashion. The comparison to XSimGCL, the method HGCL extends, is the evidence offered that the hierarchy itself, not the extra parameters, drives the gain.","feed_headline":"A cluster layer lifts graph recommender accuracy","feed_subtitle":"A two-level graph groups similar items into sectors and edges out six baselines on three benchmark datasets.","key_machinery":"The load-bearing mechanism is the two-hierarchy user-item bipartite graph built by polar-sector clustering in a t-SNE plane. Pre-trained item embeddings are compressed to two dimensions with t-SNE; the plane is partitioned into $\\rho$ radial bands and $\\theta$ angular sectors, and every item whose projected point falls in a sector joins that cluster. A separate graph links each user to the clusters containing items the user interacted with, and the two graphs are convolved with the same LightGCN-style operator. The final connecting strength $\\hat{y}_{ij}$ uses a weighted sum of the item embedding and all cluster embeddings containing the item, so the cluster representations act as learnable category prototypes that refine the item's place in ranking.","core_discovery":"On the paper's own terms, the discovery is that a two-level user-item graph, with original items plus item clusters, yields better recommendations than a flat user-item graph under the same GCL machinery. Item clusters are formed deterministically by clustering t-SNE-transformed item embeddings into polar sectors, so the hierarchy is representation-driven and needs no category labels. The final score for a user and item is the dot product of the user embedding with the item embedding plus the embedding of the item's cluster, which channels coarse-grained signals into ranking. Across three benchmark datasets the paper reports consistent, though small, gains over XSimGCL, for example Recall@20 of 0.0736 versus 0.0726 on Yelp2018, and takes these as evidence that hierarchical item structures matter for graph contrastive recommendation.","pith_inferences":["An untested consequence of the paper's reasoning is that cold-start and long-tail items should benefit disproportionately from the cluster layer, since they are the items with the least direct collaborative evidence; breaking down the reported gains by item popularity would test this directly.","The polar-sector partition is one of many possible clusterings of the t-SNE plane; a natural extension the paper does not explore is whether soft membership weights, or clustering done directly in the original embedding space, preserve or enlarge the gain.","Because sector boundaries are arbitrary, items near a boundary are assigned to one cluster only; in practice, overlapping or fuzzy sectors could smooth the boundary effect and may interact with the choice of $\\rho$ and $\\theta$.","If the gain is due to cluster prototypes acting as regularizers, similar or larger gains might be obtainable by adding a cluster-consistency contrastive loss on the item embeddings themselves, a modification that would avoid t-SNE and its stochasticity."],"forward_implications":["Any GCL recommender can be extended by adding a cluster-level graph on top of the item graph, since the hierarchy is derived from learned embeddings and requires no item attributes or external knowledge.","Recommendations become multi-resolution: a user can be matched to an item directly or through a cluster, which should let items with few interactions inherit collaborative signal from the cluster.","The number of radial and angular divisions is dataset-dependent; the sensitivity results show that too many sectors hurt performance, so practitioners need to tune $\\rho$ and $\\theta$ for each application.","The reported shift in connection-strength distributions after fine-tuning indicates the cluster layer mainly sharpens the separation between positive and negative user-item pairs.","Because HGCL inherits the pre-trained GCL weights from XSimGCL, its gains are additive to whatever uniformity and contrastive improvements the base method already provides."],"supporting_citations":[{"why":"Supplies the pre-training GCL module (XSimGCL) that HGCL extends and is the key comparison for attributing gains to the hierarchy.","marker":"[38]"},{"why":"Introduces the noise-based uniformity strategy and serves as a baseline shown in the comparisons.","marker":"[39]"},{"why":"Provides the LightGCN graph convolution operator used for propagation on both graphs.","marker":"[11]"},{"why":"Defines the SGL contrastive baseline and the self-supervised graph learning line HGCL builds on.","marker":"[25]"},{"why":"Supplies the t-SNE projection that defines the item clusters.","marker":"[21]"},{"why":"Supplies the InfoNCE contrastive loss used in pre-training and fine-tuning.","marker":"[19]"},{"why":"Supplies the Bayesian Personalized Ranking loss used for recommendation training.","marker":"[20]"}],"fun_headline_variants":["Two-level graph contrastive learning boosts recommendations","Hierarchical item clusters sharpen graph recommenders","Clustering items into sectors lifts GCL ranking","HGCL: a hierarchy-aware twist on graph contrastive learning","Item hierarchy supercharges graph contrastive recommenders"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire gain rests on the assumption that cutting a t-SNE projection of item embeddings into fixed polar sectors produces item clusters that capture similarities that actually help recommendation; the paper does not show that the result is stable under changes to the t-SNE random seed, the polar origin, or the sector boundaries.","fun_headline_variants_meta":{"raw":{"variants":["Two-level graph contrastive learning boosts recommendations","Hierarchical item clusters sharpen graph recommenders","Clustering items into sectors lifts GCL ranking","HGCL: a hierarchy-aware twist on graph contrastive learning","Item hierarchy supercharges graph contrastive recommenders"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000146,"raw_usage":{"total_tokens":1168,"prompt_tokens":914,"completion_tokens":254,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":181}},"tokens_in":530,"tokens_out":254,"duration_ms":3008,"temperature":1.0,"reasoning_tokens":181,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:20:34.408530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HGCL on Yelp2018 with multiple t-SNE random seeds and with different choices of the polar origin (coordinate origin, centroid, median), keeping $\\rho$ and $\\theta$ fixed. If the Recall@20 advantage over XSimGCL (0.0736 versus 0.0726) disappears or changes sign under any of those variations, the claim that hierarchical item structure causes the gain is not supported. A complementary check is a permutation test: assign items to sectors at random with the same cluster sizes and same number of clusters; if random sectors match HGCL's gain, the specific t-SNE-derived clusters are not doing the work.","supporting_citations":[{"cited_title":"XSimGCL: Towards extremely simple graph contrastive learning for recommendation.IEEE Transactions on Knowledge and Data Engineering, 36(2):913–926, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-training GCL module (XSimGCL) that HGCL extends and is the key comparison for attributing gains to the hierarchy."},{"cited_title":"Are graph augmentations necessary? Simple graph contrastive learning for recommendation","cited_arxiv_id":null,"evidence_quote":"Introduces the noise-based uniformity strategy and serves as a baseline shown in the comparisons."},{"cited_title":"Self-supervised graph learning for recommendation","cited_arxiv_id":null,"evidence_quote":"Defines the SGL contrastive baseline and the self-supervised graph learning line HGCL builds on."},{"cited_title":"Visualizing data using t-SNE.Journal of Machine Learning Research, 9(11), 2008","cited_arxiv_id":null,"evidence_quote":"Supplies the t-SNE projection that defines the item clusters."}],"review_version":1}