{"id":"2add3aab-423b-4fef-ade3-65d823716186","arxiv_id":"2510.12959","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PPD projects pre-trained GNN embeddings away from a popularity direction estimated from interaction-level scores, improving unbiased recommendation metrics over existing debiasing baselines.","lead":"A new post-hoc method (PPD) removes a learned 'popularity direction' from pre-trained GNN recommendation embeddings to reduce popularity bias without retraining. Experiments on three real-world datasets with unbiased test splits show consistent gains, largest under heavy skew.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper never validates that the estimated popularity direction (Eq. 10) actually aligns with popularity; reported gains could come from any embedding perturbation tuned on the unbiased validation split.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing concern: the popularity direction vector is assumed to point along the popularity confound, but this is never validated. I agree. The paper is otherwise plausible: it is a post-hoc method with a simple mechanism, evaluated on standard unbiased test sets, and it beats several strong baselines. The main problem is that the empirical gains alone cannot establish the debiasing interpretation. The projection step is a deterministic function of the pre-trained embeddings and the interaction graph; if the direction is arbitrary, the method is a learned post-processing step tuned on the unbiased validation set. Since the paper reports no error bars and no controlled experiment linking d_pop to popularity, the central claim 'removes popularity while preserving preferences' is not yet supported beyond aggregate accuracy improvements. A synthetic or ablation test could settle this directly. This does not change the reader's conditional verdict: the concern is real but addressable, and the empirical results justify caution rather than rejection.","tokens_in":20852,"tokens_out":4414,"duration_ms":42668,"concrete_test":"Run a semi-synthetic experiment on KuaiRec: take a trained LightGCN, inject a known popularity confound by adding α·log(1+item_degree)·r to item embeddings and the corresponding component to user embeddings (r a fixed random unit vector), and generate interactions from preference-plus-popularity scores. Apply PPD and measure (i) cosine similarity between the estimated d_pop(v) and the true r, and (ii) whether projection removes the r-component while preserving preference-relevant structure. Compare against a control where d_pop is replaced by a random unit vector, with β and φ retuned on the validation split; if the random-direction control achieves comparable unbiased-test gains, the popularity-specific interpretation in the paper is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PPD removes popularity components from pre-trained embeddings while preserving preferences. The mechanism hinges entirely on the popularity direction vector d_pop(v) = e_pop(v) − φ·e_pref(v) in Eq. (10), and the projection removal in Eqs. (11)–(12). But there is no independent evidence that this vector points along the popularity confound rather than an arbitrary, data-dependent direction. The vector is constructed from the same biased embeddings and interaction graph: b_ui is a normalized difference of similarity scores, and the centroids in Eqs. (8)–(9) can involve negative weights, so d_pop is not necessarily interpretable as a 'popularity direction.' If d_pop is arbitrary, PPD is just a fixed (but hyperparameter-tuned) linear transformation of the embeddings, and its success on unbiased test metrics could reflect selection on the unbiased validation split, not genuine debiasing. The absence of error bars and code means we cannot distinguish these alternatives from the aggregate metrics alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PPD, a post-hoc method for correcting popularity bias in GNN-based collaborative filtering. PPD first estimates an interaction-level popularity score b_ui = p_i - r_ui, where p_i is a global-preference measure and r_ui is a personalized-preference measure with a popularity penalty. It then constructs a per-node popularity direction d_pop(v) as the difference between a popularity centroid and a preference centroid, projects the layer-0 node embedding onto this direction, and subtracts the projected component before re-propagating through the GNN. Experiments on KuaiRec, Coat, and Yahoo! R3 with LightGCN and SGL backbones report consistent improvements over popularity-debiasing baselines, with especially large relative gains on KuaiRec.","tokens_in":21113,"tokens_out":6822,"duration_ms":54064,"significance":"If the proposed direction vector genuinely isolates popularity from preference, the method would be practically valuable because it debiases deployed embeddings without retraining and can be layered on any GNN-based CF backbone. The paper has strengths: it evaluates on three datasets with unbiased test sets, compares with a broad set of recent baselines, and includes ablations for the key hyperparameters and for a second backbone. However, the central construction is heuristic and is not validated as a popularity direction; the reported gains could in principle come from a tuned linear perturbation of the embeddings. The lack of error bars and significance tests makes it difficult to assess the small gains on Coat and Yahoo! R3. These issues are addressable with additional experiments and analysis.","major_comments":[{"comment":"The central claim that the projection removes popularity rather than an arbitrary embedding direction is not supported. b_ui = p_i - r_ui can be negative, so the centroids in Eqs. (8)–(9) are not convex combinations and d_pop(v) is not guaranteed to be interpretable as a “popularity direction”; it is a data-dependent linear functional of the same biased embeddings it is later used to transform. The paper provides no diagnostic showing that d_pop(v) aligns with item popularity (e.g., degree or interaction frequency), and the success on the unbiased test could in principle come from a tuned linear perturbation. Please add: (i) correlation/alignment analysis between d_pop and popularity measures; (ii) a random-direction or fixed-direction control; (iii) error bars over seeds. This is necessary to substantiate the debiasing interpretation.","section":"§4.1–4.2, Eqs. (7)–(12)"},{"comment":"The text states “PPD consistently achieves the best performance across both groups.” In Table 3, for Yahoo! R3 bottom 80% NDCG@20, APDA reports 0.0129 while PPD reports 0.0127; PPD is not the best. The later sentence acknowledges a –1.5% drop, but the summary claim and the conclusion that PPD improves both head and tail recommendations should be qualified. This matters because RQ2 is specifically about the absence of a head–tail trade-off.","section":"§5.2, Table 3 (Yahoo! R3, bottom 80% NDCG)"},{"comment":"No standard deviations, significance tests, or multiple-seed results are reported. The Coat and Yahoo gains over the best baseline are small (0.9–8%), while β and φ are tuned on a one-third split of the unbiased data. The reader cannot tell whether PPD is statistically better than the comparison methods or whether the gap is within noise. Report mean ± std over at least 5 seeds and paired significance tests (e.g., paired t-test or Wilcoxon) on the unbiased test folds.","section":"§5.1–5.2, Tables 2–4"},{"comment":"Popularity scores in §4.1 are computed from the final/readout embeddings e_u, e_i (Section 3.1), but the projection is applied to the layer-0 embeddings e^(0). The relation between the final-embedding popularity direction and the layer-0 representation is not discussed. If the intention is to remove a direction estimated from the final embedding from the initial embedding, the calibration of this step needs justification and an experiment (e.g., applying the debiasing at every layer or at the final layer).","section":"§4.2, Eqs. (11)–(12)"}],"minor_comments":[{"comment":"Typo: “preference centroid ¯e_pop(v)” should read “preference centroid ¯e_pref(v).”","section":"Algorithm 1, line 13"},{"comment":"The cross-reference to Figure 3 for RQ2 results is wrong; the Recall/NDCG results for RQ2 are in Table 3.","section":"Appendix B.2"},{"comment":"IPSCN is cited as [21] in Table 2 but as [16] in Section 5.1 and Appendix B.1; the citation should be [16] (with [21] as the IPW source).","section":"Table 2"},{"comment":"The ε guard only prevents exact zero denominators; it does not address negative or near-zero sums when b_ui takes negative values. Please state the range of b_ui after min–max normalization and discuss the interpretation of negative weights.","section":"Eqs. (8)–(9)"},{"comment":"The complexity simplification drops the |E| term because |E| << |U||I|, but the retained term is |E| * average degree * d; the justification is incomplete. Also, the proposed user sampling to reduce O(|U||I|d) is not used in the experiments.","section":"Appendix A.2"},{"comment":"The phrase “PPD consistently achieves the best performance across both groups” is too strong given the Yahoo tail NDCG result (see Major 2). Please rephrase.","section":"§5.2 RQ2 text"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant practical problem and has a sensible experimental framework, but the central debiasing mechanism is not validated and the empirical claims lack statistical support. The specific fixes—validation of the popularity direction, random-direction controls, error bars/significance tests, and a corrected RQ2 claim—are feasible within the scope of the manuscript. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PPD is a plausible and practically motivated method: it debiases pre-trained GNN-based CF embeddings by estimating interaction-level popularity and projecting it out, no retraining required. That's a useful niche, and the paper does several things right. The interaction-level score mixing global and personalized preference is a genuine extension over DAP's degree-based approach. The ablation PPD w/o-PP shows the personalized component matters, the SGL-backbone results show generalizability, and the layer-depth experiments are a nice check that the method doesn't just brute-force over-smoothing. Evaluating on unbiased test splits (KuaiRec, Coat, Yahoo! R3) is the right protocol.\n\nThe soft spots are real, though. The central construction, the popularity direction d_pop = e_pop − φ e_pref, is never independently validated. It is derived from the same embeddings it later alters, and the centroids can involve negative weights, so nothing guarantees the vector points along popularity rather than some arbitrary data-dependent direction. With no error bars or significance tests reported, and with β and φ tuned per dataset on the unbiased validation split, some of the reported gains could come from a well-chosen linear perturbation rather than from true debiasing. The huge relative gains on KuaiRec look impressive but also suggest the base model is very degraded there; ratios are a fragile summary.\n\nI want to be fair: the stress-test concern is legitimate but not damning. The held-out unbiased test data is independent of the debiasing construction, so the improvement is not purely circular, and the method consistently helps across three datasets. The gap is that the paper does not show the estimated direction aligns with popularity in any controlled or synthetic setting, and does not release code or seeds, so reviewers can't tell how selection-sensitive the numbers are.\n\nI'd send this to peer review. The idea deserves referee time, but the revision should add variance estimates, a synthetic experiment that verifies the direction, and ideally code/data release. If the authors can show the direction is not arbitrary, this becomes a solid contribution. For now, treat the stronger claims with caution.","headline":"Post-hoc debiasing is a promising idea, but the popularity direction is a heuristic that needs independent validation before the reported gains can be trusted.","tokens_in":21547,"tokens_out":2046,"would_cite":true,"duration_ms":18418,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that popularity bias in graph-neural-network recommenders can be removed after training by projecting node embeddings away from an estimated popularity direction, with no retraining.","keywords":["popularity bias","collaborative filtering","graph neural networks","post-hoc debiasing","embedding projection","recommender systems","unbiased evaluation","long-tail distribution"],"falsifier":"Train a GNN-based CF model on a dataset with known unbiased ratings, construct random unit vectors in embedding space, and apply the same projection-and-subtraction step (tuning phi and beta on validation data); if random directions yield similar improvements in unbiased metrics, then the specific popularity-direction construction is not what carries the performance.","tokens_in":20761,"feed_emoji":"🎯","tokens_out":3262,"duration_ms":26135,"temperature":0.7,"pith_summary":"The authors are trying to establish that a post-hoc, training-free correction can undo popularity bias that graph neural networks amplify during message passing. They define an interaction-level popularity score for each user–item pair, combine it into per-node popularity and preference centroids, and use their difference as a popularity direction vector. Projecting each learned embedding onto that direction and subtracting the component removes popularity-related signal while keeping preference signals. If correct, deployed recommender systems could be debiased cheaply by editing embeddings after training, without retraining.","feed_headline":"After-training fix removes popularity bias from GNN recommenders","feed_subtitle":"New post-hoc method estimates a popularity direction per node and subtracts it from embeddings, beating retraining-based debiasing.","key_machinery":"The popularity direction vector d_pop(v) = e_bar_pop(v) − phi * e_bar_pref(v), built from interaction-level popularity scores b_ui = p_i − r_ui. Projection of each node embedding onto this direction and subtraction of that component is the mechanism claimed to strip popularity while preserving preference.","core_discovery":"PPD works directly on embeddings from a pre-trained GNN-based collaborative filtering model. For each interaction it estimates a popularity score b_ui = p_i − r_ui, where p_i is the item's average similarity to all users (global preference) and r_ui is the item's average similarity to the user's historical items, penalized by global preference. These scores yield two centroids per node, a popularity centroid and a preference centroid; the difference, scaled by coefficient phi, defines a per-node popularity direction vector. The node embedding is then updated by subtracting its vector projection onto that direction at layer 0, and the debiased embeddings are propagated through the remaining l","pith_inferences":["If the projection direction is truly aligned with popularity, the same procedure could in principle be applied to embeddings from non-GNN recommender models, but the paper only tests GNN backbones.","The method's reliance on tunable hyperparameters phi and beta suggests the 'popularity direction' is not uniquely identifiable from the data alone; a validation set is used to pick them.","A testable extension: apply PPD to embeddings from matrix factorization or transformer-based recommenders to see whether the debiasing geometry transfers.","The paper leaves implicit that the projection could be composed with other post-hoc corrections (e.g., exposure or position bias) by defining analogous direction vectors."],"forward_implications":["Deployed GNN recommender systems can be debiased without retraining, by editing the base embeddings once.","The method applies to any GNN-based CF backbone that produces user/item embeddings, not just one architecture.","On datasets with strong popularity skew, relative gains are large (e.g., the paper reports improvements over baselines on all metrics).","Head and tail item performance can improve together, rather than trading off.","Increasing GNN depth does not necessarily hurt when debiasing is applied, mitigating bias amplification and over-smoothing."],"fun_headline_variants":["Post-hoc fix strips popularity bias from GNN embeddings","Debias GNN recommenders without retraining","Subtract popularity direction to debias GNN embeddings","After training, remove popularity bias from GNN embeddings"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that the difference between the popularity centroid and the preference centroid points precisely along the popularity confound in embedding space, so that subtracting the projection removes popularity and nothing else.","fun_headline_variants_meta":{"raw":{"variants":["Post-hoc fix strips popularity bias from GNN embeddings","Debias GNN recommenders without retraining","Subtract popularity direction to debias GNN embeddings","After training, remove popularity bias from GNN embeddings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000507,"raw_usage":{"total_tokens":2308,"prompt_tokens":747,"completion_tokens":1561,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":1499}},"tokens_in":491,"tokens_out":1561,"duration_ms":9436,"temperature":1.0,"reasoning_tokens":1499,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T09:49:04.040512+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a GNN-based CF model on a dataset with known unbiased ratings, construct random unit vectors in embedding space, and apply the same projection-and-subtraction step (tuning phi and beta on validation data); if random directions yield similar improvements in unbiased metrics, then the specific popularity-direction construction is not what carries the performance.","supporting_citations":[],"review_version":1}