Are We Really Measuring Progress? Transferring Insights from Evaluating Recommender Systems to Temporal Link Prediction

Filip Cornell; Gabriela Zarzar Gandler; Lele Cao; Oleg Smirnov

REVIEW 3 major objections 5 minor 1 cited by

Are We Really Measuring Progress? Transferring Insights from Evaluating Recommender Systems to Temporal Link Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current temporal link prediction benchmarks can rank the same models differently depending on the evaluation protocol, so measured progress may be unreliable.

desk verdict A useful, honest translation of known recsys evaluation flaws to TLP; the core concerns are solid, but the TLP-specific Simpson's paradox evidence rests on heuristics, not neural models. read the letter →

arxiv 2506.12588 v1 pith:3QRE53GX submitted 2025-06-14 cs.LG

Filip Cornell , Oleg Smirnov , Gabriela Zarzar Gandler , Lele Cao This is my paper · ORCID

classification cs.LG

keywords temporalgraphlearninglinkpredictionevaluationprotocolssampledrankingmetricshardnegativesamplingrecommendersystemsbenchmarkreliabilitySimpson'sparadox

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that current evaluation protocols for Temporal Link Prediction (TLP) are unreliable because they suffer from three identifiable design flaws: inconsistent sampled ranking metrics, hard negative sampling, and metrics that combine predictions across source nodes as if all nodes had equal base probabilities. Borrowing from recommender systems evaluation research, the authors show with illustrative examples that these flaws can produce misleading model rankings, and that different evaluation protocols can rank the same models differently, sometimes reversing order. If correct, this means existing TLP benchmark leaderboards may not actually measure progress in temporal graph learning.

What carries the argument

The central machinery is the evaluation protocol itself, decomposed into three components: sampled ranking metrics (e.g., sampled MRR, equivalent to the simplified sampled AP shown by Krichene and Rendle to be statistically inconsistent), the negative sampler (uniform vs hard; using historical edges as negatives), and the aggregation scheme (per-source-node vs combined across nodes). The paper uses scalable heuristics that score destination nodes by recency or popularity, at local and global scales, to approximate full-ranking evaluations that are computationally prohibitive for TGNNs, and Spearman rank correlation to quantify ranking consistency across protocols.

What would settle it

Run a standard TGNN (such as TGN or DyGFormer) on a public TLP dataset like Yelp; evaluate it with both full ranking and sampled MRR, under both uniform and hard negative sampling, and compute per-source-node vs combined metrics. If the pairwise ranking of several models stays stable across all these protocols, the paper's claim that these evaluation flaws reverse model rankings would be falsified for that model class and dataset.

Watch

Extended reading notes

Core claim

The central claim is stated directly in the abstract: current TLP evaluation protocols are 'often affected by one or more of' three issues: (1) inconsistent sampled metrics, (2) reliance on hard negative sampling, and (3) metrics that implicitly assume equal base probabilities across source nodes by combining predictions. The paper supports each with evidence. For issue (1), using scalable heuristics (recency and popularity at local and global scales) on the Yelp dataset, it finds a Simpson's paradox: local heuristics correlate strongly positive with full MRR, but when combined with global heuristics the overall correlation turns negative. For issue (2), it recomputes Spearman rank correlations of model AUC rankings across different negative sampling strategies and finds negative correlations on at least one dataset per strategy, meaning model rankings can reverse. For issue (3), a toy example shows perfect per-query ranks while aggregated ROC-AUC and AP report low scores. The paper hypothesizes these effects carry over to neural TGNNs, since heuristics produce sampled metric scores closely aligned with those of TGNNs.

Load-bearing premise

The illustrative examples rely on hand-designed heuristics (recency and popularity) as proxies for neural temporal graph models, because full inference on neural models is prohibitively expensive; if heuristic rankings do not behave like neural model rankings, the demonstrations of Simpson's paradox and metric inconsistency do not directly support the paper's claims about how TGNNs are evaluated.

Editorial extensions

If this is right

Existing TLP leaderboards (DGB, TGB, BenchTemp, TGB-Seq, DyGLib) can rank the same models differently under different evaluation protocols, so reported progress may be an artifact of the protocol.
Sampled MRR results from TGB and TGB-Seq should be interpreted with caution unless accompanied by correction methods or consistent alternatives such as dMR.
Hard negative sampling should be avoided or aligned with inference-time candidate generation; conclusions about model performance should be limited to settings matching the sampler.
Metrics that combine predictions across source nodes should be replaced by per-source-node evaluation, or corrected for node degree and base-probability differences.
A unified evaluation framework for TLP, borrowing consistency results from recommender systems, would let model comparisons be interpreted confidently.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The three flaws are likely to affect Temporal Knowledge Graph evaluation too, where negative sampling and ranking metrics are also standard; the paper's heuristics-based evidence suggests the problem is structural, not model-specific.
Since the paper's heuristic proxies are simpler than TGNNs, real neural models with learned embeddings might show even stronger rank inconsistencies, because embedding similarity can assign high scores to historical or popular edges in ways heuristics would not.
A direct testable extension: re-run a published TGNN on one dataset under uniform and hard negative sampling and compute the Spearman correlation across samplers; if the correlation is close to 1, the issue may be less severe than claimed for that model class.
The 'equal base probability' critique suggests a concrete fix: normalize ranking metrics per source node, or weight test edges by inverse source-node degree, before aggregating leaderboard scores.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This workshop paper translates three well-established recommender-system evaluation flaws to temporal link prediction and gives a clean benchmark characterization. It doesn't break new theoretical ground, but the translation is useful and the paper is honest about what it hasn't verified.

The best parts: Table 1 summarizes which benchmarks (DGB, TGB, BenchTemp, TGB-Seq, DyGLib) exhibit which of the three issues, and that table alone is worth having. The paper correctly leans on Krichene & Rendle for the inconsistency of sampled ranking metrics, and Table 2, which reanalyzes Poursafaei et al.'s AUC results, shows that model rankings flip under different negative samplers on several datasets. That is direct evidence for issue 2, not a hypothetical. The authors also deserve credit for stating plainly that the Simpson's paradox in Figure 1 is demonstrated with recency/popularity heuristics, not TGNNs, and that transfer to neural models is a hypothesis.

The stress-test note is on target: Figure 1 is the centerpiece for issue 1, and it is heuristic-only. The authors cite their own earlier result that heuristics align with TGNNs on these benchmarks, but that is an indirect bridge. Without at least one TGNN-based check on a small dataset, the TLP-specific demonstration of the paradox is suggestive, not confirming. The toy example in Figure 2 is fine for illustration but doesn't quantify how often combined-prediction distortions actually change leaderboard rankings in practice. The paper also provides no code or confidence intervals, which is common for an extended abstract but limits reuse. The conjecture about power-law degree effects is explicitly labeled as a conjecture, so that is fine. None of these problems sink the paper, because the core claims rest on strong recsys literature.

This is for anyone reading TGB/DGB/TGB-Seq leaderboards and wondering why rankings shift across papers. It is a workshop-level contribution that deserves a serious referee, not a desk reject. If it were submitted to a full venue, I would want the neural replication and some code. As a workshop abstract, it earns its place. I would bring it to reading group and would cite it in work on TLP evaluation. Send it to peer review, with the expectation that the authors will be pushed on the heuristics-to-TGNN gap.

Referee Report

3 major / 5 minor

Summary. The paper is an extended abstract that critiques current evaluation protocols for Temporal Link Prediction (TLP). It identifies three issues: (1) sampled ranking metrics are inconsistent under sampling, (2) hard negative sampling can distort or reverse model rankings, and (3) metrics that combine edge predictions across source nodes implicitly assume equal base probabilities and can be dominated by high-degree nodes. The authors connect these issues to existing recommender-systems literature, summarize their presence across five TLP benchmarks in Table 1, and provide illustrative evidence: a Simpson's-paradox example comparing sampled and full MRR for local versus global heuristics on the Yelp dataset, rank correlations across negative sampling strategies computed from prior results, and a toy example contrasting per-query ranking metrics with global AUC/AP. The paper closes with a discussion of remedies and states that systematic characterization and correction methods are ongoing work.

Significance. If the transfer from recommender-systems evaluation to TLP is accepted, the paper identifies a genuine and underappreciated risk: TLP leaderboards may reflect evaluation-protocol choices rather than model quality. The paper is unusually honest about the limits of its own evidence, explicitly marking the TGNN transfer as a hypothesis in Section 2.1 and describing the work as ongoing. Its strengths are the synthesis of the recsys sampling literature, the explicit benchmark catalog in Table 1, and the concrete illustrative examples that make the abstract concerns tangible. The paper does not ship machine-checked proofs or code, but it correctly anchors its central metric-inconsistency claim in the analytic results of Krichene and Rendle, which apply directly to the sampled-MRR protocols used by TGB and TGB-Seq. The contribution is a position/analysis piece rather than a completed empirical study, and it should be judged on that basis.

major comments (3)

[Section 2.1 / Figure 1] The empirical demonstration of issue 1 is performed only with hand-designed recency and popularity heuristics, not with TGNNs. The paper itself states that full TGNN inference is 'prohibitively expensive' and that 'we hypothesize that similar effects may also manifest in these models as well.' Because the abstract claims that current TLP evaluation protocols are 'often affected' by inconsistent sampled metrics, this heuristic-only evidence is load-bearing for the TLP-specific version of the claim. I recommend either adding a small neural replication (e.g., one or two TGNNs on a subset of Yelp or another TGB-Seq dataset with a bounded number of timestamps) to show sampled-versus-full MRR rank reversals, or rewording the abstract and conclusion so that the empirical demonstration is explicitly about heuristic scorers and the transfer to TGNNs is presented as a hypothesis supported only by the prior heuristic-TGNN alignment evidence cited from Cornell et al. (2025).
[Section 2.2 / Table 2] Table 2 computes Spearman correlations from seven models reported in a single external study, so each correlation is based on only seven rank pairs. With n=7, correlations such as -0.1 and 0.2 are uninformative, and the statement 'only 8 out of 39 correlations approach 0.9' overstates the strength of the evidence unless uncertainty is quantified. Please report the exact number of models per column, add confidence intervals or permutation p-values, and clarify the column labels, which currently appear duplicated as 'Historical Historical Inductive Inductive Random Random' in the typeset table.
[Section 4 / Abstract] The abstract's 'often affected' claim is not matched by a prevalence analysis. The paper provides one illustrative dataset for issue 1, one toy example for issue 3, one dataset-based mechanism for issue 2, and a manual benchmark summary in Table 1. That is sufficient to show that these problems can occur, but not that they occur 'often' across TLP benchmarks. I recommend either quantifying how many of the five benchmarks' official evaluation scripts actually use each protocol element, or softening the wording to 'can be affected' with explicitly illustrative counterexamples.

minor comments (5)

[Section 2.1 / Figure 1] The caption does not define the local and global heuristics or the exact sampling protocol (number of negatives, seed, filtering of positive edges); please add these details so the figure is reproducible.
[Section 3] The estimator written as 'dMR' in the manuscript is not defined at first use; please add a one-line definition in the style of Krichene and Rendle.
[Section 3 (EXHaustive paragraph)] The paragraph on EXHaustive says it 'may introduce its own biases' and then immediately repeats 'also introduces its own biases'; remove the redundancy.
[Appendix A] The 'surprise index' cited from Cornell et al. (2025) is not defined in this manuscript; a brief definition would help readers interpret the tgbl-review example.
[General] No code or data artifacts are provided for Figures 1 and 2; for a workshop paper this is acceptable, but a link to the heuristic definitions and plotting scripts would increase the reproducibility of the empirical illustrations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the paper's critique is grounded in external analytic results (Krichene and Rendle, Cañamares and Castells) and explicitly labeled hypotheses, not in fitted parameters or self-derived predictions.

full rationale

The paper does not present a fitted model, nor does it compute a prediction from parameters fit to data. Its central claims about inconsistent sampled metrics are imported from external, analytically established recommender-systems literature (Krichene and Rendle 2020; Cañamares and Castells 2020), and the TLP-specific illustrations are framed as examples rather than as derivations of the general conclusion. The same-author citations (Cornell et al. 2025) are used to support the effectiveness of heuristics and to supply a surprise-index observation; these are premises for illustrative demonstrations, not the conclusion that TLP evaluation is unreliable. Moreover, the paper explicitly labels the transfer to TGNNs as a hypothesis: 'we hypothesize that similar effects may also manifest in these models as well.' There is no equation in which an output quantity is defined in terms of the quantity it claims to predict, and no fitted parameter is renamed as a prediction. The toy example in Figure 2 is a constructed counterexample, not a fitted result. Overall, the derivation chain is self-contained or rests on external, independently published analytic results, so no circularity is present.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters and no invented entities. Its central claims rely on standard mathematical equivalences from cited work and on domain assumptions about network structure and heuristic representativeness, both of which are at least partially flagged by the authors.

assumptions (3)

standard math Sampled MRR is equivalent to the simplified sampled AP analyzed by Krichene and Rendle (2020), which is inconsistent under sampling.
Used in Section 2.1 to argue that TGB and TGB-Seq sampled MRR metrics are inconsistent. The equivalence is cited to Krichene and Rendle 2020 and not re-derived here.
domain assumption Hand-designed heuristics (recency and popularity, local and global) are representative of how neural temporal graph models behave in ranking evaluations.
Section 2.1 states that TGNN inference is prohibitively expensive and hypothesizes that similar effects manifest in TGNNs, supported only by alignment of scores in Cornell et al. 2025.
domain assumption Node degrees in real-world graphs follow a power-law distribution such that hub nodes dominate sampled evaluation.
Section 2.3 cites Barabási and Albert 1999 and conjectures that degree imbalance substantially affects evaluation outcomes.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Are We Really Measuring Progress? Transferring Insights from Evaluating Recommender Systems to Temporal Link Prediction." pith.science (2026). https://pith.science/paper/3QRE53GX

@misc{pith2026250612588,
  author       = {Pith},
  title        = {Pith review of: Are We Really Measuring Progress? Transferring Insights from Evaluating Recommender Systems to Temporal Link Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3QRE53GX}},
  note         = {Machine review of arXiv:2506.12588}
}

read the original abstract

Recent work has questioned the reliability of graph learning benchmarks, citing concerns around task design, methodological rigor, and data suitability. In this extended abstract, we contribute to this discussion by focusing on evaluation strategies in Temporal Link Prediction (TLP). We observe that current evaluation protocols are often affected by one or more of the following issues: (1) inconsistent sampled metrics, (2) reliance on hard negative sampling often introduced as a means to improve robustness, and (3) metrics that implicitly assume equal base probabilities across source nodes by combining predictions. We support these claims through illustrative examples and connections to longstanding concerns in the recommender systems community. Our ongoing work aims to systematically characterize these problems and explore alternatives that can lead to more robust and interpretable evaluation. We conclude with a discussion of potential directions for improving the reliability of TLP benchmarks.

Figures

Figures reproduced from arXiv: 2506.12588 by the authors.

**Figure 2.** An example where ranking metrics computed per [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

What Do Temporal Graph Learning Models Learn?
cs.LG 2025-10 conditional novelty 6.0 of 10

Temporal graph models consistently learn to favor popular nodes but fail to learn edge direction, density, and recency.

Reference graph

Works this paper leans on

15 extracted references · 10 canonical work pages · cited by 1 Pith paper

[4]

Julia Gastinger, Christian Meilicke, Federico Errica, Timo Sztyler, Anett Schülke, and Heiner Stuckenschmidt

DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models.arXiv preprint arXiv:2408.04713(2024). Julia Gastinger, Christian Meilicke, Federico Errica, Timo Sztyler, Anett Schülke, and Heiner Stuckenschmidt

arXiv 2024
[5]

Miguel A Hernán, David Clayton, and Niels Keiding

Long range propagation on continuous-time dynamic graphs.arXiv preprint arXiv:2406.02740(2024). Miguel A Hernán, David Clayton, and Niels Keiding

arXiv 2024
[8]

Yang Liu, Alan Medlar, and Dorota Glowacka

On item- sampling evaluation for recommender system.ACM Transactions on Recommender Systems2, 1 (2024), 1–36. Yang Liu, Alan Medlar, and Dorota Glowacka. 2023a. On the consistency, discriminative power and robustness of sampled metrics in offline top-n recommender system evaluation. InProceedings of the 17th ACM Conference on Recommender Systems. 1152–115...

work page 2024
[12]

InProceedings of the 15th ACM conference on recommender systems

Quality metrics in recommender systems: Do we calculate metrics consistently?. InProceedings of the 15th ACM conference on recommender systems. 708–713. Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. 2021a. Tcl: Transformer-based dynamic graph modelling via contrastive learning.arXiv ...

arXiv 2021
[13]

Wayne Xin Zhao, Zihan Lin, Zhichao Feng, Pengfei Wang, and Ji-Rong Wen

Towards better dynamic graph learning: New architecture and unified library.Advances in Neural Information Processing Systems36 (2023), 67686–67700. Wayne Xin Zhao, Zihan Lin, Zhichao Feng, Pengfei Wang, and Ji-Rong Wen

work page 2023
[14]

Han Zhu, Junqi Jin, Chang Tan, Fei Pan, Yifan Zeng, Han Li, and Kun Gai

A revisiting study of appropriate offline evaluation for top-N recommendation algorithms.ACM Transactions on Information Systems41, 2 (2022), 1–41. Han Zhu, Junqi Jin, Chang Tan, Fei Pan, Yifan Zeng, Han Li, and Kun Gai

work page 2022
[1980]

The need for biases in learning generalizations. (1980). Farimah Poursafaei, Shenyang Huang, Kellin Pelrine, and Reihaneh Rabbany. 2022a. To- wards better evaluation for dynamic link prediction.Advances in Neural Information Processing Systems35 (2022), 32928–32941. Farimah Poursafaei and Reihaneh Rabbany

work page 1980
[1999]

Maya Bechler-Speicher, Ben Finkelshtein, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Siraudin, Viktor Zaverkin, Michael M Bronstein, Mathias Niepert, Bryan Perozzi, et al

Emergence of scaling in random net- works.science286, 5439 (1999), 509–512. Maya Bechler-Speicher, Ben Finkelshtein, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Siraudin, Viktor Zaverkin, Michael M Bronstein, Mathias Niepert, Bryan Perozzi, et al

work page 1999

Show all 15 references

[2011]

Balázs Hidasi and Ádám Tibor Czapp

The Simpson’s paradox unraveled.International journal of epidemiology40, 3 (2011), 780–785. Balázs Hidasi and Ádám Tibor Czapp

2011
[2017]

InProceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining

Optimized cost per click in taobao display advertising. InProceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 2191–2200. 5 TGL Workshop, KDD 2025, August 03-07, 2025, Toronto, Canada Cornell, Smirnov, et al. A Exemplifying the Is...

2025
[2020]

Yan-Martin Tamm, Rinchin Damdinov, and Alexey Vasilev

Temporal graph networks for deep learning on dynamic graphs.arXiv preprint arXiv:2006.10637(2020). Yan-Martin Tamm, Rinchin Damdinov, and Alexey Vasilev

2020 arXiv
[2021]

Qiang Huang, Xin Wang, Susie Xi Rao, Zhichao Han, Zitao Zhang, Yongjun He, Quan- qing Xu, Yang Zhao, Zhigao Zheng, and Jiawei Jiang

OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs.arXiv preprint arXiv:2103.09430(2021). Qiang Huang, Xin Wang, Susie Xi Rao, Zhichao Han, Zitao Zhang, Yongjun He, Quan- qing Xu, Yang Zhao, Zhigao Zheng, and Jiawei Jiang. 2024b. Benchtemp: A general benchmark for...

2021 arXiv
[2022]

Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi

Offline recommender system evaluation: Challenges and new directions.AI magazine43, 2 (2022), 225–238. Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi

2022
[2023]

In2023 IEEE International Conference on Data Mining Workshops (ICDMW)

Exhaustive Evaluation of Dynamic Link Prediction. In2023 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 1121–1130. Farimah Poursafaei, Zeljko Zilic, and Reihaneh Rabbany. 2022b. A Strong Node Classi- fication Baseline for Temporal Graphs. InProceedings o...

2022
[2024]

InESANN 2024: Proceedings

Link prediction heuristics for temporal graph benchmark. InESANN 2024: Proceedings. i6doc. com, 381–386. Zifeng Ding, Yifeng Li, Yuan He, Antonio Norelli, Jingcheng Wu, Volker Tresp, Yunpu Ma, and Michael Bronstein

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [4]

Julia Gastinger, Christian Meilicke, Federico Errica, Timo Sztyler, Anett Schülke, and Heiner Stuckenschmidt

DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models.arXiv preprint arXiv:2408.04713(2024). Julia Gastinger, Christian Meilicke, Federico Errica, Timo Sztyler, Anett Schülke, and Heiner Stuckenschmidt

arXiv 2024

[2] [5]

Miguel A Hernán, David Clayton, and Niels Keiding

Long range propagation on continuous-time dynamic graphs.arXiv preprint arXiv:2406.02740(2024). Miguel A Hernán, David Clayton, and Niels Keiding

arXiv 2024

[3] [8]

Yang Liu, Alan Medlar, and Dorota Glowacka

On item- sampling evaluation for recommender system.ACM Transactions on Recommender Systems2, 1 (2024), 1–36. Yang Liu, Alan Medlar, and Dorota Glowacka. 2023a. On the consistency, discriminative power and robustness of sampled metrics in offline top-n recommender system evaluation. InProceedings of the 17th ACM Conference on Recommender Systems. 1152–115...

work page 2024

[4] [12]

InProceedings of the 15th ACM conference on recommender systems

Quality metrics in recommender systems: Do we calculate metrics consistently?. InProceedings of the 15th ACM conference on recommender systems. 708–713. Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. 2021a. Tcl: Transformer-based dynamic graph modelling via contrastive learning.arXiv ...

arXiv 2021

[5] [13]

Wayne Xin Zhao, Zihan Lin, Zhichao Feng, Pengfei Wang, and Ji-Rong Wen

Towards better dynamic graph learning: New architecture and unified library.Advances in Neural Information Processing Systems36 (2023), 67686–67700. Wayne Xin Zhao, Zihan Lin, Zhichao Feng, Pengfei Wang, and Ji-Rong Wen

work page 2023

[6] [14]

Han Zhu, Junqi Jin, Chang Tan, Fei Pan, Yifan Zeng, Han Li, and Kun Gai

A revisiting study of appropriate offline evaluation for top-N recommendation algorithms.ACM Transactions on Information Systems41, 2 (2022), 1–41. Han Zhu, Junqi Jin, Chang Tan, Fei Pan, Yifan Zeng, Han Li, and Kun Gai

work page 2022

[7] [1980]

The need for biases in learning generalizations. (1980). Farimah Poursafaei, Shenyang Huang, Kellin Pelrine, and Reihaneh Rabbany. 2022a. To- wards better evaluation for dynamic link prediction.Advances in Neural Information Processing Systems35 (2022), 32928–32941. Farimah Poursafaei and Reihaneh Rabbany

work page 1980

[8] [1999]

Maya Bechler-Speicher, Ben Finkelshtein, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Siraudin, Viktor Zaverkin, Michael M Bronstein, Mathias Niepert, Bryan Perozzi, et al

Emergence of scaling in random net- works.science286, 5439 (1999), 509–512. Maya Bechler-Speicher, Ben Finkelshtein, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Siraudin, Viktor Zaverkin, Michael M Bronstein, Mathias Niepert, Bryan Perozzi, et al

work page 1999