REVIEW 12 cited by
TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning
read the original abstract
Dynamic graph modeling has recently attracted much attention due to its extensive applications in many real-world scenarios, such as recommendation systems, financial transactions, and social networks. Although many works have been proposed for dynamic graph modeling in recent years, effective and scalable models are yet to be developed. In this paper, we propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion and enables effective dynamic node representation learning that captures both the temporal and topology information. Technically, our model contains three novel aspects. First, we generalize the vanilla Transformer to temporal graph learning scenarios and design a graph-topology-aware transformer. Secondly, on top of the proposed graph transformer, we introduce a two-stream encoder that separately extracts representations from temporal neighborhoods associated with the two interaction nodes and then utilizes a co-attentional transformer to model inter-dependencies at a semantic level. Lastly, we are inspired by the recently developed contrastive learning and propose to optimize our model by maximizing mutual information (MI) between the predictive representations of two future interaction nodes. Benefiting from this, our dynamic representations can preserve high-level (or global) semantics about interactions and thus is robust to noisy interactions. To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs. We evaluate our model on four benchmark datasets for interaction prediction and experiment results demonstrate the superiority of our model.
Forward citations
Cited by 12 Pith papers
-
Learning Long Range Spatio-Temporal Representations over Continuous Time Dynamic Graphs with State Space Models
CTDG-SSM introduces CTT-HiPPO, a Laplacian-polynomial projection of HiPPO, to create a parameter-efficient state-space formulation for continuous-time dynamic graphs that captures long-range spatio-temporal patterns.
-
Graph Retention Networks for Dynamic Graphs
Graph Retention Networks extend retention to dynamic graphs to enable parallelizable training, O(1) inference, and chunkwise long-term training while delivering competitive performance with major efficiency gains.
-
OPD-Evolver: Cultivating Holistic Agent Evolver via On-Policy Distillation
OPD-Evolver uses on-policy self-distillation in fast interaction and slow attribution loops to build agents with holistic memory competence, outperforming prior systems by up to 11.5% and allowing a 9B model to compet...
-
Forget Less, Generalize More: Unifying Temporal and Structural Adaptation for Dynamic Graphs
DSRD unifies temporal and structural adaptation for dynamic graphs via a single recurrent retentive state with learnable time-sensitivity parameters in the decay kernels.
-
ST-TGExplainer: Disentangling Stability and Transition Patterns for Temporal GNN Interpretability
ST-TGExplainer disentangles stability and transition patterns in temporal graphs via a self-explainable TGNN guided by a disentangled information bottleneck objective to produce more faithful explanations.
-
Attention Dispersion in Dynamic Graph Transformers: Diagnosis and a Transferable Fix
Diagnoses attention dispersion in CTDG Transformers under temporal shift and introduces differential attention to suppress common signals and achieve SOTA on shifted benchmarks.
-
PRISM: Iterative Cross-Modal Posterior Refinement for Dynamic Text-Attributed Graphs
PRISM iteratively transforms semantic priors into behavior-conditioned posteriors via cross-modal refinement to improve representation learning on dynamic text-attributed graphs.
-
DyGnROLE: Asymmetric Pretraining for Edge Classification on Dynamic Graphs
Role-disentangled Transformer plus temporal contrastive pretraining improves future edge classification with 10k labels on 7 of 8 DTGB datasets.
-
CoDCL: Counterfactual-Inspired Augmentation Contrastive Learning for Temporal Link Prediction in Social Networks
CoDCL combines counterfactual data augmentation with contrastive learning as a universal module to improve temporal link prediction in social networks.
-
What Do Temporal Graph Learning Models Learn?
Temporal graph models consistently learn to favor popular nodes but fail to learn edge direction, density, and recency.
-
Learning Discriminative and Generalizable Anomaly Detector for Dynamic Graph with Limited Supervision
SDGAD combines residual event representations, a two-hypersphere restriction loss, and a normalizing-flow boundary to detect dynamic-graph anomalies with little or no supervision.
-
TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism
TGFormer is a temporal graph Transformer using an auto-correlation mechanism to uncover periodic dependencies, reporting up to 9.35% precision gains over prior methods on six benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.