REVIEW 1 major objections 49 references
G2LoRA resolves task interference in continual learning on text-attributed graphs by projecting gradients in category-aware subspaces and modulating their magnitudes across encoders.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-28 15:23 UTC pith:K7VJJ7LB
load-bearing objection G2LoRA adds category-aware projection and magnitude modulation to handle forgetting in graph-text continual learning, but the abstract leaves the actual gains hard to judge without numbers or ablations. the 1 major comments →
G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
G2LoRA unifies node-, link-, and graph-level tasks under a single graph-text alignment objective for consistent optimization across incremental modes. It performs category-aware gradient projection in structured subspaces to resolve conflicting updates and enable conditional backward transfer, while introducing gradient magnitude modulation to coordinate update rates between graph and text encoders.
What carries the argument
Category-aware gradient projection in structured subspaces with gradient magnitude modulation between encoders.
Load-bearing premise
That a single graph-text alignment objective produces consistent optimization across all incremental modes and that magnitude modulation alone can coordinate the differing sensitivities of the encoders.
What would settle it
An experiment where sequential fine-tuning on new tasks still causes significant degradation in alignment between graph and text embeddings or high forgetting rates on previous tasks despite using G2LoRA.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes G2LoRA, a continual learning framework for LLM-as-Aligner models on text-attributed graphs (TAGs). It unifies node-, link-, and graph-level tasks under a single graph-text alignment objective to enable consistent optimization across incremental modes. To mitigate task interference and promote positive transfer, it performs category-aware gradient projection in structured subspaces, resolving conflicting updates while enabling conditional backward transfer. Gradient magnitude modulation is introduced to coordinate update rates between graph and text encoders and prevent cross-modal drift. Experiments on benchmark datasets are claimed to show consistent outperformance over strong baselines across backbone architectures and incremental settings.
Significance. If the empirical claims hold with proper controls, the work would address a practically relevant gap in continual learning for multimodal graph models, where catastrophic forgetting and modality misalignment are common. The combination of subspace projection for interference resolution and magnitude modulation for encoder coordination offers a concrete engineering approach that could be adopted in streaming TAG applications. The unification of heterogeneous tasks under one alignment objective is a notable design choice that, if validated, simplifies the continual learning setup.
major comments (1)
- [Abstract] Abstract and §1: the central claims of superior continual performance, effective resolution of task interference, and balanced forward/backward transfer rest on experimental results, yet the provided text supplies no ablation studies, error bars, dataset statistics, or quantitative comparisons. This prevents evaluation of whether the proposed components deliver the claimed benefits or whether the weakest assumption (unification under a single objective plus magnitude modulation suffices without task-specific loss) holds.
Simulated Author's Rebuttal
We thank the referee for the detailed review and the opportunity to clarify the presentation of our experimental validation. We address the concern point by point below.
read point-by-point responses
-
Referee: [Abstract] Abstract and §1: the central claims of superior continual performance, effective resolution of task interference, and balanced forward/backward transfer rest on experimental results, yet the provided text supplies no ablation studies, error bars, dataset statistics, or quantitative comparisons. This prevents evaluation of whether the proposed components deliver the claimed benefits or whether the weakest assumption (unification under a single objective plus magnitude modulation suffices without task-specific loss) holds.
Authors: We acknowledge that the abstract and §1 present high-level claims without embedding the full quantitative details. The complete manuscript contains these elements in later sections: dataset statistics and task settings appear in §3, quantitative comparisons (including forward/backward transfer metrics) are reported with error bars (std. dev. over 5 seeds) in Tables 1–4 of §4, and ablation studies isolating the category-aware projection, magnitude modulation, and single-objective unification are provided in §5 (Tables 5–7). These ablations directly compare against variants that retain task-specific losses, showing that the unified alignment objective plus our two components yields the reported gains without requiring per-task losses. If the referee’s copy omitted §4–5, we will ensure the camera-ready version cross-references key results from the abstract and §1. We can also insert a one-paragraph summary of the main quantitative findings into §1 upon revision. revision: partial
Circularity Check
No significant circularity detected
full rationale
The provided abstract and description frame G2LoRA as an engineering framework that unifies tasks under a single alignment objective, applies category-aware gradient projection in subspaces, and uses magnitude modulation for encoder coordination. No equations, fitted parameters, predictions derived from fits, or self-citations appear in the text. The central claims are presented as design choices for mitigating forgetting and interference rather than a derivation chain that reduces to its own inputs by construction. The method is self-contained against external benchmarks with no load-bearing self-referential steps.
Axiom & Free-Parameter Ledger
read the original abstract
LLM-as-Aligner has emerged as a prevalent pre-training paradigm for Text-Attributed Graphs(TAGS), aligning graph and text modalities into a shared embedding space via CLIP-style contrastive learning. While effective on individual downstream tasks, we observe severe catastrophic forgetting when such models are sequentially fine-tuned on streaming tasks. Although parameter-efficient fine-tuning alleviates forgetting to some extent, it remains insufficient to resolve task interference and ineffective knowledge transfer. In this work, we study graph continual learning for LLM-as-Aligner models on TAGs, with the goal of mitigating interference while promoting positive transfer across tasks. This setting introduces two fundamental challenges: (1) heterogeneous downstream tasks induce shifting optimization objectives, hindering unified fine-tuning; and (2) graph and text encoders exhibit different sensitivities to adaptation, making uncoordinated updates prone to misalignment. To address these challenges, we propose G2LoRA, a continual learning framework for TAGs. G2LoRA unifies node-, link-, and graph-level tasks under a single graph--text alignment objective, and enables consistent optimization across domain/class/task incremental modes. To reduce task interference while encouraging positive transfer, G2LoRA performs category-aware gradient projection in structured subspaces, resolving conflicting updates and enabling conditional backward transfer to balance forward and backward knowledge flow. To further prevent cross-modal drift, G2LoRA introduces gradient magnitude modulation to coordinate update rates between graph and text encoders. Extensive experiments on benchmark datasets demonstrate that G2LoRA consistently outperforms strong baselines across different backbone architectures, achieving superior continual performance and transferability.
Figures
Reference graph
Works this paper leans on
-
[1]
Hongxu Chen, Hongzhi Yin, Xiangguo Sun, Tong Chen, Bogdan Gabrys, and Katarzyna Musial. 2020. Multi-level graph convolutional networks for cross- platform anchor link prediction. InProceedings of the 26th ACM SIGKDD interna- tional conference on knowledge discovery & data mining. 1503–1511
2020
-
[2]
Junru Chen, Yang Yang, Tao Yu, Yingying Fan, Xiaolong Mo, and Carl Yang
-
[3]
InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining
Brainnet: Epileptic wave detection from seeg with hierarchical graph diffu- sion learning. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2741–2751
-
[4]
Runjin Chen, Tong Zhao, Ajay Kumar Jaiswal, Neil Shah, and Zhangyang Wang
-
[5]
InForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024
LLaGA: Large Language and Graph Assistant. InForty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net
2024
-
[6]
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et al . 2024. Exploring the potential of large language models (llms) in learning on graphs.ACM SIGKDD Explorations Newsletter25, 2 (2024), 42–61
2024
- [7]
-
[8]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186
2019
-
[9]
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783 (2024)
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[10]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs.Advances in neural information processing systems30 (2017)
2017
- [12]
- [13]
-
[14]
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.ICLR1, 2 (2022), 3
2022
-
[15]
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs.Advances in neural information processing systems 33 (2020), 22118–22133
2020
-
[16]
TN Kipf. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907(2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[17]
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks.Proceedings of the national academy of sciences114, 13 (2017), 3521– 3526
2017
-
[18]
Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting.IEEE transac- tions on pattern analysis and machine intelligence40, 12 (2017), 2935–2947
2017
-
[19]
Yan-Shuo Liang and Wu-Jun Li. 2024. InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, W A, USA, June 16-22, 2024. IEEE, 23638–23647
2024
- [20]
-
[21]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 (2019)
work page internal anchor Pith review Pith/arXiv arXiv 2019
- [22]
-
[23]
Zheyuan Liu, Xiaoxin He, Yijun Tian, and Nitesh V. Chawla. 2024. Can we Soft Prompt LLMs for Graph Learning Tasks?. InCompanion Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, Singapore, May 13-17, 2024, Tat- Seng Chua, Chong-Wah Ngo, Roy Ka-Wei Lee, Ravi Kumar, and Hady W. Lauw (Eds.). ACM, 481–484
2024
-
[24]
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. 2023. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. InPro- ceedings of the ACM web conference 2023. 417–428
2023
- [25]
-
[26]
Chaoxi Niu, Guansong Pang, Ling Chen, and Bing Liu. 2024. Replay-and-forget- free graph class-incremental learning: A task profiling and prompting approach. Advances in Neural Information Processing Systems37 (2024), 87978–88002
2024
-
[27]
Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. 2022. Balanced multimodal learning via on-the-fly gradient modulation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8238–8247
2022
-
[28]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learning. 8748–8763
2021
-
[29]
Gobinda Saha, Isha Garg, and Kaushik Roy. 2021. Gradient Projection Memory for Continual Learning. In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net
2021
-
[30]
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data.AI magazine29, 3 (2008), 93–93
2008
-
[31]
Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. 2023. All in one: Multi-task prompting for graph neural networks. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2120–2131
2023
-
[32]
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024. Graphgpt: Graph instruction tuning for large language models. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 491–500
2024
-
[33]
Qi-Wei Wang, Da-Wei Zhou, Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye
-
[34]
Advances in Neural Information Processing Systems36 (2023), 15060–15076
Few-shot class-incremental learning via training-free prototype calibration. Advances in Neural Information Processing Systems36 (2023), 15060–15076
2023
-
[35]
Ruijie Wang, Zheng Li, Dachun Sun, Shengzhong Liu, Jinning Li, Bing Yin, and Tarek Abdelzaher. 2022. Learning to sample and aggregate: few-shot reason- ing over temporal knowledge graphs. InProceedings of the 36th International Conference on Neural Information Processing Systems (NeurIPS ’22)
2022
-
[36]
Ruijie Wang, Zheng Li, Jingfeng Yang, Tianyu Cao, Chao Zhang, Bing Yin, and Tarek Abdelzaher. 2023. Mutually-paced Knowledge Distillation for Cross-lingual Temporal Knowledge Graph Reasoning. InProceedings of the ACM Web Conference 2023 (WWW ’23)
2023
-
[37]
Ruijie Wang, Yutong Zhang, Jinyang Li, Shengzhong Liu, Dachun Sun, Tianchen Wang, Tianshi Wang, Yizhuo Chen, Denizhan Kara, and Tarek Abdelzaher. 2024. MetaHKG: Meta Hyperbolic Learning for Few-shot Temporal Reasoning. In Proceedings of the 47th International Conference on Research and Development in Information Retrieval (SIGIR ’24)
2024
-
[38]
Zhihao Wen and Yuan Fang. 2023. Augmenting Low-Resource Text Classifica- tion with Graph-Grounded Pre-training and Prompting. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023, Hsin-Hsi Chen, Wei-Jou (Edward) Duh, Hen-Hsen Huang, Makoto P. Kato,...
2023
-
[39]
Nan Wu, Stanislaw Jastrzebski, Kyunghyun Cho, and Krzysztof J Geras. 2022. Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks. InInternational Conference on Machine Learning. 24043–24055
2022
- [40]
- [41]
-
[42]
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. 2023. A comprehensive study on text-attributed graphs: Benchmarking and rethinking.Advances in Neural Information Processing Systems36 (2023), 17238–17264
2023
-
[43]
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. 2019. Continual learning of context-dependent processing in neural networks.Nature Machine Intelligence1, 8 (2019), 364–372
2019
-
[44]
Peiyan Zhang, Yuchen Yan, Chaozhuo Li, Senzhang Wang, Xing Xie, Guojie Song, and Sunghun Kim. 2023. Continual learning on dynamic graphs via parameter isolation. InProceedings of the 46th international ACM SIGIR conference on research and development in information retrieval. 601–611
2023
-
[45]
Xikun Zhang, Dongjin Song, and Dacheng Tao. 2022. Hierarchical prototype networks for continual graph representation learning.IEEE Transactions on Pattern Analysis and Machine Intelligence45, 4 (2022), 4622–4636
2022
-
[46]
Xikun Zhang, Dongjin Song, and Dacheng Tao. 2023. Ricci curvature-based graph sparsification for continual graph representation learning.IEEE Transactions on Neural Networks and Learning Systems(2023). Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning KDD ’26, August 09–13, 2026, Jeju Island, Republic of Korea
2023
- [47]
-
[48]
Da-Wei Zhou, Zi-Wen Cai, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. 2025. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need.International Journal of Computer Vision133, 3 (2025), 1012–1032
2025
-
[49]
Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. 2025. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. InProceedings of the ACM on Web Conference 2025. 2183–2197
2025
-
[50]
Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang. 2024. Efficient Tuning and Inference for Large Language Models on Textual Graphs. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju, South Korea, August 3-9, 2024. ijcai.org, 5734–5742. A Appendix A.1 Proof of Theorem 1 (1) For Rule 1, we have 𝑊...
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.