REVIEW 3 major objections 44 references
Fine-tuning a language model only on the first task, then updating class anchors with uncertainty weights and graph topology, yields state-of-the-art continual node classification on streaming text-attributed graphs.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-14 13:50 UTC pith:46W4BEFK
load-bearing objection Solid incremental engineering on SimGCL: entropy-weighted EMA anchors + light structural features give consistent SOTA gains, but the first-task-only LoRA premise is untested. the 3 major comments →
UNIT: Unleash Large Language Models Potential for Graph Continual Learning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors show that a single LoRA fine-tune of an LLM on the first task, followed by continual construction of uncertainty-weighted semantic anchors that are fused with lightweight structural anchors, produces classifier weights that transfer knowledge across class-incremental graph tasks more effectively than regularization, replay or static-prototype alternatives, reaching state-of-the-art accuracy on Cora, Citeseer, WikiCS, Photo and Products.
What carries the argument
Uncertain-aware anchor generation (entropy-weighted soft average of LLM embeddings, decayed by factor λ) fused with structural confluence modeling (degree, average-neighbor-degree and clustering coefficient projected into the same space, decayed by γ) to form the final class weight w_c = β p_c + (1-β) a_c.
Load-bearing premise
Fine-tuning the language model only once, on the language and topology of the very first task, is enough to keep its embeddings reliable for every later task that arrives.
What would settle it
Construct a multi-task stream whose later tasks deliberately shift vocabulary domain or average degree by a large margin, re-run UNIT without re-tuning the LLM, and measure whether ACC_avg and ACC_N drop below a baseline that is allowed to re-fine-tune or retrain on each new task.
If this is right
- Streaming text-attributed graphs can be classified continually without storing historical node data or replaying subgraphs.
- A single frozen LLM backbone can serve as a permanent feature extractor for an open-ended sequence of new classes once it has seen the first task.
- Classifier weights that blend uncertainty-calibrated text prototypes with local topology remain stable under class-incremental arrival, reducing catastrophic forgetting relative to pure GNN or pure LLM baselines.
- Few-shot variants of the same pipeline still outperform prior methods, indicating the anchors remain informative even when labeled data per task is scarce.
Where Pith is reading between the lines
- If later tasks drift far outside the first-task language distribution, the fixed embeddings may become mis-calibrated and the entropy weights uninformative, suggesting a cheap periodic re-adaptation trigger could be added without abandoning the one-shot philosophy.
- The same uncertainty-plus-structure anchor recipe could be tried on non-text multimodal streams (image-attributed or video-attributed graphs) once a suitable frozen encoder replaces the LLM.
- Because the structural side uses only cheap local statistics, the method may remain practical on very large product or social graphs where full-neighborhood attention is prohibitive.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes UNIT, a graph continual learning framework that fine-tunes an LLM with LoRA only on the first class-incremental task, then freezes it and maintains class-level semantic anchors (entropy-weighted, exponentially decayed) and structural anchors (degree / average neighbor degree / clustering coefficient) that jointly set classifier weights. On five text-attributed graph benchmarks under the LLM4GCL class-incremental protocol, UNIT reports higher ACC_avg and ACC_N than fifteen baselines, including the prior GLM method SimGCL, with supporting ablations, few-shot results, backbone scaling, and hyper-parameter sweeps.
Significance. If the results hold under fair comparison, UNIT is a useful practical advance for class-incremental node classification on text-attributed graphs: a single first-task LoRA pass plus lightweight prototype maintenance is cheaper than per-task GNN retraining or full LLM re-tuning, and the gains over SimGCL are consistent across five datasets and few-shot settings. The ablation (Fig. 4), backbone table (Table 3), and λ/γ sensitivity (Fig. 5) give a reasonably complete empirical package for an applied ML contribution. The work does not claim formal guarantees; its value is empirical and systems-oriented.
major comments (3)
- Section 3.2 and Eq. (2): The central design choice—instruction-tuning LoRA only on T_s1, then freezing f_θ for all later tasks—is load-bearing for every subsequent module (Eqs. 3–7, 12–13). The manuscript never measures embedding or probability calibration drift across tasks, nor does it ablate re-tuning or multi-task LoRA. On the reported splits language may stay in-domain, but the SOTA attribution to “uncertain-aware” and “structural confluence” modules is conditional on an untested stationarity assumption. A minimal fix is either (i) a drift/calibration plot of h_i and α_i over task index, or (ii) an ablation that re-tunes or freezes after different first tasks, plus an explicit limitation statement.
- Section 3.4, Eq. (8): Structural confluence is claimed to fix “structural information degradation” and semantic–structural imbalance relative to SimGCL’s linearized prompts. The actual structural descriptor is only (degree, mean neighbor degree, clustering coefficient) projected by g(·). That is far weaker than the high-order topology the introduction criticizes as lost under linearization. The ablation “w/o Str” (Fig. 4) shows a drop, but does not show that these three scalars recover the missing geometric prior. Either strengthen the structural encoder (e.g., short random-walk / GNN features) or tone down the claim that confluence “explicitly integrates graph topology” in a way that resolves the stated structural degradation.
- Tables 1–2 and experimental protocol: Means ± std are reported, but the number of random seeds / runs is not stated in the main text (only “presumably multiple runs”). Without N and a clear statement that all methods share the same task splits, LLM backbone (LLaMA3-8B), and evaluation protocol as LLM4GCL, the several-point margins over SimGCL cannot be assessed for significance. Please report N, seed list, and any shared codebase settings.
Circularity Check
No circularity: empirical prototype-based continual learning method whose anchors and classifier weights are constructed from model outputs and then evaluated on held-out sequential tasks.
full rationale
UNIT is an engineering/empirical paper. Semantic anchors p_c^(T_si) (Eqs. 5–7) and structural anchors a_c (Eqs. 9–11) are defined as (decayed, uncertainty-weighted) averages of the frozen LLM embeddings and hand-crafted topological descriptors of the labeled nodes of the current task; the classifier weights are then the convex combination w_c = β p_c + (1-β) a_c (Eq. 12). Classification accuracy on subsequent tasks is measured against external baselines (Table 1). Nothing is predicted that is algebraically forced by the definition of the anchors, no parameter is fitted to a target quantity and then re-reported as a prediction, and no uniqueness theorem or ansatz is imported via self-citation to force the design. Hyper-parameters λ, γ are fixed once and subjected to a sensitivity sweep (Fig. 5); the single LoRA fine-tune on T_s1 is an explicit modeling choice whose validity is tested by the multi-task accuracy numbers, not assumed by construction. The derivation chain is therefore self-contained and non-circular.
Axiom & Free-Parameter Ledger
free parameters (3)
- λ (semantic decay) =
0.7
- γ (structural decay) =
0.6
- β (semantic/structural mix)
axioms (4)
- domain assumption Node-level class-incremental learning on text-attributed graphs with disjoint class sets across tasks is the correct formalization of graph continual learning.
- ad hoc to paper A single LoRA fine-tune on the first task’s instruction prompts is sufficient to adapt a pre-trained LLM for all future tasks.
- domain assumption Prediction entropy of the LLM is a reliable proxy for sample uncertainty that can be used as a confidence weight.
- ad hoc to paper Degree, average neighbor degree and clustering coefficient are sufficient structural descriptors for class-level confluence.
invented entities (2)
-
uncertain-aware semantic anchor
no independent evidence
-
structural confluence modeling / structural anchor
no independent evidence
read the original abstract
In real-world multimodal web scenarios, graph-structured data often arrives in a streaming manner, making graph continual learning a crucial paradigm for continuously modeling such evolving structures. However, existing graph continual learning methods still face two fundamental challenges. 1) semantic-structural separation, where the graph-based methods excel at modeling topological relationships but neglect deep semantics. 2) imbalanced knowledge transfer, where existing models fail to effectively leverage general knowledge gained from early tasks to benefit subsequent new tasks. To address above issues, we propose a novel framework, \textbf{UN}leash Large Language Models PotentIal for Graph ConTinual Learning (UNIT). By fine-tuning large language model only on the first task, we bridge the distributional gap between the pre-trained LLM corpus and the target task dataset to enhance the adaptability of LLMs for graph-structured tasks. Meanwhile, we propose an uncertain-aware anchor generation mechanism to effectively preserve representative knowledge across tasks, avoiding the neglect of universal knowledge learned from previous tasks. Additionally, we introduce structural confluence modeling to explicitly integrates graph topology information into semantic information, enhancing the collaborative capabilities between semantic understanding and structural modeling. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance in the graph continual learning task.
Figures
Reference graph
Works this paper leans on
-
[1]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in Neural Information Processing Systems33 (2020), 1877–1901
2020
-
[2]
Qiang Chen, Zhongze Wu, Xiu Su, Xi Lin, Zhe Qu, Shan You, Shuo Yang, and Chang Xu. 2025. Stable Fair Graph Representation Learning with Lipschitz Constraint. InProceedings of the International Conference on Machine Learning
2025
-
[3]
Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. 2024. Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170(2024)
Pith/arXiv arXiv 2024
-
[4]
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et al . 2024. Exploring the potential of large language models (llms) in learning on graphs.ACM SIGKDD Explorations Newsletter25, 2 (2024), 42–61
2024
-
[5]
Ziyang Cheng, Zhixun Li, Yuhan Li, Yixin Song, Kangyi Zhao, Dawei Cheng, Jia Li, and Jeffrey Xu Yu. 2025. Can LLMs Alleviate Catastrophic Forgetting in Graph Continual Learning? A Systematic Study.arXiv preprint arXiv:2505.18697(2025)
arXiv 2025
-
[6]
Seungyoon Choi, Wonjoong Kim, Sungwon Kim, Yeonjun In, Sein Kim, and Chanyoung Park. 2024. DSLR: diversity enhancement and structure learning for rehearsal-based graph continual learning. InProceedings of the ACM Web Conference 2024. 733–744
2024
-
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: human language technologies, volume 1 (long and short papers). 4171–4186
2019
-
[8]
Xiaolin Han, Yikun Zhang, Chenhao Ma, Lingyun Song, Reynold Cheng, and Xuequn Shang. 2025. TempASD: Temporal Anomalous Subgraph Discovery in Large-Scale Dynamic Financial Networks. InProceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 826–837
2025
-
[9]
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. 2023. Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning.arXiv preprint arXiv:2305.19523 (2023)
Pith/arXiv arXiv 2023
-
[10]
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.. InInternational Conference on Learning Representations. 3
2022
-
[11]
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs.Advances in Neural Information Processing Systems 33 (2020), 22118–22133
2020
-
[12]
Huang, Wang Tairan, Li Yili, He Qiutong, Gao Changlong, and Jianliang. 2025. Can LLMs Find Fraudsters? Multi-level LLM Enhanced Graph Fraud Detection. InProceedings of the ACM International Conference on Multimedia
2025
-
[13]
Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. 2019. Compacting, picking and growing for unforgetting continual learning.Advances in Neural Information Processing Systems32 (2019)
2019
-
[14]
Pengyue Jia, Jingtong Gao, Yejing Wang, Yuhao Wang, Xiaopeng Li, Qidong Liu, Yichao Wang, Bo Chen, Huifeng Guo, and Ruiming Tang. 2025. Joint Modeling in Deep Recommender Systems. InCompanion Proceedings of the ACM on Web Conference. 17–20
2025
-
[15]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. InInternational Conference on Learning Representations. 1031–1049
2017
-
[16]
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks.Proceedings of the National Academy of Sciences114, 13 (2017), 3521– 3526
2017
-
[17]
Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor W Chan, and Jia Li. 2024. Glbench: A comprehensive benchmark for graph with large language models.Advances in Neural Information Processing Systems 37 (2024), 42349–42368
2024
-
[18]
Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting.IEEE Transac- tions on Pattern Analysis and Machine Intelligence40, 12 (2017), 2935–2947
2017
-
[19]
Yan-Shuo Liang and Wu-Jun Li. 2024. Inflora: Interference-free low-rank adapta- tion for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 23638–23647
2024
-
[20]
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. 2023. One for all: Towards training one graph model for all classification tasks.arXiv preprint arXiv:2310.00149(2023)
Pith/arXiv arXiv 2023
-
[21]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 (2019)
Pith/arXiv arXiv 2019
-
[22]
Zheyuan Liu, Xiaoxin He, Yijun Tian, and Nitesh V Chawla. 2024. Can we soft prompt llms for graph learning tasks?. InCompanion Proceedings of the ACM Web Conference. 481–484
2024
-
[23]
2021.Deep learning on graphs
Yao Ma and Jiliang Tang. 2021.Deep learning on graphs. Cambridge University Press
2021
-
[24]
Chaoxi Niu, Guansong Pang, Ling Chen, and Bing Liu. 2024. Replay-and-forget- free graph class-incremental learning: A task profiling and prompting approach. Advances in Neural Information Processing Systems37 (2024), 87978–88002
2024
-
[25]
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning mul- tiple visual domains with residual adapters.Advances in Neural Information Processing Systems30 (2017)
2017
-
[26]
Yanchao Tan, Hang Lv, Pengxiang Zhan, Shiping Wang, and Carl Yang. 2025. Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining.IEEE Transactions on Pattern Analysis and Machine Intelligence(2025)
2025
-
[27]
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024. Graphgpt: Graph instruction tuning for large language models. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 491–500
2024
-
[28]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)
Pith/arXiv arXiv 2023
-
[29]
Duo Wang, Yuan Zuo, Fengzhi Li, and Junjie Wu. 2024. Llms as zero-shot graph learners: Alignment of gnn representations with llm token embeddings.Advances in Neural Information Processing Systems37 (2024), 5950–5973
2024
-
[30]
Qi-Wei Wang, Da-Wei Zhou, Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye
-
[31]
Advances in Neural Information Processing Systems36 (2023), 15060–15076
Few-shot class-incremental learning via training-free prototype calibration. Advances in Neural Information Processing Systems36 (2023), 15060–15076
2023
-
[32]
Xixi Wu, Yifei Shen, Fangzhou Ge, Caihua Shan, Yizhu Jiao, Xiangguo Sun, and Hong Cheng. 2025. A comprehensive analysis on llm-based node classification algorithms.arXiv preprint arXiv:2502.00829(2025)
Pith/arXiv arXiv 2025
-
[33]
Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. 2023. A comprehensive study on text-attributed graphs: Benchmarking and rethinking.Advances in Neural Information Processing Systems36 (2023), 17238–17264
2023
-
[34]
Hao Yan, Chaozhuo Li, Jun Yin, Zhigang Yu, Weihao Han, Mingzheng Li, Zhengxin Zeng, Hao Sun, and Senzhang Wang. 2025. When Graph Meets Multi- modal: Benchmarking and Meditating on Multimodal Attributed Graph Learning. InProceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 5842–5853
2025
-
[35]
Mengmei Zhang, Mingwei Sun, Peng Wang, Shen Fan, Yanhu Mo, Xiaoxiao Xu, Hong Liu, Cheng Yang, and Chuan Shi. 2024. Graphtranslator: Aligning graph model to large language model for open-ended tasks. InProceedings of the ACM Web Conference 2024. 1003–1014
2024
-
[36]
Peiyan Zhang, Yuchen Yan, Chaozhuo Li, Senzhang Wang, Xing Xie, Guojie Song, and Sunghun Kim. 2023. Continual learning on dynamic graphs via parameter isolation. InProceedings of the international ACM SIGIR conference on research and development in information retrieval. 601–611
2023
-
[37]
Taiyan Zhang, Renchi Yang, Yurui Lai, Mingyu Yan, Xiaochun Ye, and Dongrui Fan. 2025. Leveraging large language models for effective label-free node classifi- cation in text-attributed graphs. InProceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 698–708
2025
-
[38]
Xikun Zhang, Dongjin Song, and Dacheng Tao. 2022. Hierarchical prototype networks for continual graph representation learning.IEEE Transactions on Pattern Analysis and Machine Intelligence45, 4 (2022), 4622–4636
2022
-
[39]
Xikun Zhang, Dongjin Song, and Dacheng Tao. 2022. Sparsified subgraph memory for continual graph representation learning. In2022 IEEE International Conference on Data Mining. 1335–1340
2022
-
[40]
Xikun Zhang, Dongjin Song, and Dacheng Tao. 2023. Ricci curvature-based graph sparsification for continual graph representation learning.IEEE Transactions on Neural Networks and Learning Systems(2023)
2023
-
[41]
Da-Wei Zhou, Zi-Wen Cai, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. 2025. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need.International Journal of Computer Vision133, 3 (2025), 1012–1032
2025
-
[42]
Fan Zhou and Chengtai Cao. 2021. Overcoming catastrophic forgetting in graph neural networks with experience replay. InProceedings of the AAAI conference on artificial intelligence, Vol. 35. 4714–4722
2021
-
[43]
Hao Zhu, Yifei Zhang, Junhao Dong, and Piotr Koniusz. 2025. BiLoRA: Almost- Orthogonal Parameter Spaces for Continual Learning. InProceedings of the Com- puter Vision and Pattern Recognition Conference. 25613–25622
2025
-
[44]
Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang. 2024. Efficient tun- ing and inference for large language models on textual graphs.arXiv preprint arXiv:2401.15569(2024)
Pith/arXiv arXiv 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.