REVIEW 4 major objections 5 minor 37 references
FedOGL claims catastrophic forgetting in federated open-world multimodal graph learning is a semantic-structural memory failure, and its three-part framework cuts forgetting by 42.67% while preserving or improving downstream accuracy.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 03:41 UTC pith:MYO26OO7
load-bearing objection A solid empirical framework for a genuinely new federated open-world graph learning setting, with real gains over baselines, but reproducibility gaps and an unproven subspace assumption keep it from being a clean accept. the 4 major comments →
FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
FedOGL's core discovery is that catastrophic forgetting in federated open-world multimodal graphs is a semantic-structural memory failure with three identifiable causes, and that each cause has a corresponding remedy that can be composed into one framework. Modality-semantic overwriting is countered by anchoring the current model to a frozen task-start teacher and replaying small historical batches. Topology-induced structural erosion is countered by maintaining a global structural basis—built by aggregating per-client residual projectors—and projecting structural gradients orthogonal to that basis, so updates that would rewrite historical propagation directions are suppressed. Federated mem
What carries the argument
The load-bearing object is the global structure basis U_t: clients extract residual structural directions from centered embeddings (subtracting what the previous basis already explains), send projector matrices U_k,t U_k,t^T to the server, which sums them to form S_t and takes the top eigenvectors as the new global basis. In the next stage, structural gradients are multiplied by (I - U_t U_t^T), the orthogonal projection, so gradient components that would move parameters along protected historical directions are zeroed. The second key object is the server-side prototype memory bank: local semantic-structural descriptors per class are uploaded, matched, and merged with support-weighted averag
Load-bearing premise
The whole result rests on the structural subspace U_t capturing precisely the graph-propagation directions whose preservation keeps old classes alive; if that basis is incomplete—because local embeddings are modality-deficient or the current stage's data does not expose historical directions—the shielding either misses what needs protecting or blocks what needs learning.
What would settle it
Construct a task stream where historical classes are supported by propagation directions that do not appear in the current stage's top residual eigenvectors (e.g., the old class is only reachable through a modality that the current clients lack), then train FedOGL with replay and distillation intact; if FM still degrades as much as a no-shielding baseline, the structure basis is not carrying the memory it claims to protect.
If this is right
- Federated graph models can keep learning new classes indefinitely without a globally synchronized class catalog, as long as each client retains a bounded replay buffer and the server maintains the structure basis and prototype bank.
- The structural-gradient projection operates on gradient geometry rather than model architecture, so it should transfer to any graph encoder with a structural representation pathway.
- Unknown rejection improves as a side effect of retention: because old-class boundaries are preserved, the maximum-softmax confidence rule misclassifies fewer OOD samples as known.
- Server-side prototype consolidation provides a privacy-preserving way to share category knowledge across asynchronous clients, since only class indices and compact descriptors cross the network.
- FedOGL retains early-task knowledge throughout the incremental trajectory, not just at the final stage, as shown by the task-wise retention analysis on Grocery.
Where Pith is reading between the lines
- The projector-summing aggregation in Eq. (8) is rotation-insensitive: summing projectors rather than vectors means the global basis does not depend on arbitrary rotations of local coordinate systems, which could make the shielding robust to client drift in ways that direct averaging would not—but the paper does not test this explicitly.
- Because the structure basis is built from embeddings at the current stage, it can only protect directions that are still visible in the representation; classes whose structural signature has already been erased before the basis is computed would be missed. A testable extension is to estimate U_t from a mixture of current and replayed embeddings.
- The prototype-graph refinement (Eqs. 13–14) is essentially a tiny graph continual-learning problem on the server; hooking it into class-balanced sampling or more advanced OOD detection could further reduce FPR95 without extra client cost.
- The claim that forgetting is a semantic-structural memory failure suggests a diagnostic: monitor D_sem, D_str, and D_fed during training and check whether FedOGL's gains track all three; if only D_str drops while D_sem stays high, the three-mechanism decomposition may be incomplete.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper formulates federated open-world multimodal graph learning under class-incremental evolution, arguing that catastrophic forgetting in this setting arises from three mechanisms: modality-semantic overwriting, topology-induced structural erosion, and federated memory fragmentation. It proposes FedOGL, which combines task-anchored semantic retention (replay + distillation), structure-basis memory shielding (global structural subspace projection), and prototype-guided memory consolidation (server-side prototype matching/merging and prototype-graph refinement). Experiments on six datasets across node classification and cross-modal retrieval report that FedOGL outperforms FL/FGL/open-world baselines, with headline claims of a 42.67% FM reduction and a 28.31% FPR95 reduction.
Significance. If the results hold, this is a useful new benchmark and a practically strong framework for a realistic decentralized incremental setting. The paper introduces a clear three-mechanism diagnostic decomposition, reports five-seed means and standard deviations, includes ablations and sensitivity analysis, and proposes a distinctive structural-subspace protection idea. The strength of the empirical claims is, however, reduced by two issues: the headline improvement percentages are not computed against the best-performing baseline in Table 1, and the central structural-basis assumption is not directly validated. These concerns currently prevent acceptance.
major comments (4)
- [Abstract; Table 1; Section 5.1] The reported '42.67% FM reduction' is not the improvement over the best-performing baseline. Averaging FM over the six datasets gives FedOGL 5.24% and FedIIH 9.14%, i.e., (9.14-5.24)/9.14 = 42.67%. But FedIIH is not the strongest baseline: on RedditS, POWER achieves FM 3.49% versus FedOGL's 5.64%, and on KU, CLIPN has a lower FM (12.89%) than FedIIH (18.76%). Similarly, '28.31% FPR95 reduction' equals the reduction from POWER's average FPR95 (45.89% to 32.90%), but on RedditS, KU, and QB the best baseline is TopoOOD or CLIPN, not POWER. Please recompute the headline statistics against the true best baseline per dataset or explicitly define the comparison basis.
- [Sec. 4.2, Eqs. (7)-(9)] The structure-basis shielding assumes that U_t captures the graph-propagation directions whose preservation prevents old-class forgetting. This is not established. Eq. (7) builds residual bases from current-stage embeddings after removing U_{t-1}; historical directions not already in U_{t-1} and not re-expressible in current/replayed embeddings are irrecoverable, and the fixed rank r in Eq. (8) can drop old eigenvectors. Eq. (9) suppresses updates aligned with the leading variance directions of the current embedding covariance, but there is no argument that high-variance directions are exactly the memory-critical directions. The ablation in Table 2 shows that removing the projection hurts, but a generic gradient regularizer could produce the same pattern. Please add a direct test: measure the overlap between the protected basis U_t and the historical task bases U_i used in the D_str diag
- [Sec. 3, Eqs. (2)-(4)] The three-mechanism decomposition is supported only by descriptive drift metrics that are not tied to downstream task performance. No evidence is given that reducing D_sem, D_str, or D_fed is causally responsible for the FM improvements, rather than being a correlate of generic regularization. Since the paper's conceptual contribution is the three-mechanism interpretation, this connection needs support, e.g., a correlation analysis between each drift metric and FM across methods, or intervention experiments that isolate each mechanism.
- [Experimental Setup; Reproducibility] Essential implementation details are deferred to 'Section 2 of the supporting material' and 'Section 4 of the supporting material', but no supplementary material is included with the manuscript. Critical choices — the OOD confidence threshold delta_t in Eq. (6), the prototype matching threshold eta and memory slots S in Eq. (12), the basis rank r, loss weights, and replay memory size M — are not specified in the main text. Without these details, the five-seed results cannot be audited or reproduced. Please include all such details in the submission or state explicitly that the supplement is part of the reviewed version.
minor comments (5)
- [Abstract; Introduction] The abstract contains an unrelated passage ('As cammermay splice an authentic interview clip...') and a duplicated sentence ('In this setting, clients must learn emerging classes...'). The Introduction also contains a blank reference 'As illustrated in Sec. .' These text corruptions must be fixed before publication.
- [Eq. (7)] The class-balanced sample weights W_{k,t} are not defined. Specify their dimension and computation.
- [Eq. (6)] The OOD confidence threshold delta_t is a key free parameter, but its selection protocol is not described. The sensitivity analysis in Fig. 7 covers only basis rank and distillation weight, not delta_t, eta, or memory size.
- [Figures 5 and 7] Several figure labels appear garbled or use nonstandard character encodings, making the axis and legend text difficult to read. Please regenerate the figures with standard fonts.
- [References] Several references are dated 2026 and marked 'arXiv preprint'. If these are not yet publicly available or accepted, please clarify their status or remove them.
Circularity Check
No significant circularity: the headline forgetting reduction is an empirical comparison against external baselines on standard metrics, not a prediction forced by the paper's own definitions or fitted parameters.
full rationale
FedOGL's central claim is an empirical result: FM, FPR95, and AM are reported against three categories of external baselines (FL, FGL, open-world) on standard continual-learning protocols. The three forgetting mechanisms (modality-semantic overwriting, topology-induced structural erosion, federated memory fragmentation) are introduced through paper-defined drift diagnostics (Eqs. 2–4), and the three modules are designed to reduce those drift measures; this is an internally consistent design narrative rather than a circular prediction, because the headline 42.67% FM reduction is not computed from those diagnostics but from task-wise performance drops relative to baselines. The structure-basis shielding (Eqs. 7–9) rests on an unproven assumption that top eigenvectors of current structural embeddings span historical graph-propagation directions; that is a robustness/correctness risk, not a circularity, since the claimed FM advantage is not equivalent to the projection operation by construction. Self-citations to STAGE, PRISM, and MM-OpenFGL appear in related work and in dataset/baseline selection, but they are not used as load-bearing evidence for FedOGL's effectiveness, and no uniqueness theorem or ansatz is imported from the authors' prior work. No fitted parameter is renamed as a prediction. The skeptical concern about U_t omitting historical directions is a plausible limitation, but the paper does not define its success metric in terms of U_t, so the derivation chain is not circular.
Axiom & Free-Parameter Ledger
free parameters (5)
- OOD confidence threshold delta_t =
not reported
- Structural basis ranks r and r_l (TopEig_r) =
not reported
- Prototype matching threshold eta and memory bank slots S =
not reported
- Loss weights lambda_rep, lambda_dis, lambda_lab, lambda_tr, temperature tau, schedule gamma_t =
not reported
- Replay memory size M =
not reported
axioms (5)
- domain assumption Clients can retain a bounded local replay buffer of raw historical labeled nodes.
- domain assumption Each client observes only a subset of new classes, and unknown classes never appear during training.
- domain assumption Louvain-partitioned five-client subgraphs are a representative surrogate for real non-IID federated multimodal graphs.
- domain assumption Maximum softmax probability is a sufficiently calibrated OOD score and delta_t can be set to hold 95% TPR.
- standard math Projector summation and residual projection preserve the subspace geometry needed for structural memory shielding.
invented entities (1)
-
Prototype graph G_P = (P_t, E_P) with same-class and KNN edges
no independent evidence
read the original abstract
Federated graph learning enables collaborative training over decentralized graph data without sharing raw graph information. As such risks evolve, clients must learn emerging classes from private multimodal graph streams, retain historical categories, and reject samples outside the known class space. In this setting, clients must learn emerging classes from private multimodal graph streams while preserving historical categories and rejecting samples outside the current known class space. The core challenge is catastrophic forgetting, which in federated multimodal graphs is not merely a classifier-level failure: old knowledge can be erased through modality-semantic overwriting, topology-induced structural erosion, and federated memory fragmentation. To address this challenge, we propose \textbf{FedOGL}, a semantic-structural memory preservation framework. On the client side, FedOGL preserves historical decision behavior through replay and task-start distillation, while protecting graph-propagation memory via projection onto a globally shared structure basis. On the server side, FedOGL maintains and transfers compact category prototypes to facilitate cross-client knowledge sharing without exposing raw graph data. Extensive experiments demonstrate that, compared with the best-performing baselines, FedOGL reduces performance degradation caused by catastrophic forgetting by \textbf{42.67\%}, while maintaining or improving performance on downstream tasks.
Figures
Reference graph
Works this paper leans on
-
[1]
Artificial intelligence and statistics , pages=
Communication-efficient learning of deep networks from decentralized data , author=. Artificial intelligence and statistics , pages=. 2017 , organization=
2017
-
[2]
Proceedings of Machine learning and systems , volume=
Federated optimization in heterogeneous networks , author=. Proceedings of Machine learning and systems , volume=
-
[3]
ICLR 2021 Workshop on Distributed and Private Machine Learning (DPML) , year=
Fedgraphnn: A federated learning benchmark system for graph neural networks , author=. ICLR 2021 Workshop on Distributed and Private Machine Learning (DPML) , year=
2021
-
[4]
arXiv preprint arXiv:2408.16288 , year=
Openfgl: A comprehensive benchmark for federated graph learning , author=. arXiv preprint arXiv:2408.16288 , year=
-
[5]
arXiv preprint arXiv:2401.11755 , year=
Fedgta: Topology-aware averaging for federated graph learning , author=. arXiv preprint arXiv:2401.11755 , year=
-
[6]
Advances in Neural Information Processing Systems , volume=
FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference , author=. Advances in Neural Information Processing Systems , volume=
-
[7]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Modeling Inter-Intra Heterogeneity for Graph Federated Learning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=. 2025 , doi=
2025
-
[8]
arXiv preprint arXiv:2605.11919 , year=
STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning , author=. arXiv preprint arXiv:2605.11919 , year=
-
[9]
arXiv preprint arXiv:2606.09301 , year=
PRISM: Topology-Aware Cross-Modal Imputation for Modality-Deficient Federated Graph Learning , author=. arXiv preprint arXiv:2606.09301 , year=
-
[10]
Chen, Zekai and Lu, Haodong and Li, Xunkai and Sun, Henan and Li, Jia and Qin, Hongchao and Li, Rong-Hua and Wang, Guoren , booktitle =
-
[11]
Proceedings of the national academy of sciences , volume=
Overcoming catastrophic forgetting in neural networks , author=. Proceedings of the national academy of sciences , volume=. 2017 , publisher=
2017
-
[12]
IEEE transactions on pattern analysis and machine intelligence , volume=
Learning without forgetting , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2018 , publisher=
2018
-
[13]
Advances in Neural Information Processing Systems , volume=
CGLB: Benchmark Tasks for Continual Graph Learning , author=. Advances in Neural Information Processing Systems , volume=
-
[14]
Proceedings of the 40th International Conference on Machine Learning , series=
Towards Robust Graph Incremental Learning on Evolving Graphs , author=. Proceedings of the 40th International Conference on Machine Learning , series=
-
[15]
Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=
icarl: Incremental classifier and representation learning , author=. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition , pages=
-
[16]
arXiv preprint arXiv:2108.00158 , year=
Multiplex graph networks for multimodal brain network analysis , author=. arXiv preprint arXiv:2108.00158 , year=
-
[17]
Proceedings of the 38th International Conference on Machine Learning , series=
Learning Transferable Visual Models From Natural Language Supervision , author=. Proceedings of the 38th International Conference on Machine Learning , series=
-
[18]
International Conference on Learning Representations , year=
Adam: A Method for Stochastic Optimization , author=. International Conference on Learning Representations , year=
-
[19]
arXiv preprint arXiv:1807.03748 , year=
Representation Learning with Contrastive Predictive Coding , author=. arXiv preprint arXiv:1807.03748 , year=
-
[20]
International Conference on Learning Representations , year=
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks , author=. International Conference on Learning Representations , year=
-
[21]
2023 , doi =
Zhang, Jie and Chen, Chen and Zhuang, Weiming and Lyu, Lingjuan , booktitle =. 2023 , doi =
2023
-
[22]
Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V
Federated continual graph learning , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=
-
[23]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[24]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Towards open set deep networks , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[25]
Advances in neural information processing systems , volume=
Energy-based out-of-distribution detection , author=. Advances in neural information processing systems , volume=
-
[26]
Forty-first International Conference on Machine Learning , year=
Graph out-of-distribution detection goes neighborhood shaping , author=. Forty-first International Conference on Machine Learning , year=
-
[27]
Advances in Neural Information Processing Systems , volume=
Revisiting score propagation in graph out-of-distribution detection , author=. Advances in Neural Information Processing Systems , volume=
-
[28]
arXiv preprint arXiv:2505.13989 , year=
When LLMs meet open-world graph learning: a new perspective for unlabeled data uncertainty , author=. arXiv preprint arXiv:2505.13989 , year=
-
[29]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Clipn for zero-shot ood detection: Teaching clip to say no , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[30]
arXiv preprint arXiv:2408.11237 , year=
Out-of-Distribution Detection with Attention Head Masking for Multimodal Document Classification , author=. arXiv preprint arXiv:2408.11237 , year=
-
[31]
Advances in neural information processing systems , volume=
Delving into out-of-distribution detection with vision-language representations , author=. Advances in neural information processing systems , volume=
-
[32]
arXiv preprint arXiv:2601.22416 , year=
MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learning , author=. arXiv preprint arXiv:2601.22416 , year=
-
[33]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Mosaic of modalities: A comprehensive benchmark for multimodal graph learning , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[34]
Justifying recommendations using distantly-labeled reviews and fine-grained aspects , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=
2019
-
[35]
2023 , publisher=
Wang, Xiaodan and Wang, Chengyu and Li, Lei and Li, Zhixu and Chen, Ben and Jin, Linbo and Huang, Jun and Xiao, Yanghua and Gao, Ming , booktitle=. 2023 , publisher=
2023
-
[36]
2019 , publisher=
Wei, Yinwei and Wang, Xiang and Nie, Liqiang and He, Xiangnan and Hong, Richang and Chua, Tat-Seng , booktitle=. 2019 , publisher=
2019
-
[37]
Journal of statistical mechanics: theory and experiment , volume=
Fast unfolding of communities in large networks , author=. Journal of statistical mechanics: theory and experiment , volume=
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.