{"id":"2122cfc1-e197-4c8e-8cc6-d0d6f5760ee2","arxiv_id":"2608.10016","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Sheaf-FRL learns per-edge orthogonal or Stiefel maps to align heterogeneous agent latent spaces through a sheaf-Laplacian gluing penalty evaluated on shared pilots, with decentralized convergence guarantees.","lead":"This paper introduces a federated learning framework where each agent keeps its own latent space and alignment with neighbors is learned via geometric transport maps, regularized by a sheaf Laplacian penalty on a small shared pilot set. The authors derive a decentralized alternating algorithm with convergence guarantees and report improved classification accuracy and compression robustness on MNIST semantic-communication experiments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 does not implement the gradient step analyzed in Theorems E.2-E.4: Eq. (19) omits neighbor contributions to the gradient of R_A, so the descent and stationarity proofs do not apply to Sheaf-FRL as written.","rationale":"The paper's central theoretical claim is that Sheaf-FRL converges to first-order stationary points of Problem (SFRL). That claim rests on Theorems E.2-E.4, whose proofs use a descent lemma applied to gradient steps of the full objective J. The algorithm described in Section 5, however, replaces the full gradient by the local penalty gradient R_{A|i}, omitting the contributions of neighboring agents' penalty terms. The omission is concrete and notational: for each edge the same residual appears in two local penalties with different weights lambda_i and lambda_j, so the derivative of the sum is not the sum of derivatives of one local term. This is an internal inconsistency between Algorithm 1 and the analyzed update, not merely an assumption about the setting. Consequently the convergence guarantees, a headline contribution, are not established for the implemented method. The empirical results may still be informative, and the Procrustes step is correctly derived, but the theoretical core of the paper is currently invalid. The reader's weakest-assumption analysis focused on shared pilots and whitening; those are legitimate applicability concerns, but they are not the most load-bearing issue. The proof mismatch is more severe because it undermines the formal guarantee that distinguishes the method. A revision could fix the algorithm by including the neighbor terms (which are computable with already-exchanged pilot features plus known d_j), but as submitted the central convergence claim does not follow from the presented analysis.","tokens_in":29521,"tokens_out":15252,"duration_ms":153692,"concrete_test":"Construct a two-agent instance with d_1=2, d_2=1, K=1 and simple linear encoders f_i(x)=W_i x. After one Procrustes step, compute the full gradient of R_A with respect to W_1 by central differences and compare it with Eq. (19)'s r_1; they should differ by the missing (lambda_2/K)(grad f_1)^T(A_1 - V A_2) term whenever the residual is nonzero. Then run update (21) with eta = 1/L and check whether J(theta^1,V^1) <= J(theta^0,V^0) as claimed in Theorem E.2(i); a single violation, or a numerical equality of the two gradients, settles whether the proof applies to Algorithm 1.","verdict_should_be":"REJECT","load_bearing_attack":"The convergence theorems (E.2-E.4) analyze the compact update theta^{t+1}=theta^t - eta*grad_theta J(theta^t,V^t), with J=sum_i L_i + R_A and R_A=sum_i R_{A|i} (Eqs. 15, 27-29). But Algorithm 1 and Eq. (21) update node i using only the local penalty gradient r_i = [grad_{phi_i} R_{A|i}; 0]. These are not equal. For any edge (i,j) with d_i > d_j, the same residual ||A_i - V_{ij} A_j||^2 enters both R_{A|i} with coefficient lambda_i/(2K) and R_{A|j} with coefficient lambda_j/(2K). The full derivative of R_A with respect to phi_i therefore contains an extra term (lambda_j/K)(grad f_i)^T (A_i - V_{ij} A_j) that Eq. (19) drops; similarly when i is the lower-dimensional tail. Hence Sheaf-FRL is not performing gradient descent on the stated objective J. The descent lemma in Theorem E.2, the O(1/T) rate, and the stochastic O(1/sqrt(T)) rate all rely on the exact gradient update, so the claimed convergence to first-order stationary points of Problem (SFRL) is unsupported for the implemented algorithm. The V-block update is correctly closed-form, but the theta-block update is not a gradient step on J.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Sheaf-based Federated Representation Learning (SFRL), a framework in which each agent learns its own latent space and inter-agent consistency is encouraged through learnable orthogonal/Stiefel restriction maps and a quadratic gluing penalty built on a sheaf Laplacian. The penalty is evaluated on a small shared set of pilot samples. The authors propose Sheaf-FRL, a decentralized alternating algorithm with closed-form Procrustes updates for the restriction maps and gradient updates for local models, and prove convergence to first-order stationary points in deterministic and stochastic settings (Theorems E.2-E.4). Experiments on MNIST in a semantic-communication setup compare private and communication accuracy against ComFed, Sheaf-FMTL, FedProto, FedMuscle, and a non-cooperative baseline, and report robustness to latent-space compression.","tokens_in":29796,"tokens_out":14872,"duration_ms":142744,"significance":"If the convergence claim is valid, SFRL is a useful generalization of federated representation learning: it removes the shared-latent-space assumption, handles heterogeneous dimensions via Stiefel maps, and its pilot-based gluing penalty keeps communication cost O(d_i K). The closed-form Procrustes updates are standard and correctly derived, the convergence proofs are coherent under the stated assumptions, and the paper provides pseudocode, appendix derivations, and a public code link. The experimental design has a clear application setting. However, the convergence theorem currently does not apply to the update implemented in Algorithm 1 (the gradient of R_A is incomplete), and the empirical comparison lacks error bars and uses a memory-constrained hyperparameter for Sheaf-FMTL. These issues are correctable but must be addressed before the central claims can be accepted.","major_comments":[{"comment":"The update implemented in Algorithm 1 (line 19) is theta_i^{t+1} = theta_i^t - eta (nabla_{theta_i} L_i(theta_i^t) + r_i^t) with r_i = [nabla_{phi_i} R_{A|i}; 0]. The convergence theorems analyze the update theta^{t+1} = theta^t - eta nabla_theta J(theta^t, V^t) with J = sum_i L_i + R_A and R_A = sum_i R_{A|i}. These two updates are not equal. For each edge (i,j), the residual ||A_i - V_ij A_j||^2 contributes to both R_{A|i} (with weight lambda_i/(2K)) and R_{A|j} (with weight lambda_j/(2K)); the full gradient of R_A with respect to phi_i therefore contains, for each neighbor j, the additional term (lambda_j/K) sum_{k in A} (nabla_{phi_i} f(x^k_i))^T (f(x^k_i) - V_ij f(x^k_j)), which Eq. (19) omits. Consequently, the descent lemma in Theorem E.2 and the stochastic rates in Theorems E.3-E.4 do not apply to Algorithm 1 as written. The manuscript should either correct the algorithm to use the full gradient of R_A (the missing terms are locally computable from neighbor pilot features and lambda_j) or re-derive the convergence analysis for the actual update.","section":"Section 5.2, Eqs. (19)-(21); Appendix E, Eqs. (27)-(29)"},{"comment":"Sheaf-FMTL is evaluated at gamma = 10^{-3}, a value selected because larger gamma values exceed available memory, not because it is a good operating point for the method. The paper states that the memory ceiling binds 'well before any accuracy trade-off becomes relevant.' The reported underperformance of Sheaf-FMTL in communication accuracy may therefore reflect the memory-imposed compression of its restriction maps rather than a property of the method. Please provide results for Sheaf-FMTL under a fair memory regime (e.g., reduced architectures, gradient checkpointing, or model-parallel storage) or demonstrate that the gap persists when gamma is chosen by validation without memory constraints.","section":"Section 6 and Appendix G, Figs. 2 and 4"},{"comment":"All accuracy curves are point estimates with no error bars or multiple seeds. In Fig. 2 the private-accuracy differences between Sheaf-FRL and the best baseline are typically below one percentage point; without variance information the claim of consistent improvement is not statistically supported. Please report means and standard deviations over at least three to five independent runs (with different data partitions and initializations) in both the private and communication accuracy plots, and state the number of seeds explicitly.","section":"Section 6, Figs. 2-3"},{"comment":"The gluing penalty R_A, the Procrustes updates, and the convergence analysis all require a globally shared set of pilot indices A with known correspondence across agents. The paper only states this 'for the sake of exposition.' Because the method cannot be applied at all when no shared calibration data exist, this is a substantive assumption of Problem (SFRL) and should be stated as such in the main text, together with a discussion of failure modes when the pilot correspondence is imperfect.","section":"Section 4, Eq. (14); Appendix B"}],"minor_comments":[{"comment":"The hyperparameter lambda selected by the grid search is never reported for any method or configuration; please report the chosen values.","section":"Section 6"},{"comment":"The result is lim inf of the gradient norm, not convergence of the gradient sequence; the conclusions should refer to 'lim inf stationarity' in the stochastic case.","section":"Appendix E, Theorem E.4"},{"comment":"The symbol V_ij is reused after the reparameterization V_ij = O_ji^T V_ij with a different meaning than in Eq. (5); please introduce a new symbol or explicitly state the rename to avoid confusion.","section":"Section 3, Eq. (10)"},{"comment":"The caption says 'System RAM (31 GB)' while the text discusses GPU RAM; please clarify which memory resource is the binding constraint.","section":"Appendix G, Fig. 4"},{"comment":"The sentence 'both incoming and outgoing contributions admit a unified expression' applies to R_{A|i} only; please make this scope explicit in the main text.","section":"Section 5.2, after Eq. (19)"}],"recommendation":"major_revision","confidential_remarks":"The main technical issue (the gradient mismatch between Algorithm 1 and the convergence analysis) is serious but fixable; it blocks acceptance in the current form. The experimental section also needs the described revisions: error bars or multiple seeds, and a fairer operating point for Sheaf-FMTL. I see no signs of misattribution or duplicate submission; the paper is within scope for a machine-learning journal despite the application framing. If the authors fix the gradient update and rerun the experiments with seeds, the paper could be suitable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the core idea is genuinely good. Treating each agent's latent space as a node stalk of a learnable network sheaf, with orthogonal/Stiefel transport maps aligned on a small pilot set, is a clean and unifying framing. The Procrustes updates are closed-form, the decentralized scheme is simple, and the paper positions itself well against prior work. But there is a load-bearing bug: the convergence analysis in Appendix E analyzes the compact update theta^{t+1} = theta^t - eta * grad_theta J(theta^t, V^t), where J = sum L_i + R_A and R_A = sum_i R_{A|i}. Algorithm 1 instead updates each agent using the gradient of its local term L_i + R_{A|i}. These are not the same. For an edge (i,j), the residual ||A_i - V_ij A_j||^2 appears in both R_{A|i} and R_{A|j}, so the true gradient of R_A with respect to theta_i contains a contribution from R_{A|j} that Eq. (19) drops. The descent lemma and the O(1/T) and O(1/sqrt(T)) rates therefore do not apply to Sheaf-FRL as implemented. This is not cosmetic; it is the paper's central theoretical claim. The fix is either to change the algorithm to use the full gradient (which costs additional communication) or to analyze the local-penalty objective the algorithm actually minimizes.\n\nThe rest is in better shape. The derivation of the gradient of R_{A|i} itself checks out, the Procrustes solutions are correct, and the identifiability discussion in Appendix C is useful. The experiments are thin, though: no seeds, no error bars, and the Sheaf-FMTL baseline is run under a memory constraint the paper itself documents. The shared-pilot and whitening assumptions are enabling but plausible. Also, SheafAlign is cited but never compared, which is a miss for the closest sheaf-based method.\n\nNet: the framework deserves a serious referee and the idea is worth engaging with, but the paper needs major revision before its convergence claims can be accepted. Right now I would not cite the convergence result, and I would not hand this to a student as a reference for the theory. The empirical results might survive, but the proof as written does not cover the algorithm.","headline":"Promising sheaf-based federated representation learning idea, but the convergence theorems analyze a gradient step that Algorithm 1 does not compute — a load-bearing mismatch.","tokens_in":30392,"tokens_out":4973,"would_cite":false,"duration_ms":46721,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learnable sheaf glues together heterogeneous federated latent spaces, with provable convergence.","keywords":["federated representation learning","network sheaf","sheaf Laplacian","semantic alignment","Stiefel manifold","orthogonal Procrustes","semantic communication","decentralized optimization"],"falsifier":"Run the MNIST benchmark with the pilot indices randomly permuted per agent so the shared correspondence is broken while everything else stays fixed: if SFRL's communication accuracy does not collapse toward the non-cooperative baseline, the alignment penalty is not doing the claimed work. Alternatively, remove each encoder's whitening layer and retrain; a failure of communication accuracy to degrade would falsify the geometric-comparability premise.","tokens_in":29286,"feed_emoji":"🕸️","tokens_out":6218,"duration_ms":60041,"temperature":0.7,"pith_summary":"The paper tries to establish that heterogeneous agents—differing in data, architectures, and latent dimensions—can learn interoperable representations without being forced into a shared latent space. Its proposal, SFRL, treats each agent's latent space as the stalk of a learnable network sheaf and penalizes misalignment across edges with a quadratic sheaf-total-variation term evaluated on a small set of shared pilots. The accompanying decentralized algorithm, Sheaf-FRL, alternates local gradient steps with closed-form Procrustes updates of orthogonal and Stiefel restriction maps, and the paper proves convergence to first-order stationary points in both deterministic and stochastic settings. On MNIST cooperative classification with semantic communication, the paper reports that Sheaf-FRL beats ComFed, Sheaf-FMTL, FedProto, and FedMuscle in private and communication accuracy and degrades more gracefully under latent compression. If true, this would make semantic alignment across heterogeneous federated systems a practical alternative to shared-latent-space assumptions.","feed_headline":"Sheaf geometry aligns federated agents without a shared latent space","feed_subtitle":"Only a few shared pilots are exchanged, yet agents with different models and latent sizes learn to talk to each other.","key_machinery":"The central object is a network sheaf on the agent graph: each node i carries a latent space F(i) ~ $R^{{d_i}}$, each edge carries a stalk of dimension max(d_i,d_j), and restriction maps are orthogonal matrices O_{ji} in O(d_i) and Stiefel matrices V_{ij} in St(d_i,d_j). The coboundary operator $\\Delta$ and sheaf Laplacian L_F = $\\Delta$^T $\\Delta$ define the total variation TV(z) = ||$\\Delta$ z||^2 = sum_{e_ij} ||O_{ji} z_i - V_{ij} z_j||^2; after reparameterization this becomes sum_{e_ij} ||z_i - V_{ij} z_j||^2 under an \"embedding orientation\" from lower- to higher-dimensional stalks. The gluing penalty evaluates this on K shared pilot feature matrices A_i, and the restriction maps are updated by closed-form (semi-)orthogonal Procrustes solutions from thin SVDs of the cross-covariances A_i A_j^T.","core_discovery":"The paper claims that federated representation learning can be posed as Problem (SFRL): minimize local objectives plus a sheaf-total-variation gluing penalty evaluated on K shared pilots, with restriction maps constrained to orthogonal and Stiefel manifolds. It then claims that the decentralized alternating algorithm Sheaf-FRL—gradient updates on local models and closed-form Procrustes updates on edge maps—converges to first-order stationary points in both deterministic and stochastic settings (Theorems E.2–E.4). On MNIST semantic-communication benchmarks, the paper reports that Sheaf-FRL achieves higher private and communication accuracy than ComFed, Sheaf-FMTL, FedProto, and FedMuscle, and that its advantage grows with distribution shift and with latent-space compression.","pith_inferences":["Beyond the paper, the pilot mechanism suggests that a small public calibration set is enough to glue private latent spaces across organizations, enabling model interoperability without exposing weights or raw data.","Beyond the paper, the whitening-layer premise is testable: SFRL's communication-accuracy advantage should shrink when encoders lack whitening, which practitioners can verify before adopting the method.","Beyond the paper, the same sheaf gluing penalty could be ported to federated self-supervised learning, where common augmentations rather than shared labeled samples could define the pilots."],"forward_implications":["Agents with different latent dimensionalities can exchange task-relevant representations through isometric embeddings, so model heterogeneity no longer forces a shared latent space.","Communication per round scales as O(d_i K) because only K pilot embeddings are broadcast to neighbors.","The alternating scheme converges to first-order stationary points at O(1/T) in the deterministic case and O(1/sqrt(T)) in the stochastic case.","On the MNIST semantic-communication benchmark, the communication-accuracy gap over ComFed, Sheaf-FMTL, FedProto, and FedMuscle widens with distribution shift, and accuracy degrades gracefully as the bottleneck dimension drops toward 16."],"supporting_citations":[{"why":"Supplies the network sheaf formalism of node stalks and edge restriction maps that the framework builds on.","marker":"Curry, 2014"},{"why":"Provides the sheaf Laplacian and its spectral theory, which justify the quadratic gluing penalty.","marker":"Hansen & Ghrist, 2019"},{"why":"Gives the closed-form orthogonal Procrustes solution used for the restriction-map updates.","marker":"Schönemann, 1966"},{"why":"Supplies the descent lemma that underlies the deterministic and stochastic convergence proofs.","marker":"Nesterov, 2004; Beck, 2017"},{"why":"Provides the almost-sure convergence lemma used for the stochastic variant with diminishing step sizes.","marker":"Robbins & Siegmund, 1971"},{"why":"Motivates the gauge-freedom view of latent representations and zero-shot alignment that SFRL generalizes.","marker":"Moschella et al., 2023"},{"why":"Defines the ComFed baseline that learns compressed shared-latent projections, which SFRL compares against.","marker":"Badi et al., 2026"},{"why":"Defines the Sheaf-FMTL baseline operating on parameter spaces, which SFRL contrasts with representation-space alignment.","marker":"Issaid et al., 2025"},{"why":"Defines the FedProto prototype-alignment baseline used in the compression-robustness experiments.","marker":"Tan et al., 2022"},{"why":"Defines the FedMuscle contrastive-alignment baseline used in the two-agent compression study.","marker":"Setayesh et al., 2026"}],"fun_headline_variants":["Sheaf geometry aligns federated agents without shared latent space","Sheaf-FRL: heterogeneous agents learn together with few pilots","No shared latent space? Sheaf geometry has the answer","Sheaf-FRL outshines baselines by aligning agents with pilots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that agents share a small set of aligned pilot samples and that each encoder ends in a whitening layer, so transported latent representations are comparable on a common Euclidean reference; if either fails, the gluing penalty and the convergence analysis have nothing to act on.","fun_headline_variants_meta":{"raw":{"variants":["Sheaf geometry aligns federated agents without shared latent space","Sheaf-FRL: heterogeneous agents learn together with few pilots","No shared latent space? Sheaf geometry has the answer","Sheaf-FRL outshines baselines by aligning agents with pilots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000689,"raw_usage":{"total_tokens":3128,"prompt_tokens":960,"completion_tokens":2168,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":2097}},"tokens_in":576,"tokens_out":2168,"duration_ms":18735,"temperature":1.0,"reasoning_tokens":2097,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:29:11.128921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the MNIST benchmark with the pilot indices randomly permuted per agent so the shared correspondence is broken while everything else stays fixed: if SFRL's communication accuracy does not collapse toward the non-cooperative baseline, the alignment penalty is not doing the claimed work. Alternatively, remove each encoder's whitening layer and retrain; a failure of communication accuracy to degrade would falsify the geometric-comparability premise.","supporting_citations":[{"cited_title":"Communication-efficient and robust multi-modal federated learning via latent-space consensus","cited_arxiv_id":null,"evidence_quote":"Defines the ComFed baseline that learns compressed shared-latent projections, which SFRL compares against."}],"review_version":1}