{"id":"6e40847f-4128-441d-bca7-84c390a00fe5","arxiv_id":"2505.06497","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"FedADP unifies heterogeneous client models in federated learning by dynamically morphing them to a common architecture for aggregation, reporting accuracy improvements of up to 23.3% over FlexiFed.","lead":"FedADP lets federated-learning clients with different model shapes (for example, different widths and depths of VGG networks) transform their models into one shared global shape before aggregation, then shrink the global model back for each client. The authors report accuracy gains up to 23.3% over FlexiFed on CIFAR-100, but the method is only tested on VGG-style models and its key architecture-morphing step is not fully specified.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 3's To-Narrower/To-Shallower is not function-preserving as written: it deletes neurons without specifying downstream-layer updates, so FedADP's global-to-client morph distorts models and the reported gains are not explained.","rationale":"I agree with the reader's identification of NetChange function preservation as the weakest assumption, and I found a specific reason to doubt it rather than a mere absence of proof: Algorithm 3, as printed, is internally inconsistent with the function-preservation claim because narrowing a layer without adjusting the next layer changes the network's input-output map. This is a correctness risk in the paper's own terms, not a disagreement with any external consensus. The experimental section does not mitigate the issue: no code or hyperparameter details beyond basics are provided, and the algorithms are not mapped to convolutional layers, despite VGG being the only tested architecture. The proposed round-trip morphing test would settle whether a working implementation exists; if it fails, the central aggregation mechanism is not defined. I do not think this demands a stronger verdict than the reader's CONDITIONAL, because a corrected algorithm description plus a passing round-trip test could resolve the concern. I therefore keep the verdict unchanged.","tokens_in":7518,"tokens_out":11975,"duration_ms":130663,"concrete_test":"Run a literal implementation of Algorithms 2 and 3 on a trained VGG-19-Wider model, using the natural reading of 'neuron value' as the outgoing weight vector: first apply Algorithm 3's To-Narrower to a VGG-13 architecture, then Algorithm 2's To-Wider back to VGG-19-Wider, with no training in between. Compare the logits on a fixed validation batch before and after this round-trip. If the L2 logit distance is non-negligible or accuracy changes, Section III-B's function-preservation claim is false as written, and the missing downstream-layer/adjacent-layer update rule must be supplied. If the round-trip is near-identity, the concern is resolved; the paper should still clarify the omitted equations so the result can be reproduced without code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FedADP's central claim is that NetChange lets every client morph to and from a common global architecture while retaining the model's original functionality, so that FedAvg over the morphed models is a faithful combination of client knowledge. The load-bearing step is Section III-B's assertion that To-Narrower and To-Shallower 'ensure the model retains its original functionality and performance.' As written, Algorithm 3 cannot do this. Narrowing a layer from width M to Ntar changes the dimension of the vector consumed by the next layer. Preserving the composed function requires deleting the corresponding columns of the next layer's weight matrix (or filters) and folding the removed neurons' contributions into the surviving columns. Algorithm 3 only removes neurons after Ntar and adds s/Ntar to each remaining neuron's value; it specifies no downstream update. For a generic trained network, the layer output z_r = W_r h_{r-1}+b_r changes for every input once rows of W_r and b_r are removed, and the next layer cannot compensate without modification. To-Shallower is even less specified: removing a whole layer changes the function class, and preserving behavior requires composing the removed layer with adjacent layers, which the text does not mention. The algorithms are also stated in terms of individual neurons and 'values' and are never instantiated for convolutional filters, although all experiments use VGG CNNs. Net2Net's original construction preserves function only when new units are exact copies and their outgoing weights are split; after federated training those copies diverge, and the reverse operation in Algorithm 3 contains no mechanism to recombine them faithfully. Consequently, the claimed function-preserving morph from global VGG-19-Wider to VGG-13/VGG-16-Wider etc. is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedADP, a federated learning framework that aims to allow clients with heterogeneous model architectures to participate in a single global aggregation process. The key idea is to define a global architecture as the union of client architectures and then transform each client model to this global form before aggregation (To-Wider, To-Deeper) and transform the global model back to each client architecture before distribution (To-Narrower, To-Shallower). The transformations are collectively called NetChange and are asserted to preserve the original functionality of the models. The authors evaluate FedADP on MNIST, F-MNIST, CIFAR-10, and CIFAR-100 using VGG-family variants, reporting accuracy improvements over FlexiFed, Clustered-FL, and Standalone, with claimed improvements of up to 23.30% over FlexiFed.","tokens_in":7802,"tokens_out":4482,"duration_ms":46176,"significance":"If the NetChange transformations truly preserved model functionality, FedADP would be a valuable contribution to model-heterogeneous federated learning: it avoids discarding client-specific layers and allows all clients to contribute to a unified global model. The experimental results in Table I are internally consistent and show clear gains over the compared baselines across four datasets. However, the central mechanism of FedADP, the function-preservation property of NetChange, is asserted rather than proved, and the algorithmic descriptions are incomplete in ways that make the asserted property doubtful. Because the aggregation procedure in FedADP relies entirely on this property, the validity of the reported accuracy improvements is not established. The paper does not provide code, proofs, or sufficiently detailed experimental specifications to reproduce the results, which further weakens the contribution in its current form.","major_comments":[{"comment":"Algorithm 3 (To-Narrower) does not preserve the function of a trained network as claimed. Removing neurons r after Ntar changes the input dimension of the next layer, so the next layer's weight matrix and bias are no longer compatible. The algorithm only adds the sum of the removed neurons' values divided by Ntar to the remaining neurons; it specifies no update to the downstream layer's weights. For a generic trained network, the layer output z_r = W_r h_{r-1} + b_r changes for every input once rows of W_r and b_r are silently removed. To preserve the composed function, one must also delete the corresponding columns of the next layer's weight matrix or otherwise compensate for the removed contributions. The paper does not supply such a construction, so the load-bearing claim that the narrowed model 'retains its original functionality and performance' is unsupported.","section":"Section III-B, Algorithm 3"},{"comment":"To-Shallower is described only as the 'removal of unnecessary layers to match the client's model structure' with no algorithmic detail. Removing a layer changes the function class of the network; preserving behavior requires composing the removed layer with its adjacent layers (for example, by multiplying the relevant weight matrices and folding biases). The manuscript does not describe or justify such a composition, nor does it explain how the operation is applied to the convolutional VGG models used in the experiments. Without a precise, function-preserving shallowing operation, the distribution step of FedADP (Algorithm 1, line 6) cannot be verified and the experimental results cannot be attributed to a faithful aggregation of client knowledge.","section":"Section III-B, To-Shallower"},{"comment":"Algorithm 2 (To-Wider) is also underspecified with respect to the next layer's weights. The text states that new neurons are created by duplicating existing neurons and their incoming connections, but the pseudocode only copies scalar neuron values and scales values by |Mi|. It does not state how the outgoing connections of the new neuron are initialized or how the downstream weight matrix is adjusted to keep the layer output unchanged. For feedforward and convolutional networks, function preservation under widening requires duplicating the corresponding rows/columns of the downstream weight matrix and, in convolutional layers, handling filters and channels; none of this is specified. The paper must provide exact update rules for all affected weight tensors and a formal statement of the conditions under which the output function is preserved.","section":"Section III-B, Algorithm 2"},{"comment":"The evaluation only covers VGG-family architectures, where the notion of the 'union' of layer structures is straightforward and where the global model is a specific wider/deeper VGG variant. This does not demonstrate the general claim that FedADP can handle arbitrary heterogeneous model architectures. In addition, the experimental setup omits key reproducibility details: the data partitioning strategy (IID vs. non-IID) is not described, the exact set of eight architectures and their hyperparameters are not fully enumerated, and the implementation of the FlexiFed and Clustered-FL baselines in this heterogeneous setting is not specified. These omissions make the strong quantitative claims in Table I difficult to verify.","section":"Section IV-A"}],"minor_comments":[{"comment":"The argument order of NetChange is inconsistent: line 6 uses NetChange(ωt, ωk) while line 10 uses NetChange(ωk, ωt), but the text says NetChange(a, b) modifies a to conform to b. Clarify which argument is the source and which is the target in both calls.","section":"Algorithm 1"},{"comment":"The sentence 'the number of types of model architectures is set to 8, in this case, 6 clients will be trained using VGG-19, and the other 7 models will be adopted by two clients each' is ambiguous. With K=20, the arithmetic is 6 + 2·7 = 20, but it is unclear which of the eight architecture types are assigned to which clients. Please list the architectures explicitly.","section":"Section IV-A"},{"comment":"The phrase 'Clients will use 20% of their datasets in each round of training' is unclear: does this mean a randomly sampled 20% subset per round, the same 20% throughout, or a fraction of the local data used for each local epoch?","section":"Section IV-A"},{"comment":"There are typographical inconsistencies, e.g., 'exsiting' in the Introduction, and the inconsistent use of 'Cluster-FL' versus 'Clustered-FL' in the text and Figure 4.","section":"Throughout"},{"comment":"Table I reports means and standard deviations but no statistical significance tests. Given the small differences on MNIST and F-MNIST, a simple significance test would help support the claimed improvements.","section":"Table I and Figure 4"},{"comment":"Reference [32] for CIFAR-100 is listed as 'not found'; provide the full citation. Several references use informal arXiv identifiers without venue or version information.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central issue is that the function-preservation property of NetChange, on which the entire FedADP pipeline rests, is not established and appears incorrect as written. The reported accuracy gains may be an artifact of the subsequent local training rather than of faithful aggregation. I would encourage the editor to ask the authors to provide a correct and complete specification of the morphing operations, a proof of function preservation, and reproducible code before the paper can be considered further. The evaluation is also narrow, using only VGG variants, which is insufficient to support the general architectural-heterogeneity claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: FedADP tackles a real problem, and the experimental comparison is clean as far as it goes. But the central mechanism—NetChange's To-Narrower and To-Shallower—is not specified to the point where it can possibly be function-preserving as claimed, and all experiments are on VGG-family CNNs. The paper is worth a careful referee, but it needs major revision before the results mean what the authors say.\n\nWhat's genuinely new: combining Net2Net-style widening/deepening with narrowing/shallowing to bring heterogeneous client models to a common structure for aggregation, rather than aggregating only overlapping layers. That's a sensible direction and, as far as I can tell, not in the cited literature. The experiments are easy to follow: four datasets, 20 clients, eight VGG variants, comparing against FlexiFed, Clustered-FL, and Standalone. FedADP wins consistently, and the gap is large—up to 23.3% on CIFAR-100 over FlexiFed. That result would matter if the mechanism were sound.\n\nThe soft spot is the mechanism. Section III-B asserts that To-Narrower and To-Shallower 'ensure the model retains its original functionality and performance,' but Algorithm 3 shows no downstream-layer update when neurons are deleted, and the text's promise of 'adjustments to the input connection weights' never appears in the algorithm. Removing neurons from layer r changes the input dimension of layer r+1; preserving the composed function requires merging the removed neurons' outgoing weights into the surviving columns. Algorithm 3 only sums the removed neurons' 'values' into the survivors, which at best adjusts biases. For convolutional layers, the text doesn't even say how 'neurons' map to filters. To-Shallower is a one-line 'remove unnecessary layers,' with no description of how to compose adjacent layers. The function-preservation claim is therefore unproven, and the reported gains are not explained by the stated method. This isn't a minor omission; it's the load-bearing part of the paper.\n\nAlso minor: no code, no hyperparameter sensitivity beyond the final numbers, and the architectures used are all VGG variants, so we don't know how the morphing handles resnets or transformers.\n\nBottom line: the idea is plausible and the evaluation is honest, but the core algorithm is underspecified to the point of being incorrect as written. A serious referee should send it back for a complete specification of NetChange, ideally with a function-preservation check (e.g., equal outputs before and after morphing) and at least one non-VGG family. Worth engaging, not worth citing yet.","headline":"Real idea, honest experiments, but the paper's load-bearing morphing algorithm as written does not preserve function, so the reported gains are currently unexplained.","tokens_in":8363,"tokens_out":2945,"would_cite":false,"duration_ms":28275,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedADP claims that heterogeneous client models can be morphed into a unified global structure each round, aggregated with FedAvg, then morphed back to each client's own architecture, allowing every client to contribute its full model…","keywords":["federated learning","model heterogeneity","architecture heterogeneity","model aggregation","NetChange","VGG","personalized federated learning","resource-constrained devices"],"falsifier":"A direct test would take a trained local model, apply NetChange to the global shape and then back to the local shape, and compare predictions on a fixed validation batch; if the output distribution changes substantially (for example, classification accuracy drops more than a small tolerance), the functional-preservation assumption fails and FedADP's reported gains would not hold under repeated round-trip morphing. A second check is to re-run the CIFAR-100 experiment with the same hyperparameters and verify that the 23.30% relative improvement over FlexiFed reproduces.","tokens_in":7323,"feed_emoji":"🧠","tokens_out":2847,"duration_ms":28019,"temperature":0.7,"pith_summary":"FedADP is a federated learning framework that lets clients with different model architectures train together without discarding any layer or neuron. It works by temporarily reshaping every local model into a common global architecture before aggregation, then returning each client its own structure after the round. The paper reports that this approach outperforms existing heterogeneous-FL methods on standard image benchmarks, with accuracy gains up to 23.30% over FlexiFed and up to 46.25% over Clustered-FL. The significance is that even weak clients with small models can contribute fully to a shared global model, rather than being limited to overlapping substructures.","feed_headline":"Morphing client models into one shape lifts FL accuracy up to 23%","feed_subtitle":"FedADP widens, deepens, and trims VGG models each round so every client can join global aggregation.","key_machinery":"The central mechanism is NetChange, a set of four structure-morphing operations that extend Net2Net. To-Wider duplicates neurons and divides their incoming weights to keep layer outputs unchanged; To-Deeper inserts layers initialized as identity maps; To-Narrower removes excess neurons and redistributes their summed values to remaining neurons; To-Shallower deletes layers. These operations are applied in opposite directions before aggregation and before distribution, so all client models share one structure while retaining their original functional behavior.","core_discovery":"The paper's central claim is that model heterogeneity in federated learning can be handled by modifying model structures rather than by searching for common substructures. FedADP constructs a global model as the union of all client architectures, then applies NetChange operations: To-Wider and To-Deeper expand each local model to the global shape before aggregation, while To-Narrower and To-Shallower trim the global model back to each client's shape before distribution. After local training, the expanded models are aggregated with standard FedAvg. Experiments with VGG variants (VGG-13 through VGG-19-Wider) on MNIST, F-MNIST, CIFAR-10, and CIFAR-100 show FedADP achieving higher accuracy than FlexiFed, Clustered-FL, and Standalone, with the largest gains on CIFAR-100.","pith_inferences":["A natural testable extension is to measure the round-trip distortion of NetChange: apply To-Wider and To-Narrower in sequence to a trained model and compare its outputs on a fixed batch; if the outputs shift materially, the functional-preservation assumption is violated and cumulative drift would degrade long-horizon training.","The paper reports results on 20 clients with full participation; a likely stress test is partial participation and non-IID data, where the morphing operations may interact with client drift and require adaptive weighting.","Because the global model grows to the union of all architectures, communication and server-side memory costs scale with the largest client model; a practical variant might cap the global structure to control resource overhead.","The reported accuracy gains are on VGG-style networks; establishing similar gains on ResNet-style or transformer architectures would test whether the method generalizes beyond the evaluated family."],"forward_implications":["Every client, regardless of model size or depth, can participate in global aggregation without losing the knowledge encoded in its unique layers.","The global model is not tied to any single client architecture; it can be the union of all participating architectures, enabling full utilization of every client's capacity.","FedADP achieves higher accuracy than common-layer-only aggregation methods like FlexiFed while maintaining comparable convergence speed.","The approach applies to standard convolutional families such as VGG and could be extended to other architectures where layer-wise morphing is well-defined."],"supporting_citations":[{"why":"Net2Net provides the foundational knowledge-transfer operations that NetChange extends with widening and deepening while preserving functionality.","marker":"[21]"},{"why":"FlexiFed is the primary state-of-the-art baseline for heterogeneous model architectures, and FedADP is compared against it across all datasets.","marker":"[9]"},{"why":"Clustered-FL is the second baseline, representing methods that cluster clients with similar structures, and FedADP reports up to 46.25% higher accuracy.","marker":"[11]"},{"why":"The VGG family supplies the concrete model architectures used in all experiments, including VGG-13, VGG-16-Wider, VGG-19, and VGG-19-Wider.","marker":"[10]"},{"why":"MNIST is one of the four benchmark datasets used to evaluate accuracy and convergence against the baselines.","marker":"[30]"}],"fun_headline_variants":["Morphing client models lifts FL accuracy up to 23%","FedADP reshapes models on the fly to unify heterogeneous FL","Dynamic architecture morphing boosts FL in mixed settings","Shape-shifting neural nets for federated learning with varied clients"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the NetChange operations (To-Wider, To-Deeper, To-Narrower, To-Shallower) preserve each model's functional behavior when morphing it to and from the global architecture, so the aggregated model is a faithful combination of client knowledge rather than a distorted blend.","fun_headline_variants_meta":{"raw":{"variants":["Morphing client models lifts FL accuracy up to 23%","FedADP reshapes models on the fly to unify heterogeneous FL","Dynamic architecture morphing boosts FL in mixed settings","Shape-shifting neural nets for federated learning with varied clients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000223,"raw_usage":{"total_tokens":1406,"prompt_tokens":845,"completion_tokens":561,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":490}},"tokens_in":461,"tokens_out":561,"duration_ms":6004,"temperature":1.0,"reasoning_tokens":490,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:40:56.620386+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would take a trained local model, apply NetChange to the global shape and then back to the local shape, and compare predictions on a fixed validation batch; if the output distribution changes substantially (for example, classification accuracy drops more than a small tolerance), the functional-preservation assumption fails and FedADP's reported gains would not hold under repeated round-trip morphing. A second check is to re-run the CIFAR-100 experiment with the same hyperparameters and verify that the 23.30% relative improvement over FlexiFed reproduces.","supporting_citations":[{"cited_title":"Flexifed: Personalized federated learning for edge clients with heterogeneous model architectures","cited_arxiv_id":null,"evidence_quote":"FlexiFed is the primary state-of-the-art baseline for heterogeneous model architectures, and FedADP is compared against it across all datasets."}],"review_version":1}