{"id":"ebba0dbe-e94c-4b89-8e9b-b73c08775c5a","arxiv_id":"2504.18916","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A decentralized cross-silo federated learning framework using a private Ethereum smart contract as orchestrator and IPFS for storage reaches accuracy comparable to centralized multilevel FL while supporting synchronous and asynchronous training.","lead":"UnifyFL is a framework that lets separate organizations train machine learning models together without a single trusted middleman, using a blockchain coordinator and shared storage to exchange model updates. It reports accuracy close to a centralized setup, with an asynchronous mode that cuts training time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Private Clique PoA chain puts the decentralized-trust claim on an unstated majority-honest-validator assumption.","rationale":"The paper is a genuine systems contribution with a public repository and a testbed evaluation supporting the claim that accuracy is broadly comparable to a centralized hierarchical baseline. The accuracy numbers, while lacking error bars, are plausible and the asynchronous speedup is consistent with the design. The weakest point is the trust half of the headline claim. The architecture's decentralized trust ultimately reduces to the integrity of the Clique PoA validator set; Section 1.1.4 assumes a majority of clusters are honest, and Section 2.3 admits that private blockchains rely on trust in controlling entities. The paper does not define the validator set, its relationship to the participating organizations, or the threat model under which a validator majority is prevented from censoring scores or rewriting history. Because 'allowing trust' is one of the two advertised benefits, this assumption is load-bearing: if it fails, the system provides no trust advantage over a centralized aggregator. The concrete check would settle whether the implementation gives minority honest scorers any protection or whether the claimed trust property requires exactly the majority-honesty assumption that the abstract omits. This is a condition for acceptance, not a reason to reject; the reader's CONDITIONAL verdict remains appropriate, with the requested revisions to state the assumption and specify validator governance.","tokens_in":18697,"tokens_out":4773,"duration_ms":53896,"concrete_test":"Inspect the UnifyFL repository's genesis.json and deployment scripts to enumerate the Clique validator set and map each validator to a participating organization. Then run a controlled experiment in which a coalition controlling ceil(N/2)+1 validators attempts to censor one honest scorer's transaction for a round, and check whether that honest score appears in the smart contract's stored score list for the model. If the honest score can be excluded or the chain reorganized to drop it, the decentralized-trust claim fails without an explicit, enforced majority-honest-validator assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of 'allowing trust' rests on an unstated majority-honest-validator assumption that the architecture does not enforce. UnifyFL uses a private Ethereum chain with Clique Proof-of-Authority (Section 3.4.1). In Clique, validators finalize blocks and can reorder or censor transactions; a majority coalition can also rewrite history or bias which score submissions are recorded. The smart contract's 'random' scorer selection (Algorithm 1, startScoring) is therefore only as impartial as the validator majority. Section 1.1.4 explicitly assumes 'that a majority of clusters are honest' when discussing trustworthiness, and Section 2.3 concedes that private blockchains 'rely on trust in the controlling entities.' Yet the paper never specifies who runs the validators, how validator membership is governed, or whether each participating organization controls at most one validator. If one organization or a colluding subset controls more than half of the validator set, it can censor unfavorable scores, finalize a fabricated scoring history, and in effect become the trusted third party that the framework claims to eliminate. Thus the 'decentralized trust' component of the headline claim is not demonstrated; it is assumed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UnifyFL, a decentralized cross-silo federated learning framework in which participating FL clusters store aggregated model weights on IPFS and coordinate model sharing and scoring through a private Ethereum blockchain using Clique Proof-of-Authority. The framework supports synchronous and asynchronous orchestration, multiple aggregation and scoring policies, and allows each aggregator to choose its own policy. The evaluation uses a GPU cluster with Tiny ImageNet/VGG16 and an edge cluster with CIFAR-10/CNN, and reports that UnifyFL achieves accuracy close to a centralized multilevel FL baseline (roughly 35% vs. 36.84% on Tiny ImageNet; 51.33% vs. 50.4% on CIFAR-10) with lower wall-clock time in asynchronous mode and modest blockchain/IPFS overhead. The paper concludes that UnifyFL provides trust and efficient resource use without a third-party aggregator.","tokens_in":18906,"tokens_out":4985,"duration_ms":44919,"significance":"If the accuracy and trust claims held, UnifyFL would be a useful practical contribution: it is implemented on real testbeds, integrates with the Flower framework, ships an open-source repository, compares against independent benchmarks (Tiny ImageNet and CIFAR-10), uses no fitted parameters, and evaluates both synchronous and asynchronous modes. The reported accuracy numbers are plausibly comparable to the baseline, and the asynchronous mode's runtime advantage is qualitatively consistent with the design. However, the decentralized-trust claim is not established by the current evaluation, and the accuracy claims rest on single runs with no statistical support. The paper's value is therefore conditional on strengthening these two load-bearing points.","major_comments":[{"comment":"The central claim that UnifyFL provides trust without a third party is not demonstrated. The orchestrator is a private Ethereum chain using Clique Proof-of-Authority, and the paper itself notes in §2.3 that private blockchains rely on trust in the controlling entities and in §1.1.4 that trustworthiness assumes a majority of clusters are honest. Yet the paper never specifies who operates the Clique validators, how validator membership is governed, or what prevents one organization or a colluding subset from controlling a majority of validators. Because Clique validators can finalize arbitrary histories and the smart contract's random scorer selection in Algorithm 1 (startScoring) runs on that chain, a validator majority can censor unfavorable scores or rewrite the scoring record, effectively becoming the trusted third party the framework claims to eliminate. Please either specify a validator governance model with an explicit adversary assumption and threshold, or revise the trust claim to state the majority-honest assumption prominently as a limitation.","section":"§3.4.1, §2.3, §1.1.4, Algorithm 1"},{"comment":"The headline accuracy comparisons are single-run results: no repeated seeds, no error bars, no confidence intervals, and no significance tests are reported. The differences behind the claim of comparable accuracy are small (35% vs. 36.84% on Tiny ImageNet in §4.2.1; 51.33% vs. 50.4% on CIFAR-10 in §4.2.5), which is within the typical run-to-run variation of non-IID federated learning. Without variance information, the reader cannot judge whether UnifyFL is statistically indistinguishable from the baseline. Please report multiple seeds with mean and standard deviation (or equivalent) for at least the main comparisons in Tables 5 and 6.","section":"§4.2, Tables 5 and 6"},{"comment":"The scalability claim that with 60 clients split between 3 aggregators, UnifyFL maintained stable performance trends achieving an accuracy of roughly 30% comparable to the baseline for a similar configuration at the end of 100 rounds is asserted without a table, without the configuration details, and without a baseline that is shown for that setting. Since scalability is one of the stated evaluation questions (Q6), this needs a dedicated experimental report with the same level of detail as Tables 5 and 6.","section":"§4.2.6"}],"minor_comments":[{"comment":"Typo in the sentence 'We implement the end-to-end UnifyFL fframework' — 'fframework' should be 'framework'.","section":"§3.4.5"},{"comment":"Stray 'F' characters appear after 'FedYogi Agg 2' and after 'Agg 4' in the Config column; these should be removed.","section":"Table 5, Run 4"},{"comment":"Please clarify whether the baseline is a centralized multilevel FL implementation or an HBFL deployment; the text says both 'implement HBFL' and 'replicates the optimal scenario of centralized multilevel FL', which is confusing because §2.2 describes HBFL as having a centralized reducer.","section":"§4.2"},{"comment":"The sentence 'The results are similar for NIID dataset with other aggregation policies' is not supported by any table or figure; please add the data or remove the claim.","section":"§4.2.4"},{"comment":"The abstract claims 'optimal use of resources', but the only resource evidence is the overhead in Table 7; the resource-optimality aspect is not otherwise evaluated. Please temper the wording or add a direct resource-utilization comparison.","section":"§4.2.6"}],"recommendation":"major_revision","confidential_remarks":"The paper's systems contribution is real and the evaluation is reproducible in spirit, so I lean toward major revision rather than rejection. The main concern is that the title and abstract promise decentralized trust, but the Clique Proof-of-Authority validator assumption is unmodeled and the paper's own text admits the limitation. The authors should be asked to either document the trust assumptions and governance model or soften the trust claims accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper deserves a serious referee, not a desk reject. It is a genuine systems contribution: UnifyFL is implemented on Flower with a private Ethereum orchestrator and IPFS storage, and it actually runs on two testbeds (VGG16/Tiny ImageNet on GPUs, CNN/CIFAR-10 on heterogeneous edge nodes). The synchronous and asynchronous orchestration modes and the pluggable aggregation and scoring policies are concrete, useful features that go beyond the cited BCFL/HBFL/ChainFL baselines. The evaluation compares against a centralized multilevel FL baseline and reports comparable accuracy (e.g., 34-35% vs 36.84% on Tiny ImageNet NIID; 51.33% vs 50.4% on CIFAR-10 NIID), with async mode cutting runtime roughly from 6000 to 4000 seconds. The limitations section is candid about missing privacy techniques and Byzantine robustness.\n\nThe soft spots are real but not fatal. The biggest one is the trust claim. The orchestrator is a private Clique PoA chain, and the paper never specifies who controls the validator set or how membership is governed. If one organization or a colluding subset controls more than half the validators, it can censor unfavorable scores, finalize a fabricated history, and become exactly the trusted third party the abstract says UnifyFL eliminates. The paper's own Section 1.1.4 assumes a majority of clusters are honest, and Section 2.3 concedes private blockchains rely on controlling entities, but those admissions are not connected to the architecture's central promise. The accuracy claim is plausible; the decentralization trust claim is not demonstrated.\n\nEvaluation rigor is also thinner than it should be. Tables 5 and 6 show single runs with no repeated seeds, no error bars, and no significance tests. The 60-client scalability result appears only as a sentence (\"~30% accuracy\") with no table or figure. The HBFL baseline is their own implementation, which is acceptable but should be described enough to be checked. Minor point: the repo link is good, but no commit hash is cited, so reproducibility is harder to pin down.\n\nThe paper is, on balance, a solid engineering contribution with an overreaching headline. It should go to peer review, with requests for clarifying validator governance, tempering the trust language, and adding repeated-seed statistics for at least the headline comparisons. Researchers working on blockchain-based or cross-silo FL will get real value from this; the async mode and policy flexibility alone make it worth reading.","headline":"A real cross-silo FL framework with a plausible accuracy story and honest limitations, but the headline trust claim leans on an unstated majority-honest-validator assumption that the private PoA chain does not enforce.","tokens_in":19445,"tokens_out":1563,"would_cite":true,"duration_ms":19557,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UnifyFL claims decentralized cross-silo federated learning can match centralized accuracy without a trusted aggregator.","keywords":["Blockchain","Collaborative Learning","Federated Learning","Flower Framework","Inter-Planetary File System","Peer-to-Peer Learning","Scoring","Smart Contract"],"falsifier":"Run the same non-IID CIFAR-10 workload with more than half of the aggregators malicious, submitting poisoned model updates, and check whether the smart contract's majority-scorer selection and above-average aggregation policy keep global accuracy near 51%; a substantial drop below the centralized baseline would show that the decentralized trust claim depends on an honest majority. Alternatively, benchmark against a centralized multilevel aggregator on the same testbed and show a clear accuracy gap in the synchronous/non-IID condition.","tokens_in":18535,"feed_emoji":"🤝","tokens_out":6029,"duration_ms":55546,"temperature":0.7,"pith_summary":"The paper proposes UnifyFL, a cross-silo federated learning framework in which independent organizations collaborate without appointing a central, possibly biased aggregator. Its central claim is that this decentralized orchestration reaches model quality comparable to centralized multilevel FL while using resources more efficiently, and the experiments support that with a 51.33% global accuracy on CIFAR-10 against a 50.4% centralized baseline and about 35% on Tiny ImageNet against a 36.84% baseline. The framework adds an asynchronous mode to reduce idle time and wall-clock training time, and lets each organization choose its own aggregation algorithm and model-selection policy. If this claim holds, organizations that currently train in isolation can pool model updates through a transparent, auditable system instead of relying on a trusted third party.","feed_headline":"Federated learning without a trusted server matches central accuracy","feed_subtitle":"UnifyFL uses a blockchain orchestrator and IPFS to let rival organizations train together up to a third faster.","key_machinery":"The load-bearing mechanism is the orchestration workflow: after a cluster finishes local aggregation, it stores the serialized model on IPFS, registers the content identifier (CID) with the smart contract, and the contract then selects a majority subset of registered aggregators as scorers; scorers pull the model from IPFS, evaluate it with their own test set, and submit scores back to the contract. The contract accumulates the scores and exposes model CIDs with scores to every aggregator, which then applies its own aggregation policy (e.g., all models, top-k, above average) to build the next global model. The smart contract also implements synchronous and asynchronous orchestration phases, with the asynchronous mode triggering scoring immediately on submission to reduce idle time.","core_discovery":"UnifyFL's central discovery is that the trust-versus-resource trade-off in collaborative federated learning can be addressed by combining three pieces: a smart-contract-based decentralized orchestrator running on a private Ethereum chain with Clique proof-of-authority consensus, an IPFS distributed store for model weights, and a randomized majority-scoring protocol in which a subset of participating aggregators scores each submitted model. The claim, stated as the author would state it to a fair reader, is that this design lets independent FL clusters collaborate at accuracy comparable to an ideal centralized multilevel aggregator while removing the single point of trust, and the evaluation shows the asynchronous variant reaching 35% accuracy on Tiny ImageNet in about 4000 seconds versus 6230 seconds for the centralized baseline, and synchronous CIFAR-10 non-IID accuracy of 51.33% versus 50.4% central. Each cluster can keep its own aggregation policy and scoring algorithm, and the system runs on heterogeneous edge clusters with overhead of only a few percent CPU from the blockchain and storage layers.","pith_inferences":["If the honest-majority assumption is weakened, the same architecture could instead use a public permissionless chain or a rotating validator set to regain decentralization at the cost of throughput; the paper's private Clique chain trades full decentralization for resource efficiency, so the decentralized-trust guarantee is only as strong as the validator majority.","The random majority-scoring design decouples scoring from aggregation, which suggests a natural testbed for reputation or incentive mechanisms: tying future scoring weight to historical score accuracy could discourage lazy or malicious scorers without changing the protocol.","Because the framework stores model CIDs on-chain and weights on IPFS, the same audit trail could be reused for compliance, for example proving that a deployed model was aggregated from a certain set of contributors at a certain round.","The accuracy gap between synchronous (51%) and asynchronous (44%) non-IID runs suggests a tunable trade-off: aggregators could switch modes adaptively based on straggler presence or data skew rather than fixing one mode."],"forward_implications":["Organizations that do not trust each other can form a federation without appointing a third-party aggregator, as long as a majority of participants are honest.","Asynchronous mode cuts total training time by roughly a third in the Tiny ImageNet experiment (about 4000 seconds versus 6000+ seconds) while keeping accuracy within a few points of the centralized baseline.","Each cluster can keep its own aggregation algorithm, scoring function, and model-selection policy, so existing Flower-based FL pipelines can join without redesign.","Scoring policies such as above-average aggregation can filter out a single malicious model contributor, restoring accuracy after a temporary dip.","The framework handles IID and non-IID data, heterogeneous edge hardware, and scales to at least 60 clients with stable accuracy."],"supporting_citations":[{"why":"Supplies the HBFL baseline that replicates centralized multilevel FL and anchors the accuracy comparison.","marker":"[31]"},{"why":"Provides the Flower FL framework whose server implementation UnifyFL extends.","marker":"[7]"},{"why":"Defines the Ethereum platform that hosts the smart-contract orchestrator.","marker":"[9]"},{"why":"Defines IPFS, the distributed content-addressed storage used for model weights.","marker":"[6]"},{"why":"Supplies the FedAvg aggregation algorithm used in the main evaluation runs.","marker":"[24]"},{"why":"Defines the Clique proof-of-authority consensus used on the private Ethereum chain.","marker":"[37]"},{"why":"Provides the Tiny ImageNet dataset for the GPU-cluster workload.","marker":"[12]"},{"why":"Provides the CIFAR-10 dataset for the edge-cluster workload.","marker":"[21]"}],"fun_headline_variants":["Blockchain-based FL matches central accuracy, runs a third faster","Trustless federated learning: same accuracy, no central server","UnifyFL: decentralized FL without trusted aggregator, faster","Smart-contract FL beats multilevel central training time","Cross-silo FL without trust: accuracy on par, speed up"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The design assumes that a majority of the participating organizations, acting as validators and scorers, are honest; if they collude, the private proof-of-authority chain can finalize arbitrary histories and the scoring audit loses meaning.","fun_headline_variants_meta":{"raw":{"variants":["Blockchain-based FL matches central accuracy, runs a third faster","Trustless federated learning: same accuracy, no central server","UnifyFL: decentralized FL without trusted aggregator, faster","Smart-contract FL beats multilevel central training time","Cross-silo FL without trust: accuracy on par, speed up"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000273,"raw_usage":{"total_tokens":1655,"prompt_tokens":984,"completion_tokens":671,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":586}},"tokens_in":600,"tokens_out":671,"duration_ms":6453,"temperature":1.0,"reasoning_tokens":586,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:06:07.168691+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same non-IID CIFAR-10 workload with more than half of the aggregators malicious, submitting poisoned model updates, and check whether the smart contract's majority-scorer selection and above-average aggregation policy keep global accuracy near 51%; a substantial drop below the centralized baseline would show that the decentralized trust claim depends on an honest majority. Alternatively, benchmark against a centralized multilevel aggregator on the same testbed and show a clear accuracy gap in the synchronous/non-IID condition.","supporting_citations":[{"cited_title":"HBFL: A Hierarchical Blockchain-based Federated Learning Framework for a Collaborative IoT Intrusion Detection","cited_arxiv_id":"2204.04254","evidence_quote":"Supplies the HBFL baseline that replicates centralized multilevel FL and anchors the accuracy comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Clique proof-of-authority consensus used on the private Ethereum chain."}],"review_version":1}