Pith. sign in

REVIEW 2 major objections 2 minor 10 references

Blockchain-stored policy lets hospitals of unequal compute power run federated medical imaging via capacity-matched models and weighted ensembles.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 14:17 UTC pith:OFY64G4D

load-bearing objection Blockchain coordination cuts communication sharply but the capacity-aware model assignment is proposed without direct tests on heterogeneous hardware. the 2 major comments →

arxiv 2605.24418 v1 pith:OFY64G4D submitted 2026-05-23 cs.LG

ChainLearn: A Blockchain-Based Capacity-Aware Framework for Federated Ensemble Learning

Karan Sharma , Aditya Tripathi , Rahul Mishra , Tapas Kumar Maiti This is my paper
classification cs.LG
keywords federated learningblockchainensemble learningcapacity-awaremedical imagingheterogeneous hardwarecommunication efficiencynon-IID data
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that standard federated learning breaks when hospitals differ in hardware because it forces identical models and central aggregation. Instead, a Solidity contract records each hospital's throughput benchmark, assigns an architecture sized to that capacity, and stores ensemble weights. Hospitals train locally, submit only hashes and scalars, and the ensemble runs off-chain. Experiments on PneumoniaMNIST and DermaMNIST across non-IID partitions show the resulting system matches or exceeds FedAvg, FedProx, and FedMD in accuracy while cutting communication to 224 bytes per round.

Core claim

Capacity-aware coordination separates on-chain policy from off-chain learning: throughput measurements determine which of MobileNetV3-Small, EfficientNet-B0, or ResNet-50 each hospital trains, blockchain records the resulting weights, and a weighted ensemble produces the final prediction. This allows weak and strong hospitals to contribute without forcing uniform architectures or exchanging model parameters.

What carries the argument

The blockchain contract that stores hospital registration, benchmark hashes, metrics, and ensemble weights to coordinate capacity-appropriate model assignment and off-chain weighted inference.

Load-bearing premise

Measuring each hospital's throughput and assigning different model sizes will let the weighted ensemble maintain accuracy without the performance loss that uniform-architecture federated methods assume.

What would settle it

An experiment that runs the same hospitals on a new medical imaging task and finds the capacity-assigned ensemble produces higher calibration error or lower accuracy than a single uniform model trained under FedAvg.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Hospitals with low compute can still contribute without being excluded or forced to run oversized models.
  • Only 224 bytes travel per round instead of full model parameters, cutting communication by more than 912000 times versus FedAvg.
  • Calibration error stays the same or lower than an equal-weight ensemble across three non-IID levels.
  • Accuracy remains competitive with FedAvg, FedProx, and FedMD on both PneumoniaMNIST and DermaMNIST.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same throughput-to-architecture mapping could be applied to any device-heterogeneous setting such as edge cameras or mobile phones.
  • Storing only hashes and scalars on-chain makes the coordination layer scale to hundreds of hospitals without blockchain bloat.
  • If throughput benchmarks prove stable over time, the contract could be updated less often, further lowering overhead.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces ChainLearn, a blockchain-based framework for capacity-aware federated ensemble learning in medical imaging. Hospitals' throughput is measured to assign capacity-appropriate architectures (MobileNetV3-Small, EfficientNet-B0, ResNet-50); a Solidity contract handles registration, benchmark hashes, metrics and weights on-chain while training and weighted ensemble inference occur off-chain. Experiments on PneumoniaMNIST and DermaMNIST (5 seeds, 3 non-IID partitions) report competitive accuracy versus FedAvg/FedProx/FedMD, lower or equal calibration error versus equal-weight ensembles, and communication of only 224 bytes per round (912,000x reduction vs. FedAvg).

Significance. If the capacity-aware assignment and weighted ensemble are shown to function as described, the work would enable resource-heterogeneous hospitals to participate in privacy-preserving medical FL without forcing uniform models or high communication, while using blockchain only for lightweight coordination. The reported communication reduction is a concrete practical advantage.

major comments (2)
  1. [Experiments] Experiments section: the reported results on PneumoniaMNIST and DermaMNIST compare calibration error and accuracy but contain no evaluation, simulation, or measurement of hospital throughput, benchmark hashes, or dynamic assignment of architectures based on capacity. The central claim that capacity-aware coordination enables heterogeneous participation without standard FL degradation therefore rests on an untested assumption rather than demonstrated evidence.
  2. [Abstract / Framework description] Abstract and § on framework: the weighting procedure for the ensemble (how throughput metrics translate into ensemble weights) is not specified with sufficient detail to reproduce or verify the reported calibration-error advantage over equal-weight ensembles.
minor comments (2)
  1. [Abstract] The exact definition of the 224-byte communication figure and the 912,000x reduction calculation should be stated explicitly (e.g., which FedAvg payload size is used for the ratio).
  2. [Experiments] Error bars, exact non-IID partition generation method, and baseline hyper-parameters are referenced but not fully detailed in the provided text, limiting assessment of the accuracy and calibration claims.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. We address each major comment below. Where the comments identify gaps in the current manuscript, we will revise accordingly to strengthen the presentation of the capacity-aware mechanism and its empirical support.

read point-by-point responses

  1. Referee: [Experiments] Experiments section: the reported results on PneumoniaMNIST and DermaMNIST compare calibration error and accuracy but contain no evaluation, simulation, or measurement of hospital throughput, benchmark hashes, or dynamic assignment of architectures based on capacity. The central claim that capacity-aware coordination enables heterogeneous participation without standard FL degradation therefore rests on an untested assumption rather than demonstrated evidence.

    Authors: We agree that the current experiments section validates the ensemble accuracy and calibration results but does not include direct measurements, simulations, or evaluations of hospital throughput, benchmark hash computation, or the dynamic architecture assignment process. The framework description outlines these steps, yet the empirical demonstration of the capacity-aware coordination itself is absent from the reported results. In the revised manuscript we will add a dedicated simulation subsection that measures synthetic hospital throughput values, computes benchmark hashes, performs dynamic assignment of the three architectures, and verifies that the resulting heterogeneous ensemble maintains the reported performance levels without degradation relative to homogeneous baselines. revision: yes

  2. Referee: [Abstract / Framework description] Abstract and § on framework: the weighting procedure for the ensemble (how throughput metrics translate into ensemble weights) is not specified with sufficient detail to reproduce or verify the reported calibration-error advantage over equal-weight ensembles.

    Authors: We acknowledge that the current description of the weighting procedure is insufficiently detailed. The manuscript states that predictions are combined via a weighted ensemble derived from throughput metrics, yet it does not provide the explicit mapping, formula, or normalization steps that convert measured throughput into per-model weights. In the revised version we will expand the framework section with the precise weighting function, including how throughput values are normalized across hospitals and how the resulting weights are applied during off-chain inference, thereby enabling reproduction of the calibration-error comparison. revision: yes

Circularity Check

0 steps flagged

No circularity; derivation is self-contained empirical proposal

full rationale

The manuscript describes a framework (throughput measurement, capacity-based model assignment, weighted ensemble, on-chain Solidity policy) and reports empirical results on PneumoniaMNIST/DermaMNIST. No equations, fitted parameters presented as predictions, self-citations used as load-bearing uniqueness theorems, or ansatzes smuggled via prior work appear in the text. Claims reduce to experimental comparisons rather than any definitional equivalence or self-referential reduction. This is the normal non-circular case.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Limited information from abstract; main reliance on the assumption that capacity measurement and model assignment will yield effective ensembles in non-IID settings. No free parameters or invented entities explicitly detailed.

axioms (1)
  • domain assumption Hospitals can be assigned different model architectures based on measured throughput without loss of overall system performance.
    Central to the capacity-aware coordination claim in the abstract.

pith-pipeline@v0.9.1-grok · 5723 in / 1381 out tokens · 52709 ms · 2026-06-30T14:17:57.105888+00:00 · methodology

0 comments
read the original abstract

Federated learning is used in medical imaging where privacy prohibits centralizing data. Standard federated algorithms assume homogeneous hardware, identical architectures, and centralized aggregation, which fails when hospitals have unequal compute resources. We propose capacity-aware coordination: measure each hospital's throughput, assign capacity-appropriate architectures (MobileNetV3-Small, EfficientNet-B0, ResNet-50), and combine predictions via weighted ensemble. Weak and strong hospitals can participate without forcing uniform architectures. We separate on-chain policy from off-chain learning. A Solidity contract stores hospital registration, benchmark hashes, metrics, and weights. Hospitals train locally and submit only hashes and scalars (not parameters). Weighted ensemble inference is computed off-chain. Experiments on PneumoniaMNIST and DermaMNIST (5 seeds, 3 non-IID levels) show our method achieves lower or equal calibration error versus equal-weight ensemble and competitive accuracy versus FedAvg, FedProx, and FedMD. Communication overhead is 224 bytes per round, a reduction of over 912,000x compared to FedAvg.

Figures

Figures reproduced from arXiv: 2605.24418 by Aditya Tripathi, Karan Sharma, Rahul Mishra, Tapas Kumar Maiti.

Figure 1
Figure 1. Figure 1: Accuracy comparison on PneumoniaMNIST across all methods under [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: ECE comparison on PneumoniaMNIST (mean ± std, 5 seeds). Ours achieves directionally lower calibration error across all six (dataset, non-IID) cells. Mild (α = 1.0) Moderate (α = 0.5) Severe (α = 0.1) 0.0 0.1 0.2 0.3 0.4 0.5 ECE (lower is better) Calibration error by method and non-IID level — DermaMNIST (mean ± std, 5 seeds) FedAvg FedProx FedMD EqualWt Ours [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: ECE comparison on DermaMNIST (mean ± std, 5 seeds). Ours achieves lower ECE than EqualWt-Ens under moderate and severe hetero￾geneity. 15 true positives moves F1 by approximately 0.15. Centralized shows similar variance (0.13). Results at this severity level should be interpreted as approximately 0.70 with high variance rather than as a stable point estimate. E. Longitudinal Weighting The participation bon… view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study on PneumoniaMNIST (mean ± std, 5 seeds). Each bar removes one component of the weighting formula. Mild (α = 1.0) Moderate (α = 0.5) Severe (α = 0.1) 0.50 0.60 0.70 0.80 0.90 Accuracy Ablation study — accuracy per component removed — DermaMNIST (mean ± std, 5 seeds) Full No CapMul No Conf No ECE No Bonus No PoC [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study on DermaMNIST (mean ± std, 5 seeds) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-hospital per-round coordination payload on a log scale. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages

  1. [1]

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

    work page 1982

  2. [2]

    H. B. McMahan et al., ``Communication-Efficient Learning of Deep Networks from Decentralized Data,'' 2017

    work page 2017

  3. [3]

    Li et al., ``Federated Optimization in Heterogeneous Networks,'' MLSys, 2020

    T. Li et al., ``Federated Optimization in Heterogeneous Networks,'' MLSys, 2020

    work page 2020

  4. [4]

    Li and J

    D. Li and J. Wang, ``FedMD: Heterogeneous Federated Learning via Model Distillation,'' 2019

    work page 2019

  5. [5]

    Yang et al., ``MedMNIST v2 -- A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification,'' Scientific Data, 2023

    J. Yang et al., ``MedMNIST v2 -- A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification,'' Scientific Data, 2023

    work page 2023

  6. [6]

    Guo et al., ``On Calibration of Modern Neural Networks,'' ICML, 2017

    C. Guo et al., ``On Calibration of Modern Neural Networks,'' ICML, 2017

    work page 2017

  7. [7]

    Wood, ``Ethereum: A Secure Decentralized Generalized Transaction Ledger,'' 2014

    G. Wood, ``Ethereum: A Secure Decentralized Generalized Transaction Ledger,'' 2014

    work page 2014

  8. [8]

    Solidity Documentation, Ethereum Foundation

    work page

  9. [9]

    Kim et al., ``Blockchain-Based Federated Learning for Medical Systems,'' 2021

    M. Kim et al., ``Blockchain-Based Federated Learning for Medical Systems,'' 2021

    work page 2021

  10. [10]

    Rieke et al., ``The Future of Digital Health with Federated Learning,'' NPJ Digital Medicine, 2020

    work page 2020