{"id":"3b6be3dc-25ed-49f3-bfd8-304b5b71f95a","arxiv_id":"2508.13625","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FedOL builds a larger server model from one round of client prediction sharing, using confidence-weighted voting and iterative pseudo-label refinement to beat federated distillation baselines on CIFAR-100.","lead":"FedOL is a one-shot federated learning method in which clients send only their predictions on a shared unlabeled dataset, and the server trains a larger model by iteratively refining pseudo-labels. It reports accuracy gains over existing federated distillation baselines on CIFAR-100 with far less client communication than parameter-sharing methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (6)/(14) replaces 1[x∈D_l,k] with entropy-based λ_k(x), but public and private splits are disjoint, so this is not an approximation; no evidence confidence tracks class membership, leaving Eq. (7)'s objective on an unvalidated surrogate.","rationale":"The central claim is empirical, so I read the algorithm and setup with that in mind. FedOL has a clean one-shot communication design, and its Table I gains over FedDF/FedHKT are large enough to be worth independent reproduction. The weakest point is not the pseudo-label voting per se; it is the derivation of the distillation loss. Equations (5)-(6) purport to convert an expectation over private data into an expectation over the public set by replacing the membership indicator with λ_k(x). Because the experimental protocol defines public and private as disjoint random splits, this replacement is not an approximation; it is a different objective. The paper's only support for λ_k is an appeal to 'prediction confidence reflects training data characteristics' and a citation to FedHKT, with no measurement of that reflection. Under severe label skew, entropies are confounded by out-of-distribution confidence. This matters for the 'specialized objective function' contribution: if λ is arbitrary, the accuracy gain could be due to Algorithm 1 or to the ResNet56 server, and the paper currently has no ablation separating these. A diagnostic AUROC plus a three-way λ ablation would settle whether the mechanism is sound. I agree with the reader's weakest_assumption and would keep the verdict conditional pending that check.","tokens_in":9301,"tokens_out":8653,"duration_ms":97666,"concrete_test":"On the Dir(0.05) CIFAR-100 setup, restore labels on the public split for diagnosis only. For each client k and public sample x define m_k(x)=1 if the true class of x is among classes present in client k's private partition. (a) Measure AUROC of -H(σ(w_k^c(x))) and of λ_k(x) as predictors of m_k(x). (b) Re-run Table I's FedOL row three times: λ from Eq (14); uniform λ_k=1/K; oracle λ_k(x)∝m_k(x). If uniform≈Eq(14), the λ design is irrelevant to the gain; if oracle≫Eq(14), the objective is leaving large, known signal unused. Either outcome settles whether the Eq. (6)/(14) concern actually lands.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The server objective Eq. (7) is built on Eq. (5)->(6). In §V-A the dataset is randomly split into a public set and a private set, so for every x∈D_u the exact indicator 1[x∈D_l,k] is zero; replacing it with λ_k(x) cannot be an approximation of the expectation in Eq. (5). At best it is a domain-shift heuristic, and the paper treats it as derived. Moreover, Eq. (14) sets λ_k(x)∝exp(-H(σ(w_k^c(x)))), i.e., confidence as membership. Under Dir(0.05) or #I_c=20, clients see only a few classes; a model can be confidently wrong on public samples from classes it never trained on, so high confidence may be negatively correlated with 'this public sample's class is in this client's label set'. No experiment validates this link. Since L_d(ws) is half of Eq. (7), an incorrect surrogate can bias the server model; conversely, if the gain actually comes from Algorithm 1's voting or from the large ResNet56 server, the paper's claimed contribution of the specialized objective is not established. This does not disprove the empirical Table I results, but it makes the central mechanism unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FedOL is a one-shot federated learning method in which clients train local models on private data and share only their predictions over an unlabeled public dataset. The server then trains a larger model by optimizing an objective (Eq. (7)) that combines a per-client distillation loss with a pseudo-label loss, using an entropy-based weighting lambda_k(x) (Eq. (14)) and a class-wise confidence-weighted voting scheme (Algorithm 1). Experiments on CIFAR-100 under four heterogeneous partitions report that FedOL outperforms existing one-shot and multi-round baselines by large margins, with lower communication cost. The central technical claim is that the server objective is derived from a semi-supervised learning loss by replacing private-data membership with a confidence-based surrogate.","tokens_in":9695,"tokens_out":5500,"duration_ms":62942,"significance":"The problem addressed is practically important: federated learning with heterogeneous client models and a single communication round is an attractive scenario for mobile networks, and the paper proposes a plausible system design with a favorable communication cost. If the empirical results hold, FedOL would be a useful contribution to one-shot federated distillation. The paper also makes a specific, falsifiable design choice—using prediction confidence as a proxy for membership and weighting votes by class-wise confidence—that could be a meaningful advance if validated. However, the central derivation in Eqs. (5)-(6) rests on an unvalidated surrogate, and the evaluation does not isolate which component of the method drives the gains. The significance is therefore conditional on the authors providing direct validation of the lambda_k(x) approximation and an ablation study.","major_comments":[{"comment":"The derivation of the server objective is not mathematically grounded as written. Eq. (5) defines L_d(w_s) as an expectation over private labeled data D_l with the membership indicator 1[x in D_l,k]. Eq. (6) replaces this with an expectation over the public dataset D_u using weights lambda_k(x). Under the experimental setup in §V-A, D_u and D_l are disjoint random splits, so for every x in D_u the exact indicator in Eq. (5) is zero; lambda_k(x) in Eq. (14) is therefore not an approximation of that indicator but a new heuristic. No theorem or experiment in the paper shows that softmax confidence on a public sample correlates with that sample's class being present in the client's private training set. Under severe label skew (Dir(0.05), #I_c=20), a client model can be highly confident on public samples from classes it never trained on, so the proposed lambda could be negatively correlated","section":"§III-B, Eqs. (5)-(6); §IV-A, Eq. (14)"},{"comment":"The evaluation does not isolate the contribution of the proposed objective. FedOL differs from knowledge-distillation baselines by several algorithmic choices: the per-client distillation loss with lambda_k(x), the pseudo-label loss L_u, the entropy-gated class-wise confidence-weighted voting in Algorithm 1, and the iterative refinement schedule. Table I only reports the full system against baselines. Without ablations (e.g., uniform lambda versus Eq. (14), removing L_u, replacing the weighted vote with a simple majority vote, or using a non-iterative single-pass pseudo-label generation), the reader cannot determine whether the reported gains come from the specialized objective in Eq. (7) or from the pseudo-label voting mechanism. The paper's formal contribution is the objective function, so this missing ablation is essential.","section":"§V-B, Table I"},{"comment":"A second surrogate step is introduced without analysis: the true label y in D_l,k is replaced by the client model's soft prediction sigma(w_k^c(x)) in Eq. (4). For private samples on which the client is not perfectly confident, this changes the loss being minimized, and error propagation is not discussed. This is related to the lambda_k(x) issue but is a separate approximation that should be acknowledged and, if possible, bounded or tested.","section":"§III-B, Eq. (4)"}],"minor_comments":[{"comment":"The cost table is difficult to read. The row 'Comm/MB 0.38 65.14 0.38' does not align with the column headers, and the entry 'Comp standard supervised-learning procedure' appears truncated. Please clarify which entries correspond to communication and which to computation, and specify the values for each method.","section":"Table II"},{"comment":"The denominator notation is malformed: 'Pj=K j=1' should be sum_{j=1}^K. Please fix the typesetting.","section":"Eq. (14)"},{"comment":"Hyperparameters rho, tau, and the number of server iterations are given only as default values (rho starts at 0.1 and increases by 0.05, tau=0.2, 10 iterations). A sensitivity analysis over these parameters would improve reproducibility and help assess robustness.","section":"§V-A"},{"comment":"In the multi-round comparison, FedOL is shown as a single value over rounds. The caption should explicitly state that FedOL uses one communication round and is plotted as a horizontal line for comparison, to avoid ambiguity.","section":"Fig. 2"},{"comment":"The one-shot adaptation of baselines such as FedDF, FedHKT, FedET, and FedKT is not specified in detail. For a fair comparison, please state the number of local epochs, server-side training epochs, and public dataset size used for each baseline in the one-shot setting.","section":"§V-A, Baselines"}],"recommendation":"major_revision","confidential_remarks":"The empirical results in Table I are potentially strong and the system design is practically motivated. However, the paper currently overclaims the derivation of the server objective: the lambda_k(x) surrogate is not validated, and the ablation needed to attribute the gains is missing. I would not reject the paper on these grounds, as the heuristic may be defensible with additional experiments, but the authors should be asked to either weaken the derivation claim and provide direct validation of the membership-confidence link, or remove the approximation language and treat Eq. (6) as an explicit design choice supported by ablations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead FedOL. The contribution is real but narrower than the paper claims. The algorithm is a sensible combination of known pieces—entropy-thresholded source selection, class-wise confidence voting with -1 rejection, and iterative pseudo-label refinement in a one-shot distillation objective—and the CIFAR-100 results are strong: FedOL beats the best knowledge-based baseline by 5.8+ points under every partition. If the numbers are reproducible, this is a useful engineering recipe.\n\nThe paper's actual novelty is the specific objective in Eq. (7) and the voting in Algorithm 1, not any single component. The writing is clear and the empirical comparison is reasonably fair: knowledge-based baselines also get the larger ResNet56 server, and communication cost is quantified.\n\nThe soft spot is the derivation of Eq. (6). The stress-test note is right: the paper treats lambda_k(x) as an approximation of 1[x in D_l,k], but public and private splits are disjoint, so for any public sample the indicator is exactly zero. What lambda_k(x) actually does is replace membership with prediction confidence, and there is no evidence that a client's confidence on a public image tracks whether that image's class appeared in its training set. Under severe label skew (Dir(0.05), #Ic=20), a model can be confidently wrong on classes it never saw. So the central loss function in Eq. (7) rests on an unvalidated surrogate. This doesn't kill the empirical result—Algorithm 1's voting might be doing the real work—but the paper's stated mechanism is not established.\n\nAlso missing: ablations. No test that isolates the entropy threshold, the -1 rejection, the class-wise weights, or the iterative refinement. No sensitivity analysis for rho or tau. And no comparison against a larger server model trained directly on the same public set with simple ensemble distillation, which would help separate capacity from method.\n\nWho is this for? People working on one-shot or heterogeneous federated learning. It's a worthwhile empirical data point, and the algorithm is clearly specified enough to reimplement. But the derivation needs honest rewriting: call lambda_k(x) a heuristic weighting, not an approximation, and add the missing ablations.\n\nMy read: the central claim \"FedOL works well\" is probably true, but the \"why\" is unsubstantiated. I'd give it a serious referee, asking for ablations and a corrected derivation. I would not reject on the current evidence, but I'd be surprised if it survives as-is without changes.","headline":"Useful one-shot FL recipe with strong CIFAR-100 numbers, but the derivation of its central objective replaces private-data membership with an unvalidated confidence heuristic, and the missing ablations leave the actual mechanism unclear.","tokens_in":10114,"tokens_out":2697,"would_cite":true,"duration_ms":27958,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A one-shot federated learning method, FedOL, claims to build a larger server model from heterogeneous clients by sharing only predictions on an unlabeled public dataset, outperforming baselines in a single round.","keywords":["federated learning","one-shot learning","knowledge distillation","heterogeneous client models","pseudo-labeling","label skew","communication efficiency"],"falsifier":"Train a client on a single class and measure its average softmax confidence on public samples from classes it never saw; if that confidence is not clearly lower than on its own class, the $\\lambda_k(x)$ weighting in Eq (6) is not tracking membership. A sharper test: replace $\\lambda_k(x)$ with the true indicator $1[x \\in D_{l,k}]$ (available in simulation) and compare final accuracy; if accuracy does not drop, the confidence surrogate is not the mechanism carrying the result.","tokens_in":9229,"feed_emoji":"📡","tokens_out":5460,"duration_ms":55052,"temperature":0.7,"pith_summary":"The paper is trying to establish that a server can learn a larger, more general model than any client owns, using only one round of client predictions on an unlabeled public dataset. Knowledge distillation lets clients keep their data and even their model architectures private, while the server iteratively refines pseudo-labels and its own model to overcome skewed local data. If the claim holds, mobile networks can obtain capable server-side models at a fraction of the communication and computation cost of standard federated learning, and with a single round of client participation.","feed_headline":"One prediction round beats 15-round federated learning","feed_subtitle":"FedOL builds a larger server model from client predictions on unlabeled data, using 0.38 MB per client.","key_machinery":"The objective in Eq (7): a per-client distillation loss $L_d(w_s)$ weighted by $\\lambda_k(x) = \\exp(-H(\\sigma(w_k^c(x)))) / \\sum_j \\exp(-H(\\sigma(w_j^c(x))))$ plus a pseudo-label loss $L_u(w_s, \\hat{y})$, optimized by alternating pseudo-label generation and server updates. The pseudo-labels come from Algorithm 1's weighted vote over client and server predictions, using class-wise confidence scores $C^2_k$ and negative-learning label vectors.","core_discovery":"The paper claims that one-shot federated learning via prediction sharing on an unlabeled public dataset can build a larger server model than any client's, even when clients hold heterogeneous architectures and skewed label distributions. FedOL's server trains by minimizing Eq (7), which combines per-client distillation with iteratively refined pseudo-labels. On CIFAR-100, FedOL outperforms all baselines under one communication round in every partition setting, e.g., 37.2% vs 29.8% best baseline under Dir(1) and 29.8% vs 18.8% under Dir(0.05), and uses only 0.38 MB of communication per client compared with 65.14 MB for parameter sharing.","pith_inferences":["The confidence-as-membership surrogate in Eq (14) is untested; if confidence on a public sample does not track whether that sample's class appeared in the client's private data, the distillation loss optimizes an incorrect objective. A direct test would be replacing $\\lambda_k(x)$ with the oracle membership indicator and comparing accuracy.","The one-shot prediction-sharing protocol suggests a 'model market' where clients contribute pre-trained models or their predictions without iterative coordination; the paper's cost numbers (0.38 MB vs 65.14 MB) quantify the saving.","The paper evaluates on CIFAR-100 only; the pseudo-label voting and entropy-threshold schedule could behave differently on larger or noisier public datasets, where confident predictions are systematically wrong."],"forward_implications":["With one communication round, FedOL beats the best baseline by more than 5.8% absolute accuracy under every tested partition of CIFAR-100.","Communication per client drops to 0.38 MB for prediction sharing versus 65.14 MB for sending ResNet20 parameters.","FedOL matches or exceeds knowledge-distillation baselines that are allowed 15 communication rounds, while itself using a single round.","Clients are free to use different model architectures, since only predictions are shared, not weights.","The method requires no labeled public data, only an unlabeled dataset shared by all participants."],"supporting_citations":[{"why":"Defines the standard FedAvg parameter-sharing baseline that FedOL replaces and compares against.","marker":"[6]"},{"why":"Introduces knowledge distillation, the prediction-sharing mechanism at the core of FedOL.","marker":"[10]"},{"why":"Provides FedDF, an ensemble-distillation baseline whose uniform aggregation FedOL extends with confidence weights.","marker":"[13]"},{"why":"Inspires the confidence-based weighting in Eq (14) and serves as a hierarchical-knowledge-transfer baseline.","marker":"[16]"},{"why":"Defines the one-shot federated learning setting that FedOL targets.","marker":"[17]"},{"why":"Provides the insight that client predictions mirror local data distribution, motivating class-wise confidence scores.","marker":"[19]"},{"why":"Gives the semi-supervised pseudo-labeling formulation behind Eq (3) and the MinE baseline.","marker":"[20]"},{"why":"Supplies negative learning for the vote vectors in Algorithm 1.","marker":"[22]"},{"why":"FedKT is a one-shot knowledge-transfer baseline that FedOL compares against.","marker":"[24]"}],"fun_headline_variants":["Predictions only: one round to a larger model","Single round beats 15, uses 0.38 MB per client","FedOL: one round, heterogeneous clients, larger model","One-shot FL: larger server model from client predictions"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The server's training loss assumes that a client's prediction confidence on a public sample reflects whether that sample's class appears in the client's private data, and if confidence does not track membership the central objective is optimizing the wrong surrogate.","fun_headline_variants_meta":{"raw":{"variants":["Predictions only: one round to a larger model","Single round beats 15, uses 0.38 MB per client","FedOL: one round, heterogeneous clients, larger model","One-shot FL: larger server model from client predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000837,"raw_usage":{"total_tokens":3520,"prompt_tokens":813,"completion_tokens":2707,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":2651}},"tokens_in":557,"tokens_out":2707,"duration_ms":22369,"temperature":1.0,"reasoning_tokens":2651,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:57:51.445152+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a client on a single class and measure its average softmax confidence on public samples from classes it never saw; if that confidence is not clearly lower than on its own class, the $\\lambda_k(x)$ weighting in Eq (6) is not tracking membership. A sharper test: replace $\\lambda_k(x)$ with the true indicator $1[x \\in D_{l,k}]$ (available in simulation) and compare final accuracy; if accuracy does not drop, the confidence surrogate is not the mechanism carrying the result.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Defines the standard FedAvg parameter-sharing baseline that FedOL replaces and compares against."},{"cited_title":"Ensemble distillation for robust model fusion in federated learning,","cited_arxiv_id":null,"evidence_quote":"Provides FedDF, an ensemble-distillation baseline whose uniform aggregation FedOL extends with confidence weights."},{"cited_title":"A hierarchical knowledge transfer framework for heterogeneous federated learning,","cited_arxiv_id":null,"evidence_quote":"Inspires the confidence-based weighting in Eq (14) and serves as a hierarchical-knowledge-transfer baseline."},{"cited_title":"Towards addressing label skews in one-shot federated learning,","cited_arxiv_id":null,"evidence_quote":"Provides the insight that client predictions mirror local data distribution, motivating class-wise confidence scores."},{"cited_title":"Semi-supervised learning by entropy minimization,","cited_arxiv_id":null,"evidence_quote":"Gives the semi-supervised pseudo-labeling formulation behind Eq (3) and the MinE baseline."},{"cited_title":"Nlnl: Negative learning for noisy labels,","cited_arxiv_id":null,"evidence_quote":"Supplies negative learning for the vote vectors in Algorithm 1."},{"cited_title":"Practical one-shot federated learning for cross-silo setting,","cited_arxiv_id":null,"evidence_quote":"FedKT is a one-shot knowledge-transfer baseline that FedOL compares against."}],"review_version":1}