{"id":"04e26540-ce45-4e19-842f-33c3d60d606f","arxiv_id":"2608.01095","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"FL-OA outsources model-update auditing to a third-party server with a root dataset, adds a gradient ascent step and a correction term to local training, and reduces audit dimension by extracting critical parameters.","lead":"FL-OA is a federated learning defense that hires an outside server holding a clean dataset to audit each device's update and ignore ones that point the wrong way. It also reshapes local training so honest updates look more alike, and reports strong accuracy under attacks with up to 70% malicious devices.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's contraction factor is provably ≥1 under Assumption 1, so the stated convergence bound diverges; the referenced proof is absent.","rationale":"The reader's weakest assumption was Assumption 2, the distributional match between the root dataset and device datasets. That is a real limitation, and the paper's own two-class non-IID experiment supports it. However, my read identifies a more fundamental problem that the reader did not make central: the stated theorem that is supposed to prove Byzantine robustness has a contraction factor that cannot be less than 1 under the paper's own smoothness/strong-convexity assumptions. Because the proof is absent, this could be a typographical error rather than a fatal flaw, so the appropriate disposition remains conditional: the authors must supply the missing appendices and correct or justify Theorem 1. I therefore keep the reader's CONDITIONAL verdict, but the condition should explicitly include a corrected, verifiable Theorem 1, not merely softened claims about assumptions and additional experimental details. The experimental comparison is substantial and could still support an empirical contribution, but the formal anchor must be fixed before the central claim can be accepted.","tokens_in":52405,"tokens_out":16646,"duration_ms":147624,"concrete_test":"Obtain the missing Appendix I and independently re-derive the one-step contraction of the FL-OA aggregation recursion under Assumptions 1–3. Specifically, compute the coefficient ρ that multiplies ||W^t − W*|| in the bound. If the proof yields ρ = √(1+μ²L1²−μγ) + 2μL1 + 24μΔ2, then substituting the paper's default μ=1 and any L1 ≥ γ > 0 gives ρ > 1, confirming that Theorem 1 cannot guarantee convergence. A corrected theorem must either identify an error in the displayed τ or state and verify a parameter regime with ρ < 1; the concern is settled by checking whether such a regime exists.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section V's Theorem 1 is the formal anchor of the paper's robustness claim, but as stated it is internally inconsistent. It asserts ||W^t − W*|| ≤ (1−τ)^t ||W^init − W*|| + 12μΔ1/τ with τ = 1 − (√(1+μ²L1²−μγ) + 2μL1 + 24μΔ2). A meaningful bound requires 0 < τ < 1, i.e. ρ := √(1+μ²L1²−μγ) + 2μL1 + 24μΔ2 < 1. Under Assumption 1, the strong-convexity constant cannot exceed the smoothness constant, so γ ≤ L1. For x = μL1 > 0, ρ ≥ √(1+x²−μγ) + 2x ≥ √(1+x²−x) + 2x. If 0 < x < 1, then √(1+x²−x) = √((1−x)²+x) > 1−x, so ρ > 1+x > 1. If x ≥ 1, the square-root term is at least 1 and ρ > 1. Thus ρ > 1 for every positive μ and L1, so τ < 0 and (1−τ)^t grows exponentially. No additional condition on μ, L1, γ, or Δ2 is stated that could make τ positive, and the proof is relegated to a missing 'Appendix I'. Unless the displayed τ is a typographical error and a corrected derivation is supplied, the formal guarantee fails under the theorem's own assumptions—independent of the separate Assumption 2 distribution-matching concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FL-OA, a Byzantine-robust federated learning framework in which the task server collaborates with an outsourced server that holds a root dataset. The outsourced server audits each device update by extracting critical parameters via a parameter-importance indicator, computing a ReLU-cosine trust score against the root update, and normalizing update magnitudes; the task server then performs a trust-weighted aggregation. During local training, FL-OA adds a gradient-ascent step and a correction term intended to reduce divergence among benign updates. The manuscript claims theoretical guarantees in Theorem 1 (bounded deviation of the global model from the optimum under Byzantine attacks) and Theorem 2 (characterization of the correction term), and presents extensive experiments on CIFAR10, CIFAR100, SVHN, CINIC, and Tiny-ImageNet under Gaussian, Neurotoxin, and Focused-Flip attacks, with up to 70% malicious devices, plus ablations, sensitivity studies, scalability, stability, and complexity measurements.","tokens_in":52800,"tokens_out":7404,"duration_ms":69893,"significance":"If the theoretical claims were corrected and the missing appendices supplied, the paper would make a useful contribution: it combines outsourced root-dataset auditing with local-training modifications, and the PII-based coordinate selection is an interesting response to the curse of dimensionality. The empirical evaluation is broad and mostly consistent, showing FL-OA outperforming five baselines across several datasets and attack types. The paper is also honest about important limitations, including the two-class non-IID breakdown and the OS-collusion/privacy caveats. However, the formal anchor Theorem 1 is currently false as stated, the proofs are in absent appendices, and Assumption 2 limits the advertised 'without strong assumptions' claim. The paper is promising but not publishable without substantial revision.","major_comments":[{"comment":"The displayed contraction bound is internally inconsistent under Assumption 1. Since the global loss is both γ-strongly convex and L1-smooth, we have γ ≤ L1. Let x = μL1 > 0. The parenthesized quantity τ = 1 − (√(1+μ²L1²−μγ) + 2μL1 + 24μΔ2) satisfies τ < 0 because √(1+x²−x) + 2x > 1 for every x > 0, and 24μΔ2 is positive. Hence (1−τ)^t is exponentially growing in t, so Eq. (13) does not bound the deviation to a neighborhood of W*. Moreover, Eq. (7) shows that the aggregated update depends on attack-dependent trust scores s_t^k, but the theorem statement contains no quantity that encodes the effect of malicious updates; without such a term, or an explicit bound on it in the proof, Theorem 1 cannot support the Byzantine-robustness claim in Discussion 1. The proof is said to be in Appendix I, which is not present in the manuscript. The authors must correct the theorem statement and supply a complete proof.","section":"Section V, Theorem 1 (Eq. (13))"},{"comment":"The proof of Theorem 2 is relegated to an 'Appendix II' that is not included in the manuscript. Since Discussion 2 uses Theorem 2 to justify the correction-term design and to claim that the correction term estimates the local offset, this is a load-bearing theoretical claim. The algebra in Section V is partially checkable from Algorithm 2, but the manuscript must include the full derivation, including the indexing of λ(e) and the statement of the recurrence that leads to Eqs. (14) and (15).","section":"Section V, Theorem 2 and Algorithm 2"},{"comment":"Assumption 2 requires the root dataset and every device dataset to be independently sampled from the same training distribution χ, yet the abstract and Section I advertise robustness 'without strong assumptions.' The two-class non-IID experiment (Root2) in Table VI shows that when class sets are disjoint, FL-OA accuracy falls below 60% on CIFAR10 even without attacks, and drops to 26–52% under Gaussian and Neurotoxin attacks. The paper itself concludes that the framework is suited only to scenarios where classes overlap significantly. This is a load-bearing limitation of the outsourced-auditing trust mechanism, and the 'no strong assumptions' claim should be qualified accordingly in the abstract and introduction, or Assumption 2 should be weakened and the method adapted.","section":"Section V, Assumption 2 and Section VI.B.8, Table VI"}],"minor_comments":[{"comment":"The text alternates between 'PII' and 'PIF' (e.g., 'a corresponding PII vector' and 'A parameter with a large PIF value'); please use a single consistent abbreviation.","section":"Section IV.C, Eq. (4)"},{"comment":"The three attack types are numbered (i), (ii), (ii); the third should be (iii).","section":"Section VI.A.3"},{"comment":"The Neurotoxin rows appear to be missing the Att=30% entries for Root2, Root10, and DIR(0.1): only nine percentages are shown for the twelve columns. Please complete the table.","section":"Section VI.B.8, Table VI"},{"comment":"The fidelity discussion says FL-OA should be close to Baseline, but then states that under DIR(0.1) FL-OA is significantly higher than Baseline; this apparent tension should be clarified.","section":"Section VI.B.2"},{"comment":"The sentence 'the inconsistency of model updates is independent the local learning rate η and the importance α' should read 'independent of the local learning rate ... and ...'; also, 'exponential average' is used loosely.","section":"Section V, Discussion 2"},{"comment":"The header contains the typo 'RseNet18'; it should be 'ResNet18'.","section":"Table IV"},{"comment":"The notation for the critical-parameter selection function is inconsistent: Algorithm 3 uses 'Top(PII,κ)' while the text defines 'top(PII,κ)'; please harmonize.","section":"Algorithm 3 and Section IV.C"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the missing appendices and the impossible contraction factor in Theorem 1 are severe. If the authors cannot produce a corrected, complete proof of a meaningful convergence bound, I would not support publication. The Table VI result should also be treated as a clear bound on the advertised scope. I would also verify that the submitted source is complete, since several artifacts (missing appendices, malformed table, repeated numbering) suggest a formatting problem in the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nTwo things you should know about this one. The empirical package is genuinely broad: FL-OA is tested against five baselines, three attack types, several datasets, and up to 70% malicious devices, with ablations for each component, stability runs, complexity analysis, and even a WAN transmission-delay measurement. On the numbers, it consistently beats the baselines. That part is workmanlike and believable.\n\nWhat is actually new is the specific integration: a parameter importance indicator for selecting critical coordinates during outsourced auditing, combined with gradient-ascent and correction-term local training. The high-level outsourced-auditing idea is from FL-Auditor, and the local tricks are adapted from SAM/FedSpeed/FedProx, but this particular combination appears original. The paper does not overclaim novelty there.\n\nThe soft spots are real, and one is load-bearing. Theorem 1, the convergence guarantee, is not just missing its proof; the displayed result appears to be false as stated. The contraction factor is τ = 1 − (√(1+μ²L1²−μγ) + 2μL1 + 24μΔ2). Under Assumption 1, γ ≤ L1, and for any positive μ and L1 the parenthesized sum exceeds 1, so τ < 0 and (1−τ)^t grows exponentially rather than shrinking. The proof is deferred to a missing appendix. This is a formal inconsistency at the center of the paper's theoretical claims, independent of the distribution-matching concern.\n\nSecond, Assumption 2—that root and local datasets are independently sampled from the same distribution χ—contradicts the abstract's promise of robust aggregation \"without strong assumptions.\" The paper's own Table VI shows that when class sets are disjoint, FL-OA drops below 60% accuracy on CIFAR10 even without attacks, and attacks degrade it further. So the main robustness claim only holds under significant distribution overlap.\n\nSmaller issues: no code is released, several tables lack error bars, and the sensitivity analysis shows FL-OA is markedly sensitive to β. That is fixable but worth noting.\n\nI want to give credit where it is earned. The paper is unusually honest about its limitations—the conclusion discusses OS collusion and privacy leakage, and Section VI.B.8 explicitly concedes the disjoint-class failure. The experiments are far more comprehensive than most FL defense papers.\n\nWho should read it: people working on Byzantine-robust FL who want a concrete outsourced-auditing baseline and a thorough empirical comparison. They will find it useful even if the theory needs a complete rewrite.\n\nRecommendation: this deserves peer review, not a desk reject, because the problem is relevant and the experiments are extensive. But I would send it with a clear request: fix the theorem or withdraw the formal claims, release code, and add error bars. If the proof cannot be repaired, the paper should be reframed as an empirical study only.","headline":"Broad, well-executed empirical study undermined by a convergence theorem that contradicts its own assumptions and a central claim that overstates the framework's assumptions.","tokens_in":53310,"tokens_out":3057,"would_cite":false,"duration_ms":27996,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated learning can outsource its update auditing to a third party that holds a clean root dataset, and the paper claims this removes the need for a benign majority or a server-owned dataset.","keywords":["federated learning","Byzantine attacks","outsourced auditing","trust score","non-IID data","curse of dimensionality","gradient ascent","correction term"],"falsifier":"Run FL-OA on CIFAR10 with a two-class non-IID partition in which each device owns two disjoint classes and the outsourced server owns a different pair of classes (the paper's Root2 configuration): Table VI reports test accuracy below 60% even with no malicious devices, so the claim that robust aggregation holds without strong assumptions is contradicted in exactly this distribution-shift regime.","tokens_in":52236,"feed_emoji":"🛡️","tokens_out":8558,"duration_ms":70367,"temperature":0.7,"pith_summary":"This paper proposes FL-OA, a federated learning defense that hands the job of auditing device updates to a separate outsourced server holding a clean root dataset. The claim is that the task server can then aggregate updates safely without owning a root dataset itself and without assuming that less than half the devices are malicious. To make auditing reliable, FL-OA reshapes local training with a gradient ascent step and a correction term that pull benign updates closer together, and it audits only a selected fraction of coordinates using a parameter importance indicator, sidestepping the curse of dimensionality. A convergence theorem gives a bound on the distance from the learned model to the optimum, and experiments on CIFAR10, CIFAR100, and other benchmarks report higher accuracy than several existing defenses. The load-bearing condition is that the auditor's data and the devices' data come from the same distribution: under a two-class partition with disjoint classes, accuracy drops below 60% even with no attackers.","feed_headline":"A third-party auditor stops Byzantine attacks in federated learning","feed_subtitle":"Server outsources update auditing to a root-dataset holder, outperforming five defenses in experiments.","key_machinery":"The load-bearing object is the outsourced audit pipe: the outsourced server's root update $u^t_{\\mathrm{root}}$ is the reference vector; the Parameter Importance Indicator $\\mathrm{PII}^t_k[j] = |u^t_k[j]| + \\frac{|u^t_k[j]| - \\mathrm{med}(\\{|u^t_i[j]|\\})}{\\mathrm{med}(\\{|u^t_i[j]|\\}) + \\epsilon}$ picks the top-$\\kappa$ coordinates per device, and the trust score $s_k^t = \\mathrm{ReLU}(\\cos(u_k^t \\odot M_k^t, u_{\\mathrm{root}}^t))$ converts directional agreement into an aggregation weight. The aggregation rule $W^{t+1} = W^t + \\mu \\frac{\\sum_k s_k^t \\hat{u}_k^t}{\\sum_k s_k^t}$ then suppresses any update that disagrees with the root direction. Alongside it, the local-training formula combines a flat-minimum-seeking gradient ascent step with a correction term $\\hat{b}_k^t$ that accumulates historical local offset, which the paper claims makes the set of benign updates cohesive enough that cosine similarity against the root can distinguish malicious from benign.","core_discovery":"On its own terms, the paper establishes that Byzantine-robust federated aggregation can be outsourced: a third-party server holding a root dataset computes a root update, then scores every device update by the relu-clipped cosine similarity between the root update and the device update restricted to the most important coordinates, normalizes each update to the root norm, and returns the trust scores and normalized updates to the task server, which aggregates them as a trust-weighted average. The paper argues that this removes the server's need to hold a root dataset and removes the benign-majority assumption. It also claims that the local-training modification, a lookahead gradient ascent step combined with a momentum correction term, reduces divergence among benign updates, and that the parameter importance indicator selects coordinates on which benign and malicious updates separate most clearly. Theorem 1 states that, under the paper's assumptions, the global model satisfies the deviation bound $\\|W^t - W^*\\| \\le (1-\\tau)^t \\|W^{\\mathrm{init}} - W^*\\| + 12\\mu \\Delta_1/\\tau$, so after many rounds the model stays within a bounded neighborhood of the optimum.","pith_inferences":["The trust-score mechanism is direction-only after normalization: because each update is rescaled to the root norm, an attacker who can estimate the root direction could copy a benign direction while biasing the data it was trained on, a subtle poisoning strategy the paper does not analyze.","Since the PII masks are computed from the current batch of updates, a coordinated adversary that knows the audit protocol can in principle craft updates whose selected coordinates match benign statistics while poisoning the remaining coordinates; running an adaptive attack that optimizes against the exact mask would test whether the scheme's security holds against informed adversaries.","The paper's Theorem 1 error floor contains terms that grow with model dimension and shrink with the square root of the root dataset size, so for very large models the bound weakens; measuring whether FL-OA's empirical accuracy on ResNet50 still tracks the bound would clarify how the curse of dimensionality re-enters through the theory even if the audit uses fewer coordinates."],"forward_implications":["A task server can contract auditing out to any third party with a suitable clean dataset, so the practical bar for root-dataset defenses drops from 'server must own the data' to 'some organization must own the data and be reachable over the network.'","Defense no longer relies on a benign majority: experiments in the paper report usable accuracy when 50%, 60%, and 70% of devices are malicious, unlike statistical methods that degrade near the 50% mark.","Because only the top-$\\kappa$ coordinates are compared, the audit cost and the chance of two high-dimensional updates looking artificially similar both shrink; the paper's ablation shows auditing 30% of coordinates beats auditing 100%.","Without any attacker, FL-OA's accuracy remains close to plain averaged aggregation, so the defense does not buy robustness at the cost of degraded clean training.","The convergence bound implies the final model stays within a constant-radius ball around the optimum whose radius is controlled by the root dataset size, giving a concrete knob: larger clean root datasets tighten the achievable error floor."],"supporting_citations":[{"why":"It supplies the statistical-knowledge baseline that FL-OA must beat when the benign-majority assumption fails.","marker":"[3]"},{"why":"It introduces root-dataset trust bootstrapping, the approach FL-OA extends by moving the root dataset off the task server.","marker":"[8]"},{"why":"It is the third-party auditor scheme that FL-OA refines to handle non-IID root data and high-dimensional updates.","marker":"[10]"},{"why":"It provides the correction-term local training idea that FL-OA uses to reduce divergence among benign updates.","marker":"[20]"},{"why":"It contributes the proximal-term mechanism that keeps local models close to the global model during local training.","marker":"[28]"},{"why":"It motivates the gradient ascent step as a route to flat minima, reducing update inconsistency across devices.","marker":"[29]"},{"why":"It applies sharpness-aware minimization in federated settings, supporting the ascent-step design choice.","marker":"[30]"},{"why":"It analyzes the mixture of global and local models, supplying the local-offset view behind the correction term.","marker":"[32]"}],"fun_headline_variants":["Outsourced root-data auditing shields FL from Byzantine attacks","No benign-majority need: outsourced audit robustifies FL","Third-party root set gives federated learning an audit shield","Outsource update audits to defeat Byzantine devices in FL","Root dataset on a third-party server: FL's Byzantine fix"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scheme treats the outsourced server's root dataset as drawn from the same distribution as every device's local data; if the auditor's data does not overlap with what the devices actually hold, the trust scores lose their meaning and the defense collapses.","fun_headline_variants_meta":{"raw":{"variants":["Outsourced root-data auditing shields FL from Byzantine attacks","No benign-majority need: outsourced audit robustifies FL","Third-party root set gives federated learning an audit shield","Outsource update audits to defeat Byzantine devices in FL","Root dataset on a third-party server: FL's Byzantine fix"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2550,"prompt_tokens":995,"completion_tokens":1555,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":1472}},"tokens_in":611,"tokens_out":1555,"duration_ms":13643,"temperature":1.0,"reasoning_tokens":1472,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:12:34.271159+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FL-OA on CIFAR10 with a two-class non-IID partition in which each device owns two disjoint classes and the outsourced server owns a different pair of classes (the paper's Root2 configuration): Table VI reports test accuracy below 60% even with no malicious devices, so the claim that robust aggregation holds without strong assumptions is contradicted in exactly this distribution-shift regime.","supporting_citations":[{"cited_title":"Machine learning with adversaries: Byzantine tolerant gradient descent,","cited_arxiv_id":null,"evidence_quote":"It supplies the statistical-knowledge baseline that FL-OA must beat when the benign-majority assumption fails."},{"cited_title":"Fltrust: Byzantine-robust federated learning via trust bootstrapping,","cited_arxiv_id":null,"evidence_quote":"It introduces root-dataset trust bootstrapping, the approach FL-OA extends by moving the root dataset off the task server."},{"cited_title":"Using third- party auditor to help federated learning: An efficient byzantine-robust federated learning,","cited_arxiv_id":null,"evidence_quote":"It is the third-party auditor scheme that FL-OA refines to handle non-IID root data and high-dimensional updates."},{"cited_title":"Fedspeed: Larger local interval, less communication round, and higher generalization accuracy,","cited_arxiv_id":null,"evidence_quote":"It provides the correction-term local training idea that FL-OA uses to reduce divergence among benign updates."},{"cited_title":"Towards understanding sharpness-aware minimization,","cited_arxiv_id":null,"evidence_quote":"It motivates the gradient ascent step as a route to flat minima, reducing update inconsistency across devices."},{"cited_title":"Generalized federated learning via sharpness aware minimization,","cited_arxiv_id":null,"evidence_quote":"It applies sharpness-aware minimization in federated settings, supporting the ascent-step design choice."}],"review_version":2}