Pith. sign in

REVIEW 3 major objections 5 minor 28 references

HermesHFL jointly optimizes hierarchical LLM fine-tuning, selective unlearning, and incentives so clients can leave, erase their influence, and rejoin without full retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 04:57 UTC pith:YEZDDTY2

load-bearing objection Solid systems package for hierarchical leave–unlearn–rejoin LoRA with incentives and a usable bilevel solver; the rejoin claim rests on a thin GA+KL check that never audits residual hierarchical influence. the 3 major comments →

arxiv 2607.11528 v1 pith:YEZDDTY2 submitted 2026-07-13 cs.CE

HermesHFL: Incentive-Compatible Hierarchical Federated Unlearning for Dynamic LLM Fine-Tuning

Chenxi Sun , Minghui Liwang , Wusi He , Zhang Liu , Yuhan Su , Seyyedali Hosseinalipour , Xianbin Wang , Yiguang Hong This is my paper
classification cs.CE
keywords hierarchical federated learningmachine unlearninglarge language modelsLoRAincentive mechanismdynamic clientsbilevel evolutionary optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Fine-tuning large language models across hierarchical federated networks is hard when clients must be able to withdraw, have their contributions erased, and later rejoin, because updates pass through multiple aggregation layers and LoRA parameters stay tightly coupled. HermesHFL treats cloud, edge servers, and devices as a president–manager–worker market that co-decides participation, edge association, payments, unlearning penalties, and quality targets under a shared budget. The resulting mixed continuous–discrete bilevel problem is solved by Neogen: CMA-ES tunes continuous incentives while a CHC evolutionary search handles binary assignment, with a neural surrogate that predicts good discrete answers and shrinks the search. On GPT-2 LoRA fine-tuning of SST-2 and AGNews, the system keeps test accuracy competitive with strong baselines, balances manager and worker utilities, recovers within one or two global rounds after gradient-ascent unlearning, and cuts wall-clock cost versus full retrain. A reader who cares about privacy rules and real device churn would see a practical path to leave–unlearn–rejoin without collapsing model utility or incentive fairness.

Core claim

The paper establishes that selective unlearning, dynamic leave–unlearn–rejoin, and incentive-compatible contracts can be formulated as one hierarchical optimization for LoRA-based LLM fine-tuning, and that a neural-guided bilevel evolutionary solver (Neogen) yields higher or more balanced model utility, unlearning effectiveness, multi-agent utilities, and resource efficiency than pure evolutionary, simulated-annealing, greedy, and random baselines, with a rejoin policy matching retrain-level accuracy at substantially lower runtime.

What carries the argument

Neogen: a neural-guided bilevel evolutionary optimizer that pairs CMA-ES for continuous incentive and penalty variables with CHC (cross-generational elitist selection, heterogeneous recombination, cataclysmic mutation) for binary worker selection and manager association, using a trained neural surrogate to initialize and restrict the discrete search.

Load-bearing premise

The argument assumes that gradient-ascent steps on LoRA adapters, verified only by average KL divergence on the leaving client’s local samples plus a profile refresh before rejoin, actually erase that client’s multi-stage hierarchical influence without restoring forgotten knowledge or permanently damaging global utility.

What would settle it

After a claimed successful unlearning (KL above the paper’s 0.05 threshold), run membership-inference and retrain-distance checks on the global LoRA model for the forgotten client’s samples, then allow rejoin and measure whether forgotten labels reappear or accuracy permanently drops; failure of those audits or reappearance of erased knowledge would refute the unlearning guarantee.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Hierarchical federated LLM systems can honor erasure requests without restarting global training from scratch.
  • Recycling unlearning penalties into residual manager budgets raises manager utility when clients rejoin with refreshed profiles.
  • Neural surrogates can cut the fitness-evaluation burden of bilevel mixed continuous–discrete participation markets.
  • Leave–unlearn–rejoin becomes an operable client lifecycle rather than a permanent exit from hierarchical training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same bilevel evolutionary pattern may transfer to asynchronous or multi-modal hierarchical FL where clients stream data over time.
  • Regulators may still demand stronger erasure audits than local KL thresholds before accepting LoRA gradient-ascent unlearning as GDPR-grade.
  • Contribution-proportional budget collapse of the president layer could apply to other hierarchical multi-agent resource markets outside federated learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes HermesHFL, a hierarchical federated framework for LoRA-based LLM fine-tuning that supports selective unlearning and leave–unlearn–rejoin client dynamics under incentive contracts among a president (cloud), managers (edges), and workers (clients). It formulates a multi-objective market problem over participation, association, payments, and unlearning penalties, then reduces the three-layer problem to a two-layer form via contribution-proportional budget allocation (Eq. 32). Neogen solves the resulting mixed continuous–discrete bilevel problem with CMA-ES on incentives, CHC on binary selection/association, and a neural surrogate that restricts lower-level search. Unlearning is performed by gradient ascent on LoRA factors, verified by average KL(P_old ∥ P_new) > δ on the unlearning worker’s local samples, after which the worker’s attributes are refreshed and rejoin is allowed. Experiments on GPT-2 (124M) with SST-2 and AGNews (10×3 and 20×4 setups) compare Neogen to Pure EA, SA, Greedy, and Random, and compare Rejoin vs No Rejoin vs Retrain, reporting accuracy, manager/worker utilities, fitness evaluations, and wall-clock time.

Significance. If the claims hold, the work is a useful systems-level contribution at the intersection of hierarchical FL, PEFT for LLMs, machine unlearning, and incentive design. The leave–unlearn–rejoin lifecycle with contracts and residual-budget recycling is a practically relevant modeling step beyond flat FUL and static PEFT-FL. The bilevel evolutionary solver with a neural surrogate is a concrete engineering response to a mixed-integer hierarchical market problem, and the ablations (optimization mechanism and unlearning strategy) give a clear empirical picture of accuracy–utility–cost tradeoffs on small GPT-2 tasks. Strengths include an explicit problem reduction (Appendix A), a stated convergence sketch for Neogen (§6.3), and multi-metric reporting (Acc, MgU, WkU, Feval/r, Time). The main significance risk is that unlearning effectiveness is only weakly audited relative to the hierarchical multi-stage claim, so the “Rejoin ≈ Retrain at lower cost” headline is only partially supported.

major comments (3)
  1. §5.1.4 (Eqs. 17–24) and §7.2–7.3 (Tables 5–6, Figs. 6–8): The central claim of unlearning effectiveness and Rejoin matching Retrain rests on gradient ascent on LoRA factors plus a single local KL(P_old ∥ P_new) > δ=0.05 on the unlearning worker’s own samples, followed by attribute refresh. Unlearning updates are still FedAvg-aggregated with remaining workers and then globally re-aggregated (Eqs. 19–23). The manuscript reports only that KL exceeds threshold at g=2 and accuracy rebounds in 1–2 rounds. There is no membership-inference attack, retrain-distance / prediction-consistency audit on forgotten data, or post-rejoin leakage check that hierarchical residual influence is not re-injected. Without at least one such audit, “unlearning effectiveness” and the Rejoin≈Retrain part of the Abstract/§8 claim are overstated even if accuracy looks competitive.
  2. §7.1–7.2 and Tables 2–4: The Abstract and §1.2 claim consistent outperformance over “state-of-the-art baselines” in model utility, unlearning effectiveness, stability, and resource efficiency. The reported baselines are Pure EA (ablation of Neogen), SA, Greedy, and Random with a fixed pricing rule p=3×cost. These are optimization/search baselines, not published hierarchical federated unlearning or incentive-aware FUL methods discussed in §2 (e.g., FedRecovery, KNOT, incentive FUL works). On SST-2 Set#1, SA and Greedy actually report higher Acc than Neogen (0.877/0.834 vs 0.795); Neogen’s advantage is mainly balanced positive MgU/WkU and sometimes lower wall-clock. The SOTA claim should be narrowed to the methods actually compared, or true HFUL/incentive baselines should be added.
  3. §5.3 Eq. (32) and Appendix A: Collapsing the president into contribution-proportional budget allocation assumes president and managers share sufficiently aligned objectives. This is a modeling choice, not a derived equilibrium. The paper should either (i) state clearly that U_Pres is no longer independently optimized and that results are conditional on this alignment, or (ii) provide a sensitivity check when budget shares deviate from contribution proportions (e.g., fixed or adversarial manager budgets). As written, constraint (C9) and the two-level problem P1 bake in the alignment axiom without testing its necessity for the reported utility balance.
minor comments (5)
  1. Notation inconsistency: worker utility uses ρ_fgt and Δξ in Eq. (25), while reputation is defined via ˜x / ex indicators in Eq. (26); several places mix (g) superscripts and parenthetical forms (e.g., U^(Mana,g) vs U_Mana,(g)).
  2. Fig. 1 caption and Neogen flowchart are dense; a short algorithm-box cross-reference to Algorithm 1 in the figure caption would help readers map “Loop 1/2” to the text.
  3. §7.1: GPT-2 124M with r=8 on SST-2/AGNews is a limited LLM stress test for “strong parameter coupling”; a sentence on why results should transfer to larger models (or an explicit limitation) would strengthen §8.
  4. Typos / wording: “NEural netwOrk Guided dual network Evolutionary optimizatioN”; “cataclysmic” is standard CHC terminology but should be defined once for non-EA readers; “aloys sun@outlook.com” formatting in affiliations.
  5. Table 1 lists many free hyperparameters (δ, λ_mana, λ_pres, H, α, eη, p_mut, r_ham, κ=3 for baselines). A short sensitivity paragraph or appendix on δ and κ would reduce the free-parameter concern.

Circularity Check

0 steps flagged

No significant circularity: HermesHFL/Neogen is a self-contained systems formulation and empirical comparison, not a first-principles derivation that reduces to its inputs.

full rationale

This paper defines utilities, contracts, hierarchical LoRA training/unlearning dynamics, and a mixed-integer bilevel program (P0→P1), then solves it with a hybrid evolutionary method (CMA-ES + CHC + neural surrogate) and reports experimental comparisons. Nothing in the load-bearing chain is equivalent to its inputs by construction. The P0→P1 collapse via proportional budget allocation (eq. 32) is an explicit modeling simplification justified by aligned president/manager objectives (Appendix A); it does not claim to derive empirical accuracy or unlearning success from a tautology. Neogen’s surrogate approximates the lower-level map from data the optimizer itself generates—standard surrogate-assisted search, not a fitted quantity re-labeled as an independent prediction. Convergence arguments (§6.3) use finite discrete search, elitist monotonicity, the external universal-approximation theorem [28], and known CMA-ES properties; they do not smuggle the claimed experimental superiority into the proof. Baselines (Pure EA, SA, Greedy, Random; Rejoin/No-Rejoin/Retrain) are independent of Neogen’s definition of success. Concerns about whether KL>δ plus profile refresh truly erases hierarchical influence are correctness/verification risks, not circularity. No self-definitional loop, no fitted-input-as-prediction, no load-bearing uniqueness theorem imported from the authors, and no renaming of a known result as a derivation.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 3 invented entities

The central empirical claim rests on many hand-chosen system and optimizer hyperparameters, standard FL/LoRA/unlearning domain assumptions, and several paper-specific market constructs (president–manager–worker contracts, reputation-weighted unlearning probability, contribution-proportional budget collapse). No free parameters are fitted to claim a universal constant; they are configuration knobs that still control reported utilities and accuracy. Invented entities are architectural/algorithmic, not physical particles, but they lack independent external validation beyond this paper’s simulations.

free parameters (6)
  • KL unlearning threshold δ
    Success of unlearning and rejoin eligibility is gated by KL>δ; default δ=0.05 is chosen by hand and directly controls the central unlearning claim.
  • λ_mana and λ_pres utility normalizers
    Set to 8.0 and 0.5; they scale performance vs money in manager and president utilities and therefore shape all reported MgU/WkU tradeoffs.
  • Unlearning memory window H and decay α
    H=5, α=0.6 define reputation ρ_fgt and thus penalties and participation incentives (eq. 26).
  • CMA-ES/CHC/NN search hyperparameters
    pop_u, pop_l, σ_u, T, p_mut, r_ham, NN size/LR, etc., are hand-set and determine whether Neogen finds the reported contracts.
  • Unlearning LR eη, max GA epochs, weight scale
    eη=1e-3, max 5 epochs, weight scale 0.1 control how aggressively GA erases and thus measured KL and post-unlearning accuracy.
  • Baseline fixed payment multiplier κ=3
    Non-Neogen methods use p=3×cost chosen as midpoint of Neogen’s learned range; this choice strongly affects comparative WkU/MgU.
axioms (6)
  • ad hoc to paper President and managers share sufficiently aligned objectives that three-layer optimization can be collapsed via contribution-proportional budget allocation (eq. 32).
    Appendix A and §5.3 justify the reduction; if objectives conflict, the transformed problem P1 does not preserve the original market.
  • domain assumption FedAvg-style separate aggregation of LoRA A and B matrices is a valid hierarchical PEFT update rule.
    Used throughout §5.1; known aggregation-bias issues in heterogeneous LoRA are acknowledged in related work but not resolved here.
  • domain assumption Gradient ascent on local loss plus KL divergence on local predictive distributions is an adequate operational definition of machine unlearning for this system.
    §3 and §5.1.4; standard in some FUL work but weaker than retrain equivalence or formal privacy audits.
  • domain assumption Workers are individually rational (u_work≥0) and strategic only through the modeled utility, reputation, and contract terms.
    Constraints (C7) and utility (25); free-riding beyond the model is assumed controlled by incentives.
  • domain assumption Current-round training quality can be estimated from prior-round observations under mild temporal data variation.
    Footnote in §4.1 supporting contract design under dynamics.
  • standard math CMA-ES and CHC with elitist selection converge to useful local optima for the bilevel mixed-integer problem.
    §6.3 invokes standard evolutionary convergence arguments and universal approximation for the NN surrogate.
invented entities (3)
  • HermesHFL market (president–manager–worker with leave–unlearn–rejoin contracts) no independent evidence
    purpose: Unify hierarchical aggregation, incentives, selective unlearning, and reintegration for LoRA LLM fine-tuning.
    Core framework of the paper; no independent deployment evidence outside the reported simulations.
  • Neogen (neural-guided CMA-ES + CHC bilevel optimizer) no independent evidence
    purpose: Solve mixed continuous incentive and discrete participation/association decisions efficiently.
    Primary algorithmic contribution; performance claims are internal to this paper’s ablations.
  • Reputation score ρ_fgt via exponentially weighted unlearning window no independent evidence
    purpose: Estimate unlearning probability and scale penalties/privacy terms in utilities.
    Eq. (26); paper-specific behavioral summary statistic without external validation.

pith-pipeline@v1.1.0-grok45 · 30514 in / 4291 out tokens · 49063 ms · 2026-07-14T04:57:41.755484+00:00 · methodology

0 comments
read the original abstract

Hierarchical federated unlearning (HFUL) for large language model (LLM) fine-tuning faces significant challenges due to hierarchical aggregation, dynamic client participation, and strong parameter coupling in LLM adaptation. Selectively removing client contributions is particularly difficult because model updates propagate across multiple aggregation stages while unlearning requests may coincide with client departures and rejoining. To address these issues, we propose **HermesHFL**, a hierarchical federated learning framework that supports selective unlearning, dynamic client participation, and client reintegration for scalable LLM fine-tuning via parameter-efficient fine-tuning (PEFT) with LoRA. We formulate a unified optimization problem that jointly models client participation, edge association, incentive allocation, and unlearning under heterogeneous client behaviors. To solve this problem efficiently, we develop **Neogen**, a neural-guided bilevel evolutionary optimization framework that combines CMA-ES for continuous incentive optimization with a CHC-based evolutionary mechanism for discrete participation and association decisions. A neural surrogate further accelerates optimization and improves search efficiency. Extensive experiments on LLM fine-tuning tasks demonstrate that HermesHFL consistently outperforms state-of-the-art baselines in model utility, unlearning effectiveness, convergence stability, and resource efficiency.

Figures

Figures reproduced from arXiv: 2607.11528 by Chenxi Sun, Minghui Liwang, Seyyedali Hosseinalipour, Wusi He, Xianbin Wang, Yiguang Hong, Yuhan Su, Zhang Liu.

Figure 1
Figure 1. Figure 1: Framework of our HermesHFL (the left box) and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Global and Client-Side Architectural Workflow for [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Performance on SST-2 [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance on AGNews TABLE 2: Optimization Mechanism Comparison on SST-2 (Set #1, β (sum,g) = 30, 6 rounds) Method Acc MgU WkU Feval/r Time (s) Neogen .795 3.93 6.12 2400 3188 Pure EA .606 −0.73 7.17 3200 2232 SA .877 13.93 1.97 200 3622 Greedy .834 3.02 15.08 10 3808 Random .845 −0.68 17.25 1 4468 [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance on AGNews, Set#2 TABLE 3: Optimization Mechanism Comparison on AG￾News (Set #1, β (sum,g) = 30, 6 rounds) Method Acc MgU WkU Feval/r Time (s) Neogen .891 3.93 6.12 2400 3528 Pure EA .770 −0.73 7.17 3200 2300 SA .866 −3.18 19.05 200 3262 Greedy .874 3.02 15.08 10 3441 Random .850 −0.68 17.25 1 4592 TABLE 4: Optimization Mechanism Comparison on SST-2 (Set #2, β (sum,g) = 60, 6 rounds) Method Acc … view at source ↗
Figure 8
Figure 8. Figure 8: Utility and Accuracy Dynamics Under Unlearning [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 6
Figure 6. Figure 6: Utility and Accuracy Dynamics Under Unlearning [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Utility and Accuracy Dynamics Under Unlearning [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 2 linked inside Pith

  1. [1]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” inProceedings of the 20th International Confer- ence on Artificial Intelligence and Statistics. PMLR, 2017, pp. 1273– 1282

    2017

  2. [2]

    Hierar- chical federated learning across heterogeneous cellular networks,

    M. S. H. Abad, E. Ozfatura, D. GUndUz, and O. Ercetin, “Hierar- chical federated learning across heterogeneous cellular networks,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 8866–8870

    2020

  3. [3]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

    2017

  4. [4]

    Abdisarabshali, F

    P . Abdisarabshali, F. Nadimi, K. Borazjani, N. Khosravan, M. Li- wang, W. Ni, D. Niyato, M. Langberg, and S. Hosseinalipour, “Hi- erarchical federated foundation models over wireless networks for multi-modal multi-task intelligence: Integration of edge learning with d2d/p2p-enabled fog learning architectures,”IEEE Commu- nications Magazine, 2026

    2026

  5. [5]

    European Commission, “Regulation (eu) 2016/679 of the european parliament and of the council on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (gdpr),” Official Journal of the European Union, 2016

    2016

  6. [6]

    Federated unlearning via class- discriminative pruning,

    J. Wang, S. Guo, X. Xie, and H. Qi, “Federated unlearning via class- discriminative pruning,” inProceedings of the ACM Web Conference 2022, 2022, p. 622–632

    2022

  7. [7]

    Federated unlearning with gradient descent and conflict mitiga- tion,

    Z. Pan, Z. Wang, C. Li, K. Zheng, B. Wang, X. Tang, and J. Zhao, “Federated unlearning with gradient descent and conflict mitiga- tion,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 19, 2025, pp. 19 804–19 812

    2025

  8. [8]

    Fedrecovery: Differentially private machine unlearning for federated learning frameworks,

    L. Zhang, T. Zhu, H. Zhang, P . Xiong, and W. Zhou, “Fedrecovery: Differentially private machine unlearning for federated learning frameworks,”IEEE Transactions on Information Forensics and Secu- rity, vol. 18, pp. 4732–4746, 2023

    2023

  9. [9]

    Framu: Attention-based machine unlearning using federated reinforce- ment learning,

    T. Shaik, X. Tao, L. Li, H. Xie, T. Cai, X. Zhu, and Q. Li, “Framu: Attention-based machine unlearning using federated reinforce- ment learning,”IEEE Transactions on Knowledge and Data Engineer- ing, vol. 36, no. 10, pp. 5153–5167, 2024

    2024

  10. [10]

    F2ul: Fairness-aware federated unlearning for data trading,

    W. Su, P . Wang, Z. Wang, A. Muhammad, T. Tao, X. Tong, and Q. Zhang, “F2ul: Fairness-aware federated unlearning for data trading,”IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 13 539–13 555, 2024

    2024

  11. [11]

    Gru: Mitigating the trade-off between unlearning and retention for llms,

    Y. Wang, Q. Wang, F. Liu, W. Huang, Y. Du, X. Du, and B. Han, “Gru: Mitigating the trade-off between unlearning and retention for llms,”arXiv preprint arXiv:2503.09117, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Multi- objective large language model unlearning,

    Z. Pan, S. Zhang, Y. Zheng, C. Li, Y. Cheng, and J. Zhao, “Multi- objective large language model unlearning,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5

    2025

  13. [13]

    Asynchronous federated unlearning,

    N. Su and B. Li, “Asynchronous federated unlearning,” inIEEE INFOCOM 2023 - IEEE Conference on Computer Communications, 2023, pp. 1–10

    2023

  14. [14]

    Fate-llm: A industrial grade federated learning framework for large language models,

    T. Fan, Y. Kang, G. Ma, W. Chen, W. Wei, L. Fan, and Q. Yang, “Fate-llm: A industrial grade federated learning framework for large language models,”arXiv preprint arXiv:2310.10049, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Fedalora: Adaptive local lora aggregation for personalized federated learn- ing in llm,

    X. Yi, C. Hu, B. Cai, H. Huang, Y. Chen, and K. Wang, “Fedalora: Adaptive local lora aggregation for personalized federated learn- ing in llm,”IEEE Internet of Things Journal, vol. 12, no. 24, pp. 51 854–51 865, 2025

    2025

  16. [16]

    Fedbiot: Llm local fine- tuning in federated learning without full model,

    F. Wu, Z. Li, Y. Li, B. Ding, and J. Gao, “Fedbiot: Llm local fine- tuning in federated learning without full model,” inProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 3345–3355

    2024

  17. [17]

    Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations,

    Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu, and A. Li, “Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations,”Advances in Neural Information Processing Systems, vol. 37, pp. 22 513–22 533, 2024

    2024

  18. [18]

    Lora-fair: Federated lora fine-tuning with aggregation and initialization refinement,

    J. Bian, L. Wang, L. Zhang, and J. Xu, “Lora-fair: Federated lora fine-tuning with aggregation and initialization refinement,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 3737–3746

    2025

  19. [19]

    Resource-efficient personal large language models fine-tuning with collaborative edge computing,

    S. Ye, B. Ouyang, T. Qian, L. Zeng, J. Li, J. Du, X. Chu, G. Xing, and X. Chen, “Resource-efficient personal large language models fine-tuning with collaborative edge computing,”IEEE Transactions on Parallel and Distributed Systems, vol. 37, no. 3, pp. 680–696, 2026

    2026

  20. [20]

    Mira: A method of federated multi-task learning for large lan- guage models,

    A. Elbakary, C. Ben Issaid, T. ElBatt, K. Seddik, and M. Bennis, “Mira: A method of federated multi-task learning for large lan- guage models,”IEEE Networking Letters, vol. 7, no. 3, pp. 171–175, 2025

    2025

  21. [21]

    Edge-assisted federated learning for large language models in iot sensor systems,

    H. Chen, X. Yuan, and H. Li, “Edge-assisted federated learning for large language models in iot sensor systems,”IEEE Journal of Selected Areas in Sensors, vol. 3, pp. 125–138, 2026

    2026

  22. [22]

    Adaptive rank allocation for federated parameter-efficient fine-tuning of language models,

    F. Wu, J. Hu, G. Min, and S. Wang, “Adaptive rank allocation for federated parameter-efficient fine-tuning of language models,” IEEE Transactions on Computers, vol. 75, no. 4, pp. 1650–1663, 2026

    2026

  23. [23]

    A sustainable incentive scheme for federated learning,

    H. Yu, Z. Liu, Y. Liu, T. Chen, M. Cong, X. Weng, D. Niyato, and Q. Yang, “A sustainable incentive scheme for federated learning,” IEEE Intelligent Systems, vol. 35, no. 4, pp. 58–69, 2020

    2020

  24. [24]

    Incentive and dynamic client selection for federated unlearning,

    Y. Lin, Z. Gao, H. Du, D. Niyato, J. Kang, and X. Liu, “Incentive and dynamic client selection for federated unlearning,” inProceed- ings of the ACM Web Conference 2024, 2024, p. 2936–2944

    2024

  25. [25]

    The price of forgetting: Incentive mechanism design for machine unlearning,

    Y. Cui and M. H. Cheung, “The price of forgetting: Incentive mechanism design for machine unlearning,”IEEE Transactions on Mobile Computing, vol. 24, no. 11, pp. 11 852–11 864, 2025

    2025

  26. [26]

    Incentivized federated learning and unlearning,

    N. Ding, Z. Sun, E. Wei, and R. Berry, “Incentivized federated learning and unlearning,”IEEE Transactions on Mobile Computing, vol. 24, no. 9, pp. 8794–8810, 2025

    2025

  27. [27]

    The chc adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombina- tion,

    L. J. Eshelman, “The chc adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombina- tion,” inFoundations of genetic algorithms. Elsevier, 1991, vol. 1, pp. 265–283

    1991

  28. [28]

    Multilayer feedfor- ward networks are universal approximators,

    H. W. Kurt Hornik, Maxwell Stinchcombe, “Multilayer feedfor- ward networks are universal approximators,”Neural Networks, vol. 2, no. 5, pp. 359–366, 1989. APPENDIXA ILLUSTRATIVEEXAMPLES OFALIGNEDOBJEC- TIVES The transformation from the original three-layer optimiza- tion problem to a two-layer framework is motivated by the observation that upper-level and...

    1989