REVIEW 6 cited by
Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recently, there has been a surge in the development of advanced intelligent generative content (AIGC), especially large language models (LLMs). However, for many downstream tasks, it is necessary to fine-tune LLMs using private data. While federated learning offers a promising privacy-preserving solution to LLM fine-tuning, the substantial size of an LLM, combined with high computational and communication demands, makes it hard to apply to downstream tasks. More importantly, private edge servers often possess varying computing and network resources in real-world scenarios, introducing additional complexities to LLM fine-tuning. To tackle these problems, we design and implement an automated federated pipeline, named FedPipe, to fine-tune LLMs with minimal training cost but without adding any inference latency. FedPipe firstly identifies the weights to be fine-tuned based on their contributions to the LLM training. It then configures a low-rank adapter for each selected weight to train local low-rank adapters on an edge server, and aggregate local adapters of all edge servers to fine-tune the whole LLM. Finally, it appropriately quantizes the parameters of LLM to reduce memory space according to the requirements of edge servers. Extensive experiments demonstrate that FedPipe expedites the model training and achieves higher accuracy than state-of-the-art benchmarks.
Forward citations
Cited by 6 Pith papers
-
Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring
DUAL-Health is an uncertainty-aware multimodal fusion framework that quantifies sensor noise, customizes fusion weights accordingly, and aligns modality distributions to improve outdoor health monitoring.
-
A Satellite-Ground Synergistic Large Vision-Language Model System for Earth Observation
SpaceVerse jointly decides where to run vision-language inference in LEO satellite networks and compresses task-irrelevant image regions before downlink, improving accuracy and cutting latency versus baselines.
-
Defensive Adversarial CAPTCHA: A Semantics-Driven Framework for Natural Adversarial Example Generation
DAC and BP-DAC generate 'unsourced' adversarial CAPTCHAs from semantic prompts and report transfer attack success rates above 95% on ImageNet classifiers in black-box settings.
-
PHandover: Parallel Handover in Mobile Satellite Network
A parallel, plan-based handover using a new Satellite Synchronized Function cuts LEO satellite handover latency to about 9 ms on average in an emulated prototype.
-
HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems
HASFL jointly optimizes per-device batch sizes and neural network split points to reduce training latency in heterogeneous split federated learning, guided by a new convergence bound.
-
FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
Sparse MoE adapters with per-client expert selection and a thresholded load-balancing loss improve federated fine-tuning accuracy under non-IID data compared with LoRA baselines.
Discussion (0). Sign in to comment.