REVIEW 6 cited by
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recently, large language models (LLMs) have achieved remarkable breakthroughs, revolutionizing the natural language processing domain and beyond. Due to immense parameter sizes, fine-tuning these models with private data for diverse downstream tasks has become mainstream. Though federated learning (FL) offers a promising solution for fine-tuning LLMs without sharing raw data, substantial computing costs hinder its democratization. Moreover, in real-world scenarios, private client devices often possess heterogeneous computing resources, further complicating LLM fine-tuning. To combat these challenges, we propose HSplitLoRA, a heterogeneous parameter-efficient fine-tuning (PEFT) framework built on split learning (SL) and low-rank adaptation (LoRA) fine-tuning, for efficiently fine-tuning LLMs on heterogeneous client devices. HSplitLoRA first identifies important weights based on their contributions to LLM training. It then dynamically configures the decomposition ranks of LoRA adapters for selected weights and determines the model split point according to varying computing budgets of client devices. Finally, a noise-free adapter aggregation mechanism is devised to support heterogeneous adapter aggregation without introducing noise. Extensive experiments demonstrate that HSplitLoRA outperforms state-of-the-art benchmarks in training accuracy and convergence speed.
Forward citations
Cited by 6 Pith papers
-
Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring
DUAL-Health is an uncertainty-aware multimodal fusion framework that quantifies sensor noise, customizes fusion weights accordingly, and aligns modality distributions to improve outdoor health monitoring.
-
A Satellite-Ground Synergistic Large Vision-Language Model System for Earth Observation
SpaceVerse jointly decides where to run vision-language inference in LEO satellite networks and compresses task-irrelevant image regions before downlink, improving accuracy and cutting latency versus baselines.
-
Defensive Adversarial CAPTCHA: A Semantics-Driven Framework for Natural Adversarial Example Generation
DAC and BP-DAC generate 'unsourced' adversarial CAPTCHAs from semantic prompts and report transfer attack success rates above 95% on ImageNet classifiers in black-box settings.
-
GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems
GAPSL adds leader-gradient selection and direction-alignment regularization to parallel split learning, and its testbed runs show consistent accuracy and convergence-time gains over PSL, SFL, EPSL, and vanilla SL.
-
PHandover: Parallel Handover in Mobile Satellite Network
A parallel, plan-based handover using a new Satellite Synchronized Function cuts LEO satellite handover latency to about 9 ms on average in an emulated prototype.
-
HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems
HASFL jointly optimizes per-device batch sizes and neural network split points to reduce training latency in heterogeneous split federated learning, guided by a new convergence bound.
Discussion (0). Sign in to comment.