Sparse MoE adapters with per-client expert selection and a thresholded load-balancing loss improve federated fine-tuning accuracy under non-IID data compared with LoRA baselines.
Scalable Artificial Intelligence for Science: Perspectives, Methods and Exemplars
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In a post-ChatGPT world, this paper explores the potential of leveraging scalable artificial intelligence for scientific discovery. We propose that scaling up artificial intelligence on high-performance computing platforms is essential to address such complex problems. This perspective focuses on scientific use cases like cognitive simulations, large language models for scientific inquiry, medical image analysis, and physics-informed approaches. The study outlines the methodologies needed to address such challenges at scale on supercomputers or the cloud and provides exemplars of such approaches applied to solve a variety of scientific problems.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
FFT-MoE: Efficient Federated Fine-Tuning for Foundation Models via Large-scale Sparse MoE under Heterogeneous Edge
Sparse MoE adapters with per-client expert selection and a thresholded load-balancing loss improve federated fine-tuning accuracy under non-IID data compared with LoRA baselines.