GeMoE adaptively sets the number of experts per token via gating entropy, retaining 99.5% of static-routing performance while raising average sparsity by 36.5%.
Octavius: Mitigating task interference in mllms via lora-moe
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
representative citing papers
MoRAM learns continually by adding small rank-1 adapters that act as associative memory items, using input-key similarity to retrieve and mix only the relevant adapters at test time.
LoRA-Mixer routes modular LoRA experts into attention projection matrices with an adaptive Routing Specialization Loss to improve multi-task performance while using fewer trainable parameters than prior LoRA-MoE methods.
citing papers explorer
-
GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models
GeMoE adaptively sets the number of experts per token via gating entropy, retaining 99.5% of static-routing performance while raising average sparsity by 36.5%.
-
Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts
MoRAM learns continually by adding small rank-1 adapters that act as associative memory items, using input-key similarity to retrieve and mix only the relevant adapters at test time.
-
LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
LoRA-Mixer routes modular LoRA experts into attention projection matrices with an adaptive Routing Specialization Loss to improve multi-task performance while using fewer trainable parameters than prior LoRA-MoE methods.