GRAM selectively trains auxiliary modules so that ablating one at inference removes a targeted capability while preserving the rest, closely tracking data-filtered models at 5x lower cost across 5 capability profiles.
Title resolution pending
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
years
2026 3representative citing papers
Multi-agent LLM systems discover new Transformer and hybrid architectures that outperform Llama 3.2 at 1B scale and approach human SOTA on long-range benchmarks.
At tiny scale, MoE transformers lower validation loss versus dense models when active parameters match but raise it when total stored parameters match.
citing papers explorer
-
Modular Pretraining Enables Access Control
GRAM selectively trains auxiliary modules so that ablating one at inference removes a targeted capability while preserving the rest, closely tracking data-filtered models at 5x lower cost across 5 capability profiles.
-
Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Design
Multi-agent LLM systems discover new Transformer and hybrid architectures that outperform Llama 3.2 at 1B scale and approach human SOTA on long-range benchmarks.
-
Dense vs Sparse Pretraining at Tiny Scale: Active-Parameter vs Total-Parameter Matching
At tiny scale, MoE transformers lower validation loss versus dense models when active parameters match but raise it when total stored parameters match.