FlexServe introduces recallable secure memory and NPU to enable cooperative secure LLM inference on mobile devices, reporting 10.05X TTFT speedup over a basic TrustZone strawman.
hub
The rising costs of training frontier AI models
21 Pith papers cite this work, alongside 22 external citations. Polarity classification is still indexing.
21
Pith papers citing it
22
external citations · Pith
abstract
The costs of training frontier AI models have grown dramatically in recent years, but there is limited public data on the magnitude and growth of these expenses. This paper develops a detailed cost model to address this gap, estimating training costs using three approaches that account for hardware, energy, cloud rental, and staff expenses. The analysis reveals that the amortized cost to train the most compute-intensive models has grown precipitously at a rate of 2.4x per year since 2016 (90% CI: 2.0x to 2.9x). For key frontier models, such as GPT-4 and Gemini, the most significant expenses are AI accelerator chips and staff costs, each costing tens of millions of dollars. Other notable costs include server components (15-22%), cluster-level interconnect (9-13%), and energy consumption (2-6%). If the trend of growing development costs continues, the largest training runs will cost more than a billion dollars by 2027, meaning that only the most well-funded organizations will be able to finance frontier AI models.
hub tools
citation-role summary
background 2
citation-polarity summary
roles
background 2representative citing papers
zkComposer decomposes monolithic zkML proofs into parallel sub-proofs linked by shared boundary commitments, yielding up to 6.84× lower prover time on GPT-2 without new cryptographic primitives.
Page-granular Flex-Mem and switchable Flex-NPU cut TrustZone LLM TTFT by ~10× vs a CMA strawman and ~2.4× vs a pipelined secure-NPU strawman on RK3588.
The authors introduce definitions and a qualitative model of AI sovereignty that identifies multi-scale contributors and leverage points nations can target through kinetic and non-kinetic actions to influence AI-driven national power.
Proposes zkVM-based protocol for verifiable frontier AI pre-training with committed specs, network observations, Merkle commitments, and FP precompiles, estimating 36-month POC at single-digit overhead.
Maps common low-compute research strategies for foundation models onto statistical, internal, external, and construct validity threats via a causal-inference lens.
Exact RMT-derived formula for CoT generalization error in linear ICL reveals phase transition between exponential/polynomial improvement, saturation, and overthinking regimes depending on depth, pretraining, and context length.
Rescaled ASGD recovers convergence to the true global objective by rescaling worker stepsizes proportional to computation times, matching the known time lower bound in the leading term under non-convex smoothness and bounded heterogeneity.
Cross-model semantic disagreement adds an epistemic uncertainty term that improves total uncertainty estimation over self-consistency alone, helping flag confident errors in LLMs.
Introduces Switching Efficiency (η) decomposed into data, routing efficiency, and port utilization factors to analyze and improve communication bottlenecks in AI data center networks for LLM training.
Lightweight numerical bandits on text embeddings match or exceed LLM accuracy in contextual bandits at a fraction of the cost, with an embedding-based diagnostic to choose between them.
A graph-spectral importance score based on layer-wise structural distortion between pre- and post-activation neuron graphs identifies removable neurons for iterative pruning without intermediate updates, followed by recovery fine-tuning.
FCP shards sequences at block level with flexible P2P communication and bin-packing to achieve near-linear scaling up to 256 GPUs and 1.13x-2.21x higher attention MFU in foundation model pre-training.
AI agents convert compute capital into cognitive labor units, so on substitutable tasks the competitive human wage is bounded above by relative productivity times compute intensity times the rental rate of compute.
Stronger physics priors in neural networks for spatio-temporal shear flow forecasting yield substantially lower training carbon footprints than weak or no priors, though inference savings are less consistent.
A scaling law model derived from roofline analysis and a speedup-based efficiency factor predicts training energy for BERT models across GPU parallelism configurations.
A review of AI sustainability studies finds inconsistent life cycle definitions and predominant reliance on coarse CO2e proxies, with limited coverage of water, materials, and multi-impact assessments.
Authors propose a five-pillar sovereignty framework mapped to a five-layer AI stack with 26 components to identify gaps in EU frontier AI policy.
Open-weight models have ended the foundation model era by eliminating pre-training as a durable moat and enabling sovereign AI control through direct access to model weights.
A survey taxonomy of LLMs identifies three scaling crises and six efficiency paradigms while tracing the shift from generation to tool-using agents.
Fine-tuned LLaMA 3.1-8B variants for the energy sector outperform the base model on domain QA benchmarks, with LoRA delivering similar gains at lower training cost.
citing papers explorer
-
FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation
FlexServe introduces recallable secure memory and NPU to enable cooperative secure LLM inference on mobile devices, reporting 10.05X TTFT speedup over a basic TrustZone strawman.
-
zkComposer: Decomposing Proof Construction to Scale zkML
zkComposer decomposes monolithic zkML proofs into parallel sub-proofs linked by shared boundary commitments, yielding up to 6.84× lower prover time on GPT-2 without new cryptographic primitives.
-
FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation
Page-granular Flex-Mem and switchable Flex-NPU cut TrustZone LLM TTFT by ~10× vs a CMA strawman and ~2.4× vs a pipelined secure-NPU strawman on RK3588.
-
AI Sovereignty: A Qualitative Model of Strategic Competition as AI Becomes an Instrument of National Power
The authors introduce definitions and a qualitative model of AI sovereignty that identifies multi-scale contributors and leverage points nations can target through kinetic and non-kinetic actions to influence AI-driven national power.
-
Zero knowledge verification for frontier AI training is possible
Proposes zkVM-based protocol for verifiable frontier AI pre-training with committed specs, network observations, Merkle commitments, and FP precompiles, estimating 36-month POC at single-digit overhead.
-
Validity Threats for Foundation Model Research
Maps common low-compute research strategies for foundation models onto statistical, internal, external, and construct validity threats via a causal-inference lens.
-
An Asymptotic Theory of Chain-of-Thought in In-Context Learning
Exact RMT-derived formula for CoT generalization error in linear ICL reveals phase transition between exponential/polynomial improvement, saturation, and overthinking regimes depending on depth, pretraining, and context length.
-
Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
Rescaled ASGD recovers convergence to the true global objective by rescaling worker stepsizes proportional to computation times, matching the known time lower bound in the leading term under non-convex smoothness and bounded heterogeneity.
-
Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification
Cross-model semantic disagreement adds an epistemic uncertainty term that improves total uncertainty estimation over self-consistency alone, helping flag confident errors in LLMs.
-
Switching Efficiency: A Novel Framework for Dissecting AI Data Center Network Efficiency
Introduces Switching Efficiency (η) decomposed into data, routing efficiency, and port utilization factors to analyze and improve communication bottlenecks in AI data center networks for LLM training.
-
When Do We Need LLMs? A Diagnostic for Language-Driven Bandits
Lightweight numerical bandits on text embeddings match or exceed LLM accuracy in contextual bandits at a fraction of the cost, with an embedding-based diagnostic to choose between them.
-
Spectral structural distortion reveals redundant neurons in neural networks
A graph-spectral importance score based on layer-wise structural distortion between pre- and post-activation neuron graphs identifies removable neurons for iterative pruning without intermediate updates, followed by recovery fine-tuning.
-
Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP
FCP shards sequences at block level with flexible P2P communication and bin-packing to achieve near-linear scaling up to 256 GPUs and 1.13x-2.21x higher attention MFU in foundation model pre-training.
-
Who Prices Cognitive Labor in the Age of Agents? Compute-Anchored Wages
AI agents convert compute capital into cognitive labor units, so on substitutable tasks the competitive human wage is bounded above by relative productivity times compute intensity times the rental rate of compute.
-
Physics Priors Offer Useful Accuracy-Carbon Trade-Offs in Spatio-Temporal Forecasting
Stronger physics priors in neural networks for spatio-temporal shear flow forecasting yield substantially lower training carbon footprints than weak or no priors, though inference savings are less consistent.
-
The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling Model
A scaling law model derived from roofline analysis and a speedup-based efficiency factor predicts training energy for BERT models across GPU parallelism configurations.
-
From Cradle to Cloud: A Life Cycle Review of AI's Environmental Footprint
A review of AI sustainability studies finds inconsistent life cycle definitions and predominant reliance on coarse CO2e proxies, with limited coverage of water, materials, and multi-impact assessments.
-
Beware of GeeksBearing Gifts: Building True EU Frontier AI Sovereignty
Authors propose a five-pillar sovereignty framework mapped to a five-layer AI stack with 26 components to identify gaps in EU frontier AI policy.
-
The End of the Foundation Model Era: Open-Weight Models, Sovereign AI, and Inference as Infrastructure
Open-weight models have ended the foundation model era by eliminating pre-training as a durable moat and enabling sovereign AI control through direct access to model weights.
-
LLMOrbit: A Circular Taxonomy of Large Language Models -From Scaling Walls to Agentic AI Systems
A survey taxonomy of LLMs identifies three scaling crises and six efficiency paradigms while tracing the shift from generation to tool-using agents.
-
Towards EnergyGPT: A Large Language Model Specialized for the Energy Sector
Fine-tuned LLaMA 3.1-8B variants for the energy sector outperform the base model on domain QA benchmarks, with LoRA delivering similar gains at lower training cost.