REVIEW 27 cited by
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
read the original abstract
In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large
Forward citations
Cited by 27 Pith papers
-
ARGUS: Production-Scale Tracing and Performance Diagnosis for over 10,000-GPU Clusters
ARGUS delivers always-on, sub-2% overhead, multi-layer tracing plus 3700× kernel compression and progressive diagnosis that localizes fail-slow faults in 10k-GPU LLM training clusters.
-
Diffusing in the Right Space: A Systematic Study of Latent Diffusability
A large-scale empirical study across tokenizers and diffusion backbones identifies Velocity Irreducible Variance (VIV) as one of the most stable predictors of latent diffusion generation quality.
-
Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment
BBCritic uses contrastive learning to align GUI actions in a continuous affordance space, outperforming larger binary critic models on a new four-level hierarchical benchmark while enabling zero-shot transfer.
-
Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment
BBCritic reframes GUI critique as continuous semantic alignment via contrastive learning in an affordance space, outperforming larger binary SOTA models on a new four-level hierarchical benchmark without extra annotations.
-
SOAP, Muon, and Beyond: Pushing LLM Pretraining Scales
SOAP and Muon, stabilized by per-step QR eigenbasis updates and KL-Shampoo covariance accumulation, beat AdamW on large-batch LLM pretraining up to 100M-token batches.
-
TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation
TunerDiT adds event-partitioned masking and cross-event prompt fusion to diffusion transformers for training-free multi-event video generation, with gains scaling by event count on a new Meve benchmark.
-
From Blind Guess to Informed Judgment: Teaching LLMs to Evaluate Materials by Building Knowledge-Augmented Preference Signals
MaterEval generates paired informed and blind evaluations as preference signals to improve small open-source LLMs on high-entropy alloy assessment, approaching closed-source performance without external retrieval.
-
SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessment
SongBench is a new fine-grained benchmark for song quality assessment with seven dimensions and an expert-annotated dataset of 11,717 samples showing high correlation with professional ratings.
-
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
TSHA is a new 80,000-pair benchmark for indoor safety hazard assessment; current vision-language models score roughly 45-85, and fine-tuning on TSHA raised Qwen2.5-VL-3B by 18.3 points on TSHA's test set.
-
SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation
SegMoTE shows that adding token-level mixture-of-experts routing to a frozen SAM decoder can match or beat medical-segmentation models trained on far more data, using 0.15M curated masks and 17M trainable parameters.
-
Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention
Light Forcing accelerates autoregressive video diffusion with a chunk-aware sparsity schedule and hierarchical frame-level and block-level attention selection, matching dense-attention quality at 1.2–1.3× speedup.
-
PISCES: Annotation-free Text-to-Video Post-Training via Optimal Transport-Aligned Rewards
PISCES post-trains text-to-video models using dual optimal-transport-aligned rewards (global quality plus token-level semantic) and outperforms annotation-based and annotation-free baselines on VBench and human evaluation.
-
Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
Comprehensive profiling of expert selection in frontier MoE models reveals temporal and spatial patterns that enable 6.6x speedup on wafer-scale GPUs and 1.25x on existing systems via targeted optimizations.
-
Overfitting has a limitation: a model-independent generalization gap bound based on R\'enyi entropy
A model-independent upper bound on generalization gap is established that depends solely on the Rényi entropy of the data-generating distribution for histogram-determined algorithms such as ERM.
-
VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness
VBench-2.0 is a benchmark suite that automatically evaluates video generative models on five dimensions of intrinsic faithfulness: Human Fidelity, Controllability, Creativity, Physics, and Commonsense using VLMs, LLMs...
-
ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Acceleration
ResilPhase accelerates DiT inference via stable ODE macro-trajectory extrapolation with derivative-free barycentric Lagrange extrapolator and bounded phase mapping, claiming SOTA fidelity at high acceleration ratios o...
-
CogniRoute: Learning to Route Social Evidence in Omni-Modal Models
CogniRoute adds a cognitive schema and route-aware RL to an omni-modal MoE, reaching 59.38% accuracy on a new 118K-example social video QA benchmark and beating prior baselines by 15-27 points.
-
ARGUS: Production-Scale Tracing and Performance Diagnosis for over 10,000-GPU Clusters
ARGUS provides low-overhead always-on hierarchical tracing and progressive diagnosis for 10k+ GPU LLM training clusters with 3700x compression and six-month production deployment.
-
ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring
ARGUS uses a Prosecutor-Defender-Umpire multi-agent setup plus RAG and chain-of-thought rewards to adapt ad policy enforcement to new regulations using minimal fresh labels.
-
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
TSHA is a mixed real/AIGC/panorama/video QA benchmark showing VLMs are weak at home safety hazard assessment and that TSHA training improves scores by up to +18.3 points.
-
Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention
Sparse attention with chunk-aware sparsity growth and hierarchical frame/block selection accelerates autoregressive video diffusion at ~1.3x with VBench quality on par with dense attention.
-
DSBench: A Comprehensive Benchmark for Evaluating External and In-Cabin Risks
A new benchmark claims to be the first to test VLMs on both external and in-cabin driving risks, and reports a fine-tuned model far outperforming all baselines.
-
Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
Orthogonal growth recycles pre-trained MoE checkpoints via layer copying and noisy expert duplication, delivering 10.6% higher accuracy than training from scratch with equivalent extra compute.
-
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Technical report announcing Ling-2.6 and Ring-2.6 models with hybrid linear attention, evolutionary CoT, and KPop RL for efficient agentic intelligence at scale.
-
From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI
Autonomous AI becomes dependable when tool use is embedded in persistent workspaces with reusable skills, shifting evaluation from answers to task closure.
-
Bridging the Post-discharge Gap: A Traceable Multi-agent Framework for Safe and Continuous Care
Healink is a multi-agent framework using memory, relational databases, and constraint-based RAG to produce traceable post-discharge care responses that outperformed physician baselines in expert evaluations on 485 real cases.
-
Multi-Modal LLM based Image Captioning in ICT: Bridging the Gap Between General and Industry Domain
A 7B-parameter domain-specific image captioning model for ICT, trained in three stages on synthesized and annotated data, outperforms 32B-parameter general models on BLEU and expert accuracy metrics.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.