REVIEW 8 cited by
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
New LLM evaluation benchmarks are important to align with the rapid development of Large Language Models (LLMs). In this work, we present Chinese SimpleQA, the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions, and Chinese SimpleQA mainly has five properties (i.e., Chinese, Diverse, High-quality, Static, Easy-to-evaluate). Specifically, first, we focus on the Chinese language over 6 major topics with 99 diverse subtopics. Second, we conduct a comprehensive quality control process to achieve high-quality questions and answers, where the reference answers are static and cannot be changed over time. Third, following SimpleQA, the questions and answers are very short, and the grading process is easy-to-evaluate based on OpenAI API. Based on Chinese SimpleQA, we perform a comprehensive evaluation on the factuality abilities of existing LLMs. Finally, we hope that Chinese SimpleQA could guide the developers to better understand the Chinese factuality abilities of their models and facilitate the growth of foundation models.
Forward citations
Cited by 8 Pith papers
-
Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models
Zero-RL multi-stage constructive safety alignment with SERL and long-context training lets a 14B model match much larger models on safety without collapsing helpfulness or style.
-
Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning
Decoupling LLM-agent planning from summarization and rewarding tool-call completeness rather than final-answer correctness improves planning by 8-12% and end-to-end answers by 5-6% over end-to-end RL baselines.
-
Serving Large Language Models on Huawei CloudMatrix384
CloudMatrix384 with CloudMatrix-Infer reports prefill 6,688 tokens/s per NPU and decode 1,943 tokens/s per NPU on DeepSeek-R1, exceeding published NVIDIA-based efficiencies.
-
AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning
AdaCoT uses reinforcement learning with adjustable penalties to make LLMs trigger chain-of-thought reasoning only when needed, cutting response tokens by up to 69% with a small benchmark score loss.
-
Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking
HaluSearch reduces LLM hallucinations by generating responses through MCTS-based tree search with a reward model, outperforming CoT, self-consistency, and best-of-N baselines.
-
Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models
A 2,000-question Chinese safety factuality benchmark shows most LLMs are inaccurate on safety knowledge, with retrieval helping more than self-reflection.
-
PlanE: Meta Planning of Data, Tuning, and Inference for Extractive-based LLMs
A quadratic meta-planner trained on a few model-dataset runs selects the optimal data-tuning-inference configuration for extractive LLMs, matching grid search on three IE tasks.
-
Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
A comprehensive review that categorizes methods for shortening and adaptively triggering chain-of-thought reasoning in large language models.
Discussion (0). Continue with ORCID to comment.