Skill entropy, a reference-model-based measure of skill-switching difficulty, calibrates a new cross-skill benchmark and serves as an RL reward, more than doubling small models' scores.
STaD: Scaffolded Task Design for Identifying Compositional Skill Gaps in LLMs
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Benchmarks are often used as a standard to understand LLM capabilities in different domains. However, aggregate benchmark scores provide limited insight into compositional skill gaps of LLMs and how to improve them. To make these weaknesses visible, we propose Scaffolded Task Design (STaD) framework. STaD generates controlled variations of benchmark tasks based on the concept of scaffolding, which introduces structured, incremental support in a step-by-step manner. Rather than inspecting failures individually, this approach enables systematic and scalable probing of model behavior by identifying the specific reasoning skill compositions they lack. Treating the LLM as a black box, our experiments on six models of varying sizes reveal multiple failure points in three reasoning benchmarks and highlight each model's unique and distinct skill gaps.
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2026 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
Skill entropy, a reference-model-based measure of skill-switching difficulty, calibrates a new cross-skill benchmark and serves as an RL reward, more than doubling small models' scores.