pith. sign in

arxiv: 2505.10185 · v1 · pith:XMU6DAUVnew · submitted 2025-05-15 · 💻 cs.CL · cs.AI

The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think

classification 💻 cs.CL cs.AI
keywords reasoningmodelanalyzingbehaviorcotsdataeffectiveencyclopedia
0
0 comments X
read the original abstract

Long chain-of-thought (CoT) is an essential ingredient in effective usage of modern large language models, but our understanding of the reasoning strategies underlying these capabilities remains limited. While some prior works have attempted to categorize CoTs using predefined strategy types, such approaches are constrained by human intuition and fail to capture the full diversity of model behaviors. In this work, we introduce the CoT Encyclopedia, a bottom-up framework for analyzing and steering model reasoning. Our method automatically extracts diverse reasoning criteria from model-generated CoTs, embeds them into a semantic space, clusters them into representative categories, and derives contrastive rubrics to interpret reasoning behavior. Human evaluations show that this framework produces more interpretable and comprehensive analyses than existing methods. Moreover, we demonstrate that this understanding enables performance gains: we can predict which strategy a model is likely to use and guide it toward more effective alternatives. Finally, we provide practical insights, such as that training data format (e.g., free-form vs. multiple-choice) has a far greater impact on reasoning behavior than data domain, underscoring the importance of format-aware model design.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Diverse Thinking Schemata Elicit Better Reasoning in Large Language Models

    cs.AI 2026-06 unverdicted novelty 5.0

    DiScO enhances LLM mathematical reasoning by training for awareness of diverse thinking schemata, using RL to promote diversity, and applying it at inference, outperforming standard GRPO.