Harness-IF scores 256 rules across coding-agent runs and finds every model performs 3.6 to 7.4 points worse on rules that oppose unprompted defaults, so aggregate compliance scores overstate true instruction following.
Models overview.https://platform.claude.com/docs/claude/docs/models-overview, 2026
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.AI 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents
Harness-IF scores 256 rules across coding-agent runs and finds every model performs 3.6 to 7.4 points worse on rules that oppose unprompted defaults, so aggregate compliance scores overstate true instruction following.