Harness-IF scores 256 rules across coding-agent runs and finds every model performs 3.6 to 7.4 points worse on rules that oppose unprompted defaults, so aggregate compliance scores overstate true instruction following.
MLE-bench: Evaluating machine learning agents on machine learning engineering
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.AI 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Harness-IF: Evaluating Instruction Following Across Instruction Surfaces in Coding Agents
Harness-IF scores 256 rules across coding-agent runs and finds every model performs 3.6 to 7.4 points worse on rules that oppose unprompted defaults, so aggregate compliance scores overstate true instruction following.