A new benchmark shows LLM first-answer accuracy on procedural arithmetic drops from 63% (5 steps) to 20% (95 steps) due to execution failures like skipped steps and premature answers.
Title resolution pending
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
LLM simulations of user design preferences show significant systematic discrepancies from real aggregated user data across multiple experimental manipulations, with synthetic justifications lacking depth and relying on generic patterns instead.
citing papers explorer
-
When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models
A new benchmark shows LLM first-answer accuracy on procedural arithmetic drops from 63% (5 steps) to 20% (95 steps) due to execution failures like skipped steps and premature answers.
-
Distorted Perspectives of LLM-Simulated Preferences: Can AI Mislead Design?
LLM simulations of user design preferences show significant systematic discrepancies from real aggregated user data across multiple experimental manipulations, with synthetic justifications lacking depth and relying on generic patterns instead.