Transformers trained on random elementary cellular automata can predict unseen rules fairly well one step ahead, but multi-step planning degrades unless the model is deeper or trained with future-state or rule prediction losses.
Llms still can’t plan; can lrms? a preliminary evaluation of openai’s o1 on planbench, 2024
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.NE 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Learning Elementary Cellular Automata with Transformers
Transformers trained on random elementary cellular automata can predict unseen rules fairly well one step ahead, but multi-step planning degrades unless the model is deeper or trained with future-state or rule prediction losses.