A staged knowledge-prompting method (KAAR) improves LLM test accuracy on ARC by about 5 absolute points over repeated-sampling plan-guided code generation, reaching 35% with GPT-o3-mini.
An Approach to Solving the Abstraction and Reasoning Corpus (ARC) Challenge
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We utilise the power of Large Language Models (LLMs), in particular GPT4, to be prompt engineered into performing an arbitrary task. Here, we give the model some human priors via text, along with some typical procedures for solving the ARC tasks, and ask it to generate the i) broad description of the input-output relation, ii) detailed steps of the input-output mapping, iii) use the detailed steps to perform manipulation on the test input and derive the test output. The current GPT3.5/GPT4 prompt solves 2 out of 4 tested small ARC challenges (those with small grids of 8x8 and below). With tweaks to the prompt to make it more specific for the use case, it can solve more. We posit that when scaled to a multi-agent system with usage of past memory and equipped with an image interpretation tool via Visual Question Answering, we may actually be able to solve the majority of the ARC challenge
citation-role summary
background 1
citation-polarity summary
fields
cs.AI 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
From Reasoning to Generalization: Knowledge-Augmented LLMs for ARC Benchmark
A staged knowledge-prompting method (KAAR) improves LLM test accuracy on ARC by about 5 absolute points over repeated-sampling plan-guided code generation, reaching 35% with GPT-o3-mini.