Pith. sign in

REVIEW 4 major objections 4 minor 17 references

Some things to know about achieving artificial general intelligence

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current and foreseeable generative AI models cannot achieve general intelligence, because humans still solve the hard parts of every problem.

desk verdict A coherent synthesis of familiar AI-evaluation critiques whose central claim that GenAI cannot reach AGI rests on an unsupported distinction about insight problems. read the letter →

arxiv 2502.07828 v1 pith:DBBR6XKA submitted 2025-02-10 q-bio.NC cs.AI

Herbert Roitblat This is my paper · ORCID
classification q-bio.NCcs.AI
keywords artificialgeneralintelligenceanthropogenicdebtlargelanguagemodelsinsightproblemsill-structuredbenchmarksaffirmingtheconsequentproblemsolving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that current and foreseeable generative AI models are not on a path to artificial general intelligence because they carry what it calls 'anthropogenic debt': humans supply the problem framing, representation, architecture, training data, prompts, and evaluation criteria, leaving the model only parameter adjustment by gradient descent. Because models recast every task as a language-pattern prediction problem, they cannot autonomously handle ill-structured problems or 'insight problems' whose solutions require recognizing a new way to organize the problem rather than following a step-by-step procedure. The paper also claims that standard benchmarks cannot settle whether a model is generally intelligent, since observing a correct answer does not reveal the method that produced it, and inferring a method from success is the logical fallacy of affirming the consequent. A sympathetic reader would care because if the claim is right, scaling current models and passing benchmarks will not deliver AGI; new inventions and discoveries would be required, and safety regulations and investments based on imminent AGI would be aimed at the wrong target.

What carries the argument

The argument runs on the distinction between well-structured and ill-structured problems, taken from Simon's analysis of problem solving. A well-structured problem specifies a test for solutions, a problem space of states and moves, and a goal; these are the problems current AI can solve once a human supplies the structure. Ill-structured and 'insight' problems, illustrated by the mutilated checkerboard, have no predefined space or stepwise test, and solving them requires restructuring one's approach (the 'coloring argument' that each domino covers one square of each color). The paper's central mechanism is anthropogenic debt: a ledger of all the human contributions—representations, architecture, training tasks, prompts, labels, evaluation metrics—that remain invisible when a model's success is credited to the model. Because these contributions are not part of the model's own computation, the paper argues, benchmark scores cannot distinguish model intelligence from human intelligence embedded in the setup.

What would settle it

A concrete experiment would give a current model a brand-new insight problem whose solution does not appear anywhere in its training data—for example, a mutilated checkerboard variant where the two removed corner squares are the same color, so that the parity/coloring argument must be discovered rather than retrieved—and require the model to produce the correct 'no tiling' answer with a justification. If the model solves it without any human-provided hint, the paper's claim that GenAI models cannot solve insight problems fails; if it fails, the claim is supported.

Watch

Extended reading notes

Core claim

The central claim is that 'current and foreseeable GenAI models are not capable of achieving artificial general intelligence because they are burdened with anthropogenic debt.' The paper maintains that nearly all conceptually difficult parts of problem solving—defining the problem, choosing representations, designing the network, curating training data, writing prompts, and deciding what counts as success—are done by humans, while the model contributes only parameter adjustments. As a result, a model's apparent competence is largely a measure of how much structure humans have already imposed. General intelligence, by contrast, requires autonomy: the system must frame ill-structured problems, create its own problem space, discover insight-style solutions that cannot be reached by stepwise search, and verify its own answers. The paper further contends that no test or benchmark can certify such generality, because success on a test is compatible with many mechanisms—memorization, classification, or genuine reasoning—and inferring the mechanism from the outcome is affirming the consequent.

Load-bearing premise

The argument rests on the assumption that insight problems cannot be solved by a step-by-step procedure, such as an algorithm; if even one such problem can be solved by systematic search, the claim that a necessary class of human problems lies beyond current computational methods loses its main support.

Editorial extensions

If this is right

  • Scaling today's language models, training data, and compute will not by itself produce artificial general intelligence; overcoming anthropogenic debt requires new inventions and discoveries.
  • Benchmark results, including high scores on ARC-AGI, Winogrande, and similar tests, do not establish general intelligence, because the same score could be produced by memorization, task-specific shortcuts, or human-structured problem simplification.
  • Current models cannot be considered autonomous or able to 'escape control' in the way AGI alarm scenarios assume, because they depend on humans for the difficult parts of each problem.
  • Human intelligence tests and aptitude tests are invalid for assessing machine intelligence, since the model's 'experience' and vocabulary are designed by its creators rather than acquired in a human-like way.
  • Progress toward AGI is unlikely to be measurable on a smooth scale; it may come discontinuously through insights that cannot be predicted in advance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension of the paper's argument is to measure 'anthropogenic debt' quantitatively, for example by degrading the human-supplied components—prompt quality, problem framing, representation choices—and observing how much model performance drops; the paper predicts steep drops on ill-structured tasks.
  • The affirming-the-consequent critique applies beyond benchmarks: it also undercuts inferences from 'emergent abilities' in large language models, since observed behavior alone cannot distinguish a learned general capacity from training-set memorization.
  • If insight problems truly resist step-by-step methods, then hybrid systems that combine language models with external search, theorem provers, or symbolic planners would still inherit the debt of having the insight supplied by the tool designer; the paper's argument implies the models would need to generate new representations themselves.
  • A practical corollary for safety research: if the paper is right, the near-term risk profile shifts from autonomous superintelligent systems to humans over-trusting systems that are 'stupid' in ways the benchmarks conceal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper argues that current and foreseeable generative AI models cannot achieve artificial general intelligence (AGI) because they carry 'anthropogenic debt': humans supply problem structure, representations, training data, prompts, and evaluation criteria, leaving only parameter adjustment to the model. It further claims that a necessary class of human problems, 'insight problems,' cannot be solved by step-by-step algorithmic procedures, and that benchmarks cannot provide evidence for general intelligence because success on a test does not reveal the method of solution. The author concludes that AGI will require as-yet-undiscovered inventions, that current risk assessments are overstated, and that a theory of general intelligence is needed.

Significance. The paper contains one genuinely sound and useful logical point: observing benchmark success does not license an inference about the mechanism that produced it, so claims of emergent reasoning from language models require additional evidence. The discussion of the Goh et al. (2024) diagnostic study is a good case study of how human prompt-engineering can be conflated with model competence. However, the central negative thesis—that no foreseeable GenAI model can achieve AGI—is not established. It rests on an unsupported and in fact contradicted dichotomy between insight problems and algorithmic solvability, plus a definitional autonomy criterion that makes part of the conclusion true by stipulation. The paper offers no formal theorem, no impossibility proof, and no machine-checked argument; it is an opinion essay with illustrative examples. Because the load-bearing supporting claims fail, the main contribution is not substantiated.

major comments (4)
  1. [Insight problems and the mutilated checkerboard] The assertion that 'insight problems cannot be solved by a step-by-step procedure, like an algorithm' is load-bearing, since it supports the claim that a necessary class of human problems lies beyond current computational methods. But the author's own example undermines it: the mutilated checkerboard is decidable by brute-force enumeration of tilings or by an exact-cover/SAT solver, and the author himself cites Heule, Kiesl, and Biere (2019), who provide clausal proofs for exactly this problem. The distinction between 'specific instances' solvable by brute force and 'general solutions' requiring insight does not repair the argument: if 'general solution' means a compact insight such as the coloring argument, that is a claim about the style of human problem-solving, not a claim about computability. No formal problem class, complexity bound, or impossibility theorem is offered, so the central dichotomy remains unsupported.
  2. [Current AI models suffer from anthropogenic debt] The argument moves from 'current GenAI models depend on human-provided structure' to 'they cannot achieve AGI' without a justifying principle. The author acknowledges that AGI 'may be achievable' with future inventions, which is compatible with the possibility that future GenAI-based systems overcome anthropogenic debt. To establish impossibility, the paper would need to show that such debt is ineliminable in principle for any foreseeable model; the marathon metaphor and the Domingos equation illustrate current dependence but do not supply such a proof. The leap from 'humans currently solve the hard parts' to 'models cannot ever learn to solve those parts' is a non sequitur.
  3. [Definition of general intelligence and autonomy] Part of the conclusion follows from the paper's definitional choices rather than from empirical evidence. The claim that models 'cannot be autonomous, cannot escape control, and cannot be generally intelligent' assumes that general intelligence requires autonomy from all human-provided structure. Since the author defines the goal as covering 'the full range of human problem solving,' a model that fails the autonomy criterion fails his definition by construction. The paper should explicitly separate the definitional claim from the empirical claim and argue why this particular autonomy criterion is the correct one for AGI, rather than treating it as self-evident.
  4. [Benchmarks are unsuitable measures of general intelligence] The account of affirming the consequent in evaluating benchmarks is logically correct, but the conclusion that benchmarks 'are of no value' is overly strong. A benchmark can provide evidence about performance while remaining agnostic about mechanism; the fallacy arises only when one additionally asserts a mechanism of success. This overstatement weakens an otherwise valid epistemological point and makes the section vulnerable to the obvious reply that progress on well-designed held-out benchmarks still offers weak but non-zero evidence. The argument as written conflates 'cannot prove mechanism' with 'cannot indicate progress.'
minor comments (4)
  1. [Throughout] There are numerous typographical errors, including 'Hoftsatadter' for Hofstadter, 'provied' for provided, 'develope' for develop, 'Ardilla' for Ardila, 'Modei' for Amodei, and 'Dominos' for dominoes.
  2. [Figure 1 reference] The text refers to 'Figure 1' as an illustration of ARC-AGI tasks, but no actual figure is included; the reference is only a URL. The figure should be inserted or the citation should be made inline.
  3. [References] Several reference entries are incomplete or inconsistently formatted; for example, the Heule, Kiesl, and Biere entry lacks full publication details, and the Morris et al. entry is cited as 2024 in the text but 2025 in the reference list.
  4. [Mutilated checkerboard statistics] The author writes that 'a full checkerboard has over 12 million ways to arrange the dominos'; the exact number is 12,988,816, so the statement is true, but a precise citation or exact count would be more scholarly.

Circularity Check

3 steps flagged · score 6.0 of 10

The negative AGI thesis is substantially definitional: AGI is defined as full-range human autonomy, current models are defined as human-dependent, and insight problems are defined as not step-by-step, so the central conclusion follows by construction; the supporting theory is deferred to self-citations.

  1. self definitional [Abstract / 'Artificial General Intelligence' and 'Anthropogenic debt' sections]
    "The goal of artificial general intelligence is to build a model that is capable of the full range of human problem solving. ... General intelligence is itself an example of an ill-structured problem. It cannot be described with an explicit set of moves, because a necessary characteristic of general intelligence is that it create that set of potential moves. ... For a model to be generally intelligent or autonomous, it will have to overcome that debt and create its own representations and evaluations."

    AGI is defined as covering the full human problem range and as requiring the system to create its own moves, representations, and evaluations. Autonomy is then defined as independence from human-provided structure. The paper observes that current GenAI receives structure, representations, and evaluations from humans, and concludes that it cannot be generally intelligent or autonomous. That conclusion is the definition applied to a premise, not an independent derivation.

  2. self definitional [Section 'Simplifying complex ill-formed problems so a computer can solve them is not solving the complex problem' (insight problems and mutilated checkerboard discussion)]
    "More generally, insight problems cannot be solved by a step-by-step procedure, like an algorithm. ... Current GENAI models can report an insight solution to known problems, because descriptions of the solutions to problems are widely available on the Web. ... They would not, however be able to solve insight problems for which there is no known answer, but that is a skill that a general intelligence would need."

    The load-bearing premise that insight problems are not solvable by a step-by-step procedure is an assertion that defines insight problems by their non-algorithmic character. Since GenAI is characterized as parameter adjustment through gradient descent, a step-by-step procedure, the conclusion that GenAI cannot solve insight problems is already contained in the premise. No formal problem class or impossibility theorem is provided; the paper itself concedes brute force can solve specific instances, so the dichotomy is assumed rather than derived.

1 more flagged steps
  1. self citation load bearing [Section 'A theory of general intelligence is needed']
    "Space does not permit, nor do I have, a full theory of artificial general intelligence. Instead, I will mention a few characteristics such a theory must have. Further description is provided by Roitblat (2020, 2024). ... An adequate theory will need to provide a means for doing it, perhaps having to do with analogies and resonance."

    The paper's positive account of what an AGI theory must include, and the proposed mechanisms such as analogies and resonance, are deferred to the author's own prior work rather than derived or independently supported. The cited works are not machine-checked or externally validated, and the paper explicitly disclaims having a full theory. The claim that such a theory is needed and what it must contain therefore rests on self-citation.

full rationale

The paper is an opinion/position essay rather than a formal derivation, and it does make independent empirical observations about human contribution to AI pipelines, benchmark overfitting, and affirming the consequent in evaluation. However, the central negative thesis that current and foreseeable GenAI cannot achieve AGI is substantially definitional. AGI is defined as full-range human problem solving plus autonomy from human-provided structure; current models are then shown to depend on human-provided structure, so the conclusion follows by construction. The additional claim that insight problems are beyond step-by-step methods is asserted as a definitional dichotomy, not proved, and it is the main technical support for denying foreseeable progress. Finally, the needed theory is referred to the author's own publications. These steps make the central conclusion partly circular, though not entirely so, because the anthropogenic-debt observations and benchmark critique have independent content. Score 6 reflects the partial reduction of the central claim to definitions and self-citation.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no free parameters, fitted values, or new entities. It relies on the author's own 'anthropogenic debt' concept from prior publications, on a definition of general intelligence from Simon and Newell, and on an unproven claim about insight problems. The main intellectual burden is carried by these assumptions rather than by data or derivation.

assumptions (5)
  • ad hoc to paper Insight problems cannot be solved by a step-by-step procedure, like an algorithm.
    Asserted in the section on insight problems; no formal definition or proof is given, and the paper itself admits brute-force search can solve specific instances.
  • domain assumption General intelligence must cover the full range of problems to which the human mind has been applied.
    Relying on a quotation from Simon and Newell (1958) as the definition of general intelligence; if one adopts a different definition, the argument's target changes.
  • domain assumption Any system whose problem structure, representation, and training data are provided by humans cannot be autonomous or generally intelligent.
    Stated throughout and summarized under 'anthropogenic debt'; the equivalence between human-provided structure and lack of generality is not derived.
  • domain assumption Current GenAI models depend on humans for every conceptually difficult part and only perform parameter adjustment themselves.
    Given as a list of human contributions versus machine contribution; this is a simplification that treats prompt design, architecture search, and representation learning as always human-supplied.
  • standard math Benchmarks cannot infer the method of solution from correct outputs; affirming the consequent is invalid.
    This is a valid logical point, but the paper extends it to assert that no benchmark evidence could ever support a claim of general intelligence without additional controls.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Some things to know about achieving artificial general intelligence." pith.science (2026). https://pith.science/paper/DBBR6XKA

@misc{pith2026250207828,
  author       = {Pith},
  title        = {Pith review of: Some things to know about achieving artificial general intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DBBR6XKA}},
  note         = {Machine review of arXiv:2502.07828}
}
read the original abstract

Current and foreseeable GenAI models are not capable of achieving artificial general intelligence because they are burdened with anthropogenic debt. They depend heavily on human input to provide well-structured problems, architecture, and training data. They cast every problem as a language pattern learning problem and are thus not capable of the kind of autonomy needed to achieve artificial general intelligence. Current models succeed at their tasks because people solve most of the problems to which these models are directed, leaving only simple computations for the model to perform, such as gradient descent. Another barrier is the need to recognize that there are multiple kinds of problems, some of which cannot be solved by available computational methods (for example, "insight problems"). Current methods for evaluating models (benchmarks and tests) are not adequate to identify the generality of the solutions, because it is impossible to infer the means by which a problem was solved from the fact of its solution. A test could be passed, for example, by a test-specific or a test-general method. It is a logical fallacy (affirming the consequent) to infer a method of solution from the observation of success.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages

  1. [1]

    insight problems

    Some things to know about achieving artificial general intelligence Herbert L. Roitblat Abstract Current and foreseeable GenAI models are not capable of achieving artificial general intelligence because they are burdened with anthropogenic debt. They depend heavily on human input to provide well-structured problems, architecture, and training data. They c...

    work page 2025

  2. [2]

    experience

    describe insight problems with this metaphor: “Imagine that you are searching for a diamond in a huge, dark room. … One option is to grope blindly in the dark. … But after groping blindly for several minutes, you might decide to abandon the search for the diamond, and to search instead for a light switch. If one could be found, and the light turned on, th...

    work page 2025

  3. [4]

    artificial intelligence

    (The Gladstone Report), The recent explosion of progress in advanced artificial intelligence (AI) has brought great opportunities, but it is also creating entirely new categories of weapons of mass destruction-like (WMD-like) and WMD-enabling catastrophic risks [1–4]. A key driver of these risks is an acute competitive dynamic among the frontier AI labs t...

    work page 1965

  4. [6]

    Gould, S.J., and Lewontin, R

    Academic Press. Gould, S.J., and Lewontin, R. (1979). The Spandrels of San Marco and the Panglossian Paradigm: A Critique of the Adaptationist Programme. Proceedings of the Royal Society of London Series B, Biological Sciences 205: 581–598. https://royalsocietypublishing.org/doi/epdf/10.1098/rspb.1979.0086 Gygi, J. T., Hagmann-von, A. P., Schweizer, F., &...

    work page arXiv 1979

  5. [7]

    the standalone performance of the LLM

    or is it better to reduce the carbon? Is it better to raise Venice or install flood barriers at the entrance to the lagoon? One key problem (meta problem?) in addressing these is identifying what exactly are the specific problems we need to solve and what are the means by which we will attempt to solve them. Even after attempting to solve them, there is n...

    work page 2024

  6. [11]

    Simple off-the-shelf deep learning methods turned out to be useful for solving Bongard problems, at least in simplified classification form

    Source: https://arxiv.org/html/2412.04604v1 ARC Prize 2024: Technical Report ARC-AGI is intended to be a test of abstraction and reasoning, which is needed for general intelligence. Assuming (for sake of argument) that it is such a test, is it the model that is doing the reasoning? The concepts it tests are intended to be open-ended. Open ended means that...

    arXiv 2019

  7. [28]

    10.1016/0010-0277(88)90033-9

    195-247. 10.1016/0010-0277(88)90033-9. https://www.researchgate.net/publication/19806078_The_relation_between_linguistic_structure_and_associative_theories_of_language_learning-A_constructive_critique_of_some_connectionist_learning_models Legg , S. & Hutter, M. (2007). A collection of definitions of intelligence. In B. Goertzel and P. Wang, editors, Advan...

    work page arXiv 2007

  8. [235]

    An Essay concerning machine understanding

    JMLR.org, Article 1478, 36308–36321. Nie, W., Yu, Z., Mao, L., Patel, A. B., Zhu, Y., & Anandkumar, A. (2020). Bongard-LOGO: A new benchmark for human-level concept learning and reasoning. In Advances in Neural Information Processing Systems (pp. 16468–16480). https://papers.nips.cc/paper/2020/file/bf15e9bbff22c7719020f9df4badc20a-Paper.pdf Papers with Co...

    work page Pith review arXiv 2020

Show all 17 references
  1. [1950]

    optimizing for a single metric or set of metrics often leads to tradeoffs and shortcuts when it comes to everything that isn’t being measured and optimized for

    decided to change the representation into a tree structure of moves and represent play selection as choosing branches on that tree. The models became successful when new algorithms for tree navigation and sufficient computing resources became available to traverse the tree. Fr...

    2007

  2. [1958]

    defined general intelligence in a keynote speech: “It is not my aim to surprise or shock you—but the simplest way I can summarize is to say that there are now in the world machines that can think, that can learn and that can create. Moreover, their ability to do these things i...

    1979

  3. [1973]

    The moves could be traversing down the branches of a tree (as in chess playing) or adjusting a parameter, for example

    includes a definite criterion for testing a potential solution, a problem space representing the initial problem state, the final goal state, and all intermediate states, and a set of potential state changes (moves). The moves could be traversing down the branches of a tree (a...

    2024

  4. [2007]

    Also: https://arxiv.org/pdf/0706.3639 Leike, J., & Sutskever, I

    IOS Press. Also: https://arxiv.org/pdf/0706.3639 Leike, J., & Sutskever, I. (2023, July 5). Introducing superalignment. Openai.com. https:// openai.com/blog/introducing-superalignment Levesque, H. J.; Davis, E.; and Morgenstern, L. (2011). The Winograd schema challenge. AAAI S...

    2011 arXiv

  5. [2017]

    they” refers to the councilmen. In the second, “they

    for predicting human success, such as grades. That is, they are significant, but relatively weak predictors of what a child’s grade average will be in the years following a test. They have no validity for machines because the training set includes many similar tests and the so...

    1981

  6. [2019]

    Punctuated equilibria: An alternative to phyletic gradualism

    On the Measure of Intelligence. arXiv:1911.01547 https://arxiv.org/abs/1911.01547 College Board (n.d.). SAT Suite of Assessments: Parents FAQs: SAT Practice. https://satsuite.collegeboard.org/sat-suite-benefits-students-parents/faq/sat-practice De Groot, A. D. (1946). Het denk...

    1946 arXiv

  7. [2022]

    backpropagation,

    had the insight to adjust parameters through “backpropagation,” error correction proportionate to a parameter’s contribution to the observed error. Before that, it was well known that single-layer perceptrons (simple neural networks) could learn to solve certain problems and t...

    1946

  8. [2024]

    breakthrough

    is a symbolic representation to how close the world is to not just artificial general intelligence, but uncontrolled AGI. OpenAI recently announced a “breakthrough” on the ARC-AGI benchmark (Chollet, 2019). These predictions have also raised widespread concerns. The Future of ...

    2023

  9. [2025]

    implementing the capability to promptly enact a full shutdown

    On the other hand, Lu et al. (2024) argue that large language models only follow instructions and have no potential to autonomously master new skills without some explicit instruction. Governments around the world are considering regulations based on these threat assessments. ...

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.