{"id":"8b7b04ff-412f-4b28-9317-736453e38bfd","arxiv_id":"2607.07663","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"A survey of 1,250 papers organizes AI self-improvement along two axes—what is improved and loop closure—finding that demonstrated self-improvement strength tracks a verification hierarchy from formal verifiers down to intrinsic self-assessment.","lead":"This paper surveys 1,250 recent arXiv papers on AI self-improvement and organizes them into a taxonomy separating bounded self-refinement from open-ended recursive self-improvement. A smart generalist should read it to understand where AI self-improvement actually stands versus the hype, and what the real bottleneck is.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The verification hierarchy is the paper's load-bearing empirical claim, but it is validated only qualitatively within a corpus whose construction was partly shaped by the taxonomy that encodes the hierarchy concept — a potential circularity that no amount of transparent caveating resolves.","rationale":"The reader correctly identified the load-bearing concern: the verification hierarchy is the paper's central empirical claim, and it is validated only qualitatively within a corpus whose construction introduces potential sampling artifacts. I agree with this assessment and would sharpen it slightly to note the potential circularity between the paper's framing (every loop depends on an evaluator) and its central observation (loops with better evaluators work better), which goes beyond simple sampling bias. However, this concern does not move the verdict below CONDITIONAL for three reasons. First, the paper is transparent about the limitation — it explicitly labels the hierarchy as qualitative, not a measured law, and acknowledges the corpus is a sample shaped by its own taxonomy. Second, the within-domain evidence is genuinely supportive and drawn from independent work (Huang et al. [27], the Mirror Loop [140], Lin [92], Shumailov et al. [144]) — these are not artifacts of the corpus construction. Third, the paper's structural contributions (the two-axis taxonomy, the bounded-vs-open-ended distinction, the synthesis of failure modes) have value independent of whether the hierarchy is a measured law or a qualitative regularity. The CONDITIONAL verdict appropriately captures that the framework is useful and well-argued but its durability depends on future quantitative validation that the paper itself identifies as needed. The concrete test I propose — stratified sampling with independent annotation and statistical testing of the hierarchy ordering — is feasible with the released corpus and would settle whether the concern lands. If the hierarchy holds quantitatively, the paper's reframing of the RSI debate as a measurement program is substantially strengthened; if it does not, the framework remains useful as an organizing schema but loses its claim to identifying an empirical regularity.","tokens_in":36005,"tokens_out":3148,"duration_ms":334152,"concrete_test":"From the released corpus, draw a stratified random sample of ~150 papers (≈30 per taxonomy category, excluding foundations). For each, have two independent annotators (not the original single annotator) classify: (a) the evaluator type on the four-rung hierarchy, and (b) whether the paper reports a quantitative improvement and its effect size. Then test whether evaluator type significantly predicts improvement effect size using a rank-based test (e.g., Kruskal-Wallis across the four rungs). If the ordering does not hold at p<0.05 or the effect sizes are not monotonically ordered, the hierarchy claim weakens from 'empirical regularity' to 'plausible but unconfirmed.' Additionally, compare the domain distribution (code/math vs. other) of this sample against an independent query of all arXiv cs.AI papers mentioning self-improvement-related terms to check for systematic over-representation o","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that 'demonstrated self-improvement strength tracks' the verification hierarchy (§5.2), and that 'the direction-setting bottleneck and the verification bottleneck are the same bottleneck' (§5.4). The authors explicitly label this 'a qualitative pattern we observe throughout the corpus, not a measured law.' The concern is not merely sampling bias (though that is real: seed queries with per-query depth caps favor high-volume threads in code/math, and the supplemental harvest of 379 papers targeted directions 'the taxonomy makes first-class'). The deeper issue is a potential circularity between framing and observation. The paper's organizing premise — 'every improvement loop is a claim that some signal can substitute for human judgment' (§5) — predisposes the analysis to find that evaluator quality is the key variable. If every loop is framed as depending on an evaluator, and loops with stronger evaluators are observed to work better, the conclusion is partly built into the lens. The taxonomy elevates self-evaluation to a first-class category, the supplemental harvest fills that category, and then the hierarchy is observed across all categories including the one constructed to embody it. This does not make the claim wrong — the within-domain evidence (e.g., Huang et al.'s negative result on intrinsic self-correction [27], the Mirror Loop study [140], Lin's rise-and-collapse [92]) is genuinely supportive and drawn from independent work. But the cross-domain regularity, which is what makes the claim a 'reframing of the RSI debate,' rests on a corpus whose structure and sampling were not independent of the claim being tested. Converting the qualitative observation into a quantitative one — measuring effect sizes by evaluator type across an independently-sampled corpus — is the missing step that would either confirm or expose this circularity.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"This manuscript surveys 1,250 arXiv papers (2024–2026) on AI self-improvement, organizing them along two axes: what the system improves (deployment-time behavior, training-time policy, evaluator, or research process) and the degree of loop closure (human-in-the-loop to fully closed). The central organizing claim is a 'verification hierarchy'—that demonstrated self-improvement strength tracks the reliability of the evaluation signal, from formal verifiers (strongest) to intrinsic self-assessment (weakest)—and that characteristic failure modes (self-confirming loops, model collapse, diversity collapse) follow from violations of this hierarchy. The survey covers four technical categories (deployment-time self-evolution, training-time self-iteration, self-evaluation, auto research) plus a foundations/limits/safety family, and connects the technical literature to RSI theory and governance questions. The corpus, classification scripts, and per-paper assignments are released as reproducible artifacts.","tokens_in":36193,"tokens_out":1556,"duration_ms":332912,"significance":"The paper addresses a genuine gap: no existing survey spans the full spectrum from bounded self-refinement to open-ended RSI while treating self-evaluation as a load-bearing pillar. The two-axis taxonomy is a useful organizing contribution, and the verification hierarchy, if valid, reframes the RSI debate from speculation about capability growth into a measurement program focused on evaluator reliability. The authors ship reproducible corpus construction scripts and per-paper assignments—a concrete strength. The argumentative skeleton rests on verified anchor works (STaR, Self-Refine, FunSearch, Huang et al.'s negative result on intrinsic self-correction) and the diagnostic literature is treated as load-bearing rather than incidental. The connection to Anthropic's autonomy continuum and the identification of governance-grade measurement as an underpopulated niche are well-motivated.","major_comments":[{"comment":"§5.2 and §5.4: The verification hierarchy is the paper's central load-bearing empirical claim, but it is validated only qualitatively within a corpus whose construction was partly shaped by the taxonomy that encodes the hierarchy. The supplemental harvest (§2.3) targeted 379 papers in 'directions the taxonomy makes first-class,' including self-evaluation methods. The paper's organizing premise—'every improvement loop is a claim that some signal can substitute for human judgment' (§5)—predisposes the analysis to find that evaluator quality is the key variable. If every loop is framed as depending on an evaluator, and loops with stronger evaluators are observed to work better, the conclusion is partly built into the lens. The authors explicitly caveat this ('a qualitative pattern we observe throughout the corpus, not a measured law'), but the caveat does not resolve the structural concern.","section":null},{"comment":"§5.4: The claim that 'the direction-setting bottleneck and the verification bottleneck are the same bottleneck' is asserted rather than demonstrated. The Anthropic essay [5] identifies research direction-setting as a distinct bottleneck (choosing which problems matter), while the verification hierarchy concerns signal reliability for already-specified tasks. The conflation of these two different bottlenecks—problem selection and solution verification—is load-bearing for the paper's framing but is not separately argued. The within-domain evidence (Huang et al. [27], Mirror Loop [140], Lin [92]) supports the verification claim but does not address whether direction-setting reduces to verification.","section":null},{"comment":"§2.3: The corpus construction has a recency bias (74% from 2026) that interacts with the verification hierarchy claim in a way that is not fully addressed. If the field has recently concentrated on verifiable domains (code, math) where self-improvement works, the hierarchy could partly reflect where current research activity happens to be dense rather than a stable empirical regularity. The authors acknowledge this for citation counts but not for the hierarchy claim itself. A simple robustness check—reporting the hierarchy's support stratified by publication year—would help distinguish a real regularity from a recency artifact.","section":null}],"minor_comments":[{"comment":"Figure 1 is referenced as laying out the 4×3 grid with representative systems, but the figure content is not visible in the manuscript text. Ensure the figure is properly rendered and legible in the final version.","section":null},{"comment":"Figure 2 (semantic map) uses TF-IDF abstracts with SVD + t-SNE projection. The axes are described as 'arbitrary embedding dimensions' but no clustering quality metric (e.g., silhouette score) is reported to support the claim that 'Auto Research and foundations form coherent regions.'","section":null},{"comment":"Table 1 reports '82% posted in 2026' for the self-evaluation category, which is notably higher than other categories (57–76%). Given that the supplemental harvest specifically targeted self-evaluation methods, this percentage may reflect sampling design rather than field dynamics. A footnote clarifying this would help.","section":null},{"comment":"§4.3: The claim that on-policy self-distillation 'did not exist before 2026' is strong; the authors should verify whether precursor work (e.g., earlier self-distillation variants) existed under different names.","section":null},{"comment":"§7.1: The Whitfill and Wu result [189] is described as 'the empirical crux of the RSI-feasibility debate,' but the two specifications diverging (substitutes vs. complements) means the result is inconclusive rather than a crux. The framing should be softened.","section":null},{"comment":"The reference list includes several papers from 2026 with arXiv IDs starting '2607' (July 2026), which is the same month as this manuscript's submission. The authors should verify these are not concurrent submissions that could create citation circularity.","section":null},{"comment":"§3.5: The term 'harness' is defined in §2.1 but used extensively before its definition appears in the reading order. Consider forward-referencing the definition in §3.","section":null},{"comment":"The abstract states '1,250 arXiv papers' while §2.3 describes 871 seed + 379 supplemental = 1,250. This is consistent but the abstract could note the two-stage construction for clarity.","section":null}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about framing-observation circularity is the right concern to raise, and it does land: the supplemental harvest targeting self-evaluation as a first-class category, combined with the organizing premise that every loop depends on an evaluator, creates a genuine risk that the hierarchy is partly an artifact of the lens. However, I would frame this as a correctness-risk concern requiring a concrete robustness check (e.g., stratifying the hierarchy's support by corpus construction stage, or reporting within-domain evidence separately from cross-domain claims) rather than a fatal circularity. The within-domain evidence (Huang et al., Mirror Loop, Lin's rise-and-collapse) is genuinely supportive and drawn from independent work. The paper's contribution is real and the taxonomy is useful, but the central empirical claim needs stronger validation or more careful scoping before publication. The single-annotator classification is a limitation but not a blocker given the released per-paper assignments."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The referee identifies three major concerns: (1) a potential circularity in the verification hierarchy claim arising from the taxonomy-guided corpus construction, (2) an insufficiently argued conflation of the direction-setting bottleneck with the verification bottleneck, and (3) an unaddressed interaction between corpus recency bias and the hierarchy claim. We agree that all three points identify genuine weaknesses in the current manuscript and will revise accordingly. Specifically, we will (a) add an explicit methodological caveat about the taxonomy-corpus interaction and reframe the hierarchy's epistemic status, (b) separate the two bottlenecks and argue the reduction rather than asserting it, and (c) add a stratified robustness check by publication year. We cannot fully resolve the structural circularity concern—it is inherent to any taxonomy-guided survey—but we can make it fully transparent and distinguish what the hierarchy claims to be (an organizing empirical pattern) from what it does not claim to be (an independently validated law).","responses":[{"response":"The referee is correct that there is a structural circularity here, and we had not stated it with sufficient force. The concern has two layers, and we address each. First, the corpus-construction layer: the supplemental harvest (§2.3) targeted self-evaluation methods, test-time training, and zero-data self-play because the taxonomy made them first-class. This means the corpus is not an independent sample with respect to the taxonomy—it was partly shaped by it. We will add an explicit methodological caveat in §2.3 stating this directly and noting that the corpus proportions cannot be read as unbiased estimates of the field's composition. Second, the conceptual layer: the premise that 'every improvement loop is a claim that some signal can substitute for human judgment' does frame every loop as evaluator-dependent, which makes the finding that evaluator quality matters partly built into the lens. We acknowledge this honestly. However, we note that the premise is not merely an assumption—it is a logical observation: any improvement loop requires a criterion for 'better,' and that criterion is an evaluator by definition. What is not built into the lens is the empirical claim that stronger evaluators (formal verifiers) produce more durable improvement than weaker ones (intrinsic self-assessment). That claim is supported by independent anchor evidence (Huang et al. [27], the Mirror Loop [140], Lin [92], FunSearch [3], AlphaEvolve [4]) that was not selected by the taxonomy but by its diagnostic quality. Still, the referee is right that the qualitative pattern across the full corpus is observed through the taxonomy's lens, and we will reframe the hierarchy's epistemic status accordingly. We will (a) add a paragraph in §5.2 explicitly acknowledging the taxonomy-corpus circularly","revision_made":"partial","referee_comment":"§5.2 and §5.4: The verification hierarchy is the paper's central load-bearing empirical claim, but it is validated only qualitatively within a corpus whose construction was partly shaped by the taxonomy that encodes the hierarchy. The supplemental harvest (§2.3) targeted 379 papers in 'directions the taxonomy makes first-class,' including self-evaluation methods. The paper's organizing premise—'every improvement loop is a claim that some signal can substitute for human judgment' (§5)—predisposes the analysis to find that evaluator quality is the key variable. If every loop is framed as depending on an evaluator, and loops with stronger evaluators are observed to work better, the conclusion is partly built into the lens."},{"response":"The referee is correct. The claim as stated conflates two distinct bottlenecks: (1) verifying that a solution to a specified problem is correct, and (2) selecting which problems are worth solving. The Anthropic essay [5] treats these as distinct, and our evidence base (Huang et al. [27], Mirror Loop [140], Lin [92]) supports only the first claim. We will revise §5.4 to separate the two bottlenecks and argue the relationship rather than asserting identity. The argument we can honestly make is narrower: direction-setting is a harder problem than verification, and it sits at or above the top of the verification hierarchy because it requires evaluating not just whether a solution is correct but whether a problem is worth posing—a judgment for which no formal verifier exists. This makes direction-setting a superset of the verification problem, not the same problem. The evidence for this narrower claim comes from §6.3's diagnostic literature (ScienceAgentBench [177], ResearchArena [178], the integrity benchmarks [143]) and from Herrmann and Schmidhuber's formalization of interestingness as a bottleneck [190], which identifies problem selection as requiring inductive heuristics that go beyond verification. We will rewrite the claim to state that direction-setting subsumes the verification bottleneck and adds the further difficulty of evaluating problem importance, for which no trustworthy automated signal currently exists. We will remove the assertion of identity and replace it with this argued relationship.","revision_made":"yes","referee_comment":"§5.4: The claim that 'the direction-setting bottleneck and the verification bottleneck are the same bottleneck' is asserted rather than demonstrated. The Anthropic essay [5] identifies research direction-setting as a distinct bottleneck (choosing which problems matter), while the verification hierarchy concerns signal reliability for already-specified tasks. The conflation of these two different bottlenecks—problem selection and solution verification—is load-bearing for the paper's framing but is not separately argued. The within-domain evidence (Huang et al. [27], Mirror Loop [140], Lin [92]) supports the verification claim but does not address whether direction-setting reduces to verification."},{"response":"This is a fair and actionable suggestion. We will add the requested stratified robustness check: for each rung of the verification hierarchy, we will report the distribution of supporting evidence across publication years (2024, 2025, 2026). This will let readers assess whether the hierarchy is driven by the 2026 mass or holds across the corpus's temporal span. We expect the hierarchy to be visible in the 2024 anchor works (STaR [7], Self-Refine [6], Huang et al. [27], FunSearch [3]) and in the 2025 diagnostic literature (Mirror Loop [140], Lin [92]), not only in the 2026 wave, but the referee is right that this should be shown rather than assumed. We will also add an explicit caveat in §5.2 noting that the hierarchy's support is concentrated in verifiable domains (code, math, formal methods) and that its applicability to non-verifiable domains is inferred by absence (the lack of demonstrated durable self-improvement without external signal) rather than by positive evidence of failure. The recency interaction is real: if the field's attention has shifted toward verifiable domains, the hierarchy could partly reflect where activity is dense. The stratified check will help distinguish these possibilities, though we acknowledge it cannot fully separate a genuine regularity from a field-wide attention shift—both could produce the same stratified pattern. We will state this limitation explicitly.","revision_made":"yes","referee_comment":"§2.3: The corpus construction has a recency bias (74% from 2026) that interacts with the verification hierarchy claim in a way that is not fully addressed. If the field has recently concentrated on verifiable domains (code, math) where self-improvement works, the hierarchy could partly reflect where current research activity happens to be dense rather than a stable empirical regularity. The authors acknowledge this for citation counts but not for the hierarchy claim itself. A simple robustness check—reporting the hierarchy's support stratified by publication year—would help distinguish a real regularity from a recency artifact."}],"tokens_in":35900,"tokens_out":1757,"duration_ms":318334,"standing_objections":["The structural circularity between taxonomy-guided corpus construction and the verification hierarchy claim cannot be fully resolved. The taxonomy shapes the corpus, and the corpus is used to validate a claim encoded in the taxonomy. We can make this fully transparent, reframe the hierarchy's epistemic status as a qualitative pattern rather than a measured law, and point to independent anchor evidence, but we cannot make the corpus independent of the taxonomy without rebuilding it from scratch using taxonomy-free queries—which would lose the targeted coverage of self-evaluation that makes the survey distinctive."]},"desk_editor":{"model":"glm-5.2","letter":"Bottom line: this is a strong organizational survey of a fragmented literature. The two-axis taxonomy (what is improved × loop closure) and the dedicated treatment of self-evaluation as a load-bearing pillar are real contributions. The verification hierarchy — formal verifiers at the top, intrinsic self-assessment at the bottom — is a useful organizing claim, but it is explicitly labeled qualitative and rests on a corpus whose construction was partly shaped by the taxonomy that encodes the hierarchy concept. That circularity is the paper's main soft spot, and the authors are transparent about it, which is appropriate but does not resolve it. The paper deserves a serious referee. The taxonomy and the synthesis of failure modes across sub-fields are the durable contributions. The corpus and classification scripts are released, which is real evidence of methodological transparency. The argumentative skeleton rests on verified anchor works (STaR, Self-Refine, FunSearch, AlphaEvolve, Huang et al.'s negative result on self-correction, Shumailov et al.'s model collapse result) rather than on the 2026 mass, which is the right approach given the recency bias. The within-domain evidence for the hierarchy is genuinely supportive: Huang et al. [27] on intrinsic self-correction failure, the Mirror Loop study [140] on ungrounded self-critique, Lin's rise-and-collapse [92], and the entropy-reservoir unification [145] are all independent work that lines up with the claim. The cross-domain regularity — the claim that makes this a 'reframing of the RSI debate' — is where the circularity concern lands. The supplemental harvest targeted directions 'the taxonomy makes first-class' (§2.3), including self-evaluation as a dedicated category. If every loop is framed as depending on an evaluator, and loops with stronger evaluators are observed to work better, the conclusion is partly built into the lens. The taxonomy elevates self-evaluation, the supplemental harvest fills that category, and then the hierarchy is observed across all categories including the one constructed to embody it. This does not make the claim wrong — the within-domain evidence is independent and supportive — but the cross-domain regularity rests on a corpus whose structure was not independent of the claim being tested. Converting the qualitative observation into a quantitative one (effect sizes by evaluator type across an independently-sampled corpus) is the missing step. The single-annotator classification and the recency bias (74% from 2026) are minor concerns relative to this; the authors flag both honestly. The A-Evolve-Training episode [175] — where a loop detected its own proxy's corruption and revised its search policy — is a striking field observation and deserves the attention the paper gives it. This paper is for researchers working on self-improvement loops, RSI safety, and AI governance who need a map of a rapidly expanding field. It provides one. The framework's durability depends on future quantitative validation of the hierarchy regularity, but the taxonomy and failure-mode synthesis are useful now regardless of whether the hierarchy turns out to be a sampling artifact or a real regularity. Recommend accepting for peer review. The referee should push the authors to (1) acknowledge the framing-observation circularity more explicitly, (2) test whether the hierarchy survives within the seed corpus alone (pre-supplemental harvest), and (3) consider whether any quantitative validation is feasible even at a coarse grain. None of these are blockers for review.","headline":"A well-executed survey with a genuinely useful taxonomy, whose central empirical claim is honestly caveated but unvalidated","tokens_in":36836,"tokens_out":786,"would_cite":true,"duration_ms":221720,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Self-improving AI lives or dies by its verifier","keywords":["recursive self-improvement","verification hierarchy","self-evaluation","AI safety","model collapse","LLM self-training","evaluator co-evolution","AI governance"],"falsifier":"Find a substantial body of cases where self-improvement loops succeed reliably using only intrinsic signals (model confidence, self-consistency) without external verification — this would break the claimed hierarchy by showing the bottom rung can sustain improvement at scale.","tokens_in":36055,"feed_emoji":"","tokens_out":1276,"duration_ms":300400,"temperature":0.7,"pith_summary":"This survey of 1,250 papers argues that the entire landscape of AI self-improvement — from inference-time output revision to autonomous research agents — is governed by a single organizing principle: the strength of any self-improvement loop is bounded by the reliability of the signal that tells it whether it got better. The authors propose a two-axis taxonomy separating what a system improves (its outputs, its weights, its evaluator, or the research process itself) from how closed the loop is (human-in-the-loop to fully autonomous). Across all four categories, they observe that demonstrated improvement tracks a verification hierarchy: formal proof checkers at the top, execution feedback next, learned judges below that, and intrinsic self-assessment at the bottom. The characteristic failure modes — self-confirming loops, model collapse, diversity collapse — arise precisely when a system's evaluator sits too low on this hierarchy relative to its ambition. The survey's most consequential claim is that the bottleneck preventing fully autonomous AI research (choosing which problems matter) and the bottleneck preventing reliable self-evaluation are the same bottleneck, and that this bottleneck sits at the top of the verification hierarchy where no current system can operate without human judgment.","feed_headline":"Self-improving AI lives or dies by its verifier","feed_subtitle":"A survey of 1,250 papers argues every AI self-improvement loop is bounded by one factor: how reliable the signal is that tells it whether it","key_machinery":"The verification hierarchy: a four-level ordering of evaluator reliability from formal verifiers (strongest) through execution feedback, learned judges, to intrinsic self-assessment (weakest), which the authors claim predicts both where self-improvement works and where it fails.","core_discovery":"The central discovery is the verification hierarchy as an empirical regularity across the self-improvement literature. Formal verifiers (proof checkers, type systems) are sound by construction and permit indefinite iteration without accepting false improvements. Execution feedback (tests, benchmarks) is reliable but incomplete. Learned judges (reward models, LLM-as-judge) are bounded by the judge's own competence and are themselves gameable. Intrinsic signals (confidence, self-consistency) are cheapest and most gameable. The authors observe that every demonstrated self-improvement success in the corpus sits at the top two rungs (code, math, formal verification), while every persistent gap —,","pith_inferences":["If the verification hierarchy generalizes beyond the corpus, one would predict that self-improvement in domains with cheap verifiers (software, formal math, competitive programming) will continue to advance rapidly while domains without them (scientific research, creative work, strategic planning) will stall regardless of model scale — a domain-specific ceiling on RSI.","The exchange rate of grounding — the minimum fraction of external signal needed to prevent collapse — is a measurable quantity that could be established experimentally by systematically varying the ratio of exogenous to self-generated data in training loops and observing the collapse threshold, turning a theoretical question into an empirical one.","If research taste (identifying which problems are worth working on) is the top rung of the hierarchy and is beginning to acquire a formal theory, then benchmarks for interestingness or problem-selection quality would be the single highest-leverage intervention for the field, as they would convert the last human-in-the-loop bottleneck into a measurable target.","The A-Evolve-Training episode, where a system detected its own proxy metric corrupting and revised its search policy, may represent the first field observation of a system climbing the verification hierarchy autonomously — if this capability generalizes, it would be evidence that evaluator self-correction is achievable, weakening the claim that the verification bottleneck is permanent."],"forward_implications":["If the verification hierarchy is real, the path to more autonomous AI research runs through evaluator engineering, not raw model capability — the binding constraint is building systems that can reliably judge open-ended quality, not systems that are merely smarter.","Governance of self-improving AI becomes a measurement problem: regulators would need to audit not what a system produces but what evaluator it runs against and where that evaluator sits on the hierarchy.","The distinction between bounded self-refinement (convergent, evaluable, already industrial) and open-ended RSI (divergent, unverified, still theoretical) gives policymakers a concrete vocabulary for separating near-term engineering risks from speculative takeoff scenarios.","Evaluator co-evolution — the emerging practice of having systems improve their own verifiers alongside their policies — is identified as the pivotal empirical question: it either escapes the self-confirming loop or gives it a second story, and the answer determines whether closed-loop self-improvement stabilizes or compounds bias.","The field's smallest category (foundations, limits, and safety at 60 of 1,250 papers) represents the largest mismatch between stated stakes and research investment, suggesting the academic incentive structure is underweighting exactly the questions that governance depends on."],"fun_headline_variants":["Every AI self-improvement loop is bounded by verifier strength","Self-improvement works where formal verification exists and fails where it doesn't","AI self-improvement success tracks a verification hierarchy from proof checkers to confide","The verifier, not the loop, determines whether AI self-improvement holds or collapses","Survey of 1,250 papers: AI self-improvement strength follows verification hierarchy"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The verification hierarchy is described as a qualitative pattern observed across the corpus, not a measured law. Because the corpus was assembled by seed queries that may systematically over-represent verifiable domains (code, math) where self-improvement works and under-represent domains where weaker signals succeed, the hierarchy could partly reflect sampling bias rather than a genuine empirical regularity about AI self-improvement.","fun_headline_variants_meta":{"raw":{"variants":["Every AI self-improvement loop is bounded by verifier strength","Self-improvement works where formal verification exists and fails where it doesn't","AI self-improvement success tracks a verification hierarchy from proof checkers to confidence","The verifier, not the loop, determines whether AI self-improvement holds or collapses","Survey of 1,250 papers: AI self-improvement strength follows verification hierarchy"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":789,"prompt_tokens":703,"completion_tokens":86,"prompt_tokens_details":null},"tokens_in":703,"tokens_out":86,"duration_ms":66202,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T03:07:35.745829+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Find a substantial body of cases where self-improvement loops succeed reliably using only intrinsic signals (model confidence, self-consistency) without external verification — this would break the claimed hierarchy by showing the bottom rung can sustain improvement at scale.","supporting_citations":[],"review_version":1}