Mendel G\"odel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution

Changzhi Liu; Sikuan Yan; Volker Tresp; Yilun Liu; Yunpu Ma

REVIEW 4 major objections 5 minor 299 references

Mendel G\"odel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that coding agents learn to improve themselves faster when each self-edit is informed by comparisons across tasks and lineages rather than by a single failure.

desk verdict MGM is an honest, well-scoped extension of archive-based self-improvement for coding agents; the benchmark gains look real but the mechanism isn't yet directly tested. read the letter →

arxiv 2608.07645 v1 pith:XLKV2DEZ submitted 2026-08-07 cs.AI cs.LG

Changzhi Liu , Yilun Liu , Sikuan Yan , Volker Tresp , Yunpu Ma This is my paper · ORCID

classification cs.AIcs.LG

keywords self-improvingagentsrecursiveself-modificationevolutionarycomputationcodingreaction-normmutationcross-lineagehybridizationLLMsoftwareengineeringbenchmarks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a recursive self-improving coding agent improves faster and further when each self-modification is conditioned on comparative evidence from the archive of its own past attempts, instead of on a single failure trajectory. It adds two comparative operators to the usual clonal mutation: reaction-norm mutation, which edits an agent using its trajectories across several tasks, and cross-lineage hybridization, which edits an agent using a reference agent from another lineage on the same task. Under an additive fitness landscape, the paper proves that both operators have strictly higher effective fix probability than clonal mutation, and controlled surrogate simulations confirm faster expected convergence. Under a matched 200-evaluation budget, the method reaches 78.3% on SWE-bench Verified-60 and 93.2% on Polyglot-60, versus 73.3% and 77.9% for the single-trajectory baseline. This matters because comparative operators reuse trajectories already collected, so the gain costs no extra evaluations.

What carries the argument

The load-bearing object is the effective fix probability $$p_\sigma^f = s \cdot \Pr_{\ell \sim C_\sigma(E)}[\ell \in M(a)]$$, where $$M(a)$$ is the set of incorrect loci of agent $$a$$, $$s$$ is the probability that an edit repairs a locus it targets, and $$C_\sigma(E)$$ is the candidate set of loci the operator infers from its evidence. Clonal mutation searches the full failed task's locus set $$C_{\mathrm{CM}} = R_{\tau_t}$$; reaction-norm mutation searches the intersection $$C_{\mathrm{RM}} = R_{\tau_t} \cap R_{\tau_r}$$, which is smaller when failures share a recurring defect; cross-lineage hybridization searches the contrastively filtered set $$C_{\mathrm{CH}} = (R_\tau \cap M(a_t)) \cup N$$, where a successful reference agent removes non-causal loci. The additive fitness landscape model represents each agent as a binary genotype, with utility as the Hamming distance to an oracle, and a task is solved only if all $$k$$ loci it examines are correct. Proposition 1 converts these candidate-set containments into strict inequalities on $$p_\sigma^f$$, which is what the Monte Carlo surrogate then instantiates as the fix-probability ratio $$\rho$$.

What would settle it

Run the surrogate or the real pipeline with tasks deliberately engineered so that failures never share a common underlying defect and no reference agent supplies a meaningful contrast; if MGM's convergence then matches the single-trajectory baseline, the sound-comparison premise is falsified. A more direct measurement is to compare the actual probability that a targeted locus is truly incorrect under each operator's candidate set; if $$\Pr[\ell \in M(a) \mid \ell \sim C_{\mathrm{RM}}]$$ is not greater than the same probability for $$C_{\mathrm{CM}}$$, Proposition 1's diagnostic-compression mechanism is not operating in practice.

Watch

Extended reading notes

Core claim

The central discovery is that the bottleneck in recursive self-improvement is diagnostic: a self-modification operator only helps when it can localize a real scaffold-level defect. MGM treats the agent's source code as a genotype and its evaluation outcomes as phenotypes, then structures self-modification as Mendelian controlled comparisons. Clonal mutation edits from a single failed task; reaction-norm mutation asks the agent to compare its own trajectories across multiple tasks; cross-lineage hybridization asks a failing agent to compare itself with a reference agent that attempted the same task. The paper formalizes an additive fitness landscape in which each task requires several loci to be simultaneously correct, and shows that comparative evidence compresses the candidate set of defective loci: the intersection of two failures is smaller than one failure's task-relevant region, and a successful reference lineage filters out non-causal loci. Proposition 1 states that under sound comparative evidence, both comparative operators have strictly higher effective fix probability than clonal mutation. The agent is not a stronger editor; it is an editor given cleaner hypotheses.

Load-bearing premise

The entire advantage depends on the assumption that when the same agent fails repeatedly, the failures share an underlying weakness in the agent's own code, and that watching a successful agent on the same task reveals which parts of the task were not responsible; if most failures are just unrelated accidents, the comparative edits give no extra information and the method reverts to the standard single-failure baseline.

Editorial extensions

If this is right

Under the additive fitness model with sound comparative evidence, reaction-norm mutation and cross-lineage hybridization have strictly higher effective fix probability than clonal mutation, so each self-edit has a higher chance of correcting a real defect per unit budget.
Under a matched budget of 200 evaluations and 24 expansions, MGM improves SWE-bench Verified-60 from 68.3% to 78.3% and Polyglot-60 from 50.8% to 93.2%, against 73.3% and 77.9% for the single-trajectory baseline.
Evolved scaffolds generalize: a Polyglot-evolved MGM scaffold transfers zero-shot to SWE-bench Pro (26.7% vs. 13.3% for baseline) and SWE-bench Multilingual (55.0% vs. 43.3%), and a Qwen-evolved MGM scaffold transferred to DeepSeek backbones averages 70.8% on SWE-bench Verified-60 versus 65.0% for the baseline.
Ablations show both comparative operators matter: removing reaction-norm mutation drops Polyglot-60 from 93.2% to 79.7%, and removing cross-lineage hybridization drops it to 74.6%.
All three operators have comparable token costs, so the reported gains are attributed to diagnostic quality rather than to spending more compute on edits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper only claims results for coding-agent scaffolds, but the diagnostic-compression mechanism is generic: any archive of outcome traces for a self-modifying system could in principle use the same two comparisons, so a natural extension is to test MGM-style operators on non-coding agent loops that accumulate logged successes and failures.
The null case $$\rho=1$$ in the simulations implies a practical diagnostic: track whether comparative edits actually outperform clonal edits; if the gap narrows, the archive lacks informative contrasts and task sampling should be reweighted toward shared, repeated failure modes.
A sharp testable extension would be to seed the archive with corrupted contrasts, tasks whose failures are deliberately independent, and verify that MGM's advantage over the baseline vanishes, as Proposition 1's sound-comparison premise predicts.
Because the paper's own appendix shows a larger coding-specialized backbone can evolve worse than a smaller general-reasoning one, the limiting resource may be the editor's ability to reason over comparative evidence; an interesting follow-up is to measure how operator gains scale with backbone reasoning ability.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper probably does what it claims — two new comparative self-modification operators that improve coding-agent scaffold evolution under a matched budget — and it is honest about the gap between its theory and its evidence. The two operators, reaction-norm mutation and cross-lineage hybridization, are genuinely new relative to the DGM/HGM line, and the framing of self-modification as diagnostic compression over candidate loci is a useful way to think about the problem. The empirical section is well controlled: same ancestor, same 200-evaluation budget, same number of expansions, comparable token costs, and ablations show both operators matter. Cross-benchmark and cross-model transfer results are a strong plus.

The soft spots are real but not fatal. Proposition 1 is a conditional statement: it holds only under 'sound comparative evidence,' meaning repeated failures share a recurring defect or a successful reference filters out non-causal loci. That is acknowledged in the appendix. The surrogate simulation does not close the gap — it takes the fix-probability advantage rho as an input, so it tests the downstream dynamics of an assumed diagnostic benefit, not whether the operators actually earn that benefit in real coding failures. The benchmark gains are consistent with the diagnostic-compression story, but they could also come from added exploration or diversity that the operators introduce. The main results are single runs on fixed 60-task subsets chosen by ChatGPT, so variance is unknown. None of these issues sink the paper; they are addressable with multi-seed runs, random subset replication, and a direct measurement of comparative fix probability from archived trajectories.

If I were editing, I'd send this to peer review. The work is clear, the claims are scoped, and the contribution is useful to anyone building self-improving coding agents. I'd ask reviewers to demand multi-seed experiments and a more direct test of the mechanism before acceptance, but the paper deserves a serious referee.

Referee Report

4 major / 5 minor

Summary. The manuscript introduces the Mendel Gödel Machine (MGM), an archive-based self-improving coding-agent framework that extends the HGM/DGM lineage by replacing the single self-modification operator with three evidence-conditioned operators: clonal mutation (single failed trajectory), reaction-norm mutation (multiple trajectories of one agent across tasks), and cross-lineage hybridization (trajectories of different lineages on a shared task). The authors develop an additive binary-locus fitness model in which tasks probe random k-locus subsets, and state Proposition 1 asserting that, under 'sound comparative evidence,' reaction-norm mutation and cross-lineage hybridization have higher effective fix probability than clonal mutation. They then run Monte Carlo simulations in which the fix-probability advantage enters as an input parameter ρ, and they report empirical results on SWE-bench Verified-60, Polyglot-60, SWE-bench Pro, SWE-bench Multilingual, and cross-model transfer, together with a two-way ablation. The headline empirical claim is that, under a matched 200-evaluation budget, MGM reaches 78.3% on SWE-bench Verified-60 and 93.2% on Polyglot-60 versus 73.3% and 77.9% for HGM, and that removing either comparative operator degrades performance. The manuscript includes a detailed limitations section acknowledging conditional assumptions, single-run primary evolution, and residual subset/seed uncertainty.

Significance. If the central claims are established, the paper addresses a genuinely under-explored bottleneck in self-improving coding agents: the quality of the evidence used at each self-modification step rather than just the archive sampling policy. The operator design is clean and the idea of using archived trajectories as controlled comparisons is plausible and independent of the specific LLM backbone. The paper also ships concrete assets: code and project pages are promised; the experiments use a matched budget; the ablation includes both comparative operators; and the full Polyglot-225 evaluation provides a useful check against subset artifacts. The theoretical contribution is best read as a conditional analysis of when comparative evidence helps, and the authors are unusually candid about the assumptions (sparse defects, recurring genotype-level weaknesses, informative reference trajectories) and about the fact that primary evolution is reported on a single run. Those strengths do not, however, remove the need to test the comparative-evidence premise directly or to provide variance evidence for the empirical ordering.

major comments (4)

[§4.2 / Appendix B.1, Eqs. (38) and (48)] Proposition 1 is not a theorem about the operators under the generative model of Eqs. (20)–(23); it is a conditional statement that assumes a recurring genotype-level defect b lies in Rτt∩Rτr and that the reference comparison isolates the target-specific causal set, CCH=(Rτ∩M(at))∪N with |CCH|<k. Two failed trajectories under the model are consistent with disjoint causal loci, and a successful reference trajectory only guarantees Rτ∩M(ar)=∅; neither observation by itself identifies the target's defective loci. The derivation inAppendix B.1 is internally valid as a conditional proof, but the abstract and Section 1 overstate the result by claiming a proof of faster convergence, since the existence of 'sound comparative evidence' is an extra empirical premise rather than a consequence of the additive landscape. I recommend restating Proposition 1 as an explicit conditional theorem and providing a direct or proxy test of the sparse-defect and recurrence assumptions.
[§4.3, Eq. (26)] The Monte Carlo simulation takes the diagnostic advantage ρ=pRM_f/pCM_f=pCH_f/pCM_f as an input, so it cannot independently validate Proposition 1 or measure whether real coding failures satisfy its premise. The null case ρ=1 demonstrates only that the assumed advantage is load-bearing, and the real-benchmark gains in Table 1 are equally consistent with ΦRM and ΦCH acting as extra exploration or diversity operators rather than as diagnostic compression. To support the causal mechanism, the paper should either measure or approximate candidate-set density from actual trajectories (for example, by comparing edited loci against oracle-relevant loci in a controlled setting, or by ablating against an operator that adds diversity without comparative conditioning) or explicitly label the simulation as a consequence analysis under the assumed advantage.
[§5, Tables 1–4; Limitations] The headline empirical comparisons are all single evolution runs on fixed 60-task subsets, with no independent seeds, confidence intervals, or statistical tests. Given that the paper claims 'consistent improvement' across benchmarks, ablations, and transfer settings, the single-run status is load-bearing: Appendix E.2 shows that evolution trees contain high-utility nodes evaluated only a handful of times, so the procedure for selecting the 'best-belief agent' is noise-sensitive. I ask for at least a small number of independent evolution seeds, or a variance-sensitive analysis such as bootstrapping over the 60-task subset and over Thompson-sampling randomness, to establish that the HGM–MGM gap and the ablation ordering are stable rather than artifacts of one realization.
[Abstract / §1 / §4.2] The formal result actually obtained is a comparison of per-edit effective fix probabilities, pRM_f>pCM_f and pCH_f>pCM_f, not a theorem about expected convergence speed under the archive-search process. The claim of 'strictly faster expected convergence' is demonstrated only by simulation with ρ as an input. The manuscript should either add a theorem linking higher per-edit fix probability to faster convergence under the HGM evaluation and expansion policies, or soften the abstract and Section 1 to state that the model and simulation provide a conditional argument rather than a proof of convergence.

minor comments (5)

[§4.1, Eq. (23)] With fixed-size task subsets |Rτ|=k, the per-task success probability at edit distance d is hypergeometric, C(L−d,k)/C(L,k), rather than ((L−d)/L)^k; the current formula corresponds to independent per-locus inclusion. Please correct the expression or state the sampling convention explicitly.
[§3.2, Eq. (17) and Table 6] The experimental hyperparameter is βfail=1.0, which makes the failed-pool boost exactly neutral; the text's claim that the failed-task pool 'deliberately creates overlap across lineages' is therefore not exercised in the reported experiments. Please clarify that the pool is inert in this configuration and that overlap comes from the common task distribution.
[Appendix E.1 / Figure 2] The full Polyglot-225 evaluation reports 93.3% (210/225), while Table 1 reports 93.2% on Polyglot-60, and Figure 2 plots an asterisked HGM point from Polyglot-60 alongside full-benchmark points; the exact mapping between subset and full-benchmark numbers should be stated so the figure cannot be misread as comparing different scales.
[Appendix D] The 60-task subsets for SWE-bench Pro and SWE-bench Multilingual are selected via 'ChatGPT' without reporting the selection prompt, sampling temperature, or a seed; since the headlined gains inherit sensitivity to subset composition, please include the full selection protocol and a reproducibility statement for these subsets.
[Tables 1–4] The notation such as '78.3+10.0' is intended to show absolute percentage-point improvement over the initial agent, but as rendered it can be misread as a value plus an error or confidence interval; a separate delta column would be clearer.

Circularity Check

1 steps flagged · score 4.0 of 10

The surrogate simulation injects the comparative fix-probability advantage as an input and then reports it as a confirmed gain; the formal proposition is explicitly conditional, and the real-benchmark comparisons provide independent but single-run grounding.

other [Section 4.3, Eq. (26), Table 5, Figures 4–5]
"Guided by Proposition 1, we instantiate the diagnostic advantage of comparative evidence through a controllable fix-probability ratio rho = p_RM_f / p_CM_f = p_CH_f / p_CM_f. ... This confirms that the simulated gain is caused directly by the diagnostic-quality advantage formalized in Proposition 1."

The simulation sets p_RM_f = rho * p_CM_f and p_CH_f = rho * p_CM_f with rho > 1 (Table 5), so the plotted MGM advantage over HGM is a direct arithmetic consequence of that injected rho. The simulation therefore does not provide independent evidence that the reaction-norm or cross-lineage operators actually achieve a higher fix probability; it verifies only the conditional statement 'if p_RM, p_CH > p_CM, then MGM converges faster,' which is true by construction from Eq. (26). The rho = 1 null case merely confirms that without the injected advantage there is no advantage. This is an input restated as a confirmed prediction, though the paper honestly labels rho as a controlled parameter and the B.1 discussion explicitly disclaims unconditional superiority.

full rationale

No self-citation chain, uniqueness import, or ansatz-smuggling appears: the baselines (DGM, HGM) are prior work by other authors, and the present paper's operators are not justified by a circular citation. Proposition 1 itself is an explicit conditional proof: Appendix B.1 states that p_RM > p_CM requires the assumption that two failures share a recurring causal defect b (Eq. 38), and p_CH > p_CM requires that the contrastive candidate set C_CH = (R_tau ∩ M(a_t)) ∪ N filters out enough non-causal loci (Eq. 48). Those properties are not implied by the observation model in Eqs. (20)–(23), but the paper acknowledges them as assumptions rather than hiding them. The main circular move is the simulation: it instantiates the very diagnostic advantage that Proposition 1 is supposed to establish, so the Monte Carlo result is conditional rather than confirmatory. The real-benchmark evidence (Table 1 and ablations in Table 4) is independent of that simulation: HGM and MGM start from the same scaffold under a matched 200-evaluation budget, and the gains are externally grounded, though only single-run and on fixed 60-task subsets. The Limitations section also concedes that the surrogate is controlled and may not capture real coding-failure structure. Taking all of this together, the central empirical claim has independent content, but the controlled-simulation validation reduces by construction, giving a moderate partial circularity score.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central convergence result lives in the additive fitness model; the real-world claim additionally rests on the assumption that LLM editors can exploit comparative trajectories. The main hand-set quantities are rho, simulation parameters, and operator sampling weights; none are fitted to the benchmark results, but several directly control whether MGM appears to help.

free parameters (9)

fix-probability advantage ratio rho = 2.0 default; swept {1.0, 1.2, 1.5, 2.0}
Sets pRM_f and pCH_f as multiples of pCM_f (Eq. 26); the simulation's positive MGM result is driven by this assumed diagnostic advantage, which is not measured on real agents.
initial edit distance d0 = {10, 20, 40, 80}
Swept difficulty parameter in the additive landscape; controls how hard the simulated convergence problem is.
loci examined per task k = 5
Task subset size in Eq. (21); success probability and the size of compression both depend on it.
chain length L = 100
Genotype length in Section 4.1; sets the scale for candidate-set compression.
task pool size N = 200
Simulation environment size; affects how often shared-task comparisons are available.
break probability p_b = 0.05
Probability that an edit corrupts a correct locus; chosen by hand and affects convergence rates.
strategy weights lambda_CM, lambda_RM, lambda_CH = 0.10, 0.45, 0.45
Sampling weights in Eq. (18) used in all experiments; hand-set in Table 6 without sensitivity analysis.
failed-pool boost beta_fail = 1.0
Oversampling weight for tasks that previously failed (Eq. 17); hand-set and not swept.
minimum tasks for Phi_RM, mRM = 2
Eligibility threshold in Eq. (10); the smallest value that permits a two-task comparison.

assumptions (6)

domain assumption Agent genotypes are binary vectors and a fixed oracle g*=1 exists; improvement equals reduction in Hamming distance to the oracle.
Section 4.1, Eq. (20); the formal convergence result is defined only in this abstract fitness landscape.
domain assumption Each task examines a fixed subset R_tau of k loci, and the agent solves the task iff all loci in R_tau are correct.
Section 4.1, Eq. (21); maps coding tasks to independent capability requirements without evidence that real benchmarks behave this way.
ad hoc to paper Repeated failures of the same agent share a recurring causal defect b contained in the intersection of the tasks' examined loci.
Appendix B.1, sound-comparison assumption; without it pRM_f > pCM_f does not follow.
ad hoc to paper A successful reference agent on the same task filters out non-causal loci, leaving a small candidate set CCH.
Appendix B.1, Eq. (48); requires cross-lineage transferability of success, which the paper does not verify.
domain assumption The LLM editor can diagnose failure mechanisms from comparative trajectories and target actually incorrect loci.
Limitations section; the paper states that richer evidence only helps when the backbone can exploit it.
domain assumption HGM and MGM operate under exactly matched evaluation and expansion budgets with identical initial scaffolds.
Section 5.1; the headline comparison is valid only under this controlled protocol.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Mendel G\"odel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution." pith.science (2026). https://pith.science/paper/XLKV2DEZ

@misc{pith2026260807645,
  author       = {Pith},
  title        = {Pith review of: Mendel G\"odel Machine: Recursive Self-Improving Coding Agents via Comparative Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XLKV2DEZ}},
  note         = {Machine review of arXiv:2608.07645}
}

read the original abstract

Self-improving coding agents that iteratively rewrite their own source code have demonstrated impressive performance on coding tasks. However, existing solutions generally derive self-modification from a single failure trajectory at a time, overlooking rich comparative signals available in the agent's expanding archive of past attempts. According to Mendelian principles of controlled inheritance, we introduce Mendel G\"odel Machine (MGM). In addition to the general single-trajectory clonal mutation, MGM includes two new types of self-modification that better utilizes evidences accumulated: the reaction-norm mutation edits an agent based on its trajectories on multiple tasks simultaneously, and the cross-lineage hybridization edits an agent using the trajectory of a reference agent from another lineage on the same task. Under an additive fitness landscape model, we prove theoretically and demonstrate via controlled surrogate simulation that the new strategies facilitate a faster and better convergence over single-trajectory baselines. Experiments on SWE-bench and Polyglot confirm MGM's consistent improvement in performance, efficiency, and generalizability.

Figures

Figures reproduced from arXiv: 2608.07645 by the authors.

**Figure 1.** Mendel Gödel Machine. MGM organizes self-modification via controlled inheritance based on evidence across tasks and lineages. The archive maintains a lineage tree of agent variants; each stores its source code as genotype and evaluation outcomes as phenotype. Each iteration applies π-sampling that selects an operation to perform with the necessary resources from the archive. φ-evaluation executes the agent on untest… view at source ↗

**Figure 2.** Polyglot performance. Result marked with asterisk is from Polyglot-60; all other scores are from the complete Polyglot-2251 . Closed-source model sizes follows estimates reported by Li (2026). To isolate the contribution of each dimension rigorously, we develop a formal additive fitness landscape model in which each agent is represented as a binary vector in genotype space, and selfimprovement progress is measured … view at source ↗

**Figure 3.** Additive fitness landscape model. Each agent a carries a binary genotype with loci that are either correct or mismatched relative to an oracle a∞. The genotype is not directly observable and can only be examined through φ-evaluation, where each task τi requires a fixed subset of loci to be all correct for a successful phenotype. Φ-expansion reads phenotype records and modifies the genotype by flipping the examined l… view at source ↗

Figures from the paper (9 more)

**Figure 5.** Figure 5: Simulated final performance distribution. Results compared across task difficulty and edit effectiveness. Rows vary the initial edit distance d0; columns vary the fix-probability advantage ratio ρ. Each panel shows the distribution of final edit distance results (low…

**Figure 6.** Figure 6: Token costs of each operators for HGM and MGM evolved on Polyglot. The distribution of token costs for all evaluation and expansion operations lies in similar order of magnitude. We further analyze the token consumption of each step in the experiments [PITH_FULL_IMA…

**Figure 7.** Figure 7: Polyglot-225 performance of the MGM-discovered agent by language. The agent is evaluated after evolution without further self-modification and solves 210 out of 225 tasks overall. The area of pie slices report the proportion of resolved and unresolved tasks within each…

**Figure 8.** Figure 8: Evolution trees of MGM, HGM, and MGM’s ablated variants. Results after 200 φ-evaluations and 24 Φ-expansions on Polyglot. Nodes are independently colored by utility estimates aggregated over each tree’s φ-evaluation results. only with 15 and 4 evaluations, respectively…

**Figure 9.** Figure 9: Final per-node utilities for MGM, HGM, and the ablations. Each point is one of the 24 evolved nodes. Vertical position and fill color encode utility. Marker area is proportional to the number of evaluations. The marked node in each column is the selected final result …

**Figure 10.** Figure 10: Illustration of Cross-lineage Hybridization. Nodes are colored by their outcome on the shared diagnostic task javascript__queen-attack: red nodes fail and green nodes solve it. then preserved in node #8, which also solves javascript__queen-attack. Crucially, node #8 d…

**Figure 11.** Figure 11: Semantic visualization of evolved skills. A content-level view of the evolved “To Implement” descriptions from MGM and HGM. Each point corresponds to one evolved skill or workflow change, embedded by the semantics of its To Implement text. The marker shape indicates t…

**Figure 17.** Figure 17: Code modification on Polyglot diff --git a/coding_agent.py b/coding_agent.py index 39cb47e..d824c3d 100644 --- a/coding_agent.py +++ b/coding_agent.py @@ -172,7 +172,21 @@ class AgenticSystem: Your task is to make changes to the files in the {self.git_dir} directory t…

**Figure 18.** Figure 18: Code modification on SWE-bench diff --git a/coding_agent.py b/coding_agent.py index d19efd6..98f364a 100644 --- a/coding_agent.py +++ b/coding_agent.py @@ -3,16 +3,19 @@ import argparse import logging import os +import re import subprocess import threading from loggin…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

299 extracted references · 22 canonical work pages

[1]

2018 , title =

Korman, Eric O , journal =. 2018 , title =

2018
[2]

2026 , eprint=

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity , author=. 2026 , eprint=

2026
[3]

Transactions on Machine Learning Research , issn=

A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence , author=. Transactions on Machine Learning Research , issn=. 2026 , url=

2026
[4]

Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , journal =

J. Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , journal =. 2003 , url =

2003
[5]

2021 , title =

Korman, Eric O , booktitle =. 2021 , title =

2021
[6]

Evolutionary Principles in Self-Referential Learning

Jurgen Schmidhuber. Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta...-Hook. 1987

1987
[7]

Verhandlungen der deutschen zoologischen Gesellschaft , volume =

Woltereck, Richard , title =. Verhandlungen der deutschen zoologischen Gesellschaft , volume =
[8]

Pigliucci, Massimo , title =

Show all 299 references

[9]

2023 , title =

Bae, Sangmin and Ko, Jongwoo and Song, Hwanjun and Yun, Se-Young , volume =. 2023 , title =

2023
[10]

Glass , biburl =

Anthony Bau and Yonatan Belinkov and Hassan Sajjad and Nadir Durrani and Fahim Dalvi and James R. Glass , biburl =. Proc. of ICLR , publisher =. 2019 , title =

2019
[11]

Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwal and Ariel Herbert. Advances in Neural Information Processing Systems 33: An...

2020
[12]

2022 , title =

Burns, Collin and Ye, Haotian and Klein, Dan and Steinhardt, Jacob , volume =. 2022 , title =

2022
[13]

Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021) , publisher =

Caselli, Tommaso and Basile, Valerio and Mitrovi. Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021) , publisher =. 2021 , pages =. doi:10.18653/v1/2021.woah-1.3 , url =

2021 doi
[14]

IEEE transactions on pattern analysis and machine intelligence , volume=

Meta-learning in neural networks: A survey , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2021 , publisher=

2021
[15]

Proceedings of the 41st International Conference on Machine Learning , pages=

Promptbreeder: self-referential self-improvement via prompt evolution , author=. Proceedings of the 41st International Conference on Machine Learning , pages=
[16]

2023 , title =

Stephen Casper and Yuxiao Li and Jiawei Li and Tong Bu and Kevin Zhang and Kaivalya Hariharan and Dylan Hadfield-Menell , volume =. 2023 , title =

2023
[17]

Scientific reports , publisher =

Caucheteux, Charlotte and Gramfort, Alexandre and King, Jean-R. Scientific reports , publisher =. 2022 , pages =

2022
[18]

Manco and E

Michelangelo Ceci and Corrado Loglisci and G. Manco and E. Masciari and Z. Ras and R. Goebel and Yuzuru Tanaka , journal =. 2020 , title =. doi:10.1007/978-3-030-48861-1 , url =

2020 doi
[19]

Duo Chai and Wei Wu and Qinghong Han and Fei Wu and Jiwei Li , biburl =. Proc. of ICML , publisher =. 2020 , pages =

2020
[20]

and Yang, Qiang and Xie, Xing , journal =

Chang, Yupeng and Wang, Xu and Wang, Jindong and Wu, Yuan and Yang, Linyi and Zhu, Kaijie and Chen, Hao and Yi, Xiaoyuan and Wang, Cunxiang and Wang, Yidong and Ye, Wei and Zhang, Yue and Chang, Yi and Yu, Philip S. and Yang, Qiang and Xie, Xing , journal =. 2024 , publisher =

2024
[21]

Chen, Jiaao and Yang, Zichao and Yang, Diyi , address =. Proc. of ACL , publisher =. 2020 , pages =. doi:10.18653/v1/2020.acl-main.194 , url =

2020 doi
[22]

2023 , title =

Chen, Yanxi and Pan, Xuchen and Li, Yaliang and Ding, Bolin and Zhou, Jingren , volume =. 2023 , title =

2023
[23]

Conneau, Alexis and Kiela, Douwe and Schwenk, Holger and Barrault, Lo. Proc. of EMNLP , publisher =. 2017 , pages =. doi:10.18653/v1/D17-1070 , url =

2017 doi
[24]

Dai, Damai and Dong, Li and Hao, Yaru and Sui, Zhifang and Chang, Baobao and Wei, Furu , address =. Proc. of ACL , publisher =. 2022 , pages =. doi:10.18653/v1/2022.acl-long.581 , url =

2022 doi
[25]

Fahim Dalvi and Abdul Rafae Khan and Firoj Alam and Nadir Durrani and Jia Xu and Hassan Sajjad , biburl =. Proc. of ICLR , publisher =. 2022 , title =

2022
[26]

Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , address =. Proc. of NAACL-HLT , publisher =. 2019 , pages =. doi:10.18653/v1/N19-1423 , url =

2019 doi
[27]

2020 , title =

Dodge, Jesse and Ilharco, Gabriel and Schwartz, Roy and Farhadi, Ali and Hajishirzi, Hannaneh and Smith, Noah , volume =. 2020 , title =

2020
[28]

Durrani, Nadir and Sajjad, Hassan and Dalvi, Fahim and Belinkov, Yonatan , address =. Proc. of EMNLP , publisher =. doi:10.18653/v1/2020.emnlp-main.395 , year =

2020 doi
[29]

2022 , title =

Durrani, Nadir and Dalvi, Fahim and Sajjad, Hassan , volume =. 2022 , title =

2022
[30]

2021 , journal=

A Mathematical Framework for Transformer Circuits , author=. 2021 , journal=

2021
[31]

2022 , journal=

Softmax Linear Units , author=. 2022 , journal=

2022
[32]

2023 , title =

Elhage, Nelson and Lasenby, Robert and Olah, Christopher , journal =. 2023 , title =

2023
[33]

2024 , title =

Orgad, Hadas and Toker, Michael and Gekhman, Zorik and Reichart, Roi and Szpektor, Idan and Kotek, Hadas and Belinkov, Yonatan , journal =. 2024 , title =

2024
[34]

Findings of the Association for Computational Linguistics ACL 2024 , publisher =

Jiao, Difan and Liu, Yilun and Tang, Zhenwei and Matter, Daniel and Pfeffer, J. Findings of the Association for Computational Linguistics ACL 2024 , publisher =. doi:10.18653/v1/2024.findings-acl.277 , year =

2024 doi
[35]

2022 , pages =

Fedus, William and Zoph, Barret and Shazeer, Noam , number =. 2022 , pages =

2022
[36]

Feng, Fangxiaoyu and Yang, Yinfei and Cer, Daniel and Arivazhagan, Naveen and Wang, Wei , address =. Proc. of ACL , publisher =. 2022 , pages =. doi:10.18653/v1/2022.acl-long.62 , url =

2022 doi
[37]

Gao, Tianyu and Yao, Xingcheng and Chen, Danqi , address =. Proc. of EMNLP , publisher =. 2021 , pages =. doi:10.18653/v1/2021.emnlp-main.552 , url =

2021 doi
[38]

Geva, Mor and Schuster, Roei and Berant, Jonathan and Levy, Omer , address =. Proc. of EMNLP , publisher =. doi:10.18653/v1/2021.emnlp-main.446 , year =

2021 doi
[39]

Neural computation , publisher =

Jacobs, Robert A and Jordan, Michael I and Nowlan, Steven J and Hinton, Geoffrey E , number =. Neural computation , publisher =. 1991 , pages =

1991
[40]

2013 , title =

Eigen, David and Ranzato, Marc'Aurelio and Sutskever, Ilya , journal =. 2013 , title =

2013
[41]

Science China technological sciences , publisher =

Qiu, Xipeng and Sun, Tianxiang and Xu, Yige and Shao, Yunfan and Dai, Ning and Huang, Xuanjing , number =. Science China technological sciences , publisher =. 2020 , pages =

2020
[42]

2022 , title =

Komatsuzaki, Aran and Puigcerver, Joan and Lee-Thorp, James and Ruiz, Carlos Riquelme and Mustafa, Basil and Ainslie, Joshua and Tay, Yi and Dehghani, Mostafa and Houlsby, Neil , journal =. 2022 , title =

2022
[43]

2022 , pages =

Lu, Pan and Mishra, Swaroop and Xia, Tanglin and Qiu, Liang and Chang, Kai-Wei and Zhu, Song-Chun and Tafjord, Oyvind and Clark, Peter and Kalyan, Ashwin , volume =. 2022 , pages =

2022
[44]

2024 , eprint=

MMBench: Is Your Multi-modal Model an All-around Player? , author=. 2024 , eprint=

2024
[45]

2024 , title =

Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae , volume =. 2024 , title =

2024
[46]

2024 , pages =

Liu, Haotian and Li, Chunyuan and Li, Yuheng and Lee, Yong Jae , booktitle =. 2024 , pages =

2024
[47]

Neural computation , publisher =

Jordan, Michael I and Jacobs, Robert A , number =. Neural computation , publisher =. 1994 , pages =

1994
[48]

2017 , title =

Shazeer, Noam and Mirhoseini, Azalia and Maziarz, Krzysztof and Davis, Andy and Le, Quoc and Hinton, Geoffrey and Dean, Jeff , journal =. 2017 , title =

2017
[49]

2022 , editor =

Du, Nan and Huang, Yanping and Dai, Andrew M and Tong, Simon and Lepikhin, Dmitry and Xu, Yuanzhong and Krikun, Maxim and Zhou, Yanqi and Yu, Adams Wei and Firat, Orhan and Zoph, Barret and Fedus, Liam and Bosma, Maarten P and Zhou, Zongwei and Wang, Tao and Wang, Emma and Web...

2022
[50]

2020 , title =

Lepikhin, Dmitry and Lee, HyoukJoong and Xu, Yuanzhong and Chen, Dehao and Firat, Orhan and Huang, Yanping and Krikun, Maxim and Shazeer, Noam and Chen, Zhifeng , journal =. 2020 , title =

2020
[51]

arXiv preprint arXiv:2202.08906 , publisher =

Zoph, Barret and Bello, Irwan and Kumar, Sameer and Du, Nan and Huang, Yanping and Dean, Jeff and Shazeer, Noam and Fedus, William , number =. arXiv preprint arXiv:2202.08906 , publisher =. 2022 , pages =

2022 arXiv
[52]

2024 , eprint=

Mixtral of Experts , author=. 2024 , eprint=

2024
[53]

2024 , title =

Xue, Fuzhao and Zheng, Zian and Fu, Yao and Ni, Jinjie and Zheng, Zangwei and Zhou, Wangchunshu and You, Yang , journal =. 2024 , title =

2024
[54]

2024 , title =

Lin, Bin and Tang, Zhenyu and Ye, Yang and Cui, Jiaxi and Zhu, Bin and Jin, Peng and Zhang, Junwu and Ning, Munan and Yuan, Li , journal =. 2024 , title =

2024
[55]

and Gao, Huazuo and Chen, Deli and Li, Jiashi and Zeng, Wangding and Yu, Xingkai and Wu, Y

Dai, Damai and Deng, Chengqi and Zhao, Chenggang and Xu, R.X. and Gao, Huazuo and Chen, Deli and Li, Jiashi and Zeng, Wangding and Yu, Xingkai and Wu, Y. and Xie, Zhenda and Li, Y.K. and Huang, Panpan and Luo, Fuli and Ruan, Chong and Sui, Zhifang and Liang, Wenfeng. D eep S e...

2024 doi
[56]

2023 , title =

Hu, Zhiqiang and Wang, Lei and Lan, Yihuai and Xu, Wanyu and Lim, Ee-Peng and Bing, Lidong and Xu, Xing and Poria, Soujanya and Lee, Roy Ka-Wei , journal =. 2023 , title =

2023
[57]

2023 , title =

Zhang, Qingru and Chen, Minshuo and Bukharin, Alexander and He, Pengcheng and Cheng, Yu and Chen, Weizhu and Zhao, Tuo , booktitle =. 2023 , title =

2023
[58]

2024 , title =

Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke , volume =. 2024 , title =

2024
[59]

2024 , title =

Hao, Jitai and Sun, WeiWei and Xin, Xin and Meng, Qi and Chen, Zhumin and Ren, Pengjie and Ren, Zhaochun , journal =. 2024 , title =

2024
[60]

2020 , title =

Yang, Greg and Hu, Edward J , journal =. 2020 , title =

2020
[61]

2021 , pages =

Patel, Arkil and Bhattamishra, Satwik and Goyal, Navin , booktitle =. 2021 , pages =

2021
[62]

2014 , pages =

Hosseini, Mohammad Javad and Hajishirzi, Hannaneh and Etzioni, Oren and Kushman, Nate , booktitle =. 2014 , pages =

2014
[63]

2017 , pages =

Ling, Wang and Yogatama, Dani and Dyer, Chris and Blunsom, Phil , booktitle =. 2017 , pages =

2017
[64]

2015 , pages =

Roy, Subhro and Roth, Dan , booktitle =. 2015 , pages =

2015
[65]

Transactions of the Association for Computational Linguistics , publisher =

Koncel-Kedziorski, Rik and Hajishirzi, Hannaneh and Sabharwal, Ashish and Etzioni, Oren and Ang, Siena Dumas , volume =. Transactions of the Association for Computational Linguistics , publisher =. 2015 , pages =

2015
[66]

2022 , title =

Zoph, Barret and Bello, Irwan and Kumar, Sameer and Du, Nan and Huang, Yanping and Dean, Jeff and Shazeer, Noam and Fedus, William , journal =. 2022 , title =

2022
[67]

2021 , pages =

Aghajanyan, Armen and Gupta, Sonal and Zettlemoyer, Luke , booktitle =. 2021 , pages =

2021
[68]

2024 , title =

Lo, Ka Man and Huang, Zeyu and Qiu, Zihan and Wang, Zili and Fu, Jie , journal =. 2024 , title =

2024
[69]

2023 , pages =

Dikkala, Nishanth and Ghosh, Nikhil and Meka, Raghu and Panigrahy, Rina and Vyas, Nikhil and Wang, Xin , booktitle =. 2023 , pages =

2023
[70]

2024 , title =

Wang, Zihan and Chen, Deli and Dai, Damai and Xu, Runxin and Li, Zhuoshu and Wu, Y , journal =. 2024 , title =

2024
[71]

2023 , pages =

Liu, Zeyu and Dettmers, Tim and Lin, Xi and Stoyanov, Veselin and Li, Xian , booktitle =. 2023 , pages =

2023
[72]

2019 , pages =

Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina , booktitle =. 2019 , pages =

2019
[73]

2020 , pages =

Bisk, Yonatan and Zellers, Rowan and Gao, Jianfeng and Choi, Yejin and others , number =. 2020 , pages =

2020
[74]

2019 , pages =

Sap, Maarten and Rashkin, Hannah and Chen, Derek and Le Bras, Ronan and Choi, Yejin , booktitle =. 2019 , pages =

2019
[75]

2019 , pages =

Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin , booktitle =. 2019 , pages =

2019
[76]

2018 , title =

Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , journal =. 2018 , title =

2018
[77]

2018 , pages =

Mihaylov, Todor and Clark, Peter and Khot, Tushar and Sabharwal, Ashish , booktitle =. 2018 , pages =

2018
[78]

2017 , pages =

Dauphin, Yann N and Fan, Angela and Auli, Michael and Grangier, David , booktitle =. 2017 , pages =

2017
[79]

2020 , title =

Shazeer, Noam , journal =. 2020 , title =

2020
[80]

2024 , title =

Page-Caccia, Lucas and Ponti, Edoardo Maria and Su, Zhan and Pereira, Matheus and Le Roux, Nicolas and Sordoni, Alessandro , volume =. 2024 , title =

2024
[81]

2023 , title =

Gou, Yunhao and Liu, Zhili and Chen, Kai and Hong, Lanqing and Xu, Hang and Li, Aoxue and Yeung, Dit-Yan and Kwok, James T and Zhang, Yu , journal =. 2023 , title =

2023
[82]

2024 , title =

Luo, Tongxu and Lei, Jiahe and Lei, Fangyu and Liu, Weihao and He, Shizhu and Zhao, Jun and Liu, Kang , journal =. 2024 , title =

2024
[83]

Proceedings of the National Academy of Sciences , publisher =

Gilardi, Fabrizio and Alizadeh, Meysam and Kubli, Maël , number =. Proceedings of the National Academy of Sciences , publisher =. 2023 , title =. doi:10.1073/pnas.2305016120 , issn =

2023 doi
[84]

2023 , title =

Gurnee, Wes and Nanda, Neel and Pauly, Matthew and Harvey, Katherine and Troitskii, Dmitrii and Bertsimas, Dimitris , volume =. 2023 , title =

2023
[85]

2023 , title =

Gurnee, Wes and Tegmark, Max , volume =. 2023 , title =

2023
[86]

2003 , pages =

Guyon, Isabelle and Elisseeff, Andre , number =. 2003 , pages =

2003
[87]

IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

Han, Yizeng and Huang, Gao and Song, Shiji and Yang, Le and Wang, Honghui and Wang, Yulin , number =. IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =. 2021 , pages =

2021
[88]

2009 , title =

Hastie, Trevor and Tibshirani, Robert and Friedman, Jerome H and Friedman, Jerome H , volume =. 2009 , title =

2009
[89]

Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen , biburl =. Proc. of ICLR , publisher =. 2021 , title =

2021
[90]

2021 , title =

He, Pengcheng and Gao, Jianfeng and Chen, Weizhu , volume =. 2021 , title =

2021
[91]

Junxian He and Chunting Zhou and Xuezhe Ma and Taylor Berg. Proc. of ICLR , publisher =. 2022 , title =

2022
[92]

2017 , title =

Loshchilov, I , journal =. 2017 , title =

2017
[93]

and Sifre, Laurent , title =

Hoffmann, Jordan and Borgeaud, Sebastian and Mensch, Arthur and Buchatskaya, Elena and Cai, Trevor and Rutherford, Eliza and de Las Casas, Diego and Hendricks, Lisa Anne and Welbl, Johannes and Clark, Aidan and Hennigan, Tom and Noland, Eric and Millican, Katie and van den Dri...

2022
[94]

Neil Houlsby and Andrei Giurgiu and Stanislaw Jastrzebski and Bruna Morrone and Quentin de Laroussilhe and Andrea Gesmundo and Mona Attariyan and Sylvain Gelly , biburl =. Proc. of ICML , publisher =. 2019 , pages =

2019
[95]

Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen

Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen. Proc. of ICLR , publisher =. 2022 , title =

2022
[96]

Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada , timestamp =

Ozan Irsoy and Claire Cardie , biburl =. Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada , timestamp =. 2014 , pages =

2014
[97]

Proceedings of the 15th Conference of the

Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Mikolov, Tomas , address =. Proceedings of the 15th Conference of the. 2017 , pages =

2017
[98]

2020 , title =

Kaplan, Jared and McCandlish, Sam and Henighan, Tom and Brown, Tom B and Chess, Benjamin and Child, Rewon and Gray, Scott and Radford, Alec and Wu, Jeffrey and Amodei, Dario , volume =. 2020 , title =

2020
[99]

Kim, Yoon , address =. Proc. of EMNLP , publisher =. 2014 , pages =. doi:10.3115/v1/D14-1181 , url =

2014 doi
[100]

2023 , title =

Kirk, Hannah Rose and Yin, Wenjie and Vidgen, Bertie and R. 2023 , title =

2023
[101]

Proceedings of the 36th International Conference on Neural Information Processing Systems , publisher =

Kojima, Takeshi and Gu, Shixiang Shane and Reid, Machel and Matsuo, Yutaka and Iwasawa, Yusuke , series =. Proceedings of the 36th International Conference on Neural Information Processing Systems , publisher =. 2022 , title =

2022
[102]

PLoS biology , publisher =

Kragel, Philip A and Knodt, Annchen R and Hariri, Ahmad R and LaBar, Kevin S , number =. PLoS biology , publisher =. 2016 , pages =

2016
[103]

Hinton , biburl =

Alex Krizhevsky and Ilya Sutskever and Geoffrey E. Hinton , biburl =. Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2012, Lake Tahoe, Nevada, United States...

2012
[104]

Nature , publisher =

Kutas, Marta and Hillyard, Steven A , number =. Nature , publisher =. 1984 , pages =

1984
[105]

Siwei Lai and Liheng Xu and Kang Liu and Jun Zhao , biburl =. Proc. of AAAI , publisher =. 2015 , pages =

2015
[106]

Zhenzhong Lan and Mingda Chen and Sebastian Goodman and Kevin Gimpel and Piyush Sharma and Radu Soricut , biburl =. Proc. of ICLR , publisher =. 2020 , title =

2020
[107]

Le and Tom

Quoc V. Le and Tom. Proc. of ICML , publisher =. 2014 , pages =

2014
[108]

1989 , title =

LeCun, Yann and Denker, John and Solla, Sara , volume =. 1989 , title =

1989
[109]

Lewis, Mike and Liu, Yinhan and Goyal, Naman and Ghazvininejad, Marjan and Mohamed, Abdelrahman and Levy, Omer and Stoyanov, Veselin and Zettlemoyer, Luke , address =. Proc. of ACL , publisher =. 2020 , pages =. doi:10.18653/v1/2020.acl-main.703 , url =

2020 doi
[110]

Li, Xiang Lisa and Liang, Percy , address =. Proc. of ACL , publisher =. 2021 , pages =. doi:10.18653/v1/2021.acl-long.353 , url =

2021 doi
[111]

2022 , pages =

Mustafa, Basil and Riquelme, Carlos and Puigcerver, Joan and Jenatton, Rodolphe and Houlsby, Neil , volume =. 2022 , pages =

2022
[112]

2023 , title =

Shen, Sheng and Yao, Zhewei and Li, Chunyuan and Darrell, Trevor and Keutzer, Kurt and He, Yuxiong , journal =. 2023 , title =

2023
[113]

2021 , pages =

Riquelme, Carlos and Puigcerver, Joan and Mustafa, Basil and Neumann, Maxim and Jenatton, Rodolphe and Susano Pinto, Andr. 2021 , pages =

2021
[114]

2023 , title =

Zadouri, Ted and. 2023 , title =

2023
[115]

2023 , title =

Liu, Qidong and Wu, Xian and Zhao, Xiangyu and Zhu, Yuanshao and Xu, Derong and Tian, Feng and Zheng, Yefeng , journal =. 2023 , title =

2023
[116]

2024 , title =

Chen, Shaoxiang and Jie, Zequn and Ma, Lin , journal =. 2024 , title =

2024
[117]

2024 , title =

Wu, Xun and Huang, Shaohan and Wei, Furu , journal =. 2024 , title =

2024
[118]

2024 , title =

Li, Dengchun and Ma, Yingzi and Wang, Naizheng and Cheng, Zhiyuan and Duan, Lei and Zuo, Jie and Yang, Cal and Tang, Mingjie , journal =. 2024 , title =

2024
[119]

2024 , title =

Gao, Chongyang and Chen, Kezhen and Rao, Jinmeng and Sun, Baochen and Liu, Ruibo and Peng, Daiyi and Zhang, Yawen and Guo, Xiaoyuan and Yang, Jie and Subrahmanian, VS , journal =. 2024 , title =

2024
[120]

2023 , title =

Li, Maximilian and Davies, Xander and Nadeau, Max , journal =. 2023 , title =

2023
[121]

Zhouhan Lin and Minwei Feng and C. Proc. of ICLR , publisher =. 2017 , title =

2017
[122]

2021 , title =

Lin, Yuxiao and Meng, Yuxian and Sun, Xiaofei and Han, Qinghong and Kuang, Kun and Li, Jiwei and Wu, Fei , volume =. 2021 , title =

2021
[123]

Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence,

Pengfei Liu and Xipeng Qiu and Xuanjing Huang , biburl =. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence,. 2016 , pages =

2016
[124]

and Gardner, Matt and Belinkov, Yonatan and Peters, Matthew E

Liu, Nelson F. and Gardner, Matt and Belinkov, Yonatan and Peters, Matthew E. and Smith, Noah A. , address =. Proc. of NAACL-HLT , publisher =. 2019 , pages =. doi:10.18653/v1/N19-1112 , url =

2019 doi
[125]

2019 , title =

Liu, Yinhan and Ott, Myle and Goyal, Naman and Du, Jingfei and Joshi, Mandar and Chen, Danqi and Levy, Omer and Lewis, Mike and Zettlemoyer, Luke and Stoyanov, Veselin , volume =. 2019 , title =

2019
[126]

ACM Computing Surveys , publisher =

Liu, Pengfei and Yuan, Weizhe and Fu, Jinlan and Jiang, Zhengbao and Hayashi, Hiroaki and Neubig, Graham , number =. ACM Computing Surveys , publisher =. 2023 , pages =

2023
[127]

Lu, Yao and Bartolo, Max and Moore, Alastair and Riedel, Sebastian and Stenetorp, Pontus , address =. Proc. of ACL , publisher =. 2022 , pages =. doi:10.18653/v1/2022.acl-long.556 , url =

2022 doi
[128]

and Daly, Raymond E

Maas, Andrew L. and Daly, Raymond E. and Pham, Peter T. and Huang, Dan and Ng, Andrew Y. and Potts, Christopher , address =. Proc. of ACL , publisher =. 2011 , pages =

2011
[129]

2024 , title =

Daniel Matter and Miriam Schirmer and Nir Grinberg and Jürgen Pfeffer , volume =. 2024 , title =

2024
[130]

2013 , title =

Mikolov, Tom. 2013 , title =

2013
[131]

Mikolov, Tomas and Yih, Wen-tau and Zweig, Geoffrey , address =. Proc. of NAACL-HLT , publisher =. 2013 , pages =

2013
[132]

science , publisher =

Mitchell, Tom M and Shinkareva, Svetlana V and Carlson, Andrew and Chang, Kai-Min and Malave, Vicente L and Mason, Robert A and Just, Marcel Adam , number =. science , publisher =. 2008 , pages =

2008
[133]

2023 , title =

Naveed, Humza and Khan, Asad Ullah and Qiu, Shi and Saqib, Muhammad and Anwar, Saeed and Usman, Muhammad and Barnes, Nick and Mian, Ajmal , volume =. 2023 , title =

2023
[134]

2004 , pages =

Ng, Andrew Y , booktitle =. 2004 , pages =

2004
[135]

2022 , title =

Olsson, Catherine and Elhage, Nelson and Nanda, Neel and Joseph, Nicholas and DasSarma, Nova and Henighan, Tom and Mann, Ben and Askell, Amanda and Bai, Yuntao and Chen, Anna and others , volume =. 2022 , title =

2022
[136]

and Neumann, Mark and Logan, Robert and Schwartz, Roy and Joshi, Vidur and Singh, Sameer and Smith, Noah A

Peters, Matthew E. and Neumann, Mark and Logan, Robert and Schwartz, Roy and Joshi, Vidur and Singh, Sameer and Smith, Noah A. , address =. Proc. of EMNLP , publisher =. 2019 , pages =. doi:10.18653/v1/D19-1005 , url =

2019 doi
[137]

and Ruder, Sebastian and Smith, Noah A

Peters, Matthew E. and Ruder, Sebastian and Smith, Noah A. , address =. Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) , publisher =. 2019 , pages =. doi:10.18653/v1/W19-4302 , url =

2019 doi
[138]

2023 , title =

Pope, Reiner and Douglas, Sholto and Chowdhery, Aakanksha and Devlin, Jacob and Bradbury, James and Heek, Jonathan and Xiao, Kefan and Agrawal, Shivani and Dean, Jeff , volume =. 2023 , title =

2023
[139]

2017 , title =

Radford, Alec and Jozefowicz, Rafal and Sutskever, Ilya , volume =. 2017 , title =

2017
[140]

2018 , title =

Radford, Alec and Narasimhan, Karthik and Salimans, Tim and Sutskever, Ilya and others , publisher =. 2018 , title =

2018
[141]

2019 , pages =

Radford, Alec and Wu, Jeffrey and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya and others , number =. 2019 , pages =

2019
[142]

Liu , biburl =

Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu , biburl =. J. Mach. Learn. Res. , bibsource =. 2020 , pages =

2020
[143]

International Conference on Machine Learning,

Samyam Rajbhandari and Conglong Li and Zhewei Yao and Minjia Zhang and Reza Yazdani Aminabadi and Ammar Ahmad Awan and Jeff Rasley and Yuxiong He , biburl =. International Conference on Machine Learning,. 2022 , pages =

2022
[144]

2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) , organization =

R. 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) , organization =. 2023 , pages =

2023
[145]

Machine learning , publisher =

Read, Jesse and Pfahringer, Bernhard and Holmes, Geoff and Frank, Eibe , volume =. Machine learning , publisher =. 2011 , pages =

2011
[146]

Reimers, Nils and Gurevych, Iryna , address =. Proc. of EMNLP , publisher =. 2019 , pages =. doi:10.18653/v1/D19-1410 , url =

2019 doi
[147]

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , publisher =

Sajjad, Hassan and Durrani, Nadir and Dalvi, Fahim and Alam, Firoj and Khan, Abdul and Xu, Jia , address =. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , publisher =. 2022 , page...

2022 doi
[148]

2019 , title =

Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas , volume =. 2019 , title =

2019
[149]

2023 , eprint=

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models , author=. 2023 , eprint=

2023
[150]

2020 , title =

Suau, Xavier and Zappella, Luca and Apostoloff, Nicholas , volume =. 2020 , title =

2020
[151]

2019 , pages =

Sun, Chi and Qiu, Xipeng and Xu, Yige and Huang, Xuanjing , booktitle =. 2019 , pages =

2019
[152]

2023 , title =

Sun, Xianghui and Ji, Yunjie and Ma, Baochang and Li, Xiangang , volume =. 2023 , title =

2023
[153]

2024 , pages =

Susnjak, Teo , booktitle =. 2024 , pages =

2024
[154]

Daniel and Sumers, Theodore R

Templeton, Adly and Conerly, Tom and Marcus, Jonathan and Lindsey, Jack and Bricken, Trenton and Chen, Brian and Pearce, Adam and Citro, Craig and Ameisen, Emmanuel and Jones, Andy and Cunningham, Hoagy and Turner, Nicholas L and McDougall, Callum and MacDiarmid, Monte and Fre...

2024
[155]

Journal of the Royal Statistical Society Series B: Statistical Methodology , publisher =

Tibshirani, Robert , number =. Journal of the Royal Statistical Society Series B: Statistical Methodology , publisher =. 1996 , pages =

1996
[156]

2023 , title =

Tigges, Curt and Hollinsworth, Oskar John and Geiger, Atticus and Nanda, Neel , volume =. 2023 , title =

2023
[157]

2023 , title =

Touvron, Hugo and Martin, Louis and Stone, Kevin and Albert, Peter and Almahairi, Amjad and Babaei, Yasmine and Bashlykov, Nikolay and Batra, Soumya and Bhargava, Prajjwal and Bhosale, Shruti and others , volume =. 2023 , title =

2023
[158]

2023 , title =

Turner, Alex and Thiergart, Lisa and Udell, David and Leech, Gavin and Mini, Ulisse and MacDiarmid, Monte , volume =. 2023 , title =

2023
[159]

Proceedings of the 31st International Conference on Neural Information Processing Systems , publisher =

Attention is all you need , author =. Proceedings of the 31st International Conference on Neural Information Processing Systems , publisher =. 2017 , pages =

2017
[160]

2023 , title =

Voita, Elena and Ferrando, Javier and Nalmpantis, Christoforos , volume =. 2023 , title =

2023
[161]

Bowman , biburl =

Alex Wang and Amanpreet Singh and Julian Michael and Felix Hill and Omer Levy and Samuel R. Bowman , biburl =. Proc. of ICLR , publisher =. 2019 , title =

2019
[162]

Wang, Guoyin and Li, Chunyuan and Wang, Wenlin and Zhang, Yizhe and Shen, Dinghan and Zhang, Xinyuan and Henao, Ricardo and Carin, Lawrence , address =. Proc. of ACL , publisher =. 2018 , pages =. doi:10.18653/v1/P18-1216 , url =

2018 doi
[163]

Wang, Xiaozhi and Wen, Kaiyue and Zhang, Zhengyan and Hou, Lei and Liu, Zhiyuan and Li, Juanzi , address =. Proc. of EMNLP , publisher =. 2022 , pages =

2022
[164]

Wei, Jason and Zou, Kai , address =. Proc. of EMNLP , publisher =. 2019 , pages =. doi:10.18653/v1/D19-1670 , url =

2019 doi
[165]

2022 , pages =

Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Xia, Fei and Chi, Ed and Le, Quoc V and Zhou, Denny and others , volume =. 2022 , pages =

2022
[166]

Xin, Ji and Tang, Raphael and Lee, Jaejun and Yu, Yaoliang and Lin, Jimmy , address =. Proc. of ACL , publisher =. 2020 , pages =. doi:10.18653/v1/2020.acl-main.204 , url =

2020 doi
[167]

Findings of the Association for Computational Linguistics: EACL 2023 , publisher =

Xu, Canwen and McAuley, Julian , address =. Findings of the Association for Computational Linguistics: EACL 2023 , publisher =. 2023 , pages =

2023
[168]

Yang, Zichao and Yang, Diyi and Dyer, Chris and He, Xiaodong and Smola, Alex and Hovy, Eduard , address =. Proc. of NAACL-HLT , publisher =. 2016 , pages =. doi:10.18653/v1/N16-1174 , url =

2016 doi
[169]

Le and Jiquan Ngiam , biburl =

Brandon Yang and Gabriel Bender and Quoc V. Le and Jiquan Ngiam , biburl =. Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada , timestamp =. 2019 , pages =

2019
[170]

Carbonell and Ruslan Salakhutdinov and Quoc V

Zhilin Yang and Zihang Dai and Yiming Yang and Jaime G. Carbonell and Ruslan Salakhutdinov and Quoc V. Le , biburl =. Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vanco...

2019
[171]

The Thirty-Third

Liang Yao and Chengsheng Mao and Yuan Luo , biburl =. The Thirty-Third. 2019 , pages =. doi:10.1609/aaai.v33i01.33017370 , url =

2019 doi
[172]

IEEE Access , publisher =

Yu, Shanshan and Su, Jindian and Luo, Da , volume =. IEEE Access , publisher =. 2019 , pages =

2019
[173]

Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada , timestamp =

Xiang Zhang and Junbo Jake Zhao and Yann LeCun , biburl =. Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada , timestamp =. 2015 , pages =

2015
[174]

Liu , biburl =

Jingqing Zhang and Yao Zhao and Mohammad Saleh and Peter J. Liu , biburl =. Proc. of ICML , publisher =. 2020 , pages =

2020
[175]

Weinberger and Yoav Artzi , biburl =

Tianyi Zhang and Felix Wu and Arzoo Katiyar and Kilian Q. Weinberger and Yoav Artzi , biburl =. Proc. of ICLR , publisher =. 2021 , title =

2021
[176]

2022 , title =

Zhang, Susan and Roller, Stephen and Goyal, Naman and Artetxe, Mikel and Chen, Moya and Chen, Shuohui and Dewan, Christopher and Diab, Mona and Li, Xian and Lin, Xi Victoria and others , volume =. 2022 , title =

2022
[177]

2022 , pages =

Zhou, Yanqi and Lei, Tao and Liu, Hanxiao and Du, Nan and Huang, Yanping and Zhao, Vincent and Dai, Andrew M and Le, Quoc V and Laudon, James and others , volume =. 2022 , pages =

2022
[178]

2024 , title =

Reid, Machel and Savinov, Nikolay and Teplyashin, Denis and Lepikhin, Dmitry and Lillicrap, Timothy and Alayrac, Jean-baptiste and Soricut, Radu and Lazaridou, Angeliki and Firat, Orhan and Schrittwieser, Julian and others , journal =. 2024 , title =

2024
[179]

2024 , title =

Muennighoff, Niklas and Soldaini, Luca and Groeneveld, Dirk and Lo, Kyle and Morrison, Jacob and Min, Sewon and Shi, Weijia and Walsh, Pete and Tafjord, Oyvind and Lambert, Nathan and others , journal =. 2024 , title =

2024
[180]

2019 , title =

Ziegler, Daniel M and Stiennon, Nisan and Wu, Jeffrey and Brown, Tom B and Radford, Alec and Amodei, Dario and Christiano, Paul and Irving, Geoffrey , volume =. 2019 , title =

2019
[181]

2024 , title =

Papamarkou, Theodore and Birdal, Tolga and Bronstein, Michael and Carlsson, Gunnar and Curry, Justin and Gao, Yue and Hajij, Mustafa and Kwitt, Roland and Li. 2024 , title =

2024
[182]

2024 , title =

Wendler, Chris and Veselovsky, Veniamin and Monea, Giovanni and West, Robert , journal =. 2024 , title =

2024
[183]

2024 , title =

Schaeffer, Rylan and Lecomte, Victor and Pai, Dhruv Bhandarkar and Carranza, Andres and Isik, Berivan and Unell, Alyssa and Khona, Mikail and Yerxa, Thomas and LeCun, Yann and Chung, SueYeon and others , journal =. 2024 , title =

2024
[184]

2020 , pages =

Naitzat, Gregory and Zhitnikov, Andrey and Lim, Lek-Heng , number =. 2020 , pages =

2020
[185]

2024 , title =

Meng, Li and Goodwin, Morten and Yazidi, Anis and Engelstad, Paal , journal =. 2024 , title =

2024
[186]

2023 , pages =

Yerxa, Thomas and Kuang, Yilun and Simoncelli, Eero and Chung, SueYeon , volume =. 2023 , pages =

2023
[187]

2024 , title =

Heinzerling, Benjamin and Inui, Kentaro , journal =. 2024 , title =

2024
[188]

2024 , title =

Jiang, Yibo and Rajendran, Goutham and Ravikumar, Pradeep and Aragam, Bryon and Veitch, Victor , journal =. 2024 , title =

2024
[189]

2014 , pages =

Pennington, Jeffrey and Socher, Richard and Manning, Christopher D , booktitle =. 2014 , pages =

2014
[190]

Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 , publisher =

Distributed representations of words and phrases and their compositionality , author =. Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 , publisher =. 2013 , pages =

2013
[191]

2023 , title =

Olah, Chris and Batson, Josh , journal =. 2023 , title =

2023
[192]

2024 , title =

Shai, Adam S and Marzen, Sarah E and Teixeira, Lucas and Oldenziel, Alexander Gietelink and Riechers, Paul M , journal =. 2024 , title =

2024
[193]

Nature , publisher =

Gardner, Richard J and Hermansen, Erik and Pachitariu, Marius and Burak, Yoram and Baas, Nils A and Dunn, Benjamin A and Moser, May-Britt and Moser, Edvard I , number =. Nature , publisher =. 2022 , pages =

2022
[194]

Nature Reviews Neuroscience , publisher =

Langdon, Christopher and Genkin, Mikhail and Engel, Tatiana A , number =. Nature Reviews Neuroscience , publisher =. 2023 , pages =

2023
[195]

2024 , title =

Fitz, Stephen and Romero, Peter and Schneider, Jiyan Jonas , journal =. 2024 , title =

2024
[196]

2020 , pages =

Mamou, Jonathan and Le, Hang and Del Rio, Miguel and Stephenson, Cory and Tang, Hanlin and Kim, Yoon and Chung, Sueyeon , booktitle =. 2020 , pages =

2020
[197]

2022 , title =

Black, Sid and Sharkey, Lee and Grinsztajn, Leo and Winsor, Eric and Braun, Dan and Merizian, Jacob and Parker, Kip and Guevara, Carlos Ram. 2022 , title =

2022
[198]

2020 , title =

Han, Zhen and Ma, Yunpu and Chen, Peng and Tresp, Volker , journal =. 2020 , title =

2020
[199]

2021 , title =

Abdou, Mostafa and Kulmizev, Artur and Hershcovich, Daniel and Frank, Stella and Pavlick, Ellie and S. 2021 , title =

2021
[200]

2007 , title =

Bengio, Yoshua and LeCun, Yann , booktitle =. 2007 , title =

2007
[201]

and Osindero, Simon and Teh, Yee Whye , volume =

Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , volume =. 2006 , pages =

2006
[202]

2016 , title =

Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron and Bengio, Yoshua , volume =. 2016 , title =

2016
[203]

2025 , title =

Kim, Junsol and Evans, James and Schein, Aaron , booktitle =. 2025 , title =

2025
[204]

2023 , title =

O'Hagan, Sean and Schein, Aaron , journal =. 2023 , title =

2023
[205]

2023 , pages =

Santurkar, Shibani and Durmus, Esin and Ladhak, Faisal and Lee, Cinoo and Liang, Percy and Hashimoto, Tatsunori , booktitle =. 2023 , pages =

2023
[206]

2023 , title =

Wu, Patrick Y and Nagler, Jonathan and Tucker, Joshua A and Messing, Solomon , journal =. 2023 , title =

2023
[207]

2024 , pages =

Mochtak, Michal and Rupnik, Peter and Ljube. 2024 , pages =

2024
[208]

2024 , title =

Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Letman, Aiesha and Mathur, Akhil and Schelten, Alan and Yang, Amy and Fan, Angela and others , journal =. 2024 , title =

2024
[209]

Albert Q. Jiang and Alexandre Sablayrolles and Arthur Mensch and Chris Bamford and Devendra Singh Chaplot and Diego de las Casas and Florian Bressand and Gianna Lengyel and Guillaume Lample and Lucile Saulnier and Lélio Renard Lavaud and Marie-Anne Lachaux and Pierre Stock and...

2023
[210]

2025 , title =

An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and...

2025
[211]

2023 , title =

Kim, Junsol and Lee, Byungkyu , journal =. 2023 , title =

2023
[212]

2025 , pages =

Bernardelle, Pietro and Fr. 2025 , pages =

2025
[213]

2016 , title =

Alain, Guillaume and Bengio, Yoshua , volume =. 2016 , title =

2016
[214]

2023 , journal=

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author=. 2023 , journal=

2023
[215]

Glass , biburl =

Fahim Dalvi and Nadir Durrani and Hassan Sajjad and Yonatan Belinkov and Anthony Bau and James R. Glass , biburl =. The Thirty-Third. doi:10.1609/aaai.v33i01.33016309 , year =

doi
[216]

2018 , pages =

Gurari, Danna and Li, Qing and Stangl, Abigale J and Guo, Anhong and Lin, Chi and Grauman, Kristen and Luo, Jiebo and Bigham, Jeffrey P , booktitle =. 2018 , pages =

2018
[217]

Proceedings of the 2018

Jacovi, Alon and Sar Shalom, Oren and Goldberg, Yoav , address =. Proceedings of the 2018. 2018 , pages =. doi:10.18653/v1/W18-5408 , url =

2018 doi
[218]

2015 , title =

Karpathy, Andrej and Johnson, Justin and Fei-Fei, Li , volume =. 2015 , title =

2015
[219]

2023 , title =

Stoehr, Niklas and Cheng, Pengxiang and Wang, Jing and Preotiuc-Pietro, Daniel and Bhowmik, Rajarshi , volume =. 2023 , title =

2023
[220]

2024 , title =

Verma, Prateek and Pilanci, Mert , journal =. 2024 , title =

2024
[221]

2023 , title =

Marks, Samuel and Tegmark, Max , journal =. 2023 , title =

2023
[222]

Findings of the Association for Computational Linguistics: EMNLP 2020 , publisher =

Barbieri, Francesco and Camacho-Collados, Jose and Espinosa Anke, Luis and Neves, Leonardo , address =. Findings of the Association for Computational Linguistics: EMNLP 2020 , publisher =. 2020 , pages =. doi:10.18653/v1/2020.findings-emnlp.148 , url =

2020 doi
[223]

2018 , title =

Cer, Daniel and Yang, Yinfei and Kong, Sheng-yi and Hua, Nan and Limtiaco, Nicole and John, Rhomni St and Constant, Noah and Guajardo-Cespedes, Mario and Yuan, Steve and Tar, Chris and others , volume =. 2018 , title =

2018
[224]

Proceedings of the 15th Conference of the

Conneau, Alexis and Schwenk, Holger and Barrault, Lo. Proceedings of the 15th Conference of the. 2017 , pages =

2017
[225]

2019 , pages =

Singh, Amanpreet and Natarajan, Vivek and Shah, Meet and Jiang, Yu and Chen, Xinlei and Batra, Dhruv and Parikh, Devi and Rohrbach, Marcus , booktitle =. 2019 , pages =

2019
[226]

Communications of the ACM , publisher =

Sakaguchi, Keisuke and Bras, Ronan Le and Bhagavatula, Chandra and Choi, Yejin , number =. Communications of the ACM , publisher =. 2021 , pages =

2021
[227]

Howard, Jeremy and Ruder, Sebastian , address =. Proc. of ACL , publisher =. 2018 , pages =. doi:10.18653/v1/P18-1031 , url =

2018 doi
[228]

2024 , title =

Liu, Zhili and Chen, Kai and Han, Jianhua and Hong, Lanqing and Xu, Hang and Li, Zhenguo and Kwok, James T , journal =. 2024 , title =

2024
[229]

2021 , title =

Lester, Brian and Al-Rfou, Rami and Constant, Noah , journal =. 2021 , title =

2021
[230]

Journal of cognitive neuroscience , publisher =

O'Toole, Alice J and Jiang, Fang and Abdi, Herv. Journal of cognitive neuroscience , publisher =. 2007 , pages =

2007
[231]

2023 , title =

Panigrahi, Abhishek and Saunshi, Nikunj and Zhao, Haoyu and Arora, Sanjeev , volume =. 2023 , title =

2023
[232]

and Swayamdipta, Swabha and Wolf, Thomas , address =

Ruder, Sebastian and Peters, Matthew E. and Swayamdipta, Swabha and Wolf, Thomas , address =. Proceedings of the 2019 Conference of the North. 2019 , pages =. doi:10.18653/v1/N19-5004 , url =

2019 doi
[233]

2023 , title =

Sun, Xiaofei and Li, Xiaoya and Li, Jiwei and Wu, Fei and Guo, Shangwei and Zhang, Tianwei and Wang, Guoyin , volume =. 2023 , title =

2023
[234]

Hovy and Thang Luong and Quoc Le , biburl =

Qizhe Xie and Zihang Dai and Eduard H. Hovy and Thang Luong and Quoc Le , biburl =. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , timestamp =. 2020 , title =

2020
[236]

2023 , title =

Park, Kiho and Choe, Yo Joong and Veitch, Victor , journal =. 2023 , title =

2023
[237]

2022 , title =

Elhage, Nelson and Hume, Tristan and Olsson, Catherine and Schiefer, Nicholas and Henighan, Tom and Kravec, Shauna and Hatfield-Dodds, Zac and Lasenby, Robert and Drain, Dawn and Chen, Carol and others , journal =. 2022 , title =

2022
[238]

2023 , title =

Ballester, Rub. 2023 , title =

2023
[239]

2021 , pages =

Hernandez, Evan and Andreas, Jacob , booktitle =. 2021 , pages =

2021
[240]

2024 , title =

Park, Kiho and Choe, Yo Joong and Jiang, Yibo and Veitch, Victor , journal =. 2024 , title =

2024
[241]

2025 , title =

Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Zhang, Ruoyu and Xu, Runxin and Zhu, Qihao and Ma, Shirong and Wang, Peiyi and Bi, Xiao and others , journal =. 2025 , title =

2025
[242]

2016 , pages =

Deep Residual Learning for Image Recognition , author =. 2016 , pages =

2016
[243]

Psychological Review , publisher =

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain , author =. Psychological Review , publisher =. 1958 , pages =. doi:10.1037/h0042519 , url =

1958 doi
[244]

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , publisher =

Tenney, Ian and Das, Dipanjan and Pavlick, Ellie , editor =. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , publisher =. doi:10.18653/v1/P19-1452 , year =

doi
[245]

Computer Vision – ECCV 2014 , publisher =

Visualizing and Understanding Convolutional Networks , author =. Computer Vision – ECCV 2014 , publisher =. 2014 , pages =. doi:10.1007/978-3-319-10590-1_53 , isbn =

2014 doi
[246]

Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2 , publisher =

How transferable are features in deep neural networks? , author =. Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2 , publisher =. 2014 , pages =

2014
[247]

Distill , publisher =

Feature Visualization , author =. Distill , publisher =. 2017 , month =. doi:10.23915/distill.00007 , issn =

2017 doi
[248]

2008 , pages =

Visualizing Data Using t-SNE , author =. 2008 , pages =

2008
[249]

2018 , title =

McInnes, Leland and Healy, John and Melville, James , journal =. 2018 , title =

2018
[250]

Proceedings of the 36th International Conference on Neural Information Processing Systems , publisher =

Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , series =. Proceedings of the 36th International Conference on Neural Information Processing Systems , publisher =. 2022 , title =

2022
[251]

2023 , booktitle =

Towards Best Practices of Activation Patching in Language Models: Metrics and Methods , author =. 2023 , booktitle =

2023
[252]

Distill , publisher =

Zoom In: An Introduction to Circuits , author =. Distill , publisher =. 2020 , month =. doi:10.23915/distill.00024.001 , issn =

2020 doi
[253]

2009 , title =

Pearl, Judea , publisher =. 2009 , title =

2009
[254]

2023 , title =

Nanda, Neel and Chan, Lawrence and Lieberum, Tom and Smith, Jess and Steinhardt, Jacob , journal =. 2023 , title =

2023
[255]

Proceedings of the 35th International Conference on Neural Information Processing Systems , publisher =

Geiger, Atticus and Lu, Hanson and Icard, Thomas and Potts, Christopher , series =. Proceedings of the 35th International Conference on Neural Information Processing Systems , publisher =. 2021 , title =

2021
[256]

IEEE Transactions on information theory , publisher =

Compressed Sensing , author =. IEEE Transactions on information theory , publisher =. 2006 , pages =

2006
[257]

Signal processing , publisher =

Extensions of Compressed Sensing , author =. Signal processing , publisher =. 2006 , pages =

2006
[258]

Proceedings of the 34th International Conference on Neural Information Processing Systems , publisher =

Mu, Jesse and Andreas, Jacob , series =. Proceedings of the 34th International Conference on Neural Information Processing Systems , publisher =. 2020 , title =

2020
[259]

2023 , title =

Cunningham, Hoagy and Ewart, Aidan and Riggs, Logan and Huben, Robert and Sharkey, Lee , journal =. 2023 , title =

2023
[260]

2024 , title =

Todd, Eric and Li, Millicent and Sharma, Arnab Sen and Mueller, Aaron and Wallace, Byron C and Bau, David , booktitle =. 2024 , title =

2024
[261]

How Contextual are Contextualized Word Representations?

Ethayarajh, Kawin , editor =. How Contextual are Contextualized Word Representations?. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , publisher =....

2019 doi
[262]

2024 , title =

Bhalla, Usha and Srinivas, Suraj and Ghandeharioun, Asma and Lakkaraju, Himabindu , journal =. 2024 , title =

2024
[263]

2024 , title =

Bereska, Leonard and Gavves, Stratis , journal =. 2024 , title =

2024
[264]

IEEE Transactions on Pattern Analysis and Machine Intelligence , keywords =

Representation Learning: A Review and New Perspectives , author =. IEEE Transactions on Pattern Analysis and Machine Intelligence , keywords =. doi:10.1109/TPAMI.2013.50 , year =

2013 doi
[265]

2016 , pages =

Testing the Manifold Hypothesis , author =. 2016 , pages =

2016
[266]

Transactions of the Association for Computational Linguistics , abstract =

A Primer in BERTology: What We Know About How BERT Works , author =. Transactions of the Association for Computational Linguistics , abstract =. 2021 , month =. doi:10.1162/tacl_a_00349 , issn =

2021 doi
[267]

American journal of political science , publisher =

A Spatial Model for Legislative Roll Call Analysis , author =. American journal of political science , publisher =. 1985 , pages =

1985
[268]

2001 , month =

Mapping Policy Preferences: Estimates for Parties, Electors, and Governments 1945—1998 , author =. 2001 , month =. doi:10.1093/oso/9780199244003.001.0001 , isbn =

1945
[269]

Electoral Studies , abstract =

Estimating party policy positions: Comparing expert surveys and hand-coded content analysis , author =. Electoral Studies , abstract =. 2007 , pages =. doi:https://doi.org/10.1016/j.electstud.2006.04.008 , url =

2007 doi
[270]

American Political Science Review , publisher =

Extracting Policy Positions from Political Texts Using Words as Data , author =. American Political Science Review , publisher =. doi:10.1017/S0003055403000698 , year =

doi
[271]

Electoral Studies , abstract =

Estimating policy positions using political texts: An evaluation of the Wordscores approach , author =. Electoral Studies , abstract =. 2007 , pages =. doi:https://doi.org/10.1016/j.electstud.2007.07.006 , url =

2007 doi
[272]

Electoral studies , abstract =

Do they work? Validating computerised word frequency estimates against policy series , author =. Electoral studies , abstract =. 2007 , pages =. doi:https://doi.org/10.1016/j.electstud.2006.04.002 , url =

2007 doi
[273]

doi:10.1609/icwsm.v5i1.14126 , year =

Political Polarization on Twitter , author =. doi:10.1609/icwsm.v5i1.14126 , year =

doi
[274]

American Journal of Political Science , abstract =

A Scaling Model for Estimating Time-Series Party Positions from Texts , author =. American Journal of Political Science , abstract =. doi:https://doi.org/10.1111/j.1540-5907.2008.00338.x , year =

2008
[275]

British Journal of Political Science , publisher =

Position taking in European Parliament speeches , author =. British Journal of Political Science , publisher =. doi:10.1017/S0007123409990299 , year =

doi
[276]

ACM/IMS Trans

Political Text Scaling Meets Computational Semantics , author =. ACM/IMS Trans. Data Sci. , keywords =. 2022 , pages =. doi:10.1145/3485666 , issn =

2022 doi
[277]

Beyond Binary Labels: Political Ideology Prediction of

Preo. Beyond Binary Labels: Political Ideology Prediction of. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , publisher =. doi:10.18653/v1/P17-1068 , year =

doi
[278]

Psychological Science , publisher =

Tweeting From Left to Right: Is Online Political Communication More Than an Echo Chamber? , author =. Psychological Science , publisher =. 2015 , pages =. doi:10.1177/0956797615594620 , note =

2015 doi
[279]

Proceedings of the 2018 Conference of the North

Multi-Task Learning for Argumentation Mining in Low-Resource Settings , author =. Proceedings of the 2018 Conference of the North. doi:10.18653/v1/N18-2006 , year =

2018 doi
[280]

doi:10.18653/v1/2024.acl-long.600 , year =

Measuring Political Bias in Large Language Models: What Is Said and How It Is Said , author =. doi:10.18653/v1/2024.acl-long.600 , year =

2024 doi
[281]

PloS one , publisher =

The political preferences of LLMs , author =. PloS one , publisher =. 2024 , pages =

2024
[282]

Journal of Computational Social Science , abstract =

Assessing political bias in large language models , author =. Journal of Computational Social Science , abstract =. doi:10.1007/s42001-025-00376-w , year =

doi
[283]

Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , abstractnote =

PoliTune: Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in Large Language Models , author =. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , abstractnote =. doi:10.1609/aies.v7i1.31612 , year =

doi
[284]

Companion Proceedings of the ACM on Web Conference 2025 , publisher =

The Impact of Persona-based Political Perspectives on Hateful Content Detection , author =. Companion Proceedings of the ACM on Web Conference 2025 , publisher =. 2025 , pages =. doi:10.1145/3701716.3718383 , isbn =

2025
[285]

Hidden Persuaders:

Potter, Yujin and Lai, Shiyang and Kim, Junsol and Evans, James and Song, Dawn , editor =. Hidden Persuaders:. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , publisher =. doi:10.18653/v1/2024.emnlp-main.244 , year =

2024 doi
[286]

arXiv preprint arXiv:1803.00156 , year=

Autoencoding topology , author=. arXiv preprint arXiv:1803.00156 , year=

arXiv
[287]

ICLR 2021 Workshop on Geometrical and Topological Representation Learning , year=

Self-supervised representation learning on manifolds , author=. ICLR 2021 Workshop on Geometrical and Topological Representation Learning , year=

2021
[288]

Unsupervised Geometric and Topological Approaches for Cross-Lingual Sentence Representation and Comparison

Haim Meirom, Shaked and Bobrowski, Omer. Unsupervised Geometric and Topological Approaches for Cross-Lingual Sentence Representation and Comparison. Proceedings of the 7th Workshop on Representation Learning for NLP. 2022. doi:10.18653/v1/2022.repl4nlp-1.18

2022 doi
[289]

arXiv preprint arXiv:2402.08871 , year=

Position Paper: Challenges and Opportunities in Topological Deep Learning , author=. arXiv preprint arXiv:2402.08871 , year=

arXiv
[290]

arXiv preprint arXiv:2402.10588 , year=

Do Llamas Work in English? On the Latent Language of Multilingual Transformers , author=. arXiv preprint arXiv:2402.10588 , year=

arXiv
[291]

arXiv preprint arXiv:2406.10254 , year=

Towards Signal Processing In Large Language Models , author=. arXiv preprint arXiv:2406.10254 , year=

arXiv
[292]

arXiv preprint arXiv:2406.09366 , year=

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations , author=. arXiv preprint arXiv:2406.09366 , year=

arXiv
[293]

Journal of Machine Learning Research , volume=

Topology of deep neural networks , author=. Journal of Machine Learning Research , volume=. 2020 , url=

2020
[294]

arXiv preprint arXiv:2405.13848 , year=

Maximum Manifold Capacity Representations in State Representation Learning , author=. arXiv preprint arXiv:2405.13848 , year=

arXiv
[295]

Advances in Neural Information Processing Systems , volume=

Learning efficient coding of natural images with maximum manifold capacity representations , author=. Advances in Neural Information Processing Systems , volume=. 2023 , url=

2023
[296]

arXiv preprint arXiv:2403.10381 , year=

Monotonic representation of numeric properties in language models , author=. arXiv preprint arXiv:2403.10381 , year=

arXiv
[297]

arXiv preprint arXiv:2403.03867 , year=

On the origins of linear representations in large language models , author=. arXiv preprint arXiv:2403.03867 , year=

arXiv
[298]

arXiv preprint arXiv:2311.03658 , year=

The linear representation hypothesis and the geometry of large language models , author=. arXiv preprint arXiv:2311.03658 , year=

arXiv
[299]

arXiv preprint arXiv:2310.06824 , year=

The geometry of truth: Emergent linear structure in large language model representations of true/false datasets , author=. arXiv preprint arXiv:2310.06824 , year=

arXiv
[300]

Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages=

Glove: Global vectors for word representation , author=. Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages=. 2014 , url=

2014

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

2018 , title =

Korman, Eric O , journal =. 2018 , title =

2018

[2] [2]

2026 , eprint=

Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity , author=. 2026 , eprint=

2026

[3] [3]

Transactions on Machine Learning Research , issn=

A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence , author=. Transactions on Machine Learning Research , issn=. 2026 , url=

2026

[4] [4]

Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , journal =

J. Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , journal =. 2003 , url =

2003

[5] [5]

2021 , title =

Korman, Eric O , booktitle =. 2021 , title =

2021

[6] [6]

Evolutionary Principles in Self-Referential Learning

Jurgen Schmidhuber. Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta...-Hook. 1987

1987

[7] [7]

Verhandlungen der deutschen zoologischen Gesellschaft , volume =

Woltereck, Richard , title =. Verhandlungen der deutschen zoologischen Gesellschaft , volume =

[8] [8]

Pigliucci, Massimo , title =