Pith. sign in

REVIEW 4 major objections 5 minor 46 references

The paper proposes EELMA, a method that estimates how much control a language-model agent exerts over future states, and shows this 'empowerment' score tracks average task performance across games and web browsing.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 14:49 UTC pith:BOQ3GXOI

load-bearing objection A useful proof-of-concept with a clean toy validation, but the estimator's constant cancellation and WebArena's n=3 correlations need work before the results fully land. the 4 major comments →

arxiv 2509.22504 v3 pith:BOQ3GXOI submitted 2025-09-26 cs.AI cs.LG

Estimating the Empowerment of Language Model Agents

Jinyeop Song , Jeff Gore , Max Kleiman-Weiner This is my paper
classification cs.AI cs.LG
keywords EELMAempowermentlanguage model agentsmutual information estimationInfoNCEgoal-agnostic evaluationWebArenaagentic capability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to establish a goal-agnostic way to evaluate language-model agents: instead of measuring success on handcrafted benchmarks, it estimates 'effective empowerment'—how much influence an agent's actions have over future states. The authors introduce EELMA, which learns this quantity from multi-turn text trajectories using contrastive mutual-information estimation, and then test whether it predicts average discounted reward. Across Gridworld, Tower of Hanoi, and the realistic WebArena web environment, empowerment estimates correlate strongly with average task performance, supporting the claim that empowerment can serve as a proxy for agentic capability without any reward or goal specification. The paper also shows that high-empowerment states and actions align with pivotal moments such as successful authentication, suggesting a use for monitoring agent behavior.

Core claim

EELMA approximates effective empowerment by learning two contrastive InfoNCE critics—one that encodes a state, and one that encodes a state–action pair—and subtracting their dot products with a future-state encoder. This difference estimates the mutual information between the agent's action and a discounted future state, marginalizing over the LM policy. Empirically, the estimated empowerment strongly correlates with mean discounted reward across varied environments and agent configurations: r = 0.91 for Chain-of-Thought vs. base in Gridworld, r = 0.70–0.85 across open and closed models, and Spearman correlations of 0.83–0.94 in three WebArena domains. The paper interprets this as evidence t

What carries the argument

The central mechanism is the difference of two InfoNCE-based mutual-information estimators. State and action text are embedded with a pretrained language model plus a fine-tunable MLP; two encoders, one for state-only and one for state–action, are trained contrastively against a future-state encoder. At convergence, the log-ratio log P(s*|s,a)/P(s*|s) is recovered as the difference of the two dot products, up to additive constants that are assumed equal. This converts high-dimensional, sparse text observations into a tractable estimate of effective empowerment, defined as the average discounted mutual information between the current action and a future state sampled under geometric discounti

Load-bearing premise

The estimate treats the two InfoNCE normalizing constants as equal even though they come from different encoders, and the discount factor used to sample future states is never specified, so the exact quantity being measured is not fully pinned down.

What would settle it

Run EELMA on a fixed set of trajectories while varying the discount factor γ used to sample future states; if the resulting empowerment scores change in a way that reorders agents, then the claimed objective is not well-defined. Alternatively, construct an environment where one agent can reach many distinct states but fails all goal-relevant tasks (e.g., aimless browsing in WebArena) and test whether empowerment still tracks reward; the Shopping domain's flat relationship already hints at such a failure mode.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If EELMA's estimates reflect true empowerment, then one can rank or compare LM agents on capability without needing task-specific reward functions or human-designed goals.
  • The strong correlation with average discounted reward implies that empowerment could serve as a cheap, continuous monitoring signal during training or deployment, detecting when an agent gains or loses control over its environment.
  • The robustness to natural-language paraphrasing suggests EELMA can be applied in realistic text environments where states are described variably, not just in structured games.
  • The authentication case study shows empowerment can flag actions that expand access or control, potentially identifying power-seeking behavior without predefined threat models.
  • The method's sensitivity to chain-of-thought, memory length, and model scale gives a quantitative handle on how agentic capabilities change with design choices.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to use empowerment as an intrinsic reward or objective for training agents to be generally capable in open-ended environments, since it does not require a specified goal to be optimized.
  • The flat empowerment–reward relationship in the Shopping domain suggests empowerment alone may understate performance bottlenecks that come from reasoning rather than control; one could test this by inserting a reasoning-heavy sub-task into an otherwise high-control environment and checking whether empowerment fails to track reward.
  • Because EELMA learns from trajectory data, it could be applied in a fully offline setting to audit precollected agent logs, without any additional environment interaction.
  • The unspecified discount factor γ in the EELMA sampling procedure is a tunable knob; if the correlation with reward is sensitive to γ, then the metric's objectivity is weakened, and reporting γ across experiments would make comparisons more meaningful.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes EELMA, an InfoNCE-based estimator of effective empowerment for language model agents acting in text environments. Effective empowerment is defined as discounted mutual information between an agent's current action and a geometric-horizon future state. EELMA trains two contrastive critics—one state-only and one state-action—and estimates empowerment as the difference of their log-ratio predictions. The method is validated in Gridworld and Tower of Hanoi against direct (tabular) empowerment estimates, including under paraphrased natural-language observations, and then applied to WebArena across four domains and three LLMs. The central empirical claim is that EELMA's empowerment estimates strongly correlate with mean discounted reward across agent variations (CoT, memory, model family/scale), supporting empowerment as a goal-agnostic proxy for LM-agent capability. A case study on authentication actions is presented as evidence that high-empowerment actions mark pivotal moments.

Significance. If the central claim holds, EELMA would provide a scalable, goal-agnostic complement to benchmark-based evaluation of LM agents, with potential uses in capability monitoring and safety screening. The paper's strengths are concrete: the Gridworld/ToH validation against direct empowerment is clean and gives a rare ground-truth check for a contrastive empowerment estimator; the robustness experiments under natural-language paraphrase are informative; the comparison against a prompt-only LLM estimator is a useful negative result; and the code is publicly released. The theoretical bridge to goal-averaged reward is borrowed from Myers et al. and is stated with assumptions, which is appropriate. However, the central estimator's identifiability and the strength of the WebArena correlation claims are not yet fully established, and several statistical details need tightening before the main claim can be accepted at face value.

major comments (4)
  1. [Section 3, Eqs. (5)–(7)] The derivation of Eq. (7) assumes that the additive constants C1 and C2 in Eqs. (5)–(6) cancel. At the InfoNCE optimum, each critic recovers its log-density ratio only up to an arbitrary constant that depends on the encoder and the objective; nothing in the two separate losses ties these constants. Thus the subtracted quantity is log p(s*|s,a)/p(s*|s) + C1 - C2, not necessarily the desired mutual information. The toy-domain validation is encouraging but does not establish the general case, and WebArena has no ground truth. The paper should either prove that C1 = C2 (e.g., by construction of the encoders), impose a calibration constraint, or provide a direct way to measure or bound the offset in the settings where the metric is claimed.
  2. [Section 3, Eq. (1); Algorithm 1; Appendix D.1] The definition of effective empowerment depends on the discount factor γ through the future-state sampling distribution τ ~ Geom(1−γ), but Algorithm 1 does not specify γ or the exact empirical sampling procedure for the tuples (s_t, a_t, s_*). Appendix B.1 mentions γ=0.9 for the maximum-empowerment calculation, but not for EELMA. Without specifying γ and the sampling distribution, the estimated objective is not well defined, and values across domains and experiments cannot be compared quantitatively. Please report these hyperparameters and, if the same γ is not used everywhere, explain the implications for the cross-domain comparisons.
  3. [Section 5, Figure 7 and Table 3] The WebArena correlation claim is based on only three models per domain, with no confidence intervals, and the Shopping domain shows a flat relationship between empowerment and discounted reward. The paper's explanation—that reasoning about prices is a bottleneck independent of empowerment—is post hoc. If the metric is presented as a general goal-agnostic proxy for agentic capability, a flat relation in one of four domains is a substantive qualification, not a minor caveat. Please report bootstrap or other uncertainty intervals for the Spearman correlations, and either add more model/data points or explicitly delimit the claim to domains where environment control, rather than task-specific reasoning, is the limiting factor.
  4. [Figure 8, Figure 10, Appendix C.2; also Figure 4] The t-tests comparing valid versus invalid authentication actions treat individual typing actions as independent observations, but these actions are nested within trajectories (702 valid and 287 invalid actions across 182 trajectories). The same issue affects the valid/invalid action comparison in Figure 4. Such tests overstate statistical significance and can make the diagnostic claim look stronger than the data support. Please use trajectory-level aggregation, mixed-effects models, or cluster-robust permutation tests.
minor comments (5)
  1. [Throughout] There are multiple typos and formatting errors: 'environemnts', 'corrsponding', 'Comparment', 'authientification', and an inconsistent use of 'EELMA (NL)' vs 'NL-EELMA'. These should be corrected.
  2. [Appendix B] The sentence 'we provide supplementary information to support the WebArena results in Section 4' should refer to Section 5, where the WebArena results actually appear.
  3. [Section 3, Eq. (1)] The notation in the definition of E(π_LM) is confusing: the conditioning 'I(a_t; s* | a)' should be 'I(a_t; s* | s_t)', and the outer expectation should be made explicit. Please rewrite this equation to clarify that the mutual information is already averaged over s_t, a_t, and s*.
  4. [Appendix D.1] The temperature τ is described as 'updated over training' and 'adaptively trainable', but the loss in Eq. (8) treats it as a fixed hyperparameter. Please state explicitly whether τ is learned, what its final value is, and whether sensitivity to τ was assessed.
  5. [Figure 6] The figure reports Pearson correlations in the left panel (r = 0.91, 0.70, 0.71, 0.85) but the text discusses model families and sizes. It would be helpful to add per-panel sample sizes and confidence intervals, and to clarify whether the correlations are across the individual model points or across the ablation groups.

Circularity Check

0 steps flagged

No significant circularity: EELMA is an empirical estimator validated against direct computation, and the reward correlations are measured rather than forced by the estimator's construction.

full rationale

The paper's central derivation chain is not circular. EELMA is trained exclusively on (state, action, future state) tuples through an InfoNCE objective; reward signals and task goals never enter the training loss. The claimed correlations between estimated empowerment and mean discounted reward are therefore empirical findings, not quantities fitted by construction. The theoretical connection between empowerment and reward is imported from external prior work (Myers et al., 2025) with explicitly stated assumptions (uniform reward prior, ergodicity, Boltzmann rationality), and the paper does not rely on a self-citation chain to force its conclusions. In the toy domains, EELMA is checked against direct empowerment estimation computed from empirical transition counts, providing an independent, if same-data, validation. The main technical caveat is Eq. (7): the subtraction of the two InfoNCE critics cancels the log C1 and log C2 constants from Eqs. (5)-(6) only if those constants are equal or otherwise known, and the paper does not prove this; similarly, the discount gamma used when sampling future states is not specified. These are correctness and rigor gaps that could bias the estimates, but they are not cases where the paper's predictions reduce to its inputs by definition, nor do they involve load-bearing self-citations. The paper also openly reports a domain (Shopping) where the empowerment-reward relationship is flat, which further indicates the correlation is not an artifact of the estimator's construction. Overall, no circular step is exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central empirical claim rests on a standard InfoNCE bound, an MDP modeling choice, a convergence assumption borrowed from prior work, and a set of strong theoretical assumptions for the empowerment-reward link. The paper contributes the application and empirical measurements, not new theoretical machinery.

free parameters (4)
  • Discount factor γ for EELMA future-state sampling = not specified in paper
    Effective empowerment samples future states τ∼Geom(1−γ); γ is never given for EELMA, so the exact MI objective is undefined.
  • InfoNCE temperature τ = initialized 1.0, trainable
    Temperature controls the contrastive distribution; learned during training, affects MI estimate tightness.
  • Batch size K (negative sample count) = 256
    InfoNCE bound and its bias depend on K; chosen by hand.
  • Training steps = 10,000 (approx.)
    Convergence criterion chosen empirically; direct estimation asymptotes used as reference.
axioms (5)
  • standard math InfoNCE is a valid lower bound on mutual information
    Used in Eq. (4)-(7); the bound's tightness depends on batch size and critic capacity.
  • domain assumption Text environments are fully observable MDPs with T(s'|s,a)
    Section 3 frames LM agents in an MDP; WebArena DOM observations are treated as states, though partially observable in practice.
  • domain assumption InfoNCE critics converge to the log-density ratio up to constants (Eqs. 5-6)
    Adopted from Myers et al. (2025); the constant-cancellation C1=C2 is assumed in Eq. (7).
  • ad hoc to paper Skill Coverage, Ergodicity, Boltzmann rationality (Appendix A.2)
    These assumptions connect empowerment to goal-averaged reward; they are strong and not validated for the tested LM agents.
  • domain assumption Pretrained text embeddings preserve the state information relevant to MI
    EELMA's MI estimate depends on the fidelity of e5 and jina embeddings plus the learned projection.

pith-pipeline@v1.3.0-alltime-deepseek · 23549 in / 13967 out tokens · 119174 ms · 2026-08-04T14:49:28.610742+00:00 · methodology

0 comments
read the original abstract

As language model (LM) agents become increasingly capable and adopted in real-world applications, there is a growing need for scalable evaluation frameworks beyond costly, manually designed benchmarks. We propose information-theoretic evaluation based on empowerment, an information-theoretic measure of an agent's influence on future states through its actions. To handle the unique challenges of text-based environments, we introduce EELMA (Estimating Empowerment of Language Model Agents), an algorithm for approximating effective empowerment from multi-turn text interactions. We demonstrate EELMA on textual games and realistic web and tool-use environments, showing that empowerment strongly correlates with average task performance. We further analyze how empowerment varies across models, environment complexity, and agent configurations, and show that high-empowerment states and actions often mark pivotal moments for general capabilities. These results establish empowerment as a goal-agnostic metric that complements task-success measures for LM-agent evaluation.

Figures

Figures reproduced from arXiv: 2509.22504 by Jeff Gore, Jinyeop Song, Max Kleiman-Weiner.

Figure 1
Figure 1. Figure 1: Empowerment reflects an agent’s ability to reach diverse future states. (Top) A low-empowerment LM-agent becomes trapped in a loop and thus can access only a small fraction of states. (Bottom) A high-empowerment LM-agent effectively explores a wider range of trajectories and can successfully reach states that solve different random goals. available actions (optionality) and can effectively chain them toget… view at source ↗
Figure 2
Figure 2. Figure 2: EELMA Overview. EELMA quantifies the empowerment of LM-agent from text-based trajectories by mapping textual observations and actions to compact embeddings and estimating variational mutual information using InfoNCE (Le-Khac et al., 2020). these options unfold over multiple turns: empowerment explicitly captures an agent’s ability to sustain influence and preserve optionality across a sequence of interacti… view at source ↗
Figure 3
Figure 3. Figure 3: EELMA accurately estimates the effective empowerment. We validated the EELMA algorithm in three Gridworld scenarios and the Tower Of Hanoi(ToH). (A) State-conditional empower￾ment estimated by EELMA closely aligns with direct estimation. Heatmaps represent empowerment averaged across agent positions in the Gridworld. The graphs display empowerment for configuration (merged by permutation symmetry) in the T… view at source ↗
Figure 4
Figure 4. Figure 4: EELMA identifies influential actions. State–action conditional empowerment for valid (leading to novel states according to the game rules) and invalid actions in GridWorld (left) and ToH (right). Valid actions, which produce meaningful state transitions (e.g., moving to an empty grid in GridWorld, or placing a smaller disk onto a larger one in ToH), exhibit significantly higher empowerment than invalid act… view at source ↗
Figure 5
Figure 5. Figure 5: Environmental complexity affects effective empowerment. We vary the number of boxes from 4 to 7 in a 4-by-4 Gridworld (left), and the number of disks from 3 to 5 in the ToH of 3 rods (right). The effective empowerment of the LM-agent progressively decreases in environments compared to max empowerment (e.g., theoretical bound that optimal policy can exert influence) in higher complexity, correlating closely… view at source ↗
Figure 6
Figure 6. Figure 6: Empowerment and performance across variations of LM-agents. We evaluated how Chain-of-Thought (CoT) prompting (first column), memory context length (second column), and the choice between publicly available and closed base models (third and fourth columns) affect effective empowerment and mean discounted reward. Gridworld results are presented in the top row, and ToH results in the bottom row. 7 [PITH_FUL… view at source ↗
Figure 7
Figure 7. Figure 7: EELMA in WebArena, a realistic web browsing environment. We applied EELMA to across four domains of the WebArena benchmark using GPT-4o-mini, GPT-4o, and o3. (A) Overall performance across the four domains was quantified using mean discounted reward. (B) Domain-wise empowerment scores computed by EELMA effectively shows strong correlation with discounted rewards. Error bars indicate standard deviation acro… view at source ↗
Figure 8
Figure 8. Figure 8: EELMA Captures Valid Authentication Actions. We analyzed state-action empowerment estimates for authentication behaviors(username/password typing). Typing valid usernames and passwords resulted in high empowerment, whereas invalid actions did not (Right panel). possibility, the estimated empowerment values for the Shopping domain are already relatively high suggesting that empowerment over the environment … view at source ↗
Figure 9
Figure 9. Figure 9: Convergence of Empowerment Estimates in Gridworld and Tower of Hanoi Tasks. Empowerment estimates similarly reach convergence by 10,000 training steps for (A) three Gridworld scenarios and (B) Tower of Hanoi task. Red dashed lines indicate asymptotic empowerment levels by direct calculation. Shaded areas represent standard deviations across runs. C SUPPLEMENTARY : WEBARENA Here, we provide supplementary in… view at source ↗
Figure 10
Figure 10. Figure 10: shows the empowerment results for valid action typing in the modified shopping WebArena environment, using GPT-4o with 1-memory. Valid Authentication (n=702) Invalid Authentication (n=287) 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 Empowerment (bits) 0.365 -0.127 p < 0.001*** Merged Categories Valid Username (n=458) Invalid Username (n=22) Valid Password (n=244) Invalid Password (n=265) 0.0 0.5 1.0 1.5 2.0 0… view at source ↗
Figure 11
Figure 11. Figure 11: Mean empowerment comparison across four methods. [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: State-conditional empowerment comparison across four methods. [PITH_FULL_IMAGE:figures/full_fig_p026_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: EELMA achieves accurate empowerment estimation. In both Gridworld (left) and Tower of Hanoi (right), prompt-only LLMs substantially overestimate empowerment, whereas EELMA closely matches Direct Estimation. Error bars show standard deviation over 5 replicates. These findings underscore a methodological insight: while LLMs can articulate the theory of empowerment, they lack the computational grounding need… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 1 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Llama 3 models

    Meta AI. Llama 3 models. https://ai.facebook.com/llama-3, 2024

    2024

  3. [3]

    Rest meets react: Self-improvement for multi-step reasoning llm agent

    Renat Aksitov, Sobhan Miryoosefi, Zonglin Li, Daliang Li, Sheila Babayan, Kavya Kopparapu, Zachary Fisher, Ruiqi Guo, Sushant Prakash, Pranesh Srinivasan, Manzil Zaheer, Felix Yu, and Sanjiv Kumar. Rest meets react: Self-improvement for multi-step reasoning llm agent. arXiv preprint arXiv:2312.10003, 2023. URL https://arxiv.org/abs/2312.10003

    Pith/arXiv arXiv 2023

  4. [4]

    Swe-bench+: Enhanced coding benchmark for llms

    Reem Aleithan, Haoran Xue, Mohammad Mahdi Mohajer, Elijah Nnorom, Gias Uddin, and Song Wang. Swe-bench+: Enhanced coding benchmark for llms. arXiv preprint arXiv:2410.06992, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Balrog: Benchmarking agentic llm and vlm reasoning on games

    Anonymous. Balrog: Benchmarking agentic llm and vlm reasoning on games. In International Conference on Learning Representations, 2024

    2024

  6. [6]

    Claude 3 (haiku, sonnet)

    Anthropic. Claude 3 (haiku, sonnet). https://www.anthropic.com/claude-3, 2024

    2024

  7. [7]

    An algorithm for computing the capacity of arbitrary discrete memoryless channels

    Suguru Arimoto. An algorithm for computing the capacity of arbitrary discrete memoryless channels. IEEE Transactions on Information Theory, 18 0 (1): 0 14--20, 1972

    1972

  8. [8]

    wav2vec 2.0: A framework for self-supervised learning of speech representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, pp.\ 12449--12460, 2020

    2020

  9. [9]

    Information prioritization through empowerment in visual model-based rl

    Homanga Bharadhwaj, Mohammad Babaeizadeh, Dumitru Erhan, and Sergey Levine. Information prioritization through empowerment in visual model-based rl. In International Conference on Learning Representations, 2022

    2022

  10. [10]

    Llms struggle with perception, not reasoning (arc-agi)

    Mikel Bober-Irizar. Llms struggle with perception, not reasoning (arc-agi). https://anokas.substack.com/p/llms-struggle-with-perception-not-reasoning-arcagi, January 2025. Accessed: 2025-05-15

    2025

  11. [11]

    Qwen 2.5 models

    Alibaba Cloud. Qwen 2.5 models. https://modelscope.cn/models/qwen, 2024

    2024

  12. [12]

    Gemma models

    Google DeepMind. Gemma models. https://deepmind.google/gemma, 2023

    2023

  13. [13]

    Yilun Du, Shie Tiomkin, Emre Kiciman, Daniel Polani, Pieter Abbeel, and Anca D. Dragan. AvE : Assistance via empowerment. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020), pp.\ 4560--4571, 2020

    2020

  14. [14]

    Revisit the arimoto-blahut algorithm: New analysis with approximation, 2025

    Michail Fasoulakis, Konstantinos Varsos, and Apostolos Traganitis. Revisit the arimoto-blahut algorithm: New analysis with approximation, 2025. URL https://arxiv.org/abs/2407.06013

    Pith/arXiv arXiv 2025

  15. [15]

    Jina embeddings 2: 8192-token general-purpose text embeddings for long documents, 2023

    Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. Jina embeddings 2: 8192-token general-purpose text embeddings for long documents, 2023

    2023

  16. [16]

    Jimenez, John Yang, Alex L

    Carlos E. Jimenez, John Yang, Alex L. Zhang, Kilian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R. Narasimhan, Diyi Yang, Sida Wang, and Ofir Press. Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023

    Pith/arXiv arXiv 2023

  17. [17]

    Empowerment for continuous agent--environment systems

    Tobias Jung, Daniel Polani, and Peter Stone. Empowerment for continuous agent--environment systems. Adaptive Behavior, 19 0 (1): 0 16--39, 2011. doi:10.1177/1059712310392389

    work page doi:10.1177/1059712310392389 2011

  18. [18]

    Empowerment: A universal agent-centric measure of control

    Alexander S Klyubin, Daniel Polani, and Chrystopher L Nehaniv. Empowerment: A universal agent-centric measure of control. In 2005 IEEE Congress on Evolutionary Computation, volume 1, pp.\ 128--135. IEEE, 2005

    2005

  19. [19]

    Lucas, and Jonathan Gratch

    Deuksin Kwon, Emily Weiss, Tara Kulshrestha, Kushal Chawla, Gale M. Lucas, and Jonathan Gratch. Are llms effective negotiators? systematic evaluation of the multifaceted capabilities of llms in negotiation dialogues, 2024. URL https://arxiv.org/abs/2402.13550

    Pith/arXiv arXiv 2024

  20. [20]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP). ACM, 2023. doi:10.1145/3600006.3613165. URL https://arxiv.org/abs/2...

    arXiv 2023

  21. [21]

    Le-Khac, Graham Healy, and Alan F

    Phuc H. Le-Khac, Graham Healy, and Alan F. Smeaton. Contrastive representation learning: A framework and review. IEEE Access, 8: 0 193907–193934, 2020. ISSN 2169-3536. doi:10.1109/access.2020.3031549. URL http://dx.doi.org/10.1109/ACCESS.2020.3031549

    arXiv 2020

  22. [22]

    Stride: A tool-assisted llm agent framework for strategic and interactive decision-making, 2024

    Chuanhao Li, Runhan Yang, Tiankai Li, Milad Bafarassat, Kourosh Sharifi, Dirk Bergemann, and Zhuoran Yang. Stride: A tool-assisted llm agent framework for strategic and interactive decision-making, 2024. URL https://arxiv.org/abs/2405.16376

    Pith/arXiv arXiv 2024

  23. [23]

    Zebralogic: On the scaling limits of llms for logical reasoning

    Bill Yuchen Lin, Ronan Le Bras, Kyle Richardson, Ashish Sabharwal, Radha Poovendran, Peter Clark, and Yejin Choi. Zebralogic: On the scaling limits of llms for logical reasoning. arXiv preprint arXiv:2502.01100, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Vila: On pre-training for visual language models, 2024

    Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models, 2024. URL https://arxiv.org/abs/2312.07533

    Pith/arXiv arXiv 2024

  25. [25]

    Variational information maximisation for intrinsically motivated reinforcement learning

    Shakir Mohamed and Danilo Jimenez Rezende. Variational information maximisation for intrinsically motivated reinforcement learning. In Advances in Neural Information Processing Systems, pp.\ 2125--2133, 2015

    2015

  26. [26]

    Learning to assist humans without inferring rewards, 2025

    Vivek Myers, Evan Ellis, Sergey Levine, Benjamin Eysenbach, and Anca Dragan. Learning to assist humans without inferring rewards, 2025. URL https://arxiv.org/abs/2411.02623

    Pith/arXiv arXiv 2025

  27. [27]

    Gpt-3.5 turbo

    OpenAI. Gpt-3.5 turbo. https://platform.openai.com/docs/models/gpt-3-5, 2023 a

    2023

  28. [28]

    OpenAI. Gpt-4. https://openai.com/blog/gpt-4, 2023 b

    2023

  29. [29]

    OpenAI. Gpt-4o. https://openai.com/blog/gpt-4o, 2024

    2024

  30. [30]

    Autonomous evaluation and refinement of digital agents

    Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Autonomous evaluation and refinement of digital agents. arXiv preprint arXiv:2404.06474, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Evaluating frontier models for dangerous capabilities

    M Phuong, M Aitchison, E Catt, S Cogan, A Kaskasoli, V Krakovna, D Lindner, M Rahtz, Y Assael, S Hodkinson, et al. Evaluating frontier models for dangerous capabilities. arxiv. arXiv preprint arXiv:2403.13793, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp.\ 8748--8763. PMLR, 2021

    2021

  33. [33]

    Zimmermann, and Wieland Brendel

    Evgenia Rusak, Patrik Reizinger, Attila Juhos, Oliver Bringmann, Roland S. Zimmermann, and Wieland Brendel. Infonce: Identifying the gap between theory and practice, 2025. URL https://arxiv.org/abs/2407.00143

    Pith/arXiv arXiv 2025

  34. [34]

    Empowerment—an introduction

    Christoph Salge, Cornelius Glackin, and Daniel Polani. Empowerment—an introduction. In Guided Self-Organization: Inception, pp.\ 67--114. Springer, 2014

    2014

  35. [35]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi - Yu, Roberto Dess ` i , Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023

    Pith/arXiv arXiv 2023

  36. [36]

    Stanley and Joel Lehman

    Kenneth O. Stanley and Joel Lehman. Why Greatness Cannot Be Planned: The Myth of the Objective. Springer, 2015. ISBN 978-3-319-15523-4

    2015

  37. [37]

    Parametrically retargetable decision-makers tend to seek power

    Alexander Matt Turner and Prasad Tadepalli. Parametrically retargetable decision-makers tend to seek power. In Advances in Neural Information Processing Systems, volume 35, pp.\ 31391--31401, 2022. URL https://arxiv.org/abs/2206.13477

    Pith/arXiv arXiv 2022

  38. [38]

    Optimal policies tend to seek power

    Alexander Matt Turner, Logan Smith, Rohin Shah, Andrew Critch, and Prasad Tadepalli. Optimal policies tend to seek power. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 2021

    2021

  39. [39]

    Multilingual e5 text embeddings: A technical report

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024

    2024

  41. [41]

    A-mem: Agentic memory for llm agents

    Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023. URL https://arxiv.org/abs/2210.03629

    Pith/arXiv arXiv 2023

  43. [43]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023

    Pith/arXiv arXiv 2023

  44. [44]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  45. [45]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  46. [46]

    agent is located at x=2,y=1

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv