TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

Chengming Zhou; Dawei Wang; Di Zhao; Gary Ushaw; Marci Chi Ma; Richard Davison; Xinyuan Liu

REVIEW 3 major objections 3 minor 2 cited by

TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents

T0 review · 3 major / 3 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A tower-defense benchmark shows LLM agents reach only about 40% of human-expert scores on long-term planning and decision-making.

desk verdict A useful, lightweight TD benchmark for LLM agents with a credible human baseline, but the text-only observation may be inflating the apparent planning gap. read the letter →

arxiv 2601.05899 v2 pith:T6RQK4T5 submitted 2026-01-09 cs.AI

Dawei Wang , Chengming Zhou , Di Zhao , Xinyuan Liu , Marci Chi Ma , Gary Ushaw , Richard Davison This is my paper · ORCID

classification cs.AI

keywords TowerMinddefensegamesLLMagentslong-termplanningdecision-makinghallucinationevaluationmultimodalobservationsRTSgamebenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TowerMind is a lightweight tower defense game environment built to evaluate how well LLMs plan and decide over long horizons. The paper's central claim is empirical: across five levels, the best language-only model reaches 0.38 of the human-normalized score, the best vision-language model reaches 0.42, and humans sit at 1.0. On the hardest level, every evaluated model falls at least 84% below the human baseline. The paper also measures hallucination as the share of actions that are invalid under game rules, finding that commercial models mostly follow rules but still make strategically weak choices. If the results hold, TowerMind offers a low-cost, reproducible way to test whether LLM agents can turn correct actions into effective plans.

What carries the argument

The central object is TowerMind itself: a tower-defense game where enemies follow fixed roads and players build, upgrade, and sell towers, control knights and a hero, and collect gold through a hybrid action space of continuous coordinates plus twelve discrete action types. Two design choices carry the argument: a JSON textual observation that lists coordinates for tower points, enemy paths, and unit states without derived spatial features, and a valid-action-rate metric that treats rule-violating actions as hallucinations. Five benchmark levels use a quantitative difficulty formula and a human-expert baseline, so scores and valid-action rates are normalized relative to humans.

What would settle it

Have a competent human play TowerMind levels while seeing only the JSON observation, with no pixel image, and compare their scores to the same experts' full-screen scores; if JSON-only humans still play at expert level, the models' failures are planning failures, but if their scores collapse, the textual representation is the bottleneck. A cheaper version: append each tower point's distance to the nearest enemy path and check whether the best models stop wasting gold on useless tower points.

Watch

Extended reading notes

Core claim

The paper's central claim is empirical: on a five-level tower-defense benchmark, current LLM agents perform far below human experts. The strongest language-only model reaches 0.38 of the human-normalized score, the strongest vision-language model reaches 0.42, and on the hardest level every model is at least 84% below the human baseline. At the same time, the best models achieve valid-action rates above 0.85, meaning most of their actions obey the rules; the failure is in effectiveness, not just rule-following. Qualitative analysis attributes this to inadequate validation of plans (building on tower points that never engage enemies), a lack of multifinality (actions that serve only one goal)

Load-bearing premise

The benchmark assumes the JSON textual observation contains everything a player needs to decide well, so a model's poor tower placement is a planning failure rather than a consequence of missing spatial information.

Editorial extensions

If this is right

TowerMind can serve as a low-cost benchmark for LLM planning and decision-making, requiring far less compute and disk than heavier real-time-strategy environments while still producing measurable gaps against humans.
The gap between valid action rate and score implies that progress in rule-following does not automatically translate to strategic effectiveness; interactive benchmarks should measure both.
Hallucination, measured as invalid actions, rises with level difficulty, so TowerMind can track how reliably models behave as prompts and game states grow more complex.
Most models improve when given pixel observations alongside text, suggesting multimodal input contributes to planning in dynamic environments.
Classic RL algorithms trained for 100 million steps remain below human expert scores, indicating TowerMind is also a demanding testbed for reinforcement learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves implicit that the text-observation bottleneck is directly testable: adding a single derived field per tower point, such as distance to the nearest enemy path, could substantially change model placement choices; the included level editor makes this an easy experiment.
The valid-action-rate metric could generalize beyond games as a cheap proxy for rule-following without strategic effectiveness in other interactive LLM benchmarks.
The seeded randomness and custom level editor also enable contamination-resistant evaluation, since researchers can generate fresh levels at test time rather than reusing fixed public ones.
The observed lack of multifinality may reflect prompt objectives more than model capability; explicit subgoal instructions or reward shaping could encourage actions that simultaneously gather resources and engage enemies.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: TowerMind is a real contribution to the LLM-agent benchmark space. It gives you a cheap tower-defense environment with text, pixel, and structured observations, a level editor, and a valid-action-rate hallucination metric. The computational footprint is genuinely lower than SC2LE-based alternatives, and the appendix has enough configuration detail to get a sense of the environment. The human baseline (five experts, ~13h training, standard errors) is a nice touch, and five seeds per model is respectable for this kind of evaluation. The main quantitative finding — best LLM around 0.38–0.42 normalized score versus human 1.0 — is well supported by the tables.

The soft spot the stress-test flags is real. The JSON text observation gives raw coordinates for tower points and enemy paths, but it does not give derived relations like 'this tower point can reach the road.' So a language-only LLM has to do distance-to-segment geometry at every decision step, under a prompt that describes attack range as diameter. That is a nontrivial burden, and it makes Section 4.4's phrase 'all necessary information' too generous. The misleading-tower-point failures may be partly an artifact of the observation format, not just weak strategic planning. I don't think this kills the benchmark — the gap is large and consistent — but it means the paper should be more careful about what exactly is being measured.

There are also a few consistency issues that need cleanup: the main text tables say Gemini-2.5-Pro while the appendix tables say Gemini-1.5-Pro; Section 5 says five benchmark levels while Appendix H says nine; and the 'vision input improves performance' claim is not significance-tested. None of these are deep flaws, but they make the paper look sloppier than it is.

Who is this for? Anyone working on LLM agents who wants a fast, customizable environment with a text interface. It is not field-changing, but it fills a real gap. I would send it to peer review: the environment and benchmarks are useful, and the observation-interface issue is exactly what a good reviewer can help pin down. Recommend minor-to-major revision, not rejection.

Referee Report

3 major / 3 minor

Summary. TowerMind is a lightweight Unity-based tower defense environment intended as a benchmark for LLM agents, with pixel, textual (JSON), and structured observations, a hybrid action space, fog of war, and a level editor. The paper evaluates seven LLMs (GPT-4.1, Gemini-2.5-Pro, Claude 3.7 Sonnet, Llama 3.2 90B/11B, Qwen2.5-VL 72B/7B) on five benchmark levels under language-only and vision-language conditions, against a five-expert human baseline, using human-normalized score and valid-action-rate metrics. It also reports Ape-X DQN and PPO baselines. The central empirical claim is that LLMs are far below human experts (best average normalized score 0.38 language-only, 0.42 vision-language) and that hallucination, measured by invalid action rate, increases with level difficulty. Qualitative analysis identifies inadequate planning validation, lack of multifinality, and underuse of actions.

Significance. If the results are reliable, TowerMind is a useful contribution: it provides a low-cost, multimodal, reproducible environment for evaluating long-term planning and decision-making in LLMs, with a built-in hallucination metric and a GUI level editor. The empirical core is credible — five seeds per model, standard errors, a human baseline, and a random baseline are reported, and the source code is public. The performance gap between LLMs and human experts is consistent across levels and modalities. The main value is as a benchmark and environment rather than as a theoretical contribution; its significance depends on the soundness of the observation interface and the correctness of the reported model identities.

major comments (3)

[Section 4.4 and Appendix E] The qualitative claim that LLMs 'consistently chose to build towers on misleading tower points' is presented as evidence of inadequate planning validation, with the assertion that 'despite having access to all necessary information in the prompt' the models failed. However, the textual observation (Listing 6) provides only raw coordinates and does not supply derived spatial relations such as 'this tower point can reach any enemy path segment.' More importantly, the prompt in Appendix E states: 'the attack range described above is the diameter.' This is a non-standard convention; if the game engine treats the listed AttackRange as a radius (as is conventional), then the prompt provides misleading information, and the observed tower placements could result from this ambiguity rather than from a planning deficit. Please clarify whether the diameter convention is intentional and matches the
[Tables 2/3 vs Appendix G Tables 7/8] The main text and Section 4.1 evaluate 'Gemini-2.5-Pro', but the appendix tables reporting the same numerical results list 'Gemini-1.5-Pro'. Since the values are identical, this is likely a naming error, but it affects which model is actually benchmarked and must be corrected for the results to be reproducible.
[Appendix H] The RL experiments section states that 'each episode was initialized by randomly selecting one of the nine benchmark levels,' while the abstract, Section 5, and Table 5 describe exactly five benchmark levels. This internal inconsistency about the number of levels used in RL training is load-bearing for the RL benchmark claims and must be resolved.

minor comments (3)

[Listing 6] The JSON field 'Level_Dropped_Gold_Coins_3_Realtime_Status' appears to contain a stray '3'; this is likely a typo. Other typos include 'Is_Bulit' and 'Countdwon'.
[Section 3.3] The difficulty metric D(l) is an ad hoc linear combination of four normalized components. The paper does not validate this metric against perceived difficulty or empirical success rates. It is descriptive, but the authors should state this more explicitly to avoid over-interpretation.
[Appendix E] The zero-shot prompt contains the line 'The following is the information about this level, organized in Json format' followed by a level state that does not include the full real-time status fields shown in Listing 6 (e.g., fog of war position is absent in the level-state example). This may be an abbreviated example, but it is confusing and should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: TowerMind's benchmark results are direct measurements; no fitted parameter is renamed as a prediction.

full rationale

The paper's central claims are empirical measurements of LLM and human performance on a new game environment. The score and valid-action-rate metrics are defined directly from environment reward and action validity (Section 4.1), and the human-normalized scores are a transparent scaling convention stated in Appendix G: s_l = (s_raw - s_min)/(s_human - s_min). No parameter is fitted to model outputs and then reported as a prediction. The level difficulty metric D(l) in Section 3.3 is a descriptive design metric composed of map/tower/enemy/resource statistics, not fitted to LLM performance; the observed monotonic relation with hallucination is an empirical correlation across five levels, not a consequence of the metric's construction. The LLM-human performance gap is a direct comparison to a human baseline collected independently (Appendix F), and the qualitative findings are behavioral observations of trajectories. There is no load-bearing self-citation: the cited StarCraft II LLM benchmarks are by different author groups, and no uniqueness theorem or ansatz is imported from the authors' prior work. The skeptical concern that the JSON textual observation leaves spatial relations implicit is a validity or interface-design threat to interpretation of the gap, not a circularity: the benchmark does not define model failure in terms of the observation format, and the claim that all necessary information is present is an assumption about information content, not a derivation that reduces to its own input. Therefore the derivation chain is self-contained and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims are empirical benchmark measurements, so no fitted parameters or invented theoretical entities are required. The main assumptions are about observational sufficiency and the validity of the hallucination metric, not mathematical axioms.

assumptions (4)

domain assumption The textual and structured observations are a faithful, sufficient rendering of the game state for planning purposes.
Used in Section 4.4 to interpret LLM placement failures as reasoning errors; the JSON in Appendix B, Listing 6 provides coordinates and paths but no derived spatial features.
domain assumption Valid action rate is an acceptable operationalization of hallucination in this setting.
Defined in Section 4.1; invalid actions are treated as conflicts with game state or rules, which is a narrow, environment-specific notion of hallucination.
domain assumption Tower-defense game mechanics exercise the same long-term planning and decision-making capabilities as RTS games.
Stated in Section 1 to justify TowerMind as an RTS substitute; the game removes opponent unpredictability, which changes the strategic setting.
domain assumption Five human experts after 13 hours of training provide a stable reference baseline for normalization.
Appendix F reports five participants and standard errors; the normalized scores in Tables 2 and 3 treat this small sample as the reference value.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents." pith.science (2026). https://pith.science/paper/T6RQK4T5

@misc{pith2026260105899,
  author       = {Pith},
  title        = {Pith review of: TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T6RQK4T5}},
  note         = {Machine review of arXiv:2601.05899}
}

read the original abstract

Recent breakthroughs in Large Language Models (LLMs) have positioned them as a promising paradigm for agents, with long-term planning and decision-making emerging as core general-purpose capabilities for adapting to diverse scenarios and tasks. Real-time strategy (RTS) games serve as an ideal testbed for evaluating these two capabilities, as their inherent gameplay requires both macro-level strategic planning and micro-level tactical adaptation and action execution. Existing RTS game-based environments either suffer from relatively high computational demands or lack support for textual observations, which has constrained the use of RTS games for LLM evaluation. Motivated by this, we present TowerMind, a novel environment grounded in the tower defense (TD) subgenre of RTS games. TowerMind preserves the key evaluation strengths of RTS games for assessing LLMs, while featuring low computational demands and a multimodal observation space, including pixel-based, textual, and structured game-state representations. In addition, TowerMind supports the evaluation of model hallucination and provides a high degree of customizability. We design five benchmark levels to evaluate several widely used LLMs under different multimodal input settings. The results reveal a clear performance gap between LLMs and human experts across both capability and hallucination dimensions. The experiments further highlight key limitations in LLM behavior, such as inadequate planning validation, a lack of multifinality in decision-making, and inefficient action use. We also evaluate two classic reinforcement learning algorithms: Ape-X DQN and PPO. By offering a lightweight and multimodal design, TowerMind complements the existing RTS game-based environment landscape and introduces a new benchmark for the AI agent field. The source code is publicly available on GitHub(https://github.com/tb6147877/TowerMind).

Figures

Figures reproduced from arXiv: 2601.05899 by the authors.

**Figure 2.** Left: A screenshot of the TowerMind environment with key game elements annotated. The coordinate axes illustrate [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** See Appendix H for detailed information about the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Training curves, the horizontal axis shows the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Screenshots of the 15 enemy types, with IDs from [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 7.** Figure 7: Each predefined tower point is associated with a [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 6.** Figure 6: Pixel-based Observation of the TowerMind Envi [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

**Figure 8.** Figure 8: This is a screenshot of the TowerMind level editor. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: The game guide for human participants. Each participant was compensated at a rate of £17.89 per hour for their time. All five participants were provided with a game guide for the TowerMind environment (as shown in [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

**Figure 11.** Figure 11: Model architecture of the MLP model, the struc [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

**Figure 10.** Figure 10: Model architecture of the multiple-layers CNN [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models
cs.AI 2026-06 unverdicted novelty 7.0 of 10

RTSGameBench is a new extensible benchmark for VLMs using diverse RTS matchups, diagnostic mini-games targeting individual competencies, and a self-evolving query-to-game generator, with results showing poor VLM perfo...
MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation
cs.AI 2026-07 conditional novelty 6.0 of 10

Credit assignment via LMM pairwise comparisons plus Bradley–Terry rank aggregation and potential-based shaping improves cooperative MARL under sparse rewards and dynamic agent counts.

Reference graph

Works this paper leans on

8 extracted references · 7 linked inside Pith · cited by 2 Pith papers

[4]

Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Gold- berg, K.; Gonzalez, J

LLM-PySC2: Starcraft II learning environment for Large Language Models.arXiv e-prints, arXiv:2411.05348. Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Gold- berg, K.; Gonzalez, J. E.; Jordan, M. I.; and Stoica, I. 2018. RLlib: Abstractions for Distributed Reinforcement Learn- ing. InInternational Conference on Machine Learning (ICML). Lingo, R....

arXiv 2018
[6]

Renze, M.; and Guven, E

Instruction tuning with gpt-4.arXiv preprint arXiv:2304.03277. Renze, M.; and Guven, E. 2024. Self-reflection in llm agents: Effects on problem-solving performance.arXiv preprint arXiv:2405.06682. Rummell, P. A. 2011. Adaptive ai to play tower defense game. In2011 16th International Conference on Computer Games (CGAMES), 38–40. IEEE. Russell, S. J.; and N...

arXiv 2024
[2013]

InInternational Multi- Conference on Artificial Intelligence Technology, 135–146

Automated evaluation for AI controllers in tower de- fense game using genetic algorithm. InInternational Multi- Conference on Artificial Intelligence Technology, 135–146. Springer. Tian, Y .; Gong, Q.; Shang, W.; Wu, Y .; and Zitnick, C. L
[2017]

Towers": 2[ 3{

Elf: An extensive, lightweight and flexible research platform for real-time strategy games.Advances in Neural Information Processing Systems, 30. Vinyals, O.; Ewalds, T.; Bartunov, S.; Georgiev, P.; Vezhn- evets, A. S.; Yeo, M.; Makhzani, A.; K¨uttler, H.; Agapiou, J.; Schrittwieser, J.; et al. 2017. Starcraft ii: A new challenge for reinforcement learnin...

arXiv 2017
[2018]

Unity: A general platform for intelligent agents.arXiv preprint arXiv:1809.02627. Koc, V . 2025. Tiny QA Benchmark++: Ultra-Lightweight, Synthetic Multilingual Dataset Generation & Smoke- Tests for Continuous LLM Evaluation.arXiv preprint arXiv:2505.12058. Kruglanski, A. W.; Chernikova, M.; Babush, M.; Dugas, M.; and Schumpe, B. M. 2015. The architecture ...

arXiv 2025
[2019]

InProceedings of the 14th International Conference on the Foundations of Digital Games, 1–9

Automatic generation of tower defense levels using PCG. InProceedings of the 14th International Conference on the Foundations of Digital Games, 1–9. Liu, X.; Yu, H.; Zhang, H.; Xu, Y .; Lei, X.; Lai, H.; Gu, Y .; Ding, H.; Men, K.; Yang, K.; et al. 2023. Agentbench: Evaluating llms as agents.arXiv preprint arXiv:2308.03688. Ma, W.; Fu, Y .; Zhang, Z.; and...

arXiv 2023
[2023]

C ¸ elen, A.; Han, G.; Schindler, K.; Van Gool, L.; Armeni, I.; Obukhov, A.; and Wang, X

Exploring the intersection of Generative AI and Soft- ware Development.arXiv preprint arXiv:2312.14262. C ¸ elen, A.; Han, G.; Schindler, K.; Van Gool, L.; Armeni, I.; Obukhov, A.; and Wang, X. 2024. I-design: Personalized llm interior designer.arXiv preprint arXiv:2404.02838. Comanici, G.; Bieber, E.; Schaekermann, M.; Pasupat, I.; Sachdeva, N.; Dhillon,...

arXiv 2024
[2024]

Haduong, N.; Wang, I.; Lu, B.-R.; Ammanabrolu, P.; and Smith, N

Length-controlled alpacaeval: A simple way to debias automatic evaluators.arXiv preprint arXiv:2404.04475. Haduong, N.; Wang, I.; Lu, B.-R.; Ammanabrolu, P.; and Smith, N. A. 2024. CPS-TaskForge: Generating Collabora- tive Problem Solving Environments for Diverse Communi- cation Tasks. InProceedings of the 1st Workshop on Cus- tomizable NLP: Progress and ...

arXiv 2024

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [4]

Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Gold- berg, K.; Gonzalez, J

LLM-PySC2: Starcraft II learning environment for Large Language Models.arXiv e-prints, arXiv:2411.05348. Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Gold- berg, K.; Gonzalez, J. E.; Jordan, M. I.; and Stoica, I. 2018. RLlib: Abstractions for Distributed Reinforcement Learn- ing. InInternational Conference on Machine Learning (ICML). Lingo, R....

arXiv 2018

[2] [6]

Renze, M.; and Guven, E

Instruction tuning with gpt-4.arXiv preprint arXiv:2304.03277. Renze, M.; and Guven, E. 2024. Self-reflection in llm agents: Effects on problem-solving performance.arXiv preprint arXiv:2405.06682. Rummell, P. A. 2011. Adaptive ai to play tower defense game. In2011 16th International Conference on Computer Games (CGAMES), 38–40. IEEE. Russell, S. J.; and N...

arXiv 2024

[3] [2013]

InInternational Multi- Conference on Artificial Intelligence Technology, 135–146

Automated evaluation for AI controllers in tower de- fense game using genetic algorithm. InInternational Multi- Conference on Artificial Intelligence Technology, 135–146. Springer. Tian, Y .; Gong, Q.; Shang, W.; Wu, Y .; and Zitnick, C. L

[4] [2017]

Towers": 2[ 3{

Elf: An extensive, lightweight and flexible research platform for real-time strategy games.Advances in Neural Information Processing Systems, 30. Vinyals, O.; Ewalds, T.; Bartunov, S.; Georgiev, P.; Vezhn- evets, A. S.; Yeo, M.; Makhzani, A.; K¨uttler, H.; Agapiou, J.; Schrittwieser, J.; et al. 2017. Starcraft ii: A new challenge for reinforcement learnin...

arXiv 2017

[5] [2018]

Unity: A general platform for intelligent agents.arXiv preprint arXiv:1809.02627. Koc, V . 2025. Tiny QA Benchmark++: Ultra-Lightweight, Synthetic Multilingual Dataset Generation & Smoke- Tests for Continuous LLM Evaluation.arXiv preprint arXiv:2505.12058. Kruglanski, A. W.; Chernikova, M.; Babush, M.; Dugas, M.; and Schumpe, B. M. 2015. The architecture ...

arXiv 2025

[6] [2019]

InProceedings of the 14th International Conference on the Foundations of Digital Games, 1–9

Automatic generation of tower defense levels using PCG. InProceedings of the 14th International Conference on the Foundations of Digital Games, 1–9. Liu, X.; Yu, H.; Zhang, H.; Xu, Y .; Lei, X.; Lai, H.; Gu, Y .; Ding, H.; Men, K.; Yang, K.; et al. 2023. Agentbench: Evaluating llms as agents.arXiv preprint arXiv:2308.03688. Ma, W.; Fu, Y .; Zhang, Z.; and...

arXiv 2023

[7] [2023]

C ¸ elen, A.; Han, G.; Schindler, K.; Van Gool, L.; Armeni, I.; Obukhov, A.; and Wang, X

Exploring the intersection of Generative AI and Soft- ware Development.arXiv preprint arXiv:2312.14262. C ¸ elen, A.; Han, G.; Schindler, K.; Van Gool, L.; Armeni, I.; Obukhov, A.; and Wang, X. 2024. I-design: Personalized llm interior designer.arXiv preprint arXiv:2404.02838. Comanici, G.; Bieber, E.; Schaekermann, M.; Pasupat, I.; Sachdeva, N.; Dhillon,...

arXiv 2024

[8] [2024]

Haduong, N.; Wang, I.; Lu, B.-R.; Ammanabrolu, P.; and Smith, N

Length-controlled alpacaeval: A simple way to debias automatic evaluators.arXiv preprint arXiv:2404.04475. Haduong, N.; Wang, I.; Lu, B.-R.; Ammanabrolu, P.; and Smith, N. A. 2024. CPS-TaskForge: Generating Collabora- tive Problem Solving Environments for Diverse Communi- cation Tasks. InProceedings of the 1st Workshop on Cus- tomizable NLP: Progress and ...

arXiv 2024