{"id":"5ba4b1ac-415c-47bf-8887-69061f2ffdc7","arxiv_id":"2502.05632","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A system paper describing a collaboratively designed online platform for FSM-based multi-agent environments, with a pilot study showing limited early usage.","lead":"This paper introduces a web platform where users design, play, and share tiny environments built from simple finite-state-machine agents. A short usage study shows the site works, but community adoption so far is limited, with only five fortresses submitted by non-authors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The database-diversity claim is not yet supported: Section IV reports only 33 fortresses, five from non-authors, and no measure of thematic diversity. Community adoption is therefore the load-bearing unverified premise of the central claim.","rationale":"I read the paper in good faith as a system description plus a small pilot study. The engine description is concrete, the editors, X-ray feature, backpack, and text-editor compiler are checkable artifacts, and the authors are unusually honest about the low submission count. The strongest claim, however, is stated as a current service: the system 'serves to provide a database of thematically diverse AI and game environments.' For that claim to be true, the database must actually contain a diverse corpus, not merely have the capacity to accumulate one. Section IV provides the only evidence, and it shows 33 fortresses, 28 of which are author-created, with no analysis of themes, genres, or domains. The absence of a diversity metric is not a stylistic omission; it is the missing evidence for an adjective in the central claim. The reader's weakest_assumption identifies the same load-bearing point: the platform's stated value depends on users outside the author team submitting a diverse set of fortresses, and that premise is not yet supported. I agree with that assessment. I did not find a more fundamental technical flaw: the 16x8 versus 14x6 grid statements are plausibly reconciled as outer grid versus interior placement area, and no derivation or correctness claim in the paper depends on a contested formula. The repository and open-source release are promised rather than delivered, but that affects reproducibility more than the central claim's current truth. A future audit of the live database with thematic labeling would settle whether the concern lands. Until then, the conditional verdict is appropriate, and my read does not change it.","tokens_in":8997,"tokens_out":4460,"duration_ms":49192,"concrete_test":"Take a fixed snapshot of the live site one year after the arXiv release, download every submitted fortress with its author, date, parent/remix link, notes, and entity definitions. Have at least two independent raters label each fortress's theme or genre from the notes and entity names, and compute inter-rater agreement, the number of non-author submitters, and the number of distinct themes contributed by non-author users. If the non-author submitter count and the distinct theme count are not materially larger than the five external fortresses and the handful of themes visible in Section IV, then the claim that the system provides a thematically diverse database is not supported. If the site is unavailable, the claim should be treated as unverified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract states that the system 'serves to provide a database of thematically diverse AI and game environments.' The data offered in Section IV cannot carry that statement. Of 33 fortresses, 28 come from the six authors; only five external fortresses arrived after launch. No content analysis, genre/topic labels, or diversity metric is reported, and the node-frequency histogram in Fig. 11 addresses node-type coverage, not thematic diversity. Thus the central claim depends on an adoption-and-diversity premise that the paper itself explicitly acknowledges as unfulfilled. This is not an internal inconsistency or a computational error; it is a scope-of-evidence gap. The platform may still be useful, and the proposed mixed-initiative tools may improve adoption, but the current claim that the system provides a diverse database is conditional on further community growth and on a demonstration that submitted fortresses span distinct themes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Amorphous Fortress Online, a web-based platform for designing, playing, and sharing small multi-agent environments (\"fortresses\") driven by finite-state machine (FSM) agents. The system includes an entity editor, fortress editor, text editor with compiler validation, a play screen with an X-ray debugging feature, and a sharing/remixing pipeline with a \"backpack\" for saving entities. The authors describe the engine's 13 action nodes and five edge conditions, then report a short post-launch study: 174 plays, 33 fortresses, 143 entities, and 298 nodes. The abstract claims the system will provide a database of thematically diverse AI and game environments and lays groundwork for future AI training and generative modeling experiments.","tokens_in":9142,"tokens_out":4110,"duration_ms":38524,"significance":"If the platform were to attract a sustained user community, it could provide a valuable, heterogeneous corpus of simple, transparent multi-agent environments, and the X-ray explainability feature is a nice design choice for human-understandable agent behavior. The paper reports a working, open-source system and honest usage statistics rather than hiding the limited external adoption. However, the central claim of delivering a \"thematically diverse\" database is not supported by the presented data: the corpus is small, mostly author-made, and no diversity measure is given. The significance is therefore currently prospective rather than demonstrated; the contribution is a systems paper with a credible foundation, not yet a validated data-resource paper.","major_comments":[{"comment":"The abstract states that the system \"serves to provide a database of thematically diverse AI and game environments,\" but the data reported in Section IV cannot support this claim. The study found only 33 fortresses total, with the text indicating that six authors submitted 28 of them and only five external fortresses arrived after launch. No content analysis, genre/topic labels, or diversity metric is reported; the node-frequency histogram in Fig. 11 addresses node-type coverage, not thematic diversity. The authors should either reframe this claim as an intended future outcome (as the Conclusion already does) or provide an operationalized definition of thematic diversity and demonstrate that the existing corpus satisfies it.","section":"Abstract and Section IV"},{"comment":"The usage statistics are internally inconsistent: the text first reports that six authors submitted 33 fortresses and that \"only five additional fortresses were submitted after the release of the site,\" which implies 28 author fortresses, but the following sentence says \"Outside of the author's submission of 21 fortresses, users submitted an average of two fortresses.\" This contradicts the earlier counts and also conflicts with the statement that there were five external submissions. Please correct the numbers and clarify the exact split between author and non-author fortresses, as well as the average external submissions.","section":"Section IV"}],"minor_comments":[{"comment":"The engine description in Section III.A says the fortress grid is 16 x 8 spaces, while Section III.D states that the fortress editor uses a \"14 by 6 grid space area.\" Since the text-editor compiler checks fortress dimensions, this discrepancy is confusing and should be aligned.","section":"Section III.A and Section III.D"},{"comment":"The sentence \"There were a total of 143 entities made in the website and a total of 298 nodes\" is ambiguous: does it count distinct entity definitions or placed instances, and does it include author submissions? Please clarify the counting unit.","section":"Section IV"},{"comment":"In the X-ray paragraph, \"transtitioned\" should be \"transitioned\" (Figure 8 caption area).","section":"Section III.F"},{"comment":"The histogram has no descriptive caption clarifying whether it counts nodes in all fortresses or per-entity; a short caption would improve interpretability.","section":"Figure 11"},{"comment":"The phrase \"The stark difference of fortress plays vs. fortress creation\" would be clearer as something like \"The contrast between the number of plays and the number of submissions.\"","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a system-description paper from a known group, and the platform itself appears functional. The main concern is that the abstract overclaims a diverse database without evidence of diversity or external adoption; the data inconsistencies in Section IV compound the problem. A revision that tempers the claim and corrects the internal inconsistencies would make this publishable as a systems/demonstration paper. The grid-size discrepancy suggests the manuscript was not carefully checked before submission, so I recommend the authors do a thorough pass before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: Amorphous Fortress Online is a real, playable system that ports the authors' earlier FSM-based artificial-life engine to the browser and adds collaborative editing tools. The system description is clear, and the X-ray FSM viewer is a legitimately nice debugging affordance. But the abstract's headline claim—that the site provides a database of thematically diverse AI environments—is not backed by the paper's own pilot data, which shows almost no external adoption and no attempt to measure thematic diversity.\n\nWhat's new: the online entity/fortress/text editors, the backpack entity-reuse mechanism, and the real-time X-ray view are new relative to the prior Amorphous Fortress papers. The port itself is not a scientific result, but the platform is a working infrastructure piece. The paper also does the right thing by openly acknowledging the adoption problem and proposing mixed-initiative tools to address it. That honesty deserves credit.\n\nSoft spots: first, the diversity claim. Thirty-three fortresses, mostly from the six authors, is not a diverse database, and no content analysis or genre labels are offered. The node-frequency histogram speaks to node-type coverage, not themes. This is the load-bearing gap: the system's future value for training generative or recommendation models depends on a community that hasn't formed yet. The authors know this, but the abstract overstates the present state. Second, the submission counts are hard to reconcile: the text says six authors submitted 33 fortresses, then says only five additional fortresses came in, then later mentions \"outside of the author's submission of 21 fortresses.\" A referee should ask for a clean breakdown. Third, there's a minor grid-size mismatch between \"16 x 8\" in the engine section and \"14 by 6\" in the fortress editor description—cosmetic, but sloppy.\n\nWho's it for: people building game-AI platforms, researchers studying mixed-initiative design, and anyone wanting a lightweight FSM multi-agent testbed. It's not a paper with a strong empirical result; it's a systems paper with an honest pilot.\n\nRecommendation: send it to peer review, but reviewers should ask for corrected counts, a more careful framing of the database claim, and ideally a plan for how diversity will be measured once more fortresses arrive. The infrastructure is real and the authors are candid about its current limits; that's deserving of referee time.","headline":"Genuine browser-based FSM environment platform with honest pilot data, but the 'diverse database' claim is ahead of the evidence.","tokens_in":9667,"tokens_out":2274,"would_cite":false,"duration_ms":20348,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Amorphous Fortress Online is a browser platform where users design, play, and share small multi-agent worlds built from simple finite-state machines, building toward a community database of open-ended AI environments.","keywords":["multi-agent systems","finite-state machines","open-endedness","user-generated content","collaborative design","artificial life","web platform","emergent behavior"],"falsifier":"Track all fortress submissions for one year after the public release: if the share of non-author fortresses stays below ten percent, or if submitted fortresses reuse essentially the same few entity themes and remix chains never exceed depth four, then the claim that a thematically diverse community database is forming would be refuted.","tokens_in":8809,"feed_emoji":"🧫","tokens_out":7180,"duration_ms":61933,"temperature":0.7,"pith_summary":"Amorphous Fortress Online is a browser-based platform where anyone can design, play, and share small petri-dish-like worlds made of simple finite-state machine agents. The paper argues that such a community platform can grow into a database of thematically diverse AI and game environments, with emergent behavior arising from transparent FSM logic that users can inspect while the simulation runs. If the platform gains a user base, it would give AI researchers a shared, remixable collection of small open-ended environments for training agents and generative models. A short study of the first release shows modest uptake: 33 fortresses total, only five from non-authors, alongside 174 plays.","feed_headline":"Build, play, and remix tiny FSM-driven AI worlds in the browser","feed_subtitle":"The web platform lets anyone design, play, and share petri-dish simulations of finite-state-machine agents.","key_machinery":"The load-bearing mechanism is the entity FSM: every entity class is an ASCII character plus a finite-state machine made of action nodes and condition edges, evaluated each timestep against the fortress grid. Thirteen node types cover autonomous behaviors (move, chase, take, add, clone, transform, and others) and player-controlled actions, while edges trigger on no condition, a step count, proximity within a distance, adjacency, or contact. This FSM formulation is simple enough for users to design in a drag-and-drop canvas yet expressive enough to reproduce game interactions such as a Bokoblin chasing Link when within five spaces. The engine's termination conditions (extinction, overpopulation, inactivity) bound every simulation and let users observe emergent behavior in a compact setting.","core_discovery":"The central claim is that a carefully designed web interface can turn a simple FSM-based simulation engine into a collaborative repository of open-ended AI environments. Each fortress consists of a 16-by-8 grid of entities whose behaviors are defined by action nodes (idle, move, die, clone, push, take, chase, add, transform, move wall, plus three player-controlled variants) and condition edges (none, step, within, nextTo, touch). Users can write these definitions through drag-and-drop graph editing, a fortress placement screen, or a text editor with a validating compiler, then play the simulation and see the active FSM states highlighted in real time via the X-ray feature. The authors present this working platform as the foundation for future work in training reinforcement learning agents and generative models on the submitted fortress data.","pith_inferences":["A natural next test is whether remix chains and backpack reuse predict the emergence of genuinely new behaviors; the platform's submission logs would allow that analysis without new infrastructure.","The gap between 174 plays and five non-author submissions suggests that a mixed-initiative generator that proposes a starting fortress may matter more than editor polish for growing the database.","Because each fortress is a compact, parseable artifact, the corpus could double as an open-endedness benchmark: one could measure how often submitted entities use new combinations of node types or produce long extinction-free runs.","The planned sprite-generation step implies that, if adoption grows, the site also becomes a continuous source of labeled small images paired with gameplay data, extending its value beyond the game AI community."],"forward_implications":["Entities saved to a backpack can be dropped into a new fortress, with dangling character references randomly replaced, which supports cross-environment remixing and long design lineages.","The X-ray feature shows active nodes and recently used edges during play, making agent behavior inspectable and helping designers debug before submission.","The database of fortress definitions — layouts, entity FSMs, and submitted notes — is intended to serve as training data for reinforcement learning agents, recommendation systems, and generative models.","The node-frequency data reported (298 nodes across 143 entities) gives an early quantitative picture of which behaviors designers choose, a pattern the platform could track over time.","Remix depth, already reaching four in the short study, provides a measurable indicator of community reuse that future releases can grow."],"supporting_citations":[{"why":"Supplies the original Amorphous Fortress Python engine and the base set of FSM action nodes that the online engine ports and extends.","marker":"[14]"},{"why":"Contributes the quality-diversity evolutionary method from prior iterations, planned to power the mixed-initiative evolver and entity recommendation.","marker":"[15]"},{"why":"Provides the model of a collaborative online platform where users guide and share generated content, which shapes the site's sharing database.","marker":"[5]"},{"why":"Documents ZZT, a community-driven game engine whose user-created games across genres motivate the open-ended design goal.","marker":"[8]"},{"why":"Is the textbook reference grounding the choice of finite-state machines for game AI and their explainability.","marker":"[16]"},{"why":"Provides the Zelda: Tears of the Kingdom scenarios used to demonstrate the engine's ability to emulate game-like interactions.","marker":"[17]"},{"why":"Gives the collaborative mixed-initiative level design system whose feedback loop is adapted for planned fortress evolution.","marker":"[6]"}],"fun_headline_variants":["Collaboratively craft AI life in a web petri dish","Build tiny FSM AI worlds and share them online","Open-ended AI environments designed by the crowd","Petri-dish AI: design, play, and remix in-browser","FSM agents, collaborative worlds: Amorphous Fortress"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The platform's value as a diverse environment database depends on outside users actually submitting many varied fortresses; the paper's own study found only five non-author fortresses among 33, so this adoption premise is not yet supported.","fun_headline_variants_meta":{"raw":{"variants":["Collaboratively craft AI life in a web petri dish","Build tiny FSM AI worlds and share them online","Open-ended AI environments designed by the crowd","Petri-dish AI: design, play, and remix in-browser","FSM agents, collaborative worlds: Amorphous Fortress"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000555,"raw_usage":{"total_tokens":2567,"prompt_tokens":793,"completion_tokens":1774,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":409,"completion_tokens_details":{"reasoning_tokens":1692}},"tokens_in":409,"tokens_out":1774,"duration_ms":14078,"temperature":1.0,"reasoning_tokens":1692,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:34:23.692228+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Track all fortress submissions for one year after the public release: if the share of non-author fortresses stays below ten percent, or if submitted fortresses reuse essentially the same few entity themes and remix chains never exceed depth four, then the claim that a thematically diverse community database is forming would be refuted.","supporting_citations":[{"cited_title":"Amorphous fortress: Observing emergent behavior in multi-agent fsms,","cited_arxiv_id":null,"evidence_quote":"Supplies the original Amorphous Fortress Python engine and the base set of FSM action nodes that the online engine ports and extends."},{"cited_title":"Quality diversity in the amorphous fortress: Evolving for complexity in 0-player games,","cited_arxiv_id":null,"evidence_quote":"Contributes the quality-diversity evolutionary method from prior iterations, planned to power the mixed-initiative evolver and entity recommendation."},{"cited_title":"Picbreeder: evolving pictures collaboratively online,","cited_arxiv_id":null,"evidence_quote":"Provides the model of a collaborative online platform where users guide and share generated content, which shapes the site's sharing database."},{"cited_title":"Anthropy, ZZT: Boss Fight Books# 3, vol","cited_arxiv_id":null,"evidence_quote":"Documents ZZT, a community-driven game engine whose user-created games across genres motivate the open-ended design goal."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the textbook reference grounding the choice of finite-state machines for game AI and their explainability."},{"cited_title":"The legend of zelda: Tears of the kingdom","cited_arxiv_id":null,"evidence_quote":"Provides the Zelda: Tears of the Kingdom scenarios used to demonstrate the engine's ability to emulate game-like interactions."},{"cited_title":"Baba is y’all: Collaborative mixed-initiative level design,","cited_arxiv_id":null,"evidence_quote":"Gives the collaborative mixed-initiative level design system whose feedback loop is adapted for planned fortress evolution."}],"review_version":1}