{"id":"4fad8d8f-8a7f-4795-9a10-5375f5e11157","arxiv_id":"2608.12125","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Showing LLM agents a graded similarity score strongly increases cooperation in social dilemmas, and a new equilibrium concept makes high-similarity cooperation stable.","lead":"LLM agents who are told they are similar to their opponent cooperate far more in one-shot games like the Prisoner's Dilemma, with most models switching from full defection to full cooperation as the similarity score rises. A new behavioral equilibrium model explains this as rational when each agent believes the other will mirror its deviations with probability equal to the similarity score.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The similarity prompt does not fix the probabilistic mirroring reading assumed by Definition 1; the paper's own CoT excerpts show causal, acausal-mirroring, and opposite-action readings, so the theory-to-experiment link for the central claim is not established.","rationale":"The reader's weakest_assumption is exactly the concern that lands. I read the paper in good faith: the empirical pattern is credible and the formal theory is internally consistent; Appendix E proofs (Lemma 2, Proposition 3, Theorem 1, existence results) appear correct, and the random-benchmark control is a thoughtful design. However, the central claim as stated is about a cooperation mechanism—similarity signals should support mutually beneficial outcomes because similar agents can anticipate each other's deviations. That mechanism requires the signal to mean the same thing to each model. The paper's own Appendix B shows the authors iterated on prompt wording specifically to avoid multiple interpretations, and Appendix D shows the final wording still produces incompatible readings. This is not an outside-consensus disagreement; it is an internal mismatch between the operationalization and the theoretical model. The consequence is that the reported cooperation rates mix at least three decision rules. The theoretical welfare bound (Theorem 1) then does not explain the data; it only explains what would happen under one of the readings. I therefore do not call for rejection: the empirical direction may survive a prompt rewrite, and the paper already discloses some of these issues. But the manuscript should remain CONDITIONAL on adding either an explicit-mirroring prompt condition or a comprehension/classification check that establishes the interpretation distribution. Statistical issues (n=10 per condition, no significance tests, unreleased raw data) are secondary because they are fixable without changing the conceptual claim; the interpretation gap is load-bearing for the mechanism.","tokens_in":37977,"tokens_out":6729,"duration_ms":60921,"concrete_test":"Run a prompt-variant control on RQ1: replace the default similarity sentence with an explicit mirroring rule—'If you choose an action, the other agent will choose the same action with probability X%, and otherwise will choose the action that maximizes its own points independently'—and compare cooperation rates across the same 9 models and X levels. If the curves differ materially (e.g., by more than one similarity step at the transition, or models that were monotonic become non-monotonic), the default prompt was not inducing the Definition 1 interpretation, and the paper's theoretical explanation of the measured cooperation fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—higher similarity scores induce cooperation—is interpreted through the b-similarity equilibrium of Section 3, where b_ij is the probability that co-player j deviates in the same way as i (Definition 1). For Theorem 1 to explain the experimental results, the sentence 'the other agent's decision-making is X% similar to yours' (Prompt 1, Appendix J.4) must make each LLM adopt that exact counterfactual. The paper's own evidence shows it does not. Appendix B reports that earlier prompts were abandoned for allowing multiple readings, but the final wording still leaves the key inference implicit: 'how similar you and the other agent reason and come to conclusions' does not say that a contemplated action by i is mirrored with probability X. Appendix D contains CoT traces in which the same model alternates between causal readings ('A1 is strictly dominant ... regardless of similarity'—GPT at s=100%, Claude at s=80%) and acausal EV readings ('40% chance opponent copies A0 ...'—Qwen, Gemma, Gemini). RQ3 also notes that some models assume the co-player plays the opposite action. Aggregating cooperation rates over these heterogeneous rules conflates distinct mechanisms. The random-noise benchmarks in RQ4 compound the problem: Gemini and Claude cooperate at high similarity when the score is generated by coin flips or die rolls, so part of the measured effect is a response to the label 'similarity' rather than to evidence about co-player behavior. The formal results in Appendix E appear sound, but they are connected to the experiments only if the prompt operationalizes the b-mirroring rule; absent that, the headline mechanism is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether graded similarity signals can induce cooperation in LLM agents. It reports experiments across nine LLMs in several symmetric mixed-motive games, using both abstract similarity percentages and scores grounded in benchmark responses, and it introduces a formal 'b-similarity equilibrium' concept. The central empirical claim is that higher similarity scores usually induce more cooperative behavior, with large variation across models; the paper also finds that the benchmark domain used to compute similarity has little effect, that LLMs rate themselves as highly similar to co-players, and that some models cooperate even under random-noise similarity signals. The theoretical contribution is Theorem 1, which bounds the welfare loss of any b-similarity equilibrium in terms of the product of similarity scores.","tokens_in":38270,"tokens_out":2039,"duration_ms":20242,"significance":"If the central claim is correct, similarity signaling could be a useful mechanism for promoting cooperation among LLM agents, with direct relevance to the emerging literature on multi-agent AI. The formal framework is a clean interpolation between Nash equilibrium and evidential/superrational behavior, and the proof of Theorem 1 is short and plausible. The paper also ships an open-source evaluation framework, which is a valuable community resource. However, the empirical-to-theoretical link is not yet established: the natural-language similarity prompt does not fix the probabilistic mirroring interpretation assumed by Definition 1, and the experimental results are reported with very small samples and without significance tests for several key comparisons. The qualitative pattern is interesting and worth further study, but it currently supports a weaker claim than the paper's stated conclusion.","major_comments":[{"comment":"The connection between the experiments and the b-similarity equilibrium is load-bearing but not established. Definition 1 requires that a similarity score b_ij is the probability that player j deviates in the same way as player i, yet Prompt 1 (Appendix J.4) says only that the other agent's decision-making is X% similar 'in reasoning and conclusions.' The paper's own CoT excerpts in Appendix D show that models read this in incompatible ways: some use a causal reading ('A1 is strictly dominant regardless of similarity'), some use an acausal expected-value calculation with a mirroring probability, and some assume the co-player plays the opposite action. Aggregating cooperation rates over these heterogeneous decision rules conflates distinct mechanisms, so the measured rates cannot be attributed to a single well-defined similarity signal. The authors should either constrain the prompt to explicitly define a probabilistic mirroring rule, or analyze cooperation rates separately for each identified reading, or revise the central claim to be about similarity labels rather than b-similarity equilibrium.","section":"§2, RQ1, and Appendix D"},{"comment":"The claim that the benchmark domain has 'small to no impact' on induced cooperation is not supported by statistical evidence. Each condition uses only 10 samples per model, and Figure 3/Figure 12 are presented without significance tests or confidence intervals. Table 2 aggregates payoffs across models without error bars, which makes it impossible to assess whether the ranking of benchmarks is meaningful. The authors should report per-condition confidence intervals and perform hypothesis tests (or equivalence tests) for the null claim that domain does not matter.","section":"RQ4 and Table 2"},{"comment":"The finding that Gemini and Claude cooperate at high similarity when the similarity signal is based on random die rolls or coin tosses is a serious alternative explanation for the central result. If models respond to the label 'similarity' rather than to evidence about co-player behavior, then the mechanism is not the b-similarity equilibrium but a form of persuasive framing. The paper acknowledges this as a 'trustworthiness problem,' but the abstract and introduction still state the stronger conclusion that similarity signals induce cooperation. The authors should explicitly temper the central claim to distinguish between evidence-responsive and label-responsive behavior, and discuss the implications for deploying similarity signaling as a cooperation mechanism.","section":"§4.1, random-noise benchmarks"},{"comment":"Theorem 1 is a bound on any b-similarity equilibrium that exists, but the paper itself shows in Appendix E.3 that b-similarity equilibria need not exist for intermediate b, including in the Traveler's Dilemma. This is not an error, but it limits the interpretive power of the theory for the empirical sweeps, where models are asked to act at every similarity level. The paper should state more clearly that the formal model predicts behavior only in the ranges where equilibria are guaranteed to exist, and that the empirical data may include out-of-equilibrium behavior in the existence gap.","section":"§3, Theorem 1 and existence caveat"}],"minor_comments":[{"comment":"The appendix reports that earlier prompts were abandoned because they admitted multiple incompatible readings, but the final wording still does not pin down the probabilistic mirroring rule. A short validation study, even with a few human or LLM judges, would strengthen the claim that the final prompt yields a single dominant interpretation.","section":"Appendix B"},{"comment":"The text states that all experiments report 'the mean and standard error,' but Figure 2 does not show error bars. Adding error bars or shaded bands would help the reader assess the stability of the monotonic trends.","section":"Figure 2"},{"comment":"The row label 'Trait' is inconsistent with the benchmark name 'TRAIT' used elsewhere; consider using the same capitalization throughout.","section":"Table 2"},{"comment":"The paper alternates between 'Prisoners' and 'Prisoner's Dilemma' without a consistent abbreviation. Standardizing this would improve readability.","section":"§1 and §5"},{"comment":"The comparison to the CoopEval leaderboard is described as 'rough,' but it is not clear how the different model sets and sample sizes affect the comparison. A sentence explaining the limitations of this comparison would be useful.","section":"§4.2, RQ6"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and addresses an important question. The main concern is not the formal theory, which appears sound, but the gap between the formal model and the experimental measurement. I recommend major revision with a clear request to either fix the prompt to match Definition 1, or soften the theoretical interpretation of the empirical results. The small sample sizes and missing significance tests are also important but fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is worth your time for two reasons: the b-similarity equilibrium is a clean formal interpolation between Nash and evidential/Kantian reasoning, with proofs that look careful, and the graded similarity sweep across 9 LLMs is genuinely new. The empirical pattern—cooperation mostly ramps up as the advertised similarity goes from 0 to 100%—is credible and likely to be cited.\n\nThe formal side is in good shape. Definition 1 is clear, Theorem 1's welfare bound is sensible, and the existence results (including the non-existence example and the Traveler's threshold) are real contributions. The authors also did a good thing by running random-noise controls; those reveal that some models are reacting to the label 'similarity' rather than evidence about the co-player, which is a useful caution.\n\nThe soft spot is the bridge between the theory and the experiments. The b-similarity equilibrium assumes the agent interprets the signal as: if I deviate, my co-player deviates the same way with probability b. Prompt 1 (Appendix J.4) doesn't say that. It says decision-making is X% similar, meaning how similarly you reason and come to conclusions. That leaves the probabilistic mirroring assumption implicit. The paper's own CoT excerpts in Appendix D show models reading it causally, acausally, and some even assuming the co-player plays the opposite action. So the measured cooperation rates mix several distinct decision rules, and the headline mechanism—that the b-similarity equilibrium explains the results—is not supported by the data. The authors are aware of some of this; they report abandoning earlier, vaguer prompts, but the final wording still doesn't fix the key inference.\n\nThe statistics are also thin: 10 samples per condition, no significance tests for the benchmark-irrelevance null claim, aggregate payoff tables without error bars, and raw data not released. The LLM judge is unvalidated. These are fixable rather than fatal.\n\nBottom line: this is a serious paper with a valuable formal concept and a broad empirical map. But the central claim that similarity signals induce cooperation through something like b-similarity reasoning is only loosely connected to the experiments. The right referee would ask for a prompt that explicitly states the mirroring rule (or separates interpretations), more samples, and error bars. I'd send it to review rather than desk reject.","headline":"Solid new equilibrium concept and broad empirical sweep, but the prompt doesn't pin down the mirroring rule the theory needs, so the headline mechanism is not yet established.","tokens_in":38827,"tokens_out":2251,"would_cite":true,"duration_ms":20275,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91A10","91A80"],"pacs":[],"model":"deepseek-v4-flash","headline":"Graded similarity signals can induce cooperation in LLM agents, though the effect varies sharply by model.","keywords":["similarity signaling","cooperation","large language models","Prisoner's Dilemma","evidential decision theory","b-similarity equilibrium","social dilemmas","multi-agent systems"],"falsifier":"Run the same Prisoner's Dilemma experiment with the similarity score explicitly defined in two alternative ways: 'the chance that the other agent copies your exact action' versus 'how much your general reasoning styles overlap'; if cooperation thresholds shift with the definition, then the reported score is not being interpreted as one fixed signal and the observed behavior cannot be attributed to a single similarity mechanism. A sharper test is to measure each model's true mirroring rate from a pretest and then feed it an accurate or an inverted similarity score; if cooperation follows the label rather than the true mirroring rate, the mechanism is label compliance, not evidential reasoning.","tokens_in":37794,"feed_emoji":"🤝","tokens_out":10721,"duration_ms":90171,"temperature":0.7,"pith_summary":"This paper asks whether telling one AI agent that its decision-making is X percent similar to its co-player's can induce cooperation in one-shot social dilemmas. Across nine large language models and five mixed-motive games, the authors find that cooperation rates usually rise as the reported similarity score rises, although the effect varies drastically from model to model and is sensitive to prompt framing. The paper formalizes the observed behavior as a $b$-similarity equilibrium, in which a player evaluates a deviation from the common strategy assuming each co-player makes the same deviation with probability equal to the similarity score. Its central theorem shows that as similarity approaches 100 percent, every such equilibrium is within a payoff-range error term $R_i(1-P)$ of the best symmetric outcome, where $P$ is the product of the similarities; cooperative welfare is therefore approximately recovered at the limit. If correct, the result makes similarity signaling a practical, information-only cooperation mechanism for AI agents, with no need for contracts, repetition, or third-party mediation.","feed_headline":"A similarity score nudges most LLMs to cooperate","feed_subtitle":"Across nine models and five dilemmas, higher similarity raises cooperation; an equilibrium model explains why.","key_machinery":"The central object is the $b$-similarity equilibrium, a solution concept in which a player considering a deviation from the symmetric profile assumes that each other player deviates in the same way with probability equal to the known similarity score, and otherwise stays at the original profile. The concept is defined so that $b=0$ reproduces the standard unilateral-deviation equilibrium and $b=1$ requires the globally best symmetric profile, making it a continuous interpolation between independent-decision reasoning and exact-copy reasoning. The proof of the high-similarity welfare bound treats a deviation as a random coalition of co-deviators: with probability equal to the product of the similarities, all co-players join the deviation, and on every other outcome the loss is bounded by the payoff range. This machine carries the paper's central argument by connecting the observed LLM behavior to a rigorous equilibrium statement: higher reported similarity shrinks the welfare gap, and below a threshold the cooperative profile is not stable.","core_discovery":"The paper's central claim is that large language models often treat a reported similarity score as evidence that their co-player will make the same choice they make, and that this evidential connection is enough to move many models from defection to cooperation even in the one-shot Prisoner's Dilemma, where defection is strictly dominant. The authors demonstrate the effect with a graded-similarity sweep across nine models, finding monotonic rises to full cooperation in most models, with abrupt switches typically between 60 and 80 percent similarity, alongside two notable exceptions: one model defects at every reported similarity, and one shows a non-monotonic peak near 80 percent. They then encode the apparent reasoning as a formal solution concept: a symmetric profile is a $b$-similarity equilibrium if no player benefits by deviating when every co-player joins the deviation with probability $b$. For any such equilibrium, each player's payoff is at least the best symmetric payoff minus $R_i(1-P)$, where $R_i$ is the player's payoff range and $P$ is the product of the similarity scores to all co-players; the cooperative outcome becomes the unique equilibrium in the Prisoner's Dilemma once $b>1/2$. Experiments that ground the score in ten different benchmarks (moral dilemmas, personality tests, expert questions, random coin flips, and others) find that the grounding domain has little effect on cooperation, and models asked to judge a co-player's reasoning tend to rate themselves highly similar; some models even cooperate under scores derived from pure random noise.","pith_inferences":["If models react to the label 'similar' more than to the evidence behind it, an adversarial party could inflate similarity scores to steer deployed agents toward cooperation on unfavorable terms; the paper's random-noise results already hint that some frontier models do not check the grounding.","Because models judging a co-player's reasoning self-report high similarity even when their actual answers diverge, ecosystems with shared model families or overlapping training data may spontaneously converge on mutual cooperation, with collusion and correlated-failure risks accompanying the cooperative upside.","A direct testable extension is to calibrate the reported score to the co-player's empirically measured mirroring probability and compare cooperation thresholds to the model's predicted value $b^*$; the main theory is stated for arbitrary reported similarity, and its quantitative predictions are not yet tested with calibrated scores.","Similarity signaling could be made dynamic, updating the score from observed choices in repeated play, which would connect this one-shot result to learning agents and to the broader family of experience-based cooperation mechanisms."],"forward_implications":["In the standard Prisoner's Dilemma, the model predicts an abrupt switch from defection to full cooperation once the reported similarity passes $1/2$, and several tested models show exactly this sharp threshold.","With sufficiently high similarity, the welfare-maximizing outcome becomes the unique $b$-similarity equilibrium, so mutual cooperation among LLM agents is an equilibrium outcome, not a fragile or accidental behavior.","The same mechanism extends to public-goods games: with three players and the paper's payoff parameters, full contribution becomes the unique equilibrium once similarity to each co-player exceeds $1/2$, while many-player settings are correspondingly harder.","Grounding the similarity score in real benchmark responses preserves the effect: moral-reasoning and personality benchmarks recover roughly 72 percent of optimal welfare in pairwise matchups, putting similarity signaling among the more effective tested cooperation mechanisms.","Framing the signal as difference instead of similarity, or increasing the number of co-players, predictably lowers cooperation, giving system designers a tunable dial rather than a binary effect."],"supporting_citations":[{"why":"Provides the theoretical starting point: learned policies can cooperate under a credible difference signal, but its construction is taken as given; this paper operationalizes it for LLMs.","marker":"Oesterheld et al. 2023"},{"why":"Supplies the baseline single-shot social-dilemma evaluation framework and the leaderboard of alternative cooperation mechanisms against which similarity signaling is ranked.","marker":"Tewolde et al. 2026"},{"why":"Frames cooperation mechanisms for AI agents and motivates the study of agents that share decision-making components, which is the premise of similarity signaling.","marker":"Conitzer and Oesterheld 2023"},{"why":"Foundational statement of evidential decision theory, the reasoning pattern the paper identifies in cooperative LLM chain-of-thought traces.","marker":"Jeffrey 1965"},{"why":"Defines Kantian equilibrium, which is the exact-copy endpoint of the paper's solution concept and the formal anchor for cooperation among identical agents.","marker":"Roemer 2010"},{"why":"Provides the LLM-as-a-judge framework used to classify chain-of-thought justifications across seventeen categories in the paper's large-scale reasoning analysis.","marker":"Guzman Piedrahita et al. 2025"}],"fun_headline_variants":["Similarity score flips most LLMs to cooperate in dilemmas","Telling LLMs they're similar nudges them toward cooperation","Similarity signal turns LLM defection into cooperation","For most LLMs, a similarity score triggers cooperation","Similarity nudge makes LLMs cooperate even in one-shot dilemmas"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume that the wording telling one agent the other agent's decision-making is X percent similar to theirs is read by each model as the probabilistic mirroring rule, meaning the co-player will make the same deviation with probability X, but the paper's own chain-of-thought excerpts show that models sometimes read the signal causally, acausally, or as evidence of opposite play, so the measured cooperation mixes several distinct decision rules.","fun_headline_variants_meta":{"raw":{"variants":["Similarity score flips most LLMs to cooperate in dilemmas","Telling LLMs they're similar nudges them toward cooperation","Similarity signal turns LLM defection into cooperation","For most LLMs, a similarity score triggers cooperation","Similarity nudge makes LLMs cooperate even in one-shot dilemmas"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000559,"raw_usage":{"total_tokens":2712,"prompt_tokens":1056,"completion_tokens":1656,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":672,"completion_tokens_details":{"reasoning_tokens":1573}},"tokens_in":672,"tokens_out":1656,"duration_ms":11151,"temperature":1.0,"reasoning_tokens":1573,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:14:41.722595+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same Prisoner's Dilemma experiment with the similarity score explicitly defined in two alternative ways: 'the chance that the other agent copies your exact action' versus 'how much your general reasoning styles overlap'; if cooperation thresholds shift with the definition, then the reported score is not being interpreted as one fixed signal and the observed behavior cannot be attributed to a single similarity mechanism. A sharper test is to measure each model's true mirroring rate from a pretest and then feed it an accurate or an inverted similarity score; if cooperation follows the label rather than the true mirroring rate, the mechanism is label compliance, not evidential reasoning.","supporting_citations":[],"review_version":1}