{"id":"be31babc-16fa-4a9c-9814-e57a946d702b","arxiv_id":"2412.12409","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Bayesian inference over a set of language models and cognitive hierarchy levels improves Codenames spymaster performance with out-of-distribution teammates, while Bayesian guessers remain mostly worse than static baselines.","lead":"A new Codenames AI reasons about its teammate by holding a probability distribution over possible language models and strategy levels, updating it with Bayes' rule as the game progresses. The Bayesian spymaster beats the best static agents when playing with teammates whose language models it has never seen, a long-standing failure mode in Codenames AI.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No ablation isolates Bayesian posterior updating from noise-robust clue selection; Table 1 gains may stem from ensemble/hedging, not inference.","rationale":"The reader's weakest assumption concerned the fidelity of the Gaussian noise model for cross-agent semantic variation. That is a legitimate concern, but the more decisive issue is attribution: Table 1 compares S-hat-MB to the best static spymaster, not to an equally expressive non-adaptive ensemble. Without a no-update control, the headline result conflates the effect of Bayesian updating with the effect of noise-robust clue selection and expected-utility hedging. If the no-update control performs as well, the paper's claimed novelty ('first example of an agent having a probability distribution over a set of language models, updating this distribution... and adapting') is not established. The manuscript also contains algorithmic artifacts (e.g., 'dosomethings' in Algorithm 2, 'Require: What is the input to this function?' in Algorithm 14) and lacks error bars, but those are secondary. The concern is addressable with a straightforward ablation, so CONDITIONAL rather than REJECT is appropriate. I partially agree with the reader: the Gaussian noise assumption is also untested, but the ablation control is the single check that would most directly settle whether the central claim holds.","tokens_in":16034,"tokens_out":4680,"duration_ms":47465,"concrete_test":"Add a no-update control: implement S-hat-MB exactly as described (noise sigma=1.0, 10 samples per clue, same clue-filter optimization, same four internal embeddings, same heuristic utility) but skip Algorithm 11 so that P(g) stays uniform for the entire game. Evaluate against the same guesser set in both deterministic and stochastic environments with at least 500 games, and compare win rates to S-hat-MB using a bootstrap or chi-square test. Also run the analogous no-update control for SMB (sigma=0). If the no-update control matches S-hat-MB within sampling error, the improvement over 'Best Model' is not due to Bayesian inference; if S-hat-MB is significantly better, the adaptation claim is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that Bayesian inference over a set of language models lets a spymaster adapt to an unknown guesser and outperform static baselines, particularly out-of-distribution (Table 1). The most load-bearing condition is that the posterior updates are causally responsible for the reported gains. But S-hat-MB differs from the 'Best Model' baseline in two confounded ways: it maintains an updated posterior, and it also uses Gaussian-perturbed clue embeddings (sigma=1.0) plus expected-utility maximization across the internal model set (Algorithms 12–15). A static spymaster that uses the same noise and a fixed uniform posterior (a non-adaptive ensemble) is not reported. The observed improvement could therefore come entirely from robust clue selection under embedding noise or from the heuristic utility function, with Bayesian updating contributing little or nothing. The Discussion attributes the gain to learning ('rapidly learns which of its model guessers is the best fit'), but no belief-convergence analysis or ablation is provided to support that attribution. Additionally, the written likelihood update in Algorithm 11 multiplies by an unnormalized pseudo-count from the current turn's sampling, which would make the posterior sensitive to the number of Monte Carlo samples; this needs clarification before the Bayesian mechanism can be evaluated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Bayesian agents for the cooperative language game Codenames. The agents maintain a probability distribution over teammate models that combine semantic uncertainty (a finite set of word embeddings) with pragmatic uncertainty (levels of a deductive/cognitive hierarchy), and add Gaussian noise to embeddings to model fine-grained uncertainty. The Bayesian spymaster updates beliefs from observed guesses and selects clues by maximizing expected heuristic utility; the Bayesian guesser updates beliefs from observed clues and uses thresholds to decide guesses. Experiments compare these agents against static baselines across in-distribution and out-of-distribution word embeddings, in deterministic and stochastic environments. The main positive result is that the noise-assuming Bayesian spymaster (S-hat-MB) improves win rates over the best static spymaster, especially out-of-distribution (Table 1: 0.872 vs 0.718 deterministic; 0.625 vs 0.468 stochastic). The Bayesian guessers, however, mostly underperform the best static guesser (Table 2).","tokens_in":16383,"tokens_out":4087,"duration_ms":39991,"significance":"If the spymaster improvement is robust and can be attributed to Bayesian inference, the paper would be a useful contribution to ad hoc teamwork in language games, demonstrating a principled way to combine semantic and pragmatic uncertainty and adapt to an unknown teammate. The paper's strengths include the explicit construction of a model set over embeddings and hierarchy levels, evaluation with external out-of-distribution embeddings, 500 games per pairing, and released source code. The significance is currently conditional: the central claim lacks an ablation that isolates posterior updating from the other mechanisms that differ between the Bayesian and baseline spymasters, and the guesser results contradict any broad claim that the Bayesian agents generally perform better. With the missing attribution and statistical detail supplied, the framework would be a solid contribution to the Codenames AI literature.","major_comments":[{"comment":"The central claim that Bayesian inference improves cooperation is not supported by an ablation that isolates Bayesian posterior updating from the other differences between S-hat-MB and the Best Model baseline. S-hat-MB differs from the baseline in at least four ways: it maintains and updates a posterior, it uses Gaussian-perturbed clue embeddings, it aggregates expected utility across the internal model set, and it uses the heuristic utility function with assassin avoidance. The Discussion attributes the gain to the spymaster 'rapidly learning which of its model guessers is the best fit,' but a static spymaster using the same noise and a fixed uniform posterior (a non-adaptive ensemble) is not reported. Without this control, the observed improvement could come entirely from noise-robust clue selection or from the heuristic, with Bayesian learning contributing little or nothing. Please add an ablation with a uniform-posterior, same-noise spymaster, and ideally a version that updates beliefs but uses noiseless clue selection.","section":"Section 6, Table 1 and Section 7"},{"comment":"The belief update in Algorithm 11 multiplies the previous posterior by h_t(g,l,n), an unnormalized pseudo-count that grows with the number of Monte Carlo samples s used during clue selection. As written, the posterior depends on the arbitrary sampling count: with s=10, the likelihood factor is roughly an integer between 1 and 10, and multiplying by this factor each turn can inflate the belief values without normalization. The appendix says the pseudo-count is used as the likelihood 'since it does not need to be normalized,' but a product of unnormalized counts is not a likelihood and its magnitude is not invariant to s. Please specify the normalized likelihood (or the Dirichlet posterior mean) actually used in the implementation, or demonstrate that the update is invariant to s in the reported experiments.","section":"Appendix B, Algorithm 11 and 'Estimating Likelihood'"},{"comment":"The broad claim that the Bayesian agents 'perform better' in experiments is contradicted by the guesser results. In Table 2, all Bayesian guessers have lower average win rates than the Best Model for out-of-distribution spymasters, and in the stochastic in-distribution environment only DGB-hat marginally exceeds the baseline (0.733 vs 0.708). The Discussion acknowledges this, but the abstract and conclusion still present a general improvement claim. The paper should either restrict the contribution to the Bayesian spymaster or provide a clear explanation and supporting evidence for why the guesser results do not undermine the claim that Bayesian inference improves cooperation.","section":"Abstract, Section 1, and Section 6, Table 2"},{"comment":"No error bars, confidence intervals, or significance tests are reported despite 500 games per pairing. Some comparisons in Table 1 are close (e.g., SMB vs Best Model in the stochastic in-distribution column, 0.370 vs 0.370, and 0.477 vs 0.468 out-of-distribution), so it is impossible to tell which differences are reliable. Please report standard errors or confidence intervals for the win rates, and ideally paired or per-board variance information, especially for the headline out-of-distribution comparisons.","section":"Section 6, 'Experimental Results'"},{"comment":"The likelihood model assumes isotropic Gaussian perturbation of word embeddings with a fixed variance (sigma=1.0 in the experiments), but no evidence is provided that inter-agent semantic variation is well described by this noise model. A misspecified likelihood can cause the posterior to concentrate on the wrong teammate model. Since the Bayesian mechanism is the paper's contribution, please add a sensitivity analysis over sigma and, if feasible, a comparison with a heavier-tailed noise distribution, to show that the reported gains are not an artifact of this specific Gaussian assumption.","section":"Section 3 and Section 5"}],"minor_comments":[{"comment":"The entry '604' in the stochastic in-distribution column for BGB appears to be missing a leading decimal point and should likely be 0.604.","section":"Table 2"},{"comment":"Algorithm 14 contains the placeholder line 'Require: What is the input to this function?' and does not specify its inputs; please complete the pseudocode.","section":"Appendix B, Algorithm 14"},{"comment":"Algorithm 2 contains the placeholder line 'dosomethings', and Algorithm 6 has typos such as 'vecor' and an unclosed parenthesis in 'order.insert(c,i'; these should be fixed for reproducibility.","section":"Appendix A, Algorithms 2 and 6"},{"comment":"The likelihood integral in Section 5 uses the notation N(m_t(w), sigma, x) without defining whether sigma is a scalar, vector, or covariance matrix, and without specifying the normalization; please clarify the notation.","section":"Section 5"},{"comment":"The claim that this is 'the first example of an agent having a probability distribution over a set of language models' is qualified as 'to our knowledge,' but the paper would be stronger if it briefly stated the scope of the literature search or cited related work on probabilistic mixtures of embedding spaces, so readers can evaluate the novelty claim.","section":"Section 1 and Section 2"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal and the source code is a clear asset. The main issue is attribution: the headline result may be due to robust clue selection rather than Bayesian learning, and the guesser results weaken the general claim. These are fixable with an ablation, a corrected likelihood description, and scoped claims, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look. It gives the first Bayesian agent that keeps a posterior over language models and updates it from teammate behavior in Codenames, which is a genuine step beyond the static spymasters and guessers that dominate prior work. The Table 1 spymaster results are the main event: with out-of-distribution guessers, the noise-perturbed Bayesian spymaster (S-hat-MB) beats the best static spymaster by a wide margin in both deterministic (0.872 vs. 0.718) and stochastic (0.625 vs. 0.468) settings. That is a practically meaningful improvement for cross-embedding play, a known failure mode in Codenames AI. Full credit for releasing code and using a broad set of internal and external embeddings.\n\nThe soft spots are real, but they are mostly addressable rather than fatal. The stress-test concern about confounding is the important one: S-hat-MB differs from the best-model baseline by retaining an updating posterior, but it also uses Gaussian-perturbed clue embeddings and expected-utility maximization over the internal model set. There is no ablation with a static uniform-posterior spymaster that uses the same noise. The authors attribute the gain to “rapidly learns which of its model guessers is the best fit,” but the data do not isolate that mechanism. They need either a no-update baseline or a belief-convergence analysis. Similarly, Algorithm 11 multiplies by unnormalized pseudo-counts from the Monte Carlo samples; this is not obviously a well-normalized Bayesian update, and the dependence on sample count should be clarified. The Table 2 guesser results are also weaker than the abstract implies—the Bayesian guessers usually lose to the best static model—and the Discussion says this, but the abstract overstates the case. Minor but real: there are unfinished manuscript artifacts (e.g., “dosomethings” in Algorithm 2, “What is the input to this function?” in Algorithm 14). No error bars or significance tests are reported, though 500 games per pairing should be enough to make them meaningful.\n\nWho gets value from this? Researchers in ad hoc teamwork, language games, and Codenames AI. The central spymaster finding is likely to survive a rewrite; the hard part is stating what exactly the Bayesian inference contributes versus robust clue selection under noise. I would send this to peer review with a clear request for an ablation and cleanup.\n\nFor me: bring it to reading group, yes; cite it when discussing cross-embedding Codenames agents; and treat it as an interesting empirical result whose mechanism needs one more experiment.","headline":"Bayesian spymaster in Codenames shows real out-of-distribution gains, but the paper doesn't isolate whether Bayesian updating or noise-robust clue selection drives them.","tokens_in":16789,"tokens_out":2577,"would_cite":true,"duration_ms":26968,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Bayesian spymaster that tracks probabilities over possible teammate embeddings and hierarchy levels outperforms the best static spymaster against out-of-distribution guessers in Codenames.","keywords":["Codenames","Bayesian inference","cognitive hierarchy","word embeddings","cooperative language games","teammate modeling","ad hoc teamwork","semantics and pragmatics"],"falsifier":"Pair the noisy Bayesian spymaster with a guesser whose embedding is a deterministic rotation or translation of one of the internal embeddings, a clearly non-Gaussian semantic shift. If the Bayesian spymaster does not beat the best static spymaster in this setting, the claim that it adapts to any unseen semantics fails. A complementary check is to collect the actual distribution of inter-agent guess differences and test whether they are Gaussian with variance near 1.0.","tokens_in":15822,"feed_emoji":"🎯","tokens_out":8323,"duration_ms":68121,"temperature":0.7,"pith_summary":"This paper argues that an agent in a cooperative language game can adapt to an unknown partner by keeping a probability distribution over possible partner models, updating it with Bayes' rule after each observed message, and choosing actions that maximize expected utility under that posterior. The authors build such Bayesian agents for Codenames, using a small set of word embeddings for semantic uncertainty and a deductive/cognitive hierarchy for pragmatic uncertainty, with Gaussian noise added to embeddings so every observation stays possible. The central result is that the Bayesian spymaster that assumes noise outperforms the best static spymaster against out-of-distribution guessers, raising average win rate from 0.718 to 0.872 in the deterministic environment and from 0.468 to 0.625 in the stochastic one. This matters because previous Codenames AI required knowing the partner's embedding in advance; a single adaptive agent can now play well with unseen partners.","feed_headline":"Bayesian spymaster beats best static partner on unseen guessers","feed_subtitle":"Tracking probabilities over teammate models beats any fixed embedding in Codenames, even with unknown partners.","key_machinery":"The engine is a posterior distribution over partner models, where each model is a pair of a word embedding and a cognitive-hierarchy level. The likelihood of an observed guess or clue is evaluated by Monte Carlo sampling: the agent perturbs the relevant word's embedding with isotropic Gaussian noise (sigma = 1.0 for the noisy agents), simulates the partner's deterministic strategy, and uses the frequency of the observed action as its probability. The agent then chooses the clue or guess that maximizes the expected value of a turn-level heuristic utility function. This turns a discrete set of language models into a smoothly adaptable policy.","core_discovery":"The central claim is that a Bayesian spymaster can learn which of a set of candidate guessers it is playing with, and that this inference, combined with hedging under embedding noise, yields better cooperation than any fixed-embedding spymaster when the partner is out of distribution. Concretely, the noisy Bayesian spymaster (S-hat-MB) achieves average win rates of 0.872 against guessers built from embeddings not in its model set, versus 0.718 for the best static spymaster; in the stochastic environment the gap is 0.625 versus 0.468. The paper also claims this is the first agent with a probability distribution over a set of language models, updated from observations, and that the same framework extends to guessers, although the Bayesian guessers are robust rather than dominant.","pith_inferences":["A large part of the gain may come from the smoothing that Gaussian noise injects rather than from accurate model identification; comparing the noisy Bayesian spymaster to a non-Bayesian ensemble that averages over all candidate models without updating would isolate the value of the posterior itself.","If the Gaussian-noise assumption holds, the same recipe could be applied to human partners by treating human semantic and pragmatic variation as a mixture over embedding-based hierarchy models, a step the authors leave untested.","A productive stress test would be to use deliberately structured, non-Gaussian semantic shifts such as rotating or translating one internal embedding; the current design has no mechanism for such shifts, so the claim of adapting to any unseen semantics would be sharply tested."],"forward_implications":["A single Bayesian spymaster can outperform any fixed-embedding spymaster against out-of-distribution guessers, so Codenames agents no longer need to know their partner's embedding in advance.","The framework generalizes the deductive hierarchy: with one model and no noise, the Bayesian spymaster reduces exactly to a level-k spymaster, and with multiple models it hedges across them.","Even in a deterministic environment with an in-distribution partner (the d2v guesser), the Bayesian spymaster beats the matching static spymaster by distinguishing card types and avoiding the assassin when forced to give a bad clue.","Bayesian guessers are more robust than static ones but rarely beat the best static guesser, because clues carry less information about the spymaster's identity than guesses carry about the guesser; the paper identifies this as a limitation."],"supporting_citations":[{"why":"Defines the deductive hierarchy of spymasters and guessers that supplies the pragmatic component of the teammate models and the level-k baselines.","marker":"[4]"},{"why":"Establishes the embedding-based Codenames AI framework and the static spymaster/guesser baselines the Bayesian agents are compared against.","marker":"[18]"},{"why":"Motivates the use of Gaussian noise added to word embeddings to model fine-grained semantic uncertainty and to give every action non-zero likelihood.","marker":"[20]"},{"why":"Provides the word2vec embedding used as one of the four internal language models in the Bayesian agents' model set.","marker":"[23]"},{"why":"Provides the GloVe embedding (g3) used as an internal language model in the Bayesian agents' model set.","marker":"[26]"},{"why":"Provides the ConceptNet Numberbatch embedding (cnnb) used as an internal language model.","marker":"[30]"},{"why":"Provides the Dict2vec embedding (d2v) used as an internal language model, the one where the Bayesian spymaster beats the matching static spymaster in the deterministic environment.","marker":"[36]"}],"fun_headline_variants":["Bayesian spymaster learns partner, beats fixed strategies","Inference over teammate models wins against unseen partners","Noisy Bayesian agent outperforms static embedding in Codenames","Adaptive Bayesian guessing improves cooperation in language games","Bayesian updating over partner types yields better Codenames play"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach hinges on the assumption that any real teammate's behavior is well approximated by one of a small finite set of word-embedding plus hierarchy-level models, with all leftover variation being isotropic Gaussian noise added to word embeddings with a fixed standard deviation of 1.0; real semantic differences that are not noise-like would send the Bayesian updates off course.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian spymaster learns partner, beats fixed strategies","Inference over teammate models wins against unseen partners","Noisy Bayesian agent outperforms static embedding in Codenames","Adaptive Bayesian guessing improves cooperation in language games","Bayesian updating over partner types yields better Codenames play"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000346,"raw_usage":{"total_tokens":1853,"prompt_tokens":855,"completion_tokens":998,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":919}},"tokens_in":471,"tokens_out":998,"duration_ms":9486,"temperature":1.0,"reasoning_tokens":919,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:06:28.577462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pair the noisy Bayesian spymaster with a guesser whose embedding is a deterministic rotation or translation of one of the internal embeddings, a clearly non-Gaussian semantic shift. If the Bayesian spymaster does not beat the best static spymaster in this setting, the claim that it adapts to any unseen semantics fails. A complementary check is to collect the actual distribution of inter-agent guess differences and test whether they are Gaussian with variance near 1.0.","supporting_citations":[{"cited_title":"Bills and C","cited_arxiv_id":null,"evidence_quote":"Defines the deductive hierarchy of spymasters and guessers that supplies the pragmatic component of the teammate models and the level-k baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the embedding-based Codenames AI framework and the static spymaster/guesser baselines the Bayesian agents are compared against."},{"cited_title":"Mikolov, I","cited_arxiv_id":null,"evidence_quote":"Provides the word2vec embedding used as one of the four internal language models in the Bayesian agents' model set."},{"cited_title":"Pennington, R","cited_arxiv_id":null,"evidence_quote":"Provides the GloVe embedding (g3) used as an internal language model in the Bayesian agents' model set."},{"cited_title":"Speer, J","cited_arxiv_id":null,"evidence_quote":"Provides the ConceptNet Numberbatch embedding (cnnb) used as an internal language model."},{"cited_title":"Tissier, C","cited_arxiv_id":null,"evidence_quote":"Provides the Dict2vec embedding (d2v) used as an internal language model, the one where the Bayesian spymaster beats the matching static spymaster in the deterministic environment."}],"review_version":1}