{"id":"4653d0e3-2328-4b5d-a179-bb05cdc02895","arxiv_id":"2506.15756","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RecBayes identifies known teams and tasks from partial observations alone using a recurrent Bayesian classifier, scaling to roughly one million states, and then assists with pre-trained policies.","lead":"RecBayes trains a recurrent neural classifier on past partial observations so an ad hoc agent can recognize known teammate teams and tasks and then pick a pre-trained helper policy. It matters because previous methods either needed full states or teammate actions, or only worked in small tabular environments, while RecBayes claims to scale to about one million states.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The recurrent classifier is trained on trajectories from individual team-task policies but evaluated on a mixture policy; this unanalyzed covariate shift is load-bearing for the posterior updates.","rationale":"I chose the training/evaluation distribution shift because it directly attacks the bridge between the two components of the method: the classifier trained in Algorithm 2 and the policy mixture in Eq. 3. For the central claim to hold, ŝp_θ must output a reliable posterior not just on training rollouts but on exactly the closed-loop rollouts produced by the mixture. The paper does not measure this. The reader's weakest assumption identifies the same issue; my reading agrees. I do not think this warrants REJECT: the empirical results in Table 2 and Figure 4 are consistent with the method working in the tested domains, and the shift may be small there because team behavior dominates the observations and the policy mixture stays close to the training policies after convergence. But the absence of any analysis of the shift, plus the closed-loop compounding, is exactly why the verdict should remain CONDITIONAL rather than ACCEPT. Other concerns, such as the overgeneralized scaling claim and lack of error bars in Figure 4, are secondary and do not threaten the mechanism itself.","tokens_in":11288,"tokens_out":7477,"duration_ms":76798,"concrete_test":"Run evaluation episodes under the Eq. 3 mixture for each team-task, recording the classifier's prefix predictions and true labels. Compute prefix classification accuracy and expected calibration error (ECE) on these evaluation-distribution prefixes and compare against held-out training-distribution prefixes. Then retrain the classifier on trajectories collected by the mixture policy itself and re-run Table 2. If evaluation-distribution ECE is substantially higher than training-distribution ECE, or if retraining on mixture trajectories materially improves normalized scores, the training/evaluation covariate shift is the cause and the posterior updates in Algorithm 3 are not reliable as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RecBayes's central mechanism is that the recurrent classifier ŝp_θ approximates the Bayesian posterior p_t(k | a_{0:t-1}, z_{1:t}) well enough to weight the policy mixture in Eq. 3. However, Algorithm 1 collects training trajectories by executing the single best-response policy π_ψk for each team-task k, while Algorithm 3 evaluates using the mixture π_t = Σ_k p_t(k) π_ψk. The classifier therefore learns the distribution of action-observation pairs under the individual training policies; at evaluation, the same action-observation pair is generated under a different action distribution. Because the agent's own action is part of the evidence fed to ŝp_θ, and because π_t itself depends on p_t, this is a closed-loop covariate shift: any early miscalibration changes future actions, which changes future observations and compounds the error. The paper reports aggregate normalized returns and averaged identification curves, but no calibration or classification accuracy on evaluation-distribution prefixes, so the validity of the online posterior update is not established. If this shift is large in other domains, the mixed policy can degrade without any detection mechanism, undermining the claim that the method approximates Eq. 1 'well enough' in general.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes RecBayes, an ad hoc teamwork method for partially observable environments. RecBayes trains a recurrent classifier on trajectories collected during previous interactions with known team-task combinations, where the agent uses only its own partial observations and actions; at deployment, the classifier's output is used as an approximate posterior over known team-tasks to weight a library of best-response policies. The paper claims that this is the first approach that never requires states of the environment or teammates' actions at any stage and that can scale to large domains, and it evaluates on Level-Based Foraging and Predator-Prey with up to 970,200 states and 2^125 observations, reporting near-optimal normalized performance in most settings.","tokens_in":11401,"tokens_out":8187,"duration_ms":78526,"significance":"If the empirical results hold, RecBayes is a practically useful contribution: it removes the privileged-information requirement and the tabular-model bottleneck of prior ad hoc teamwork methods, and the experimental protocol—including normalization against original teammate and random policies, two domains, and three identification sub-problems—is a reasonable first test. The paper also discloses hyperparameters and reports a large number of trials. The key open question is whether the learned recurrent classifier is a reliable substitute for the exact Bayesian update under the deployment policy, and the current evidence does not yet establish that beyond the tested settings.","major_comments":[{"comment":"The training and deployment distributions over the agent's own actions differ: Algorithm 1 collects trajectories by executing the dedicated policy π_ψk for each team-task, whereas Algorithm 3 executes the mixture π_t = Σ_k p_t(k) π_ψk, which itself depends on the posterior being estimated. Because the action a_t is part of the evidence fed to p̂_θ, any early error changes future actions and therefore future observations, creating a closed-loop covariate shift. The paper does not analyze this shift or report calibration or classification accuracy on evaluation-distribution prefixes; without such evidence, the central claim that p̂_θ approximates Eq. (1) 'well enough' in general is not supported.","section":"Section 4, Algorithms 1 and 3"},{"comment":"Algorithm 2 states only that the cross-entropy loss is computed on complete trajectories τ_i, and it does not specify whether the loss is applied at every recurrent step or only after the full sequence. Algorithm 3, however, queries p̂_θ after each prefix (a_0, z_1, ..., a_{t-1}, z_t) and uses those intermediate outputs as the posterior for the mixture policy. If the classifier is trained only on full-sequence logits, the intermediate beliefs are never directly supervised, and the online posterior updates may be miscalibrated. Please clarify the training objective for prefixes and report prefix-level accuracy or calibration.","section":"Section 4, Algorithm 2"},{"comment":"The evaluation compares RecBayes only against general RL baselines, the original teammate, and random actions; it does not compare against any existing ad hoc teamwork algorithm such as ATPO, PO-GPL, FEAT, or ODITS. Since the paper's stated contribution is to overcome the limitations of these methods (tabular modeling and privileged information), a direct comparison on the 7x7 settings, where ATPO is feasible, is necessary to validate the claimed advantage over the state of the art.","section":"Section 6"},{"comment":"The near-optimality claim is based on normalized means with no confidence intervals or statistical tests. Table 2 reports a single number per condition, and the text asserts 'no statistically significant differences' without any test. Given the small number of trials and the high standard deviations in Table 1, values such as 0.86 and 0.85 in the 10x10 Task Identification rows may not be reliably below or above the 0.90 threshold; please add error bars or significance tests.","section":"Section 6.1, Table 2"}],"minor_comments":[{"comment":"The observation space is described as 2^125, but this is only an upper bound on the 5x5x5 tuple observation space; the reachable observation distribution is far smaller, and the phrase 'up to 2^125 observations' may overstate the actual diversity. Please clarify.","section":"Section 5.1"},{"comment":"The initial prior p_0 is never defined; please state how it is initialized (for example, as uniform over the K team-tasks).","section":"Section 4 and Algorithm 3"},{"comment":"The factor π_t(a_t | h_t) appears inside the sum in Eq. (1), but since it is independent of k it cancels in the normalization constant ρ; removing it would make the update easier to read.","section":"Eq. (1)"},{"comment":"The sums run from k=0 to K, while the team-tasks are indexed k ∈ {1,...,K}; this off-by-one notation should be corrected.","section":"Eqs. (2) and (3)"},{"comment":"The belief curves appear to be averaged over episodes, but the figure does not report error bars, the number of episodes, or per-episode classification accuracy; the statement that 'all team and tasks were in the end correctly classified' is stronger than what the averaged curves show.","section":"Figure 4"},{"comment":"The sentence 'All three non-optimal cases were related with Task Identification' should be quantified as two cases for RecBayes-MF and one for RecBayes-MB, based on the Table 2 values.","section":"Section 6.1"},{"comment":"The phrase 'arbitrarily large' is too strong for an empirical evaluation up to 970,200 states; suggest replacing it with 'substantially larger' or another more precise qualifier.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The paper is readable and the experiments are substantial, but the missing analysis of the training/deployment distribution shift and the lack of comparison with prior ad hoc teamwork methods are the main obstacles. I do not see any indication of misconduct; the issues are of empirical rigor and conceptual precision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RecBayes is the first ad hoc teamwork method under partial observability that never needs the state or teammates' actions and can handle environments with a million states, and the experiments mostly back that up. The core move is straightforward and credible: instead of doing the exact tabular Bayesian update from Ribeiro et al. (2023a), train a GRU-based sequence classifier on trajectories labeled by team-task, then use its posterior to weight a library of best-response policies. In Table 2, RecBayes is near-oracle in most 7x7 cases and in several 10x10 cases, and clearly above random everywhere. Figure 4 shows the correct team-task rising to near 1 over time. That is a real empirical result for a real restriction.\n\nThe paper is also honest about its scope: it only handles known teams and tasks, and says so in the introduction. The notation error in Eq. 3 is cosmetic.\n\nNow the soft spots. The stress-test concern is the right one. Algorithm 1 collects training trajectories by executing each single best-response policy; Algorithm 3 evaluates under the posterior-weighted mixture. The agent's own action is part of the classifier's evidence, so the action distribution shift changes the observation distribution, and the posterior affects the policy which affects the observations. The authors don't report calibration or classification accuracy on evaluation-distribution prefixes, so the online update's validity is not directly established. This matters, but in the tested settings the aggregate results indicate the shift did not destroy performance. It is a gap, not a refutation.\n\nThe other issues are milder. 'Arbitrarily large' is overclaiming: two grid sizes, up to ~1M states, do not demonstrate arbitrary scale. Figure 4 has no error bars or statistical aggregation, so the identification claim is weaker than it looks. The repository is only named as 'Anonymised for Review'; no code link in the preprint, which makes the empirical claims harder to check. All of these are fixable.\n\nWho this is for: people working in ad hoc teamwork, POMDP-based team identification, or MARL with pre-trained teammate libraries. A serious referee should get it; the central idea is clear, the experiments are appropriate, and the main weakness is a missing analysis rather than a fatal flaw. I would accept it with major revision, asking for code, error bars, a calibration analysis, and a softer scaling claim.","headline":"RecBayes is a credible step forward for partial-observability ad hoc teamwork, with a real distribution-shift gap that needs analysis but is not fatal.","tokens_in":12054,"tokens_out":2538,"would_cite":true,"duration_ms":25242,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that a recurrent classifier trained only on the ad hoc agent's own partial-observation trajectories can identify known teams and tasks in real time, reaching near-optimal assistance without ever seeing environment states…","keywords":["ad hoc teamwork","partial observability","recurrent Bayesian classifier","team identification","task identification","POMDP","multi-agent systems","RecBayes"],"falsifier":"Retrain or evaluate RecBayes in a moderately sized partially observable domain where the exact posterior of Equation 1 can be computed from the POMDP models; if the classifier's belief consistently lags behind or disagrees with the exact posterior, or if normalized performance drops below the 0.90 near-optimal threshold when training and evaluation policies differ, the approximation claim would be falsified.","tokens_in":10980,"feed_emoji":"🤝","tokens_out":10505,"duration_ms":95693,"temperature":0.7,"pith_summary":"RecBayes tackles ad hoc teamwork under partial observability: an agent is dropped into a team it has not coordinated with and must help. The paper's claim is that the agent can identify which known team and task it faces using only its own actions and partial observations, with no access to the environment's full state and no access to teammates' actions at any stage. That identification is learned by a recurrent Bayesian classifier trained on past trajectories, which approximates the exact Bayesian update over candidate team-task models. In two benchmark domains scaled to roughly 970,000 states and $2^{125}$ possible observations, the classifier's beliefs drive a mixed best-response policy that reaches near-optimal normalized performance in most settings. The payoff is a route to ad hoc teamwork that does not need tabular models of the world or privileged information.","feed_headline":"Partial observations alone identify teams and tasks on the fly","feed_subtitle":"RecBayes needs no environment states and no teammate actions, and scales to nearly one million states.","key_machinery":"The load-bearing object is the recurrent Bayesian classifier $\\hat{p}_{\\theta}$, a recurrent network trained with categorical cross-entropy on labelled trajectories to output a distribution over team-tasks. It stands in for the exact Bayesian update of Equation 1, which would require transition and observation probabilities for every candidate POMDP and is infeasible for large spaces. The classifier's belief $\\hat{p}_{\\theta}(k \\mid \\phi(z_t), h_t)$ is combined in Equation 3 with the best-response policy $\\pi_{\\psi_k}$ for each known team-task to form the agent's mixed policy. Relaxing observations to arbitrary features $\\phi(z_t)$ is what removes the need for a tabular model.","core_discovery":"The central claim is that a recurrent Bayesian classifier trained on trajectories of the form $\\tau=(a_0,z_1,a_1,\\ldots,a_{L-1},z_L)$, collected while the ad hoc agent explored each known team-task under partial observability, approximates the exact Bayesian posterior over team-tasks that Equation 1 would compute with full POMDP models. At run time the classifier's output distribution over team-tasks weights the best-response policies of all known team-tasks, producing the mixed policy of Equation 3. Because the classifier consumes arbitrary features $\\phi(z_t)$ of the observation rather than a tabular entry, the approach scales to domains with roughly $970{,}200$ states and $2^{125}$ observations. The paper presents evidence that this identifies the correct team-task on the fly in both team-identification and task-identification settings, and that the resulting assistance is near-optimal in most evaluated cases.","pith_inferences":["Beyond the paper's own experiments, an immediate test of the underlying approximation is to compare the classifier's belief trajectory with the exact posterior of Equation 1 in a small domain where that posterior is computable; a systematic divergence would mean the method succeeds despite, not because of, its Bayesian justification.","The paper trains the classifier on trajectories collected under exploratory policies and evaluates it under the mixed policy of Equation 3; retraining on evaluation-policy trajectories would test whether that distribution shift matters for identification accuracy.","Because the classifier accepts arbitrary observation features $\\phi(z_t)$, a natural next step is high-dimensional or continuous observations, such as raw images; the algorithm itself would not need to change.","A scaling stress test the paper does not report is holding the domain fixed while increasing the number of known team-task combinations; identification accuracy should degrade with more alternatives, and the rate of that degradation would guide practical use."],"forward_implications":["An ad hoc agent that has only ever seen its own past interactions with a team can later assist that same team without any privileged state information being available during training or deployment.","The tabular bottleneck of earlier partial-observability approaches, which were limited to domains of a few thousand states, is broken; the same pipeline applies to much larger state and observation spaces as long as trajectories can be collected.","The team-task identification problem is reduced to recurrent sequence classification, so the classifier and the best-response policies can be trained independently and combined only at run time.","Because the method never needs teammate actions, it applies to settings where other agents' decisions are private or unobservable, which is the rule in many real-world deployments rather than the exception."],"supporting_citations":[{"why":"Formalizes ad hoc teamwork as task identification, team identification, and planning, the framing RecBayes builds on.","marker":"Melo & Sardinha (2016)"},{"why":"The tabular partial-observability approach whose models and limitations RecBayes takes as its starting point.","marker":"Ribeiro et al. (2023a)"},{"why":"Introduces the ad hoc teamwork problem and the evaluation protocol of replacing one teammate that the experiments follow.","marker":"Stone et al. (2010)"},{"why":"ODITS, a prior partial-observability method that RecBayes says requires fully observable pre-training with states and teammate actions.","marker":"Gu et al. (2021)"},{"why":"PO-GPL, a partial-observability baseline that still requires teammate actions for identification.","marker":"Rahman et al. (2023)"},{"why":"FEAT, which extends PO-GPL with meta-learning and also relies on teammate actions, one of the baselines RecBayes must avoid needing.","marker":"Fang et al. (2024)"},{"why":"Provides the limited field-of-view observation function adapted to make the benchmark domains partially observable.","marker":"Santos et al. (2022)"},{"why":"PERSEUS, the point-based POMDP solver used to compute best-response policies in the tabular predecessor approach.","marker":"Spaan & Vlassis (2005)"}],"fun_headline_variants":["Bayesian ad hoc teamwork from observations alone, no states or actions","On-the-fly team ID from observations only, scales to 1M states","RecBayes: identify teams and tasks without states or teammate actions","Partial observations suffice: RecBayes scales to 1M states","No states, no teammate actions—just observations for teamwork"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that trajectories collected while the agent is learning with exploratory policies are representative enough for the classifier to keep identifying team-tasks at evaluation time, when the agent is instead executing the mixed policy and thereby changing what it observes.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian ad hoc teamwork from observations alone, no states or actions","On-the-fly team ID from observations only, scales to 1M states","RecBayes: identify teams and tasks without states or teammate actions","Partial observations suffice: RecBayes scales to 1M states","No states, no teammate actions—just observations for teamwork"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000306,"raw_usage":{"total_tokens":1783,"prompt_tokens":1002,"completion_tokens":781,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":689}},"tokens_in":618,"tokens_out":781,"duration_ms":7755,"temperature":1.0,"reasoning_tokens":689,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:34:37.629710+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain or evaluate RecBayes in a moderately sized partially observable domain where the exact posterior of Equation 1 can be computed from the POMDP models; if the classifier's belief consistently lags behind or disagrees with the exact posterior, or if normalized performance drops below the 0.90 near-optimal threshold when training and evaluation policies differ, the approximation claim would be falsified.","supporting_citations":[],"review_version":2}