{"id":"14a58878-b3cd-41d6-8226-fae7809e644d","arxiv_id":"2501.01463","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DRACO infers an agent's goal by comparing its observed actions to learned goal-specific policies using Z-score or absolute-deviation distance metrics.","lead":"Researchers propose DRACO, a goal recognition method that trains a separate reinforcement learning policy for each possible goal, then scores observed actions against those policies to infer which goal an agent is pursuing. It runs on raw, continuous observations without hand-built domain models and reports large savings in memory and training time versus tabular baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'state-of-the-art' claim rests on comparisons only against GRAQL and R&G; existing deep-RL goal recognizers (Fang 2023, Chiari 2022, Maynard 2019) are cited but never benchmarked, so the central claim is unsupported.","rationale":"The reader's verdict was CONDITIONAL, and this stress-test agrees with that. The reader identified distinguishability as the weakest assumption; that is a valid necessary-condition concern and the paper lists it in Section 7. However, the more load-bearing problem for the headline claims is the absence of any comparison with the nearest prior work. The paper's own related-work section names three deep-RL or deep-learning GR methods, yet none appear in Tables or Figures; GRAQL and R&G are not continuous-domain state-of-the-art. Thus the falsifiable prediction 'state-of-the-art in continuous settings' is untested. Adding those baselines is the concrete check that would settle the claim. The double-normalization in Eqs. 2-3 and the mislabeled Wasserstein metric are also issues, but they are secondary because the experimental results are consistent with the intended ranking behavior. No internal inconsistency was found that would invalidate the method; the condition is that the claims be softened or the baselines added.","tokens_in":13970,"tokens_out":7220,"duration_ms":70879,"concrete_test":"Benchmark DRACO against Fang et al. (2023), Chiari et al. (2022), and Maynard et al. (2019) on the same MiniGrid and Panda-gym configurations, using identical observation sequences, observability levels, and noise conditions. Use the released code or reimplementations with the hyperparameters reported in those papers. If DRACO does not outperform them on the continuous Panda-gym tasks, the 'state-of-the-art' and 'first' claims should be revised; if it does, the claims are provisionally supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DRACO achieves state-of-the-art performance in discrete and especially continuous settings. The evidence compares DRACO only with GRAQL, a tabular Q-learning method forced into a discretization of Panda-gym, and Ramirez & Geffner, a symbolic planner not applied to continuous domains. The related work cites three deep-RL/neural GR methods that are directly relevant competitors: Chiari et al. 2022 (GRNet), Maynard et al. 2019, and Fang et al. 2023, which uses deep RL for continuous goal recognition. None are included in the experiments. Without them, the phrase 'state-of-the-art' is not established; beating a discretized Q-table baseline and a planner used only where it is inapplicable does not demonstrate superiority over modern GR systems. In addition, the 'first goal recognition algorithm that learns policy networks from unstructured data' claim is undercut by Fang et al. 2023, which the paper itself describes as extending goal recognition with deep RL to robotics simulators; at minimum, the priority claim needs a prior-art analysis. The distinguishability assumption in Section 3.1 is real but is partially validated by the reported F-scores and is explicitly acknowledged in Section 7 as a limitation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DRACO, a goal recognition algorithm that learns one actor-critic policy per candidate goal via deep RL and then infers the observed agent's goal by comparing its trajectory with these policies using two proposed distance metrics: a so-called Wasserstein distance and a z-score. The approach is evaluated in MiniGrid (discrete) and Panda-gym (continuous) against Ramirez and Geffner's plan-recognition baseline and GRAQL, a tabular RL method, reporting accuracy, precision, recall, F-score, confidence, and resource usage.","tokens_in":14236,"tokens_out":6988,"duration_ms":65484,"significance":"If the technical issues are corrected, the idea of using learned policy networks for goal recognition is a promising step toward scaling GR to continuous and unstructured domains, and the Panda-gym testbed is a useful addition. The authors explicitly train with one RL algorithm (PPO) and test on trajectories from another (A2C), which is a good generalization check. However, the current empirical evidence does not justify the 'state-of-the-art' claim, as the only comparisons are against a tabular method and a symbolic planner, omitting the deep-learning GR baselines cited in the paper itself.","major_comments":[{"comment":"The term P(O|g) is defined as a softmin over distances to all goals in G, so the value for one goal inherently depends on the other hypotheses and the vector sums to 1. Applying Bayes' rule in Eq. (3) then merely renormalizes this already-normalized vector, making the Bayesian step redundant with a uniform prior. Consequently P(O|g) is not a likelihood in the usual sense; the posterior is a softmin score, not the result of a generative model. Please reframe the inference as score-based ranking or define a per-goal likelihood that factorizes over observations.","section":"Section 3.2, Eqs. (2)-(3)"},{"comment":"The method labeled 'Wasserstein distance' is not a Wasserstein distance. Eq. (6) is the L1 norm between a single observed action and a single random sample from the policy, and Eq. (7) averages these per-step deviations; this is a Monte Carlo estimator of the mean absolute deviation, not the Wasserstein distance defined in Eq. (5). The stochastic sampling also introduces avoidable variance. Please either compute a true Wasserstein distance or rename the metric and justify the approximation.","section":"Section 3.3, Eqs. (5)-(7)"},{"comment":"Several R&G rows in Table 2 are internally inconsistent under standard definitions. For instance, in the 10% Lava row, accuracy is 1.0 ± 0.0 while precision is 0.77 ± 0.46 and recall is 1.0 ± 0.0; with accuracy = (TP+TN)/(TP+TN+FP+FN), recall = 1.0 and accuracy = 1.0 force FP = 0, which contradicts precision = 0.77. The 2-Goals and 3-Goals rows (accuracy 1.0 with precision 0.5) are similarly impossible. Please clarify the exact computation of these metrics for multi-goal problems and correct the table, as these numbers undermine the comparative claims.","section":"Section 4-5, Table 2"},{"comment":"The 'state-of-the-art' claim is not supported by the experimental comparison. The evaluation includes only GRAQL (tabular RL) and R&G (symbolic planning), while the related work itself identifies deep-learning GR methods: Chiari et al. 2022 (GRNet), Maynard et al. 2019, and Fang et al. 2023, the last of which applies deep RL to continuous-domain goal recognition. None of these appears in the experiments. Please add at least one direct deep-RL baseline or provide a clear justification for their omission, and temper the abstract's 'state-of-the-art' wording accordingly. The priority claim of being 'first' to learn a set of policy networks also requires a detailed prior-art discussion relative to Fang et al. 2023.","section":"Section 5 and Section 6"}],"minor_comments":[{"comment":"The softmin expression does not specify a temperature parameter; if the temperature is a free parameter, its value and sensitivity should be reported, and if it is fixed, the value should be stated.","section":"Section 3.2, Eq. (2)"},{"comment":"The z-score metric for continuous multi-dimensional actions is ambiguous: it should specify whether μ and σ are computed per action dimension and how the absolute value is computed for vector-valued actions.","section":"Section 3.3, Eq. (8)"},{"comment":"The notation a(s) in the reward function is not defined; it appears to denote the successor state after applying action a in state s, and should be clarified.","section":"Section 3.1, Eq. (1)"},{"comment":"The sentence 'In Panda-Gym, GRAQL's learning rate (γ) was 0.01' uses γ, which elsewhere in the paper denotes the discount factor; rename one of these variables to avoid confusion.","section":"Section 4, Hyperparameters"},{"comment":"The phrase 'GRAQL and DRACO load their policies before inference' should specify that GRAQL loads Q-tables rather than policies; the term 'policy' is used loosely throughout.","section":"Section 5, Scalability"},{"comment":"There are several typos and formatting issues, including 'differnece' and 'precieved' in Section 3.2, and garbled equation formatting in Eqs. (4) and (5) (e.g., '1P' and 'kX'). A careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the framework is generally reproducible, but the paper currently overclaims its empirical standing. The missing comparisons with deep-RL GR baselines and the inconsistent Table 2 are load-bearing issues that must be addressed before the claims can be accepted. I believe these are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a worthwhile contribution to goal recognition, but read the claims with salt. The core idea—train one PPO policy per candidate goal and score an observation by how well its state-action pairs fit each policy's action distribution—is sensible and genuinely different from prior tabular and neural GR work. The continuous-domain testbed, the WCD-based ambiguity analysis, and the memory/runtime comparison against GRAQL are useful concrete contributions. The paper also deserves credit for generating observations with A2C rather than the trained PPO, and for explicitly listing the distinguishability and simulator assumptions as limitations in Section 7.\n\nWhere it gets soft: the 'first' and 'state-of-the-art' claims are not supported by the evidence. The paper itself cites Fang et al. 2023, Chiari et al. 2022, and Maynard et al. 2019 as deep-learning GR methods, but benchmarks none of them. Beating GRAQL (tabular Q-learning) and R&G (a planner that does not handle continuous domains) does not establish SOTA. The low-observability MiniGrid results are also mixed: R&G actually gets higher accuracy in the 10% Lava condition, even if DRACO has better precision and confidence. The abstract's sweeping superiority claim is too strong.\n\nThe inference math is sloppier than it looks. Equation 2 defines P(O|g) as a softmin over all goals, so each 'likelihood' already depends on every other goal; Equation 3's Bayes normalization is then redundant. And Equation 6 is a single-sample L1 absolute deviation, not a Wasserstein distance between distributions. The Z-score metric is more honestly named. Both are fixable with renaming and a clearer derivation, but as written they invite over-interpretation.\n\nThe distinguishability assumption in Section 3.1 is load-bearing, but the paper acknowledges it and the reported F-scores suggest it holds in these domains. That is a fair limitation, not a fatal one. Missing code and data is a real obstacle for reproduction, not a minor issue.\n\nBottom line: worth a serious referee. The construction is new enough to matter to the goal-recognition community, and the paper is honest about many of its own limitations. I would send it to review with a request to soften the priority and SOTA claims, add at least one deep-RL baseline to the experiments, fix the metric naming, and release the testbed and policies. With that revision, I would be comfortable citing it.","headline":"A genuinely new way to do goal recognition with per-goal learned policies, but the 'first' and 'state-of-the-art' claims outrun the evidence; worth reviewing seriously with revision.","tokens_in":14774,"tokens_out":2632,"would_cite":false,"duration_ms":27252,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that goal recognition can be done by learning one policy network per candidate goal in a simulator and scoring an observed trajectory by its distance to each policy, removing hand-built domain models and discretization.","keywords":["goal recognition","reinforcement learning","actor-critic","policy networks","Wasserstein distance","z-score","continuous state spaces","partial observability"],"falsifier":"Create a two-goal continuous domain where the trained policies yield the same action distribution at every state the observed agent visits; if DRACO's posterior over the true goal stays at chance while a separated-policy control succeeds, the distinguishing-policy assumption is the point of failure.","tokens_in":13762,"feed_emoji":"🎯","tokens_out":11361,"duration_ms":101024,"temperature":0.7,"pith_summary":"DRACO is a goal-recognition method that replaces hand-written domain theories with neural-network policies learned offline by reinforcement learning. The paper claims this is the first goal-recognition algorithm that learns a set of policy networks from unstructured data and uses them for inference. Its core idea is to train one policy per candidate goal in a simulator, then score an observed trajectory by how closely its actions match each policy, using either a Wasserstein-based or a Z-score-based distance. On a grid-navigation domain it matches or beats a planner-based recognizer and a tabular Q-learning recognizer while giving much sharper confidence separation, and on a simulated robot-arm domain it outperforms the tabular baseline at a fraction of the memory and training cost. If correct, the approach would extend goal recognition to continuous, high-dimensional, partially observed settings without an expert-built planning model.","feed_headline":"Learned policies infer goals from raw observations, no domain model","feed_subtitle":"No domain expert, no discretization, and a small memory footprint: learned policies do the recognizing.","key_machinery":"The load-bearing object is the set of goal-conditioned actor networks $\\pi_g$, one neural network policy per candidate goal, trained offline with an actor-critic reinforcement learning algorithm. The actor network outputs an action distribution in the same space as the observations, so the comparison between an observed action and the policy can be done directly, without symbolic domain theory or discretization. At inference, DRACO aggregates the per-step distance into $\\Delta(O,\\pi_g)$, turns it into $P(O|g)$ by softmin, and normalizes to $P(g|O)$. The critic network is used only during training; the actor's output carries the recognition signal.","core_discovery":"The paper's central claim is that goal recognition can be solved end-to-end from raw state-action observations by learning a policy $\\pi_g$ for each candidate goal and measuring how well an observed trajectory fits each policy. The likelihood $P(O|g)$ is computed from an aggregate distance $\\Delta(O,\\pi_g)$ between observed actions and the actions the policy would take, and $P(g|O)$ comes from softmin normalization over goals. Two distance metrics are introduced: a Wasserstein-based distance that compares each observed action to a single action sampled from the policy, and a Z-score-based distance that compares the observed action to the mean of the policy's Gaussian action distribution divided by its standard deviation. The paper reports that in discrete grid-navigation problems DRACO reaches or exceeds the accuracy of the planner-based and tabular baselines while providing far higher confidence, and in a continuous robot-arm environment it outperforms the discretized tabular baseline, with memory measured in a few megabytes versus gigabytes and training times of about thirteen minutes versus hundreds of minutes.","pith_inferences":["The paper leaves implicit that the per-goal policy-distance mechanism could apply to human intent prediction if policy training used demonstrations rather than simulator rewards, removing the environment-model requirement at deployment.","A testable extension is to train one goal-conditioned policy that takes the candidate goal as input and ask whether the Wasserstein and Z-score distances still separate hypotheses; success would cut training cost from one network per goal to a single network.","The distinguishability requirement yields a practical diagnostic: before deployment, compute pairwise distances between learned policies on the states the observed agent is expected to visit and flag goal pairs that overlap, since no distance-based recognizer can separate them."],"forward_implications":["Continuous state and action spaces no longer need to be discretized for recognition, because the policy networks operate directly on the raw observation format.","Memory and inference cost stop depending on the size of the state-action space: what is stored is a set of neural networks, not a table or a planning domain.","Adding a new candidate goal can reuse existing policies through goal-conditioned RL or transfer learning, rather than requiring a new hand-built domain theory.","The two inference metrics give practitioners a choice: Wasserstein distance holds up better under missing and noisy observations, while the Z-score metric becomes more accurate as the learned policies approach optimality."],"supporting_citations":[{"why":"Supplies the Bayesian goal-recognition formulation $P(g|O)$ and the planner-based baseline compared in the discrete domain.","marker":"Ramirez and Geffner 2011"},{"why":"Defines the tabular Q-learning approach (GRAQL) that serves as the main RL baseline in both discrete and discretized-continuous evaluations.","marker":"Amado, Mirsky, and Meneguzzi 2022"},{"why":"Supplies the PPO algorithm used to train DRACO's goal-conditioned policies.","marker":"Schulman et al. 2017"},{"why":"Supplies the A2C algorithm used to generate the observed trajectories in the experiments.","marker":"Mnih et al. 2016"},{"why":"Supplies the discrete grid-navigation benchmark environments used in the discrete evaluation.","marker":"Chevalier-Boisvert et al. 2023"},{"why":"Supplies the continuous robot-arm benchmark used for the main continuous evaluation.","marker":"Gallouedec et al. 2021"},{"why":"Supplies the worst-case distinctiveness metric used to quantify ambiguity of the tested problems.","marker":"Keren, Gal, and Karpas 2014"},{"why":"Source of the Wasserstein distance that one of DRACO's inference metrics adapts.","marker":"Vallender 1974"},{"why":"Provides the stable-baselines implementation used to run the PPO training.","marker":"Raffin et al. 2021"}],"fun_headline_variants":["DRACO learns goal policies from raw data, no domain model","Deep RL infers goals from raw observations without handcrafting","Policy-based goal recognition outperforms planner baselines","No discretization needed: policies recognize goals in raw state-action space","Goal recognition via actor-critic: cheaper and memory-light"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the policy learned for each goal is more similar to an observed agent's behavior toward that goal than toward any other goal, and that a simulator or environment model is available to train those policies.","fun_headline_variants_meta":{"raw":{"variants":["DRACO learns goal policies from raw data, no domain model","Deep RL infers goals from raw observations without handcrafting","Policy-based goal recognition outperforms planner baselines","No discretization needed: policies recognize goals in raw state-action space","Goal recognition via actor-critic: cheaper and memory-light"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000682,"raw_usage":{"total_tokens":3067,"prompt_tokens":887,"completion_tokens":2180,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":2096}},"tokens_in":503,"tokens_out":2180,"duration_ms":17062,"temperature":1.0,"reasoning_tokens":2096,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:48:04.238210+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Create a two-goal continuous domain where the trained policies yield the same action distribution at every state the observed agent visits; if DRACO's posterior over the true goal stays at chance while a separated-policy control succeeds, the distinguishing-policy assumption is the point of failure.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Bayesian goal-recognition formulation $P(g|O)$ and the planner-based baseline compared in the discrete domain."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the tabular Q-learning approach (GRAQL) that serves as the main RL baseline in both discrete and discretized-continuous evaluations."},{"cited_title":"P.; Mirza, M.; Graves, A.; Lillicrap, T.; Harley, T.; Silver, D.; and Kavukcuoglu, K","cited_arxiv_id":null,"evidence_quote":"Supplies the A2C algorithm used to generate the observed trajectories in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the worst-case distinctiveness metric used to quantify ambiguity of the tested problems."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the Wasserstein distance that one of DRACO's inference metrics adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the stable-baselines implementation used to run the PPO training."}],"review_version":1}