{"id":"3d63b5f4-29b5-4a6d-9542-e68e16e88637","arxiv_id":"2507.06690","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A knowledge-graph-embedding 'skill graph' selects, combines, or fine-tunes pre-trained multi-agent skills across unrelated tasks, beating a hierarchical MAPPO baseline in a staged flock-then-combat scenario.","lead":"This paper proposes a skill graph, a knowledge-graph-based memory, to let multi-robot teams switch between unrelated tasks such as flocking and combat by retrieving or combining previously trained skills. It offers a modular way to reuse low-level reinforcement learning policies across tasks without retraining from scratch.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The comparative claim is unsupported: the skill graph's ρsucc is never measured, and the stated 100% follows from a definition that counts any decision as success.","rationale":"The reader's weakest assumption—that hand-crafted feature vectors are sufficient for embedding-space distance to reflect skill suitability—is a genuine limitation of the method's generalization claims. However, the single most load-bearing concern for the paper's central comparative claim is more immediate: the paper never actually measures the skill graph's decision success rate. Section V-C defines ρsucc as a success rate, reports it only for high-level MAPPO, and then asserts the skill graph achieves 100% because its scoring mechanism always returns a highest-ranked skill. This conflates 'made a decision' with 'made the correct decision'. If the metric is meant to measure whether the chosen skill matches the task, then the skill graph's value is unknown and the claimed superiority has no quantitative basis. If the metric literally counts any output as success, then Table II is not a comparison at all, since MAPPO is evaluated on correctness. Either way, the central claim is not supported by the presented evidence. This concern is addressable with a straightforward experiment, so it does not warrant rejection; the reader's CONDITIONAL verdict is appropriate. I partially agree with the reader because their identified weakness, while relevant, is not the assumption on which the headline comparison hinges.","tokens_in":13791,"tokens_out":4869,"duration_ms":57561,"concrete_test":"Compute ρsucc for the skill graph under the same success criterion used for MAPPO in Section V-C: a decision is successful only if the selected (highest-scoring) skill matches the ground-truth expected skill for that query, as defined by the scenario design (for interpolation queries, define ground truth by running each candidate skill on the query and taking the best-performing one). Place this result as a new row in Table II alongside the MAPPO rows, for the same base case and the same generalization perturbations (init num and leader v variations). If the skill graph's ρsucc under this corrected definition is materially below the reported MAPPO values, the central claim fails; if it is near 100%, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that the skill graph outperforms the latest hierarchical MAPPO—rests entirely on Section V-C, but the comparison is never actually carried out. ρsucc is defined in V-C.1 as nsucc/ntotal with 'successful decisions'; the MAPPO high-level receives a −1 reward when the selected skill does not match the expected one, so Table II is a genuine accuracy measure (≈88–95%). For the skill graph, no row is reported. Instead, V-C.2 asserts ρsucc = 100% because the graph 'selects skills based on their scores without requiring a perfect score, (i.e., 1)'—i.e., because the scoring function always returns a highest-scoring skill. Under that definition, even a random skill selector that always outputs a skill would have ρsucc = 100%, so the metric cannot distinguish the method from a trivial baseline. The claimed superiority is therefore a definitional artifact unless the skill graph is evaluated under the same success criterion used for MAPPO. The reader's feature-sufficiency concern (Sec. IV-E) is real, but it is secondary: even if the features are perfect, this paper does not contain the evidence needed to substantiate the headline comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hierarchical multi-task multi-agent reinforcement learning (MT-MARL) architecture in which a knowledge-graph-style \"skill graph\" serves as the high-level module and a local-critic MADDPG variant serves as the low-level module. Environments, tasks, and skills are represented as entities in a graph embedded with TransH; given a query environment and task, the graph scores all skills and selects, combines, or further trains the highest-scoring ones. The authors claim the method handles unrelated tasks (adversarial and cooperative), transfers knowledge better than standard hierarchical RL, and outperforms hierarchical MAPPO baselines. The paper reports simulation experiments with 50-agent teams and a qualitative six-robot real-world demonstration, and it makes code and videos publicly available.","tokens_in":14063,"tokens_out":4504,"duration_ms":54470,"significance":"If the comparative claims were fully substantiated, the paper would be a useful contribution: it addresses the under-studied setting of unrelated tasks in MT-MARL, provides a modular graph-based selection mechanism, and includes both simulation and real-robot demonstrations. The availability of code and videos is a clear strength, as is the explicit acknowledgment in Section VII that entirely unseen scenarios require learning from scratch. However, the central comparative claim against hierarchical MAPPO is not currently supported by the reported evidence. The skill graph's ρ_succ=100% is asserted on the basis of its definition rather than measured under the same success criterion used for the baseline, and the baseline receives less informative observations than the skill graph. These issues are load-bearing because the abstract and introduction cite \"outperforms the latest hierarchical MAPPO\" as the main empirical result. The underlying idea remains plausible, and the missing evidence is in principle obtainable, so the paper is best treated as requiring major revision rather than rejection.","major_comments":[{"comment":"The claim that the skill graph achieves ρ_succ=100% is definitional, not empirical. The text states that the skill graph selects skills based on their scores without requiring a perfect score (i.e., 1), resulting in ρ_succ=100%; under that definition, any selector that always returns a highest-scoring skill would also have 100% success. The ρ_succ reported for high-level MAPPO in Table II is a genuine accuracy measure: a decision is successful only if the selected skill matches the expected one, with a -1 reward for mismatch. Unless the skill graph is evaluated under the same matching-based criterion on the same set of test queries, the paper's headline claim that the proposed method outperforms hierarchical MAPPO is unsupported. Please report the skill graph's ρ_succ under the identical success definition, including variance over seeds, and define the expected skill for each query in the combination and further-training regimes.","section":"V-C.2"},{"comment":"The comparison between Scheme 1 and Scheme 2 is confounded by input information. The high-level MAPPO observes only o_g = [x_rg, n_g, n_r] (relative team-center position and team sizes), whereas the skill graph receives the full ground-truth query: the environment features (y, L) and the task features (v_max, v_min, Δh, n_o, r_att) or (v_max, v_min, d_ref, r_perc). The skill graph therefore has privileged information about the exact task parameters, while the baseline must infer the task from trajectories. The two frameworks are also not controlled at the low level: local-MADDPG versus MAPPO can change the distribution of states observed by the high level. These differences alone could explain the reported gap. Please either provide the same task-feature inputs to the high-level MAPPO baseline, or evaluate the skill graph from partial observations, and ideally match the low-level algorithm.","section":"V-C.1 and V-B.2"},{"comment":"The skill graph's generalization behavior rests on the assumption that hand-crafted feature vectors and the weighted similarity δ in Eq. (1) make embedding-space distance correspond to skill suitability, but no sensitivity analysis is provided for the chosen weights k_j or the thresholds α_high and α_low. Moreover, for queries that exactly match a training triple (e.g., stage 1 query (1,6)+(1,0,0.4,3) matching 'floc 3 fixed'), a score near 1 is directly enforced by the positive-sample term (S_posi - 1)^2 in Eq. (1), so the stage-1 and stage-2 results in Fig. 5(d)-(e) demonstrate retrieval of memorized samples rather than predictive generalization. The d_ref=0.6 combination case in Fig. 5(f) is the only genuinely interpolated query presented, and it is not evaluated with a quantitative success metric. Please add held-out queries and report how often the graph's top selection matches the independently verified best skill.","section":"IV-E and Eq. (1)"},{"comment":"Several effectiveness claims are supported only by single illustrative trials. Fig. 5(h) reports that further training of the highest-scoring skill requires less training time and better sampling efficiency than training from scratch, but no reward curves with numerical values, no number of seeds, and no definition of the convergence criterion are given. Similarly, the real-world experiment in Section VI is described through snapshots only; there are no quantitative metrics such as task completion rate, time to convergence, or number of repeated trials. These claims should either be quantified with statistics or explicitly labeled as qualitative demonstrations.","section":"V-B.2 and VI"}],"minor_comments":[{"comment":"The section heading contains a typo: 'Feature Constructon' should be 'Feature Construction'.","section":"IV"},{"comment":"The caption contains garbled text ('0.6refdd' and inconsistent d_ref formatting) and the mapping of subfigures (c)-(f) to periodic versus fixed boundaries is unclear; please clean up the caption and ensure the labels match the images.","section":"Fig. 4 caption"},{"comment":"The table header repeats 'Adve/Floc' in a way that makes it difficult to tell which hyperparameters belong to local-MADDPG and which to MAPPO; please reformat with clearer column groupings.","section":"Table I"},{"comment":"The text says 'dim' indicates the number of low-level skills, but Table II rows range from dim=2 to dim=12; please clarify how these skill sets were constructed and how the 'expected' skill for the -1 reward is determined in each case.","section":"V-C.1"},{"comment":"The weight notation k_j is not fully defined: the text gives values such as k_1,t=0, k_2,t=0, ..., but does not explain the mapping from j to the attributes of the feature vectors, nor how k_j values for environment features interact with those for task features.","section":"Eq. (1) and V-B.1"},{"comment":"The thresholds α_high and α_low are used in the utilization procedure but are only assigned concrete values in Section V-B.1; please either define them in Section III-B or state that they are application-specific hyperparameters.","section":"III-B"},{"comment":"The skill graph idea is credited to reference [24] (RSG), but the paper does not discuss how the present construction differs from RSG; a brief comparison would help position the contribution.","section":"Related work"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is clearly written and the experimental platform is impressive, but the central comparison is currently not valid as reported. The most serious issue is that the skill graph's ρ_succ=100% is a definitional artifact rather than a measured quantity under the same criterion applied to the MAPPO baseline. If the authors can supply a fair comparison with matching information and a proper success metric, the paper could become acceptable; as it stands, the headline claim cannot be verified from the text."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe paper applies the skill-graph idea from RSG [24] to multi-agent multi-task RL, specifically to the unrelated-task setting (adversarial plus cooperative) in a swarm context. The staged flocking/combat/flocking simulation and the six-robot hardware demo are real; the high-level graph being trained independently of the low-level policies is clean and the code is provided. If the evaluation held up, this would be a useful modular alternative to hierarchical MAPPO for switching between unrelated swarm behaviors.\n\nThe centerpiece comparison, however, is not supported. Section V-C defines ρ_succ for the MAPPO baseline as the fraction of decisions matching an expected skill, and the baseline's numbers are genuine accuracies (about 88–95%). For the skill graph, no such number is reported. Instead the paper asserts ρ_succ=100% because the graph 'selects skills based on their scores without requiring a perfect score.' That definition makes any selector that always outputs a skill achieve 100%. So the abstract's 'outperforms the latest hierarchical MAPPO' rests on a metric artifact. The stress-test note is correct; the graph's selection accuracy is never measured under the same criterion.\n\nThe feature-sufficiency worry is secondary but real. The encoders map hand-crafted feature vectors (e.g., d_ref, r_att, boundary type) into embedding space, and there is no sensitivity analysis for the feature weights k_j or the thresholds α_high/α_low. Exact-match queries get near-1 scores by construction of the soft-sample loss, so those cases are memorized lookups more than predictions. The combination and further-training cases are more interesting, but the paper doesn't quantify how much they help over picking the best single skill.\n\nThe concluding paragraph honestly notes that the graph cannot transfer to entirely unseen scenarios, which I credit. The method itself is coherent and the hardware demo shows a functional pipeline. But the headline comparative claim is a definitional artifact, and the missing numbers are easy to supply. That is a major-revision issue, not a reject issue.\n\nThis paper is for swarm robotics and MARL researchers working on modular skill reuse; they would get value from it if the evaluation is redone. I would send it to peer review with a clear expectation of rework, and I would want to see the skill graph's ρ_succ computed under the same success criterion used for the MAPPO baseline.","headline":"A plausible modular skill-graph system for unrelated multi-agent tasks, but the headline comparison against MAPPO rests on a metric artifact (ρ_succ=100% by definition).","tokens_in":14593,"tokens_out":2370,"would_cite":false,"duration_ms":27323,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By embedding environments, tasks, and skills in a shared vector space and scoring every skill against a query, the paper shows that one hierarchical controller can handle adversarial and cooperative tasks in the same library and…","keywords":["multi-task multi-agent reinforcement learning","skill graph","knowledge graph embedding","TransH","hierarchical reinforcement learning","multi-robot systems","flocking","adversarial tasks"],"falsifier":"Take a pair of queries with identical feature vectors but different true dynamics, for example two tasks that share the reported boundary type, speed limits, attack radius, and perception range but differ in robot mass or in an unlisted environmental factor. If the skill graph still rates an old skill near 1 and the execution fails, the hand-built features are not sufficient to make embedding distance track real skill suitability, and the ranking mechanism would not transfer to unseen queries.","tokens_in":13593,"feed_emoji":"🤖","tokens_out":9042,"duration_ms":147117,"temperature":0.7,"pith_summary":"This paper tries to establish that a multi-agent robot team can learn many unrelated tasks, such as fighting an adversary and flocking with teammates, using one hierarchical controller whose top layer is a skill graph. The graph treats environments, tasks, and learned skills as entities in a knowledge graph, embeds them in a shared vector space, and answers each new query by scoring every known skill. High scores mean a skill is selected as-is, middle scores mean several skills' actions are blended, and low scores mean the best existing skill is fine-tuned. The authors claim this decouples task selection from low-level skill training, so unrelated tasks do not need shared attributes, and they report better decision success and generalization than hierarchical MAPPO. They also demonstrate the full fight-then-flock sequence on a real six-robot swarm.","feed_headline":"Skill graph lets robot swarms switch from combat to flocking","feed_subtitle":"One graph ranks learned skills, so combat and flocking live in one library with no shared-task assumptions.","key_machinery":"The skill graph is a knowledge graph whose entities are environments, tasks, and skills and whose relations are \"environment to skill\" and \"task to skill\". TransH embeds each relation as a hyperplane with a translation vector and scores a triple by $\\exp(-\\lambda \\| (h - w_r^\\top h w_r) + d_r - (b - w_r^\\top b w_r)\\|)$, supporting one-to-many and many-to-many relations. A query is scored by multiplying the task-skill and environment-skill scores; the score tier decides direct selection, weighted combination, or further training. The construction is independent of the low-level policies, which is what gives the high-level module its generalization and lets unrelated tasks share one library.","core_discovery":"The central discovery is that a knowledge-graph embedding of skills turns multi-task multi-agent learning into a retrieval-and-refinement problem. The paper builds a skill graph with environment, task, and skill entities and relations $e\\to s$ and $t\\to s$, learns their representations with TransH under a loss that mixes positive, negative, and soft samples, and then answers a query $(e_{\\mathrm{new}}, t_{\\mathrm{new}})$ by computing $S = S(t, r_{t\\to s}, s) \\cdot S(e, r_{e\\to s}, s)$ for every skill. Depending on where the score falls relative to thresholds $\\alpha_{\\mathrm{high}}$ and $\\alpha_{\\mathrm{low}}$, the controller chooses the top skill, combines the actions of top skills with normalized weights, or further trains the top skill with RL. This mechanism, with low-level policies trained by a local-critic version of MADDPG, is what lets adversarial and cooperative tasks live in the same skill library and lets new but similar tasks be solved faster than training from scratch.","pith_inferences":["Editorial extension: the score margin between the top-ranked and second-ranked skills could be used as a confidence signal for online mode switching, a use the paper does not explore.","Editorial extension: the combination step assumes that blending actions by normalized weights is safe; a natural stress test is blending an attack skill with a flocking skill and checking whether the mixed behavior preserves both objectives.","Editorial extension: the paper's own conclusion notes that totally unseen scenarios yield low scores and require new skills; this suggests the graph should be able to grow by inserting newly trained skills, turning one-shot adaptation into a lifelong learning process.","Editorial extension: because the ranking depends on hand-crafted features, a testable improvement is to learn task and environment features end-to-end from observations instead of specifying vectors like $(y, L)$ and $(v_{\\max}, v_{\\min}, \\Delta h, n_o, r_{\\mathrm{att}})$ by hand."],"forward_implications":["A team trained once on a library of flocking and combat skills can be reused for a new mission that mixes both; the skill graph supplies the right skill for each stage without retraining the high level.","When a new task resembles two stored skills, the robot action is a normalized weighted sum of those skills' actions, giving a cheap adaptation path that does not require new reinforcement learning.","When the new task falls outside the stored distribution, warm-starting from the top-ranked skill reaches a working policy in fewer episodes than training from scratch, as shown in the reward curves.","Because the graph's construction is independent of the low-level policies, changes to low-level parameters such as initial counts or leader speed do not degrade high-level decision success, whereas the hierarchical MAPPO baseline's success rate drops as the skill count grows and parameters vary.","The high-level decision layer never needs centralized observations; the low-level local-critic MADDPG handles 50-agent tasks, so the method is compatible with distributed robot swarms."],"supporting_citations":[{"why":"Brings the skill graph concept from single-agent quadruped skill learning; the paper reuses it as the high-level multi-task module.","marker":"[24]"},{"why":"Supplies the knowledge-graph definition and the rationale that graphs enable knowledge transfer, motivating the entity and relation design.","marker":"[25]"},{"why":"Provides the TransH hyperplane-translation embedding and its scoring function, the core of the skill-graph ranking mechanism.","marker":"[26]"},{"why":"Presents MADDPG, the low-level MARL algorithm the paper modifies to a local critic for collecting the skills stored in the graph.","marker":"[27]"},{"why":"The hierarchical MAPPO baseline whose decision success and generalization the skill graph is compared against.","marker":"[34]"}],"fun_headline_variants":["Skill graph lets agents reuse skills for unrelated tasks","Retrieve skills, not retrain: multi-task MARL via graph","One skill graph for combat and flocking in multi-agent RL","Skill graph retrieval accelerates multi-agent RL on new tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole ranking rests on the assumption that the hand-built feature vectors for environments and tasks make embedding-space distance track which skill will actually work.","fun_headline_variants_meta":{"raw":{"variants":["Skill graph lets agents reuse skills for unrelated tasks","Retrieve skills, not retrain: multi-task MARL via graph","One skill graph for combat and flocking in multi-agent RL","Skill graph retrieval accelerates multi-agent RL on new tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000768,"raw_usage":{"total_tokens":3399,"prompt_tokens":935,"completion_tokens":2464,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":2396}},"tokens_in":551,"tokens_out":2464,"duration_ms":18063,"temperature":1.0,"reasoning_tokens":2396,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:56:53.230165+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a pair of queries with identical feature vectors but different true dynamics, for example two tasks that share the reported boundary type, speed limits, attack radius, and perception range but differ in robot mass or in an unlisted environmental factor. If the skill graph still rates an old skill near 1 and the execution fails, the hand-built features are not sufficient to make embedding distance track real skill suitability, and the ranking mechanism would not transfer to unseen queries.","supporting_citations":[{"cited_title":"Unlock Reliable Skill Inference for Quadruped Adaptive Behavior by Skill Graph","cited_arxiv_id":"2311.06015","evidence_quote":"Brings the skill graph concept from single-agent quadruped skill learning; the paper reuses it as the high-level multi-task module."},{"cited_title":"A survey on knowledge graphs: Representation, acquisition, and applications,","cited_arxiv_id":null,"evidence_quote":"Supplies the knowledge-graph definition and the rationale that graphs enable knowledge transfer, motivating the entity and relation design."},{"cited_title":"Knowledge graph embedding by translating on hyperplanes,","cited_arxiv_id":null,"evidence_quote":"Provides the TransH hyperplane-translation embedding and its scoring function, the core of the skill-graph ranking mechanism."},{"cited_title":"Multi-agent actor-critic for mixed cooperative-competitive environments,","cited_arxiv_id":null,"evidence_quote":"Presents MADDPG, the low-level MARL algorithm the paper modifies to a local critic for collecting the skills stored in the graph."},{"cited_title":"The surprising effectiveness of PPO in cooperative multi- agent games,","cited_arxiv_id":null,"evidence_quote":"The hierarchical MAPPO baseline whose decision success and generalization the skill graph is compared against."}],"review_version":1}