{"id":"9b637577-60dc-48ba-9e07-41046cb75f61","arxiv_id":"2608.05970","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"SkillMemo couples MoE-based skill discovery with episodic memory retrieval and reports consistent success-rate gains on diffusion and VLA policies for simulated and real manipulation tasks.","lead":"The paper presents SkillMemo, an add-on that lets robot manipulation policies split demonstrations into latent skills and recall them from an episodic memory during inference; on the LIBERO benchmark it reports a 98.0% success rate when attached to π0.5, versus 96.8% for π0.5 alone. A generalist might read it because it tests whether cheap, reusable skill memories can substitute for larger robot datasets, which matters for real-world robotics.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (7)'s temporal alignment t' is undefined and the gating-to-segment partition behind memory values is unspecified, so the memory-fusion mechanism that drives the reported gains cannot be implemented as written.","rationale":"The reader's weakest assumption correctly identifies the undefined temporal alignment t' in Eq. (7) as the most load-bearing gap. I agree with that diagnosis, and I extend it to the closely related fact that the memory values v_n in Eq. (5) are defined over 'skill segments' whose extraction from the MoE gating time series is never specified. Both omissions concern the exact same mechanism: the paper claims that retrieved expert activation profiles are fused into the current gating distribution, but no reproducible rule is given for either segment formation or temporal alignment, so the fusion term cannot be computed from the text alone. This concern is not an attack on the empirical results; the reported gains could be real. But the paper provides no code, no checkpoints, no multi-seed statistics, and the expert count in Table 2 is selected on the evaluation benchmark, so there is no independent way to determine whether the SLMA component, rather than hyperparameter selection or an implicit alignment heuristic, produces the improvements. Because the reader's conditional verdict already requires the authors to specify the alignment and segmentation procedures, my read does not move the verdict; it reinforces the same conditions. A direct reimplementation with and without specified alignments would settle whether the concern is fatal or merely cosmetic, and that is the check I recommend.","tokens_in":10685,"tokens_out":5123,"duration_ms":54174,"concrete_test":"Request the authors' exact specification of (a) how the MoE gating time series is cut into skill segments used as memory values in Eq. (5) and (b) the full alignment map from current timestep t to the retrieved index t' in Eq. (7), then re-run the π0.5-SkillMemo LIBERO experiment from scratch with those specifications. As a control, run the same experiment with three plausible alignments (t' set to the segment start, the segment end, and the timestep whose stored hidden feature is nearest to the current hidden feature); if the average LIBERO success varies by more than 0.5% across these choices, the reported 98.0% figure is not well-defined by the paper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SkillMemo's skill-level memory fusion improves compositional generalization. The operative equation is Eq. (7), g'(q_t) = λg(q_t) + (1-λ)(1/N)Σ v_n[t']. The index t' is described only as 'the aligned timestep within the retrieved skill sequence', and no alignment rule is given anywhere in the paper or appendices. The problem is compounded by Eq. (5): the stored value v_n is the sequence of gating weights over a skill segment, but the paper never specifies how the continuous MoE gating time series is partitioned into discrete segments. Eq. (1) gives an explicit segmentation that the method explicitly replaces, and no implicit boundary-detection or thresholding procedure is provided. Because g'(q_t) is the only mechanism by which memory enters the policy, an unspecified t' and unspecified segment boundaries make the SLMA results in Tables 1, 4, and 5 irreproducible in a non-arbitrary way. This is a load-bearing omission, not a stylistic one: without a well-defined fusion step, the reported improvements cannot be attributed to the claimed compositional memory mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SkillMemo, a skill-level memory framework for robotic manipulation policies. It introduces an MoE-based trajectory segmentation module (EGTS) that implicitly partitions demonstrations into latent atomic skills via gating coefficients, a skill-level episodic memory bank that stores centroid keys and gating-sequence values, and an inference-time fusion mechanism that retrieves and averages stored gating profiles into the policy's current gating distribution. The authors evaluate SkillMemo with Diffusion Policy and VLA backbones (UniAct, UniVLA, π0.5) on LIBERO, Push-T, BlockPush, Kitchen, and real-world UR5e tasks, reporting consistent improvements and a headline 98.0% average success on LIBERO for π0.5-SkillMemo versus 96.8% for π0.5.","tokens_in":10957,"tokens_out":6668,"duration_ms":67821,"significance":"Should the mechanism be fully specified and the results reproducible, SkillMemo would be a useful plug-in skill-retrieval module for both DP and VLA policies. The paper's strengths include its breadth of backbones, the inclusion of zero-shot cross-suite and real-world compositional generalization experiments, and explicit component ablations. The claim that memory of gating profiles improves compositional generalization is falsifiable and not guaranteed by construction: no computed quantity reduces directly to a fitted parameter. However, the key memory-fusion equations are under-specified as written, and all quantitative claims lack variance estimates, which substantially weakens the evidence for the headline improvements.","major_comments":[{"comment":"The index t′ in Eq. (7) is never defined. The phrase 'vn[t′] corresponds to the aligned timestep within the retrieved skill sequence' is not an alignment rule. Because g′(qt) is the only pathway through which memory enters the policy, without a concrete rule mapping the current rollout time to a position inside each retrieved skill sequence, the memory-fusion step has no grounded meaning and the reported memory gains in Tables 1, 4, and 5 cannot be reproduced. Please specify the alignment procedure (e.g., relative progress, dynamic time warping, or a learned time embedding) and state how it handles retrieved skills whose durations differ from the current skill.","section":"Section 3.2, Eq. (7)"},{"comment":"The paper claims implicit trajectory segmentation but never defines the mapping from the gating time series to the stored segments τm. Eq. (1) defines fixed-length non-overlapping segments with chunk size L, yet Section 3.1 states that explicit segmentation is replaced by implicit boundary detection, and no boundary-detection or thresholding procedure is given. Eq. (4) divides by L and Eq. (5) stores 'the complete sequence of gating weights across all time steps within the segment,' which presupposes a well-defined segment of known length. Please state how boundaries are detected from g(ht), whether segments are fixed-length or variable-length, how L is chosen, and how the memory values in Eq. (5) are constructed from the implicit partition.","section":"Sections 3.1–3.2, Eqs. (1), (4), (5)"},{"comment":"All success rates are reported as point estimates with no error bars, seed counts, or significance tests. The headline result (π0.5-SkillMemo 98.0% vs π0.5 96.8%) is a 1.2% difference based on 50 rollouts per task across four LIBERO suites; without variance or multiple seeds this could be within sampling noise. Please report mean ± std across at least three seeds for the simulation benchmarks and confidence intervals for the 40-trial real-world results.","section":"Sections 4.2–4.4, Tables 1–7"},{"comment":"The LPID regularization term is not operationalized. The notation I({A,B};G) and the mutual-information terms involving high-dimensional expert/feature variables are not defined, no estimator is provided, and this loss is not ablated separately from the rest of the EGTS module. As written, this component of the method cannot be reimplemented. Please specify the exact variables, the mutual-information estimator used, and an ablation isolating the contribution of LPID.","section":"Section 3.1, Eq. (3)"}],"minor_comments":[{"comment":"The row and column labels of Table 4 are ambiguous: the rows are training suites, but the caption reads as if they are datasets. Please make the 'trained on / evaluated on' structure explicit.","section":"Table 4"},{"comment":"The text says the method 'outperforms π0.5'; more precisely, SkillMemo is applied on top of π0.5, so the comparison is π0.5-SkillMemo versus π0.5. Please clarify this wording.","section":"Abstract and Section 1"},{"comment":"The hyperparameter list omits the memory fusion weight λ, the reliability threshold δ, and the retrieval top-N value used in the experiments. Please add these values and describe how they were selected.","section":"Appendix A.1"},{"comment":"The inference-time measurements in Table 2 are reported without standard deviations or a description of how they were aggregated across runs. Please specify the measurement procedure.","section":"Table 2"},{"comment":"The caption says 'Marks with !' but the table uses exclamation marks; please use a more descriptive marker and explain in the caption what each mark denotes.","section":"Table 7 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable engineering contribution, but the missing alignment rule for Eq. (7) and the missing segment-boundary definition are serious reproducibility gaps. I would encourage the editor to request a rewritten method section with explicit definitions of the alignment and segmentation procedures, along with variance/seed reporting. If those are supplied, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea here is genuinely worth a look: store MoE gating sequences as skill values in an episodic memory bank and average them into the current routing distribution at inference. That specific mechanism is not in MemoryVLA, HAMLET, or Sparse Diffusion Policy, and the paper earns credit for trying it on both DP and VLA backbones, with real-world validation on the DP side. The PID-based regularization is a nice touch. The empirical sweep is broad, even if the headline gains over π0.5 are modest (1.2% average) and not uniform across LIBERO suites (Object is flat).\n\nThe soft spots are real, and one is load-bearing. The stress-test note is right: Eq. (7) uses an alignment index t' that is never defined, and Eq. (5) never specifies how the continuous gating time series is partitioned into the skill segments that become memory values. Since g'(q_t) is the only mechanism by which memory affects the policy, an undefined t' and undefined segment boundaries mean the SLMA results in Tables 1, 4, and 5 cannot be reproduced from the paper as written. This is not a stylistic gap; it is the core of the method. The authors need to state how t' is computed (e.g., relative position, dynamic time warping, or something else) and how segments are extracted from gating profiles (threshold? change-point detection?). Given that Eq. (1) offers an explicit segmentation that the method explicitly replaces, the omission is glaring.\n\nMinor but not trivial: there are no error bars or seed counts anywhere, so the 1–3% gains could be within run-to-run noise. The expert count N=5 is selected on the evaluation benchmark itself (Table 2), inflating apparent performance. The memory update/pruning mechanism is described but never ablated; it may simply be doing nothing. The real-world evaluation covers only the DP backbone, so the VLA story is entirely simulated.\n\nWho gets value from this? Researchers working on memory-augmented policies and skill discovery will find the architecture schematic appealing and the empirical comparisons informative. It deserves a serious referee, but a major revision is needed: define the alignment and segmentation procedures, add multi-seed statistics, and release code or checkpoints. I would not cite it in its current form, since the central mechanism cannot be built upon. Send it to peer review with a clear request for these specifics, not a desk reject.","headline":"A plausible memory-augmented skill framework whose central fusion equation is under-specified; needs a major revision before the reported gains can be trusted.","tokens_in":11494,"tokens_out":1534,"would_cite":false,"duration_ms":18323,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SkillMemo claims that a skill-level episodic memory lets visuomotor policies recombine known movement primitives, lifting zero-shot success on unseen task compositions across DP and VLA backbones.","keywords":["skill learning","memory-augmented policy","robot manipulation","mixture-of-experts","compositional generalization","vision-language-action models","diffusion policy","episodic memory"],"falsifier":"Run LIBERO with the retrieved gating sequences in Eq. (7) replaced by randomly selected time steps within each segment; if the 98.0% success rate survives, the temporal alignment is irrelevant, and if it drops, the unpublished alignment rule is carrying the reported gain.","tokens_in":10482,"feed_emoji":"🤖","tokens_out":5020,"duration_ms":45687,"temperature":0.7,"pith_summary":"SkillMemo is a framework for adding a skill-level memory to robotic manipulation policies. The paper claims that by implicitly splitting demonstrations into latent atomic skills with a Mixture-of-Experts routing module and storing those skills as retrievable gating profiles in an episodic memory bank, both Diffusion Policy and Vision-Language-Action models improve on LIBERO and on real-world tasks. The headline result is that augmenting the π0.5 VLA with SkillMemo reaches 98.0% average success versus 96.8% for π0.5, and that models trained on one LIBERO suite generalize zero-shot to unseen suites. The reason a reader might care is that this offers a route to compositional generalization from limited demonstrations: recombining known skills instead of memorizing whole trajectories.","feed_headline":"Skill memory lifts robot success on LIBERO to 98%","feed_subtitle":"Recombining stored skill primitives beats π0.5 and helps diffusion policies on unseen tasks.","key_machinery":"The load-bearing mechanism is the skill-level episodic memory built on Mixture-of-Experts gating. The MoE layer's gating coefficients over K experts act as an implicit, label-free segmentation of a trajectory into atomic skill primitives; a memory bank stores each discovered segment as a key-value pair, with the segment's mean latent feature as the key and the sequence of gating weights as the value. During inference the bank retrieves the most similar keys and fuses the retrieved gating sequences into the current gating distribution via a weighted average (Eq. 7), using an alignment index t' that maps the current time step into each retrieved sequence. This fusion is what the paper claims turns stored experience into a contextual prior for action prediction.","core_discovery":"The paper's central claim is that long-horizon manipulation demonstrations can be decomposed, without any action labels, into reusable atomic skills, and that storing these skills in an episodic memory bank lets a policy synthesize unseen behaviors by recombining them. The decomposition is done implicitly by a Mixture-of-Experts layer whose gating weights over five experts vary sharply at semantic boundaries such as grasping or turning on a stove. Each skill segment is stored as a key-value pair: the key is the mean latent feature of the segment, and the value is the full sequence of gating weights over the segment. At inference, the current observation's latent feature retrieves the top-N most similar keys by cosine similarity, and if the average similarity exceeds a threshold, the retrieved gating profiles are averaged into the current gating distribution. The paper reports that this consistently improves several backbones, including Diffusion Policy, UniAct, UniVLA, and π0.5, and that the resulting model outperforms π0.5 on LIBERO (98.0% vs 96.8%).","pith_inferences":["The alignment index t' in Eq. (7) is never defined, so the core fusion equation is underspecified; the reported gains may depend on an unpublished alignment heuristic.","If the memory bank is simply retrieving near-duplicate gating profiles, the 'compositional' claim reduces to nearest-neighbor smoothing; a diagnostic would be to compare against retrieving random segments from the same task.","The real-world gains are shown only on a single UR5e setup with 50 demos per task; scaling to more diverse demonstrations and tasks would test whether the memory bank remains useful.","The PID-based synergy loss is presented as a regularizer, but its contribution is not isolated in the ablations; ablating it would show whether expert specialization comes from the loss or from the MoE architecture itself."],"forward_implications":["Both diffusion-based and VLA-based policies are claimed to improve when the SkillMemo memory is added, so the mechanism is presented as backbone-agnostic.","Models trained on one LIBERO suite show zero-shot gains on unseen suites; the Goal-trained model reaches 75.6% on Spatial, above the in-distribution Spatial baseline's 72.1%.","On real-world unseen object-receptacle combinations, success rates improve by up to 12.5 percentage points over the DP baseline, e.g., Strawberry on Plate from 65.0% to 75.0%.","The dynamic pruning strategy (timestamp, usage, and reliability-based removal) is claimed to keep the finite-capacity memory bank compact and relevant for lifelong learning.","Increasing expert count from 1 to 10 steadily improves accuracy with diminishing returns beyond N=5, which the paper sets as the default."],"supporting_citations":[{"why":"Diffusion Policy: the DP backbone and baseline that SkillMemo enhances.","marker":"[1]"},{"why":"π0.5: the strongest VLA baseline that SkillMemo claims to outperform (98.0 vs 96.8).","marker":"[7]"},{"why":"MemoryVLA: closest memory-based VLA baseline, used to argue skill-level memory improves on holistic memory.","marker":"[18]"},{"why":"UniAct: a 0.5B VLA backbone used in ablations, showing component-wise gains.","marker":"[15]"},{"why":"UniVLA: an 8.5B VLA backbone enhanced by SkillMemo in the LIBERO comparison.","marker":"[6]"},{"why":"LIBERO: the simulation benchmark providing the main evaluation suites and cross-suite protocol.","marker":"[34]"},{"why":"OpenVLA: the VLA evaluation setup and comparison baseline.","marker":"[8]"},{"why":"Sparse Diffusion Policy: a baseline and a prior use of MoE-like sparse activation in diffusion policies.","marker":"[2]"}],"fun_headline_variants":["Skill memory recombines latent primitives for unseen tasks","Implicit skill decomposition hits 98% on LIBERO","Stored skill keys beat π0.5 in compositional manipulation","Expert-guided memory boosts OOD robot generalization","Episodic skill bank reuses primitives to solve new tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fusion step in Eq. (7) assumes the current time step can be aligned to a specific time step t' inside each retrieved skill sequence, but the paper never states how that alignment is computed.","fun_headline_variants_meta":{"raw":{"variants":["Skill memory recombines latent primitives for unseen tasks","Implicit skill decomposition hits 98% on LIBERO","Stored skill keys beat π0.5 in compositional manipulation","Expert-guided memory boosts OOD robot generalization","Episodic skill bank reuses primitives to solve new tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1397,"prompt_tokens":997,"completion_tokens":400,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":319}},"tokens_in":613,"tokens_out":400,"duration_ms":4133,"temperature":1.0,"reasoning_tokens":319,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:04:40.698817+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LIBERO with the retrieved gating sequences in Eq. (7) replaced by randomly selected time steps within each segment; if the 98.0% success rate survives, the temporal alignment is irrelevant, and if it drops, the unpublished alignment rule is carrying the reported gain.","supporting_citations":[{"cited_title":"Zheng, J","cited_arxiv_id":null,"evidence_quote":"UniAct: a 0.5B VLA backbone used in ablations, showing component-wise gains."}],"review_version":1}