Pith.

Artificial Intelligence · arXiv:2611.12109 v2 · submitted 9 Nov 2026

SkillDuet: Co-Evolving Models and Skills through Response Feedback

Xingguang Ji, Ziqiang Dong, Yubo Zhang, Xi Leng, Ninghua Yan, Yang Yang, Guanjun Jiang

The paper says

“During reinforcement learning, experience can be both internalized in LLM parameters and retained externally as reusable textual skills. Parameter updates alter the model's behavior and the interaction experience from which new skills are extracted.”

more of the abstract

These skills, in turn, guide the model's subsequent interactions and parameter updates. As the model's capabilities evolve, previously beneficial skills may become redundant or detrimental. We present SkillDuet, a framework that coordinates parameter updates and skill evolution through model response feedback. Skills extracted from on-policy trajectories guide parameter updates through OPD at every step. A response-based score, Strength, ranks candidate skills by the magnitude of the shift they induce in the current model's responses, and the highest-scoring skills are periodically consolidated into a persistent global skill set (Global). The updated model and Global jointly guide subsequent interactions, generating experience for further parameter and skill updates. With Qwen3-4B, SkillDuet achieves success rates of 98.57% on ALFWorld and 92.97% on WebShop, exceeding the matched SEED-API baseline by 2.86 and 5.47 percentage points, respectively. Ablations show that response-based skill selection and Global consolidation improve task performance. Further experiments show that the benefits of retained skills depend on both their content and their compatibility with the model. SkillDuet provides a concrete mechanism for model-skill co-evolution and skill-level recursive self-improvement.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 10 Nov 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. SkillDuet's core idea is sound and well-motivated, but the central performance claim rests on unreplicated training runs, which makes the reported gains uninterpretable without variance estimates. This line is wrong

3 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at Appendix A and Sections 4.1–4.2

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.75):

“Scope of the comparison The three evaluation seeds measure execution variability at these checkpoints, not variability across independent training runs.”

Pith says: Objection 1 of 3 · major

The central empirical claim—that SkillDuet outperforms SEED-API by 2.86 pp on ALFWorld and 5.47 pp on WebShop—rests on a single training run per configuration. The paper explicitly states (Appendix A, b492) that 'the three seeds in Tables 3 and 8 are repeated execution evaluations of fixed checkpoints, not independent training runs.'

more of objection 1

On environments where trajectory sampling, skill extraction, and synthesis introduce stochasticity, single-run differences can be large (e.g., the paper's own Figure 5 shows a 7.03 pp gap between WebShop Top25% and Top50% in a single run). Without repeated independent training runs (e.g., 3–5 seeds) and an estimate of training variance, the observed gaps cannot be distinguished from random variation. This is the most load-bearing weakness and must be addressed for the empirical conclusions to be reliable.

The paper says, at Appendix A, b332

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.82):

“The verified synthesis requests use the endpoint alias edu-vlm-rm, which does not identify an immutable backend weight version for all earlier experiments.”

Pith says: Objection 2 of 3 · major

The external analyzer endpoint (edu-vlm-rm) 'does not identify an immutable backend weight version for all earlier experiments.' Because both skill extraction and skill synthesis depend on this analyzer, uncontrolled changes in the backend during or across experiments could affect the content and quality of Local and Global skills, introducing a confound. The authors should either fix the analyzer version (by snapshotting the model or pinning a release) and report it, or demonstrate that the observed results are robust across versions.

The paper says, at Appendix A, b327, b413

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.89):

“Complete deployment records are unavailable for some earlier and original configurations.”

Pith says: Objection 3 of 3 · major

The paper notes that 'complete deployment records are unavailable for some earlier and original configurations' and that 'complete deployment and task-level probe records are unavailable.' This limits the reproducibility of the ablations and the controlled Global evaluations. For a journal-level contribution, the authors should provide full logs or a clearly documented release of the training and evaluation pipeline, including all configuration files, to allow independent verification.

5 minor objections

The paper says, at Equation (1) and surrounding text

Pith says: minor objection 1 of 5

The confidence-gate scale β is set to 5, but no sensitivity analysis is provided. A brief note or reference to prior work justifying this value would be helpful.

The paper says, at Table 2 and Figure 5

Pith says: minor objection 2 of 5

The 'Random Top50%' baseline in Figure 5 uses a different OPD coefficient than the SEED-API baseline (0.03 vs 0.05). Clarify whether the random selection results in Table 2 use the same coefficient as SkillDuet. If not, state this explicitly and discuss potential impacts.

show 3 more

The paper says, at Appendix A, Algorithm 1

Pith says: minor objection 3 of 5

Line 10 says 'Clear temporary windows for all Globals and start the next window.' It would be useful to state whether the clearing occurs before or after the synthesis step; the text suggests after, but the pseudocode could be more explicit.

The paper says, at Appendix C, historical pool experiment

Pith says: minor objection 4 of 5

The normalized AUC computation sums trapezoidal areas between steps 30, 60, ..., 300. Clarify whether the denominator is 270 or 300 (since the first point is at step 30). Currently it says 'divide by the 270-step interval,' but the interval from step 30 to step 300 is 270 steps; confirm this is correct.

The paper says, at General

Pith says: minor objection 5 of 5

The paper uses 'OPD' to refer to both the distillation objective and the overall OPD step; consider using a different term for the objective (e.g., L_OPD) to avoid ambiguity.

What would settle it

Run each of the five configurations from Tables1and2 (SkillDuet, SEED-API, Paired Local-only, Random Top50%, GRPO) with 5-10 different random training seeds (varying the random seed for trajectory sampling, skill extraction, and optimization). For each run, record peak validation success. Compute the mean and standard deviation per configuration. If the mean of SkillDuet exceeds SEED-API by more than two pooled standard deviations, the result is robust; other wise the reported advantage may be within noise.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 10 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 23 s · 20,650 tokens read, 2,731 written
  3. Editor wrote the verdict above 12 s · 21,529 tokens read, 1,348 written
  4. Referee major revision: 3 major objections, 5 minor 19 s · 21,064 tokens read, 2,052 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper introduces a clean extension to the SEED family: using response-shift magnitude (Strength) to select skills for a persistent global set, and coordinating this with on-policy distillation. The motivation (fixed skills lose value as the model learns) is well supported by the probe in Section 3.1. The ablation design is thoughtful—comparing Strength to reward weighting and random selection, testing retention ratios, and separating the effects of content versus compatibility. The writing is clear and the authors are upfront about limitations.

Here is the problem: the headline comparison to SEED-API (2.86 pp on ALFWorld, 5.47 on WebShop) comes from one training run per configuration. Appendix A says so explicitly. The three seeds in Table 3 measure execution variability at fixed checkpoints, not training variability. On these benchmarks, single-run differences of several percent are not unusual (their own Figure 5 shows a 7 pp gap between two selection ratios from single runs). Without repeated training runs, we cannot tell whether SkillDuet’s gains reflect the method or random seed luck. The authors acknowledge the scope of the comparison, but the paper’s central claim still leans on unreplicated results.

Minor concerns: the external analyzer endpoint (edu-vlm-rm) is not pinned to a version, which makes reproduction fragile. The GPU model is not fully documented. These are secondary next to the statistical issue.

This is an honest, well-framed piece of work. The idea is worth pursuing, and the ablations show the framework behaves as expected. But the main empirical conclusion is under-supported. The paper deserves peer review because the concept is novel and the experiments are otherwise careful—but the editor should ask for multiple training seeds and, ideally, a public release of code or detailed run scripts. With those, the paper could become a solid contribution.

The strongest objection

The central empirical claim—that SkillDuet outperforms SEED-API by 2.86pp on ALFWorld and 5.47pp on WebShop—is supported by exactly one training run per configuration (Appendix A: 'Each training configuration is represented by one training run'). The paper provides no estimate of training variance. The three evaluation seeds in Table3/8 measure only execution variability at fixed checkpoints, not training variability (Appendix F, b492). On environments like ALFWorld and WebShop, where trajectory sampling and skill extraction introduce stochasticity, individual runs can differ by several percentage points. Without repeated runs, the observed gaps could arise from chance rather than the framework. The Strength proxy concern (reader's weakest_assumption) is secondary because even if Strength were perfect, the insufficient statistical evidence would still undermine the claim. The paper's own ablations (Figure5) show WebShop Top25% achieving 85.94%, Top50% 92.97%, a gap of 7.03pp from a single run, illustrating that single-run differences can be large. The comparison to SEED-API is similarly single-run. Therefore, the most load-bearing concern is the absence of repeated independent training runs to estimate variance and confirm significance.

The paper's central empirical claim rests on single training runs. The reader's conditional verdict already flags this and requests multiple seeds (reader's rationale). Our analysis identifies the same experimental-design shortcoming as the most load-bearing concern because without variance estimates, the reported gains could be noise. However, we do not lowert the verdict to REJECT because the paper otherwise shows careful ablations and reasoned design; the CONDITIONAL verdict with the request for multiple seeds is appropriate. The Strength proxy concern is valid but secondary: even a perfect proxy would not rescue an experiment with no replications. Thus we recommend keeping the CONDITIONAL verdict, with the condition that the authors provide repeated runs with different seeds.

The referee's report

If the reported gains hold, SkillDuet provides a principled mechanism for co-evolving a policy and its external skill library during training, addressing the real observation that fixed skills lose utility as the policy changes. The idea of using the magnitude of response shift (rather than execution outcome) as a proxy for skill relevance is novel within the line of work on skill-augmented RL for LLM agents. The framework is presented clearly, and the ablation that isolates Strength from random selection is well-motivated. The work advances the broader goal of skill-level recursive self-improvement.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tries to establish that during reinforcement learning with large language models, the external textual skills used to guide behavior must be updated as the model itself changes. The authors propose SkillDuet, a framework that couples continuous parameter updates via on-policy distillation with periodic skill consolidation driven by a response-based score called Strength. The central claim is that this co-evolution loop outperforms methods that either keep skills fixed or update them without considering the current model's state. A sympathetic reader would care because it addresses a practical problem: skills that help early in training can become redundant or harmful later, and SkillDuet provides a concrete mechanism for deciding which skills to retain.

What carries the argument. The central mechanism is the Strength score, which ranks candidate Local skills by the average absolute magnitude of the log-probability shift they cause in the model's responses on held-out trajectories, using neutral text as a baseline. The highest-scoring skills from a ten-step window are consolidated into a persistent Global skill set via hierarchical synthesis. This Global set then guides subsequent interactions, while all eligible Local skills provide token-level on-policy distillation (OPD) supervision for parameter updates at every training step.

Core claim. SkillDuet achieves 98.57% success on ALFWorld and 92.97% on WebShop with Qwen3-4B, outperforming the matched SEED-API baseline by 2.86 and 5.47 percentage points respectively. The key claim is that this improvement comes from coordinating parameter updates (through on-policy distillation guided by trajectory-specific Local skills) with periodic consolidation of the most useful Local skills into a persistent Global skill set, where usefulness is measured by Strength—the magnitude of shift a skill induces in the current model's response probabilities on held-out trajectories. Ablations show that response-based selection beats random selection, and that Global consolidation adds value beyond Local-guided parameter updates alone.

The premise it rests on. The Strength score assumes that the magnitude of response shift a skill induces on held-out trajectories reliably indicates the skill's future usefulness for the evolving policy, even though the authors acknowledge that response shift does not directly measure benefit and that skills with high shift can still be detrimental.

If the paper is right (Pith's inferences, not the authors')

If SkillDuet's approach is correct, then the standard practice of maintaining a static library of skills during LLM agent training may be suboptimal; skills should be dynamically selected and synthesized as the policy evolves.

The Strength metric provides a low-cost proxy for skill utility that requires no additional environment interaction—only re-scoring already sampled responses—making it practical for large-scale training.

The co-evolution loop could be extended to multi-task or lifelong learning settings where the policy encounters new tasks over time, requiring continuous skill renewal.

The finding that a skill set trained for one model can harm another (Section 4.6) implies that skill-transfer methods must account for the recipient model's current capability state.

The paper's ablation showing that even sentence-shuffled skills provide some benefit for OPD suggests that the textual content's procedural structure is not the sole driver of distillation gains; the simple presence of relevant text may also help.

The SkillDuet architecture suggests a direct analogy to biological memory consolidation: Local skills correspond to short-term episodic memories, Strength scores to salience signals, and Global consolidation to long-term memory formation. This analogy could inspire neurobiologically grounded selection rules.

Because Strength relies on absolute response shift regardless of sign, it may inadvertently retain skills that push the policy in an undesirable direction. Future work could test a signed variant weighted by outcome advantage, though the paper's own experiments show Strength outperforms such variants in the current setting.

The paper demonstrates co-evolution on two interactive benchmarks with clear success criteria. Extending the framework to tasks with sparse or continuous rewards (e.g., open-ended dialogue or robotics) would test whether Strength remains a useful signal when outcome feedback is delayed or absent.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

During reinforcement learning, experience can be both internalized in LLM parameters and retained externally as reusable textual skills. Parameter updates alter the model's behavior and the interaction experience from which new skills are extracted. These skills, in turn, guide the model's subsequent interactions and parameter updates. As the model's capabilities evolve, previously beneficial skills may become redundant or detrimental. We present SkillDuet, a framework that coordinates parameter updates and skill evolution through model response feedback. Skills extracted from on-policy trajectories guide parameter updates through OPD at every step. A response-based score, Strength, ranks candidate skills by the magnitude of the shift they induce in the current model's responses, and the highest-scoring skills are periodically consolidated into a persistent global skill set (Global). The updated model and Global jointly guide subsequent interactions, generating experience for further parameter and skill updates. With Qwen3-4B, SkillDuet achieves success rates of 98.57% on ALFWorld and 92.97% on WebShop, exceeding the matched SEED-API baseline by 2.86 and 5.47 percentage points, respectively. Ablations show that response-based skill selection and Global consolidation improve task performance. Further experiments show that the benefits of retained skills depend on both their content and their compatibility with the model. SkillDuet provides a concrete mechanism for model-skill co-evolution and skill-level recursive self-improvement.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-10

The authors' main claim, as the classifier picked it out 0.88

“SkillDuet provides a concrete mechanism for model-skill co-evolution and skill-level recursive self-improvement.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.53 · human participants 0.05 · controlled comparison 0.96 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
During reinforcement learning, experience can be both internalized in LLM parameters and retained externally as reusable textual skills. background 1.00 0.11 0.04 asserted 0.96
Parameter updates alter the model's behavior and the interaction experience from which new skills are extracted. background 0.98 0.16 0.03 asserted 1.00
These skills, in turn, guide the model's subsequent interactions and parameter updates. background 0.96 0.14 0.05 asserted 1.00
As the model's capabilities evolve, previously beneficial skills may become redundant or detrimental. background 0.89 0.11 0.04 hedged possibility 0.98
We present SkillDuet, a framework that coordinates parameter updates and skill evolution through model response feedback. contribution 0.99 0.89 0.14 asserted 1.00
Skills extracted from on-policy trajectories guide parameter updates through OPD at every step. method 0.99 0.82 0.07 asserted 1.00
A response-based score, Strength, ranks candidate skills by the magnitude of the shift they induce in the current model's responses, and the highest-scoring skills are periodically consolidated into a persistent global skill set (Global). method 1.00 0.92 0.08 asserted 1.00
The updated model and Global jointly guide subsequent interactions, generating experience for further parameter and skill updates. method 0.99 0.67 0.08 asserted 1.00
With Qwen3-4B, SkillDuet achieves success rates of 98.57% on ALFWorld and 92.97% on WebShop, exceeding the matched SEED-API baseline by 2.86 and 5.47 percentage points, respectively. result 1.00 0.96 0.24 asserted 1.00
Ablations show that response-based skill selection and Global consolidation improve task performance. result 0.99 0.95 0.14 asserted 1.00
Further experiments show that the benefits of retained skills depend on both their content and their compatibility with the model. result 0.84 0.94 0.12 asserted 1.00
SkillDuet provides a concrete mechanism for model-skill co-evolution and skill-level recursive self-improvement. interpretation 0.81 0.70 0.59 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-10: 1 request, 8,734 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (8)
Figure 6 of the paper
Figure 6. Optional execution-based acceptance checks for Global. Direct deployment and Commit32 are compared at matched retention ratios. At Top50% retention, direct deployment outperforms Commit32 on both benchmarks. Commit controls Global deployment without reversing Local-guided policy updates. Appendix E. Detailed results at the selected checkpoints For ALFWorld, we first compute each category’s success rate by pooling its evaluation trajectories, then take the unweighted mean of the six rates to obtain Avg. ALL is the fraction of successful trajectories across the full evaluation set.…
Figure 7 of the paper
Figure 7. Effect of removing Global from SkillDuet on ALFWorld, with model parameters fixed. The largest gains from Global occur on Clean, Heat, and Cool. Paired action-sequence example The Clean example in Section 4.6 is the first pair in which learned Global succeeds and removal fails, ordered by eval￾uation seed, batch, and slot (0/0/0; ButterKnife–DiningTable). With Global, the agent follows the clean-before-place procedure and completes the task in 7 actions. Without Global, it reaches the 30-action limit without completing the task. Full trajectories, prompts, and deployment…
Figure 8 of the paper
Figure 8. Hierarchical skill consolidation on WebShop. Thirty-seven selected Local skills produce fourteen first-layer and eight second-layer summaries, which are combined with five existing Global rules to yield five updated rules. The update uses eight requests, including one repair. Deployment at step 21 and reuse as the prior Global at step 31 are verified in the records. The updated skill set guides subsequent interactions and further consolidation. Skill excerpts are paraphrased. 22

Showing 3 of the paper's 8 figures. Figures are the authors', reproduced from the paper.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment