Pith.

Software Engineering · arXiv:2607.03238 v1 · submitted 3 Jul 2026 · 40 references

An Empirical Study of Downstream Adaptation for Agent Skills

Xinjian Wu, Jingzhi Gong, Gunel Jahangirova, Zhenpeng Chen, Jie M. Zhang

The paper says

“As Large Language Model (LLM) agents become integral to modern software systems, “skills” have emerged as a novel unit of software reuse, enabling developers to package workflows, decision procedures, and prompt-based policies.”

more of the abstract

While skills are intended for reuse, downstream developers frequently modify published skills to fit local contexts, yet little is known about the nature of such adaptations. This paper presents the first empirical study of downstream skill adaptation in public forks, to understand how published skills are adapted, and to provide implications for researchers and engineers on improving skill design, evolution, and orchestration. Specifically, we analyze 1,126 skill-adaptation instances from six widely adopted skill repositories and develop a taxonomy comprising 46 adaptation patterns organized into 13 families. Our key findings reveal a reuse paradox: although skills are intended to be easily imported and reused, developers spend a lot of effort rewriting what the skills do, fixing skill discoverability, and translating them for different tools and languages, indicating a need for better abstractions, standardized interfaces, and automated support for skill adaptation. Furthermore, adaptations are highly interdependent, with changes in one component often requiring coordinated updates elsewhere, motivating automated support for detecting inconsistent modifications. We also find that nearly one-fifth of adaptations introduce security-sensitive content within the same instruction text that governs behavior.

from the abstract

Pith says

Machine review · grok-4.5 · 12 Jul 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Solid first MSR map of how people actually rewrite agent skills in public forks; the reuse paradox and NL security surface are real within that slice, selection bias is the main limit. This line is wrong

2 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 40 references say.

The paper says, at Section III-A, Table IV, Tables VIII–XII

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

Section III-A and Table IV: obra/superpowers alone supplies 61.6% of the 1,126 instances. Repository-adjusted prevalence and upstream coverage are reported, but the headline reuse-paradox rates (lifecycle 40.9%, procedure 40.8%, procedure+decision 18.7%, security-sensitive 18.6%) and the co-occurrence lifts in Table XI are not shown with leave-one-out or per-repository breakdowns.

more of objection 1

Because these quantities underwrite the abstract’s central claims, a short sensitivity table (or appendix) excluding the dominant repository is needed so readers can judge whether the paradox and multi-family coupling are ecosystem-wide or community-specific.

The paper says, at Section III-F, Section IV-D, Table XII

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

Section III-F and RQ4 (Section IV-D): the 18.6% security-sensitive rate is produced by regex rules adapted from a code-oriented catalog (Liu et al.), applied to unified diffs that are predominantly natural-language SKILL.md/docs (73.1% of matches). The authors correctly frame matches as review surfaces rather than confirmed exploits, and they already restricted the token rule.

more of objection 2

Still, without a manual audit of a sample of the 209 added-line instances (precision/false-positive estimate, stratified by rule and file type), the quantitative security claim that anchors the abstract and implications remains only weakly calibrated. A modest audited subsample would make the attack-surface argument load-bearing rather than suggestive.

6 minor objections

The paper says, at Abstract, Section I, RQ1 summary

Pith says: minor objection 1 of 6

The phrase “reuse paradox” is effective but slightly overstated relative to classical software-reuse literature (libraries and frameworks routinely require local adaptation). A short paragraph situating the finding against fork/adaptation studies of conventional libraries would sharpen the novelty claim without weakening it.

The paper says, at Section V-C

Pith says: minor objection 2 of 6

RADAR is presented as an implication-driven checklist, not an evaluated method. State explicitly in Section V-C that it is unvalidated guidance so readers do not treat it as an empirical contribution.

show 4 more

The paper says, at Figure 3, Section IV-E

Pith says: minor objection 3 of 6

Figure 3 (Why-category × family lift heatmap) is dense; adding a brief caption note on how multi-label Why categories affect cell denominators would aid interpretation.

The paper says, at Table VII, Section III-B

Pith says: minor objection 4 of 6

Table VII pattern descriptions are useful but very terse; releasing the full decision rules (already promised in the artifact) is essential—consider pointing more prominently to them in the main text.

The paper says, at Section III-B, Section VI-d

Pith says: minor objection 5 of 6

Minor wording: “claude-opus-4-8” and “Claude Code CLI” should be cited with model/version provenance as available; the threats section already notes missing snapshot/temperature controls.

The paper says, at Abstract

Pith says: minor objection 6 of 6

In the abstract, “nearly one-fifth” and “18.6%” both appear later; keep the abstract figure consistent with the added-line definition used in RQ4.

What would settle it

Recompute headline prevalences (lifecycle 40.9%, procedure 40.8%, security-sensitive 18.6%) after excluding obra/superpowers (61.6% of instances) and re-weighting the remaining five repos equally; if any shifts >10 percentage points or the top-pattern ranking changes, the reuse-paradox framing weakens even for the public-fork population.

from the skeptic's reading

How the review went

4 readings by grok-4.5 · 12 Jul 2026

  1. Reader holds up, with conditions confidence high
  2. Skeptic agrees 5 s · 17,679 tokens read, 506 written
  3. Editor wrote the verdict above 6 s · 18,587 tokens read, 550 written
  4. Referee minor revision: 2 major objections, 6 minor 19 s · 17,996 tokens read, 1,219 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This is the first careful look at how published agent skills get rewritten after adoption, and that alone makes it worth your time if you care about agent packaging or skill marketplaces.

What is new is the object of study: not skill creation or retrieval, but downstream adaptation of existing packages. They mine public forks of six high-star skill repos, keep only unique in-skill changes against the upstream default branch, and end up with 1,126 labeled instances. The taxonomy (46 patterns / 13 families) is induced, saturated, and dual-audited on 318 records with strong agreement (κ ~0.83–0.85). The headline numbers hold inside that corpus: metadata/discoverability edits, procedure/decision/policy rewrites dominate; multi-family bundles are common; ~18.6% of instances add security-sensitive content, mostly in SKILL.md rather than scripts; commit messages capture functional outcome better than the underlying need. They ship the corpus, prompts, audit materials, and scripts. That is real empirical work, not a thin case study.

Soft spots are the ones the paper already flags. One repo is 61.6% of instances; the seed set is six popular public GitHub projects after star/fork thresholds. Public forks are not private reuse, abandoned forks, or non-GitHub practice. Security findings are regex surface matches, not confirmed exploits—they say so. Labeling is LLM-first with partial human audit; residual error is bounded but not zero. None of that breaks the descriptive claims inside the studied slice; it just means you should not treat the prevalences as ecosystem-wide rates.

Citation pattern is appropriate: skill surveys, MSR fork work, prompt evolution, and the Liu et al. security catalog they adapt. No circular math; this is observational taxonomy work.

Who it is for: people building skill formats, harnesses, marketplaces, or security review for agent instructions. If that is your lane, read it and cite the taxonomy and the reuse-paradox framing. It deserves a serious referee, not a desk reject—standard MSR contribution with clear methods and released artifacts. I would bring it to reading group and expect useful discussion on selection and on whether SKILL.md should become a first-class review surface.

The strongest objection

The paper's central descriptive claim (reuse paradox via high rates of metadata/procedure/decision/policy rewrites, multi-family co-occurrence, and 18.6% security-sensitive additions mostly in SKILL.md) is supported by a large labeled corpus, dual-auditor validation (κ≈0.83–0.85), repository-adjusted metrics, and released artifacts. The reader's weakest assumption—that six highly starred public forks (one at 61.6%) generalize to all skill adaptation—is already the primary external-validity limit (Threats VI-c, Table IV). No deeper internal inconsistency or hidden assumption undermines the claim *within* that slice: the taxonomy is inductively saturated, security rules are explicitly surface-matching not confirmed exploits, and commit-message incompleteness is measured rather than assumed. The argument does not overclaim ecosystem-wide rates once the CONDITIONAL framing is kept.

The reader's CONDITIONAL verdict already correctly isolates selection toward popular public forks (and residual LLM-labeling uncertainty) as the main limit while treating the descriptive taxonomy, co-occurrence, and security-surface findings as solid within the studied slice. My stress-test finds no additional load-bearing flaw—e.g., no circular definition of 'adaptation,' no unvalidated security overclaim, no failure of the dual-auditor process—that would move the verdict to REJECT or require a stronger downgrade. The concrete leave-one-repo-out check is the natural sensitivity test already partially anticipated by the paper's adjusted-prevalence metric; if it holds, confidence in the public-fork characterization rises further without changing the CONDITIONAL stance on generalization. Agreement with the reader is therefore full on both the strongest claim and the weakest assumption.

The referee's report

The work is timely and fills a clear gap: prior skill literature focuses on representation, acquisition, retrieval, and upstream evolution, not systematic downstream modification. Strengths include a carefully filtered MSR pipeline (trivial-fork removal, unique-commit filtering, in-skill retention), inductive taxonomy with saturation, dual-auditor reconciliation on 318 records with strong agreement (κ≈0.83–0.85, α≈0.87), Wilson intervals, repository-adjusted prevalence, and a full replication package (corpus, prompts, audit materials, scripts). The descriptive findings—metadata/procedure rewrites, coupled behavioral families, and security-sensitive content embedded in instruction text—are concrete and actionable for skill format designers, harness engineers, and security researchers. If the rates hold within the studied public-fork slice, the paper usefully reframes skills as living, multi-surface artifacts rather than plug-and-play packages.

Pith's reading of the paper (a machine's summary, not the authors')

Agent skills package reusable workflows, decision rules, and policies so large-language-model agents can load them instead of rebuilding behavior from scratch. This paper shows that, in practice, published skills rarely transfer cleanly. Mining 1,126 public-fork adaptations across six popular skill repositories, the authors find developers systematically rewrite how skills are discovered, what steps they run, and what constraints they enforce, and that these changes tend to travel together. They also find that nearly one-fifth of adaptations newly introduce security-sensitive content, mostly inside the natural-language instruction file rather than executable scripts. A sympathetic reader cares because skills are already being treated like libraries for agents; if reuse routinely requires heavy rewriting and injects invisible security surfaces, skill formats, tooling, and review practice need to change.

What carries the argument. A validated two-level taxonomy of 46 adaptation patterns in 13 families (lifecycle, procedure, decision, policy, retarget, resource, script, and others), applied to 1,126 labeled skill-adaptation instances from public GitHub forks and measured by prevalence, co-occurrence lift, component location, and security-sensitive patch matches.

Core claim. The first large-scale look at how published agent skills are adapted after adoption shows a reuse paradox: skills meant for plug-and-play import are repeatedly rewritten for discoverability, procedure, decision logic, and policy, with adaptations tightly coupled across families, and 18.6% of adaptation instances newly adding security-sensitive content largely inside SKILL.md and documentation rather than code.

The premise it rests on. The central rates and patterns are assumed to generalize from public forks of six highly starred skill repositories, even though one repository alone supplies most instances and private or non-GitHub reuse is invisible.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by grok-4.5, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

As Large Language Model (LLM) agents become integral to modern software systems, “skills” have emerged as a novel unit of software reuse, enabling developers to package workflows, decision procedures, and prompt-based policies. While skills are intended for reuse, downstream developers frequently modify published skills to fit local contexts, yet little is known about the nature of such adaptations. This paper presents the first empirical study of downstream skill adaptation in public forks, to understand how published skills are adapted, and to provide implications for researchers and engineers on improving skill design, evolution, and orchestration. Specifically, we analyze 1,126 skill-adaptation instances from six widely adopted skill repositories and develop a taxonomy comprising 46 adaptation patterns organized into 13 families. Our key findings reveal a reuse paradox: although skills are intended to be easily imported and reused, developers spend a lot of effort rewriting what the skills do, fixing skill discoverability, and translating them for different tools and languages, indicating a need for better abstractions, standardized interfaces, and automated support for skill adaptation. Furthermore, adaptations are highly interdependent, with changes in one component often requiring coordinated updates elsewhere, motivating automated support for detecting inconsistent modifications. We also find that nearly one-fifth of adaptations introduce security-sensitive content within the same instruction text that governs behavior.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 1.00

“Our key findings reveal a reuse paradox: although skills are intended to be easily imported and reused, developers spend a lot of effort rewriting what the skills do, fixing skill discoverability, and translating them for different tools and languages, indicating a need for better abstractions, standardized interfaces, and automated support for skill adaptation.”

How the main claim is stated: asserted 0.79

What the abstract itself states, as probabilities: claims first 0.97 · states limitation 0.17 · human participants 0.12 · controlled comparison 0.03 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.06 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
As Large Language Model (LLM) agents become integral to modern software systems, ``skills'' have emerged as a novel unit of software reuse, enabling developers to package workflows, decision procedures, and prompt-based policies. background 1.00 0.04 0.02 asserted 1.00
While skills are intended for reuse, downstream developers frequently modify published skills to fit local contexts, yet little is known about the nature of such adaptations. gap 0.98 0.05 0.02 asserted 0.99
This paper presents the first empirical study of downstream skill adaptation in public forks, to understand how published skills are adapted, and to provide implications for researchers and engineers on improving skill design, evolution, and orchestration. contribution 1.00 0.86 0.07 asserted 0.98
Specifically, we analyze 1,126 skill-adaptation instances from six widely adopted skill repositories and develop a taxonomy comprising 46 adaptation patterns organized into 13 families. method 0.81 0.97 0.05 asserted 1.00
Our key findings reveal a reuse paradox: although skills are intended to be easily imported and reused, developers spend a lot of effort rewriting what the skills do, fixing skill discoverability, and translating them for different tools and languages, indicating a need for better abstractions, standardized interfaces, and automated support for skill adaptation. interpretation 0.51 0.95 0.80 asserted 0.79
Furthermore, adaptations are highly interdependent, with changes in one component often requiring coordinated updates elsewhere, motivating automated support for detecting inconsistent modifications. result 0.80 0.91 0.07 asserted 0.99
We also find that nearly one-fifth of adaptations introduce security-sensitive content within the same instruction text that governs behavior. result 1.00 0.95 0.06 asserted 1.00

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from analysis of existing data 0.76

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 5,764 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (3)
Figure 2 of the paper
Figure 2. Overview of the study pipeline. TABLE I. Selected upstream repositories. Stars and forks are GitHub counts captured on 2026-04-28. Repository Stars Forks anthropics/skills 125,742 14,697 obra/superpowers 172,158 15,025 affaan-m/everything-claude-code 169,700 26,239 mattpocock/skills 40,040 2,764 vercel-labs/agent-skills 25,892 2,353 ComposioHQ/awesome-claude-skills 56,990 6,186 skill package already present in the upstream repository, and therefore exclude whole-skill additions or deletions, upstream synchronization, repository maintenance changes, and files outside skill packages.…
Figure 3 of the paper
Figure 3. Relationship between commit-message Why categories and patch

Showing 2 of the paper's 3 figures. Figures are the authors', reproduced from the paper.

References (40)
  1. [1] Y . Liu, W. Wang, R. Feng, Y . Zhang, G. Xu, G. Deng, Y . Li, and L. Zhang, “Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale,” 2026 on Pith
  2. [2] Y . Zhou, W. Shu, Y . Su, W. Du, Y . Fang, and X. Lin, “A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications,” May 2026. [Online]. Available: http://arxiv.org/abs/2605.07358 on PithPith's reviewarXiv
  3. [3] R. Xu and Y . Yan, “Agent Skills for Large Language Models: Architec- ture, Acquisition, Security, and the Path Forward,” 2026 on Pith
  4. [4] Y . Jiang, D. Li, H. Deng, B. Ma, X. Wang, Q. Wang, and G. Yu, “SoK: Agentic Skills – Beyond Tool Use in LLM Agents,” 2026 on Pith
  5. [5] Y . Zheng, Z. Zhang, C. Ma, Y . Yu, J. Zhu, Y . Wu, T. Xu, B. Dong, H. Zhu, R. Huang, and G. Yu, “SkillRouter: Skill Routing for LLM Agents at Scale,” Apr. 2026 on Pith
  6. [6] H. Li, C. Mu, J. Chen, S. Ren, Z. Cui, Y . Zhang, L. Bai, and S. Hu, “Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale,” 2026 on Pith
  7. [7] K. Ding, Y . Zhou, C. Jin, F. Tong, M. Zhou, and D. N. Metaxas, “Agent Skill Evaluation and Evolution: Frameworks and Benchmarks,” Jun. 2026. [Online]. Available: http://arxiv.org/abs/2606.11435 on PithPith's reviewarXiv
  8. [8] T. Han, Y . Zhang, W. Song, C. Fang, Z. Chen, Y . Sun, and L. Hu, “SWE- Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?” Mar. 2026 on Pith
  9. [9] X. Li, W. Chen, Y . Liu, S. Zheng, X. Chen, Y . He, Y . Li, B. You, H. Shen, J. Sun, S. Wang, B. Li, Q. Zeng, D. Wang, X. Zhao, Y . Wang, R. B. Chaim, Z. Di, Y . Gao, J. He, Y . He, L. Jing, L. Kong, X. Lan, J. Li, S. Li, Y . Li, Y . Lin, X. Liu, X. Liu, H. Lyu, Z. Ma, B. Wang... on Pith
  10. [10] Y . Li, H. Wen, Y . Chen, H. Liu, Y . Tian, and Y . Feng, “No Attack Re- quired: Semantic Fuzzing for Specification Violations in Agent Skills,” 2026 on Pith
  11. [11] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing Reasoning and Acting in Language Models,” inThe Eleventh International Conference on Learning Representations (ICLR), 2023 on Pith
  12. [12] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024 on Pith
  13. [13] X. Wang, B. Li, Y . Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y . Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y . Shao, N. Muennighoff, Y . Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig, “OpenHands: An Open Platform for AI Software D... on Pith
  14. [14] C. Zhou, H. Chai, W. Chen, Z. Guo, R. Shan, Y . Song, T. Xu, Y . Yang, A. Yu, W. Zhang, C. Zheng, J. Zhu, Z. Zheng, Z. Zhang, X. Lou, C. Zhang, Z. Fu, J. Wang, W. Liu, J. Lin, and W. Zhang, “Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harne... on Pith
  15. [15] Anthropic and the Agent Skills community, “Agent Skills: A standardized way to give AI agents new capabilities and expertise,” on Pith
  16. [16] [Online]. Available: https://www.anthropic.com/news/skills on Pith
  17. [17] S. Alzubi, N. Provenzano, J. Bingham, W. Chen, and T. Vu, “EvoSkill: Automated Skill Discovery for Multi-Agent Systems,” 2026 on Pith
  18. [18] Y . Yang, J. Li, Q. Pan, B. Zhan, Y . Cai, L. Du, J. Zhou, K. Chen, Q. Chen, X. Li, B. Zhang, and L. He, “AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution,” 2026 on Pith
  19. [19] H. Lin, P. Li, J. Song, F. Jiang, and T. Zhang, “MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation,” May 2026 on Pith
  20. [20] J. Gong, R. Gu, Z. Fei, Y . Cao, L. Twist, A. Geiger, S. Han, D. Sobania, F. Sarro, and J. M. Zhang, “Skillmoo: Multi-objective optimization of agent skills for software engineering,”arXiv preprint arXiv:2604.09297, 2026 on PithPith's reviewarXiv
show 20 more
  1. [21] A. E. Hassan, “The road ahead for Mining Software Repositories,” in 2008 Frontiers of Software Maintenance. Beijing, China: IEEE, Sep. 2008, pp. 48–57 on Pith
  2. [22] J. Gong, G. Pinna, Y . Bian, and J. M. Zhang, “Analyzing message-code inconsistency in ai coding agent-authored pull requests,”MSR, 2026 on Pith
  3. [23] J. Jiang, D. Lo, J. He, X. Xia, P. S. Kochhar, and L. Zhang, “Why and how developers fork what from whom in GitHub,”Empirical Software Engineering, vol. 22, no. 1, pp. 547–578, Feb. 2017 on Pith
  4. [24] E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian, “The promises and perils of mining github,” inProceedings of the 11th Working Conference on Mining Software Repositories, 2014, pp. 92–101 on Pith
  5. [25] V . Kovalenko, F. Palomba, and A. Bacchelli, “Mining file histories: Should we consider branches?” inProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. Mont- pellier France: ACM, Sep. 2018, pp. 202–213 on Pith
  6. [26] J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, and D. C. Schmidt, “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT,” Feb. 2023 on Pith
  7. [27] J. White, S. Hays, Q. Fu, J. Spencer-Smith, and D. C. Schmidt, “ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design,” Mar. 2023 on Pith
  8. [28] M. Tafreshipour, A. Imani, E. Huang, E. Almeida, T. Zimmermann, and I. Ahmed, “Prompting in the Wild: An Empirical Study of Prompt Evo- lution in Software Repositories,” inProceedings of the 22nd IEEE/ACM International Conference on Mining Software Repositories (MSR), 2025 on Pith
  9. [29] H. Li, H. Masri, F. R. Cogo, A. A. Bangash, B. Adams, and A. E. Hassan, “Understanding Prompt Management in GitHub Repositories: A Call for Best Practices,”IEEE Software, vol. 43, no. 2, pp. 85–93, Mar. 2026 on Pith
  10. [30] GitHub, “GitHub REST API documentation,” 2026. [Online]. Available: https://docs.github.com/en/rest on Pith
  11. [31] T. Ahmed, P. Devanbu, C. Treude, and M. Pradel, “Can LLMs Replace Manual Annotation of Software Engineering Artifacts?” Feb. 2025 on Pith
  12. [32] R. Wang, J. Guo, C. Gao, G. Fan, C. Y . Chong, and X. Xia, “Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a- Judge in Software Engineering,”Proceedings of the ACM on Software Engineering, vol. 2, no. ISSTA, pp. 1955–1977, Jun. 2025 on Pith
  13. [33] Anthropic, “Effort. Claude API docs,” 2025. [Online]. Available: https://platform.claude.com/docs/en/build-with-claude/effort on Pith
  14. [34] X. Wang, H. Kim, S. Rahman, K. Mitra, and Z. Miao, “Human- LLM Collaborative Annotation Through Effective Verification of LLM Labels,” inProceedings of the CHI Conference on Human Factors in Computing Systems. Honolulu HI USA: ACM, May 2024, pp. 1–21 on Pith
  15. [35] W. G. Cochran, “Sampling techniques,”Johan Wiley & Sons Inc, 1977 on Pith
  16. [36] J. Cohen, “A coefficient of agreement for nominal scales,”Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960 on Pith
  17. [37] K. Krippendorff, “Reliability in content analysis: Some common mis- conceptions and recommendations,”Human Communication Research, vol. 30, no. 3, pp. 411–433, 2004 on Pith
  18. [38] A. Gelman and J. Hill,Data Analysis Using Regression and Multi- level/Hierarchical Models. Cambridge university press, 2007 on Pith
  19. [39] Y . Tian, Y . Zhang, K.-J. Stol, L. Jiang, and H. Liu, “What makes a good commit message?” inProceedings of the 44th International Conference on Software Engineering. Pittsburgh Pennsylvania: ACM, May 2022, pp. 2389–2401. [Online]. Available: https://dl.acm.org/doi/ 10.1145/35... on Pithdoi:10.1145/3510003.3510205
  20. [40] Y . Yang, Z. Gong, W. Huang, Q. Yang, Z. Zhou, Z. Huang, Y . Li, X. Gao, Q. Dai, B. Liu, K. Qiu, Y . Yang, D. Chen, X. Yang, and C. Luo, “SkillOpt: Executive Strategy for Self-Evolving Agent Skills,” 2026. 11 on Pith
Cited by 2 Pith papers
How the literature uses it

What this paper does with the arXiv papers it cites

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 3 sentences without a confident label

Cites as background 9 papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment