Pith. sign in

REVIEW 3 major objections 7 minor 37 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

216,938-Skill Library Tests When Injected Knowledge Actually Helps Agents

2026-07-09 02:45 UTC pith:FWCX6ZPE

load-bearing objection Solid systems paper with a genuinely useful negative result about retrieval; the source-grounding guarantee is narrower than the paper claims, but the evaluation is honest enough that this matters less than it sounds. the 3 major comments →

arxiv 2607.07676 v1 pith:FWCX6ZPE submitted 2026-07-08 cs.AI

SkillCenter: A Large-Scale Source-Grounded Skill Library for Autonomous AI Agents

Tianming Sha , Yue Zhao , Lichao Sun , Yushun Dong This is my paper
classification cs.AI
keywords skill libraryautonomous agentssource groundingretrieval-augmented generationknowledge injectionLLM evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

SkillCenter introduces a library of 216,938 structured, source-grounded skills for autonomous AI agents, built through an automated pipeline that extracts actionable guidance from academic papers, GitHub repositories, web documentation, and forums. The pipeline's central mechanism is a five-stage process: source acquisition, an LLM-based quality gate called SkillGate that filters sources before generation, template-driven skill generation, iterative refinement with deterministic source-grounding checks that confirm each claim traces to an exact quotation in its source, and a publish gate enforcing quality-score, license, plagiarism, and placeholder thresholds. The library ships as offline-searchable SQLite FTS5 bundles across 24 domains. The paper's controlled evaluation isolates the conditions under which injected skills improve agent correctness: skills help only when the task exceeds the solver's pretrained knowledge, the needed knowledge exists in the library, and retrieval surfaces the right skill. On tasks within the solver's competence, keyword-retrieved skill injection is statistically indistinguishable from injecting irrelevant text and is mildly harmful for mid-tier models. On tasks requiring knowledge the solver lacks, a precisely targeted skill lifts pass rates from zero to 61–100%. Critically, the paper identifies retrieval—not corpus coverage or the injection mechanism—as the binding constraint: when the needed skill is present in the searched bundles but title-only FTS5 retrieval fails to surface it, the solver scores 0%.

Core claim

The paper's central empirical finding is a three-condition gate on skill value: a skill helps an agent only when (i) the task requires knowledge the solver lacks, (ii) that knowledge exists in the library, and (iii) retrieval surfaces it. The paper demonstrates this through a paired A/B evaluation across four LLM solvers. On general algorithmic tasks where models are near ceiling, keyword-retrieved skills provide no benefit over a length-matched placebo of irrelevant skills. On synthetic knowledge-gap tasks and on real-corpus parameter-recall tasks drawn from actual SkillCenter research skills, baseline and keyword-retrieval arms score 0% while oracle injection of the correct skill scores 61

What carries the argument

The SkillGate pipeline: a five-stage automated process (source acquisition, SkillGate pre-filtering, template-driven generation, iterative source-grounding with deterministic substring matching, and quality-controlled publishing) that converts raw documents into structured, retrievable skills. The source-grounding check uses deterministic substring matching (≤20 words) to verify that each claim in a generated skill maps to an exact quotation in its source document, providing traceability but not factual verification.

Load-bearing premise

The quality guarantee of the pipeline subset rests on a source-grounding check that uses deterministic substring matching (≤20 words) to confirm each claim maps to an exact quotation in its source, plus a single-LLM quality score that assigns 4 to 82% of skills. The substring check verifies that isolated quoted phrases exist in the source but does not verify that the skill's actionable guidance is fully supported by the source or that the source is factually correct.

What would settle it

Run the real-corpus knowledge-gap probe (§4.4) with full-content semantic retrieval instead of title-only FTS5. If pass rates remain at 0% despite the needed skill being present and retrievable, the problem is not retrieval but something deeper about how skills are structured or how agents parse them.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Corpus size alone does not improve agent performance; the binding constraint is retrieval precision. A 216,938-skill library is useless to an agent if the right skill cannot be found at query time.
  • Gap-aware injection—detecting when a solver lacks task-relevant knowledge before retrieving and injecting skills—could avoid the mild harm observed when irrelevant or redundant skills are injected into already-competent solvers.
  • The shift from raw RAG document chunks to structured, quality-scored, source-grounded skill units represents a different retrieval paradigm: the artifact is curated actionable guidance rather than contextual text the agent must interpret.
  • The safety surface area of a skill library scales with corpus size and agent autonomy: skills containing prompt-injection phrases, destructive commands, or hardcoded secrets were found predominantly in the unscreened community bundles, not the pipeline subset.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If retrieval is the binding constraint, then extending the FTS5 index from title-only to full-content (as the paper plans) may be necessary but not sufficient: the real-corpus probe showed that even when the needed skill was present in searched bundles, keyword retrieval returned generic off-topic skills. Semantic retrieval or task-aware retrieval may be required.
  • The source-grounding check verifies traceability (each claim maps to a source quotation) but not factual correctness or completeness of the actionable guidance. A skill could pass grounding while its operational steps are only partially supported by the source, or while the source itself is outdated or incorrect.
  • The single-LLM quality scorer assigning score 4 to 82% of skills means the quality signal carries little discriminative information; downstream agents using quality scores to prioritize skills may not be able to distinguish adequate from excellent guidance.
  • The paper's evaluation uses single-shot LLM solvers on algorithmic and parameter-recall tasks. Whether skills improve multi-turn agentic workflows on operational tasks (the corpus's primary content type) remains unmeasured.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. SkillCenter introduces a 216,938-skill library for autonomous AI agents across 24 domain bundles, of which 114,565 are produced by an automated pipeline (SkillGate) from peer-reviewed journals, ArXiv, GitHub, web, and forum sources, and 102,373 are integrated from community collections. The pipeline has five stages: source acquisition, an LLM-based quality gate, template-driven generation, iterative source-grounding, and quality-controlled publishing. Skills ship as offline SQLite FTS5 bundles. A controlled downstream evaluation (§4) uses paired McNemar tests across four solvers with placebo, keyword-retrieval, and oracle arms, finding that skills help only when the task exceeds the solver's knowledge and retrieval surfaces the right skill. The paper is notably honest in reporting that keyword retrieval never beats placebo and scores 0% on the real-corpus probe.

Significance. The scale of the released library (216,938 skills, open-source on GitHub and Hugging Face) is a genuine contribution to the agent infrastructure ecosystem. The downstream evaluation is well-designed: paired McNemar tests, a placebo control to separate context-token effects from skill content, an oracle probe with synthetic conventions, and a real-corpus probe using 18 source-grounded research skills. The honest reporting of negative results (keyword retrieval indistinguishable from placebo, 0% recall on real-corpus tasks) is a credit to the paper. The falsifiable claim that retrieval, not coverage or injection, is the binding constraint is well-supported. The redundancy analysis (MinHash + LSH) and the pipeline-vs-community separation are useful for practitioners.

major comments (3)
  1. §3.4, step 4 and §2.1/§3.4/§7: The paper repeatedly states that 'each retained claim maps to an exact quotation in its source' (Abstract, §2.1, §3.4, §7). However, the actual mechanism in §3.4 step 4 checks only that 'each claimed improvement must map to an exact quote (≤20 words)' — i.e., it verifies changes made during the iterative refinement loop, not all claims in the skill body. A skill that reaches composite score 0 on the first pass (no lint issues, no unaddressed suggestions) terminates immediately and proceeds to the publish gate (§3.5) without any source-grounding check on its content. The publish gate itself checks quality score, license, plagiarism ratio, and placeholder count — but not source-grounding. The worked example in Appendix E confirms this: 'Evidence verification confirms 7/8 via deterministic substring match' refers to 7 of 8 addressed improvement suggestions,not
  2. §4.4, Table 17: The real-corpus probe uses only 18 tasks, each a narrow parameter-recall item. While the pooled McNemar result (p≈2×10⁻¹⁵) is statistically significant, the sample is too small to support the broad claim that 'the corpus genuinely carries operational knowledge the models lack' (§4.4). The paper acknowledges this ('deliberately narrow fact-recall tasks and a lower bound'), but the framing in the abstract and conclusion is stronger than what 18 tasks can support. Consider softening the language or expanding the probe.
  3. §2.6 and §3.5: The quality score distribution (82.1% at score 4, mean 3.91) means the score carries little discriminative information. The paper acknowledges this as 'score inflation' and an 'internal QA signal, not external validation.' However, the publish gate threshold of ≥3.0 excludes only ~3% of skills, and the paper still reports per-domain and per-source score averages (Tables 1, 4, 15) as if they carry meaning. Given the acknowledged inflation, these averages should either be removed or more prominently flagged as non-informative.
minor comments (7)
  1. §2.7: The FTS5 index currently covers only title and domain, not full skill body. This is a significant limitation for a released artifact. The paper acknowledges it as a 'planned extension' (§6.5), but users of the current release should be more prominently warned.
  2. Table 1: The 'Other (residual)' row for research sources has avg score 3.52, notably lower than all other research sources. Consider briefly explaining why.
  3. §3.4: The composite score formula (lint_count×100 + missing_count) is stated, but the choice of 100× weight is not justified beyond intuition. A brief sensitivity analysis or justification would strengthen this.
  4. Appendix E: The worked example states 'Self-review: overall_score = 4, issues = 1, improvement_suggestions = 12' but then 'Pass 1: lint_count = 0, missing_count = 8 → score = 8.' The relationship between 12 suggestions and missing_count=8 is unclear; clarify whether 4 were already addressed.
  5. §2.3, Table 5: The cloud-domain web pages average 2.25 across only 8 skills. This is too few to report as a meaningful average; consider noting the small n.
  6. The term 'ClawHub' and 'OpenClaw' appear to be introduced without clear prior context in the literature. If these are the authors' own artifacts, this should be disclosed.
  7. §4.1: The solver names (gemini-3.5-flash, claude-haiku-4.5, etc.) appear to be fictional future model versions. If this is a simulated evaluation, clarify; if real, confirm availability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee identifies three substantive issues: (1) a gap between the paper's universal source-grounding claim and the actual mechanism, which only verifies claims modified during iterative refinement; (2) framing of the 18-task real-corpus probe that exceeds what the sample can support; and (3) the low discriminative value of quality-score averages given acknowledged inflation. We agree with all three points and will revise accordingly.

read point-by-point responses

  1. Referee: §3.4 step 4 and §2.1/§3.4/§7: The paper repeatedly states 'each retained claim maps to an exact quotation in its source,' but the actual mechanism only verifies changes made during the iterative refinement loop, not all claims in the skill body. A skill that reaches composite score 0 on the first pass terminates without any source-grounding check on its content. The publish gate checks quality score, license, plagiarism, and placeholder count — but not source-grounding.

    Authors: The referee is correct. The source-grounding check in §3.4 step 4 verifies only claims introduced or modified during the iterative improvement loop, not the full skill body. A skill that reaches composite score 0 on the first pass (no lint issues, no unaddressed suggestions) exits the loop immediately, and the publish gate (§3.5) does not independently verify source-grounding of the skill's content. The universal language in the Abstract ('each retained claim maps to an exact quotation in its source'), §2.1, §3.4, and §7 overstates what the mechanism actually guarantees. We will correct this throughout the paper. Specifically: (a) The Abstract and §2.1 will be revised to state that source-grounding verification applies to claims introduced or modified during iterative refinement, not to the full skill body. (b) §3.4 step 4 will be clarified to explicitly note that skills terminating at composite score 0 on the first pass undergo no source-grounding check. (c) §7 (Limitations) will be expanded to state this as a known gap. (d) The Appendix E worked example will be annotated to clarify that '7/8' refers to improvement-suggestion evidence, not whole-skill grounding. We will also add a note in §6.5 (roadmap) that a full-content source-grounding check at the publish gate is a planned extension. We do not claim that the current mechanism provides whole-skill grounding, and the revision will make this explicit. revision: yes

  2. Referee: §4.4, Table 17: The real-corpus probe uses only 18 tasks, each a narrow parameter-recall item. While the pooled McNemar result is statistically significant, the sample is too small to support the broad claim that 'the corpus genuinely carries operational knowledge the models lack' (§4.4). The framing in the abstract and conclusion is stronger than what 18 tasks can support.

    Authors: The referee is right that 18 narrow parameter-recall tasks cannot support a broad claim about the corpus as a whole. The paper already acknowledges this locally in §4.4 ('deliberately narrow fact-recall tasks and a lower bound'), but the framing elsewhere — particularly the phrase 'the corpus genuinely carries operational knowledge the models lack' in §4.4, and any implied broader claims in the abstract and conclusion — is stronger than the evidence warrants. We will make the following changes: (a) Soften the §4.4 conclusion to state that the probe demonstrates that specific source-grounded research skills carry parameter-level knowledge the tested models lack, and that this is a lower bound on a narrow class of verifiable gaps, not a claim about the corpus as a whole. (b) Ensure the abstract and conclusion do not generalize from the 18-task probe to the full library. (c) Add an explicit scope qualifier noting that the probe covers only the research subset and only parameter-recall tasks. We considered expanding the probe but determined that doing so properly — designing additional offline-verifiable tasks grounded in real skills across diverse domains — requires substantial new work that belongs in the planned multi-turn benchmark (§6.5, item 8) rather than a quick addition to this revision. revision: yes

  3. Referee: §2.6 and §3.5: The quality score distribution (82.1% at score 4, mean 3.91) means the score carries little discriminative information. The publish gate threshold of ≥3.0 excludes only ~3% of skills, and the paper still reports per-domain and per-source score averages (Tables 1, 4, 15) as if they carry meaning. Given the acknowledged inflation, these averages should either be removed or more prominently flagged as non-informative.

    Authors: The referee's observation is accurate. With 82.1% of skills at score 4 and the publish gate excluding only ~3%, the per-domain and per-source averages (Tables 1, 4, 15) carry little discriminative information. The paper already acknowledges this in §2.6 ('score inflation,' 'internal QA signal, not external validation') and §7, but the tables themselves do not make this sufficiently visible. Rather than removing the tables — which practitioners may still find useful as completeness records — we will add a prominent caveat directly in the table captions and in the surrounding text stating that, given the documented score-4 concentration, the averages should not be interpreted as meaningful quality differentiators across domains or sources. We will also strengthen the §2.6 discussion to explicitly state that the narrow band of averages (roughly 3.7–4.2 across most domains) is not interpretable given the single-LLM scorer and score-4 collapse, and that the only clearly separated case (the residual 'other' domain at 3.39) reflects a catch-all category rather than a meaningful quality signal. The §7 Limitations entry on this point will be expanded to reference the tables by number. revision: yes

Circularity Check

0 steps flagged

No significant circularity found; the paper's central claims are supported by independent external evaluation, and self-assessed quality scores are acknowledged as internal signals rather than presented as validated predictions.

full rationale

This is a systems/data paper, not a theoretical derivation, and its load-bearing claims do not reduce to their inputs by construction. The three main contributions are: (1) a corpus of 216,938 skills (a factual count, not a derived prediction), (2) an automated pipeline (described architecturally, not 'derived'), and (3) a controlled evaluation showing skills help only when a knowledge gap exists and retrieval surfaces the right skill. The downstream evaluation (§4) uses externally graded tasks with hidden reference solutions, four independent LLM solvers from three providers, and a placebo arm that controls for context-token effects. The oracle probes (§4.3, §4.4) test whether real corpus content carries knowledge solvers lack, using exact-McNemar paired tests. The quality scores assigned by GPT-5.2 are explicitly acknowledged as 'an internal QA signal, not external validation' (§2.6, §7), and the authors note the 82% score-4 collapse as a limitation rather than presenting the scores as validated predictions. The source-grounding check (§3.4) is acknowledged as 'traceability, not verification' (§7). The one minor concern is that the same LLM class (GPT-5.2) generates, scores, and is used as a solver in some arms of the evaluation, but the paper is transparent about this, the evaluation includes non-OpenAI solvers, and the quality scores are not used as 'predictions' of anything—they are metadata for filtering. No self-citation chain is load-bearing for the central claims. The paper is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

9 free parameters · 4 axioms · 3 invented entities

The pipeline has 9 hand-set thresholds and 4 domain assumptions. The most consequential free parameters are the publish-gate thresholds (min_score=3.0, max_plagiarism=0.35), which directly determine what enters the corpus. The most fragile axiom is the LLM-as-judge quality scoring, which the paper itself flags as inflated. The title-only FTS5 indexing is an ad-hoc choice that the evaluation shows is inadequate.

free parameters (9)
  • publish_gate min_score = 3.0
    Threshold below which skills are excluded; chosen by hand (§3.5).
  • publish_gate max_plagiarism_ratio = 0.35
    Maximum verbatim copy ratio allowed; chosen by hand (§3.5).
  • publish_gate max_not_provided_count = 15
    Maximum placeholder occurrences allowed; chosen by hand (§3.5).
  • improvement max_passes = 3
    Cap on iterative refinement passes; configurable, default 3 (§3.4).
  • source_truncation = 12000
    Character limit applied to all sources before generation (§3.2).
  • skillgate_excerpt_truncation = 4000
    Character limit for SkillGate pre-filter input (§3.3).
  • min_source_length = 200
    Below this, sources auto-fail SkillGate without LLM call (§3.3).
  • github_min_stars = 50-500 (domain-specific)
    Per-domain threshold for repository inclusion (§3.2, Appendix C).
  • dedup_jaccard_threshold = 0.8
    MinHash Jaccard threshold for near-duplicate clustering (§2.5).
axioms (4)
  • domain assumption GPT-5.2 is a reliable quality scorer for generated skills
    All quality scores (1-to-5) are assigned by GPT-5.2 at the publish gate (§3.5). The paper itself notes 82.1% of skills receive score 4, calling this 'score inflation' (§2.6).
  • domain assumption Deterministic substring matching (≤20 words) confirms source grounding
    The source-grounding check in §3.4 step 4 verifies that claims map to exact quotations via substring match. This confirms traceability, not factual correctness (§7).
  • ad hoc to paper Title-only FTS5 search is a sufficient retrieval mechanism for skill lookup
    The current release indexes only skill title and domain, not full body (§2.7). The downstream evaluation (§4) shows this is the binding constraint: keyword retrieval scores 0% on the real-corpus probe even when the needed skill is present.
  • domain assumption LLM-generated skills from source documents carry operational knowledge distinct from raw RAG chunks
    The paper's framing (§1.4) posits that structured skills are more actionable than RAG document chunks. The real-corpus probe (§4.4) provides partial support: oracle injection lifts pass rates from 0% to 61-78%.
invented entities (3)
  • SkillGate independent evidence
    purpose: LLM-based pre-generation filter that classifies source suitability as pass/maybe/fail
    The gate is a concrete pipeline component with a reproduced prompt (Appendix D) and observable behavior on case studies (§3.3). Its filtering effect is measurable.
  • Skill (as a data structure) independent evidence
    purpose: Structured Markdown document with Background, Use Cases, Inputs, Outputs, Steps, Verification, Evidence sections
    The schema is concretely specified (Appendix B) and three full examples are reproduced (Appendix G). Its utility is tested in the downstream evaluation (§4).
  • ClawHub marketplace no independent evidence
    purpose: Community skill marketplace integrated as one bundle
    Referenced as an external platform [10] but appears to be authored by the same group (OpenClaw [7]). 11,389 skills are integrated from it without quality gating (§2.4).

pith-pipeline@v1.1.0-glm · 37014 in / 3534 out tokens · 752790 ms · 2026-07-09T02:45:58.151554+00:00 · methodology

0 comments
read the original abstract

Autonomous AI agents can execute complex tasks with limited human review, yet they often lack the grounded operational knowledge to make their outputs not just executable but correct, secure, and maintainable. We introduce SkillCenter, to our knowledge the largest open skill library for agents by total count: 216,938 structured skills across 24 domain bundles. A SkillGate-filtered pipeline contributes 114,565 source-grounded skills from peer-reviewed journals, ArXiv, and over 24,000 technical sources, integrated with 102,373 community skills from GitHub and the ClawHub marketplace. We present the end-to-end framework that builds the pipeline subset: multi-source acquisition, an LLM-based quality gate (SkillGate), template-driven generation, iterative source-grounding, and quality-controlled publishing. Source grounding is a traceability guarantee: each retained claim maps to an exact quotation in its source. All skills ship as offline-searchable SQLite FTS5 bundles.

Figures

Figures reproduced from arXiv: 2607.07676 by Lichao Sun, Tianming Sha, Yue Zhao, Yushun Dong.

Figure 1
Figure 1. Figure 1: Treemap visualization of the SkillCenter library. Area is proportional to skill count. The library comprises pipeline-produced Research (41.5%) and Technical (11.3%) bundles plus an integrated Community category of GitHub SkillMD and the ClawHub marketplace (47.2%). 2.1 Corpus Overview The SkillCenter library contains 216,938 published skills organized into 24 domain bundles, drawn from three broad categor… view at source ↗
Figure 2
Figure 2. Figure 2: Concept-feature profiles of the 24 corpus bundles. Rows (bundles) and columns (22 concept features) are reordered by seriation so that high relevance concentrates along the diagonal; each cell shows the row-normalized relevance (0 to 1) of a concept to a bundle, so every bundle’s dominant concepts read brightest. The left strip marks each bundle’s macro-category (Technical, Research, or Community; see lege… view at source ↗
Figure 3
Figure 3. Figure 3: Skill kind composition across PLOS journals. Each bar shows the percentage of skills in four categories: idea/intro, experiment, method, and picture. Comp Bio and Genetics tilt toward methods; Medicine and NTD toward visual communication. figures (14%), consistent with its computational and mathematical emphasis. PLOS Genetics (4,241 skills, avg 3.77) follows a similar pattern: method-heavy (26%) with low … view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the SkillCenter pipeline. The data collection pipeline (top) ingests sources from academic publications, GitHub, web pages, and forums, then applies SkillGate filtering, template￾driven generation, iterative refinement, and quality-controlled release gating to produce SQLite FTS5 skill bundles distributed via Hugging Face. The application pipeline (bottom) consumes these skills at inference tim… view at source ↗
Figure 5
Figure 5. Figure 5: Quality score distribution by source type (left) and mean score (right). Score 4 dominates across all source types. GitHub has the highest mean (4.01) and score-5 share (16.1%). 4 Downstream Evaluation: When Do Skills Help? Our motivating claim is that skills improve the correctness of agent outputs, not merely whether they execute. We now report a controlled evaluation of that claim. Rather than asking on… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 27 internal anchors

  1. [1]

    vibe coding

    Andrej Karpathy.Vibe Coding. Post on X (formerly Twitter). First articulation of the “vibe coding” concept. Feb. 2025.url:https://x.com/karpathy/status/18861921848 08149383

    work page arXiv 2025

  2. [2]

    Blog post

    OpenAI.Vibe Research 2026: The New Paradigm of Human-AI Collaborative Research. Blog post. Non-archival source cited for anecdotal context only; no quantitative claims in this report depend on this source. Examines the emerging vibe research paradigm. 2026

    work page 2026

  3. [3]

    VibeX 2026: 1st International Workshop on Vibe Coding and Vibe Researching

    “VibeX 2026: 1st International Workshop on Vibe Coding and Vibe Researching”. In: Proceedings of the International Conference on Evaluation and Assessment in Software Engineering (EASE 2026). Workshop exploring the emerging paradigm of vibe coding and vibe researching in software engineering. 2026.url:https://conf.researchr.org/ home/ease-2026/vibex-2026

    work page 2026

  4. [4]

    OpenAI announcement

    OpenAI and Ginkgo Bioworks.GPT-5 Lowers the Cost of Cell-Free Protein Synthesis. OpenAI announcement. Autonomous-laboratory collaboration in which a GPT-5 model ran over 36,000 cell-free protein synthesis experiments across six iterative cycles, reducing benchmark-protein cost by 40%. Corroborated by Ginkgo Bioworks press release, 5 Feb

    work page

  5. [5]

    2026.url:https://openai.com/index/gpt-5-lowers-protein-synthesis- cost/

    Feb. 2026.url:https://openai.com/index/gpt-5-lowers-protein-synthesis- cost/

    work page 2026

  6. [6]

    Streamline automated biomedical discoveries with agentic bioin- formatics

    Juexiao Zhou et al. “Streamline automated biomedical discoveries with agentic bioin- formatics”. In:Briefings in Bioinformatics26.5 (2025). Reviews agentic bioinformatics pipelines in which autonomous LLM-driven agents formulate hypotheses, design exper- iments, and interpret results, bbaf505.doi:10.1093/bib/bbaf505.url:https:// academic.oup.com/bib/artic...

    work page doi:10.1093/bib/bbaf505.url:https:// 2025

  7. [7]

    The Rise and Potential of Large Language Model Based Agents: A Survey

    Zhiheng Xi et al. “The Rise and Potential of Large Language Model Based Agents: A Survey”. In:arXiv preprint arXiv:2309.07864(2023).url:https://arxiv.org/abs/ 2309.07864

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  8. [8]

    GitHub repository

    OpenClaw.OpenClaw: Open-Source AI Agent Framework. GitHub repository. Modular framework for building autonomous AI agents with composable skill architectures. Ac- cessed March 2026. 2026.url:https://github.com/openclaw/openclaw

    work page 2026

  9. [9]

    Preprint

    OPAL Consortium.OPAL: Orchestrated Platform for Autonomous Laboratories. Preprint. Non-archival source cited for context. No stable URL available at time of writing. A platform for orchestrating autonomous laboratory workflows integrating AI planning with robotic execution. 2026

    work page 2026

  10. [10]

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    Carlos E. Jimenez et al. “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” In:International Conference on Learning Representations (ICLR). 2024.url: https://arxiv.org/abs/2310.06770. [10]ClawHub: Marketplace for OpenClaw Skills. Online platform. A community marketplace for sharing, discovering, and composing reusable skills for the OpenCl...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [11]

    SemDeDup: Data-efficient learning at web-scale through semantic deduplication

    Amro Abbas et al. “SemDeDup: Data-Efficient Learning at Web-Scale through Semantic Deduplication”. In:arXiv preprint arXiv:2303.09540(2023).url:https://arxiv.org/ abs/2303.09540

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  12. [12]

    Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

    Nils Reimers and Iryna Gurevych. “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks”. In:Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 2019.url:https://arxiv.org/abs/1908.10084. 45

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  13. [13]

    Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

    Lianmin Zheng et al. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena”. In:Advances in Neural Information Processing Systems (NeurIPS), Datasets and Bench- marks Track. Documents position, verbosity, and self-enhancement biases in LLM-based evaluation. 2023.url:https://arxiv.org/abs/2306.05685

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  14. [14]

    G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment

    Yang Liu et al. “G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment”. In:arXiv preprint arXiv:2303.16634(2023).url:https://arxiv.org/abs/2303.16634

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  15. [15]

    Self-Refine: Iterative Refinement with Self-Feedback

    Aman Madaan et al. “Self-Refine: Iterative Refinement with Self-Feedback”. In:arXiv preprint arXiv:2303.17651(2023).url:https://arxiv.org/abs/2303.17651

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  16. [16]

    Survey of Hallucination in Natural Language Generation

    Ziwei Ji et al. “Survey of Hallucination in Natural Language Generation”. In:ACM Com- puting Surveys(2023).url:https://arxiv.org/abs/2202.03629

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  17. [17]

    Voyager: An Open-Ended Embodied Agent with Large Language Models

    Guanzhi Wang et al. “Voyager: An Open-Ended Embodied Agent with Large Language Models”. In:arXiv preprint arXiv:2305.16291(2023).url:https://arxiv.org/abs/ 2305.16291

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  18. [18]

    Self-Instruct: Aligning Language Models with Self-Generated Instructions

    Yizhong Wang et al. “Self-Instruct: Aligning Language Models with Self-Generated In- structions”. In:Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). 2023.url:https://arxiv.org/abs/2212.10560

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  19. [19]

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

    Patrick Lewis et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”. In:Advances in Neural Information Processing Systems (NeurIPS). 2020.url: https://arxiv.org/abs/2005.11401

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  20. [20]

    Generative Agents: Interactive Simulacra of Human Behavior

    Joon Sung Park et al. “Generative Agents: Interactive Simulacra of Human Behavior”. In: Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). 2023.url:https://arxiv.org/abs/2304.03442

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    Reflexion: Language Agents with Verbal Reinforcement Learning

    Noah Shinn et al. “Reflexion: Language Agents with Verbal Reinforcement Learning”. In:Advances in Neural Information Processing Systems (NeurIPS). 2023.url:https: //arxiv.org/abs/2303.11366

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  22. [22]

    ReAct: Synergizing Reasoning and Acting in Language Models

    Shunyu Yao et al. “ReAct: Synergizing Reasoning and Acting in Language Models”. In: International Conference on Learning Representations (ICLR). 2023.url:https://arx iv.org/abs/2210.03629

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  23. [23]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Jason Wei et al. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Mod- els”. In:arXiv preprint arXiv:2201.11903(2022).url:https://arxiv.org/abs/2201. 11903

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  24. [24]

    Toolformer: Language Models Can Teach Themselves to Use Tools

    Timo Schick et al. “Toolformer: Language Models Can Teach Themselves to Use Tools”. In:Advances in Neural Information Processing Systems (NeurIPS). 2023.url:https: //arxiv.org/abs/2302.04761

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  25. [25]

    Gorilla: Large Language Model Connected with Massive APIs

    Shishir G. Patil et al. “Gorilla: Large Language Model Connected with Massive APIs”. In: arXiv preprint arXiv:2305.15334(2023).url:https://arxiv.org/abs/2305.15334

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

    Akari Asai et al. “Self-RAG: Learning to Retrieve, Generate, and Critique through Self- Reflection”. In:International Conference on Learning Representations (ICLR). 2024.url: https://arxiv.org/abs/2310.11511

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  27. [27]

    The Pile: An 800GB Dataset of Diverse Text for Language Modeling

    Leo Gao et al. “The Pile: An 800GB Dataset of Diverse Text for Language Modeling”. In: arXiv preprint arXiv:2101.00027(2020).url:https://arxiv.org/abs/2101.00027

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  28. [28]

    The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only

    Guilherme Penedo et al. “The RefinedWeb Dataset for Falcon LLM: Outperforming Cu- rated Corpora with Web Data, and Web Data Only”. In:arXiv preprint arXiv:2306.01116 (2023).url:https://arxiv.org/abs/2306.01116

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  29. [29]

    Deduplicating Training Data Makes Language Models Better

    Katherine Lee et al. “Deduplicating Training Data Makes Language Models Better”. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). 2022.url:https://arxiv.org/abs/2107.06499

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  30. [30]

    On the Resemblance and Containment of Documents

    Andrei Z. Broder. “On the Resemblance and Containment of Documents”. In:Proceedings of the Compression and Complexity of Sequences (SEQUENCES). IEEE, 1997

    work page 1997

  31. [31]

    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

    Kai Greshake et al. “Not What You’ve Signed Up For: Compromising Real-World LLM- Integrated Applications with Indirect Prompt Injection”. In:Proceedings of the ACM 46 Workshop on Artificial Intelligence and Security (AISec@CCS). First systematic taxonomy of indirect prompt injection attacks against LLM-integrated applications. 2023.url:htt ps://arxiv.org/...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  32. [32]

    InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

    Qiusi Zhan et al. “InjecAgent: Benchmarking Indirect Prompt Injections in Tool- Integrated Large Language Model Agents”. In:Findings of the Association for Computa- tional Linguistics: ACL 2024. 1,054 test cases across 17 user tools and 62 attacker tools evaluating indirect prompt injection vulnerability in tool-augmented LLM agents. 2024. url:https://arx...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  33. [33]

    AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

    Edoardo Debenedetti et al. “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents”. In:Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track. Benchmark with 97 re- alistic tasks and 629 security test cases for evaluating LLM agent robustness to prompt injection. 2024.url:http...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    Identifying the Risks of LM Agents with an LM-Emulated Sandbox

    Yangjun Ruan et al. “Identifying the Risks of LM Agents with an LM-Emulated Sand- box”.In:Proceedings of the International Conference on Learning Representations (ICLR). Spotlight paper. ToolEmu framework for scalable risk identification in tool-integrated LM agents using emulated sandboxes. 2024.url:https://arxiv.org/abs/2309.15817

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [35]

    R-Judge: Benchmarking Safety Risk Awareness for LLM Agents

    Tongxin Yuan et al. “R-Judge: Benchmarking Safety Risk Awareness for LLM Agents”. In: Findings of the Association for Computational Linguistics: EMNLP 2024. Safety bench- mark with 569 multi-turn agent interaction records covering 27 risk scenarios. 2024.url: https://arxiv.org/abs/2401.10019

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  36. [36]

    SkillCenter Contributors.SkillCenter: A Large-Scale Source-Grounded Skill Library for Autonomous AI Agents. GitHub. Open-source framework and corpus. Pre-built bundles available athttps://huggingface.co/datasets/Tommysha/skillcenter-bundles. 2026.url:https://github.com/LabRAI/SkillCenter

    work page 2026

  37. [37]

    Datasheets for Datasets

    Timnit Gebru et al. “Datasheets for Datasets”. In:Communications of the ACM64.12 (2021), pp. 86–92.url:https://arxiv.org/abs/1803.09010. 47

    work page internal anchor Pith review Pith/arXiv arXiv 2021