Pith.

Information Retrieval · arXiv:2610.32336 v1 · submitted 23 Oct 2026 · 67 references

SkillRep: Characterizing and Understanding the Representation Design Space of Agent Skill Retrieval

Tianyi Sun, Pengfei Yao, Yanze Wang, Hongda Chen, Yunyun Han, Jun Sun, Chuanrui Hu, Yafeng Deng

The paper says

“Retrieving the right skill from a large library is essential for LLM agents, yet requests and skills are written for different purposes and in different words.”

more of the abstract

Existing retrievers narrow this request-skill gap with one or two representation choices each, so it is unknown whether their gains hold in combination. No prior work measures the gap itself, so it is also unclear which part of it each choice narrows. We define a representation design space by five choices – encoder fine-tuning, query-side and document-side expansion, background projection and a lexical channel – and evaluate them together for the first time, as modules of one retriever, SkillRep. With a 0.6B encoder and LLM expansion, SkillRep improves MRR@10 on SRA-Bench over the strongest same-size retriever by 7.02 points with the official skill pool and 8.79 with UniPool, a 111,247-skill pool we build from public registries, and is best on SkillsBench. The value of each choice depends on the others. Fine-tuning, for instance, reverses the ranking of the four training-free modules and turns them from complementary into overlapping. To understand why, we measure the gap with three quantities: background, the embedding component that does not separate skills; unsaid vocabulary, the gold skill's content a request leaves unsaid; and dilution, the information lost when a skill is encoded into a single vector. They show which part each module narrows and by what mechanism. This reveals that modules narrowing the same part compete for it, while those narrowing different parts complement each other.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 26 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. The SOTA claim is well supported by same-pool runs and honest ablations; the causal overlap/complementarity principle is the soft spot and should be treated as a plausible hypothesis, not an established explanation. This line is wrong

3 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 67 references say.

The paper says, at Section 6, Eqs. 2-3; Sections 3.2-3.3, 7.1-7.3

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.76):

“Becauseℓ subtracts the one-vector score cos(𝑞,𝑑) itself, part of this correlation is built into the measure.”

Pith says: Objection 1 of 3 · major

The three diagnostic quantities are partly defined by the modules they are used to explain, so the causal reading of Table 6 ('narrows') is stronger than the measurements support. B_D in Eq. 2 is the top-two singular subspace that background projection removes in Section 3.3; B_Q is the top singular direction of the pseudo-request matrix in Section 6.1, which is …

more of objection 1

… the same text document-side expansion mixes into documents; the coverage counts concern gold-document words, and the Section 7.2 deletion probe shows those words carry query-side expansion's gain; and the dilution loss of Eq. 3 subtracts cos(q,d), the one-vector score, as the paper acknowledges (b194). The paper also acknowledges the small partial correlations (b158), and the projection, deletion, and padding controls are genuine evidence for the mechanisms. However, most of these controls intervene on the very quantities the modules are defined against, so they show that each module moves the quantity it was built around rather than that the three quantities are separable causes of retrieval failure. I ask for either a predictive test of Table 6 on a setting not used to build the taxonomy (for example, one of the BEIR collections or a held-out module combination), or a reframing of the quantities as descriptive probes with the causal 'narrows' language qualified accordingly.

The paper says, at Section 7.4, Tables 6-7, 10

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The overlap/complementarity principle is induced from exactly the four modules and two outcomes it claims to explain. The partial correlations in Table 7 are small (|rho| <= 0.20) and are computed post hoc on the same 5,400 requests on which the paper also selects and evaluates its settings.

more of objection 2

Table 10's support is in the predicted direction — after query-side expansion, document-side expansion and background projection narrow the B_Q gap by less and gain less — but the deltas are modest (B_Q narrowing 0.56 to 0.25; gains 3.10 to 2.10 and 1.63 to 0.96), so the principle still has enough freedom to be retold as a summary of the tables rather than a tested explanation. A concrete strengthening is available in the paper's own Appendix C: SkillRet's requests have high coverage, and the principle predicts query-side expansion should add little in the dense channel there, which Table 14 confirms with a best dense weight of w=0; making such predictions before measurement, on a held-out pool or module combination, would substantially raise confidence in the principle.

The paper says, at Sections 5-7

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

No confidence intervals or significance tests are reported anywhere, and several deltas that support the mechanism claims are small enough to matter: the SkillsBench comparisons rest on 75 requests (Table 3), the B_Q gap narrowings in Table 8 are 0.56 to 0.99, and the Table 10 comparisons are 0.25 to 0.68 in gap narrowing and 0.63 to 1.63 in MRR@10 gain.

more of objection 3

The headline margins of 7.02 and 8.79 MRR@10 are large enough that the SOTA claim does not depend on significance testing, but the mechanism sections' quantitative assertions are exactly where uncertainty quantification is needed; bootstrap intervals over the 5,400 requests or over the six SRA-Bench subsets would show which of those deltas are distinguishable from noise.

7 minor objections

The paper says, at Abstract

Pith says: minor objection 1 of 7

The abstract's phrase 'and is best on SkillsBench' should be qualified, since Table 3 shows SkillRep is not best on easy R@10 (SkillRet-Emb. leads by 0.84); the body's formulation 'best on five of the six metrics' (sentence b113) is accurate and should be used in the abstract.

The paper says, at Section 7.4

Pith says: minor objection 2 of 7

The claim that 'the three modules overlap by 2.11 MRR@10, nearly all of the 2.18 by which the four overlap in Table 5' is not derivable from the printed tables; the computation should be shown.

show 5 more

The paper says, at Section 4 vs. Appendix C

Pith says: minor objection 3 of 7

The main text's statement that 'all settings are chosen on UniPool and kept the same on every benchmark' can be read as conflicting with sentence b359 ('the settings were also chosen on the same SRA-Bench requests they are tested on'); the appendix's 3,150-setting search is an overfitting bound, and the main text should clarify this in the same paragraph.

The paper says, at Section 3.1 and Appendix A

Pith says: minor objection 4 of 7

The encoder is fine-tuned on documents taken from the library it later searches (30% of UniPool and 13% of the official pool); the Appendix A mitigation is reasonable, but this transductive setup should be stated in Section 3.1 so that Table 2 is not read without it.

The paper says, at Table 3

Pith says: minor objection 5 of 7

The table mixes rows copied from [20] (R@10 only, 'as published') with the authors' own runs reporting all metrics, which makes the non-starred cells easy to misread as comparable; a footnote listing which cells are directly comparable would help.

The paper says, at Equation 4

Pith says: minor objection 6 of 7

The definition of the partial correlation should state which covariates enter the regressions for c and g, and whether the other measures enter only through z, since the current wording 'together with the other measures' is ambiguous.

The paper says, at Figure 3

Pith says: minor objection 7 of 7

Reporting the per-fifth request counts and the standard error of the micro-averaged RR would help interpret the spread from 77.3 on the least diluted fifth to 56.1 on the most diluted fifth.

What would settle it

Hold out half of the 5,400 SRA-Bench requests, stratified by source dataset. On the other half, fix all SkillRep hyperparameters and estimate each module's gain partial correlations (Table 7) and B_D/B_Q gap changes (Table 10). Pre-specify the Table 6 mapping and the overlap/complementarity predictions, then apply the frozen modules to the held-out half and compare predicted versus observed gain patterns, reporting sign agreement with bootstrap confidence intervals. Run the same procedure with matched non-informative surrogates: random orthonormal subspaces in place of B_D and B_Q, and random word subsets in place of coverage. If held-out sign agreement is not significantly above the surrogate level, Table 6 should be reported as an exploratory taxonomy rather than an established mechanism.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 26 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 1 min 27 s · 23,288 tokens read, 11,461 written
  3. Editor wrote the verdict above 12 s · 24,256 tokens read, 1,212 written
  4. Referee major revision: 3 major objections, 7 minor 1 min 49 s · 23,728 tokens read, 16,681 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The empirical core holds up. SkillRep sets a real SOTA at 0.6B on SRA-Bench and SkillsBench, with same-pool runs of released baselines, ablations in both directions, a disclosed 3,150-setting half-split bound on tuning, and direct intervention probes. The paper is unusually honest about its own soft spots: it states that settings were chosen on the same SRA-Bench requests they are tested on (b359), that partial correlations are small because most requests' RR does not move (b158), and that Eq. 3 subtracts the very score it explains (b194). Those acknowledgements are in the text, not buried.

The genuinely new content is narrower than the title suggests: the five modules are all known, and the first half is a well-executed engineering combination. The novel part is the joint evaluation of the five choices in one retriever plus the three gap measures and the attempt to explain overlap and complementarity. The empirical result is not the problem. The causal principle in Section 7.4 is the soft spot, and it is soft in a specific way: the three quantities are partly defined by the modules they explain. BD is the subspace that background projection removes, dilution subtracts the one-vector score, and coverage counts gold words that query expansion supplies. The paper's deletion, padding, and projection controls are real evidence and soften the entanglement, but the partial correlations sustaining Table 6 are small (|rho| <= 0.20), computed post hoc on the same 5,400 requests used to select settings. With four modules and two observed outcomes, the overlap/complementarity principle is plausible but underdetermined. I would not call it circular, but it is a hypothesis supported by suggestive evidence, not a closed explanation.

Minor but real: no confidence intervals or significance tests anywhere, and on 75-request SkillsBench the easy R@10 gap is less than one request. The artifacts are not fully released; UniPool, training pairs, expansions, and a pinned commit hash would materially help reproducibility.

Who this is for: anyone working on agent skill retrieval or module composition in dense retrieval. It deserves a serious referee, and I would send it out, but the revision should add confidence intervals, release with a pinned commit hash, and ideally cross-validate the mechanism predictions on a held-out split.

The strongest objection

The SOTA claim is well supported by same-pool baseline runs, the disclosed 3,150-setting selection bound, and the direct intervention probes. The load-bearing weakness is the second claim: that the three quantities are separable causes and that modules narrowing the same part overlap while modules narrowing different parts complement each other (Section 7.4). The supporting evidence is Table 7 partial correlations with |rho| <= 0.20, computed post hoc on the same 5,400 requests used to choose the settings. More importantly, each diagnostic is entangled with the module it explains: B_D in Eq. 2 is the same top-two subspace that background projection removes; Eq. 3 subtracts cos(q,d), the one-vector score used by the single-vector retriever; and coverage is defined from gold-document words that query-side expansion is known to add. The paper openly acknowledges the small correlations (sentence b158) and the built-in dilution term (b194), and its deletion, padding, and projection controls are real evidence, so this is not a hidden circularity. The unresolved issue is whether those controls establish the claimed invariant or merely show that modules act on quantities they were designed around. The taxonomy in Table 6 has exactly four modules and two outcomes, so without a held-out test many arrangements of Table 6 and Table 10 can be retold as a principle. This does not undermine the empirical result, but it leaves the central explanatory contribution conditional.

The empirical headline is credible: same-pool baseline runs, ablations in both directions, direct intervention probes, and the Appendix C selection bound covering the grid choices all support the SOTA claim. My concern is not with those numbers. The explanation claim is different in kind: it asserts a causal structure and a general principle. The paper is honest about its weak points, including the small partial correlations and the built-in dilution term, and its control experiments go beyond what most systems papers do. However, the crucial step from Table 6 to the Section 7.4 principle rests on small, partly definitional correlations, a four-module taxonomy with considerable freedom to fit observations, and post hoc reading of the same data used to select the modules. A pre-specified held-out sign test with surrogate nulls would settle whether the diagnostics isolate separable failure causes or simply re-describe the geometry of the modules. This agrees with the reader's weakest_assumption and keeps the verdict at CONDITIONAL; it does not justify rejection, because the empirical and reproducibility concerns are separate from the explanatory overreach.

The referee's report

The empirical contribution is strong and unusually well disciplined for this area: baselines are rerun on the same pools and metrics with released code, ablations are measured both alone and last, random-direction projection controls are reported, the grid-search selection procedure is disclosed with a half-split overfitting bound (winner within 0.60 MRR@10 of the best setting on the held-out half, against a 7-to-9 point lead), and the public code URL gives the results a reproducible basis. The headline gains (7.02 and 8.79 MRR@10) are large, and Appendix F's demonstration that the background gap and dilution are specific to skill retrieval rather than generic IR phenomena is a valuable external check. The weak point is the causal reading of the explanatory framework: the diagnostics are partly defined by the modules they explain, the partial correlations in Table 7 are small (|rho| <= 0.20), and the overlap/complementarity principle is fitted post hoc to four modules and two outcomes. The SOTA claim does not depend on the explanation; the explanatory claim, which the abstract and title foreground, is defensible but conditional on further testing or a more careful framing.

Pith's reading of the paper (a machine's summary, not the authors')

An LLM agent that must pick a skill from a library of more than a hundred thousand documents faces the same mismatch every time: the request states a task while the skill describes a procedure, and the two rarely share vocabulary. Existing retrievers each narrow this request–skill gap with one or two representation choices — fine-tuning the encoder, expanding the query or the document with LLM-written text, projecting out generic background directions, or adding a lexical channel — so nobody knew whether the choices still help in combination. This paper claims they do: stacked into one retriever, SkillRep, the five choices lift MRR@10 on SRA-Bench by 7.02 points on the official pool and 8.79 on UniPool, a 111,247-skill library built from public registries, against the strongest same-size retrievers, and are best on five of six SkillsBench metrics. The paper's second and more general claim is that the gap decomposes into three measurable quantities — background, unsaid vocabulary, and dilution — and that modules narrowing the same quantity compete while modules narrowing different quantities add up. If that principle holds, retrieval design becomes a matter of measuring which part of the gap is still open rather than choosing tricks by habit.

What carries the argument. The load-bearing machinery is a set of three per-request diagnostics together with one fusion rule. The background is the component the library's embeddings share, estimated as the top two right singular vectors of the uncentered document matrix; the background gap of Eq. 2 measures how much more that shared direction adds to competitors' scores than to the gold document's, and on the untuned encoder it is genuinely harmful, with 89% of documents scoring above the gold one on the background component alone. Unsaid vocabulary is measured by coverage, the share of the gold document's content words that the request expresses, counted three ways (lexical, semantic, soft). Dilution is measured by the dilution loss $\ell(q)=\cos(q,s^*)-\cos(q,d)$ of Eq. 3, the similarity to the best-matching section that is lost when the document is encoded as a single vector; section-level retrieval and BM25 do not lose accuracy as the dilution loss grows, only the one-vector retriever does, and padding experiments with irrelevant sections confirm that dilution, not length, is the cause. Attribution of a module's gain to a quantity runs through the partial correlation of Eq. 4, which controls for baseline reciprocal rank and for the other quantities. The fusion rule is Eq. 1, z-score score fusion of the dense and BM25 channels, chosen over reciprocal rank fusion because rank fusion discards the dense margin and lets the lexical channel overturn a wide dense lead.

Core claim. The paper's central claim is that the request–skill gap in agent skill retrieval can be measured, and that measuring it explains how five representation choices interact. SkillRep, which stacks contrastive fine-tuning of a 0.6B encoder, query-side and document-side LLM expansion, background projection, and a BM25 lexical channel, reaches MRR@10 of 79.21 on SRA-Bench's official pool and 76.46 on UniPool, against 72.19 and 67.67 for the strongest same-size baselines. The interaction claim is the more general one: on the fine-tuned encoder the four training-free modules overlap, with their individual gains summing to +15.33 while all four together give +13.15, whereas on the untuned encoder they complement each other (+16.56 summed against +17.30 together), and fine-tuning reverses which module matters most, demoting background projection and elevating query-side expansion. The paper attributes this to three per-request quantities — background, the shared embedding component that does not separate skills; unsaid vocabulary, the gold skill's content a request leaves unstated; and dilution, information lost when a whole procedure is compressed into one vector — and shows by partial correlation and direct manipulation which quantity each module narrows. The principle that emerges is stated plainly: modules that narrow the same part of the gap overlap and compete, while modules that narrow different parts complement each other.

The premise it rests on. The explanation assumes that the three measured quantities — background, unsaid vocabulary, and dilution — are separable causes of retrieval failure rather than re-descriptions of what the modules themselves remove; the gains-to-quantity correlations are small ($|\rho| \le 0.20$), the background gap is defined on the very subspace that background projection removes, and the dilution loss subtracts the one-vector score itself, so part of its apparent effect is built into the measure, as the paper acknowledges.

If the paper is right (Pith's inferences, not the authors')

A retriever can be designed by measurement: first gauge which of background, unsaid vocabulary, or dilution is still large for the request set, then add one module for each remaining part, since a second module aimed at an already-narrowed part is known in advance to be wasted effort.

Module evaluations are only meaningful together with the encoder beneath them: fine-tuning changes not only the baseline but which part of the gap each training-free module acts on, so a module's reported gain cannot be transplanted to another encoder.

For hybrid retrieval generally, score fusion that preserves the dense margin should be preferred over rank fusion when one channel is meant to rescue near-misses, because the lexical rescue cannot be targeted at the requests that need it and rank fusion's gains and losses nearly cancel on average.

Applied to three BEIR collections with the same measurements, the background is harmless and dilution is small, which localizes the paper's claims to settings where documents describe whole procedures rather than short facts.

If the decomposition is causal, it should transfer without re-attribution: a library with heavy template boilerplate (large background) should make background projection indispensable even on a fine-tuned encoder, and a library of short single-purpose documents should make dilution vanish and shrink the lexical channel's edge — predictions the paper's own corpora do not yet vary enough to test.

The dilution result points to an architectural alternative the paper does not pursue: a section-level dense index was immune to dilution in the paper's own control retriever, so it might replace most of the lexical channel's rescue function rather than merely complement it.

The overlap principle suggests a cheap stopping rule for retrieval engineering: once all three quantities are near zero on a hold-out request set, further representation work is being spent on the same part of the gap and should be redirected to data, reranking, or the agent's downstream use of the retrieved skill.

The same three quantities could be measured wherever query and document are written for different purposes, such as tool selection, API retrieval, or function calling, and the paper's BEIR comparison gives a template for checking whether the gap decomposition even applies in a new setting.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Retrieving the right skill from a large library is essential for LLM agents, yet requests and skills are written for different purposes and in different words. Existing retrievers narrow this request-skill gap with one or two representation choices each, so it is unknown whether their gains hold in combination. No prior work measures the gap itself, so it is also unclear which part of it each choice narrows. We define a representation design space by five choices – encoder fine-tuning, query-side and document-side expansion, background projection and a lexical channel – and evaluate them together for the first time, as modules of one retriever, SkillRep. With a 0.6B encoder and LLM expansion, SkillRep improves MRR@10 on SRA-Bench over the strongest same-size retriever by 7.02 points with the official skill pool and 8.79 with UniPool, a 111,247-skill pool we build from public registries, and is best on SkillsBench. The value of each choice depends on the others. Fine-tuning, for instance, reverses the ranking of the four training-free modules and turns them from complementary into overlapping. To understand why, we measure the gap with three quantities: background, the embedding component that does not separate skills; unsaid vocabulary, the gold skill's content a request leaves unsaid; and dilution, the information lost when a skill is encoded into a single vector. They show which part each module narrows and by what mechanism. This reveals that modules narrowing the same part compete for it, while those narrowing different parts complement each other.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-26

The authors' main claim, as the classifier picked it out 0.70

“This reveals that modules narrowing the same part compete for it, while those narrowing different parts complement each other.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.89 · states limitation 0.34 · human participants 0.04 · controlled comparison 0.84 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Retrieving the right skill from a large library is essential for LLM agents, yet requests and skills are written for different purposes and in different words. background 0.99 0.06 0.02 asserted 1.00
Existing retrievers narrow this request-skill gap with one or two representation choices each, so it is unknown whether their gains hold in combination. gap 0.80 0.05 0.02 asserted 1.00
No prior work measures the gap itself, so it is also unclear which part of it each choice narrows. gap 0.99 0.07 0.02 asserted 0.94
We define a representation design space by five choices -- encoder fine-tuning, query-side and document-side expansion, background projection and a lexical channel -- and evaluate them together for the first time, as modules of one retriever, SkillRep. contribution 0.78 0.97 0.08 asserted 1.00
With a 0.6B encoder and LLM expansion, SkillRep improves MRR@10 on SRA-Bench over the strongest same-size retriever by 7.02 points with the official skill pool and 8.79 with UniPool, a 111,247-skill pool we build from public registries, and is best on SkillsBench. result 1.00 0.97 0.25 asserted 1.00
The value of each choice depends on the others. result 0.78 0.86 0.28 asserted 1.00
Fine-tuning, for instance, reverses the ranking of the four training-free modules and turns them from complementary into overlapping. result 0.86 0.92 0.12 asserted 1.00
To understand why, we measure the gap with three quantities: background, the embedding component that does not separate skills; unsaid vocabulary, the gold skill's content a request leaves unsaid; and dilution, the information lost when a skill is encoded into a single vector. method 0.80 0.93 0.08 asserted 0.98
They show which part each module narrows and by what mechanism. result 0.37 0.87 0.09 asserted 1.00
This reveals that modules narrowing the same part compete for it, while those narrowing different parts complement each other. interpretation 0.97 0.93 0.48 asserted 1.00

What kind of paper the classifier reports this is: method or tool 0.63 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-26: 1 request, 7,699 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (3)
Figure 1 of the paper
Figure 1. The three quantities on one SRA-Bench [44] request and its gold skill. The background is drawn schematically.
Figure 2 of the paper
Figure 2. The five modules of SkillRep, where 𝑞ˆ and ˆℎ are the mean embeddings of the LLM-written pseudo-requests and hypothetical skills. 3 SkillRep Five representation choices make up the space SkillRep covers (
Figure 3 of the paper
Figure 3. RR of each retriever by fifths of the gold document’s

Figures are the authors', reproduced from the paper.

References (67)
  1. [1] Mehmet Akçay, Ismail Sengor Altingovde, Craig Macdonald, and Iadh Ounis on Pith
  2. [2] Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel. 2009. Improvements That Don’t Add Up: Ad-Hoc Retrieval Results Since 1998. InPro- ceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM). 601–610. doi:10.1145/1645953.1646031 on Pithdoi:10.1145/1645953.1646031
  3. [3] Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. 2022. InPars: Data Augmentation for Information Retrieval using Large Language Models. arXiv:2202.05144 [cs.CL] https://arxiv.org/abs/2202.05144 on PithPith's reviewarXiv
  4. [4] Sebastian Bruch, Siyu Gai, and Amir Ingber. 2023. An Analysis of Fusion Func- tions for Hybrid Retrieval.ACM Transactions on Information Systems42, 1 (2023), 1–35 on Pith
  5. [5] Ruishuo Chen, Xun Wang, Yu Chen, Zhuoran Li, and Longbo Huang. 2026. The Router Within: Eliciting Native Skill Routing from a Frozen LLM. arXiv:2609.15982 [cs.LG] https://arxiv.org/abs/2609.15982 on PithPith's reviewarXiv
  6. [6] Tingwei Chen, Yunxiao Shi, Zhengdong Chu, Qingsong Wen, and Min Xu. 2026. Adapting Embedding Models for Agent Capability Retrieval. arXiv:2607.17347 [cs.IR] on PithPith's reviewarXiv
  7. [7] Yanping Chen, Weijie Shi, Wen Yang, and Jiajie Xu. 2026. Task Decomposition- Guided Reranking for Adaptive Agent Skill Retrieval. arXiv:2607.06283 [cs.AI] https://arxiv.org/abs/2607.06283 on PithPith's reviewarXiv
  8. [8] Gordon V. Cormack, Charles L. A. Clarke, and Stefan Büttcher. 2009. Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. InProceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 758... on Pithdoi:10.1145/1571941.1572
  9. [9] Zhuyun Dai, Vincent Y. Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, and Ming-Wei Chang. 2023. Promptagator: Few-shot Dense Retrieval From 8 Examples. InThe Eleventh International Conference on Learning Representations (ICLR). https://open... on Pith
  10. [10] Jiandong Ding, Honglei Ji, Ming Liu, and Tao Duan. 2026. Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval.arXiv preprint arXiv:2606.10388(2026) on PithPith's reviewarXiv
  11. [11] Kexin Ding, Yang Zhou, Can Jin, Feng Tong, Mu Zhou, and Dimitris N. Metaxas on Pith
  12. [12] Lihui Ding, Zihan Guo, Bingwei Lu, Chenyu Zhou, Yuanjian Zhou, Weinan Zhang, Jianghao Lin, and Dongdong Ge. 2026. Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrieval. arXiv:2608.16071 [cs.IR] on PithPith's reviewarXiv
  13. [13] Kawin Ethayarajh. 2019. How Contextual are Contextualized Word Represen- tations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. In Proceedings of EMNLP-IJCNLP. 55–65 on Pith
  14. [14] Edward A. Fox and Joseph A. Shaw. 1994. Combination of Multiple Searches. InProceedings of the Second Text REtrieval Conference (TREC-2). NIST Special Publication 500-215, 243–252 on Pith
  15. [15] Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Precise Zero-Shot Dense Retrieval without Relevance Labels. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL). 1762–1777. doi:10 .18653/v1/2023.acl-long.99 on Pith
  16. [16] Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online and Punta Cana, Domi... on Pith
  17. [17] Mitko Gospodinov, Sean MacAvaney, and Craig Macdonald. 2023. Doc2Query– : When Less is More. InAdvances in Information Retrieval – 45th European Conference on Information Retrieval (ECIR) (Lecture Notes in Computer Science, Vol. 13980). Springer, 414–422. doi:10.1007/978-3-031... on Pithdoi:10.1007/978-3-031-28238-6_31
  18. [18] Paimon Goulart, Liang Wu, Kelly Wan, Evangelos E. Papalexakis, and Liangjie Hong. 2026. Field Aware Agent Skill Retrieval. InLifelong Agent Workshop at the Conference on Language Modeling (COLM) on Pith
  19. [19] Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Infor- mation Retrieval with Contrastive Learning.Transactions on Machine Learning Research(2022). https://openreview.net/forum?id=... on Pith
  20. [20] Donghong Jiang, Endian Lin, Luoping Cui, Hanqing Liu, Mingjie Liu, Fan Yang, Hong Wang, Zhao Yang, and Chuang Zhu. 2026. SkillRea- son: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests. arXiv:2608.08640 [cs.IR] on PithPith's reviewarXiv
show 47 more
  1. [21] Ryangkyung Kang, Hongcheol Cho, and Youngeun Kim. 2026. SkillRet: A Large- Scale Benchmark for Skill Retrieval in LLM Agents. arXiv:2605.05726 [cs.AI] on PithPith's reviewarXiv
  2. [22] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)... on Pithdoi:10.18653/v1/
  3. [23] Sadegh Kharazmi, Falk Scholer, David Vallet, and Mark Sanderson. 2016. Exam- ining Additivity and Weak Baselines.ACM Transactions on Information Systems 34, 4 (2016), 23:1–23:18. doi:10.1145/2882782 on Pithdoi:10.1145/2882782
  4. [24] Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li on Pith
  5. [25] Fangzhou Li, Pagkratios Tagkopoulos, and Ilias Tagkopoulos. 2025. SkillFlow: Scalable and Efficient Agent Skill Retrieval System. arXiv:2504.06188 [cs.IR] on PithPith's reviewarXiv
  6. [26] Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyoung Whan Choe, Jiankai Sun, and Shuyi Wang. 2026. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. arXiv:2602.12670 [cs.AI] on PithPith's reviewarXiv
  7. [27] Xinze Li, Yuhang Zang, Yixin Cao, and Aixin Sun. 2026. Skill-as- Pseudocode: Refactoring Skill Libraries to Pseudocode for LLM Agents. arXiv:2605.27955 [cs.CL] on PithPith's reviewarXiv
  8. [28] Qiliang Liang, Hansi Wang, Zhong Liang, and Yang Liu. 2026. From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills. arXiv:2604.24026 [cs.CL] on PithPith's reviewarXiv
  9. [29] Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Zou. 2022. Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning. InAdvances in Neural Information Processing Systems on Pith
  10. [30] Jimmy Lin. 2018. The Neural Hype and Comparisons Against Weak Baselines. SIGIR Forum52, 2 (2018), 40–51. doi:10.1145/3308774.3308781 on Pithdoi:10.1145/3308774.3308781
  11. [31] Shuo Liu, Yutong Yang, Haohao Xiao, Mouxing Yang, and Xi Peng. 2026. Imagine Before Retrieval: Prospective Skill Retrieval for LLM Agents. arXiv:2609.01642 [cs.AI] on PithPith's reviewarXiv
  12. [32] Yujian Liu, Jiabao Ji, Li An, Tommi Jaakkola, Yang Zhang, and Shiyu Chang. 2026. How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings. arXiv:2604.04323 [cs.CL] on PithPith's reviewarXiv
  13. [33] Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query Rewriting in Retrieval-Augmented Large Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Singapo... on Pith
  14. [34] Xueguang Ma, Kai Sun, Ronak Pradeep, Minghan Li, and Jimmy Lin. 2022. Another Look at DPR: Reproduction of Training and Replication of Retrieval. InAdvances in Information Retrieval – 44th European Conference on Informa- tion Retrieval (ECIR) (Lecture Notes in Computer Science... on Pithdoi:10.1007/978-3-030-99736-6_41
  15. [35] Xiangcheng Meng, Shu Wang, and Yixiang Fang. 2026. SkillRAE: Agent Skill-Based Context Compilation for Retrieval-Augmented Execution. arXiv:2605.10114 [cs.CL] https://arxiv.org/abs/2605.10114 on PithPith's reviewarXiv
  16. [36] Mark Montague and Javed A. Aslam. 2001. Relevance Score Normalization for Metasearch. InProceedings of the 10th ACM International Conference on Information and Knowledge Management (CIKM). 427–433. doi:10.1145/502585.5 02657 on Pithdoi:10.1145/502585.5
  17. [37] Jiaqi Mu and Pramod Viswanath. 2018. All-but-the-Top: Simple and Effective Postprocessing for Word Representations. InInternational Conference on Learning Representations. Sun et al on Pith
  18. [38] Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022. Large Dual Encoders Are Generalizable Retrievers. InProceedings of the 2022 Conference on Empirical Methods in Natural Language P... on Pithdoi:10.18653/v1/2022.emnlp-main.669
  19. [39] Rodrigo Nogueira and Jimmy Lin. 2019.From doc2query to docTTTTTquery. Technical Report. David R. Cheriton School of Computer Science, University of Waterloo. https://cs.uwaterloo.ca/~jimmylin/publications/Nogueira_Lin_2019 _docTTTTTquery-v2.pdf on Pith
  20. [40] Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document Expansion by Query Prediction. arXiv:1904.08375 [cs.IR] on PithPith's reviewarXiv
  21. [41] Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Frame- work: BM25 and Beyond.Foundations and Trends in Information Retrieval3, 4 (2009), 333–389. doi:10.1561/1500000019 on Pithdoi:10.1561/1500000019
  22. [42] Nazib Sorathiya, Daniel Zhang, and Bardiya Akhbari. 2026. Enrich- Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routing. arXiv:2608.22695 [cs.CL] on PithPith's reviewarXiv
  23. [43] Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whiten- ing Sentence Representations for Better Semantics and Faster Retrieval. arXiv:2103.15316 [cs.CL] https://arxiv.org/abs/2103.15316 on PithPith's reviewarXiv
  24. [44] Weihang Su, Jianming Long, Qingyao Ai, Qiaozhi He, Yichen Tang, Changyue Wang, Yiteng Tu, Yingbo Wang, and Yiqun Liu. 2026. Skill Retrieval Augmenta- tion for Agentic AI. arXiv:2604.24594 [cs.IR] on PithPith's reviewarXiv
  25. [45] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks on Pith
  26. [46] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748 [cs.LG] https://arxiv.org/ abs/1807.03748 on PithPith's reviewarXiv
  27. [47] Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv:2212.03533 [cs.CL] https://arxiv.org/abs/2212.0 3533 on PithPith's reviewarXiv
  28. [48] Liang Wang, Nan Yang, and Furu Wei. 2023. Query2doc: Query Expansion with Large Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Singapore, 9414–9423. doi:10.18653/v1/2... on Pithdoi:10.18653/v1/2023.emnlp-main.585
  29. [49] Tianle Wang, Yanghe Zou, Xiang Liu, Ziyao Huang, Chenchen Fu, and Wei- wei Wu. 2026. Beyond Task-Only Matching: Personalized Skill Routing with Counterfactual Evaluation. arXiv:2608.28241 [cs.IR] on PithPith's reviewarXiv
  30. [50] Yanze Wang, Pengfei Yao, Tianyi Sun, Chuanrui Hu, Yan Xiao, Xiaotian Luo, Yunyun Han, Yifan Chen, Jun Sun, and Yafeng Deng. 2026. SkillCorpus: Consol- idating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents. arXiv preprint arXiv:2607.15557(2026) on PithPith's reviewarXiv
  31. [51] Zifei Wang, Wei Wen, Qiang Ji, Keyu Chen, Ruizhi Qiao, and Xing Sun. 2026. Skill Is Not Document: Query-Conditioned Compatibility for LLM Agent Skill Routing. arXiv:2606.03565 [cs.IR] https://arxiv.org/abs/2606.03565 on PithPith's reviewarXiv
  32. [52] Zifei Wang, Wei Wen, Qiang Ji, and Ruizhi Qiao. 2026. Skills Know Their Neighbors: Cluster-Contrastive Capability Pages for Skill Retrieval. arXiv:2608.04482 [cs.IR] on PithPith's reviewarXiv
  33. [53] Jinying Xiao, Bin Li, Xiaopeng Li, Jianling Li, Jiacheng Jie, Xiaodong Liu, Ma Jun, Chao Wang, Nyima Tashi, and Jie Yu. 2026. SkillSight: Calibrating Generic Content Bias for Skill Retrieval. arXiv:2607.18785 [cs.IR] on PithPith's reviewarXiv
  34. [54] An Yang et al. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] on PithPith's reviewarXiv
  35. [55] Guanqun Yang, Wenlong Zhang, Tian Shi, and Ping Wang. 2026. SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale. arXiv:2609.38822 [cs.IR] https://arxiv.org/abs/2609.38822 on PithPith's reviewarXiv
  36. [56] Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. 2019. Critically Examining the “Neural Hype”: Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. InProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Inform... on Pithdoi:10.1145/3331184.3331340
  37. [57] Barry Zhang, Keith Lazuka, and Mahesh Murag. 2025. Equipping Agents for the Real World with Agent Skills. Anthropic Engineering Blog. https://www.anthro pic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills on Pith
  38. [58] Chi Zhang, Yimin Liu, Xinze Chen, and Ping Ji. 2026. What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files. arXiv:2608.08453 [cs.AI] https://arxiv.org/abs/2608.08453 on PithPith's reviewarXiv
  39. [59] Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou on Pith
  40. [60] Jingyuan Zheng, Dongjing Wang, Xin Zhang, Hao Chen, Youhuizi Li, Xudong Shen, Haiping Zhang, Butian Huang, Dongjin Yu, and Guandong Xu. 2026. SkillSelect-Serve: QoS-Aware Budgeted Skill Service Recommendation for LLM Agents.arXiv preprint arXiv:2607.00011(2026) on PithPith's reviewarXiv
  41. [61] YanZhao Zheng, ZhenTao Zhang, Chao Ma, YuanQiang Yu, JiHuai Zhu, Yong Wu, Tianze Xu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, and Gang Yu. 2026. SkillRouter: Skill Routing for LLM Agents at Scale. arXiv:2603.22455 [cs.LG] A Encoder fine-tuning Each training pair comes with up... on PithPith's reviewarXiv
  42. [2017] On the Additivity and Weak Baselines for Search Result Diversification Research. InProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR). 109–116. doi:10.1145/3121050.3121066 on Pithdoi:10.1145/3121050.3121066
  43. [2020] On the Sentence Embeddings from Pre-trained Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 9119– on Pith
  44. [2025] Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models. arXiv:2506.05176 [cs.CL] on PithPith's reviewarXiv
  45. [2026] Agent Skill Evaluation and Evolution: Frameworks and Benchmarks. arXiv:2606.11435 [cs.CL] https://arxiv.org/abs/2606.11435 on PithPith's reviewarXiv
  46. [6910] doi:10.18653/v1/2021.emnlp-main.552 on Pithdoi:10.18653/v1/2021.emnlp-main.552
  47. [9130] doi:10.18653/v1/2020.emnlp-main.733 on Pithdoi:10.18653/v1/2020.emnlp-main.733

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment