REVIEW 4 major objections 5 minor 51 references
SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structure
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read SkillZip claims that an evolved agent skill can be compressed by finding repeated structure inside the skill itself, with no tasks, rollouts, rewards, or verifiers, shrinking skills by 27–37% while preserving behavior.
desk verdict SkillZip is a clean, honest formalization of a real problem whose load-bearing empirical assumption — parser coverage of every normative span — is clearly flagged in the text but never actually measured. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the typed contract $\mathcal{C}(S)=\langle I,G,T,C,O,E\rangle$ extracted from a skill: interface entries, workflow nodes and edges, tool protocol, scoped rules with modality and guard, output contract, and supporting evidence. Around this contract SkillZip builds a deterministic pipeline: a scanner that turns Markdown structure into numbered blocks, a schema-constrained extractor that returns typed units with source citations, type-compatible reuse proposal using hash matches plus a frozen relation checker, a min-cost cover optimizer that solves scope placement by dynamic programming and workflow reuse by weighted set packing, and fixed-template rendering with an optional structural audit. The objective it minimizes is $L(K)+L(R|K)$ subject to $\forall a\in A_{\mathrm{req}}(S),\ a\preceq (K,R)$, and the cost model charges for definitions, references, and scope notation so that a shared abstraction must actually pay for itself. The same objective drives both one-shot compression and the Zip-on-Write updater, which classifies each incoming patch as absorb, refine, extend, or refactor, then periodically repacks when cross-scope reuse accrues.
What would settle it
Take evolved skills whose normative spans have been annotated by humans, run SkillZip's extraction, and measure recall of the typed units against those annotations; then delete the units the parser missed and test the rare branches those units govern. If any deleted unit changes behavior on tasks that activate its guard, the evaluation-free fidelity guarantee is falsified.
Extended reading notes
Core claim
The central discovery is formalized as a typed minimum-description-length objective: choose a library $K$ of reusable contract elements and a residual $R$ of unique or uncertain content that minimizes $L(K)+L(R|K)$ subject to a hard coverage constraint for every extracted trigger, workflow node and edge, tool requirement, scoped rule, and output field. The paper argues that this single objective unifies four forms of reuse—collapsing equivalent paraphrases, lifting a repeated rule to its nearest common scope, factoring a repeated action sequence into a shared procedure, and writing guarded variants as one common rule plus explicit exceptions—and that each is accepted only when the shared form is literally shorter after paying for definitions, references, and scope notation. Because coverage is enforced for the parsed contract rather than for sampled tasks, preservation of a rare guard, tool argument, exception, or output field does not depend on how often any compression-time task distribution activates it (Corollary IV.2). The guarantee is deliberately limited to what the parser extracts: uncertain spans are locked verbatim, and an optional structural audit reparses the rendered skill and restores any missing span. The paper's headline claim is that compression may change how a requirement is written, but not whether it remains represented.
Load-bearing premise
The load-bearing premise is that the method's structural parser recovers the complete set of requirements hidden in natural-language skill text; if any requirement is missed, the hard coverage constraint protects the wrong set and the fidelity guarantee collapses.
Editorial extensions
If this is right
- Compression no longer depends on a sampled evaluation set, so a rare guard or output field survives simply because it is part of the extracted contract.
- Running compression requires no rollouts, cutting measured end-to-end cost by about 3.5× compared with evaluation-guided skill compression.
- Activating compression from round one of self-evolution keeps skill length near 1.6–1.9× its seed size instead of growing to 2.5–3.7×, with no test-accuracy loss in the reported runs.
- Compressed skills transfer across agent backbones with about 0.97 retention on one benchmark, suggesting that explicit rules, guards, and output contracts travel better than raw evolved prose.
- The structural audit can detect and restore any missing trigger, guard, workflow edge, tool argument, or output field, making under-compression the intended failure mode rather than silent deletion.
Reading between the lines
- We infer that the practical reach of the fidelity guarantee is set by parser recall, which the paper leaves unmeasured; a human-annotated span-recall study would pin down how much of the guarantee is real in deployment.
- We infer that the explain-once-reference-many objective is not tied to agent skills and could be applied to other evolving text artifacts such as memory banks, runbooks, or documentation, though the paper does not make that claim.
- We infer that repeated application of the structural audit across many evolution rounds could serve as a drift detector, flagging when a restored span keeps being re-inserted by new patches; the paper does not report such longitudinal audit statistics.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SkillZip, a compression method for self-evolving agent skills. It treats a skill as a typed contract (interface, workflow, tool protocol, scoped rules, output contract, and evidence), extracts this contract with a schema-constrained LLM while placing uncertain spans into a locked residual, and then selects the shortest representation of the contract under a hard coverage constraint. The method is evaluation-free in that compression never observes tasks, rewards, rollouts, or behavioral verifiers. Two modes are presented: one-shot compression and Zip-on-Write continual compression. Experiments on three benchmarks and three backbone models report 27.1–36.9% compression with roughly preserved task performance, a 3.5× speedup over SkillReducer, cross-model transfer, and bounded continual growth. The formal preservation guarantee (Prop. IV.1) is explicitly conditional on the parser-extracted contract, a boundary the paper acknowledges in Sections III.C and Appendix D.
Significance. The paper contributes a clean and well-documented formulation: an MDL-style objective over typed contract content, a hard coverage constraint, and deterministic optimization after a single structured extraction call. The evaluation-free design is a genuine point of distinction from SkillReducer's task-based feedback loop, and the conservative locked-residual mechanism is principled. The appendices are unusually detailed for reproducibility, including schemas, prompts, a CLI, cache keys, and an atomic transaction protocol. If parser coverage can be demonstrated empirically, the method would be practically useful for maintaining evolved agent skills. However, the central fidelity claim currently rests on unmeasured parser coverage and on point estimates without variance, so the empirical core is thinner than the paper's stated conclusions.
major comments (4)
- [Section III-C, Eq. (3); Section V.A.2; Appendix D.C] The hard coverage constraint in Eq. (3) is defined over A_req(S), the contract produced by the parser, and Section V.A.2 states that contract recovery 'can be evaluated against human annotations,' yet no parser precision, recall, or coverage numbers are reported anywhere; Appendix D.C repeats the promise without results. The central claim in Section IV.A—that compression changes how a requirement is written but not whether it remains represented—therefore holds only relative to an extraction step whose error rate is unknown. The optional structural audit in Section V.A.5 and Algorithm 1 is not an independent check, since it reparses the rendered skill with the same parser family; a systematic parser blind spot (e.g., for implicit, cross-referenced, or highly condensed requirements such as the duplicated rules in the appendix example) would be invisible to it. Please report per-type extraction coverage against human annotations on evolved skills, including recall for rare rules and calibration of the confidence scores that route spans to the locked residual, and report renderer-level preservation using an independent re-annotation or manual audit.
- [Table I, Section VI.C (RQ2)] All nine task scores are single point estimates with no error bars, number of seeds, or significance tests. In three cells SkillZip is numerically below the Evolved Skill reference (Qwen3.7-Max BFCL-V4 0.863 vs 0.869 and Spreadsheet 0.519 vs 0.525; Kimi K2.6 BFCL-V4 0.747 vs 0.772), so the statement that SkillZip 'matches or improves the evolved skill in five of nine settings' is not established as a preservation claim. The macro-average 0.577 vs 0.570 could easily be within evaluation noise, and the held-out test-set sizes are not reported. Please provide multiple evaluation runs or bootstrap confidence intervals, report test-set sizes, and state whether each task score is a single pass or an aggregate.
- [Section IV.B; Appendix A.B; Appendix B.E] The four sharing mechanisms (equivalent requirements, scope lifting, workflow reuse, and guarded variants) are presented as unified under the objective in Eq. (4), but no ablation isolates any of them. The reported compression rate and fidelity could come mostly from one mechanism, or from simple deduplication, and the claim that the typed coverage constraint rather than token pruning is responsible for preserving behavior is untested. Add ablations that disable each of the four mechanisms and that replace the coverage constraint with a length-only objective, and report compression rate and task scores for each variant.
- [Section IV.C, Prop. IV.1 and Corollary IV.2] Corollary IV.2 is presented as 'the key theoretical benefit of evaluation-free compression,' but it is a direct logical consequence of the hard coverage constraint over the extracted contract; it is a specification of the method rather than an empirical discovery. This is not an internal inconsistency, since the paper states that the guarantee is limited to the parsed contract, but the framing should be adjusted so that the rare-rule guarantee is not read as evidence of parser fidelity. The empirical question of whether rare requirements are actually extracted from evolved skills is exactly what needs to be measured.
minor comments (5)
- [Figure 4, Section VI.B] The Figure 4 caption names both SkillOpt and Memento-Skills, but Section VI.B only describes skill construction with SkillOpt; please clarify whether RQ1 includes Memento-Skills and how those skills are obtained, or remove the extra name from the caption.
- [Appendix D.D] Appendix D.D promises that idempotence 'will be tested empirically by applying one-shot compression twice and reporting token and contract differences,' but no such experiment appears in the paper; either add the promised results or mark the property as a conjecture to be tested in future work.
- [Table II, Section VI.D] The 'LLM calls' column counts only compressor-model calls, while SkillReducer's rollout column implicitly includes additional agent calls; this asymmetry should be stated in the table caption for a fairer cost comparison.
- [Section VII and Appendix C.E] The conclusion contains missing spaces ('SkillZiptreats', 'SkillZipuses'), and the appendix example asserts that 'every distinct output contract [is preserved] verbatim in meaning' without verification; mark example-level statements as illustrative rather than measured.
- [Section VI.C] The takeaway 'preserving or improving their overall performance' overstates what point estimates without error bars can support; please soften this to be consistent with the requested uncertainty reporting.
Circularity Check
One definitional guarantee (rare-rule preservation) reduces to the paper's own coverage constraint; the empirical compression and fidelity claims are otherwise self-contained and not circular.
-
self definitional
[Section IV-A Eq. (4), Prop. IV.1, Cor. IV.2; Appendix A-C proof]
"SkillZip selects the shortest representation that still covers every required contract unit: (K∗,R∗)=arg min ... s.t. a⪯(K,R),∀a∈A req(S). (4) ... compression may change how a requirement is written, but not whether it remains represented. ... Proposition IV.1: If every normative source span is represented by a typed unit or residual, any feasible solution of Eq.(4) preserves all extracted requirements. ... Corollary IV.2: The preservation of a unique requirement does not depend on how often its branch appears in any compression-time task distribution."
The preservation result is the feasibility condition of Eq. (4) restated. A_req(S) is defined (Def. III.1) as the set of extracted interface, workflow, tool, rule, and output units; coverage ⪯ is the relation Eq. (4) mandates for every such unit. Prop. IV.1's proof says 'Removing the only representation of any required unit violates the constraint in Eq.(4)', so any feasible solution preserves A_req(S) by construction. Corollary IV.2 then labels this same feasibility guarantee as frequency-independent rare-rule preservation. The paper is transparent ('preserves unique rare rules by construction'), and the guarantee is conditional on parser completeness, which the paper flags as the main semantic uncertainty but does not measure.
full rationale
The paper's empirical core is not circular: compression rates are measured against token counts of externally produced evolved skills, held-out task scores come from test sets disjoint from the evolution split, no length-model weights are fitted to task performance (the λ values equal template delimiter costs, Appendix B-D), and SkillReducer is an external baseline. The only definitional entailment is the rare-rule preservation guarantee, which follows directly from the hard coverage constraint and is explicitly labeled 'by construction'. The paper also honestly delimits the guarantee: Section III-C calls the structural parser 'the main source of semantic uncertainty', and Appendix D-C states that Prop. IV.1 protects only the contract produced by the parser. The absence of parser precision/recall measurements is a real correctness risk, but it is an unverified empirical assumption, not a circular derivation. Because the central empirical claims are tested externally and no fitted parameter is renamed as a prediction, the overall circularity is mild.
Assumptions & free parameters
free parameters (3)
- length-model weights lambda_def, lambda_ref, lambda_scope =
template delimiter token costs (not task-fitted)
- repack thresholds theta_repack, rho, B =
not specified numerically in paper
- candidate embedding top-k and relation-checker thresholds =
not specified
assumptions (5)
- domain assumption A skill decomposes into a typed contract (I, G, T, C, O, E) with scoped units
- domain assumption Coverage of every extracted unit implies behavioral preservation sufficient for deployment
- domain assumption Relation checker correctly decides equivalence/implication/conflict for near-duplicate units
- standard math MDL is the appropriate model-selection principle
- domain assumption Rendered token length is a valid cost proxy
Cite this review
Pith. "Pith review of SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structure." pith.science (2026). https://pith.science/paper/T365EAWQ
@misc{pith2026260811079,
author = {Pith},
title = {Pith review of: SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structure},
year = {2026},
howpublished = {\url{https://pith.science/paper/T365EAWQ}},
note = {Machine review of arXiv:2608.11079}
}
read the original abstract
Self-evolving agents accumulate reusable skills by appending successful procedures and failure fixes. Over time, the same requirement is often restated in several branches, examples, and warnings, while common action sequences are copied rather than reused. The resulting skill becomes expensive to inject and difficult to maintain. Generic prompt compression is ill-suited to this setting because a skill is not a flat passage: its name and description define when it applies, its workflow controls execution, its tool and output contracts constrain validity, and rare exceptions may remain essential even when no sampled task activates them. Evaluation-guided compression can test these behaviors, but it introduces rollouts, cost, and dependence on the compression-time evaluation set. We present SkillZip, an evaluation-free method that compresses a skill by finding its shortest faithful structural explanation. The intuition is explain once, reference many: state a repeated rule once at the scope where it applies, factor a repeated action sequence into a shared procedure, and keep only the differences as explicit exceptions. We formalize this intuition as a typed minimum description-length objective over a skill contract and a residual, subject to a hard coverage constraint for every extracted trigger, workflow edge, tool requirement, obligation, and output field. The formulation provides simple sharing thresholds, preserves unique rare rules by construction, and supports efficient local updates. SkillZip has a one-shot mode with one structured extraction call and deterministic optimization, and a continual Zip-on-Write mode that integrates each self-evolution patch without replaying tasks or reparsing the full history. Through comprehensive experimental evaluations, we demonstrate the effectiveness and superiority of SkillZip in compression performance, generalizability, and cost overhead.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Skillreducer: Optimizing llm agent skills for token efficiency,
Y. Gao, Z. Li, Y. Yuan, Z. Ji, P. Ma, and S. Wang, “Skillreducer: Optimizing llm agent skills for token efficiency,”arXiv preprint arXiv:2603.29919, 2026
arXiv 2026
-
[2]
Skillopt: Executive strategy for self-evolving agent skills,
Y. Yang, Z. Gong, W. Huang, Q. Yang, Z. Zhou, Z. Huang, Y. Li, X. Gao, Q. Dai, B. Liu, K. Qiu, Y. Yang, D. Chen, X. Yang, and C. Luo, “Skillopt: Executive strategy for self-evolving agent skills,” arXiv preprint arXiv:2605.23904, 2026
arXiv 2026
-
[3]
Skillclaw: Let skills evolve collectively with agentic evolver,
Z. Ma, S. Yang, Y. Ji, X. Wang, Y. Wang, Y. Hu, T. Huang, and X. Chu, “Skillclaw: Let skills evolve collectively with agentic evolver,”arXiv preprint arXiv:2604.08377, 2026
arXiv 2026
-
[4]
Llmlingua: Compressingpromptsforacceleratedinferenceoflargelanguagemodels,
H. Jiang, Q. Wu, C.-Y. Lin, Y. Yang, and L. Qiu, “Llmlingua: Compressingpromptsforacceleratedinferenceoflargelanguagemodels,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 13358–13376
work page 2023
-
[5]
LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression,
H. Jiang, Q. Wu, X. Luo, D. Li, C.-Y. Lin, Y. Yang, and L. Qiu, “LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for ...
work page 2024
-
[6]
Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression,
Z. Pan, Q. Wu, H. Jiang, M. Xia, X. Luoet al., “Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024
work page 2024
-
[7]
Voyager: An open-ended embodied agent with large language models,
G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,”Transactions on Machine Learning Research, 2023
work page 2023
-
[8]
Reflexion: Language agents with verbal reinforcement learning,
N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” in Advances in Neural Information Processing Systems, vol. 36, 2023
2023
Show all 51 references
-
[9]
A survey of self- evolving agents: On path to artificial super intelligence,
H.-a. Gao, J. Geng, W. Hua, M. Hu, X. Juanet al., “A survey of self- evolving agents: On path to artificial super intelligence,”arXiv preprint arXiv:2507.21046, 2025
2025 arXiv
-
[10]
Agentic context engineering: Evolving contexts for self-improving language models,
Q. Zhang, C. Hu, S. Upasani, B. Ma, F. Hong, V. Kamanuru, J. Rainton, C. Wu, M. Ji, H. Li, U. Thakker, J. Zou, and K. Olukotun, “Agentic context engineering: Evolving contexts for self-improving language models,”arXiv preprint arXiv:2510.04618, 2025
-
[11]
Skillrl: Evolving agents via recursive skill-augmented reinforcement learning,
P. Xia, J. Chen, H. Wang, J. Liu, K. Zeng, Y. Wang, S. Han, Y. Zhou, X. Zhao, H. Chen, Z. Zheng, C. Xie, and H. Yao, “Skillrl: Evolving agents via recursive skill-augmented reinforcement learning,”arXiv preprint arXiv:2602.08234, 2026
2026 arXiv
-
[12]
Skillrevise: Improving llm-authored agent skills via trace-conditioned skill revision,
Y. Liu, Z. Su, L. Xie, Y. Zhang, Q. Zong, J. Guo, Z. Xie, Y. Ji, Y. Yim, H. Luo, X. Ren, R. Chenyu, H. Li, and Y. Song, “Skillrevise: Improving llm-authored agent skills via trace-conditioned skill revision,”arXiv preprint arXiv:2606.01139, 2026
2026 arXiv
-
[13]
Skillgrad: Optimizing agent skills like gradient descent,
H. Wang, Y. Lan, B. Cao, L. Lin, and J. Chen, “Skillgrad: Optimizing agent skills like gradient descent,”arXiv preprint arXiv:2605.27760, 2026
2026 arXiv
-
[14]
Compressing context to enhance inference efficiency of large language models,
Y. Li, B. Dong, F. Guerin, and C. Lin, “Compressing context to enhance inference efficiency of large language models,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 6342–6353
2023
-
[15]
Learning to compress prompts with gist tokens,
J. Mu, X. L. Li, and N. Goodman, “Learning to compress prompts with gist tokens,” inAdvances in Neural Information Processing Systems, vol. 36, 2023, pp. 19327–19352
2023
-
[16]
Recomp: Improving retrieval-augmented lms with compression and selective augmentation,
F. Xu, W. Shi, and E. Choi, “Recomp: Improving retrieval-augmented lms with compression and selective augmentation,” inInternational Conference on Learning Representations, 2024
2024
-
[17]
Generalization in adaptive data analysis and holdout reuse,
C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. Roth, “Generalization in adaptive data analysis and holdout reuse,” inAdvances in Neural Information Processing Systems, vol. 28, 2015
2015
-
[18]
Adaptive multi-resolution procedural knowledge compression for large language models,
C. Wang, W. Su, Q. Ai, Y. Tang, R. Qiao, X. Li, M. Zhang, and Y. Liu, “Adaptive multi-resolution procedural knowledge compression for large language models,”arXiv preprint arXiv:2606.12203, 2026
2026 arXiv
-
[19]
Tokmem: Tokenized procedural memory for large language models,
Z. Wu, Y. Hao, and L. Mou, “Tokmem: Tokenized procedural memory for large language models,”arXiv preprint arXiv:2510.00444, 2025
2025
-
[20]
Skill-to-lora: From using skills to learning behav- iors for token-efficient llm agents,
T. Zhang and Z. Qi, “Skill-to-lora: From using skills to learning behav- iors for token-efficient llm agents,”arXiv preprint arXiv:2606.16769, 2026
2026
-
[21]
Formal skill: Programmable runtime skills for efficient and accurate llm agents,
X. Zhang, M. Gao, Y. Zhao, X. Tan, Y. Yao, F. Wang, Y. Wang, Dingsiyi, and T. Yang, “Formal skill: Programmable runtime skills for efficient and accurate llm agents,”arXiv preprint arXiv:2605.19604, 2026
2026 arXiv
-
[22]
Skillrt: Compiling skills for efficient execution everywhere,
L. Chen, E. Feng, Y. Xia, and H. Chen, “Skillrt: Compiling skills for efficient execution everywhere,”arXiv preprint arXiv:2604.03088, 2026
2026 arXiv
-
[23]
P. D. Gr"unwald,The Minimum Description Length Principle. MIT Press, 2007
2007
-
[24]
The minimum description length principle for pattern mining: A survey,
E. Galbrun, “The minimum description length principle for pattern mining: A survey,”Data Mining and Knowledge Discovery, vol. 36, no. 5, pp. 1679–1727, 2022
2022
-
[25]
Identifying hierarchical structure in sequences: A linear-time algorithm,
C. G. Nevill-Manning and I. H. Witten, “Identifying hierarchical structure in sequences: A linear-time algorithm,”Journal of Artificial Intelligence Research, vol. 7, pp. 67–82, 1997
1997
-
[26]
Off-line dictionary-based compression,
N. J. Larsson and A. Moffat, “Off-line dictionary-based compression,” Proceedings of the IEEE, vol. 88, no. 11, pp. 1722–1732, 2000
2000
-
[27]
Qwen3.7: The agent frontier,
Qwen Team, “Qwen3.7: The agent frontier,” Qwen Research Blog, May 2026, introduces the Qwen3.7 series, including Qwen3.7-Max. Accessed: 2026-07-23. [Online]. Available: https://qwen.ai/blog?id=qwen3.7
2026
-
[28]
Qwen3.6-Plus: Towards real world agents,
——, “Qwen3.6-Plus: Towards real world agents,” Qwen Research Blog, Apr. 2026, accessed: 2026-07-23. [Online]. Available: https: //qwen.ai/blog?id=qwen3.6 9
2026
-
[29]
Kimi K2.6: Advancing open-source coding,
Moonshot AI, “Kimi K2.6: Advancing open-source coding,” Kimi Technical Blog, 2026, accessed: 2026-07-23. [Online]. Available: https://www.kimi.com/blog/kimi-k2-6
2026
-
[30]
The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models,
S. G. Patil, H. Mao, F. Yan, C. C.-J. Ji, V. Suresh, I. Stoica, and J. E. Gonzalez, “The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models,” in Proceedings of the 42nd International Conference on Machine Learning, ser. P...
2025
-
[31]
BFCL V4: Agentic part 1—web search,
Gorilla Team, “BFCL V4: Agentic part 1—web search,” Berkeley Function Calling Leaderboard Technical Blog, 2025, accessed: 2026- 07-23. [Online]. Available: https://gorilla.cs.berkeley.edu/blogs/15_bfcl_ v4_web_search.html
2025
-
[32]
LiveMathematicianBench: A live benchmark for mathematician-level reasoning with proof sketches,
L. He, Q. Yu, H. Dong, B. Liao, X. Xu, M. Goldblum, J. Bian, and N. Mesgarani, “LiveMathematicianBench: A live benchmark for mathematician-level reasoning with proof sketches,”arXiv preprint arXiv:2604.01754, 2026
2026
-
[33]
Spreadsheetbench: Towards challenging real world spreadsheet manipulation,
Z. Ma, B. Zhang, J. Zhang, J. Yu, X. Zhang, X. Zhang, S. Luo, X. Wang, and J. Tang, “Spreadsheetbench: Towards challenging real world spreadsheet manipulation,” inAdvances in Neural Information Processing Systems, vol. 37, 2024
2024
-
[34]
Memento-skills: Let agents design agents,
H. Zhou, S. Guo, A. Liu, Z. Yu, Z. Gong, B. Zhao, Z. Chen, M. Zhang, Y. Chen, J. Li, R. Yang, Q. Liu, X. Yu, J. Zhou, N. Wang, C. Sun, and J. Wang, “Memento-skills: Let agents design agents,” 2026. [Online]. Available: https://arxiv.org/abs/2603.18743 10 Appendix A Additional ...
2026
-
[35]
parse and validate the patch
-
[36]
record proposed operations and savings
-
[37]
apply them to a copy of the sidecar
-
[38]
validate coverage and JSON schema
-
[39]
render a temporary skill
-
[40]
atomically replace state and text; and
-
[41]
interface
commit the log entry. A crash before the final step leaves the previous skill unchanged. Repacking is performed in a separate transaction so failure cannot corrupt patch ingestion. Appendix C Contract Schema and Prompts A. Core Sidecar Schema { "interface": { "name": "string",...
-
[42]
Restate the problem in your own words; list every given quantity and the exact quantity to find
-
[43]
Classify the problem and choose a suitable method (direct computation, algebraic manipulation, casework, invariants, symmetry, recursion, or a known theorem) and briefly justify the choice
-
[44]
Introduce explicit notation for all unknowns and constraints before computing
-
[45]
Solve step by step, keeping exact values (fractions, radicals, symbolic constants such as pi or e) rather than rounding early
-
[46]
## Rules - Never round intermediate results unless the problem explicitly asks for a decimal approximation
Verify the result: substitute back into the original relations, sanity-check magnitude and units, and confirm every constraint and edge case holds. ## Rules - Never round intermediate results unless the problem explicitly asks for a decimal approximation. - Reduce fractions to...
-
[47]
Restate problem; list every given and exact quantity to find
-
[48]
Classify problem; choose suitable method (direct computation, algebraic manipulation, casework, invariants, symmetry, recursion, known theorem); justify
-
[49]
Introduce explicit notation all unknowns and constraints before computing
-
[50]
Solve step by step; keep exact values (fractions, radicals, symbolic constants such as pi or e) not rounding early
-
[51]
## Output - Show concise working; put final answer own line as \boxed{...}
Verify result: substitute original relations, sanity-check magnitude and units, confirm every constraint and edge case holds. ## Output - Show concise working; put final answer own line as \boxed{...}. - Give simplest exact form (e.g., 1/2 not 0.5) unless decimal explicitly re...
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.