Pith. sign in

REVIEW 5 cited by

Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.02319 v2 pith:RH5EG2Z4 submitted 2024-04-02 cs.CL cs.AIcs.LG

Tobias Schnabel , Jennifer Neville This is my paper
classification cs.CLcs.AIcs.LG
keywords promptprogramssammoprogramsymboliccompile-timedifferentoptimization
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In many modern LLM applications, such as retrieval augmented generation, prompts have become programs themselves. In these settings, prompt programs are repeatedly called with different user queries or data instances. A big practical challenge is optimizing such prompt programs. Recent work has mostly focused on either simple prompt programs or assumed that the general structure of a prompt program is fixed. We introduce SAMMO, a framework to perform symbolic prompt program search for compile-time optimizations of prompt programs. SAMMO represents prompt programs on a symbolic level which allows for a rich set of transformations that can be searched over during optimization. We show that SAMMO generalizes previous methods and improves the performance of complex prompts on (1) instruction tuning, (2) RAG pipeline tuning, and (3) prompt compression, across several different LLMs. We make all code available open-source at https://github.com/microsoft/sammo .

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TSCG: Deterministic Tool-Schema Compilation for Agentic LLM Deployments

    cs.SE 2026-05 unverdicted novelty 7.0 of 10

    TSCG compiles JSON tool schemas into token-efficient structured text, raising tool-use accuracy for small LLMs from 0% to 84.4% on benchmarks while cutting tokens by 52-57%.

  2. Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

    cs.SE 2025-09 unverdicted novelty 7.0 of 10

    Clotho ranks LLM test inputs by failure likelihood using pre-generation hidden states and GMMs, achieving 0.716 ROC-AUC after labeling 5.4% of inputs on average across eight tasks and three models, with transfer to pr...

  3. Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

    cs.SE 2025-09 conditional novelty 6.0 of 10

    Clotho ranks LLM test inputs by failure likelihood using a Gaussian mixture model over pre-generation hidden states of passing inputs, reaching 0.716 ROC-AUC with 5.4% labelling and transferring from open-weight to pr...

  4. Prompt Orchestration Markup Language

    cs.HC 2025-08 conditional novelty 6.0 of 10

    POML is a markup language that structures LLM prompts, embeds multimodal data, and decouples formatting via stylesheets, with case studies showing strong prompt format sensitivity.

  5. Type-Driven Prompt Programming: From Typed Interfaces to a Calculus of Constraints

    cs.PL 2025-08 conditional novelty 6.0 of 10

    The paper proposes a not-yet-complete dependently typed calculus for prompt programming with probabilistic refinements, and identifies gaps in constraint expressiveness and optimization algorithms.

Pith tools