REVIEW 8 cited by
Language Model Cascades
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are probabilistic models, and may be expressed in the language of graphical models with random variables whose values are complex data types such as strings. Cases with control flow and dynamic structure require techniques from probabilistic programming, which allow implementing disparate model structures and inference strategies in a unified language. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use. We refer to the resulting programs as language model cascades.
Forward citations
Cited by 8 Pith papers
-
Why Ask One When You Can Ask $k$? Learning-to-Defer to the Top-$k$ Experts
This paper introduces top-k learning-to-defer, where a single k-independent policy routes each query to the k highest-scored experts, plus an adaptive variant that chooses k per query.
-
Auto: The AGI Compiler
AUTO compiles witnessed-deterministic LLM-agent spans into verified WASM cognition binaries and recompiles on deopt, cutting cost 6.4× at 96.9% parity on a 300-item shifted stream.
-
Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference
Using conformal prediction-set size as the cascade deferral rule yields distribution-free cascade accuracy bounds and usually beats confidence-threshold heuristics on multiple-choice LLM benchmarks.
-
Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models
A hybrid language-model and probabilistic-program architecture predicts human judgments on novel open-world reasoning vignettes better than language-model-only baselines.
-
Understanding the Logic of Direct Preference Alignment through Logic
Direct preference alignment losses can be expressed as logical programs over model predictions, yielding an organized landscape of billions of definable losses and a route to new variants.
-
What makes prompts a graph: necessary and sufficient conditions for prompt graph engineering
The paper defines prompt graph engineering via four necessary and sufficient conditions (explicit structure, structure/content separation, executable semantics, first-class artifact) and an inclusion/exclusion test th...
-
Template-Based Financial Report Generation in Agentic and Decomposed Information Retrieval
Decomposing a report template into per-section subqueries yields more detailed and better-scored financial reports than a multi-agent system that uses the full template at once.
-
MultiQ&A: An Analysis in Measuring Robustness via Automated Crowdsourcing of Question Perturbations and Answers
MultiQ&A automatically rephrases questions and crowdsources answers from LLM agents, showing gpt-3.5-turbo is fairly stable on extractive and multiple-choice QA but more variable on free-form answers.
Discussion (0). Continue with ORCID to comment.