REVIEW 10 cited by
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
read the original abstract
A popular approach for improving the correctness of output from large language models (LLMs) is Self-Consistency - poll the LLM multiple times and output the most frequent solution. Existing Self-Consistency techniques always generate a constant number of samples per question, where a better approach will be to non-uniformly distribute the available budget based on the amount of agreement in the samples generated so far. In response, we introduce Adaptive-Consistency, a cost-efficient, model-agnostic technique that dynamically adjusts the number of samples per question using a lightweight stopping criterion. Our experiments over 17 reasoning and code generation datasets and three LLMs demonstrate that Adaptive-Consistency reduces sample budget by up to 7.9 times with an average accuracy drop of less than 0.1%. Our code and data are available at https://www.sample-step-by-step.info
Forward citations
Cited by 10 Pith papers
-
VecFontLLM: Anchor-Guided Direct Synthesis of Chinese Vector Fonts
An anchor-guided MLLM generates complex Chinese vector glyphs from one or a few style exemplars, decoupling coarse layout from Bézier curve completion to improve structure and editability.
-
TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning
A heterogeneous GNN that jointly routes each time-series query to the best modality–model pair under a user-chosen accuracy–cost trade-off, yielding large gains on four reasoning tasks.
-
TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning
A heterogeneous-graph router jointly selects the optimal modality (text, vision, or both) and model per time series query, beating prior routing baselines and generalizing to unseen models and tasks.
-
Small RL Controller, Large Language Model: RL-Guided Adaptive Sampling for Test-Time Scaling
RL-trained lightweight controller using answer statistics improves trade-offs among correctness, latency, and total samples in adaptive sampling for LLM test-time scaling.
-
Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scaling
DDC couples confidence-weighted Bayesian early stopping with trend-aware pruning of reasoning paths, cutting inference tokens by over 10x on some benchmarks while roughly maintaining accuracy.
-
Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answers
Tracking only the counts of the two most frequent LLM answers is asymptotically as sample-efficient as the full Bayesian posterior in an adaptive self-consistency procedure, cutting LLM calls by up to half.
-
Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling
C3RL is a new RL algorithm combining correctness, calibration, and reference accuracy rewards to improve LLM confidence calibration, enabling CAS to outperform majority voting with up to 12.33x lower inference cost.
-
Dual-Dimensional Consistency: Balancing Budget and Quality in Adaptive Inference-Time Scaling
DDC reduces token consumption by over 10x in LLM reasoning while maintaining or exceeding baseline accuracy across five benchmarks via adaptive path quality filtering.
-
Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
Empirical evaluation on Gemini 2.5 models shows self-consistency yields only 0.4% gain on HotpotQA and 1.6% on MATH-500 across 20 samples while token costs scale linearly, with performance plateauing or declining at h...
-
The Prompt Engineering Report Distilled: Quick Start Guide for Life Sciences
The paper reduces a broad set of prompt engineering techniques to six core approaches and applies them to life sciences use cases while addressing common LLM pitfalls.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.