REVIEW 1 cited by
Self-Consistency Boosts Calibration for Math Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Self-Consistency Boosts Calibration for Math Reasoning
read the original abstract
Calibration, which establishes the correlation between accuracy and model confidence, is important for LLM development. We design three off-the-shelf calibration methods based on self-consistency (Wang et al., 2022) for math reasoning tasks. Evaluation on two popular benchmarks (GSM8K and MathQA) using strong open-source LLMs (Mistral and LLaMA2), our methods better bridge model confidence and accuracy than existing methods based on p(True) (Kadavath et al., 2022) or logit (Kadavath et al., 2022).
Forward citations
Cited by 1 Pith paper
-
Quantifying Consistency in LLM Logical Reasoning via Structural Uncertainty
Structural uncertainty from self-preference-induced rankings of LLM reasoning paths complements answer dispersion for identifying unreliable instances on logical tasks while collapsing on factual retrieval.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.