Pith. sign in

REVIEW 4 cited by

Diversity of Thought Improves Reasoning Abilities of LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.07088 v2 pith:RJ5Y7WVB submitted 2023-10-11 cs.CL cs.AI

Ranjita Naik , Varun Chandrasekaran , Mert Yuksekgonul , Hamid Palangi , Besmira Nushi This is my paper · ORCID
classification cs.CLcs.AI
keywords reasoningdiversediversityapproachespromptcalldecodingensembling
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large language models (LLMs) are documented to struggle in settings that require complex reasoning. Nevertheless, instructing the model to break down the problem into smaller reasoning steps, or ensembling various generations through modifying decoding steps boosts performance. However, these methods assume that the input prompt is fixed and expect the decoding strategies to introduce the diversity needed for ensembling. In this work, we discuss how one can create and leverage variations of the input prompt as a means of diversity of thought. We propose a method that automatically improves prompt diversity by soliciting feedback from the LLM to ideate approaches that are apt for the problem. We then ensemble the diverse prompts in our method DIVSE (DIVerse reasoning path Self-Ensemble) across multiple inference calls, or use diverse approaches within a single inference call; we call the latter IDIV-SE (In-call DIVerse reasoning path Self-Ensemble). Apart from our approaches outperforming prior work, DIV-SE(in particular) advances state-of-the-art performance on the challenging planning and graph coloring benchmarks. Our results improve the Pareto frontier of the accuracy-cost trade-off.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation

    cs.CL 2025-02 conditional novelty 6.0 of 10

    BARE generates diverse, high-quality synthetic training data from only three seed examples by having a base model draft and an instruction-tuned model refine, improving downstream fine-tuning accuracy in few-shot settings.

  2. Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration

    cs.CL 2025-01 conditional novelty 6.0 of 10

    RAC ranks label descriptions by semantic similarity, then runs iterative binary LLM checks one label at a time, improving zero-shot labelling accuracy and enabling a precision-coverage trade-off.

  3. Optimizing Sequential Multi-Step Tasks with Parallel LLM Agents

    cs.MA 2025-07 conditional novelty 5.0 of 10

    Running multiple multi-agent teams in parallel, stopping at the first successful answer, cuts end-to-end latency by up to 2.2x with roughly unchanged completion rates on GAIA.

  4. Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models

    cs.LG 2024-12 conditional novelty 4.0 of 10

    This survey organizes LLM synthetic data research around quality, diversity, and complexity, claiming quality mainly helps in-distribution generalization, diversity mainly helps out-of-distribution generalization, and...

Pith tools