Pith. sign in

REVIEW 2 cited by

Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.10691 v4 pith:UR3WJ2FG submitted 2024-02-16 cs.CL

Xianzhen Luo , Qingfu Zhu , Zhiming Zhang , Libo Qin , Xuanyu Zhang , Qing Yang , Dongliang Xu , Wanxiang Che This is my paper · ORCID
classification cs.CL
keywords languagelanguagespythonachievesacrossapproachbestmodels
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Currently, PoT primarily uses Python. However, relying solely on a single language may result in suboptimal solutions and overlook the potential benefits of other programming languages. In this paper, we conduct comprehensive experiments on the programming languages used in PoT and find that no single language consistently delivers optimal performance across all tasks and models. The effectiveness of each language varies depending on the specific scenarios. Inspired by this, we propose a task and model agnostic approach called MultiPoT, which harnesses strength and diversity from various languages. Experimental results reveal that it significantly outperforms Python Self-Consistency. Furthermore, it achieves comparable or superior performance compared to the best monolingual PoT in almost all tasks across all models. In particular, MultiPoT achieves more than 4.6% improvement on average on ChatGPT (gpt-3.5-turbo-0701).

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Format-Adapter: Improving Reasoning Capability of LLMs by Adapting Suitable Format

    cs.CL 2025-06 conditional novelty 5.0 of 10

    FORMAT-ADAPTER automatically generates and selects per-question reasoning formats for LLMs, improving vote-based accuracy by around 4.3% over prior multi-format methods.

  2. Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering

    cs.AI 2024-11 conditional novelty 4.0 of 10

    A survey and position paper that reframes post-training of foundation models as a search, verify, and feedback pipeline called verifier engineering.

Pith tools