Pith Number

pith:EJVCKAVC

pith:2023:EJVCKAVCH6TQ4G5ONYV5RXEWIT

not attested not anchored not stored refs resolved

ART: Automatic multi-step reasoning and tool-use for large language models

Bhargavi Paranjape, Hannaneh Hajishirzi, Luke Zettlemoyer, Marco Tulio Ribeiro, Sameer Singh, Scott Lundberg

ART lets large language models automatically generate multi-step reasoning programs that call external tools.

arxiv:2303.09014 v1 · 2023-03-16 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{EJVCKAVCH6TQ4G5ONYV5RXEWIT}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

ART achieves a substantial improvement over few-shot prompting and automatic CoT on unseen tasks in the BigBench and MMLU benchmarks, and matches performance of hand-crafted CoT prompts on a majority of these tasks.

C2weakest assumption

That a fixed task library contains sufficiently diverse and high-quality demonstrations so that nearest-neighbor selection reliably supplies useful programs for entirely new, unseen tasks.

C3one line summary

ART automatically generates multi-step reasoning programs with tool integration for LLMs, yielding substantial gains over few-shot and auto-CoT prompting on BigBench and MMLU while matching hand-crafted CoT on most tasks.

References

267 extracted · 267 resolved · 38 Pith anchors

[15] Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2022. https://doi.org/10.18653/v1/2022.acl-long.579 I nternet-augmented dialogue generation . In Proceedings of the 60th Annual Meeting of the Associat 2022 · doi:10.18653/v1/2022.acl-long.579

[22] Are NLP Models really able to Solve Simple Math Word Problems? 2021 · doi:10.18653/v1/2021.naacl-main.168

[33] Chain-of-thought prompting elicits reasoning in large language models

[37] Finetuned Language Models Are Zero-Shot Learners · arXiv:2109.01652

[38] Chain of Thought Prompting Elicits Reasoning in Large Language Models , author=. ArXiv , year=

Formal links

2 machine-checked theorem links

Cited by

29 papers in Pith

Bridging Language Models and Financial Analysis

Large Language Model Agent: A Survey on Methodology, Applications and Challenges

When control meets large language models: From words to dynamics

A Comprehensive Overview of Large Language Models

TSGuard: Automated User-Centric Incident Diagnosis for AI Workloads in the Cloud

Receipt and verification

First computed	2026-05-17T23:38:46.903881Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

226a2502a23fa70e1bae6e2bd8dc9644d215cffc162551cfe5ac94e7997a9c91

Aliases

arxiv: 2303.09014 · arxiv_version: 2303.09014v1 · doi: 10.48550/arxiv.2303.09014 · pith_short_12: EJVCKAVCH6TQ · pith_short_16: EJVCKAVCH6TQ4G5O · pith_short_8: EJVCKAVC

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/EJVCKAVCH6TQ4G5ONYV5RXEWIT \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 226a2502a23fa70e1bae6e2bd8dc9644d215cffc162551cfe5ac94e7997a9c91

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "fc18cfe94626e98794805db81b978304805750ed8622ad370687008a6b807ec8",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2023-03-16T01:04:45Z",
    "title_canon_sha256": "cad4a37e49b9b66ccfa0ac42edf9a2ea6626c8aa4df881435177bfde702d5028"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2303.09014",
    "kind": "arxiv",
    "version": 1
  }
}