Pith Number

pith:7IK6ILWQ

pith:2026:7IK6ILWQAPXKMPENTXFUTGBANA

not attested not anchored not stored refs pending

From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents

Jonas Robertson, Md Tahmid Rahman Laskar, Quinten McNamara, Seyyed Saeed Sarfjoo, Shashi Bhushan TN, Xue-Yong Fu

A dataset-agnostic framework converts text tool-calling benchmarks to paired audio versions via TTS and noise, showing model-dependent performance with small text-to-voice gaps of 1.8-4.8 points on Confetti and When2Call.

arxiv:2605.15104 v1 · 2026-05-14 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{7IK6ILWQAPXKMPENTXFUTGBANA}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Our dataset-agnostic framework uses text-to-speech, speaker variation, and environmental noise to create paired text-audio instances while preserving the original dataset annotations.

C2weakest assumption

That adding TTS, speaker variation, and environmental noise does not introduce new biases or artifacts that change how models interpret tool arguments or intent in ways that the preserved gold labels fail to capture.

C3one line summary

Receipt and verification

First computed	2026-05-17T21:40:25.803035Z
Last reissued	2026-05-17T21:57:19.134793Z
Builder	pith-number-builder-2026-05-17-v1
Signature	unsigned_v0
Schema	pith-number/v1.0

Canonical hash

fa15e42ed003eea63c8d9dcb49982068273784102384f097c27a1235d436fc3c

Aliases

arxiv: 2605.15104 · arxiv_version: 2605.15104v1 · pith_short_12: 7IK6ILWQAPXK · pith_short_16: 7IK6ILWQAPXKMPEN · pith_short_8: 7IK6ILWQ

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/7IK6ILWQAPXKMPENTXFUTGBANA \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: fa15e42ed003eea63c8d9dcb49982068273784102384f097c27a1235d436fc3c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "0ca004fe3ea5eccd10deb2c2fa2bcf5406961f23c64776f168d631d878553e81",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2026-05-14T17:22:42Z",
    "title_canon_sha256": "8e5b9e5718a5ec88afa0d84104e3d88ac3579debb8b5b46b0b714e9896a94ae2"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.15104",
    "kind": "arxiv",
    "version": 1
  }
}