Pith Number

pith:VTS4FYGP

pith:2026:VTS4FYGPL5HDKSOANCOC5C3PYD

not attested not anchored not stored refs resolved

NodeSynth: Socially Aligned Synthetic Data for AI Evaluation

Darlene Neal, Erin van Liemt, Jamila Smith-Loud, Kshitij Pancholi, Qazi Mamunur Rashid, Xuan Yang, Yanzhou Pan, Zhengzhe Yang

NodeSynth generates synthetic queries via a fine-tuned taxonomy generator that cause mainstream LLMs to fail at rates up to five times higher than human benchmarks.

arxiv:2605.14381 v1 · 2026-05-14 · cs.LG · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{VTS4FYGPL5HDKSOANCOC5C3PYD}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Evaluated against four mainstream LLMs, NodeSynth elicited failure rates up to five times higher than human-authored benchmarks.

C2weakest assumption

The synthetic queries produced by the fine-tuned TaG are representative of genuine sociotechnical risks without introducing new biases or artifacts that inflate failure rates.

C3one line summary

NodeSynth generates evidence-anchored synthetic queries that trigger up to five times higher failure rates in mainstream LLMs than human-authored benchmarks.

References

53 extracted · 53 resolved · 3 Pith anchors

[1] On llms-driven synthetic data generation, curation, and evaluation: A survey 2024

[2] Synthetic data in ai: Challenges, applications, and ethical implications.arXiv preprint arXiv:2401.01629, 2024 2024

[3] Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing 2024 · arXiv:2406.08464

[4] Self-instruct: Aligning language models with self-generated instruc- tions 2023

[5] Examining the expanding role of synthetic data throughout the ai development pipeline 2025

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-05-17T23:39:07.722958Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

ace5c2e0cf5f4e3549c0689c2e8b6fc0c31af8791d9f9c7e955ea666a80aa92c

Aliases

arxiv: 2605.14381 · arxiv_version: 2605.14381v1 · doi: 10.48550/arxiv.2605.14381 · pith_short_12: VTS4FYGPL5HD · pith_short_16: VTS4FYGPL5HDKSOA · pith_short_8: VTS4FYGP

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/VTS4FYGPL5HDKSOANCOC5C3PYD \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: ace5c2e0cf5f4e3549c0689c2e8b6fc0c31af8791d9f9c7e955ea666a80aa92c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "e6dad8f25c97add073feca94bb5ff6baac0b8f85b79c684b69bb68ace65cf674",
    "cross_cats_sorted": [
      "cs.CL"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-05-14T05:06:50Z",
    "title_canon_sha256": "263d734e60d3ddbd1ba6b2d605a0e9370e07088b3965fa962d22dcc9127bb2e1"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.14381",
    "kind": "arxiv",
    "version": 1
  }
}