Pith Number

pith:OSJNOZG2

pith:2023:OSJNOZG2B2GXW2D6J46HAMVB5Q

not attested not anchored not stored refs pending

LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks

Ayse Coskun, Gianluca Stringhini, Hammond Pearce, Mingji Han, Saad Ullah, Saurabh Pujar

arxiv:2312.12575 v3 · 2023-12-19 · cs.CR

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{OSJNOZG2B2GXW2D6J46HAMVB5Q}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

4 papers in Pith

An Empirical Study of Security Calibration in Large Language Models for Code

XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants

ASSEMBLAGE-DEEPHISTORY: A Cross-Build Binary Dataset with Temporal Coverage

LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software

Receipt and verification

First computed	2026-07-05T08:47:48.832037Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

7492d764da0e8d7b687e4f3c7032a1ec2474582f68792403db9ea7cc1bf6e932

Aliases

arxiv: 2312.12575 · arxiv_version: 2312.12575v3 · doi: 10.48550/arxiv.2312.12575 · pith_short_12: OSJNOZG2B2GX · pith_short_16: OSJNOZG2B2GXW2D6 · pith_short_8: OSJNOZG2

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/OSJNOZG2B2GXW2D6J46HAMVB5Q \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 7492d764da0e8d7b687e4f3c7032a1ec2474582f68792403db9ea7cc1bf6e932

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "46474d9a05ae9efddb4ebb1dec700f48333f99663dd9d28c41477887ed22ac2d",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CR",
    "submitted_at": "2023-12-19T20:19:43Z",
    "title_canon_sha256": "1b05f00e8976c23902841cb6bba74290344182f75cb5a55d7769be7c21c8e8d2"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2312.12575",
    "kind": "arxiv",
    "version": 3
  }
}