Pith Number

pith:Y3FKY7FQ

pith:2026:Y3FKY7FQC3KK5B7Z3FBARM3A6X

not attested not anchored not stored refs pending

Approximate Next Policy Sampling: Replacing Conservative Target Policy Updates in Deep RL

Dillon Sandhu, Ronald Parr

Approximate Next Policy Sampling lets deep RL agents make larger policy updates safely by approximating the next policy's state distribution.

arxiv:2605.05481 v2 · 2026-05-06 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{Y3FKY7FQC3KK5B7Z3FBARM3A6X}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

If certain stability criteria are met, the update is guaranteed to be safe; otherwise, it remains no less safe than standard approximate policy iteration. Applying SV-API to PPO yields SV-PPO, which matches or improves performance on high-dimensional discrete (Atari) and continuous control benchmarks while executing substantially larger target policy updates.

C2weakest assumption

That the iteratively updated behavioral policy can produce a training distribution that sufficiently approximates the next policy's state-visitation distribution, and that the stability criteria can be reliably defined and checked in practice without introducing new instabilities.

C3one line summary

Approximate Next Policy Sampling approximates the next policy's state distribution during training to enable larger safe policy updates in deep RL, demonstrated by SV-PPO matching or exceeding standard PPO on Atari and continuous control tasks.

Receipt and verification

First computed	2026-06-19T16:12:20.544354Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

c6caac7cb016d4ae87f9d94208b360f5e2d980f058109bb59c8601ca090d6c9c

Aliases

arxiv: 2605.05481 · arxiv_version: 2605.05481v2 · doi: 10.48550/arxiv.2605.05481 · pith_short_12: Y3FKY7FQC3KK · pith_short_16: Y3FKY7FQC3KK5B7Z · pith_short_8: Y3FKY7FQ

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/Y3FKY7FQC3KK5B7Z3FBARM3A6X \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: c6caac7cb016d4ae87f9d94208b360f5e2d980f058109bb59c8601ca090d6c9c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "45a1fcfab309665f999bf197fc9067f73ad0fee6e5a48de52acc5a56adb05c6c",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-05-06T22:02:35Z",
    "title_canon_sha256": "f7963600e474df518e50b85ebdc6222fd1afc458b39f64097e09183dc5d15e3b"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.05481",
    "kind": "arxiv",
    "version": 2
  }
}