Pith Number

pith:YLYZW2EP

pith:2026:YLYZW2EP2NILYIOZY2B3LFFY3C

not attested not anchored not stored refs resolved

What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs

Fei Shen, Fei Yu, Haizhou Li, Jiaping Lin, Junzhe Li, Ming Li, Ping Nie

GUI grounding in VLMs follows a two-stage process where the prefill stage selects candidate UI elements that the decoding stage cannot correct.

arxiv:2605.12549 v1 · 2026-05-10 · cs.CV

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{YLYZW2EP2NILYIOZY2B3LFFY3C}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

we show that grounding follows a two-stage paradigm: the prefill stage determines candidate UI elements, while the decoding stage subsequently refines the final coordinates. This asymmetry establishes prefill as the critical step, as errors in candidate selection cannot be effectively corrected during decoding.

C2weakest assumption

That visual tokens receiving consistently high attention from the query (final) position across layers form a reliable preliminary target hypothesis, and that re-appending them with instruction hidden states enables effective re-thinking without adding noise or bias.

C3one line summary

GUI grounding in VLMs is bottlenecked by prefill-stage candidate selection that decoding cannot fix, so Re-Prefill uses attention to extract and re-inject target tokens for up to 4.3% gains on ScreenSpot-Pro.

References

41 extracted · 41 resolved · 4 Pith anchors

[1] Gui agents: A survey 2025

[2] Large language model-brained gui agents: A survey 2024

[3] Gui agents with foundation models: A comprehensive survey 2024

[4] GTA1: GUI test-time scaling agent 2026

[5] Gui-g2: Gaussian reward modeling for gui grounding 2026

Formal links

1 machine-checked theorem link

Receipt and verification

First computed	2026-05-18T03:10:02.164922Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

c2f19b688fd350bc21d9c683b594b8d89422b33c9315cffdad8c87e6c88f5d5b

Aliases

arxiv: 2605.12549 · arxiv_version: 2605.12549v1 · doi: 10.48550/arxiv.2605.12549 · pith_short_12: YLYZW2EP2NIL · pith_short_16: YLYZW2EP2NILYIOZ · pith_short_8: YLYZW2EP

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/YLYZW2EP2NILYIOZY2B3LFFY3C \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: c2f19b688fd350bc21d9c683b594b8d89422b33c9315cffdad8c87e6c88f5d5b

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "39b07e6b1acba186aa8e866003a84fa071b9bb523e7eec9096a8a203f7dc0a28",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CV",
    "submitted_at": "2026-05-10T07:04:07Z",
    "title_canon_sha256": "07560c2ec7a79bc099ccfc84abaad10b7e4cf1a639680238e83555b1289482bf"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.12549",
    "kind": "arxiv",
    "version": 1
  }
}