Pith. sign in

Paper Citation Record · LEDGER

Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

As of 7 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 5 inbound Pith citation observations for arXiv:2403.06265.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2403.06265 v2

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 5 of 5 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-07T06:34:17.273281+00:00

measured 5 of 5 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-07T05:40:47.736679Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-18T05:52:24.931394Z

Reference resolution

0 of 0 outbound references displayed

  • verified exact0
  • verified fuzzy0
  • unresolved0
  • parse uncertain0
  • malformed identifier0
  • metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation 750354bf-6abd-424d-8f91-2a2e8c6c0b95 · inbound

Bit-level BPE: Below the byte boundary cites this paper.

Bit-level BPE: Below the byte boundary Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-07T05:40:47.736679Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-07T05:40:47.736679Z digest=sha256:d959480f52ffb00a47b27d3cdb61d3a6493e32ac626cd06a67ccab8a404618f2

Observation 2b14736c-197a-4e1b-8386-6d34f635b68d · inbound

Improving Large Language Models with Concept-Aware Fine-Tuning cites this paper.

Improving Large Language Models with Concept-Aware Fine-Tuning Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

Reference 18

Resolution
unresolved
no resolver link, observed 2026-08-07T05:28:23.856911Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-07T05:28:23.856911Z digest=sha256:01195167161255e0e2654c5960f31edc5c6d15966a3414e285c5763ed01fb698

Observation 1d5a7a66-27f0-4744-9fa0-329ba24fd0b1 · inbound

Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models cites this paper.

Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-05T22:43:22.326571Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-05T22:43:22.326571Z digest=sha256:63b1e9209e7e55a1fd7960985d805517d83751cbeb548f792968cc31c18a84bf

Observation 3804cdc8-8cc3-4706-8333-c75845585452 · inbound

Foundation Models for Discovery and Exploration in Chemical Space cites this paper.

Foundation Models for Discovery and Exploration in Chemical Space Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

Reference 156

Resolution
verified exact
arxiv_id, observed 2026-05-18T05:52:24.933919Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

source=pdf_text observed=2026-05-18T05:52:10.848118Z digest=sha256:dc27b6328f17c74c1617d50019d27ab1fba45bd1e3b2bc98a6168e68391b02c7

Observation a47a92df-9d5d-4660-a792-89b3ac0c11a7 · inbound

The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices? cites this paper.

The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices? Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

Reference 3

Resolution
metadata mismatch
arxiv_id, observed 2026-05-16T15:01:03.886358Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

source=pdf_text observed=2026-05-16T14:59:02.533789Z digest=sha256:a16e172dd863eee4f6be65e27a6740ec2c3ffb04435d2676f36b9c3c3dd5f600