Pith. sign in

Paper Citation Record · LEDGER

Explicit Boundary Markers for Subword Vocabularies

As of 14 August 2026, this Paper Citation Record lists 67 of 67 outbound references and 0 inbound Pith citation observations for arXiv:2608.08847.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2608.08847 v2

Coverage vector

measured 67 of 67 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-14T04:32:08.796515Z

measured 67 of 67 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-14T06:32:32.682623+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

67 of 67 outbound references displayed

  • verified exact1
  • verified fuzzy18
  • unresolved46
  • parse uncertain1
  • malformed identifier0
  • metadata mismatch1

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation c18acb7d-cdee-4cae-891f-e7f30f64bfb9 · outbound

This paper cites 2025 , booktitle=.

Explicit Boundary Markers for Subword Vocabularies 2025 , booktitle=

Reference 1

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.767994Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.501308Z digest=sha256:4e8a69aff0d4f91a27d249243692a72daeaedb958d2f62630f9e170ae3adbf5e

Observation b965d0a2-8d35-4304-8e8f-11df574d9d22 · outbound

This paper cites 2019 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2019 , eprint=

Reference 2

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.752877Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.507787Z digest=sha256:31ac4eca12afd20f0ced69f07bf6dcdfbdc5c4c818f1019feab298a815166b08

Observation 1781d9c0-41cd-4564-9c08-e37ee98cffb0 · outbound

This paper cites Say Anything but This: When Tokenizer Betrays Reasoning in.

Explicit Boundary Markers for Subword Vocabularies Say Anything but This: When Tokenizer Betrays Reasoning in

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.512381Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.512381Z digest=sha256:62bbc0163b796cf6cd9074c1bcea653d23907bc6242b41c6aae5ccd096519202

Observation 2c241548-dcac-4f8b-98da-2743803f11f9 · outbound

This paper cites Improving Tokenisation by Alternative Treatment of Spaces.

Explicit Boundary Markers for Subword Vocabularies Improving Tokenisation by Alternative Treatment of Spaces

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.516904Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.516904Z digest=sha256:602717a181371750bc3dd500f34226343a72d51f57c900201484db7d5ad4314a

Observation 369574cf-3149-43b3-8458-6025fd598a84 · outbound

This paper cites Tokenization Falling Short: On Subword Robustness in Large Language Models.

Explicit Boundary Markers for Subword Vocabularies Tokenization Falling Short: On Subword Robustness in Large Language Models

Reference 5

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.521538Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.521538Z digest=sha256:d18ef2a880bbd4ebdaaa82159e84331f651526c2810fef03377c73c4d57bcc2d

Observation 4741079b-0c12-47ba-8368-08568c8ec030 · outbound

This paper cites MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment.

Explicit Boundary Markers for Subword Vocabularies MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment

Reference 6

Resolution
metadata mismatch
local_arxiv, observed 2026-08-14T04:32:09.160993Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.526111Z digest=sha256:50c5954e1113486a13d802d08e0caa5152e8fada29ee4d775cd7b2ae8bcbdb20

Observation f126473a-1ca8-42d0-a87d-eba91bb316a3 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.530890Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.530890Z digest=sha256:13e0ee28fefeca5f2d15f3f1244a6d5a3175293cbf2f9dedd88b56561a9efaa2

Observation f8b9c1ec-f24c-40e9-a0d9-a30e3ff7a6eb · outbound

This paper cites doi:10.5281/zenodo.7199437 , url =.

Explicit Boundary Markers for Subword Vocabularies doi:10.5281/zenodo.7199437 , url =

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.535120Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.535120Z digest=sha256:2cb92057f07bac43748f0ba4554fe9515f15c2e150b362cb5f40ce168d193f8c

Observation ca47e64a-b520-4106-8efd-088025906699 · outbound

This paper cites 2026 , month = jul, day =.

Explicit Boundary Markers for Subword Vocabularies 2026 , month = jul, day =

Reference 9

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.726777Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.540015Z digest=sha256:80321adca203d871f456503ff30d2cb42aca1b9908a612eddf62eb9562cc270d

Observation 0f4643fe-4c1b-4d5c-862a-a36d4891f421 · outbound

This paper cites 2022 , url =.

Explicit Boundary Markers for Subword Vocabularies 2022 , url =

Reference 10

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.544531Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.544531Z digest=sha256:313b6268b6eb62cf1e24584419760cefab093332671e5b71ca663048f5750f90

Observation f0e05985-c6c4-4c71-83c0-933baa27fb00 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 11

Resolution
unresolved
raw_fallback, observed 2026-08-14T04:32:09.700968Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.549296Z digest=sha256:82a18228cdae1461a182e211f7688c2fb9f633499d8e6076dec4cb1fb211fcd7

Observation bc50c1dc-19df-4e6d-8e45-084ed9ce70b6 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 12

Resolution
unresolved
raw_fallback, observed 2026-08-14T04:32:09.685802Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.553633Z digest=sha256:2da4c83f405eb7cec734ffde386992c3d04e11af98113ff2056899b5e75b33cc

Observation 0401d9a3-23e0-4a9e-81ae-b0fbfa592440 · outbound

This paper cites and Reddy, Varshini and Zhang, Haoran and Alameddine, Alec and Uzan, Omri and Pinter, Yuval and Tanner, Chris.

Explicit Boundary Markers for Subword Vocabularies and Reddy, Varshini and Zhang, Haoran and Alameddine, Alec and Uzan, Omri and Pinter, Yuval and Tanner, Chris

Reference 13

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.557843Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.557843Z digest=sha256:f84d0d538da9fc83d755337f7c82a6525aac44af37f6131812e22bd2dfe8bc2b

Observation 21c68cfd-e18d-4124-8c88-8aadd65df67a · outbound

This paper cites and Tanner, Chris and Pinter, Yuval.

Explicit Boundary Markers for Subword Vocabularies and Tanner, Chris and Pinter, Yuval

Reference 14

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.562168Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.562168Z digest=sha256:6270c5140437ed2de4c2425375f0da8b63029698cff55b40b4c00b84c856b5c7

Observation a578bef0-9049-423c-879b-261ce4d84ab1 · outbound

This paper cites Tokenization and the Noiseless Channel.

Explicit Boundary Markers for Subword Vocabularies Tokenization and the Noiseless Channel

Reference 15

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.566529Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.566529Z digest=sha256:f52c01386b7444e460b1ff5de3c13223341d3614ccb3ab2793fd36c24b2ec6ed

Observation 503dc684-d99a-40b5-b1bc-f5d98f35277c · outbound

This paper cites Two Counterexamples to Tokenization and the Noiseless Channel.

Explicit Boundary Markers for Subword Vocabularies Two Counterexamples to Tokenization and the Noiseless Channel

Reference 16

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.570953Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.570953Z digest=sha256:bdac978300f442897de80c33d514bf477e43d3c893f8a1754fa69a6f6bfefcd4

Observation 42f779e3-75b3-4435-a198-4b85e4b60a15 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 17

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.575493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.575493Z digest=sha256:ce2cc480387b5883b3875d8059b750ad586d4fe5a3835d23790c9739de63f260

Observation ee460b06-900d-456b-838c-95ee4bc929d4 · outbound

This paper cites Byte Pair Encoding is Suboptimal for Language Model Pretraining.

Explicit Boundary Markers for Subword Vocabularies Byte Pair Encoding is Suboptimal for Language Model Pretraining

Reference 18

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.579754Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.579754Z digest=sha256:8c2821ecd6a2255713e0ad1fb3613de9575eae15be64cfa6fb4381945ac9322d

Observation 935f6ca1-ac75-433b-8454-e1685b4d0de1 · outbound

This paper cites Incorporating Context into Subword Vocabularies.

Explicit Boundary Markers for Subword Vocabularies Incorporating Context into Subword Vocabularies

Reference 19

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.584043Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.584043Z digest=sha256:846f296ace3736bf88fbaa3ffc15c7108977e2cd228b91f73965c5890adc628f

Observation 1d556313-8b9e-4f1b-a396-9c90749a5b8d · outbound

This paper cites Rethinking Tokenization for Rich Morphology: The Dominance of U nigram over BPE and Morphological Alignment.

Explicit Boundary Markers for Subword Vocabularies Rethinking Tokenization for Rich Morphology: The Dominance of U nigram over BPE and Morphological Alignment

Reference 20

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.588319Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.588319Z digest=sha256:3f24550c9935e026cc89b2b92460a776938bd9cdd7e1ffae22fc785e7b244b54

Observation d281eeb7-d8ef-43ab-9e61-28bda6313ae5 · outbound

This paper cites MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies.

Explicit Boundary Markers for Subword Vocabularies MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies

Reference 21

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.592496Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.592496Z digest=sha256:1f9502523f66ca243ff6fb05f4afb4ed54f35219d383699791f4cf6a11c8a4a5

Observation c8ad4a85-151b-4e07-bcc4-e9b5e2f0c03f · outbound

This paper cites Which Pieces Does U nigram Tokenization Really Need?.

Explicit Boundary Markers for Subword Vocabularies Which Pieces Does U nigram Tokenization Really Need?

Reference 22

Resolution
verified exact
doi, observed 2026-08-14T04:32:08.978562Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.597667Z digest=sha256:07f008317f08a3b81f8b79a3b90c02b4f52257aeb471df6b8ce1641e23fce6f4

Observation b2c1c6b8-4ce0-4d9c-9333-f76cd98655c2 · outbound

This paper cites 2025 , publisher =.

Explicit Boundary Markers for Subword Vocabularies 2025 , publisher =

Reference 23

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.649697Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.601886Z digest=sha256:e3560b64d55d7008dda3d149aba7fb3bbaab16ca26eddc272acd4bad9efec549

Observation 3f3c5379-d3e3-43f9-96de-9085c8a8ec8c · outbound

This paper cites 2020 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2020 , eprint=

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.606012Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.606012Z digest=sha256:2d3d3a94e1583f8a2905977856adf18248a838bf73bbc22be73a4988b741b1a1

Observation 440d6026-698e-43b0-85f0-b7f591eb124f · outbound

This paper cites Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features.

Explicit Boundary Markers for Subword Vocabularies Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features

Reference 25

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.610131Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.610131Z digest=sha256:91a2605c989daeec42d8d4ab547422af2d2663372841fc827be074e2cd7a1689

Observation 35571c34-544c-47a2-94ff-48bf0d9fcc16 · outbound

This paper cites 2025 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2025 , eprint=

Reference 26

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.621724Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.614523Z digest=sha256:bae80db20091154250d244a3192ca832e869ad4a2b52db05815ec9b531e4d612

Observation 0b301b75-2c55-43e7-bcfe-68543be61bd1 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 27

Resolution
unresolved
raw_fallback, observed 2026-08-14T04:32:09.605669Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.618825Z digest=sha256:32d381cb8c9339272d0ac5f33c70eed8cabad09c2edbb65400089d4f2701350e

Observation c288f69d-93ca-498d-b1f5-e7626179dd85 · outbound

This paper cites SentencePiece : A simple and language independent subword tokenizer and detokenizer for Neural Text Processing.

Explicit Boundary Markers for Subword Vocabularies SentencePiece : A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

Reference 28

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.623089Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.623089Z digest=sha256:89ff65aa16000d3b5728fa9fa22de1b8e25e0a967ce26565a2c415b39043348e

Observation 3c8f8c9e-c8b2-43ac-adbf-037a289fb8e5 · outbound

This paper cites Combinatorial Pattern Matching , pages=.

Explicit Boundary Markers for Subword Vocabularies Combinatorial Pattern Matching , pages=

Reference 29

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.590287Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.627520Z digest=sha256:e3b95217890f647810b4165764ebaff985a1eee2a531f792f22a6e131b68bfce

Observation cc839931-7172-4887-bbed-02d59c92bdb6 · outbound

This paper cites ICML 2025 Tokenization Workshop , url=.

Explicit Boundary Markers for Subword Vocabularies ICML 2025 Tokenization Workshop , url=

Reference 30

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.573221Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.631915Z digest=sha256:f8a9fdf2792f110adc7397362b2944d0bcafa9692e2897c89a13bcb3cc18ba87

Observation e8c01c50-d811-4217-8f87-aeef26c8643c · outbound

This paper cites BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training.

Explicit Boundary Markers for Subword Vocabularies BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training

Reference 31

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.636165Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.636165Z digest=sha256:23fe1dc98a0dd7474b4e482f87c30a53c547b42afac1b884cd7a34dc2b18a6ac

Observation 210332ec-3e1a-4438-989d-fb19c090adc4 · outbound

This paper cites and Bergen, Benjamin.

Explicit Boundary Markers for Subword Vocabularies and Bergen, Benjamin

Reference 32

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.555570Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.640588Z digest=sha256:25601a15d46893c0a5f76544da6ffac79d1cd5c2e1bf5efe3c5b3c43cef5440f

Observation 36dcc0f7-5aaf-4d9d-ac4f-1a9101310ff2 · outbound

This paper cites Neural Machine Translation of Rare Words with Subword Units.

Explicit Boundary Markers for Subword Vocabularies Neural Machine Translation of Rare Words with Subword Units

Reference 33

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.644857Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.644857Z digest=sha256:343f944657edb591b367e4d4186663495da57a73ca5e6a667160efc910622ed5

Observation 12d5a6d2-733c-4b29-bd8e-ce3d6cd24e35 · outbound

This paper cites Proceedings of the 31st International Conference on Computational Linguistics.

Explicit Boundary Markers for Subword Vocabularies Proceedings of the 31st International Conference on Computational Linguistics

Reference 34

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.538164Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.650141Z digest=sha256:4f97c0d4d0cba0b1b8715faa1cb33468a877494912a67b53aab9fe76fe6dbfb8

Observation b1eade6e-3477-46a0-9915-cc72bdc2c8ba · outbound

This paper cites Pre-tokenization on punctuation in.

Explicit Boundary Markers for Subword Vocabularies Pre-tokenization on punctuation in

Reference 35

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.522373Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.654554Z digest=sha256:15e101b672b169595da25c302077a90b2e808fefdd6c31cb766311a66c6c06e1

Observation f1eccd93-d700-4501-ba30-f13f45e3191f · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 36

Resolution
parse uncertain
raw_fallback, observed 2026-08-14T04:32:09.506421Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.658729Z digest=sha256:90a27d377b0c208527af6e60cbbd037a47ae5d36a62029d18afed3e6224e019c

Observation bbf3f645-201b-4ed4-b11e-48485c39c1b9 · outbound

This paper cites A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages.

Explicit Boundary Markers for Subword Vocabularies A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

Reference 37

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.490232Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.663227Z digest=sha256:556a716d51d03e9174996740fa601a24b61dd24a36f797d438c5e6d5723404f3

Observation 3a908081-7600-456d-973d-27f1334a3d36 · outbound

This paper cites Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures , series =.

Explicit Boundary Markers for Subword Vocabularies Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures , series =

Reference 38

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.667481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.667481Z digest=sha256:e9b12afb142431f5079b6f2f646578876272b76d1bd83f5d52866c80190a39cb

Observation 1f33e711-3560-4865-ac4a-3ee19212a5bc · outbound

This paper cites CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages.

Explicit Boundary Markers for Subword Vocabularies CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

Reference 39

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.671910Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.671910Z digest=sha256:d922a45c49fdc8a1bb98224366d7b1bc8b9c21e0b983179598a8a3ca0c4cd45c

Observation d657a32b-281c-40f8-b587-8c858bb377da · outbound

This paper cites Jamo-Level Subword Tokenization in Low-Resource K orean Machine Translation.

Explicit Boundary Markers for Subword Vocabularies Jamo-Level Subword Tokenization in Low-Resource K orean Machine Translation

Reference 40

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.473882Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.676436Z digest=sha256:26ca532878f68c48d8d70810039870aa2d8e58d7b8f330a015daedd1395d3ef1

Observation be0f5dd0-5c53-4488-8ce6-56c4948b9316 · outbound

This paper cites Fishing for M agikarp: Automatically Detecting Under-trained Tokens in Large Language Models.

Explicit Boundary Markers for Subword Vocabularies Fishing for M agikarp: Automatically Detecting Under-trained Tokens in Large Language Models

Reference 41

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.680957Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.680957Z digest=sha256:463c9513b639d998c3a1e6b1f487aa944fba463521c557d3995b1d22777ee28d

Observation dcb03b0c-dc5b-4a0d-b9e3-1262318dea22 · outbound

This paper cites HuggingFace's Transformers: State-of-the-art Natural Language Processing.

Explicit Boundary Markers for Subword Vocabularies HuggingFace's Transformers: State-of-the-art Natural Language Processing

Reference 42

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.685393Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.685393Z digest=sha256:e65968e50ff07ea4659b998e8a83093fd15ba7532aa0b8b53746a14a36dd9fe2

Observation 7b983339-8701-4503-9b88-14ecdd2ed58b · outbound

This paper cites 2017 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2017 , eprint=

Reference 43

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.690142Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.690142Z digest=sha256:6726f261984c02203571744e769d8ebd7cbf026e76292bab24440e5fe3ae24c2

Observation a3fd1f5b-7114-4a39-a317-7c7abe58e897 · outbound

This paper cites Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates.

Explicit Boundary Markers for Subword Vocabularies Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

Reference 44

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.694268Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.694268Z digest=sha256:95c12f2a12d0f959d29ea8915e721f1eb218ca8e35564cd930a7f0b7d1f45492

Observation 95e4aa0c-f6ae-4952-acbc-4ac8f4c3e651 · outbound

This paper cites BPE -Dropout: Simple and Effective Subword Regularization.

Explicit Boundary Markers for Subword Vocabularies BPE -Dropout: Simple and Effective Subword Regularization

Reference 45

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.698756Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.698756Z digest=sha256:1d88f3a4e1912d34e7e8f5a5e6a2f84d0368cabfef246ab7da562113d09c7f3e

Observation e0b483f3-2cbf-4b91-9682-8e714fa9f6a0 · outbound

This paper cites Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP.

Explicit Boundary Markers for Subword Vocabularies Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP

Reference 46

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.702904Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.702904Z digest=sha256:55fe14f16cbca35e8e99080ae46b933c6b1abe19297671136066ac29ad18079f

Observation e3311064-0e79-4bd9-bcd1-8fcad81a62af · outbound

This paper cites Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications.

Explicit Boundary Markers for Subword Vocabularies Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications

Reference 47

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.707592Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.707592Z digest=sha256:40ca622e18430de75d3d3bbe455520c1ffbfc3cf266052be0276989139f46997

Observation 70936cb0-be81-4845-bc76-81940b2704cd · outbound

This paper cites 2025 , url=.

Explicit Boundary Markers for Subword Vocabularies 2025 , url=

Reference 48

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.441602Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.711821Z digest=sha256:32e21fecd64f3c36d207685fbaee06197b48f3ab0cb06124967869863f2012f7

Observation b2f14f14-e1a0-4f06-b2eb-1f86850af6b1 · outbound

This paper cites 2025 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2025 , eprint=

Reference 49

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.715984Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.715984Z digest=sha256:c3fcc0d06ab1f786c69d253ccdf85b96fa61512dc876f8bee62167b3281ee7ce

Observation 680412f5-6602-4c60-bab0-15de5187e626 · outbound

This paper cites 2026 , url =.

Explicit Boundary Markers for Subword Vocabularies 2026 , url =

Reference 50

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.414943Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.720445Z digest=sha256:7e639533214fca2375742b3550e5dd4073bc8d43fbc61528a288a1dcdae22c68

Observation 754e9bd6-3320-4a17-828b-dd7f9851e9a8 · outbound

This paper cites Language Model Tokenizers Introduce Unfairness Between Languages , url =.

Explicit Boundary Markers for Subword Vocabularies Language Model Tokenizers Introduce Unfairness Between Languages , url =

Reference 51

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.399264Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.724598Z digest=sha256:c7883155249b63e1a5bfaea4f481151cf9628f5160a794560c69015ec0c414ff

Observation 1b8213fc-73a6-4f7d-bf7c-3dbeb8697782 · outbound

This paper cites Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models.

Explicit Boundary Markers for Subword Vocabularies Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Reference 52

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.728737Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.728737Z digest=sha256:c5cc0dea6a85b86476687f160c2e612bd909757e8f5b528241fc4ea0c830aac2

Observation fa60bca4-a8a8-4e4e-950b-a6e56d713518 · outbound

This paper cites Tokenizer Choice For LLM Training: Negligible or Crucial?.

Explicit Boundary Markers for Subword Vocabularies Tokenizer Choice For LLM Training: Negligible or Crucial?

Reference 53

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.733224Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.733224Z digest=sha256:c8406cae3abfa24c011ca55998cd9d9e83e721881525e9f0755018250d17db0b

Observation 59ee554e-81eb-4021-ae32-95bf33ea029c · outbound

This paper cites and Lopes, Ant \'o nio V.

Explicit Boundary Markers for Subword Vocabularies and Lopes, Ant \'o nio V

Reference 54

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.737507Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.737507Z digest=sha256:186a21ac9bb84ab65bf64604d65537424e5e090f27798f532b96d64d46c74799

Observation 84aeb7f1-bfd1-4ec8-a4ee-418263d4d8e6 · outbound

This paper cites Why do language models perform worse for morphologically complex languages?.

Explicit Boundary Markers for Subword Vocabularies Why do language models perform worse for morphologically complex languages?

Reference 55

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.742742Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.742742Z digest=sha256:6fdd9c75f78831d2fc467797c1841e008665b5735c8c26a7f6e597477b0fe914

Observation f99981db-50d8-4299-b9ba-f84ab92104e3 · outbound

This paper cites How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models.

Explicit Boundary Markers for Subword Vocabularies How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Reference 56

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.747007Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.747007Z digest=sha256:bd87a0458034b99209a40c24dda6445b46cfd8911ce104f7e2eee1f24e4408f9

Observation 7d456155-f456-4f34-8358-8b8b1f4eb84d · outbound

This paper cites Superbizarre Is Not Superb: Derivational Morphology Improves BERT ' s Interpretation of Complex Words.

Explicit Boundary Markers for Subword Vocabularies Superbizarre Is Not Superb: Derivational Morphology Improves BERT ' s Interpretation of Complex Words

Reference 57

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.751597Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.751597Z digest=sha256:a1ce76b980fd20c7bebc76c42b8ebc4cf1bb01b6338ca899dc1683bc62a5034e

Observation 58e8f36a-afe7-4284-8eb1-892bc93d8580 · outbound

This paper cites 2026 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2026 , eprint=

Reference 58

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.362442Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.756259Z digest=sha256:de34fc97b8b72ecc4334bc02353f7de0c9b375893ebca2cce06a4d6a0a749a64

Observation 66764e8d-ade3-404b-a12a-feb8294fc7b4 · outbound

This paper cites Proceedings of the 15th Language Resources and Evaluation Conference (LREC) , year=.

Explicit Boundary Markers for Subword Vocabularies Proceedings of the 15th Language Resources and Evaluation Conference (LREC) , year=

Reference 59

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.347160Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.760424Z digest=sha256:15f69b1b4c676a9dfeae8af7bdf231cdaea32f0987d55c6abe12e25397933d6f

Observation 3b852b5c-c50e-4855-a8df-44ed3d46629d · outbound

This paper cites 2025 , isbn =.

Explicit Boundary Markers for Subword Vocabularies 2025 , isbn =

Reference 60

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.764743Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.764743Z digest=sha256:33a7470dc206901e207c1ad8b04d01dc2cf39715e14c88b01fddcd1e83a4379f

Observation 9ab2392b-dc5a-41bc-86b7-e3a33530cf9a · outbound

This paper cites The U niversity of E dinburgh ' s Neural MT Systems for WMT 17.

Explicit Boundary Markers for Subword Vocabularies The U niversity of E dinburgh ' s Neural MT Systems for WMT 17

Reference 61

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.769918Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.769918Z digest=sha256:a9e81d0c177ea0a0ed748430e37d31e29cba301b86cf825899c9ef3cc6102c75

Observation d00f0e03-4c7f-41b5-a388-37bcc9f6c541 · outbound

This paper cites Second Conference on Language Modeling , year=.

Explicit Boundary Markers for Subword Vocabularies Second Conference on Language Modeling , year=

Reference 62

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.774272Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.774272Z digest=sha256:3e3d505cfc98541e4c2608b15ff3a0947cb8b02a9bd0f637774f1e144bdf1d22

Observation b5ad4ce1-9dfe-465f-8b92-94144e0894e2 · outbound

This paper cites Investigating the Effectiveness of BPE : The Power of Shorter Sequences.

Explicit Boundary Markers for Subword Vocabularies Investigating the Effectiveness of BPE : The Power of Shorter Sequences

Reference 63

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.778637Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.778637Z digest=sha256:7d2640d6caaa1a720ce9c0b41407554ad9cc2dfc6f907745e24908de486b2e7f

Observation 3b740281-38cb-46ab-8081-0770132ecb1e · outbound

This paper cites 2026 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2026 , eprint=

Reference 64

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.322667Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.783008Z digest=sha256:9cf32982b713f78dfb85eca3e06613b3cb13f184dbfd0ea8f31b3ac40513d271

Observation 14593466-7d5c-4aba-b9fc-66e16f0ac3b7 · outbound

This paper cites Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training.

Explicit Boundary Markers for Subword Vocabularies Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training

Reference 65

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.787165Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.787165Z digest=sha256:408e2611b359f9d0228737ca0f4af155ef39c1dfb448f3bfa945ad85afdaa324

Observation af02c198-1418-4931-b94a-8c17ea00192a · outbound

This paper cites 2025 , publisher =.

Explicit Boundary Markers for Subword Vocabularies 2025 , publisher =

Reference 66

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.792085Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.792085Z digest=sha256:be50fa27d7c0f31fd7ffa1a6d8e19f5ad7b712f15ed893b61133293a18a78726

Observation 3ac71059-15bc-4015-8e14-f79695bf3a15 · outbound

This paper cites and Carmon, Yair and Dave, Achal and Schmidt, Ludwig and Shankar, Vaishaal , booktitle =.

Explicit Boundary Markers for Subword Vocabularies and Carmon, Yair and Dave, Achal and Schmidt, Ludwig and Shankar, Vaishaal , booktitle =

Reference 67

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.796515Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.796515Z digest=sha256:5375f2cab11789dfdfa68e5182e6475211cd3f9bb2175cb27f3f763343636e1d

Pith citing papers

No inbound Pith citation observations are available.