Pith. sign in

Paper Citation Record · LEDGER

Explicit Boundary Markers for Subword Vocabularies

As of 15 August 2026, this Paper Citation Record lists 67 of 67 outbound references and 0 inbound Pith citation observations for arXiv:2608.08847.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2608.08847 v2

Coverage vector

measured 67 of 67 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-14T04:32:08.796515Z

measured 67 of 67 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-14T06:32:32.682623+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

67 of 67 outbound references displayed

  • verified exact1
  • verified fuzzy18
  • unresolved46
  • parse uncertain1
  • malformed identifier0
  • metadata mismatch1

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation c18acb7d-cdee-4cae-891f-e7f30f64bfb9 · outbound

This paper cites 2025 , booktitle=.

Explicit Boundary Markers for Subword Vocabularies 2025 , booktitle=

Reference 1

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.767994Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.501308Z digest=sha256:aafb837f06489df1258498c2abfed95783c3be3eb6d7be1f2439eb1efc394a25

Observation b965d0a2-8d35-4304-8e8f-11df574d9d22 · outbound

This paper cites 2019 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2019 , eprint=

Reference 2

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.752877Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.507787Z digest=sha256:357ed91944c67309e7f4761b0e533077fb58e84a40fe5725e5467d4922959261

Observation 1781d9c0-41cd-4564-9c08-e37ee98cffb0 · outbound

This paper cites Say Anything but This: When Tokenizer Betrays Reasoning in.

Explicit Boundary Markers for Subword Vocabularies Say Anything but This: When Tokenizer Betrays Reasoning in

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.512381Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.512381Z digest=sha256:eb018b3628b2d61c781cc69c3eccb57f36e6c6348d1c357a6aa18c17c19f1848

Observation 2c241548-dcac-4f8b-98da-2743803f11f9 · outbound

This paper cites Improving Tokenisation by Alternative Treatment of Spaces.

Explicit Boundary Markers for Subword Vocabularies Improving Tokenisation by Alternative Treatment of Spaces

Reference 4

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.516904Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.516904Z digest=sha256:60c5247315e0164e28007dc812a961cf1507e4240ceeab6fedc4b6df1bdbbef2

Observation 369574cf-3149-43b3-8458-6025fd598a84 · outbound

This paper cites Tokenization Falling Short: On Subword Robustness in Large Language Models.

Explicit Boundary Markers for Subword Vocabularies Tokenization Falling Short: On Subword Robustness in Large Language Models

Reference 5

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.521538Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.521538Z digest=sha256:7c5d47c3bf21b02aa0ca4f5babc1273e10dd4665b513c3d425020062d04a774f

Observation 4741079b-0c12-47ba-8368-08568c8ec030 · outbound

This paper cites MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment.

Explicit Boundary Markers for Subword Vocabularies MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment

Reference 6

Resolution
metadata mismatch
local_arxiv, observed 2026-08-14T04:32:09.160993Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.526111Z digest=sha256:4963613d23c24a9cd8ae19cb8ca8dd067b8f4d7a04a80f899bda899070f112d9

Observation f126473a-1ca8-42d0-a87d-eba91bb316a3 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.530890Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.530890Z digest=sha256:92f1d570f638937ea13c7044adcedddaa4fb266f0dbf8f3c95865849fe770584

Observation f8b9c1ec-f24c-40e9-a0d9-a30e3ff7a6eb · outbound

This paper cites doi:10.5281/zenodo.7199437 , url =.

Explicit Boundary Markers for Subword Vocabularies doi:10.5281/zenodo.7199437 , url =

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.535120Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.535120Z digest=sha256:38714758d81f6473cf697623558e50eb897e747ec81fd8f3903d77b410645f74

Observation ca47e64a-b520-4106-8efd-088025906699 · outbound

This paper cites 2026 , month = jul, day =.

Explicit Boundary Markers for Subword Vocabularies 2026 , month = jul, day =

Reference 9

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.726777Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.540015Z digest=sha256:4500a9b46b652590930ac449cd2f1d84ab130f74fcc15fa23263573121f248a3

Observation 0f4643fe-4c1b-4d5c-862a-a36d4891f421 · outbound

This paper cites 2022 , url =.

Explicit Boundary Markers for Subword Vocabularies 2022 , url =

Reference 10

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.544531Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.544531Z digest=sha256:8606ee6cc8615bded84134fb9c96601520ec042ad6bce87ea3b152aa96c37b97

Observation f0e05985-c6c4-4c71-83c0-933baa27fb00 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 11

Resolution
unresolved
raw_fallback, observed 2026-08-14T04:32:09.700968Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.549296Z digest=sha256:282447fe77da2b18dca62eaa56cb5a9c9f1fc34cfcdfe9c72d9ab9ede893f179

Observation bc50c1dc-19df-4e6d-8e45-084ed9ce70b6 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 12

Resolution
unresolved
raw_fallback, observed 2026-08-14T04:32:09.685802Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.553633Z digest=sha256:671ad092d6cc790611d57f78fd7d0ce37c54579a4a03fed8957579203af1dac0

Observation 0401d9a3-23e0-4a9e-81ae-b0fbfa592440 · outbound

This paper cites and Reddy, Varshini and Zhang, Haoran and Alameddine, Alec and Uzan, Omri and Pinter, Yuval and Tanner, Chris.

Explicit Boundary Markers for Subword Vocabularies and Reddy, Varshini and Zhang, Haoran and Alameddine, Alec and Uzan, Omri and Pinter, Yuval and Tanner, Chris

Reference 13

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.557843Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.557843Z digest=sha256:bbe7a319bb6dce2c3738dd80183ff22a2b6fa0b4130bd48db01bed979028eced

Observation 21c68cfd-e18d-4124-8c88-8aadd65df67a · outbound

This paper cites and Tanner, Chris and Pinter, Yuval.

Explicit Boundary Markers for Subword Vocabularies and Tanner, Chris and Pinter, Yuval

Reference 14

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.562168Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.562168Z digest=sha256:22370f2b898a1bf6d6e2e5ca3e96d7c8be008955adfbfa115745064273c26546

Observation a578bef0-9049-423c-879b-261ce4d84ab1 · outbound

This paper cites Tokenization and the Noiseless Channel.

Explicit Boundary Markers for Subword Vocabularies Tokenization and the Noiseless Channel

Reference 15

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.566529Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.566529Z digest=sha256:4cfb209b238152736c8c5a36ffc6d88a081f3c6d88b4d2bdf280d68584f4a9d9

Observation 503dc684-d99a-40b5-b1bc-f5d98f35277c · outbound

This paper cites Two Counterexamples to Tokenization and the Noiseless Channel.

Explicit Boundary Markers for Subword Vocabularies Two Counterexamples to Tokenization and the Noiseless Channel

Reference 16

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.570953Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.570953Z digest=sha256:0b141aab7cb09fb11d6aa5f26ae8aa22d575cf10ee1a019be5748d5c04dc34a5

Observation 42f779e3-75b3-4435-a198-4b85e4b60a15 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 17

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.575493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.575493Z digest=sha256:e148f6a8c9922daaca6759aeca947815cdde250946cbad653fbe189e4f904abf

Observation ee460b06-900d-456b-838c-95ee4bc929d4 · outbound

This paper cites Byte Pair Encoding is Suboptimal for Language Model Pretraining.

Explicit Boundary Markers for Subword Vocabularies Byte Pair Encoding is Suboptimal for Language Model Pretraining

Reference 18

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.579754Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.579754Z digest=sha256:424a27332bb1caae7298732856b774303f7aa9573f873b2523b79baa28bf7b8e

Observation 935f6ca1-ac75-433b-8454-e1685b4d0de1 · outbound

This paper cites Incorporating Context into Subword Vocabularies.

Explicit Boundary Markers for Subword Vocabularies Incorporating Context into Subword Vocabularies

Reference 19

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.584043Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.584043Z digest=sha256:3d151f55be41c3c291886e262054d615f6acac5ef5cde85bf6a2c66eec6b0dad

Observation 1d556313-8b9e-4f1b-a396-9c90749a5b8d · outbound

This paper cites Rethinking Tokenization for Rich Morphology: The Dominance of U nigram over BPE and Morphological Alignment.

Explicit Boundary Markers for Subword Vocabularies Rethinking Tokenization for Rich Morphology: The Dominance of U nigram over BPE and Morphological Alignment

Reference 20

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.588319Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.588319Z digest=sha256:ac8f9a2ffe35a019ab0476c5ac38c7a05ce53ddf875df3f3706ee8f3fc5626d3

Observation d281eeb7-d8ef-43ab-9e61-28bda6313ae5 · outbound

This paper cites MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies.

Explicit Boundary Markers for Subword Vocabularies MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies

Reference 21

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.592496Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.592496Z digest=sha256:5c54f22b0b980a535e2ab10178e634b1fa21c444c1c1b83cd1b350dd6d3486d8

Observation c8ad4a85-151b-4e07-bcc4-e9b5e2f0c03f · outbound

This paper cites Which Pieces Does U nigram Tokenization Really Need?.

Explicit Boundary Markers for Subword Vocabularies Which Pieces Does U nigram Tokenization Really Need?

Reference 22

Resolution
verified exact
doi, observed 2026-08-14T04:32:08.978562Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.597667Z digest=sha256:c852e8cba1a0cb9faac67fd87eecd9f551894a361ebefa837c57648f48e2cabf

Observation b2c1c6b8-4ce0-4d9c-9333-f76cd98655c2 · outbound

This paper cites 2025 , publisher =.

Explicit Boundary Markers for Subword Vocabularies 2025 , publisher =

Reference 23

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.649697Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.601886Z digest=sha256:d1109268c4a8c9c796ded22d5947e0dbf4a640cfeac176bad8e3886b8913d1d3

Observation 3f3c5379-d3e3-43f9-96de-9085c8a8ec8c · outbound

This paper cites 2020 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2020 , eprint=

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.606012Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.606012Z digest=sha256:e96c613c9f66a66ac925d59bf2223edd4692ebcea40a0bee752dea7302e6ca86

Observation 440d6026-698e-43b0-85f0-b7f591eb124f · outbound

This paper cites Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features.

Explicit Boundary Markers for Subword Vocabularies Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features

Reference 25

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.610131Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.610131Z digest=sha256:7cc0a7f28cd3a19b03df12e462a4ac5a8d05f8c81fb1c6c44f7b1f62ac2acb9c

Observation 35571c34-544c-47a2-94ff-48bf0d9fcc16 · outbound

This paper cites 2025 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2025 , eprint=

Reference 26

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.621724Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.614523Z digest=sha256:10793c6d392f917ccfa844d5d465cb6df303a594c351f4798bdf574726c9158e

Observation 0b301b75-2c55-43e7-bcfe-68543be61bd1 · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 27

Resolution
unresolved
raw_fallback, observed 2026-08-14T04:32:09.605669Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.618825Z digest=sha256:c192f5ae4d0903745eaf1b96258a944df2ac7e9530820a2f8eafd2de42d9f6b4

Observation c288f69d-93ca-498d-b1f5-e7626179dd85 · outbound

This paper cites SentencePiece : A simple and language independent subword tokenizer and detokenizer for Neural Text Processing.

Explicit Boundary Markers for Subword Vocabularies SentencePiece : A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

Reference 28

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.623089Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.623089Z digest=sha256:b8a3369ce161e4effd55063c025d1b7b342555b1cc9c16058dc2914b3ace36fa

Observation 3c8f8c9e-c8b2-43ac-adbf-037a289fb8e5 · outbound

This paper cites Combinatorial Pattern Matching , pages=.

Explicit Boundary Markers for Subword Vocabularies Combinatorial Pattern Matching , pages=

Reference 29

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.590287Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.627520Z digest=sha256:3b3b6be8f6c8e7266e3d179be43e663ac556ed2c9fb86370d76b93fa716e7a0d

Observation cc839931-7172-4887-bbed-02d59c92bdb6 · outbound

This paper cites ICML 2025 Tokenization Workshop , url=.

Explicit Boundary Markers for Subword Vocabularies ICML 2025 Tokenization Workshop , url=

Reference 30

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.573221Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.631915Z digest=sha256:1623e53669da48a29f7925f0c5a42f04faef794b5092a7e44af0072f00e69c36

Observation e8c01c50-d811-4217-8f87-aeef26c8643c · outbound

This paper cites BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training.

Explicit Boundary Markers for Subword Vocabularies BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training

Reference 31

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.636165Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.636165Z digest=sha256:8de6be179200983f8fce2850fa964afdf390ab6ad83fa5c1d9f109a437e253de

Observation 210332ec-3e1a-4438-989d-fb19c090adc4 · outbound

This paper cites and Bergen, Benjamin.

Explicit Boundary Markers for Subword Vocabularies and Bergen, Benjamin

Reference 32

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.555570Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.640588Z digest=sha256:91dbf1cc9eb98c75f9a8ea1f10e2911162e05b7e2277b587ddbed6117af4425d

Observation 36dcc0f7-5aaf-4d9d-ac4f-1a9101310ff2 · outbound

This paper cites Neural Machine Translation of Rare Words with Subword Units.

Explicit Boundary Markers for Subword Vocabularies Neural Machine Translation of Rare Words with Subword Units

Reference 33

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.644857Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.644857Z digest=sha256:5344e2715cf20d281ba4b798e9043014fc46326b9cf56d8efe8702039fb18633

Observation 12d5a6d2-733c-4b29-bd8e-ce3d6cd24e35 · outbound

This paper cites Proceedings of the 31st International Conference on Computational Linguistics.

Explicit Boundary Markers for Subword Vocabularies Proceedings of the 31st International Conference on Computational Linguistics

Reference 34

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.538164Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.650141Z digest=sha256:3e21c01379e5820afaa023001d60d425fb4ff42e3a5770b867325099b299ede3

Observation b1eade6e-3477-46a0-9915-cc72bdc2c8ba · outbound

This paper cites Pre-tokenization on punctuation in.

Explicit Boundary Markers for Subword Vocabularies Pre-tokenization on punctuation in

Reference 35

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.522373Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.654554Z digest=sha256:b81d4937009919b930aa2bfe6a07b9f8fd25c3f9956640fdc04792a2cb9fb55d

Observation f1eccd93-d700-4501-ba30-f13f45e3191f · outbound

This paper cites an unresolved cited work.

Explicit Boundary Markers for Subword Vocabularies Unresolved cited work

Reference 36

Resolution
parse uncertain
raw_fallback, observed 2026-08-14T04:32:09.506421Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.658729Z digest=sha256:498c949f557b2eadef05c1bf7190735649bffb2a43e1b87973fe00261ba7421b

Observation bbf3f645-201b-4ed4-b11e-48485c39c1b9 · outbound

This paper cites A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages.

Explicit Boundary Markers for Subword Vocabularies A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

Reference 37

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.490232Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.663227Z digest=sha256:c57bc582deff670baa3721abbbada8b529ec2c41103e13985d51cb5afe1e9955

Observation 3a908081-7600-456d-973d-27f1334a3d36 · outbound

This paper cites Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures , series =.

Explicit Boundary Markers for Subword Vocabularies Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures , series =

Reference 38

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.667481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.667481Z digest=sha256:74accc153c76ddaaf6bd598f6049c4ec2eef700c61a81870b0f796d8af555241

Observation 1f33e711-3560-4865-ac4a-3ee19212a5bc · outbound

This paper cites CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages.

Explicit Boundary Markers for Subword Vocabularies CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages

Reference 39

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.671910Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.671910Z digest=sha256:0ab6e5ec6a375670e41df64a755297265f24c71cd80c96bc59bf5e17375e60ee

Observation d657a32b-281c-40f8-b587-8c858bb377da · outbound

This paper cites Jamo-Level Subword Tokenization in Low-Resource K orean Machine Translation.

Explicit Boundary Markers for Subword Vocabularies Jamo-Level Subword Tokenization in Low-Resource K orean Machine Translation

Reference 40

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.473882Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.676436Z digest=sha256:cf3282b82d3ad54a7b313295d9df1f533a3c86c4f07ef067f8122ce578ce2151

Observation be0f5dd0-5c53-4488-8ce6-56c4948b9316 · outbound

This paper cites Fishing for M agikarp: Automatically Detecting Under-trained Tokens in Large Language Models.

Explicit Boundary Markers for Subword Vocabularies Fishing for M agikarp: Automatically Detecting Under-trained Tokens in Large Language Models

Reference 41

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.680957Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.680957Z digest=sha256:de07b596e79864428e8c0d5ef2181b4dda2825d2f535f93ec0a38fb6949efcde

Observation dcb03b0c-dc5b-4a0d-b9e3-1262318dea22 · outbound

This paper cites HuggingFace's Transformers: State-of-the-art Natural Language Processing.

Explicit Boundary Markers for Subword Vocabularies HuggingFace's Transformers: State-of-the-art Natural Language Processing

Reference 42

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.685393Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.685393Z digest=sha256:151fe82023fa15cdf8840f84c54816833a297b91d0af213b70625809d5754462

Observation 7b983339-8701-4503-9b88-14ecdd2ed58b · outbound

This paper cites 2017 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2017 , eprint=

Reference 43

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.690142Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.690142Z digest=sha256:7c6fca09b7d61bf1ba0e17082a85b877b1b7b38bfa674eed0b09375a97801f41

Observation a3fd1f5b-7114-4a39-a317-7c7abe58e897 · outbound

This paper cites Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates.

Explicit Boundary Markers for Subword Vocabularies Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

Reference 44

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.694268Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.694268Z digest=sha256:f851e9dab7bab96eeefdd4d2f42ea8deaf5b0ef66340b5b02c85d356570b1c5e

Observation 95e4aa0c-f6ae-4952-acbc-4ac8f4c3e651 · outbound

This paper cites BPE -Dropout: Simple and Effective Subword Regularization.

Explicit Boundary Markers for Subword Vocabularies BPE -Dropout: Simple and Effective Subword Regularization

Reference 45

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.698756Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.698756Z digest=sha256:8935fa343ff3b696c44e349bd8a36a1fadf469382dfec0b3011f1613162d56dc

Observation e0b483f3-2cbf-4b91-9682-8e714fa9f6a0 · outbound

This paper cites Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP.

Explicit Boundary Markers for Subword Vocabularies Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP

Reference 46

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.702904Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.702904Z digest=sha256:7b56c2f11a53052978afca2879c990e8b4cd3e02b385be4ebcddd21d4d4a0a66

Observation e3311064-0e79-4bd9-bcd1-8fcad81a62af · outbound

This paper cites Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications.

Explicit Boundary Markers for Subword Vocabularies Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications

Reference 47

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.707592Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.707592Z digest=sha256:6c89bf4fe19534866fd6886d3d4c927bffa5a3b30a28d7b97a7002f0c4e37f60

Observation 70936cb0-be81-4845-bc76-81940b2704cd · outbound

This paper cites 2025 , url=.

Explicit Boundary Markers for Subword Vocabularies 2025 , url=

Reference 48

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.441602Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.711821Z digest=sha256:3bddc31482b42a0f906727b3706d3b2c9c5e6fb0f21b0a70a575612bc563c8f2

Observation b2f14f14-e1a0-4f06-b2eb-1f86850af6b1 · outbound

This paper cites 2025 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2025 , eprint=

Reference 49

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.715984Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.715984Z digest=sha256:fcbb228daed943a58dbbae56470b568970a4e3e1cc27999e55790abd241ac629

Observation 680412f5-6602-4c60-bab0-15de5187e626 · outbound

This paper cites 2026 , url =.

Explicit Boundary Markers for Subword Vocabularies 2026 , url =

Reference 50

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.414943Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.720445Z digest=sha256:41df69b4d173f26ba6eb5845d858604d397b57f6c75c94ed7f9acd807cb5df15

Observation 754e9bd6-3320-4a17-828b-dd7f9851e9a8 · outbound

This paper cites Language Model Tokenizers Introduce Unfairness Between Languages , url =.

Explicit Boundary Markers for Subword Vocabularies Language Model Tokenizers Introduce Unfairness Between Languages , url =

Reference 51

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.399264Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.724598Z digest=sha256:738c309d118b112c73c077f4f339a62ab9d08101f9ceec531bf153183843f941

Observation 1b8213fc-73a6-4f7d-bf7c-3dbeb8697782 · outbound

This paper cites Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models.

Explicit Boundary Markers for Subword Vocabularies Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Reference 52

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.728737Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.728737Z digest=sha256:3ba895f601210820878a5e653a81c32de412dd15e258985c049f162384cd1d41

Observation fa60bca4-a8a8-4e4e-950b-a6e56d713518 · outbound

This paper cites Tokenizer Choice For LLM Training: Negligible or Crucial?.

Explicit Boundary Markers for Subword Vocabularies Tokenizer Choice For LLM Training: Negligible or Crucial?

Reference 53

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.733224Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.733224Z digest=sha256:1aab05aca866a211ad2acfcde74481d7211489e20b3f3511084c5325720f2748

Observation 59ee554e-81eb-4021-ae32-95bf33ea029c · outbound

This paper cites and Lopes, Ant \'o nio V.

Explicit Boundary Markers for Subword Vocabularies and Lopes, Ant \'o nio V

Reference 54

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.737507Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.737507Z digest=sha256:0c64bfb05aa56f6b2de2a194797cd230f2e904922b335b5ae3bf11ae3cbf392c

Observation 84aeb7f1-bfd1-4ec8-a4ee-418263d4d8e6 · outbound

This paper cites Why do language models perform worse for morphologically complex languages?.

Explicit Boundary Markers for Subword Vocabularies Why do language models perform worse for morphologically complex languages?

Reference 55

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.742742Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.742742Z digest=sha256:2191947973f86f42c5031c1af942a12998bb4985f3cbe08954610d60f2ef5616

Observation f99981db-50d8-4299-b9ba-f84ab92104e3 · outbound

This paper cites How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models.

Explicit Boundary Markers for Subword Vocabularies How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Reference 56

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.747007Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.747007Z digest=sha256:d8d97e241aa7e7ff402c2872b7aec2567d5cc3e2761c2897fed26590f40e400f

Observation 7d456155-f456-4f34-8358-8b8b1f4eb84d · outbound

This paper cites Superbizarre Is Not Superb: Derivational Morphology Improves BERT ' s Interpretation of Complex Words.

Explicit Boundary Markers for Subword Vocabularies Superbizarre Is Not Superb: Derivational Morphology Improves BERT ' s Interpretation of Complex Words

Reference 57

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.751597Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.751597Z digest=sha256:1d25cb831f86ec5ba89ce67f3185b0185d371704ee3b1c61b3412c97738a8e84

Observation 58e8f36a-afe7-4284-8eb1-892bc93d8580 · outbound

This paper cites 2026 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2026 , eprint=

Reference 58

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.362442Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.756259Z digest=sha256:282d6d4acf6ad93fa2f25845151dc4ed8cbc7b01b1db8c5626256f78ad18ed0c

Observation 66764e8d-ade3-404b-a12a-feb8294fc7b4 · outbound

This paper cites Proceedings of the 15th Language Resources and Evaluation Conference (LREC) , year=.

Explicit Boundary Markers for Subword Vocabularies Proceedings of the 15th Language Resources and Evaluation Conference (LREC) , year=

Reference 59

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.347160Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.760424Z digest=sha256:b681794a9efce9ad19504cb53c1e714a6157442f100e89830f6546a078407b5d

Observation 3b852b5c-c50e-4855-a8df-44ed3d46629d · outbound

This paper cites 2025 , isbn =.

Explicit Boundary Markers for Subword Vocabularies 2025 , isbn =

Reference 60

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.764743Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.764743Z digest=sha256:927b12e126edf11a1dc0617fb41583b0f5bec5eb45e956f29f0d92f3ace9de8e

Observation 9ab2392b-dc5a-41bc-86b7-e3a33530cf9a · outbound

This paper cites The U niversity of E dinburgh ' s Neural MT Systems for WMT 17.

Explicit Boundary Markers for Subword Vocabularies The U niversity of E dinburgh ' s Neural MT Systems for WMT 17

Reference 61

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.769918Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.769918Z digest=sha256:b348b691977420ef7600a698180381c9646341bb984147cb07576bdeec090eb3

Observation d00f0e03-4c7f-41b5-a388-37bcc9f6c541 · outbound

This paper cites Second Conference on Language Modeling , year=.

Explicit Boundary Markers for Subword Vocabularies Second Conference on Language Modeling , year=

Reference 62

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.774272Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.774272Z digest=sha256:1c2abc4363d20acdf0af67b930c1e572e84ada0575dbf2ae842ac6c1b4bce8f8

Observation b5ad4ce1-9dfe-465f-8b92-94144e0894e2 · outbound

This paper cites Investigating the Effectiveness of BPE : The Power of Shorter Sequences.

Explicit Boundary Markers for Subword Vocabularies Investigating the Effectiveness of BPE : The Power of Shorter Sequences

Reference 63

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.778637Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.778637Z digest=sha256:cc6935059ef6441fc6b5303347e8e4c2bb076045352e151a7123c9bfe6e51a22

Observation 3b740281-38cb-46ab-8081-0770132ecb1e · outbound

This paper cites 2026 , eprint=.

Explicit Boundary Markers for Subword Vocabularies 2026 , eprint=

Reference 64

Resolution
verified fuzzy
raw_fallback, observed 2026-08-14T04:32:09.322667Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-14T06:32:32.682623+00:00.

source=arxiv_source observed=2026-08-14T04:32:08.783008Z digest=sha256:664a1f938779fe0d32c9d720bbaa842f8107cf5596d9556f7a1b65baa5604c7e

Observation 14593466-7d5c-4aba-b9fc-66e16f0ac3b7 · outbound

This paper cites Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training.

Explicit Boundary Markers for Subword Vocabularies Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training

Reference 65

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.787165Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.787165Z digest=sha256:9a01a259b95f33a17c9f3822f82e8b9f0023618ebaed6043c88cf77d1d2d437b

Observation af02c198-1418-4931-b94a-8c17ea00192a · outbound

This paper cites 2025 , publisher =.

Explicit Boundary Markers for Subword Vocabularies 2025 , publisher =

Reference 66

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.792085Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.792085Z digest=sha256:4e37d48e51e73f895ecfe2ef56e991f6f3f17b42d5d678002d2a40e96dd0192c

Observation 3ac71059-15bc-4015-8e14-f79695bf3a15 · outbound

This paper cites and Carmon, Yair and Dave, Achal and Schmidt, Ludwig and Shankar, Vaishaal , booktitle =.

Explicit Boundary Markers for Subword Vocabularies and Carmon, Yair and Dave, Achal and Schmidt, Ludwig and Shankar, Vaishaal , booktitle =

Reference 67

Resolution
unresolved
no resolver link, observed 2026-08-14T04:32:08.796515Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-14T04:32:08.796515Z digest=sha256:ca491d2e5885e4e848cad698cb6be3083006dcb096d98aeb913dfc77667657b7

Pith citing papers

No inbound Pith citation observations are available.