SUNTA uses surprise-driven chunk boundaries and decoupled training in hierarchical state-space models to sustain accurate video predictions over 250 timesteps where baselines fail after 10.
Dynamic chunking for end-to-end hierarchical sequence modeling, 2025
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
citation-role summary
background 2
method 1
citation-polarity summary
years
2026 11representative citing papers
MultiHashFormer enables hash-based autoregression in LMs by encoding tokens as multi-hash signatures, outperforming standard Transformers at 100M-3B scales while keeping parameter count constant for multilingual expansion.
LH-NeF learns tokenized neural-field representations via a locality-preserving hierarchical encoder, achieving 42× lower memory and 133× larger batches than modality-agnostic meta-learning baselines while matching or exceeding performance on reconstruction and downstream tasks.
Scratchpad Patching decouples compute from patch size in byte-level language models by inserting entropy-triggered scratchpads to update patch context dynamically.
DC-DiT learns dynamic chunking to allocate fewer tokens to smooth or noisy regions and more to detailed or late-stage areas, cutting inference FLOPs up to 36.8% while improving FID up to 37.8% on class-conditional ImageNet generation.
LDARNet learns adaptive token boundaries via dynamic chunking in a genomic foundation model and reports gains on histone modification tasks over larger models.
Training transformers with KV sparsification during continued pretraining produces representations that admit better post-hoc KV cache compression, improving quality under memory budgets for long-context tasks.
Byte-level simulations show subword tokenization improves LLM training mainly via increased throughput and boundary priors.
MMHNet enables video-to-audio models trained on short clips to generalize and generate audio for videos over 5 minutes long.
Byte modeling incurs greater scaling overhead for masked diffusion than autoregressive models because the diffusion objective destroys local byte contiguity needed to resolve semantics.
Token-Superposition Training combines multiple tokens into bags for multi-hot cross-entropy pre-training followed by a recovery phase, yielding up to 2.5x reduction in training time at 10B scale under equal-loss conditions.
citing papers explorer
-
Training Transformers for KV Cache Compressibility
Training transformers with KV sparsification during continued pretraining produces representations that admit better post-hoc KV cache compression, improving quality under memory budgets for long-context tasks.