Pith. sign in

REVIEW 2 major objections 5 minor 48 references

ReconSpan: Reconstruction-Guided Adaptive Latent Tokenization

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ReconSpan chunks text at points where a backward decoder can no longer reconstruct it, preserving more text per latent token than random boundaries.

desk verdict New allocation rule is real, but the headline comparison against random boundaries is confounded by chunk-length distribution; the paper deserves a serious revision. read the letter →

arxiv 2608.12756 v1 pith:SAVN5UMC submitted 2026-08-13 cs.CL cs.LG

Lixing Li This is my paper · ORCID
classification cs.CLcs.LG
keywords adaptivelatenttokenizationreconstruction-guidedchunkingbackwarddecodercontextualprefixcodestokenstokenizergranularitydirectreadoutautoencoderreconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes treating reconstruction fidelity as the criterion for adaptive latent tokenization: instead of fixed subword units or entropy spikes, text is chunked wherever a backward decoder, starting from a single contextual prefix code, stops being able to reconstruct the recent tokens. Each retained boundary code becomes a latent token, so difficult spans get shorter chunks and easy spans get longer ones. On held-out text, this allocation preserves more content than length-matched random boundaries: at mean chunk length 6.50, exact-token recovery is 0.907 versus 0.618. The same trained autoencoder can be re-granularized after training by loosening the stopping rule, reaching mean chunk lengths from 6.5 to 12.2 input tokens. A reader trained on the latent sequence recovers topic information well but exact details only weakly, so the paper separates what the autoencoder retains from what a downstream model can access.

What carries the argument

The load-bearing object is backward-decoder reach: the longest contiguous run of most-recent tokens that a single prefix code can exactly reproduce when decoded in reverse under teacher forcing. The chunking algorithm computes all prefix codes in one encoder pass, repeatedly teacher-forces the decoder from the current endpoint, and cuts a chunk at the first position where the stopping rule fires—Failure($m$), the $m$-th wrong token, or Logit-gap($\tau$), an accumulated margin. The code at the boundary is the latent token, so chunk length is a direct measurement of how far one code reaches back; this is what makes allocation adaptive and post-training-controllable.

What would settle it

Run the same chunking algorithm with a different backward decoder, or with a forward decoder given an oracle starting position, and compare reconstruction-guided boundaries against matched random boundaries using the protocol of Table 3; if the advantage disappears or reverses, backward reconstruction reach is not a general allocation signal.

Watch

Extended reading notes

Core claim

ReconSpan's central claim is that a chunk boundary is best placed where backward reconstruction stops being reliable. A causal encoder produces a code $c_t = E(x_{1:t})$ at every position; a backward decoder teacher-forced against the true text reconstructs newest-first from the final code until a stopping rule fires, and the longest successfully reconstructed suffix becomes a chunk with $c_t$ as its latent token. The paper reports that this capacity-based boundary rule beats a random rule matched in chunk count and mean length: 0.907 exact-token recovery versus 0.618, with better suffix recovery and lower conditional perplexity. Because the stopping rule is applied only at chunking time, one trained model supplies mean chunk lengths from 6.50 to 12.17 by choosing Failure($m$) or Logit-gap($\tau$). It also claims that the resulting latent sequences are readable by separately trained language models for coarse semantic information, with task adaptation improving exact-detail access, and that reconstruction fidelity is therefore a viable allocation criterion for adaptive latent tokenization.

Load-bearing premise

The argument rests on the premise that a backward decoder's reconstruction reach is a good proxy for what a latent token should preserve for downstream models; if chunking merely optimizes for that decoder's own roundtrip behavior, the allocation advantage may not transfer to other readers.

Editorial extensions

If this is right

  • The same trained autoencoder can produce several granularities, with mean chunk lengths from 6.5 to 12.2 input tokens, by relaxing the stopping rule with no retraining.
  • Boundary placement, not just boundary count, determines how much text survives: reconstruction-guided boundaries outperform random boundaries matched in count and mean length on exact-token and suffix recovery.
  • Downstream language models can consume the contextual latent sequence directly; they match raw-text-level topic classification on AG News but lag on exact-detail tasks such as LAMBADA.
  • Task adaptation on top of generic readout narrows the gap between direct readout and native roundtrip, showing the latent codes carry task-relevant information a generic reader leaves unused.
  • Reconstruction fidelity is proposed as a viable allocation criterion even though the selected boundaries are not claimed to align with linguistic units.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If backward reconstruction reach tracks local information density, ReconSpan boundaries could serve as an unsupervised proxy for difficulty; a direct test would correlate chunk lengths with surprisal from an independent language model.
  • The paper leaves open whether boundaries align with syntax or discourse; a comparison of boundary placement against constituent and clause boundaries would settle that question.
  • The native roundtrip's strong exact-detail performance suggests a better-trained reader, not a different tokenizer, may close the direct-access gap; a reader trained with retrieval-style objectives is a natural next test.
  • If boundary-selection cost is amortized, the same latent sequence could double as a context-compression interface, trading prefill and KV-cache memory for a one-time reconstruction scan.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. ReconSpan proposes an adaptive latent tokenization method in which chunk boundaries are placed where a backward decoder, teacher-forced from a prefix code, begins to reconstruct tokens incorrectly. The prefix code at each boundary is retained as a contextual latent token. The paper contributes a new allocation criterion (backward reconstruction reach), shows that one trained autoencoder supports a range of post-training granularities via stopping rules, and presents an experimental characterization of the resulting latent sequences via native reconstruction and separately trained reader models. The headline empirical claim is that at matched average chunk length, reconstruction-guided boundaries preserve more text than random boundaries (Table 3: exact-token recovery 0.907 vs 0.618), and that downstream readers recover semantic topic information more readily than exact lexical details (Table 5). The paper is thorough in reporting limitations, including the short-chunk spike, the reader access gap, and compute costs.

Significance. If the central claim holds, the paper introduces a genuinely new, post-training-controllable signal for latent tokenization that does not require training a boundary predictor or an explicit length target, and it provides a clean way to trade granularity against reconstruction quality. The manuscript is honest and unusually detailed on limitations and compute accounting, and it reports negative results (the code-to-code reader) as well as positive ones, which is a strength. However, the empirical support for the central claim is currently weakened by the design of the random control and by the mismatch between the teacher-forced selection signal and the free-run evaluation metric. These are fixable within the scope of the paper, so the significance of the contribution is real but the evidence needs strengthening.

major comments (2)
  1. [Section 4.4, Table 3] The random control matches only the per-document chunk count, not the chunk-length distribution. Failure(1) produces 28.1% one-token chunks (Section 6.1), and one-token chunks are far easier to reconstruct exactly than longer chunks. The headline gap in exact-token recovery (0.907 vs 0.618) may therefore be a consequence of the length distribution rather than of where boundaries fall. This concern is sharpened by the fact that the exact-chunk rates are nearly identical (0.797 vs 0.794), so the token-level advantage is driven by which lengths are exact. Please add a random baseline that matches the empirical chunk-length distribution (for example, by randomly permuting the actual chunk boundary positions, or by sampling chunk lengths from the Failure(1) distribution and then placing boundaries accordingly) and report the token-level metrics under that control.
  2. [Section 3.2, Appendix C, Section 4.4] Boundary selection in Algorithm 1 is teacher-forced: the backward decoder is fed the true previous tokens at each reverse step. The evaluation in Table 3, by contrast, uses free-run reconstruction of each chunk encoded in isolation (Appendix C). The paper never shows that teacher-forced reach predicts free-run reach; free decoding compounds errors, so the selection signal could be exploiting autoencoder idiosyncrasies under teacher forcing rather than genuine reconstruction capacity. Please provide a direct test, for example by measuring the correlation between teacher-forced reach and free-run reach on a sample of positions, or by running the boundary selection with an approximate free-run or sampling-based reach and comparing the resulting Table 3 metrics. The manuscript itself acknowledges this distinction in Appendix C, but it does not close the gap.
minor comments (5)
  1. [Section 4.4] The phrase "length-matched random control" is misleading because only the mean chunk length (via per-document chunk count) is matched, not the length distribution. The Table 3 caption is accurate; please make the main text consistent.
  2. [Table 3] Please report the number of documents and chunks, and provide confidence intervals or a significance test for the exact-token difference against the random baseline, since the claim is load-bearing.
  3. [Section 3.2] The statement "O(1 + n)" sequential model invocations is confusing; it is equivalent to O(n) for n>1. Please simplify or clarify what the constant term refers to.
  4. [Table 5] The superscript stars mark significance at 95% confidence, but the caption does not define how multiple comparisons are handled. Given the number of rows, please state whether any correction was applied.
  5. [Appendix B] The statement "Code and checkpoints are not publicly released" limits reproducibility; at minimum, please provide pseudocode or a reference implementation of the random-control procedure so reviewers can verify the distribution-matching issue.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline native-reconstruction advantage reduces to the same decoder's selection signal; reader-based evidence is independent, but the central 'viable allocation criterion' claim is partially circular.

  1. fitted input called prediction [Section 3.2 Algorithm 1; Section 4.4 Table 3; Appendix C]
    "“Because the text being tokenized is already known, the decoder is teacher-forced against it: at each reverse step it is fed the true previous tokens and we record only whether its own greedy (argmax) prediction matches — nothing is sampled or generated.” “After ReconSpan selects the boundaries, each resulting chunk is encoded in isolation and decoded autoregressively; Appendix C defines this protocol formally.” “Boundary selection is teacher-forced, but this round-trip decode conditions on its own previous outputs.”"

    Algorithm 1 places each boundary at the point where the backward decoder's teacher-forced argmax first fails, so each chunk is defined as a suffix the decoder can reconstruct in a conditional, teacher-forced sense. Table 3 then evaluates those chunks by re-encoding them in isolation and running the same decoder autoregressively; exact-token and suffix scores measure free-run reconstruction on chunks selected for teacher-forced reconstruction. The 0.907-vs-0.618 gap thus shows that conditioning the selection on the decoder's own reach transfers to the decoder's native decoding, not that reconstruction fidelity is a general allocation criterion independent of the decoder.

full rationale

The central empirical claim, stated in the abstract and supported by Table 3, is that at matched average length reconstruction-guided boundaries preserve more text than random boundaries under native reconstruction. But the boundaries are selected by the same backward decoder's teacher-forced argmax reach (Algorithm 1), and the evaluation is that same decoder's free-run reconstruction of the selected chunks (Appendix C). This is a fitted-input-called-prediction pattern: the selection rule optimizes a conditional version of the evaluation metric, so the reported advantage is largely a consequence of the selection rule rather than an independent demonstration that reconstruction reach is a generally useful allocation criterion. The paper is otherwise careful: there are no load-bearing self-citations; it explicitly distinguishes the native reconstruction route from the direct-code reader route; and its reader experiments (Table 5) honestly report that exact-detail recovery is poor, with LAMBADA readout at 20.1% versus a 72.5% round-trip reference. Those reader-based measurements are independent evidence and are not circular. The random-boundary control is fair as a count-matched baseline, but the failure to match the induced length distribution (28.1% one-token chunks under Failure(1)) weakens the quantitative comparison. Overall the 'viable allocation criterion' claim is partially circular; the paper's own limitation statements support a moderate score rather than a higher one.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the behavior of a trained backward decoder; the stopping thresholds and training-sampling choices act as hand-set controls of granularity, and the key background assumption is that teacher-forced reconstruction reach is a meaningful proxy for latent-token usefulness. No new physical or architectural entities are invented.

free parameters (4)
  • Failure(m) stopping threshold m = m=1,2
    Chosen by hand to produce different granularities (mean chunk lengths 6.50 and 12.28 in Table 3); not fit to an optimization target but controls the central granularity dial.
  • Logit-gap threshold tau = tau=1,3
    Continuous stop rule tested at two hand-chosen values; at similar mean length it matches Failure rules in reconstruction quality (Section 4.4).
  • Decode block size W = 32
    Teacher-forced decoding window per endpoint; caps reach, causes the length-32 pile-up noted in Figure 1, and sets the O(W n) work bound.
  • Encode-length sampling mixture = 25% uniform [1,20], 75% log-uniform [20,4096]
    Training-time sampling bias that prioritizes short windows and, with the k<=L+1 cap, biases the decoder toward recent-token reconstruction (Section 3.3, Appendix A).
assumptions (5)
  • domain assumption A causal encoder's final hidden state c_t = E(x_{1:t}) is a sufficient code to reconstruct the most recent tokens of its prefix.
    The whole method assumes the prefix code retains recent-token information; Section 3.1 defines the autoencoding path and Section 5.2 tests its limits.
  • domain assumption Teacher-forced greedy backward decoding accuracy is a valid measure of how far a code reconstructs.
    Algorithm 1 counts greedy argmax mismatches under teacher forcing; the paper does not compare greedy vs sampling reach.
  • domain assumption The backward decoder's reconstruction ability on the training distribution transfers to new corpora (Wikipedia, WikiText, FineWeb, task datasets).
    Chunk lengths shift across corpora (Section 4.3), so transfer is assumed rather than guaranteed.
  • domain assumption Pythia-410M and Mamba2-130M, loaded from pretrained weights, are adequate base models for the autoencoder.
    Reliance on third-party pretrained models is standard practice in the field.
  • domain assumption Random boundaries matched for per-document chunk count are an appropriate baseline for boundary quality.
    Table 3 uses this control; it isolates boundary placement from granularity, but is only one comparison class.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReconSpan: Reconstruction-Guided Adaptive Latent Tokenization." pith.science (2026). https://pith.science/paper/SAVN5UMC

@misc{pith2026260812756,
  author       = {Pith},
  title        = {Pith review of: ReconSpan: Reconstruction-Guided Adaptive Latent Tokenization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SAVN5UMC}},
  note         = {Machine review of arXiv:2608.12756}
}
read the original abstract

Adaptive latent tokenization maps a fine-grained input to a shorter sequence of continuous representations associated with input-dependent spans. We introduce ReconSpan, which divides text into chunks that a backward decoder can reconstruct from a single contextual prefix code and retains one such code as the latent token for each chunk. The reconstruction criterion is applied when chunks are formed, allowing one trained autoencoder to produce average chunk lengths from 6.5 to 12.2. At matched average length, reconstruction-guided boundaries preserve more text than random boundaries. Readers of the resulting latent sequence recover topic information reliably but struggle to extract exact details.

Figures

Figures reproduced from arXiv: 2608.12756 by the authors.

Figure 1
Figure 1. Chunk-length distributions (raw counts) for four stopping rules on the same 500 Wikipedia [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Single-code reconstruction ROUGE-L versus encode length, on 500 held-out Wikipedia [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Per-position backward decode reach for one 63-token passage under [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 23 canonical work pages

  1. [1]

    Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Neural Machine Translation of Rare Words with Subword Units , author=. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2016 , publisher=. doi:10.18653/v1/P16-1162 , url=

    doi:10.18653/v1/p16-1162 2016

  2. [2]

    The Tenth International Conference on Learning Representations , year=

    Charformer: Fast Character Transformers via Gradient-based Subword Tokenization , author=. The Tenth International Conference on Learning Representations , year=

    work page

  3. [3]

    2023 , url=

    Yu, Lili and Simig, Daniel and Flaherty, Colin and Aghajanyan, Armen and Zettlemoyer, Luke and Lewis, Mike , booktitle=. 2023 , url=

    work page 2023

  4. [4]

    Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization

    Flexibly Scaling Large Language Models Contexts Through Extensible Tokenization , author=. arXiv preprint arXiv:2401.07793 , year=. doi:10.48550/arXiv.2401.07793 , url=

    work page Pith review arXiv doi:10.48550/arxiv.2401.07793

  5. [5]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Efficient Transformers with Dynamic Token Pooling , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2023 , publisher=. doi:10.18653/v1/2023.acl-long.353 , url=

    doi:10.18653/v1/2023.acl-long.353 2023

  6. [6]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Byte Latent Transformer: Patches Scale Better Than Tokens , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2025 , publisher=. doi:10.18653/v1/2025.acl-long.453 , url=

    doi:10.18653/v1/2025.acl-long.453 2025

  7. [7]

    The Fourteenth International Conference on Learning Representations , year=

    Dynamic Chunking for End-to-End Hierarchical Sequence Modeling , author=. The Fourteenth International Conference on Learning Representations , year=

    work page

  8. [8]

    2026 , publisher=

    Owodunni, Abraham Toluwase and Ahia, Orevaoghene and Kumar, Sachin , booktitle=. 2026 , publisher=. doi:10.18653/v1/2026.findings-acl.848 , url=

    work page doi:10.18653/v1/2026.findings-acl.848 2026

Show all 48 references
  1. [9]

    2026 , publisher=

    Liu, Dong and Yu, Yanxuan , booktitle=. 2026 , publisher=. doi:10.18653/v1/2026.starsem-conference.1 , url=

    2026 doi

  2. [10]

    The Twelfth International Conference on Learning Representations , year=

    In-context Autoencoder for Context Compression in a Large Language Model , author=. The Twelfth International Conference on Learning Representations , year=

  3. [11]

    Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics , pages=

    Muennighoff, Niklas and Tazi, Nouamane and Magne, Lo. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics , pages=. 2023 , publisher=. doi:10.18653/v1/2023.eacl-main.148 , url=

    2023 doi

  4. [12]

    Li, Zongqian and Su, Yixuan and Collier, Nigel , booktitle=. 500x. 2025 , publisher=. doi:10.18653/v1/2025.acl-long.1219 , url=

    2025 doi

  5. [13]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Learning to Compress Prompts with Gist Tokens , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  6. [14]

    Proceedings of EMNLP , year=

    Adapting Language Models to Compress Contexts , author=. Proceedings of EMNLP , year=

  7. [15]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Compressing Context to Enhance Inference Efficiency of Large Language Models , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=. 2023 , publisher=. doi:10.18653/v1/2023.emnlp-main.391 , url=

    2023 doi

  8. [16]

    arXiv preprint arXiv:2603.25340 , year=

    Large Language Model as Token Compressor and Decompressor , author=. arXiv preprint arXiv:2603.25340 , year=

    arXiv

  9. [17]

    2025 , publisher=

    Chen, Shaoshen and Li, Yangning and Xu, Zishan and Zeng, Yongqin and Wu, Shunlong and Hu, Xinshuo and Shan, Zifei and Su, Xin and Tang, Jiwei and Li, Yinghui and Zheng, Hai-Tao , booktitle=. 2025 , publisher=. doi:10.18653/v1/2025.findings-acl.1055 , url=

    2025 doi

  10. [18]

    arXiv preprint arXiv:2603.25926 , year=

    Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio , author=. arXiv preprint arXiv:2603.25926 , year=

  11. [19]

    International Conference on Learning Representations (ICLR) , year=

    Compressed Context Memory for Online Language Model Interaction , author=. International Conference on Learning Representations (ICLR) , year=

  12. [20]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Recurrent Memory Transformer , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  13. [21]

    arXiv preprint arXiv:2502.06139 , year=

    LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs , author=. arXiv preprint arXiv:2502.06139 , year=

    arXiv

  14. [22]

    arXiv preprint arXiv:2406.18173 , year=

    UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs , author=. arXiv preprint arXiv:2406.18173 , year=

    arXiv

  15. [23]

    arXiv preprint arXiv:2510.20797 , year=

    No Mean Feat: Simple, Strong Baselines for Context Compression , author=. arXiv preprint arXiv:2510.20797 , year=

    arXiv

  16. [24]

    arXiv preprint arXiv:2602.09789 , year=

    When Less is More: The LLM Scaling Paradox in Context Compression , author=. arXiv preprint arXiv:2602.09789 , year=

    arXiv

  17. [25]

    arXiv preprint arXiv:2308.11466 , year=

    SONAR: Sentence-Level Multimodal and Language-Agnostic Representations , author=. arXiv preprint arXiv:2308.11466 , year=. doi:10.48550/arXiv.2308.11466 , url=

    doi arXiv

  18. [26]

    arXiv preprint arXiv:2412.08821 , year=

    Large Concept Models: Language Modeling in a Sentence Representation Space , author=. arXiv preprint arXiv:2412.08821 , year=. doi:10.48550/arXiv.2412.08821 , url=

    doi arXiv

  19. [27]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Semi-supervised Sequence Learning , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  20. [28]

    Transformers are

    Dao, Tri and Gu, Albert , booktitle=. Transformers are. 2024 , publisher=

    2024

  21. [29]

    Proceedings of the 40th International Conference on Machine Learning , pages=

    Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling , author=. Proceedings of the 40th International Conference on Machine Learning , pages=. 2023 , publisher=

    2023

  22. [30]

    Paperno, Denis and Kruszewski, Germ. The. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2016 , publisher=. doi:10.18653/v1/P16-1144 , url=

    2016 doi

  23. [31]

    Advances in Neural Information Processing Systems , volume=

    Character-level Convolutional Networks for Text Classification , author=. Advances in Neural Information Processing Systems , volume=. 2015 , url=

    2015

  24. [32]

    Advances in Neural Information Processing Systems , volume=

    The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale , author=. Advances in Neural Information Processing Systems , volume=. 2024 , url=

    2024

  25. [33]

    and Salakhutdinov, Ruslan and Manning, Christopher D

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William W. and Salakhutdinov, Ruslan and Manning, Christopher D. , booktitle=. 2018 , publisher=. doi:10.18653/v1/D18-1259 , url=

    2018 doi

  26. [34]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  27. [35]

    arXiv preprint arXiv:2412.15115 , year=

    Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year=

    arXiv

  28. [36]

    Advances in Neural Information Processing Systems , volume=

    Attention Is All You Need , author=. Advances in Neural Information Processing Systems , volume=. 2017 , url=

    2017

  29. [37]

    2002 , publisher=

    Papineni, Kishore and Roukos, Salim and Ward, Todd and Zhu, Wei-Jing , booktitle=. 2002 , publisher=. doi:10.3115/1073083.1073135 , url=

    2002

  30. [38]

    2004 , publisher=

    Lin, Chin-Yew , booktitle=. 2004 , publisher=

    2004

  31. [39]

    Advances in Neural Information Processing Systems , volume=

    Root Mean Square Layer Normalization , author=. Advances in Neural Information Processing Systems , volume=. 2019 , url=

    2019

  32. [40]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=. 2022 , url=

    2022

  33. [41]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    The Power of Scale for Parameter-Efficient Prompt Tuning , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=. 2021 , publisher=. doi:10.18653/v1/2021.emnlp-main.243 , url=

    2021 doi

  34. [42]

    The Seventh International Conference on Learning Representations , year=

    Decoupled Weight Decay Regularization , author=. The Seventh International Conference on Learning Representations , year=

  35. [43]

    The Fifth International Conference on Learning Representations , year=

    Pointer Sentinel Mixture Models , author=. The Fifth International Conference on Learning Representations , year=

  36. [44]

    arXiv preprint arXiv:1604.06174 , year=

    Training Deep Nets with Sublinear Memory Cost , author=. arXiv preprint arXiv:1604.06174 , year=

    arXiv

  37. [45]

    arXiv preprint arXiv:2407.21783 , year=

    The Llama 3 Herd of Models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  38. [46]

    Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) , pages=

    Cer, Daniel and Diab, Mona and Agirre, Eneko and Lopez-Gazpio, I. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) , pages=. 2017 , publisher=. doi:10.18653/v1/S17-2001 , url=

    2017 doi

  39. [47]

    Marelli, Marco and Menini, Stefano and Baroni, Marco and Bentivogli, Luisa and Bernardi, Raffaella and Zamparelli, Roberto , booktitle=. A. 2014 , publisher=

    2014

  40. [48]

    Advances in Information Retrieval: 38th European Conference on IR Research , pages=

    A Full-Text Learning to Rank Dataset for Medical Information Retrieval , author=. Advances in Information Retrieval: 38th European Conference on IR Research , pages=. 2016 , publisher=

    2016

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.