PRISM: Efficient Long-Range Reasoning With Short-Context LLMs

Beliz Gunel; Dulhan Jayalath; James Bradley Wendt; Nicholas Monath; Sandeep Tata

REVIEW 3 major objections 5 minor 1 cited by

PRISM: Efficient Long-Range Reasoning With Short-Context LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PRISM, a structured in-context memory method, lets short-context LLMs beat incremental and hierarchical merging baselines on long-range tasks while using 4 to 50 times less context than a long-context model.

desk verdict PRISM's quality gains over simple in-context baselines look real, but its headline token-efficiency and cost-reduction claims rest on KV-cache accounting that may not generalize. read the letter →

arxiv 2412.18914 v3 pith:KVR4Q7OV submitted 2024-12-25 cs.AI

Dulhan Jayalath , James Bradley Wendt , Nicholas Monath , Sandeep Tata , Beliz Gunel This is my paper · ORCID

classification cs.AI

keywords long-rangereasoningshort-contextLLMsstructuredmemoryin-contextlearningKVcachereuseincrementalprocessingschemageneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PRISM is a training-free, task-agnostic method for long-range reasoning that works with short-context LLMs. Instead of feeding an entire document into the model, it processes chunks one by one, maintaining a compact structured memory whose shape is defined by a user-supplied typed hierarchical schema. The model proposes programmatic revisions to that memory, paths to add or update, rather than rewriting a running summary in natural language. The paper claims this beats incremental and hierarchical merging baselines on summarization, code retrieval, and database question answering, while using 4 to 50 times less context than a long-context model and cutting inference cost by up to 54 percent through key-value cache reuse. If correct, long-range reasoning does not require long-context models, training data, or heavy retrieval pipelines.

What carries the argument

The central object is the typed hierarchical memory, a nested key-value map whose fields are specified by a dataclass-style schema. The revision tuple, path, operation, value, is what lets the LLM update this memory without regenerating it. The amendments representation is the cache-specific piece: instead of modifying a value in place, PRISM appends the new value as a separate memory object at the end, so the prefix of the prompt up to the newest change is identical to the previous step and its key-value activations can be reused. This trade-off, larger memory in exchange for fewer tokens re-encoded, carries the cost reductions reported in the paper.

What would settle it

Give PRISM a synthetic task in which each chunk contains key-value pairs and some values point to keys that appeared in earlier chunks, using a schema that stores only a limited subset of keys. The paper itself notes this type of multi-hop tracing requires a complete, lossless memory; if PRISM's accuracy on such a task collapses below an incremental natural-language baseline, it confirms that the schema, not the revision mechanism, determines what the method can remember.

Watch

Extended reading notes

Core claim

PRISM's central claim is that a short-context LLM can solve long-range tasks by maintaining a typed hierarchical structured memory instead of a natural-language running summary. At each chunk, the model emits a programmatic revision, an addressable path, an add or update operation, and a value, which is validated and applied to the memory. Because the memory is structured and the model only proposes differences, outputs are concise and directly relevant to the task. The paper reports that this beats both incremental and hierarchical merging baselines on all three benchmarks, and on BooookScore it reaches 97 percent of the long-context model's score with a 50 times smaller context window. Efficiency comes from amendments: appending changes to the end of the memory preserves the longest matching prefix, so key-value activations are reused across steps, yielding cache hit rates around 69 to 75 percent and cutting cost by up to 54 percent.

Load-bearing premise

The method depends on the schema being rich enough to encode all information a query might need; the paper only tests three hand-crafted schemas and three LLM-generated ones, and admits that designing an optimal schema is not trivial.

Editorial extensions

If this is right

Book-length summarization can be done with short-context LLMs at 97 percent of a long-context model's score while using 50 times less context.
Code retrieval from repositories larger than 100k tokens becomes feasible with short-context models, reaching 58 percent of the long-context ceiling compared with 24 to 26 percent for incremental baselines.
SQL-database question answering can be handled chunk by chunk, roughly doubling the accuracy of an adapted incremental merging baseline.
Because LLM-generated schemas match hand-crafted ones on two of three tasks, new domains can be tackled with just a task description and an example query.
The training-free, weight-free, task-agnostic design means PRISM works with any black-box LLM API without fine-tuning or retrieval infrastructure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct extension suggested by the paper's structure is schema adaptation: let the model revise the schema itself when a chunk contains information the current memory cannot encode; the paper only studies static schemas.
The key-value cache trick is separable from the schema design, so one could apply amendments to any incremental memory method and expect some cache saving, independent of whether the structured memory itself helps.
On hardware without prefix caching, amendments can backfire because they inflate memory size; the reported 54 percent saving assumes a serving stack that reuses prefix activations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The interesting part of PRISM is the method: a typed, hierarchical structured memory updated by programmatic revisions, plus the amendments representation that reuses prefix KV cache activations. That combination is a genuine step beyond incremental and hierarchical merging, and beyond your own prior structured summarization work. The appendices are detailed enough that the method could be reimplemented, and the paper is honest about several limitations.

The quality results are the solid core. On BooookScore, RepoQA, and LOFT-Spider, PRISM beats incremental and hierarchical merging with statistical significance at p<0.02, and on BooookScore it nearly matches a long-context Gemini 1.5 Pro. Those are meaningful wins, even with only 50 examples per dataset. Generated schemas also land close to hand-crafted ones, which supports the task-agnostic claim. If the paper only claimed better quality per unit of carefully managed cost, I would be fairly convinced.

The soft spot is the efficiency headline. The abstract's "4x shorter contexts" compares only chunk size to the long-context model's full input, ignoring that PRISM's memory grows over time. The cost index subtracts KV cache hits, which is fair only when a cache is actually available and priced that way. Under conservative accounting—total tokens encoded, no cache reuse—PRISM is not cheaper than the baselines; the paper's own Table 3 numbers imply PRISM encodes about 559k tokens versus 227-249k for hierarchical and incremental merging. And in LOFT-Spider, even with caching, PRISM's cost index is no better than incremental. So the "up to 54% cost reduction" and the broader "token-efficient" framing are not yet demonstrated. That needs to be fixed in revision, either by reporting total-token costs and cache sensitivity, or by narrowing the claims.

Other soft spots are minor by comparison: 50 examples per dataset, no PEFT or RAG baselines despite Table 1's qualitative comparison, and no code release. None of these undermine the central quality result, but they do limit how much weight the efficiency conclusion can carry.

This paper is for people working on in-context memory, long-range reasoning with small contexts, and cost-sensitive LLM serving. It deserves a serious referee. I would send it to review, but the review must push the authors to redo the efficiency analysis with conservative accounting and to release code or at least detailed token logs.

Recommendation: engage with it, but treat the quality claims as stronger than the cost claims.

Referee Report

3 major / 5 minor

Summary. The paper introduces PRISM, an in-context method for long-range tasks that processes input in chunks using a typed hierarchical structured memory and programmatic memory revisions. The memory is updated via add/update operations, and an 'amendments' variant appends changes to maximize prefix KV-cache reuse. Experiments on BooookScore, RepoQA, and LOFT-Spider compare PRISM against incremental and hierarchical merging baselines using Gemini 1.5 Pro with a 32k context, and against a long-context ceiling (Gemini 1.5 Pro, 1M context). The paper reports statistically significant quality improvements over the baselines, up to 50x smaller chunk sizes than the long-context input, a cost reduction of up to 54% via cache reuse, and competitive performance with LLM-generated schemas.

Significance. If the quality results hold, PRISM provides a useful training-free, task-agnostic approach that narrows the gap between short-context and long-context models on summarization and structured retrieval tasks, with a clearly described and ablated algorithmic contribution. The detailed prompts and schema definitions in the appendices are a practical strength for reproducibility. However, the central efficiency claim is contingent on KV-cache reuse and specific pricing assumptions; the current cost index can make PRISM appear dramatically cheaper than baselines even when its total token processing is substantially higher. The study is limited to 50 examples per dataset, a single model family, and no empirical comparison against PEFT or RAG despite Table 1 positioning PRISM against them.

major comments (3)

[Section 4.2, Table 3] The cost index is defined as (Net Tokens + 3×Output) / 10^6, where 'Net' subtracts KV-cache hits (Table 3). Under this accounting PRISM's BooookScore cost is 0.31 versus 0.67 (incremental) and 0.43 (hierarchical). However, PRISM's total encoded tokens are 559k versus 227k (hierarchical) and 249k (incremental). With the same 3:1 output weighting but without cache reuse, PRISM's cost would be 559 + 3×47 = 700, above both hierarchical (437) and incremental (672). The abstract's claim of 'reduc[ing] costs by up to 54%' is therefore contingent on prefix-cache availability, the 3:1 output/input price ratio, and the choice of the least favorable baseline. Please report total-token and no-cache costs, and clearly state the cache and pricing assumptions on which the headline efficiency claim rests.
[Table 2] The caption and abstract claim '4-50x smaller contexts' by comparing the 'Ch. Tokens' column (2-8k chunks) with the long-context model's full input (30-121k). This comparison omits the memory tokens: the per-step prompt is (T, S, m_i, d_i), and m_i can be large (Table 3 shows PRISM-BooookScore total encoded tokens of 559k, more than the baselines' 227-249k). Thus '4x shorter context' conflates chunk size with actual input size. Please report the mean per-step prompt length (including memory) and the total number of tokens processed for each method.
[Section 4.2, Table 5] For LOFT-Spider, Table 5 shows PRISM's net encoded tokens (33k) exceed incremental (31k), and its cost index (0.034-0.035) is essentially indistinguishable from incremental (0.035). In the main text, Section 4.2 says 'variants of our method achieve the best results for all metrics across both datasets', which is misleading because Table 5 appears only in Appendix E and the cost advantage over incremental is negligible on this third dataset. The efficiency claim should be qualified per-dataset rather than as a general property.

minor comments (5)

[Table 1] The checkmark alignment is ambiguous: 'Long-context models ✓ ✓ ✓' appears to have three marks for four columns, and the row contradicts the abstract's statement that long-context models require large compute budgets. Please redraw the table with explicit ✓/✗ per column and define each column.
[Section 4] The paper reports p-values ('at worst p = 0.02') but does not describe the statistical test, whether it is paired, or whether it is computed over the 50 examples or over the five solutions. Please specify the test and the unit of analysis.
[Section 4.3, Table 4] The generated-schema performance on LOFT-Spider drops to 58% of manual (0.15 vs 0.26). The discussion ('it is impressive that a strong representation can be constructed') would be strengthened by an analysis of whether this gap reflects schema quality or task difficulty.
[Section 6] The limitation section acknowledges only three hand-crafted schemas and one dataset for the chunk-size scaling study; this is appropriate, but the 'task-agnostic' claim in Table 1 and the conclusion should be tempered accordingly until more domains and schemas are tested.
[Throughout] The paper does not state whether code or schemas will be released. Since the appendices contain detailed prompts and schemas, a public release would considerably aid reproducibility and adoption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; PRISM's empirical claims rest on external benchmarks and measured token counts, not on self-referential definitions.

full rationale

PRISM's contributions are empirical: a method plus evaluations on three external benchmarks. The quality claims (Tables 2 and 4) compare against external baselines (Chang et al.) and a long-context model; no parameter is fitted to the test labels and then renamed as a prediction. The token-efficiency claim (Section 4.2, Tables 3 and 5) is a cost-index calculation from measured token counts and cache-hit rates; although the cost index's reliance on KV-cache reuse is contestable under conservative accounting, that is an evaluation-methodology concern, not a self-referential derivation. The method is defined by Algorithm 1; the output is not equal to its input by construction: the memory is constructed from chunks and the final answer is generated from the final memory. The self-citations (Hwang et al., 2024; Hwang et al., 2025) appear only in related work and are not used as evidence for PRISM's effectiveness. The limitations section explicitly notes schema design is hard and only three schemas were tested, which weakens generalization claims but does not create a circular step. No quoted equation or construction reduces a prediction to a fitted input.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or ontological entities; its novel constructs are algorithmic (structured memory, revisions, amendments). The free parameters are design choices that materially affect the results, and the axioms are the working assumptions about LLM behavior, evaluation benchmarks, and transformer inference that the method depends on.

free parameters (4)

chunk_size = 2k (BooookScore), 8k (RepoQA, LOFT-Spider); 0.5k to 8k in ablation
Chosen per dataset; controls number of incremental steps, memory size, and cost. The paper shows cost stays flat as chunk size varies, so it is not a sensitive fitted constant, but it is a hand-selected design choice.
memory_schema = Hand-crafted per task (Appendix A) or LLM-generated (Appendix B)
Determines what information is stored and how it is organized; results depend strongly on schema quality, as seen when the LLM-generated schema for LOFT-Spider reaches only 58% of the expert schema's accuracy. No principled selection rule is given.
cost_index_output_weight = 3
Cost Index = (Net Tokens + 3 x Output) / 10^6, reflecting assumed API pricing. A different weight would change the relative cost comparisons and the reported 54% reduction.
memory_variant = Amendments with or without updates, or in-place, chosen per dataset
The headline cost reduction (up to 54%) selects the best variant per dataset (BooookScore amendments with updates, RepoQA amendments without updates), so the efficiency claim is a best-case selection across configurations.

assumptions (4)

domain assumption An LLM can reliably emit valid JSONPath revision tuples (add/update) that pass programmatic validation.
Algorithm 1 assumes revisions are valid; the paper does not report parse or validation failure rates, nor a fallback for malformed outputs.
standard math Prefix KV cache reuse for unchanged memory prefixes is semantically correct in the serving stack used.
Transformer attention computes identical KV activations for identical token prefixes; PRISM relies on this property for the amendments representation.
domain assumption The three datasets and their metrics (BooookScore, RepoQA, LOFT-Spider) are representative of long-range reasoning tasks.
The paper explicitly excludes RULER and InfiniteBench as mostly synthetic, so generalization beyond these three domains is not established.
domain assumption Gemini 1.5 Pro with a 32k context is a fair short-context model and the 1M version a fair long-context ceiling.
All comparisons isolate context length but use the same model family; results may not transfer to other base LLMs or to models with different tokenizers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PRISM: Efficient Long-Range Reasoning With Short-Context LLMs." pith.science (2026). https://pith.science/paper/KVR4Q7OV

@misc{pith2026241218914,
  author       = {Pith},
  title        = {Pith review of: PRISM: Efficient Long-Range Reasoning With Short-Context LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KVR4Q7OV}},
  note         = {Machine review of arXiv:2412.18914}
}

read the original abstract

Long-range tasks demand reasoning over long inputs. However, existing solutions are limited, e.g., long-context models require large compute budgets, parameter-efficient fine-tuning (PEFT) needs training data, and retrieval-augmented generation (RAG) entails complex task-specific designs. Though in-context approaches overcome many of these issues, methods with short-context LLMs are inefficient, trading context for processing more tokens. We introduce PRISM, a highly token-efficient in-context method based on structured schemas that outperforms baselines on diverse tasks with 4x shorter contexts. This approach produces concise outputs and efficiently leverages key-value (KV) caches to reduce costs by up to 54%. PRISM scales down to tiny contexts without increasing costs or sacrificing quality, and generalizes to new tasks with minimal effort by generating schemas from task descriptions.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Cartridges: Lightweight and general-purpose long context representations via self-study
cs.CL 2025-06 conditional novelty 6.0 of 10

A per-corpus trained KV cache, called a Cartridge, matches full-context in-context learning quality on long-document benchmarks while using up to 38.6x less serving memory.

Reference graph

Works this paper leans on

19 extracted references · 6 canonical work pages · cited by 1 Pith paper

[1]

Chang, K

Y. Chang, K. Lo, T. Goyal, and M. Iyyer. BooookScore : A systematic exploration of book-length summarization in the era of LLM s. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=7Ttk3RzDeu

work page 2024
[2]

W. Fei, X. Niu, G. Xie, Y. Zhang, B. Bai, L. Deng, and W. Han. Retrieval meets reasoning: Dynamic in-context editing for long-text understanding. CoRR, abs/2406.12331, 2024. doi:10.48550/ARXIV.2406.12331. URL https://doi.org/10.48550/arXiv.2406.12331

arXiv doi:10.48550/arxiv.2406.12331 2024
[3]

Z. He, Z. Qin, N. Prakriya, Y. Sun, and J. Cong. HMT: hierarchical memory transformer for long context language processing. CoRR, abs/2405.06067, 2024. doi:10.48550/ARXIV.2405.06067. URL https://doi.org/10.48550/arXiv.2405.06067

arXiv doi:10.48550/arxiv.2405.06067 2024
[4]

Hsieh, S

C.-P. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, and B. Ginsburg. Ruler: What's the real context size of your long-context language models? ArXiv, abs/2404.06654, 2024

arXiv 2024
[5]

Hwang, Y

E. Hwang, Y. Zhou, J. B. Wendt, B. Gunel, N. Vo, J. Xie, and S. Tata. Enhancing incremental summarization with structured representations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3830--3842, Miami, Florida, USA, Nov. 2024. doi:10.18653/v1/2024.findings-emnlp.220. URL https://aclanthology.org/2024.findings-emnlp.220/

doi:10.18653/v1/2024.findings-emnlp.220 2024
[6]

Hwang, Y

E. Hwang, Y. Zhou, B. Gunel, J. B. Wendt, and S. Tata. SUMIE : A synthetic benchmark for incremental entity summarization. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10839--10864, Abu Dhabi, UAE, Jan. 2025. URL https://aclanthology.org/2025.coling-main.721/

work page 2025
[7]

M. Ivgi, U. Shaham, and J. Berant. Efficient long-text understanding with short-text models. Trans. Assoc. Comput. Linguistics, 11: 0 284--299, 2023. doi:10.1162/TACL\_A\_00547. URL https://doi.org/10.1162/tacl\_a\_00547

doi:10.1162/tacl 2023
[8]

W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP 2023, Koblenz, Germany, October 23-26, 2023 , pages 611--626. ACM , 2023. doi:10.1145/3600006.3613165. UR...

arXiv 2023

Show all 19 references

[9]

J. Lee, A. Chen, Z. Dai, D. Dua, D. S. Sachan, M. Boratko, Y. Luan, S. M. R. Arnold, V. Perot, S. Dalmia, H. Hu, X. Lin, P. Pasupat, A. Amini, J. R. Cole, S. Riedel, I. Naim, M. Chang, and K. Guu. Can long-context language models subsume retrieval, rag, sql, and more? CoRR, ab...

2024 doi arXiv
[10]

J. Liu, J. L. Tian, V. Daita, Y. Wei, Y. Ding, Y. K. Wang, J. Yang, and L. Zhang. Repo QA : Evaluating long context code understanding. CoRR, abs/2406.06025, 2024. doi:10.48550/ARXIV.2406.06025. URL https://doi.org/10.48550/arXiv.2406.06025

2024 doi arXiv
[11]

N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12: 0 157--173, 2023

2023
[12]

Munkhdalai, M

T. Munkhdalai, M. Faruqui, and S. Gopal. Leave no context behind: Efficient infinite context transformers with infini-attention. CoRR, abs/2404.07143, 2024. doi:10.48550/ARXIV.2404.07143. URL https://doi.org/10.48550/arXiv.2404.07143

2024 doi arXiv
[13]

Packer, V

C. Packer, V. Fang, S. G. Patil, K. Lin, S. Wooders, and J. E. Gonzalez. Mem GPT : Towards LLM s as operating systems. CoRR, abs/2310.08560, 2023. doi:10.48550/ARXIV.2310.08560. URL https://doi.org/10.48550/arXiv.2310.08560

2023 doi arXiv
[14]

R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean. Efficiently scaling transformer inference. In Proceedings of the Sixth Conference on Machine Learning and Systems, MLSys 2023, Miami, FL, USA, June 4-8, 2023. mlsys.org, 2023

2023
[15]

H. Qian, Z. Liu, P. Zhang, K. Mao, Y. Zhou, X. Chen, and Z. Dou. Are long-llms a necessity for long-context tasks? ArXiv, abs/2405.15318, 2024

2024 arXiv
[16]

M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. P. Lillicrap, J. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, I. Antonoglou, R. Anil, S. Borgeaud, A. M. Dai, K. Millican, E. Dyer, M. Glaese, T. Sottiaux, B. Lee, F. Viola, M. Reynolds, Y. Xu, J. Molloy, J....

2024 doi arXiv
[17]

W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, and F. Wei. Augmenting language models with long-term memory. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, Decem...

2023
[18]

Zhang, Y

X. Zhang, Y. Chen, S. Hu, Z. Xu, J. Chen, M. K. Hao, X. Han, Z. L. Thai, S. Wang, Z. Liu, and M. Sun. Infinitebench: Extending long context evaluation beyond 100k tokens. ArXiv, abs/2402.13718, 2024

2024 arXiv
[19]

Zheng, L

L. Zheng, L. Yin, Z. Xie, J. Huang, C. Sun, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, C. W. Barrett, and Y. Sheng. Efficiently programming large language models using sglang. CoRR, abs/2312.07104, 2023. doi:10.48550/ARXIV.2312.07104. URL https://doi.org/10.485...

2023 doi arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Chang, K

Y. Chang, K. Lo, T. Goyal, and M. Iyyer. BooookScore : A systematic exploration of book-length summarization in the era of LLM s. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=7Ttk3RzDeu

work page 2024

[2] [2]

W. Fei, X. Niu, G. Xie, Y. Zhang, B. Bai, L. Deng, and W. Han. Retrieval meets reasoning: Dynamic in-context editing for long-text understanding. CoRR, abs/2406.12331, 2024. doi:10.48550/ARXIV.2406.12331. URL https://doi.org/10.48550/arXiv.2406.12331

arXiv doi:10.48550/arxiv.2406.12331 2024

[3] [3]

Z. He, Z. Qin, N. Prakriya, Y. Sun, and J. Cong. HMT: hierarchical memory transformer for long context language processing. CoRR, abs/2405.06067, 2024. doi:10.48550/ARXIV.2405.06067. URL https://doi.org/10.48550/arXiv.2405.06067

arXiv doi:10.48550/arxiv.2405.06067 2024

[4] [4]

Hsieh, S

C.-P. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, and B. Ginsburg. Ruler: What's the real context size of your long-context language models? ArXiv, abs/2404.06654, 2024

arXiv 2024

[5] [5]

Hwang, Y

E. Hwang, Y. Zhou, J. B. Wendt, B. Gunel, N. Vo, J. Xie, and S. Tata. Enhancing incremental summarization with structured representations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3830--3842, Miami, Florida, USA, Nov. 2024. doi:10.18653/v1/2024.findings-emnlp.220. URL https://aclanthology.org/2024.findings-emnlp.220/

doi:10.18653/v1/2024.findings-emnlp.220 2024

[6] [6]

Hwang, Y

E. Hwang, Y. Zhou, B. Gunel, J. B. Wendt, and S. Tata. SUMIE : A synthetic benchmark for incremental entity summarization. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10839--10864, Abu Dhabi, UAE, Jan. 2025. URL https://aclanthology.org/2025.coling-main.721/

work page 2025

[7] [7]

M. Ivgi, U. Shaham, and J. Berant. Efficient long-text understanding with short-text models. Trans. Assoc. Comput. Linguistics, 11: 0 284--299, 2023. doi:10.1162/TACL\_A\_00547. URL https://doi.org/10.1162/tacl\_a\_00547

doi:10.1162/tacl 2023

[8] [8]

W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP 2023, Koblenz, Germany, October 23-26, 2023 , pages 611--626. ACM , 2023. doi:10.1145/3600006.3613165. UR...

arXiv 2023