Pith. sign in

REVIEW 4 major objections 5 minor 36 references

Making LLM serving aware of semantic query plans lets intermediate KV-cache state be reused across operators, cutting query time without changing results.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Query-aware LLM serving with adaptive memory-aware pipelining reuses cross-operator KV-cache state and speeds semantic queries by up to 4.57×.

T0 review reviewed 2026-07-30 challenge →

load-bearing objection Real systems contribution on query-aware KV scheduling; the 4.57× headline oversells a cleaner ~1.2–1.8× mechanism gain plus baseline noise. the 4 major comments →

arxiv 2607.23815 v1 pith:VRRPTJJX submitted 2026-07-26 cs.DB cs.AIcs.CL

Kalypso: Relational LLM Serving

Hojae Son , Md Ashraful Islam , Huy Gia Cao , Hui Guan , Marco Serafini This is my paper
classification cs.DB cs.AIcs.CL
keywords relational LLM servingsemantic operatorsKV-cache reusepipelined query executionmemory-aware schedulingprefix cachingsemantic query processing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Semantic queries treat large language models as relational operators over unstructured text—filters, maps, joins, and so on—but today’s stacks hand each operator call to a request-centric serving engine that never sees the plan. Prefill work on shared tuple prefixes is therefore often thrown away and recomputed. This paper argues that a relational serving layer between the query system and the engine can pipeline operators so a tuple’s cached attention state is consumed by the next operator before eviction, while still preserving exact query semantics and accuracy. Kalypso implements that idea with an API for operator plans and an adaptive scheduler that assigns GPU memory budgets per pipeline stage, rebalancing online to avoid both starving downstream work and saturating the cache with upstream tasks. Across fact verification, medical notes, biomedical matching, and contract entailment, the system reports end-to-end speedups up to about 4.6× over operator-at-a-time baselines that issue a similar number of model calls.

Core claim

Relational LLM serving—query-plan-aware admission and memory control over pipelined semantic operators—can substantially reduce end-to-end query completion time by raising cross-operator KV-cache hit rates, without changing query semantics or output accuracy. Kalypso shows this with speedups up to 4.57× versus request-centric, materializing baselines on diverse workloads.

What carries the argument

Adaptive memory-aware scheduling of pipeline stages and tasks: each stage gets a dynamic KV-cache budget; the scheduler admits tasks only when budget remains, tracks waiting-queue pressure to detect starvation versus saturation, and rebalances memory between upstream and downstream stages so reusable prefixes are consumed before eviction.

Load-bearing premise

Consecutive operators must share a long stable prompt prefix for each tuple, and that prefix must stay in GPU memory until the next operator runs—either by pinning or by careful launch timing under ordinary cache eviction.

What would settle it

Run the same multi-operator plans with prompts rewritten so shared system-plus-tuple prefixes disappear, or force immediate eviction of every completed request’s cache: if Kalypso’s speedups over operator-at-a-time baselines vanish while call counts stay similar, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Semantic query systems can treat serving-layer pipelining as a free efficiency win complementary to cascades, proxy models, and plan rewrites.
  • GPU memory for KV cache becomes a first-class query-execution resource that must be budgeted across stages, not only across independent requests.
  • Operator APIs need to declare pipelining, predicate pruning, and join structure so the serving layer can form stages and dependent tasks.
  • Under tight KV-cache budgets, plan-aware admission control can avoid the sharp latency cliffs that materializing intermediate tables causes when tables exceed cache capacity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If prompt templates stay prefix-aligned by convention, relational serving could become a default substrate for multi-step agent and RAG pipelines, not only declarative SQPS plans.
  • The same starvation/saturation tradeoff likely appears whenever any multi-stage LLM workflow shares long context across steps under a finite paged KV pool.
  • Virtual pinning nearly matching explicit pinning suggests many engines could gain most of the benefit without new pinning APIs, if higher-level schedulers control admission tightly enough.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces relational LLM serving and Kalypso, a layer between semantic query processing systems and request-centric engines such as vLLM. Query plans are divided into pipelines and stages; stages run as tuple-level tasks, with dependent tasks launched promptly so downstream operators can reuse upstream KV-cache prefixes. An adaptive scheduler assigns and rebalances per-stage memory budgets to avoid starvation and saturation, while token-bound estimation, explicit or virtual pinning, retry handling, and deadlock recovery address uncertain memory demand. Experiments on FEVER, MEDEC, BioDEX, and ContractNLI compare Kalypso with Lotus and Palimpzest over vLLM and Llama-3.3-70B. Kalypso reports speedups up to 4.57× with broadly similar oracle-call counts, supported by ablations for pipelining, adaptive budgeting, pinning, token bounds, and memory pressure.

Significance. Making LLM serving query-aware is a timely and useful systems direction. Kalypso’s stage/task abstraction, adaptive budgets, and virtual pinning could be adopted beneath multiple semantic-query systems, particularly because the approach is intended to complement rather than replace cascades and query optimization. The evaluation has substantial strengths: four distinct workloads, a common vLLM/model configuration, LLM-call counts, repeated runs, blocking and static-budget ablations, virtual-versus-explicit pinning, token-budget sensitivity, and memory-pressure sweeps. If the gains are shown to preserve outputs and are properly attributed to the serving layer, the work would be a significant contribution. At present, the magnitude and mechanism of the largest end-to-end claim remain insufficiently isolated.

major comments (4)
  1. [§7.3, Figure 7; §7.4, Figure 9; Table 2] §7.3/Fig. 7 versus §7.4/Fig. 9 and Table 2: the 4.57× ContractNLI headline is not fully attributable to relational serving. Lotus takes 4,854s, Palimpzest 2,373.4s, Kalypso 1,062.4s, and blocking Kalypso 1,940s. Thus the pipelined/blocking ablation shows 1.83×, while Kalypso is 2.23× faster than the stronger request-centric baseline. Lotus and Palimpzest differ by about 2× despite similar oracle-call counts. Please add a lean request-centric control using the same UDFs, prompts, engine, and batching, or otherwise decompose orchestration, batching, and serving effects; report the strongest-baseline speedup alongside the headline.
  2. [§2.1, §4, §5.2, §6, §7.4/Figure 9] The central mechanism assumes exact shared prompt prefixes and timely retention (§2.1, §4, §5.2, §6). However, Fig. 9’s blocking-versus-pipelined comparison confounds operator overlap with KV reuse, and the evaluation does not report cache-hit rates, reusable-prefix lengths, recomputed prefill tokens, premature evictions, or retries caused by eviction. Since virtual pinning depends on LRU behavior and prompt formatting is delegated to SQPS UDFs, please measure these quantities directly and include a sensitivity test in which prompt layouts reduce prefix sharing or eviction timing violates the assumed retention window.
  3. [Abstract; §1; §7.1–§7.3; Table 2] The abstract and §1 claim that query semantics and output accuracy are preserved, but §7 reports latency and call counts rather than output equivalence or task quality. This matters because prompts may be reformatted for prefix sharing, predicate calls are capped at 8 tokens, and cascade use differs: FEVER has 183.3 Lotus oracle fallbacks versus 243.6 for Kalypso, while cascading is disabled for Kalypso on MEDEC and ContractNLI. Please compare final outputs under deterministic settings and report workload accuracy/F1, with matched oracle-only and matched-cascade configurations.
  4. [§7.1; §7.3/Figure 7] §7.1 states that all systems use the same operator implementations and manually optimized plans, but it is unclear whether Palimpzest’s optimizer was disabled, how its default batch limit of 64 interacts with vLLM admission, and where the large ContractNLI gap between the two request-centric systems arises. Per-operator timelines and baseline orchestration/batching settings are needed to establish that the end-to-end comparison isolates the serving layer rather than framework overhead.
minor comments (5)
  1. [§5.1, Figure 4] §5.1/Figure 4 contains an apparent unresolved editorial note: the figure is said to place outputs in the queue, while the text and Algorithm 1 place new tasks there. Please make the queue semantics consistent.
  2. [§5.3, Algorithm 1; §7.1] Algorithm 1 and §7.1 use α=1 and β=0.5 without a sensitivity study. A small sweep or explanation of why the results are insensitive to these thresholds would strengthen the adaptive-scheduling claims.
  3. [§7.1] The static 8-token bound for predicate calls should be justified empirically. Please report whether truncation or retries occurred and how often the bound was reached.
  4. [§7; §9] The evaluation uses one model, one GPU platform, and left-deep static plans. This is reasonable for a prototype, but the conclusion should state these scope limits more explicitly, especially for operators with variable fanout or non-prefix-compatible prompts.
  5. [§4, §5.1, §7.2, §7.4, §8] Several typographical errors should be corrected, including “dowstream,” “runing,” “Speeups,” “in is a separate pipeline,” and “we the small LLM.”

Circularity Check

0 steps flagged

No significant circularity: systems paper with measured speedups, not a fitted derivation presented as prediction.

full rationale

Kalypso is an engineering/systems paper. Its central claim is empirical end-to-end latency improvement from query-aware pipelined scheduling and KV-cache reuse, measured against external baselines (Lotus, Palimpzest) on defined workloads with similar oracle call counts. The scheduler (Algorithm 1), memory estimator (99th-percentile output/input ratio), and rebalance thresholds (α, β) are design knobs whose effects are ablated (blocking vs pipelined, static vs adaptive budgets, virtual vs explicit pinning, fixed token budgets). Nothing in the paper presents a quantity as a first-principles prediction that is definitionally equal to a fitted input, nor does it rest a uniqueness or impossibility claim on self-citation. Baseline-sensitivity and magnitude of the 4.57× headline are evaluation-interpretation issues, not circularity. Score 0; steps empty.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 4 invented entities

The result rests on standard LLM serving mechanics (prefill/decode, paged KV cache, opportunistic prefix caching), on SQPS operator semantics, and on a few scheduler tuning constants. No new physical entities; invented pieces are software abstractions (relational serving API, stages/tasks, virtual pinning, adaptive rebalance rules).

free parameters (4)
  • rebalance high/low queue factors α, β = α=1, β=0.5
    Stage starving/saturated thresholds are high_s=α·budget/minBudget and low_s=β·budget/minBudget; evaluation fixes α=1, β=0.5 without a broad sensitivity study across many queries.
  • output-to-input token-ratio percentile for memory bounds = 99th percentile
    For generative operators, Kalypso uses the calibrated 99th percentile of output/input tokens as the admission bound; the percentile choice trades retries vs concurrency.
  • per-stage minBudget_s and initial last-stage-heavy split = min one-task budget; remainder to last stage
    Each stage keeps at least one-task memory; initially non-last stages get minBudget and the last stage gets the remainder—an intentional policy prior adapted online.
  • static max tokens=8 for predicate LLM calls = 8 tokens
    Fixed short bound for filter/join boolean-style calls across systems to limit runaway generations; affects admission and retry behavior.
axioms (6)
  • domain assumption Autoregressive LLM inference splits into prefill (materialize KV) and decode; KV cache is the primary GPU memory bottleneck and grows with sequence length.
    Section 2.1 background; load-bearing for why prefix reuse matters.
  • domain assumption Modern engines (vLLM/SGLang) provide opportunistic automatic prefix caching but may evict completed-request blocks under pressure with no cross-request retention guarantee.
    Section 2.1; motivates memory-aware admission rather than relying on cache alone.
  • domain assumption Semantic operator prompts can be written so consecutive operators on the same tuple share a long prefix (system + tuple context) with only the task instruction differing.
    Figure 1 and Section 4; reuse is only as good as prompt structure supplied by UDFs.
  • ad hoc to paper Virtual pinning works under an LRU-like eviction policy when the scheduler limits concurrent admissions so still-needed prefixes remain hot.
    Section 6; alternative to explicit pin support, validated empirically but policy-dependent.
  • ad hoc to paper Query plans are static and left-deep; Cartesian products pipeline on the left input with a static right table (or ICP UDF subset).
    Sections 4–5 API/execution model; scopes the scheduler.
  • domain assumption Scheduling and memory control do not change operator semantics or output accuracy relative to the same UDFs and model.
    Stated in abstract/intro; frames the work as pure execution optimization.
invented entities (4)
  • Relational LLM serving layer independent evidence
    purpose: Interpose between SQPS and request-centric engines to schedule plan-aware pipelined inference and KV retention.
    Core abstraction named in the title and Section 3; software architecture, not a physical postulate.
  • Stage/task pipeline model with CP-separated stages independent evidence
    purpose: Unit of admission control and dependency tracking for prefix reuse across operators.
    Section 5.1 execution framework; standard systems structuring, specific to this design.
  • Adaptive memory-budget rebalancer (starve/saturate transfers) independent evidence
    purpose: Online shift of KV budget between upstream and downstream stages without static ratios.
    Algorithm 1 Rebalance; central algorithmic invention evaluated against static splits.
  • Virtual pinning independent evidence
    purpose: Best-effort prefix retention via admission control without engine pin APIs.
    Section 6; optional counterpart to explicit pinning.

reviewed 2026-07-30 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Kalypso: Relational LLM Serving." pith.science (2026). https://pith.science/paper/VRRPTJJX

@misc{pith2026260723815,
  author       = {Pith},
  title        = {Pith review of: Kalypso: Relational LLM Serving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VRRPTJJX}},
  note         = {Machine review of arXiv:2607.23815}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large language models are increasingly used as semantic operators for filtering, extracting, ranking, joining, and transforming unstructured data. Existing semantic query processing systems invoke request-centric LLM serving systems that are unaware of the query plan, leaving substantial performance opportunities unused. This paper introduces relational LLM serving, an abstraction that makes LLM serving aware of semantic query structure while preserving query semantics and output accuracy. The key opportunity is pipelined execution across semantic operators: when intermediate tuples flow directly from one operator to the next, their KV-cache state can be reused instead of recomputed. We present Kalypso, a relational LLM serving system that exposes an API for semantic query plans and executes them using an adaptive, memory-aware scheduling algorithm. Kalypso addresses a new online scheduling problem in which pipelined operator execution is coupled with GPU memory pressure management to reuse KV-cache state in the serving engine before eviction. Its scheduler continuously adjusts memory allocations to balance upstream parallelism, downstream progress, and GPU utilization. Our evaluation shows that Kalypso improves query completion time over baselines using request-centric LLM serving, with speedups up to 4.57x across diverse workloads, demonstrating that query-aware LLM serving can substantially improve the efficiency of semantic query execution.

Figures

Figures reproduced from arXiv: 2607.23815 by Hojae Son, Hui Guan, Huy Gia Cao, Marco Serafini, Md Ashraful Islam.

Figure 1
Figure 1. Figure 1: Prompt structure for a filter → map query. 10 30 50 70 90 110 130 # Tuples 0 10 20 30 40 50 Runtime (s) SemOp_1 SemOp_2 0.0 0.5 1.0 1.5 2.0 Throughput (tuples/sec) Throughput [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The first operator prefills the KV cache for each input [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Executing a semantic query plan with pipelining. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: A parallel depth-first policy can starve downstream [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A parallel breadth-first policy can saturate the KV [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: End-to-end latency and speedup across workloads. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Effect of GPU memory utilization on end-to-end latency across workloads. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Latency for Kalypso in blocking mode (operator [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 12
Figure 12. Figure 12: Latency under fixed output-token budgets. The [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 1 canonical work pages

  1. [1]

    Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Tam- ing Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, Santa Clara, CA, 117–134. https://www.useni...

    2024

  2. [2]

    Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, et al. 2022. Deepspeed-inference: enabling efficient inference of trans- former models at unprecedented scale. InSC22: International Conference for High Performance Computing, Networking, Storage and Analysi...

    2022

  3. [3]

    Deter- ministic

    Berk Atıl, Sarp Aykent, Alexa Chittams, Lisheng Fu, Rebecca J. Passonneau, Evan Radcliffe, Guru Rajan Rajagopal, Adam Sloan, Tomasz Tudrej, Ferhan Ture, Zhe Wu, Lixinyu Xu, and Breck Baldwin. 2025. Non-Determinism of “Deter- ministic” LLM System Settings in Hosted Environments. InProceedings of the 5th Workshop on Evaluation and Comparison of NLP Systems,...

    2025

  4. [4]

    Asma Ben Abacha, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, and Thomas Lin. 2025. MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes. InFindings of the Association for Computa- tional Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025. Associ- ation for Computational Linguistics, 22539–22550. h...

    2025

  5. [5]

    Shu Chen, Deepti Raghavan, and Uğur Çetintemel. 2025. Continuous Prompts: LLM-Augmented Pipeline Processing over Unstructured Streams. arXiv:2512.03389 [cs.DB] https://arxiv.org/abs/2512.03389

    arXiv 2025

  6. [6]

    Yeounoh Chung, Rushabh Desai, Jian He, Yu Xiao, Thibaud Hottelier, Yves- Laurent Kom Samo, Pushkar Khadilkar, Xianshun Chen, Sam Idicula, Fatma Oz- can, Alon Halevy, and Yannis Papakonstantinou. 2026. 100x Cost & Latency Re- duction: Performance Analysis of AI Query Approximation using Lightweight Proxy Models: [Experiments & Analysis].Proceedings of the ...

    doi:10.1145/3802002 2026

  7. [7]

    Karel D’Oosterlinck, François Remy, Johannes Deleu, Thomas Demeester, Chris Develder, Klim Zaporojets, Aneiss Ghodsi, Simon Ellershaw, Jack Collins, and Christopher Potts. 2023. BioDEX: Large-Scale Biomedical Adverse Drug Event Extraction for Real-World Pharmacovigilance. InFindings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouam...

    work page doi:10.18653/v1/2023.findings-emnlp.8 2023

  8. [8]

    In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, and Lin Zhong. 2024. Prompt cache: Modular attention reuse for low-latency infer- ence.Proceedings of Machine Learning and Systems6 (2024), 325–338

    2024

  9. [9]

    Guoyu Hu, Shaofeng Cai, Tien Tuan Anh Dinh, Zhongle Xie, Cong Yue, Gang Chen, and Beng Chin Ooi. 2025. HAKES: Scalable Vector Database for Em- bedding Search Service.Proceedings of the VLDB Endowment18, 9 (May 2025), 3049–3062. doi:10.14778/3746405.3746427

    arXiv 2025

  10. [10]

    Floudas, Fangyuan Chen, Changlin Gong, Dara Bracken-Clarke, Elisabetta Xue, Yifan Yang, Jimeng Sun, and Zhiy- ong Lu

    Qiao Jin, Zifeng Wang, Charalampos S. Floudas, Fangyuan Chen, Changlin Gong, Dara Bracken-Clarke, Elisabetta Xue, Yifan Yang, Jimeng Sun, and Zhiy- ong Lu. 2024. Matching patients to clinical trials with large language models. Nature Communications15, 1 (2024), 9074. doi:10.1038/s41467-024-53081-z

    doi:10.1038/s41467-024-53081-z 2024

  11. [11]

    Saehan Jo and Immanuel Trummer. 2024. Thalamusdb: Approximate query pro- cessing on multi-modal data.Proceedings of the ACM on Management of Data2, 3 (2024), 1–26

    2024

  12. [12]

    Yuta Koreeda and Christopher Manning. 2021. ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts. InFindings of the Association for Computational Linguistics: EMNLP 2021, Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistics, Punta Cana, Dominican Republic,...

    doi:10.18653/v1/2021.findings-emnlp.164 2021

  13. [13]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. InProceedings of the 29th symposium on operating systems principles. 611–626

    2023

  14. [14]

    Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baile Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, Rana Shahout, et al. 2025. Palimpzest: Optimizing ai-powered analytics with declarative query processing. InProceedings of the Conference on Innovative Database Research (CIDR). 2

    2025

  15. [15]

    Gonzalez, and Matei Zaharia

    Shu Liu, Asim Biswal, Amog Kamsetty, Audrey Cheng, Luis Gaspar Schroeder, Liana Patel, Shiyi Cao, Xiangxi Mo, Ion Stoica, Joseph E. Gonzalez, and Matei Zaharia. 2025. Optimizing LLM Queries in Relational Data Analytics Workloads. InEighth Conference on Machine Learning and Systems. https://openreview.net/ forum?id=R7bK9yycHp

    2025

  16. [16]

    Shicheng Liu, Jialiang Xu, Wesley Tjangnaka, Sina Semnani, Chen Yu, and Mon- ica Lam. 2024. SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models. InFindings of the Association for Computa- tional Linguistics: NAACL 2024, Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, M...

    2024

  17. [17]

    Gonzalez, M Waleed Kadous, and Ion Stoica

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. 2025. RouteLLM: Learning to Route LLMs from Preference Data. InThe Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id= 8sSqNntaMr

    2025

  18. [18]

    Olga Ovcharenko, Matthias Boehm, and Sebastian Schelter. 2026. SemPipes: Optimizable Semantic Data Operators for Tabular Machine Learning Pipelines. arXiv:2602.05134 [cs.LG] doi:10.48550/arXiv.2602.05134 VLDB Demo 2026

    doi:10.48550/arxiv.2602.05134 2026

  19. [19]

    Liana Patel, Siddharth Jha, Melissa Pan, Harshit Gupta, Parth Asawa, Carlos Guestrin, and Matei Zaharia. 2025. Semantic Operators and Their Optimiza- tion: Enabling LLM-Based Data Processing with Accuracy Guarantees in LO- TUS.Proceedings of the VLDB Endowment18, 11 (2025), 4171–4184

    2025

  20. [20]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented lan- guage models.Transactions of the Association for Computational Linguistics11 (2023), 1316–1331

    2023

  21. [21]

    Matthew Russo and Tim Kraska. 2026. Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics. InProceedings of the Conference on Innovative Data Systems Research (CIDR). arXiv:2509.02751. Son et al

    Pith/arXiv arXiv 2026

  22. [22]

    Shreya Shankar, Tristan Chambers, Tarak Shah, Aditya G Parameswaran, and Eugene Wu. 2025. DocETL: Agentic Query Rewriting and Evaluation for Com- plex Document Processing.Proceedings of the VLDB Endowment18, 9 (2025), 3035–3048

    2025

  23. [23]

    Shreya Shankar, Sepanta Zeighami, and Aditya Parameswaran. 2026. Task Cascades for Efficient Unstructured Data Processing. InProceedings of the 2026 ACM SIGMOD International Conference on Management of Data. arXiv:2601.05536 [cs.DB] https://arxiv.org/abs/2601.05536

    arXiv 2026

  24. [24]

    Biao Sun, Ziming Huang, Hanyu Zhao, Wencong Xiao, Xinyi Zhang, Yong Li, and Wei Lin. 2024. Llumnix: Dynamic Scheduling for Large Language Model Serving. In18th USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI 24). https://arxiv.org/abs/2406.03243

    Pith/arXiv arXiv 2024

  25. [25]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mit- tal. 2018. FEVER: a Large-scale Dataset for Fact Extraction and VERifica- tion. InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), Marilyn Walker, Heng Ji, and Ama...

    doi:10.18653/v1/n18-1074 2018

  26. [26]

    Immanuel Trummer. 2025. Implementing Semantic Join Operators Efficiently. arXiv:2510.08489 [cs.DB] doi:10.48550/arXiv.2510.08489

    doi:10.48550/arxiv.2510.08489 2025

  27. [27]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. InAdvances in Neural Information Processing Systems, Vol. 30

    2017

  28. [28]

    vLLM Project. 2026. Reproducibility - vLLM. https://docs.vllm.ai/en/latest/ usage/reproducibility/

    2026

  29. [29]

    Gyeong-In Yu, Jeongmin Jeong, Gyuhong Kim, Soojeong Shin, and Byung-Gon Kim. 2022. Orca: A Distributed Serving System for Transformer-Based Gener- ative Models.16th USENIX Symposium on Operating Systems Design and Imple- mentation (OSDI 22)(2022), 521–538

    2022

  30. [30]

    Jiayi Yuan, Hao Li, Xinheng Ding, Wenya Xie, Yu-Jhe Li, Wentian Zhao, Kun Wan, Jing Shi, Xia Hu, and Zirui Liu. 2026. Understanding and Mitigating Nu- merical Sources of Nondeterminism in LLM Inference. InThe Thirty-ninth An- nual Conference on Neural Information Processing Systems. https://openreview. net/forum?id=Q3qAsZAEZw

    2026

  31. [31]

    Sepanta Zeighami, Shreya Shankar, and Aditya Parameswaran. 2026. Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees. In Proceedings of the 2026 International Conference on Management of Data. arXiv:2509.02896 [cs.DB] doi:10.48550/arXiv.2509.02896 To appear

    Pith/arXiv arXiv doi:10.48550/arxiv.2509.02896 2026

  32. [32]

    Sepanta Zeighami, Shreya Shankar, and Aditya Parameswaran. 2026. Featurized-Decomposition Join: Low-Cost Semantic Joins with Guaran- tees.Proceedings of the VLDB Endowment(2026). arXiv:2512.05399 [cs.DB] doi:10.48550/arXiv.2512.05399 To appear

    doi:10.48550/arxiv.2512.05399 2026

  33. [33]

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. 2024. Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems37 (2024), 62557–62583

    2024

  34. [34]

    Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang. 2024. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. In18th USENIX Sym- posium on Operating Systems Design and Implementation (OSDI 24). https: //arxiv.org/abs/2401.09670

    Pith/arXiv arXiv 2024

  35. [148]

    doi:10.18653/v1/2025.eval4nlp-1.12

    doi:10.18653/v1/2025.eval4nlp-1.12 2025

  36. [4555]

    doi:10.18653/v1/2024.findings-naacl.283

    doi:10.18653/v1/2024.findings-naacl.283 2024

This paper was first reviewed by grok-4.5 on July 30, 2026.