{"id":"cd408be5-037f-4c55-8443-a08d45b16420","arxiv_id":"2607.19214","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Keepalive pings at the largest interval safely under a provider's cache TTL keep agentic prompts warm and profitable, with savings bounded by ~tau(w/r - 1), while universal adoption creates an unpriced congestion externality.","lead":"This paper measures how long LLM API providers keep a cached prompt alive and shows that clients can \"ping\" the cache during tool pauses to cut costs up to 12.5x. It then derives the optimal ping interval and warns that if everyone does this, shared cache tiers may degrade until providers start charging for cache residency.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"OpenAI τ*=480s saving (2.45x) assumes every ping hits; stochastic retention at 480s—unmeasured—could shrink the saving to ~1.5x or break-even.","rationale":"I focused on the OpenAI τ*=480s result because the central claim's headline savings numbers are the load-bearing evidence for 'keepalive pays.' The reader's weakest assumption (refresh-on-read semantics and congestion) is related but broader; refresh-on-read is actually supported by the keepalive holding at 600s where the baseline is evicted, and the congestion extrapolation is explicitly argued as a prediction in §6, not part of the current per-client saving. The unmodeled stochasticity of retention is a sharper, currently unaddressed gap: §5.2 shows OpenAI decays gradually, so a deterministic ping-cost model is not appropriate for the very case where the paper claims the largest saving (2.45x). The expected-cost calculation shows the result is sensitive to plausible miss rates, and the 6-sample run cannot bound that risk. This supports the reader's CONDITIONAL verdict and adds a specific validation: measure q at 480s and re-compute expected costs. I also noticed Table 1's 1-hour Anthropic row is internally inconsistent (τ*≈50min implies I_max≈15.8h, not 3.3h), but this is a minor numerical issue that does not affect the central claim.","tokens_in":10205,"tokens_out":18665,"duration_ms":206949,"concrete_test":"Run OpenAI baseline retention at 480s idle with n≥30 (same protocol as §4, no keepalive), estimate q = fraction of cold re-reads. Compute expected keepalive cost 0.4+2.7q for the 1800s pause; if q>~0.15, the 2.45x saving is not supported and τ*=240s is the better policy. Also re-analyze the existing Table 4 run ping-by-ping: count how many individual keepalive pings (not just final reqB) were warm; if any ping missed, the median-cost table understates expected cost, and the saving factor should be recomputed with the empirical miss rate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest point in the central economic claim is the assumption, embedded in the §5.3 cost model (cost=(I/τ+1)r, break-even I_max=τ(w/r−1)), that every keepalive ping is a cache read at rate r. This treats provider retention as deterministic below the eviction point. But §5.1/§5.2 describe OpenAI as a 'soft, lossy cache' with gradual decay: baseline warm rate is 39/48 (81%) at 600s and ~50% at 15–20min, not a hard cliff. The paper nevertheless sets OpenAI τ*=480s and reports a 2.45x saving from a single 6/6-warm run (Table 4). Let q be the probability a 480s-gap ping misses. Normalizing w=1.0, r=0.1, a 1800s idle with 3 pings plus a final read has expected keepalive cost 0.4+2.7q, against baseline 1.0; break-even q≈0.22. The 6-cell run cannot rule out q=0.2 (95% CI lower bound on p is ~0.54), and q=0.19 is exactly the observed miss rate at 600s. If q=0.1, the saving drops from 2.45x to ~1.5x; if q=0.19, it is near break-even and τ*=240s becomes optimal. The paper never measures retention at 480s or reports per-ping misses, so the headline OpenAI saving is an upper-bound estimate, not a tested one. This is a measurement gap in the paper's own terms (§7: 'within-run correlation'), not a derivation error.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper measures prompt-cache retention and client-side keepalive efficacy across Anthropic, DeepSeek, Google, and OpenAI. It derives a cost model I_max = τ(w/r − 1), recommends pinging at the largest interval safely under each provider's measured retention (≈240 s for Anthropic/DeepSeek, ≈480 s for OpenAI) rather than the common 30 s convention, and reports that at a 30-minute pause a keepalive saves 1.56× on Anthropic and up to 2.45× on OpenAI, while DeepSeek and Google only get latency benefits. It further argues that universal rational adoption would degrade shared prompt-cache tiers and push providers toward metering cache residency.","tokens_in":10512,"tokens_out":17892,"duration_ms":161144,"significance":"The paper addresses a practical, timely problem and is methodologically careful: timing-integrity gates, warm-reference sanity checks, separate baseline/keepalive time blocks, per-run Fisher exact tests with Bonferroni correction, explicit threats-to-validity, and a transparent, self-contained derivation of the break-even horizon. The cross-provider retention curves and the per-provider policy prescription are useful contributions. However, the central quantitative claims are not yet established at the advertised precision: the cost model treats provider retention as deterministic, the paper's own DeepSeek data contradict that model, and the headline OpenAI saving rests on one small, unreplicated run. The errors are local and fixable, so the paper is a credible major revision rather than a rejection.","major_comments":[{"comment":"The claimed 2.45× OpenAI saving assumes every keepalive ping is a cached read at price ratio r. But §5.1/§5.2 describe OpenAI as a 'soft, lossy cache' with gradual decay, and the τ=480s column is a single n=6 run. Under the paper's own §7 within-run-correlation caveat, 6/6 warm cells cannot tightly bound a per-ping miss probability. With per-ping miss probability q, normalized w=1, r=0.1, three pings plus the final read have expected cost 4(q + (1−q)·0.1); break-even is q≈0.17. A miss rate of 0.1 reduces the saving to ≈1.3×, and q=0.2 makes it a loss. The paper reports no per-ping miss counts. The headline saving is therefore an upper bound, and τ*=480s for OpenAI is not established. Please add per-ping hit/miss data at candidate intervals or present a stochastic cost model with confidence intervals.","section":"§5.3, Table 4 (OpenAI τ=480s)"},{"comment":"The measured DeepSeek costs contradict the cost model in §5.3. Using Table 1's r=0.10, w=1.00, the 1800s/τ=240s cell should cost roughly 8–9 read-priced pings ≈ $0.018–0.019 per 100k tokens, below the $0.022 re-prefill, so the model predicts a small saving. Table 4 reports $0.108, i.e., a large loss. The paper does not reconcile this or report whether the pings actually hit. Either DeepSeek's effective read price on this backend is not 0.1×, or pings miss at a substantial rate. Since this model is the basis for I_max and the 'too cheap to insure' conclusion, the discrepancy must be explained; otherwise the model is not predictive for a provider where it fails.","section":"Table 4 (DeepSeek) vs. §5.3"},{"comment":"There is an internal inconsistency in OpenAI's τ* and I_max. Table 1 lists OpenAI τ*≈240s and I_max≈36min, and the abstract repeats the ~36min figure. Section 5.4 and Table 4 recommend τ*≈480s for OpenAI. With τ=480s, the formula gives I_max=480·(1/0.1 −1)=4320s=72min, more than double the stated band. The paying band is a central claim; the paper must adopt one τ* for OpenAI and recompute the abstract, Table 1, and §5.3 consistently.","section":"Table 1 vs. §5.4/Abstract"},{"comment":"For the Anthropic 1-hour tier, Table 1 gives w=2.00, r=0.10, τ*≈50min, and I_max≈3.3h. But the formula I_max=τ(w/r −1) yields 19τ = 19·50min ≈15.8h, not 3.3h. This is inconsistent with the formula displayed in the same table and with its use for the 5-minute tier. Please correct the arithmetic and adjust any discussion that relies on the paid tier's break-even horizon.","section":"Table 1 (Anthropic 1-hour tier)"}],"minor_comments":[{"comment":"The 'up to 12.5×' reduction on post-pause request cost is simply the list-price ratio w/r (1.25/0.1). It should be labeled as a price-ratio ceiling, not a measured end-to-end saving, to avoid confusion with the whole-strategy savings reported in Table 4.","section":"Abstract"},{"comment":"The retention curves are plotted as point estimates without confidence intervals. Given n=6 per cell and the paper's own 'lottery cache' characterization of Google, adding binomial CIs would materially strengthen the 'never converges' claim.","section":"Figure 2"},{"comment":"The n annotations differ between Table 3 (240s column n=4) and Table 4 (τ=240s n=8). Please clarify whether these are separate runs and how the per-run Fisher-exact bar was applied to the 1800s paying-band run, which appears to be reported without formal statistical tests.","section":"Tables 3 and 4"},{"comment":"The 'universal adoption is rational' equilibrium and the degradation argument are flagged in §7 as extrapolations, but the Discussion would benefit from an explicit statement that the retention curves were measured at single-client load and may not describe a keepalive-saturated tier.","section":"§6"},{"comment":"References [2] and [3] appear to point to the same Anthropic prompt-caching documentation under different URLs; please consolidate or distinguish them.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is borderline between major revision and reject. The measurement hygiene is genuinely good and the practical message is likely right qualitatively, but the OpenAI headline saving is supported by one small run under an unverified deterministic-hit assumption, and the DeepSeek data contradict the paper's own cost model. Both are fixable with additional per-ping measurements and a corrected stochastic cost model; the table arithmetic errors also need correction. I recommend major revision rather than rejection because the central methodology is sound and the remaining work is within the manuscript's stated scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: a solid, reproducible measurement study that turns a known trick (keepalives) into a per-provider policy, and the headline OpenAI saving is real but softer than the 2.45x number suggests.\n\nWhat's actually new: the technique isn't new — Aider shipped one in 2024, Anthropic recommends pre-warming, and practitioners had break-even arithmetic. Khailo adds the first measured cross-provider retention curves (Anthropic's hard 5-min cliff, DeepSeek's collapse by ~10min, OpenAI's soft decay to fully cold by ~30min, Google's lifetime lottery), a clean derivation that the economical interval is the provider's measured retention minus a margin, and a paying band bounded by τ(w/r−1). The protocol is careful: timing gates, separate baseline/keepalive time blocks, per-run Fisher exact tests with Bonferroni, and an unusually honest threats-to-validity section. The math is self-contained and correct under its stated assumptions.\n\nThe soft spots: the OpenAI τ*=480s and 2.45x saving is an upper bound, not a tested number. The paper's own retention sweep shows OpenAI is a 'soft, lossy cache': 81% warm at 600s and ~50% at 15–20min, so a 480s-gap ping has a non-trivial miss probability. The single 6/6-warm run in Table 4 can't rule out q≈0.2, and at q≈0.19 the saving is roughly break-even. They should have measured retention at 480s directly and reported per-ping misses. That's a measurement gap, not a fatal flaw — the core claims hold, but the OpenAI headline should be read carefully. The other limits are the ones the paper flags: DeepSeek is one reseller backend, Google's cells are quota-thin, and the paper's own per-run significance bar is met only by Anthropic and DeepSeek 100k. The whole mechanism also depends on cached reads refreshing TTL and recency; the paper cites provider docs but doesn't independently stress that under concurrent traffic. The §6 argument about all-adoption degrading the shared cache is argued, not measured, but it's clearly labeled as a prediction.\n\nVerdict: worth a serious referee. The measurement discipline and the economic derivation make this more than a blog post, even though the strongest empirical claims need more samples before being taken at face value. I'd cite it for the retention curves and the paying-band framing, and I'd bring it to a reading group for the threats-to-validity section alone.","headline":"Measured cross-provider keepalive economics with a genuinely useful policy — but the star OpenAI saving is an upper bound, not a tested number.","tokens_in":11067,"tokens_out":3472,"would_cite":true,"duration_ms":33151,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Client-side keepalives for LLM prompt caches are individually rational and pay only inside a per-provider band bounded by eviction time and a break-even horizon.","keywords":["prompt caching","cache keepalive","agentic workloads","LLM inference","cache eviction","cache TTL","cost optimization","token-hour pricing"],"falsifier":"Run the same 30-minute pause experiment with a 240s keepalive while many clients ping concurrently; if the follow-up request arrives cold (cached fraction below 90%) or the provider's effective retention drops so far that no interval under the documented lifetime holds warmth, the central claim is falsified.","tokens_in":10000,"feed_emoji":"⏱️","tokens_out":12214,"duration_ms":113536,"temperature":0.7,"pith_summary":"Agentic workloads repeatedly pause for tools or approval, long enough for LLM providers to evict the cached prompt prefix and charge full prefill price on the next request. The paper claims that a client-side keepalive—replaying the prefix on a timer—holds the cache warm through these gaps and is individually rational on every provider whose cold re-prefill is expensive enough to insure. The economics reduce to one identity: the keepalive saves only when the pause is longer than the provider's measured eviction point and shorter than Imax = τ(w/r − 1), roughly 46 minutes at Anthropic and 36 at OpenAI. It also shows that the common 30-second ping convention spends about 8x more than needed; the economical interval is the provider's own retention minus a margin. A reader should care because it converts a folklore trick into a measured, per-provider policy and warns that universal adoption will push providers to meter cache residency directly.","feed_headline":"Timed cache pings cut post-pause LLM costs by up to 12.5x","feed_subtitle":"Measured across four providers: savings exist only between a cache's eviction point and its break-even horizon.","key_machinery":"The load-bearing object is the keepalive: re-sending the exact prompt prefix on a timer so each cached read refreshes the entry's lifetime and recency in the eviction policy. The companion identity is the break-even condition Imax = τ(w/r − 1), where r is the cached-read price ratio, w is the re-prefill price ratio, and τ is the ping interval; it converts the provider's retention curve into a paying band with a hard ceiling. The same identity shows why intervals past the provider's retention turn toxic: each ping lands on a dead cache and pays the full re-prefill it was meant to prevent.","core_discovery":"On its own terms, the paper establishes that keepalives work everywhere measured: a 30s ping kept ≥98% of prompt tokens cached across four providers through gaps that evicted idle baselines, cutting the post-pause request cost up to 12.5x. The strategic claim is sharper: keepalive cost falls as the interval grows, so the optimum is the largest interval safely under the provider's own retention (about 240s at Anthropic and DeepSeek, 480s at OpenAI), not the 30s convention. The saving exists only inside a paying band bounded by the provider's eviction point and Imax ≈ τ(w/r − 1), about 6–46 minutes for Anthropic and 25–36 for OpenAI; measured savings at a 30-minute pause are 1.56x and 2.45x. D","pith_inferences":["This suggests the same keepalive logic applies beyond tool pauses—to any scheduled idle inside the band, including human-approval waits and overnight batch gaps, provided the user accepts the per-read rent.","A testable extension: under fleet-wide keepalive load, effective retention curves should shorten; operators should re-measure their provider's eviction point before fixing the interval, otherwise the band may close.","If providers adopt per-token-hour residency pricing, the break-even formula would gain a residency-cost term, shifting the optimal interval and making speculative warmth unprofitable; that suggests operators should prefer the bounded keepalive policy even before pricing changes.","The per-provider interval policy suggests multi-provider agent frameworks should treat cache keepalive as a routing parameter, choosing ping intervals and providers based on measured retention rather than a single global default."],"forward_implications":["The 30-second keepalive convention is roughly an 8x overspend; the right interval is the provider's measured retention minus a margin, about 4 minutes at Anthropic and 8 at OpenAI.","Keepalives pay only inside the band: below the eviction point they are waste, past Imax they exceed the re-prefill, and past the provider's retention each ping re-prefills a dead cache at full price.","At a 30-minute pause, measured marginal savings are 1.56x on Anthropic and 2.45x on OpenAI, with the post-pause request cost cut by up to 12.5x.","For DeepSeek and Google the cost band is empty: DeepSeek's re-prefill is too cheap to insure, and Google's cache never reliably evicts, so keepalives buy latency only.","Universal rational adoption will make the shared cache tier degrade toward first-in-first-out-of-luck, forcing providers to meter cache residency per token-hour; one provider already does."],"fun_headline_variants":["Keepalive pings trim LLM post-pause costs by 12.5x","Optimal cache ping interval? About 4 minutes, not 30s","Timed pings beat cache eviction, cut costs 12.5x","Forget 30s pings: 4-minute keepalives save more","Cache pings that cost less: 4-min beat 30-sec"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole argument rests on the assumption that reading a cached prefix refreshes its lifetime and recency, and that the retention curves measured at one client's load remain valid when every client runs keepalives; if either fails, the paying band and the measured savings collapse.","fun_headline_variants_meta":{"raw":{"variants":["Keepalive pings trim LLM post-pause costs by 12.5x","Optimal cache ping interval? About 4 minutes, not 30s","Timed pings beat cache eviction, cut costs 12.5x","Forget 30s pings: 4-minute keepalives save more","Cache pings that cost less: 4-min beat 30-sec"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000144,"raw_usage":{"total_tokens":1085,"prompt_tokens":891,"completion_tokens":194,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":88}},"tokens_in":635,"tokens_out":194,"duration_ms":2933,"temperature":1.0,"reasoning_tokens":88,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T13:02:54.555590+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 30-minute pause experiment with a 240s keepalive while many clients ping concurrently; if the follow-up request arrives cold (cached fraction below 90%) or the provider's effective retention drops so far that no interval under the documented lifetime holds warmth, the central claim is falsified.","supporting_citations":[],"review_version":1}