Pith. sign in

REVIEW 4 major objections 4 minor 4 cited by

Redbench: Workload Synthesis From Cloud Traces

T0 review · 4 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Redbench turns cloud traces into executable SQL workloads that preserve query repetitions, temporal burstiness, and read/write interleaving, and thereby reveals caching gains that standard benchmarks hide.

desk verdict Redbench is a real step forward in workload synthesis, but its fidelity check measures inputs it optimizes for, and the headline caching gains rest on anonymous systems plus an unvalidated schema mapping. read the letter →

arxiv 2511.13059 v2 pith:Y3KC2YG2 submitted 2025-11-17 cs.DB

Johannes Wehrstein , Roman Heinrich , Mihail Stoian , Skander Krid , Martin Stemmer , Andreas Kipf , Carsten Binnig , Muhammad El-Hindi This is my paper
classification cs.DB
keywords workloadsynthesisclouddatawarehousequeryrepetitionresultcachingtemporalpatternsread/writeinterleavingtrace-drivenbenchmarking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that standard analytical benchmarks systematically understate the value of workload-driven optimizations such as result caching, because they lack the query repetition, temporal burstiness, and read/write interleaving found in real cloud workloads. To fix this, it introduces Redbench, a workload generator that consumes an anonymized cloud trace (e.g., Redset) and a user-provided schema, and outputs executable SQL streams that preserve those trace properties. Two synthesis modes are offered: matching-based reuse of existing benchmark queries, and generation-based creation of new SQL guided by trace statistics such as bytes-read, scansets, and join shapes. On four commercial cloud systems the paper finds caching speedups of 1.10–2.98× under Redbench workloads versus at most 1.08× under a traditional benchmark baseline. If correct, Redbench provides a practical way to test caching and similar reuse optimizations without access to proprietary SQL or data.

What carries the argument

The load-bearing machinery is the trace-shape synthesis pipeline. From an anonymized trace it extracts per-query features—query hash, scanset, join count, bytes read, arrival timestamp—and then either (a) finds similar queries in a support benchmark pool and reorders them to match the trace's repetitions and timing (matching-based synthesis), or (b) builds new SQL from scratch: tables are mapped by size to the support schema, join paths are generated by random walks over the schema's join graph, and predicate selectivities are tuned so the generated queries hit the trace's bytes-read targets (generation-based synthesis). What carries the argument is the preservation of repetition types (exac

What would settle it

Take a workload whose original SQL text is available, strip it to only the anonymized fields Redbench uses (hash, scanset, bytes read, timestamps), synthesize a Redbench workload on a chosen support schema, and replay both the original and synthesized workloads on the same system with result caching enabled. If the caching speedups diverge by more than noise (e.g., >20%), the metadata-preservation premise fails; equivalently, if changing the support schema (star vs. wide) changes speedups dramatically, the size-based table mapping is the culprit.

Watch

Extended reading notes

Core claim

The central discovery is that the workload-relevant signal in a cloud trace—query repetitions (exact and structural), temporal dynamics, and read/write interleaving—can be transferred onto a different schema by synthesizing SQL that matches trace metadata, and that doing so changes system evaluation outcomes. Specifically, Redset's repetition rates (query repetition rate ≈0.72, DML-aware ≈0.73) are reproduced closely by generation-based synthesis (0.73/0.79) while a straightforward round-robin baseline yields 0% DML-aware repetition. When executed on four commercial cloud data warehouses, the baseline workloads show negligible result-caching speedups (≤1.08×), whereas Redbench workloads expo

Load-bearing premise

The load-bearing premise is that the anonymized metadata in a trace—query hash, scanset, bytes read, and timestamps—together with a size-based mapping from inferred trace tables to the user's support schema, is enough to synthesize SQL that preserves the caching-relevant behavior of the unknown original queries; the paper itself flags this mapping as speculative in Section 4.3.

Editorial extensions

If this is right

  • If Redbench is right, evaluations of result caching on TPC-style benchmarks have been systematically missing real gains: the same systems show ≤1.08× on traditional workloads but up to 2.98× on trace-shaped ones.
  • The two synthesis modes bracket reality: generation-based gives trace-faithful estimates of caching benefit; matching-based (higher repetition) gives an upper bound useful for stress-testing caches.
  • DML-aware repetition metrics (QRR_DML, SRR_DML) provide a way to quantify how interleaved writes limit reuse opportunities, making them a useful reporting standard for workload benchmarks.
  • Redbench workloads expose system-specific architectural trade-offs (e.g., write-optimized vs read-optimized caching configurations) that single-workload or single-system evaluations hide.
  • The same trace-shaping approach can be applied to other workload-driven optimizations—adaptive clustering, materialization, plan caching, incremental view maintenance—where temporal locality and repetition matter.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • It is an open question whether the caching benefits Redbench reveals are typical of production workloads or of this particular Redset cluster; the paper selects one representative cluster, and the method's generality across trace types is untested.
  • The size-based table mapping from inferred trace tables to the support schema is the least constrained step; a natural test is to vary the support database (e.g., a small vs large schema) and measure how caching speedups change, which would show how sensitive conclusions are to schema choice.
  • Because the generated SQL is validated only to avoid empty results and match byte counts, not to match the original queries' semantics, Redbench workloads cannot be assumed to preserve query-planner behavior (join order, operator selection); comparisons against workloads where original SQL is known would clarify this.
  • By construction, matching-based synthesis overstates repetition (QRR≈0.98 vs 0.72), so if used carelessly it could mislead practitioners into over-investing in caching; the generation-based mode should be treated as the more production-representative choice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. Redbench is a benchmark and workload generator that takes an anonymized cloud workload trace (Redset) and a user-supplied support database (e.g., IMDb), and produces a replayable SQL workload. It supports two synthesis modes: matching-based, which maps trace entries to existing benchmark queries and reorders them; and generation-based, which creates new SQL from table statistics and trace-derived features such as scansets, join counts, bytes-read, and repetitions. The paper evaluates the generated workloads on four commercial cloud data warehouse systems (Systems A–D), reporting that Redbench reproduces repetition and temporal patterns of the original trace, and that it exposes substantial result-caching speedups (generation-based 1.10–2.00×, matching-based 2.12–2.98×) that a round-robin baseline derived from JOB and CEB fails to reveal (≤1.08×). The authors argue that Redbench bridges synthetic and real workloads and provides a foundation for evaluating workload-driven optimizations.

Significance. If the claims hold, Redbench would be a practically valuable contribution: it turns otherwise unusable anonymized traces into executable, trace-shaped workloads, and it provides evidence that traditional benchmarks substantially understate the benefit of workload-driven optimizations such as result caching. The paper's strengths include the use of a real published trace (Redset), a public artifact, two complementary synthesis strategies with different fidelity/complexity trade-offs, and an evaluation on four commercial systems that reveals qualitatively different caching behaviors. However, the validity of the central fidelity claims is weakened by several issues: the fidelity metrics partly measure properties that the generator optimizes, the schema-mapping step is explicitly flagged as speculative and appears to inflate scanset repetition, and Table 4 contains internal inconsistencies. These issues are fixable, but they currently prevent the paper from fully supporting its strongest conclusions.

major comments (4)
  1. [§4.3(4), §5.3 (Table 4)] The generation-based synthesis maps inferred Redset tables to the user schema by size, and the paper itself states in §4.3(4) that 'This approach is speculative.' This mapping collapses multiple distinct Redset tables into one IMDb table, mechanically inflating scanset-level repetition and widening DML invalidation. Table 4 shows exactly this: generation-based SRR is 0.95 vs. Redset's 0.86, and DML-aware SRR is 0.98 vs. 0.87. Since §5.4 attributes the headline caching speedups to 'production-like' repetition and read/write interleaving, the unvalidated mapping could make the synthesized stream more reuse-friendly than the actual trace. The paper needs either a validation of the mapping (e.g., robustness across different support schemas, a sensitivity analysis, or a comparison on a schema with similar table count) or substantially tempered claims.
  2. [§5.3, Table 4] The repetition metrics are internally inconsistent. By the definitions in §3, DML-aware repetition rates must be at most the non-DML-aware rates, because DML-aware counts only repetitions not invalidated by intervening writes. Yet Table 4 reports DML-aware QRR/SRR greater than or equal to the non-DML-aware values in almost every row: generation QRR 0.73 vs. DML QRR 0.79; matching QRR 0.97 vs. DML QRR 0.98; baseline QRR 0.00 vs. DML QRR 0.74; Redset SRR 0.86 vs. DML SRR 0.87. The baseline's DML QRR of 0.74 also contradicts the text in §5.3, which states the baseline 'achieving 0% DML-aware QRR.' This makes the central fidelity table uninterpretable and must be corrected with consistent definitions and recomputed values.
  3. [§4.1–§4.3 vs. §5.3] Part of the fidelity evaluation is circular. The generator uses arrival times, query repetitions (hash/scanset), and bytes-read as synthesis inputs, and §5.3 then reports closeness on those same metrics (QRR, SRR, bytes-read, temporal patterns) as evidence of realism. For example, §4.3(2) explicitly uses bytes-read per scanset to steer predicate selectivities, and Figure 6 then shows a close bytes-read CDF as a fidelity result. The non-circular evidence is limited: the runtime distribution in Figure 6 is more indirect, but Figure 7 compares Redset trace runtimes (from Redshift) to Redbench runtimes on Systems A–D, which are different systems, so this is not a clean validation either. The authors should add hold-out metrics that are not optimized by the generator (e.g., operator mix, intermediate result cardinalities, cache hit/miss patterns) or otherwise demonstrate that the synthesis do
  4. [§5.2, §5.4, Figures 8–9] The experimental claims lack reproducibility details that are essential for a benchmark paper. The four systems are anonymous, no error bars or reruns are reported, raw timings are not given, and the execution uses 'compressed idle gaps' (§5.2), which conflicts with the §5.3 claim that Redbench 'preserves these patterns by retaining the exact arrival times.' With gaps compressed, cache eviction behavior changes, so the temporal-dynamics claims are not directly supported by the experiment. At minimum, the paper should report variance across runs, describe the exact execution protocol, and clarify how arrival times were used after gap compression.
minor comments (4)
  1. [§5.2] Typo: 'in our valuation' should be 'in our evaluation.'
  2. [Figure 8] The SELECT-only speedup values (e.g., 22.02×, 50.09×) are not discussed in the text and appear far outside the ranges quoted in §5.4. Please clarify whether these are per-query-type speedups and why they are so much larger than the SELECT+DML numbers.
  3. [Figure 4] The x-axis labels '136/0', '6/0', etc. are cryptic; please explain the cluster/database naming convention in the caption or text.
  4. [§2.2] The related-work discussion is thorough, but Table 1 would benefit from a footnote explaining what the '(✓)' for PBench means; currently it is ambiguous.

Circularity Check

4 steps flagged · score 6.0 of 10

Fidelity validation is circular: Section 5.3 reports matches on QRR, SRR, arrival times, and bytes-read that Section 4 uses as synthesis inputs; the caching experiments are real measurements but inherit the unsupported realism claim.

  1. fitted input called prediction [§4.1/§4.2/§5.3, Table 4]
    "Redbench then analyzes the workload trace to infer features like query frequencies, complexities, scansets, repetitions, and temporal locality. ... If two SELECT queries from the workload trace share the same hash and constitute full query repetitions, they will be mapped to the same benchmark query instance generated previously. ... Table 4 shows that Redbench's generation-based mode closely matches Redset's repetition characteristics (QRRDML: 0.73 vs. 0.72)."

    QRR and SRR are not independent outcome variables: the trace-processing step extracts repetitions and scansets, and the synthesis step re-creates them (same hash to same query instance, same scanset to same template). Reporting 'closely matches' on Table 4 confirms the generator followed its inputs, not that the synthesized stream approximates unknown production properties. This is the fitted-input-called-prediction pattern.

  2. fitted input called prediction [§5.3 'Temporal patterns and query type distributions', Figure 4]
    "Redbench preserves these patterns by retaining the exact arrival times from the Redset trace. ... Figure 4 demonstrates that Redbench faithfully reproduces the bursty activity patterns observed in Redset, with distinct periods of high query load followed by idle intervals."

    Arrival timestamps are taken verbatim from Redset as synthesis input, so burst/idle structure in Figure 4 is copied, not discovered. The fidelity claim on temporal dynamics is a tautology: the generator is given the exact timestamps and then is credited with preserving them.

2 more flagged steps
  1. fitted input called prediction [§5.3 'System load characteristics', Figure 6]
    "The generation-based approach achieves close alignment with Redset's load distribution by explicitly steering filter selectivity to match target bytes-read values."

    Bytes-read is a steering target of the generation-based mode: selectivities are tuned so that generated queries hit target bytes-read extracted from Redset. Matching the bytes-read CDF in Figure 6 is therefore a consistency check on the optimizer, not independent validation that the synthesized queries have realistic semantics or structure.

  2. other [§4.3(4), §5.3, Table 4]
    "This approach is speculative since the schemas of the redset database can only be inferred (by aggregating scansets of the redset queries) and depend on the schema of the support database provided to Redbench. ... the IMDb support schema contains fewer tables than the original Redset database, forcing multiple Redset tables to be mapped onto a single IMDb table."

    The table mapping is inferred from the same scansets whose repetition rate (SRR) is later reported as fidelity. Because the size-based collapse merges distinct Redset tables into one IMDb table, the generation-based SRR of 0.95 (vs Redset's 0.86) is partly an artifact of the mapping. The 'closely matches' claim on scanset repetition therefore does not independently validate the synthesized queries.

full rationale

The paper's central realism claim is supported by metrics that the synthesis procedure explicitly optimizes. Section 4.1 extracts repetitions, scansets, temporal locality, and bytes-read from Redset; §4.2 maps same-hash queries to identical query instances; §4.3 tunes selectivities to hit target bytes-read and preserves arrival times. Section 5.3 then reports closeness on QRR, SRR, temporal burst plots, and bytes-read CDFs as evidence of faithfulness. These are consistency checks on the generator, not independent confirmation that the mapping to a different schema preserves workload semantics. I separately flag the paper's own caveat in §4.3(4) that the Redset schema is only inferred and the mapping is 'speculative,' together with the §5.3 admission that IMDb has fewer tables, forcing multiple Redset tables onto one IMDb table; this inflates generation-based SRR and further weakens the SRR fidelity evidence. The caching speedup experiments in §5.4 are genuine system measurements and are not circular: injecting repetition and DML interleaving is the stated purpose, and the round-robin baseline provides a contrast. However, the characterization of generation-based speedups as 'trace-faithful' and 'realistic' inherits the circular fidelity validation, so the paper's headline claim that Redbench produces realistic workloads is only partially supported. No load-bearing self-citation was found; reference [6] describes the authors' previous Redbench-v1 and is not used to justify the current results.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The ledger shows that the paper's trace-fidelity claims rest on the sufficiency of Redset metadata and on speculative schema mapping. The headline caching evaluation additionally depends on the round-robin representation of classic benchmarks and on heuristic DML row sizes. These are domain assumptions rather than parameters fitted to a physical theory, but they carry the load of the benchmark's realism claim.

free parameters (4)
  • Predicate selectivities = Tuned per query to match Redset bytes-read
    Section 4.3(5): 'Selectivities are then tuned to match the target bytes-read per query.' Figure 6 then reports bytes-read/runtime CDF closeness as fidelity evidence.
  • Per-table maximum scan-size estimates = Derived from largest observed scan size per inferred Redset table
    Section 4.3(2): aggregated bytes-read yield 'an estimated maximum scan size for each table'; these drive table mapping and selectivity targets.
  • Join-count normalization mapping = Linear map from min/max Redset joins to min/max support-benchmark joins
    Section 4.2: 'the minimum and maximum number of joins in the Redset workload are linearly mapped to the minimum and maximum number of joins in the support benchmark.'
  • DML row-update count = Randomly selected 'reasonable' number of rows
    Section 4.3 DML generation: Redset lacks row-level write counts, so update/delete sizes are chosen heuristically to stabilize data distributions.
assumptions (5)
  • domain assumption Redset's anonymized metadata (query hash, scanset, bytes-read, timestamps) is a sufficient representation of production workload semantics for benchmarking.
    Section 3: Redbench infers repetitions from hash+scanset and steers synthesis from these statistics. The realism of every generated workload rests on this proxy.
  • ad hoc to paper The inferred schema and size-based table mapping from Redset to the user's support database preserve workload characteristics.
    Section 4.3(3)–(4): tables are mapped by scan size and join paths are random walks over an inferred join graph. The paper itself calls this 'speculative.'
  • domain assumption DML invalidation of caches is adequately modeled by table-level read/write dependencies, without row-count granularity.
    Section 4.3 DML: since Redset does not contain write row counts, writes invalidate caches at table granularity and modify 'reasonable' random row subsets.
  • domain assumption IMDb and JOB/CEB queries are representative enough to stand in for the unknown Redset data and SQL.
    Section 5.2: evaluation uses IMDb; schema mismatch forces multiple Redset tables onto single IMDb tables, yet fidelity is claimed despite the mismatch.
  • domain assumption A round-robin ordering of JOB/CEB queries with uniformly spaced DMLs adequately represents traditional benchmarks for caching evaluation.
    Section 5.2 baseline construction: the baseline deliberately removes repetitions and bursts, which is the basis for the claim that traditional benchmarks show no caching speedup.
invented entities (1)
  • Staging tables
    purpose: Copies of base tables with PK/FK constraints carried forward and unique suffix values on non-key columns, enabling INSERT/SELECT queries that satisfy constraints and maintain data distributions.
    Introduced in Section 4.3 for DML generation. It is an implementation device, not a physical claim, and is not externally validated against Redset behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Redbench: Workload Synthesis From Cloud Traces." pith.science (2026). https://pith.science/paper/Y3KC2YG2

@misc{pith2026251113059,
  author       = {Pith},
  title        = {Pith review of: Redbench: Workload Synthesis From Cloud Traces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y3KC2YG2}},
  note         = {Machine review of arXiv:2511.13059}
}
read the original abstract

Workload traces from cloud data warehouse providers reveal that standard benchmarks such as TPC-H and TPC-DS fail to capture key characteristics of real-world workloads, including query repetition and string-heavy queries. In this paper, we introduce Redbench, a novel benchmark featuring a workload generator that reproduces real-world workload characteristics derived from traces released by cloud providers. Redbench integrates multiple workload generation techniques to tailor workloads to specific objectives, transforming existing benchmarks into realistic query streams that preserve intrinsic workload characteristics. By focusing on inherent workload signals rather than execution-specific metrics, Redbench bridges the gap between synthetic and real workloads. Our evaluation shows that (1) Redbench produces more realistic and reproducible workloads for cloud data warehouse benchmarking, and (2) Redbench reveals the impact of system optimizations across four commercial data warehouse platforms. We believe that Redbench provides a crucial foundation for advancing research on optimization techniques for modern cloud data warehouses.

Figures

Figures reproduced from arXiv: 2511.13059 by the authors.

Figure 1
Figure 1. Result caching speedups for traditional benchmarks [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Redbench uses a given workload trace and data [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Redbench transforms real workload traces [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Redbench faithfully preserves the temporal arrival patterns from Redset, including distinct bursts of activity followed [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Comparison of query repetition rate (QRR) and scan [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: CDF of runtime and bytes-read (log-scale) for Red [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Overall caching speedups across systems for Red [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Runtime under different caching configurations for generation-based, matching-based, and baseline workloads. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Bi-Channel Networking Paradigm for Database Systems in the Cloud

    cs.DB 2026-06 unverdicted novelty 7.0 of 10

    The bi-channel paradigm separates database networking into a high-performance UDP data path and a TCP control path to reduce kernel overhead while preserving reliability on fast cloud networks.

  2. IDSTune: A Multi-Agent Collaborative Framework for Integrated Database System Tuning

    cs.DB 2026-07 conditional novelty 6.0 of 10

    IDSTune jointly tunes knobs, indexes, and materialized views via coordinated LLM agents, reporting up to 38% latency gains and 57% faster tuning on benchmarks.

  3. Hollywood: Towards a Large Movie Dataset for Database Benchmarking

    cs.DB 2026-07 conditional novelty 6.0 of 10

    Hollywood is a synthetic IMDb-compatible benchmark generator whose 200K-movie instance induces cardinality-estimation errors comparable to or exceeding the original JOB/IMDb workload, enabling generalization testing b...

  4. AutoSLO: Practical Latency SLOs on Cloud Data Warehouses -- Extended Version

    cs.DB 2026-07 conditional novelty 6.0 of 10

    AutoSLO meets query latency SLOs on multi-cluster cloud warehouses while cutting cost ~26% versus the next-best baseline by combining forecast-driven planning, reactive scaling, and concurrency-aware routing.

Reference graph

Works this paper leans on

24 extracted references · 2 canonical work pages · cited by 4 Pith papers

  1. [1]

    Dana Van Aken, Djellel Eddine Difallah, Andrew Pavlo, Carlo Curino, and Philippe Cudré-Mauroux. 2015. BenchPress: Dynamic Workload Control in the OLTP-Bench Testbed. InProceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, Victoria, Australia, May 31 - June 4, 2015, Timos K. Sellis, Susan B. Davidson, and Zachary G. ...

    arXiv 2015

  2. [2]

    Leilani Battle, Philipp Eichmann, Marco Angelini, Tiziana Catarci, Giuseppe Santucci, Yukun Zheng, Carsten Binnig, Jean-Daniel Fekete, and Dominik Moritz

  3. [3]

    TPC Benchmark. 2021. TPC BENCHMARK TM DS Standard Specification Version 3.2.0. https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-DS_v3. 2.0.pdf

    2021

  4. [4]

    TPC Benchmark. 2021. TPC BENCHMARK TM H Standard Specification Version 3.0.1. https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3. 0.1.pdf

    2021

  5. [5]

    Narasayya

    Bailu Ding, Surajit Chaudhuri, Johannes Gehrke, and Vivek R. Narasayya. 2021. DSB: A Decision Support Benchmark for Workload-Driven and Traditional Database Systems.Proc. VLDB Endow.14, 13 (2021), 3376–3388. doi:10.14778/ 3484224.3484234

    arXiv 2021

  6. [6]

    Skander Krid, Mihail Stoian, and Andreas Kipf. 2025. Redbench: A Benchmark Reflecting Real Workloads.CoRRabs/2506.12488 (2025). arXiv:2506.12488 doi:10. 48550/ARXIV.2506.12488

    arXiv doi:10.48550/arxiv.2506.12488 2025

  7. [7]

    Jiale Lao and Immanuel Trummer. 2025. Demonstrating SQLBarber: Leveraging Large Language Models to Generate Customized and Realistic SQL Workloads. InCompanion of the 2025 International Conference on Management of Data, SIG- MOD/PODS 2025, Berlin, Germany, June 22-27, 2025, Volker Markl, Joseph M. Hellerstein, and Azza Abouzied (Eds.). ACM, 151–154. doi:1...

    work page doi:10.1145/3722212 2025

  8. [8]

    Jiale Lao and Immanuel Trummer. 2025. SQLBarber: A System Leveraging Large Language Models to Generate Customized and Realistic SQL Workloads.CoRR abs/2507.06192 (2025). arXiv:2507.06192 doi:10.48550/ARXIV.2507.06192

    work page doi:10.48550/arxiv.2507.06192 2025

Show all 24 references
  1. [9]

    Viktor Leis, Andrey Gubichev, Atanas Mirchev, Peter Boncz, Alfons Kemper, and Thomas Neumann. 2015. How Good Are Query Optimizers, Really?Proc. VLDB Endow.9, 3 (2015), 204–215. doi:10.14778/2850583.2850594

    2015

  2. [10]

    Guanli Liu and Renata Borovica-Gajic. 2025. DriftBench: Defining and Generating Data and Query Workload Drift for Benchmarking. arXiv:2510.10858 [cs.DB] https://arxiv.org/abs/2510.10858

    2025

  3. [11]

    Parimarjan Negi, Ryan Marcus, Andreas Kipf, Hongzi Mao, Nesime Tatbul, Tim Kraska, and Mohammad Alizadeh. 2021. Flow-Loss: Learning Cardi- nality Estimates That Matter.Proc. VLDB Endow.14, 11 (2021), 2019–2032. doi:10.14778/3476249.3476259

    2021

  4. [12]

    Tobias Schmidt, Viktor Leis, Peter Boncz, and Thomas Neumann. 2025. SQLStorm: Taking Database Benchmarking into the LLM Era.Proc. VLDB Endow.18, 11 (2025), 4144–4157. https://www.vldb.org/pvldb/vol18/p4144-schmidt.pdf

    2025

  5. [13]

    Jan Vincent Szlang, Sebastian Breß, Sebastian Cattes, Jonathan Dees, Florian Funke, Max Heimel, Michel Oleynik, Ismail Oukid, and Tobias Maltenberger

  6. [14]

    Alexander van Renen, Dominik Horn, Pascal Pfeil, Kapil Vaidya, Wenjian Dong, Murali Narayanaswamy, Zhengchun Liu, Gaurav Saxena, Andreas Kipf, and Tim Kraska. 2024. Why TPC Is Not Enough: An Analysis of the Amazon Redshift Fleet.Proc. VLDB Endow.17, 11 (2024), 3694–3706. doi:1...

    2024

  7. [15]

    Alexander van Renen and Viktor Leis. 2023. Cloud Analytics Benchmark.Proc. VLDB Endow.16, 6 (2023), 1413–1425. doi:10.14778/3583140.3583156

    2023

  8. [16]

    Adrian Vogelsgesang, Michael Haubenschild, Jan Finis, Alfons Kemper, Viktor Leis, Tobias Mühlbauer, Thomas Neumann, and Manuel Then. 2018. Get Real: How Benchmarks Fail to Represent the Real World. InProceedings of the 7th International Workshop on Testing Database Systems, DB...

    2018

  9. [17]

    Midhul Vuppalapati, Justin Miron, Rachit Agarwal, Dan Truong, Ashish Moti- vala, and Thierry Cruanes. 2020. Building An Elastic Query Engine on Dis- aggregated Storage. In17th USENIX Symposium on Networked Systems De- sign and Implementation, NSDI 2020, Santa Clara, CA, USA, F...

    2020

  10. [18]

    Ciortea, Konstantinos Karanasos, and Subru Krishnan

    Chengcheng Wan, Yiwen Zhu, Joyce Cahoon, Wenjing Wang, Katherine Lin, Sean Liu, Raymond Truong, Neetu Singh, Alexandra M. Ciortea, Konstantinos Karanasos, and Subru Krishnan. 2023. Stitcher: Learned Workload Synthesis from Historical Performance Footprints. InProceedings 26th ...

    2023

  11. [19]

    Johannes Wehrstein, Timo Eckmann, Roman Heinrich, and Carsten Binnig

  12. [20]

    Jiashu Zhang, Wen Jiang, Bo Tang, Haoxiang Ma, Lixun Cao, Zhongbin Jiang, Yuanyuan Nie, Fan Wang, Lei Zhang, and Yuming Liang. 2023. CDSBen: Benchmarking the Performance of Storage Services in Cloud-native Data- base System at ByteDance.Proc. VLDB Endow.16, 12 (2023), 3584–359...

    2023

  13. [21]

    Yan Zhou, Chunwei Liu, Bhuvan Urgaonkar, Zhengle Wang, Magnus Mueller, Chao Zhang, Songyue Zhang, Pascal Pfeil, Dominik Horn, Zhengchun Liu, Davide Pagano, Tim Kraska, Samuel Madden, and Ju Fan. 2025. PBench: Workload Synthesizer with Real Statistics for Cloud Analytics Benchm...

    2025

  14. [22]

    arXiv:2507.07471 doi:10.48550/ARXIV

    JOB-Complex: A Challenging Benchmark for Traditional & Learned Query Optimization.CoRRabs/2507.07471 (2025). arXiv:2507.07471 doi:10.48550/ARXIV. 2507.07471

    2025 doi arXiv

  15. [2020]

    Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data. InProceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14-19, 2020, David Maier, Rachel Pottinger, AnHai Do...

    2020

  16. [2025]

    VLDB Endow.18, 12 (2025), 5126–5138

    Workload Insights From the Snowflake Data Cloud: What Do Production Analytic Queries Really Look Like?Proc. VLDB Endow.18, 12 (2025), 5126–5138. https://www.vldb.org/pvldb/vol18/p5126-bress.pdf

    2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.