REVIEW 5 major objections 6 minor 20 references
A Unified Architecture for Efficient Binary and Worst-Case Optimal Join Processing
T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A single compilation pipeline unifies binary and worst-case optimal joins, beating Free Join by up to 4.8x.
desk verdict A solid engineering integration with an unverifiable baseline; the architecture and ablation are worth refereeing, but the headline speedups are provisional. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is SDQL, a statically typed intermediate language for functional collection programming with semi-ring dictionaries, extended here with annotated dictionary types that select physical representations. A DuckDB-optimized binary plan is converted into a Free Join plan; the plan is then compiled into SDQL programs that build hash tries or sorted dictionaries and execute nested iterations, and finally translated into C++. The performance case rests on dictionary specialization (vector and stack-allocated SmallVector leaves), early projection and aggregation (dead code elimination, eliminating redundant offsets, loop-invariant code motion), and a Range representation that stores only the first and last offsets of consecutive sorted runs.
What would settle it
Run the upstream Free Join implementation on LSQB Q1 with its factorization optimization enabled and record the runtime; if it reproduces close to the authors' system numbers, the 80x speedup evaporates. Similarly, complete LSQB Q3 and re-run JOB with vectorization enabled in Free Join; the geometric-mean speedups should drop if the baseline was handicapped.
Extended reading notes
Core claim
The paper's central claim is that a Free Join plan, which already generalizes both binary join plans and Generic Join plans, can be executed through a single compiled pipeline that is faster than the reference Free Join implementation on standard benchmarks. The route is: take a DuckDB binary join plan, convert it to a Free Join plan, generate an SDQL program with tries, apply dictionary specialization and early projection/aggregation optimizations, and compile to C++ with either hash-based or sort-based physical data structures. Against the Free Join framework, the authors measure geometric-mean speedups of 1.49x for Generic Join and 1.42x for Free Join on JOB, and larger gains on LSQB, including up to 27.5x (Generic Join) and 80x (Free Join) on query Q1, whose baseline results they could not reproduce. The authors also show a hybrid mode that uses sorted dictionaries for pre-sorted base relations and hash tables for intermediate results, removing the need to sort intermediates.
Load-bearing premise
The reported speedups assume the Free Join framework baseline was configured fairly; the paper could not reproduce Free Join's LSQB Q1 and Q3 results, and it compares against a non-vectorized Free Join for part of the JOB results, so a misconfigured baseline would inflate the gains.
Editorial extensions
If this is right
- Single-pipeline engines can expose both binary and worst-case optimal execution without maintaining two separate join subsystems.
- Sort-based WCOJ becomes practical when inputs are pre-sorted or sortable; hybrid hash/sort execution removes the need to sort intermediate results.
- The same Free Join plans used by the state of the art can be compiled through a functional IR and still outperform the reference implementation, suggesting the IR is not a performance tax.
- Early projection and aggregation is decisive on queries whose output is much larger than input, such as LSQB Q1; engines that delay aggregation miss large constant-factor gains.
- Because the system does not yet support lazy trie construction or vectorization, the measured speedups are a lower bound on what the architecture could deliver with those features added.
Reading between the lines
- We infer that the same SDQL pipeline could be retargeted beyond C++ to parallel or GPU runtimes, since trie construction and query execution are separable phases with read-only intermediates.
- We infer the hybrid hash/sort decision invites a purpose-built optimizer cost model: pre-sorted base relations favor sorted dictionaries with Range compression, while intermediate results favor hash tables.
- We infer the LSQB Q1 and Q3 reproducibility gap means the headline 80x result should be treated as provisional until re-run on upstream Free Join with vectorization and factorization enabled.
- We infer a natural testable extension is to apply the same IR to graph pattern matching workloads, since WCOJ algorithms are widely used there and the system's cyclic-query support is demonstrated on LSQB Q2.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a unified compilation architecture for binary and worst-case optimal joins. Queries are first converted from an optimized binary join plan (from DuckDB) into a Free Join plan, then translated into an SDQL intermediate representation, optimized by a series of transformations (dictionary specialization with vectors and SmallVectors, early projection, dead-code elimination, redundant-offset elimination, loop-invariant code motion), and finally compiled to C++. The system supports both hash-based and sort-based WCOJ implementations, plus a hybrid mode in which base relations use sorted dictionaries and intermediate results use hash tables. The experiments compare the system against the Free Join framework on JOB and LSQB, reporting geometric-mean speedups of about 1.49x/1.42x on JOB for Generic Join/Free Join and much larger, more variable speedups on LSQB (including an 80.05x outlier for Q1), while noting that LSQB Q3 was excluded because the Free Join baseline could not be reproduced.
Significance. If the experimental claims are correct, the paper makes a useful systems contribution: it demonstrates that a semi-ring-dictionary intermediate representation can express both traditional binary joins and WCOJ algorithms, can be compiled to efficient C++ code, and can accommodate both hash-based and sort-based paradigms within a single pipeline. The ablation study (O1-O5) provides concrete evidence about which optimizations matter. The claimed main benefit is architectural unification plus a set of concrete implementation optimizations, rather than a new algorithmic asymptotic improvement. The evaluation, however, is currently the weakest link, and several load-bearing experimental decisions would need to be repaired before the headline speedups can be considered established.
major comments (5)
- [§5.1, §5.2.2] The LSQB comparison is not reproducible on the baseline side. The paper explicitly states that LSQB Q3 was excluded because the Free Join framework's results could not be reproduced, and that for Q1 the Free Join results could not be reproduced either, yet Q1 contributes the largest reported outlier (80.05x for Free Join) and the large average speedups for Q1 (23.13x). When the baseline for the most dramatic speedup cannot be independently verified, the LSQB claims are unsupported as stated. The paper should provide a working baseline configuration, raw per-query timings, or a reproducible artifact, and should temper or remove the unverifiable Q1 speedup claims.
- [§5.4 vs. §5.2.1, Figure 14c] There is an internal inconsistency about which system variant produced the JOB Free Join numbers. Section 5.2.1 describes Figure 14c as comparing 'our system's Free Join implementation' with the Free Join framework with vectorization, while Section 5.4 says 'Using this hybrid approach, we achieve superior performance ... as shown in Figure 14c.' If Figure 14c and its 1.42x average / 4.78x maximum speedups refer to the hybrid approach, then the paper's headline 'Free Join' comparison is actually a hybrid comparison; if they refer to the non-hybrid Free Join implementation, then Section 5.4's attribution is wrong. The manuscript must clarify, with consistent figure labels and text, which configuration produced the headline numbers.
- [§5.2.1, §6] The conclusion that the system 'consistently outperforms or matches state-of-the-art solutions' is contradicted by the paper's own reported minimum speedups on JOB: 0.71x (40% slowdown) for Generic Join and 0.30x (3.33x slowdown) for Free Join. A system that is over three times slower on some benchmark queries cannot be described as consistently matching or outperforming the baseline. The conclusion should be revised to state the range of observed performance, including the substantial regressions.
- [§4.3, §5.4] The sort-based and hybrid experiments assume that input relations are 'always provided in sorted order,' and the reported sort-based speedups (up to 6.25x) do not appear to include the cost of sorting the base relations. For a benchmark like JOB, whose base tables are not typically sorted by the relevant join keys, this assumption excludes a potentially dominant cost. Please state explicitly whether the reported timings include all sorting work; if they do not, the sort-based and hybrid claims are not end-to-end comparisons and should be presented as such.
- [§5 (entire evaluation)] The paper provides no code, no artifact, no raw per-query timings, and no build/configuration details for the Free Join baseline. Given that the central claim is empirical and the paper itself reports failure to reproduce the baseline for two LSQB queries, the absence of a reproducibility package makes it impossible for a reader to check the numbers. A systems paper making state-of-the-art performance claims should supply an artifact or, at minimum, a detailed reproducibility appendix with exact versions, flags, and measured timings.
minor comments (6)
- [Global] There are several typos and grammar issues: 'apple-to-apple' should be 'apples-to-apples'; 'as can be realized' should be 'as can be seen'; and 'In Figure 18a, illustrates' is ungrammatical and should read 'Figure 18a illustrates'.
- [Figure 1b] In the generated C++ code for the binary-join example, line 22 assigns to 'Tx' from 'S_ht.at(x)' but should use 'T_ht.at(x)'.
- [Abstract and §5.2.1] The abstract's 'on average 1.5x and 1.4x' speedups are geometric means over JOB only, not 'across ... standard query benchmarks' as the abstract implies. Please state explicitly that these aggregate numbers are JOB-only and report LSQB aggregates separately.
- [§5.3] The sentence 'O5 ... is 6.5% faster than O2' is imprecise; the ratio 1.124/1.056 is approximately 1.064, so the text should say '6.4% higher speedup than O2' or give the exact ratio.
- [§4.1.2, §5.3] The SmallVector inline capacity N is a tunable parameter (shown as N=4 in the examples). The experiments should report the chosen value and, ideally, a brief sensitivity check, since this is the one free parameter in the implementation.
- [§5.2.1] The statement 'the majority of data points appear below the diagonal' is vague; please report the number of queries faster and slower than the baseline for each figure panel.
Circularity Check
No significant circularity: the paper is an implementation-and-benchmark systems contribution; no derived claim reduces to its own inputs, and the acknowledged baseline issues are correctness risks, not circularity.
full rationale
The paper's central claim is an empirical performance comparison: its SDQL-based pipeline achieves speedups over the Free Join framework's Generic Join and Free Join implementations on JOB and LSQB. This is a measurement claim against an external baseline, not a derivation from fitted parameters or from the paper's own definitions. The paper explicitly adopts the competitor's plans for comparability ('For an apple-to-apple comparison, we use the same query plans as the Free Join framework'), which is a standard experimental control rather than a circular construction; the speedups are not definitionally forced by that choice. The only self-citations are to prior work by the authors: SDQL [18], used as the intermediate representation, and hinted dictionaries [17], mentioned only as future work. These are building blocks or suggested extensions, not uniqueness theorems or ansatzes smuggled in to forbid alternatives, and the experimental results stand independently of whether those citations are accepted. The paper's own acknowledgments of non-reproducible Free Join results for LSQB Q1 and Q3, and its use of a non-vectorized Free Join baseline in one JOB comparison, are threats to the validity of the reported speedups and should be treated as correctness risks, but they do not make the paper's argument circular. No fitted parameter is renamed as a prediction, no result is equivalent by construction to its input, and no load-bearing step reduces to a self-citation chain. Therefore, the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (1)
- SmallVector inline capacity N =
4
assumptions (3)
- domain assumption Sort-based WCOJ assumes input relations are always provided in sorted order on the join attributes.
- domain assumption Selections are pushed down and projections and aggregations are deferred until after the full join.
- domain assumption For all JOB queries, each relation involves at most one attribute lookup before iteration, making eager trie construction behave like lazy evaluation.
Cite this review
Pith. "Pith review of A Unified Architecture for Efficient Binary and Worst-Case Optimal Join Processing." pith.science (2026). https://pith.science/paper/ZRRH7ADA
@misc{pith2026250519918,
author = {Pith},
title = {Pith review of: A Unified Architecture for Efficient Binary and Worst-Case Optimal Join Processing},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZRRH7ADA}},
note = {Machine review of arXiv:2505.19918}
}
read the original abstract
Join processing is a fundamental operation in database management systems; however, traditional join algorithms often encounter efficiency challenges when dealing with complex queries that produce intermediate results much larger than the final query output. The emergence of worst-case optimal join (WCOJ) algorithms represents a significant advancement, offering asymptotically better performance by avoiding the enumeration of potentially exploding intermediate results. In this paper, we propose a unified architecture that efficiently supports both traditional binary joins and WCOJ processing. As opposed to the state-of-the-art, which only focuses on either hash-based or sort-based join implementations, our system accommodates both physical implementations of binary joins and WCOJ algorithms. Experimental evaluations demonstrate that our system achieves performance gains of up to 3.1x (on average 1.5x) and 4.8x (on average 1.4x) over the state-of-the-art implementation of Generic Join and Free Join methods, respectively, across acyclic and cyclic queries in standard query benchmarks.
Figures
Figures from the paper (15 more)
Reference graph
Works this paper leans on
-
[1]
Aberger, Andrew Lamb, Susan Tu, Andres Nötzli, Kunle Oluko- tun, and Christopher Ré
Christopher R. Aberger, Andrew Lamb, Susan Tu, Andres Nötzli, Kunle Oluko- tun, and Christopher Ré. 2017. EmptyHeaded: A Relational Engine for Graph Processing. ACM Trans. Database Syst. 42, 4, Article 20 (oct 2017), 44 pages. https://doi.org/10.1145/3129246
doi:10.1145/3129246 2017
-
[2]
Albert Atserias, Martin Grohe, and Dániel Marx. 2013. Size Bounds and Query Plans for Relational Joins. SIAM J. Comput. 42, 4 (2013), 1737–1767. https: //doi.org/10.1137/110859440 arXiv:https://doi.org/10.1137/110859440
-
[3]
Michael Freitag, Maximilian Bandle, Tobias Schmidt, Alfons Kemper, and Thomas Neumann. 2020. Adopting worst-case optimal joins in relational database systems. Proc. VLDB Endow. 13, 12 (jul 2020), 1891–1904. https://doi.org/10.14778/3407790. 3407797
doi:10.14778/3407790 2020
-
[4]
Viktor Leis, Andrey Gubichev, Atanas Mirchev, Peter Boncz, Alfons Kemper, and Thomas Neumann. 2015. How good are query optimizers, really? Proc. VLDB Endow. 9, 3 (Nov. 2015), 204–215. https://doi.org/10.14778/2850583.2850594
arXiv 2015
-
[5]
LLVM Project. [n. d.]. LLVM SmallVector Documentation. https://llvm.org/ doxygen/classllvm_1_1SmallVector.html. Accessed: 2024-10-15
work page 2024
-
[6]
Amine Mhedhbi, Matteo Lissandrini, Laurens Kuiper, Jack Waudby, and Gábor Szárnyas. 2021. LSQB: a large-scale subgraph query benchmark. In Proceedings of the 4th ACM SIGMOD Joint International Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA) (Virtual Event, China) (GRADES-NDA ’21). Association for Computing...
-
[7]
Amine Mhedhbi and Semih Salihoglu. 2019. Optimizing subgraph queries by combining binary and worst-case optimal joins. Proc. VLDB Endow. 12, 11 (jul 2019), 1692–1704. https://doi.org/10.14778/3342263.3342643
arXiv 2019
-
[8]
Hung Q. Ngo. 2018. Worst-Case Optimal Join Algorithms: Techniques, Results, and Open Problems. In Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems (Houston, TX, USA) (PODS ’18). Association for Computing Machinery, New York, NY, USA, 111–124. https: //doi.org/10.1145/3196959.3196990
arXiv 2018
Show all 20 references
-
[9]
Ngo, Ely Porat, Christopher Ré, and Atri Rudra
Hung Q. Ngo, Ely Porat, Christopher Ré, and Atri Rudra. 2012. Worst-case optimal join algorithms: [extended abstract]. In Proceedings of the 31st ACM SIGMOD- SIGACT-SIGAI Symposium on Principles of Database Systems (Scottsdale, Arizona, USA) (PODS ’12). Association for Computi...
2012
-
[10]
Hung Q Ngo, Christopher Ré, and Atri Rudra. 2014. Skew strikes back: new developments in the theory of join algorithms. SIGMOD Rec. 42, 4 (feb 2014), 5–16. https://doi.org/10.1145/2590989.2590991
2014
-
[11]
Gregory Popovitch. 2024. The Parallel Hashmap C++ library . https://github.com/ greg7mdp/parallel-hashmap
2024
-
[12]
Mark Raasveldt. 2022. DuckDB - A Modern Modular and Extensible Database System. In CDMS@VLDB. https://api.semanticscholar.org/CorpusID:252384081
2022
-
[13]
Mark Raasveldt and Hannes Mühleisen. 2019. DuckDB: an Embeddable Analytical Database. In Proceedings of the 2019 International Conference on Management of Data (Amsterdam, Netherlands) (SIGMOD ’19). Association for Computing Machinery, New York, NY, USA, 1981–1984. https://doi...
2019 doi
-
[14]
Mark Raasveldt and Hannes Mühleisen. 2020. Data Management for Data Science - Towards Embedded Analytics. InConference on Innovative Data Systems Research. https://api.semanticscholar.org/CorpusID:210712240
2020
-
[15]
Rust Crate Developers. [n. d.]. SmallVec: A Rust crate for Small Vector Optimiza- tion. https://docs.rs/smallvec/. Accessed: 2024-10-15
2024
-
[16]
Ngo, and XuanLong Nguyen
Maximilian Schleich, Dan Olteanu, Mahmoud Abo Khamis, Hung Q. Ngo, and XuanLong Nguyen. 2019. A Layered Aggregate Engine for Analytics Workloads. In Proceedings of the 2019 International Conference on Management of Data (Ams- terdam, Netherlands) (SIGMOD ’19). Association for ...
2019
-
[17]
Amir Shaikhha, Mahdi Ghorbani, and Hesam Shahrokhi. 2023. Hinted Dictionar- ies: Efficient Functional Ordered Sets and Maps. In 37th European Conference on Object-Oriented Programming (ECOOP 2023) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 263), Karim Ali...
2023 doi
-
[18]
Amir Shaikhha, Mathieu Huot, Jaclyn Smith, and Dan Olteanu. 2022. Functional collection programming with semi-ring dictionaries. Proceedings of the ACM on Programming Languages 6, OOPSLA1 (2022), 1–33
2022
-
[19]
Veldhuizen
Todd L. Veldhuizen. 2013. Leapfrog Triejoin: a worst-case optimal join algorithm. arXiv:1210.0481 [cs.DB] https://arxiv.org/abs/1210.0481
2013 arXiv
-
[20]
Yisu Remy Wang, Max Willsey, and Dan Suciu. 2023. Free Join: Unifying Worst- Case Optimal and Traditional Joins. Proc. ACM Manag. Data 1, 2, Article 150 (jun 2023), 23 pages. https://doi.org/10.1145/3589295
2023 doi
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.