{"id":"7ac0f854-3cf2-42ce-bf74-d2164cc91696","arxiv_id":"2505.07595","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"On small benchmark datasets, SQL/PGQ is faster than SQL in DuckDB but slower in Spanner, so query performance depends on the system's internal optimizations rather than on the query language itself.","lead":"This paper compares SQL and SQL/PGQ graph queries on DuckDB, Google Cloud Spanner, and Neo4j, finding that the same logical query runs at very different speeds depending on which formalism is used. It argues that database systems should optimize SQL and graph patterns together rather than forcing users to guess which style is faster.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SQL rewrites in §3 are not faithful translations of the PGQ queries they are compared against, so the reported latency ratios may measure different workloads rather than the effect of query formalism.","rationale":"Good-faith reading: the paper is a workshop paper making a modest empirical point and a research agenda. The agenda does not depend on these particular numbers. The load-bearing empirical claim, however, does. I looked for the condition that must hold for the claim and found it least secure not in the acknowledged environmental noise but in the equivalence of the SQL rewrites. The abstract and Section 4 say the comparison is between SQL and SQL/PGQ queries written in both formalisms. If the SQL version is a different query, then differences in latency are explained by the query, not the formalism. The SQL for Q1 visibly returns fewer columns; the recursive SQL for Q2 visibly changes the semantics. This is a correctness issue, not a consensus issue. It can be settled by direct result-set comparison. I also note Table 1's internal inconsistency as a warning that the numerical layer needs verification. The reader's weakest_assumption targeted measurement noise; I agree that is real, but I would put query equivalence upstream. Hence partial agreement. Verdict remains CONDITIONAL: the direction is reasonable and the future-work discussion is useful, but the experimental support for the central claim needs corrected, equivalent rewrites and repeated runs before the claim can be taken as established.","tokens_in":10043,"tokens_out":6649,"duration_ms":67489,"concrete_test":"On the published DuckDB datasets, execute the §3 SQL rewrite and the SQL/PGQ Query 1 side by side in the same DuckDB process and compare the full result relations (columns, tuples, cardinality); repeat for Query 2 vs. Query 3. If the result sets differ, as the paper's own text suggests for Q1, then the reported latency ratios compare different queries and the cross-model conclusion is unsupported until equivalent rewrites are used and re-timed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that performance is 'often tied to how the query is expressed,' and the only direct evidence is the SQL-vs-SQL/PGQ latency comparison. That comparison is interpretable only if each SQL rewrite is the same query as its PGQ counterpart. Section 3's rewrite of Query 1 is not: the PGQ query matches a directed triangle and returns (x.name, y.name, z.name), while the SQL returns only the two endpoints of a friend edge that have some common friend, omitting the third vertex and changing the result shape. Likewise, Query 3 is presented as the translation of Query 2, but it enumerates all directed cycles up to depth 2000 and returns only starting accounts, whereas Query 2 uses ANY SHORTEST; these are different computations. If the rewrites are not equivalent, the ratios in Table 1 and the corresponding figures do not isolate the query formalism. The absence of repetitions, tiny datasets, and heterogeneous cloud environments noted by the reader compounds the problem, but it is not the primary issue: re-running the same mismatched queries many times would not make the comparison valid. The data also show internal inconsistencies, e.g., the DuckDB Q5/size-150 ratio in Table 1 (250.00) is not what Table 4 implies (12,420/55.5 ≈ 223.8), which reinforces that the empirical layer needs correction before it can support the abstract's conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the SQL/PGQ standard, which lets users query property graphs through either SQL-style joins or graph-pattern matching. It reports an experimental comparison of six queries (three bounded friend-triangle-style patterns and three unbounded transfer-cycle patterns) on synthetic datasets of 50, 100, and 150 rows, using DuckDB with DuckPGQ, Google Cloud Spanner, and Neo4j. The main empirical claim, stated in the abstract and Section 4, is that performance is often tied to how a query is expressed, which suggests that current systems are not yet fully decoupling SQL and SQL/PGQ optimization. The paper also proposes two future directions: applying and combining relational and graph algorithms, and internal query rewriting between the two formalisms. The authors acknowledge their results should be read qualitatively because of heterogeneous execution environments.","tokens_in":10241,"tokens_out":3938,"duration_ms":36923,"significance":"If the central claim were established, the paper would provide a useful data point for the SQL/PGQ and GQL community: it would indicate that users cannot currently choose between SQL and graph patterns based purely on convenience without performance consequences, and it would motivate holistic, cross-formalism optimizers. The paper is honest about several limitations and makes its experimental scripts available, which is commendable. However, the empirical evidence as presented does not support the central claim, because the SQL rewrites in Section 3 are not faithful translations of the SQL/PGQ queries, and because the experimental protocol (single unreplicated runs, tiny synthetic datasets, uncontrolled cloud environments) is too weak to separate query-formalism effects from noise. The paper also contains an internal inconsistency between Table 1 and Table 4 for DuckDB Q5 at size 150. With corrected query translations and a more rigorous experimental design, the underlying question is worth investigating.","major_comments":[{"comment":"The SQL rewrite of Query 1 is not equivalent to the PGQ triangle query. The PGQ query matches a directed triangle (x->y, y->z, z->x) and returns (x.name, y.name, z.name). The SQL version returns only distinct pairs (f.pid1, f.pid2) that have a common friend through FriendPairs, and it does not require the closing edge z->x or return the third vertex. The two queries therefore compute different results, and the latency ratio in Table 1 for Q1 measures different workloads rather than the effect of SQL versus SQL/PGQ expression. This undermines the bounded-query comparison.","section":"Section 3, Query 1"},{"comment":"Query 3 is not a faithful translation of Query 2. Query 2 uses ANY SHORTEST to find a shortest directed cycle matching the transfer pattern, whereas Query 3 recursively enumerates all directed walks up to depth 2000 and returns every account that appears in a cycle of length at least 2. These are different computations: Query 3 can return cycles that are not shortest, and the depth limit of 2000 is an ad hoc free parameter that is not shown to approximate ANY SHORTEST on the experimental datasets. Consequently, the unbounded-query ratio comparisons in Table 1 and the associated figures do not isolate the query formalism.","section":"Section 3, Query 2 vs. Query 3"},{"comment":"There is an internal inconsistency in the reported DuckDB results. Table 1 reports the DuckDB Q5 ratio at dataset size 150 as 250.00, but Table 4 gives SQL latency 12,420 ms and SQL/PGQ latency 55.5 ms, whose ratio is approximately 223.8, not 250. Smaller discrepancies also appear for other cells (e.g., Q2 at size 100 and Q4 at size 150). Since Table 1 is the main quantitative evidence for the paper's central claim, these discrepancies need to be resolved and the tables audited.","section":"Tables 1 and 4"},{"comment":"The experimental protocol is too weak to support the paper's qualitative conclusions. Each query appears to have been run once per cell, on datasets of only 50, 100, and 150 rows, and the cloud-based runs for Spanner and Neo4j were executed on hardware and configurations outside the authors' control, as the paper itself acknowledges. With no repetitions, no confidence intervals, and no statistical test, the latency differences in Figures 1 and 2 and Table 1 are indistinguishable from environment noise. This is particularly important because the entire central claim rests on these measurements.","section":"Section 4, Experimental Setting"}],"minor_comments":[{"comment":"Query 3's title contains a typo ('Recusive'), and the identifiers 'accoun_in_cycl e' and 'accou nt_in_cy cle' are broken by spaces. Please fix these formatting errors.","section":"Section 3"},{"comment":"The query definitions in the appendix and the paper would benefit from a precise statement of the six queries Q1-Q6; currently the reader must infer which figure corresponds to which query and how the bounded and unbounded variants relate. In particular, Query 2's 'RETURN ;' and the undefined variables px and pz in its WHERE clause should be corrected or explained.","section":"Section 4"},{"comment":"The entry '36.3m' for SQL Q5 at size 100 appears to be a typo; it should likely be '36.3', and the units should be stated consistently.","section":"Table 4"},{"comment":"There is a typo in the Introduction: 'feasable translations' should be 'feasible translations'.","section":"Section 1"},{"comment":"The statements about the expressiveness asymmetry between recursive SQL and SQL/PGQ rely on references [3] and [6], which are the authors' own prior work. A brief independent explanation of the separating query class would help readers assess this claim without consulting the cited papers.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is at the workshop stage and addresses a timely question, but the central empirical claim is not currently supported because the SQL rewrites are not equivalent to the PGQ queries and because the measurements are unreplicated and partially uncontrolled. These issues are fixable within the paper's scope: the authors could replace the rewrites with genuinely equivalent SQL (or explicitly argue for a weaker comparison), rerun the experiments with repetitions on controlled hardware, and correct the table inconsistencies. If those changes are made, the revised version could make a useful contribution; as it stands, the quantitative comparisons should not be interpreted as evidence about query formalism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the experiments are the heart of the paper and they don't hold up: the SQL rewrites in Section 3 are not equivalent to the PGQ queries they're meant to mirror. Query 1's SQL returns ordered friend pairs with a common friend; the PGQ returns directed triangles. Query 3 enumerates all cycles up to depth 2000; the PGQ uses ANY SHORTEST. So the ratios in Table 1 compare different workloads, not just different notation. Second, even setting that aside, the empirical base is thin—single runs, 50–150 row synthetic datasets, three heterogeneous environments—and the authors admit only to reading the results qualitatively. There's also a small arithmetic slip: Table 1 lists DuckDB Q5 at size 150 as 250.00, but Table 4's raw numbers imply about 224.\n\nWhat's genuinely useful: this is the first direct latency comparison of SQL and SQL/PGQ across DuckDB, Spanner, and Neo4j that I know of, with scripts on GitHub. The authors are upfront about the uncontrolled setup and don't oversell absolute runtimes. The framing—that system-internal optimization matters more than the user's choice of formalism—is a reasonable and testable hypothesis, and the future-work section points at concrete steps (internal rewriting, join algorithms for traversal). The citation of prior work, including Lou et al.'s converged optimization framework and their own expressiveness results, is honest.\n\nThe soft spots are addressable rather than fatal. Correct the rewrites so they actually capture the same pattern, run with repetitions and error bars, use larger datasets, and fix the ratio inconsistency. Then the paper would become a useful evidence point. As it stands, the central claim—that the two models are 'not yet fully decoupled'—is not supported by the data as presented.\n\nWho's this for? People thinking about whether SQL/PGQ is ready for production, and researchers working on unified relational-graph optimization. It's a workshop paper with a vision, not a definitive benchmark. I'd send it to review with a request for major revision; the question is timely and the authors are asking the right things. If they fix the translation issue, it's a solid contribution to the discussion.","headline":"The paper's central comparison is undermined by SQL rewrites that don't match the PGQ queries, so the numbers don't support the headline claim, but the research direction is worth watching.","tokens_in":10836,"tokens_out":3034,"would_cite":false,"duration_ms":28049,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SQL and SQL/PGQ are not yet \"fully decoupled\": query latency still depends on how the query is expressed.","keywords":["SQL/PGQ","property graphs","query optimization","cross-model efficiency","GQL","graph pattern matching","performance evaluation","query rewriting"],"falsifier":"Re-run the six queries on a single controlled machine with the same dataset sizes plus larger ones (thousands to millions of rows), repeating each measurement several times. If DuckDB no longer favors SQL/PGQ, Spanner no longer favors SQL, or the gaps shrink below the noise level, the conclusion that the models are not yet decoupled loses its experimental support.","tokens_in":9757,"feed_emoji":"⏱️","tokens_out":6294,"duration_ms":53833,"temperature":0.7,"pith_summary":"This paper tries to establish that the SQL/PGQ standard's promise of free switching between relational and graph querying is not yet met, because measured latency depends on which formalism the user chooses. The authors run six query pairs — bounded friend-triangle patterns and unbounded transfer cycles — written in both plain SQL and SQL/PGQ across DuckDB, Google Cloud Spanner, and Neo4j. They find that DuckDB often runs SQL/PGQ faster than the equivalent SQL, while Spanner runs SQL faster than SQL/PGQ, and Neo4j shows mixed results. From these gaps they conclude that current systems optimize each formalism separately, and that a holistic optimizer choosing the best internal algorithm regardless of syntax would shrink the gap. A sympathetic reader would care because the commercial value of SQL/PGQ depends on the user's choice of formalism being a matter of clarity, not performance.","feed_headline":"Same query, two formalisms, two speeds","feed_subtitle":"Benchmarks on DuckDB, Spanner, and Neo4j show each engine favors a different way to write the same graph query.","key_machinery":"The central mechanism is the paired SQL/SQL/PGQ rewriting of six queries, split into bounded patterns that reduce to ordinary joins and unbounded patterns that need recursive SQL or graph traversal. Running both versions on the same engine and taking the ratio of SQL latency to SQL/PGQ latency isolates the effect of formalism on each system's optimizer. The bounded/unbounded split matters because it distinguishes cases where relational join techniques should suffice from cases where graph-specific algorithms like multi-source BFS and Bellman-Ford are needed, which lets the paper pinpoint where each engine's optimization effort actually goes.","core_discovery":"The paper's central claim is that performance is often tied to how a query is expressed, so the SQL and SQL/PGQ execution paths are not yet fully decoupled. Evidence comes from a small benchmark where the same logical queries are encoded twice, once in SQL and once in SQL/PGQ, and run on DuckDB and Spanner, with Neo4j covering only the graph side. Results show the advantage flips by system: DuckDB's SQL/PGQ executions are generally faster than its SQL ones, including on unbounded traversals, while Spanner's SQL executions beat its SQL/PGQ ones on bounded patterns, which are the only ones Spanner supports. The paper argues that these differences are signs of separate optimizations for each formalism, and that systems should instead translate internally between the two models so the user's choice of language does not dictate performance.","pith_inferences":["One extension not tested in the paper: if the same paired queries are run on an engine that compiles both formalisms to a common intermediate representation, the SQL/SQL/PGQ latency ratio should approach 1, a direct way to confirm the decoupling diagnosis.","The paper's graph-creation numbers suggest that virtual graph views (compiled to relational plans) are viable in DuckDB but costly in Spanner and Neo4j; a system designer might therefore choose to keep views virtual only when the compilation path is present, and materialize them elsewhere.","If the observed gaps persist with larger datasets and repeated trials, cloud providers would face both an optimization opportunity and a standards-compliance risk, since SQL/PGQ adoption depends on the formalism not being a performance liability.","The one-run-per-cell methodology implies the true effect sizes are likely smaller than the reported ratios, so any future fix should be validated statistically, not with single latency samples."],"forward_implications":["If the decoupling claim is right, engines should automatically rewrite SQL/PGQ patterns into joins when the pattern is bounded and relational plans are cheaper.","Unbounded Kleene-star queries should be executed with graph traversal algorithms (CSR-based BFS, shortest path) rather than recursive SQL where the engine supports them, because DuckDB's SQL/PGQ shows the speedup such structures can provide.","Systems like Spanner that only optimize the SQL path will need native pattern-matching acceleration before they can honor SQL/PGQ's promise.","Internal rewriting must be partial: since recursive SQL is strictly more expressive than SQL/PGQ, the optimizer can always rewrite patterns into SQL but can only sometimes rewrite SQL into patterns."],"supporting_citations":[{"why":"Supplies the SQL/PGQ implementation and its CSR/BFS machinery; the central DuckDB system under test.","marker":"[21]"},{"why":"The second system under test; its SQL vs SQL/PGQ gap defines the main cross-system comparison.","marker":"[7]"},{"why":"Establishes that recursive SQL is more expressive than SQL/PGQ, setting the one-way limit on internal rewriting.","marker":"[6]"},{"why":"Provides the translation of core SQL/PGQ to first-order logic with transitive closure, grounding the SQL rewrites used in the benchmark.","marker":"[3]"},{"why":"Defines the pattern calculus that underlies the SQL/PGQ MATCH syntax for bounded and unbounded patterns.","marker":"[4]"},{"why":"Worst-case optimal join algorithms, the relational technique the paper proposes to reuse for graph edge traversal.","marker":"[16]"}],"fun_headline_variants":["SQL vs PGQ: Speed varies by engine","Same query, two optimized paths","Format choice shifts query performance","Your graph query's speed depends on its wording","One query, two formalisms, no consistent winner"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the measured latencies capture real engine behavior, even though each query ran once, on only 50–150 synthetic rows, and on environments that mix a local PC with cloud machines of unknown hardware.","fun_headline_variants_meta":{"raw":{"variants":["SQL vs PGQ: Speed varies by engine","Same query, two optimized paths","Format choice shifts query performance","Your graph query's speed depends on its wording","One query, two formalisms, no consistent winner"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000148,"raw_usage":{"total_tokens":1120,"prompt_tokens":807,"completion_tokens":313,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":423,"completion_tokens_details":{"reasoning_tokens":248}},"tokens_in":423,"tokens_out":313,"duration_ms":3863,"temperature":1.0,"reasoning_tokens":248,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:12:30.848938+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the six queries on a single controlled machine with the same dataset sizes plus larger ones (thousands to millions of rows), repeating each measurement several times. If DuckDB no longer favors SQL/PGQ, Spanner no longer favors SQL, or the gaps shrink below the noise level, the conclusion that the models are not yet decoupled loses its experimental support.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The second system under test; its SQL vs SQL/PGQ gap defines the main cross-system comparison."},{"cited_title":"Lin, and Liat Peterfreund","cited_arxiv_id":null,"evidence_quote":"Provides the translation of core SQL/PGQ to first-order logic with transitive closure, grounding the SQL rewrites used in the benchmark."}],"review_version":1}