{"id":"1a38f532-d79b-4c04-8a67-9bb5fa06e10f","arxiv_id":"2411.13278","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Schema inference is now a native SQL aggregate in Apache AsterixDB, using local schema trees and global merging, and benchmarks show speedups of up to two orders of magnitude over Spark-based tools.","lead":"This paper builds schema inference into Apache AsterixDB as a SQL function, so users can ask a database to describe the structure of its own JSON data without exporting it to Spark. For database practitioners, this promises faster, simpler schema discovery for NoSQL workloads, with measured speedups of up to two orders of magnitude over external tools in the paper's benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Performance claim is confounded: §4.6 credits the ADM format for avoiding JSON parsing, so the two-orders-of-magnitude speedup may not reflect the SQL-function integration itself.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern I find: the benchmark confounds storage format with the SQL-function integration. This is not an external or nitpicky objection; the paper's own Section 4.6 states that Open-SI 'uses the ADM format, which avoids the costly parsing Spark implementations require.' Thus the paper effectively concedes that the measured speedup includes a large parsing-avoidance component. The central engineering contribution—exposing schema inference as a SQL aggregate in AsterixDB—is credible and is supported by the described implementation, so I would not reject the paper on this basis. However, the abstract and conclusion elevate the 'two orders of magnitude' result to headline status, and that claim is not substantiated by the current experiment. A controlled baseline isolating parsing cost from algorithm/execution would settle the matter. Other issues, such as the absence of variance bars and the difficult-to-read pseudocode, are secondary and do not alter the main concern. Since the reader already reached a conditional-acceptance verdict, my recommendation is unchanged: the paper should be accepted only if the authors add a controlled comparison or significantly soften the performance claim.","tokens_in":12337,"tokens_out":2432,"duration_ms":27331,"concrete_test":"Re-run Expr. 3 with two matched baselines: (A) AsterixDB Open-SI over raw JSON files exposed as external datasets (no ADM conversion) using the same 2-NC / 8-partition configuration; (B) Spark implementations reading the same data pre-parsed into a binary format such as Parquet (or a serialized ADM-equivalent). If Open-SI's advantage over Spark drops below one order of magnitude on the Pharma dataset, the published 'two orders of magnitude' is primarily a storage-format effect; if the margin persists, the native-execution claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline performance claim ('up to two orders of magnitude' over external methods) rests on Expr. 3, which compares AsterixDB Open-SI scanning records already stored in native ADM format against Spark-based implementations (Baazizi, Spoth, Mior) that parse raw JSON text from HDFS. Section 4.6 explicitly attributes Open-SI's advantage to using the ADM format, which 'avoids the costly parsing Spark implementations require.' This makes the measured speedup a compound of two differences: (i) the SI algorithm and parallel execution model, and (ii) the storage/parsing layer. Because the abstract and conclusion present the speedup as evidence for the value of native SQL-function integration, the comparison does not isolate the contribution of the proposed function. The existence of the SQL function is credible and not in question. What is in question is whether the empirical support for the headline speedup would survive a controlled comparison. Since the paper itself names the parsing difference as the source of the advantage, the burden is on the authors to show the speedup is not primarily a format artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes schema inference (SI) as a native SQL function in Apache AsterixDB. It introduces two operations: Open-SI, which scans records of an open dataset and builds a Schema Intermediate Structure (SIS) tree via local aggregation per partition and global merging, and Closed-SI, which reads a predefined schema from metadata. The authors evaluate speed-up and scale-up on a Twitter dataset and compare against Spark-based SI implementations (Baazizi, Spoth, Mior) on GitHub, Yelp, and Pharma datasets, reporting up to two orders of magnitude speedup.","tokens_in":12529,"tokens_out":5234,"duration_ms":53895,"significance":"If the performance result is substantiated, the paper makes a useful systems contribution: turning SI into a declarative SQL function that runs in parallel inside a DBMS, with a real implementation (publicly available on GitHub). The algorithmic design of a local/global SIS merge is sensible and builds on prior work. The experiments use real-world datasets and address scalability in a multi-node cluster. However, the headline speedup currently rests on a comparison that conflates native execution with the storage format (ADM versus raw JSON parsing), and the lack of variance reporting weakens the empirical claims. The core idea of native SQL-function SI is credible and should be preserved; the experimental evidence needs to be sharpened.","major_comments":[{"comment":"The comparison against Spark baselines is confounded by storage representation. AsterixDB scans records already stored in native ADM format, while the Spark implementations parse raw JSON text from HDFS. Section 4.6 explicitly credits the ADM format for avoiding 'the costly parsing Spark implementations require.' Because the abstract and conclusion present the two-orders-of-magnitude result as evidence for the value of the SQL-function integration, the experiment does not isolate the contribution of the proposed function. Please add a controlled condition, e.g., having AsterixDB ingest and scan raw JSON, or having Spark read a deserialized/binary representation, and report the resulting speedup ratios.","section":"§4.6, Expr. 3"},{"comment":"Closed-SI is not directly comparable to the Spark baselines. It returns a schema already present in metadata without scanning records, so its 0.03-second runtime is essentially a metadata lookup rather than an inference cost. Presenting Closed-SI in the same aggregated comparison with Spark-based scanners overstates its advantage. Please either separate Closed-SI into its own evaluation or compare it against an equivalent metadata-only approach.","section":"§4.5, Figure 6"},{"comment":"The experiments report only the mean over 10 iterations, with no standard deviation, per-iteration values, or confidence intervals. Given the claim of 'up to two orders of magnitude' and the confounded comparison, variability reporting is necessary to assess whether the observed differences are stable and to support the strength of the claim.","section":"§4.1"},{"comment":"The pseudo-code is difficult to verify because of font-substitution artifacts in variable names (e.g., 𝑀𝑁𝑀, 𝑂𝑃𝑄𝑅𝑂𝑆, 𝑇𝑈𝑃𝑉𝑆) and inconsistent notation between the algorithms and the surrounding text. Additionally, there is no complexity analysis or memory-footprint characterization of the SIS tree. At minimum, provide readable pseudo-code and state the asymptotic complexity in terms of the number of records, total fields, and tree size.","section":"§3.3, Algorithms 1–4"}],"minor_comments":[{"comment":"The text contains numerous ligature artifacts (e.g., '!exibility' instead of 'flexibility', '!eld' instead of 'field') that should be fixed in the camera-ready version.","section":"Throughout"},{"comment":"Table 2 lists dataset sizes as '274M', '3.2G', etc. without an explicit unit; please specify whether these are bytes, and clarify whether the size refers to the raw text size or the parsed size.","section":"Table 2"},{"comment":"The related-work discussion should explicitly distinguish the claimed 'first' native SQL-function SI from Wang et al. [30], which already supports inferring, managing, and querying schemas inside a repository; the current wording may be defensible but needs a precise differentiation.","section":"§5"},{"comment":"The statement that existing SI approaches 'depend on external data processing frameworks, such as Apache Spark' is too broad, since Section 5 describes several standalone algorithms that are not tied to Spark. Please qualify the sentence.","section":"§1"},{"comment":"The speed-up and scale-up figures report normalized behavior without numerical execution times; including the underlying times (or a table) would allow quantitative verification of the claimed trends.","section":"Figures 4 and 5"},{"comment":"The conclusion repeats the 'two orders of magnitude' claim without mentioning the parsing confound identified in Section 4.6; please adjust the wording to match the evidence presented in the evaluation.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable fit for an experimental database venue and the system contribution appears real, but the experimental methodology needs substantial tightening before publication. I would not reject on the basis of the confound alone, since the existence of the SQL function is credible and the implementation is available; however, the headline performance claim must be re-evaluated under a controlled comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Calvin,\n\nQuick take: the paper is a real systems contribution — schema inference exposed as a SQL aggregate in AsterixDB, with both a scan-based version (Open-SI) and a metadata-based one (Closed-SI). The integration story is credible and the code is on GitHub, which is more than many papers do. The two-phase local/global merge with union nodes is standard from Baazizi and friends, but that's fine; the new thing is packaging it as a query operator, and that part works.\n\nWhat's weaker is the empirical support for the headline \"two orders of magnitude\" claim. The comparison in Expr. 3 runs AsterixDB over records already stored in native ADM and Spark over raw JSON text on HDFS. The authors themselves say in Section 4.6 that ADM \"avoids the costly parsing Spark implementations require,\" which means the speedup is partly a storage-format effect, not a pure win for the SQL-function design. If both systems read raw JSON, or both read binary, the gap could shrink a lot. The stress-test note is right about this, and I don't think it's a manufactured flaw — the paper's own text supports it. There are also no error bars, and the algorithm listings in the PDF are garbled, which makes verification harder than it should be.\n\nOn the positive side, the scale-up and speed-up experiments look plausible and use real datasets at reasonable sizes. The related work is honest and the self-citations (e.g., to AsterixDB architecture) are appropriate and not doing load-bearing work. The core engineering claim — that you can implement SI as an aggregate and have it work on both base datasets and query results — is not in question.\n\nWho should read this? People working on NoSQL schema management, semi-structured query processing, or benchmarking parallel DBMS features. It's a useful data point, with a cautionary tale about benchmark control built in.\n\nMy recommendation: this deserves a serious referee. The right verdict is conditional acceptance, with a request for a controlled comparison (ideally AsterixDB reading raw JSON, or Spark reading ADM) and variance reporting. The function exists and is plausibly useful; the speedup claim needs to be re-measured before it's repeated.","headline":"Genuine systems contribution with a headline speedup that is confounded by the ADM-vs-raw-JSON comparison; referee it but require a controlled benchmark.","tokens_in":13028,"tokens_out":3091,"would_cite":true,"duration_ms":32033,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims schema inference can run as a native SQL function inside a parallel DBMS, and that doing so beats external Spark-based pipelines by up to two orders of magnitude on real datasets.","keywords":["schema inference","JSON Schema","SQL functions","Apache AsterixDB","semi-structured data","NoSQL databases","parallel query processing","schema discovery"],"falsifier":"Run the same schema-inference benchmark with AsterixDB reading raw JSON from HDFS, or Spark reading pre-parsed binary records, holding all other settings equal; if the reported two-order gap collapses to a small factor, the load-bearing part of the performance claim is the storage format rather than the SQL-function integration.","tokens_in":12142,"feed_emoji":"🗄️","tokens_out":4939,"duration_ms":49524,"temperature":0.7,"pith_summary":"The paper argues that schema inference—automatically deriving a dataset's structure and types from raw semi-structured records—should be a built-in SQL operation rather than a job for an external data-processing pipeline. It introduces two SQL functions in Apache AsterixDB: Open-SI, which scans records and infers a JSON Schema, and Closed-SI, which reads the schema already stored in metadata. The paper reports that on real-world datasets the native implementation runs up to two orders of magnitude faster than existing Spark-based approaches. If this holds, users of NoSQL and semi-structured systems could discover schemas with familiar SQL queries, no data export, and no separate framework.","feed_headline":"Schema inference as a native SQL function - 100x faster","feed_subtitle":"A new SQL function infers JSON schemas inside AsterixDB, skipping external Spark pipelines and data export.","key_machinery":"The Schema Intermediate Structure (SIS) is a tree whose nodes—Primitive, Array, Multiset, Object, Union—model the AsterixDB Data Model (ADM), a superset of JSON. SIS carries the argument because it makes local-to-global merging simple: local SIS trees built per partition are merged by recursively combining nodes and creating Union nodes when the same field has different types across records. The same structure is converted into JSON Schema through a depth-first traversal that emits 'type', 'properties', 'items', and 'oneOf' keywords. The second piece of machinery is the aggregate-function execution model, which lets the existing parallel query engine pipeline records into local inference and then merge only the small SIS trees, avoiding movement of raw data.","core_discovery":"The central claim is that schema inference can be natively supported as an on-demand SQL function in a DBMS, demonstrated through an implementation in Apache AsterixDB. The Open-SI function works like an aggregate query: records are scanned in parallel across data partitions, each partition builds a local Schema Intermediate Structure (SIS) tree, a central node merges these into a global SIS, and the tree is converted to a JSON Schema. Fields whose types conflict across records become Union nodes under the JSON Schema 'oneOf' construct. Closed-SI avoids scanning entirely by generating the JSON Schema from the dataset's declared metadata. The paper presents this design as the first of its kind and reports that, in experiments on Twitter, GitHub, Yelp, and Pharma datasets, it outperforms Spark-based schema inference methods by an order of magnitude in most cases and by up to two orders of magnitude on the Pharma dataset.","pith_inferences":["A large part of the reported speedup likely comes from the storage-format comparison: AsterixDB scans already-parsed native ADM records while the Spark baselines parse raw JSON text, as the paper itself credits ADM for avoiding costly parsing; this is an inference about the benchmark, not a paper claim.","A testable extension is to run Open-SI on datasets with heavy type heterogeneity and observe whether the growth of Union nodes slows the global merge phase enough to erode the speedup.","The SIS merge path could be reused to collect richer statistics such as min/max values and null percentages, turning schema inference into a lightweight profiling tool without a separate scan."],"forward_implications":["If the paper is right, schema inference becomes a query-time operation on both base datasets and query results, rather than a separate pipeline step.","Users can invoke schema inference with a SQL function call, so it composes with filters, joins, and aggregations inside the database.","Closed-SI can return a schema for structured datasets in milliseconds by reading metadata, something external tools cannot do without scanning records.","Other parallel DBMSs with distributed storage and aggregate execution could adopt the same two-phase local-then-global design for schema inference."],"supporting_citations":[{"why":"Supplies the schema inference algorithm and the local/global merging strategy that the paper's SIS design builds on.","marker":"[7]"},{"why":"Provides a Spark-based bi-clustering schema inference baseline that the paper compares against.","marker":"[27]"},{"why":"Provides a Spark-based monoid-enriched schema inference baseline and the 'Min' and 'Simple' modes used in experiments.","marker":"[22]"},{"why":"Describes Apache AsterixDB's architecture, which the implementation leverages for parallel record processing.","marker":"[3]"},{"why":"Defines the Apache Spark framework that the external baseline approaches run on.","marker":"[34]"},{"why":"Establishes JSON Schema as the de-facto standard output format that the inferred schemas follow.","marker":"[24]"},{"why":"Documents the ADM format that the paper credits for avoiding costly JSON parsing.","marker":"[6]"},{"why":"Prior work on columnar schemaless storage that the SIS tree structure builds on.","marker":"[2]"}],"fun_headline_variants":["Schema inference as SQL function in AsterixDB, 100x faster","Native SQL function infers schemas, skips Spark, 100x speedup","AsterixDB gets built-in schema inference via SQL, up to 100x","Schema inference on demand: SQL function beats Spark by 100x","Two orders faster: schema inference as a native SQL function"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The big speedup comes from comparing the native database scan of already-parsed records with external tools that must parse raw JSON text from disk; if both sides were given the same input format, the advantage could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Schema inference as SQL function in AsterixDB, 100x faster","Native SQL function infers schemas, skips Spark, 100x speedup","AsterixDB gets built-in schema inference via SQL, up to 100x","Schema inference on demand: SQL function beats Spark by 100x","Two orders faster: schema inference as a native SQL function"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1615,"prompt_tokens":815,"completion_tokens":800,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":431,"completion_tokens_details":{"reasoning_tokens":702}},"tokens_in":431,"tokens_out":800,"duration_ms":8030,"temperature":1.0,"reasoning_tokens":702,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:37:04.571790+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same schema-inference benchmark with AsterixDB reading raw JSON from HDFS, or Spark reading pre-parsed binary records, holding all other settings equal; if the reported two-order gap collapses to a small factor, the load-bearing part of the performance claim is the storage format rather than the SQL-function integration.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the schema inference algorithm and the local/global merging strategy that the paper's SIS design builds on."},{"cited_title":"JSONoid: Monoid-based Enrichment for Configurable and Scalable Data-Driven Schema Discovery","cited_arxiv_id":"2307.03113","evidence_quote":"Provides a Spark-based monoid-enriched schema inference baseline and the 'Min' and 'Simple' modes used in experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes Apache AsterixDB's architecture, which the implementation leverages for parallel record processing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the ADM format that the paper credits for avoiding costly JSON parsing."},{"cited_title":"Columnar Formats for Schemaless LSM-based Document Stores","cited_arxiv_id":"2111.11517","evidence_quote":"Prior work on columnar schemaless storage that the SIS tree structure builds on."}],"review_version":1}