{"id":"3401b31d-a738-4b85-aa31-5992e6f391a6","arxiv_id":"2504.14322","paper_version":1,"verdict":"UNVERDICTED","confidence":"HIGH","novelty_score":1.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper is a survey arguing that Hadoop components can be orchestrated as a polyglot big data processing platform, with illustrative but untested use cases.","lead":"This paper reviews how the Hadoop ecosystem can combine storage and processing tools, such as Spark, Hive, HBase, Kafka, and others, into a 'polyglot' big data platform. It is a survey of known components and hypothetical use cases, with no measurements or new technique.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The efficiency/scalability claim for polyglot Hadoop is unsupported: Section 5 use cases are hypothetical, the promised monoglot comparison is absent, and Section 7 defers all tests to future work.","rationale":"The Reader's verdict of UNVERDICTED is appropriate, and my stress-test pass does not change it. The paper is best read as a survey and position piece: it describes the Hadoop ecosystem, explains polyglot persistence concepts, and proposes hypothetical use cases. The central claim that polyglot Hadoop processing is efficient and scalable is not a theorem or a derivable result; it is an empirical performance claim. The manuscript provides no experiments, benchmarks, datasets, or reproducible artifacts to support it, and it explicitly defers comprehensive tests to future work. The only in-text reference to evaluation, Section 5.3's citation of the authors' companion paper [1], does not bring evidence into this manuscript. My concern is therefore not that the argument is internally inconsistent, but that its load-bearing empirical premise is entirely unverified. This does not warrant REJECT in the sense of demonstrating the claim is false; it warrants the paper remaining UNVERDICTED as a research contribution. A controlled benchmark comparing a representative polyglot pipeline against a monoglot baseline would settle whether the claimed advantage exists. The Reader and I identify the same weakest point, so agreement is 'agree.'","tokens_in":17020,"tokens_out":2669,"duration_ms":26158,"concrete_test":"Reproduce the Section 5.3 social-network pipeline on a fixed 8-node cluster: ingest a public Twitter-like corpus via Kafka into Hive internal and external tables, run Presto ad-hoc queries and a Spark/MapReduce term-frequency job, and measure end-to-end latency, throughput, and resource usage at two data scales (e.g., 1x and 4x corpus size). Compare against a monoglot baseline running the same analytical workloads in Spark SQL over Parquet files on HDFS without Kafka, Hive, or Presto. If the polyglot pipeline shows no measurable advantage in at least one metric (latency, throughput, or cost per query), the Section 4 efficiency and scalability claim has no empirical support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim in Section 4, that orchestrating HDFS, Hive, HBase, Kafka, Spark/MLlib and related components as a polyglot platform 'allows for efficient and scalable data processing,' rests entirely on the assumed effectiveness of the component combinations sketched in Sections 5.1 through 5.4. Those use cases are explicitly hypothetical ('Consider...', 'Imagine...'), and no implementation, dataset, benchmark, or measured comparison is reported anywhere in the manuscript. The conclusion in Section 7 concedes this by listing 'executing comprehensive tests' as future work. The only pointer to an evaluation, the reference [1] in Section 5.3, is to a separate companion paper whose results are not included here. Meanwhile, the introduction promises 'a comparative analysis of these polyglot strategies against traditional, or monoglot, systems,' but no such comparison appears. Thus the load-bearing premise is an unverified empirical claim: that integrating multiple storage and processing engines yields better efficiency, scalability, or real-time performance than a simpler monoglot stack. This is not internally inconsistent, but it is unsupported. If the integrations impose operational and data-movement overhead without measurable gains, the paper's main practical claim collapses. This is the same weak point the Reader identified, and it is decisive for any assessment of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that the Hadoop ecosystem, centered on HDFS, can serve as a polyglot big data processing platform in which different storage systems (HBase, Hive, Cassandra, etc.) and processing engines (MapReduce, Spark, Storm, Kafka) are selected per workload. It reviews the ecosystem's storage, resource management, processing, programming, SQL, and NoSQL components; discusses polyglot persistence and its relationship to the CAP theorem; proposes a general polyglot architecture with ingestion, storage, and processing phases; and illustrates the approach with four hypothetical use cases (healthcare, stock market, social networks, smart cities). The paper concludes that the approach enables efficient and scalable processing and lists benchmarking and comparative studies as future work.","tokens_in":17262,"tokens_out":3851,"duration_ms":33324,"significance":"If the manuscript's empirical claims were substantiated, the paper would offer a useful synthesis of how Hadoop ecosystem components can be orchestrated as a polyglot platform, and the use cases would be a helpful orientation for practitioners. The manuscript is clearly written, covers a broad set of technologies, and provides instructive diagrams. Its strength is conceptual: the claim that HDFS can act as a central repository with multiple processing engines and storage systems is defensible from the cited documentation. However, the paper does not yet support the stronger claim of efficiency and scalability advantages over monoglot systems, and the comparative analysis promised in the introduction is absent. The contribution is currently at the level of a survey or position paper rather than an empirically validated research advance.","major_comments":[{"comment":"The introduction promises \"a comparative analysis of these polyglot strategies against traditional, or monoglot, systems,\" but the manuscript never delivers such a comparison. Sections 4 and 5 only describe the polyglot architecture and hypothetical use cases, and Section 7 explicitly lists \"the comparative analysis of different polystores within the Hadoop ecosystem\" as future work. This is a load-bearing gap because the title and abstract claim that the approach \"advances\" polyglot big data processing beyond conventional monoglot systems.","section":"Section 1 (Introduction)"},{"comment":"The use cases are presented as illustrations (\"Consider...\", \"Imagine...\") and contain no implementation, dataset, or measured benchmark. The central assertion in Section 4 that polyglot processing \"allows for efficient and scalable data processing\" is therefore unsupported by evidence in the manuscript. The conclusion (Section 7) concedes this when it lists \"executing comprehensive tests across the various domains\" as future work, so the paper's main practical claim is currently a hypothesis rather than an established result.","section":"Sections 5.1-5.4"},{"comment":"The only reference to an evaluation in the use-case sections is the sentence \"The evaluation of the jobs related to this use case can be found in [1],\" which defers to a companion paper by the same authors. Because [1] is not included and its methodology and results are not summarized, readers cannot verify the claimed benefits for the social network scenario, and the manuscript is not self-contained.","section":"Section 5.3"}],"minor_comments":[{"comment":"The acronym \"Y ARN (Yet Another Resource Negotiator)\" contains a typo; it should be \"YARN\", and the name is spelled inconsistently as \"Yarn\" elsewhere in the same section.","section":"Section 2.2"},{"comment":"The text says that Resilient Distributed Datasets \"today they are called just Dataframes.\" RDDs and DataFrames are distinct abstractions in Spark; this conflation should be corrected.","section":"Section 2.3.1"},{"comment":"The CAP theorem discussion is generic and does not explain how the consistency-availability-partition trade-offs apply to the specific Hadoop components selected in the proposed architecture; consider tying it concretely to HBase, Hive, and Kafka.","section":"Section 3.2"},{"comment":"The section heading and opening sentence claim \"real-world examples or case studies,\" but the scenarios that follow are hypothetical as written. The wording should be changed to \"illustrative scenarios\" unless actual deployments are described.","section":"Section 5"},{"comment":"The reference list is inconsistent: some entries include access dates for URLs and others do not, and some entries (e.g., [36]) appear to have odd metadata. Please standardize citation formatting.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is close to a survey/position paper, and the editor may want to assess whether the venue's scope welcomes such contributions without empirical validation. The self-citation [1] carries a key evaluation claim in Section 5.3; it would be worth confirming that this companion paper is published and accessible, since the present manuscript is not self-contained on that point."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Couple of things to know. This is a survey/position paper, not a research paper. It walks through the Hadoop ecosystem—HDFS, YARN, MapReduce, Spark, Hive, HBase, Kafka, and several NoSQL stores—and argues that combining them in a polyglot fashion supports efficient, scalable big data processing. That argument is repeated confidently, but it is never tested.\n\nWhat it does well: the component descriptions are accurate and readable, with mostly sensible citations to primary sources (GFS, MapReduce, RDD, Calcite, Fowler's polyglot persistence). A newcomer to the ecosystem would come away with a reasonable map of the terrain. The authors are also honest in the conclusion that no comprehensive tests have been run.\n\nThe soft spot is load-bearing. Section 1 promises \"a comparative analysis of these polyglot strategies against traditional, or monoglot, systems.\" That analysis never appears. Section 5's use cases—healthcare, stock market, social networks, smart cities—are explicitly hypothetical, using \"Consider\" and \"Imagine,\" with no implementation, dataset, benchmark, or measured results. The only pointer to an evaluation is reference [1], the authors' companion paper on the social network use case, but that evaluation is not included here. Section 7 defers \"executing comprehensive tests\" to future work, which means the claim that polyglot Hadoop \"allows for efficient and scalable data processing\" is an assumption, not a finding. The stress-test note is exactly right on this point.\n\nThere are also minor issues: some references feel like padding rather than used support, and the paper overstates the novelty of combining known tools. But the main gap is the missing evaluation.\n\nWho is it for? As an introductory tutorial or a course handout on Hadoop components, it would be fine. As a research contribution, there is nothing new to build on, and I would not cite it in my own work.\n\nIf I were the editor, I would desk reject it. A serious referee would spend their time on something with an actual contribution, and the promised comparison simply isn't there. To make this a real paper, the authors should run the social network evaluation from [1] (or any other use case), do the monoglot comparison, and report actual numbers.","headline":"A competent but unoriginal survey of Hadoop components that promises a monoglot comparison it never delivers, and asserts a polyglot efficiency claim without testing it.","tokens_in":17778,"tokens_out":2705,"would_cite":false,"duration_ms":23070,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The Hadoop ecosystem can serve as a polyglot big data platform, pairing each data type with the right store and engine.","keywords":["polyglot persistence","polyglot data processing","Hadoop ecosystem","HDFS","big data","data lakes","real-time analytics","NoSQL data stores"],"falsifier":"Run the same workload twice on one cluster: once with a polyglot stack (for example, Kafka for ingestion, HBase for live reads, Hive for warehousing, and Spark MLlib for modeling) and once with a single monoglot alternative (for example, Spark alone reading from HDFS), using identical data, queries, and hardware, then measure end-to-end latency, throughput, and resource use. If the monoglot configuration matches or beats the polyglot one on the metrics the paper cites, the central practical claim is not supported.","tokens_in":16800,"feed_emoji":"🐘","tokens_out":4634,"duration_ms":41405,"temperature":0.7,"pith_summary":"The paper sets out to establish that the Hadoop ecosystem, with HDFS at its center, can serve as a polyglot big data processing platform: one where different storage systems and processing engines are chosen for each data type and task. It argues that mixing polyglot persistence (HBase for real-time read/write, Hive for warehouse queries, external tables over raw files) with polyglot computing (MapReduce, Spark, Storm/Kafka streaming, SQL engines) yields efficient and scalable processing, and it illustrates the combination through healthcare, stock market, social network, and smart city scenarios. The paper's evidence is architectural: it assembles the ecosystem's components, explains each one's role, and maps them to workload needs. A sympathetic reader would take the claim as a design thesis: a single HDFS-centered platform can replace several specialized monoglot systems, but measurement of that advantage is left to future work.","feed_headline":"One Hadoop platform can serve every big-data task with its best tool","feed_subtitle":"Matching data types to the right Hadoop component, from HBase real-time reads to Hive warehousing and Spark analytics.","key_machinery":"The mechanism that carries the argument is a four-phase workflow centered on HDFS: an Ingestion Phase that lands raw data in a Landing Zone, a transformation step that produces an Enriched Zone, a Storage Phase that routes data to Hive internal tables, Hive external tables, or HBase tables, and a processing phase where distinct tasks read from the stores best matched to their needs. The load-bearing design pattern is HDFS as a shared raw-data repository combined with per-task engine and store selection, supported by integration points such as Hive external tables (a schema over shared files), Hive-HBase integration, and mediator systems like Apache Calcite that expose one SQL interface over heterogeneous stores.","core_discovery":"Polyglot Big Data Processing is the paper's central concept: using various data stores and processing technologies suited to different data types within one platform. The specific claim is that the Hadoop ecosystem, centered on HDFS, already contains the building blocks for this: HDFS as common storage; YARN for resource management; MapReduce and Spark for batch and in-memory processing; Hive, Impala, and Presto as SQL engines; HBase, Cassandra, Neo4j, MongoDB, and others as specialized stores; Kafka, Storm, Flume, and NiFi for ingestion and streaming; and Calcite-like mediators for unified query access. Arranged well, these components let each task read from the most appropriate store and process with the most appropriate engine, enabling efficient and scalable data processing and real-time analytics.","pith_inferences":["Editorial inference: if the thesis holds, the practical adoption path is incremental—teams can add specialized stores and engines around an existing HDFS data lake rather than replacing their stack, which lowers the barrier to polyglot processing.","Editorial inference: the claimed advantage likely depends on workload heterogeneity; a monoglot system may remain competitive for uniform workloads, so the strongest test cases are mixed-velocity, mixed-model datasets.","Editorial inference: a concrete next experiment suggested by the paper would be a benchmark comparing mediator-based querying (for example, Calcite across Hive, MongoDB, and HBase) against hand-written federation logic, measuring query latency and development effort.","Editorial inference: the architecture also implies operational costs, because many components must run in one cluster, so a fair comparison should include administration overhead and not only query speed."],"forward_implications":["If the claim is correct, a single HDFS-centered data lake can host both batch and streaming workloads, replacing separate monoglot clusters for warehousing, real-time access, and machine learning.","Organizations can route data by type: HBase for low-latency key access, Hive external tables for SQL over raw files, Hive internal tables for managed warehouse data, and Spark or MapReduce for heavy computation.","Domains with mixed data velocity, such as healthcare monitoring and smart cities, can keep all their data in one platform and still get real-time alerts alongside historical analytics.","The polyglot approach defines a comparison agenda: which component pairs perform best, and where polystore mediators are needed, which the paper explicitly names as future work."],"supporting_citations":[{"why":"Supplies the MapReduce programming model that is the core of Hadoop's distributed processing.","marker":"[14]"},{"why":"Provides the Google File System architecture that inspired HDFS distributed storage.","marker":"[19]"},{"why":"Supplies the Resilient Distributed Dataset abstraction that underpins Spark's in-memory processing advantage.","marker":"[64]"},{"why":"Defines polyglot persistence and the CAP-based argument for why SQL alone is insufficient in distributed systems.","marker":"[29]"},{"why":"Provides the polystore classification and the Calcite example the paper uses for mediator-based unified querying.","marker":"[20]"},{"why":"Coins the term polyglot persistence and supplies the principle of choosing storage technology by data model.","marker":"[38]"},{"why":"Supplies the strengths of document, wide-column, key-value, and graph stores that justify per-type storage choices.","marker":"[46]"},{"why":"Provides the canonical description of Hadoop, HDFS, and MapReduce execution used throughout the paper.","marker":"[60]"}],"fun_headline_variants":["Hadoop ecosystem unifies polyglot big data tools","Polyglot big data: one platform, many specialized tools","Hadoop: the polyglot platform for diverse data tasks","Polyglot processing: Hadoop brings diverse tools together"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that the proposed component combinations, such as Kafka ingestion plus HBase storage plus Hive warehousing plus Spark MLlib, actually deliver the promised efficiency, scalability, and real-time benefits in real deployments.","fun_headline_variants_meta":{"raw":{"variants":["Hadoop ecosystem unifies polyglot big data tools","Polyglot big data: one platform, many specialized tools","Hadoop: the polyglot platform for diverse data tasks","Polyglot processing: Hadoop brings diverse tools together"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000538,"raw_usage":{"total_tokens":2519,"prompt_tokens":819,"completion_tokens":1700,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":435,"completion_tokens_details":{"reasoning_tokens":1632}},"tokens_in":435,"tokens_out":1700,"duration_ms":10563,"temperature":1.0,"reasoning_tokens":1632,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:50:24.739615+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same workload twice on one cluster: once with a polyglot stack (for example, Kafka for ingestion, HBase for live reads, Hive for warehousing, and Spark MLlib for modeling) and once with a single monoglot alternative (for example, Spark alone reading from HDFS), using identical data, queries, and hardware, then measure end-to-end latency, throughput, and resource use. If the monoglot configuration matches or beats the polyglot one on the metrics the paper cites, the central practical claim is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the canonical description of Hadoop, HDFS, and MapReduce execution used throughout the paper."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MapReduce programming model that is the core of Hadoop's distributed processing."},{"cited_title":"Ghemawat, H","cited_arxiv_id":null,"evidence_quote":"Provides the Google File System architecture that inspired HDFS distributed storage."},{"cited_title":"Zaharia, M","cited_arxiv_id":null,"evidence_quote":"Supplies the Resilient Distributed Dataset abstraction that underpins Spark's in-memory processing advantage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines polyglot persistence and the CAP-based argument for why SQL alone is insufficient in distributed systems."},{"cited_title":"Polyglot Persistence","cited_arxiv_id":null,"evidence_quote":"Coins the term polyglot persistence and supplies the principle of choosing storage technology by data model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the strengths of document, wide-column, key-value, and graph stores that justify per-type storage choices."}],"review_version":1}