{"id":"53949ea6-a0cf-4161-9b1e-6bc7d91ce597","arxiv_id":"2504.14314","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A three-node Hadoop-Spark benchmark demonstrates that polyglot processing can run social-network tasks, but the reported scalability evidence is too weak to support the paper's conclusions.","lead":"The paper benchmarks three social-media analytics tasks, influencer ranking, term counting, and user-graph construction, across Hive, HBase, MapReduce, and GraphX on a three-node Hadoop-Spark cluster. It reports execution times for 500K and 5M tweet datasets and argues the polyglot setup is practical for social network research.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scalability conclusion rests on unreported 5M-tweet timings and uncontrolled warm-up/caching; decreasing times suggest experimental artifact, not capacity.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the scalability inference assumes that the lower execution times at 5M tweets reflect system capacity rather than experimental confounds. My stress-test confirms this: the paper provides no numerical 5M times, no variance measures, and no controls for warm-up, caching, or dataset composition. The internal text even flags the result as surprising and calls for further investigation, which weakens the certainty of the scalability conclusion. Because the central quantitative claim rests on sparse and unreported measurements, the REJECT verdict is appropriate. I recommend no change to the reader's verdict, as the concern is already central and correctly identified.","tokens_in":7962,"tokens_out":1657,"duration_ms":16186,"concrete_test":"Request the raw per-run execution times for every task on both the 500K and 5M datasets, together with the run order and cluster state (e.g., whether jobs were submitted back-to-back on a warm cluster, cache status, and any cleanup between runs). Recompute means and variances from these raw logs. Then rerun the 5M workload on a freshly restarted cluster with cold caches before each trial, interleaved with 500K trials, and compare. If the 5M times are no longer lower than the 500K times, the scalability claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that the polyglot Hadoop-Spark approach is efficient and scalable—depends on the Section 5.4 observation that execution times decrease when the dataset grows from 500K to 5M tweets. This inference is load-bearing and insecure. First, the 5M-tweet execution times are never reported numerically in the text; the paper only says they are 'notably lower than initially expected' and refers to figures. Without the actual numbers, the quantitative basis for the scalability claim is absent. Second, although Section 5.4 states each job was executed ten times and averages were reported, no variance, per-trial values, or confidence intervals are given, so the 5M vs. 500K comparison could be within noise. Third, the experimental design does not control for cluster warm-up, OS page cache, JVM JIT compilation, HDFS block caching, or dataset-composition differences. A monotone decrease in execution time with a 10x larger dataset is the classic signature of a caching/warm-up effect, not of scalable throughput. The paper itself signals surprise ('notably lower than initially expected') and asks for further investigation, so the conclusion that the solution 'is likely scalable' exceeds what the measurements can support. The weakest link is therefore the causal claim from decreasing times to scalability, rather than the raw fact that jobs ran.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a 'polyglot data processing' approach within the Hadoop-Spark ecosystem, combining storage engines (HDFS, Hive, HBase) and processing frameworks (MapReduce, Spark, GraphX/GraphFrames) for three Twitter analytics tasks: identifying influential users, counting frequent terms, and building a user-follows graph. Experiments are conducted on a 3-node Google Dataproc cluster with datasets of approximately 500K and 5M tweets; execution times are averaged over ten runs. The authors conclude that the approach is efficient, feasible, and 'likely scalable' because execution times appeared to decrease from the smaller to the larger dataset.","tokens_in":8155,"tokens_out":6086,"duration_ms":49165,"significance":"If the empirical results were reliable, the paper would provide a useful practical benchmark for practitioners selecting Hadoop/Spark components for social-network analytics, and the task-to-engine mapping is a reasonable engineering contribution. The paper has strengths: it uses real Twitter data, describes a clear evaluation cycle, reports a concrete cluster configuration, and makes its workloads explicit. However, the quantitative core is currently too weak to support the stated efficiency and scalability claims: the 5M-dataset timings are missing, no variability is reported, experimental confounds (caching, warm-up, dataset composition) are uncontrolled, and the Task 1 influence query contains a formula error. The soundest part is the qualitative mapping of tasks to ecosystem tools, which may retain value if the quantitative claims are substantially revised or removed.","major_comments":[{"comment":"The scalability inference is presented in Section 5.4: 'The observation that execution times decrease as the dataset volume increases suggests that the solution is likely scalable.' Yet the execution times for the 5M-tweet dataset are never reported numerically; the text only says they were 'notably lower than initially expected' and points to Figure 6, which is not legible in the manuscript as provided. Without the actual values, the reader cannot verify the direction or magnitude of the change, so the central quantitative claim is unsubstantiated.","section":"5.4, Figure 6"},{"comment":"Section 5.4 states that each job was executed ten times and that the reported times are averages, but no variance, per-trial measurements, confidence intervals, or statistical tests are provided. With n=10 and no dispersion information, the comparison between the 500K and 5M timings could easily be within run-to-run noise, so the observed difference does not constitute evidence of scalability.","section":"5.4"},{"comment":"The experimental design does not control for cluster warm-up, OS page cache effects, JVM JIT compilation, HDFS block caching, or differences in tweet composition between the two datasets. A monotone decrease in execution time with a 10x larger dataset is the classic signature of a caching or warm-up artifact, and the paper's own expression of surprise ('notably lower than initially expected') indicates that the authors did not expect this outcome. The conclusion that the solution is likely scalable therefore exceeds what the measurements can support.","section":"5.4"},{"comment":"The SQL query in Section 5.1 defines influence as `SUM(impression_count) + SUM(like_count) + SUM(like_count) + SUM(reply_count) + SUM(retweet_count)`, which double-counts `like_count` and omits `quote_count`, contradicting the task definition in Section 4.1 that influence is based on impressions, likes, quotes, replies, and retweets. This error means the reported Task 1 results do not actually measure the stated influence metric.","section":"5.1, Task 1 query"}],"minor_comments":[{"comment":"The caption of Figure 1 contains a typo: 'Understanting' should be 'Understanding.'","section":"Figure 1 caption"},{"comment":"The query begins with 'SELECT author_id.' where a comma after the column list is intended; as printed, the query is not syntactically valid.","section":"5.1, SQL query"},{"comment":"The text says that all graphs are generated using 'the Spark GraphX library and GraphFrames extension,' but the code fragment only uses GraphFrames; GraphX and GraphFrames are distinct APIs, and the manuscript should clarify which one was used.","section":"5.3"},{"comment":"The Python code snippet contains non-ASCII characters inside the `replace` calls and omits the surrounding Spark session and stopword definitions, making the snippet hard to reproduce.","section":"5.2"},{"comment":"The phrase 'the above-mentioned findings' is vague; the results should be anchored to specific figure numbers and, better, to a table with numeric execution times.","section":"5.4"},{"comment":"The reference to Dean and Ghemawat (2004) lacks a publication venue, and the Apache Spark RDD reference points to a tutorialspoint URL rather than an authoritative specification; both should be updated.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like an extended technical report rather than a fully developed research article. The load-bearing issues are the missing 5M-dataset timings, the absence of variance reporting, and the uncontrolled warm-up/caching confounds; these require new measurements or a substantial reframing of the claims. The SQL formula error in Task 1 should also be verified and corrected. The authors are advised to either provide a rigorous experimental appendix with raw per-trial times, cache-clearing procedures, and additional dataset sizes, or to restrict the conclusions to a feasibility study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked for a read on arXiv:2504.14314. My take: it's a workshop-level case study that runs three standard social-network analytics jobs across Hive, HBase, MapReduce, Spark, and GraphX on a three-node Dataproc cluster. That specific measurement combination is the only thing I'd call new. The polyglot framing comes from the literature they cite, and the tasks are textbook applications of each tool. The authors deserve credit for describing the setup and code clearly, and for not overselling what they did beyond one questionable inference. No self-citation, no equations to check, and the runs at least look real.\n\nThe soft spots are the ones the stress-test flags. The 5M-tweet execution times never appear in the text; we only get 'notably lower than initially expected' and a chart. No variance or per-trial values are given despite saying each job ran ten times. No controls for warm-up, caching, or JIT are mentioned. A monotone decrease in time as data grows 10x is the classic signature of a cold-start artifact, not scalable throughput. The paper itself says the result 'suggests' scalability and calls for further investigation, which is at least a hedged version, but the inference still doesn't follow from the data they show. Also, Task 1's SQL has a duplicated like_count in the influence sum—a minor but embarrassing bug. Task 2 compares MapReduce and Spark, but the results only mention MapReduce; the Spark number is missing. For Task 3 we get a graph build time but no comparison to anything else. The quantitative core is thin.\n\nThe reader's REJECT with moderate confidence is fair. The paper is not a serious research contribution on its own terms: the central empirical claim is unsupported, and the missing raw data makes it impossible to verify. That said, it is not incoherent and the authors are honestly reporting what they did. I would not send this to peer review at a journal or a competitive conference. It could be a decent workshop poster or a technical report, and the authors could strengthen it substantially by releasing measurements, adding error bars, controlling for cluster state, and fixing the SQL. As it stands, a serious editor should desk-reject rather than spend referee time on it.\n\nFor your own work: not something you'd cite, but worth a glance if you ever teach a class on benchmark pitfalls. I wouldn't bring it to reading group; the interesting conversation is about the methodology, not the results.","headline":"A small, honest benchmark case study that overreaches on scalability; useful as a cautionary example of inferring capacity from wall-clock time.","tokens_in":8690,"tokens_out":1913,"would_cite":false,"duration_ms":18493,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A polyglot Hadoop-Spark pipeline can rank influencers, count terms, and build follower graphs, with execution times that fall as tweet volume grows.","keywords":["polyglot data processing","Hadoop","Spark","social network analysis","Hive","HBase","GraphX","scalability"],"falsifier":"Run the same three tasks on fresh clusters at 1M, 2M, and 10M tweets, recording per-run times and variance; if execution time rises at any step or varies widely across identical runs, the claimed scalability is not supported.","tokens_in":7735,"feed_emoji":"📊","tokens_out":5575,"duration_ms":46936,"temperature":0.7,"pith_summary":"The paper argues that social-network data processing should not rely on a single compute or storage engine. Instead, it proposes a polyglot approach within the Hadoop-Spark ecosystem, pairing each analytical task—ranking influential users, counting frequent terms, discovering user-group relationships—with the engine best suited to it: Hive or HBase for querying, MapReduce or Spark for term counting, GraphX for graph construction. To support this, the authors ran the three tasks on 500K and 5M tweet datasets and report average execution times of roughly 11 seconds and 16 seconds for the influence query, 28 seconds for term counting, and 42 seconds for graph generation on the smaller set, with lower times on the larger set. The paper's central claim is that the approach is feasible, efficient, and likely scalable, letting practitioners choose engines per task while keeping performance stable or improving as data grows.","feed_headline":"5M tweets process faster than 500K in polyglot Hadoop","feed_subtitle":"Mixing Hive, HBase, MapReduce and GraphX keeps social-network queries fast as data grows.","key_machinery":"The load-bearing mechanism is Polyglot Data Processing: a combination of polyglot persistence (storing the same Twitter data simultaneously in HDFS, Hive external or internal tables, and HBase) and polyglot computing (choosing Spark, MapReduce, or GraphX per task). The paper treats HDFS as the central repository and ingestion landing zone, Hive as a SQL-like warehouse, HBase as a column-family NoSQL store for read-intensive queries, and Spark's GraphX as the in-memory graph layer. This machinery lets the authors attribute the observed performance to matching each task's data access pattern to the appropriate engine, rather than to any single technology.","core_discovery":"The central discovery is that a polyglot processing pipeline built from HDFS storage plus Hive, HBase, MapReduce, Spark, and GraphX can execute the three selected social-network tasks end to end, and that the measured average execution times for the 5M-tweet dataset were lower than those for the 500K-tweet dataset. The authors interpret this decrease as evidence that the solution is likely scalable. In their view, the experiment demonstrates that different engines have complementary strengths: Hive over HDFS handles the influence query in about 11 seconds versus about 16 seconds for HBase, MapReduce and Spark count frequent terms in about 28 seconds, and GraphX builds the user graph in about 42 seconds, with ingestion and storage under a minute for the smaller set and about seven minutes for the larger one.","pith_inferences":["The reported decrease in execution time from 500K to 5M tweets is likely driven by fixed overheads such as cluster startup, query planning, and JVM warm-up being amortized over more data, rather than by the engines becoming intrinsically faster; a fair scalability test would keep cluster state and cache conditions identical and report variance.","The paper's future-work list already points to Flink, Kafka, and Hudi; the untested implication is that the polyglot assignment rule could extend to streaming and incremental processing, routing real-time feeds through streaming engines while batch analytics continue on Hive and HBase.","The Hive-versus-HBase comparison is workload-specific: with different query shapes such as point lookups, scans, or updates, HBase's column-family storage could beat Hive, so the engine rankings should not be generalized beyond the three tasks.","A testable extension is to run the same three tasks at intermediate volumes of 1M, 2M, and 10M tweets and plot execution time against dataset size; the paper's scalability claim predicts flat or falling times, while an eventual rise would locate the crossover point."],"forward_implications":["Practitioners can build a social-media analytics pipeline where the same raw tweets in HDFS are queried through Hive for warehouse-style analysis and through HBase for column-oriented lookups without moving data between systems.","Term-frequency and influence-ranking tasks can be run with MapReduce or Spark over HDFS files, while graph tasks such as follower-community discovery use GraphX, and the choice per task is guided by measured execution time.","If the scalability observation holds, increasing the dataset volume should not require redesigning the pipeline; the same polyglot cycle of ingestion, storage, and evaluation can be repeated with larger collections.","The cyclical methodology gives a template for future benchmark studies: store data in multiple engines, run the same logical query on each, and compare average execution times to decide engine assignment.","The measured difference between Hive and HBase on the influence query suggests that for SQL-like aggregation over this workload, the warehouse engine should be preferred, and this kind of per-task comparison is the practical output of the approach."],"supporting_citations":[{"why":"Supplies the polyglot persistence concept that motivates combining multiple data stores.","marker":"[Sadalage and Fowler 2013]"},{"why":"Defines the MapReduce model that the term-counting job and Hadoop processing layer are built on.","marker":"[Dean and Ghemawat 2004]"},{"why":"Provides the Google File System architecture that HDFS replicates for distributed storage.","marker":"[Ghemawat et al. 2003]"},{"why":"Introduces Resilient Distributed Datasets, the in-memory abstraction credited for Spark's speed over MapReduce.","marker":"[Zaharia et al. 2012]"},{"why":"Cited to support the claim that Spark's in-memory buffers make it faster than Hadoop MapReduce.","marker":"[Aziz et al. 2018]"},{"why":"Cited for quantitative performance gains of Spark over Hadoop in disk and memory-intensive processing.","marker":"[Garg and Kaur 2019]"},{"why":"Frames polyglot persistence as an emerging data-management approach and provides an overview on which the paper builds.","marker":"[Glake et al. 2022]"}],"fun_headline_variants":["Polyglot Hadoop-Spark stack scales: 5M tweets faster than 500K","Mixed engines speed social queries with data growth","Hadoop-Spark polyglot pipeline beats scale barrier","5M tweets process faster than 500K in polyglot stack","Social data scales: polyglot Hadoop-Spark outperforms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scalability conclusion rests on comparing the 500K and 5M tweet runs as if they differ only in dataset volume; if the larger run benefited from a warmed-up cluster, cached data, different job plans, or random variation, the drop in execution time would not prove the system scales.","fun_headline_variants_meta":{"raw":{"variants":["Polyglot Hadoop-Spark stack scales: 5M tweets faster than 500K","Mixed engines speed social queries with data growth","Hadoop-Spark polyglot pipeline beats scale barrier","5M tweets process faster than 500K in polyglot stack","Social data scales: polyglot Hadoop-Spark outperforms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000142,"raw_usage":{"total_tokens":1107,"prompt_tokens":825,"completion_tokens":282,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":193}},"tokens_in":441,"tokens_out":282,"duration_ms":2977,"temperature":1.0,"reasoning_tokens":193,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:51:32.254847+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same three tasks on fresh clusters at 1M, 2M, and 10M tweets, recording per-run times and variance; if execution time rises at any step or varies widely across identical runs, the claimed scalability is not supported.","supporting_citations":[{"cited_title":"and Kaur, D","cited_arxiv_id":null,"evidence_quote":"Cited for quantitative performance gains of Spark over Hadoop in disk and memory-intensive processing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the polyglot persistence concept that motivates combining multiple data stores."},{"cited_title":"and Ghemawat, S","cited_arxiv_id":null,"evidence_quote":"Defines the MapReduce model that the term-counting job and Hadoop processing layer are built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Google File System architecture that HDFS replicates for distributed storage."},{"cited_title":"J., Shenker, S., and Stoica, I","cited_arxiv_id":null,"evidence_quote":"Introduces Resilient Distributed Datasets, the in-memory abstraction credited for Spark's speed over MapReduce."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cited to support the claim that Spark's in-memory buffers make it faster than Hadoop MapReduce."}],"review_version":1}