{"id":"6a45fd19-d97e-415a-ae48-afca34f4310f","arxiv_id":"1909.02203","paper_version":5,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The full text proposes Elastic HH, a heavy-hitter-only variant of the Elastic sketch that drops the light part, uses lambda=1 eviction, and reports lower error and 2.5x speedup on one CAIDA trace.","lead":"The abstract and title describe a '2FA Sketch' with a theoretically derived optimal lambda, but the full text contains a different paper, 'Elastic HH', which simplifies the Elastic sketch by dropping its light part and setting the eviction threshold to 1. The reported gain is 5.7 to 8.1 times lower error and 2.5 times faster than Elastic on a CAIDA trace, but the comparison is confounded by memory allocation and the key parameter is fitted, not derived.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 5.7-8.1x error reduction is confounded by unequal heavy-part memory: with total memory fixed at 300KB, Elastic HH devotes all 300KB to its heavy part while Elastic devotes only 225KB, so the improvement may be a memory artifact rather than an algorithmic gain.","rationale":"The reader's weakest_assumption already flags both the lambda selection and the memory allocation confound. I focus on the memory confound because it is the most load-bearing for the paper's quantitative headline: it directly undermines the causal attribution of the 5.7-8.1x error reduction to the algorithmic change. A controlled comparison with equal heavy-part memory would settle whether the claim survives. The abstract/body mismatch about 2FA Sketch versus Elastic HH is a serious coherence problem but is secondary to the empirical confound for the strongest claim as stated in the full text. Since the confound lands and the mismatch remains, the reader's REJECT verdict is preserved; no adjustment is needed.","tokens_in":8092,"tokens_out":8383,"duration_ms":82654,"concrete_test":"Rerun Figure 3 on the same CAIDA trace with Elastic total memory = 400KB (heavy part 300KB, light part 100KB) versus Elastic HH at 300KB, so the heavy-part budgets are equal. If Elastic's AAE/ARE are still 5.7-8.1x worse than Elastic HH, the algorithmic change is supported; if the gap collapses, the headline improvement is due to giving Elastic HH more heavy-part memory. As a secondary check, sweep lambda on a second trace (e.g., MAWI) and report means with error bars over multiple runs to test the generality of lambda=1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-A fixes total memory at 300KB and sets the Elastic heavy:light ratio to 3:1, while Elastic HH (Section III-A) keeps only the heavy part. Hence Elastic's heavy part gets 225KB and Elastic HH's gets 300KB, a 33% capacity difference in exactly the component whose collision rate determines AAE/ARE in these bucketized sketches. The claimed 7.3-8.1x smaller AAE and 5.7-7.5x smaller ARE are therefore not cleanly attributable to the modified eviction rule and new-flow size initialization. Section III-C picks lambda=1 on the same CAIDA trace used in Section IV-B, so the parameter is fitted to the test workload rather than independently justified, and no error bars are reported for the accuracy metrics (throughput alone is repeated 100 times, Section IV-A). The abstract additionally promises a 'theoretically derived' optimal lambda for a '2FA Sketch' that does not appear in the full text; the body only gives an empirical justification. The most direct threat to the quantitative headline is the memory confound, which a controlled comparison can settle.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Elastic HH, a specialization of the Elastic sketch for the single task of heavy-hitter detection. The design drops the light part of Elastic and changes the eviction rule inside the heavy part: when a bucket is full and an incoming flow is not stored, the smallest flow is evicted once the bucket's negative-vote counter exceeds the smallest flow's positive votes, and the incoming flow is initialized with the evicted flow's count plus one. Experiments on a CAIDA trace report that Elastic HH reduces AAE by 7.3-8.1x and ARE by 5.7-7.5x compared with Elastic, while being about 2.5x faster. The abstract supplied with the submission, however, describes a \"2FA Sketch\" with a theoretically derived optimal lambda and different performance numbers (2.5-19.7x error reduction and 1.03x speedup), and that content does not appear in the full text.","tokens_in":8328,"tokens_out":5742,"duration_ms":59845,"significance":"If the claimed improvements were established under controlled comparisons, the paper would make a practically useful point: specializing a general-purpose sketch to heavy-hitter detection and simplifying its eviction logic is a plausible way to gain accuracy and speed, and the availability of source code on Github is a concrete strength. The reported F1/PR/RR results and the use of multiple baselines are also in the paper's favor. However, the central quantitative claim is currently confounded by unequal memory allocation between Elastic HH and Elastic, and the key parameter lambda is selected on the same workload used to report the headline accuracy results. These issues prevent the significance from being assessed on the present evidence.","major_comments":[{"comment":"The headline accuracy comparison is confounded by unequal heavy-part memory. Section IV-A fixes total memory at 300KB for all algorithms and sets Elastic's heavy:light ratio to 3:1, so Elastic's heavy part receives 225KB while Elastic HH, which has no light part, receives 300KB for its only (heavy) part. Since AAE and ARE in these bucketized sketches are governed largely by collisions in the heavy part, the claimed 7.3-8.1x smaller AAE and 5.7-7.5x smaller ARE in Section IV-B are not cleanly attributable to the algorithmic changes. A controlled comparison should give the two sketches equal heavy-part memory (for example, run Elastic with 300KB heavy plus 100KB light, or run Elastic HH with 225KB total) and report both perspectives.","section":"Section IV-A and IV-B"},{"comment":"The choice lambda=1 is parameter fitting rather than a validated design decision. Section III-C justifies lambda=1 by \"our experimental results on different datasets,\" and Fig. 4 shows the accuracy comparison across lambda values, but the only dataset described in Section IV-A is the CAIDA Equinix-Chicago trace, which is also the dataset used for the main accuracy results in Section IV-B. The abstract further promises a \"theoretically derived optimal lambda,\" but no such derivation appears in the body. Please provide either an analytic derivation or an out-of-sample validation on held-out traces with distinct traffic characteristics, and report variance across runs.","section":"Section III-C and Fig. 4"},{"comment":"The eviction condition is stated inconsistently. Section I says replacement is activated when vote- > vote+_min + 1, while Section III-B defines the rule as vote- > vote+_min with lambda=1, and the worked example in Section III-B follows the latter (vote- becomes 12 and evicts a flow with vote+=11). The text also says \"vote+_min is incremented by 1\" and then the example sets the new flow's size to vote-, which equals vote+_min+1. Because this rule is the core algorithmic contribution, the exact condition and the initial counter value of the inserted flow must be stated unambiguously and must match the implementation used in the experiments.","section":"Section I vs. Section III-B"},{"comment":"The submitted abstract and the full text describe different artifacts. The abstract promises a \"2FA Sketch\" with dual-layer protection, an improved Arbitration strategy, a cross-bucket Avoidance hashing scheme, and a theoretically derived optimal lambda, and it reports error reduction of 2.5-19.7x and speedup of 1.03x. The full text is titled \"Elastic HH,\" contains no \"2FA\" nomenclature, no Arbitration/Avoidance scheme, no theoretical derivation, and reports 5.7-8.1x error reduction and 2.5x speedup. The title, abstract, and body must be aligned before the paper can be evaluated as a coherent submission.","section":"Abstract and title"}],"minor_comments":[{"comment":"Only the throughput experiments are described as repeated 100 times; the accuracy metrics in Figs. 3-5 are reported as single curves with no error bars or run-to-run variability. Please add repetitions for the accuracy metrics or justify why they are deterministic.","section":"Section IV-A"},{"comment":"The phrase \"on different datasets\" is not supported by the experimental setup, which names only the CAIDA Equinix-Chicago trace. Please specify how many traces were used and give their traffic characteristics.","section":"Section III-C"},{"comment":"The query set Phi in the AAE and ARE definitions is not specified. Clarify which flows are included in Phi and how they are selected from the stream.","section":"Section IV-A, metrics"},{"comment":"The label \"F1 socre\" contains a typo; it should read \"F1 score.\"","section":"Figure 5(c)"},{"comment":"In the Elastic sketch example, the text says no eviction occurs when vote-/vote+ < lambda and eviction occurs when the ratio \"equals\" lambda; the boundary condition should be stated consistently with the original Elastic reference.","section":"Section II-A"},{"comment":"Reference [1] formats the Github repository name as \"Elastic HH\" with a space, which will not resolve as a URL; please provide the correct repository path.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The discrepancy between the supplied abstract and the full text is substantial enough that I would ask the editor to verify the submission metadata; if the \"2FA Sketch\" abstract is the official abstract, the manuscript is not the artifact it describes. The memory-confounding issue in Section IV-A is the most serious technical problem and will require a new experimental campaign, but it is fixable within the scope of a major revision. I would not recommend rejection solely on the grounds of disagreement with the Elastic sketch baseline; the paper's direction is reasonable, but the current evidence does not support the quantitative claims as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the abstract and title describe a '2FA Sketch' with dual-layer protection, arbitration, avoidance hashing, and a theoretically derived lambda. The full text is a different paper about 'Elastic HH', which simply removes the light part of Elastic, sets lambda=1 empirically, and initializes new flows to min+1. That mismatch alone is disqualifying for a submission as a coherent record.\n\nWhat the full text does well: it identifies a reasonable specialization of Elastic for heavy hitter detection, evaluates against five baselines on a CAIDA trace, and ships code. The basic idea—spend all memory on the heavy part and evict more aggressively—is plausible and might help practitioners.\n\nThe soft spots, in order: (1) The headline error reduction is confounded by memory allocation. Elastic uses a 3:1 heavy:light ratio, so with 300KB total, its heavy part gets 225KB while Elastic HH gets the full 300KB. A controlled comparison would give both the same heavy-part memory. (2) Lambda=1 is chosen on the same dataset used for the main results, so the parameter is fitted, not validated. No error bars, single trace. (3) The eviction condition is written as vote− > min+1 in Section I but as vote− > min in Section III-B; the example follows the latter. Minor, but sloppy. (4) The abstract promises a theoretical derivation; the paper gives only heuristics.\n\nNone of this means the idea is worthless. Removing the light part and using min+1 initialization is a sensible tweak. But the reported 5.7–8.1x error reduction cannot be trusted as an algorithmic gain given the memory confound, and the mismatch between the advertised and actual algorithm is fatal for the current submission. I'd reconsider a revised version that fixes the abstract, adds a controlled memory comparison, and validates lambda on multiple traces. As it stands, it deserves a desk reject rather than referee time.","headline":"The abstract and title describe a different algorithm than the full text, and once you set that aside, the empirical claim is a small tweak confounded by unequal memory allocation.","tokens_in":8874,"tokens_out":3473,"would_cite":false,"duration_ms":33836,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Stripping Elastic down to its heavy part and evicting at $\\lambda=1$ cuts heavy-hitter error by 5.7–8.1 times and speeds processing by about 2.5 times.","keywords":["heavy hitter detection","data streams","Elastic sketch","Elastic HH","stream summarization","eviction strategy","network measurement","top-k flows"],"falsifier":"Run Elastic and Elastic HH on synthetic streams with different levels of skew while giving both sketches the same effective heavy-part memory, and compare AAE and ARE; if any skew value makes Elastic HH no better than Elastic, the claimed generalization of $\\lambda=1$ fails. Alternatively, give Elastic a 25 percent larger total memory so its heavy part matches Elastic HH's, and check whether the reported 5.7 to 8.1 times error reduction persists.","tokens_in":7877,"feed_emoji":"⚡","tokens_out":9504,"duration_ms":81294,"temperature":0.7,"pith_summary":"This paper argues that a general-purpose streaming sketch can be made much better at one specific task, finding heavy hitters, by tailoring the data structure to that task. The authors take the Elastic sketch, discard its light part, and change the bucket-eviction rule: replace the smallest flow as soon as a bucket's negative-vote count exceeds that flow's positive votes, and give the incoming flow a starting count of one more than the evicted flow's count. On a public Internet trace, this tailored sketch, called Elastic HH, reports 5.7 to 8.1 times smaller average absolute and relative error than the standard Elastic sketch and processes about 2.5 times more packets per second. The gain is purchased by giving up Elastic's generality: Elastic HH only answers heavy-hitter queries.","feed_headline":"One eviction rule cuts heavy-hitter sketch error 5.7–8.1x","feed_subtitle":"Dropping Elastic's light part and evicting at λ=1 also speeds packet processing by about 2.5 times.","key_machinery":"The central mechanism is the eviction rule change in a fixed-size bucket of flow counters: instead of Elastic's ratio test $vote^-/vote^+_{min} > \\lambda$ with $\\lambda=8$, Elastic HH uses $vote^- > vote^+_{min}$ and sets the new flow's initial size to $vote^+_{min}+1$. This is equivalent to $\\lambda=1$, makes the replacement decision a single integer comparison, and lets the discarded flow's count carry over into the incoming flow so the new counter does not start from zero. The second piece of machinery is dropping the light part, so the entire memory budget goes to the heavy buckets and each insertion touches only one contiguous bucket.","core_discovery":"The central claim is that the Elastic sketch over-invests in generality, and that this generality actively hurts heavy-hitter detection. If the light part is removed and the heavy part alone is kept, with eviction triggered when the shared negative-vote counter exceeds the smallest positive vote and the new flow's counter initialized to that minimum plus one, each bucket behaves like a small top-k tracker whose estimates are closer to true flow sizes. The paper reports that on real traffic, compared with the standard Elastic sketch at the same total memory, Elastic HH reduces average absolute error by 7.3 to 8.1 times and average relative error by 5.7 to 7.5 times, while raising throughput by roughly 2.5 times; precision, recall, and F1 score reach almost 100 percent even at small memory.","pith_inferences":["The replace-with-min-plus-one rule makes Elastic HH structurally similar to classic top-k frequent-item algorithms, so it may inherit a formal error bound if the comparison is drawn out explicitly.","The same tailoring recipe, removing the general-purpose companion structure and tightening the eviction threshold, could be tested on other multi-purpose sketches to produce task-specific variants for heavy change or cardinality detection.","The speed advantage depends on one-bucket cache-friendly access, so re-measuring on programmable switch hardware or with packet reordering would show whether the 2.5 times throughput gain survives outside a software CPU setting.","Because $\\lambda=1$ is validated on a single trace, the paper's parameter-free claim could be probed by sweeping synthetic flow-size distributions with different skew and checking whether accuracy stays near-optimal."],"forward_implications":["At equal memory, heavy-hitter detection error drops by roughly 5.7 to 8.1 times compared with the standard Elastic sketch.","Throughput is about 2.5 times higher than Elastic on the tested CPU, and more than ten times higher than the other compared sketches when SIMD instructions are used.","Precision, recall, and F1 score reach nearly 100 percent on the tested trace even with a 100 KB memory budget.","The accuracy and speed advantages are specific to heavy-hitter queries; other measurement tasks supported by Elastic are no longer served by the tailored structure.","Setting $\\lambda=1$ is reported as near-optimal and costs no extra computation per insertion."],"supporting_citations":[{"why":"Defines the Elastic sketch used as the baseline and supplies the bucket parameters (7 flows per bucket) and the 3:1 heavy-to-light memory ratio used in the experiments.","marker":"[23]"},{"why":"Provides the public Internet traffic trace on which $\\lambda$ is tuned to 1 and on which all reported accuracy and throughput numbers are measured.","marker":"[2]"},{"why":"Supplies the Space-Saving baseline against which Elastic HH is compared on precision, recall, F1 score, and throughput.","marker":"[21]"},{"why":"Supplies the Count sketch plus min-heap baseline used in the AAE, ARE, and CDF accuracy comparisons.","marker":"[6]"},{"why":"Supplies the Count-Min sketch plus min-heap baseline used in the AAE, ARE, and CDF accuracy comparisons.","marker":"[7]"}],"fun_headline_variants":["Dual-layer sketch: heavy hitters found with 19.7x less error","Heavy hitter detection gets 2FA: one rule, 5.7–8.1x less error","Sketch upgrade: drop light part, cut heavy-hitter error 19.7x","Optimal eviction rule cuts heavy-hitter sketch error 7x","Two-factor sketch finds heavy hitters 19.7x more precisely"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's error-reduction numbers assume both that $\\lambda=1$ is near-optimal on unseen traffic patterns, which is tested only on one public trace, and that comparing at equal total memory is fair even though the standard Elastic spends 25 percent of that memory on its light part; weakening either assumption shrinks the reported gains.","fun_headline_variants_meta":{"raw":{"variants":["Dual-layer sketch: heavy hitters found with 19.7x less error","Heavy hitter detection gets 2FA: one rule, 5.7–8.1x less error","Sketch upgrade: drop light part, cut heavy-hitter error 19.7x","Optimal eviction rule cuts heavy-hitter sketch error 7x","Two-factor sketch finds heavy hitters 19.7x more precisely"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001596,"raw_usage":{"total_tokens":6335,"prompt_tokens":896,"completion_tokens":5439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":5327}},"tokens_in":512,"tokens_out":5439,"duration_ms":41949,"temperature":1.0,"reasoning_tokens":5327,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:56:48.547227+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Elastic and Elastic HH on synthetic streams with different levels of skew while giving both sketches the same effective heavy-part memory, and compare AAE and ARE; if any skew value makes Elastic HH no better than Elastic, the claimed generalization of $\\lambda=1$ fails. Alternatively, give Elastic a 25 percent larger total memory so its heavy part matches Elastic HH's, and check whether the reported 5.7 to 8.1 times error reduction persists.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Elastic sketch used as the baseline and supplies the bucket parameters (7 flows per bucket) and the 3:1 heavy-to-light memory ratio used in the experiments."},{"cited_title":"http://www.caida.org/data/ overview/","cited_arxiv_id":null,"evidence_quote":"Provides the public Internet traffic trace on which $\\lambda$ is tuned to 1 and on which all reported accuracy and throughput numbers are measured."},{"cited_title":"Metwally, D","cited_arxiv_id":null,"evidence_quote":"Supplies the Space-Saving baseline against which Elastic HH is compared on precision, recall, F1 score, and throughput."},{"cited_title":"Charikar, K","cited_arxiv_id":null,"evidence_quote":"Supplies the Count sketch plus min-heap baseline used in the AAE, ARE, and CDF accuracy comparisons."},{"cited_title":"Cormode and S","cited_arxiv_id":null,"evidence_quote":"Supplies the Count-Min sketch plus min-heap baseline used in the AAE, ARE, and CDF accuracy comparisons."}],"review_version":1}