Pith. sign in

REVIEW 3 major objections 3 minor 63 references

Vectorized Sequence-Based Chunking for Data Deduplication

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SeqCDC achieves roughly 10x higher throughput than unaccelerated chunking and 1.25–1.35x over vector-accelerated chunking while keeping space savings within a few percent of the best alternatives.

desk verdict The vectorization angle is new, but the paper's stated boundary rule cannot produce its reported chunk sizes, so the whole evaluation needs an explanation before the results can be trusted. read the letter →

arxiv 2505.21194 v1 pith:KLBM6MAV submitted 2025-05-27 cs.DC

Sreeharsha Udayashankar , Samer Al-Kiswany This is my paper
classification cs.DC
keywords datadeduplicationcontent-definedchunkingSIMDvectorizationAVX-512throughputstoragesystemsbyte-ordersequences
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SeqCDC is a content-defined chunking algorithm for data deduplication that tries to make chunking throughput scale with chunk size. It detects chunk boundaries from fixed-length runs of monotonically increasing or decreasing byte values, skips over regions whose byte order opposes the target run, and accelerates both phases with SSE/AVX vector instructions. The paper reports roughly 10x higher throughput than unaccelerated chunking algorithms and 1.25–1.35x higher than vector-accelerated ones at the 8–16 KB chunk sizes production deduplication favors, with space savings within a few percent of the best alternatives. If true, this removes a throughput bottleneck in the deduplication pipeline without sacrificing deduplication ratio.

What carries the argument

The load-bearing object is the boundary-triggering sequence: a run of SeqLength consecutive bytes whose values are monotonically increasing in Increasing mode or decreasing in Decreasing mode, with a chunk boundary placed immediately after the sequence ends. Two auxiliary parameters regulate how much data the scanner skips: SkipTrigger, the number of opposing byte pairs that marks a region as unfavorable, and SkipSize, the number of bytes jumped over. Under SIMD, the scanner loads SeqLength shifted copies of a 64-byte block into vector registers, performs pairwise greater-than comparisons, and ANDs the resulting masks so a single non-zero mask reports every candidate boundary at once; a separate comparison mask counts opposing pairs that trigger content-defined skipping. This turns per-byte boundary judgement into a handful of vector comparisons per 64 bytes, which is what lets throughput rise with chunk size.

What would settle it

Run SeqCDC on a low-entropy workload such as a directory of compressed logs or sparse VM images, measure the chunk-size distribution and space savings against a Rabin-based CDC, and look for average chunk sizes far above the configured target or space savings more than a few points lower; that would falsify the claim that content-defined skipping minimally impacts deduplication efficiency.

Watch

Extended reading notes

Core claim

The central claim is that sequence-based boundary detection plus content-defined skipping plus SIMD vectorization together make chunking throughput grow with target chunk size, instead of staying flat as in prior CDC algorithms. SeqCDC treats each byte as a value and declares a chunk boundary at the end of the first fixed-length sequence of SeqLength bytes that is monotone in the chosen direction. Because most bytes scanned at large chunk sizes do not form boundaries, SeqCDC skips unfavorable regions, where byte order opposes the chosen direction, after SkipTrigger opposing pairs are seen, using a SkipSize of 256–512 bytes to limit missed boundaries. The paper argues this preserves byte-shifting resistance for ordinary edits, and its measurements show SeqCDC either beats or comes within 6% of the best space savings on five real datasets across 4, 8, and 16 KB chunks.

Load-bearing premise

The parameter settings for skipping and sequence length were tuned on randomized data plus one dataset and then applied unchanged to all datasets, so the claimed speedups and minimal space-savings impact depend on those settings remaining appropriate for workloads with different byte statistics.

Editorial extensions

If this is right

  • At 16 KB chunks, the AVX-512 version reaches about 30.5 GB/s, roughly 10x the fastest non-vector alternative evaluated.
  • Unaccelerated SeqCDC reaches 5.3–5.4 GB/s at 8 KB and 9.9–10 GB/s at 16 KB, beating the fastest prior native algorithm by 1.5x and 2.8x at those sizes.
  • Space savings stay within 6% of the best tested CDC algorithm on every dataset, and SeqCDC is the best or tied at some configurations.
  • Because vector acceleration does not change the chunk boundaries, SSE-128 and AVX-256 versions preserve space savings and retain most of the speedup on CPUs without AVX-512.
  • Throughput scales with chunk size, so deduplication systems can move to larger chunks to shrink fingerprint databases without paying a chunking-speed penalty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: the parameter values in Table I were chosen on randomized data plus the DEB dataset, so workloads with very different byte statistics, such as low-entropy logs or encrypted streams, may need per-workload parameter search; too-aggressive skipping would inflate chunk sizes and erode space savings.
  • Extension: because boundary detection is a pure function of local byte order, an adversarial or pathological input could be engineered to suppress boundary sequences for long stretches and force maximum-size chunks, a worst case the paper acknowledges for all CDC algorithms but does not quantify for SeqCDC.
  • Extension: the vectorized shifted-comparison pattern could be reused outside deduplication, for example in compression or delta-encoding systems that need to find runs or turning points quickly.
  • Extension: the reported results are single-threaded, so combining SeqCDC with existing multithreaded chunking or locality-based duplicate-skip optimizations would likely compound the gains, though that combination is not evaluated here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper presents SeqCDC, a content-defined chunking algorithm that detects chunk boundaries via fixed-length monotonically increasing/decreasing byte sequences, uses content-based skipping and sub-minimum skipping, and accelerates processing with SSE/AVX vector instructions. The authors evaluate SeqCDC against seven unaccelerated and three vector-accelerated CDC algorithms on five datasets, reporting throughput improvements of 10x over unaccelerated algorithms and 1.2x-1.35x over vector-accelerated algorithms at 8-16 KB chunk sizes, with deduplication space savings within 6% of the best alternatives. The paper also includes a chunk size distribution analysis and a discussion of byte-shift resistance.

Significance. If the reported results are correct, SeqCDC would be a valuable contribution: it offers a hashless, vector-friendly chunking method that scales throughput with chunk size, unlike existing CDC algorithms. The work builds on and extends the authors' earlier VectorCDC and SeqCDC papers, and the code is released with DedupBench. The central idea of using simple monotonicity tests instead of hashing or extrema searches is elegant and potentially practical. However, the internal inconsistency identified below in the boundary-detection parameters casts doubt on whether the implementation matches the description, so the significance cannot be assessed without resolving that issue.

major comments (3)
  1. [§III-A, Table I, §V, §VI-E] The described boundary condition is a run of SeqLength strictly increasing bytes, and Table I sets SeqLength=5 for all chunk sizes. For i.i.d. uniform random bytes, the probability that a given position starts such a run is 1/5! = 1/120. Even with the sub-minimum skip (25-50% of average chunk size) and content-defined skipping (SkipTrigger=50-55 opposing pairs, SkipSize=256-512 bytes), the expected chunk size on random data would be roughly the minimum chunk size plus a few hundred bytes, far below the 4-16 KB targets. For example, for the 8 KB target with a 4 KB minimum, the expected chunk size would be about 4.6 KB, yet Figure 11 and Table III report chunk size distributions centered near the nominal sizes. The Monte-Carlo calibration in §V would have immediately contradicted these parameters if it used the described rule. This suggests that the implementation uses a different boundary condition or different parameters than the paper states, or the calibration is erroneous. Consequently, the reported throughput and space-savings numbers cannot be reproduced from the manuscript's description, and this must be resolved before the central claims can be accepted.
  2. [Abstract, §I, §VI-C] The abstract claims 15x higher throughput over unaccelerated algorithms, while §I and §VI-C report 10x, and the data in Figure 8b support about 10x (30.5 GB/s vs. approximately 3 GB/s for FCDC). This inconsistency in the headline speedup figure should be corrected.
  3. [§V, §VI] The parameter values for SeqLength, SkipTrigger, and SkipSize are chosen by a search on the DEB dataset (§V), and DEB is then used as one of the five evaluation datasets (§VI, Table II). The DEB results are therefore not an independent test of the parameter configuration; the paper should either exclude DEB from the main evaluation or justify why the tuning does not materially affect the reported comparisons.
minor comments (3)
  1. [Abstract and Section I] The typographical spacing in 'SSE/A VX' should read 'SSE/AVX'.
  2. [Section V] The sentence 'We will release this micro benchmark with the final version of our paper' indicates that an artifact is incomplete; please include it or note its availability at submission.
  3. [Section VI-E] The claim that 'SeqCDC exhibits a chunk size distribution similar to hash-based algorithms' needs to be re-examined after correcting the SeqLength parameter or the boundary detection description, given the inconsistency raised in the major comments.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: SeqCDC's results are direct measurements against external datasets and baselines, and the tuned parameters are implementation settings, not disguised predictions.

full rationale

I walked the paper's derivation chain and found no step in which a claimed prediction reduces by construction to an input, a fitted parameter, or a self-citation. The SeqLength, SkipTrigger, and SkipSize parameters are obtained by Monte-Carlo simulation and a search on the DEB dataset, but the paper's throughput and space-savings claims are reported measurements on five real-world datasets against external baseline algorithms; they are not derived quantities forced by those parameter fits. The vector-acceleration claims are measured with SSE/AVX implementations and compared to independently implemented baselines. Self-citations to the authors' prior VectorCDC and SeqCDC work are used as background and comparison points, not as load-bearing justification for the central result; in fact, the paper explicitly argues that VectorCDC's approach is incompatible with SeqCDC's boundary detection. The only notable concern I found is a potential internal inconsistency: the described SeqLength=5 boundary rule may not be able to produce the reported chunk-size distributions on random data, making Table I and Figure 11 hard to reconcile. That is a reproducibility or correctness issue, not a circularity issue, so it does not raise the circularity score. Overall, the paper's evaluation is self-contained with respect to external benchmarks, and no circular derivation chain is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on parameter tuning and generalization assumptions. SeqLength, SkipTrigger, and SkipSize are fitted values. No invented entities are introduced. The main risk is that the one-time parameter selection on DEB, plus qualitative assumptions about byte-shifting behavior, do not transfer to all production workloads.

free parameters (3)
  • SeqLength = 5 (for 4/8/16 KB)
    Length of the monotonic byte run that triggers a chunk boundary; tuned via Monte-Carlo simulation and the DEB dataset in Section V, Table I.
  • SkipTrigger = 55 at 4 KB, 50 at 8/16 KB
    Number of opposing byte pairs that triggers a content-defined skip; tuned per chunk size in Section V, Table I.
  • SkipSize = 256 bytes at 4/8 KB, 512 bytes at 16 KB
    Number of bytes skipped after a trigger; larger values improve throughput but may reduce space savings; tuned per chunk size in Section V, Table I.
assumptions (4)
  • domain assumption Randomized data streams are representative for tuning chunk-size parameters.
    Section V uses Monte-Carlo simulations on randomized data to find parameter combinations for each average chunk size; if real data has different byte-order statistics, the tuned parameters may not produce the intended chunk sizes.
  • domain assumption Real workloads have limited byte-shifting, so content-defined skipping rarely hides or reveals boundaries.
    Section IV argues this qualitatively and reports only indirect evidence via space savings; no direct measurement of boundary movement is provided.
  • domain assumption Parameters chosen on DEB generalize to other datasets.
    Section V selects the best combination on DEB and uses it for DEV, LNX, RDS, and TPCC; cross-dataset results support but do not prove generalization to arbitrary workloads.
  • domain assumption Monotonic byte sequences occur frequently enough in tested data to keep chunks near the target average size.
    The boundary detection mechanism depends on such sequences; the chunk-size distributions in Section VI-E suggest this holds for these datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vectorized Sequence-Based Chunking for Data Deduplication." pith.science (2026). https://pith.science/paper/KLBM6MAV

@misc{pith2026250521194,
  author       = {Pith},
  title        = {Pith review of: Vectorized Sequence-Based Chunking for Data Deduplication},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KLBM6MAV}},
  note         = {Machine review of arXiv:2505.21194}
}
read the original abstract

Data deduplication has gained wide acclaim as a mechanism to improve storage efficiency and conserve network bandwidth. Its most critical phase, data chunking, is responsible for the overall space savings achieved via the deduplication process. However, modern data chunking algorithms are slow and compute-intensive because they scan large amounts of data while simultaneously making data-driven boundary decisions. We present SeqCDC, a novel chunking algorithm that leverages lightweight boundary detection, content-defined skipping, and SSE/AVX acceleration to improve chunking throughput for large chunk sizes. Our evaluation shows that SeqCDC achieves 15x higher throughput than unaccelerated and 1.2x-1.35x higher throughput than vector-accelerated data chunking algorithms while minimally affecting deduplication space savings.

Figures

Figures reproduced from arXiv: 2505.21194 by the authors.

Figure 1
Figure 1. Chunking throughput on randomized data the window. By repeating this across the entire file, they divide it into data chunks. Each CDC algorithm has parameters that can be tuned to change the average size of generated chunks. CDC algorithms can be classified into hash-based and hashless algorithms [27]. Hash-based chunking algorithms, such as Rabin’s chunk￾ing [15], insert chunk boundaries only when the hash value o… view at source ↗
Figure 2
Figure 2. An example of a chunk generated by SeqCDC [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Handling byte-shifts with SeqCDC boundaries. Byte shifting can span one or more bytes and take the form of insertions or deletions. In general, sub-minimum and content-defined skipped regions affect SeqCDC’s byte￾shifting resistance [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: SeqCDC’s Space Savings vs Chunk Size across datasets [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Space Savings with 8KB chunks. Note that [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 8
Figure 8. Figure 8: shows the throughput of native algorithms with their respective AVX-512 accelerated versions; CRC and Gear accelerated with SS-CDC [18], RAM accelerated with Vector￾CDC [25] and SeqCDC accelerated with the method described 0 0.4 0.8 1.2 1.6 GB/s (a) CRC and Gear with S…
Figure 9
Figure 9. Figure 9: Chunking throughput of vector accelerated CDC algo [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: SeqCDC’s throughput breakdown at 16 KB with native [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 12
Figure 12. Figure 12: Chunking Throughput on DEB with SSE-128, AVX-256 and AVX-512 Instructions families, further cementing its effectiveness and backward compatibility. E. Chunk Size Distribution CDC algorithms are expected to achieve a uniform chunk size distribution, loosely centered ar…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 63 canonical work pages

  1. [1]

    V olume of data/information created, captured, copied, and consumed worldwide from 2010 to 2025,

    A. Holst, “V olume of data/information created, captured, copied, and consumed worldwide from 2010 to 2025,” Statista, June, 2021

    work page 2010

  2. [2]

    The Google File System,

    S. Ghemawat, H. Gobioff, and S.-T. Leung, “The Google File System,” SIGOPS Operating Systems Review , vol. 37, p. 29–43, oct 2003

    work page 2003

  3. [3]

    The Hadoop Distributed File System,

    K. Shvachko, H. Kuang, S. Radia, and R. Chansler, “The Hadoop Distributed File System,” in 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) , pp. 1–10, 2010

    work page 2010

  4. [4]

    RAID: High-performance, reliable secondary storage,

    P. M. Chen, E. K. Lee, G. A. Gibson, R. H. Katz, and D. A. Patterson, “RAID: High-performance, reliable secondary storage,” ACM Computing Surveys (CSUR) , vol. 26, no. 2, pp. 145–185, 1994

    work page 1994

  5. [5]

    Network attached storage architecture,

    G. A. Gibson and R. Van Meter, “Network attached storage architecture,” Communications of the ACM , vol. 43, no. 11, pp. 37–45, 2000

    work page 2000

  6. [6]

    Data compression,

    D. A. Lelewer and D. S. Hirschberg, “Data compression,” ACM Com- puting Surveys (CSUR) , vol. 19, no. 3, pp. 261–296, 1987

    work page 1987

  7. [7]

    Salomon, Data compression

    D. Salomon, Data compression. Springer, 2002

    work page 2002

  8. [8]

    A comprehensive study of the past, present, and future of data deduplication,

    W. Xia, H. Jiang, D. Feng, F. Douglis, P. Shilane, Y . Hua, M. Fu, Y . Zhang, and Y . Zhou, “A comprehensive study of the past, present, and future of data deduplication,” Proceedings of the IEEE , vol. 104, no. 9, pp. 1681–1710, 2016

    work page 2016

Show all 63 references
  1. [9]

    Demys- tifying data deduplication,

    N. Mandagere, P. Zhou, M. A. Smith, and S. Uttamchandani, “Demys- tifying data deduplication,” in The ACM/IFIP/USENIX Middleware’08 Conference Companion, pp. 12–17, 2008

    2008

  2. [10]

    A study of practical deduplication,

    D. T. Meyer and W. J. Bolosky, “A study of practical deduplication,” ACM Transactions on Storage (ToS) , vol. 7, no. 4, pp. 1–20, 2012

    2012

  3. [11]

    Characteristics of backup workloads in production systems,

    G. Wallace, F. Douglis, H. Qian, P. Shilane, S. Smaldone, M. Cham- ness, and W. Hsu, “Characteristics of backup workloads in production systems,” in 10th USENIX Conference on File and Storage Technologies (FAST 12), (San Jose, CA), USENIX Association, Feb. 2012

    2012

  4. [12]

    Coyne, S

    L. Coyne, S. Moulton, and C. Alvarez, IBM System Storage N Series Data Compression and Deduplication: Data ONTAP 8.1 Operating in 7-mode. IBM Redbooks, 2012

    2012

  5. [13]

    Venti: A New Approach to Archival Data Storage,

    S. Quinlan and S. Dorward, “Venti: A New Approach to Archival Data Storage,” in Conference on File and Storage Technologies (FAST 02) , (Monterey, CA), USENIX Association, Jan. 2002

    2002

  6. [14]

    Data deduplication techniques,

    Q. He, Z. Li, and X. Zhang, “Data deduplication techniques,” in 2010 international conference on future information technology and management engineering, vol. 1, pp. 430–433, IEEE, 2010

    2010

  7. [15]

    A low-bandwidth network file system,

    A. Muthitacharoen, B. Chen, and D. Mazieres, “A low-bandwidth network file system,” in The eighteenth ACM symposium on Operating systems principles, pp. 174–187, 2001. Dataset CDC 4KB 8KB 16KB AE 41.99% 33.69% 21.94% CRC 41.65% 35.23% 26.59% FCDC 43.83% 36.10% 26.47% GEAR 39.4...

    2001

  8. [16]

    Oceanstore: An architecture for global-scale persistent storage,

    J. Kubiatowicz, D. Bindel, Y . Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S. Rhea, H. Weatherspoon, W. Weimer,et al., “Oceanstore: An architecture for global-scale persistent storage,” ACM SIGOPS Operating Systems Review , vol. 34, no. 5, pp. 190–201, 2000

    2000

  9. [17]

    AE: An asymmetric extremum content defined chunking algorithm for fast and bandwidth-efficient data deduplication,

    Y . Zhang, H. Jiang, D. Feng, W. Xia, M. Fu, F. Huang, and Y . Zhou, “AE: An asymmetric extremum content defined chunking algorithm for fast and bandwidth-efficient data deduplication,” in 2015 IEEE Conference on Computer Communications (INFOCOM) , pp. 1337–1345, IEEE, 2015

    2015

  10. [18]

    SS-CDC: a two-stage parallel content- defined chunking for deduplicating backup storage,

    F. Ni, X. Lin, and S. Jiang, “SS-CDC: a two-stage parallel content- defined chunking for deduplicating backup storage,” in Proceedings of the 12th ACM International Conference on Systems and Storage , SYS- TOR ’19, (New York, NY , USA), p. 86–96, Association for Computing 12 M...

    2019

  11. [19]

    FastCDC: A fast and efficient content-defined chunking approach for data deduplication,

    W. Xia, Y . Zhou, H. Jiang, D. Feng, Y . Hua, Y . Hu, Q. Liu, and Y . Zhang, “FastCDC: A fast and efficient content-defined chunking approach for data deduplication,” in 2016 USENIX Annual Technical Conference (USENIX ATC 16) , pp. 101–114, 2016

    2016

  12. [20]

    Ddelta: A deduplication-inspired fast delta compression approach,

    W. Xia, H. Jiang, D. Feng, L. Tian, M. Fu, and Y . Zhou, “Ddelta: A deduplication-inspired fast delta compression approach,” Performance Evaluation, vol. 79, pp. 258–272, 2014. Special Issue: Performance 2014

    2014

  13. [21]

    A new content-defined chunking algorithm for data deduplication in cloud storage,

    R. N. Widodo, H. Lim, and M. Atiquzzaman, “A new content-defined chunking algorithm for data deduplication in cloud storage,” Future Generation Computer Systems , vol. 71, pp. 145–156, 2017

    2017

  14. [22]

    A framework for analyzing and improving content-based chunking algorithms,

    K. Eshghi and H. K. Tang, “A framework for analyzing and improving content-based chunking algorithms,” Hewlett-Packard Labs Technical Report TR, vol. 30, no. 2005, 2005

    2005

  15. [23]

    Content-dependent chunking for differential compression, the local maximum approach,

    N. Bjørner, A. Blass, and Y . Gurevich, “Content-dependent chunking for differential compression, the local maximum approach,” Journal of Computer and System Sciences , vol. 76, no. 3-4, pp. 154–203, 2010

    2010

  16. [24]

    Verification of a cryptographic primitive: SHA-256,

    A. W. Appel, “Verification of a cryptographic primitive: SHA-256,” ACM Transactions on Programming Languages and Systems (TOPLAS), vol. 37, no. 2, pp. 1–31, 2015

    2015

  17. [25]

    VectorCDC: Accelerat- ing Data Deduplication with Vector Instructions,

    S. Udayashankar, A. Baba, and S. Al-Kiswany, “VectorCDC: Accelerat- ing Data Deduplication with Vector Instructions,” inUSENIX Conference on File and Storage Technologies , 2025

    2025

  18. [26]

    Vector instruction set support for conditional operations,

    J. E. Smith, G. Faanes, and R. Sugumar, “Vector instruction set support for conditional operations,” ACM SIGARCH Computer Architecture News, vol. 28, no. 2, pp. 260–269, 2000

    2000

  19. [27]

    Dedup- bench: A Benchmarking Tool for Data Chunking Techniques,

    A. Liu, A. Baba, S. Udayashankar, and S. Al-Kiswany, “Dedup- bench: A Benchmarking Tool for Data Chunking Techniques,” in 2023 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), pp. 469–474, 2023

    2023

  20. [28]

    The MD5 Message-Digest Algorithm

    R. L. Rivest, “The MD5 Message-Digest Algorithm.” RFC 1321, 1992

    1992

  21. [29]

    The Impact of Low-Entropy on Chunking Techniques for Data Deduplication,

    M. A. Jarah, S. Udayashankar, A. Baba, and S. Al-Kiswany, “The Impact of Low-Entropy on Chunking Techniques for Data Deduplication,” in 2024 IEEE 17th International Conference on Cloud Computing (CLOUD), pp. 134–140, 2024

    2024

  22. [30]

    Effective implementation of matrix-vector multiplication on intel’s avx multicore processor,

    S. A. Hassan, M. M. Mahmoud, A. Hemeida, and M. A. Saber, “Effective implementation of matrix-vector multiplication on intel’s avx multicore processor,” Computer Languages, Systems & Structures , vol. 51, pp. 158–175, 2018

    2018

  23. [31]

    Fast quicksort implementation using A VX instructions,

    S. Gueron and V . Krasnov, “Fast quicksort implementation using A VX instructions,” The Computer Journal , vol. 59, no. 1, pp. 83–90, 2016

    2016

  24. [32]

    Vector LLV A: a virtual vector instruc- tion set for media processing,

    R. L. Bocchino Jr and V . S. Adve, “Vector LLV A: a virtual vector instruc- tion set for media processing,” in Proceedings of the 2nd international conference on Virtual execution environments , pp. 46–56, 2006

    2006

  25. [33]

    A running time improvement for the two thresholds two divisors algorithm,

    T.-S. Moh and B. Chang, “A running time improvement for the two thresholds two divisors algorithm,” in The 48th Annual Southeast Regional Conference, pp. 1–6, 2010

    2010

  26. [34]

    Intel® Intrinsics Guide

    Intel, “Intel® Intrinsics Guide.” https://www.intel.com/content/www/us/ en/docs/intrinsics-guide/index.html

  27. [35]

    C. Z. Mooney, Monte carlo simulation . Sage, 1997

    1997

  28. [36]

    VMWare marketplace

    VMWare, “VMWare marketplace.” https://marketplace.cloud.vmware. com/services, 2023

    2023

  29. [37]

    GitHub - rust-lang/rust: Empowering everyone to build reliable and efficient software

    Rust, “GitHub - rust-lang/rust: Empowering everyone to build reliable and efficient software..” https://github.com/rust-lang/rust, 2023

    2023

  30. [38]

    The Linux Kernel Archives

    Linux, “The Linux Kernel Archives.” https://www.kernel.org/, 2023

    2023

  31. [39]

    Redis, “Redis.” https://redis.io/, 2023

    2023

  32. [40]

    MySQL, “MySQL.” https://www.mysql.com/, 2023

    2023

  33. [41]

    TPC-C Overview

    T. P. Council, “TPC-C Overview.” https://www.tpc.org/tpcc/detail5.asp, 2023

    2023

  34. [42]

    The Design and Operation of CloudLab,

    D. Duplyakin, R. Ricci, A. Maricq, G. Wong, J. Duerig, E. Eide, L. Stoller, M. Hibler, D. Johnson, K. Webb, A. Akella, K. Wang, G. Ricart, L. Landweber, C. Elliott, M. Zink, E. Cecchet, S. Kar, and P. Mishra, “The Design and Operation of CloudLab,” in 2019 USENIX Annual Techni...

    2019

  35. [43]

    VM Images for Dedu- plication

    S. Udayashankar, A. Baba, and S. Al-Kiswany, “VM Images for Dedu- plication.” https://www.kaggle.com/dsv/10561721, 2025

    2025

  36. [44]

    Intel’s Haswell CPU microarchitecture,

    D. Kanter, “Intel’s Haswell CPU microarchitecture,” Real World Tech- nologies, vol. 17, 2012

    2012

  37. [45]

    A VX overhead profiling: How much does your fast code slow you down?,

    M. Gottschlag, T. Schmidt, and F. Bellosa, “A VX overhead profiling: How much does your fast code slow you down?,” in Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems , APSys ’20, (New York, NY , USA), p. 59–66, Association for Computing Machinery, 2020

    2020

  38. [46]

    StoreGPU: Exploiting Graphics Processing Units to Accelerate Distributed Storage Systems,

    S. Al-Kiswany, A. Gharaibeh, E. Santos-Neto, G. Yuan, and M. Ri- peanu, “StoreGPU: Exploiting Graphics Processing Units to Accelerate Distributed Storage Systems,” in The 17th International Symposium on High Performance Distributed Computing , HPDC ’08, (New York, NY , USA), p...

    2008

  39. [47]

    Similarity and Locality Based Indexing for High Performance Data Deduplication,

    W. Xia, H. Jiang, D. Feng, and Y . Hua, “Similarity and Locality Based Indexing for High Performance Data Deduplication,” IEEE Transactions on Computers, vol. 64, no. 4, pp. 1162–1176, 2015

    2015

  40. [48]

    Balancing storage efficiency and data confidentiality with tunable encrypted deduplication,

    J. Li, Z. Yang, Y . Ren, P. P. Lee, and X. Zhang, “Balancing storage efficiency and data confidentiality with tunable encrypted deduplication,” in Proceedings of the Fifteenth European Conference on Computer Systems, pp. 1–15, 2020

    2020

  41. [49]

    Optimizing the Performance of Consistency-Aware Deduplication Using Persistent Memory,

    C. Song, X. Chen, D. Liu, J. Li, Y . Tan, and A. Ren, “Optimizing the Performance of Consistency-Aware Deduplication Using Persistent Memory,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2023

    2023

  42. [50]

    Accelerating restore and garbage collection in deduplication- based backup systems via exploiting historical information,

    M. Fu, D. Feng, Y . Hua, X. He, Z. Chen, W. Xia, F. Huang, and Q. Liu, “Accelerating restore and garbage collection in deduplication- based backup systems via exploiting historical information,” in 2014 USENIX Annual Technical Conference (USENIX ATC 14), (Philadelphia, PA), pp...

    2014

  43. [51]

    Im- proving Restore Performance in Deduplication-Based Backup Systems via a Fine-Grained Defragmentation Approach,

    Y . Tan, B. Wang, J. Wen, Z. Yan, H. Jiang, and W. Srisa-an, “Im- proving Restore Performance in Deduplication-Based Backup Systems via a Fine-Grained Defragmentation Approach,” IEEE Transactions on Parallel and Distributed Systems, vol. 29, no. 10, pp. 2254–2267, 2018

    2018

  44. [52]

    HYDRAstor: A scalable secondary storage.,

    C. Dubnicki, L. Gryz, L. Heldt, M. Kaczmarczyk, W. Kilian, P. Strzel- czak, J. Szczepkowski, C. Ungureanu, and M. Welnicki, “HYDRAstor: A scalable secondary storage.,” in FAST, vol. 9, pp. 197–210, 2009

    2009

  45. [53]

    Extreme Binning: Scalable, parallel deduplication for chunk-based file backup,

    D. Bhagwat, K. Eshghi, D. D. E. Long, and M. Lillibridge, “Extreme Binning: Scalable, parallel deduplication for chunk-based file backup,” in 2009 IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, pp. 1–9, 2009

    2009

  46. [54]

    RapidCDC: Leveraging Duplicate Locality to Accelerate Chunking in CDC-Based deduplication systems,

    F. Ni and S. Jiang, “RapidCDC: Leveraging Duplicate Locality to Accelerate Chunking in CDC-Based deduplication systems,” in The ACM Symposium on Cloud Computing , SoCC ’19, (New York, NY , USA), p. 220–232, Association for Computing Machinery, 2019

    2019

  47. [55]

    QuickCDC: A Quick Content Defined Chunk- ing Algorithm Based on Jumping and Dynamically Adjusting Mask Bits,

    Z. Xu and W. Zhang, “QuickCDC: A Quick Content Defined Chunk- ing Algorithm Based on Jumping and Dynamically Adjusting Mask Bits,” in 2021 IEEE Intl Conf on Parallel & Distributed Pro- cessing with Applications, Big Data & Cloud Computing, Sustain- able Computing & Communicati...

    2021

  48. [56]

    MUCH: Multithreaded Content-Based File Chunking,

    Y . Won, K. Lim, and J. Min, “MUCH: Multithreaded Content-Based File Chunking,” IEEE Transactions on Computers , vol. 64, no. 5, pp. 1375– 1388, 2015

    2015

  49. [57]

    P-dedupe: Exploiting parallelism in data deduplication system,

    W. Xia, H. Jiang, D. Feng, L. Tian, M. Fu, and Z. Wang, “P-dedupe: Exploiting parallelism in data deduplication system,” in 2012 IEEE Seventh International Conference on Networking, Architecture, and Storage, pp. 338–347, IEEE, 2012

    2012

  50. [58]

    MII: A Novel Content Defined Chunking Algorithm for Finding Incremental Data in Data Synchronization,

    C. Zhang, D. Qi, Z. Cai, W. Huang, X. Wang, W. Li, and J. Guo, “MII: A Novel Content Defined Chunking Algorithm for Finding Incremental Data in Data Synchronization,” IEEE Access, vol. 7, pp. 86932–86945, 2019

    2019

  51. [59]

    SeqCDC: Hashless Content-Defined Chunking for Data Deduplication,

    S. Udayashankar, A. Baba, and S. Al-Kiswany, “SeqCDC: Hashless Content-Defined Chunking for Data Deduplication,” in Proceedings of the 25th International Middleware Conference , Middleware ’24, (New York, NY , USA), p. 292–298, Association for Computing Machinery, 2024

    2024

  52. [60]

    A Survey of Secure Data Deduplication Schemes for Cloud Storage Systems,

    Y . Shin, D. Koo, and J. Hur, “A Survey of Secure Data Deduplication Schemes for Cloud Storage Systems,” ACM Computing Surveys, vol. 49, Jan 2017

    2017

  53. [61]

    Message-locked encryp- tion and secure deduplication,

    M. Bellare, S. Keelveedhi, and T. Ristenpart, “Message-locked encryp- tion and secure deduplication,” in Annual international conference on the theory and applications of cryptographic techniques , pp. 296–312, Springer, 2013

    2013

  54. [62]

    Secure Deduplication of Encrypted Data without Additional Independent Servers,

    J. Liu, N. Asokan, and B. Pinkas, “Secure Deduplication of Encrypted Data without Additional Independent Servers,” in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS ’15, (New York, NY , USA), p. 874–885, Association for Computing Mac...

    2015

  55. [63]

    Side Channels in Cloud Services: Deduplication in Cloud Storage,

    D. Harnik, B. Pinkas, and A. Shulman-Peleg, “Side Channels in Cloud Services: Deduplication in Cloud Storage,” IEEE Security and Privacy , vol. 8, no. 6, pp. 40–47, 2010

    2010

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.