Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Mycelium: A Transformation-Embedded LSM-Tree

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Data transformations embedded in LSM compaction can cost about 20% write overhead and speed reads by up to 425%.

desk verdict A useful generalization of compaction-embedded transformation, but the headline write-overhead claim is confounded and update/delete semantics are unspecified. read the letter →

arxiv 2506.08923 v1 pith:MY5AA7X3 submitted 2025-06-10 cs.DC

Holly Casaletto , Jeff Lefevre , Aldrin Montana , Peter Alvaro This is my paper
classification cs.DC
keywords LSM-treecompactiondatatransformationcolumnarstoragesecondaryindexwriteamplificationreadoptimizationRocksDB
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the background compaction step of an LSM-tree can do more than merge sorted runs: it can reshape data for future reads at nearly the same I/O cost as compaction alone. The authors built Mycelium, a RocksDB-based prototype whose transformer interface splits rows into column groups, converts value formats, and builds secondary indexes while data is being compacted. On YCSB workloads, Mycelium reports a write-throughput overhead of about 20% for embedded transformations, compared with 35-60% when the same transformations are done outside compaction, and read-latency improvements up to 425% over the RocksDB baseline. If this holds, write-optimized storage could become read-optimized without a separate ETL pipeline, making real-time analytics on freshly ingested data more practical.

What carries the argument

The load-bearing mechanism is transformation-embedded compaction: a compaction job in a source column family reads input SST files, applies a Transformer to each key-value pair in memory, and installs the outputs as new SST files in one or more internal destination column families that are hidden from user writes. The hybrid merge policy called Tierveling uses tiering for transformed compactions and leveling for identity compactions, so transformed data flows between column families while untransformed data keeps single-run levels. A Transformer interface with Prepare, Transform, and Retrieve methods makes split (row-to-column), convert (JSON-to-FlatBuffers), augment (secondary index), and composed transformations expressible as compaction-time functions.

What would settle it

Run a mixed workload that inserts keys, lets compaction transform them, then updates or deletes some of those keys, and issue point reads and range scans: if any read returns a stale or duplicate version, the engine is not transparent and the reported read improvements are not obtained on live data. Section 5.3.1 covers only Q1 (INSERT), so this experiment lies outside the paper's measurements.

Watch

Extended reading notes

Core claim

The paper's central claim is that data transformation can be made a byproduct of LSM compaction. Instead of writing compacted output back into the same column family, a compaction job applies a transformer to each key-value pair and installs the results into internal destination column families, so the reads and writes that compaction already performs are shared by the transformation. The reported result is a write-throughput penalty of 10.09% to 21.25% for single and combined transformations, versus 34.84% to 57.98% for naive external transformations, with read-latency gains of up to 425% and index-query gains of several hundred thousand times over full-table scans.

Load-bearing premise

Mycelium's transparency rests on later updates and deletes to keys that have already been transformed still returning the correct current value, but the paper tests only inserts and does not specify how a post-compaction write or delete propagates to the transformed copies.

Editorial extensions

If this is right

  • A write-optimized store can progressively become read-optimized as data ages through compaction, without a separate extraction step and without delaying availability.
  • Transformations that shrink values or reduce per-query I/O transfer their benefit directly to read latency, with the largest gains on column-retrieving range queries and index queries.
  • Multiple transformations can share the same compaction I/O budget: the paper measures about 21% write overhead for split-plus-convert, versus 10-15% for a single transformation.
  • Applying even a no-op transformation under tierveling can slightly raise write throughput (about 5%) by moving data out of Level-0 faster and reducing write stalls, at the cost of potentially higher read costs.
  • Data transformations become composable, declarative operations on column families rather than externally scheduled jobs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If update and delete transparency is verified, TE-LSM could become a general substrate for materialized views, schema normalization, and compression, not just the three transformations demonstrated.
  • The 425% read improvement applies to column-retrieving range queries; a workload dominated by full-row reads would likely shrink or reverse that gain because split rows must be reassembled.
  • A testable extension is to compare Mycelium against an engine that runs the same transformations during idle time, separating the benefit of amortizing I/O from the benefit of doing transformation work earlier.
  • The cost model's assumption that transformation is CPU-bound relative to compaction I/O could be probed by embedding a deliberately expensive transformer and measuring where the write overhead begins to exceed 20%.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents Transformation-Embedded LSM-trees (TE-LSM), a design that runs user-defined data transformations such as column splitting, format conversion, and secondary-index building during LSM-tree compaction, inside a RocksDB-based prototype called Mycelium. The claimed benefit is that the I/O cost of transformation is amortized by compaction I/O, reducing the write-throughput penalty of transformation to about 20% (versus 35-60% for transformations done outside compaction) while improving read latency by up to 425%. The paper includes a cost model and an experimental evaluation using YCSB workloads on 100GB of data.

Significance. The idea of embedding transformations into compaction is a useful generalization of prior work and may have practical value for mixed OLTP/OLAP workloads. The paper demonstrates substantial engineering: a working prototype, three transformation types, and a composition mechanism, with a large-scale evaluation. However, the headline quantitative claims are currently weakened by (i) the confounding of the write-overhead measurement with the Tierveling merge strategy and (ii) the absence of defined update/delete semantics for transformed data, without which the 'transparent' claim is not established. Both issues are addressable in a revision, and the qualitative direction of the results is likely to survive, so the paper has clear potential.

major comments (3)
  1. [§5.4.1, Table 2] The write-throughput overhead figures are confounded by the Tierveling merge strategy. Table 2 reports Mycelium-Identity as -5.25% (i.e., 5.25% faster than RocksDB) even though it performs no transformation. The transformation-specific overhead should therefore be measured relative to Mycelium-Identity, not to the RocksDB baseline. Recomputing from the table, the single-transformation penalties become approximately 15-20% and Split-Converting becomes ~25%, rather than the 10.09-21.25% range stated. The paper should report the number of runs and error bars for these measurements, and either compare TE-LSM configurations against Mycelium-Identity or adjust the baseline to isolate the transformation cost from the Tierveling effect. Without this adjustment, the abstract's '20% write throughput overhead' claim is not established.
  2. [§3.1, §3.2, §4.3, §4.4, Algorithm 3] The 'transparent' embedding claim is unsupported for update and delete workloads. The design moves transformed data to internal destination column families (Sec. 3.3) and deletes the source files after transformation (Algorithm 3), while Sec. 3.1 forbids external writes to those destination column families. Sec. 4.3 states only that writes 'function the same way as in standard RocksDB,' which cannot be true for a key that already has a transformed copy in a destination column family: a later write, update, or delete in the user-facing source column family has no defined propagation path to that copy. Sec. 4.4 describes row reassembly from split column families but does not describe merging a newer source-column-family version with the transformed destination copy. The evaluation (Sec. 5.3.1) exercises only Q1 (INSERT) and never tests update or delete. As a result, the system may return stale or resurrected values for update/delete workloads, and the headline performance claims are currently scoped to append-only workloads. The authors must either implement and test update/delete propagation or explicitly restrict the paper's claims to append-only, insert-only usage.
  3. [Appendix B] The cost model is calibrated to the experimental results rather than used as an independent predictor. The write-throughput example picks n=2 extra writes during cross-column-family compaction and obtains the same ~20% overhead that appears in the abstract, while the point-query examples assume a 30% record-size reduction after format conversion, close to the measured 34.76% SST size reduction in Sec. 5.4.2. The values of n and the record-size reduction are not independently measured or justified. Since Sec. 3.6 claims the model can 'predict whether a given transformation is beneficial,' the paper should validate the model with measured parameters or explicitly describe it as an illustrative calculation rather than a predictive model.
minor comments (5)
  1. [§5.3.2] The test-data description refers to 'Baseline-1' through 'Baseline-4,' but Section 5.2.1 defines baselines as 'Baseline,' 'Baseline-Splitting,' 'Baseline-Converting,' and 'Baseline-Augmenting.' Please align the naming.
  2. [§5.4.2] The read-latency results report P50/P99 values but do not state how many measurement repetitions were performed or the run-to-run variance. Reporting confidence intervals or at least the number of batches would strengthen the claimed speedups, especially for Q2 and Q3.
  3. [§3.5 and §4.2.5] The claimed associativity and commutativity of transformers (Eqs. 1-2) appear inconsistent with the transformer-assignment rules in Sec. 4.2.5, which impose that gradual transformers are applied first and that at most one gradual transformer exists per logical column family. Please clarify whether the algebra describes logical equivalence of composed functions or the scheduling policy, and note that the evaluation only demonstrates one composition order.
  4. [§4.4] The column merge operator that reassembles full rows from split column families is described only in one sentence. A brief description of how the merge is performed and how it interacts with range scans would make the read path understandable without requiring the reader to infer the implementation.
  5. [Appendix B] In Eq. (4), the text says the write bandwidth is 'determined by the minimum' of two terms, but the subsequent simplification to Eq. (5) silently assumes the write-disk bandwidth is the binding constraint. Please state that assumption explicitly and discuss the regime in which compute-bound transformation throughput would change the result.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are direct measurements against RocksDB baselines, and the cost model is an illustrative analysis, not a load-bearing prediction.

full rationale

The paper's headline results (20% write throughput overhead; read latency improvements up to 4.25x) are empirical outcomes of YCSB experiments comparing Mycelium configurations to RocksDB and to naive external-transformation baselines (Sec. 5.4). No derivation chain is claimed: TE-LSM is a design proposal, and its evaluation is self-contained against external baselines. The cost model in Appendix B is explicitly an example-based analysis: it assumes n=2 extra cross-column-family writes and a 30% size reduction, which loosely track the measured values, but the paper does not use the model to generate or validate the experimental numbers, nor does it claim the model's example outputs were predicted before measurement. There are no load-bearing self-citations: the only closely related prior work (Saxena et al., Real-Time LSM-Tree) is cited for inspiration and generalization, not as a theorem that forces the design. The transparency gap for later updates and deletes to keys already transformed into internal destination column families (Secs. 3.1, 4.3, 4.4; the write evaluation uses only Q1 INSERT in Sec. 5.3.1) is a correctness and scope concern, not a circularity: it does not make any stated claim equivalent to its own inputs by construction. Accordingly, no circular step meets the required evidence bar.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central results are empirical, so the ledger is short. The cost model needs two hand-set parameters (n and the converted-size reduction) that line up with the headline measurements, and the evaluation implicitly assumes transformation compute is cheap and that workloads are insert-only.

free parameters (2)
  • n (extra write amplification per cross-column-family compaction) = n=2 in cost model example
    The number of extra writes per transformation compaction is stated as 1 <= n < T/2 without derivation from the implementation; the example uses n=2, which produces the same ~20% write overhead as the headline result.
  • Converted record size reduction = 30% in cost model; 34.76% measured in Section 5.4.2
    The model's point/range query examples use a 30% size reduction, close to the measured 34.76% SST reduction, so the 'prediction' is calibrated to the measurement.
assumptions (4)
  • domain assumption Transformation compute is negligible relative to compaction I/O, so added in-memory processing during compaction contributes little to write cost.
    Section 3 states compaction's second (in-memory) step is rate-limited by I/O and that transformations add overhead only to this step; this is the core amortization premise and is only tested for the three simple transformers.
  • domain assumption All data arrives in random key ranges, so all Level-0 sorted runs overlap, giving Z overlapping runs in the cost model.
    Appendix B (Table 4 notes) assumes random key ranges; this simplifies L0 cost but may not hold for time-ordered ingestion.
  • standard math Standard LSM write amplification formula WA = 1 + T/(T-1) log_T(N/B) applies.
    Used as baseline in Appendix B.0.1; standard result from LSM literature.
  • domain assumption Updates and deletes are assumed not to affect the transformed copies; all workloads are insert-only.
    Section 5.3.1 only issues INSERT; the paper does not define semantics for propagating overwrites/deletes to destination column families.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mycelium: A Transformation-Embedded LSM-Tree." pith.science (2026). https://pith.science/paper/MY5AA7X3

@misc{pith2026250608923,
  author       = {Pith},
  title        = {Pith review of: Mycelium: A Transformation-Embedded LSM-Tree},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MY5AA7X3}},
  note         = {Machine review of arXiv:2506.08923}
}
read the original abstract

Compaction is a necessary, but often costly background process in write-optimized data structures like LSM-trees that reorganizes incoming data that is sequentially appended to logs. In this paper, we introduce Transformation-Embedded LSM-trees (TE-LSM), a novel approach that transparently embeds a variety of data transformations into the compaction process. While many others have sought to reduce the high cost of compaction, TE-LSMs leverage the opportunity to embed other useful work to amortize IO costs and amplification. We illustrate the use of a TE-LSM in Mycelium, our prototype built on top of RocksDB that extends the compaction process through a cross-column-family merging mechanism. Mycelium enables seamless integration of a transformer interface and aims to better prepare data for future accesses based on access patterns. We use Mycelium to explore three types of transformations: splitting column groups, converting data formats, and index building. In addition to providing a cost model analysis, we evaluate Mycelium's write and read performance using YCSB workloads. Our results show that Mycelium incurs a 20% write throughput overhead - significantly lower than the 35% to 60% overhead observed in naive approaches that perform data transformations outside of compaction-while achieving up to 425% improvements in read latency compared to RocksDB baseline.

Figures

Figures reproduced from arXiv: 2506.08923 by the authors.

Figure 1
Figure 1. In a cross-column-family compaction, the user-facing [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. illustrates the high-level architecture of Mycelium, along with an example code path for open￾ing the database and writing to a column family. The core object structs for Mycelium are the MemTable, the SST files, and the Write Ahead Log (WAL). The compo￾nents that have been modified to support transformation￾embedded compaction are highlighted in green, which includes: 1, the Options to manage the configuration pa￾r… view at source ↗
Figure 4
Figure 4. Split transformer splits column groups while com [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (4 more)
Figure 6
Figure 6. Figure 6: Augment transformer creates secondary index while [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Column-retrieving queries benefit significantly from [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Row-retrieving queries showing negative impact from [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Mycelium-Identity (M-Identity) and Mycelium [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 44 canonical work pages

  1. [1]

    Compaction management in distributed key-value datastores.Proceedings of the VLDB Endowment, 8(8):850–861, 2015

    Muhammad Yousuf Ahmad and Bettina Kemme. Compaction management in distributed key-value datastores.Proceedings of the VLDB Endowment, 8(8):850–861, 2015

    work page 2015

  2. [2]

    H2o: a hands-free adaptive store

    Ioannis Alagiannis, Stratos Idreos, and Anastasia Ailamaki. H2o: a hands-free adaptive store. In Proceedings of the 2014 ACM SIGMOD interna- 12 tional conference on Management of data, pages 1103–1114, 2014

    work page 2014

  3. [3]

    Bridging the archipelago between row-stores and column-stores for hybrid workloads

    Joy Arulraj, Andrew Pavlo, and Prashanth Menon. Bridging the archipelago between row-stores and column-stores for hybrid workloads. InProceed- ings of the 2016 International Conference on Man- agement of Data, pages 583–598, 2016

    work page 2016

  4. [4]

    Designing access meth- ods: The rum conjecture

    Manos Athanassoulis, Michael S Kester, Lukas M Maas, Radu Stoica, Stratos Idreos, Anastasia Aila- maki, and Mark Callaghan. Designing access meth- ods: The rum conjecture. InEDBT, volume 2016, pages 461–466, 2016

    work page 2016

  5. [5]

    Efficient phrase querying with an auxiliary index

    Dirk Bahle, Hugh E Williams, and Justin Zobel. Efficient phrase querying with an auxiliary index. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, pages 215–221, 2002

    work page 2002

  6. [6]

    In2017 USENIX Annual Technical Conference (USENIX ATC 17), pages 363–375, 2017

    Oana Balmau, Diego Didona, Rachid Guerraoui, Willy Zwaenepoel, Huapeng Yuan, Aashray Arora, Karan Gupta, and Pavan Konka.{TRIAD}: Creat- ing synergies between memory, disk and log in log structured {Key-Value} stores. In2017 USENIX Annual Technical Conference (USENIX ATC 17), pages 363–375, 2017

    work page 2017

  7. [7]

    Polynesia: En- abling effective hybrid transactional/analytical databases with specialized hardware/software co- design.arXiv preprint arXiv:2103.00798, 2021

    Amirali Boroumand, Saugata Ghose, Geraldo F Oliveira, and Onur Mutlu. Polynesia: En- abling effective hybrid transactional/analytical databases with specialized hardware/software co- design.arXiv preprint arXiv:2103.00798, 2021

    arXiv 2021

  8. [8]

    Optimizing binary serial- ization with an independent data definition format

    David Carrera Castillo, Jonathan Rosales, and Gus- tavo A Torres Blanco. Optimizing binary serial- ization with an independent data definition format. International Journal of Computer Applications, 975:8887, 2018

    work page 2018

Show all 46 references
  1. [9]

    Work- load diversity and dynamics in big data analytics: implications to system designers

    Jichuan Chang, Kevin T Lim, John Byrne, Laura Ramirez, and Parthasarathy Ranganathan. Work- load diversity and dynamics in big data analytics: implications to system designers. InProceedings of the 2nd Workshop on Architectures and Systems for Big Data, pages 21–26, 2012

    2012

  2. [10]

    CRC press, 2002

    David Collett.Modelling binary data. CRC press, 2002

    2002

  3. [11]

    Optimizing space amplification in rocksdb

    Siying Dong, Mark Callaghan, Leonidas Galanis, Dhruba Borthakur, Tony Savor, and Michael Strum. Optimizing space amplification in rocksdb. In CIDR, volume 3, page 3, 2017

    2017

  4. [12]

    Data preparation: A technologi- cal perspective and review.SN Computer Science, 4(4):425, 2023

    Alvaro AA Fernandes, Martin Koehler, Nikolaos Konstantinou, Pavel Pankin, Norman W Paton, and Rizos Sakellariou. Data preparation: A technologi- cal perspective and review.SN Computer Science, 4(4):425, 2023

    2023

  5. [13]

    A survey on indexing techniques for big data: taxonomy and performance evaluation.Knowledge and information systems, 46:241–284, 2016

    Abdullah Gani, Aisha Siddiqa, Shahaboddin Shamshirband, and Fariza Hanum. A survey on indexing techniques for big data: taxonomy and performance evaluation.Knowledge and information systems, 46:241–284, 2016

    2016

  6. [14]

    Key-value stor- age engines

    Stratos Idreos and Mark Callaghan. Key-value stor- age engines. InProceedings of the 2020 ACM SIG- MOD International Conference on Management of Data, pages 2667–2672, 2020

    2020

  7. [15]

    A technical comprehensive survey of etl tools.Inter- national Journal of Applied Engineering Research, 11(4):2557–2559, 2016

    Vaishali A Kherdekar and Pravin S Metkewar. A technical comprehensive survey of etl tools.Inter- national Journal of Applied Engineering Research, 11(4):2557–2559, 2016

    2016

  8. [16]

    Extract-transform-load for video streams

    Ferdinand Kossmann, Ziniu Wu, Eugenie Lai, Nes- ime Tatbul, Lei Cao, Tim Kraska, and Samuel Mad- den. Extract-transform-load for video streams. arXiv preprint arXiv:2310.04830, 2023

    2023 arXiv

  9. [17]

    Mainlining databases: Supporting fast transactional workloads on universal columnar data file formats

    Tianyu Li, Matthew Butrovich, Amadou Ngom, Wan Shen Lim, Wes McKinney, and Andrew Pavlo. Mainlining databases: Supporting fast transactional workloads on universal columnar data file formats. arXiv preprint arXiv:2004.14471, 2020

    2004 arXiv

  10. [18]

    Wisckey: Separating keys from values in ssd-conscious stor- age.ACM Transactions On Storage (TOS), 13(1):1– 28, 2017

    Lanyue Lu, Thanumalayan Sankaranarayana Pil- lai, Hariharan Gopalakrishnan, Andrea C Arpaci- Dusseau, and Remzi H Arpaci-Dusseau. Wisckey: Separating keys from values in ssd-conscious stor- age.ACM Transactions On Storage (TOS), 13(1):1– 28, 2017

    2017

  11. [19]

    Greenplum: a hybrid database for transac- tional and analytical workloads

    Zhenghua Lyu, Huan Hubert Zhang, Gang Xiong, Gang Guo, Haozhou Wang, Jinbao Chen, Asim Praveen, Yu Yang, Xiaoming Gao, Alexandra Wang, et al. Greenplum: a hybrid database for transac- tional and analytical workloads. InProceedings of the 2021 International Conference on Manage...

    2021

  12. [20]

    Dod-etl: dis- tributed on-demand etl for near real-time business intelligence.Journal of Internet Services and Ap- plications, 10(1):21, 2019

    Gustavo V Machado, Ítalo Cunha, Adriano CM Pereira, and Leonardo B Oliveira. Dod-etl: dis- tributed on-demand etl for near real-time business intelligence.Journal of Internet Services and Ap- plications, 10(1):21, 2019

    2019

  13. [21]

    Enhancing data warehouse efficiency by optimizing etl processing in near real time data integration environment

    Kunal Maharaj and Kunal Kumar. Enhancing data warehouse efficiency by optimizing etl processing in near real time data integration environment. In International Conference on Big Data Intelligence and Computing, pages 289–304. Springer, 2022. 13

    2022

  14. [22]

    Dataxformer: An interactive data transfor- mation tool

    John Morcos, Ziawasch Abedjan, Ihab Francis Ilyas, Mourad Ouzzani, Paolo Papotti, and Michael Stone- braker. Dataxformer: An interactive data transfor- mation tool. InProceedings of the 2015 ACM SIG- MOD International Conference on Management of Data, pages 883–888, 2015

    2015

  15. [23]

    The log-structured merge-tree (lsm-tree).Acta Informatica, 33:351–385, 1996

    Patrick O’Neil, Edward Cheng, Dieter Gawlick, and Elizabeth O’Neil. The log-structured merge-tree (lsm-tree).Acta Informatica, 33:351–385, 1996

    1996

  16. [24]

    dcompaction: Speeding up compaction of the lsm- tree via delayed compaction.Journal of Computer Science and Technology, 32:41–54, 2017

    Feng-Feng Pan, Yin-Liang Yue, and Jin Xiong. dcompaction: Speeding up compaction of the lsm- tree via delayed compaction.Journal of Computer Science and Technology, 32:41–54, 2017

    2017

  17. [25]

    Fast access to columnar, hierarchically nested data via code transformation

    Jim Pivarski, Peter Elmer, Brian Bockelman, and Zhe Zhang. Fast access to columnar, hierarchically nested data via code transformation. In2017 IEEE International Conference on Big Data (Big Data), pages 253–262. IEEE, 2017

    2017

  18. [26]

    Scaling up mixed workloads: a battle of data freshness, flexibility, and scheduling

    Iraklis Psaroudakis, Florian Wolf, Norman May, Thomas Neumann, Alexander Böhm, Anastasia Ail- amaki, and Kai-Uwe Sattler. Scaling up mixed workloads: a battle of data freshness, flexibility, and scheduling. InPerformance Characterization and Benchmarking. Traditional to Big Da...

    2014

  19. [27]

    Pebblesdb: Building key-value stores using fragmented log-structured merge trees

    Pandian Raju, Rohan Kadekodi, Vijay Chi- dambaram, and Ittai Abraham. Pebblesdb: Building key-value stores using fragmented log-structured merge trees. InProceedings of the 26th Symposium on Operating Systems Principles, pages 497–514, 2017

    2017

  20. [28]

    Relational memory: Native in-memory accesses on rows and columns.arXiv preprint arXiv:2109.14349, 2021

    Shahin Roozkhosh, Denis Hoornaert, Ju Hyoung Mun, Tarikul Islam Papon, Ahmed Sanaullah, Ul- rich Drepper, Renato Mancuso, and Manos Athanas- soulis. Relational memory: Native in-memory accesses on rows and columns.arXiv preprint arXiv:2109.14349, 2021

    2021 arXiv

  21. [29]

    Improv- ing data latency in etl with filtering algorithms for stream processing an experimental setup

    Adilah Sabtu, Mahadi Bahari, Nurulhuda Fir- daus Mohd Azmi, Nor Azizah Ali, Zuraidah Su- laiman, and Nur Aqidah Mohd Fauzi. Improv- ing data latency in etl with filtering algorithms for stream processing an experimental setup. InAIP Conference Proceedings, volume 2991. AIP Pub...

    2024

  22. [30]

    L- store: A real-time oltp and olap system.arXiv preprint arXiv:1601.04084, 2016

    Mohammad Sadoghi, Souvik Bhattacherjee, Bish- waranjan Bhattacharjee, and Mustafa Canim. L- store: A real-time oltp and olap system.arXiv preprint arXiv:1601.04084, 2016

    2016 arXiv

  23. [31]

    Muneer Ahmed Salamkar and Karthik Allam. Data integration techniques: Exploring tools and method- ologies for harmonizing data across diverse systems and sources.Distributed Learning and Broad Ap- plications in Scientific Research, 6, 2020

    2020

  24. [32]

    Dis- secting, designing, and optimizing lsm-based data stores

    Subhadeep Sarkar and Manos Athanassoulis. Dis- secting, designing, and optimizing lsm-based data stores. InProceedings of the 2022 International Conference on Management of Data, pages 2489– 2497, 2022

    2022

  25. [33]

    Constructing and ana- lyzing the lsm compaction design space (updated version).arXiv preprint arXiv:2202.04522, 2022

    Subhadeep Sarkar, Dimitris Staratzis, Zichen Zhu, and Manos Athanassoulis. Constructing and ana- lyzing the lsm compaction design space (updated version).arXiv preprint arXiv:2202.04522, 2022

    2022 arXiv

  26. [34]

    Real-time lsm-trees for htap work- loads

    Hemant Saxena, Lukasz Golab, Stratos Idreos, and Ihab F Ilyas. Real-time lsm-trees for htap work- loads. In2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 1208–1220. IEEE, 2023

    2023

  27. [35]

    blsm: a general purpose log structured merge tree

    Russell Sears and Raghu Ramakrishnan. blsm: a general purpose log structured merge tree. InPro- ceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pages 217– 228, 2012

    2012

  28. [36]

    Building {Workload-Independent} storage with {VT-Trees}

    Pradeep J Shetty, Richard P Spillane, Ravikant R Malpani, Binesh Andrews, Justin Seyster, and Erez Zadok. Building {Workload-Independent} storage with {VT-Trees}. In11th USENIX Conference on File and Storage Technologies (FAST 13), pages 17–30, 2013

    2013

  29. [37]

    Data qual- ity in etl process: A preliminary study.Procedia Computer Science, 159:676–687, 2019

    Manel Souibgui, Faten Atigui, Saloua Zammali, Samira Cherfi, and Sadok Ben Yahia. Data qual- ity in etl process: A preliminary study.Procedia Computer Science, 159:676–687, 2019

    2019

  30. [38]

    Learned in- dexes for dynamic workloads.arXiv preprint arXiv:1902.00655, 2019

    Chuzhe Tang, Zhiyuan Dong, Minjie Wang, Zhaoguo Wang, and Haibo Chen. Learned in- dexes for dynamic workloads.arXiv preprint arXiv:1902.00655, 2019

    1902 arXiv

  31. [39]

    A survey of extract–transform– load technology.International Journal of Data Warehousing and Mining (IJDWM), 5(3):1–27, 2009

    Panos Vassiliadis. A survey of extract–transform– load technology.International Journal of Data Warehousing and Mining (IJDWM), 5(3):1–27, 2009

    2009

  32. [40]

    The aha-tree: An adaptive index for htap workloads.arXiv preprint arXiv:2406.08746, 2024

    Lu Xing and Walid G Aref. The aha-tree: An adaptive index for htap workloads.arXiv preprint arXiv:2406.08746, 2024

    2024 arXiv

  33. [41]

    The trip to the enterprise gourmet data product marketplace through a self-service data platform.arXiv preprint arXiv:2107.13212, 2021

    Michal Zasadzinski, Michael Theodoulou, Markus Thurner, and Kshitij Ranganath. The trip to the enterprise gourmet data product marketplace through a self-service data platform.arXiv preprint arXiv:2107.13212, 2021. 14

    2021 arXiv

  34. [42]

    Pipelined compaction for the lsm-tree

    Zigang Zhang, Yinliang Yue, Bingsheng He, Jin Xiong, Mingyu Chen, Lixin Zhang, and Ninghui Sun. Pipelined compaction for the lsm-tree. In 2014 IEEE 28th International Parallel and Dis- tributed Processing Symposium, pages 777–786. IEEE, 2014. A Algorithms In this section, we s...

    2014

  35. [43]

    When the entire value associated with the key is required

  36. [44]

    We denote the former as PQRA and the latter as PQRC

    When only a single field is needed. We denote the former as PQRA and the latter as PQRC. Let L represent the number of levels in the LSM tree, determined by: L=log T N B where N is the total data size, B is the write buffer size, andTis the size ratio. Let si represent the num...

  37. [45]

    TEC embedded with convert transformation: ≈ 97.78 block reads

  38. [46]

    TEC embedded with split transformation: ≈ 17.78 block reads For comparison, the range query cost for CWT is ap- proximately 138.88 block reads, resulting in: - An87.2% improvement for thesplittransformation case - A 29.6% improvement for theconverttransformation case B.0.4 Spa...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.