A Performance Optimization Method for Key-Value Store Based on LSM-tree

被引：0
|
作者：
Wang H. [1 ,2 ]
Li Z. [1 ,2 ]
Zhang X. [1 ,2 ]
Zhao X. [1 ,2 ]
机构：
[1] School of Computer Science, Northwestern Polytechnical University, Xi'an
[2] Key Laboratory of Big Data Storage and Management, Northwestern Polytechnical University, Ministry of Industry and Information Technology, Xi'an
来源：
Jisuanji Yanjiu yu Fazhan/Computer Research and Development | 2019年 / 56卷 / 08期
基金：
中国国家自然科学基金;
关键词：
File system; Key-value store; LSM-tree; Write performance; Write-ahead log (WAL);
D O I：
10.7544/issn1000-1239.2019.20190110
中图分类号：
学科分类号：
摘要：
Nowadays, persistent key-value (KV) stores play a critical role in a variety of modern data-intensive applications, such as Web indexing, e-commerce, and cloud data storage systems, etc. KV stores that are based on log-structured merge tree (LSM-tree) have attracted growing attention because of their ability to eliminate random writes and maintain acceptable read performance. However, they also suffer from some performance issues. On one hand, they need to leverage write-ahead log (WAL) files to guarantee the atomicity and safety of write operations to enable recovery in case of a crash. This will result in severe write amplification and metadata overhead because of frequent WAL file update, leading to performance degradation. On the other hand, these KV stores usually use a conventional local filesystem to store KV data, which can harm the performance due to unnecessary operations in the filesystem. In this paper, we present RocksFS, an optimized filesystem for KV stores based on LSM-tree. We simplify the filesystem to remove unnecessary functions and attributes to reduce filesystem overhead and redesign the format and I/O path of WAL file to decrease metadata overhead. We compare RocksFS with conventional filesystems in the environment of RocksDB, a popular LSM-tree-based KV store. The experimental results demonstrate that RocksFS can observably improve the small key-value data write performance of RocksDB by 8x at most compared with traditional filesystems on both hard disk drive and solid state disk. © 2019, Science Press. All right reserved.
引用
收藏
页码：1792 / 1802
页数：10
相关论文
共 20 条
  • [11] Lu L., Pillai T.S., Gopalakrishnan H., Et al., WiscKey: Separating keys from values in SSD-conscious storage, ACM Transactions on Storage, 13, 1, (2017)
  • [12] Raju P., Kadekodi R., Chidambaram V., Et al., PebblesDB: Building key-value stores using fragmented log-structured merge trees, Proc of the 26th Symp on Operating Systems Principles, pp. 497-514, (2017)
  • [13] Yue Y., He B., Li Y., Et al., Building an efficient put-intensive key-value store with skip-tree, IEEE Transactions on Parallel & Distributed Systems, 28, 4, pp. 961-973, (2017)
  • [14] Zhang J., Lu Y., Shu J., Et al., FlashKV: Accelerating KV performance with open-channel SSDs, ACM Transactions on Embedded Computing Systems, 16, 5, (2017)
  • [15] Qin X., Zhang J., Lu Y., Et al., A Key-Value database optimization method based on raw flash device, Journal of Computer Research and Development, 54, 6, pp. 1326-1336, (2017)
  • [16] Bjorling M., Madsen J., Bonnet P., Et al., LightNVM: Lightning fast evaluation platform for non-volatile memories, Proc of the 5th Annual UCSD Non-Volatile Memories Workshop, pp. 28-29, (2014)
  • [17] Bjorling M., Gonzalez J., Bonnet P., LightNVM: The Linux open-channel SSD subsystem, Proc of the 15th USENIX Conf on File and Storage Technologies, pp. 359-374, (2017)
  • [18] Howard B., Bloom filter
  • [19] Weil S.A., Brandt S.A., Miller E.L., Et al., Ceph: A scalable, high-performance distributed file system, Proc of the 7th Symp on Operating Systems Design and Implementation, pp. 307-320, (2006)
  • [20] Atikoglu B., Xu Y., Frachtenberg E., Et al., Workload analysis of a large-scale key-value store, Proc of the 12th ACM SIGMETRICS/PERFORMANCE Joint Int Conf on Measurement and Modeling of Computer Systems, pp. 53-64, (2012)
12