STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV

Dongdong Tang; Weilan Wang; Yu Mao; Jinghuan YU; Tei-Wei Kuo; Chun Jason Xue

doi:10.1109/ICDE60146.2024.00298

STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV

Dongdong Tang
, Weilan Wang^*
, Yu Mao^*
, Jinghuan YU
, Tei-Wei Kuo
, Chun Jason Xue

^*Corresponding author for this work

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

5 Citations (Scopus)

Abstract

Log-Structured-Merge-tree (LSM-tree) has been extensively adopted because of its exceptional write efficiency and high space utilization. Compaction is invoked periodically in LSM-tree based key-value(LSM KV) systems to maintain good system performance. As the size of LSM-KV grows, large-scale compaction is now frequently seen. Compaction throughput significantly degrades with larger inputs, leading to frequent write stalls and decrement in overall write throughput. This paper proposes STEM, a stream-based compaction framework with FPGA to address this issue. A clean-cut algorithm is introduced to enable streaming-based compaction for large-scale data. With a multi-unit pipeline and dynamic pipeline schedule, STEM can handle large-scale compaction tasks efficiently. Based on the experiment result, the compaction throughput of STEM can achieve 27× on average and up to 35× improvement compared with the current RocksDB compaction, 2.09× to 2.27× improvement compared with the state-of-the-art FPGA accelerator. © 2024 IEEE.

Original language	English
Title of host publication	Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024
Place of Publication	Los Alamitos, Calif.
Publisher	IEEE Computer Society
Pages	3893-3905
ISBN (Electronic)	9798350317152
ISBN (Print)	9798350317169
DOIs	https://doi.org/10.1109/ICDE60146.2024.00298
Publication status	Published - 2024
Event	40th IEEE International Conference on Data Engineering (ICDE 2024) - Utrecht, Netherlands Duration: 13 May 2024 → 17 May 2024 https://icde2024.github.io/

Publication series

Name	Proceedings - International Conference on Data Engineering
ISSN (Print)	1084-4627
ISSN (Electronic)	2375-026X

Conference

Conference	40th IEEE International Conference on Data Engineering (ICDE 2024)
Place	Netherlands
City	Utrecht
Period	13/05/24 → 17/05/24
Internet address	https://icde2024.github.io/

Funding

The work described in this paper was supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. CityU 11209122).

Research Keywords

Compaction
FPGA
LSM KV

RGC Funding Information

RGC-funded

Access Output

10.1109/ICDE60146.2024.00298

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Tang, D., Wang, W., Mao, Y., YU, J., Kuo, T.-W., & Xue, C. J. (2024). STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV. In Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024 (pp. 3893-3905). (Proceedings - International Conference on Data Engineering). IEEE Computer Society. https://doi.org/10.1109/ICDE60146.2024.00298

@inproceedings{f022ffc064574980ae1b94db97a8a755,

title = "STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV",

abstract = "Log-Structured-Merge-tree (LSM-tree) has been extensively adopted because of its exceptional write efficiency and high space utilization. Compaction is invoked periodically in LSM-tree based key-value(LSM KV) systems to maintain good system performance. As the size of LSM-KV grows, large-scale compaction is now frequently seen. Compaction throughput significantly degrades with larger inputs, leading to frequent write stalls and decrement in overall write throughput. This paper proposes STEM, a stream-based compaction framework with FPGA to address this issue. A clean-cut algorithm is introduced to enable streaming-based compaction for large-scale data. With a multi-unit pipeline and dynamic pipeline schedule, STEM can handle large-scale compaction tasks efficiently. Based on the experiment result, the compaction throughput of STEM can achieve 27× on average and up to 35× improvement compared with the current RocksDB compaction, 2.09× to 2.27× improvement compared with the state-of-the-art FPGA accelerator. {\textcopyright} 2024 IEEE.",

keywords = "Compaction, FPGA, LSM KV",

author = "Dongdong Tang and Weilan Wang and Yu Mao and Jinghuan YU and Tei-Wei Kuo and Xue, \{Chun Jason\}",

year = "2024",

doi = "10.1109/ICDE60146.2024.00298",

language = "English",

isbn = "9798350317169",

series = "Proceedings - International Conference on Data Engineering",

publisher = "IEEE Computer Society",

pages = "3893--3905",

booktitle = "Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024",

address = "United States",

note = "40th IEEE International Conference on Data Engineering (ICDE 2024) ; Conference date: 13-05-2024 Through 17-05-2024",

url = "https://icde2024.github.io/",

}

Tang, D , Wang, W , Mao, Y , YU, J, Kuo, T-W & Xue, CJ 2024, STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV. in Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024. Proceedings - International Conference on Data Engineering, IEEE Computer Society, Los Alamitos, Calif., pp. 3893-3905, 40th IEEE International Conference on Data Engineering (ICDE 2024), Utrecht, Netherlands, 13/05/24. https://doi.org/10.1109/ICDE60146.2024.00298

STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV. / Tang, Dongdong ; Wang, Weilan ; Mao, Yu et al.
Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024. Los Alamitos, Calif.: IEEE Computer Society, 2024. p. 3893-3905 (Proceedings - International Conference on Data Engineering).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - STEM

T2 - 40th IEEE International Conference on Data Engineering (ICDE 2024)

AU - Tang, Dongdong

AU - Wang, Weilan

AU - Mao, Yu

AU - YU, Jinghuan

AU - Kuo, Tei-Wei

AU - Xue, Chun Jason

PY - 2024

Y1 - 2024

N2 - Log-Structured-Merge-tree (LSM-tree) has been extensively adopted because of its exceptional write efficiency and high space utilization. Compaction is invoked periodically in LSM-tree based key-value(LSM KV) systems to maintain good system performance. As the size of LSM-KV grows, large-scale compaction is now frequently seen. Compaction throughput significantly degrades with larger inputs, leading to frequent write stalls and decrement in overall write throughput. This paper proposes STEM, a stream-based compaction framework with FPGA to address this issue. A clean-cut algorithm is introduced to enable streaming-based compaction for large-scale data. With a multi-unit pipeline and dynamic pipeline schedule, STEM can handle large-scale compaction tasks efficiently. Based on the experiment result, the compaction throughput of STEM can achieve 27× on average and up to 35× improvement compared with the current RocksDB compaction, 2.09× to 2.27× improvement compared with the state-of-the-art FPGA accelerator. © 2024 IEEE.

AB - Log-Structured-Merge-tree (LSM-tree) has been extensively adopted because of its exceptional write efficiency and high space utilization. Compaction is invoked periodically in LSM-tree based key-value(LSM KV) systems to maintain good system performance. As the size of LSM-KV grows, large-scale compaction is now frequently seen. Compaction throughput significantly degrades with larger inputs, leading to frequent write stalls and decrement in overall write throughput. This paper proposes STEM, a stream-based compaction framework with FPGA to address this issue. A clean-cut algorithm is introduced to enable streaming-based compaction for large-scale data. With a multi-unit pipeline and dynamic pipeline schedule, STEM can handle large-scale compaction tasks efficiently. Based on the experiment result, the compaction throughput of STEM can achieve 27× on average and up to 35× improvement compared with the current RocksDB compaction, 2.09× to 2.27× improvement compared with the state-of-the-art FPGA accelerator. © 2024 IEEE.

KW - Compaction

KW - FPGA

KW - LSM KV

UR - https://www.scopus.com/pages/publications/85200459419

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85200459419&origin=recordpage

U2 - 10.1109/ICDE60146.2024.00298

DO - 10.1109/ICDE60146.2024.00298

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9798350317169

T3 - Proceedings - International Conference on Data Engineering

SP - 3893

EP - 3905

BT - Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024

PB - IEEE Computer Society

CY - Los Alamitos, Calif.

Y2 - 13 May 2024 through 17 May 2024

ER -

Tang D , Wang W , Mao Y , YU J, Kuo TW, Xue CJ. STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV. In Proceedings - 2024 IEEE 40th International Conference on Data Engineering, ICDE 2024. Los Alamitos, Calif.: IEEE Computer Society. 2024. p. 3893-3905. (Proceedings - International Conference on Data Engineering). Epub 2024 Jul 23. doi: 10.1109/ICDE60146.2024.00298

STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV

Abstract

Publication series

Conference

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Towards Unified-storage-memory-enabled Mobile Devices

Cite this

STEM: Streaming-Based FPGA Acceleration for Large-Scale Compactions in LSM KV

Abstract

Publication series

Conference

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Towards Unified-storage-memory-enabled Mobile Devices

Cite this