HRaft : Adaptive Erasure Coded Data Maintenance for Consensus in Distributed Networks
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Title of host publication | Proceedings - 2022 IEEE 36th International Parallel and Distributed Processing Symposium, IPDPS 2022 |
Publisher | Institute of Electrical and Electronics Engineers, Inc. |
Pages | 1316-1326 |
ISBN (electronic) | 978-1-6654-8106-9 |
ISBN (print) | 978-1-6654-8107-6 |
Publication status | Published - 2022 |
Publication series
Name | Proceedings - IEEE International Parallel and Distributed Processing Symposium, IPDPS |
---|---|
ISSN (Print) | 1530-2075 |
ISSN (electronic) | 530-2075 |
Conference
Title | 36th IEEE International Parallel & Distributed Processing Symposium, IPDPS 2022 |
---|---|
Location | Virtual |
Place | France |
City | Lyon |
Period | 30 May - 3 June 2022 |
Link(s)
Abstract
Distributed data services usually rely on consensus protocols like Paxos and Raft to provide fault-tolerance and data consistency across global and local-distributed data centers. Erasure coding replication has appealing storage and network cost saving compared with full copy replication, which helps consensus protocols achieve low latency, high fault tolerance, and high throughput for data access. Applying erasure coding in consensus protocols directly will degrade the liveness level when the number of failure servers reaches a certain level. To address the challenge, CRaft just stores full copy replication instead of erasure coding replication when the number of failed servers reaches a certain threshold. In such situation, CRaft will be downgraded sharply to the same storage and network costs as Raft. To overcome the shortcoming of CRaft, we propose a protocol, called HRaft, which can adapt the placement of data blocks in order to always have enough blocks to recover the stored value when servers fail. By replenishing some coded blocks in healthy servers instead of full copy replication, it can avoid switching to the full replication when a certain threshold on the number of failures is reached. We designed and implemented a key-value (KV) storage prototype to validate the proposed protocol and evaluate its performance. The experimental results show HRaft can significantly reduce storage and network costs and improve write performance while keeping the liveness level compared to CRaft.
Research Area(s)
- Consensus protocol, Erasure coding, Fault tolerance, Network storage, Paxos, Raft
Citation Format(s)
HRaft: Adaptive Erasure Coded Data Maintenance for Consensus in Distributed Networks. / Jia, Yulei; Xu, Guangping; Sung, Chi Wan et al.
Proceedings - 2022 IEEE 36th International Parallel and Distributed Processing Symposium, IPDPS 2022. Institute of Electrical and Electronics Engineers, Inc., 2022. p. 1316-1326 (Proceedings - IEEE International Parallel and Distributed Processing Symposium, IPDPS).
Proceedings - 2022 IEEE 36th International Parallel and Distributed Processing Symposium, IPDPS 2022. Institute of Electrical and Electronics Engineers, Inc., 2022. p. 1316-1326 (Proceedings - IEEE International Parallel and Distributed Processing Symposium, IPDPS).
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review