On the Synchronization Bottleneck of OpenStack Swift-Like Cloud Storage Systems

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal

2 Scopus Citations
View graph of relations

Author(s)

  • Mingkang Ruan
  • Thierry Titcheu
  • Ennan Zhai
  • Zhenhua Li
  • Yao Liu
  • Jinlong E
  • Yong Cui

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)2059-2074
Journal / PublicationIEEE Transactions on Parallel and Distributed Systems
Volume29
Issue number9
Online published27 Feb 2018
Publication statusPublished - Sep 2018

Abstract

As one type of the most popular cloud storage services, OpenStack Swift and its follow-up systems replicate each object across multiple storage nodes and leverage object sync protocols to achieve high reliability and eventual consistency. The performance of object sync protocols heavily relies on two key parameters: r (number of replicas for each object) and n (number of objects hosted by each storage node). In existing tutorials and demos, the configurations are usually r = 3 and n < 1000 by default, and the sync process seems to perform well. However, we discover in data-intensive scenarios, e.g., when r > 3 and n 1000, the sync process is significantly delayed and produces massive network overhead, referred to as the sync bottleneck problem. By reviewing the source code of OpenStack Swift, we find that its object sync protocol utilizes a fairly simple and network-intensive approach to check the consistency among replicas of objects. Hence in a sync round, the number of exchanged hash values per node is Θ(n × r). To tackle the problem, we propose a lightweight and practical object sync protocol, LightSync, which not only remarkably reduces the sync overhead, but also preserves high reliability and eventual consistency. LightSync derives this capability from three novel building blocks: 1) Hashing of Hashes, which aggregates all the h hash values of each data partition into a single but representative hash value with the Merkle tree; 2) Circular Hash Checking, which checks the consistency of different partition replicas by only sending the aggregated hash value to the clockwise neighbor; and 3) Failed Neighbor Handling, which properly detects and handles node failures with moderate overhead to effectively strengthen the robustness of LightSync. The design of LightSync offers provable guarantee on reducing the per-node network overhead from Θ(n × r) to Θ(n/h). Furthermore, we have implemented LightSync as an open-source patch and adopted it to OpenStack Swift, thus reducing the sync delay by up to 879× and the network overhead by up to 47.5×.

Research Area(s)

  • Cloud computing, Cloud storage, Delays, Electronic mail, object synchronization, Open source software, OpenStack Swift, performance bottleneck, Protocols, Reliability, Synchronization

Citation Format(s)

On the Synchronization Bottleneck of OpenStack Swift-Like Cloud Storage Systems. / Ruan, Mingkang; Titcheu, Thierry; Zhai, Ennan; Li, Zhenhua; Liu, Yao; E, Jinlong; Cui, Yong; Xu, Hong.

In: IEEE Transactions on Parallel and Distributed Systems, Vol. 29, No. 9, 09.2018, p. 2059-2074.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal