A highly efficient data locality aware task scheduler for cloud-based systems

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)Not applicablepeer-review

1 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE International Conference on Cloud Computing - IEEE CLOUD 2019 - Part of the 2019 IEEE World Congress on Services
PublisherIEEE
Pages496-498
ISBN (Electronic)978-1-7281-2705-7
Publication statusPublished - Jul 2019

Publication series

NameIEEE International Conference on Cloud Computing, CLOUD
Volume2019-July
ISSN (Print)2159-6182
ISSN (Electronic)2159-6190

Conference

Title12th IEEE International Conference on Cloud Computing, CLOUD 2019
PlaceItaly
CityMilan
Period8 - 13 July 2019

Abstract

Scheduling tasks in the vicinity of stored data can significantly diminish network traffic. Scheduling optimisation can improve data locality by attempting to locate a task and its related data on the same node. Existing schedulers tend to ignore overhead and tradeoff between data transfer and task placement, and bandwidth consumption, by only emphasising data locality without considering other factors. We present a novel data locality aware scheduler for balancing time consumption and network bandwidth traffic-DLAforBT-to improve data locality for tasks and throughput, with the optimal placement policy exhibiting a threshold-based structure. DLAforBT uses bipartite graph modelling to represent data placement, adopts a judgment mechanism and a precise prediction model to determine moving data or moving computation. It integrates an improved Dominant Resource Fairness (DRF) resource allocation to capture tenants' resource allocation and run as many jobs as possible. DLAforBT improves by 16% of data locality rate, and 25% of throughput.

Research Area(s)

  • Bipar tite graph modelling, Cloud computing, Data locality, Multi-tenancy, Scheduling

Citation Format(s)

A highly efficient data locality aware task scheduler for cloud-based systems. / Ru, Jia; Yang, Yun; Grundy, John; Keung, Jacky; Hao, Li.

Proceedings - 2019 IEEE International Conference on Cloud Computing - IEEE CLOUD 2019 - Part of the 2019 IEEE World Congress on Services. IEEE, 2019. p. 496-498 8814565 (IEEE International Conference on Cloud Computing, CLOUD; Vol. 2019-July).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)Not applicablepeer-review