Entropy-Based Scheduling Policy for Cross Aggregate Ranking Workloads

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal

View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)507-520
Journal / PublicationIEEE Transactions on Services Computing
Volume11
Issue number3
Online published29 Jun 2016
Publication statusPublished - May 2018

Abstract

Many data exploration applications require the ability to identify the top-k results according to a scoring function. We study a class of top-k ranking problems where top-k candidates in a dataset are scored with the assistance of another set. We call this class of workloads cross aggregate ranking. Example computation problems include evaluating the Hausdorff distance between two datasets, finding the medoid or radius within one dataset, and finding the closest or farthest pair between two datasets. In this paper, we propose a parallel and distributed solution to process cross aggregate ranking workloads. Our solution subdivides the aggregate score computation of each candidate into tasks while constantly maintains the tentative top-k results as an uncertain top-k result set. The crux of our proposed approach lies in our entropy-based scheduling technique to determine result-yielding tasks based on their abilities to reduce the uncertainty of the tentative result set. Experimental results show that our proposed approach consistently outperforms the best existing one in two different types of cross aggregate rank workloads using real datasets.

Research Area(s)

  • knowledge and data engineering tools and techniques, Query processing