On Improving Constrained Single and Group Operator Placement Using Evictions in Big Data Environments

Abstract

With an ever increasing amount of data generated by scientific experiments, social networks and mobile as well as wireless sensor networks, reducing resource consumption by big data applications becomes of paramount importance. Towards this end, filtering data close to the data sources is a common strategy in order to reduce network traffic. Assuming a network of nodes, each potentially generating data and a query in the form of a single operator to be applied in these data, the basic statement of the operator placement problem is: find the best node to place the operator so that the network traffic is minimized. In this paper we study the problem of placing a set of communicating operators exhibiting a tree structure over a tree network of nodes with capacity constraints. We take advantage of our previous work on unconstrained placement in order to develop a new approach enabling both single and group operator migrations using evictions of hosted operators if free space is required. To enhance their applicability, the algorithms work in a distributed asynchronous manner, requiring only minimal knowledge at each network node. Results from simulation experiments show that the proposed algorithms reduce considerably network overhead against their counterparts. © 2016 IEEE.