A Continuous Data Imputation Mechanism based on Streams Correlation

Abstract

The increased adoption of the Internet of Things (IoT) for the delivery of intelligent applications over huge volumes of data opens new opportunities to draw conclusions from data and support efficient decision making. For this reason many applications have been developed for data collection and processing. A large part of them are aligned with the requirements of the vast infrastructure of IoT. However, one of the biggest problems occurring at real-time applications is that they are prone to missing values. Missing values can negatively affect the outcomes of any processing activity, thus, they can limit the performance of IoT applications. In this paper, we depart from the relevant literature and propose a data imputation model that is based on the correlation of data reported by different IoT devices. Our aim is to support data imputation using the 'knowledge' of a team of IoT devices over their reports for various phenomena. Our scheme adopts a continuous correlation detection methodology applied at real time reports of the involved devices. Hence, any missing value can be replaced by the aggregated outcome of data reported by correlated devices. We provide the description of our approach and evaluate it through a high number of simulations adopting various experimental scenarios. © 2020 IEEE.