Managing Heterogeneous Sensor Data on a Big Data Platform: IoT Services for Data-intensive Science

Big data has emerged as a key connecting point between things and objects on the internet. In this cyber-physical space, different types of sensors interact over wireless networks, collecting data and delivering services ranging from environmental pollution monitoring, disaster management and recovery, improving the quality of life in homes, to enabling smart cities to function. However, despite the perceived benefits we are realizing from these sensors, the dawn of the Internet of Things (IoT) brings fresh challenges. Some of these have to do with designing the appropriate infrastructure to capture and store the huge amount of heterogeneous sensor data, finding practical use of the collected sensor data, and managing IoT communities in such a way that users can seamlessly search, find, and utilize their sensor data. In order to address these challenges, this paper describes an integrated IoT architecture that combines the functionalities of Service-Controlled Networking (SCN) with cloud computing. The resulting community-driven big data platform helps environmental scientists easily discover and manage data from various sensors, and share their knowledge and experience relating to air pollution impacts. Our experience in managing the platform and communities provides a proof of concept and best practice guidelines on how to manage IoT services in a data-intensive research environment.