Effective Metadata Management in Federated Sensor Networks

As the Sensor Internet starts to become reality, increasingly heterogeneous sensor networks are becoming interconnected into federated sensor networks and provide huge volumes of sensor data for a variety of applications to large user communities, such as in science and engineering. Effective metadata management plays a crucial role in processing and properly interpreting raw sensor measurement data, and needs to be performed in a collaborative fashion. Previous work on metadata management for experimental and measurement data has not provided specific support for joint real-time processing of metadata and sensor data. In this paper we propose a framework that allows effective sensor metadata management based on realtime metadata creation and processing over federated sensor networks. The framework is based on three key mechanisms: (i) Distributed metadata joins to allow streaming sensor data to be efficiently processed with associated metadata, regardless of their location in the network. (ii) Automated metadata generation to permit users to define monitoring conditions or operations for extracting and storing metadata from streaming sensor data. (iii) Advanced metadata search utilizing various techniques specifically designed for sensor metadata querying and visualization. The framework is currently deployed and used as the backbone of a concrete application in environmental science and engineering, the Swiss Experiment, which runs a wide variety of measurements and experiments for environmental hazard forecasting and warning.