Detecting anomalies and de-noising monitoring data from sensors: A smart data approach

Weili Fang , Yixiao Shao , Peter E.D. Love , Timo Hartmann , Wenli Liu

Abstract

When monitoring safety levels in deep pit foundations using sensors, anomalies (e.g., highly correlated variables) and noise (e.g., high dimensionality) exist in the extracted time series data, impacting the ability to assess risks. Our research aims to address the following question: How can we detect anomalies and de-noise monitoring data from sensors in real time to improve its quality and use it to assess geotechnical safety risks? In addressing this research question, we develop a hybrid smart data approach that integrates Extended Isolation Forest and Variational Mode Decomposition models to detect anomalies and de-noise data effectively. We use real-life data obtained from sensors to validate our smart data approach while constructing a deep pit foundation. Our smart data approach can detect anomalies with a root mean square error and signal-to-noise ratio of 0.0389 and 24.09, respectively. To this end, our smart data approach can effectively pre-process data enabling improved decision-making and the management of safety risks.