Adding value to sensor data of civil engineering structures: automatic outlier detection
1st Workshop on Machine Learning, Intelligent Systems and Statistical Analysis for Pattern Recognition in Real-life Scenarios, ML-ISAPR 2018
This paper discusses the problem of outlier detection in datasets generated by sensors installed in large civil engineering structures. Since outlier detection can be implemented after the acquisition process, it is fully independent of particular acquisition processes as well as it scales to new or updated sensors. It shows a method of using machine learning techniques to implement an automatic outlier detection procedure, demonstrating and evaluating the results in a real environment, following the Design Science Research Methodology. The proposed approach makes use of Manual Acquisition System measurements and combine them with a clustering algorithm (DBSCAN) and baseline methods (Multiple Linear Regression and thresholds based on standard deviation) to create a method that is able to identify and remove most of the outliers in the datasets used for demonstration and evaluation. This automatic procedure improves data quality having a direct impact on the decision processes with regard to structural safety.
Outlier detection,Sensor data,Machine learning,Data mining
Classificação Fields of Science and Technology
- Ciências Físicas - Ciências Naturais