Adding value to sensor data of civil engineering structures: Automatic outlier detection
1st Workshop on Machine Learning, Intelligent Systems and Statistical Analysis for Pattern Recognition in Real-life Scenarios (ML-ISAPR 2018), organized in the 9th International Conference on Information, Intelligence, Systems and Applications (IISA 2018)
This paper discusses the problem of outlier detection in datasets generated by sensors installed in large civil engi- neering structures. Since outlier detection can be implemented after the acquisition process, it is fully independent of particular acquisition processes as well as it scales to new or updated sensors. It shows a method of using machine learning tech- niques to implement an automatic outlier detection procedure, demonstrating and evaluating the results in a real environ- ment, following the Design Science Research Methodology. The proposed approach makes use of Manual Acquisition System measurements and combine them with a clustering algorithm (DBSCAN) and baseline methods (Multiple Linear Regression and thresholds based on standard deviation) to create a method that is able to identify and remove most of the outliers in the datasets used for demonstration and evaluation. This automatic procedure improves data quality having a direct impact on the decision processes with regard to structural safety.
outlier detection,sensor data,machine learning,data mining