scholarly journals Indirectly Supervised Anomaly Detection of Clinically Meaningful Health Events from Smart Home Data

2021 ◽  
Vol 12 (2) ◽  
pp. 1-18
Author(s):  
Jessamyn Dahmen ◽  
Diane J. Cook
Keyword(s):  
Anomaly Detection ◽  
Time Series Data ◽  
Bayesian Optimization ◽  
Detection Methods ◽  
Series Data ◽  
Detection Techniques ◽  
Health Events ◽  
Warm Start ◽  
Health Related ◽  
Unsupervised Algorithms

Anomaly detection techniques can extract a wealth of information about unusual events. Unfortunately, these methods yield an abundance of findings that are not of interest, obscuring relevant anomalies. In this work, we improve upon traditional anomaly detection methods by introducing Isudra, an Indirectly Supervised Detector of Relevant Anomalies from time series data. Isudra employs Bayesian optimization to select time scales, features, base detector algorithms, and algorithm hyperparameters that increase true positive and decrease false positive detection. This optimization is driven by a small amount of example anomalies, driving an indirectly supervised approach to anomaly detection. Additionally, we enhance the approach by introducing a warm-start method that reduces optimization time between similar problems. We validate the feasibility of Isudra to detect clinically relevant behavior anomalies from over 2M sensor readings collected in five smart homes, reflecting 26 health events. Results indicate that indirectly supervised anomaly detection outperforms both supervised and unsupervised algorithms at detecting instances of health-related anomalies such as falls, nocturia, depression, and weakness.

scholarly journals Recent Advances in Anomaly Detection Methods Applied to Aviation

Aerospace ◽  
2019 ◽  
Vol 6 (11) ◽  
pp. 117 ◽  
Author(s):  
Luis Basora ◽  
Xavier Olive ◽  
Thomas Dubot
Keyword(s):  
Anomaly Detection ◽  
Time Series Data ◽  
Data Driven ◽  
Sensor Data ◽  
Detection Methods ◽  
Series Data ◽  
Detection Techniques ◽  
Advantages And Disadvantages ◽  
Recent Advances ◽  
Flight Operations

Anomaly detection is an active area of research with numerous methods and applications. This survey reviews the state-of-the-art of data-driven anomaly detection techniques and their application to the aviation domain. After a brief introduction to the main traditional data-driven methods for anomaly detection, we review the recent advances in the area of neural networks, deep learning and temporal-logic based learning. In particular, we cover unsupervised techniques applicable to time series data because of their relevance to the aviation domain, where the lack of labeled data is the most usual case, and the nature of flight trajectories and sensor data is sequential, or temporal. The advantages and disadvantages of each method are presented in terms of computational efficiency and detection efficacy. The second part of the survey explores the application of anomaly detection techniques to aviation and their contributions to the improvement of the safety and performance of flight operations and aviation systems. As far as we know, some of the presented methods have not yet found an application in the aviation domain. We review applications ranging from the identification of significant operational events in air traffic operations to the prediction of potential aviation system failures for predictive maintenance.

scholarly journals Recent Advances in Anomaly Detection Methods applied to Aviation

2019 ◽  
Author(s):  
Luis Basora ◽  
Xavier Olive ◽  
Thomas Dubot
Keyword(s):  
Anomaly Detection ◽  
Time Series Data ◽  
Data Driven ◽  
Sensor Data ◽  
Detection Methods ◽  
Series Data ◽  
Detection Techniques ◽  
Advantages And Disadvantages ◽  
Recent Advances ◽  
Flight Operations

Anomaly detection is an active area of research with numerous methods and applications. This survey reviews the state-of-the-art of data-driven anomaly detection techniques and their application to the the aviation domain. After a brief introduction to the main traditional data-driven methods for anomaly detection, we review the recent advances in the area of neural networks, deep learning and temporal-logic based learning. We cover especially unsupervised techniques applicable to time series data because of their relevance to the aviation domain, where the lack of labeled data is the most usual case, and the nature of flight trajectories and sensor data is sequential, or temporal. The advantages and disadvantages of each method are presented in terms of computational efficiency and detection efficacy. The second part of the survey explores the application of anomaly detection techniques to aviation and their contributions to the improvement of the safety and performance of flight operations and aviation systems. As far as we know, some of the presented methods have not yet found an application in the aviation domain. We review applications ranging from the identification of significant operational events in air traffic operations to the prediction of potential aviation system failures for predictive maintenance.

scholarly journals A Hybrid Deep Learning Framework for Unsupervised Anomaly Detection in Multivariate Spatio-Temporal Data

2020 ◽  
Vol 10 (15) ◽  
pp. 5191
Author(s):  
Yıldız Karadayı ◽  
Mehmet N. Aydin ◽  
A. Selçuk Öğrenci
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Hurricane Katrina ◽  
Time Series Data ◽  
Hybrid Approach ◽  
Ground Truth ◽  
Outbreak Detection ◽  
Series Data ◽  
Detection Techniques ◽  
Spatio Temporal

Multivariate time-series data with a contextual spatial attribute have extensive use for finding anomalous patterns in a wide variety of application domains such as earth science, hurricane tracking, fraud, and disease outbreak detection. In most settings, spatial context is often expressed in terms of ZIP code or region coordinates such as latitude and longitude. However, traditional anomaly detection techniques cannot handle more than one contextual attribute in a unified way. In this paper, a new hybrid approach based on deep learning is proposed to solve the anomaly detection problem in multivariate spatio-temporal dataset. It works under the assumption that no prior knowledge about the dataset and anomalies are available. The architecture of the proposed hybrid framework is based on an autoencoder scheme, and it is more efficient in extracting features from the spatio-temporal multivariate datasets compared to the traditional spatio-temporal anomaly detection techniques. We conducted extensive experiments using buoy data of 2005 from National Data Buoy Center and Hurricane Katrina as ground truth. Experiments demonstrate that the proposed model achieves more than 10% improvement in accuracy over the methods used in the comparison where our model jointly processes the spatial and temporal dimensions of the contextual data to extract features for anomaly detection.

Variance error of multi-classification based anomaly detection for time series data

2020 ◽  
pp. 1-16
Author(s):  
Baoquan Wang ◽  
Tonghai Jiang ◽  
Xi Zhou ◽  
Bo Ma ◽  
Fan Zhao ◽  
...  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Short Term Memory ◽  
Computational Cost ◽  
Reconstruction Error ◽  
Detection Methods ◽  
Series Data ◽  
Data Set

For abnormal detection of time series data, the supervised anomaly detection methods require labeled data. While the range of outlier factors used by the existing semi-supervised methods varies with data, model and time, the threshold for determining abnormality is difficult to obtain, in addition, the computational cost of the way to calculate outlier factors from other data points in the data set is also very large. These make such methods difficult to practically apply. This paper proposes a framework named LSTM-VE which uses clustering combined with visualization method to roughly label normal data, and then uses the normal data to train long short-term memory (LSTM) neural network for semi-supervised anomaly detection. The variance error (VE) of the normal data category classification probability sequence is used as outlier factor. The framework enables anomaly detection based on deep learning to be practically applied and using VE avoids the shortcomings of existing outlier factors and gains a better performance. In addition, the framework is easy to expand because the LSTM neural network can be replaced with other classification models. Experiments on the labeled and real unlabeled data sets prove that the framework is better than replicator neural networks with reconstruction error (RNN-RS) and has good scalability as well as practicability.

scholarly journals Evaluation metrics for anomaly detection algorithms in time-series

2019 ◽  
Vol 11 (2) ◽  
pp. 113-130 ◽  
Author(s):  
György Kovács ◽  
Gheorghe Sebestyen ◽  
Anca Hangan
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Confusion Matrix ◽  
Series Data ◽  
Time Data ◽  
Temporal Dimension ◽  
Detection Techniques ◽  
Detection Algorithms

Abstract Time-series are ordered sequences of discrete-time data. Due to their temporal dimension, anomaly detection techniques used in time-series have to take into consideration time correlations and other time-related particularities. Generally, in order to evaluate the quality of an anomaly detection technique, the confusion matrix and its derived metrics such as precision and recall are used. These metrics, however, do not take this temporal dimension into consideration. In this paper, we propose three metrics that can be used to evaluate the quality of a classification, while accounting for the temporal dimension found in time-series data.

scholarly journals Deep Quantile Regression for Unsupervised Anomaly Detection in Time-Series

2021 ◽  
Vol 2 (6) ◽  
Author(s):  
Ahmad Idris Tambuwal ◽  
Daniel Neagu
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Quantile Regression ◽  
Gaussian Distribution ◽  
Time Series Data ◽  
Prediction Interval ◽  
Data Interpretation ◽  
Research Literature ◽  
Detection Methods ◽  
Series Data

AbstractTime-series anomaly detection receives increasing research interest given the growing number of data-rich application domains. Recent additions to anomaly detection methods in research literature include deep neural networks (DNNs: e.g., RNN, CNN, and Autoencoder). The nature and performance of these algorithms in sequence analysis enable them to learn hierarchical discriminative features and time-series temporal nature. However, their performance is affected by usually assuming a Gaussian distribution on the prediction error, which is either ranked, or threshold to label data instances as anomalous or not. An exact parametric distribution is often not directly relevant in many applications though. This will potentially produce faulty decisions from false anomaly predictions due to high variations in data interpretation. The expectations are to produce outputs characterized by a level of confidence. Thus, implementations need the Prediction Interval (PI) that quantify the level of uncertainty associated with the DNN point forecasts, which helps in making better-informed decision and mitigates against false anomaly alerts. An effort has been made in reducing false anomaly alerts through the use of quantile regression for identification of anomalies, but it is limited to the use of quantile interval to identify uncertainties in the data. In this paper, an improve time-series anomaly detection method called deep quantile regression anomaly detection (DQR-AD) is proposed. The proposed method go further to used quantile interval (QI) as anomaly score and compare it with threshold to identify anomalous points in time-series data. The tests run of the proposed method on publicly available anomaly benchmark datasets demonstrate its effective performance over other methods that assumed Gaussian distribution on the prediction or reconstruction cost for detection of anomalies. This shows that our method is potentially less sensitive to data distribution than existing approaches.

scholarly journals Towards Machine Learning-based Anomaly Detection on Time-Series Data

2021 ◽  
Vol 13 (1) ◽  
pp. 35-44
Author(s):  
Daniel Vajda ◽  
Adrian Pekar ◽  
Karoly Farkas
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Anomaly Detection ◽  
Time Series Data ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Detection Methods ◽  
Series Data ◽  
Rich Information

The complexity of network infrastructures is exponentially growing. Real-time monitoring of these infrastructures is essential to secure their reliable operation. The concept of telemetry has been introduced in recent years to foster this process by streaming time-series data that contain feature-rich information concerning the state of network components. In this paper, we focus on a particular application of telemetry — anomaly detection on time-series data. We rigorously examined state-of-the-art anomaly detection methods. Upon close inspection of the methods, we observed that none of them suits our requirements as they typically face several limitations when applied on time-series data. This paper presents Alter-Re2, an improved version of ReRe, a state-of-the-art Long Short- Term Memory-based machine learning algorithm. Throughout a systematic examination, we demonstrate that by introducing the concepts of ageing and sliding window, the major limitations of ReRe can be overcome. We assessed the efficacy of Alter-Re2 using ten different datasets and achieved promising results. Alter-Re2 performs three times better on average when compared to ReRe.

scholarly journals STATISTICAL ANOMALY DETECTION FOR MONITORING OF HUMAN DYNAMICS

2015 ◽  
Vol XL-4/W5 ◽  
pp. 93-98
Author(s):  
K. Kamiya ◽  
T. Fuse
Keyword(s):  
Anomaly Detection ◽  
Traffic Control ◽  
Time Series Data ◽  
Detection Method ◽  
Population Data ◽  
Detection Methods ◽  
Series Data ◽  
Location Information ◽  
Detection Problem ◽  
Human Dynamics

Understanding of human dynamics has drawn attention to various areas. Due to the wide spread of positioning technologies that use GPS or public Wi-Fi, location information can be obtained with high spatial-temporal resolution as well as at low cost. By collecting set of individual location information in real time, monitoring of human dynamics is recently considered possible and is expected to lead to dynamic traffic control in the future. Although this monitoring focuses on detecting anomalous states of human dynamics, anomaly detection methods are developed ad hoc and not fully systematized. This research aims to define an anomaly detection problem of the human dynamics monitoring with gridded population data and develop an anomaly detection method based on the definition. According to the result of a review we have comprehensively conducted, we discussed the characteristics of the anomaly detection of human dynamics monitoring and categorized our problem to a semi-supervised anomaly detection problem that detects contextual anomalies behind time-series data. We developed an anomaly detection method based on a sticky HDP-HMM, which is able to estimate the number of hidden states according to input data. Results of the experiment with synthetic data showed that our proposed method has good fundamental performance with respect to the detection rate. Through the experiment with real gridded population data, an anomaly was detected when and where an actual social event had occurred.

Sign in / Sign up

Export Citation Format

Share Document