WMFP-Outlier: An Efficient Maximal Frequent-Pattern-Based Outlier Detection Approach for Weighted Data Streams

Since outliers are the major factors that affect accuracy in data science, many outlier detection approaches have been proposed for effectively identifying the implicit outliers from static datasets, thereby improving the reliability of the data. In recent years, data streams have been the main form of data, and the data elements in a data stream are not always of equal importance. However, the existing outlier detection approaches do not consider the weight conditions; hence, these methods are not suitable for processing weighted data streams. In addition, the traditional pattern-based outlier detection approaches incur a high time cost in the outlier detection phase. Aiming at overcoming these problems, this paper proposes a two-phase pattern-based outlier detection approach, namely, WMFP-Outlier, for effectively detecting the implicit outliers from a weighted data stream, in which the maximal frequent patterns are used instead of the frequent patterns to accelerate the process of outlier detection. In the process of maximal frequent-pattern mining, the anti-monotonicity property and MFP-array structure are used to accelerate the mining operation. In the process of outlier detection, three deviation indices are designed for measuring the degree of abnormality of each transaction, and the transactions with the highest degrees of abnormality are judged as outliers. Last, several experimental studies are conducted on a synthetic dataset to evaluate the performance of the proposed WMFP-Outlier approach. The results demonstrate that the accuracy of the WMFP-Outlier approach is higher compared to the existing pattern-based outlier detection approaches, and the time cost of the outlier detection phase of WMFP-Outlier is lower than those of the other four compared pattern-based outlier detection approaches.

Approximate Maximal Frequent Pattern Mining with Weight Conditions and Error Tolerance

Since the concept of frequent pattern mining was proposed, there have been many efforts to obtain useful pattern information from large databases. As one of them, applying weight conditions allows us to mine weighted frequent patterns considering unique importance of each item composing databases, and the result of analysis for the patterns provides more useful information than that of considering only frequency or support information. However, although this approach gives us more meaningful pattern information, the number of patterns found from large databases is extremely large in general; therefore, analyzing all of them may become inefficient and hard work. Thus, it is essential to apply a method that can selectively extract representative patterns from the enormous ones. Moreover, in the real-world applications, unexpected errors such as noise may occur, which can have a negative effect on the values of databases. Although the changes by the error are quite small, the characteristics of generated patterns can be turned definitely. For this reason, we propose a novel algorithm that can solve the above problems, called AWMax (an algorithm for mining Approximate weighted maximal frequent patterns (AWMFPs) considering error tolerance). Through the algorithm, we can obtain useful AWMFPs regardless of noise because of the consideration of error tolerance. Comprehensive performance experiments present that the proposed algorithm has more outstanding performance than previous state-of-the-art ones.

Mining Maximal Frequent Patterns over Data Stream Based on Time Decaying

A new algorithm DSMFP-Miner was proposed. When the data stream reach continuously, a maximal frequent pattern tree called MFP-Tree is adopted to maintain the transactions in data screams dynamically. Transactions in the same Transaction-sensitive sliding window are set to own the same “importance”. Besides, the support of the transactions in old window is decayed to reduce their influence to mining results, and infrequent patterns and overdue patterns are deleted periodically. In the mining process, the algorithm put an enumeration tree with each node of MFP-Tree as root as the search space, and use the "depth-first" search strategy to mining the maximal frequent patterns with this node as a suffix.

Diversifying Search Results through Pattern-Based Subtopic Modeling

Traditional information retrieval models do not necessarily provide users with optimal search experience because the top ranked documents may contain excessively redundant information. Therefore, satisfying search results should be not only relevant to the query but also diversified to cover different subtopics of the query. In this paper, the authors propose a novel pattern-based framework to diversify search results, where each pattern is a set of semantically related terms covering the same subtopic. They first apply a maximal frequent pattern mining algorithm to extract the patterns from retrieval results of the query. The authors then propose to model a subtopic with either a single pattern or a group of similar patterns. A profile-based clustering method is adapted to group similar patterns based on their context information. The search results are then diversified using the extracted subtopics. Experimental results show that the proposed pattern-based methods are effective to diversify the search results.