Please use this identifier to cite or link to this item:
https://hdl.handle.net/2440/113370
Citations | ||
Scopus | Web of Science® | Altmetric |
---|---|---|
?
|
?
|
Type: | Conference paper |
Title: | LSHiForest: A generic framework for fast tree isolation based ensemble anomaly analysis |
Author: | Zhang, X. Dou, W. He, Q. Zhou, R. Leckie, C. Kotagiri, R. Salcic, Z. |
Citation: | Proceedings / International Conference on Data Engineering. International Conference on Data Engineering, 2017, pp.983-994 |
Publisher: | IEEE |
Publisher Place: | Online |
Issue Date: | 2017 |
Series/Report no.: | International Conference on Data Engineering. Proceedings |
ISBN: | 150906544X 9781509065448 |
ISSN: | 1084-4627 2375-026X |
Conference Name: | IEEE 33rd International Conference on Data Engineering (ICDE 2017) (19 Apr 2017 - 22 Apr 2017 : San Diego, CA) |
Statement of Responsibility: | Xuyun Zhang, Wanchun Dou, Qiang He, Rui Zhou, Christopher Leckie, Ramamohanarao Kotagiri, Zoran Salcic |
Abstract: | Anomaly or outlier detection is a major challenge in big data analytics because anomaly patterns provide valuable insights for decision-making in a wide range of applications. Recently proposed anomaly detection methods based on the tree isolation mechanism are very fast due to their logarithmic time complexity, making them capable of handling big data sets efficiently. However, the underlying similarity or distance measures in these methods have not been well understood. Contrary to the claims that these methods never rely on any distance measure, we find that they have close relationships with certain distance measures. This implies that the current use of this fast isolation mechanism is only limited to these distance measures and fails to generalise to other commonlyused measures. In this paper, we propose a generic framework named LSHiForest for fast tree isolation based ensemble anomaly analysis with the use of a Locality-Sensitive Hashing (LSH) forest. Being generic, the proposed framework can be instantiated with a diverse range of LSH families, and the fast isolation mechanism can be extended to any distance measures, data types and data spaces where an LSH family is defined. In particular, the instances of our framework with kernelised LSH families or learning based hashing schemes can detect complicated anomalies like local or surrounded anomalies. We also formally show that the existing tree isolation based detection methods are special cases of our framework with the corresponding distance measures. Extensive experiments on both synthetic and real-world benchmark data sets show that the framework can achieve both high time efficiency and anomaly detection quality. |
Rights: | © 2017 IEEE |
DOI: | 10.1109/ICDE.2017.145 |
Grant ID: | http://purl.org/au-research/grants/arc/DP170104747 http://purl.org/au-research/grants/arc/DP170101932 |
Published version: | https://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7929494 |
Appears in Collections: | Aurora harvest 8 Computer Science publications |
Files in This Item:
There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.