Ordinal Anomaly Probability Algorithm for Anomaly Detection Problems of Massive Data Sets

陈刚 1

您当前的位置：

首页 >

文章列表页 >

Ordinal Anomaly Probability Algorithm for Anomaly Detection Problems of Massive Data Sets

更新时间：2025-11-17

- Ordinal Anomaly Probability Algorithm for Anomaly Detection Problems of Massive Data Sets
- Vol. 45, Issue 4, Pages: 36-40(2011)
- 作者机构：
  
  西安交通大学电子与信息工程学院,西安,710049
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP393
- Published：2011
- 稿件说明：
移动端阅览
陈刚 1. Ordinal Anomaly Probability Algorithm for Anomaly Detection Problems of Massive Data Sets[J]. 2011, 45(4): 36-40.
DOI：

陈刚 1. Ordinal Anomaly Probability Algorithm for Anomaly Detection Problems of Massive Data Sets[J]. 2011, 45(4): 36-40. DOI：

摘要

针对异常检测算法速度慢、精度低、稳定性差等问题

提出了一种通过异常概率排序提取异常点的算法(OAP).由于异常点相对正常点更容易通过对数据空间的均匀分割而孤立出来

所以OAP通过数据点在均匀N叉分割树中的孤立深度估算异常概率的大小

从而得到异常概率的排序

最终构造由k个异常概率最大的点组成的列表

列表中的数据就是所求的异常点.OAP不需要距离或密度的计算

复杂度被降到O(n)级.实验结果表明

对于规模线性增加的海量实验数据集

OAP消耗的CPU时间也线性增加; 相对iForest算法

其速度提高了30倍

精度提高了20%～30%

且同一数据集上的多次实验结果一致

稳定性高.

Abstract

An ordinal anomaly probability method(OAP)is proposed to improve the efficiency

effectiveness and stability of existing anomaly detection algorithms. Since anomalies are easier to be isolated by uniformly partitioning the data space

the order of anomaly probabilities can be evaluated in terms of isolation depths in uniform N-ary partition trees. Then

the k largest probable anomalies are extracted. OAP can ignore the evaluation of distance and density

and hence reduces the complexity to O(n). Experimental results show that the CPU time of OAP increases linearly with a linearly growing data set. Furthermore

comparisons show that OAP is 30 times faster than iForest and is much more stable

while its accuracy is improved by 20%-30%.

关键词

Keywords

references

ESKIN E, ARNOLD A, PRERAU M, et al. A geometric framework for unsupervised anomaly detection [J]. Advances in Information Security, 2002, 56(4): 21-31.

RICHARD B, DAVID H. Statistical fraud detection: a review [J]. Statistical Science,2002,17(3): 235-255.

LI Xiaolie, LI Zhenhui, HAN Jiawei. Temporal outlier detection in vehicle traffic data [C]∥Proc 2009 Int Conf on Data Engineering(ICDE'09). Piscataway, NJ, USA: IEEE, 2009: 1319-1322.

MARCEL P, ELIZABETH B, SEAN H, et al. A brain tumor segmentation framework based on outlier detection [J]. Medical Image Analysis, 2004, 8(3): 275-283.

HAN Jiawei, MICHELINE K. Data mining: concepts and techniques [M]. Singapore: Elsevier, 2006: 5-9.

MARKUS M, HANS P, RAYMOND T, et al. LOF: identifying density-based local outliers [C]∥Proc ACM SIGMOD'00.New York, USA: ACM, 2000: 93-104.

LIU Feitong, TING Kaiming. Isolation forest [C]∥Proceedings of IEEE International Conference on Data Mining ICDM'08. Piscataway, NJ, USA: IEEE, 2008: 413-422.

YÜ Xiao, TANG Lu'an, HAN Jiawei. Filtering and refinement: a two-stage approach for efficient and effective anomaly detection [C]∥Proc Int Conf on Data Mining ICDM'09. Piscataway, NJ, USA: IEEE, 2009: 90-104.

HO Y C, SREENIVAS R. Ordinal optimization of DEDS [J]. Discrete Event Dynamic Systems, 1992, 45(9): 61-88.

祝颂和, 陆诗娣. 离散数学 [M]. 西安: 西安交通大学出版社, 1991: 6.

KNUTH D, Art of computer programming [M]. New York, USA: Addison-Wesley, 1998: 51-53.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Sensitivity Analysis of Foil Geometrical Parameters for the Third-Generation CO2 Gas Foil Bearings Based on Design of Experiments Method

Study on the Thermodynamic Properties in the Anomalous Region of Supercritical Water

Turbulent Kinetic Energy Dissipation Mechanism of Supercritical Carbon Dioxide Leakage Flow in a Scallop Bionic Seal Cavity

Effects of Expansion Angle Crater Film Hole on the Film Cooling Characteristics of a Double-Wall Structure

Knowledge Graph Representation Learning Driven by Semantic Discrimination and Subgraph Integration

Related Author

No data

Related Institution

No data

⁰