Distance-based outlier detection: Consolidation and renewed bearing

Gustavo H. Orair, Carlos H.C. Teixeira, Wagner Meira, Ye Wang, Srinivasan Parthasarathy

Research output: Contribution to journalArticle

67 Citations (Scopus)

Abstract

Detecting outliers in data is an important problem with interesting applications in a myriad of domains ranging from data cleaning to financial fraud detection and from network intrusion detection to clinical diagnosis of diseases. Over the last decade of research, distance-based outlier detection algorithms have emerged as a viable, scalable, parameter-free alternative to the more traditional statistical approaches. In this paper we assess several distance-based outlier detection approaches and evaluate them. We begin by surveying and examining the design landscape of extant approaches, while identifying key design decisions of such approaches. We then implement an outlier detection framework and conduct a factorial design experiment to understand the pros and cons of various optimizations proposed by us as well as those proposed in the literature, both independently and in conjunction with one another, on a diverse set of real-life datasets. To the best of our knowledge this is the first such study in the literature. The outcome of this study is a family of state of the art distance-based outlier detection algorithms. Our detailed empirical study supports the following observations. The combination of optimization strategies enables significant efficiency gains. Our factorial design study highlights the important fact that no single optimization or combination of optimizations (factors) always dominates on all types of data. Our study also allows us to characterize when a certain combination of optimizations is likely to prevail and helps provide interesting and useful insights for moving forward in this domain.

Original languageEnglish
Pages (from-to)1469-1480
Number of pages12
JournalProceedings of the VLDB Endowment
Volume3
Issue number2
Publication statusPublished - 1 Sep 2010

    Fingerprint

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Orair, G. H., Teixeira, C. H. C., Meira, W., Wang, Y., & Parthasarathy, S. (2010). Distance-based outlier detection: Consolidation and renewed bearing. Proceedings of the VLDB Endowment, 3(2), 1469-1480.