Group anomaly detection using deep generative models

Raghavendra Chalapathy, Edward Toth, Sanjay Chawla

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Unlike conventional anomaly detection research that focuses on point anomalies, our goal is to detect anomalous collections of individual data points. In particular, we perform group anomaly detection (GAD) with an emphasis on irregular group distributions (e.g. irregular mixtures of image pixels). GAD is an important task in detecting unusual and anomalous phenomena in real-world applications such as high energy particle physics, social media and medical imaging. In this paper, we take a generative approach by proposing deep generative models: Adversarial autoencoder (AAE) and variational autoencoder (VAE) for group anomaly detection. Both AAE and VAE detect group anomalies using point-wise input data where group memberships are known a priori. We conduct extensive experiments to evaluate our models on real world datasets. The empirical results demonstrate that our approach is effective and robust in detecting group anomalies. Code related to this paper is available at: https://github.com/raghavchalapathy/gad, https://www.cs.cmu.edu/~lxiong/gad/gad.html, https://github.com/jorjasso/SMDD-group-anomaly-detection, https://github.com/cjlin1/libsvm.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings
EditorsFrancesco Bonchi, Thomas Gärtner, Neil Hurley, Georgiana Ifrim, Michele Berlingerio
PublisherSpringer Verlag
Pages173-189
Number of pages17
ISBN (Print)9783030109240
DOIs
Publication statusPublished - 1 Jan 2019
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2018 - Dublin, Ireland
Duration: 10 Sep 201814 Sep 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11051 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2018
CountryIreland
CityDublin
Period10/9/1814/9/18

Fingerprint

Generative Models
High energy physics
Anomaly Detection
Medical imaging
Pixels
Anomaly
Anomalous
Irregular
Experiments
Particle Physics
Social Media
Medical Imaging
Real-world Applications
High Energy
Pixel
Evaluate

Keywords

  • Adversarial
  • Auto-encoders
  • Group anomaly detection
  • Variational

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Chalapathy, R., Toth, E., & Chawla, S. (2019). Group anomaly detection using deep generative models. In F. Bonchi, T. Gärtner, N. Hurley, G. Ifrim, & M. Berlingerio (Eds.), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings (pp. 173-189). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11051 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-030-10925-7_11

Group anomaly detection using deep generative models. / Chalapathy, Raghavendra; Toth, Edward; Chawla, Sanjay.

Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings. ed. / Francesco Bonchi; Thomas Gärtner; Neil Hurley; Georgiana Ifrim; Michele Berlingerio. Springer Verlag, 2019. p. 173-189 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11051 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chalapathy, R, Toth, E & Chawla, S 2019, Group anomaly detection using deep generative models. in F Bonchi, T Gärtner, N Hurley, G Ifrim & M Berlingerio (eds), Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11051 LNAI, Springer Verlag, pp. 173-189, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD 2018, Dublin, Ireland, 10/9/18. https://doi.org/10.1007/978-3-030-10925-7_11
Chalapathy R, Toth E, Chawla S. Group anomaly detection using deep generative models. In Bonchi F, Gärtner T, Hurley N, Ifrim G, Berlingerio M, editors, Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings. Springer Verlag. 2019. p. 173-189. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-10925-7_11
Chalapathy, Raghavendra ; Toth, Edward ; Chawla, Sanjay. / Group anomaly detection using deep generative models. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings. editor / Francesco Bonchi ; Thomas Gärtner ; Neil Hurley ; Georgiana Ifrim ; Michele Berlingerio. Springer Verlag, 2019. pp. 173-189 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{43cff415ea494081b56159d7b31baf23,
title = "Group anomaly detection using deep generative models",
abstract = "Unlike conventional anomaly detection research that focuses on point anomalies, our goal is to detect anomalous collections of individual data points. In particular, we perform group anomaly detection (GAD) with an emphasis on irregular group distributions (e.g. irregular mixtures of image pixels). GAD is an important task in detecting unusual and anomalous phenomena in real-world applications such as high energy particle physics, social media and medical imaging. In this paper, we take a generative approach by proposing deep generative models: Adversarial autoencoder (AAE) and variational autoencoder (VAE) for group anomaly detection. Both AAE and VAE detect group anomalies using point-wise input data where group memberships are known a priori. We conduct extensive experiments to evaluate our models on real world datasets. The empirical results demonstrate that our approach is effective and robust in detecting group anomalies. Code related to this paper is available at: https://github.com/raghavchalapathy/gad, https://www.cs.cmu.edu/~lxiong/gad/gad.html, https://github.com/jorjasso/SMDD-group-anomaly-detection, https://github.com/cjlin1/libsvm.",
keywords = "Adversarial, Auto-encoders, Group anomaly detection, Variational",
author = "Raghavendra Chalapathy and Edward Toth and Sanjay Chawla",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-10925-7_11",
language = "English",
isbn = "9783030109240",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "173--189",
editor = "Francesco Bonchi and Thomas G{\"a}rtner and Neil Hurley and Georgiana Ifrim and Michele Berlingerio",
booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings",

}

TY - GEN

T1 - Group anomaly detection using deep generative models

AU - Chalapathy, Raghavendra

AU - Toth, Edward

AU - Chawla, Sanjay

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Unlike conventional anomaly detection research that focuses on point anomalies, our goal is to detect anomalous collections of individual data points. In particular, we perform group anomaly detection (GAD) with an emphasis on irregular group distributions (e.g. irregular mixtures of image pixels). GAD is an important task in detecting unusual and anomalous phenomena in real-world applications such as high energy particle physics, social media and medical imaging. In this paper, we take a generative approach by proposing deep generative models: Adversarial autoencoder (AAE) and variational autoencoder (VAE) for group anomaly detection. Both AAE and VAE detect group anomalies using point-wise input data where group memberships are known a priori. We conduct extensive experiments to evaluate our models on real world datasets. The empirical results demonstrate that our approach is effective and robust in detecting group anomalies. Code related to this paper is available at: https://github.com/raghavchalapathy/gad, https://www.cs.cmu.edu/~lxiong/gad/gad.html, https://github.com/jorjasso/SMDD-group-anomaly-detection, https://github.com/cjlin1/libsvm.

AB - Unlike conventional anomaly detection research that focuses on point anomalies, our goal is to detect anomalous collections of individual data points. In particular, we perform group anomaly detection (GAD) with an emphasis on irregular group distributions (e.g. irregular mixtures of image pixels). GAD is an important task in detecting unusual and anomalous phenomena in real-world applications such as high energy particle physics, social media and medical imaging. In this paper, we take a generative approach by proposing deep generative models: Adversarial autoencoder (AAE) and variational autoencoder (VAE) for group anomaly detection. Both AAE and VAE detect group anomalies using point-wise input data where group memberships are known a priori. We conduct extensive experiments to evaluate our models on real world datasets. The empirical results demonstrate that our approach is effective and robust in detecting group anomalies. Code related to this paper is available at: https://github.com/raghavchalapathy/gad, https://www.cs.cmu.edu/~lxiong/gad/gad.html, https://github.com/jorjasso/SMDD-group-anomaly-detection, https://github.com/cjlin1/libsvm.

KW - Adversarial

KW - Auto-encoders

KW - Group anomaly detection

KW - Variational

UR - http://www.scopus.com/inward/record.url?scp=85061156104&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061156104&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-10925-7_11

DO - 10.1007/978-3-030-10925-7_11

M3 - Conference contribution

SN - 9783030109240

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 173

EP - 189

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2018, Proceedings

A2 - Bonchi, Francesco

A2 - Gärtner, Thomas

A2 - Hurley, Neil

A2 - Ifrim, Georgiana

A2 - Berlingerio, Michele

PB - Springer Verlag

ER -