Minor Probability Events' Detection in Big Data

An Integrated Approach with Bayes Detection and MIM

Shuo Wan, Jiaxun Lu, Pingyi Fan, Khaled Letaief

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

In some common scenarios, rare events tend to be more important and worth careful detection. In this case, minor probability event detection rises up as an important issue in big data analytics. With the existence of empirical prior probability distributions of events, one common method to minimize the average detection error is Bayes detection. However, in extreme cases where the prior probability is small, the Bayes method may result in high miss detection rate. To overcome this problem, we proposed a modified detection algorithm by integrating the Bayes method with the message importance measure. Simulation results show that the algorithm significantly reduces the miss detection rate of extremely rare events, which may be helpful to dig rarely occurring phenomenons in big data.

Original languageEnglish
Article number8629291
Pages (from-to)418-421
Number of pages4
JournalIEEE Communications Letters
Volume23
Issue number3
DOIs
Publication statusPublished - 1 Mar 2019
Externally publishedYes

Fingerprint

Event Detection
Bayes
Minor
Bayes Method
Error detection
Prior Probability
Rare Events
Probability distributions
Error Detection
Prior distribution
Extremes
Probability Distribution
Big data
Tend
Minimise
Scenarios
Simulation

Keywords

  • false alarm rate
  • Message importance
  • minor probability
  • miss detection rate

ASJC Scopus subject areas

  • Modelling and Simulation
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Minor Probability Events' Detection in Big Data : An Integrated Approach with Bayes Detection and MIM. / Wan, Shuo; Lu, Jiaxun; Fan, Pingyi; Letaief, Khaled.

In: IEEE Communications Letters, Vol. 23, No. 3, 8629291, 01.03.2019, p. 418-421.

Research output: Contribution to journalArticle

@article{3377c4f70f68407d952aa89646374827,
title = "Minor Probability Events' Detection in Big Data: An Integrated Approach with Bayes Detection and MIM",
abstract = "In some common scenarios, rare events tend to be more important and worth careful detection. In this case, minor probability event detection rises up as an important issue in big data analytics. With the existence of empirical prior probability distributions of events, one common method to minimize the average detection error is Bayes detection. However, in extreme cases where the prior probability is small, the Bayes method may result in high miss detection rate. To overcome this problem, we proposed a modified detection algorithm by integrating the Bayes method with the message importance measure. Simulation results show that the algorithm significantly reduces the miss detection rate of extremely rare events, which may be helpful to dig rarely occurring phenomenons in big data.",
keywords = "false alarm rate, Message importance, minor probability, miss detection rate",
author = "Shuo Wan and Jiaxun Lu and Pingyi Fan and Khaled Letaief",
year = "2019",
month = "3",
day = "1",
doi = "10.1109/LCOMM.2019.2895828",
language = "English",
volume = "23",
pages = "418--421",
journal = "IEEE Communications Letters",
issn = "1089-7798",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "3",

}

TY - JOUR

T1 - Minor Probability Events' Detection in Big Data

T2 - An Integrated Approach with Bayes Detection and MIM

AU - Wan, Shuo

AU - Lu, Jiaxun

AU - Fan, Pingyi

AU - Letaief, Khaled

PY - 2019/3/1

Y1 - 2019/3/1

N2 - In some common scenarios, rare events tend to be more important and worth careful detection. In this case, minor probability event detection rises up as an important issue in big data analytics. With the existence of empirical prior probability distributions of events, one common method to minimize the average detection error is Bayes detection. However, in extreme cases where the prior probability is small, the Bayes method may result in high miss detection rate. To overcome this problem, we proposed a modified detection algorithm by integrating the Bayes method with the message importance measure. Simulation results show that the algorithm significantly reduces the miss detection rate of extremely rare events, which may be helpful to dig rarely occurring phenomenons in big data.

AB - In some common scenarios, rare events tend to be more important and worth careful detection. In this case, minor probability event detection rises up as an important issue in big data analytics. With the existence of empirical prior probability distributions of events, one common method to minimize the average detection error is Bayes detection. However, in extreme cases where the prior probability is small, the Bayes method may result in high miss detection rate. To overcome this problem, we proposed a modified detection algorithm by integrating the Bayes method with the message importance measure. Simulation results show that the algorithm significantly reduces the miss detection rate of extremely rare events, which may be helpful to dig rarely occurring phenomenons in big data.

KW - false alarm rate

KW - Message importance

KW - minor probability

KW - miss detection rate

UR - http://www.scopus.com/inward/record.url?scp=85062978387&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062978387&partnerID=8YFLogxK

U2 - 10.1109/LCOMM.2019.2895828

DO - 10.1109/LCOMM.2019.2895828

M3 - Article

VL - 23

SP - 418

EP - 421

JO - IEEE Communications Letters

JF - IEEE Communications Letters

SN - 1089-7798

IS - 3

M1 - 8629291

ER -