Decaying telco big data with data postdiction

Constantinos Costa, Andreas Charalampous, Andreas Konstantinidis, Demetrios Zeinalipour-Yazti, Mohamed Mokbel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages106-115
Number of pages10
Volume2018-June
ISBN (Electronic)9781538641330
DOIs
Publication statusPublished - 13 Jul 2018
Event19th IEEE International Conference on Mobile Data Management, MDM 2018 - Aalborg, Denmark
Duration: 26 Jun 201828 Jun 2018

Other

Other19th IEEE International Conference on Mobile Data Management, MDM 2018
CountryDenmark
CityAalborg
Period26/6/1828/6/18

Fingerprint

Learning algorithms
Learning systems
Big data

Keywords

  • big data
  • data decaying
  • data reduction
  • machine learning
  • spatio-temporal analytics
  • telco

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Costa, C., Charalampous, A., Konstantinidis, A., Zeinalipour-Yazti, D., & Mokbel, M. (2018). Decaying telco big data with data postdiction. In Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018 (Vol. 2018-June, pp. 106-115). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MDM.2018.00027

Decaying telco big data with data postdiction. / Costa, Constantinos; Charalampous, Andreas; Konstantinidis, Andreas; Zeinalipour-Yazti, Demetrios; Mokbel, Mohamed.

Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018. Vol. 2018-June Institute of Electrical and Electronics Engineers Inc., 2018. p. 106-115.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Costa, C, Charalampous, A, Konstantinidis, A, Zeinalipour-Yazti, D & Mokbel, M 2018, Decaying telco big data with data postdiction. in Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018. vol. 2018-June, Institute of Electrical and Electronics Engineers Inc., pp. 106-115, 19th IEEE International Conference on Mobile Data Management, MDM 2018, Aalborg, Denmark, 26/6/18. https://doi.org/10.1109/MDM.2018.00027
Costa C, Charalampous A, Konstantinidis A, Zeinalipour-Yazti D, Mokbel M. Decaying telco big data with data postdiction. In Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018. Vol. 2018-June. Institute of Electrical and Electronics Engineers Inc. 2018. p. 106-115 https://doi.org/10.1109/MDM.2018.00027
Costa, Constantinos ; Charalampous, Andreas ; Konstantinidis, Andreas ; Zeinalipour-Yazti, Demetrios ; Mokbel, Mohamed. / Decaying telco big data with data postdiction. Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018. Vol. 2018-June Institute of Electrical and Electronics Engineers Inc., 2018. pp. 106-115
@inproceedings{79a6af817ad14ca1b219d530eadb2aca,
title = "Decaying telco big data with data postdiction",
abstract = "In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.",
keywords = "big data, data decaying, data reduction, machine learning, spatio-temporal analytics, telco",
author = "Constantinos Costa and Andreas Charalampous and Andreas Konstantinidis and Demetrios Zeinalipour-Yazti and Mohamed Mokbel",
year = "2018",
month = "7",
day = "13",
doi = "10.1109/MDM.2018.00027",
language = "English",
volume = "2018-June",
pages = "106--115",
booktitle = "Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Decaying telco big data with data postdiction

AU - Costa, Constantinos

AU - Charalampous, Andreas

AU - Konstantinidis, Andreas

AU - Zeinalipour-Yazti, Demetrios

AU - Mokbel, Mohamed

PY - 2018/7/13

Y1 - 2018/7/13

N2 - In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.

AB - In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.

KW - big data

KW - data decaying

KW - data reduction

KW - machine learning

KW - spatio-temporal analytics

KW - telco

UR - http://www.scopus.com/inward/record.url?scp=85050810761&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050810761&partnerID=8YFLogxK

U2 - 10.1109/MDM.2018.00027

DO - 10.1109/MDM.2018.00027

M3 - Conference contribution

AN - SCOPUS:85050810761

VL - 2018-June

SP - 106

EP - 115

BT - Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -