Abstract
In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 106-115 |
| Number of pages | 10 |
| Volume | 2018-June |
| ISBN (Electronic) | 9781538641330 |
| DOIs | |
| Publication status | Published - 13 Jul 2018 |
| Event | 19th IEEE International Conference on Mobile Data Management, MDM 2018 - Aalborg, Denmark Duration: 26 Jun 2018 → 28 Jun 2018 |
Other
| Other | 19th IEEE International Conference on Mobile Data Management, MDM 2018 |
|---|---|
| Country | Denmark |
| City | Aalborg |
| Period | 26/6/18 → 28/6/18 |
Fingerprint
Keywords
- big data
- data decaying
- data reduction
- machine learning
- spatio-temporal analytics
- telco
ASJC Scopus subject areas
- Engineering(all)
Cite this
Decaying telco big data with data postdiction. / Costa, Constantinos; Charalampous, Andreas; Konstantinidis, Andreas; Zeinalipour-Yazti, Demetrios; Mokbel, Mohamed.
Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018. Vol. 2018-June Institute of Electrical and Electronics Engineers Inc., 2018. p. 106-115.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Decaying telco big data with data postdiction
AU - Costa, Constantinos
AU - Charalampous, Andreas
AU - Konstantinidis, Andreas
AU - Zeinalipour-Yazti, Demetrios
AU - Mokbel, Mohamed
PY - 2018/7/13
Y1 - 2018/7/13
N2 - In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.
AB - In this paper, we present a novel decaying operator for Telco Big Data (TBD), coined TBD-DP (Data Postdiction). Unlike data prediction, which aims to make a statement about the future value of some tuple, our formulated data postdiction term, aims to make a statement about the past value of some tuple, which does not exist anymore as it had to be deleted to free up disk space. TBD-DP relies on existing Machine Learning (ML) algorithms to abstract TBD into compact models that can be stored and queried when necessary. Our proposed TBD-DP operator has the following two conceptual phases: (i) in an offline phase, it utilizes a LSTM-based hierarchical ML algorithm to learn a tree of models (coined TBD-DP tree) over time and space; (ii) in an online phase, it uses the TBD-DP tree to recover data within a certain accuracy. In our experimental setup, we measure the efficiency of the proposed operator using a ∼10GB anonymized real telco network trace and our experimental results in Tensorflow over HDFS are extremely encouraging as they show that TBD-DP saves an order of magnitude storage space while maintaining a high accuracy on the recovered data.
KW - big data
KW - data decaying
KW - data reduction
KW - machine learning
KW - spatio-temporal analytics
KW - telco
UR - http://www.scopus.com/inward/record.url?scp=85050810761&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050810761&partnerID=8YFLogxK
U2 - 10.1109/MDM.2018.00027
DO - 10.1109/MDM.2018.00027
M3 - Conference contribution
VL - 2018-June
SP - 106
EP - 115
BT - Proceedings - 2018 IEEE 19th International Conference on Mobile Data Management, MDM 2018
PB - Institute of Electrical and Electronics Engineers Inc.
ER -