Towards a Microblogs Data Management System

Amr Magdy, Mohamed Mokbel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages271-278
Number of pages8
Volume1
ISBN (Electronic)9781479999729
DOIs
Publication statusPublished - 11 Sep 2015
Externally publishedYes
Event16th IEEE International Conference on Mobile Data Management, MDM 2015 - Pittsburgh, United States
Duration: 15 Jun 201518 Jun 2015

Other

Other16th IEEE International Conference on Mobile Data Management, MDM 2015
CountryUnited States
CityPittsburgh
Period15/6/1518/6/15

Fingerprint

Information management
Data storage equipment
Response time (computer systems)
Recovery
Processing

Keywords

  • Big Data
  • Data Management Systems
  • Data Stream
  • Indexing
  • Keyword
  • Microblogs
  • Query Processing
  • Spatial
  • Temporal
  • Top-k

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Magdy, A., & Mokbel, M. (2015). Towards a Microblogs Data Management System. In Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015 (Vol. 1, pp. 271-278). [7264332] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MDM.2015.24

Towards a Microblogs Data Management System. / Magdy, Amr; Mokbel, Mohamed.

Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015. Vol. 1 Institute of Electrical and Electronics Engineers Inc., 2015. p. 271-278 7264332.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Magdy, A & Mokbel, M 2015, Towards a Microblogs Data Management System. in Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015. vol. 1, 7264332, Institute of Electrical and Electronics Engineers Inc., pp. 271-278, 16th IEEE International Conference on Mobile Data Management, MDM 2015, Pittsburgh, United States, 15/6/15. https://doi.org/10.1109/MDM.2015.24
Magdy A, Mokbel M. Towards a Microblogs Data Management System. In Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015. Vol. 1. Institute of Electrical and Electronics Engineers Inc. 2015. p. 271-278. 7264332 https://doi.org/10.1109/MDM.2015.24
Magdy, Amr ; Mokbel, Mohamed. / Towards a Microblogs Data Management System. Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015. Vol. 1 Institute of Electrical and Electronics Engineers Inc., 2015. pp. 271-278
@inproceedings{7a213dee4f804ab89338283ca3b7978a,
title = "Towards a Microblogs Data Management System",
abstract = "This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.",
keywords = "Big Data, Data Management Systems, Data Stream, Indexing, Keyword, Microblogs, Query Processing, Spatial, Temporal, Top-k",
author = "Amr Magdy and Mohamed Mokbel",
year = "2015",
month = "9",
day = "11",
doi = "10.1109/MDM.2015.24",
language = "English",
volume = "1",
pages = "271--278",
booktitle = "Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Towards a Microblogs Data Management System

AU - Magdy, Amr

AU - Mokbel, Mohamed

PY - 2015/9/11

Y1 - 2015/9/11

N2 - This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

AB - This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

KW - Big Data

KW - Data Management Systems

KW - Data Stream

KW - Indexing

KW - Keyword

KW - Microblogs

KW - Query Processing

KW - Spatial

KW - Temporal

KW - Top-k

UR - http://www.scopus.com/inward/record.url?scp=84958169629&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84958169629&partnerID=8YFLogxK

U2 - 10.1109/MDM.2015.24

DO - 10.1109/MDM.2015.24

M3 - Conference contribution

AN - SCOPUS:84958169629

VL - 1

SP - 271

EP - 278

BT - Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -