Towards a Microblogs Data Management System

Amr Magdy, Mohamed Mokbel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

This paper advocates for the need to build a Microblogs Data Management System (MDMS) as an end-to-end data management system to support indexing, querying, and analyzing microblogs, e.g., Tweets, comments, or check-in's. We identify a set of characteristics for microblogging environments that are distinguishing from any other data management environment. Then, we propose a system architecture for the first Microblogs Data Management System, which includes indexing, querying, and recovery components. The indexing component is responsible for indexing recent data in memory, indexing older data in disk, and synchronizing the flow of data from memory to disk without affecting the query response time. The querying component is responsible for retrieving the query answer from both memory and disk storage as well as employing online selectivity estimation techniques tuned to the behavior of microblogs data. The recovery module allows for efficiently storing and processing incoming microblogs in memory without worrying about data loss.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages271-278
Number of pages8
Volume1
ISBN (Electronic)9781479999729
DOIs
Publication statusPublished - 11 Sep 2015
Externally publishedYes
Event16th IEEE International Conference on Mobile Data Management, MDM 2015 - Pittsburgh, United States
Duration: 15 Jun 201518 Jun 2015

Other

Other16th IEEE International Conference on Mobile Data Management, MDM 2015
CountryUnited States
CityPittsburgh
Period15/6/1518/6/15

    Fingerprint

Keywords

  • Big Data
  • Data Management Systems
  • Data Stream
  • Indexing
  • Keyword
  • Microblogs
  • Query Processing
  • Spatial
  • Temporal
  • Top-k

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Magdy, A., & Mokbel, M. (2015). Towards a Microblogs Data Management System. In Proceedings - 2015 IEEE 16th International Conference on Mobile Data Management, MDM 2015 (Vol. 1, pp. 271-278). [7264332] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MDM.2015.24