M 3: Stream processing on main-memory MapReduce

Ahmed M. Aly, Asmaa Sallam, Bala M. Gnanasekaran, Long Van Nguyen-Dinh, Walid G. Aref, Mourad Ouzzani, Arif Ghafoor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

41 Citations (Scopus)

Abstract

The continuous growth of social web applications along with the development of sensor capabilities in electronic devices is creating countless opportunities to analyze the enormous amounts of data that is continuously steaming from these applications and devices. To process large scale data on large scale computing clusters, MapReduce has been introduced as a framework for parallel computing. However, most of the current implementations of the MapReduce framework support only the execution of fixed-input jobs. Such restriction makes these implementations inapplicable for most streaming applications, in which queries are continuous in nature, and input data streams are continuously received at high arrival rates. In this demonstration, we showcase M 3, a prototype implementation of the MapReduce framework in which continuous queries over streams of data can be efficiently answered. M 3 extends Hadoop, the open source implementation of MapReduce, bypassing the Hadoop Distributed File System (HDFS) to support main-memory-only processing. Moreover, M 3 supports continuous execution of the Map and Reduce phases where individual Mappers and Reducers never terminate.

Original languageEnglish
Title of host publicationProceedings - International Conference on Data Engineering
Pages1253-1256
Number of pages4
DOIs
Publication statusPublished - 2012
EventIEEE 28th International Conference on Data Engineering, ICDE 2012 - Arlington, VA, United States
Duration: 1 Apr 20125 Apr 2012

Other

OtherIEEE 28th International Conference on Data Engineering, ICDE 2012
CountryUnited States
CityArlington, VA
Period1/4/125/4/12

    Fingerprint

ASJC Scopus subject areas

  • Information Systems
  • Signal Processing
  • Software

Cite this

Aly, A. M., Sallam, A., Gnanasekaran, B. M., Nguyen-Dinh, L. V., Aref, W. G., Ouzzani, M., & Ghafoor, A. (2012). M 3: Stream processing on main-memory MapReduce. In Proceedings - International Conference on Data Engineering (pp. 1253-1256). [6228181] https://doi.org/10.1109/ICDE.2012.120