M 3

Stream processing on main-memory MapReduce

Ahmed M. Aly, Asmaa Sallam, Bala M. Gnanasekaran, Long Van Nguyen-Dinh, Walid G. Aref, Mourad Ouzzani, Arif Ghafoor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

40 Citations (Scopus)

Abstract

The continuous growth of social web applications along with the development of sensor capabilities in electronic devices is creating countless opportunities to analyze the enormous amounts of data that is continuously steaming from these applications and devices. To process large scale data on large scale computing clusters, MapReduce has been introduced as a framework for parallel computing. However, most of the current implementations of the MapReduce framework support only the execution of fixed-input jobs. Such restriction makes these implementations inapplicable for most streaming applications, in which queries are continuous in nature, and input data streams are continuously received at high arrival rates. In this demonstration, we showcase M 3, a prototype implementation of the MapReduce framework in which continuous queries over streams of data can be efficiently answered. M 3 extends Hadoop, the open source implementation of MapReduce, bypassing the Hadoop Distributed File System (HDFS) to support main-memory-only processing. Moreover, M 3 supports continuous execution of the Map and Reduce phases where individual Mappers and Reducers never terminate.

Original languageEnglish
Title of host publicationProceedings - International Conference on Data Engineering
Pages1253-1256
Number of pages4
DOIs
Publication statusPublished - 2012
EventIEEE 28th International Conference on Data Engineering, ICDE 2012 - Arlington, VA, United States
Duration: 1 Apr 20125 Apr 2012

Other

OtherIEEE 28th International Conference on Data Engineering, ICDE 2012
CountryUnited States
CityArlington, VA
Period1/4/125/4/12

Fingerprint

Data storage equipment
Processing
Cluster computing
Parallel processing systems
Demonstrations
Sensors

ASJC Scopus subject areas

  • Information Systems
  • Signal Processing
  • Software

Cite this

Aly, A. M., Sallam, A., Gnanasekaran, B. M., Nguyen-Dinh, L. V., Aref, W. G., Ouzzani, M., & Ghafoor, A. (2012). M 3: Stream processing on main-memory MapReduce. In Proceedings - International Conference on Data Engineering (pp. 1253-1256). [6228181] https://doi.org/10.1109/ICDE.2012.120

M 3 : Stream processing on main-memory MapReduce. / Aly, Ahmed M.; Sallam, Asmaa; Gnanasekaran, Bala M.; Nguyen-Dinh, Long Van; Aref, Walid G.; Ouzzani, Mourad; Ghafoor, Arif.

Proceedings - International Conference on Data Engineering. 2012. p. 1253-1256 6228181.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Aly, AM, Sallam, A, Gnanasekaran, BM, Nguyen-Dinh, LV, Aref, WG, Ouzzani, M & Ghafoor, A 2012, M 3: Stream processing on main-memory MapReduce. in Proceedings - International Conference on Data Engineering., 6228181, pp. 1253-1256, IEEE 28th International Conference on Data Engineering, ICDE 2012, Arlington, VA, United States, 1/4/12. https://doi.org/10.1109/ICDE.2012.120
Aly AM, Sallam A, Gnanasekaran BM, Nguyen-Dinh LV, Aref WG, Ouzzani M et al. M 3: Stream processing on main-memory MapReduce. In Proceedings - International Conference on Data Engineering. 2012. p. 1253-1256. 6228181 https://doi.org/10.1109/ICDE.2012.120
Aly, Ahmed M. ; Sallam, Asmaa ; Gnanasekaran, Bala M. ; Nguyen-Dinh, Long Van ; Aref, Walid G. ; Ouzzani, Mourad ; Ghafoor, Arif. / M 3 : Stream processing on main-memory MapReduce. Proceedings - International Conference on Data Engineering. 2012. pp. 1253-1256
@inproceedings{20741b6658d34bbd84f5c19aa903def7,
title = "M 3: Stream processing on main-memory MapReduce",
abstract = "The continuous growth of social web applications along with the development of sensor capabilities in electronic devices is creating countless opportunities to analyze the enormous amounts of data that is continuously steaming from these applications and devices. To process large scale data on large scale computing clusters, MapReduce has been introduced as a framework for parallel computing. However, most of the current implementations of the MapReduce framework support only the execution of fixed-input jobs. Such restriction makes these implementations inapplicable for most streaming applications, in which queries are continuous in nature, and input data streams are continuously received at high arrival rates. In this demonstration, we showcase M 3, a prototype implementation of the MapReduce framework in which continuous queries over streams of data can be efficiently answered. M 3 extends Hadoop, the open source implementation of MapReduce, bypassing the Hadoop Distributed File System (HDFS) to support main-memory-only processing. Moreover, M 3 supports continuous execution of the Map and Reduce phases where individual Mappers and Reducers never terminate.",
author = "Aly, {Ahmed M.} and Asmaa Sallam and Gnanasekaran, {Bala M.} and Nguyen-Dinh, {Long Van} and Aref, {Walid G.} and Mourad Ouzzani and Arif Ghafoor",
year = "2012",
doi = "10.1109/ICDE.2012.120",
language = "English",
pages = "1253--1256",
booktitle = "Proceedings - International Conference on Data Engineering",

}

TY - GEN

T1 - M 3

T2 - Stream processing on main-memory MapReduce

AU - Aly, Ahmed M.

AU - Sallam, Asmaa

AU - Gnanasekaran, Bala M.

AU - Nguyen-Dinh, Long Van

AU - Aref, Walid G.

AU - Ouzzani, Mourad

AU - Ghafoor, Arif

PY - 2012

Y1 - 2012

N2 - The continuous growth of social web applications along with the development of sensor capabilities in electronic devices is creating countless opportunities to analyze the enormous amounts of data that is continuously steaming from these applications and devices. To process large scale data on large scale computing clusters, MapReduce has been introduced as a framework for parallel computing. However, most of the current implementations of the MapReduce framework support only the execution of fixed-input jobs. Such restriction makes these implementations inapplicable for most streaming applications, in which queries are continuous in nature, and input data streams are continuously received at high arrival rates. In this demonstration, we showcase M 3, a prototype implementation of the MapReduce framework in which continuous queries over streams of data can be efficiently answered. M 3 extends Hadoop, the open source implementation of MapReduce, bypassing the Hadoop Distributed File System (HDFS) to support main-memory-only processing. Moreover, M 3 supports continuous execution of the Map and Reduce phases where individual Mappers and Reducers never terminate.

AB - The continuous growth of social web applications along with the development of sensor capabilities in electronic devices is creating countless opportunities to analyze the enormous amounts of data that is continuously steaming from these applications and devices. To process large scale data on large scale computing clusters, MapReduce has been introduced as a framework for parallel computing. However, most of the current implementations of the MapReduce framework support only the execution of fixed-input jobs. Such restriction makes these implementations inapplicable for most streaming applications, in which queries are continuous in nature, and input data streams are continuously received at high arrival rates. In this demonstration, we showcase M 3, a prototype implementation of the MapReduce framework in which continuous queries over streams of data can be efficiently answered. M 3 extends Hadoop, the open source implementation of MapReduce, bypassing the Hadoop Distributed File System (HDFS) to support main-memory-only processing. Moreover, M 3 supports continuous execution of the Map and Reduce phases where individual Mappers and Reducers never terminate.

UR - http://www.scopus.com/inward/record.url?scp=84864197919&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84864197919&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2012.120

DO - 10.1109/ICDE.2012.120

M3 - Conference contribution

SP - 1253

EP - 1256

BT - Proceedings - International Conference on Data Engineering

ER -