Efficient big data processing in Hadoop MapReduce

Research output: Chapter in Book/Report/Conference proceedingChapter

127 Citations (Scopus)

Abstract

This tutorial is motivated by the clear need of many organizations, companies, and researchers to deal with big data volumes efficiently. Examples include web analytics applications, scientific applications, and social networks. A popular data processing engine for big data is Hadoop MapReduce. Early versions of Hadoop MapReduce suffered from severe performance problems. Today, this is becoming history. There are many techniques that can be used with Hadoop MapReduce jobs to boost performance by orders of magnitude. In this tutorial we teach such techniques. First, we will briefly familiarize the audience with Hadoop MapReduce and motivate its use for big data processing. Then, we will focus on different data management techniques, going from job optimization to physical data organization like data layouts and indexes. Throughout this tutorial, we will highlight the similarities and differences between Hadoop MapReduce and Parallel DBMS. Furthermore, we will point out unresolved research problems and open issues.

Original languageEnglish
Title of host publicationProceedings of the VLDB Endowment
Pages2014-2015
Number of pages2
Volume5
Edition12
Publication statusPublished - Aug 2012
Externally publishedYes

    Fingerprint

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Dittrich, J., & Quiane Ruiz, J. A. (2012). Efficient big data processing in Hadoop MapReduce. In Proceedings of the VLDB Endowment (12 ed., Vol. 5, pp. 2014-2015)