SpatialHadoop: A MapReduce framework for spatial data

Ahmed Eldawy, Mohamed Mokbel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

243 Citations (Scopus)

Abstract

This paper describes SpatialHadoop; a full-fledged MapReduce framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, MapReduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the MapReduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

Original languageEnglish
Title of host publication2015 IEEE 31st International Conference on Data Engineering, ICDE 2015
PublisherIEEE Computer Society
Pages1352-1363
Number of pages12
Volume2015-May
ISBN (Electronic)9781479979639
DOIs
Publication statusPublished - 1 Jan 2015
Externally publishedYes
Event2015 31st IEEE International Conference on Data Engineering, ICDE 2015 - Seoul, Korea, Republic of
Duration: 13 Apr 201517 Apr 2015

Other

Other2015 31st IEEE International Conference on Data Engineering, ICDE 2015
CountryKorea, Republic of
CitySeoul
Period13/4/1517/4/15

Fingerprint

High level languages
Experiments

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Cite this

Eldawy, A., & Mokbel, M. (2015). SpatialHadoop: A MapReduce framework for spatial data. In 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015 (Vol. 2015-May, pp. 1352-1363). [7113382] IEEE Computer Society. https://doi.org/10.1109/ICDE.2015.7113382

SpatialHadoop : A MapReduce framework for spatial data. / Eldawy, Ahmed; Mokbel, Mohamed.

2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. Vol. 2015-May IEEE Computer Society, 2015. p. 1352-1363 7113382.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Eldawy, A & Mokbel, M 2015, SpatialHadoop: A MapReduce framework for spatial data. in 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. vol. 2015-May, 7113382, IEEE Computer Society, pp. 1352-1363, 2015 31st IEEE International Conference on Data Engineering, ICDE 2015, Seoul, Korea, Republic of, 13/4/15. https://doi.org/10.1109/ICDE.2015.7113382
Eldawy A, Mokbel M. SpatialHadoop: A MapReduce framework for spatial data. In 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. Vol. 2015-May. IEEE Computer Society. 2015. p. 1352-1363. 7113382 https://doi.org/10.1109/ICDE.2015.7113382
Eldawy, Ahmed ; Mokbel, Mohamed. / SpatialHadoop : A MapReduce framework for spatial data. 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. Vol. 2015-May IEEE Computer Society, 2015. pp. 1352-1363
@inproceedings{1f509acb897b4b0683abf995e09e7649,
title = "SpatialHadoop: A MapReduce framework for spatial data",
abstract = "This paper describes SpatialHadoop; a full-fledged MapReduce framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, MapReduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the MapReduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.",
author = "Ahmed Eldawy and Mohamed Mokbel",
year = "2015",
month = "1",
day = "1",
doi = "10.1109/ICDE.2015.7113382",
language = "English",
volume = "2015-May",
pages = "1352--1363",
booktitle = "2015 IEEE 31st International Conference on Data Engineering, ICDE 2015",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - SpatialHadoop

T2 - A MapReduce framework for spatial data

AU - Eldawy, Ahmed

AU - Mokbel, Mohamed

PY - 2015/1/1

Y1 - 2015/1/1

N2 - This paper describes SpatialHadoop; a full-fledged MapReduce framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, MapReduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the MapReduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

AB - This paper describes SpatialHadoop; a full-fledged MapReduce framework with native support for spatial data. SpatialHadoop is a comprehensive extension to Hadoop that injects spatial data awareness in each Hadoop layer, namely, the language, storage, MapReduce, and operations layers. In the language layer, SpatialHadoop adds a simple and expressive high level language for spatial data types and operations. In the storage layer, SpatialHadoop adapts traditional spatial index structures, Grid, R-tree and R+-tree, to form a two-level spatial index. SpatialHadoop enriches the MapReduce layer by two new components, SpatialFileSplitter and SpatialRecordReader, for efficient and scalable spatial data processing. In the operations layer, SpatialHadoop is already equipped with a dozen of operations, including range query, kNN, and spatial join. Other spatial operations are also implemented following a similar approach. Extensive experiments on real system prototype and real datasets show that SpatialHadoop achieves orders of magnitude better performance than Hadoop for spatial data processing.

UR - http://www.scopus.com/inward/record.url?scp=84940864358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940864358&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2015.7113382

DO - 10.1109/ICDE.2015.7113382

M3 - Conference contribution

AN - SCOPUS:84940864358

VL - 2015-May

SP - 1352

EP - 1363

BT - 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015

PB - IEEE Computer Society

ER -