ST-Hadoop

A mapreduce framework for spatio-temporal data

Louai Alarabi, Mohamed Mokbel, Mashaal Musleh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

Original languageEnglish
Title of host publicationAdvances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings
PublisherSpringer Verlag
Pages84-104
Number of pages21
ISBN (Print)9783319643663
DOIs
Publication statusPublished - 1 Jan 2017
Externally publishedYes
Event15th International Symposium on Spatial and Temporal Databases, SSTD 2017 - Arlington, United States
Duration: 21 Aug 201723 Aug 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10411 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other15th International Symposium on Spatial and Temporal Databases, SSTD 2017
CountryUnited States
CityArlington
Period21/8/1723/8/17

Fingerprint

Spatio-temporal Data
MapReduce
Indexing
Distributed File System
Experiments
Query
Open Source
Expand
Join
Divides
Framework
Vertex of a graph
Range of data
Experiment

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Alarabi, L., Mokbel, M., & Musleh, M. (2017). ST-Hadoop: A mapreduce framework for spatio-temporal data. In Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings (pp. 84-104). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10411 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-64367-0_5

ST-Hadoop : A mapreduce framework for spatio-temporal data. / Alarabi, Louai; Mokbel, Mohamed; Musleh, Mashaal.

Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings. Springer Verlag, 2017. p. 84-104 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10411 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Alarabi, L, Mokbel, M & Musleh, M 2017, ST-Hadoop: A mapreduce framework for spatio-temporal data. in Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10411 LNCS, Springer Verlag, pp. 84-104, 15th International Symposium on Spatial and Temporal Databases, SSTD 2017, Arlington, United States, 21/8/17. https://doi.org/10.1007/978-3-319-64367-0_5
Alarabi L, Mokbel M, Musleh M. ST-Hadoop: A mapreduce framework for spatio-temporal data. In Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings. Springer Verlag. 2017. p. 84-104. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-64367-0_5
Alarabi, Louai ; Mokbel, Mohamed ; Musleh, Mashaal. / ST-Hadoop : A mapreduce framework for spatio-temporal data. Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings. Springer Verlag, 2017. pp. 84-104 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{5c27545b683e4ed483fce90f6d460099,
title = "ST-Hadoop: A mapreduce framework for spatio-temporal data",
abstract = "This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10{\^A} TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.",
author = "Louai Alarabi and Mohamed Mokbel and Mashaal Musleh",
year = "2017",
month = "1",
day = "1",
doi = "10.1007/978-3-319-64367-0_5",
language = "English",
isbn = "9783319643663",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "84--104",
booktitle = "Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings",

}

TY - GEN

T1 - ST-Hadoop

T2 - A mapreduce framework for spatio-temporal data

AU - Alarabi, Louai

AU - Mokbel, Mohamed

AU - Musleh, Mashaal

PY - 2017/1/1

Y1 - 2017/1/1

N2 - This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

AB - This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10 TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.

UR - http://www.scopus.com/inward/record.url?scp=85028452248&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028452248&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-64367-0_5

DO - 10.1007/978-3-319-64367-0_5

M3 - Conference contribution

SN - 9783319643663

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 84

EP - 104

BT - Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings

PB - Springer Verlag

ER -