SHAHED

A MapReduce-based system for querying and visualizing spatio-temporal satellite data

Ahmed Eldawy, Mohamed Mokbel, Saif Alharthi, Abdulhadi Alzaidy, Kareem Tarek, Sohaib Ghani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

37 Citations (Scopus)

Abstract

Remote sensing data collected by satellites are now made publicly available by several space agencies. This data is very useful for scientists pursuing research in several applications including climate change, desertification, and land use change. The benefit of this data comes from its richness as it provides an archived history for over 15 years of satellite observations for natural phenomena such as temperature and vegetation. Unfortunately, the use of such data is very limited due to the huge size of archives (> 500TB) and the limited capabilities of traditional applications. This paper introduces SHAHED; a MapReduce-based system for querying, visualizing, and mining large scale satellite data. SHAHED considers both the spatial and temporal aspects of the data to provide efficient query processing at large scale. The core of SHAHED is composed of four main components. The uncertainty component recovers missing data in the input which comes from cloud coverage and satellite mis-alignment. The indexing component provides a novel multi-resolution quad-tree-based spatio-temporal index structure, which indexes satellite data efficiently with minimal space overhead. The querying component answers selection and aggregate queries in real-time using the constructed index. Finally, the visualization component uses MapReduce programs to generate heat map images and videos for user queries. A set of experiments running on a live system deployed on a cluster of machines show the efficiency of the proposed design. All the features supported by SHAHED are made accessible through an easy to use web interface that hides the complexity of the system and provides a nice user experience.

Original languageEnglish
Title of host publication2015 IEEE 31st International Conference on Data Engineering, ICDE 2015
PublisherIEEE Computer Society
Pages1585-1596
Number of pages12
Volume2015-May
ISBN (Electronic)9781479979639
DOIs
Publication statusPublished - 1 Jan 2015
Externally publishedYes
Event2015 31st IEEE International Conference on Data Engineering, ICDE 2015 - Seoul, Korea, Republic of
Duration: 13 Apr 201517 Apr 2015

Other

Other2015 31st IEEE International Conference on Data Engineering, ICDE 2015
CountryKorea, Republic of
CitySeoul
Period13/4/1517/4/15

Fingerprint

Satellites
Query processing
Land use
Climate change
Remote sensing
Visualization
Experiments
Temperature

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Cite this

Eldawy, A., Mokbel, M., Alharthi, S., Alzaidy, A., Tarek, K., & Ghani, S. (2015). SHAHED: A MapReduce-based system for querying and visualizing spatio-temporal satellite data. In 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015 (Vol. 2015-May, pp. 1585-1596). [7113427] IEEE Computer Society. https://doi.org/10.1109/ICDE.2015.7113427

SHAHED : A MapReduce-based system for querying and visualizing spatio-temporal satellite data. / Eldawy, Ahmed; Mokbel, Mohamed; Alharthi, Saif; Alzaidy, Abdulhadi; Tarek, Kareem; Ghani, Sohaib.

2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. Vol. 2015-May IEEE Computer Society, 2015. p. 1585-1596 7113427.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Eldawy, A, Mokbel, M, Alharthi, S, Alzaidy, A, Tarek, K & Ghani, S 2015, SHAHED: A MapReduce-based system for querying and visualizing spatio-temporal satellite data. in 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. vol. 2015-May, 7113427, IEEE Computer Society, pp. 1585-1596, 2015 31st IEEE International Conference on Data Engineering, ICDE 2015, Seoul, Korea, Republic of, 13/4/15. https://doi.org/10.1109/ICDE.2015.7113427
Eldawy A, Mokbel M, Alharthi S, Alzaidy A, Tarek K, Ghani S. SHAHED: A MapReduce-based system for querying and visualizing spatio-temporal satellite data. In 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. Vol. 2015-May. IEEE Computer Society. 2015. p. 1585-1596. 7113427 https://doi.org/10.1109/ICDE.2015.7113427
Eldawy, Ahmed ; Mokbel, Mohamed ; Alharthi, Saif ; Alzaidy, Abdulhadi ; Tarek, Kareem ; Ghani, Sohaib. / SHAHED : A MapReduce-based system for querying and visualizing spatio-temporal satellite data. 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015. Vol. 2015-May IEEE Computer Society, 2015. pp. 1585-1596
@inproceedings{5357a508ca3740ed921f585f017d1f5f,
title = "SHAHED: A MapReduce-based system for querying and visualizing spatio-temporal satellite data",
abstract = "Remote sensing data collected by satellites are now made publicly available by several space agencies. This data is very useful for scientists pursuing research in several applications including climate change, desertification, and land use change. The benefit of this data comes from its richness as it provides an archived history for over 15 years of satellite observations for natural phenomena such as temperature and vegetation. Unfortunately, the use of such data is very limited due to the huge size of archives (> 500TB) and the limited capabilities of traditional applications. This paper introduces SHAHED; a MapReduce-based system for querying, visualizing, and mining large scale satellite data. SHAHED considers both the spatial and temporal aspects of the data to provide efficient query processing at large scale. The core of SHAHED is composed of four main components. The uncertainty component recovers missing data in the input which comes from cloud coverage and satellite mis-alignment. The indexing component provides a novel multi-resolution quad-tree-based spatio-temporal index structure, which indexes satellite data efficiently with minimal space overhead. The querying component answers selection and aggregate queries in real-time using the constructed index. Finally, the visualization component uses MapReduce programs to generate heat map images and videos for user queries. A set of experiments running on a live system deployed on a cluster of machines show the efficiency of the proposed design. All the features supported by SHAHED are made accessible through an easy to use web interface that hides the complexity of the system and provides a nice user experience.",
author = "Ahmed Eldawy and Mohamed Mokbel and Saif Alharthi and Abdulhadi Alzaidy and Kareem Tarek and Sohaib Ghani",
year = "2015",
month = "1",
day = "1",
doi = "10.1109/ICDE.2015.7113427",
language = "English",
volume = "2015-May",
pages = "1585--1596",
booktitle = "2015 IEEE 31st International Conference on Data Engineering, ICDE 2015",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - SHAHED

T2 - A MapReduce-based system for querying and visualizing spatio-temporal satellite data

AU - Eldawy, Ahmed

AU - Mokbel, Mohamed

AU - Alharthi, Saif

AU - Alzaidy, Abdulhadi

AU - Tarek, Kareem

AU - Ghani, Sohaib

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Remote sensing data collected by satellites are now made publicly available by several space agencies. This data is very useful for scientists pursuing research in several applications including climate change, desertification, and land use change. The benefit of this data comes from its richness as it provides an archived history for over 15 years of satellite observations for natural phenomena such as temperature and vegetation. Unfortunately, the use of such data is very limited due to the huge size of archives (> 500TB) and the limited capabilities of traditional applications. This paper introduces SHAHED; a MapReduce-based system for querying, visualizing, and mining large scale satellite data. SHAHED considers both the spatial and temporal aspects of the data to provide efficient query processing at large scale. The core of SHAHED is composed of four main components. The uncertainty component recovers missing data in the input which comes from cloud coverage and satellite mis-alignment. The indexing component provides a novel multi-resolution quad-tree-based spatio-temporal index structure, which indexes satellite data efficiently with minimal space overhead. The querying component answers selection and aggregate queries in real-time using the constructed index. Finally, the visualization component uses MapReduce programs to generate heat map images and videos for user queries. A set of experiments running on a live system deployed on a cluster of machines show the efficiency of the proposed design. All the features supported by SHAHED are made accessible through an easy to use web interface that hides the complexity of the system and provides a nice user experience.

AB - Remote sensing data collected by satellites are now made publicly available by several space agencies. This data is very useful for scientists pursuing research in several applications including climate change, desertification, and land use change. The benefit of this data comes from its richness as it provides an archived history for over 15 years of satellite observations for natural phenomena such as temperature and vegetation. Unfortunately, the use of such data is very limited due to the huge size of archives (> 500TB) and the limited capabilities of traditional applications. This paper introduces SHAHED; a MapReduce-based system for querying, visualizing, and mining large scale satellite data. SHAHED considers both the spatial and temporal aspects of the data to provide efficient query processing at large scale. The core of SHAHED is composed of four main components. The uncertainty component recovers missing data in the input which comes from cloud coverage and satellite mis-alignment. The indexing component provides a novel multi-resolution quad-tree-based spatio-temporal index structure, which indexes satellite data efficiently with minimal space overhead. The querying component answers selection and aggregate queries in real-time using the constructed index. Finally, the visualization component uses MapReduce programs to generate heat map images and videos for user queries. A set of experiments running on a live system deployed on a cluster of machines show the efficiency of the proposed design. All the features supported by SHAHED are made accessible through an easy to use web interface that hides the complexity of the system and provides a nice user experience.

UR - http://www.scopus.com/inward/record.url?scp=84940826528&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940826528&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2015.7113427

DO - 10.1109/ICDE.2015.7113427

M3 - Conference contribution

VL - 2015-May

SP - 1585

EP - 1596

BT - 2015 IEEE 31st International Conference on Data Engineering, ICDE 2015

PB - IEEE Computer Society

ER -