Pigeon: A spatial MapReduce language

Ahmed Eldawy, Mohamed Mokbel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

39 Citations (Scopus)

Abstract

With the huge amounts of spatial data collected everyday, MapReduce frameworks, such as Hadoop, have become a common choice to analyze big spatial data for scientists and people from industry. Users prefer to use high level languages, such as Pig Latin, to deal with Hadoop for simplicity. Unfortunately, these languages are designed for primitive non-spatial data and have no support for spatial data types or functions. This demonstration presents Pigeon, a spatial extension to Pig which provides spatial functionality in Pig. Pigeon is implemented through user defined functions (UDFs) making it easy to use and compatible with all recent versions of Pig. This also allows it to integrate smoothly with existing non-spatial functions and operations such as Filter, Join and Group By. Pigeon is compatible with the Open Geospatial Consortium (OGC) standard which makes it easy to learn and use for users who are familiar with existing OGC-compliant tools such as PostGIS. This demonstrations shows to audience how to work with Pigeon through some interesting applications running on large scale real datasets extracted from OpenStreetMap.

Original languageEnglish
Title of host publication2014 IEEE 30th International Conference on Data Engineering, ICDE 2014
PublisherIEEE Computer Society
Pages1242-1245
Number of pages4
ISBN (Print)9781479925544
DOIs
Publication statusPublished - 1 Jan 2014
Externally publishedYes
Event30th IEEE International Conference on Data Engineering, ICDE 2014 - Chicago, IL, United States
Duration: 31 Mar 20144 Apr 2014

Other

Other30th IEEE International Conference on Data Engineering, ICDE 2014
CountryUnited States
CityChicago, IL
Period31/3/144/4/14

Fingerprint

Demonstrations
High level languages
Industry

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Cite this

Eldawy, A., & Mokbel, M. (2014). Pigeon: A spatial MapReduce language. In 2014 IEEE 30th International Conference on Data Engineering, ICDE 2014 (pp. 1242-1245). [6816751] IEEE Computer Society. https://doi.org/10.1109/ICDE.2014.6816751

Pigeon : A spatial MapReduce language. / Eldawy, Ahmed; Mokbel, Mohamed.

2014 IEEE 30th International Conference on Data Engineering, ICDE 2014. IEEE Computer Society, 2014. p. 1242-1245 6816751.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Eldawy, A & Mokbel, M 2014, Pigeon: A spatial MapReduce language. in 2014 IEEE 30th International Conference on Data Engineering, ICDE 2014., 6816751, IEEE Computer Society, pp. 1242-1245, 30th IEEE International Conference on Data Engineering, ICDE 2014, Chicago, IL, United States, 31/3/14. https://doi.org/10.1109/ICDE.2014.6816751
Eldawy A, Mokbel M. Pigeon: A spatial MapReduce language. In 2014 IEEE 30th International Conference on Data Engineering, ICDE 2014. IEEE Computer Society. 2014. p. 1242-1245. 6816751 https://doi.org/10.1109/ICDE.2014.6816751
Eldawy, Ahmed ; Mokbel, Mohamed. / Pigeon : A spatial MapReduce language. 2014 IEEE 30th International Conference on Data Engineering, ICDE 2014. IEEE Computer Society, 2014. pp. 1242-1245
@inproceedings{4a432ca0693a4360a77213b72de46ae9,
title = "Pigeon: A spatial MapReduce language",
abstract = "With the huge amounts of spatial data collected everyday, MapReduce frameworks, such as Hadoop, have become a common choice to analyze big spatial data for scientists and people from industry. Users prefer to use high level languages, such as Pig Latin, to deal with Hadoop for simplicity. Unfortunately, these languages are designed for primitive non-spatial data and have no support for spatial data types or functions. This demonstration presents Pigeon, a spatial extension to Pig which provides spatial functionality in Pig. Pigeon is implemented through user defined functions (UDFs) making it easy to use and compatible with all recent versions of Pig. This also allows it to integrate smoothly with existing non-spatial functions and operations such as Filter, Join and Group By. Pigeon is compatible with the Open Geospatial Consortium (OGC) standard which makes it easy to learn and use for users who are familiar with existing OGC-compliant tools such as PostGIS. This demonstrations shows to audience how to work with Pigeon through some interesting applications running on large scale real datasets extracted from OpenStreetMap.",
author = "Ahmed Eldawy and Mohamed Mokbel",
year = "2014",
month = "1",
day = "1",
doi = "10.1109/ICDE.2014.6816751",
language = "English",
isbn = "9781479925544",
pages = "1242--1245",
booktitle = "2014 IEEE 30th International Conference on Data Engineering, ICDE 2014",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Pigeon

T2 - A spatial MapReduce language

AU - Eldawy, Ahmed

AU - Mokbel, Mohamed

PY - 2014/1/1

Y1 - 2014/1/1

N2 - With the huge amounts of spatial data collected everyday, MapReduce frameworks, such as Hadoop, have become a common choice to analyze big spatial data for scientists and people from industry. Users prefer to use high level languages, such as Pig Latin, to deal with Hadoop for simplicity. Unfortunately, these languages are designed for primitive non-spatial data and have no support for spatial data types or functions. This demonstration presents Pigeon, a spatial extension to Pig which provides spatial functionality in Pig. Pigeon is implemented through user defined functions (UDFs) making it easy to use and compatible with all recent versions of Pig. This also allows it to integrate smoothly with existing non-spatial functions and operations such as Filter, Join and Group By. Pigeon is compatible with the Open Geospatial Consortium (OGC) standard which makes it easy to learn and use for users who are familiar with existing OGC-compliant tools such as PostGIS. This demonstrations shows to audience how to work with Pigeon through some interesting applications running on large scale real datasets extracted from OpenStreetMap.

AB - With the huge amounts of spatial data collected everyday, MapReduce frameworks, such as Hadoop, have become a common choice to analyze big spatial data for scientists and people from industry. Users prefer to use high level languages, such as Pig Latin, to deal with Hadoop for simplicity. Unfortunately, these languages are designed for primitive non-spatial data and have no support for spatial data types or functions. This demonstration presents Pigeon, a spatial extension to Pig which provides spatial functionality in Pig. Pigeon is implemented through user defined functions (UDFs) making it easy to use and compatible with all recent versions of Pig. This also allows it to integrate smoothly with existing non-spatial functions and operations such as Filter, Join and Group By. Pigeon is compatible with the Open Geospatial Consortium (OGC) standard which makes it easy to learn and use for users who are familiar with existing OGC-compliant tools such as PostGIS. This demonstrations shows to audience how to work with Pigeon through some interesting applications running on large scale real datasets extracted from OpenStreetMap.

UR - http://www.scopus.com/inward/record.url?scp=84901786420&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84901786420&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2014.6816751

DO - 10.1109/ICDE.2014.6816751

M3 - Conference contribution

AN - SCOPUS:84901786420

SN - 9781479925544

SP - 1242

EP - 1245

BT - 2014 IEEE 30th International Conference on Data Engineering, ICDE 2014

PB - IEEE Computer Society

ER -