Combining human computing and machine learning to make sense of big (Aerial) data for disaster response

Ferda Ofli, Patrick Meier, Muhammad Imran, Carlos Castillo, Devis Tuia, Nicolas Rey, Julien Briant, Pauline Millet, Friedrich Reinhard, Matthew Parkan, Stéphane Joost

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

Aerial imagery captured via unmanned aerial vehicles (UAVs) is playing an increasingly important role in disaster response. Unlike satellite imagery, aerial imagery can be captured and processed within hours rather than days. In addition, the spatial resolution of aerial imagery is an order of magnitude higher than the imagery produced by the most sophisticated commercial satellites today. Both the United States Federal Emergency Management Agency (FEMA) and the European Commission's Joint Research Center (JRC) have noted that aerial imagery will inevitably present a big data challenge. The purpose of this article is to get ahead of this future challenge by proposing a hybrid crowdsourcing and real-time machine learning solution to rapidly process large volumes of aerial data for disaster response in a time-sensitive manner. Crowdsourcing can be used to annotate features of interest in aerial images (such as damaged shelters and roads blocked by debris). These human-annotated features can then be used to train a supervised machine learning system to learn to recognize such features in new unseen images. In this article, we describe how this hybrid solution for image analysis can be implemented as a module (i.e., Aerial Clicker) to extend an existing platform called Artificial Intelligence for Disaster Response (AIDR), which has already been deployed to classify microblog messages during disasters using its Text Clicker module and in response to Cyclone Pam, a category 5 cyclone that devastated Vanuatu in March 2015. The hybrid solution we present can be applied to both aerial and satellite imagery and has applications beyond disaster response such as wildlife protection, human rights, and archeological exploration. As a proof of concept, we recently piloted this solution using very high-resolution aerial photographs of a wildlife reserve in Namibia to support rangers with their wildlife conservation efforts (SAVMAP project, http://lasig.epfl.ch/savmap). The results suggest that the platform we have developed to combine crowdsourcing and machine learning to make sense of large volumes of aerial images can be used for disaster response.

Original languageEnglish
Pages (from-to)47-59
Number of pages13
JournalBig Data
Volume4
Issue number1
DOIs
Publication statusPublished - 1 Mar 2016

Fingerprint

Disasters
Learning systems
Antennas
Satellite imagery
Machine learning
Imagery
Disaster response
Unmanned aerial vehicles (UAV)
Debris
Image analysis
Artificial intelligence
Conservation
Satellites

Keywords

  • Big Data analytics
  • crowdsourcing
  • machine learning
  • remote sensing
  • UAV

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems
  • Information Systems and Management

Cite this

Combining human computing and machine learning to make sense of big (Aerial) data for disaster response. / Ofli, Ferda; Meier, Patrick; Imran, Muhammad; Castillo, Carlos; Tuia, Devis; Rey, Nicolas; Briant, Julien; Millet, Pauline; Reinhard, Friedrich; Parkan, Matthew; Joost, Stéphane.

In: Big Data, Vol. 4, No. 1, 01.03.2016, p. 47-59.

Research output: Contribution to journalArticle

Ofli, F, Meier, P, Imran, M, Castillo, C, Tuia, D, Rey, N, Briant, J, Millet, P, Reinhard, F, Parkan, M & Joost, S 2016, 'Combining human computing and machine learning to make sense of big (Aerial) data for disaster response', Big Data, vol. 4, no. 1, pp. 47-59. https://doi.org/10.1089/big.2014.0064
Ofli, Ferda ; Meier, Patrick ; Imran, Muhammad ; Castillo, Carlos ; Tuia, Devis ; Rey, Nicolas ; Briant, Julien ; Millet, Pauline ; Reinhard, Friedrich ; Parkan, Matthew ; Joost, Stéphane. / Combining human computing and machine learning to make sense of big (Aerial) data for disaster response. In: Big Data. 2016 ; Vol. 4, No. 1. pp. 47-59.
@article{d06dfd63ae9549fd8ed61d8f09059c61,
title = "Combining human computing and machine learning to make sense of big (Aerial) data for disaster response",
abstract = "Aerial imagery captured via unmanned aerial vehicles (UAVs) is playing an increasingly important role in disaster response. Unlike satellite imagery, aerial imagery can be captured and processed within hours rather than days. In addition, the spatial resolution of aerial imagery is an order of magnitude higher than the imagery produced by the most sophisticated commercial satellites today. Both the United States Federal Emergency Management Agency (FEMA) and the European Commission's Joint Research Center (JRC) have noted that aerial imagery will inevitably present a big data challenge. The purpose of this article is to get ahead of this future challenge by proposing a hybrid crowdsourcing and real-time machine learning solution to rapidly process large volumes of aerial data for disaster response in a time-sensitive manner. Crowdsourcing can be used to annotate features of interest in aerial images (such as damaged shelters and roads blocked by debris). These human-annotated features can then be used to train a supervised machine learning system to learn to recognize such features in new unseen images. In this article, we describe how this hybrid solution for image analysis can be implemented as a module (i.e., Aerial Clicker) to extend an existing platform called Artificial Intelligence for Disaster Response (AIDR), which has already been deployed to classify microblog messages during disasters using its Text Clicker module and in response to Cyclone Pam, a category 5 cyclone that devastated Vanuatu in March 2015. The hybrid solution we present can be applied to both aerial and satellite imagery and has applications beyond disaster response such as wildlife protection, human rights, and archeological exploration. As a proof of concept, we recently piloted this solution using very high-resolution aerial photographs of a wildlife reserve in Namibia to support rangers with their wildlife conservation efforts (SAVMAP project, http://lasig.epfl.ch/savmap). The results suggest that the platform we have developed to combine crowdsourcing and machine learning to make sense of large volumes of aerial images can be used for disaster response.",
keywords = "Big Data analytics, crowdsourcing, machine learning, remote sensing, UAV",
author = "Ferda Ofli and Patrick Meier and Muhammad Imran and Carlos Castillo and Devis Tuia and Nicolas Rey and Julien Briant and Pauline Millet and Friedrich Reinhard and Matthew Parkan and St{\'e}phane Joost",
year = "2016",
month = "3",
day = "1",
doi = "10.1089/big.2014.0064",
language = "English",
volume = "4",
pages = "47--59",
journal = "Big Data",
issn = "2167-6461",
publisher = "Mary Ann Liebert Inc.",
number = "1",

}

TY - JOUR

T1 - Combining human computing and machine learning to make sense of big (Aerial) data for disaster response

AU - Ofli, Ferda

AU - Meier, Patrick

AU - Imran, Muhammad

AU - Castillo, Carlos

AU - Tuia, Devis

AU - Rey, Nicolas

AU - Briant, Julien

AU - Millet, Pauline

AU - Reinhard, Friedrich

AU - Parkan, Matthew

AU - Joost, Stéphane

PY - 2016/3/1

Y1 - 2016/3/1

N2 - Aerial imagery captured via unmanned aerial vehicles (UAVs) is playing an increasingly important role in disaster response. Unlike satellite imagery, aerial imagery can be captured and processed within hours rather than days. In addition, the spatial resolution of aerial imagery is an order of magnitude higher than the imagery produced by the most sophisticated commercial satellites today. Both the United States Federal Emergency Management Agency (FEMA) and the European Commission's Joint Research Center (JRC) have noted that aerial imagery will inevitably present a big data challenge. The purpose of this article is to get ahead of this future challenge by proposing a hybrid crowdsourcing and real-time machine learning solution to rapidly process large volumes of aerial data for disaster response in a time-sensitive manner. Crowdsourcing can be used to annotate features of interest in aerial images (such as damaged shelters and roads blocked by debris). These human-annotated features can then be used to train a supervised machine learning system to learn to recognize such features in new unseen images. In this article, we describe how this hybrid solution for image analysis can be implemented as a module (i.e., Aerial Clicker) to extend an existing platform called Artificial Intelligence for Disaster Response (AIDR), which has already been deployed to classify microblog messages during disasters using its Text Clicker module and in response to Cyclone Pam, a category 5 cyclone that devastated Vanuatu in March 2015. The hybrid solution we present can be applied to both aerial and satellite imagery and has applications beyond disaster response such as wildlife protection, human rights, and archeological exploration. As a proof of concept, we recently piloted this solution using very high-resolution aerial photographs of a wildlife reserve in Namibia to support rangers with their wildlife conservation efforts (SAVMAP project, http://lasig.epfl.ch/savmap). The results suggest that the platform we have developed to combine crowdsourcing and machine learning to make sense of large volumes of aerial images can be used for disaster response.

AB - Aerial imagery captured via unmanned aerial vehicles (UAVs) is playing an increasingly important role in disaster response. Unlike satellite imagery, aerial imagery can be captured and processed within hours rather than days. In addition, the spatial resolution of aerial imagery is an order of magnitude higher than the imagery produced by the most sophisticated commercial satellites today. Both the United States Federal Emergency Management Agency (FEMA) and the European Commission's Joint Research Center (JRC) have noted that aerial imagery will inevitably present a big data challenge. The purpose of this article is to get ahead of this future challenge by proposing a hybrid crowdsourcing and real-time machine learning solution to rapidly process large volumes of aerial data for disaster response in a time-sensitive manner. Crowdsourcing can be used to annotate features of interest in aerial images (such as damaged shelters and roads blocked by debris). These human-annotated features can then be used to train a supervised machine learning system to learn to recognize such features in new unseen images. In this article, we describe how this hybrid solution for image analysis can be implemented as a module (i.e., Aerial Clicker) to extend an existing platform called Artificial Intelligence for Disaster Response (AIDR), which has already been deployed to classify microblog messages during disasters using its Text Clicker module and in response to Cyclone Pam, a category 5 cyclone that devastated Vanuatu in March 2015. The hybrid solution we present can be applied to both aerial and satellite imagery and has applications beyond disaster response such as wildlife protection, human rights, and archeological exploration. As a proof of concept, we recently piloted this solution using very high-resolution aerial photographs of a wildlife reserve in Namibia to support rangers with their wildlife conservation efforts (SAVMAP project, http://lasig.epfl.ch/savmap). The results suggest that the platform we have developed to combine crowdsourcing and machine learning to make sense of large volumes of aerial images can be used for disaster response.

KW - Big Data analytics

KW - crowdsourcing

KW - machine learning

KW - remote sensing

KW - UAV

UR - http://www.scopus.com/inward/record.url?scp=84991821750&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84991821750&partnerID=8YFLogxK

U2 - 10.1089/big.2014.0064

DO - 10.1089/big.2014.0064

M3 - Article

VL - 4

SP - 47

EP - 59

JO - Big Data

JF - Big Data

SN - 2167-6461

IS - 1

ER -