Nazr-CNN

Fine-Grained classification of UAV imagery for damage assessment

Nazia Attari, Ferda Ofli, Mohammad Awad, Ji Lucas, Sanjay Chawla

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage. To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines.

Original languageEnglish
Title of host publicationProceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages50-59
Number of pages10
Volume2018-January
ISBN (Electronic)9781509050048
DOIs
Publication statusPublished - 16 Jan 2018
Event4th International Conference on Data Science and Advanced Analytics, DSAA 2017 - Tokyo, Japan
Duration: 19 Oct 201721 Oct 2017

Other

Other4th International Conference on Data Science and Advanced Analytics, DSAA 2017
CountryJapan
CityTokyo
Period19/10/1721/10/17

Fingerprint

Damage Assessment
Unmanned aerial vehicles (UAV)
Damage
Neural Networks
Neural networks
Object Detection
Labeling
Pipelines
Pixel
Pixels
Typhoon
Transfer Learning
Aerial Image
Disaster
Hits
Disasters
Encoding
Infrastructure
Monitoring
Antennas

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems and Management
  • Statistics, Probability and Uncertainty
  • Computer Networks and Communications

Cite this

Attari, N., Ofli, F., Awad, M., Lucas, J., & Chawla, S. (2018). Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment. In Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017 (Vol. 2018-January, pp. 50-59). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DSAA.2017.72

Nazr-CNN : Fine-Grained classification of UAV imagery for damage assessment. / Attari, Nazia; Ofli, Ferda; Awad, Mohammad; Lucas, Ji; Chawla, Sanjay.

Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 50-59.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Attari, N, Ofli, F, Awad, M, Lucas, J & Chawla, S 2018, Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment. in Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 50-59, 4th International Conference on Data Science and Advanced Analytics, DSAA 2017, Tokyo, Japan, 19/10/17. https://doi.org/10.1109/DSAA.2017.72
Attari N, Ofli F, Awad M, Lucas J, Chawla S. Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment. In Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 50-59 https://doi.org/10.1109/DSAA.2017.72
Attari, Nazia ; Ofli, Ferda ; Awad, Mohammad ; Lucas, Ji ; Chawla, Sanjay. / Nazr-CNN : Fine-Grained classification of UAV imagery for damage assessment. Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 50-59
@inproceedings{d0e4ccf685e74bf29dc70db3bae44655,
title = "Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment",
abstract = "We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage. To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines.",
author = "Nazia Attari and Ferda Ofli and Mohammad Awad and Ji Lucas and Sanjay Chawla",
year = "2018",
month = "1",
day = "16",
doi = "10.1109/DSAA.2017.72",
language = "English",
volume = "2018-January",
pages = "50--59",
booktitle = "Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Nazr-CNN

T2 - Fine-Grained classification of UAV imagery for damage assessment

AU - Attari, Nazia

AU - Ofli, Ferda

AU - Awad, Mohammad

AU - Lucas, Ji

AU - Chawla, Sanjay

PY - 2018/1/16

Y1 - 2018/1/16

N2 - We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage. To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines.

AB - We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage. To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines.

UR - http://www.scopus.com/inward/record.url?scp=85046265272&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046265272&partnerID=8YFLogxK

U2 - 10.1109/DSAA.2017.72

DO - 10.1109/DSAA.2017.72

M3 - Conference contribution

VL - 2018-January

SP - 50

EP - 59

BT - Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -