Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment

Nazia Attari, Ferda Ofli, Mohammad Awad, Ji Lucas, Sanjay Chawla

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different levels of damage. To showcase our approach we use data from UAVs that were deployed to assess the level of damage in the aftermath of a devastating cyclone that hit the island of Vanuatu in 2015. The collected images were labeled by a crowdsourcing effort and the labeling categories consisted of fine-grained levels of damage to built structures. Since our data set is relatively small, a pre-trained network for pixel-level classification and FV encoding was used. Nazr-CNN attains promising results both for object detection and damage assessment suggesting that the integrated pipeline is robust in the face of small data sets and labeling errors by annotators. While the focus of Nazr-CNN is on assessment of UAV images in a post-disaster scenario, our solution is general and can be applied in many diverse settings. We show one such case of transfer learning to assess the level of damage in aerial images collected after a typhoon in Philippines.

Original languageEnglish
Title of host publicationProceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages50-59
Number of pages10
ISBN (Electronic)9781509050048
DOIs
Publication statusPublished - 2 Jul 2017
Event4th International Conference on Data Science and Advanced Analytics, DSAA 2017 - Tokyo, Japan
Duration: 19 Oct 201721 Oct 2017

Publication series

NameProceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017
Volume2018-January

Other

Other4th International Conference on Data Science and Advanced Analytics, DSAA 2017
CountryJapan
CityTokyo
Period19/10/1721/10/17

    Fingerprint

ASJC Scopus subject areas

  • Signal Processing
  • Information Systems and Management
  • Statistics, Probability and Uncertainty
  • Computer Networks and Communications

Cite this

Attari, N., Ofli, F., Awad, M., Lucas, J., & Chawla, S. (2017). Nazr-CNN: Fine-Grained classification of UAV imagery for damage assessment. In Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017 (pp. 50-59). (Proceedings - 2017 International Conference on Data Science and Advanced Analytics, DSAA 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DSAA.2017.72