Towards black box testing of android apps

Yury Zhauniarovich, Anton Philippov, Olga Gadyatskaya, Bruno Crispo, Fabio Massacci

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Citations (Scopus)

Abstract

Many state-of-art mobile application testing frameworks (e.g., Dynodroid, Evo Droid) enjoy Emma or other code coverage libraries to measure the coverage achieved. The underlying assumption for these frameworks is availability of the app source code. Yet, application markets and security researchers face the need to test third-party mobile applications in the absence of the source code. There exists a number of frameworks both for manual and automated test generation that address this challenge. However, these frameworks often do not provide any statistics on the code coverage achieved, or provide coarse-grained ones like a number of activities or methods covered. At the same time, given two test reports generated by different frameworks, there is no way to understand which one achieved better coverage if the reported metrics were different (or no coverage results were provided). To address these issues we designed a framework called BBox Tester that is able to generate code coverage reports and produce uniform coverage metrics in testing without the source code. Security researchers can automatically execute applications exploiting current state-of-art tools, and use the results of our framework to assess if the security-critical code was covered by the tests. In this paper we report on design and implementation of BBox Tester and assess its efficiency and effectiveness.

Original languageEnglish
Title of host publicationProceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages501-510
Number of pages10
ISBN (Electronic)9781467365901
DOIs
Publication statusPublished - 16 Oct 2015
Externally publishedYes
Event10th International Conference on Availability, Reliability and Security, ARES 2015 - Toulouse, France
Duration: 24 Aug 201527 Aug 2015

Other

Other10th International Conference on Availability, Reliability and Security, ARES 2015
CountryFrance
CityToulouse
Period24/8/1527/8/15

Fingerprint

Black-box testing
Application programs
Testing
Statistics
Availability
Android (operating system)

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture

Cite this

Zhauniarovich, Y., Philippov, A., Gadyatskaya, O., Crispo, B., & Massacci, F. (2015). Towards black box testing of android apps. In Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015 (pp. 501-510). [7299958] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ARES.2015.70

Towards black box testing of android apps. / Zhauniarovich, Yury; Philippov, Anton; Gadyatskaya, Olga; Crispo, Bruno; Massacci, Fabio.

Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 501-510 7299958.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhauniarovich, Y, Philippov, A, Gadyatskaya, O, Crispo, B & Massacci, F 2015, Towards black box testing of android apps. in Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015., 7299958, Institute of Electrical and Electronics Engineers Inc., pp. 501-510, 10th International Conference on Availability, Reliability and Security, ARES 2015, Toulouse, France, 24/8/15. https://doi.org/10.1109/ARES.2015.70
Zhauniarovich Y, Philippov A, Gadyatskaya O, Crispo B, Massacci F. Towards black box testing of android apps. In Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 501-510. 7299958 https://doi.org/10.1109/ARES.2015.70
Zhauniarovich, Yury ; Philippov, Anton ; Gadyatskaya, Olga ; Crispo, Bruno ; Massacci, Fabio. / Towards black box testing of android apps. Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 501-510
@inproceedings{c66132f6689142b18a7df66b3e1bc1b1,
title = "Towards black box testing of android apps",
abstract = "Many state-of-art mobile application testing frameworks (e.g., Dynodroid, Evo Droid) enjoy Emma or other code coverage libraries to measure the coverage achieved. The underlying assumption for these frameworks is availability of the app source code. Yet, application markets and security researchers face the need to test third-party mobile applications in the absence of the source code. There exists a number of frameworks both for manual and automated test generation that address this challenge. However, these frameworks often do not provide any statistics on the code coverage achieved, or provide coarse-grained ones like a number of activities or methods covered. At the same time, given two test reports generated by different frameworks, there is no way to understand which one achieved better coverage if the reported metrics were different (or no coverage results were provided). To address these issues we designed a framework called BBox Tester that is able to generate code coverage reports and produce uniform coverage metrics in testing without the source code. Security researchers can automatically execute applications exploiting current state-of-art tools, and use the results of our framework to assess if the security-critical code was covered by the tests. In this paper we report on design and implementation of BBox Tester and assess its efficiency and effectiveness.",
author = "Yury Zhauniarovich and Anton Philippov and Olga Gadyatskaya and Bruno Crispo and Fabio Massacci",
year = "2015",
month = "10",
day = "16",
doi = "10.1109/ARES.2015.70",
language = "English",
pages = "501--510",
booktitle = "Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Towards black box testing of android apps

AU - Zhauniarovich, Yury

AU - Philippov, Anton

AU - Gadyatskaya, Olga

AU - Crispo, Bruno

AU - Massacci, Fabio

PY - 2015/10/16

Y1 - 2015/10/16

N2 - Many state-of-art mobile application testing frameworks (e.g., Dynodroid, Evo Droid) enjoy Emma or other code coverage libraries to measure the coverage achieved. The underlying assumption for these frameworks is availability of the app source code. Yet, application markets and security researchers face the need to test third-party mobile applications in the absence of the source code. There exists a number of frameworks both for manual and automated test generation that address this challenge. However, these frameworks often do not provide any statistics on the code coverage achieved, or provide coarse-grained ones like a number of activities or methods covered. At the same time, given two test reports generated by different frameworks, there is no way to understand which one achieved better coverage if the reported metrics were different (or no coverage results were provided). To address these issues we designed a framework called BBox Tester that is able to generate code coverage reports and produce uniform coverage metrics in testing without the source code. Security researchers can automatically execute applications exploiting current state-of-art tools, and use the results of our framework to assess if the security-critical code was covered by the tests. In this paper we report on design and implementation of BBox Tester and assess its efficiency and effectiveness.

AB - Many state-of-art mobile application testing frameworks (e.g., Dynodroid, Evo Droid) enjoy Emma or other code coverage libraries to measure the coverage achieved. The underlying assumption for these frameworks is availability of the app source code. Yet, application markets and security researchers face the need to test third-party mobile applications in the absence of the source code. There exists a number of frameworks both for manual and automated test generation that address this challenge. However, these frameworks often do not provide any statistics on the code coverage achieved, or provide coarse-grained ones like a number of activities or methods covered. At the same time, given two test reports generated by different frameworks, there is no way to understand which one achieved better coverage if the reported metrics were different (or no coverage results were provided). To address these issues we designed a framework called BBox Tester that is able to generate code coverage reports and produce uniform coverage metrics in testing without the source code. Security researchers can automatically execute applications exploiting current state-of-art tools, and use the results of our framework to assess if the security-critical code was covered by the tests. In this paper we report on design and implementation of BBox Tester and assess its efficiency and effectiveness.

UR - http://www.scopus.com/inward/record.url?scp=84961639115&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961639115&partnerID=8YFLogxK

U2 - 10.1109/ARES.2015.70

DO - 10.1109/ARES.2015.70

M3 - Conference contribution

AN - SCOPUS:84961639115

SP - 501

EP - 510

BT - Proceedings - 10th International Conference on Availability, Reliability and Security, ARES 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -