Towards a methodical evaluation of antivirus scans and labels "if you're not confused, you're not paying attention"

Aziz Mohaisen, Omar Alrawi, Matt Larson, Danny McPherson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

In recent years, researchers have relied heavily on labels provided by antivirus companies in establishing ground truth for applications and algorithms of malware detection, classification, and clustering. Furthermore, companies use those labels for guiding their mitigation and disinfection efforts. However, ironically, there is no prior systematic work that validates the performance of antivirus vendors, the reliability of those labels (or even detections), or how they affect the said applications. Equipped with malware samples of several malware families that are manually inspected and labeled, we pose the following questions: How do different antivirus scans perform relatively? How correct are the labels given by those scans? How consistent are AV scans among each other? Our answers to these questions reveal alarming results about the correctness, completeness, coverage, and consistency of the labels utilized by much existing research. We invite the research community to challenge the assumption of relying on antivirus scans and labels as a ground truth for evaluating malware analysis and classification techniques.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Pages231-241
Number of pages11
Volume8267 LNCS
ISBN (Print)9783319051482
DOIs
Publication statusPublished - 2014
Externally publishedYes
Event14th International Workshop on Information Security Applications, WISA 2013 - Jeju Island, Korea, Republic of
Duration: 19 Aug 201321 Aug 2013

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8267 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other14th International Workshop on Information Security Applications, WISA 2013
CountryKorea, Republic of
CityJeju Island
Period19/8/1321/8/13

    Fingerprint

Keywords

  • Automatic analysis
  • Evaluation
  • Labeling
  • Malware

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Mohaisen, A., Alrawi, O., Larson, M., & McPherson, D. (2014). Towards a methodical evaluation of antivirus scans and labels "if you're not confused, you're not paying attention". In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8267 LNCS, pp. 231-241). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8267 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-05149-9_15