AV-meter: An evaluation of antivirus scans and labels

Aziz Mohaisen, Omar Alrawi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

36 Citations (Scopus)

Abstract

Antivirus scanners are designed to detect malware and, to a lesser extent, to label detections based on a family association. The labeling provided by AV vendors has many applications such as guiding efforts of disinfection and countermeasures, intelligence gathering, and attack attribution, among others. Furthermore, researchers rely on AV labels to establish a baseline of ground truth to compare their detection and classification algorithms. This is done despite many papers pointing out the subtle problem of relying on AV labels. However, the literature lacks any systematic study on validating the performance of antivirus scanners, and the reliability of those labels or detection. In this paper, we set out to answer several questions concerning the detection rate, correctness of labels, and consistency of detection of AV scanners. Equipped with more than 12,000 malware samples of 11 malware families that are manually inspected and labeled, we pose the following questions. How do antivirus vendors perform relatively on them? How correct are the labels given by those vendors? How consistent are antivirus vendors among each other? We answer those questions unveiling many interesting results, and invite the community to challenge assumptions about relying on antivirus scans and labels as a ground truth for malware analysis and classification. Finally, we stress several research directions that may help addressing the problem.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Pages112-131
Number of pages20
Volume8550 LNCS
ISBN (Print)9783319085081
DOIs
Publication statusPublished - 1 Jan 2014
Event11th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, DIMVA 2014 - Egham, United Kingdom
Duration: 10 Jul 201411 Jul 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8550 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other11th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, DIMVA 2014
CountryUnited Kingdom
CityEgham
Period10/7/1411/7/14

    Fingerprint

Keywords

  • Automatic Analysis
  • Evaluation
  • Labeling
  • Malware

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Mohaisen, A., & Alrawi, O. (2014). AV-meter: An evaluation of antivirus scans and labels. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 8550 LNCS, pp. 112-131). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8550 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-08509-8_7