ACME: A scalable parallel system for extracting frequent patterns from a very long sequence

Majed Sahli, Essam Mansour, Panos Kalnis

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Modern applications, including bioinformatics, time series, and web log analysis, require the extraction of frequent patterns, called motifs, from one very long (i.e., several gigabytes) sequence. Existing approaches are either heuristics that are error-prone, or exact (also called combinatorial) methods that are extremely slow, therefore, applicable only to very small sequences (i.e., in the order of megabytes). This paper presents ACME, a combinatorial approach that scales to gigabyte-long sequences and is the first to support supermaximal motifs. ACME is a versatile parallel system that can be deployed on desktop multi-core systems, or on thousands of CPUs in the cloud. However, merely using more compute nodes does not guarantee efficiency, because of the related overheads. To this end, ACME introduces an automatic tuning mechanism that suggests the appropriate number of CPUs to utilize, in order to meet the user constraints in terms of run time, while minimizing the financial cost of cloud resources. Our experiments show that, compared to the state of the art, ACME supports three orders of magnitude longer sequences (e.g., DNA for the entire human genome); handles large alphabets (e.g., English alphabet for Wikipedia); scales out to 16,384 CPUs on a supercomputer; and supports elastic deployment in the cloud.

Original languageEnglish
Pages (from-to)871-893
Number of pages23
JournalVLDB Journal
Volume23
Issue number6
DOIs
Publication statusPublished - 15 Nov 2014

Keywords

  • Automatic tuning
  • Cache efficient
  • Cloud
  • Elastic
  • Motif
  • Suffix tree

ASJC Scopus subject areas

  • Information Systems
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'ACME: A scalable parallel system for extracting frequent patterns from a very long sequence'. Together they form a unique fingerprint.

  • Cite this