ACME

A scalable parallel system for extracting frequent patterns from a very long sequence

Majed Sahli, Essam Mansour, Panos Kalnis

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Modern applications, including bioinformatics, time series, and web log analysis, require the extraction of frequent patterns, called motifs, from one very long (i.e., several gigabytes) sequence. Existing approaches are either heuristics that are error-prone, or exact (also called combinatorial) methods that are extremely slow, therefore, applicable only to very small sequences (i.e., in the order of megabytes). This paper presents ACME, a combinatorial approach that scales to gigabyte-long sequences and is the first to support supermaximal motifs. ACME is a versatile parallel system that can be deployed on desktop multi-core systems, or on thousands of CPUs in the cloud. However, merely using more compute nodes does not guarantee efficiency, because of the related overheads. To this end, ACME introduces an automatic tuning mechanism that suggests the appropriate number of CPUs to utilize, in order to meet the user constraints in terms of run time, while minimizing the financial cost of cloud resources. Our experiments show that, compared to the state of the art, ACME supports three orders of magnitude longer sequences (e.g., DNA for the entire human genome); handles large alphabets (e.g., English alphabet for Wikipedia); scales out to 16,384 CPUs on a supercomputer; and supports elastic deployment in the cloud.

Original languageEnglish
Pages (from-to)871-893
Number of pages23
JournalVLDB Journal
Volume23
Issue number6
DOIs
Publication statusPublished - 1 Jan 2014

Fingerprint

Program processors
Supercomputers
DNA sequences
Bioinformatics
Time series
Tuning
Genes
Costs
Experiments

Keywords

  • Automatic tuning
  • Cache efficient
  • Cloud
  • Elastic
  • Motif
  • Suffix tree

Cite this

ACME : A scalable parallel system for extracting frequent patterns from a very long sequence. / Sahli, Majed; Mansour, Essam; Kalnis, Panos.

In: VLDB Journal, Vol. 23, No. 6, 01.01.2014, p. 871-893.

Research output: Contribution to journalArticle

@article{f6b2dc4685d54842bc3a26820ac22c25,
title = "ACME: A scalable parallel system for extracting frequent patterns from a very long sequence",
abstract = "Modern applications, including bioinformatics, time series, and web log analysis, require the extraction of frequent patterns, called motifs, from one very long (i.e., several gigabytes) sequence. Existing approaches are either heuristics that are error-prone, or exact (also called combinatorial) methods that are extremely slow, therefore, applicable only to very small sequences (i.e., in the order of megabytes). This paper presents ACME, a combinatorial approach that scales to gigabyte-long sequences and is the first to support supermaximal motifs. ACME is a versatile parallel system that can be deployed on desktop multi-core systems, or on thousands of CPUs in the cloud. However, merely using more compute nodes does not guarantee efficiency, because of the related overheads. To this end, ACME introduces an automatic tuning mechanism that suggests the appropriate number of CPUs to utilize, in order to meet the user constraints in terms of run time, while minimizing the financial cost of cloud resources. Our experiments show that, compared to the state of the art, ACME supports three orders of magnitude longer sequences (e.g., DNA for the entire human genome); handles large alphabets (e.g., English alphabet for Wikipedia); scales out to 16,384 CPUs on a supercomputer; and supports elastic deployment in the cloud.",
keywords = "Automatic tuning, Cache efficient, Cloud, Elastic, Motif, Suffix tree",
author = "Majed Sahli and Essam Mansour and Panos Kalnis",
year = "2014",
month = "1",
day = "1",
doi = "10.1007/s00778-014-0370-1",
language = "English",
volume = "23",
pages = "871--893",
journal = "VLDB Journal",
issn = "1066-8888",
publisher = "Springer New York",
number = "6",

}

TY - JOUR

T1 - ACME

T2 - A scalable parallel system for extracting frequent patterns from a very long sequence

AU - Sahli, Majed

AU - Mansour, Essam

AU - Kalnis, Panos

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Modern applications, including bioinformatics, time series, and web log analysis, require the extraction of frequent patterns, called motifs, from one very long (i.e., several gigabytes) sequence. Existing approaches are either heuristics that are error-prone, or exact (also called combinatorial) methods that are extremely slow, therefore, applicable only to very small sequences (i.e., in the order of megabytes). This paper presents ACME, a combinatorial approach that scales to gigabyte-long sequences and is the first to support supermaximal motifs. ACME is a versatile parallel system that can be deployed on desktop multi-core systems, or on thousands of CPUs in the cloud. However, merely using more compute nodes does not guarantee efficiency, because of the related overheads. To this end, ACME introduces an automatic tuning mechanism that suggests the appropriate number of CPUs to utilize, in order to meet the user constraints in terms of run time, while minimizing the financial cost of cloud resources. Our experiments show that, compared to the state of the art, ACME supports three orders of magnitude longer sequences (e.g., DNA for the entire human genome); handles large alphabets (e.g., English alphabet for Wikipedia); scales out to 16,384 CPUs on a supercomputer; and supports elastic deployment in the cloud.

AB - Modern applications, including bioinformatics, time series, and web log analysis, require the extraction of frequent patterns, called motifs, from one very long (i.e., several gigabytes) sequence. Existing approaches are either heuristics that are error-prone, or exact (also called combinatorial) methods that are extremely slow, therefore, applicable only to very small sequences (i.e., in the order of megabytes). This paper presents ACME, a combinatorial approach that scales to gigabyte-long sequences and is the first to support supermaximal motifs. ACME is a versatile parallel system that can be deployed on desktop multi-core systems, or on thousands of CPUs in the cloud. However, merely using more compute nodes does not guarantee efficiency, because of the related overheads. To this end, ACME introduces an automatic tuning mechanism that suggests the appropriate number of CPUs to utilize, in order to meet the user constraints in terms of run time, while minimizing the financial cost of cloud resources. Our experiments show that, compared to the state of the art, ACME supports three orders of magnitude longer sequences (e.g., DNA for the entire human genome); handles large alphabets (e.g., English alphabet for Wikipedia); scales out to 16,384 CPUs on a supercomputer; and supports elastic deployment in the cloud.

KW - Automatic tuning

KW - Cache efficient

KW - Cloud

KW - Elastic

KW - Motif

KW - Suffix tree

UR - http://www.scopus.com/inward/record.url?scp=84912111055&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84912111055&partnerID=8YFLogxK

U2 - 10.1007/s00778-014-0370-1

DO - 10.1007/s00778-014-0370-1

M3 - Article

VL - 23

SP - 871

EP - 893

JO - VLDB Journal

JF - VLDB Journal

SN - 1066-8888

IS - 6

ER -