ERA: Efficient serial and parallel suffix tree construction for very long strings

Essam Mansour, Amin Allam, Spiros Skiadopoulos, Panos Kalnis

Research output: Chapter in Book/Report/Conference proceedingChapter

24 Citations (Scopus)

Abstract

The suffix tree is a data structure for indexing strings. It is used in a variety of applications such as bioinformatics, time series analysis, clustering, text editing and data com-pression. However, when the string and the resulting suffix tree are too large to fit into the main memory, most existing construction algorithms become very inefficient. This paper presents a disk-based suffix tree construction method, called Elastic Range (ERa), which works efficiently with very long strings that are much larger than the available memory. ERa partitions the tree construction process hor-izontally and vertically and minimizes I/Os by dynamically adjusting the horizontal partitions independently for each vertical partition, based on the evolving shape of the tree and the available memory. Where appropriate, ERa also groups vertical partitions together to amortize the I/O cost. We developed a serial version; a parallel version for shared-memory and shared-disk multi-core systems; and a parallel version for shared-nothing architectures. ERa indexes the entire human genome in 19 minutes on an ordinary desk-top computer. For comparison, the fastest existing method needs 15 minutes using 1024 CPUs on an IBM BlueGene supercomputer.

Original languageEnglish
Title of host publicationProceedings of the VLDB Endowment
Pages49-60
Number of pages12
Volume5
Edition1
Publication statusPublished - Sep 2011
Externally publishedYes

Fingerprint

Data storage equipment
Partitions (building)
Time series analysis
Supercomputers
Bioinformatics
Program processors
Data structures
Genes
Costs

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Mansour, E., Allam, A., Skiadopoulos, S., & Kalnis, P. (2011). ERA: Efficient serial and parallel suffix tree construction for very long strings. In Proceedings of the VLDB Endowment (1 ed., Vol. 5, pp. 49-60)

ERA : Efficient serial and parallel suffix tree construction for very long strings. / Mansour, Essam; Allam, Amin; Skiadopoulos, Spiros; Kalnis, Panos.

Proceedings of the VLDB Endowment. Vol. 5 1. ed. 2011. p. 49-60.

Research output: Chapter in Book/Report/Conference proceedingChapter

Mansour, E, Allam, A, Skiadopoulos, S & Kalnis, P 2011, ERA: Efficient serial and parallel suffix tree construction for very long strings. in Proceedings of the VLDB Endowment. 1 edn, vol. 5, pp. 49-60.
Mansour E, Allam A, Skiadopoulos S, Kalnis P. ERA: Efficient serial and parallel suffix tree construction for very long strings. In Proceedings of the VLDB Endowment. 1 ed. Vol. 5. 2011. p. 49-60
Mansour, Essam ; Allam, Amin ; Skiadopoulos, Spiros ; Kalnis, Panos. / ERA : Efficient serial and parallel suffix tree construction for very long strings. Proceedings of the VLDB Endowment. Vol. 5 1. ed. 2011. pp. 49-60
@inbook{5c0999a001354eba83bac21087f52e8a,
title = "ERA: Efficient serial and parallel suffix tree construction for very long strings",
abstract = "The suffix tree is a data structure for indexing strings. It is used in a variety of applications such as bioinformatics, time series analysis, clustering, text editing and data com-pression. However, when the string and the resulting suffix tree are too large to fit into the main memory, most existing construction algorithms become very inefficient. This paper presents a disk-based suffix tree construction method, called Elastic Range (ERa), which works efficiently with very long strings that are much larger than the available memory. ERa partitions the tree construction process hor-izontally and vertically and minimizes I/Os by dynamically adjusting the horizontal partitions independently for each vertical partition, based on the evolving shape of the tree and the available memory. Where appropriate, ERa also groups vertical partitions together to amortize the I/O cost. We developed a serial version; a parallel version for shared-memory and shared-disk multi-core systems; and a parallel version for shared-nothing architectures. ERa indexes the entire human genome in 19 minutes on an ordinary desk-top computer. For comparison, the fastest existing method needs 15 minutes using 1024 CPUs on an IBM BlueGene supercomputer.",
author = "Essam Mansour and Amin Allam and Spiros Skiadopoulos and Panos Kalnis",
year = "2011",
month = "9",
language = "English",
volume = "5",
pages = "49--60",
booktitle = "Proceedings of the VLDB Endowment",
edition = "1",

}

TY - CHAP

T1 - ERA

T2 - Efficient serial and parallel suffix tree construction for very long strings

AU - Mansour, Essam

AU - Allam, Amin

AU - Skiadopoulos, Spiros

AU - Kalnis, Panos

PY - 2011/9

Y1 - 2011/9

N2 - The suffix tree is a data structure for indexing strings. It is used in a variety of applications such as bioinformatics, time series analysis, clustering, text editing and data com-pression. However, when the string and the resulting suffix tree are too large to fit into the main memory, most existing construction algorithms become very inefficient. This paper presents a disk-based suffix tree construction method, called Elastic Range (ERa), which works efficiently with very long strings that are much larger than the available memory. ERa partitions the tree construction process hor-izontally and vertically and minimizes I/Os by dynamically adjusting the horizontal partitions independently for each vertical partition, based on the evolving shape of the tree and the available memory. Where appropriate, ERa also groups vertical partitions together to amortize the I/O cost. We developed a serial version; a parallel version for shared-memory and shared-disk multi-core systems; and a parallel version for shared-nothing architectures. ERa indexes the entire human genome in 19 minutes on an ordinary desk-top computer. For comparison, the fastest existing method needs 15 minutes using 1024 CPUs on an IBM BlueGene supercomputer.

AB - The suffix tree is a data structure for indexing strings. It is used in a variety of applications such as bioinformatics, time series analysis, clustering, text editing and data com-pression. However, when the string and the resulting suffix tree are too large to fit into the main memory, most existing construction algorithms become very inefficient. This paper presents a disk-based suffix tree construction method, called Elastic Range (ERa), which works efficiently with very long strings that are much larger than the available memory. ERa partitions the tree construction process hor-izontally and vertically and minimizes I/Os by dynamically adjusting the horizontal partitions independently for each vertical partition, based on the evolving shape of the tree and the available memory. Where appropriate, ERa also groups vertical partitions together to amortize the I/O cost. We developed a serial version; a parallel version for shared-memory and shared-disk multi-core systems; and a parallel version for shared-nothing architectures. ERa indexes the entire human genome in 19 minutes on an ordinary desk-top computer. For comparison, the fastest existing method needs 15 minutes using 1024 CPUs on an IBM BlueGene supercomputer.

UR - http://www.scopus.com/inward/record.url?scp=84863749939&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863749939&partnerID=8YFLogxK

M3 - Chapter

AN - SCOPUS:84863749939

VL - 5

SP - 49

EP - 60

BT - Proceedings of the VLDB Endowment

ER -