Semantics-based distributed I/O for mpiBLAST

P. Balaji, W. Feng, J. Archuleta, H. Lin, R. Kettimuthu, R. Thakur, X. Ma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

BLAST is a widely used software toolkit for genomic sequence search. mpiBLAST is a freely available, open-source parallelization of BLAST that uses database segmentation to allow different worker processes to search (in parallel) unique segments of the database. After searching, the workers write their output to a filesystem. While mpiBLAST has been shown to achieve high performance in clusters with fast local filesystems, its I/O processing remains a concern for scalability, especially in systems having limited I/O capabilities such as distributed filesystems spread across a wide-area network. Thus, we present ParaMEDIC - a novel environment that uses applicationspecific semantic information to compress I/O data and improve performance in distributed environments. Specifically, for mpiBLAST, ParaMEDIC partitions worker processes into compute and I/O workers. Compute workers, instead of directly writing the output to the filesystem, the workers process the output using semantic knowledge about the application to generate metadata and write the metadata to the filesystem. I/O workers, which physically reside closer to the actual storage, then process this metadata to re-create the actual output and write it to the filesystem. This approach allows ParaMEDIC to reduce I/O time, thus accelerating mpiBLAST by as much as 25-fold.

Original languageEnglish
Title of host publicationPPoPP'08 - Proceedings of the 2008 ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
Pages293-294
Number of pages2
Publication statusPublished - 1 Dec 2008
Externally publishedYes
Event13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP'08 - Salt Lake City, UT, United States
Duration: 20 Feb 200823 Feb 2008

Publication series

NameProceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

Other

Other13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP'08
CountryUnited States
CitySalt Lake City, UT
Period20/2/0823/2/08

    Fingerprint

Keywords

  • Distributed filesystem
  • I/O
  • MpiBLAST

ASJC Scopus subject areas

  • Software

Cite this

Balaji, P., Feng, W., Archuleta, J., Lin, H., Kettimuthu, R., Thakur, R., & Ma, X. (2008). Semantics-based distributed I/O for mpiBLAST. In PPoPP'08 - Proceedings of the 2008 ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (pp. 293-294). (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP).