Automatic tuning of bag-of-tasks applications

Majed Sahli, Essam Mansour, Tariq Alturkestani, Panos Kalnis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.

Original languageEnglish
Title of host publicationProceedings - International Conference on Data Engineering
PublisherIEEE Computer Society
Pages843-854
Number of pages12
Volume2015-May
ISBN (Print)9781479979639
DOIs
Publication statusPublished - 26 May 2015
Event2015 31st IEEE International Conference on Data Engineering, ICDE 2015 - Seoul, Korea, Republic of
Duration: 13 Apr 201517 Apr 2015

Other

Other2015 31st IEEE International Conference on Data Engineering, ICDE 2015
CountryKorea, Republic of
CitySeoul
Period13/4/1517/4/15

Fingerprint

Program processors
Tuning
Computational chemistry
Supercomputers
Bioinformatics
Application programming interfaces (API)
Data mining
Simulators
Decomposition
Experiments
Linux

ASJC Scopus subject areas

  • Information Systems
  • Signal Processing
  • Software

Cite this

Sahli, M., Mansour, E., Alturkestani, T., & Kalnis, P. (2015). Automatic tuning of bag-of-tasks applications. In Proceedings - International Conference on Data Engineering (Vol. 2015-May, pp. 843-854). [7113338] IEEE Computer Society. https://doi.org/10.1109/ICDE.2015.7113338

Automatic tuning of bag-of-tasks applications. / Sahli, Majed; Mansour, Essam; Alturkestani, Tariq; Kalnis, Panos.

Proceedings - International Conference on Data Engineering. Vol. 2015-May IEEE Computer Society, 2015. p. 843-854 7113338.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sahli, M, Mansour, E, Alturkestani, T & Kalnis, P 2015, Automatic tuning of bag-of-tasks applications. in Proceedings - International Conference on Data Engineering. vol. 2015-May, 7113338, IEEE Computer Society, pp. 843-854, 2015 31st IEEE International Conference on Data Engineering, ICDE 2015, Seoul, Korea, Republic of, 13/4/15. https://doi.org/10.1109/ICDE.2015.7113338
Sahli M, Mansour E, Alturkestani T, Kalnis P. Automatic tuning of bag-of-tasks applications. In Proceedings - International Conference on Data Engineering. Vol. 2015-May. IEEE Computer Society. 2015. p. 843-854. 7113338 https://doi.org/10.1109/ICDE.2015.7113338
Sahli, Majed ; Mansour, Essam ; Alturkestani, Tariq ; Kalnis, Panos. / Automatic tuning of bag-of-tasks applications. Proceedings - International Conference on Data Engineering. Vol. 2015-May IEEE Computer Society, 2015. pp. 843-854
@inproceedings{3fa28bb7b9a64ae5bc62e7511e66df1f,
title = "Automatic tuning of bag-of-tasks applications",
abstract = "This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.",
author = "Majed Sahli and Essam Mansour and Tariq Alturkestani and Panos Kalnis",
year = "2015",
month = "5",
day = "26",
doi = "10.1109/ICDE.2015.7113338",
language = "English",
isbn = "9781479979639",
volume = "2015-May",
pages = "843--854",
booktitle = "Proceedings - International Conference on Data Engineering",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Automatic tuning of bag-of-tasks applications

AU - Sahli, Majed

AU - Mansour, Essam

AU - Alturkestani, Tariq

AU - Kalnis, Panos

PY - 2015/5/26

Y1 - 2015/5/26

N2 - This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.

AB - This paper presents APlug, a framework for automatic tuning of large scale applications of many independent tasks. APlug suggests the best decomposition of the original computation into smaller tasks and the best number of CPUs to use, in order to meet user-specific constraints. We show that the problem is not trivial because there is large variability in the execution time of tasks, and it is possible for a task to occupy a CPU by performing useless computations. APlug collects a sample of task execution times and builds a model, which is then used by a discrete event simulator to calculate the optimal parameters. We provide a C++ API and a stand-alone implementation of APlug, and we integrate it with three typical applications from computational chemistry, bioinformatics, and data mining. A scenario for optimizing resources utilization is used to demonstrate our framework. We run experiments on 16,384 CPUs on a supercomputer, 480 cores on a Linux cluster and 80 cores on Amazon EC2, and show that APlug is very accurate with minimal overhead.

UR - http://www.scopus.com/inward/record.url?scp=84940868788&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940868788&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2015.7113338

DO - 10.1109/ICDE.2015.7113338

M3 - Conference contribution

AN - SCOPUS:84940868788

SN - 9781479979639

VL - 2015-May

SP - 843

EP - 854

BT - Proceedings - International Conference on Data Engineering

PB - IEEE Computer Society

ER -