Parallel graph mining with dynamic load balancing

Nilothpal Talukder, Mohammed J. Zaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Frequent subgraph mining (FSM) has important applications in areas such as bioinformatics, social networks and others. In this paper, we present a highly scalable approach called ParGraph that can efficiently mine from a single graph in both distributed as well as shared-memory based systems. In a distributed environment, we can leverage the local memory of multiple compute nodes for storing a large number of intermediate states for enumerating patterns. To address the skewness in the pattern generation tree, our approach uses a novel hybrid load balancing scheme to efficiently distribute workload across both processes and threads. Our experiments demonstrate good speedups using message passing interface (MPI) and OpenMP threads.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3352-3359
Number of pages8
ISBN (Electronic)9781467390040
DOIs
Publication statusPublished - 2 Feb 2017
Event4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States
Duration: 5 Dec 20168 Dec 2016

Other

Other4th IEEE International Conference on Big Data, Big Data 2016
CountryUnited States
CityWashington
Period5/12/168/12/16

Fingerprint

Dynamic loads
Resource allocation
Data storage equipment
Message passing
Bioinformatics
Experiments

Keywords

  • Dynamic Load Balancing
  • High Performance Computing
  • Parallel Frequent Graph Mining

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Hardware and Architecture

Cite this

Talukder, N., & Zaki, M. J. (2017). Parallel graph mining with dynamic load balancing. In Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 3352-3359). [7840995] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7840995

Parallel graph mining with dynamic load balancing. / Talukder, Nilothpal; Zaki, Mohammed J.

Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc., 2017. p. 3352-3359 7840995.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Talukder, N & Zaki, MJ 2017, Parallel graph mining with dynamic load balancing. in Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7840995, Institute of Electrical and Electronics Engineers Inc., pp. 3352-3359, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 5/12/16. https://doi.org/10.1109/BigData.2016.7840995
Talukder N, Zaki MJ. Parallel graph mining with dynamic load balancing. In Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 3352-3359. 7840995 https://doi.org/10.1109/BigData.2016.7840995
Talukder, Nilothpal ; Zaki, Mohammed J. / Parallel graph mining with dynamic load balancing. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 3352-3359
@inproceedings{aa458c71613048758b55bbd16357c337,
title = "Parallel graph mining with dynamic load balancing",
abstract = "Frequent subgraph mining (FSM) has important applications in areas such as bioinformatics, social networks and others. In this paper, we present a highly scalable approach called ParGraph that can efficiently mine from a single graph in both distributed as well as shared-memory based systems. In a distributed environment, we can leverage the local memory of multiple compute nodes for storing a large number of intermediate states for enumerating patterns. To address the skewness in the pattern generation tree, our approach uses a novel hybrid load balancing scheme to efficiently distribute workload across both processes and threads. Our experiments demonstrate good speedups using message passing interface (MPI) and OpenMP threads.",
keywords = "Dynamic Load Balancing, High Performance Computing, Parallel Frequent Graph Mining",
author = "Nilothpal Talukder and Zaki, {Mohammed J.}",
year = "2017",
month = "2",
day = "2",
doi = "10.1109/BigData.2016.7840995",
language = "English",
pages = "3352--3359",
booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Parallel graph mining with dynamic load balancing

AU - Talukder, Nilothpal

AU - Zaki, Mohammed J.

PY - 2017/2/2

Y1 - 2017/2/2

N2 - Frequent subgraph mining (FSM) has important applications in areas such as bioinformatics, social networks and others. In this paper, we present a highly scalable approach called ParGraph that can efficiently mine from a single graph in both distributed as well as shared-memory based systems. In a distributed environment, we can leverage the local memory of multiple compute nodes for storing a large number of intermediate states for enumerating patterns. To address the skewness in the pattern generation tree, our approach uses a novel hybrid load balancing scheme to efficiently distribute workload across both processes and threads. Our experiments demonstrate good speedups using message passing interface (MPI) and OpenMP threads.

AB - Frequent subgraph mining (FSM) has important applications in areas such as bioinformatics, social networks and others. In this paper, we present a highly scalable approach called ParGraph that can efficiently mine from a single graph in both distributed as well as shared-memory based systems. In a distributed environment, we can leverage the local memory of multiple compute nodes for storing a large number of intermediate states for enumerating patterns. To address the skewness in the pattern generation tree, our approach uses a novel hybrid load balancing scheme to efficiently distribute workload across both processes and threads. Our experiments demonstrate good speedups using message passing interface (MPI) and OpenMP threads.

KW - Dynamic Load Balancing

KW - High Performance Computing

KW - Parallel Frequent Graph Mining

UR - http://www.scopus.com/inward/record.url?scp=85015208336&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015208336&partnerID=8YFLogxK

U2 - 10.1109/BigData.2016.7840995

DO - 10.1109/BigData.2016.7840995

M3 - Conference contribution

AN - SCOPUS:85015208336

SP - 3352

EP - 3359

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -