Transparent runtime parallelization of the R scripting language

Jiangtian Li, Xiaosong Ma, Srikanth Yoginath, Guruprasad Kora, Nagiza F. Samatova

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Scripting languages such as R and Matlab are widely used in scientific data processing. As the data volume and the complexity of analysis tasks both grow, sequential data processing using these tools often becomes the bottleneck in scientific workflows. We describe pR, a runtime framework for automatic and transparent parallelization of the popular R language used in statistical computing. Recognizing scripting languages' interpreted nature and data analysis codes' use pattern, we propose several novel techniques: (1) applying parallelizing compiler technology to runtime, whole-program dependence analysis of scripting languages, (2) incremental code analysis assisted with evaluation results, and (3) runtime parallelization of file accesses. Our framework does not require any modification to either the source code or the underlying R implementation. Experimental results demonstrate that pR can exploit both task and data parallelism transparently and overall has better performance as well as scalability compared to an existing parallel R package that requires code modification.

Original languageEnglish
Pages (from-to)157-168
Number of pages12
JournalJournal of Parallel and Distributed Computing
Volume71
Issue number2
DOIs
Publication statusPublished - 1 Feb 2011
Externally publishedYes

    Fingerprint

Keywords

  • Incremental analysis
  • Runtime parallelization
  • Scripting languages

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Software
  • Theoretical Computer Science

Cite this