RSVM: A region-based software virtual memory for GPU

Feng Ji, Heshan Lin, Xiaosong Ma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Citations (Scopus)

Abstract

While Graphics Processing Units (GPU) have gained much success in general purpose computing in recent years, their programming is still difficult, due to, particularly, explicitly managed GPU memory and manual CPU-GPU data transfer. Despite recent calls for managing GPU resources as first-class citizens in the operating system, a mature GPU memory management mechanism is still missing, which leads to reinventing the wheels in various GPU system software. Meanwhile, due to ever enlarging problem sizes, we urgently need a system-level mechanism for unified CPU-GPU memory management. In this work, we present the design of Region-based Software Virtual Memory (RSVM), a software virtual memory running on both CPU and GPU in a distributed and cooperative way. In addition to automatic GPU memory management and GPU-CPU data transfer, RSVM offers two novel features: 1) GPU kernel-issued on-demand data fetching from the host into the GPU memory, and 2) intra-kernel transparent GPU memory swapping into the main memory. Our study reveals important insights on the challenges and opportunities of building unified virtual memory systems for heterogeneous computing. Experimental results on real GPU benchmarks demonstrate that, though it incurs a small overhead, RSVM can transparently scale GPU kernels to large problem sizes exceeding the device memory size limit; developers write the same code for different problem sizes, but still can optimize on data layout definition accordingly. Our evaluation also identifies missing GPU architecture features for better system software efficiency.

Original languageEnglish
Title of host publicationParallel Architectures and Compilation Techniques - Conference Proceedings, PACT
Pages269-278
Number of pages10
DOIs
Publication statusPublished - 18 Nov 2013
Externally publishedYes
Event22nd International Conference on Parallel Architectures and Compilation Techniques, PACT 2013 - Edinburgh, United Kingdom
Duration: 7 Sep 201311 Sep 2013

Other

Other22nd International Conference on Parallel Architectures and Compilation Techniques, PACT 2013
CountryUnited Kingdom
CityEdinburgh
Period7/9/1311/9/13

Fingerprint

Virtual Memory
Graphics Processing Unit
Data storage equipment
Software
Memory Management
Program processors
Graphics processing unit
Data Transfer
Data transfer
kernel
Software System
Heterogeneous Computing
Computer operating systems

Keywords

  • GPGPU
  • GPU Memory Mangement
  • Heterogeneous System

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Theoretical Computer Science

Cite this

Ji, F., Lin, H., & Ma, X. (2013). RSVM: A region-based software virtual memory for GPU. In Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT (pp. 269-278). [6618823] https://doi.org/10.1109/PACT.2013.6618823

RSVM : A region-based software virtual memory for GPU. / Ji, Feng; Lin, Heshan; Ma, Xiaosong.

Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT. 2013. p. 269-278 6618823.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ji, F, Lin, H & Ma, X 2013, RSVM: A region-based software virtual memory for GPU. in Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT., 6618823, pp. 269-278, 22nd International Conference on Parallel Architectures and Compilation Techniques, PACT 2013, Edinburgh, United Kingdom, 7/9/13. https://doi.org/10.1109/PACT.2013.6618823
Ji F, Lin H, Ma X. RSVM: A region-based software virtual memory for GPU. In Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT. 2013. p. 269-278. 6618823 https://doi.org/10.1109/PACT.2013.6618823
Ji, Feng ; Lin, Heshan ; Ma, Xiaosong. / RSVM : A region-based software virtual memory for GPU. Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT. 2013. pp. 269-278
@inproceedings{a6e7ee13420148b38471e4c3a5db1722,
title = "RSVM: A region-based software virtual memory for GPU",
abstract = "While Graphics Processing Units (GPU) have gained much success in general purpose computing in recent years, their programming is still difficult, due to, particularly, explicitly managed GPU memory and manual CPU-GPU data transfer. Despite recent calls for managing GPU resources as first-class citizens in the operating system, a mature GPU memory management mechanism is still missing, which leads to reinventing the wheels in various GPU system software. Meanwhile, due to ever enlarging problem sizes, we urgently need a system-level mechanism for unified CPU-GPU memory management. In this work, we present the design of Region-based Software Virtual Memory (RSVM), a software virtual memory running on both CPU and GPU in a distributed and cooperative way. In addition to automatic GPU memory management and GPU-CPU data transfer, RSVM offers two novel features: 1) GPU kernel-issued on-demand data fetching from the host into the GPU memory, and 2) intra-kernel transparent GPU memory swapping into the main memory. Our study reveals important insights on the challenges and opportunities of building unified virtual memory systems for heterogeneous computing. Experimental results on real GPU benchmarks demonstrate that, though it incurs a small overhead, RSVM can transparently scale GPU kernels to large problem sizes exceeding the device memory size limit; developers write the same code for different problem sizes, but still can optimize on data layout definition accordingly. Our evaluation also identifies missing GPU architecture features for better system software efficiency.",
keywords = "GPGPU, GPU Memory Mangement, Heterogeneous System",
author = "Feng Ji and Heshan Lin and Xiaosong Ma",
year = "2013",
month = "11",
day = "18",
doi = "10.1109/PACT.2013.6618823",
language = "English",
isbn = "9781479910212",
pages = "269--278",
booktitle = "Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT",

}

TY - GEN

T1 - RSVM

T2 - A region-based software virtual memory for GPU

AU - Ji, Feng

AU - Lin, Heshan

AU - Ma, Xiaosong

PY - 2013/11/18

Y1 - 2013/11/18

N2 - While Graphics Processing Units (GPU) have gained much success in general purpose computing in recent years, their programming is still difficult, due to, particularly, explicitly managed GPU memory and manual CPU-GPU data transfer. Despite recent calls for managing GPU resources as first-class citizens in the operating system, a mature GPU memory management mechanism is still missing, which leads to reinventing the wheels in various GPU system software. Meanwhile, due to ever enlarging problem sizes, we urgently need a system-level mechanism for unified CPU-GPU memory management. In this work, we present the design of Region-based Software Virtual Memory (RSVM), a software virtual memory running on both CPU and GPU in a distributed and cooperative way. In addition to automatic GPU memory management and GPU-CPU data transfer, RSVM offers two novel features: 1) GPU kernel-issued on-demand data fetching from the host into the GPU memory, and 2) intra-kernel transparent GPU memory swapping into the main memory. Our study reveals important insights on the challenges and opportunities of building unified virtual memory systems for heterogeneous computing. Experimental results on real GPU benchmarks demonstrate that, though it incurs a small overhead, RSVM can transparently scale GPU kernels to large problem sizes exceeding the device memory size limit; developers write the same code for different problem sizes, but still can optimize on data layout definition accordingly. Our evaluation also identifies missing GPU architecture features for better system software efficiency.

AB - While Graphics Processing Units (GPU) have gained much success in general purpose computing in recent years, their programming is still difficult, due to, particularly, explicitly managed GPU memory and manual CPU-GPU data transfer. Despite recent calls for managing GPU resources as first-class citizens in the operating system, a mature GPU memory management mechanism is still missing, which leads to reinventing the wheels in various GPU system software. Meanwhile, due to ever enlarging problem sizes, we urgently need a system-level mechanism for unified CPU-GPU memory management. In this work, we present the design of Region-based Software Virtual Memory (RSVM), a software virtual memory running on both CPU and GPU in a distributed and cooperative way. In addition to automatic GPU memory management and GPU-CPU data transfer, RSVM offers two novel features: 1) GPU kernel-issued on-demand data fetching from the host into the GPU memory, and 2) intra-kernel transparent GPU memory swapping into the main memory. Our study reveals important insights on the challenges and opportunities of building unified virtual memory systems for heterogeneous computing. Experimental results on real GPU benchmarks demonstrate that, though it incurs a small overhead, RSVM can transparently scale GPU kernels to large problem sizes exceeding the device memory size limit; developers write the same code for different problem sizes, but still can optimize on data layout definition accordingly. Our evaluation also identifies missing GPU architecture features for better system software efficiency.

KW - GPGPU

KW - GPU Memory Mangement

KW - Heterogeneous System

UR - http://www.scopus.com/inward/record.url?scp=84887490622&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887490622&partnerID=8YFLogxK

U2 - 10.1109/PACT.2013.6618823

DO - 10.1109/PACT.2013.6618823

M3 - Conference contribution

AN - SCOPUS:84887490622

SN - 9781479910212

SP - 269

EP - 278

BT - Parallel Architectures and Compilation Techniques - Conference Proceedings, PACT

ER -