NVMalloc

Exposing an aggregate SSD store as a memory partition in extreme-scale machines

Chao Wang, Sudharshan S. Vazhkudai, Xiaosong Ma, Fei Meng, Youngjae Kim, Christian Engelmann

Research output: Chapter in Book/Report/Conference proceedingConference contribution

43 Citations (Scopus)

Abstract

DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-peta flop and exa flop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. In this paper, we propose a novel approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, we propose an NVMalloc library with a suite of services that enables applications to access a distributed NVM storage system. We have devised ways within NVMalloc so that the storage system, built from compute node-local NVM devices, can be accessed in a byte-addressable fashion using the memory mapped I/O interface. Our approach has the potential to re-energize out-of-core computations on large-scale machines by having applications allocate certain variables through NVMalloc, thereby increasing the overall memory capacity available. Our evaluation on a 128-core cluster shows that NVMalloc enables applications to compute problem sizes larger than the physical memory in a cost-effective manner. It can bring more performance/efficiency gain with increased computation time between NVM memory accesses or increased data access locality. In addition, our results suggest that while NVMalloc enables transparent access to NVM-resident variables, the explicit control it provides is crucial to optimize application performance.

Original languageEnglish
Title of host publicationProceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012
Pages957-968
Number of pages12
DOIs
Publication statusPublished - 4 Oct 2012
Externally publishedYes
Event2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012 - Shanghai, China
Duration: 21 May 201225 May 2012

Other

Other2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012
CountryChina
CityShanghai
Period21/5/1225/5/12

Fingerprint

Data storage equipment
Dynamic random access storage
Interfaces (computer)
Costs

ASJC Scopus subject areas

  • Software

Cite this

Wang, C., Vazhkudai, S. S., Ma, X., Meng, F., Kim, Y., & Engelmann, C. (2012). NVMalloc: Exposing an aggregate SSD store as a memory partition in extreme-scale machines. In Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012 (pp. 957-968). [6267902] https://doi.org/10.1109/IPDPS.2012.90

NVMalloc : Exposing an aggregate SSD store as a memory partition in extreme-scale machines. / Wang, Chao; Vazhkudai, Sudharshan S.; Ma, Xiaosong; Meng, Fei; Kim, Youngjae; Engelmann, Christian.

Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012. 2012. p. 957-968 6267902.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, C, Vazhkudai, SS, Ma, X, Meng, F, Kim, Y & Engelmann, C 2012, NVMalloc: Exposing an aggregate SSD store as a memory partition in extreme-scale machines. in Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012., 6267902, pp. 957-968, 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012, Shanghai, China, 21/5/12. https://doi.org/10.1109/IPDPS.2012.90
Wang C, Vazhkudai SS, Ma X, Meng F, Kim Y, Engelmann C. NVMalloc: Exposing an aggregate SSD store as a memory partition in extreme-scale machines. In Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012. 2012. p. 957-968. 6267902 https://doi.org/10.1109/IPDPS.2012.90
Wang, Chao ; Vazhkudai, Sudharshan S. ; Ma, Xiaosong ; Meng, Fei ; Kim, Youngjae ; Engelmann, Christian. / NVMalloc : Exposing an aggregate SSD store as a memory partition in extreme-scale machines. Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012. 2012. pp. 957-968
@inproceedings{60aeaeed38344aae937a5b3c43cc4681,
title = "NVMalloc: Exposing an aggregate SSD store as a memory partition in extreme-scale machines",
abstract = "DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-peta flop and exa flop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. In this paper, we propose a novel approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, we propose an NVMalloc library with a suite of services that enables applications to access a distributed NVM storage system. We have devised ways within NVMalloc so that the storage system, built from compute node-local NVM devices, can be accessed in a byte-addressable fashion using the memory mapped I/O interface. Our approach has the potential to re-energize out-of-core computations on large-scale machines by having applications allocate certain variables through NVMalloc, thereby increasing the overall memory capacity available. Our evaluation on a 128-core cluster shows that NVMalloc enables applications to compute problem sizes larger than the physical memory in a cost-effective manner. It can bring more performance/efficiency gain with increased computation time between NVM memory accesses or increased data access locality. In addition, our results suggest that while NVMalloc enables transparent access to NVM-resident variables, the explicit control it provides is crucial to optimize application performance.",
author = "Chao Wang and Vazhkudai, {Sudharshan S.} and Xiaosong Ma and Fei Meng and Youngjae Kim and Christian Engelmann",
year = "2012",
month = "10",
day = "4",
doi = "10.1109/IPDPS.2012.90",
language = "English",
isbn = "9780769546759",
pages = "957--968",
booktitle = "Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012",

}

TY - GEN

T1 - NVMalloc

T2 - Exposing an aggregate SSD store as a memory partition in extreme-scale machines

AU - Wang, Chao

AU - Vazhkudai, Sudharshan S.

AU - Ma, Xiaosong

AU - Meng, Fei

AU - Kim, Youngjae

AU - Engelmann, Christian

PY - 2012/10/4

Y1 - 2012/10/4

N2 - DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-peta flop and exa flop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. In this paper, we propose a novel approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, we propose an NVMalloc library with a suite of services that enables applications to access a distributed NVM storage system. We have devised ways within NVMalloc so that the storage system, built from compute node-local NVM devices, can be accessed in a byte-addressable fashion using the memory mapped I/O interface. Our approach has the potential to re-energize out-of-core computations on large-scale machines by having applications allocate certain variables through NVMalloc, thereby increasing the overall memory capacity available. Our evaluation on a 128-core cluster shows that NVMalloc enables applications to compute problem sizes larger than the physical memory in a cost-effective manner. It can bring more performance/efficiency gain with increased computation time between NVM memory accesses or increased data access locality. In addition, our results suggest that while NVMalloc enables transparent access to NVM-resident variables, the explicit control it provides is crucial to optimize application performance.

AB - DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-peta flop and exa flop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. In this paper, we propose a novel approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, we propose an NVMalloc library with a suite of services that enables applications to access a distributed NVM storage system. We have devised ways within NVMalloc so that the storage system, built from compute node-local NVM devices, can be accessed in a byte-addressable fashion using the memory mapped I/O interface. Our approach has the potential to re-energize out-of-core computations on large-scale machines by having applications allocate certain variables through NVMalloc, thereby increasing the overall memory capacity available. Our evaluation on a 128-core cluster shows that NVMalloc enables applications to compute problem sizes larger than the physical memory in a cost-effective manner. It can bring more performance/efficiency gain with increased computation time between NVM memory accesses or increased data access locality. In addition, our results suggest that while NVMalloc enables transparent access to NVM-resident variables, the explicit control it provides is crucial to optimize application performance.

UR - http://www.scopus.com/inward/record.url?scp=84866862083&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84866862083&partnerID=8YFLogxK

U2 - 10.1109/IPDPS.2012.90

DO - 10.1109/IPDPS.2012.90

M3 - Conference contribution

SN - 9780769546759

SP - 957

EP - 968

BT - Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012

ER -