Enabling high-fidelity neutron transport simulations on petascale architectures

Dinesh Kaushik, Micheal Smith, Allan Wollaber, Barry Smith, Andrew Siegel, Won Sik Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

The UNIC code is being developed as part of the DOE's Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. UNIC is an unstructured, deterministic neutron transport code that allows a highly detailed description of a nuclear reactor. The primary goal of our simulation efforts is to reduce the uncertainties and biases in reactor design calculations by progressively replacing existing multilevel averaging (homogenization) techniques with more direct solution methods based on first principles. Since the neutron transport equation is seven dimensional (three in space, two in angle, one in energy, and one in time), these simulations are among the most memory and computationally intensive in all of computational science. In order to model the complex physics of a reactor core, billions of spatial elements, hundreds of angles, and thousands of energy groups are necessary, leading to problem sizes with petascale degrees of freedom. Therefore, these calculations exhaust memory resources on current and even next-generation architectures. In this paper, we present UNIC simulation results for two important representative problems in reactor design and analysis - -PHENIX and ZPR-6. In each case, UNIC shows good weak scalability on up to 163,840 cores of Blue Gene/P (Argonne) and 122,800 cores of XT5 (Oak Ridge). While our current per processor performance is less than ideal, we demonstrate a clear ability to effectively utilize the leadership computing platforms. Over the coming months, we aim to improve the per processor performance while maintaining the high parallel efficiency by employing better algorithms such as spatial p- and h-multigrid preconditioners, optimized matrix-tensor operations, and weighted partitioning for better load balancing. Combining these additional algorithmic improvements with the availability of larger parallel machines should allow us to realize our long-term goal of explicit geometry coupled multiphysics reactor simulations. In the long run, these high-fidelity simulations will be able to replace expensive mockup experiments and reduce the uncertainty in crucial reactor design and operational parameters.

Original languageEnglish
Title of host publicationProceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09
DOIs
Publication statusPublished - 2009
Externally publishedYes
EventConference on High Performance Computing Networking, Storage and Analysis, SC '09 - Portland, OR, United States
Duration: 14 Nov 200920 Nov 2009

Other

OtherConference on High Performance Computing Networking, Storage and Analysis, SC '09
CountryUnited States
CityPortland, OR
Period14/11/0920/11/09

Fingerprint

Neutrons
Data storage equipment
Reactor cores
Nuclear reactors
Nuclear energy
Resource allocation
Tensors
Scalability
Physics
Genes
Availability
Geometry
Experiments
Uncertainty

Keywords

  • Discrete ordinates
  • Memory bandwidth
  • Neutron transport
  • Nuclear reactors
  • Parallel scalability

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Kaushik, D., Smith, M., Wollaber, A., Smith, B., Siegel, A., & Yang, W. S. (2009). Enabling high-fidelity neutron transport simulations on petascale architectures. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09 [1654128] https://doi.org/10.1145/1654059.1654128

Enabling high-fidelity neutron transport simulations on petascale architectures. / Kaushik, Dinesh; Smith, Micheal; Wollaber, Allan; Smith, Barry; Siegel, Andrew; Yang, Won Sik.

Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09. 2009. 1654128.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kaushik, D, Smith, M, Wollaber, A, Smith, B, Siegel, A & Yang, WS 2009, Enabling high-fidelity neutron transport simulations on petascale architectures. in Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09., 1654128, Conference on High Performance Computing Networking, Storage and Analysis, SC '09, Portland, OR, United States, 14/11/09. https://doi.org/10.1145/1654059.1654128
Kaushik D, Smith M, Wollaber A, Smith B, Siegel A, Yang WS. Enabling high-fidelity neutron transport simulations on petascale architectures. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09. 2009. 1654128 https://doi.org/10.1145/1654059.1654128
Kaushik, Dinesh ; Smith, Micheal ; Wollaber, Allan ; Smith, Barry ; Siegel, Andrew ; Yang, Won Sik. / Enabling high-fidelity neutron transport simulations on petascale architectures. Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09. 2009.
@inproceedings{3f3e49712780456fa79207a4641fcf75,
title = "Enabling high-fidelity neutron transport simulations on petascale architectures",
abstract = "The UNIC code is being developed as part of the DOE's Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. UNIC is an unstructured, deterministic neutron transport code that allows a highly detailed description of a nuclear reactor. The primary goal of our simulation efforts is to reduce the uncertainties and biases in reactor design calculations by progressively replacing existing multilevel averaging (homogenization) techniques with more direct solution methods based on first principles. Since the neutron transport equation is seven dimensional (three in space, two in angle, one in energy, and one in time), these simulations are among the most memory and computationally intensive in all of computational science. In order to model the complex physics of a reactor core, billions of spatial elements, hundreds of angles, and thousands of energy groups are necessary, leading to problem sizes with petascale degrees of freedom. Therefore, these calculations exhaust memory resources on current and even next-generation architectures. In this paper, we present UNIC simulation results for two important representative problems in reactor design and analysis - -PHENIX and ZPR-6. In each case, UNIC shows good weak scalability on up to 163,840 cores of Blue Gene/P (Argonne) and 122,800 cores of XT5 (Oak Ridge). While our current per processor performance is less than ideal, we demonstrate a clear ability to effectively utilize the leadership computing platforms. Over the coming months, we aim to improve the per processor performance while maintaining the high parallel efficiency by employing better algorithms such as spatial p- and h-multigrid preconditioners, optimized matrix-tensor operations, and weighted partitioning for better load balancing. Combining these additional algorithmic improvements with the availability of larger parallel machines should allow us to realize our long-term goal of explicit geometry coupled multiphysics reactor simulations. In the long run, these high-fidelity simulations will be able to replace expensive mockup experiments and reduce the uncertainty in crucial reactor design and operational parameters.",
keywords = "Discrete ordinates, Memory bandwidth, Neutron transport, Nuclear reactors, Parallel scalability",
author = "Dinesh Kaushik and Micheal Smith and Allan Wollaber and Barry Smith and Andrew Siegel and Yang, {Won Sik}",
year = "2009",
doi = "10.1145/1654059.1654128",
language = "English",
isbn = "9781605587448",
booktitle = "Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09",

}

TY - GEN

T1 - Enabling high-fidelity neutron transport simulations on petascale architectures

AU - Kaushik, Dinesh

AU - Smith, Micheal

AU - Wollaber, Allan

AU - Smith, Barry

AU - Siegel, Andrew

AU - Yang, Won Sik

PY - 2009

Y1 - 2009

N2 - The UNIC code is being developed as part of the DOE's Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. UNIC is an unstructured, deterministic neutron transport code that allows a highly detailed description of a nuclear reactor. The primary goal of our simulation efforts is to reduce the uncertainties and biases in reactor design calculations by progressively replacing existing multilevel averaging (homogenization) techniques with more direct solution methods based on first principles. Since the neutron transport equation is seven dimensional (three in space, two in angle, one in energy, and one in time), these simulations are among the most memory and computationally intensive in all of computational science. In order to model the complex physics of a reactor core, billions of spatial elements, hundreds of angles, and thousands of energy groups are necessary, leading to problem sizes with petascale degrees of freedom. Therefore, these calculations exhaust memory resources on current and even next-generation architectures. In this paper, we present UNIC simulation results for two important representative problems in reactor design and analysis - -PHENIX and ZPR-6. In each case, UNIC shows good weak scalability on up to 163,840 cores of Blue Gene/P (Argonne) and 122,800 cores of XT5 (Oak Ridge). While our current per processor performance is less than ideal, we demonstrate a clear ability to effectively utilize the leadership computing platforms. Over the coming months, we aim to improve the per processor performance while maintaining the high parallel efficiency by employing better algorithms such as spatial p- and h-multigrid preconditioners, optimized matrix-tensor operations, and weighted partitioning for better load balancing. Combining these additional algorithmic improvements with the availability of larger parallel machines should allow us to realize our long-term goal of explicit geometry coupled multiphysics reactor simulations. In the long run, these high-fidelity simulations will be able to replace expensive mockup experiments and reduce the uncertainty in crucial reactor design and operational parameters.

AB - The UNIC code is being developed as part of the DOE's Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. UNIC is an unstructured, deterministic neutron transport code that allows a highly detailed description of a nuclear reactor. The primary goal of our simulation efforts is to reduce the uncertainties and biases in reactor design calculations by progressively replacing existing multilevel averaging (homogenization) techniques with more direct solution methods based on first principles. Since the neutron transport equation is seven dimensional (three in space, two in angle, one in energy, and one in time), these simulations are among the most memory and computationally intensive in all of computational science. In order to model the complex physics of a reactor core, billions of spatial elements, hundreds of angles, and thousands of energy groups are necessary, leading to problem sizes with petascale degrees of freedom. Therefore, these calculations exhaust memory resources on current and even next-generation architectures. In this paper, we present UNIC simulation results for two important representative problems in reactor design and analysis - -PHENIX and ZPR-6. In each case, UNIC shows good weak scalability on up to 163,840 cores of Blue Gene/P (Argonne) and 122,800 cores of XT5 (Oak Ridge). While our current per processor performance is less than ideal, we demonstrate a clear ability to effectively utilize the leadership computing platforms. Over the coming months, we aim to improve the per processor performance while maintaining the high parallel efficiency by employing better algorithms such as spatial p- and h-multigrid preconditioners, optimized matrix-tensor operations, and weighted partitioning for better load balancing. Combining these additional algorithmic improvements with the availability of larger parallel machines should allow us to realize our long-term goal of explicit geometry coupled multiphysics reactor simulations. In the long run, these high-fidelity simulations will be able to replace expensive mockup experiments and reduce the uncertainty in crucial reactor design and operational parameters.

KW - Discrete ordinates

KW - Memory bandwidth

KW - Neutron transport

KW - Nuclear reactors

KW - Parallel scalability

UR - http://www.scopus.com/inward/record.url?scp=74049110481&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=74049110481&partnerID=8YFLogxK

U2 - 10.1145/1654059.1654128

DO - 10.1145/1654059.1654128

M3 - Conference contribution

SN - 9781605587448

BT - Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09

ER -