Fully relativistic particle-in-cell (PIC) simulations are crucial for advancing knowledge of plasma physics. Modern state-of-the-art GPU supercomputers offer the potential to perform PIC simulations of unprecedented scale, but require robust and feature-rich codes which can fully leverage the computational resources. We have addressed this demand by adding GPU acceleration to the PIC code OSIRIS. In this article, we provide an overview of our algorithm, which features a CUDA extension to the underlying Fortran architecture. We present detailed performance benchmarks for thermal plasmas, which demonstrate excellent weak scaling on NERSC's Perlmutter supercomputer and high levels of absolute performance. We also show simulations of Weibel filamentation and laser-wakefield acceleration run with dynamic load balancing, which illustrate the robustness of the code to model a variety of physical systems. Finally, we show measurements of energy consumption, which indicate that the GPU algorithm is ~14 times faster and is ~7 times more energy-efficient than the optimized CPU algorithm on a node-to-node basis. Our development provides an important contribution to the PIC simulation community's computational demands both by enabling the use of OSIRIS on modern supercomputers and by providing a path to GPU acceleration of other PIC codes.

Work conducted under the auspices of DOE, LLNL, NSF, LLE