Towards the optimization of a gyrokinetic Particle-In-Cell (PIC) code on large-scale hybrid architectures

With the aim of enabling state-of-the-art gyrokinetic PIC codes to benefit from the performance of recent multithreaded devices, we developed an application from a platform called the "PIC-engine" [1, 2, 3] embedding simplified basic features of the PIC method. The application solves the gyrokinetic equations in a sheared plasma slab using B-spline finite elements up to fourth order to represent the self-consistent electrostatic field. Preliminary studies of the so-called Particle-In-Fourier (PIF) approach, which uses Fourier modes as basis functions in the periodic dimensions of the system instead of the real-space grid, show that this method can be faster than PIC for simulations with a small number of Fourier modes. Similarly to the PIC-engine, multiple levels of parallelism have been implemented using MPI+OpenMP [2] and MPI+OpenACC [1], the latter exploiting the computational power of GPUs without requiring complete code rewriting. It is shown that sorting particles [3] can lead to performance improvement by increasing data locality and vectorizing grid memory access. Weak scalability tests have been successfully run on the GPU-equipped Cray XC30 Piz Daint (at CSCS) up to 4,096 nodes. The reduced time-to-solution will enable more realistic and thus more computationally intensive simulations of turbulent transport in magnetic fusion devices.

Gyrokinetic particle-in-cell global simulations of ion-temperature-gradient and collisionless-trapped-electron-mode turbulence in tokamaks

Sébastien Jolliet

The goal of thermonuclear fusion research is to provide power plants, that will be able to produce one gigawatt of electricity. Among the different ways to achieve fusion, the tokamak, based on magnetic confinement, is the most promising one. A gas is heated up to hundreds of millions of degrees and becomes a plasma, which is maintained – or confined – in a toroidal vessel by helical magnetic field lines. Then, deuterium and tritium are injected and fuse to create an α particle and an energetic neutron. In order to have a favorable power balance, the power produced by fusion reactions must exceed the power needed to heat the plasma and the power losses. This can be cast in a very simple expression which stipulates that the product of the density, the temperature and the energy confinement time must exceed some given value. Unfortunately, present-days tokamaks are not able to reach this condition, mostly due to plasma turbulence. The latter phenomenon enhances the heat losses and degrades the energy confinement time, which cannot be predicted by analytical theories such as the so-called neoclassical theory in which the heat losses are caused by Coulomb collisions. Therefore, numerical simulations are being developed to model plasma turbulence, mainly caused by the Ion and Electron Temperature-Gradient and the Trapped-Electron-Mode instabilities. The plasma is described by a distribution function which evolves according to the Vlasov equation. The electromagnetic fields created by the particles are self-consistently obtained through Maxwell's equations. The resulting Vlasov-Maxwell system is greatly simplified by using the gyrokinetic theory, which consists, through an appropriate ordering, of eliminating the fast gyromotion (compared to the typical frequency of instabilities). Nevertheless, it is still extremely difficult to solve this system numerically due to the large range of time and spatial scales to be resolved. In this thesis, the Vlasov-Maxwell system is solved in the electrostatic and collisionless limit with the Particle-In-Cell (PIC) ORB5 code in global tokamak geometry. This Monte-Carlo approach suffers from statistical noise which unavoidably degrades the quality of the simulation. Consequently, the first part of this work has been devoted to the optimization of the code with a view to reduce the numerical noise. The code has been rewritten in a new coordinate system which takes advantage of the anisotropy of turbulence, which is mostly aligned with the magnetic field lines. The overall result of the optimization is that for a given accuracy, the CPU time has been decreased by a factor two thousand, the total memory has been decreased by a factor ten and the numerical noise has been reduced by a factor two hundred. In addition, the scaling of the code with respect to plasma size is presently optimal, suggesting that ORB5 could compute heat transport for future fusion devices such as ITER. The second part of this thesis presents the validation of the code with numerical convergence tests, linear (including dispersion relations) and nonlinear benchmarks. Furthermore, the code has been applied to important issues in gyrokinetic theory. It is shown for the first time that a 5D global delta-f PIC code can achieve a thermodynamic steady state on the condition that some dissipation is present. This is a fundamental result as the main criticism against delta-f PIC codes is their inability to deal with long time simulations. Next, the role of the parallel nonlinearity is studied and it is demonstrated in this work that this term has no real influence on turbulence, provided the numerical noise is sufficiently low. This result should put an end to the controversy that recently occurred, in which gyrokinetic simulations using different numerical approaches yielded contradictory results. Finally, thanks to the optimization of the code, the gyrokinetic model has been extended to include the kinetic response of trapped-electrons, in place to the usual adiabatic (Boltzmann) approximation. For the first time, global TEM nonlinear simulations are presented, and the role of the zonal flow on heat transport is analyzed. This study will help in acquiring some knowledge on the less-known TEM turbulence (as compared to ITG). In conclusion, this thesis is one of the main steps of the development of ORB5, which is now a state-of-the-art gyrokinetic code for collisionless ITG and TEM turbulence, and has brought several contributions to the understanding of these phenomena.

EPFL2009

Development of a Global Version of the Gyrokinetic Microturbulence Code GENE

Stephan Brunner, Xavier Lapillonne, Ben McMillan, Laurent Villard

In order to address non-local effects in turbulent transport of magnetic-fusion devices, the gyrokinetic code GENE has been extended from a flux tube to a more global geometry. This global version of GENE includes radial variations of temperature and density profiles, as well as of magnetic equilibrium quantities. Non-periodic radial boundary conditions allow for profile evolution. Thanks to an interface with the MHD equilibrium code CHEASE, as well as the implementation of various types of source and sink terms, the global GENE version will provide a realistic model for carrying out gyrokinetic, quasi-stationary, microturbulence simulations over a major fraction of the core plasma. Validation of the code has in particular been ensured thanks to detailed comparisons with the global PIC codes GYGLES and ORB5.

2010

Investigating profile stiffness and critical gradients in shaped TCV discharges using local gyrokinetic simulations of turbulent transport

Stephan Brunner, Yann Camenen, Alessandro Marinoni, Gabriele Merlo, Olivier Sauter, Laurent Villard

The experimental observation made on the TCV tokamak of a significant confinement improvement in plasmas with negative triangularity (delta < 0) compared to those with standard positive triangularity has been interpreted in terms of different degrees of profile stiffness (Sauter et al 2014 Phys. Plasmas 21 055906) and/or different critical gradients. Employing the Eulerian gyrokinetic code GENE (Jenko et al 2000 Phys. Plasmas 7 1904), profile stiffness and critical gradients are studied under TCV relevant conditions. For the considered experimental discharges, trapped electron modes (TEMs) and electron temperature gradient (ETG) modes are the dominant microinstabilities, with the latter providing a significant contribution to the non-linear electron heat fluxes near the plasma edge. Two series of simulations with different levels of realism are performed, addressing the question of profile stiffness at various radial locations. Retaining finite collisionality, impurities and electromagnetic effects, as well as the physical electron-to-ion mass ratio are all necessary in order to approach the experimental flux measurements. However, flux-tube simulations are unable to fully reproduce the TCV results, pointing towards the need to carry out radially nonlocal (global) simulations, i.e. retaining finite machine size effects, in a future study. Some conclusions about the effect of triangularity can nevertheless be drawn based on the flux-tube results. In particular, the importance of considering the sensitivity to both temperature and density gradient is shown. The flux tube results show an increase of the critical gradients towards the edge, further enhanced when d < 0, and they also appear to indicate a reduction of profile stiffness towards plasma edge.