GEAR-RT: Towards Exa-Scale Moment Based Radiative Transfer For Cosmological Simulations Using Task-Based Parallelism And Dynamic Sub-Cycling with SWIFT

Numerical simulations have become an indispensable tool in astrophysics and cosmology. The constant need for higher accuracy, higher resolutions, and models ofever-increasing sophistication and complexity drives the development of modern toolswhich target largest computing systems and employ state-of-the-art numerical methods and algorithms. Hence modern tools need to be developed while keeping optimization and parallelization strategies in mind from the start.In this work, the development and implementation of Gear-RT, a radiative transfersolver using the M1 closure in the open source code Swift, is presented, and validatedusing standard tests for radiative transfer. Gear-RT is modeled after Ramses-RT (Rosdahl et al. (2013)) with some key differences. Firstly, while Ramses-RT uses finitevolume methods and an adaptive mesh refinement (AMR) strategy, Gear-RT employsparticles as discretization elements and solves the equations using a finite volume particle method (FVPM). Secondly, Gear-RT makes use of the task-based parallelizationstrategy of Swift, which allows for optimized load balancing, increased cache efficiency, asynchronous communications, and a domain decomposition based on workrather than on data.Gear-RT is able to perform sub-cycles of radiative transfer steps w.r.t. a single hydrodynamics step. Radiation requires much smaller time step sizes than hydrodynamics,and sub-cycling permits calculations which are not strictly necessary to be skipped.Indeed, in a test case with gravity, hydrodynamics, and radiative transfer, the subcycling is able to reduce the runtime of a simulation by over 90%. Allowing only apart of the involved physics to be sub-cycled is a contrived matter when task-basedparallelism is involved, and it required the development of a secondary time steppingscheme parallel to the one employed for other physics. It is an entirely novel featurein Swift.Since Gear-RT uses a FVPM, a detailed introduction into finite volume methods andfinite volume particle methods is presented. In astrophysical literature, two FVPMmethods are written about: Hopkins (2015) have implemented one in their Gizmocode, while the one mentioned in Ivanova et al. (2013) isn't used to date. In this work,I test an implementation of the Ivanova et al. (2013) version, and conclude that in itscurrent form, it is not suitable for use with particles which are co-moving with thefluid, which in turn is an essential feature for cosmological simulations.Finally, the implementation of Acacia, a new algorithm to generate dark matter halomerger trees with the AMR code Ramses, is presented. As opposed to most availablemerger tree tools, it works on the fly during the course of the N-body simulation. Itcan track dark matter substructures individually using the index of the most boundparticles in the clump. Once a halo (or a sub-halo) merges into another one, the algorithm still tracks it through the last identified most bound particle in the clump,allowing to check at later snapshots whether the merging event was definitive. another one. The performance of the method is compared using standard validationdiagnostics, demonstrating that it reaches a quality similar to the best available andcommonly used merger tree tools. As proof of concept, Acacia is used together witha parametrized stellar-mass-to-halo-mass relation to generate a mock galaxy cataloguethat shows good agreement with observational data.