May 2016

Megaparsecs of progress

At the University of Texas, Paul R. Shapiro and his team – including scientists from France, Germany, Spain, Switzerland and Texas – used Titan, the Cray XK7 supercomputer at the Department of Energy’s Oak Ridge Leadership Computing Facility to run a simulation called Cosmic Dawn, or CoDa.

This snapshot (above) from a 3-D simulation of the universe, shows hot ionized gases (bright patches) distributed around millions of galaxies, centered on the Milky Way, when reionization was half-over. Red represents high-density gas, blue low-density gas.

This simulation ran in a fully-coupled manner, combining calculations of gravitational pull between matter, hydrogen gas flows, and patterns of radiation. The team started with a cube – 90 megaparsecs, or about 300 million light-years, on a side – filled with neutral hydrogen gas to simulate what existed in the universe just hundreds of thousands of years after the Big Bang. Shapiro, UT’s Frank N. Edmonds Jr. Regents Professor in Astronomy, described the project in a talk at last fall’s SC15 supercomputing conference.

The simulation required high resolution to see small changes and high dynamic range to account for processes that took place on disparate time scales. To accomplish that, the team used a new code called Ramses-Cudation, which ran the simulation’s changes in velocity, density, temperature and the like on standard computer processors, or CPUs. It used associated graphics processing units (GPUs) to calculate the faster-changing radiation and ionization, working at 100 times the speed of the CPUs.

The simulation tracked 69 billion cells across cosmic reionization, consuming 2.1 million node hours – or 11 days – on Titan. The results, Shapiro explained at SC15, revealed bursts of radiation. As more of the volume lit up over time, starlight leaked out of the forming galaxies, and hydrogen reionized exponentially.

The results simulated some of cosmic evolution’s interactions. For example, the creation of stars drove reionization, which also constrained the formation of some other stars. The simulation’s results also showed reionization ending later than the period found in observations, but modifications in CoDa II should adjust for that, making an even more accurate simulation of how the universe reionized as galaxies and stars formed.

Image courtesy of Hyunbae Park, Paul Shapiro, Junhwan Choi and Anson D’Aloisio of the Cosmology Group in the University of Texas Department of Astronomy, using data supplied by Pierre Ocvirk of the University of Strasbourg.