Dishing up the early universe

When it comes to observing exploding stars, evolving galaxies and other celestial mysteries, combining the planet’s largest radio telescope with America’s most powerful supercomputer seems like a heavenly match.

The massive new radio telescope now under construction, the Square Kilometre Array (SKA), will span two continents and scour the universe for such cosmic objects as supermassive black holes, stellar nurseries, galaxy clusters and quasars. Researchers also plan to use the SKA to peer back in time and space toward the dawn of the universe.

SKA’s first phase will include nearly 200 mid-frequency radio telescope dish antennas, each 15 meters across and joined with fiber optics in South Africa. There already are 64 operational SKA precursor dishes there in the Karoo Desert, built as part of a project known as MeerKAT.

Phase one also will include more than 130,000 low-frequency, cone-shaped radio antennas, each about two meters tall, in Western Australia.

To process the first radio astronomy simulation data from the SKA, researchers will use the Summit supercomputer, developed by IBM and located at the Department of Energy’s (DOE’s) ’s Oak Ridge Leadership Computing Facility in Tennessee. Summit can perform 200,000 trillion calculations a second, or 200 petaflops.

Radio waves are much longer and weaker than visible light waves, so radio telescopes must be far larger than optical telescopes to make comparable observations. Fortunately, astronomers determined decades ago that if they combined signals from widely separated radio telescope antennas – a technique known as interferometry – they could produce images as bright and sharp as those from a single large radio antenna.

Roughly the size of two tennis courts, Summit has 185 miles of fiber-optic cable and weighs about 340 tons – roughly the weight of 75 African elephants. Its file system can store 250 petabytes of data, the equivalent of 74 years of high-definition video.

To train Summit to process SKA data, the team, with support from DOE’s SciDAC (Scientific Discovery through Advanced Computing) program, used a software simulator University of Oxford scientists designed to mimic the telescope array’s data collection, says Ruonan (Jason) Wang, a software engineer in Oak Ridge National Laboratory’s (ORNL’s) Scientific Data Group. The team, which also included Australia’s International Centre for Radio Astronomy Research (ICRAR) and China’s Shanghai Astronomical Observatory, fed Summit a cosmological model of the early universe and a low-frequency antenna-array configuration model to generate data similar to what radio telescopes observing the sky would produce, he says.

‘There is no way we could do science if we were unable to process all of the data.’

The simulation target was the Epoch of Reionization, a period from about 300,000 years after the Big Bang to a billion years later, the time that astronomers call First Light – when the earliest stars and galaxies began to flicker.

The goal was to simulate a real astronomical observation that could be verified, says Andreas Wicenec, director of ICRAR’s Data Intensive Astronomy program. “This was the first time that radio astronomical data has been processed at this scale.”

How are radio waves converted to images? For a parabolic telescope, radio waves from space bounce from the dish to a focal point at its tip, where they enter a receiver that measures and amplifies tiny voltage wave-induced fluctuations. These are digitized for processing and storage in a computer, which converts the data into images.

Scientists hope that mapping the early universe’s cold, primordial hydrogen gas, which emits telltale radio waves invisible to optical telescopes, will indicate how and when the cosmos first fired up. They also plan to overlay optical, ultraviolet, infrared and gamma-ray telescope images on the SKA pictures to better understand individual astronomical targets.

“There is no way we could do science if we were unable to process all of the data,” Wicenec says. “Once construction of the SKA is completed, we will have not only the world’s largest radio telescope, but also one of world’s largest data generators.”

Despite many hurdles, the researchers finally created an end-to-end SKA data-processing workflow in 2019 with Summit, the only machine in the world capable of such a breakthrough. The feat also will help the world’s radio astronomy community design future radio telescopes, like the proposed next-generation Very Large Array in New Mexico, Wicenec says.

To process simulated data, the researchers partly relied on the Adaptable I/O System (ADIOS), an open-source input/output framework developed by Scott Klasky and an ORNL team. ADIOS, which also receives SciDAC support, increased the efficiency of I/O operations and enabled data transfers between high-performance computing systems and other facilities to speed up Summit simulations.

The end-to-end workflow included more than 27,000 graphics processing units, or GPUs – specialized computer chips that quickly manipulate memory to accelerate certain computations. The GPUs deliver the bulk of Summit’s processing capability but require dedicated software for efficient use, Wicenec says.

Summit also is a workhorse for DOE supercomputing studies in biomedicine (including COVID-19 research), energy, and advanced materials – all using artificial intelligence. In machine learning, an AI subfield, computer systems recognize patterns and learn from data, often with no human intervention. “Machine learning is very likely the start of the next industrial revolution,” Wang says.

SKA’s second phase, expected to start near the end of the decade, will grow to thousands of parabolic dishes across Africa and more than a million low-frequency antennas in Australia, allowing astronomers to survey the sky much more quickly and efficiently than today.

Involving 16 countries, the SKA project will eventually employ thousands of scientists, engineers, support staff and students to chart the skies as never before. The configuration of both the low- and mid-frequency SKA antennas will allow astronomers to see the universe in a new light, far exceeding the resolution of current space- and ground-based observatories. And, Wang notes, “the faster we can process the data, the better we can understand the universe.”