Virtual wind farms

A transformative gale is blowing through the U.S. power industry. The Energy Information Administration puts wind power’s share of America’s electricity generation at 8.4 percent in 2020, up from less than 1 percent in 1990. Increasingly competitive on cost and with demand for low-carbon energy sources rising, by 2050 wind power is expected to account for at least a third – and perhaps as much as half – of U.S. electricity production.

Achieving this energy revolution requires not just more wind farms but better ones, a 2019 review in the journal Science concluded. For example, a 2015 Department of Energy (DOE) workshop noted that as wind-generated electricity approaches 30 percent of U.S. power supply, just a two-percent efficiency improvement will save more than $1 billion annually.

To optimize performance, wind researchers are turning to DOE supercomputers.

“The wind energy problem has a lot of unknowns,” says Michael Sprague, a computational scientist at DOE’s National Renewable Energy Laboratory (NREL) in Golden, Colorado. “And if you don’t have models that can predict well enough the way a system is going to behave, under various conditions, it’s impossible to optimize that system. The wind problem is really a grand-challenge-type exascale computing problem.”

Since 2016, Sprague has led the development team for ExaWind, a high-performance, open-source code that aims to be the world’s most powerful and highest-fidelity computational tool for predictively simulating wind farms. ExaWind is a joint project of the Exascale Computing Project of DOE’s Office of Science and its National Nuclear Security Administration, and the Wind Energy Technologies Office of DOE’s Office of Energy Efficiency and Renewable Energy. The project involves more than 40 participants, primarily from NREL and Sandia National Laboratories.

NREL researchers have already demonstrated how lower-fidelity simulations can optimize wind-farm efficiency, Sprague notes. The Simulator fOr Wind Farm Applications (SOWFA) model let them predict how best to steer turbines’ wakes – orienting them to avoid dead zones around downwind turbines. Applying the model to the real world produced a predicted 10 percent reduction in wake losses.

Sprague emphasizes that the wind-energy industry requires the push higher-fidelity models provide because today’s turbines increasingly operate in the uncharted world of extreme science and engineering. Wind turbines are the largest rotating structures ever built, with blades longer than a football field on towers as tall as a 60-story skyscraper. And there are plans for even larger ones.

‘For gold-standard predictive models of a wind farm your model has billions of equations to solve.’

Simulating a wind farm of massive turbines, particularly in complex terrains, is an equally large challenge in computational fluid dynamics, Sprague says. It’s necessary to model across a vast range of scales and time steps – from the millimeter-level turbulence around a rotating blade to the hundreds-of-square kilometers of weather systems.

That means that “for gold-standard predictive models of a wind farm your model has billions of equations to solve,” Sprague says. “And you have to solve them again and again as you move through time.”

Each equation represents a grid point, the description of a localized moment of fluid dynamics. Previous leading wind energy models used tens of millions of grid points. “With ExaWind we’re getting into the tens of billions of grid points,” Sprague says – a  detail level that will enable predictions of wind farm dynamics and performance. ExaWind has already achieved a 51.5 billion grid-point simulation using 90 percent of the graphics processing units (GPUs) on Oak Ridge National Laboratory’s Summit, DOE’s most powerful supercomputer. Yet this demonstration involved simulating only the atmospheric boundary layer, no turbines included.

ExaWind integrates two core software codes with well-supported open-source libraries. The first is Nalu, an advanced computational fluid dynamics code developed at Sandia National Laboratories and adapted to become Nalu-Wind. The second is OpenFAST, an NREL wind-turbine simulation tool.

“Performance portability is really important in our software stack,” Sprague says. “We’ve gone through a lot of effort to make sure it will run on CPUs [central processing units or] and multiple flavors of GPUs.” Although Summit uses NVIDIA-designed GPUs, DOE’s first exascale machines, due to boot-up next year, will have both CPUs and GPUs, the latter designed by AMD and Intel.

In 2019, the ExaWind team significantly boosted the model’s speed and resolution by dividing the computational dynamics modeling into two parts, using different solvers for each – close to the turbine blade and the larger-scale turbulence.

Sprague says “this hybrid solver strategy is a game changer.” It lets Nalu-Wind use an unstructured grid to simulate near-blade fluid dynamics. Larger-scale atmospheric turbulence and turbine wakes are modeled using AMR-Wind. The ExaWind team based this structured-grid computational fluid dynamics code on the AMReX software framework developed at DOE’s Berkeley Lab. The software takes advantage of adaptive mesh refinement, which builds a fine computational grid in areas of greatest interest, concentrating computing power where it’s needed most. With this addition, ExaWind can solve equations for larger scale wind dynamics at least five times faster.

The researchers now are validating ExaWind against both proprietary wind-farm data and the best international public turbine-data sources, including the Danish DanAero turbine experiment, available through the International Energy Agency’s global wind energy research collaboration.

Sprague says that once it’s fully validated, researchers will run ExaWind’s extremely high-fidelity simulations to build knowledge and datasets as inputs to lower-fidelity for wind-farm design models – which will run on laptops.

These simulations will soon portray next-generation wind farms in increasingly extreme environments.

“If we’re really going to capture wind energy’s potential, we need to go offshore,” says Sprague, referring to future additions to ExaWind’s simulation ability. The turbines of offshore wind farms such as those along the Danish coast are anchored in shallow marine waters, but “around the United States we have very deep waters.”

This means that when ExaWind runs on DOE’s first exascale computers, it will provide data to optimize performance for a whole new kind of wind farm: skyscraper-sized, flexible turbines bobbing in the ocean on floating platforms, producing electricity for Americans.