Adding up alloys

The future of additive manufacturing (AM), commonly called 3D printing, hinges on learning metals’ microstructure. Demands for more fuel-efficient transportation and cleaner energy, for example, require components that perform better than ever. AM researchers are particularly interested in metals called superalloys to meet these needs.

“Nickel-based superalloys are really common for high-temperature applications,” says Stephen DeWitt, a computational scientist at Oak Ridge National Laboratory (ORNL). “Essentially, the turbine blades in either a jet engine or a natural-gas turbine are incredibly expensive, and a small increase in the properties can save a ton of money and a ton of pollution.”

Materials scientists must better understand how molten metals solidify before they can use AM to build components with precise properties. DeWitt and six co-investigators at ORNL, Los Alamos National Laboratory (LANL) and Lawrence Livermore National Laboratory (LLNL) will use Summit, an IBM AC922 supercomputer at Oak Ridge, to create the most detailed model yet of a solidifying molten-metal pool. To conduct the research, DeWitt has received 370,000 node-hours on the machine from the Department of Energy (DOE) Office of Science’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program.

Using what’s called a phase-field model, the team can simulate phase transformation in the narrow region around the pool’s outer edge where solidification begins and moves to the center. The “field” in the model’s name denotes whether the material in each location is a solid or liquid (the “phase”) or in transition between the two. Several computational science groups have used similar phase-field models for simpler solidification simulations. The models, done in two dimensions due to limited computing power, represented metal crystals as they grow in branching, treelike shapes called dendrites, but couldn’t fully depict how they collide and interact.

With Summit’s 27,000-plus graphics processing units (GPUs), capable of 200,000 trillion operations a second, DeWitt and his colleagues will bring more computing power to the problem than ever. “Through this INCITE allocation,” he says, “we’re actually able to do this whole thing in 3D so that we can get the complex interactions of these dendrites in all three dimensions, not projected into a plane.”

The simulations are running in parallel with work by the Colorado School of Mines’ Amy Clarke and the University of California Santa Barbara’s Tresa Pollack, who “are doing the experiment version of what we’re doing, looking at these spot melts in single crystals,” DeWitt says. “The main takeaway is that they are doing in-situ radiography at the Advanced Photon Source at Argonne National Laboratory to watch the (microscopic) re-solidification of spot melts of a single crystal. That setup is what our simulations are based on.”

DeWitt and colleagues will conduct four simulations of spot melts in a nickel-aluminum alloy using Tusas, a phase-field code that breaks simulations into pieces and solves them in parallel, making efficient use of massive machines such as Summit. LANL mathematician and INCITE team member Christopher Newman is lead developer of the code.

Each simulation will demand Summit’s full computational resources for a total of about 24 hours.

The team will model a spot melt 100 microns in diameter – 10 grains wide with each grain comprised of 10 dendrites. Dewitt explains: “We’re going to simulate the process of shining a laser on a piece of solid metal, creating a melt pool, and then having that melt pool solidify, and do that all at a dendrite-resolved scale.”

The first simulation will model solidification of a single grain, which is composed of dendrites that all have the same the orientation. The second will model multiple grains as they grow, with their dendrites interacting from different orientations. The third simulation will feature a single grain solidifying with random additional crystals popping up through nucleation and interacting with dendrites growing from the original grain.

“The fourth one is to jam that all together,” DeWitt says. “So now we have a substrate with multiple grains, and we have nucleation, so we have a whole bunch of grains growing at the beginning and then new grains can be popping in over the course of the simulation.”

The simulation must handle five physics problems at once, DeWitt says. It will track the transformation from liquid to solid for various crystal orientations, the formation of new grains, the motions of the elements, heat transport, and heat input from the laser. Each simulation will demand Summit’s full computational resources for a total of about 24 hours, done in six- to eight-hour segments. “That’s why we’re only doing four simulations,” DeWitt says with a laugh. “Essentially, four of these simulations blows our entire allocation, so we have to be very, very selective and very strategic about how we’re doing it.”

Once the simulations are done and the datasets are published, DeWitt and colleagues, as well as other groups, can begin to design more efficient models that generate the most relevant data needed to design AM materials with specific properties.

The project is part of the Exascale Computing Project (ECP), a collaboration between DOE’s Office of Science and National Nuclear Security Administration. The ECP aims to deliver a computing ecosystem that can perform 1 million trillion operations per second. The effort within ECP that DeWitt leads falls under the Exascale Additive Manufacturing (ExaAM) project, which is developing exascale computing capability to precisely design AM components with specified properties, including different ones in various parts of the same component. John Turner, ORNL’s Computational Engineering Program director, and LLNL Senior Scientist James Belak are the project’s co-investigators as well as ExaAM’s principal investigator and co-principal investigator.

Co-investigators Jean-Luc Fattebert and Balasubramaniam Radhakrishnan also bring phase-field modeling expertise to the spot melt project. ORNL Research Scientist Fattebert focuses on efficient ways to track the formation of multiple grains. Radhakrishnan, an ORNL senior researcher, has “deep material science expertise,” DeWitt says, and will lead the modeling of crystals that arise randomly among the dendrites. Stephen Nichols, a computational scientist at the Oak Ridge Leadership Computer Facility, Summit’s home, will work with the team to ensure it makes the most of Summit, DeWitt says.

In fact, making the most of Summit may be the project’s biggest challenge. The code must simultaneously handle all five key physics challenges and will have to run efficiently at scale on 27,000 GPUs at once, DeWitt notes. “We need the expertise of seven people to pull this off.”