January 2022

Sustainable cities

Oak Ridge researchers harness Argonne’s Theta supercomputer to build energy-efficiency models for all U.S. buildings.

AutoBEM, software developed at Oak Ridge National Laboratory, has created a digital twin of nearly all the United States’ nearly 130 million buildings. This snapshot shows some of the nearly 180,000 buildings modeled in Chattanooga, Tennessee.

Five years ago, Joshua New of Oak Ridge National Laboratory had an idea for an ambitious project: an energy model of every building in the United States. At the time, urban-scale simulation was getting off the ground and programmers were beginning to move from studying single buildings to modeling city blocks or urban centers. “Looking at a city is much more interesting,” New says. Sufficiently accurate digital twins of buildings could give governments, corporations and utilities the data and scenarios that would allow them to invest confidently in energy efficiency strategies, reduce costs and help the environment.

New’s team has succeeded. Its tool, called Automatic Building Energy Modeling, or AutoBEM (, relies on open-source Department of Energy (DOE) software: EnergyPlus and OpenStudio.

Before AutoBEM, the largest urban energy models included just a couple thousand structures. To validate their work and demonstrate that their building models were realistic, the ORNL researchers partnered with EPB, a public electric company in Chattanooga, Tennessee, to model more than 178,000 structures and compare simulated energy use with the utility’s anonymized data. In the journal Energies, they described how a handful of strategies, such as using smart thermostats and changing water heater and HVAC systems, could improve energy efficiency and curtail demand in some building types.

New’s love of computer science started in high school, where he fixed machines all over campus during his senior year. As a graduate student at the University of Tennessee, Knoxville, he focused on supercomputing and scientific visualization. In 2004, he started at ORNL, where his Ph.D. research included bioinformatics applied to genetic changes in mice. In 2009 the lab offered him a job in building technologies, so New chose to shift gears to work on energy efficiency.

As the lone computer scientist among mechanical and electrical engineers, New initially felt out of place. But he found his niche as he applied his computational expertise to traditional building science. First New built simulation-based tools that help consumers estimate energy use and efficiency strategies’ effects on individual homes and businesses. His team built a web-based tool for predicting the cost-effectiveness of roof and attic-cooling technologies, winning an R&D 100 Award in 2016. He then began coordinating with teams at other national labs as part of ORNL’s Building Energy Modeling program. The team simulates and analyzes buildings that incorporate individual retrofits or technologies and calculates how those strategies affect certain building types. Much of that work uses EnergyPlus, open-source DOE software that captures the physics of buildings to estimate energy use.

To implement AutoBEM, New’s team had to think about energy data differently and about how to manage privacy. The researchers couldn’t audit the energy of individual buildings, and for privacy reasons they couldn’t look inside buildings with surveillance instruments. Instead, they relied on satellite imaging, street-view mapping and LIDAR (laser imaging, detection and ranging) that helped them map features such as building size, height and age.

A typical building model requires thousands of data inputs. Some of those data are critically important, but other pieces are negligible for understanding energy use. AutoBEM uses sensitivity analysis to collect information on factors that have the largest effects.

The team also had to consider how a building’s use changes energy consumption. A supermarket, with its refrigerated display cases, coolant systems and freezers, uses energy differently from other types of commercial spaces or homes.

AutoBEM can perform a year-long simulation of more than a million buildings in one hour.

To manage such data and fill the gaps, the team used a variety of artificial intelligence algorithms, cloud computing resources, university supercomputers and DOE systems. The work leading up to and including a model of all U.S. buildings used 45 million core-hours on the Theta supercomputer at Argonne National Laboratory, the country’s most powerful all-CPU system.

A critical piece of model development was ensuring that its predictions matched energy use from a real city. The partnership with EPB in Chattanooga, 100 miles southwest of ORNL, began in 2016 after Jim Ingraham, the utility’s strategic research vice president, attended a New presentation. EPB was already working on a variety of projects – integrating smart thermostats in homes and businesses, adding solar power and storage batteries and increasing grid resilience against natural disasters. After the talk, Ingraham followed New back to his office and asked “what do we have to do to work together on this?”

EPB’s smart grid records power usage information at 15-minute intervals for all customers, providing a way for New’s team to validate its models with a real-world test case. The researchers modeled 178,000 Chattanooga buildings, simulated their projected energy use and then compared the results with anonymized data from those structures.

The Energies paper is just a small slice of the results, New says. AutoBEM can perform a year-long simulation of more than a million buildings in one hour on Theta. This let the team assess how various retrofitting strategies would affect energy use for every building in EPB’s service area ( And with support from the ASCR Leadership Computing Challenge, the ORNL team has achieved its initial AutoBEM goal of creating a virtual model for nearly every building in America (

AutoBEM results don’t yet consistently reach the high bar of investment-grade models, an industry standard that would put these methods in wider use. New and his colleagues are working on algorithms and data strategies to reach that goal, and they aim to improve their models with a 2022 award of 800,000 node-hours, again on Theta, from the ASCR Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program.

Insights from the AutoBEM work are shaping EPB’s projects, Ingraham says. The utility already had partnered with the Tennessee Valley Authority and local nonprofits on a so-called energy-uplift program to help residents struggling to pay bills. The program provides education and improvements to help save energy and lower costs – insulation upgrades in attics and crawlspaces, crack-closure around doors and windows and smart thermostats. “We can do the analysis individually” to calculate which homes would benefit from the program, Ingraham says. But with the AutoBEM models, “we’re expanding that now to look at all the homes on that circuit.” As a result, EPB has identified many houses the program could help.

U.S. companies and utilities already invest nearly $30 billion each year in improving building energy efficiency, New says, but more is needed. “The thing I’m most passionate about is the potential for automated financing,” he says. With refined building models that project the savings from new technologies, lenders could easily see the potential benefits. “I think that’s the way to unlock major investment to make our built environment more sustainable.”