July 2011

The nuts & bolts of Argonne’s cloud

Argonne National Laboratory is home to one of two cloud computing facilities named for circumnavigating explorer Ferdinand Magellan. The other is at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. The Department of Energy (DOE) launched the testbeds in 2010 with an American Recovery and Reinvestment Act allocation.

The machines are designed to test cloud computing’s capacity to meet the demand for scientific computing. Researchers with projects too small for DOE’s supercomputers but too large for in-house desktop or cluster computers can access Magellan as needed. By creating a “virtual machine,” the researchers can choose the number of processors on which to run their application. Cloud computers can be used to complete projects or to test code before scaling up to run on DOE’s biggest machines.

The Argonne Magellan cloud’s base system is comprised of IBM iDataPlex servers with 504 compute nodes, each equipped with dual quad-core processor chips. All told, the machine has 4,032 processor cores that combined provide a theoretical peak power of 40 teraflops (trillion mathematical operations per second). The processors access a total of 12 terabytes (trillion bytes) of system memory plus 250 terabytes of disk storage.

The system also has active storage servers with 200 compute/storage nodes, 40 terabytes of solid-state FLASH storage, 9.6 terabytes of system memory and 1.6 petabytes (quadrillion bytes) of disk space. Another 133 servers incorporate 266 graphics processing units (GPUs), a total of 8.5 terabytes of system memory and 133 terabytes of disc memory. Fifteen memory servers hold a total of 15 terabytes of system memory and 15 terabytes of disk space. To archive data, eight file servers provide 160 terabytes of storage. The systems are linked with a high-performance fiber-optic cable interconnect.

Researchers access Magellan via DOE’s ESNet network at a rate of up to 10 gigabits per second (thousands of times faster than a typical home cable Internet connection). Some can move data to and from Magellan at up to 100 gigabits per second via another DOE testbed, the Advanced Networking Initiative.