IBM Roadrunner

What is IBM Roadrunner?

Roadrunner was a supercomputer developed by IBM at the Los Alamos National Laboratory in New Mexico. This hybrid supercomputer cost $133 million to develop. In 2008, Roadrunner became the first computer to break what has been called the petaflop barrier -- the ability to operate at a rate of a thousand trillion floating point operations per second (FLOPS), which is a crucial measure of computing power and speed.

Since 2008, other supercomputers built by IBM and other companies have hit the petaflop mark. However, Roadrunner remains well-known as the first supercomputer to do so. Despite its visionary status as a powerful supercomputer, the Roadrunner was decommissioned in 2013 due to its low energy efficiency.

Development of IBM Roadrunner

Development on the IBM Roadrunner started in 2002. Named for the state bird of New Mexico, the supercomputer went online in 2006 and became operational in 2008. IBM built the computer for the U.S. Department of Energy's (DOE) National Nuclear Security Administration at the Los Alamos National Laboratory (LANL). Its predecessor, called "Dark Horse" was also developed at the LANL.

Constructed in three phases, Roadrunner occupied approximately 296 IBM BladeCenter racks and cabinets which covered 560 square meters, the area of a moderately sized warehouse. After assembly, 21 tractor trailer trucks were required to deliver the supercomputer to the LANL.

When Roadrunner was first conceived in 2002, its three-phase design was planned to eventually reach 1.7 petaflops. The first phase, a standard AMD Opteron 2210 (dual-core) cluster went online in 2006 and managed to reach just 71 teraflops. In the second phase, which went online in 2007, some older Cell architecture multicore processors were introduced in a hybrid design. Finally, in the third phase, thousands of Opterons and 12,960 PowerXCell processors were combined and the Roadrunner was able to break the 1-petaflop barrier in summer 2008.

Roadrunner design and architecture

Unlike previous supercomputers, Roadrunner was a hybrid supercomputer, meaning it featured a hybrid 3D design with integrated 3D memory, networking, processors and other technologies. Also, it used two types of processor architectures: IBM PowerXCell 8i chip, an enhanced Cell Broadband Engine chip running at 3.2 GHz, and x86 AMD Opteron 2210 dual-core processors running at 1.8 GHz.

Its PowerXCell processors were used as accelerators, also known as turbochargers, for some calculations. Due to its hybrid format, the Roadrunner consumed 2.35 megawatts of power. It also delivered 437 million calculations per watt.

Roadrunner's PowerXCell and Opteron processors were placed in specially designed IBM blade servers connected by InfiniBand, a networking communications standard typically used in high-performance computing (HPC). The supercomputer was considered an Opteron cluster with Cell accelerators, with each of its nodes consisting of a Cell attached to an Opteron core and the Opteron processors attached to each other. The Opterons were connected by HyperTransport links.

Roadrunner, which occupied a footprint of approximately 12,000 square feet, used the Red Hat Enterprise Linux (RHEL) and Fedora operating systems. All its software was written specially for its hybrid architecture. It was managed with XCAT distributed computing software and used Open MPI, an open source message passing interface implementation that is developed and maintained by numerous research, academic and industry partners.

Like other modern supercomputers, Roadrunner used cluster computing to achieve high levels of performance. This approach is both cheaper and more extensible than the supercomputers of the early 90s, as commodity parts are used to achieve the final machine.

Breaking the petaflop barrier

Roadrunner was built to full scale and broke the 1 petaflop barrier in 2008 at IBM's Poughkeepsie, N.Y., facility. In testing, the system had demonstrated 1.026 quadrillion calculations per second, as recorded by the Linpack measurement system. To illustrate its speed in everyday terms, IBM said that it would take 100,000 high-performance laptop computers to equal Roadrunner's processing power. In another comparison, the company said it would take all six billion people on the planet (world population at the time), working on handheld computers (24 hours a day, seven days a week) 46 years to equal what Roadrunner can do in a single day.

Crossing the petaflop barrier has long been considered as a crucial milestone for HPC. And Roadrunner was the first supercomputer to ever achieve this feat. In May 2008, Roadrunner achieved 1.026 petaflops. Then in November of the same year, it achieved a top performance of 1.456 petaflops, retaining its top spot in the TOP500 list of supercomputers. It also achieved a spot on the Supermicro Green500 list as the world's fourth-most energy-efficient supercomputer at the time.

In addition to being the first to achieve the petaflop milestone, Roadrunner was also notable for being the world's first hybrid supercomputer. Although the hybrid design was contentious at the time, it has since emerged as the preferred design for supercomputers, with numerous such machines now using this design paradigm.

Decommissioning of Roadrunner

In 2013, the LANL decommissioned Roadrunner due to its low energy efficiency compared to newer supercomputers. Roadrunner required 2,345 kilowatts to hit 1.042 petaflops, much more energy than comparable supercomputers of the time. It produced 444 megaflops of computer power per watt of energy consumed, whereas newer supercomputers delivered upward of 888 megaflops of computing power per watt. In other words, other supercomputers consumed less than half the power to deliver the same performance as Roadrunner.

A few years prior, a new supercomputer had been commissioned to replace Roadrunner. Built by Cray, the supercomputer, called Cielo, was smaller and more energy-efficient than Roadrunner, and cost less than half the Roadrunner's cost to develop and deploy (approx. $54 million). Cray was acquired by Hewlett Packard Enterprise (HPE) in 2019.

Although it was decommissioned, Roadrunner remains one of IBM's most famous innovations because it achieved many firsts. After its decommissioning, researchers and companies like IBM turned their attention to exascale computing, that is, building supercomputers that would be capable of exaflop-level performance (1,000 petaflops) or more. These machines are expected to be able to deliver a thousandfold performance increase by using new technologies like light pulses for chip communication, nanoscale carbon tubes and the integration of optical devices and functions directly onto silicon chips. Those innovations could result in smaller, faster and more power-efficient chips and a new generation of HPC that conventional technologies cannot deliver.

See top considerations for HPC infrastructure in the data center.