After Years of Work IBM, NCSA, Cancel "Blue Waters" Supercomputer

Four years ago almost to the day, the National Science Board handed the National science Foundation a mandate to build the most powerful petaflop-class supercomputer in the world. The NSF announced in turn that the system would be based on IBM's Power7 processor technology. One of the features that won IBM the contract was the fact that the cluster doesn't require message passing and could theoretically be programmed as a single system.

As of today, Blue Waters is officially canceled. The news comes less than a month after IBM announced it would begin commercial shipments of the individual nodes that were originally supposed to make up the building blocks of the Blue Waters system.


The MCM modules are in the left-hand "column"; RAM is stored in the central column and to the right-hand side.

The nodes themselves were incredibly dense. The Power 775 modules packed up to 3,072 Power7 processors per rack, with each core running at 3.83GHz. The system was designed to scale up to 524,288 cores, with each rack capable of handling up to 24TB of RAM and 230TB of storage. The ultradense packaging method required IBM to design a custom water-cooling system, as in the image above, wherein the stack of what look like radiator fins are individual DRAM modules. Each module integrates a copper pipe; the modules themselves are packed so tightly that a credit card can't even pass between them. Each node is 100 percent water cooled.


There's an MCM module on the left and a hub/switch on the right. The two are pin-compatible.

Each CPU MCM (multi chip module) contains four octal-core Power7 processors; each Power7 core is capable of handling four threads for a total of 128 threads per module. Each MCM supports up to 16 memory slots; an eight-MCM node like the one pictured below supports 16GB DIMMs for a total of 2TB of RAM per node. The MCMs interface incorporates 5,336 pins.


The joint statement from the NCSA and IBM states:
NCSA is confident that its goal of building a sustained-petascale supercomputer remains achievable in a timely manner. NCSA is coordinating with the National Science Foundation to ensure project continuity and that the goals of the project are achieved.

The University of Illinois and NCSA selected IBM in 2007 as the supercomputer vendor for the Blue Waters project based on projections of future technology development. The innovative technology that IBM ultimately developed was more complex and required significantly increased financial and technical support by IBM beyond its original expectations. NCSA and IBM worked closely on various proposals to retain IBM's participation in the project but could not come to a mutually agreed-on plan concerning the path forward.
The Power 775 system IBM designed for Blue Waters will go on to other projects. The server is also designated as PERCS (Productive, Easy-to-use, Reliable Computer System) and was designed to meet the goals of DARPA's High Productivity Computing Systems (HPCS) initiative. The project is tasked with creating a new generation of petaflop-scale HPC systems that enable next-generation research and analysis while keeping the US at the top of the supercomputing industry.

With that said, governments and universities don't typically snap up supercomputers the way they purchase commercial laptops. IBM has yet to announce if it has any other HPC clients lined up to purchase a Power 775 server cluster.