CFT Project Spotlight

Janus HPC Super Computer

Critical Facilities Technology receives an honorable mention at the 2012 Uptime Institute Symposium

The National Center for Atmospheric Research (NCAR) joined with the University of Colorado, Boulder, and the University of Colorado, Denver, to bring additional supercomputing resources to the Colorado Front Range area. Through this collaboration, NCAR has obtained access to approximately 10% of a new 184-teraflops Dell supercomputer system called “Janus.”

The Janus supercomputer at the University of Colorado, Boulder is one of the largest container-based supercomputers in the world with 342 Dell c6100 servers and an estimated peak performance of 184 Teraflops.

Faced with extreme time constraints, CFT collaborated on this HPC project that leveraged rapid deployment technologies to design, build, and test off-site a fully customized pre- fabricated data center. “We were extremely excited to collaborate on such a challenging project.” stated Robert Strong of Critical Facilities Technology He went on to say “We knew that we were on an especially aggressive time schedule, with a very sophisticated array of power and cooling systems to be implemented. The only way we could accomplish such a daunting task was through the co-design process. This process specifies the design of the facility in parallel to the design of the HPC resources. The primary benefit is that the complete solution can be designed and implemented in approximately the same amount of time as the design and implementation of a large-scale HPC resource alone.”

Janus is housed on the CU-Boulder campus and has a high-speed networking connection to the computing and data storage systems at the NCAR Mesa Laboratory. Allocations of a portion of this resource are now available to the university community through the proposal process outlined below.

Janus is GREEN with Holistic Design

The holistic design process enabled by the co-design of the Janus system involves examining every element available { structural, civil, electrical, mechanical, available cooling technology, local climate conditions, and IT technology

{ and not stopping at optimizing any one element, but as many as possible. In a holistic design, the engineering team has access to as many of the actual datacenter load data specifics as possible, such as kW per Rack, CFM/kW, or server design differential temperatures on a rack-by-rack basis. This knowledge allows the engineer to maximize the efficiency of the space.

PUE Below 1.2

The PUE design target of the facility was 1.2, but has consistently provided a PUE of 1.059 under load, and 1.12 while the resource has been idle (approximately 240kW). During this period, the facility has been operating in free-cooling mode, utilizing the external evaporative water tower and at-plate heat-exchanger only.

For much of this period even the cooling tower fan was operating at a minimal power draw, and the action of cycling the water through the tower was sufficient to discharge the majority of the waste heat.

The team systematically modified the facility parameters to stress individual components, allowing us to verify and match each component's power draw and load to the manufacturer's specifications.

Cherry Creek Instructional Support Facility (CCISF)

Critical Facilities Technology Designs 1,300 Sq.Ft. Data Center for Cherry Creek Instructional Support Facility

The Cherry Creek School District engaged CFT to help produce a data center design for their new 1,300 square foot facility. Working alongside Cator, Ruma & Associates and the Rimrock Group CFT was able to meet and exceed the school districts’ criteria for rack space, efficiency and modularity.

The final solution consists of 36 racks, in-row cooling architecture, a 250kW UPS, rack power distribution and data center infrastructure management (DCIM) software to monitor all aspects of the facility. All racks are fed in a 2N configuration utilizing dual 150kW PDU’s in each of the two (2) pods. The UPS can be scaled from as low as 100kW to 250kW using modular, hot-swappable, 25kW power modules. The CCISF is yet another example for how CFT can tailor fit a solution to fit any customer requirement.