As heterogeneity increases in future HPC systems, so does the risk of underutilizing expensive hardware resources if we preserve today’s rigid node configuration and reservation strategies. This has sparked interest in resource disaggregation to enable finer-grain allocation of hardware resources to applications. However, there is currently no data-driven study of what range of disaggregation is appropriate in HPC. To that end, we perform a detailed analysis of key metrics sampled in NERSC’s Cori, a production HPC system that executes a diverse open-science HPC workload. In addition, we profile a variety of deep-learning applications to represent an emerging workload. We show that for a rack (cabinet) configuration and applications similar to Cori, a CPU with intra-rack disaggregation has a 99.5% probability to find all resources it requires inside its rack. In addition, ideal intra-rack resource disaggregation in Cori could reduce memory and NIC resources by 5.36% to 69.01% and still satisfy the worst-case average rack utilization.
George Michelogiannakis is a research scientist for the computer architecture group (CAG) in the computational research division (CRD) of Lawrence Berkeley National Laboratory. He has extensive work on networking (both off- and on-chip) and computer architecture. His latest work focuses on the post Moore's law era looking into specialization, emerging devices (transistors), superconducting digital computing, memories, photonics, and 3D integration. He is also currently working on optics and architecture for HPC and datacenter networks.
