High Performance Computing Systems and Applications

- Overview

High Performance Computing (HPC) most generally refers to the practice of aggregating computing power in a way that delivers much higher performance than one could get out of a typical desktop computer or workstation in order to solve large problems in science, engineering, or business. 

HPC is the use of parallel processing and supercomputers to run advanced and complex application programs. The system focuses on developing parallel processing systems by integrating both administration and computational methods. And for decades, HPC and supercomputing were intrinsically linked. 

Please refer to the following for more information:

• Wikipedia: High Performance Computing

- Advanced AI Solutions and High Performance Architecture

Artificial intelligence (AI) solutions require purpose-built architecture to enable rapid model development, training, tuning and real-time intelligent interaction. 

High Performance Architecture (HPA) is critical at every stage of the AI ​​workflow, from model development to deployment. Without the right foundation, companies will struggle to realize the full value of their investments in AI applications.

HPA strategically integrates high-performance computing workflows, AI/ML development workflows, and core IT infrastructure elements into a single architectural framework designed to meet the intense data requirements of advanced AI solutions. 

• High-Performance Computing (HPC): Training and running modern AI engines requires the right amount of combined CPU and GPU processing power. 
• High-performance storage: Training AI/ML models requires the ability to reliably store, clean, and scan large amounts of data. 
• High-performance networks: AI/ML applications require extremely high-bandwidth and low-latency network connections. 
• Automation, orchestration, software and applications: HPA requires the right mix of data science tools and infrastructure optimization. 
• Policy and governance: Data must be protected and easily accessible to systems and users. 
• Talent and skills: Building AI models and maintaining HPA infrastructure requires the right mix of experts.

- High-performance Computing (HPC) vs. Supercomputers

High-performance computing (HPC) and supercomputers are both types of computing that use advanced techniques and systems to solve complex problems. 

HPC is sometimes used as a synonym for supercomputing. However, in other contexts, "supercomputer" is used to refer to a more powerful subset of "high-performance computers". 

Here are some differences between HPC and supercomputers:

• Hardware configuration and architecture: HPC systems are typically more flexible and scalable than supercomputers.
• Hardware: Supercomputers are made up of thousands of compute nodes that work together to complete tasks. HPC systems are built using massive clusters of servers with one or more central processing units (CPUs).
• Security: HPC is more of a headache for security. With connections that may run the gamut from secure onsite servers to publicly managed cloud instances and private cloud deployments, ensuring security across the HPC continuum is inherently challenging.

HPC systems are designed to handle large amounts of data, perform intricate simulations, and deliver results at high speeds. They are often measured in terms of floating-point operations per second (FLOPS). 

Supercomputers are a class of computing machines that are built to handle tasks of unprecedented complexity. Supercomputing clusters are used by AI researchers to test model designs, hyperparameters, and datasets. They can improve AI models and advance AI capabilities. 

HPC can be used for tasks such as: Handling the complex partial differential equations used to express the physics of weather. Supercomputers can be used for tasks such as: Real-time AI applications like autonomous cars and robotics

[Lake Brienz, Switzerland]

- Why Do We  Need the HPC?

Specialized computing resources were necessary to help researchers and scientists extracts insights from massive data sets. Scientist need HPC because they hit a tipping point. At some point in research, there is a need to:

• Expand the current study area (regional → national → global)
• Integrate new data
• Increase model resolution

But … processing on my desktop or a single server no longer works. Some typical computational barriers:

• Time – processing on local systems is too slow or not feasible
• CPU Capacity -- Can only run one model at a time
• Develop, implement, and disseminate state-of-the-art techniques and tools so that models are more effectively applied to today’s decision-making
• Management of Computer Systems – Science Groups don’t want to purchase and manage local computer systems – they want to focus on science

- Exascale Computing

- Memory Centric Computing