CS 455: Introduction to High Performance Computing
Introduction to high performance computing covering fundamental HPC architecture concepts, parallel computing systems software techniques, and hands-on experience with supercomputers.
Course Overview
This course aims to introduce students to high performance computing (HPC) in a general way that is useful to computer science students and all STEM fields. The course covers fundamental HPC architecture concepts and parallel computing systems software techniques. Students will complete several assignments investigating the use of parallel processing systems.1
Course Material
- R. Robey and Y. Zamora, Parallel and High Performance Computing, Manning Publishers, 2021
- T. Sterling, M. Anderson, and M. Brodowicz, High-Performance Computing Modern Systems and Practices, Morgan Kaufmann Publishers, 2018
Grading
- 50% Homeworks
- 30% Exams
- 20% Quizzes and Class Participation
-
Was taught as CS 494 in the Spring of 2023 and 2024. ↩
Schedule
| Week | Topic |
|---|---|
| 1 | History of High Performance Computing Course overview, what HPC is for, its societal role. |
| 2 | HPC Systems: Shared-Memory Architecture Processors, caches, memory hierarchy, threads on a single node. |
| 3 | HPC Systems: Distributed-Memory Architecture Nodes, interconnects, network topologies, message passing. |
| 4 | Benchmarking Top500 and LINPACK, measuring what a machine actually delivers. |
| 5 | Resource Management Batch schedulers, queues, job allocation on shared systems. |
| 6 | Parallel Programming: OpenMP Threads, work sharing, data races in shared memory. |
| 7 | Parallel Programming: OpenMP (continued) Loop scheduling, synchronization, scaling within a node. |
| 8 | Parallel Programming: MPI Point-to-point communication, running across many nodes. |
| 9 | Parallel Programming: MPI (continued) Collectives, communicators, hybrid MPI with OpenMP. |
| 10 | Parallel Algorithms Decomposition strategies, load balance, communication cost. |
| 11 | Performance Analysis and Tuning Profiling tools, finding bottlenecks, strong and weak scaling. |
| 12 | Debugging and Checkpointing Debugging parallel codes, fault tolerance, restart from checkpoint. |
| 13 | Visualization: Performance/System and Scientific ParaView, scientific visualization, visualizing system behavior. |
| 14 | Future Systems Exascale, heterogeneous architectures, where HPC is heading. |