CMS/CAIMS Books in Mathematics15 Extreme Scale Computing

Scientific computing is essential for tackling complex problems across many domains—but how can scientists develop high-performance and high-quality software that scales efficiently? This book serves as an accessible introduction to extreme-scale computing, specifically designed for domain scientists who may not have formal computer science training but need to harness the power of C++ and parallel computing for large-scale applications. The book begins by covering the fundamentals of scientific computing software management, including essential tools like Linux, Git, and CMake, before diving into a detailed exploration of C++ for extreme-scale computing. Readers familiar with languages like Python will gain the necessary skills to transition to C++ and build scalable, efficient software. Beyond basic programming, this book delves into hardware-aware computing, teaching readers how to optimize software performance by understanding the underlying architecture of modern computational systems. It then introduces parallel computing techniques, covering MPI for distributed memory parallelism, shared memory parallelism, CUDA for GPU programming, and Kokkos for performance portability. Further chapters focus on efficient I/O, debugging, and profiling, which all address aspects of the critical challenge of performance optimization in extreme-scale computing. The book concludes with an overview of popular libraries for extreme-scale computing, equipping readers with the tools they need to solve real-world computational problems. With a balance of theory, practical applications, and illustrative case studies, this book provides domain scientists with a comprehensive roadmap to mastering extreme-scale computing and developing highly parallel and performant software.