Introduction to High Performance Computing

Introduction to High Performance Scientific Computing

by David Chopp

This book is available for purchase in the SIAM Bookstore.

This book introduces methods for adding parallelism to numerical methods for solving differential equations. It contains exercises and programming projects that facilitate learning as well as examples and discussions based on the C programming language, with additional comments for those already familiar with C++.

The text provides an overview of concepts and algorithmic techniques for modern scientific computing and is divided into six self-contained parts that can be assembled in any order to create an introductory course using available computer hardware.

Part I introduces the C programming language for those not already familiar with programming in a compiled language.
Part II describes parallelism on shared memory architectures using OpenMP.
Part III details parallelism on computer clusters using MPI for coordinating a computation.
Part IV demonstrates the use of graphical programming units (GPUs) to solve problems using the CUDA language for NVIDIA graphics cards.
Part V addresses programming on GPUs for non-NVIDIA graphics cards using the OpenCL framework.
Part VI contains a brief discussion of numerical methods and applications, giving the reader an opportunity to test the methods on typical computing problems.

Introduction to High Performance Scientific Computing is intended for advanced undergraduate or beginning graduate students who have limited exposure to programming or parallel programming concepts. Extensive knowledge of numerical methods is not assumed. The material can be adapted to the available computational hardware, from OpenMP on simple laptops or desktops to MPI on computer clusters or CUDA and OpenCL for computers containing NVIDIA or other graphics cards. Experienced programmers unfamiliar with parallel programming will benefit from comparing the various methods to determine the type of parallel programming best suited for their application.

The book can be used for courses on parallel scientific computing, high performance computing, and numerical methods for parallel computing.

The examples in the textbook can be downloaded using the links below. A list of errata is here.

Book Section	Example Codes in C	Example Codes in C++
Part I - C Programming	All C Examples	All C++ Examples
	Example2.1.c	Example2.1.cpp
	Example3.2.c	Example3.2.cpp
	Example3.3.c	Example3.3.cpp
	Example3.5.c	Example3.5.cpp
	Example3.6.c	Example3.6.cpp
	Example4.1.c	Example4.1.cpp
	Example4.2.c	Example4.2.cpp
	Example5.1.c	Example5.1.cpp
	Example6.1.c	Example6.1.cpp
	Example6.6.c	Example6.6.cpp
	Example7.1.c	Example7.1.cpp
	Example7.2.c	Example7.2.cpp
	Example7.3.c	Example7.3.cpp
	Example7.4.c	Example7.4.cpp
	Example7.5.c	Example7.5.cpp
Part II - OpenMP	All C Examples	All C++ Examples
	Example9.1.c	Example9.1.cpp
	Example9.2.c	Example9.2.cpp
	Example10.1.c	Example10.1.cpp
	Example10.2.c	Example10.2.cpp
	Example10.3.c	Example10.3.cpp
	Example10.4.c	Example10.4.cpp
	Example10.5.c	Example10.5.cpp
	Example11.1.c	Example11.1.cpp
	Example11.2.c	Example11.2.cpp
	Example11.3.c	Example11.3.cpp
	Example12.1.c	Example12.1.cpp
	Example13.1.c	Example13.1.cpp
	Example13.2.c	Example13.2.cpp
	Example14.1.c	Example14.1.cpp
	Example14.2.c	Example14.2.cpp
	Example14.3.c	Example14.3.cpp
	Example14.4.c	Example14.4.cpp
Part III - MPI	All C Examples	All C++ Examples
	Example16.1.c	Example16.1.cpp
	Example17.1.c	Example17.1.cpp
	Example17.2.c	Example17.2.cpp
	Example17.3.c	Example17.3.cpp
	Example17.4.c	Example17.4.cpp
	Example18.1.c	Example18.1.cpp
	Example18.2.c	Example18.2.cpp
	Example18.3.c	Example18.3.cpp
	Example20.1.c	Example20.1.cpp
	Example20.2.c	Example20.2.cpp
Part IV - CUDA	All C Examples	All C++ Examples
	checkerror.h	checkerror.h
	Example22.1.cu	Example22.1.cu
	Example22.2.cu	Example22.2.cu
	Example23.1.cu	Example23.1.cu
	Example23.2.cu	Example23.2.cu
	Example23.3.h	Example23.3.h
	Example24.2.cu	Example24.2.cu
	Example24.3.cu	Example24.3.cu
	Example24.4.cu	Example24.4.cu
	Example24.5.cu	Example24.5.cu
	Example24.6.cu	Example24.6.cu
	Example24.7.cu	Example24.7.cu
	Example24.8.cu	Example24.8.cu
	Example25.1.cu	Example25.1.cu
	Example25.2.cu	Example25.2.cu
	Example25.3.cu	Example25.3.cu
	Example26.1.cu	Example26.1.cu
	Example26.2.cu	Example26.2.cu
	Example26.3.cu	Example26.3.cu
	Example26.4.cu	Example26.4.cu
	Example26.5.cu	Example26.5.cu
	Example26.6.cu	Example26.6.cu
	Example26.7.cu	Example26.7.cu
Part V - OpenCL	All C Examples	All C++ Examples
	cl_handle_error.h	cl_handle_error.h
	getkernel.h	getkernel.h
	getkernel.c	getkernel.cpp
	get_rand.cl	get_rand.cl
	Example28.1.c	Example28.1.cpp
	Example28.2.c	Example28.2.cpp
	Example29.1.c	Example29.1.cpp
	Example29.2.c	Example29.2.cpp
	Example29.2_diff.cl	Example29.2_diff.cl
	Example29.4.c	Example29.4.cpp
	Example29.4_diff.cl	Example29.4_diff.cl
	Example30.1.c	Example30.1.cpp
	Example30.1_diff.cl	Example30.1_diff.cl
	Example30.2.c	Example30.2.cpp
	Example30.2_diff.cl	Example30.2_diff.cl
	Example30.3.c	Example30.3.cpp
	Example30.3_diff.cl	Example30.3_diff.cl
	Example31.1.c	Example31.1.cpp
	Example31.1_diff.cl	Example31.1_diff.cl
	Example31.3.c	Example31.3.cpp
	Example31.3_diff.cl	Example31.3_diff.cl
	Example32.1.c	Example32.1.cpp
	Example32.2.cl	Example32.2.cl
	Example32.3.c	Example32.3.cpp
	Example32.4.c	Example32.4.cpp
	Example32.5.c	Example32.5.cpp