High Performance Computing For MU Semester 8 Computer Engineering Course Code :- (Code :CSDC8022) Academic Year 2022-2023  (Paperback, Dr. Pankaj Agarkar, Prof. Yogesh Mali)

1 Introduction to Parallel Computing 5 1.1 1.2 1.3 1.4 Parallelism (What, Why, Applications), Levels of parallelism (instruction, transaction, task, thread, memory, function). Classification Models : Architectural Schemes (Flynn’s, Shore’s, Feng’s, Handler’s). Memory Access : Distributed Memory, Shared Memory, Hybrid Distributed Shared Memory. Parallel Architecture : Pipeline Architecture : Arithmetic pipelines, Floating Point, Array Processor. (Refer Chapter 1) 2 Parallel Programming Platform and Algorithm Design 11 2.1 2.2 Parallel Programming Platform : Physical Organization of Parallel Platforms, Communication Costs in Parallel Machines. Algorithm Design : Preliminaries, Decomposition Techniques, Characteristics of Tasks and Interactions, Mapping Techniques for Load Balancing, Methods for Containing Interaction Overheads, Parallel Algorithm Models. (Refer Chapter 2) 3 Performance Measures 3 Performance Measures : Speedup, execution time, efficiency, cost, scalability, Effect of granularity on performance, Scalability of Parallel Systems, Amdahl’s Law, Gustavson’s Law, Performance Bottlenecks, The Karp Flatt Metric. (Refer Chapter 3) 4 HPC Programming : OpenMP and MPI 10 4.1 4.2 4.3 4.4 4.5 4.6 4.7 HPC Programming : OpenMP Introduction : Threads, Share memory Architecture, Multi-core processors and Hyperthreading, Fork and join model. OpenMP directives : #pragma omp parallel, Hello world with openMP, #pragma omp for, #pragma omp for schedule, Serial vs Parallel PI program. Synchronisation : Introduction, Private vs Shared variables. Critical section, #pragma omp critical, #pragma omp atomic, #pragma omp barrier, #pragma omp reduction. HPC Programming : MPI Introduction : Processes, Multiprocessor programming model, Distributed system programming model, Inter-process communication using message passing : Asynchronous and Synchronous. MPI Programming : Hello world problem, mpi_initMPI_sendMPI_Recv, Synchronisation : MPI_Barrier Hybrid (MPI + OpenMP) programming, Hardware requirement, Threads inside Processes, Hybrid Matrix multiplication Message passing vs Share memory communication : Advantages and disadvantage. (Refer Chapter 4) 5 Parallel Programming using Accelerators 4 An Overview of GPGPUs, Introduction to CUDA, Introduction to Heterogeneous Computing using OpenCL, An Overview of OpenCL API, Heterogeneous Programming in OpenCL. (Refer Chapter 5) 6 High Performance Computing in the Cloud 4 Virtualization and Containerization, Parallel Computing Frameworks, Scaling, HPC in the Cloud Use Cases. (Refer Chapter 6)