Parallel & Distributed Computing  (Paperback, I. A. Dhotre)

1. Introduction to Parallel Computing : The Idea of Parallelism, Power and potential of parallelism, Examining sequential and parallel programs, Scope and issues of parallel and distributed computing, Goals of parallelism, Parallelism and concurrency using multiple instructions streams. (Chapter - 1) 2. Parallel Architecture : Pipeline architecture, Array processor, Multi processor architecture, Systolic architecture, Dataflow architecture, Architectural classification schemes, Memory access classification, Memory Issues : Shared vs. distributed, Symmetric multiprocessing (SMP), SIMD, Vector processing, GPU co-processing, Flynn’s Taxonomy, Instruction Level support for parallel programming, Multiprocessor caches and Cache Coherence, Non-Uniform Memory Access (NUMA). (Chapter - 2) 3. Parallel Algorithm Design Principles and Programming : Need for communication and coordination/synchronization, Scheduling and contention, Independence and partitioning, Task- Based Decomposition, Data Parallel Decomposition, Characteristics of task and interaction, Load balancing, Data Management, parallel algorithm models, Sources of overhead in parallel programs, Performance metrics for parallel algorithm implementations, Parallel algorithmic patterns like divide and conquer, Map and Reduce, Specific algorithms like parallel Merge Sort, Parallel graph Algorithms. (Chapter - 3) 4. Introduction to Distributed Systems : Goals of the Distributed Systems, Relation to parallel systems, synchronous versus asynchronous execution, design issues and challenges, Types of Distributed Systems, Distributed System Models, Hardware and software concepts related to distributed systems, middleware models. (Chapter - 4) 5. Distributed Computing and Communication design principles : A Model of distributed executions, Models of communication networks, Global state of distributed system, Models of process communication. Communication and Coordination: Shared Memory, Consistency, Atomicity, Message - Passing, Consensus, Conditional Actions, Critical Paths, Scalability, and cache coherence in multiprocessor systems, synchronization mechanism. (Chapter - 5) 6. Parallel and Distributed Programming Frameworks : Overview of CUDA, OpenMP, POSIX Threads, Apache Hadoop (DFS), and current trends in parallel and distributed computing. (Chapter - 6)