Course Outline
Overview of GPU-Accelerated Compute Infrastructure
- Heterogeneous computing paradigms and the integration of CPU and GPU architectures
- CUDA execution models and the management of distinct memory spaces
- Compilation workflows for CUDA C++ utilizing nvcc and CMake toolchains
- Validation protocols for establishing a robust GPU development environment
Implementing Parallel Algorithms via Thrust and CUB
- Execution of accelerated sorting, reduction, and transformation operations on GPUs
- Refactoring Standard Template Library (STL) algorithms to support GPU-based execution
- Utilizing Thrust device vectors and specifying execution policies
- Leveraging CUB device-wide primitives to construct custom computational pipelines
GPU Memory Architecture and Resource Management
- Differentiation of global, constant, and texture memory hierarchies
- Strategies for explicit device memory allocation and host-device data transfers
- Implementation of Unified Memory to streamline data accessibility
- Optimization of memory coalescing and access patterns for enhanced throughput
Managing Asynchronous Operations with CUDA Streams
- Creation and lifecycle management of CUDA stream objects
- Techniques for overlapping kernel execution with data transfer operations
- Employing CUDA events for precise dependency tracking and management
- Configuration of stream priorities and concurrency tuning for optimal resource utilization
Development of Custom CUDA Kernels
- Understanding the SIMT programming model and warp-level execution semantics
- Kernel launch configuration strategies and the implementation of grid-stride loops
- Thread indexing methodologies and the management of multidimensional grids
- Error handling protocols and comprehensive CUDA runtime API validation
Thread Hierarchy and Execution Semantics
- Structural analysis of grids, blocks, and threads within device code
- Application of warp-level primitives and ballot operations for collective logic
- Block-level synchronization mechanisms and the use of barriers
- Analysis of occupancy metrics and hardware resource utilization
Utilizing Cooperative Groups for Adaptive Parallelism
- Integration of the cooperative_groups API and management of distinct group types
- Implementation of thread-block tiles and the tiled_partition facility
- Execution of grid-level cooperative launches for synchronized operations
- Development of multi-grid synchronization patterns for complex workflows
Optimization Strategies for Shared Memory
- Analysis of shared memory bank structures and strategies to avoid bank conflicts
- Application of tiling strategies to enhance matrix operation performance
- Treating shared memory as a user-managed cache for data locality
- Use of cuda::shared_memory_mdspan for efficient multidimensional memory views
Kernel Fusion and Advanced Parallel Design Patterns
- Fusing multiple kernel operations to minimize launch overhead
- Implementation of scan, reduce-by-key, and segmented algorithmic approaches
- Utilization of atomic operations and lock-free data structures for concurrency
- Application of warp-aggregated atomics to maximize throughput
Performance Analysis and Optimization via Nsight Systems
- Timeline-based analysis of concurrent CPU and GPU activity
- Identification and mitigation of memory transfer bottlenecks
- Profiling of kernel performance metrics and occupancy levels
- Iterative performance refinement using Nsight Compute tools
Integration of Modern C++ Features in CUDA Device Code
- Employment of lambdas, constexpr evaluation, and automatic type deduction in kernels
- Implementation of C++17 parallel algorithms and standardized execution policies
- Application of C++20 concepts and ranges within the device environment
- Support for C++23 features in nvcc and the CCCL 3.x ecosystem
Advanced Asynchrony through CUDA Graphs
- Definition and deployment of CUDA graph execution models
- Capture of execution graphs directly from active stream operations
- Management of graph updates and the inclusion of conditional execution nodes
- Reduction of launch latency to improve efficiency in iterative workloads
Integration Patterns for Existing Application Stacks
- Encapsulation of GPU-specific code behind standardized C++ interfaces
- Management of multi-GPU environments and NUMA-aware system configurations
- Seamless build system integration utilizing CMake and CUDA tooling
- Debugging of device code utilizing cuda-gdb instrumentation
Summary and Operational Best Practices
- Criteria for selecting between Thrust, CUB, and custom kernel implementations
- Ensuring performance portability across diverse GPU architectures
- Code organization principles and RAII practices for CUDA resource management
- Guidance for next steps and advanced pathways in CUDA expertise
Requirements
- Foundational proficiency in C++, including proficiency with lambda expressions, templates, and the Standard Template Library (STL)
- Familiarity with standard algorithms, container structures, and iterator semantics
- Proficiency in control flow structures, conditional logic, and function encapsulation
- No prior experience in CUDA or GPU programming is required
Target Audience
- C++ developers seeking to enhance compute-intensive applications through GPU acceleration
- Software engineers transitioning from CPU-only architectures to heterogeneous parallel programming models
- Performance engineers and quantitative analysts managing large-scale datasets in public sector environments
Testimonials (3)
Initially, the trainer's pace seemed a bit too fast for me, but after offering the feedback during the training, he acknowledged it and slowed down the pace without sacrificing any of the lectures. Good rapport with the audience, very friendly and open for discussions.
Alexandru Ostafi - Siemens
Course - Advanced C++ : Practical workshop
Detailed explanation, re-iteration of points in a quite subtle way that really drove the knowledge home very well. Rod's willingness to double-check the odd obscure question that we rasied, to be sure that his answers were 100% right. Also, his interest in discussing the pros & cons of alternate coding styles so that we learnt not only how to use C++ in our intended way, but why it should be done that way.
Nick Dillon - cellxica Ltd
Course - Using C++ in Embedded Systems - Applying C++11/C++14
Experience sharing, it's teacher's know-how and valuable.
