Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Computing and CUDA Architecture

• Architectural distinctions between CPU and GPU processing units
• NVIDIA GPU streaming multiprocessor (SM) framework
• Overview of the CUDA programming paradigm
• Heterogeneous computing systems and host-device interaction models

Configuring the CUDA Development Environment

• Installation procedures for CUDA Toolkit 13.x
• NVCC compiler integration and build process management
• Environment validation through device capability queries
• Integration with Integrated Development Environments (IDEs) and associated tools

Development and Execution of CUDA Kernels

• Syntax specifications and qualifier usage for kernel functions
• Kernel launch parameters and execution flow
• Implementation of vector addition and fundamental data-parallel algorithms
• Implementation of CUDA error handling macros

CUDA Thread Hierarchical Structure and Execution Logic

• Organization of grids, blocks, and individual threads
• Thread indexing strategies and global identifier computation
• Warp-level execution mechanics and the SIMT (Single Instruction, Multiple Threads) model
• Analysis of occupancy and resource allocation efficiency

GPU Memory Architecture and Management Strategies

• Memory hierarchy components: global, shared, constant, and register memory
• Procedures for device memory allocation and deallocation
• Data transfer protocols between host and device systems
• Utilization of shared memory for intra-block coordination

Unified Memory Models and Data Migration

• Unified memory architecture and managed allocation techniques
• Page migration mechanisms and on-demand paging behavior
• Implementation of asynchronous prefetching via cudaMemPrefetchAsync
• Application of memory advice hints to optimize access patterns

System-Level Profiling with Nsight Systems

• Timeline-based performance analysis using Nsight Systems
• Detection of CPU-GPU synchronization points
• Visualization of kernel execution sequences and memory operations
• Interpretation of system-wide performance metrics

Kernel Optimization using Nsight Compute

• Interactive kernel-level profiling with Nsight Compute
• Assessment of memory throughput and bandwidth utilization
• Analysis of compute intensity and warp state statistics
• Application of guided analysis and automated optimization recommendations

Concurrent Streams and Asynchronous Operation Models

• CUDA stream management and the default stream context
• Techniques for overlapping kernel execution with data transfers
• Stream synchronization mechanisms and the use of CUDA events
• Design patterns for multi-stream pipeline architectures

Error Handling and Diagnostic Tooling

• Interpretation of CUDA API error codes and recovery procedures
• Memory access validation using compute-sanitizer
• Kernel debugging capabilities via cuda-gdb
• Implementation of assertions and synchronous error detection

Profile-Driven Optimization Methodology

• Structured approach to iterative profiling
• Identification and prioritization of performance bottlenecks
• Execution of performance regression tests
• Documentation of optimization decisions and outcomes

End-to-End Accelerated Application Development

• Architectural design of a comprehensive GPU-accelerated solution
• Integration of profiling tools throughout the development lifecycle
• Execution of performance benchmarks and reporting standards
• Deployment considerations for production-grade environments