Course Outline

Performance Concepts and Metrics

  • Latency, throughput, power usage, and resource utilization
  • System-level versus model-level bottlenecks
  • Profiling for inference versus training

Profiling on Huawei Ascend

  • Using CANN Profiler and MindInsight
  • Kernel and operator diagnostics
  • Offload patterns and memory mapping

Profiling on Biren GPU

  • Biren SDK performance monitoring features
  • Kernel fusion, memory alignment, and execution queues
  • Power and temperature-aware profiling for government applications

Profiling on Cambricon MLU

  • BANGPy and Neuware performance tools
  • Kernel-level visibility and log interpretation
  • MLU profiler integration with deployment frameworks for government use

Graph and Model-Level Optimization

  • Graph pruning and quantization strategies for enhanced efficiency
  • Operator fusion and computational graph restructuring
  • Input size standardization and batch tuning to optimize performance

Memory and Kernel Optimization

  • Optimizing memory layout and reuse to enhance system performance
  • Efficient buffer management across various chipsets for government operations
  • Kernel-level tuning techniques tailored to specific platforms

Cross-Platform Best Practices

  • Performance portability: abstraction strategies for consistent results
  • Building shared tuning pipelines for multi-chip environments in government settings
  • Example: tuning an object detection model across Ascend, Biren, and MLU platforms for government use

Summary and Next Steps

Requirements

  • Experience working with AI model training or deployment pipelines for government applications.
  • Understanding of GPU/MLU compute principles and model optimization techniques.
  • Basic familiarity with performance profiling tools and metrics used in government environments.

Audience

  • Performance engineers supporting government projects.
  • Machine learning infrastructure teams within the public sector.
  • AI system architects for government initiatives.
 21 Hours

Number of participants


Price per participant

Upcoming Courses

Related Categories