Course Outline
Fundamentals of Artificial Intelligence Red Teaming
- Analyzing the current threat environment for AI systems
- Defining the operational role of red teams in safeguarding AI infrastructure
- Addressing ethical standards and statutory compliance requirements
Adversarial Machine Learning Dynamics
- Classifying attack vectors: evasion, data poisoning, model extraction, and inference
- Developing adversarial instances using methodologies such as FGSM and PGD
- Differentiating between targeted and untargeted interventions and defining performance metrics
Evaluating Model Resilience
- Testing system stability under data perturbation scenarios
- Identifying model limitations, blind spots, and potential failure states
- Conducting stress tests on classification, computer vision, and natural language processing models
Red Teaming AI System Architectures
- Mapping the threat surface across data ingestion, model training, and deployment phases
- Probing for vulnerabilities in model application programming interfaces and network endpoints
- Analyzing model logic and outputs through reverse engineering techniques
Simulation Environments and Technical Tools
- Leveraging the Adversarial Robustness Toolbox (ART) for testing
- Employing frameworks such as TextAttack and IBM ART for adversarial assessment
- Implementing sandboxing, continuous monitoring, and observability protocols
Strategic Planning and Cross-Functional Defense
- Formulating objectives and protocols for red team exercises
- Establishing feedback loops between offensive and defensive security units
- Embedding red teaming practices within broader AI risk management frameworks
Conclusion and Implementation Pathways
Requirements
- Proficiency in machine learning and deep learning architectural concepts
- Practical experience with Python programming and major ML frameworks (such as TensorFlow or PyTorch)
- Working knowledge of cybersecurity principles and offensive security methodologies
Intended Audience
- Security analysts and researchers
- Members of offensive security and penetration testing teams
- Professionals specializing in AI assurance and red team operations for government
Testimonials (3)
inventory and identifying the different risk exposures within AI
Gary Cook - Cybersecurity and Information Technology Risk Division
Course - Introduction to AI Trust, Risk, and Security Management (AI TRiSM)
I really enjoyed learning about AI attacks and the tools out there to begin practicing and actively using for security testing. I took a lot of knowledge away which I didn't have at the beginning and the course met what I hoped it would be. My favorite part shown from the training was Comet Browser and was amazed at what it could do. Definitely something will be looking into more. Overall it was a great course and enjoyed learning all OWASP GenAI Top 10.
Patrick Collins - Optum
Course - OWASP GenAI Security
The profesional knolage and the way how he presented it before us