Course Outline
Overview of Artificial Intelligence Red Teaming
- Assessment of the artificial intelligence threat environment
- Functions of red teams in securing AI systems for government
- Ethical and statutory frameworks governing operations
Adversarial Machine Learning Techniques
- Categories of vulnerabilities: evasion, data poisoning, model extraction, and membership inference
- Methodologies for constructing adversarial inputs (e.g., Fast Gradient Sign Method, Projected Gradient Descent)
- Differentiating targeted from untargeted attacks and establishing success criteria
Evaluating Model Resilience
- Measuring system robustness against input perturbations
- Identifying system limitations and failure points
- Conducting stress tests on classification, computer vision, and natural language processing models
Testing Artificial Intelligence Workflow Integrity
- Mapping attack surfaces across data ingestion, model training, and deployment phases
- Assessing vulnerabilities in insecure model application programming interfaces and network endpoints
- Analyzing model behavior through output examination and reverse engineering
Simulation Environments and Technical Tools
- Utilization of the Adversarial Robustness Toolbox (ART)
- Executing red team assessments using platforms such as TextAttack and IBM ART
- Deployment of sandboxing, monitoring, and observability solutions for secure testing
Red Team Operational Strategy and Interagency Collaboration
- Designing red team exercises and defining operational objectives
- Reporting findings to defensive (blue) teams for remediation
- Incorporating red teaming activities into comprehensive AI risk management frameworks for government entities
Executive Summary and Future Actions
Requirements
- Proficiency in the principles and structures underlying machine learning and deep learning models
- Hands-on experience utilizing Python alongside major machine learning frameworks such as TensorFlow or PyTorch
- Working knowledge of cybersecurity fundamentals, including offensive security methodologies
Audience
- Security researchers
- Offensive security teams
- AI assurance and red team professionals
Testimonials (2)
I really enjoyed learning about AI attacks and the tools out there to begin practicing and actively using for security testing. I took a lot of knowledge away which I didn't have at the beginning and the course met what I hoped it would be. My favorite part shown from the training was Comet Browser and was amazed at what it could do. Definitely something will be looking into more. Overall it was a great course and enjoyed learning all OWASP GenAI Top 10.
Patrick Collins - Optum
Course - OWASP GenAI Security
The profesional knolage and the way how he presented it before us