Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Overview of the Artificial Intelligence Threat Environment
- Distinguishing factors in AI security: unpredictability, lack of transparency in decision-making, and the vulnerability of prompts as an attack vector
- Categorization of threats: attacks targeting training processes, inference stages, and supply chain integrity
- Adversary profiles for machine learning systems: identifying potential actors and their motivations
OWASP Top 10 Framework for Large Language Model Applications
- Prompt injection techniques: analyzing direct and indirect vectors
- Risks related to insecure output management and cross-plugin request forgery
- Vulnerabilities stemming from training data poisoning and supply chain weaknesses
- Threats including model denial of service, unauthorized disclosure of sensitive information, and excessive system autonomy
- Practical laboratory exercise: demonstrating vulnerabilities within each OWASP category using a test application for government contexts
Red Teaming for Prompt Injection and Jailbreak Techniques
- Classification of injection methods: direct, indirect, multi-turn, and multi-modal approaches
- Automated red-team operations utilizing Giskard, Garak, and custom fuzzing utilities
- Classification of jailbreak attempts and evaluation of defensive measures
- Development of a red-team framework for continuous security assessment of LLMs
Threats and Defenses at the Model Level
- Model extraction: acquiring model weights and functionality through API queries
- Membership inference attacks: determining whether specific data was included in training sets
- Adversarial examples: inputs designed to mislead classifiers and embeddings
- Data poisoning: manipulating training data to create backdoors or reduce system performance
Security Controls for Input and Output Data
- Input sanitization strategies that extend beyond traditional web application defenses
- Output filtering mechanisms to prevent toxicity, personal identifiable information (PII) leakage, and execution of hallucinated code
- Implementation of guardrails as critical security infrastructure: utilizing NeMo, Guardrails AI, and custom policies
- Enforcement of structured outputs as a definitive security boundary
Security within the AI Supply Chain
- Establishing model provenance to verify authenticity and integrity
- Scanning dependencies within machine learning frameworks and model formats
- Securing model serving environments through sandboxing, network isolation, and least-privilege access controls
- Vetting fine-tuned and community-contributed models for embedded malware
Operational Security for Artificial Intelligence Systems
- Access management for model endpoints, vector databases, and agent tooling
- Comprehensive audit logging of all model interactions and decisions
- Incident response procedures specific to AI breaches, including scenarios where the model itself is compromised
- Integration of continuous security testing into CI/CD pipelines for machine learning workflows
Establishment of an AI Security Program
- Development of a maturity model and strategic roadmap for AI security capabilities
- Integration of AI security protocols into existing application security (AppSec) and cloud security frameworks
- Alignment with governance standards and emerging regulatory requirements for AI systems
- Creation and maintenance of an organizational playbook for AI security, tailored for government use
Requirements
- Proven track record of deploying machine learning models or large language model applications within operational environments.
- Demonstrated understanding of security frameworks, encompassing authentication protocols, authorization controls, and threat modeling methodologies.
- Advanced proficiency in Python programming for the execution of adversarial testing exercises.
Target Participants
- Security professionals transitioning into the AI/ML domain to address emerging threat vectors.
- MLOps practitioners accountable for ensuring model safety and operational robustness.
- Red team specialists incorporating artificial intelligence systems into their assessment scopes.
14 Hours
