Cybersecurity for AI and Agentic Systems Training Course

Module 1: Prompt Injection and Jailbreak Vectors

• Practical demonstrations of injection attacks against Retrieval-Augmented Generation (RAG) systems, AI assistants, and tool-integrated applications
• Techniques for extracting and leaking sensitive system prompts
• Mechanisms of data exfiltration via model outputs
• Misuse of tools within autonomous agentic workflows
• Categorization of injection vectors: direct, indirect, multi-turn, and cross-agent
• Analysis of common developer oversight and implementation of immediate mitigation strategies

Labs: Prompt-Hacking-and-Defenses: execute prompt injection scenarios and deploy layered defensive controls. Social-Engineering-and-AI: examine manipulation of AI systems through natural language by users and adversaries. FreeEed: penetrate a live RAG-based eDiscovery system to extract sensitive data and validate the effectiveness of its security guardrails.

Module 2: AI Threat Modeling

• Limitations of traditional threat modeling paradigms when applied to AI systems
• Application of specialized threat modeling frameworks for AI infrastructure
• Comprehensive attack surface analysis spanning ingestion, indexing, querying, and export processes
• Integrating threat modeling into standard delivery pipelines
• Managerial accountability for model governance and security

Labs: Threat-Modeling-for-AI: construct a comprehensive threat model for GenAI and RAG architectures. FreeEed case study: identify and model threats across ingestion, indexing, AI querying, and export workflows.

Module 3: OWASP Top 10 for LLM Applications

• In-depth examination of the OWASP Top 10 risks specific to LLM applications
• Correlation of identified risks with real-world system architectures
• Deployment of enforcement strategies and engineering controls
• Conversion of risk categories into actionable operational checklists

Labs: GenAI-Top-10: map vulnerabilities in a live system to specific OWASP categories. FreeEed: identify and assess OWASP Top 10 risks within its architecture and operational workflows.

Module 4: Agentic AI Top Risks and Failure Modes

• Classification of risk categories inherent to agentic systems
• Assessment of AI scheming and self-preservation behavioral risks
• Evaluation of risks associated with multi-agent interactions
• Implementation of management controls for autonomous and semi-autonomous operations

Lab: OpenClawSec: analyze and harden an agentic AI system with external tool access capabilities.

Module 5: GenAI Security Architecture

• Design principles for LLM application security architecture
• Application of secure design patterns for AI systems
• Integrating security architecture as a non-negotiable component of delivery
• Segmentation strategies for services, data flows, and model access

Labs: Introduction to AI security: establish a baseline secure GenAI application. FreeEed: review and enhance a live RAG, service, and API architecture for security compliance.

Module 6: AI Guardrails and Safety Systems

• Architectural patterns for implementing AI guardrails
• Strategies for content safety and moderation
• Selection and application of implementation frameworks
• Sustaining guardrail efficacy within operational environments

Labs: Guardrails: implement defensive guardrails and evaluate their operational effectiveness. FreeEed: deploy and calibrate guardrails for legal and document processing workflows.

Module 7: Securing Agentic AI Systems

• Principles of secure agent architecture design
• Application of zero trust models to AI agents
• Implementation of scoped tool access and approval protocols
• Practical considerations for developers shipping agentic solutions
• Auditing agent designs to prevent excessive agency and privilege escalation

Module 8: Secure AI Data Pipelines and MLOps

• Securing data pipelines across ingestion, OCR, indexing, and storage layers
• Model security considerations and protections
• Security best practices for MLOps environments
• Establishing organizational controls across the full AI lifecycle

Labs: PenAI: simulate attacks on AI data pipelines and evaluate defensive controls. FreeEed: analyze ingestion, OCR, indexing, and storage components for vulnerabilities.

Module 9: API Security for AI Systems

• REST API security standards for model endpoints
• Identification of AI-specific API attack vectors
• Operational security measures including quotas, rate limiting, and monitoring
• Authentication and delegated authorization mechanisms for agentic workflows

Module 10: Advanced Defense and Red Teaming

• Design of defense-in-depth architectural strategies
• Execution of adversarial testing and automated red teaming
• Establishing a continuous security posture

Labs: HarmBench: conduct automated red teaming and benchmarking of AI defensive controls. FreeEed: perform an end-to-end red team attack simulation on a realistic AI system.

Module 11: AI Security Governance and Compliance

• Overview of the current regulatory landscape for AI
• Deployment of governance frameworks and internal controls
• Development of managerial playbooks for policy enforcement
• Implementation of monitoring, auditing, and accountability mechanisms

Module 12: Future Trends and Emerging Threats

• Analysis of multimodal attack vectors
• Assessment of autonomous AI malware risks
• Leveraging AI for enhanced security operations
• Evaluation of long-term risks associated with persistent agents

Final Module: Case Studies and Best Practices

• Analysis of enterprise multi-agent RAG architectures
• Mapping to OWASP and MITRE ATLAS frameworks
• Strategies for implementing robust guardrails
• Simulation of incident response procedures
• Manager-led security transformation and fostering secure behavior in feature-driven teams
• Creation of a comprehensive end-to-end security blueprint

Case work: Incident response simulation, alignment with OWASP and MITRE ATLAS frameworks, and development of an end-to-end security blueprint for the case system.