AI Security for Security Teams Training Course

Module 1 — Artificial Intelligence Systems for Security Engineering

Lab: Lab 01 — 01-Introduction

Analyzing system architecture.

Core Topics:

• Differentiating Large Language Models (LLMs) from traditional applications
• Architecting AI inference pipelines
• Prompt flow management
• Retrieval-Augmented Generation (RAG) structures
• Vector databases and embeddings
• Agentic workflow design
• Tool calling mechanisms
• AI gateway infrastructure
• Copilot integration
• MCP and agent protocol standards
• Points of WAF visibility within AI architectures
• Limitations of WAF visibility in AI contexts

Key Insight: Traditional Web Application Firewalls (WAFs) frequently lack visibility into content once the prompt is processed by the model.

Module 2 — OWASP Top 10 for Generative AI

Lab: None — Interactive review and discussion

Fundamental categories of AI vulnerabilities.

Core Topics:

• Prompt Injection
• Insecure Output Handling
• Training Data Poisoning
• Model Denial of Service (DoS)
• Supply Chain Vulnerabilities
• Sensitive Information Disclosure
• Excessive Agency
• Vector and Embedding Weaknesses
• Misinformation Risks
• Unbounded Resource Consumption

Framework Integration:

• Differentiating AI-specific risks from classic OWASP guidelines
• Mapping vulnerabilities to defensive controls (WAF, gateways, application-layer)
• Evaluating the efficacy of each control mechanism
• Identifying limitations where controls fail

Module 3 — Detecting Prompt Injections

Lab: Lab 02 — 02-Prompt-Injection

Establishing baseline security for AI inputs.

Threat Vectors:

• Direct prompt injection
• Indirect prompt injection
• Covert instruction insertion
• Document-based attack vectors
• HTML and Markdown injection techniques
• Jailbreak pattern recognition
• Context override attacks
• Role confusion strategies

Detection Methodologies:

• Keyword-based heuristics
• Semantic classification models
• Prompt linting processes
• Instruction boundary enforcement
• Allow and deny policy implementation
• AI-optimized regular expressions

Practical Application:

• Executing attacks against chatbot interfaces
• Evaluating the effectiveness of naive filtering mechanisms
• Developing layered detection frameworks for government systems where applicable

Module 4 — Developing AI-Aware WAF Rules

Lab: Lab 03 — 03-WAF-Basics

Evolution of WAF rules to support AI workloads.

Core Topics:

• Securing LLM endpoints
• Inference API protection strategies
• Token-aware rate limiting
• Payload size inspection
• AI-specific signature development
• Conversation anomaly detection
• Multi-turn abuse pattern identification
• Model enumeration prevention
• Inference scraping mitigation
• Denial-of-Wallet protection measures

Implementation Examples:

• Securing /v1/chat/completions endpoints
• Defending streaming API architectures
• Preventing recursive agent call cycles

Module 5 — Securing Retrieval-Augmented Generation (RAG) Pipelines

Lab: Lab 04 — 04-RAG-Security

Mitigating critical new attack surfaces.

Threat Vectors:

• Vector database vulnerabilities
• Embedding poisoning techniques
• Malicious document ingestion (PDFs, etc.)
• Retrieval manipulation tactics
• Semantic poisoning
• Hidden instructions within documents
• Cross-document contamination
• Data exfiltration via retrieval channels

Defensive Measures:

• Ingestion sanitization protocols
• Trust scoring algorithms
• Metadata isolation techniques
• Document provenance verification
• Retrieval policy enforcement
• Data segmentation strategies

Case Study: "Upload a poisoned PDF and take over the AI assistant." This scenario illustrates the operational risk to government operations.

Module 6 — Security for Agentic AI Systems

Lab: Lab 05 — 05-Agent-Security

Mitigating high-risk autonomous behaviors.

Threat Vectors:

• Excessive agency privileges
• Tool abuse and misuse
• API chaining exploits
• Autonomous loop vulnerabilities
• Permission escalation attempts
• Memory poisoning techniques
• Indirect tool execution risks
• Agent impersonation attacks
• Credential leakage vectors
• Multi-agent coordination attacks

Defensive Measures:

• Least privilege principles for agents
• Approval gate mechanisms
• Runtime policy engine implementation
• Sandboxing technologies
• Scoped credential management
• Tool whitelisting protocols
• Human-in-the-loop oversight requirements

Strategic Importance: This area is critical for agency leadership, as the risks transition from technical vulnerabilities to direct operational and business impact.

Module 7 — API Security for Artificial Intelligence

Lab: Lab 06 — 06-Denial-of-Wallet

Securing the API-heavy nature of AI systems.

Core Topics:

• API gateway configuration for AI
• GraphQL risks specific to AI
• MCP and API abuse prevention
• JWT protection standards
• AI plugin security controls
• Agent authentication mechanisms
• Delegated authorization frameworks
• Secret management practices
• Signed prompt integrity verification
• Comprehensive AI API inventory maintenance

Framework Alignment: Integration with OWASP API Security Top 10 standards.

Module 8 — Detection Engineering and SOC Integration

Lab: Lab 07 — 07-Detection

Implementing operational defense capabilities.

Core Topics:

• AI telemetry collection
• Prompt logging standards
• Token usage analytics
• Anomaly detection techniques
• Semantic SIEM pipeline development
• Identification of AI attack indicators
• Threat hunting for LLM abuse
• Runtime observability frameworks

Detection Scenarios:

• Identifying coordinated jailbreak campaigns
• Spotting automated agent misuse
• Detecting model scraping activities

Module 9 — Cloud WAFs and AI Security Implementation

Lab: None — Interactive review and discussion

Evaluating vendor-specific implementations for government use.

Core Topics:

• AWS WAF configurations for AI APIs
• Azure WAF capabilities
• Cloudflare AI Gateway features
• General API gateway strategies
• Envoy AI filtering patterns
• Kong AI Gateway implementations
• NGINX security patterns for AI

Comparative Analysis:

• Evaluating traditional WAFs vs. AI gateways vs. application-layer guardrails
• Contrasting proxy-based filtering with semantic filtering approaches

Module 10 — Constructing a Layered AI Defense Strategy

Lab: Lab 08 — 08-Layered-Defense

Strategic conclusion on defense-in-depth.

Core Principle: No single layer, including WAFs alone, can secure an AI system. A comprehensive approach is required.

Integrated Defense Model:

1. Web Application Firewall (WAF)
2. API Gateway
3. AI Gateway
4. Application Guardrails
5. Runtime Monitoring Systems
6. Identity and Access Management
7. Sandbox Environments
8. Human Approval Workflows
9. Ongoing Observability
10. Incident Response Procedures

This model aligns with established multi-layer security frameworks.

Module and Lab Mapping

Labs are conducted in sequential order, corresponding to the module progression.

The curriculum consists of 10 modules and 8 labs; Modules 2 and 9 serve as interactive recaps and do not include laboratory exercises.

Each lab is explicitly tagged with its corresponding module throughout this outline.

• Lab 01 (Module 1)
  • Directory: 01-Introduction
  • Objective: Analyze an AI system and inspect network traffic
• Lab 02 (Module 3)
  • Directory: 02-Prompt-Injection
  • Objective: Execute attacks on chatbots and bypass naive filtering
• Lab 03 (Module 4)
  • Directory: 03-WAF-Basics
  • Objective: Develop AI-aware WAF rules
• Lab 04 (Module 5)
  • Directory: 04-RAG-Security
  • Objective: Demonstrate RAG pipeline poisoning
• Lab 05 (Module 6)
  • Directory: 05-Agent-Security
  • Objective: Secure autonomous agent operations
• Lab 06 (Module 7)
  • Directory: 06-Denial-of-Wallet
  • Objective: Detect denial-of-wallet attacks
• Lab 07 (Module 8)
  • Directory: 07-Detection
  • Objective: Monitor AI abuse patterns within logs
• Lab 08 (Module 10)
  • Directory: 08-Layered-Defense
  • Objective: Architect a layered AI defense strategy for government applications

Capstone Exercise

Participants defend a simulated enterprise AI assistant environment.

Adversarial teams attempt the following attacks:

1. Prompt injection
2. Tool abuse
3. Credential theft
4. Retrieval poisoning
5. Excessive API consumption (Denial-of-Wallet)
6. Agent privilege escalation

Defensive teams implement:

• WAF rule sets
• AI gateway policies
• Runtime detection mechanisms
• Application guardrails
• Incident response protocols