Building AI Applications Training Course

Module 1: Overview of Contemporary AI Applications

• Course orientation and experiential learning methodology
• Evolution of AI architectures, foundation models, and large language models
• Reasoning capabilities and comparative analysis of hosted versus local AI deployment
• Economic drivers of AI adoption, specifically declining token costs
• Enterprise-level use cases, operational capabilities, and inherent constraints
• Assessment of hallucination risks and reliability considerations
• Selection criteria within the Claude model ecosystem

Instructional segment: Orientation and strategic discussion. Establishing course objectives and pedagogical approaches, while reviewing the historical development of AI systems and the current model landscape relevant to subsequent practical exercises.

Module 2: Prompt and Context Engineering Protocols

• Principle 1: Formulating precise directives utilizing delimiters, structured output formats, conditional logic, and few-shot examples
• Principle 2: Facilitating model reasoning through multi-step tasks and self-verification processes
• Iterative refinement of prompt structures
• Techniques for summarization, inference, transformation, and expansion
• Chat interface mechanics, system prompt configuration, and multi-turn dialogue management
• Mitigation strategies for model limitations and hallucination prevention

Practical Exercise 01: Comprehensive Prompt Engineering. Application of formatting guidelines, iterative improvement techniques, and core language tasks to construct a functional chatbot interface.

Module 3: Implementation of the Claude API

• Messages API architecture: request structures, response handling, and content block management
• Token consumption metrics, cost optimization, and model selection strategies
• Generation of structured data and JSON outputs
• Implementation of streaming response protocols
• Secure response handling and fundamental integration patterns

Practical Exercise 02: Development of a Claude-Integrated Assistant. Programmatic invocation of the Messages API, management of content blocks and streaming data, and secure retrieval of structured JSON output.

Module 4: Embeddings and Semantic Search Implementation

• Limitations of traditional keyword-based search methodologies
• Conceptual foundations of embeddings and semantic similarity
• Dense retrieval techniques and approximate nearest neighbor algorithms
• Optimization of retrieved result sets through reranking
• Synthesis of contextually grounded responses from retrieved information

Practical Exercise 03: Semantic Search and Retrieval Logic. Comparative analysis of keyword versus embedding-based retrieval methods, followed by result optimization and generation of grounded responses.

Module 5: Retrieval-Augmented Generation with LangChain

• Document ingestion from PDFs, web sources, and other formats, including chunking strategies
• Embedding processes and integration with vector stores
• Retrieval methodologies: similarity scoring and Maximum Marginal Relevance
• Construction of question-answering chains for proprietary documents
• Conversational retrieval systems incorporating memory mechanisms
• Overview of vector database solutions: Chroma, pgvector, OpenSearch, Pinecone, and Weaviate

Practical Exercise 04: RAG Implementation for Proprietary Documents. Document processing, embedding into vector stores, retrieval using similarity and MMR, and construction of a question-answering chain with conversational memory.

Module 6: LangChain Core Components

• Integration of models, prompts, and output parsers
• Memory management types: buffer, summary, and token-based approaches
• Chain architectures: sequential and router-based structures
• Question-answering systems utilizing product catalog datasets
• LLM-based evaluation frameworks for output quality

Practical Exercise 05: LangChain Component Integration. Application of models, prompts, parsers, memory, and chain structures to product catalog data, incorporating LLM-based performance evaluation.

Module 7: Function Calling, Tools, and Agent Design

• Implementation of tool use and function calling with Claude
• Conversion of standard Python functions into model-accessible tools
• Structured data extraction and tagging using tool interfaces
• Tool routing mechanisms and agent executor patterns
• Development of conversational agents with persistent memory

Practical Exercise 06: Advanced Tool and Function Integration. Transformation of Python functions into model tools, implementation of structured extraction and routing, and construction of a memory-equipped conversational agent.

Module 8: Architecting Agentic AI Systems

• Defining agentic capabilities: distinctions between assistants and agents, and levels of autonomy
• Implementation of planning, reasoning, and reflection loops
• Self-correcting code generation workflows
• Tool-utilizing agents for weather data, file operations, and research tasks
• Coordination of multi-agent workflows
• Human-in-the-loop protocols and agent performance evaluation

Practical Exercise 07: Construction of Agentic Workflows. Integration of planning, reflection, and self-correction loops, culminating in a multi-agent workflow with human oversight checkpoints.

Module 9: AI Agents for Structured Data Interaction

• Natural language querying of CSV data utilizing pandas agents
• SQL agent implementation for relational database interaction
• Function calling protocols for database operations
• Security measures: injection prevention, role-based access control, and read-only enforcement

Practical Exercise 08: Natural Language Database Agent. Querying CSV data via pandas agents and relational databases via SQL agents, with implementation of read-only access and role-based security controls.

Module 10: Serverless LLM Applications on AWS

• Integration of Amazon Bedrock and Claude services
• Text generation using foundation models via boto3
• Development of audio transcription and summarization pipelines using Claude
• Implementation of CloudWatch logging and observability standards
• Deployment of processing pipelines as AWS Lambda functions
• Event-driven architectural patterns triggered by data uploads

Practical Exercise 09: Serverless AI Deployment on AWS Lambda. Execution of audio transcription, summarization via Bedrock, deployment as Lambda functions, and configuration of CloudWatch logging.

Module 11: Agent Configuration and Guardrails on Bedrock

• Provisioning of Bedrock agents
• System integration using action groups and Lambda functions
• Enhancement of agent capabilities with calculation and code execution features
• Implementation of Bedrock Guardrails for topic and content policy enforcement
• Attachment of knowledge bases for context-grounded responses

Practical Exercise 10: Bedrock Agent with Integrated Guardrails. Creation of the agent, system integration via action groups, capability enhancement, and application of guardrails and knowledge bases.

Module 12: Multimodal and Advanced Model Capabilities (Elective)

• Multimodal reasoning: analysis of images and documents
• Long-context summarization, multi-document processing, and codebase interrogation
• Prompt optimization techniques and synthetic data generation
• Elective Practical Exercise 11: Multimodal and Long-Context Applications

Elective Practical Exercise 11: Multimodal and Long-Context Applications. Execution of image and document analysis, long-context summarization, and prompt optimization, utilizing provider-native features or compliant alternatives as required by the client environment.

Module 13: AI Application Threat Modeling

• Distinguishing AI applications from conventional software: semantic versus syntactic attack vectors
• Analysis of the OWASP Top 10 for GenAI and LLMs from a development perspective
• Identification of risk zones: prompts, retrieval, tools, outputs, and identity management
• Mapping prior system components to their respective attack surfaces

Instructional Segment: Interactive Threat Modeling. Participants conduct threat assessments of their applications against surfaces identified in prior modules, utilizing the OWASP LLM Top 10 as the evaluation framework.

Module 14: Prompt Injection Defense Mechanisms

• Classification of direct and indirect prompt injection attacks
• Detection of hidden instructions within documents, HTML, and Markdown
• Analysis of jailbreak techniques, role confusion, and context override attacks
• Implementation of layered defenses: instruction-data separation, input inspection, operator channels, and allow/deny policies

Practical Exercise 12: Chatbot Attack and Defense Simulation. Execution of attacks on a chatbot, bypassing basic filters, and implementing layered detection mechanisms using instruction-data separation and policy enforcement.

Module 15: Securing RAG Pipelines

• Assessment of the RAG attack surface: ingestion, embedding, retrieval, and generation
• Mitigation of poisoned documents and indirect injection via retrieved content
• Prevention of data exfiltration and cross-document contamination
• Implementation of defenses: ingestion sanitization, provenance scoring, metadata isolation, retrieval policies, and output validation

Practical Exercise 13: RAG Pipeline Security Hardening. Simulation of PDF poisoning and assistant compromise, followed by implementation of sanitization, provenance scoring, and retrieval scoping to neutralize threats.

Module 16: Secure Tool and Agent Execution

• Management of excessive agency and tool misuse risks
• Prevention of unsafe tool execution, including SQL and command injection via agents
• Application of least-privilege design principles and scoped credential management
• Implementation of approval gates, human-in-the-loop controls, and sandboxing
• Secrets management: exclusion of credentials from prompts, tools, and memory stores

Practical Exercise 14: Tool-Using Agent Security Hardening. Strengthening of database and tool-using agents through least-privilege enforcement, approval gates, sandboxing, and credential scoping.

Module 17: Guardrails, Validation, and Responsible AI Practices

• Output validation and enforcement of structured data formats
• Implementation of guardrails for topic, content, and safety policies
• Handling of refusals and unsafe generative outputs
• Establishment of governance frameworks, auditability, logging, and human oversight
• Cost and abuse control mechanisms, including denial-of-wallet mitigation

Practical Exercise 15: Guardrail and Output Validation Implementation. Configuration of topic and safety policies, enforcement of structured outputs, handling of refusals, and implementation of rate limits and cost budgets.

Module 18: Capstone: Comprehensive Application Hardening

• Implementation of input inspection and prompt injection detection
• Deployment of sanitized, provenance-aware retrieval pipelines
• Configuration of least-privilege, approval-gated tools
• Application of output guardrails and validation protocols
• Establishment of logging and abuse monitoring systems

Practical Exercise 16: Application Hardening Capstone. Application of input inspection, provenance-aware retrieval, least-privilege tools, output guardrails, and abuse monitoring to systems constructed during the training period.