Course Outline
1. Introduction to Deep Reinforcement Learning
- Defining Reinforcement Learning principles
- Distinguishing between Supervised, Unsupervised, and Reinforcement Learning paradigms
- 2025 Applications of DRL in public sector and industrial contexts (robotics, healthcare, finance, logistics)
- Comprehending the agent-environment interaction framework
2. Reinforcement Learning Fundamentals
- Markov Decision Processes (MDP) concepts
- State, Action, Reward, Policy, and Value function components
- The Exploration vs. Exploitation trade-off mechanism
- Monte Carlo methods and Temporal-Difference (TD) learning techniques
3. Implementing Basic RL Algorithms
- Tabular methods: Dynamic Programming, Policy Evaluation, and Iteration processes
- Q-Learning and SARSA algorithm implementations
- Epsilon-greedy exploration and decay strategies
- Configuring RL environments utilizing OpenAI Gymnasium
4. Transition to Deep Reinforcement Learning
- Limitations inherent in tabular methods
- Leveraging neural networks for function approximation
- Deep Q-Network (DQN) architecture and operational workflow
- Experience replay mechanisms and target network structures
5. Advanced DRL Algorithms
- Double DQN, Dueling DQN, and Prioritized Experience Replay enhancements
- Policy Gradient Methods: REINFORCE algorithm application
- Actor-Critic architectures (A2C, A3C) design
- Proximal Policy Optimization (PPO) implementation
- Soft Actor-Critic (SAC) integration
6. Working with Continuous Action Spaces
- Challenges associated with continuous control systems
- Utilizing DDPG (Deep Deterministic Policy Gradient) models
- Twin Delayed DDPG (TD3) application
7. Practical Tools and Frameworks
- Utilizing Stable-Baselines3 and Ray RLlib frameworks
- Logging and monitoring procedures via TensorBoard
- Hyperparameter tuning strategies for DRL models
8. Reward Engineering and Environment Design
- Reward shaping and penalty balancing techniques
- Sim-to-real transfer learning concepts
- Custom environment creation within Gymnasium
9. Partially Observable Environments and Generalization
- Handling incomplete state information (POMDPs)
- Memory-based approaches utilizing LSTMs and RNNs
- Enhancing agent robustness and generalization capabilities
10. Game Theory and Multi-Agent Reinforcement Learning
- Introduction to multi-agent operational environments
- Dynamics of Cooperation vs. Competition
- Applications in adversarial training and strategy optimization
11. Case Studies and Real-World Applications
- Autonomous driving simulation models
- Dynamic pricing and financial trading strategy analysis
- Robotics and industrial automation integration
12. Troubleshooting and Optimization
- Diagnosing unstable training processes
- Managing reward sparsity and overfitting risks
- Scaling DRL models on GPUs and distributed systems
13. Summary and Next Steps
- Recap of DRL architecture and key algorithmic components
- Industry trends and research directions (e.g., RLHF, hybrid models)
- Further resources and reference materials
Requirements
- Proficiency in Python programming
- Understanding of Calculus and Linear Algebra concepts
- Basic knowledge of Probability and Statistics
- Experience building machine learning models using Python and NumPy or TensorFlow/PyTorch
Intended Audience
- Developers focused on AI and intelligent systems
- Data Scientists exploring reinforcement learning frameworks
- Machine Learning Engineers working with autonomous systems
Testimonials (2)
Getting people that never used AI some repetition in prompting and people that do use AI to consider different methods to using it.
Matthew Gay - Tarsus Pharmaceuticals
Course - Artificial Intelligence (AI) Overview
Working from first principles in a focused way, and moving to applying case studies within the same day
