Course Outline

Introduction to Open-Source Large Language Models (LLMs)

  • Overview of open-weight models and their significance for government applications
  • Detailed review of LLaMA, Mistral, Qwen, and other community-developed models
  • Exploration of use cases for private, on-premise, or secure deployments in the public sector

Environment Setup and Tools for Government Use

  • Installation and configuration of Transformers, Datasets, and PEFT libraries for government systems
  • Selection of appropriate hardware for fine-tuning large language models in a government setting
  • Procedures for loading pre-trained models from Hugging Face or other trusted repositories

Data Preparation and Preprocessing for Government Applications

  • Understanding dataset formats, including instruction tuning, chat data, and text-only datasets
  • Tokenization techniques and sequence management for government-specific data
  • Steps to create custom datasets and data loaders tailored to public sector needs

Fine-Tuning Techniques for Government Use

  • Comparison of standard full fine-tuning versus parameter-efficient methods in a government context
  • Application of LoRA and QLoRA for efficient fine-tuning in secure environments
  • Utilization of the Trainer API to facilitate rapid experimentation for government projects

Model Evaluation and Optimization for Government Use

  • Methods for assessing fine-tuned models using generation and accuracy metrics relevant to public sector operations
  • Strategies for managing overfitting, ensuring generalization, and utilizing validation sets in government applications
  • Tips for performance tuning and logging practices specific to government use cases

Deployment and Private Use for Government Operations

  • Procedures for saving and loading models for inference in secure government systems
  • Best practices for deploying fine-tuned models in secure enterprise environments within the public sector
  • Evaluation of on-premise versus cloud deployment strategies for government applications

Case Studies and Use Cases for Government

  • Examples of how LLaMA, Mistral, and Qwen are utilized in enterprise settings, including government agencies
  • Approaches to handling multilingual and domain-specific fine-tuning for public sector needs
  • Discussion on the trade-offs between open-source and closed models in a government context

Summary and Next Steps for Government Applications

Requirements

  • An understanding of large language models (LLMs) and their architecture for government applications
  • Experience with Python and PyTorch
  • Basic familiarity with the Hugging Face ecosystem

Audience

  • Machine learning practitioners
  • Artificial intelligence developers
 14 Hours

Number of participants


Price per participant

Upcoming Courses

Related Categories