Get in Touch
 Duration 14 hours

Course Outline

Introduction to Open-Source Large Language Models

  • Definition of open-weight models and their strategic significance
  • Survey of LLaMA, Mistral, Qwen, and other community-driven architectures
  • Applications for private, on-premise, or secure institutional deployments

Environment Configuration and Tooling

  • Installation and setup of Transformers, Datasets, and PEFT libraries
  • Selection of appropriate hardware resources for fine-tuning operations
  • Acquisition of pre-trained models from Hugging Face or equivalent repositories

Data Curation and Preprocessing

  • Dataset structures (instruction tuning, conversational data, text-only)
  • Tokenization strategies and sequence management protocols
  • Development of custom datasets and data loading pipelines

Fine-Tuning Methodologies

  • Comparison of standard full fine-tuning versus parameter-efficient methods
  • Implementation of LoRA and QLoRA for resource-efficient optimization
  • Utilization of the Trainer API for rapid experimentation cycles

Model Assessment and Optimization

  • Verification of fine-tuned models using generation quality and accuracy metrics
  • Mitigation of overfitting, generalization checks, and validation set management
  • Performance tuning techniques and comprehensive logging practices

Deployment and Secure Internal Utilization

  • Procedures for saving and loading models for inference tasks
  • Integration of fine-tuned models into secure enterprise environments
  • Strategic comparison of on-premise versus cloud-based deployment models

Case Studies and Operational Use Cases

  • Instances of enterprise adoption of LLaMA, Mistral, and Qwen
  • Management of multilingual and domain-specific fine-tuning requirements
  • Analysis: Strategic trade-offs between open-source and proprietary models

Conclusion and Future Directions

Requirements

  • Proficiency in large language models (LLMs) and their underlying architecture
  • Practical experience with Python and PyTorch frameworks
  • Fundamental familiarity with the Hugging Face ecosystem

Intended Audience

  • ML practitioners
  • AI developers

Number of participants


Price per participant

Upcoming Courses

Related Categories