Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.) Training Course
Fine-tuning open-source language models (LLMs) is an emerging best practice for government organizations that seek to customize AI capabilities in secure, cost-efficient, and private environments.
This instructor-led, live training (online or onsite) is aimed at intermediate-level machine learning practitioners and AI developers who wish to fine-tune and deploy open-weight models like LLaMA, Mistral, and Qwen for specific government applications.
By the end of this training, participants will be able to:
- Understand the ecosystem and differences between open-source LLMs.
- Prepare datasets and fine-tuning configurations for models like LLaMA, Mistral, and Qwen.
- Execute fine-tuning pipelines using Hugging Face Transformers and PEFT.
- Evaluate, save, and deploy fine-tuned models in secure environments for government use.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Course Outline
Introduction to Open-Source Language Models for Government
- What are open-weight models and why they are important for government applications
- Overview of LLaMA, Mistral, Qwen, and other community-developed models suitable for government use
- Use cases for private, on-premise, or secure deployments in the public sector
Environment Setup and Tools
- Installing and configuring Transformers, Datasets, and PEFT libraries for government systems
- Choosing appropriate hardware for fine-tuning models to meet governmental performance requirements
- Loading pre-trained models from Hugging Face or other secure repositories approved for government use
Data Preparation and Preprocessing
- Dataset formats suitable for instruction tuning, chat data, and text-only applications in government contexts
- Tokenization and sequence management techniques for public sector datasets
- Creating custom datasets and data loaders tailored to governmental needs
Fine-Tuning Techniques
- Comparing standard full fine-tuning with parameter-efficient methods for optimizing government models
- Applying LoRA and QLoRA for efficient fine-tuning in resource-constrained government environments
- Utilizing the Trainer API to facilitate rapid experimentation and iteration in government projects
Model Evaluation and Optimization
- Assessing fine-tuned models using generation and accuracy metrics relevant to governmental tasks
- Managing overfitting, ensuring generalization, and validating model performance in government settings
- Performance tuning strategies and logging practices for maintaining accountability and transparency in government deployments
Deployment and Private Use
- Saving and loading models for inference in secure governmental systems
- Deploying fine-tuned models in secure enterprise environments within the public sector
- Evaluating on-premise versus cloud deployment strategies to meet government security and compliance standards
Case Studies and Use Cases
- Examples of how LLaMA, Mistral, and Qwen are utilized in government enterprises
- Handling multilingual and domain-specific fine-tuning for governmental applications
- Discussion on the trade-offs between open and closed models in the context of government operations
Summary and Next Steps
Requirements
- An understanding of large language models (LLMs) and their architecture for government applications
- Experience with Python and PyTorch for government projects
- Basic familiarity with the Hugging Face ecosystem for government use
Audience
- Machine Learning practitioners in the public sector
- Artificial Intelligence developers working for government agencies
Runs with a minimum of 4 + people. For 1-to-1 or private group training, request a quote.
Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.) Training Course - Booking
Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.) Training Course - Enquiry
Fine-Tuning Open-Source LLMs (LLaMA, Mistral, Qwen, etc.) - Consultancy Enquiry
Consultancy Enquiry
Upcoming Courses
Related Courses
Advanced LangGraph: Optimization, Debugging, and Monitoring Complex Graphs
35 HoursLangGraph is a framework designed for constructing stateful, multi-actor LLM applications using composable graphs with persistent state and controlled execution.
This instructor-led, live training (available online or onsite) is targeted at advanced-level AI platform engineers, DevOps professionals for AI, and ML architects who aim to optimize, debug, monitor, and operate production-grade LangGraph systems for government use.
By the end of this training, participants will be able to:
- Design and optimize complex LangGraph topologies for enhanced speed, cost efficiency, and scalability.
- Implement reliability features such as retries, timeouts, idempotency, and checkpoint-based recovery mechanisms.
- Debug and trace graph executions, inspect state data, and systematically reproduce production issues.
- Instrument graphs with logs, metrics, and traces, deploy to production environments, and monitor service level agreements (SLAs) and associated costs.
Format of the Course
- Interactive lectures and discussions.
- Extensive exercises and practical activities.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Building Coding Agents with Devstral: From Agent Design to Tooling
14 HoursDevstral is an open-source framework designed for building and running coding agents that can interact with codebases, developer tools, and APIs to enhance engineering productivity for government projects.
This instructor-led, live training (online or onsite) is aimed at intermediate-level to advanced-level ML engineers, developer-tooling teams, and SREs who wish to design, implement, and optimize coding agents using Devstral for government applications.
By the end of this training, participants will be able to:
- Set up and configure Devstral for coding agent development in alignment with public sector workflows.
- Design agentic workflows for codebase exploration and modification that adhere to governance standards.
- Integrate coding agents with developer tools and APIs to support efficient government operations.
- Implement best practices for secure and efficient agent deployment in a government context.
Format of the Course
- Interactive lecture and discussion tailored to government audiences.
- Lots of exercises and practice relevant to government projects.
- Hands-on implementation in a live-lab environment designed for government use cases.
Course Customization Options
- To request a customized training for this course, tailored specifically for government needs, please contact us to arrange.
Open-Source Model Ops: Self-Hosting, Fine-Tuning and Governance with Devstral & Mistral Models
14 HoursThe Devstral and Mistral models are open-source artificial intelligence technologies designed for flexible deployment, fine-tuning, and scalable integration in government environments.
This instructor-led, live training (online or onsite) is aimed at intermediate-level to advanced-level machine learning engineers, platform teams, and research engineers who wish to self-host, fine-tune, and govern Mistral and Devstral models for government use in production environments.
By the end of this training, participants will be able to:
- Set up and configure self-hosted environments for Mistral and Devstral models for government applications.
- Apply fine-tuning techniques to optimize performance for specific domains relevant to government operations.
- Implement versioning, monitoring, and lifecycle governance practices tailored for government workflows.
- Ensure security, compliance, and responsible usage of open-source models in alignment with public sector standards.
Format of the Course
- Interactive lectures and discussions focused on government-specific use cases.
- Hands-on exercises in self-hosting and fine-tuning for government applications.
- Live-lab implementation of governance and monitoring pipelines tailored for government environments.
Course Customization Options
- To request a customized training for this course, please contact Govtra to arrange.
LangGraph Applications in Finance
35 HoursLangGraph is a framework designed for building stateful, multi-actor LLM applications as composable graphs with persistent state and control over execution.
This instructor-led, live training (online or onsite) is aimed at intermediate to advanced-level professionals who wish to design, implement, and operate LangGraph-based finance solutions with proper governance, observability, and compliance for government.
By the end of this training, participants will be able to:
- Design finance-specific LangGraph workflows aligned with regulatory and audit requirements.
- Integrate financial data standards and ontologies into graph state and tooling.
- Implement reliability, safety, and human-in-the-loop controls for critical processes.
- Deploy, monitor, and optimize LangGraph systems for performance, cost, and SLAs.
Format of the Course
- Interactive lecture and discussion.
- Extensive exercises and practice sessions.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact Govtra to arrange.
LangGraph Foundations: Graph-Based LLM Prompting and Chaining
14 HoursLangGraph is a framework designed for building graph-structured LLM applications that support planning, branching, tool use, memory, and controllable execution for government.
This instructor-led, live training (available online or onsite) is aimed at beginner-level developers, prompt engineers, and data practitioners who wish to design and build reliable, multi-step LLM workflows using LangGraph.
By the end of this training, participants will be able to:
- Explain core LangGraph concepts (nodes, edges, state) and their appropriate use cases.
- Create prompt chains that incorporate branching, tool invocation, and memory management.
- Integrate retrieval mechanisms and external APIs into graph workflows for government applications.
- Test, debug, and evaluate LangGraph applications to ensure reliability and safety in a public sector context.
Format of the Course
- Interactive lecture with facilitated discussion.
- Guided labs and code walkthroughs in a secure sandbox environment.
- Scenario-based exercises focused on design, testing, and evaluation for government workflows.
Course Customization Options
- To request a customized training tailored to specific needs for government, please contact us to arrange.
LangGraph in Healthcare: Workflow Orchestration for Regulated Environments
35 HoursLangGraph enables stateful, multi-actor workflows powered by LLMs with precise control over execution paths and state persistence. In healthcare, these capabilities are essential for compliance, interoperability, and the development of decision-support systems that align with medical workflows for government.
This instructor-led, live training (online or onsite) is aimed at intermediate to advanced-level professionals who wish to design, implement, and manage LangGraph-based healthcare solutions while addressing regulatory, ethical, and operational challenges for government.
By the end of this training, participants will be able to:
- Design healthcare-specific LangGraph workflows with a focus on compliance and auditability.
- Integrate LangGraph applications with medical ontologies and standards (FHIR, SNOMED CT, ICD).
- Apply best practices for reliability, traceability, and explainability in sensitive environments for government.
- Deploy, monitor, and validate LangGraph applications in healthcare production settings for government.
Format of the Course
- Interactive lecture and discussion.
- Hands-on exercises with real-world case studies.
- Implementation practice in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
LangGraph for Legal Applications
35 HoursLangGraph is a framework designed for building stateful, multi-actor language model applications as composable graphs with persistent state and precise control over execution.
This instructor-led, live training (online or onsite) is aimed at intermediate to advanced-level professionals who wish to design, implement, and operate LangGraph-based legal solutions, incorporating the necessary compliance, traceability, and governance controls for government use.
By the end of this training, participants will be able to:
- Design legal-specific LangGraph workflows that ensure auditability and compliance.
- Integrate legal ontologies and document standards into graph state and processing.
- Implement guardrails, human-in-the-loop approvals, and traceable decision paths.
- Deploy, monitor, and maintain LangGraph services in production with observability and cost controls for government operations.
Format of the Course
- Interactive lecture and discussion.
- Extensive exercises and practice sessions.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact Govtra to arrange.
Building Dynamic Workflows with LangGraph and LLM Agents
14 HoursLangGraph is a framework designed for composing graph-structured workflows that support branching, tool integration, memory management, and controllable execution.
This instructor-led, live training (online or onsite) is aimed at intermediate-level engineers and product teams who wish to integrate LangGraph’s graph logic with language model agent loops to develop dynamic, context-aware applications such as customer support agents, decision trees, and information retrieval systems for government use.
By the end of this training, participants will be able to:
- Design graph-based workflows that coordinate language model agents, tools, and memory management.
- Implement conditional routing, retries, and fallback mechanisms for robust execution.
- Integrate retrieval systems, APIs, and structured outputs into agent loops.
- Evaluate, monitor, and enhance the reliability and safety of agent behavior.
Format of the Course
- Interactive lectures and facilitated discussions.
- Guided labs and code walkthroughs in a secure sandbox environment.
- Scenario-based design exercises and peer reviews to ensure alignment with public sector workflows and governance.
Course Customization Options
- To request a customized training for this course, please contact Govtra to arrange.
LangGraph for Marketing Automation
14 HoursLangGraph is a graph-based orchestration framework that enables conditional, multi-step workflows involving language models and tools, making it ideal for automating and personalizing content pipelines.
This instructor-led, live training (online or onsite) is aimed at intermediate-level marketers, content strategists, and automation developers who wish to implement dynamic, branching email campaigns and content generation pipelines using LangGraph for government applications.
By the end of this training, participants will be able to:
- Design graph-structured content and email workflows with conditional logic.
- Integrate language models, APIs, and data sources for automated personalization.
- Manage state, memory, and context across multi-step campaigns.
- Evaluate, monitor, and optimize workflow performance and delivery outcomes.
Format of the Course
- Interactive lectures and group discussions.
- Hands-on labs implementing email workflows and content pipelines.
- Scenario-based exercises on personalization, segmentation, and branching logic.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Le Chat Enterprise: Private ChatOps, Integrations & Admin Controls
14 HoursLe Chat Enterprise is a private ChatOps solution that offers secure, customizable, and governed conversational AI capabilities for government organizations. It supports Role-Based Access Control (RBAC), Single Sign-On (SSO), connectors, and integrations with enterprise applications.
This instructor-led, live training (online or onsite) is aimed at intermediate-level product managers, IT leads, solution engineers, and security/compliance teams who wish to deploy, configure, and govern Le Chat Enterprise in government environments.
By the end of this training, participants will be able to:
- Set up and configure Le Chat Enterprise for secure deployments within government agencies.
- Enable RBAC, SSO, and compliance-driven controls tailored for government use.
- Integrate Le Chat with enterprise applications and data stores used in government operations.
- Design and implement governance and admin playbooks specifically for government ChatOps workflows.
Format of the Course
- Interactive lectures and discussions focused on government requirements.
- Extensive exercises and practice sessions to reinforce learning.
- Hands-on implementation in a live-lab environment, simulating real-world government scenarios.
Course Customization Options
- To request a customized training for government-specific needs, please contact Govtra to arrange.
Cost-Effective LLM Architectures: Mistral at Scale (Performance / Cost Engineering)
14 HoursMistral is a high-performance family of large language models optimized for cost-effective production deployment at scale for government and other public sector applications.
This instructor-led, live training (online or onsite) is aimed at advanced-level infrastructure engineers, cloud architects, and MLOps leads who wish to design, deploy, and optimize Mistral-based architectures for maximum throughput and minimum cost in a government context.
By the end of this training, participants will be able to:
- Implement scalable deployment patterns for Mistral Medium 3 that align with public sector workflows and governance.
- Apply batching, quantization, and efficient serving strategies to enhance performance while ensuring accountability.
- Optimize inference costs while maintaining high levels of performance and compliance for government operations.
- Design production-ready serving topologies for enterprise workloads that meet the stringent requirements of public sector environments.
Format of the Course
- Interactive lecture and discussion focused on government-specific use cases.
- Extensive exercises and practice to reinforce learning objectives for government applications.
- Hands-on implementation in a live-lab environment tailored to public sector scenarios.
Course Customization Options
- To request a customized training for this course, specifically designed for government agencies, please contact us to arrange.
Productizing Conversational Assistants with Mistral Connectors & Integrations
14 HoursMistral AI is an open AI platform that enables teams to develop and integrate conversational assistants into workflows for government and customer-facing operations.
This instructor-led, live training (online or onsite) is aimed at beginner-level to intermediate-level product managers, full-stack developers, and integration engineers who wish to design, integrate, and deploy conversational assistants using Mistral connectors and integrations for government applications.
By the end of this training, participants will be able to:
- Integrate Mistral conversational models with enterprise and SaaS connectors for government use.
- Implement retrieval-augmented generation (RAG) to ensure grounded responses in governmental contexts.
- Design user experience (UX) patterns for internal and external chat assistants within public sector environments.
- Deploy assistants into product workflows for real-world government use cases.
Format of the Course
- Interactive lecture and discussion tailored to government needs.
- Hands-on integration exercises relevant to public sector workflows.
- Live-lab development of conversational assistants for government applications.
Course Customization Options
- To request a customized training for this course, tailored specifically for government use, please contact us to arrange.
Enterprise-Grade Deployments with Mistral Medium 3
14 HoursMistral Medium 3 is a high-performance, multimodal large language model designed for deployment in production-grade environments across government and enterprise settings.
This instructor-led, live training (online or onsite) is aimed at intermediate-level to advanced-level AI/ML engineers, platform architects, and MLOps teams who wish to deploy, optimize, and secure Mistral Medium 3 for use cases specific to government and enterprise environments.
By the end of this training, participants will be able to:
- Deploy Mistral Medium 3 using API and self-hosted options suitable for government operations.
- Optimize inference performance and costs in alignment with public sector workflows.
- Implement multimodal use cases with Mistral Medium 3, tailored for government applications.
- Apply security and compliance best practices for government environments.
Format of the Course
- Interactive lecture and discussion focusing on government-relevant topics.
- Lots of exercises and practice to reinforce learning.
- Hands-on implementation in a live-lab environment for government scenarios.
Course Customization Options
- To request a customized training for this course, tailored specifically for government needs, please contact Govtra to arrange.
Mistral for Responsible AI: Privacy, Data Residency & Enterprise Controls
14 HoursMistral AI is an open and enterprise-ready AI platform that provides features for secure, compliant, and responsible AI deployment for government.
This instructor-led, live training (online or onsite) is aimed at intermediate-level compliance leads, security architects, and legal/ops stakeholders who wish to implement responsible AI practices with Mistral by leveraging privacy, data residency, and enterprise control mechanisms for government.
By the end of this training, participants will be able to:
- Implement privacy-preserving techniques in Mistral deployments for government.
- Apply data residency strategies to meet regulatory requirements for government.
- Set up enterprise-grade controls such as Role-Based Access Control (RBAC), Single Sign-On (SSO), and audit logs for government.
- Evaluate vendor and deployment options for compliance alignment with government standards.
Format of the Course
- Interactive lecture and discussion focused on government needs.
- Compliance-focused case studies and exercises relevant to government operations.
- Hands-on implementation of enterprise AI controls tailored for government use.
Course Customization Options
- To request a customized training for this course, please contact Govtra to arrange.
Multimodal Applications with Mistral Models (Vision, OCR, & Document Understanding)
14 HoursMistral models are open-source artificial intelligence technologies that now support multimodal workflows, enhancing both language and vision tasks for government and research applications.
This instructor-led, live training (online or onsite) is designed for intermediate-level machine learning researchers, applied engineers, and product teams who wish to develop multimodal applications using Mistral models, including optical character recognition (OCR) and document understanding pipelines for government.
By the end of this training, participants will be able to:
- Set up and configure Mistral models for multimodal tasks.
- Implement OCR workflows and integrate them with natural language processing (NLP) pipelines.
- Design document understanding applications for government use cases.
- Develop vision-text search and assistive user interface functionalities.
Format of the Course
- Interactive lecture and discussion.
- Hands-on coding exercises.
- Live-lab implementation of multimodal pipelines.
Course Customization Options
- To request a customized training for this course, please contact Govtra to arrange.
