Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Deployment in Offline Environments
- Enabling offline configurations to eliminate runtime reliance on external network resources
- Sourcing model artifacts from vetted internal repositories and pre-loading them into designated read-only directories
- Ensuring the integrity of model weights through SHA-256 checksum validation and the use of signed model documentation
- Executing workflows within air-gapped networks to remove dependency on external model hosting platforms
Access Management for Dashboards and APIs
- Configuring reverse proxy servers with TLS termination to secure web interfaces and API endpoints
- Applying role-based access control protocols to restrict privileges for dashboard and REST API interactions
- Utilizing system-level credential managers to securely store authentication secrets for API access
- Limiting administrative access to verified source IP ranges to enhance endpoint security
Cluster Segmentation and Network Protection
- Segmenting cluster resources using namespace isolation and VLAN configurations to limit lateral movement
- Applying host-based firewall rules to restrict traffic on specific service ports
- Preventing unauthorized device discovery and the injection of unapproved nodes into the network
- Securing peer-to-peer traffic between nodes using encryption when direct memory access is not available
Model Governance and Source Verification
- Maintaining an internal registry of approved models with associated metadata for governance
- Versioning and tagging quantized model weights to ensure traceability against source checkpoints
- Restricting model loading capabilities to authorized repositories or verified internal artifacts only
- Documenting model lineage, licensing conditions, and acceptable use policies for compliance
Audit Trails and Regulatory Compliance
- Forwarding system logs to immutable audit storage systems such as SIEM platforms or write-once media
- Correlating API activity logs with user identities and precise timestamps for accountability
- Recording events related to model instance lifecycle changes and inference requests
- Producing periodic reports to support internal and external compliance audits
Risk Assessment and Incident Response
- Identifying potential threats, including data leakage via model outputs, prompt manipulation, and side-channel vulnerabilities
- Deploying monitoring and filtering mechanisms to detect and mitigate malicious inputs
- Establishing standardized procedures for responding to cluster security compromises
- Isolating compromised components, preserving forensic evidence, and restoring secure operational environments
Physical Security and Hardware Controls
- Protecting high-speed data ports from unauthorized peripheral connections
- Leveraging secure enclaves and hardware attestation features to verify system integrity
- Implementing strict physical access controls for hardware clusters and shared storage media
- Maintaining records of hardware lifecycle management and secure decommissioning processes
Regulatory Alignment
- Aligning deployment practices with GDPR, HIPAA, and SOC 2 standards
- Ensuring data residency requirements are met by keeping inference operations on-premise
- Assessing and documenting risks associated with third-party software dependencies and supply chains
- Preparing operational procedures to meet emerging AI governance frameworks
Requirements
- Practical experience with EXO or similar local large language model runtimes
- Working knowledge of Unix filesystem permissions and network access control lists
- Familiarity with TLS/SSL certificate management and fundamental encryption principles
Intended Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators responsible for handling sensitive government data
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
