Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Overview of the EXO Framework and Local Artificial Intelligence Clustering
- Introduction to the EXO framework and the exo-explore ecosystem for distributed computing resources.
- Evaluation of centralized cloud-based inference against decentralized, localized inference models for government applications.
- Technical architecture components, including libp2p device discovery protocols, MLX backend integration, dashboard interfaces, and application programming interface (API) layers.
- Hardware specifications required for optimal performance, specifically Apple Silicon architectures (M3 Ultra, M4 Pro/Max), Thunderbolt 5 connectivity, and shared storage solutions suitable for secure data processing.
Deployment of EXO on macOS Environments
- Configuration of prerequisites, including Xcode, the Metal ToolChain, and associated macOS dependencies.
- Installation of essential development tools, including uv, Node.js, and the Rust nightly toolchain.
- Setup of a customized macmon fork to facilitate hardware monitoring for Apple Silicon devices.
- Procedures for cloning the source repository and compiling the management dashboard using npm.
- Execution of EXO from source code and verification of system accessibility via the localhost:52415 dashboard interface.
Deployment of EXO on Linux Environments
- Installation of system dependencies through apt or Homebrew package managers.
- Configuration of runtime environments, including uv, Node.js (version 18 or higher), and the Rust nightly toolchain.
- Compilation of the dashboard and initialization of EXO in central processing unit (CPU)-only operational mode.
- Overview of directory structures, adhering to XDG Base Directory standards for configuration, data storage, caching, and logging.
Automated Device Discovery and Cluster Formation Protocols
- Mechanisms for automatic node discovery across local area networks using libp2p protocols.
- Configuration of isolated cluster namespaces via the EXO_LIBP2P_NAMESPACE environment variable to enhance security and segmentation.
- Verification of node integration within the cluster view presented in the management dashboard.
- Procedures for addressing discovery failures and mitigating issues related to network segmentation.
Implementation of Remote Direct Memory Access (RDMA) over Thunderbolt 5
- Overview of RDMA architecture and its capacity to reduce latency by approximately 99 percent, supporting high-throughput data transfers.
- Steps for enabling RDMA functionality within macOS Recovery mode using rdma_ctl utilities.
- Specifications for cabling requirements and port topology configurations on Mac Studio hardware.
- Requirement for version consistency across all cluster nodes running compatible macOS releases.
- Troubleshooting methodologies for RDMA discovery processes and dynamic host configuration protocol (DHCP) settings.
Deployment of Frontier Artificial Intelligence Models
- Utilization of the management dashboard to load and partition frontier models, including DeepSeek v3.1, Qwen3-235B, and the Llama family.
- Pre-deployment analysis of instance placement using the /instance/previews API endpoint.
- Creation of model instances utilizing pipeline or tensor-parallel sharding techniques for efficient resource allocation.
- Configuration protocols for custom model cards sourced from the HuggingFace hub, tailored for specific mission requirements.
Operational Monitoring and Diagnostic Procedures
- Analysis of EXO log outputs and interpretation of distributed tracing data for operational visibility.
- Assessment of cluster health metrics via the dashboard’s cluster view interface.
- Diagnosis of worker node failures and management of automatic reconnection behaviors.
- Leveraging the EXO_TRACING_ENABLED flag to identify and address performance bottlenecks within government infrastructure.
Cluster Maintenance and System Updates
- Procedures for updating EXO binaries and rebuilding dashboard interfaces to ensure system integrity.
- Management of model cache migrations and handling of pre-downloaded models across Network File System (NFS) shares.
- Protocols for the graceful removal of nodes and rebalancing of computational workloads to maintain operational efficiency.
Requirements
**Prerequisite Knowledge**
* Foundational concepts in network architecture, including Internet Protocol (IP) addressing, subnetting methodologies, and firewall configuration.
* Proficiency in command-line interface administration within macOS or Linux environments.
* Competence in Python package management systems (e.g., pip/uv) and Node.js development tooling.
**Target Audience**
* System administrators managing institutional IT resources.
* DevOps engineers optimizing deployment pipelines.
* AI infrastructure architects tasked with the governance and implementation of on-premise Large Language Model (LLM) solutions for government applications.
21 Hours
