Course Outline
Introduction
- Comprehensive overview of the Elastic Stack (ELK) infrastructure
Module 1: ELK Stack Architecture and Evaluation of Current Environment
- Assessment of the existing Altor CB architectural framework
- Core ELK components: Elasticsearch, Logstash, Kibana, and Beats
- Comparative analysis of Ingest nodes versus Logstash
- Scalability and performance factors specific to on-premise deployments for government use
- Best practices for administrative oversight and compliance
Module 2: Beats – Distributed Monitoring (2 hours)
- Configuration and deployment of Filebeat, Auditbeat, Winlogbeat, and Packetbeat
- Securing data transmission via SSL encryption
- Evaluation of preconfigured modules against custom input requirements
- Integration with Logstash and Ingest Pipelines for standardized workflows
Module 3: Parsing and Ingesting Logs from Applications and Databases (4 hours)
- Ingestion of custom application logs
- Leveraging Logstash for data parsing and transformation processes
- Application of filters: grok, dissect, kv, mutate, and date
- Establishing database connections (Oracle, PostgreSQL, SQL Server) using the JDBC input plugin
- Practical applications: error logs, audit trails, transaction traces, and slow query analysis
Module 4: Advanced Search and Regular Expressions (2 hours)
- Advanced query syntax implementation in Kibana
- Utilization of regular expressions (regex) for precise data extraction
- Construction of filters using OR/AND logical operators
- Management of nested fields and array structures
- Persistence of reusable queries and filter configurations
Module 5: Custom Dashboards and Visualizations in Kibana (3 hours)
- Visualization methodologies: bar charts, line graphs, maps, and tables
- Implementation of aggregations and key performance metrics
- Deployment of dynamic filters, controls, and drill-down capabilities
- Protocols for dashboard distribution and sharing
- Exercises: Constructing dashboards from database and system logs for operational reporting
Module 6: Alerts and Email Notifications (3 hours)
- Overview of Watcher and alternative tools (ElastAlert, Kibana Alerts)
- Definition of custom conditions and alert triggers
- Configuration of email output channels for notification
- Exercise: Generating alerts upon detection of critical events in Windows or database logs
Module 7: User and Permission Management (2 hours)
- Introduction to X-Pack security features and available options
- Creation and management of users and roles
- Implementation of access controls at the index, dashboard, and query levels
- Exercise: Defining specific roles for audit and operational teams
Module 8: Elasticsearch REST API (3 hours)
- Foundational concepts of the Elasticsearch RESTful API
- Execution of GET and POST queries
- Manual and automated indexing procedures
- Utilization of standard tools such as curl and Postman
- Exercises: Searching, inserting, deleting, and updating document records
Summary and Next Steps
Requirements
- Fundamental understanding of the ELK Stack architecture and core components
- Practical experience in ingesting and visualizing logs using Kibana and Logstash
- Proficiency with Linux command-line interfaces and basic scripting capabilities
Target Audience
- System administrators
- Infrastructure engineers
- Technical teams requiring advanced log centralization capabilities for government compliance and efficiency
Testimonials (2)
The content is very helpful, and the trainer makes it more easier to understand
Ibrahim Al mayahi - Vastech SA
Course - Advanced Elasticsearch and Kibana Administration
the profesionalism of the trainer; the way he tried to respond to all the questions; the review questions we had to ask: engaging us in conversations
