Course Outline
Introduction
- General overview of the Elastic Stack (ELK)
Module 1: ELK Stack Architecture and Review of Existing Environment
- Review of the current architecture of Altor CB for government
- ELK architecture: Elasticsearch, Logstash, Kibana, Beats
- Ingest node vs. Logstash
- Scalability and performance considerations in on-premise installations for government
- Administration best practices for government environments
Module 2: Beats – Distributed Monitoring (2 hours)
- Configuration and use of Filebeat, Auditbeat, Winlogbeat, and Packetbeat in government systems
- Secure shipping with SSL for government data
- Preconfigured modules vs. custom inputs for government-specific requirements
- Integration with Logstash and Ingest Pipelines for enhanced security and compliance
Module 3: Parsing and Ingesting Logs from Applications and Databases (4 hours)
- Ingesting custom logs from applications in government environments
- Using Logstash for data parsing and transformation to meet regulatory standards
- Use of filters: grok, dissect, kv, mutate, date for detailed log analysis
- Database connections (Oracle, PostgreSQL, SQL Server) using JDBC input plugin for government databases
- Practical cases: error logs, audit trails, traces, slow queries in government applications
Module 4: Advanced Search and Regular Expressions (2 hours)
- Advanced search syntax in Kibana for efficient data retrieval in government settings
- Use of regular expressions (regex) for complex query patterns
- Filters and OR/AND combinations to refine search results
- Nested fields and arrays for structured data analysis
- Saving reusable queries and filters for consistent reporting in government operations
Module 5: Custom Dashboards and Visualizations in Kibana (3 hours)
- Visualization types: bar, line, maps, tables for government data representation
- Aggregations and metrics to provide insights into government operations
- Dynamic filters, controls, and drill-down features for interactive dashboards in government
- Dashboard sharing to facilitate collaboration among government teams
- Exercises: creating dashboards from database and system logs for government use cases
Module 6: Alerts and Email Notifications (3 hours)
- Introduction to Watcher and alternatives (ElastAlert, Kibana Alerts) for government monitoring
- Creating custom conditions and triggers for timely alerts in government systems
- Email output configuration to ensure rapid response in government operations
- Exercise: send alert when a critical event is detected in Windows or database logs for government environments
Module 7: User and Permission Management (2 hours)
- Introduction to X-Pack and free options for government use
- Creating users and roles with specific permissions for government staff
- Access control by index, dashboard, and query to ensure data security in government
- Exercise: define roles for audit and operations to meet government compliance requirements
Module 8: Elasticsearch REST API (3 hours)
- Foundations of Elasticsearch RESTful API for government applications
- GET / POST queries for data manipulation in government systems
- Manual and automated indexing to optimize data management for government operations
- Using tools like curl and Postman for testing and deployment in government environments
- Exercises: searching, inserting, deleting, and updating documents in government databases
Summary and Next Steps
Requirements
- A comprehensive understanding of the fundamental ELK Stack architecture and its components
- Practical experience in ingesting and visualizing logs using Kibana and Logstash
- Proficiency with Linux command line operations and basic scripting
Audience
- System administrators for government agencies
- Infrastructure engineers in the public sector
- Technical teams within government organizations seeking advanced log centralization capabilities
Testimonials (5)
it is all
Assad Alshabibi - Vastech SA
Course - Advanced Elasticsearch and Kibana Administration
the practice learning
Presiyan Petrov
Course - ELK Training
I thought the training was very thorough and while we covered a lot of material, Martin made ample time for questions and gave good focus to each individual and their different requirements.
Jean Thysse - Quidco
Course - Elasticsearch for Developers
I enjoyed the exercices gives a good insight.
Andreas Kukacka
Course - ELK: Elasticsearch, Logstash and Kibana for Administrators
I genuinely liked learning a new skill.
