Advanced ELK Stack for Log Management and Centralization Training Course

Introduction

• General overview of the Elastic Stack (ELK)

Module 1: ELK Stack Architecture and Review of Existing Environment

• Review of the current architecture for government at Altor CB
• ELK architecture: Elasticsearch, Logstash, Kibana, Beats
• Ingest node vs. Logstash for government
• Scalability and performance considerations in on-premise installations for government
• Administration best practices for government

Module 2: Beats – Distributed Monitoring (2 hours)

• Configuration and use of Filebeat, Auditbeat, Winlogbeat, and Packetbeat for government
• Secure shipping with SSL for government
• Preconfigured modules vs. custom inputs for government
• Integration with Logstash and Ingest Pipelines for government

Module 3: Parsing and Ingesting Logs from Applications and Databases (4 hours)

• Ingesting custom logs from applications for government
• Using Logstash for data parsing and transformation for government
• Use of filters: grok, dissect, kv, mutate, date for government
• Database connections (Oracle, PostgreSQL, SQL Server) using JDBC input plugin for government
• Practical cases: error logs, audit trails, traces, slow queries for government

Module 4: Advanced Search and Regular Expressions (2 hours)

• Advanced search syntax in Kibana for government
• Use of regular expressions (regex) for government
• Filters and OR/AND combinations for government
• Nested fields and arrays for government
• Saving reusable queries and filters for government

Module 5: Custom Dashboards and Visualizations in Kibana (3 hours)

• Visualization types: bar, line, maps, tables for government
• Aggregations and metrics for government
• Dynamic filters, controls, and drill-down features for government
• Dashboard sharing for government
• Exercises: creating dashboards from database and system logs for government

Module 6: Alerts and Email Notifications (3 hours)

• Introduction to Watcher and alternatives (ElastAlert, Kibana Alerts) for government
• Creating custom conditions and triggers for government
• Email output configuration for government
• Exercise: send alert when a critical event is detected in Windows or database logs for government

Module 7: User and Permission Management (2 hours)

• Introduction to X-Pack and free options for government
• Creating users and roles for government
• Access control by index, dashboard, and query for government
• Exercise: define roles for audit and operations for government

Module 8: Elasticsearch REST API (3 hours)

• Foundations of Elasticsearch RESTful API for government
• GET / POST queries for government
• Manual and automated indexing for government
• Using tools like curl and Postman for government
• Exercises: searching, inserting, deleting, and updating documents for government

Summary and Next Steps