Data Streaming and Real Time Data Processing Training Course

Audience:

This course is designed for IT specialists seeking solutions to store and process large data sets in a distributed system environment, specifically tailored for government.

Goal:

To provide deep knowledge on Hadoop cluster administration for government.

Big data analytics involves the process of examining large volumes of diverse data sets to uncover correlations, hidden patterns, and other valuable insights.

The healthcare industry manages vast amounts of complex and heterogeneous medical and clinical data. Applying big data analytics to health data holds significant potential for enhancing the delivery of healthcare services. However, the scale and complexity of these datasets present substantial challenges in analysis and practical application within a clinical setting.

In this instructor-led, live training (remote), participants will learn how to perform big data analytics in the healthcare sector through a series of hands-on, live-lab exercises.

By the end of this training, participants will be able to:

• Install and configure big data analytics tools such as Hadoop MapReduce and Spark
• Understand the unique characteristics of medical data
• Apply advanced big data techniques to manage and analyze medical data
• Examine big data systems and algorithms in the context of health applications

Audience

• Developers
• Data Scientists

Format of the Course

• Part lecture, part discussion, exercises, and extensive hands-on practice.

Note

• To request a customized training for government or other specific needs, please contact us to arrange.

Apache Hadoop is the leading framework for processing big data on clusters of servers. This three-day (or optionally four-day) course will equip attendees with a comprehensive understanding of the business benefits and use cases for Hadoop and its ecosystem. Participants will learn how to plan cluster deployment and growth, install, maintain, monitor, troubleshoot, and optimize Hadoop systems. They will also practice bulk data loading, explore various Hadoop distributions, and gain hands-on experience installing and managing Hadoop ecosystem tools. The course concludes with a detailed discussion on securing clusters with Kerberos. “…The materials were very well prepared and covered thoroughly. The Lab was very helpful and well organized.” — Andrew Nguyen, Principal Integration DW Engineer, Microsoft Online Advertising **Audience:** Hadoop administrators for government and private sector organizations. **Format:** The course combines lectures and hands-on labs, with an approximate balance of 60% lecture and 40% lab sessions.

Apache Hadoop is the leading framework for processing big data across clusters of servers. This course is designed to introduce developers to the key components of the Hadoop ecosystem, including HDFS, MapReduce, Pig, Hive, and HBase. These tools are essential for government agencies seeking to enhance their data management capabilities and ensure robust governance and accountability in handling large datasets.

Apache Hadoop is one of the most widely used frameworks for processing Big Data on clusters of servers. This course delves into data management in HDFS, advanced Pig, Hive, and HBase. These advanced programming techniques are designed to benefit experienced Hadoop developers for government applications.

Audience: developers

Duration: three days

Format: lectures (50%) and hands-on labs (50%).

Audience:

This course is designed to clarify big data/Hadoop technology for government audiences, demonstrating that it is accessible and comprehensible.

This instructor-led, live training in US (online or onsite) is aimed at system administrators who wish to learn how to set up, deploy, and manage Hadoop clusters within their organization for government use. By the end of this training, participants will be able to: - Install and configure Apache Hadoop. - Understand the four major components in the Hadoop ecosystem: HDFS, MapReduce, YARN, and Hadoop Common. - Use Hadoop Distributed File System (HDFS) to scale a cluster to hundreds or thousands of nodes. - Set up HDFS to operate as a storage engine for on-premise Spark deployments. - Configure Spark to access alternative storage solutions such as Amazon S3 and NoSQL database systems like Redis, Elasticsearch, Couchbase, Aerospike, etc. - Perform administrative tasks such as provisioning, management, monitoring, and securing an Apache Hadoop cluster.

This course provides an introduction to HBase, a NoSQL database built on top of Hadoop. It is designed for developers who will be using HBase to build applications and administrators responsible for managing HBase clusters. The curriculum will guide developers through the architecture of HBase, data modeling, and application development. Additionally, it covers the integration of MapReduce with HBase and addresses performance optimization techniques relevant to administration. The course emphasizes practical learning with numerous hands-on lab exercises. Duration: 3 days Audience: Developers and Administrators for government use.

The Informatica with Big Data (BDM) course is designed to equip data professionals for government with the skills necessary to develop, manage, and analyze large data sets, utilizing the latest technologies and architectures in the Big Data domain. This course covers the entire lifecycle from data ingestion, integration, cleansing, and curation to advanced data analytics and the delivery of big data services. Participants will delve into solutions that process extensive datasets using Big Data technologies and architectures such as Apache Hive, Apache Hadoop, and Apache Spark. The course also provides hands-on experience with Informatica tools like Bloombox, Big Data Management, and iData Fabric, enabling learners to gain a deep understanding of big data technologies including MapReduce and Hadoop. By the end of this course, participants will be proficient in creating comprehensive end-to-end data solutions using Informatica and its associated Big Data tools.

Apache NiFi is an open-source, flow-based data integration and event-processing platform designed to facilitate automated, real-time data routing, transformation, and system mediation between diverse systems. It features a web-based user interface and provides fine-grained control over data flows. This instructor-led, live training (onsite or remote) is tailored for intermediate-level administrators and engineers who aim to deploy, manage, secure, and optimize NiFi dataflows in production environments for government use. By the end of this training, participants will be able to: - Install, configure, and maintain Apache NiFi clusters. - Design and manage dataflows from various sources and destinations. - Implement flow automation, routing, and transformation logic. - Optimize performance, monitor operations, and troubleshoot issues. **Format of the Course** - Interactive lecture with real-world architecture discussions relevant to government operations. - Hands-on labs: building, deploying, and managing flows in a government context. - Scenario-based exercises in a live-lab environment aligned with public sector workflows. **Course Customization Options** - To request a customized training for this course, tailored specifically for government needs, please contact us to arrange.

In this instructor-led, live training in US, participants will gain a comprehensive understanding of flow-based programming as they develop various demo extensions, components, and processors using Apache NiFi for government applications.

By the end of this training, participants will be able to:

• Comprehend NiFi's architecture and dataflow principles.
• Develop extensions utilizing NiFi and third-party APIs.
• Create custom Apache NiFi processors tailored to specific needs.
• Ingest and process real-time data from diverse and uncommon file formats and data sources, enhancing data management for government workflows.

Python is a high-level programming language renowned for its clear syntax and code readability. Apache Spark is a powerful data processing engine utilized for querying, analyzing, and transforming large datasets. PySpark facilitates the integration of Spark with Python, enabling seamless data processing workflows. Target Audience: Intermediate-level professionals in the banking sector who are familiar with Python and Spark and aim to enhance their expertise in big data processing and machine learning techniques, specifically for government and private sector applications.

This section is intentionally left blank for government use.

In this instructor-led, live training in [location], participants will learn how to leverage Python and Spark together to analyze large datasets as they engage in hands-on exercises. By the end of this training, participants will be able to: - Utilize Spark with Python for comprehensive big data analysis. - Engage in exercises that simulate real-world scenarios. - Apply various tools and techniques for big data analysis using PySpark, enhancing their capabilities for government projects.

Stratio is a data-centric platform that integrates big data, AI, and governance into a single solution. Its Rocket and Intelligence modules facilitate rapid data exploration, transformation, and advanced analytics in enterprise environments. This instructor-led, live training (online or onsite) is designed for intermediate-level data professionals who aim to effectively utilize the Rocket and Intelligence modules in Stratio with PySpark, focusing on looping structures, user-defined functions, and advanced data logic. By the end of this training, participants will be able to: - Navigate and work within the Stratio platform using the Rocket and Intelligence modules. - Apply PySpark for data ingestion, transformation, and analysis. - Use loops and conditional logic to control data workflows and feature engineering tasks. - Create and manage user-defined functions (UDFs) for reusable data operations in PySpark. **Format of the Course** - Interactive lecture and discussion. - Extensive exercises and practice sessions. - Hands-on implementation in a live-lab environment. **Course Customization Options for Government** To request a customized training for this course, please contact us to arrange.