Preparing with Cloudera Data Engineering

This four-day hands-on training course delivers the key concepts and expertise developers need to use Apache Spark to develop high-performance parallel applications.

Participants will learn how to use Spark SQL to query structured data and Spark Streaming to perform real-time processing on streaming data from a variety of sources. Developers will also practice writing applications that use core Spark to perform ETL processing and iterative algorithms.

Note: Scala and Python developers will learn key concepts and gain the expertise needed to ingest and process data, and develop high-performance applications using Apache Spark 2.

1:1 Coaching

24*7 Support

Cloud Labs

High Success Rate

Globally Renowned Trainer

Real-time code analysis and feedback

Course Description

The course covers how to work with “big data” stored in a distributed file system, and execute Spark applications on a Hadoop cluster. After taking this course, participants will be prepared to face real-world challenges and build applications to execute faster decisions, better decisions, and interactive analysis, applied to a wide variety of use cases, architectures, and industries.

Learning Objectives

How to write, configure, and deploy Apache Spark applications on a Hadoop cluster
How to use the Spark shell and Spark applications to explore, process, and analyze distributed data
How the Apache Hadoop ecosystem fits in with the data processing lifecycle

How data is distributed, stored, and processed in a Hadoop cluster
How to query data using Spark SQL, DataFrames, and Datasets
How to use Spark Streaming to process a live data stream

Schedule

Enroll your course from

$3,564.00

6th – 9th May, 2024

09:30AM – 05:30PM

Trainer

Ajit Kumar Amit

Singapore

Weekday

Enroll your course from

$3,564.00

26th – 29th Aug, 2024

09:30AM – 05:30PM

Trainer

Ajit Kumar Amit

Singapore

Weekday

Enroll your course from

$3,564.00

11th – 14th Nov, 2024

09:30AM – 05:30PM

Trainer

Ajit Kumar Amit

Singapore

Weekday

Certification Curriculum

Introduction

Introduction to Apache Hadoop and the Hadoop Ecosystem

Apache Hadoop Overview
Data Processing
Introduction to the Hands-On Exercises

Module 1

Apache Hadoop File Storage

Apache Hadoop Cluster Components
HDFS Architecture
Using HDFS

Module 2

Distributed Processing on an Apache Hadoop Cluster

YARN Architecture
Working With YARN

Module 3

Apache Spark Basics

What is Apache Spark?
Starting the Spark Shell
Using the Spark Shell
Getting Started with Datasets and DataFrames
DataFrame Operations

Module 4

Working with DataFrames and Schemas

Creating DataFrames from Data Sources
Saving DataFrames to Data Sources
DataFrame Schemas
Eager and Lazy Execution

Module 5

Analyzing Data with DataFrame Queries

Querying DataFrames Using Column Expressions
Grouping and Aggregation Queries
Joining DataFrames

Module 6

RDD Overview

RDD Overview
RDD Data Sources
Creating and Saving RDDs
RDD Operations

Module 7

Transforming Data with RDDs

Writing and Passing Transformation Functions
Transformation Execution
Converting Between RDDs and DataFrames

Module 8

Aggregating Data with Pair RDDs

Key-Value Pair RDDs
Map-Reduce
Other Pair RDD Operations

Module 9

Querying Tables and Views with SQL

Querying Tables in Spark Using SQL
Querying Files and Views
The Catalog API

Module 10

Working with Datasets in Scala

Datasets and DataFrames
Creating Datasets
Loading and Saving Datasets
Dataset Operations

Module 11

Writing, Configuring, and Running Spark Applications

Writing a Spark Application
Building and Running an Application
Application Deployment Mode
The Spark Application Web UI
Configuring Application Properties

Module 12

Spark Distributed Processing

Review: Apache Spark on a Cluster
RDD Partitions
Example: Partitioning in Queries
Stages and Tasks
Job Execution Planning
Example: Catalyst Execution Plan
Example: RDD Execution Plan

Module 13

Distributed Data Persistence

DataFrame and Dataset Persistence
Persistence Storage Levels
Viewing Persisted RDDs

Module 14

Common Patterns in Spark Data Processing

Common Apache Spark Use Cases
Iterative Algorithms in Apache Spark
Machine Learning
Example: k-means

Module 15

Introduction to Structured Streaming

Apache Spark Streaming Overview
Creating Streaming DataFrames
Transforming DataFrames
Executing Streaming Queries

Module 16

Structured Streaming with Apache Kafka

Overview
Receiving Kafka Messages
Sending Kafka Messages

Module 17

Aggregating and Joining Streaming DataFrames

Streaming Aggregation
Joining Streaming DataFrames

Conclusion

Message Processing with Apache Kafka

What Is Apache Kafka?
Apache Kafka Overview
Scaling Apache Kafka
Apache Kafka Cluster Architecture
Apache Kafka Command Line Tools

Prerequisites

This course is designed for developers and engineers who have programming experience, but prior knowledge of Spark and Hadoop is not required. Apache Spark examples and hands-on exercises are presented in Scala and Python. The ability to program in one of those languages is required. Basic familiarity with the Linux command line is assumed. Basic knowledge of SQL is helpful.

Certification Assessment

When you complete all of the courses in the program, you’ll earn a Certificate to share with your professional network as well as unlock access to career support resources to help you kickstart your new career. Many Professional Certificates have hiring partners that recognize the Professional Certificate credential and others can help prepare you for a certification exam. You can find more information on individual Professional Certificate pages where it applies.

Testimonials

Our clients praise us for our great results, personable service, expert knowledge, and on-time delivery. Here are what just a few of them had to say:

Sarbojit Bose

The course is one of the two in the track of Agile Professional Coach. It is designed to provide both wide and deep knowledge to become a competent Coach with the additional skills of a Trainer and a Mentor. The two trainers, Preeth Panday and Naveen K Singh, are excellent Facilitators and Coaches with patience and promptness.

Tiong Yeow Tan

It was a good introductory course to agile coaching, providing a good overview and potential deep dive into other courses if one wishes to learn more about a particular topic. Trainer was able to address most of the questions from the participants and demonstrate coaching in his interaction with us.

Chun Ngee Boo Alson

The course is well structure. Timing is also right. The trainer Mr Raj is professional. And he asnwer all my question and doubts.

Puli

One of best I have encountered in my life. Freedom to interact and respond candidly and with courage for every question is not an easy task for Trainers which they did it exceptionally well.

Training FAQ's

Is this training conducted as an online/ virtual course?

As a response to the COVID-19, we have moved our classes online. Our Live Virtual format delivers the same benefits as our face-to-face training: expert instruction, hands-on labs and exercises, peer-to-peer collaboration, and high-quality instructional material.

If I cancel the enrollment, will I get a refund?

Cancellation requests received within 24 hours of registration would be offered a full refund (minus payment gateway charges), please reach out to our support team through drop a refund request to enquiry@agilitics.sg. Visit our page for more details about Cancellation & Refund Policy.

What Payment Options Are Available?

The process of enrolling for this classroom training is simple. The payment can be made through different options by using a debit/credit card which includes MasterCard, Visa Card, American Express or through PayPal. Acknowledgment will be issued automatically via email to the candidates once payment is done.

What are the modes of payment available for payment of accreditation fees?

Payment can be made via credit card, debit card, UPI, and internet banking.

Have more doubts?

Please send in an email to enquiry@agilitics.sg, and we will answer any queries you may have!

Preparing with Cloudera Data Engineering

Course Description

Learning Objectives

Schedule

Certification Curriculum

Introduction to Apache Hadoop and the Hadoop Ecosystem

Apache Hadoop File Storage

Distributed Processing on an Apache Hadoop Cluster

Apache Spark Basics

Working with DataFrames and Schemas

Analyzing Data with DataFrame Queries

RDD Overview

Transforming Data with RDDs

Aggregating Data with Pair RDDs

Querying Tables and Views with SQL

Working with Datasets in Scala

Writing, Configuring, and Running Spark Applications

Spark Distributed Processing

Distributed Data Persistence

Common Patterns in Spark Data Processing

Introduction to Structured Streaming

Structured Streaming with Apache Kafka

Aggregating and Joining Streaming DataFrames

Message Processing with Apache Kafka

Prerequisites

Download Brochure

Certification Assessment

Testimonials

Our clients praise us for our great results, personable service, expert knowledge, and on-time delivery. Here are what just a few of them had to say:

Training FAQ's

Trending Course

Leading Safe®️ 5.1

Cloudera Data Analyst

Certified Agile Coaching

We'd loveto hear from you

Download Brochure

Quick Links

Trending Courses

Discover

Contact Information

Singapore

Social

We'd love
to hear from you