Real-Time Streaming with Apache Kafka, Spark, and Storm

Build a platform using Apache Kafka, Spark, and Storm to generate real-time data insights and view them through Dashboards.

Key Features

● Extensive practical demonstration of Apache Kafka concepts, including producer and consumer examples.

● Includes graphical examples and explanations of implementing Kafka Producer and Kafka Consumer commands and methods.

● Covers integration and implementation of Spark-Kafka and Kafka-Storm architectures.

Description

Real-Time Streaming with Apache Kafka, Spark, and Storm is a book that provides an overview of the real-time streaming concepts and architectures of Apache Kafka, Storm, and Spark. The readers will learn how to build systems that can process data streams in real time using these technologies. They will be able to process a large amount of real-time data and perform analytics or generate insights as a result of this.

The architecture of Kafka and its various components are described in detail. A Kafka Cluster installation and configuration will be demonstrated. The Kafka publisher-subscriber system will be implemented in the Eclipse IDE using the Command Line and Java. The book discusses the architecture of Apache Storm, the concepts of Spout and Bolt, as well as their applications in a Transaction Alert System. It also describes Spark’s core concepts, applications, and the use of Spark to implement a microservice. To learn about the process of integrating Kafka and Storm, two approaches to Spark and Kafka integration will be discussed.

This book will assist a software engineer to transition to a Big Data engineer and Big Data architect by providing knowledge of big data processing and the architectures of Kafka, Storm, and Spark Streaming.

What you will learn

● Creation of Kafka producers, consumers, and brokers using command line.

● End-to-end implementation of Kafka messaging system with Java in Eclipse.

● Perform installation and creation of a Storm Cluster and execute Storm Management commands.

● Implement Spouts, Bolts and a Topology in Storm for Transaction alert application system.

Who this book is for

This book is intended for Software Developers, Data Scientists, and Big Data Architects who want to build software systems to process data streams in real time. To understand the concepts in this book, knowledge of any programming language such as Java, Python, etc. is needed.

Table of Contents

1. Introduction to Kafka

2. Installing Kafka

3. Kafka Messaging

4. Kafka Producers

5. Kafka Consumers

6. Introduction to Storm

7. Installation and Configuration

8. Spouts and Bolts

9. Introduction to Spark

10. Spark Streaming

11. Kafka Integration with Storm

12. Kafka Integration with Spark

About the Authors

Brindha Priyadarshini Jeyaraman has more than 12+ years of work experience in Software Development and building Data analytics systems. She has completed her M.Tech in Knowledge Engineering with a gold medal from the National University of Singapore. She is an expert in understanding business problems, designing, and implementing solutions using Machine Learning. She has a strong software development background with extensive experience in implementing data analytics systems. She has worked on several Data Science projects in Transportation, E-commerce, Healthcare, Insurance, Banking and Finance Domains. She has completed her SCJP and SCWCD certification