Sumit Gupta

Building scalable and fault-tolerant streaming applications made easy with Spark streamingAbout This Book• Process live data streams more efficiently with better fault recovery using Spark Streaming• Implement and deploy real-time log file analysis• Learn about integration with Advance Spark Libraries – GraphX, Spark SQL, and MLib.Who This Book Is ForThis book is intended for big data developers with basic knowledge of Scala but no knowledge of Spark. It will help you grasp the basics of developing real-time applications with Spark and understand efficient programming of core elements and applications.What You Will Learn• Install and configure Spark and Spark Streaming to execute applications• Explore the architecture and components of Spark and Spark Streaming to use it as a base for other libraries• Process distributed log files in real-time to load data from distributed sources• Apply transformations on streaming data to use its functions• Integrate Apache Spark with the various advance libraries like MLib and GraphX• Apply production deployment scenarios to deploy your applicationIn DetailUsing practical examples with easy-to-follow steps, this book will teach you how to build real-time applications with Spark Streaming.Starting with installing and setting the required environment, you will write and execute your first program for Spark Streaming. This will be followed by exploring the architecture and components of Spark Streaming along with an overview of libraries/functions exposed by Spark. Next you will be taught about various client APIs for coding in Spark by using the use-case of distributed log file processing. You will then apply various functions to transform and enrich streaming data. Next you will learn how to cache and persist datasets. Moving on you will integrate Apache Spark with various other libraries/components of Spark like Mlib, GraphX, and Spark SQL. Finally, you will learn about deploying your application and cover the different scenarios ranging from standalone mode to distributed mode using Mesos, Yarn, and private data centers or on cloud infrastructure.Style and approachA Step-by-Step approach to learn Spark Streaming in a structured manner, with detailed explanation of basic and advance features in an easy-to-follow Style. Each topic is explained sequentially and supported with real world examples and executable code snippets that appeal to the needs of readers with the wide range of experiences.

Learning Real Time Processing with Spark Streaming

Rajesh Gupta

Nikil D. Dutt

Alexandru Nicolau

Rapid advances in microelectronic integration and the advent of Systems-on-Chip have fueled the need for high-level synthesis, i.e., an automated approach to the synthesis of hardware from behavioral descriptions. SPARK: A Parallelizing Approach to the High - Level Synthesis of Digital Circuits presents a novel approach to the high-level synthesis of digital circuits -- that of parallelizing high-level synthesis (PHLS). This approach uses aggressive code parallelizing and code motion techniques to discover circuit optimization opportunities beyond what is possible with traditional high-level synthesis. This PHLS approach addresses the problems of the poor quality of synthesis results and the lack of controllability over the transformations applied during the high-level synthesis of system descriptions with complex control flows, that is, with nested conditionals and loops. Also described are speculative code motion techniques and dynamic compiler transformations that optimize the circuit quality in terms of cycle time, circuit size and interconnect costs. We describe the SPARK parallelizing high-level synthesis framework in which we have implemented these techniques and demonstrate the utility of SPARK's PHLS approach using designs derived from multimedia and image processing applications. We also present a case study of an instruction length decoder derived from the Intel Pentium-class of microprocessors. This case study serves as an example of a typical microprocessor functional block with complex control flow and demonstrates how our techniques are useful for such designs. SPARK: A Parallelizing Approach to the High - Level Synthesis of Digital Circuits is targeted mainly to embedded system designers and researchers. This includes people working on design and design automation. The book is useful for researchers and design automation engineers who wish to understand how the main problems hindering the adoption of high-level synthesis among designers.

SPARK: A Parallelizing Approach to the High-Level Synthesis of Digital Circuits

Bhupendra Prakash Sharma

G. Srinivasa Rao

Pallav Gupta

Anamika Prasad

This book presents select proceedings of the International Conference on Future Learning Aspects of Mechanical Engineering (FLAME 2020). This book, in particular, focuses on characterizing materials using novel techniques. It covers a variety of advanced materials, viz. composites, coatings, nanomaterials, materials for fuel cells, biomaterials among others. The book also discusses advanced characterization techniques like X-ray photoelectron, UV spectroscopy, scanning electron, atomic power, transmission electron and laser confocal scanning fluorescence microscopy, and gel electrophoresis chromatography. This book gives the readers an insight into advanced material processes and characterizations with special emphasis on nanotechnology.

Select Proceedings of FLAME 2020

Advances in Engineering Materials

Shilpi,

Design, process, and analyze large sets of complex data in real time About This Book Get acquainted with transformations and database-level interactions, and ensure the reliability of messages processed using Storm Implement strategies to solve the challenges of real-time data processing Load datasets, build queries, and make recommendations using Spark SQL Who This Book Is For If you are a Big Data architect, developer, or a programmer who wants to develop applications/frameworks to implement real-time analytics using open source technologies, then this book is for you. What You Will Learn Explore big data technologies and frameworks Work through practical challenges and use cases of real-time analytics versus batch analytics Develop real-word use cases for processing and analyzing data in real-time using the programming paradigm of Apache Storm Handle and process real-time transactional data Optimize and tune Apache Storm for varied workloads and production deployments Process and stream data with Amazon Kinesis and Elastic MapReduce Perform interactive and exploratory data analytics using Spark SQL Develop common enterprise architectures/applications for real-time and batch analytics In Detail Enterprise has been striving hard to deal with the challenges of data arriving in real time or near real time. Although there are technologies such as Storm and Spark (and many more) that solve the challenges of real-time data, using the appropriate technology/framework for the right business use case is the key to success. This book provides you with the skills required to quickly design, implement and deploy your real-time analytics using real-world examples of big data use cases. From the beginning of the book, we will cover the basics of varied real-time data processing frameworks and technologies. We will discuss and explain the differences between batch and real-time processing in detail, and will also explore the techniques and programming concepts using Apache Storm. Moving on, we'll familiarize you with “Amazon Kinesis” for real-time data processing on cloud. We will further develop your understanding of real-time analytics through a comprehensive review of Apache Spark along with the high-level architecture and the building blocks of a Spark program. You will learn how to transform your data, get an output from transformations, and persist your results using Spark RDDs, using an interface called Spark SQL to work with Spark. At the end of this book, we will introduce Spark Streaming, the streaming library of Spark, and will walk you through the emerging Lambda Architecture (LA), which provides a hybrid platform for big data processing by combining real-time and precomputed batch data to provide a near real-time view of incoming data. Style and approach This step-by-step is an easy-to-follow, detailed tutorial, filled with practical examples of basic and advanced features. Each topic is explained sequentially and supported by real-world examples and executable code snippets.

Sumit Gupta

About

Books

Join the club