PluralSight - Optimizing Apache Spark on Databricks torrent

Optimizing Apache Spark on Databricks
https://TutGee.com

Duration: 2h 19s | Video: .MP4, 1280x720 30 fps | Audio: AAC, 48 kHz, 2ch | Size: 291 MB
Genre: eLearning | Language: English

This course will teach you how to optimize the performance of Spark clusters on Azure Databricks by identifying and mitigating various issues such as data ingestion problems and performance bottlenecks

What you'll learn
The Apache Spark unified analytics engine is an extremely fast and performant framework for big data processing. However, you might find that your Apache Spark code running on Azure Databricks still suffers from a number of issues. These could be due to the difficulty in ingesting data in a reliable manner from a variety of sources or due to performance issues that you encounter because of disk I/O, network performance, or computation bottlenecks.

In this course, Optimizing Apache Spark on Databricks, you will first explore and understand the issues that you might encounter ingesting data into a centralized repository for data processing and insight extraction. Then, you will learn how Delta Lake on Azure Databricks allows you to store data for processing, insights, as well as machine learning on Delta tables and you will see how you can mitigate your data ingestion problems using Auto Loader on Databricks to ingest streaming data.