糖心vlog官网观看

In this module, you'll gain a fundamental understanding of the Apache Hadoop architecture, ecosystem, practices, and commonly used applications, including Distributed File System (HDFS), MapReduce, Hive, and HBase. You鈥檒l also gain practical skills in hands-on labs when you query the data added using Hive, launch a single-node Hadoop cluster using Docker, and run MapReduce jobs.

In this module, you鈥檒l turn your attention to the popular Apache Spark platform, where you will explore the attributes and benefits of Apache Spark and distributed computing. You'll gain key insights about functional programming and Lambda functions. You鈥檒l also explore Resilient Distributed Datasets (RDDs), parallel programming, resilience in Apache Spark, and relate RDDs and parallel programming with Apache Spark. Then, you鈥檒l dive into additional Apache Spark components and learn how Apache Spark scales with Big Data. Working with Big Data signals the need for working with queries, including structured queries using SQL. You鈥檒l also learn about the functions, parts, and benefits of Spark SQL and DataFrame queries, and discover how DataFrames work with Spark SQL.

In this module, you鈥檒l learn about Resilient Distributed Datasets (RDDs), their uses in Apache Spark, and RDD transformations and actions. You'll compare the use of datasets with Spark's latest data abstraction, DataFrames. You'll learn to identify and apply basic DataFrame operations. You鈥檒l explore Apache Spark SQL optimization and learn how Spark SQL and memory optimization benefit from using Catalyst and Tungsten. Finally, you鈥檒l fortify your skills with guided hands-on lab to create a table view and apply data aggregation techniques.

In this module, you鈥檒l explore how Spark processes the requests that your application submits and learn how you can track work using the Spark Application UI. Because Spark application work happens on the cluster, you need to be able to identify Apache Cluster Managers, their components, and benefits. You鈥檒l also know how to connect with each cluster manager and how and when you might want to set up a local, standalone Spark instance. Next, you鈥檒l learn about Apache Spark application submission, including the use of Spark鈥檚 unified interface, 鈥渟park-submit,鈥� and learn about options and dependencies. You鈥檒l also describe and apply options for submitting applications, identify external application dependency management techniques, and list Spark Shell benefits. You鈥檒l also look at recommended practices for Spark's static and dynamic configuration options and perform hands-on labs to use Apache Spark on IBM Cloud and run Spark on Kubernetes.

Platforms and applications require monitoring and tuning to manage issues that inevitably happen. In this module, you'll learn about connecting the Apache Spark user interface web server and using the same UI web server to manage application processes. You鈥檒l also identify common Apache Spark application issues and learn about debugging issues using the application UI and locating related log files. Further, you鈥檒l discover and gain real-world knowledge about how Spark manages memory and processor resources using the hands-on lab.

In this module, you鈥檒l perform a practice lab where you鈥檒l explore two critical aspects of data processing using Spark: working with Resilient Distributed Datasets (RDDs) and constructing DataFrames from JSON data. You will also apply various transformations and actions on both RDDs and DataFrames to gain insights and manipulate the data effectively. Further, you鈥檒l apply your knowledge in a final project where you will create a DataFrame by loading data from a CSV file and applying transformations and actions using Spark SQL. Finally, you鈥檒l be assessed based on your learning from the course.