[Activity] Running the Similar Movies Script using Spark's Cluster Manager - Machine Learning and Big Data Training with Frank Kane

[Activity] Running the Similar Movies Script using Spark’s Cluster Manager

November 5, 2019

Back to: Apache Spark 3 with Scala – Hands On with Big Data!

Superhero Degrees of Separation: Review the code, and run it!

Getting Started
- Tip: Apply for a Twitter Developer Account now!
- Introduction, and Getting Set Up
- [Activity] Create a Histogram of Real Movie Ratings with Spark!
Scala Crash Course [Optional]
- [Activity] Scala Basics, Part 1
- [Exercise] Scala Basics, Part 2
- [Exercise] Flow Control in Scala
- [Exercise] Functions in Scala
- [Exercise] Data Structures in Scala
Spark Basics and Simple Examples
- Introduction to Spark
- The Resilient Distributed Dataset
- Ratings Histogram Walkthrough
- Spark Internals
- Key / Value RDD’s, and the Average Friends by Age example
- [Activity] Running the Average Friends by Age Example
- Filtering RDD’s, and the Minimum Temperature by Location Example
- [Activity] Running the Minimum Temperature Example, and Modifying it for Maximum
- [Activity] Counting Word Occurrences using Flatmap()
- [Activity] Improving the Word Count Script with Regular Expressions
- [Activity] Sorting the Word Count Results
- [Exercise] Find the Total Amount Spent by Customer
- [Exercise] Check your Results, and Sort Them by Total Amount Spent
- Check Your Results and Implementation Against Mine
Advanced Examples of Spark Programs
- [Activity] Find the Most Popular Movie
- [Activity] Use Broadcast Variables to Display Movie Names
- [Activity] Find the Most Popular Superhero in a Social Graph
- Superhero Degrees of Separation: Introducing Breadth-First Search
- Superhero Degrees of Separation: Accumulators, and Implementing BFS in Spark
- Superhero Degrees of Separation: Review the code, and run it!
- Item-Based Collaborative Filtering in Spark, cache(), and persist()
- [Activity] Running the Similar Movies Script using Spark’s Cluster Manager
- [Exercise] Improve the Quality of Similar Movies
Running Spark on a Cluster
- [Activity] Using spark-submit to run Spark driver scripts
- [Activity] Packaging driver scripts with SBT
- Introducing Amazon Elastic MapReduce
- Creating Similar Movies from One Million Ratings on EMR
- Partitioning
- Best Practices for Running on a Cluster
- Troubleshooting, and Managing Dependencies
SparkSQL, DataFrames, and DataSets
- Introduction to SparkSQL
- [Activity] Using SparkSQL
- [Activity] Using DataFrames and DataSets
- [Activity] Using DataSets instead of RDD’s
Machine Learning with MLLib
- Introducing MLLib
- [Activity] Using MLLib to Produce Movie Recommendations
- [Activity] Linear Regression with MLLib
- [Activity] Using DataFrames with MLLib
Intro to Spark Streaming
- Spark Streaming Overview
- [Activity] Set up a Twitter Developer Account, and Stream Tweets
- Structured Streaming
Intro to GraphX
- GraphX, Pregel, and Breadth-First-Search with Pregel.
- [Activity] Superhero Degrees of Separation using GraphX
You Made It! Where to Go from Here.
- Learning More, and Career Tips
- Continue your Learning Journey!

[Activity] Running the Similar Movies Script using Spark’s Cluster Manager

[Activity] Running the Similar Movies Script using Spark’s Cluster Manager

Superhero Degrees of Separation: Review the code, and run it!

[Exercise] Improve the Quality of Similar Movies