A/B Test Gotchas

November 1, 2019

Back to: Machine Learning, Data Science, and Deep Learning with Python

No HTML was returned.

Determining How Long to Run an Experiment

Getting Started
- Introduction
- Installation: Getting Started
- [Activity] WINDOWS: Installing and Using Anaconda & Course Materials
- [Activity] MAC: Installing and Using Anaconda & Course Materials
- [Activity] LINUX: Installing and Using Anaconda & Course Materials
- Python Basics, Part 1 [Optional]
- [Activity] Python Basics, Part 2 [Optional]
- [Activity] Python Basics, Part 3 [Optional]
- [Activity] Python Basics, Part 4 [Optional]
- Introducing the Pandas Library [Optional]
Statistics and Probability Refresher, and Python Practice
- Types of Data
- Mean, Median, Mode
- [Activity] Using mean, median, and mode in Python
- [Activity] Variation and Standard Deviation
- Probability Density Function; Probability Mass Function
- Common Data Distributions
- [Activity] Percentiles and Moments
- [Activity] A Crash Course in matplotlib
- [Activity] Advanced Visualization with Seaborn
- [Activity] Covariance and Correlation
- [Exercise] Conditional Probability
- Exercise Solution: Conditional Probability
- Bayes’ Theorem
Predictive Models
- [Activity] Linear Regression
- [Activity] Polynomial Regression
- [Activity] Multiple Regression, and Predicting Car Prices
- Multi-Level Models
Machine Learning with Python
- Supervised vs. Unsupervised Learning, and Train/Test
- [Activity] Using Train/Test to Prevent Overfitting
- Bayesian Methods: Concepts
- [Activity] Implementing a Spam Classifier with Naive Bayes
- K-Means Clustering
- [Activity] Clustering People Based on Income and Age
- Measuring Entropy
- [Activity] WINDOWS: Installing GraphViz
- [Activity] MAC: Installing GraphViz
- [Activity] LINUX: Installing GraphViz
- Decision Trees: Concepts
- [Activity] Decision Trees: Predicting Hiring Decisions
- Ensemble Learning
- Support Vector Machines (SVM) Overview
- [Activity] Using SVM to Cluster People using scikit-learn
Recommender Systems
- User-Based Collaborative Filtering
- Item-Based Collaborative Filtering
- [Activity] Finding Movie Similarities
- [Activity] Improving the Results of Movie Similarities
- [Activity] Making Movie Recommendations to People
- [Exercise] Improve the Recommender’s Results
More Data Mining and Machine Learning Techniques
- K-Nearest-Neighbors: Concepts
- [Activity] Using KNN to Predict a Rating for a Movie
- Dimensionality Reduction; Principal Component Analysis (PCA)
- [Activity] PCA Example with the Iris Data Set
- Data Warehousing Overview: ETL and ELT
- Reinforcement Learning
- [Activity] Reinforcement Learning and Q-Learning with Gym
- Understanding a Confusion Matrix
- Measuring Classifiers (Precision, Recall, F1, AUC, ROC)
Dealing with Real-World Data
- Bias / Variance Tradeoff
- [Activity] K-Fold Cross-Validation
- Data Cleaning and Normalization
- [Activity] Cleaning Web Log Data
- Normalizing Numerical Data
- [Activity] Detecting Outliers
- Feature Engineering and the Curse of Dimensionality
- Imputation Techniques for Missing Data
- Handling Unbalanced Data: Oversampling, Undersampling, and SMOTE
- Binning, Transforming, Encoding, Scaling, and Shuffling
Apache Spark: Machine Learning on Big Data
- Warning about Java 11!
- Spark Installation Notes for MacOS and Linux Users
- [Activity] Installing Spark – Part 1
- [Activity] Installing Spark – Part 2
- Spark Introduction
- Spark and the Resilient Distributed Dataset
- Introducing MLLib
- Introduction to Decision Trees in Spark
- [Activity] K-Means Clustering in Spark
- TF / IDF
- [Activity] Searching Wikipedia with Spark
- [Activity] Using the Spark 2.x DataFrame API for MLLib
Experimental Design / ML in the Real World
- Deploying Models to Real-Time Systems
- A/B Testing Concepts
- T-Tests and P-Values
- [Activity] Hands-on With T-Tests
- Determining How Long to Run an Experiment
- A/B Test Gotchas
Deep Learning and Neural Networks
- Deep Learning Pre-Requisites
- The History of Artificial Neural Networks
- [Activity] Deep Learning in the Tensorflow Playground
- Deep Learning Details
- Introducing Tensorflow
- [Activity] Using Tensorflow, Part 1
- [Activity] Using Tensorflow, Part 2
- [Activity] Introducing Keras
- [Activity] Using Keras to Predict Political Affiliations
- Convolutional Neural Networks (CNN’s)
- [Activity] Using CNN’s for Handwriting Recognition
- Recurrent Neural Networks (RNN’s)
- [Activity] Using a RNN for Sentiment Analysis
- [Activity] Transfer Learning
- Tuning Neural Networks: Learning Rate and Batch Size Hyperparameters
- Deep Learning Regularization with Dropout and Early Stopping
- The Ethics of Deep Learning
- Learning More about Deep Learning
Final Project
- Your Final Project Assignment
- Final Project Review
You Made It!
- More to Explore
- Continue Your Learning!

A/B Test Gotchas

A/B Test Gotchas

Determining How Long to Run an Experiment

Deep Learning Pre-Requisites