Machine Learning

First Steps

• Scope, Terminology, Prediction, and Data
• Putting the Machine in Machine Learning
• Examples of Learning Systems
• Evaluating Learning Systems
• A Process for Building Learning Systems
• Assumptions and Reality of Learning
• About Our Setup
• The Need for Mathematical Language
• Our Software for Tackling Machine Learning
• Probability
• Linear Combinations, Weighted Sums, and Dot Products
• A Geometric View: Points in Space
• Notation and the Plus-One Trick
• Getting Groovy, Breaking the Straight-Jacket, and Nonlinearity
• NumPy versus “All the Maths”
• Floating-Point Issues
• EOC

Predicting: Getting started with Classification and Regression

• Classification Tasks
• A Simple Classification Dataset
• Training and Testing: Don’t Teach to the Test
• Evaluation: Grading the Exam
• Simple Classifier #1: Nearest Neighbors, Long Distance Relationships, and Assumptions
• Simple Classifier #2: Naive Bayes, Probability, and Broken Promises
• Simplistic Evaluation of Classifiers
• A Simple Regression Dataset
• Nearest-Neighbors Regression and Summary Statistics
• Linear Regression and Errors
• Optimization: Picking the Best Answer
• Simple Evaluation and Comparison of Regressors
• EOC

Evaluation

• Evaluation and Why Less Is More
• Terminology for Learning Phases
• Major Tom, There’s Something Wrong: Overfitting and Underfitting
• From Errors to Costs
• (Re)Sampling: Making More from Less
• Break-It-Down: Deconstructing Error into Bias and Variance
• Graphical Evaluation and Comparison
• Comparing Learners with Cross-Validation
• Baseline Classifiers
• Beyond Accuracy: Metrics for Classification
• ROC Curves
• Another Take on Multiclass: One-versus-One
• Precision-Recall Curves
• Cumulative Response and Lift Curves
• More Sophisticated Evaluation of Classifiers: Take Two
• Baseline Regressors
• Additional Measures for Regression
• Residual Plots
• A First Look at Standardization
• Evaluating Regressors in a More Sophisticated Way: Take Two
• EOC

More Methods and Fundamentals

• Revisiting Classification
• Decision Trees
• Support Vector Classifiers
• Logistic Regression
• Discriminant Analysis
• Assumptions, Biases, and Classifiers
• Comparison of Classifiers: Take Three
• Linear Regression in the Penalty Box: Regularization
• Support Vector Regression
• Piecewise Constant Regression
• Regression Trees
• Comparison of Regressors: Take Three
• Feature Engineering Terminology and Motivation
• Feature Selection and Data Reduction: Taking out the Trash
• Feature Scaling
• Discretization
• Categorical Coding
• Relationships and Interactions
• Target Manipulations
• Models, Parameters, Hyperparameters
• Tuning Hyperparameters
• Down the Recursive Rabbit Hole: Nested Cross-Validation
• Pipelines
• Pipelines and Tuning Together
• EOC

Adding Complexity

• Ensembles
• Voting Ensembles
• Bagging and Random Forests
• Boosting
• Comparing the Tree-Ensemble Methods
• Feature Selection
• Feature Construction with Kernels
• Principal Components Analysis: An Unsupervised Technique
• Working with Text
• Clustering
• Working with Images
• Optimization
• Linear Regression from Raw Materials
• Building Logistic Regression from Raw Materials
• SVM from Raw Materials
• Neural Networks
• Probabilistic Graphical Models
• EOC

Appendix A: mlwpy.py Listing