Lecture 5 - Overfitting, underfitting, Generalization and Regularization - Machine Learning Course
Machine Learning course taught by Dr. Mohamed-Rafik Bouguelia, at Halmstad University, Sweden.
This lecture of the machine learning course presents the problem of overfitting in machine learning, and how to avoid it and achieve a good generalization. The lecture first explains what is overfitting through classification and regression examples. It explains the relation between model complexity and underfitting (high bias, low variance) and overfitting (low bias, high variance).
Then, it explains why and when overfitting can occur (not enough training examples, a large number of features, model parameters etc.).
The lecture then presents how to estimate the generalization error using k-fold cross-validation, and how to address the problem of overfitting by: (1) fine-tuning the hyper-parameters of the model, (2) features selection, (3) using ensemble methods, and (4) using regularization. Then, the lecture presents a regularized linear regression (Ridge Regression and LASSO Regression), as well as a regularized logistic regression (for classification).