AI & Machine Learning
Machine Learning, Part I
Understand the foundations of regression and classification and analyze real-life datasets.
01
Course overview
Understand the foundations of regression and classification and analyze real-life datasets.
02
What you will learn
This course focuses on teaching students the fundamentals of learning the relationships between an output (response variable) and a set of inputs (predictors) in specific problems. The two major sub-fields covered are regression, where the response is quantitative, and classification, where the response is qualitative with labels such as diseased or non-diseased.
Another objective is to equip students with the skills to analyze real-life datasets and design appropriate learning functions to minimize prediction errors. This will be accomplished through computer exercises and a course project.
The course begins with a review of probability theory, including advanced topics like multivariate analysis with an emphasis on multinormal distributions. Basics of Linear Algebra will be introduced, covering important topics such as the four spaces of a matrix, Singular Value Decomposition (SVD), and its connection to Principal Component Analysis (PCA).
The course follows a breadth-first approach, starting with Statistical Decision Theory and transitioning into Linear Models and Regression. Bayes' classifier will be derived and explained, along with its application to multinormal distributions. Various statistical concepts will be defined, including estimation, loss function, and risk minimization.
This course is just Part 1 that prepares students for Part 2, which will cover basic methods for regression and classification will be covered at different levels of detail, such as Neural Networks (NN), K-Nearest Neighbor, logistic regression, and Classification and Regression Trees (CART), exploring different assessment metrics, such as Receiver Operating Characteristics Curve (ROC) and Area Under the Curve (AUC), and Cross Validation (CV).
03
Lecture syllabus
Video titles and durations as published on Arabsera. The advertised lecture total may differ from the number of video entries, which can include introductions, tutorials, and split sessions.
Video titles and durations
- 01 Introduction to Pattern Recognition
- 02 Introduction to Statistical Decision Theory I (Regression)
- 03 Introduction to Statistical Decision Theory II (Classification)
- 04 Introduction to Statistical Decision Theory III (Classification)
- 05 Getting to "Learning" I (Regression)
- 06 Getting to "Learning" II (Classification)
- 07 Linear Models for Regression: Least Mean Square
- 08 Linear Models for Regression: Centered Model
- 09 Linear Models for Regression: Performance
- 10 Linear Models for Regression: Data Preprocessing and Transformation
- 11 Bias-Variance Decomposition
- 12-a Bias-Variance: illustration, model complexity, and model selection
- 12-b Bias-Variance: illustration, model complexity, and model selection
- 12-c Bias-Variance: illustration, model complexity, and model selection
- 13 Reducing Complexity: subset selection
- 14-a Reducing Complexity: regularization, shrinkage, ridge regression
- 14-b Reducing Complexity: regularization, shrinkage, ridge regression
- Appendix A: fast revision on basics of Probability
- Appendix B: fast revision on basics of Statistics
- Pattern Recognition: the big picture (1/3)
- Pattern Recognition: the big picture (2/3)
- Pattern Recognition: the big picture (3/3)
04
Teaching support (check your plan)
- Discussion groups
- TA-human texting for Q&A
- TA-GPT (coming soon)
05
Textbook
Hastie, T., Tibshirani, R., & Friedman, J. H. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics.
06
Certificate
Awarded after passing a brief sample exam, which you may attempt multiple times.