Foundations of Statistical Machine Learning (ECE 246)
The goal of this class is to introduce the students to the foundations of statistical machine learning. We start with an overview of several widely used learning algorithms including logistic and linear regression, kernel methods and SVM, ensemble learning methods, decision trees and nearest neighbor classifiers. We then build the connections to information theory through PAC learning, stability, bias-complexity trade-off, structural risk minimization, MDL, and universal learning. We give an introduction to representation learning with topics including unsupervised learning, clustering, (non-linear) dimensionality reduction, sketching, parametric distribution estimation including Gaussian mixtures, Expectation Maximization, non-parametric distribution estimation, property testing and neural networks focussed on distribution sampling (VAEs, GANs).
This course was taught at UCLA in Spring 2018-19, and Winter 2019-20.