Statistical Learning and Graphical Models

ECE, CS 4501/6501: Statistical Learning and Graphical Models (Fall 2020)

Welcome to statistical learning and graphical models! This course is focused on the foundations of estimation theory, statistical learning, probabilistic graphical models, and relevant computational algorithms. The course maintains a probabilistic view of these topics, aiming to represent complex systems via probabilistic models and using these models to draw conclusions about hidden values from observation and data.

We will start with the fundamentals of inference, including frequentist and Bayesian estimation methods, and explore these further in the context of regression and classification problems. We will then turn our attention to probabilistic graphical models, which provide a flexible mechanism for representing statistical relationships between variables and processes. Finally, we will study computational methods for inference and learning, enabling us to analyze, interpret, and explain patterns in complex data.

Pre-requisites: Fluency in basic probability (e.g., APMA 3100) is needed for the course as is familiarity with linear algebra. For probability, you should be already comfortable with 70-80% of these Probability Review Notes. The programming exercises are based on Python.

Tentative Grading Scheme: HW/Labs = 50%; Quizzes/In-class activities = 20%; Midterm Exam = 20%; Project = 10%

Notes (from previous offering):

  1. Review of Probability
  2. Probability, Inference, and Learning
  3. Frequentist Parameter Estimation (partial *)
  4. Bayesian Parameter Estimation
  5. Multivariate random variables
  6. Linear Regression
  7. Linear Classification
  8. Expectation-Maximization *
  9. Basics of Graphical Models
  10. Independence in Graphical Models
  11. Parameter Estimation in Graphical Models
  12. Inference in Graphical Models
  13. Inference in Hidden Markov Models
  14. Factor Graphs and Sum/Max-product Algorithms **
  15. Markov Chains
  16. Sampling Methods
  17. Appendix

*: material only for graduate students. **: optional material for completeness (not part of the course)
Other Material: