📅 Course Schedule

📚 Course Description

This is a comprehensive, math-intensive machine learning course that bridges theoretical foundations with practical applications. The course covers core supervised learning algorithms including linear/logistic regression, support vector machines, neural networks, and ensemble methods like boosting. Students will dive deep into optimization techniques (gradient descent, Newton's method), convex analysis, kernel methods, dimensionality reduction (PCA), clustering, and modern topics like deep learning architectures (CNNs, RNNs, LSTMs) and online learning algorithms.

The course emphasizes rigorous mathematical understanding alongside hands-on implementation. Students will engage through programming assignments in Python/MATLAB, scribe detailed lecture notes using LaTeX, and work on a significant course project. Topics progress from fundamental concepts like loss functions and regularization to advanced methods including spectral clustering, Gaussian processes, and transformer architectures. The curriculum balances theoretical analysis (convergence proofs, mistake bounds, duality theory) with practical machine learning skills.

Upon completion, students will have a solid theoretical foundation in machine learning mathematics, practical experience implementing algorithms from scratch, and the ability to analyze and apply ML methods to real-world problems. Success requires a strong background in probability, statistics, and linear algebra, along with programming proficiency. This course prepares students for advanced ML research, industry roles requiring deep technical understanding, or pursuing research-level graduate studies in ML and AI.

⚠️ Prerequisites

Expect this to be a fairly math intensive course. Please familiarize yourself with the basics of Probability-Statistics (PS) and Linear-Algebra (LA). Recommended introductory lectures to check if you are comfortable with the basics:

• PS review:
  1. https://cs229.stanford.edu/section/cs229-prob.pdf
  2. https://www2.isye.gatech.edu/~sman/courses/6761/6761-1-ProbReview.pdf
  3. https://users.ssc.wisc.edu/~ctaber/410/statrev.pdf
• LA review:
  1. https://cs229.stanford.edu/section/cs229-linalg.pdf
  2. https://james-chuang.github.io/notes/linalg_review.pdf
  3. https://www.cs.cmu.edu/~jingx/docs/linearalgebra.pdf

Familiarity with LaTeX for scientific writing for scribing the lecture notes (only for MS students). You can learn the basics from A Simple Quickstart Guide. Many other online tutorials are available for beginners — feel free to explore and use whichever best suits your needs.

Programming assignments (in Python or Matlab). Be prepared to code: [Python ML Tutorials], [Google Colab] (many online tutorials available for beginners).

A strong grasp of the foundational material outlined above is expected of all students taking the course for credit. Insufficient preparation may adversely affect your ability to engage with the course content and perform successfully in assessments, which may impact your final grades.

📖 Resources

Class lecture will be based on, but not limited to, the following books:

• [ESL] The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman [Book website]
• [MLTM] Machine Learning by Tom Mitchell [Online copy]
• [PML] Probabilistic Machine Learning: An Introduction, by Kevin Murphy [Book website]
• [PLG] Prediction, Learning and Games by Nicolo Cesa-Bianchi and Gabor Lugosi, Cambridge University Press, 2006 [Local Copy from E1 245]
• [CvxO] Convex Optimization: Algorithms and Complexity by Sebastien Bubeck [Book Website]
• [PyAG] (Optional): Hands-On Machine Learning with Scikit-Learn & Tensorflow by Aurelien Geron [Online, Github]
• [PRML] (Optional): Pattern Recognition and Machine Learning by Christopher Bishop [Free copy]
• [CIML] (Optional): A Course in Machine Learning by Hal Daume III [Online copy], [Errata]
• [UML] (Optional): Understanding Machine Learning: From Theory to Algorithms by Shai Ben-David, Shai Shalev-Shwartz [Online copy]
• [OCO] (Optional) Introduction to Online Convex Optimization by Elad Hazan [Book website]

🎯 Course Logistics

• 📍 Location: Lecture Complex F6
• ⏰ Schedule: Tuesday & Thursday, 2:00 - 3:15 PM
• 🏛️ Office Hours: Thurdays 5:00–6:00 PM or by appointment
• 📧 Contact: Email or Piazza