MACHINE LEARNING – CS671

SPRING 2013

 

Prof. Dan Simovici, Office hours: M 2:00-4:00, W 2:00 -3:00; other time can be arranged individually.

Machine Learning is a foundational discipline for data mining and artificial intelligence grounded in probability theory and linear algebra.  This course is intended to develop a context for placing various algorithms  and techniques developed in subsequent applied courses, especially in data mining.

Topics Covered in the Course

Concepts, Hypotheses, Learning Algorithms (one week)

The Vapnik-Chervonenkis Dimension (2 weeks)

Boolean Formulas and Representation of Hypothesis Spaces  ( one week)

Probabilistic Learning (3 weeks)

Complexity of Learning Algorithms (3 weeks)

Linear Threshold Networks (1 week)

Parametric Training Methods(2 weeks)

Support Vector Machines (2 weeks)

Students will be evaluated based on their homework ,  a take-home midterm and a take-home final examination.   All students are invited to attend the data mining seminar (Mondays at 5:30, starting February 4th  2013).

The main reference for the course are the lectures.  Slides used in the lectures and other handouts will be posted below.  In addition we recommend:

1.        M. Anthony and N. Biggs: Computational Learning Theory, Cambridge University Press, Cambridge, UK, 1997

2.        M. J. Kearns and U. V. Vazirani: An Introduction to Computational Learning Theory, The MIT Press, Cambridge, MA, 1997

HANDOUTS:

  • Handout 1
  • Vapnik-Chervonenkis 2a
  • Vapnik-Chervonenkis 2b
  • PAC0
  • PAC1
  • PAC2
  • PAC3
  • Support Vector Machines 0
  • Convex Sets I (revised)
  • Convex Sets II
  • Optimization 0
  • Optimization 1 (first revision)
  • Duality
  • Perceptrons
  • Linear Classifiers
  • Fisher Linear Discriminant
  • Principal Component Analysis
  • Biplots
  • Scaling

    SUPPLEMENTARY HANDOUTS:

  • Hoeffding inequalities
  • Listing monomials in lexicographic order 2a
  • Optimization and Rayleigh Quotients
  • Matrix Norms
  • Singular Value Decompositions

     

    HOMEWORK:

  • Homework 1
  • Homework 2
  • Homework 3
  • Homework 4