Time: Tue/Thu 9:30--11:00am. Place: Bell 224

Course Blog -- Updated often. Please subscribe to its RSS feed.

Computer Science and Engineering

Instructors: Hung Q. Ngo and Atri Rudra.

This is a year-long seminar on
several central topics in the general umbrella of *Computational
Learning Theory*. The topics are
chosen partly to fit the (research) interests of the instructors.
However, they were also chosen to follow two central themes.

- (Fall 2010) We explore two
foundational questions: "what is learning?" and "what is learnable?"
However, unlike the more experimental approach of statistical machine
learning, our focus is on formal theoretical model for the prediction
capability
**and**algorithmic efficiency, both in terms of time and space. The complexity emphasis is the hallmark of COLT. - (Spring 2011) We explore many interesting connections between learning (on-line learning, in particular), algorithmic complexity, game theory, and electronic markets.

- Hung Q.
Ngo ( hungngo [at] buffalo )
- Office hours: 9-10am, Mondays and Wednesdays, 238 Bell.

- Office hours: 9-10am, Mondays and Wednesdays, 238 Bell.
- Atri Rudra
( atri [at] buffalo )
- Office hours: by appointment

Basic knowledge of probability theory. (We assume that you have studied some introductory probability course/book before.)

Students are expected to participate in class, and make at least one presentation. Instructors will assign the topic and material to be presented. No A/F grade will be given, only S/U grades.

- Michael J. Kearns and Umesh V. Vazirani, "An Introduction to Computational Learning Theory", MIT Press.
- Vapnik, V. N. "The Nature of Statistical Learning Theory". Springer-Verlag New York, Inc.
- M. Anthony and N. Biggs. "Computational Learning Theory." Cambridge Univ. Press, 1992.
- O. Bousquet, S. Boucheron, and G. Lugosi, Introduction to Statistical Learning Theory. [ pdf ]
- N. Cristianini and J. Shawe-Taylor, Kernel Methods for Pattern Analysis, 2004.
- N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines (and other kernel-based learning methods), 2000, CUP.
- M. Anthony and P. Bartlett. Learning in Neural Networks : Theoretical Foundations. Cambridge University Press, 1999.
- Avrim Blum's COLT Tutorial at FOCS 2003. [ ppt slides ]
- Robert E. Schapire. "The boosting approach to machine learning: An overview." Nonlinear Estimation and Classification. Springer, 2003. [ pdf ]
- Valiant's original "Theory of the Learnable" paper. [ pdf ]
- Robert E. Schapire, Yoav Freund, Peter Bartlett and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5):1651-1686, 1998. [ pdf ]
- Prediction, learning, and games by Nicolò Cesa-Bianchi and Gábor Lugosi. Cambridge University Press, 2006
- Avrim Blum. "On-line algorithms in machine learning." In Dagstuhl Workshop on On-Line Algorithms, June, 1996. [ ps ]
- Chris Burges' SVM tutorial. [ pdf ]
- Shai Shalev-Shwartz and Yoram Singer, Tutorial on Theory and Applications of Online Learning, ICML 2008.
- Kivinen, Warmuth, Exponentiated Gradient versus Gradient Descent for Linear Predictors 1997. [ pdf ].
- Stephen Della Pietra, Vincent Della Pietra and John Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):380-393, April, 1997. [ ps ]