Introduction: A Brief Tour of Concepts, Techniques, and Applications
(5 weeks)
 [ pdf ]
Discrete Probability Space:
Events, The Probabilistic Method, and the Union Bound
 [ pdf ]
Expectation: definition, linearity of expectation,
decomposition into indicator variables, the argument from expectation.
 [ pdf ]
(Conditional) Probability and Expectation.
Independence and Conditional Probability,
Random Variables,
Expectation and Conditional Expectation,
Law of Total Probability,
Law of Total Expectation,
Randomized Algorithms,
Derandomization Using Conditional Expectation.

[ pdf ]
The Second Moment:
Sampling,
(Co)Variance,
Moments and Deviation,
the Second Moment Method,
Basic Concentration/Tail Inequalities (Markov, Chebyshev, Chernoff).

[ pdf ]
Tail and Concentration Inequalities:
Markov, Chebyshev, Chernoff, ChernoffHoefding.
The Bernstein moment generating function technique.
Computational Learning Theory (10 weeks)
 [ pdf ] Introduction to Learning Theory, The Consistency Model, PAC learning
[ presentation candidate ] [ pdf ] Pitt, L. and Valiant, L. G. 1988. Computational limitations on learning from examples. J. ACM 35, 4 (Oct. 1988), 965984.
 [ pdf ]
Sample complexity and Occam's razor. Sample complexity for finite
hypothesis classes.
 [ pdf ] VCdimension and SauerShelah Lemma
 [ pdf ] Sample complexity for
infinite hypothesis classes.
VapnikChervonenkis theorem. The double sampling trick.
Lower bound on the sample complexity through VCdimension
 [ pdf ] Dealing with Noises.
Inconsistent Hypothesis Model.
Empirical error and Generalization error.
Empirical loss minimization strategy.
Estimation error and approximation error.
Uniform convergence theorem.
 [ pdf ] Rademacher complexity and the uniform convergence theorem.
 [ pdf ] Weak and Strong PAClearning. Boosting and AdaBoost, training error bound, generalization error bound: naive and marginbased. General margin bound.
 A brief introduction to optimization.
 Support Vector Machines.
 [ No time for this ] Online learning. The mistakebound model. Learning from expert advices.
WMA and RWMA. The Hedging algorithm.