### CSE 736 - Markov Chains: Foundations and Applications

Spring 2003

**Instructor**: Dr. Hung
Q. Ngo

Office: 239 Bell Hall
Office Hours: Thursdays 1:00-3:00pm
Phone: 645-3180 x 160
Email: hungngo@cse.buffalo.edu

**Website:** http://www.cse.buffalo.edu/~hungngo/classes/MarkovChains_Spring03/

**Grading**: to be done on an S/N (or S/U) basis only.

**Time and place**: Mondays 2pm-3:20pm
and Tuesdays 7pm-8:20pm. Bell 242.

#### Description:

I shall spend roughly half of the semester presenting the foundations of Markov
Chains, both discrete and continuous. An introductory course on probability
theory is recommended, but not absolutely required. I shall provide reading
materials on basic probability theory.

At the second half, each member of the class presents a paper or a topic which
uses Markov chains. They should be in one of two areas: **queueing theory**
and **randomized algorithms**.

Also, each class member prepares scribe note for at least one lecture. The LATEX
template shall be provided. Part of the grading is based on how much effort
I have to spend modifying the scribe note. All notes shall be shared to the
class.

A list of recommended papers and topics shall also be provided.

#### Objectives:

- Have fun learning!
- Learn the foundations of stochastic processes and Markov Chains
- Learn their applications to queueing networks and randomized algorithms

#### Topics (tentative):

- A Brief Introduction to Linear Algebra
- Stochastic processes
- Poisson Processes and the Exponential Distribution
- Discrete time Markov Chains
- Continuous time Markov Chains
- Martingales

#### Course Load:

- About 50 pages of dense reading per week (i.e. no bed-time reading).
- Each student is expected to do a presentation at some point during the semester,
based on:
- A survey done by the student on a particular topic to be suggested by
the instructor or picked by the student but approved by the instructor.
- A (few) very dense mathematical paper.
- An open problem solved by the student.

#### Prerequisites:

- Find learning new things fun.
- Find learning mathematics fun.
- Find solving mathematical puzzles fun.
- Find the instructor funny.

- OK, the real deals are: rudimentary knowledge on linear algebra, algorithms,
probability theory. They are not entirely essential to follow things presented
in the seminars. Related background materials shall be provided in the forms
of small tutorials/notes. You'd have to read them though.