CSE 676 A: Deep Learning

Course description. Deep Learning algorithms learn multi-level representations of data, with each level explaining the data in a hierarchical manner. Such algorithms have been effective at uncovering underlying structure in data, e.g., features to discriminate between classes. They have been successful in many artificial intelligence problems including image classification, speech recognition and natural language processing. The course, which will be taught through lectures and projects, will cover the underlying theory, the range of applications to which it has been applied, and learning from very large data sets. The course will cover connectionist architectures commonly associated with deep learning, e.g., basic neural networks, convolutional neural networks, and recurrent neural networks.

Course objective and learning outcomes. This course is to help students understand basic components in deep learning including loss functions, optimization, various neural network architectures, modern deep learning paradigms, and further learn how to use them in practical problems such as emotion recognition from facial expressions, fake news detection, language translation service, audio signal denoising. Some other skills such as mathematical analysis, presentation, report writing, programming, etc.

Session: [A], syllabus can be found in click this link

Time: Tuesday and Thursday 12:30PM - 1:50PM

Logistics information: (Office hours, location, TA information): can be found in syllabus

Others: class recordings and slides are available at UB Learns.

Suggested Readings

• Pattern Classification, David G. Stork, Peter E. Hart, and Richard O. Duda

• Pattern Recognition and Machine Learning, Christopher Bishop

• Attention Is All You Need Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion

• Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin

• Deep Learning, Goodfellow Ian, Yoshua Bengio, and Aaron Courville

Class outline:

1/25, 1/30, 2/1, 2/6

• Math and Linear Regression

2/8, 2/13, 2/15, 2/20

• Softmax Regression, MLP and Convex Optimization

2/22, 2/27, 2/29, 3/5, 3/7

• Convolutional Neural Networks

3/12

• In-person Midterm (Coverage on all previous lectures)

3/14, 3/26, 3/28,4/2

• Recurrent Neural Networks and Transformer

4/14, 4/9, 4/11,4/16

• Continual Learning and Meta-Learning

4/18, 4/23

• Bilevel Optimization and Applications

4/25, 4/30,5/2,5/7

• Project Presentation

5/14 at Academ 322

• Final exam (Coverage on all lectures after midterm)