Seminar: Machine Learning and Cybersecurity

General Information

Instructor

Dr. Hongxin Hu

Homepage: https://cse.buffalo.edu/~hongxinh/

Office Hours: By Appointment

Time and Location: R 10:00 AM-11:30 PM, Davis 113A.

Overview

In this seminar class, we will discuss the use of machine learning, especially deep learning, for detecting and mitigating cyber threats arising in commercial systems and applications. We will also discuss security issues in machine learning (adversarial attacks and defenses on deep learning, backdoor attacks and defenses on deep learning, Security in Large Language Models including ChatGPT etc.). Our ability to identify the type of machine learning algorithms that are useful for specific security applications can help us improve our defenses against attacks such as credit card fraud, malware, and spam, and also anticipate the potential attack variants that may arise in the future. In addition to lectures, you'll participate in hands-on projects that will simulate a cyber threat and defense. You'll learn how to extract essential features, preprocess data and then identify a suitable suite of machine learning algorithms that can be used to detect and mitigate the cyber threat.

The main goal of the seminar is to help students understand the state of the art in a variety of topics in emerging machine learning and cybersecurity. As a secondary goal, students will learn how to read research papers and how to communicate technical material effectively.

The seminar is suitable for students who have strong interest in achine learning and cybersecurity and intent to pursue a career in the area, e.g., PhD students already working in ML and cybersecurity or MS students interested in pursuing a PhD or doing research in the field (in the form of independent studies and/or MS Thesis). One of the goals of this seminar is to identify, by the end of the semester, a set of open research problems on which students can work during the next semester, e.g., in the form of independent studies.

Tentative Schedule

Week	Date	Topic	Papers (Presenters)	Notes
1	Aug 28	Class Overview	N/A
2	Sept 4	Labor Day	N/A
2	Sept 11	Cybersecurity Overview I	N/A
3	Sept 18	Cybersecurity Overview II	N/A
4	Sept 25	Network Security Overview I	N/A
5	Oct 2	Network Security Overview II	N/A
6	Oct 9	Fall Break	N/A
6	Oct 16	Adversarial Machine Learning I	N/A	Ke Yan
7	Oct 23	Adversarial Machine Learning II & Labs	N/A	Ke Yan
10	Nov 6	Topic #2: Large Language Models for Vulnerability Detection and Repair I	Universal Fuzzing via Large Language Models (Rupam Patir) No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation (Pragathi Makkena)	Review #1 (for the paper assigned to you) is due.
9	Oct 30	Topic #1: Large Language Models for Reasoning	Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Sushanth Grandhi, Mamatha Yarramaneni) Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (Tanmay Gupta, Deepanshu)
11	Nov 13	Topic #2: Large Language Models for Vulnerability Detection and Repair II	Prompting Is All You Need: Automated Android Bug Replay with Large Language Models (Venkata Satvik Gogineni, Praveen Kumar Ramineni) Enhancing Automated Program Repair through Fine-tuning and Prompt Engineering (Mohit Sunil Gandhi, Kashyap Deveshbhai Soni)	The 1st lab is due: Lab Instruction Lab Colab
12	Nov 20	Topic #3: Large Language Models for Security Applications	Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content (Priya Pandharinath More, Siddhant Gupta) FlowTransformer: A Transformer Framework for Flow-based Network Intrusion Detection Systems (Muhammad Adil, Shubham Sunita Ambavale) DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models (Manikanta Kalyan Gokavarapu, Shubham Kulkarni)
13	Nov 27	Topic #4: Security in Large Language Models I	Universal and Transferable Adversarial Attacks on Aligned Language Models (Shijie Zhou) GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher (Saema Nadim, Prakhar Bhatnagar)	Review #2 (for any paper presented by someone else) is due.
14	Dec 4	Topic #4: Security in Large Language Models II	Do Anything Now: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models (Narasimha Prabhu, Manudeep Herle) On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective (Dhvani Mitesh Kothari, Shawna Saha)
15	Dec 11	Topic #5: Security in Large Vision-Language Models	Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models (Blessy Boby, Tushar Goyal) Prompt Stealing Attacks Against Text-to-Image Generation Models (Sai Kiran Janga)	The 2nd lab is due: Lab Instruction Lab Colab

Seminar Structure and Assignments

I will present material during the first 3 classes, followed by 2 lectures present by one of my PhD student Keyan Guo. Students then present selected papers during the remaining classes including two invited talks. A list of papers from top security and ML conferences (IEEE S&P - Oakland, USENIX Security, ACM CCS, NDSS, NeurIPS, ICML, etc.) will be provided for each topic. All students are encouraged to read all papers, submit reviews for a subset of them, and participate in discussions in class.

The course includes the following assignments:

Paper reading: if you are not familar with reading research papers, I recommend reading "How to Read a Paper" by S. Keshav.
Paper presentation: each student will present around 1 research paper at a class. You can find a set of recommendations on how to give a good presentation here.
Paper reviews: you will write reviews for 2 papers (you will choose which ones). The template of a review can be download here. You can find a set of recommendations on how to write a good reviews here.
Labs: 3-credit students will finish 2 labs.

Course Cridits

You can take this seminar for either 1 credit or 3 credits. With 1 credit, your responsibility is to read papers, write reviews, and present papers to the class. With 3 credits, you will also finish labs.