Kaiyi Ji
Home
Publication
Teaching
Group
Service

Welcome to Kaiyi Ji's Homepage

alt text 

About Me

I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022, working with Prof. Lei Ying. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016. I received CSE Junior Faculty Research Award (2023) and NSF CAREER Award (2025).

Research

I am particularly interested in the intersection of optimization, machine learning, and deep learning, from both theoretical and algorithmic perspectives. My current focus is on:

  • Large-Scale Optimization: We design provably efficient optimization algorithms for machine (deep) learning tasks with strong empirical performance. Recent efforts focus on preconditioned methods for large language model (LLM) training, bilevel optimization for hierarchical tasks, and multi-objective optimization for tasks with multiple competing objectives.

  • Machine (Deep) Learning: We develop algorithms to tackle key challenges in modern machine learning problems. Our current interest lies in multi-task learning (such as conflict resolution, balancing, and scalability) and life-long/continual learning (such as data selection, forgetting, computing cost). We also pursue theoretical foundations to understand the mechanisms, generalization behavior, and computational complexity of these methods.

  • Foundation Models: We explore efficient and theoretically grounded approaches for training LLMs under resource constraints. Current directions include continual prompt tuning, structured low-rank adaptation, and multi-objective decoding for LLMs.

  • Applications: We seek to ground our algorithmic designs in real-world conditions by actively collaborating with experts in robotics, ad recommendation, and natural language processing.

Recent News!

  • [Talk] 07/2025 I gave a talk on tuning-free bilevel optimization at ICCOPT 2025, Los Angeles, CA. Thanks for the invite from Prof. Shiqian Ma and Prof. Tong Zhang.

  • [Service] 07/2025 I will be serving as a reviewer for SODA 2026.

  • [Manuscript] 07/2025 Our new paper on Task-Agnostic Continual Prompt Tuning for LLM training is available online. We propose GRID by integrating a task-aware decoding mechanism that improves backward transfer by leveraging representative inputs, automatic task identification, and constrained decoding. It reduces forgotten tasks by up to 80% without sacrificing forward transfer performance. Check our code: GitHub.

  • [Award] 04/2025 Glad to receive NSF CAREER Award [news]. Thanks to my students!

  • [Talk] 03/2025 I will organize a special session entitled “Recent advances in optimization for machine learning and networking” at Asilomar 2025. Please join us if you attend Asilomar at CA.

  • [Manuscript] 02/2025 Our new multi-task learning (MTL) paper is out! We present LDC-MTL, a balanced MTL approach with O(1) time and memory complexity, offering both high accuracy and efficiency. It reduces loss discrepancy, minimizes gradient conflicts, and outperforms weight-swept linear scalarization through dynamic weight adjustment. Check out our paper and code. Please star us if you find it helpful!

  • [Manuscript] 02/2025 A new AI4Science manuscript on Space-Aware Crystal Prediction is available online. We explore reciprocal space to encode long-range interactions and leverage a mixture-of-experts (MoE) approach for multi-property prediction.

  • [Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).

  • [Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!

  • [Talk] 10/2024 I gave a talk on federated optimization at the INFORMS Annual Meeting in Seattle.

  • [Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!

Recent Featured Works

Selected Publications