Kaiyi Ji
Home
Publication
Teaching
Group
Awards
Service

Welcome to Kaiyi Ji's Homepage

alt text 

About Me

I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016. I received CSE Junior Faculty Research Award (2023) and NSF CAREER Award (2025).

Research

I am particularly interested in the theoretical and algorithmic foundations for optimization, machine learning, and deep learning. My current focus is on:

  • Large-scale optimization. We design provably efficient optimization methods for machine (deep) learning with strong empirical performance. Recent efforts focus on parameter-efficient optimization for LLM fine-tuning, bilevel optimization for tasks with hierarchical structures, and multi-objective optimization for tasks with multiple possibly competing objectives.

  • Foundations for Machine (deep) learning. We design algorithms to tackle fundamental challenges in modern machine learning. Our recent efforts span multi-task learning (conflict resolution, balancing, scalability), continual learning (data selection, forgetting, memory efficiency), and meta-learning (task heterogeneity, convergence stability, computational cost). We also pursue theoretical advances to characterize the mechanisms, generalization behavior, and computational complexity of these methods.

  • Foundation models. We aim to develop efficient and theoretically grounded training and fine-tuning of large foundation models in resource-constrained settings, including continual prompt tuning, computationally efficient low-rank adaptation, distributed parameter-efficient fine-tuning, and training-free methods (decoding and prompt-based).

  • Applications. We aim to anchor our algorithmic designs in real-world settings through collaborations in robotics, recommendation, and natural language processing.

Recent News!

  • [Service] 08/2025 I will be serving as an Area Chair for ICLR 2026.

  • [Library] 07/2025 Our open source project DeepMTL2R led by Amazon is available at GitHub. DeepMTL2R is a deep learning framework for multi-task learning-to-rank tasks, integrating a wide range of existing preference-based and non-preference-based MTL algorithms.

    • DeepMTL2R: A Library for Deep Multi-task Learning to Rank. Chaosheng Dong, Peiyao Xiao, Kaiyi Ji and Aleix Martinez.

  • [Talk] 07/2025 I gave a talk on tuning-free bilevel optimization at ICCOPT 2025, Los Angeles, CA. Thanks for the invite from Prof. Shiqian Ma and Prof. Tong Zhang.

  • [Service] 07/2025 I will be serving as a reviewer for SODA 2026.

  • [Manuscript] 07/2025 Our new paper on Task-Agnostic Continual Prompt Tuning for LLM training is available online. We propose GRID by integrating a task-aware decoding mechanism that improves backward transfer by leveraging representative inputs, automatic task identification, and constrained decoding. It reduces forgotten tasks by up to 80% without sacrificing forward transfer performance. Check our code: GitHub.

  • [Award] 04/2025 Glad to receive NSF CAREER Award [news]. Thanks to my students!

  • [Manuscript] 02/2025 Our new multi-task learning (MTL) paper is out! We present LDC-MTL, a balanced MTL approach with O(1) time and memory complexity, offering both high accuracy and efficiency. It reduces loss discrepancy, minimizes gradient conflicts, and outperforms weight-swept linear scalarization through dynamic weight adjustment. Check out our paper and code. Please star us if you find it helpful!

  • [Manuscript] 02/2025 A new AI4Science manuscript on Space-Aware Crystal Prediction is available online. We explore reciprocal space to encode long-range interactions and leverage a mixture-of-experts (MoE) approach for multi-property prediction.

  • [Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).

  • [Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!

  • [Talk] 10/2024 I gave a talk on federated optimization at the INFORMS Annual Meeting in Seattle.

  • [Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!

Recent Featured Works

Selected Publications