Kaiyi Ji
Home
Publication
Teaching
Group
Service

Welcome to Kaiyi Ji's Homepage

alt text 

About Me

I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016. I received CSE Junior Faculty Research Award (2023) and NSF CAREER Award (2025).

Research

I am particularly interested in the intersection of optimization, machine learning, and deep learning, from both theoretical and algorithmic perspectives. My current focus is on:

  • Large-Scale Optimization: We design provably efficient optimization algorithms for machine (deep) learning tasks with strong empirical performance. Recent efforts focus on parameter-efficient optimization for large language model (LLM) training, bilevel optimization for hierarchical tasks, and multi-objective optimization for tasks with multiple possibly competing objectives.

  • Machine (Deep) Learning: We develop algorithms to tackle key challenges in modern machine learning problems. Our current interest lies in multi-task learning (such as conflict resolution, balancing, and scalability), life-long/continual learning (such as data selection, forgetting, memory efficiency) and meta-learning (such as task heterogeneity, convergence stability and computational cost). We also pursue theoretical foundations to understand the mechanisms, generalization behavior, and computational complexity of these methods.

  • Foundation Models: We explore computationally efficient and theoretically grounded approaches for training and fine-tuning LLMs in resource-constrained scenarios. Current directions include continual prompt tuning, computationally efficient low-rank adaptation, and training-free methods for LLMs.

  • Applications: We seek to ground our algorithmic designs in real-world conditions by collaborating with experts in robotics, ad recommendation, and natural language processing.

Recent News!

  • [Service] 08/2025 I will be serving as an Area Chair for ICLR 2026.

  • [Library] 07/2025 Our open source project DeepMTL2R led by Amazon is available at GitHub. DeepMTL2R is a deep learning framework for multi-task learning-to-rank tasks, integrating a wide range of existing preference-based and non-preference-based MTL algorithms.

    • DeepMTL2R: A Library for Deep Multi-task Learning to Rank. Chaosheng Dong, Peiyao Xiao, Kaiyi Ji and Aleix Martinez.

  • [Talk] 07/2025 I gave a talk on tuning-free bilevel optimization at ICCOPT 2025, Los Angeles, CA. Thanks for the invite from Prof. Shiqian Ma and Prof. Tong Zhang.

  • [Service] 07/2025 I will be serving as a reviewer for SODA 2026.

  • [Manuscript] 07/2025 Our new paper on Task-Agnostic Continual Prompt Tuning for LLM training is available online. We propose GRID by integrating a task-aware decoding mechanism that improves backward transfer by leveraging representative inputs, automatic task identification, and constrained decoding. It reduces forgotten tasks by up to 80% without sacrificing forward transfer performance. Check our code: GitHub.

  • [Award] 04/2025 Glad to receive NSF CAREER Award [news]. Thanks to my students!

  • [Manuscript] 02/2025 Our new multi-task learning (MTL) paper is out! We present LDC-MTL, a balanced MTL approach with O(1) time and memory complexity, offering both high accuracy and efficiency. It reduces loss discrepancy, minimizes gradient conflicts, and outperforms weight-swept linear scalarization through dynamic weight adjustment. Check out our paper and code. Please star us if you find it helpful!

  • [Manuscript] 02/2025 A new AI4Science manuscript on Space-Aware Crystal Prediction is available online. We explore reciprocal space to encode long-range interactions and leverage a mixture-of-experts (MoE) approach for multi-property prediction.

  • [Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).

  • [Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!

  • [Talk] 10/2024 I gave a talk on federated optimization at the INFORMS Annual Meeting in Seattle.

  • [Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!

Recent Featured Works

Selected Publications