Kaiyi Ji
Home
Publication
Teaching
Group
Service

Welcome to Kaiyi Ji's Homepage

alt text 

About Me

I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016. I received CSE Junior Faculty Research Award (2023) and NSF CAREER Award (2025).

Research

I am particularly interested in the intersection of optimization, machine learning, and deep learning, from both theoretical and algorithmic perspectives. My current focus is on:

  • Large-Scale Optimization: We design provably efficient optimization algorithms for machine (deep) learning tasks with strong empirical performance. Recent efforts focus on preconditioned methods for large language model (LLM) training, bilevel optimization for hierarchical tasks, and multi-objective optimization for tasks with multiple competing objectives.

  • Machine (Deep) Learning: We develop algorithms to tackle key challenges in modern machine learning problems. Our current interest lies in multi-task learning (such as conflict resolution, balancing, and scalability), life-long/continual learning (such as data selection, forgetting, memory efficiency) and meta-learning (such as task heterogeneity, convergence stability and computational cost). We also pursue theoretical foundations to understand the mechanisms, generalization behavior, and computational complexity of these methods.

  • Foundation Models: We explore computationally efficient and theoretically grounded approaches for training and fine-tuning LLMs in resource-constrained scenarios. Current directions include continual prompt tuning, computationally efficient low-rank adaptation, and training-free methods for LLMs.

  • Applications: We seek to ground our algorithmic designs in real-world conditions by actively collaborating with experts in robotics, ad recommendation, and natural language processing.

Recent News!

  • [Service] 08/2025 I will be serving as an Area Chair for ICLR 2026.

  • [Library] 07/2025 Our open source project DeepMTL2R led by Amazon is available at GitHub. DeepMTL2R is a deep learning framework for multi-task learning-to-rank tasks, integrating a wide range of existing preference-based and non-preference-based MTL algorithms.

    • DeepMTL2R: A Library for Deep Multi-task Learning to Rank. Chaosheng Dong, Peiyao Xiao, Kaiyi Ji and Aleix Martinez.

  • [Talk] 07/2025 I gave a talk on tuning-free bilevel optimization at ICCOPT 2025, Los Angeles, CA. Thanks for the invite from Prof. Shiqian Ma and Prof. Tong Zhang.

  • [Service] 07/2025 I will be serving as a reviewer for SODA 2026.

  • [Manuscript] 07/2025 Our new paper on Task-Agnostic Continual Prompt Tuning for LLM training is available online. We propose GRID by integrating a task-aware decoding mechanism that improves backward transfer by leveraging representative inputs, automatic task identification, and constrained decoding. It reduces forgotten tasks by up to 80% without sacrificing forward transfer performance. Check our code: GitHub.

  • [Award] 04/2025 Glad to receive NSF CAREER Award [news]. Thanks to my students!

  • [Manuscript] 02/2025 Our new multi-task learning (MTL) paper is out! We present LDC-MTL, a balanced MTL approach with O(1) time and memory complexity, offering both high accuracy and efficiency. It reduces loss discrepancy, minimizes gradient conflicts, and outperforms weight-swept linear scalarization through dynamic weight adjustment. Check out our paper and code. Please star us if you find it helpful!

  • [Manuscript] 02/2025 A new AI4Science manuscript on Space-Aware Crystal Prediction is available online. We explore reciprocal space to encode long-range interactions and leverage a mixture-of-experts (MoE) approach for multi-property prediction.

  • [Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).

  • [Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!

  • [Talk] 10/2024 I gave a talk on federated optimization at the INFORMS Annual Meeting in Seattle.

  • [Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!

Recent Featured Works

Selected Publications