Welcome to Kaiyi Ji's Homepage

Kaiyi Ji (吉凯意) Assistant Professor Department of Computer Science and Engineering University at Buffalo, The State University of New York Address: 338G Davis Hall Email: kaiyiji at buffalo.edu [Google Scholar]

About Me

I am an Assistant Professor in Computer Science and Engineering at the University at Buffalo. Previously, I was a Postdoctoral Researcher in Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor, and a Visiting Student Research Collaborator in Electrical Engineering at Princeton University. I received my Ph.D. in Electrical and Computer Engineering from The Ohio State University and my B.E. from the University of Science and Technology of China. I received the UB CSE Junior Faculty Research Award (2023) and the NSF CAREER Award (2025).

Research Interests

I am particularly interested in the theoretical and algorithmic foundations for optimization, machine learning, and deep learning. My current focus is on:

• Optimization & theory for modern ML: parameter-efficient optimization, multi-objective and bilevel optimization.

• Foundation of AI and ML: multi-task learning, continual learning and meta-learning.

• Efficient foundation-model adaptation: PEFT/LoRA for multi-task and federated settings, continual prompt tuning, training-free (multi-objective) decoding.

• Applied side: methods validated in robotics, recommendation, and NLP.

Recent News!

• [Manuscript] 10/2025 A new paper, Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs <GitHub>, is now online. This work revisits how LoRA modules share parameters across tasks and reveals several surprising findings that inspire new algorithms for both multi-task and federated fine-tuning.

• [Award] 08/2025 Our paper, heretogeneous meta-learning, received Best Paper Candidate Award from IJCAI 2025 GLOW Workshop. Congrats to Zhaofeng!

• [Service] 08/2025 I will serve as an Area Chair for ICLR 2026.

• [Library] 07/2025 Our open-source multi-task learning-to-rank library DeepMTL2R led by Amazon is now online.

• [Talk] 07/2025 I gave a talk on tuning-free bilevel optimization at ICCOPT 2025, Los Angeles, CA. Thanks for the invite from Prof. Shiqian Ma and Prof. Tong Zhang.

• [Service] 07/2025 I served as a reviewer for SODA 2026.

• [Manuscript] 07/2025 A new paper, Scalable Task-Agnostic Prompt-based Continual Learning for LLMs <GitHub>, is now online.

• [Publication] 05/2025 Our paper “Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy” has been formally accepted to The International Journal of Robotics Research (IJRR) — one of the most prestigious journals in robotics! This work marks the first successful use of bilevel optimization to develop a unified, interpretable, and self-supervised framework for robot autonomy. Read our <blog> and <Library> for a quick overview.

• [Award] 04/2025 Glad to receive NSF CAREER Award [news].

• [Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL.

Recent Featured Works

• Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs <Code>
  Hao Ban and Kaiyi Ji, Preprint, 2025.

• Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis <Code>
  Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma, Kaiyi Ji.
  International Conference on Learning Representations (ICLR) 2025.

• MGDA Converges under Generalized Smoothness, Provably
  Qi Zhang, Peiyao Xiao, Shaofeng Zou, Kaiyi Ji.
  International Conference on Learning Representations (ICLR) 2025.

• Understanding Forgetting in Continual Learning with Linear Regression
  Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
  International Conference on Machine Learning (ICML) 2024.

Selected Publications

• Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy <Open-Source Library>
  Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen,
  Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao.
  The International Journal of Robotics Research (IJRR), 2025.

• Fair Resource Allocation in Multi-Task Learning <Code>
  Hao Ban and Kaiyi Ji.
  International Conference on Machine Learning (ICML) 2024.

• Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
  Peiyao Xiao, Hao Ban, Kaiyi Ji.
  Conference on Neural Information Processing Systems (NeurIPS) 2023.

• Will Bilevel Optimizers Benefit from Loops
  Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying
  Conference on Neural Information Processing Systems (NeurIPS) 2022. (Spotlight)

• Lower Bounds and Accelerated Algorithms for Bilevel Optimization
  Kaiyi Ji, Yingbin Liang
  Journal of Machine Learning Research (JMLR) 2022.

• Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
  Kaiyi Ji, Junjie Yang, Yingbin Liang.
  International Conference on Machine Learning (ICML) 2021.