Welcome to Kaiyi Ji's Homepage
 |
|
About Me
I am an Assistant Professor in Computer Science and Engineering at the University at Buffalo. Previously, I was a Postdoctoral Researcher in Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor, and a Visiting Student Research Collaborator in Electrical Engineering at Princeton University. I received my Ph.D. in Electrical and Computer Engineering from The Ohio State University, supervised by Prof. Yingbin Liang, and my B.E. from the University of Science and Technology of China. I received the UB CSE Junior Faculty Research Award (2023), the NSF CAREER Award (2025), CSE Excellence in Research Award (2025) and SEAS Early Career Researcher of the Year Award (2026).
Research Interests
I am particularly interested in the theoretical and algorithmic foundations for optimization and machine learning. My current focus is on:
Optimization & theory for modern ML: parameter-efficient optimization, multi-objective and bilevel optimization.
Foundation of AI and ML: multi-task learning, continual learning and meta-learning.
Efficient foundation-model adaptation: PEFT/LoRA for multi-task and federated settings, parameter-efficient continual tuning, multi-objective alignment.
Applied side: methods validated in robotics, recommendation, and NLP.
Recent News!
[Publication] 05/2026 Two paper on Lower bounds for bilevel optimization and Backward knowledge transfer for LLMs have been accepted by ICML 2026. Congrats to Anushka on her first paper at a top ML conference!
[Award] 04/2026 I am honored to receive SEAS Early Career Researcher of the Year Award from the School of Engineering and Applied Sciences.
[Publication] 04/2026 Our paper, Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs <GitHub>, has been accepted to ACL 2026 Findings. Congrats to Hao!
This work clarifies an important detail in prior multi-LoRA sharing results: the similarity in A may largely stem from identical initialization rather than genuinely shared knowledge, offering a useful diagnostic for future method design. It further proposes a simple asymmetric design (sharing B) that improves task balance without degrading average performance, and extends this approach to federated fine-tuning.
[Library] 02/2026 Our open-source multi-task learning-to-rank library DeepMTL2R <paper> <code> with Amazon is now online.
[Publication] 01/2026 Our paper, Minimax Optimal Adversarial Reinforcement Learning, has been accepted at ICLR 2026. Congrats to Yudan!
[Award] 01/2026 My proposal on continual learning received a $15,000 award from the Research, Innovation and Economic Development Grant at UB. I sincerely thank UB for the support.
[Publication] 01/2026 Our paper, Provable Effects of Data Replay in Continual Learning: A Feature Learning Perspective, has been accepted at AISTATS 2026. Congrats to Meng!
[Service] 12/2025 I am serving on the Organization Program Committee for the International Conference on Bilevel Optimization (ICBO), which will take place on August 2–5, 2026, in Pittsburgh, USA. Please consider submitting your work through this link. Thanks!
[Award] 12/2025 I received the CSE Excellence in Research Award from the department. I am truly grateful to my fantastic students for their great contributions and to the department for its support.
[Manuscript] 11/2025 I wrote a paper on complexity lower bounds for bilevel optimization with first-order oracles. I hope it offers useful insights and helps stimulate further progress in this underexplored direction.
[Publication] 11/2025 Our paper ’Understanding Fine-tuning in Approximate Unlearning: A Theoretical Perspective’ has been accepted at TMLR 2025! Congrats to Meng.
[Award] 08/2025 Our paper, heretogeneous meta-learning, received Best Paper Candidate Award from IJCAI 2025 GLOW Workshop. Congrats to Zhaofeng!
[Service] 08/2025 I will serve as an Area Chair for ICLR 2026.
[Talk] 07/2025 I gave a talk on tuning-free bilevel optimization at ICCOPT 2025, Los Angeles, CA. Thanks for the invite from Prof. Shiqian Ma and Prof. Tong Zhang.
[Manuscript] 07/2025 A new paper, Scalable Task-Agnostic Prompt-based Continual Learning for LLMs <GitHub>, is now online.
[Publication] 05/2025 Our paper “Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy” has been formally accepted to The International Journal of Robotics Research (IJRR) — one of the most prestigious journals in robotics! This work marks the first successful use of bilevel optimization to develop a unified, interpretable, and self-supervised framework for robot autonomy. Read our <blog> and <Library> for a quick overview.
Recent Featured Works
Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs <Code>
Hao Ban and Kaiyi Ji.
Association for Computational Linguistics (ACL) 2026 Findings.
Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis <Code>
Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025.
Lower Complexity Bounds for Nonconvex-Strongly-Convex Bilevel Optimization with First-Order Oracles
Kaiyi Ji.
International Conference on Machine Learning (ICLM) 2026.
Understanding Forgetting in Continual Learning with Linear Regression
Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
International Conference on Machine Learning (ICML) 2024.
Selected Publications
Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy <Open-Source Library>
Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen,
Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao.
The International Journal of Robotics Research (IJRR), 2025.
Fair Resource Allocation in Multi-Task Learning <Code>
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.
Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
Peiyao Xiao, Hao Ban, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
Will Bilevel Optimizers Benefit from Loops
Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying
Conference on Neural Information Processing Systems (NeurIPS) 2022. (Spotlight)
Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.
Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
Kaiyi Ji, Junjie Yang, Yingbin Liang.
International Conference on Machine Learning (ICML) 2021.