Welcome to Kaiyi Ji's Homepage
 |
|
About Me
I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022, working with Prof. Lei Ying. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016. I received CSE Junior Faculty Research Award (2023) and NSF CAREER Award (2025).
Prospective students: I am now looking for PhD and Intern students with rich math and coding experiences in LLMs (optimization, alignment, prompt tuning, etc). Please send me an email with your CV and Transcript.
Research
My current research interests include:
Large-scale optimization: bilevel optimization, multi-objective optimization, and optimization for LLMs.
Machine Learning and deep learning: multi-task learning, model merging, continual learning and AI4Science
Foundation models: progressive prompts, parameter-efficient fine-tuning, multi-objective LLMs
Recent News!
[Publication] 06/2025 Our following paper on multi-task learning has been accepted to ICCV 2025! Congratulations to my students Hao and Gokul!
SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation. Hao Ban, Gokul Ram Subramani and Kaiyi Ji.
[Publication] 06/2025 Our paper “Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning” has been accepted to IEEE Transactions on Information Theory! Congratulations to my students and other fantastic collaborators!
[Publication] 05/2025 Our paper “Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy” has been formaly accepted to The International Journal of Robotics Research (IJRR) — one of the most prestigious journals in robotics! This work marks the first successful use of bilevel optimization to develop a unified, interpretable, and self-supervised framework for robot autonomy. Congratulations to the team! For a quick overview, visit blog and Library.
[Talk] 03/2025 I will organize a special session entitled “Recent advances in optimization for machine learning and networking” at Asilomar 2025. Please join us if you attend Asilomar at CA.
[Manuscript] 02/2025 Our new multi-task learning (MTL) paper is out! We present LDC-MTL, a balanced MTL approach with O(1) time and memory complexity, offering both high accuracy and efficiency. It reduces loss discrepancy, minimizes gradient conflicts, and outperforms weight-swept linear scalarization through dynamic weight adjustment. Check out the code: GitHub. Please star us if you find it helpful!
LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control. Peiyao Xiao, Chaosheng Dong, Shaofeng Zou, Kaiyi Ji.
[Manuscript] 02/2025 A new AI4Science manuscript is available online. We explore reciprocal space to encode long-range interactions and leverage a mixture-of-experts (MoE) approach for multi-property prediction.
ReGNet: Reciprocal Space-Aware Long-Range Modeling and Multi-Property Prediction for Crystals. Jianan Nie*, Peiyao Xiao*, Kaiyi Ji, Peng Gao.
[Publication] 01/2025 Three papers have been accepted by ICLR 2025 and JMLR 2025. Congratulations to my students Yifan, Hao, Peiyao and all collaborators.
[Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).
[Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!
[Talk] 10/2024 I'll be giving a talk at the INFORMS Annual Meeting in Seattle from October 20-23. Love to connect if you are also attending.
[Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!
[Job] 05/2024 Peiyao starts his internship at Amazon working on multi-objective optimization for recommendation.
Understanding Forgetting in Continual Learning with Linear Regression
Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
International Conference on Machine Learning (ICML) 2024.Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm <Code>
Jie Hao, Kaiyi Ji, Mingrui Liu.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
(New!) LDC-MTL: Balancing Multi-Task Learning through Scalable Loss Discrepancy Control <Code>
Peiyao Xiao, Chaosheng Dong, Shaofeng Zou, Kaiyi Ji.(New!) MGDA Converges under Generalized Smoothness, Provably.
Qi Zhang, Peiyao Xiao, Shaofeng Zou, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025.(New!) Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning
Yudan Wang, Peiyao Xiao, Hao Ban, Kaiyi Ji, Shaofeng Zou.
IEEE Transactions on Information Theory 2025, under revision.Fair Resource Allocation in Multi-Task Learning <Code>
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
Peiyao Xiao, Hao Ban, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
(New!) Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis <Code>
Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025(New!) Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy <Open-Source Library>
Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen,
Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao. arXiv:2406.16087, 2024.Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
Kaiyi Ji, Junjie Yang, Yingbin Liang.
International Conference on Machine Learning (ICML) 2021.
SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023. (Spotlight)Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Minhui Huang, Dewei Zhang, Kaiyi Ji.
International Conference on Machine Learning (ICML) 2023.