Welcome to Kaiyi Ji's Homepage
 |
|
About Me
I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022, working with Prof. Lei Ying. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016.
Prospective students: I am now looking for PhD and Intern students with rich math and coding experiences in LLMs (optimization, alignment, prompt tuning, etc). Please send me an email with your CV and Transcript.
Research
My current research interests include:
Large-scale optimization: bilevel optimization, multi-objective optimization, and optimization for LLMs.
Machine Learning and deep learning: multi-task learning, model merging, continual learning and AI4Science
Foundation models: progressive prompts, parameter-efficient fine-tuning, multi-objective RLHF
Recent News!
[Talk] 03/2025 I will organize a special session entitled “Recent advances in optimization for machine learning and networking” at Asilomar 2025. Please join us if you attend Asilomar at CA.
[Manuscript] 02/2025 A new MTL paper is out! We introduce a scalable loss balancing approach for multi-task learning with O(1) time and memory cost, achieving superior accuracy and efficiency. Check out the code GitHub. Please star us if you find it useful!
Scalable Bilevel Loss Balancing for Multi-Task Learning. Peiyao Xiao, Chaosheng Dong, Shaofeng Zou, Kaiyi Ji.
[Manuscript] 02/2025 A new AI4Science manuscript is available online. We explore reciprocal space to encode long-range interactions and leverage a mixture-of-experts (MoE) approach for multi-property prediction.
ReGNet: Reciprocal Space-Aware Long-Range Modeling and Multi-Property Prediction for Crystals. Jianan Nie*, Peiyao Xiao*, Kaiyi Ji, Peng Gao.
[Publication] 01/2025 Three papers have been accepted by ICLR 2025 and JMLR 2025. Congratulations to my students Yifan, Hao, Peiyao and all collaborators.
[Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).
[Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!
[Talk] 10/2024 I'll be giving a talk at the INFORMS Annual Meeting in Seattle from October 20-23. Love to connect if you are also attending.
[Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!
[Manuscript] 07/2024 Our paper, Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy, presents a neural-symbolic approach to robot autonomy using bilevel optimization. We validate our framework on path planning, rule induction, optimal control, visual odometry, and multi-robot routing. For a quick overview, visit blog and Library. Thanks to our fantastic collaborators!
[Job] 05/2024 Peiyao starts his internship at Amazon working on multi-objective optimization for recommendation.
[Award] 12/2023 Glad to receive CSE Junior Faculty Research Award from UB CSE. Thanks to the department and my students!
Understanding Forgetting in Continual Learning with Linear Regression
Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
International Conference on Machine Learning (ICML) 2024.Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm <Code>
Jie Hao, Kaiyi Ji, Mingrui Liu.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
(New!) Scalable Bilevel Loss Balancing for Multi-Task Learning <Code>
Peiyao Xiao, Chaosheng Dong, Shaofeng Zou, Kaiyi Ji.(New!) MGDA Converges under Generalized Smoothness, Provably.
Qi Zhang, Peiyao Xiao, Shaofeng Zou, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025.(New!) Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning
Yudan Wang, Peiyao Xiao, Hao Ban, Kaiyi Ji, Shaofeng Zou.
IEEE Transactions on Information Theory 2025, under revision.Fair Resource Allocation in Multi-Task Learning <Code>
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
Peiyao Xiao, Hao Ban, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
(New!) Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis <Code>
Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma, Kaiyi Ji.
International Conference on Learning Representations (ICLR) 2025(New!) Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy <Open-Source Library>
Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen,
Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao. arXiv:2406.16087, 2024.Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
Kaiyi Ji, Junjie Yang, Yingbin Liang.
International Conference on Machine Learning (ICML) 2021.
SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023. (Spotlight)Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Minhui Huang, Dewei Zhang, Kaiyi Ji.
International Conference on Machine Learning (ICML) 2023.