Welcome to Kaiyi Ji's Homepage
 |
|
About Me
I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022, working with Prof. Lei Ying. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016.
Prospective students: I do not have PhD positions now. However, intern and visiting students are welcome! Please send me an email with your CV and Transcript or fill this form.
Recent News!
[Services] 12/2024 Couple of upcoming services include NSF panelist, TPC member for ACM Mobihoc 2025 and TPC member for IEEE Information Theory Workshops (ITW’25).
[Publication] 12/2024 Our paper on efficiently escaping saddle points in bilevel optimization was accepted by JMLR after minor revision.
[Talk] 12/2024 I gave a talk on bilevel optimization for machine learning at the Computer Science Seminar Series at Johns Hopkins University. Thanks for the invitation!
[Talk] 10/2024 I'll be giving a talk at the INFORMS Annual Meeting in Seattle from October 20-23. Love to connect if you are also attending.
[Manuscript] 10/2024 Check our recent interesting results in: (I) tuning-free bilevel optimization <code> and (II) understanding fine-tuning in machine unlearning from a theoretical perspective.
[Publication] 09/2024 Our work on minimax bilevel optimization and rank-based robust meta-learning has been accepted to NeurIPS 2024. Congratulations to Yifan, Zhaofeng and all collaborators.
[Software] 09/2024 Our FairGrad for multi-task/objective learning is now supported by open-source MTL Library LibMTL. Feel free to explore it and see if it can benefit your research!
[Manuscript] 07/2024 Our paper, Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy, presents a neural-symbolic approach to robot autonomy using bilevel optimization. We validate our framework on path planning, rule induction, optimal control, visual odometry, and multi-robot routing. For a quick overview, visit blog and Library. Thanks to our fantastic collaborators!
[Job] 05/2024 Peiyao starts his internship at Amazon working on multi-objective optimization for recommendation.
[Publication] 05/2024 Two papers on understanding forgetting in continual learning and fair resource allocation in multi-task learning accepted to ICML 2024. Congratulations to Hao, Meng and collaborators.
[Award] 12/2023 Glad to receive CSE Junior Faculty Research Award from UB CSE. Thanks to the department and my students!
Research
My research focuses on developing foundational algorithms and theories in machine learning, optimization, deep learning, and wireless networking. My current interests include: (i) hierarchical machine learning, including bilevel optimization, meta-learning and hyperparameter optimization; (ii) multi-objective learning, including multi-objective optimization, multi-task learning, and model fusion; and (iii) knowledge transfer and unlearning, with emphasis on continual learning, fine-tuning, and machine unlearning. I am also exploring applications of these approaches in communication, robotics, and signal processing. Here are selected publications:
(New!) Why Fine-Tuning Struggles with Forgetting in Machine Unlearning? Theoretical Insights and a Remedial Approach
Meng Ding, Jinhui Xu, Kaiyi Ji. arXiv:2410.03833, 2024.Understanding Forgetting in Continual Learning with Linear Regression
Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
International Conference on Machine Learning (ICML) 2024.Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm <Code>
Jie Hao, Kaiyi Ji, Mingrui Liu.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
(New!) Fair Resource Allocation in Multi-Task Learning <Code>
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.AUC-CL: A Batchsize-Robust Framework for Self-Supervised Contrastive Representation Learning <Code>
Rohan Sharma, Kaiyi Ji, Zhiqiang Xu, Changyou Chen.
International Conference on Learning Representations (ICLR) 2024.Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
Peiyao Xiao, Hao Ban, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
(New!) Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis <Code>
Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma, Kaiyi Ji. arXiv:2410.05140, 2024.(New!) Imperative Learning: A Self-supervised Neural-Symbolic Learning Framework for Robot Autonomy <Open-Source Library>
Chen Wang, Kaiyi Ji, Junyi Geng, Zhongqiang Ren, Taimeng Fu, Fan Yang, Yifan Guo, Haonan He, Xiangyu Chen,
Zitong Zhan, Qiwei Du, Shaoshu Su, Bowen Li, Yuheng Qiu, Yi Du, Qihang Li, Yifan Yang, Xiao Lin, Zhipeng Zhao. arXiv:2406.16087, 2024.Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
Kaiyi Ji, Junjie Yang, Yingbin Liang.
International Conference on Machine Learning (ICML) 2021.
SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023. (Spotlight)Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Minhui Huang, Dewei Zhang, Kaiyi Ji.
International Conference on Machine Learning (ICML) 2023.