Welcome to Kaiyi Ji's Homepage
 |
|
About Me
I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022, working with Prof. Lei Ying. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016.
Prospective students: I do not have PhD positions now. However, intern and visiting students are welcome! Please send me an email with your CV and Transcript or fill this form.
Research
I have been working at the intersection of optimization, machine learning and wireless networking. In particular, I work on bilevel optimization, multi-task learning, transfer learning (meta-learning, continual learning, machine unlearning), distributed learning over networks, stochastic optimization and control, and their applications in signal processing, communication and image processing. Here are selected publications showcasing my current interest:
Understanding Forgetting in Continual Learning with Linear Regression
Meng Ding, Kaiyi Ji, Di Wang, Jinhui Xu.
International Conference on Machine Learning (ICML) 2024.Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm <Code>
Jie Hao, Kaiyi Ji, Mingrui Liu.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
On the Convergence of Multi-Objective Optimization under Generalized Smoothness
Qi Zhang, Peiyao Xiao, Kaiyi Ji, Shaofeng Zou. arXiv:2405.19440, 2024Fair Resource Allocation in Multi-Task Learning <Code>
Hao Ban and Kaiyi Ji.
International Conference on Machine Learning (ICML) 2024.Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
Peiyao Xiao, Hao Ban, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.
Achieving O(\epsilon^{-1.5}) Complexity in Hessian/Jacobian-free Stochastic Bilevel Optimization
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023.Will Bilevel Optimizers Benefit from Loops
Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying
Conference on Neural Information Processing Systems (NeurIPS) 2022. (Spotlight)Lower Bounds and Accelerated Algorithms for Bilevel Optimization
Kaiyi Ji, Yingbin Liang
Journal of Machine Learning Research (JMLR) 2022.Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
Kaiyi Ji, Junjie Yang, Yingbin Liang.
International Conference on Machine Learning (ICML) 2021.
SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
Yifan Yang, Peiyao Xiao, Kaiyi Ji.
Conference on Neural Information Processing Systems (NeurIPS) 2023. (Spotlight)Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Minhui Huang, Dewei Zhang, Kaiyi Ji.
International Conference on Machine Learning (ICML) 2023.
Recent News!
[Manuscript] 06/2024 Two new manuscripts: finite-time analysis for multi-task reinforcement learning and multi-objective optimization with generalized smoothness are available online. Please check them out!
[Publication] 05/2024 Two papers on continual learning theory and multi-task learning accepted to ICML 2024. Congratulations to Hao, Meng and collaborators.
[Publication] 03/2024 Our paper Boosting One-Point Derivative-Free Online Optimization via Residual Feedback was accepted by IEEE Transactions on Automatic Control. Thanks to all collaborators!
[Manuscript] 02/2024 One manuscript “Fair Resource Allocation in Multi-Task Learning” is available online. We connect fair resource allocation in wireless communication with multi-task learning, and propose an optimization method named FairGrad. This method implements different ideas of fairness and achieves SOTA performance among gradient manipulation MTL methods with performance guarantee. The idea has also been incorporated into existing MTL methods with significant improvements observed. Check our codes: Click.
[Manuscript] 02/2024 One manuscript “Discriminative Adversarial Unlearning” is available online. We introduce a novel machine unlearning framework founded on an attacker network and a defender network, where the attacker teases out the information of the data to be unlearned, and the defender unlearns to defend the network against the attack. We also incorporate a self-supervised objective to address the feature space discrepancies between the forget and validation sets. This method closely approximates the ideal benchmark of retraining from scratch in various scenarios. Code is available at Click.
[Publication] 01/2024 One paper on resource-efficient self-supervised contrastive learning accepted in ICLR 2024! We achieve competitive or SOTA results on ImageNet and other standard datasets with an impressively small batchsize. This method also has the promising downstream viability on transfer learning and few-shot learning. Code and paper will come soon! Big congratulations to Rohan and other coauthors!
[Award] 12/2023 Glad to receive CSE Junior Faculty Research Award from UB CSE. Thanks to the department and my students!
[Talk] 12/2023 Glad to visit RPI ECSE and give a talk on bilevel optimization for machine learning and beyond. Many thanks to Tianyi's invitation and host!
[Talk] 10,11/2023 Glad to give multiple invited talks at INFORMS 2023 (Phoenix), Asilomar 2023 (Pacific Grove), MobiHoc 2023 (Washionton DC) about our recent progress on bilevel optimization for continual learning and network resource allocation.