Welcome to Kaiyi Ji's Homepage
 |
|
About Me
I am an assistant professor at the Department of Computer Science and Engineering of the University at Buffalo, The State University of New York. I was a postdoctoral research fellow at the Electrical Engineering and Computer Science Department of the University of Michigan, Ann Arbor, in 2022, working with Prof. Lei Ying. I received my Ph.D. degree from the Electrical and Computer Engineering Department of The Ohio State University in December, 2021, advised by Prof. Yingbin Liang. I was a visiting student research collaborator at the department of Electrical Engineering, Princeton University working with Prof. H. Vincent Poor. Previously I obtained my B.S. degree from University of Science and Technology of China in 2016.
Research
I have been working at the intersection of optimization, machine learning and networks, on theory, algorithm and application sides. My major research focuses include:
Bilevel optimization and its application in deep learning
Meta-learning, multi-task learning and continual learning
Large-scale stochastic optimization
Federated learning and communication networks
To Prospective Students
I am looking for highly motivated students with strong mathematical backgrounds and/or programming skills in machine learning and optimization to work with me.
Moreover, intern and visiting students are highly welcome!
One PhD position is avaliable immediately. Please fill the following form if you are interested. I will contact you if there is a good match!
Recent News!
[Publication] 03/2024 Our paper Boosting One-Point Derivative-Free Online Optimization via Residual Feedback was accepted by IEEE Transactions on Automatic Control. Thanks to all collaborators!
[Manuscript] 02/2024 One manuscript “Fair Resource Allocation in Multi-Task Learning” is available online. We connect fair resource allocation in wireless communication with multi-task learning, and propose an optimization method named FairGrad. This method implements different ideas of fairness and achieves SOTA performance among gradient manipulation MTL methods with performance guarantee. The idea has also been incorporated into existing MTL methods with significant improvements observed. Check our codes: Click.
[Manuscript] 02/2024 One manuscript “Discriminative Adversarial Unlearning” is available online. We introduce a novel machine unlearning framework founded on an attacker network and a defender network, where the attacker teases out the information of the data to be unlearned, and the defender unlearns to defend the network against the attack. We also incorporate a self-supervised objective to address the feature space discrepancies between the forget and validation sets. This method closely approximates the ideal benchmark of retraining from scratch in various scenarios. Code is available at Click.
[Publication] 01/2024 One paper on resource-efficient self-supervised contrastive learning accepted in ICLR 2024! We achieve competitive or SOTA results on ImageNet and other standard datasets with an impressively small batchsize. This method also has the promising downstream viability on transfer learning and few-shot learning. Code and paper will come soon! Big congratulations to Rohan and other coauthors!
[Award] 12/2023 Glad to receive CSE Junior Faculty Research Award from UB CSE. Thanks to the department and my students!
[Talk] 12/2023 Glad to visit RPI ECSE and give a talk on bilevel optimization for machine learning and beyond. Many thanks to Tianyi's invitation and host!
[Talk] 10,11/2023 Glad to give multiple invited talks at INFORMS 2023 (Phoenix), Asilomar 2023 (Pacific Grove), MobiHoc 2023 (Washionton DC) about our recent progress on bilevel optimization for continual learning and network resource allocation.
[New Member] 11/2023 New Ph.D. student Hao Ban (BS and MS from Southeast University, China) will join our group in Spring 2024. Welcome, Hao!
[Publication] 09/2023 Five papers accepted in NeurIPS 2023 with one spotlight presentation! The topics span over Hessian-free bilevel optimization, federated learning, continual learning and multi-objective learning. Big congratulations to my students Yifan, Peiyao and Hao, and many thanks to my collaborators!
[Grant] 07/2023 I received a grant as site PI @ UB in collaboration with Dr. Shiqian Ma (Rice University) on Distributed Optimization in Multi-Agent Systems. Many thanks to NSF!
[Grant] 07/2023 I received a grant as Lead PI @ UB in collaboration with Dr. Shiqian Ma (Rice University) on Large-Scale Bilevel Optimization. Many thanks to NSF!
[Publication] 07/2023 Our work on fair resource allocation over communication network has been accepted by ACM MobiHoc 2023! (acceptance rate: 21.9%)
[Talk] 06/2023 Gave an invited talk on bilevel optimization and continual learning at SIAM Conference on Optimization.
[Manuscript] 06/2023 Two new manuscripts on Simple federated bilevel learning and Direction-oriented multi-objective/task learning are available online. Codes can be found in GitHub.
[Publication] 04/2023 Our two submissions on federated bilevel optimization: Achieving Linear Speedup and Aggregated Iterative Differentiation have been accepted at ICML 2023 (acceptance rate: 27.9%). Congrats to my students Peiyao and Dewei! This is also Peiyao's first work. Keep going!
[Award] 02/2023 Selected as Top Reviewer in AISTATS 2023.
[Manuscript] 01/2023 A manuscript on fair resource allocation over communication network is available online: Network Utility Maximization with Bilevel Learning. We propose a new distributed data-driven bilevel optimization approach for network utility maximization with unknown user utilities!
[Award] 11/2022 Our NeurIPS 2022 paper on Unified Analysis for Bilevel Optimization received Spotlight (5% acceptance rate).
[Service] 10/2022 Will serve as TPC member of ACM MobiHoc 2023. Please consider submitting your best work to the conference!
[Talk] 10/2022 Gave an invited talk in the session “Bilevel Stochastic Methods for Optimization and Learning” at the 2022 INFORMS Annual Meeting, Indianapolis, IN.
[Publication] 09/2022 Two papers on Hessian-Free Bilevel Optimization and Unified Analysis for Bilevel Optimization have been accepted by NeurIPS 2022.
[Publication] 09/2022 Our paper, “Lower Bounds and Accelerated Algorithms for Bilevel Optimization”, has been accepted by JMLR 2022 with minor revision.
Funded Projects
Our research is generously supported by NSF and University at Buffalo.
Collaborative Research: CIF: Small: New Theory, Algorithms and Applications for Large-Scale Bilevel Optimization
Lead PI (share $300,190 out of $600,000)
Aug 2023 - Jul 2026
Collaborative Research: Distributed Bilevel Optimization in Multi-Agent Systems
Site PI (share $250,000 out of $500,000)
Aug 2023 - Jul 2026