Publications

* indicates equal contribution

** indicates corresponding author

Preprints

• (New!) Fair Resource Allocation in Multi-Task Learning <Code>
  Hao Ban and Kaiyi Ji**. Submitted. 2024.

• (New!) Discriminative Adversarial Unlearning <Code>
  Rohan Sharma, Shijie Zhou, Kaiyi Ji, Changyou Chen. Submitted. 2024.

• A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima
  Daouda Sow, Kaiyi Ji, Ziwei Guan, Yingbin Liang. Submitted.

2024

• (New!) Boosting One-Point Derivative-Free Online Optimization via Residual Feedback
  Yan Zhang, Yi Zhou, Kaiyi Ji, Yi Shen, and Michael M. Zavlanos. Accepted by IEEE Transactions on Automatic Control, 2024.

• (New!) AUC-CL: A Batchsize-Robust Framework for Self-Supervised Contrastive Representation Learning <Code>
  Rohan Sharma, Kaiyi Ji, Zhiqiang Xu, Changyou Chen. Accepted by ICLR 2024.

2023

• (New!) Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms <Code>
  Peiyao Xiao*, Hao Ban*, Kaiyi Ji**. Accepted by NeurIPS 2023.

• (New!) SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning
  Yifan Yang, Peiyao Xiao, Kaiyi Ji**. Accepted by NeurIPS 2023. (Spotlight)

• (New!) Achieving O(\epsilon^{-1.5}) Complexity in Hessian-free Stochastic Bilevel Optimization
  Yifan Yang, Peiyao Xiao, Kaiyi Ji**. Accepted by NeurIPS 2023.

• (New!) Non-Convex Bilevel Optimization with Time-Varying Objective Functions
  Sen Lin, Daouda Sow, Kaiyi Ji, Yingbin Liang, Ness Shroff. Accepted by NeurIPS 2023.

• (New!) Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm
  Jie Hao, Kaiyi Ji, Mingrui Liu. Accepted by NeurIPS 2023.

• (New!) Network Utility Maximization with Unknown Utility Functions: A Distributed, Data-Driven Bilevel Optimization Approach
  Kaiyi Ji and Lei Ying. ACM MobiHoc 2023! (acceptance rate: 21.9%)

• Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiation.
  Peiyao Xiao and Kaiyi Ji**. International Conference on Machine Learning (ICML) 2023.

• Achieving Linear Speedup in Non-IID Federated Bilevel Learning
  Minhui Huang*, Dewei Zhang*, Kaiyi Ji**. International Conference on Machine Learning (ICML) 2023.

• Efficiently Escaping Saddle Points in Bilevel Optimization
  Minhui Huang, Kaiyi Ji, Shiqian Ma, Lifeng Lai. JMLR 2023, in revision.

2022

• On the Convergence Theory for Hessian-Free Bilevel Algorithms <Code>
  Daouda Sow, Kaiyi Ji, Yingbin Liang
  Conference on Neural Information Processing Systems (NeurIPS) 2022.

• Will Bilevel Optimizers Benefit from Loops
  Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying
  Conference on Neural Information Processing Systems (NeurIPS) 2022. (Spotlight, 5% acceptance rate)

• Lower Bounds and Accelerated Algorithms for Bilevel Optimization
  Kaiyi Ji, Yingbin Liang
  Journal of Machine Learning Research (JMLR) 2022.

• Data Sampling Affects the Complexity of Online SGD over Dependent Data
  Shaocong Ma, Ziyi Chen, Yi Zhou, Kaiyi Ji, Yingbin Liang.
  Conference on Uncertainty in Artificial Intelligence (UAI) 2022.

• Boosting One-Point Derivative-Free Online Optimization via Residual Feedback
  Yan Zhang, Yi Zhou, Kaiyi Ji, Michael M. Zavlanos
  IEEE Transactions on Automatic Control 2022, in revision.

2021

• Provably Faster Algorithms for Bilevel Optimization <Code>
  Junjie Yang, Kaiyi Ji, Yingbin Liang
  Conference on Neural Information Processing Systems (NeurIPS) 2021. (Spotlight, 3% acceptance rate)

• Bilevel Optimization: Nonasymptotic Analysis and Faster Algorithms <Code>
  Kaiyi Ji, Junjie Yang, Yingbin Liang
  International Conference on Machine Learning (ICML) 2021 (21.4% acceptance rate)
  Short version appears at ICML 2021 Workshop: Beyond First-Order Methods in ML Systems.

• A New One-Point Residual-Feedback Oracle For Black-Box Learning and Control
  Yan Zhang, Yi Zhou, Kaiyi Ji, Michael M. Zavlanos
  Automatica 2021.

• Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning
  Kaiyi Ji, Junjie Yang, Yingbin Liang
  Journal of Machine Learning Research (JMLR) 2021
  Short version appears at ICML 2021 Workshop: Beyond First-Order Methods in ML Systems

• Understanding Estimation and Generalization Error of Generative Adversarial Networks
  Kaiyi Ji, Yi Zhou, Yingbin Liang
  IEEE Trans. on Information Theory (TIT), 2021.

• When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models?
  Tengyu Xu, Yi Zhou, Kaiyi Ji, Yingbin Liang
  Stat, Special Issue of Deep Learning from Statistical Perspective, 2021

2020

• Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters
  Kaiyi Ji, Jason D. Lee, Yingbin Liang, H. Vincent Poor
  Conference on Neural Information Processing Systems (NeurIPS) 2020 (20.1% acceptance rate).

• History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms
  Kaiyi Ji, Zhe Wang, Bowen Weng, Yi Zhou, Wei Zhang, Yingbin Liang,
  International Conference on Machine Learning (ICML) 2020 (21.8% acceptance rate).

• Learning Latent Features with Pairwise Penalties in Matrix Completion
  Kaiyi Ji, Jian Tan, Yuejie Chi, Jinfeng Xu
  IEEE Trans. on Signal Processing (TSP), 2020
  Short version as an invited paper to a special session at the IEEE SAM 2020

• Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization
  Yi Zhou, Zhe Wang, Kaiyi Ji, Yingbin Liang, Vahid Tarokh
  International Joint Conferences on Artificial Intelligence Organization (IJCAI) 2020 (12.6% acceptance rate).

• Robust Stochastic Bandit Algorithms under Probabilistic Unbounded Adversarial Attack
  Ziwei Guan, Kaiyi Ji, Donald J. Bucci Jr, Timothy Hu, Joseph Palombo, Michael Liston, Yingbin Liang
  AAAI Conference on Artificial Intelligence (AAAI) 2020 (20.6% acceptance rate).

2019

• SpiderBoost and Momentum: Faster Stochastic Variance Reduction Algorithms
  Zhe Wang, Kaiyi Ji, Yi Zhou, Yingbin Liang, Vahid Tarokh
  Conference on Neural Information Processing Systems (NeurIPS) 2019 (21% acceptance rate).

• Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization
  Kaiyi Ji, Zhe Wang, Yi Zhou, Yingbin Liang
  International Conference on Machine Learning (ICML) 2019 (22.5% acceptance rate).

• Minimax Estimation of Neural Net Distance
  Kaiyi Ji, Yingbin Liang
  Conference on Neural Information Processing Systems (NeurIPS) 2018 (21% acceptance rate).

2018

• Asymptotic Miss Ratio of LRU Caching with Consistent Hashing
  Kaiyi Ji, Guocong Quan, Jian Tan
  IEEE INFOCOM 2018 (19% acceptance rate).

• LRU Caching with Dependent Competing Requests
  Guocong Quan, Kaiyi Ji, Jian Tan
  IEEE INFOCOM 2018 (19% acceptance rate).

• On Resource Pooling and Separation for LRU Caching
  Jian Tan, Guocong Quan, Kaiyi Ji, Ness Shroff
  ACM SIGMETRICS 2018 (20% acceptance rate).

• On Resource Pooling and Separation for LRU Caching
  Jian Tan, Guocong Quan, Kaiyi Ji, Ness Shroff
  Proceedings of the ACM on Measurement and Analysis of Computing Systems (POMACS), 2018