----------------------------------------------------------------------------------------------------

       Department of Systems Engineering and Engineering Management

                           The Chinese University of Hong Kong

----------------------------------------------------------------------------------------------------

Date: Friday, December 6, 2024, 4:30 pm HKT

Venue: ERB 513, The Chinese University of Hong Kong

Title: Algorithms for bilevel optimization programs with applications in hyperparameter learning

Speaker: Prof. Zhang Jin, SUSTech, China

Abstract:

This work focuses on addressing two major challenges in large-scale Bi-Level Optimization (BLO) problems, which are increasingly applied in machine learning due to their ability to model nested structures. These challenges involve ensuring computational efficiency and providing theoretical guarantees. Recent advances in scalable BLO algorithms have primarily relied on lower-level simplifications and, inevitably, on computationally intensive calculations related to the Hessian matrix. We address both computational and theoretical challenges simultaneously by introducing an innovative single-loop gradient-based algorithm, utilizing the Moreau envelope-based reformulation, and providing convergence analysis for large-scale BLO problems with weakly convex and constrained lower levels. Notably, our algorithm relies solely on first-order gradient information, enhancing its practicality and efficiency, especially for large-scale BLO learning tasks. We validate the effectiveness of our approach through experiments on various synthetic problems and real-world applications, demonstrating its superior performance.

Biography:

Zhang Jin, Associate Professor in the Department of Mathematics at Southern University of Science and Technology and the National Center for Applied Mathematics Shenzhen, specializes in optimization theory and applications. His representative work has been published in journals such as Math Program, SIAM J Optim, Math Oper Res, SIAM J Numer Anal, J Mach Learn Res, IEEE Trans Pattern Anal Mach Intell, Sci. China. Math, as well as conferences like ICML, NeurIPS, and ICLR in the fields of optimization and machine learning.

Everyone is welcome to attend the talk!

SEEM-5201 Website: https://seminar.se.cuhk.edu.hk

Email: seem5201@se.cuhk.edu.hk