----------------------------------------------------------------------------------------------------------------------

 

 

      Department of Systems Engineering and Engineering Management

                             The Chinese University of Hong Kong

 

----------------------------------------------------------------------------------------------------------------------

 

Date: 4:30pm to 5:30pm, Feb. 17, 2017 (Friday)

 

Title: Stochastic Algorithms for Large-Scale Machine Learning Problems

 

Speaker: Prof. Shiqian Ma from The Chinese University of Hong Kong

 

Abstract:

 

Stochastic gradient descent (SGD) method and its variants are the main approaches for solving machine learning problems that involve large-scale training dataset. This talk addresses two issues in SGD. (i) One of the major issues in SGD is how to choose the step size while running the algorithm. Since the traditional line search technique does not apply for stochastic optimization algorithms, the common practice in SGD is either to use a diminishing step size, or to tune a fixed step size by hand, which can be time consuming in practice. We propose to use the Barzilai-Borwein method to automatically compute step sizes for SGD and its variant: stochastic variance reduced gradient (SVRG) method, which leads to two algorithms: SGD-BB and SVRG-BB. We prove that SVRG-BB converges linearly for strongly convex objective functions. Numerical results on standard machine learning problems are reported to demonstrate the advantages of our methods. (ii) Another issue is how to incorporate the second-order information to SGD. We propose a stochastic quasi-Newton method for solving nonconvex learning problems. Note that all existing stochastic quasi-Newton methods can only handle convex problems. Convergence and complexity results of our method are established. Numerical results on classification problems using SVM and neural networks are reported. 

 

Biography:

 

Shiqian Ma received his B.S. from Peking University in 2003, M.S. from Chinese Academy of Sciences in 2006 and Ph.D. in Industrial Engineering and Operations Research from Columbia University in 2011. He then spent one and half years in the Institute for Mathematics and Its Applications at University of Minnesota as an NSF postdoctoral fellow. Shiqian Ma joined the Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong in December 2012. His current research interests include theory and algorithms for large-scale optimization and its applications in big data analytics, statistics, machine learning, bioinformatics, signal processing and image processing.

 

Shiqian Ma received the INFORMS Optimization Society best student paper prize in 2010, honorable mention of INFORMS George Nicholson student paper competition in 2011. He was one of the finalists of the 2011 IBM Herman Goldstine fellowship. He received the Journal of the Operations Research Society of China Excellent Paper Award in 2016. 

 

Everyone is welcome to attend the talk!

 

 

 

Venue: Room 513,

      William M.W. Mong Engineering Building (ERB),

      The Chinese University of Hong Kong.

 

SEEM-5202 Website: http://seminar.se.cuhk.edu.hk

Email: seem5202@se.cuhk.edu.hk