Randomized coordinate descent methods: a unified theory

                 Department of Systems Engineering and Engineering Management
                              The Chinese University of Hong Kong
Date: Friday, October 10, 2014 - 16:30 - 17:30
Speaker: Prof. Peter Richtarik, School of Mathematics, University of Edinburgh.
Title: Randomized coordinate descent methods: a unified theory
Many big data applications can be cast as convex optimization problems and solved by a suitable optimization algorithm. Due to the size of such problems, simple methods able to progress while investigating only a small portion of the data are more desirable and efficient. I will outline a unified theory of a large class of randomized block coordinate descent methods. These methods have recently become extremely popular in areas such as machine learning, optimization and engineering due to their simplicity, versatility, scalability and ability to take advantage of sparsity in data. In the special case of trivial randomization, these methods become deterministic and include algorithms such as gradient descent, projected gradient descent, proximal gradient descent, iterative soft thresholding algorithm (ISTA), Nesterov's accelerated gradient method and FISTA. In the general randomized setting, the methods specialize to serial, parallel, distributed, proximal and accelerated coordinate descent. I will conclude with presenting the results of some big data experiments.
Peter Richtarik is an Assistant Professor of Mathematical Optimization at the University of Edinburgh. He received his PhD in Operations Research in 2007 from Cornell University and prior to his current appointment spent two years at the Center for Operations Research and Econometrics, Belgium, as a postdoctoral scholar, hosted by Prof Yurii Nesterov. He has recently spent a semester at the University of California, Berkeley, as an invited researcher, participating in the Theoretical Foundations of Big Data Analysis program run at the Simons Institute for the Theory of Computing.        Dr Richtarik is an expert in stochastic methods for big data convex optimization. In a series of papers he developed and analyzed randomized coordinate descent methods of various flavours---serial, parallel, distributed, proximal, nonsmooth, accelerated and nonuniform. These methods are scalable to problems involving billions of variables. The algorithms (e.g., PCDM, Hydra, APPROX) were successfully implemented on serial and parallel architectures, ranging from single and multicore processors, through GPUs, to supercomputers. He has also worked on topics such as sparse principal component analysis (GPower), optimization in relative scale, cutting plane methods, image classification, truss topology design and optimal planet formation.       The papers of Dr Richtarik have won several prizes, including the 16th Leslie Fox Prize (2nd prize), awarded biennially since 1985 to a numerical analyst worldwide under the age 31, and the 2012 INFORMS Computing Society Best Student Paper Prize (sole runner up). These prizes were awarded to his co-author and PhD student Martin Takac. Dr Richtarik has delivered more than 80 research talks in many countries around the world. He is often invited to give keynote and plenary addresses at events focused on the theory and practice of big data analysis, machine learning, computational statistics and optimization. He has organized several international scientific workshops and symposia.       Dr Richtarik is a member of the Edinburgh Research Group in Optimization, Edinburgh Compressed Sensing and the Algorithms and Complexity Research group. He serves on the steering committees of the Centre for Doctoral Training in Data Science and the Centre for Numerical Algorithms and Intelligent Software, and is affiliated with the Maxwell Institute Graduate School in Mathematical Analysis and Applications --- all three being institutions funded by £5m grants from the UK government. His recent research is funded by grants from EPSRC and other sources totalling more than £1m. He is a member of the EPSRC Peer Review College, and serves as an evaluator of EU proposals.
Everyone is welcome to attend the talk!
Venue: Room 513,
       William M.W. Mong Engineering Building (ERB),
       (Engineering Building Complex Phase 2)
       The Chinese University of Hong Kong.
The talk will be hosted by:
Prof. Shiqian Ma,
Department of Systems Engineering and Engineering Management,
The Chinese University of Hong Kong,
Telephone Number: (852) 3943-8240
SEEM-5201 Website:
Friday, October 10, 2014 - 08:30 to 09:30