Date: Friday, Feburary 14, 4:00 pm – 5:30 pm



Venue: ERB 513, The Chinese University of Hong Kong



Title: Accommodating LLM Service over Heterogeneous Computational Resources



Speaker: Professor Binhang YUAN, HKUST





Abstract:



Serving large-scale language model service is crucial to contemporary 

AI applications. We focus on deploying such services in a 

heterogeneous and potentially decentralized setting to mitigate the 

substantial costs typically associated with centralized data centers. 

Our work relies on carefully designed scheduling algorithms where we 

model the computation capacity and inter-machine connection precisely 

and propose an efficient searching algorithm to find the optimal 

allocations for different LLM serving paradigms. Our empirical study 

suggests that the proposed method can efficiently reduce the service 

cost while preserving the service quality.



Biography:



Binhang YUAN is an Assistant Professor at the Department of Computer 

Science and Engineering (CSE), the Hong Kong University of Science and 

Technology (HKUST). He received his Ph.D. and master's degrees from 

Rice University and his bachelor's degree from Fudan University. 

Before joining HKUST, he was a Postdoc at the Swiss Federal Institute 

of Technology Zurich (ETH Zurich). His main research interests are in 

data management systems for machine learning and distributed and 

decentralized machine learning systems. He won the VLDB Best Paper 

Honorable Mention Award in 2019 and the SIGMOD Research Highlight 

Award in 2020.

Everyone is welcome to attend the talk!



SEEM-5202 Website: http://seminar.se.cuhk.edu.hk

Email: seem5202@se.cuhk.edu.hk