----------------------------------------------------------------------------------------------------

    Department of Systems Engineering and Engineering Management

                       The Chinese University of Hong Kong

----------------------------------------------------------------------------------------------------

Date: Friday, February 3, 4:30 pm – 6:00 pm

Venue: ERB 513, The Chinese University of Hong Kong

Title: Algorithmic Decision-making with Endogenous Rewards

Speaker: Prof. Xiaowei Zhang, University of Hong Kong

Abstract:

In the standard data analysis framework, data is collected (once for all), and then data analysis is carried out. However, with the advancement of digital technology, decision-makers constantly analyze past data and generate new data through their decisions. We model this as a Markov decision process and show that the dynamic interaction between data generation and data analysis leads to a new type of bias (reinforcement bias) that exacerbates the endogeneity problem in standard data analysis. We propose a class of instrument variable (IV)-based reinforcement learning (RL) algorithms to correct for the bias and establish their theoretical properties by incorporating them into a stochastic approximation (SA) framework. Our analysis accommodates iterate-dependent Markovian structures and, therefore, can be used to study RL algorithms with policy improvement. Furthermore, we derive a sharper trajectory concentration bound: with a polynomial rate, the entire future trajectory of the SA iterates, after a given finite time, falls within a ball centered at the true parameter and shrinking at another polynomial rate. We also provide formulas for inference on optimal policies of the IV-RL algorithms. These formulas highlight how intertemporal dependencies of the Markovian environment affect the inference.

Biography:

Xiaowei Zhang is an assistant professor in the Faculty of Business and Economics, University of Hong Kong. He received his PhD in Management Science and Engineering from Stanford University and B.S. in Mathematics from Nankai University. His recent research interests include high-dimensional simulation optimization and causal reinforcement learning. His research has been published at top journals such as Management Science, Operations Research, and Mathematics of Operations Research.

Everyone is welcome to attend the talk!

SEEM-5202 Website: http://seminar.se.cuhk.edu.hk

Email: seem5202@se.cuhk.edu.hk