“Reinforcement Learning for Treatment Recommendations in Healthcare: Challenges, Algorithms and Analyses”
Wednesday, Feb. 28 at 1:00pm
LAR 234
Add to Calendar

Abstract

Reinforcement learning (RL) offers promising personalized treatment recommendations in healthcare because it adapts to patient’s health states, preferences and previous treatment history, accounts for possibly delayed treatment effects and optimizes long-term patient outcome. However, there are significant challenges when applying RL to treatment recommendations. In this talk, we focus on two challenges of (1) distribution mismatch and (2) limited offline clinical data. First, existing RL approaches usually assume that a learned policy will be deployed in the same environment as the one it was trained in. This is often violated due to population heterogeneity and non-stationarity, which could lead to a significant performance degradation. Second, most RL approaches require exploration of the actual environment, which is oftentimes infeasible in healthcare settings, and existing clinical datasets are offline, and usually only offer a limited coverage.

I will first introduce our recent results on the robust average-reward RL under model mismatch, including the fundamentals of the robust average-reward RL and further comprehensive design and analyses for both the model-based and model-free approaches. I will then present our results on offline RL, to which we provide a solution with minimax optimality based on distributionally robust optimization. Our approach effectively addresses the challenges of limited data availability and distributional shifts. In the end, I will discuss our ongoing efforts of intervention recommendation for children with speech and language challenges using RL.

Biography

Dr. Shaofeng Zou is an Assistant Professor at the Department of Electrical Engineering, University at Buffalo, the State University of New York. He received the Ph.D. degree in Electrical and Computer Engineering from Syracuse University in 2016. He received the B.E. degree (with honors) from Shanghai Jiao Tong University, Shanghai, China, in 2011. He was a postdoctoral research associate at the Coordinated Science Lab, University of Illinois at Urbana-Champaign during 2016-2018. Dr. Zou’s research interests include reinforcement learning, machine learning, statistical signal processing and information theory and their applications in health science and autonomous systems. He received the National Science Foundation CRII award in 2019 and the 2023 AAAI Distinguished Paper Award. His NSF CAREER proposal was recommended for funding recently in Jan. 2024. He has been serving as Associate Editor for IEEE Transactions on Signal Processing since 2023.