“Clever Exploration for Machine Learning”
Wednesday, March 29 at 3:00pm
NEB 409

Abstract

Many machine learning and optimization algorithms solve hidden root-finding problems through the framework of stochastic approximation (SA). SA is generally slow to converge: the optimal rate at which the mean squared error (MSE) vanishes is only of order O(n⁻¹).

Recent advances in theory show that a deterministic analog of SA, known as quasi-stochastic approximation (QSA), has led to algorithms with superior performance in terms of convergence rates and estimation error. For example, rates arbitrarily close to O(n⁻⁴) are achieved for the MSE in QSA through careful algorithm design.

New representations of the solution to QSA algorithms provide a clear path to obtaining transient bounds and guidelines for algorithm design. This is especially valuable in applications where the designer of the algorithm also designs the algorithm’s “noise” or “exploration” sequence (e.g. reinforcement learning and gradient-free optimization).

This talk will survey general SA/QSA theory, provide insights on how to design algorithms to improve performance, and illustrate design principles through numerical studies.

Biography

Caio Kalil Lauand is a Ph.D. student at the University of Florida under the supervision of Dr. Sean Meyn. His focus is on stochastic approximation and applications such as optimization and reinforcement learning. He received the B.S.E.E. degree from the University of North Florida.