“Interactive Segmentation with Deep Metric Learning”
Wednesday, March 8 at 3:00pm
NEB 409

Abstract

Interactive segmentation/annotation is a technique in which humans can dynamically provide instruction to aid in segmenting regions of interest. It helps to facilitate fine-scale (pixel-wise) annotation by allowing the user to dynamically modify and correct coarse annotations with feedback from the training model in an interactive framework. Most existing classical interactive segmentation methods rely on low-level features such as color and texture. On the other hand, most advanced deep interactive segmentation techniques require the user to interact with every single image containing the same category of the region of interest, making it difficult to process a batch of images.

In this talk, I will present my proposed interactive segmentation/annotation framework with deep metric learning. Our metric-based loss function clusters annotated data in the embedding feature space, allowing for determination of labels of unknown pixels in other testing images by measuring their distance to the anchor set in the embedding space. We also propose a prototype learning algorithm to avoid intra-class variance and prevent data distribution, making a pre-trained network transferable to a new image with unseen objects. Furthermore, we integrate the algorithm with different annotation tools and develop a user-friendly interactive annotation APP to study how human behavior and model output evolve during the interaction loop.

Biography

Xiaolei Guo is a Ph.D. candidate in the Department of Electrical & Computer Engineering. She received the Master of Science in Electrical & Computer Engineering from the University of Florida in 2018 and the Bachelor of Science Electronic & Information Engineering from Beihang University, Beijing, China in 2016.