Activity Recognition from a First Person Perspective

Tuesday, November 20, 2018 -
4:00pm to 5:00pm
Biotechnology Center Auditorium, 425 Henry Mall

Speaker Name: 

Yin Li

Speaker Institution: 

Assistant Professor, Biostatistics and Medical Informatics, UW-Madison




Advances in sensor miniaturization, low-power computing, and battery life have enabled the first generation of mainstream wearable cameras. Millions of hours of videos have been captured by these devices, creating a record of our daily visual experiences at an unprecedented scale. This has created a major opportunity to develop new capabilities and products based on First Person Vision (FPV)--the automatic analysis of videos captured from wearable cameras. Meanwhile, vision technology is at a tipping point. Major progress has been made over the last few years in both visual recognition and 3D reconstruction. The stage is set for a grand challenge of activity recognition in FPV. My research focuses on understanding naturalistic daily activities of the camera wearer in FPV to advance both computer vision and mobile health.

My talk has three parts. First, I will demonstrate that first person video has the unique property of encoding the intentions and goals of the camera wearer. I will introduce first person visual cues that capture the users' intent and can be used to predict their point of gaze and the actions they are performing during activities of daily living. This technology provides a new set of tools to analysis highly skill activities, such as those involved in biomedical experiments. Second, I will describe a novel approach to measure children’s social behaviors during naturalistic face-to-face interactions with an adult partner, who is wearing a camera. I will show that first person video can support fine-grained coding of gaze for autism research. Going further, I will present a method for automatically detecting moments of eye contact. Finally, I will briefly cover my work on measuring physiological parameters such as heart rate and respiration rate using wearable devices.