Real-time human gaze estimation

Abstract

This paper describes a system for estimating the course gaze or 1D head pose of multiple people in a video stream from a moving camera in an indoor scene. The system runs at 30 Hz and can detect human heads with a F-Score of 87.2% and predict their gaze with an average error 20.9° including when they are facing directly away from the camera. The system uses two Convolutional Neural Networks (CNNs) for head detection and gaze estimation respectively and uses common tracking and filtering techniques for smoothing predictions over time. This paper is application-focused and so describes the individual components of the system as well as the techniques used for collecting data and training the CNNs.