Low-Cost Eye Tracking for Foveated Rendering Using Machine Learning

Abstract

This paper outlines a $50 head-mounted real-time eye tracker to track the user's eye, and uses foveated rendering reduce bandwidth costs and improve immersion. This is accomplished by using two cameras for detecting the iris and locating the beacons around the screen which tracks head movement in three dimensions. A neural net is trained on this data which then predicts where the user is looking based on the inputs from the two cameras. Foveated rendering is performed by only rendering the area of the screen currently being focused and blurring the area that falls under peripheral vision. Maturing this technology could be integrated into virtual reality headsets and for other immersive media experiences while dramatically decreasing bandwidth costs and increasing the overall functionality and capabilities of the devices. This would allow for a more robust as well as wireless virtual reality experience, which would mitigate the largest drawback to mass adoption: clunky wires and lack of a truly realistic experience.