TVCG Best Journal Nominees Award
Human Eye Fixation Analysis and Forecasting in Task-Oriented Virtual Environments
Zhiming Hu, Andreas Bulling, Sheng Li, Guoping Wang
Dataset PDF PPT Code Experimental Scenes Supplemental Material
Human visual attention in immersive virtual reality (VR) is key for many important applications, such as content design, gaze-contingent rendering, or gaze-based interaction. However, prior works typically focused on free-viewing conditions that have limited relevance for practical applications. We first collect eye tracking data of 27 participants performing a visual search task in four immersive VR environments. Based on this dataset, we provide a comprehensive analysis of the collected data and reveal correlations between users' eye fixations and other factors, i.e. users' historical gaze positions, task-related objects, saliency information of the VR content, and users' head rotation velocities. Based on this analysis, we propose FixationNet -- a novel learning-based model to forecast users' eye fixations in the near future in VR. We evaluate the performance of our model for free-viewing and task-oriented settings and show that it outperforms the state of the art by a large margin of 19.8% (from a mean error of 2.93° to 2.35°) in free-viewing and of 15.1% (from 2.05° to 1.74°) in task-oriented situations. As such, our work provides new insights into task-oriented attention in virtual environments and guides future work on this important topic in VR research.
Our related work:
EHTask: Recognizing User Tasks from Eye and Head Movements in Immersive Virtual Reality
Research progress of user task prediction and algorithm analysis (in Chinese)
Eye Fixation Forecasting in Task-Oriented Virtual Reality
Gaze Analysis and Prediction in Virtual Reality
DGaze: CNN-Based Gaze Prediction in Dynamic Scenes
Temporal Continuity of Visual Attention for Future Gaze Prediction in Immersive Virtual Reality
SGaze: A Data-Driven Eye-Head Coordination Model for Realtime Gaze Prediction