Eye Fixation Forecasting in Task-Oriented Virtual Reality

Eye Fixation Analysis and Forecasting in Task-Oriented Virtual Reality

Zhiming Hu



In immersive virtual reality (VR), users' visual attention is crucial for many important applications, including VR content design, gaze-based interaction, and gaze-contingent rendering. Especially, information on users' future eye fixations is key for intelligent user interfaces and has significant relevance for many areas, such as visual attention enhancement, dynamic event triggering, and human-computer interaction. However, previous works typically focused on free-viewing conditions and paid less attention to task-oriented attention. This paper aims at forecasting users' eye fixations in task-oriented virtual reality. To this end, a VR eye tracking dataset that corresponds to different users performing a visual search task in immersive virtual environments is built. A comprehensive analysis of users' eye fixations is performed based on the collected data. The analysis reveals that eye fixations are correlated with users' historical gaze positions, task-related objects, saliency information of the VR content, and head rotation velocities. Based on this analysis, a novel learning-based model is proposed to forecast users' eye fixations in the near future in immersive virtual environments.

Presentation Video

Related Work

Our related work:

EHTask: Recognizing User Tasks from Eye and Head Movements in Immersive Virtual Reality

Research progress of user task prediction and algorithm analysis (in Chinese)

FixationNet: Forecasting Eye Fixations in Task-Oriented Virtual Environments

Gaze Analysis and Prediction in Virtual Reality

DGaze: CNN-Based Gaze Prediction in Dynamic Scenes

Temporal Continuity of Visual Attention for Future Gaze Prediction in Immersive Virtual Reality

SGaze: A Data-Driven Eye-Head Coordination Model for Realtime Gaze Prediction


@inproceedings{hu21_Eye, author={Hu, Zhiming}, title = {Eye Fixation Forecasting in Task-Oriented Virtual Reality}, booktitle={Proceedings of the 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops}, year = {2021}, pages={707-708}, organization={IEEE} }