MedGaze: Gaze Estimation on WCE Images Based on a CNN Autoencoder

Abstract

The interpretation of medical images depends on physicians' experience. Over time, physicians develop their ability to examine the images, and this is usually reflected on gaze patterns they follow to observe visual cues, which lead them to diagnostic decisions. In the context of gaze prediction, graph and machine learning methods have been proposed for the visual saliency estimation on generic images. In this work we preset a novel and robust gaze estimation methodology based on physicians' eye fixations, using convolutional neural networks combined with regularization methods, on medical images taken during Wireless Capsule Endoscopy (WCE). Furthermore, we present a novel dataset of physicians' eye fixation patterns which was used for the training of the neural network model. The model was able to achieve 68.5% Judd's Area Under the receiver operating Characteristic (AUC-J). © 2019 IEEE.