Explainable Classification of Weakly Annotated Wireless Capsule Endoscopy Images Based on a Fuzzy Bag-of-Colour Features Model and Brain Storm Optimization

Abstract

Wireless capsule endoscopy (WCE) constitutes a medical imaging technology developed for the endoscopic exploration of the gastrointestinal (GI) tract, whereas it provides a more comfortable examination method, in comparison to the conventional endoscopy technologies. In this paper, we propose a novel Explainable Fuzzy Bag-of-Words (XFBoW) feature extraction model, for the classification of weakly annotated WCE images. A comparative advantage of the proposed model over state-of-the-art feature extractors is that it can provide an explainable classification outcome, even with conventional classification schemes, such as Support Vector Machines. The explanations that can be derived are based on the similarity of the image content with the content of the training images, used for the construction of the model. The feature extraction process relies on data clustering and fuzzy sets. Clustering is used to encode the image content into visual words. These words are subsequently used for the formation of fuzzy sets to enable a linguistic characterization of similarities with the training images. A state-of-the-art Brain Storm Optimization algorithm is used as an optimizer to define the most appropriate number of visual words and fuzzy sets and also the fittest parameters of the classifier, in order to optimally classify the WCE images. The training of XFBoW is performed using only image-level, semantic labels instead of detailed, pixel-level annotations. The proposed method is investigated on real datasets that include a variety of GI abnormalities. The results show that XFBoW outperforms several state-of-the-art methods, while providing the advantage of explainability. © 2021, Springer Nature Switzerland AG.