Software for automated classification of probe-based confocal laser endomicroscopy videos of colorectal polyps

Figure 1 Imaging modalities of colonic polyps. A: Setup of probe-based confocal laser endomicroscopy (pCLE) imaging system (Cellvizio, Mauna Kea Technologies); B: Endoscopic image of tubular adenoma, and the pCLE miniprobe; C: An image of the pCLE video sequence; D: A pCLE mosaic image built with the video mosaicing tool; E: Histopathology image.

Figure 2 Adjusting bag-of-visual-words technique for probe-based confocal laser endomicroscopy video retrieval. A: Neoplastic probe-based confocal laser endomicroscopy mosaic obtained with non-rigid registration; B: Colored visual words mapped to the disk regions of radius 60 pixels in the mosaic image; C: Overlap scores of the local regions in the mosaic space, computed from the translation results of mosaicing.

Figure 3 Pipeline of the probe-based confocal laser endomicroscopy retrieval-based software classification framework. From the acquisition of the probe-based confocal laser endomicroscopy (pCLE) video query by the Cellvizio system to the on-line automated diagnosis estimation.

Figure 4 Typical results of automated probe-based confocal laser endomicroscopy video retrieval. The probe-based confocal laser endomicroscopy (pCLE) videos are represented by mosaic images; they are annotated with their histopathology diagnosis. Video queries are highlighted in gray and followed by their 3 most similar videos. Automated software classification (hyperplastic vs neoplastic) of query videos is based on the votes of the similar videos. With respect to histopathology, both the automated software classification and the pCLE diagnosis established by expert endoscopists are correct for these queries.

Figure 5 Results of automated probe-based confocal laser endomicroscopy video retrieval represented as mosaics. With respect to histology: the automated software classification is correct for video query Q6 but incorrect for video queries Q5 and Q7, whereas the off-line diagnosis of probe-based confocal laser endomicroscopy videos established by the expert endoscopists is correct for video queries Q5 but incorrect for video queries Q6 and Q7 (for which this disagreement is marked by *).