br Figure Data augmentation for
Figure 4. Data augmentation for a representative image.
postoperative pathologic examination results. Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated by standard definitions. To compare classification performance between the CNN-CAD system and endoscopists, 17 endoscopists of varying experi-ence also evaluated the invasion depth of the lesion in the test dataset. Among them, 8 were classified as experienced en-doscopists who SC 560 had performed more than 5000 conventional endoscopy examinations, whereas the others were classified as junior endoscopists who had also performed more than 2000 conventional endoscopy examinations. One of the au-thors (J.C.) not involved in the construction of the CNN-CAD system made the statistical comparative analysis of the classification performance between the CNN-CAD system and endoscopists. The endoscopists received original endo-scopic images without any frames or annotations and gave every single picture their own classification with no informing of CAD classification results. The classification results from en-doscopists were compared with the label of the pictures. After that, a bootstrap resampling method was used to estimate the mean differences between the CNN-CAD system and the en-doscopists and their 95% confidence intervals (CIs) for sensi-tivity, specificity, and accuracy. One thousand bootstrap samples were simulated from original data to obtain robust estimates.
We also attempted to understand how the CNN-CAD system recognized the input images by performing an oc-clusion technique16 to determine which area of the image was most essential to the classification result. We performed iterations over regions of an image setting a 60 60 patch of the image to and recorded the
classification probability of the occluded image output by the system. The occluded area was slid from top-left to bottom-right with a stride of 6 pixels to generate a new da-taset consisting of the same images with different areas occluded. We then plotted the classification probability of each occluded image based on its occluded position and created a heatmap.
Applying a CNN-CAD system to determine invasion depth for endoscopic resection Zhu et al
TABLE 1. Clinicopathologic characteristics of patients in development and test datasets
Degree of differentiation
M, Mucosa; SM, submucosa.
Figure 5. Receiver operating characteristic curve for the test dataset. The area under the curve was 94%. Each point represents the prediction of a single endoscopist.
Performance of the CNN-CAD system
We plotted a receiver operating characteristic curve of the probability of P0 (negative) and P1 (positive) classifica-tions for each test image (Fig. 5) to assess the robustness of our CNN-CAD system. The area under the curve was .94 (95% CI, .90-.97).
Lesion invasion depth is one of the most important prognostic factors in gastric cancer. However, there is still
Zhu et al
Applying a CNN-CAD system to determine invasion depth for endoscopic resection
TABLE 2. Diagnostic accuracy of CNN-CAD system versus endoscopists
Values in parenthesis are standard deviations. CNN-CAD, Convolutional neural network computer-aided detection.
Figure 6. Heatmap of occlusion analysis.
no consensus on the best procedure for evaluating inva-sion depth preoperatively. To our knowledge, this is the first report of the use of a CNN-CAD system to evaluate the invasion depth of gastric cancer.
EUS was once considered an accurate technique for predicting invasive depth.17 Kim et al18 found that miniprobe EUS showed a higher accuracy for lesions with SM features, such as an irregular depressed surface or fold changes on conventional endoscopy. EUS might be more helpful for less-experienced endo-scopists because hydrogen bond provides objective evidence. Howev-er, EUS is not available in all hospitals and may require additional time and impose a financial burden. Further-more, it is sometimes difficult to obtain high-quality im-ages considering the size or location of the lesion, because it is highly operator-dependent. A previous study reported that low-quality EUS images lead to incor-rect determinations of invasion depth of EGC.19
In terms of overall accuracy in evaluating invasion depth, EUS is not superior to conventional endoscopy.20,21 In most clinical cases, experienced endoscopists can esti-mate invasion depth by macroscopic features using con-ventional endoscopy. Many endoscopic features, such as abrupt cutting of converging folds, remarkable redness, or nonextension sign, are reportedly associated with SM or deeper invasion.22-24 Although the accuracy of conven-tional endoscopy is relatively low,5,6 an advantage of using macroscopic features to evaluate invasion depth is that they are related to the results of pathologic examination. For example, remarkable redness can be explained by dilated tumor vessels,4 and the abrupt cutting of convergent folds with shallow or depressed areas is related to fibrosis of the SM layer caused by the tumor. Tsujii et al22 report a practical strategy for determining invasion depth using both conventional endoscopy and EUS to improve accuracy.