Published online Jun 18, 2020.
https://doi.org/10.5624/isd.2020.50.2.169
Automatic detection of periodontal compromised teeth in digital panoramic radiographs using faster regional convolutional neural networks
Abstract
Purpose
Periodontal disease causes tooth loss and is associated with cardiovascular diseases, diabetes, and rheumatoid arthritis. The present study proposes using a deep learning-based object detection method to identify periodontally compromised teeth on digital panoramic radiographs. A faster regional convolutional neural network (faster R-CNN) which is a state-of-the-art deep detection network, was adapted from the natural image domain using a small annotated clinical data- set.
Materials and Methods
In total, 100 digital panoramic radiographs of periodontally compromised patients were retrospectively collected from our hospital's information system and augmented. The periodontally compromised teeth found in each image were annotated by experts in periodontology to obtain the ground truth. The Keras library, which is written in Python, was used to train and test the model on a single NVidia 1080Ti GPU. The faster R-CNN model used a pretrained ResNet architecture.
Results
The average precision rate of 0.81 demonstrated that there was a significant region of overlap between the predicted regions and the ground truth. The average recall rate of 0.80 showed that the periodontally compromised teeth regions generated by the detection method excluded healthiest teeth areas. In addition, the model achieved a sensitivity of 0.84, a specificity of 0.88 and an F-measure of 0.81.
Conclusion
The faster R-CNN trained on a limited amount of labeled imaging data performed satisfactorily in detecting periodontally compromised teeth. The application of a faster R-CNN to assist in the detection of periodontally compromised teeth may reduce diagnostic effort by saving assessment time and allowing automated screening documentation.
Introduction
Periodontal disease is an inflammatory pathological condition of the gum and bone support surrounding teeth that causes tooth loss. Other inflammatory conditions, such as coronary artery disease, insulin resistance, and arthritis are also present in a considerable proportion of patients with periodontal disease.1 Periodontal disease is an important oral health problem among elderly patients. The biological changes associated with age, including changes in the immune response, may contribute to alterations in the physiological response to microbial plaque that favor the development of periodontal disease.2 The detection and diagnosis of periodontal disease play and essential role in dental care. It is therefore recommended that dentists should perform periodontal assessments of all patients as part of routine oral examinations. A high level of clinical examination skill is required to detect and diagnose periodontal disease. Therefore, it is generally accepted by clinicians that radiographs play a valuable role in supplementing clinical examinations to establish the diagnosis and to guide a patient's treatment plan.
For comprehensive diagnosis and treatment planning, panoramic radiography has been suggested as the standard protocol. Information from panoramic radiographs allows the appropriate selection of additional intraoral periapical radiographs in more specific areas so that the teeth and surrounding bone can be completely examined while administering a lower dose of radiation to the patient. The principal strength of panoramic radiographs is that they show the complete dentition, thereby enabling the diagnosis of impacted teeth; foreign bodies in the jaws; and gross abnormalities in the number, position, and anatomy of the teeth.3 There has been a significant shift toward panoramic radiographs in the workup of patients with periodontitis due to factors including time efficiency, patient tolerance, and low radiation exposure. In conventional periodontal examinations, periapical radiographs and periodontal probes are standard diagnostic tools for diagnosing and predicting periodontally compromised teeth. However, clinical diagnosis and prognostication judgment take time and depend on the clinician having a suitably high skill level.4
Deep learning methods are remarkably beneficial in a broad range of applications, such as natural language processing and image processing. Deep learning, specifically, as implemented using convolutional neural networks (CNNs), has become a conventional technique in medical image detection and classification.5 Computer-based diagnosis is gaining momentum due to the ability of computing systems to detect and diagnose lesions that may be undetected by the human eye.6 In dentistry, a deep CNN algorithm using digital dental radiographic images to predict periodontal disease performed as effectively as experienced periodontists.7 As a recent advancement, deep CNNs have been replaced by faster regional CNNs (faster R-CNNs) because the latter perform reasonably well with a small dataset due to the use of pretrained models.8 In the detection of early esophageal adenocarcinoma, a faster R-CNN performed better than an R-CNN, a fast R-CNN, and a single-shot multibox detector using only 100 images from 39 patients.9
Traditional object detection methods rely on handcrafted features determined by studying the performance achieved by extracting different features and applying a proposed classification or search method.10 Deep learning, specifically as implemented by CNNs, has proven to be effective in various fields, such as detection, classification, segmentation, and several advanced object detection methods use deep learning. In this article, the faster R-CNN proposed by Ren et al. was used,8 which is a combination of the region proposal network (RPN) and the fast R-CNN model.
The performance of the faster R-CNN method was evaluated on our dataset of periodontally compromised teeth.
Materials and Methods
Data set
This retrospective study used our institutional general dentistry registry database with approval from the institutional ethical committee (DENTSWU-EC16/2561). All patients with periodontal disease who were examined and treated between January 2012 and December 2017 were identified. The dataset consisted of 100 anonymized digital panoramic radiographs collected using an Orthophos XG (Sirona, Bensheim, Germany). The mean age of the patients, who consisted of 42 men and 58 women, at the time of the oral examinations was 51.24±12.61 years. The radiographic image files were recorded anonymously according to the research ethics protocol.
Reference data
Our reference data were the clinical and radiographic findings and diagnoses recorded in each patient's periodontal examination chart. The Fédération Dentaire Internationale system uses a 2-digit numbering system, in which the first number represents a tooth's quadrant and the second number represents the number of the tooth from the midline of the face. For permanent teeth, the upper right teeth begin with the number 1. The upper left teeth begin with the number 2. The lower left teeth begin with the number 3. The lower right teeth begin with the number 4. The World Health Organization standardized Community Periodontal Index probe was used to classify whether teeth were healthy or periodontally compromised.11 Teeth with a clinical attachment level of less than 3 mm were diagnosed as healthy. Teeth with bleeding on probing and a clinical attachment level below 6 mm or bone loss of less than 4 mm were classified as moderately periodontally compromised, and teeth with a clinical attachment level of greater than 6 mm and bone loss of more than 4 mm were diagnosed as severely periodontally compromised.11 Moderately and severely periodontally compromised teeth were grouped together to form the periodontally compromised teeth group. In our dataset, 56.3% of teeth were healthy, and 43.7% were periodontally compromised.
The periodontally compromised teeth found on the panoramic radiographs, as shown in Figure 1, were annotated by 3 experts in periodontology to obtain the ground truth. The experts were asked to draw a rectangular bounding box to frame each intact tooth (including the crown and root). Due to differences in the manual segmentation performed by the 3 experts, this study used the largest intersection area of the annotations made by all 3 experts during the training and testing phase.
Fig. 1
Example of a panoramic radiograph from the provided dataset showing periodontally compromised teeth (numbers 14, 23, 25, 26, 37, 35, and 47) annotated by experts.
Experiment
The RPN generates region proposals for each location using the last feature map produced from the CNN based on anchor boxes. The anchor boxes are detection boxes, which have different sizes and ratios that are compared to the ground truth during the training process. By default, this study used 3 scales and 3 aspect ratios, yielding 9 anchors at each sliding position for a convolutional feature map with a typical size of ~2,400. During training, each generated anchor box is compared to the ground truth object location. Boxes that overlap the ground truth with an intersection over union (IoU) exceeding a certain threshold are considered as an object (no class specified). The selected anchor boxes are passed on as region proposals from the RPN stage with a classification score for each box and 4 coordinates that represent the location of this object. Later on, the selected region proposals are fed into the next phase, as in a fast R-CNN. The base CNN used in our model was ResNet-101. Further details on the network structure can be found in the original article and the source code. One of the main benefits of the faster R-CNN is that the convolutional layer between the two networks (RPN and fast R-CNN) is shared; therefore, the faster R-CNN does not involve learning by 2 separate networks.
Due to the limited number of publicly available datasets and to reduce overfitting, the training data were argumented by flipping each image horizontally before feeding it into the machine learning model. The manually cropped input images of periodontally compromised teeth were used to train the network in a minibatch manner. In this way, 70% of the data were randomly chosen for training, 10% for validation, and 20% to determine accuracy. Five-fold cross-validation was applied to evaluate the robustness of the faster R-CNN performance. For implementation, we used the Keras library12 (which is written in Python) to train and test the deep learning object detection model on a single NVidia 1080Ti GPU. The faster R-CNN model used a pretrained ResNet architecture and was trained for 5000 iterations with the learning rate set to 0.0001. The images were used in their original size (1,612×856 pixels).
The input images and their corresponding segmentation maps were used to train the network in a minibatch manner. Gradient descent computation and updates were carried out by a stochastic gradient descent optimizer. To assess the performance of the faster R-CNN object detection method in detecting periodontally compromised teeth on panoramic radiographs, this study employed the average recall rate (ARR) and average precision rate (APR)13 to evaluate the bounding box accuracy. Moreover, the sensitivity, specificity, and F-measure were calculated, as metrics that reflect the number of missed regions in periodontally compromised teeth and any false predictions for healthy teeth. Additionally, if the IoU value between the generated bounding box and the ground truth was less than 0.5, then the produced bounding box was considered to be a false prediction.
Results
This approach yielded a relatively high performance in the detection of periodontally compromised teeth and the lack of detection of healthy teeth. A high APR (0.81) was achieved, demonstrating a significant overlap between the predicted region and the ground truth. A high ARR (0.80) was also observed, showing that the periodontally compromised teeth region generated by the detection method excluded most areas of healthy teeth. In addition, the model achieved a sensitivity of 0.84, a specificity of 0.88, and an F-measure of 0.81.
Some qualitative results of the faster R-CNN object detection method for different cases were randomly chosen as representative results. The bounding box results from the faster R-CNN object detection deep learning method are provided on sample images shown in Figures 2 and 3 in comparison to the bounding box ground truth. An example of a healthy tooth (number 45) image that was falsely predicted by the faster R-CNN method as periodontally compromised is shown in Figure 2. Figure 3 shows examples of the false-negative detections of periodontally compromised teeth (number 17 and 31).
Fig. 2
The original panoramic radiograph (A) and the bounding box output from the faster R-CNN (B) when using 5-fold cross validation showing correct prediction (C) of all periodontally compromised teeth and a false-positive prediction of 1 healthy tooth (number 45). P: periodontally compromised tooth, H: healthy tooth.
Fig. 3
The original panoramic radiograph (A) and the bounding box output from the faster R-CNN (B) when using 5-fold cross validation showing correct prediction (C) of all healthy teeth and a false-negative prediction of 2 periodontally compromised teeth (numbers 17 and 31). P: periodontally compromised tooth, H: healthy tooth.
Discussion
Deep learning requires a large amount of data. However, transfer learning may be accomplished by training the first layers of the deep learning model using other images that contain similar characteristics.14 Although doing so may seem restrictive, using a deep learning model trained on other images and then training the last few layers with radiographic images to perform radiological diagnoses has been reported to be successful.15 The amount of data required for training depends on the variability observed in the process being studied. In dentistry, the radiographic images have a fixed scale for periapical films and panoramic radiographs, and the variation in the appearance of the tooth and jaw structures are small.16 In this study, the input data were panoramic radiographs taken from the same machine. The protocol used to take panoramic radiographs strictly followed the manufacturer's instructions. In addition, the quadrant and symmetrical alignment of the teeth reduced variation in the dataset.
CNNs have been widely used in image classification tasks in dentistry.17, 18 Image classification assigns a class label to an image, while object localization draws a bounding box around 1 or more objects in an image. Object detection combines these 2 tasks by drawing a bounding box around each object of interest in the image and assigning a class label to the image. The R-CNN, fast R-CNN, and faster R-CNN methods have achieved increasingly superior results in object detection tasks.19
Faster R-CNNs have recently been used in tooth detection and numbering.20, 21 Chen et al.20 proposed 3 post-processing techniques to supplement the baseline faster R-CNN according to certain prior domain knowledge. First, a filtering algorithm was constructed to delete overlapping boxes (specifically, those associated with the same tooth) detected by the faster R-CNN. Next, a neural network model was implemented to detect missing teeth. Finally, a rule-based module based on a teeth numbering system was proposed to match the labels of detected teeth boxes to modify the detected results that violate certain intuitive rules. Tuzoff et al.21 demonstrated that a faster R-CNN model could be trained to detect and classify teeth based on tooth number on panoramic radiographs, with possible applications for automated dental recording. The performance of the faster R-CNN was comparable to that of experts.
In our study, a faster R-CNN deep learning model was used for object detection in images. The use of the network yielded a relatively high accuracy of the detected bounding box compared to the accuracy of the ground truth region on the panoramic radiographs. The sensitivity, specificity, and the F-measure demonstrated acceptable performance for the binary classification which supporting the appropriateness of this method. In this study, the performance of the faster R-CNN could not be directly compared to the performance of experts since periodontally compromised teeth are not identified using only radiographic images. We plan to conduct further clinical trials comparing the identification of periodontally compromised teeth using conventional clinical and radiographic findings with and without the support of the faster R-CNN. Other topics for future research include additional techniques for further improving the system output, such as applying more advanced augmentation techniques, extending the dataset, and using more recent CNN architectures.
In summary, this study found that a faster R-CNN trained on a limited number of labeled imaging data had satisfactory detection ability for periodontally compromised teeth. The application of a faster R-CNN to assist in the detection of periodontally compromised teeth may reduce diagnostic effort by saving assessment time and enabling automated screening documentation.
This work was supported in part by Strategic Wisdom and Research Institute, Srinakharinwirot University for the scholarships.
Conflicts of Interest:None
References
-
Ainamo J, Barmes D, Beagrie G, Cutress T, Martin J, Sardo-Infirri J. Development of the World Health Organization (WHO) community periodontal index of treatment needs (CPITN). Int Dent J 1982;32:281–291.
-
-
Keras. The python deep learning library. [cited at 2019 Oct 20].Available from: https://keras.io/.
-
-
Xian M, Zhang Y, Cheng HD. Fully automatic segmentation of breast ultrasound images based on breast characteristics in space and frequency domains. Pattern Recognit 2015;48:485–497.
-
-
Erhan D, Bengio Y, Courville A, Manzagol PA, Vincent P, Bengio S. Why does unsupervised pre-training help deep learning? J Mach Learn Res 2010;11:625–660.
-
-
Greenspan H, van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging 2016;35:1153–1159.
-
-
Yang J, Xie Y, Liu L, Xia B, Cao Z, Guo C. Automated dental image analysis by deep learning on small dataset; Proceedings of 2018 IEEE 42nd Annual Computer Software and Applications Conference; 2018 Jul 23–27; Tokyo, Japan. IEEE; 2018.
-
Cited by
MeSH Terms
Figures
1 / 3
