Login Register Cart 0
Generic placeholder image

Current Medical Imaging

Editor-in-Chief

ISSN (Print): 1573-4056
ISSN (Online): 1875-6603

Back
Journal Home About Journal Editorial Board Journal Insight Current Issue Volumes/Issues
Subscribe
Research Article

Quality Assurance based on Deep Learning for Pelvic OARs Delineation in Radiotherapy

Author(s): Hang Yu, Yisong He, Yuchuan Fu*, Xia Li, Jun Zhang and Huan Liu

Volume 19, Issue 4, 2023

Published on: 26 August, 2022

Article ID: e210622206246 Pages: 9

DOI: 10.2174/1573405618666220621121225

Price: $65

Abstract

Background: Correct delineation of organs at risk (OARs) is an important step for radiotherapy and it is also a time-consuming process that depends on many factors.

Objective: An automatic quality assurance (QA) method based on deep learning (DL) was proposed to improve efficiency for detecting contouring errors of OARs.

Materials and Methods: A total of 180 planning CT scan sets at the pelvic site and the corresponding OARs contours from clinics were enrolled in this study. Among them, 140 cases were randomly chosen as the training datasets, 20 cases were used as the validation datasets, and the remaining 20 cases were used as the test datasets. DL-based models were trained through data curation for data cleaning based on the Dice similarity coefficient and the 95th percentile Hausdorff distance between the original contours and the predicted contours. All contouring errors could be classified into two types; minor modification required and major modification required. The pass criteria were established using Bias- Corrected and Accelerated bootstrap on 20 manually reviewed validation datasets. The performance of the QA method was evaluated with the metrics of sensitivity, specificity, the area under the receiving operator characteristic curve (AUC), and detection rate sensitivity on the 20 test datasets.

Results: For all OARs, segmentation results after data curation were superior to those without. The sensitivity of the QA method was greater than 0.890 and the specificity was higher than 0.975. The AUCs were 0.948, 0.966, 0.965, and 0.932 for the bladder, right femoral head, left femoral head, and rectum, respectively. Almost all major errors could be detected by the automatic QA method, and the lowest detection rate sensitivity of minor errors was 0.863 for the rectum.

Conclusions: QA of OARs is an important step for the correct implementation of radiotherapy. The DL-based QA method proposed in this study showed a high potential to automatically detect contouring errors with high precision. The method can be integrated into the existing radiotherapy procedures to improve the efficiency of delineating the OARs.

Keywords: Quality assurance, deep learning, radiotherapy, contouring, bias-corrected and accelerated bootstrap, organs at risk

Graphical Abstract

[1]
Vinod SK, Jameson MG, Min M, Holloway LC. Uncertainties in volume delineation in radiation oncology: A systematic review and recommendations for future studies. Radiother Oncol 2016; 121(2): 169-79.
[http://dx.doi.org/10.1016/j.radonc.2016.09.009] [PMID: 27729166]
[2]
Vaassen F, Hazelaar C, Canters R, Peeters S, Petit S, van Elmpt W. The impact of organ-at-risk contour variations on automatically generated treatment plans for NSCLC. Radiother Oncol 2021; 163: 136-42.
[http://dx.doi.org/10.1016/j.radonc.2021.08.014] [PMID: 34461185]
[3]
Moghaddasi L, Bezak E, Marcu LG. Current challenges in clinical target volume definition: Tumour margins and microscopic extensions. Acta Oncol 2012; 51(8): 984-95.
[http://dx.doi.org/10.3109/0284186X.2012.720381] [PMID: 22998477]
[4]
Marks LB, Adams RD, Pawlicki T, et al. Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: Executive summary. Pract Radiat Oncol 2013; 3(3): 149-56.
[http://dx.doi.org/10.1016/j.prro.2012.11.010] [PMID: 24175002]
[5]
Lo AC, Liu M, Chan E, et al. The impact of peer review of volume delineation in stereotactic body radiation therapy planning for primary lung cancer: A multicenter quality assurance study. J Thorac Oncol 2014; 9(4): 527-33.
[http://dx.doi.org/10.1097/JTO.0000000000000119] [PMID: 24736076]
[6]
Altman MB, Kavanaugh JA, Wooten HO, et al. A framework for automated contour quality assurance in radiation therapy including adaptive techniques. Phys Med Biol 2015; 60(13): 5199-209.
[http://dx.doi.org/10.1088/0031-9155/60/13/5199] [PMID: 26083863]
[7]
Hui CB, Nourzadeh H, Watkins WT, et al. Quality assurance tool for organ at risk delineation in radiation therapy using a parametric statistical approach. Med Phys 2018; 45(5): 2089-96.
[http://dx.doi.org/10.1002/mp.12835] [PMID: 29481703]
[8]
McIntosh C, Svistoun I, Purdie TG. Groupwise conditional random forests for automatic shape classification and contour quality assessment in radiotherapy planning. IEEE T Med Imaging 2013; 32(6): 1043-57.
[http://dx.doi.org/10.1109/TMI.2013.2251421] [PMID: 23475352]
[9]
Chen HC, Tan J, Dolly S, et al. Automated contouring error detection based on supervised geometric attribute distribution models for radiation therapy: A general strategy. Med Phys 2015; 42(2): 1048-59.
[http://dx.doi.org/10.1118/1.4906197] [PMID: 25652517]
[10]
Joskowicz L, Cohen D, Caplan N, Sosna J. Automatic segmentation variability estimation with segmentation priors. Med Image Anal 2018; 50: 54-64.
[http://dx.doi.org/10.1016/j.media.2018.08.006] [PMID: 30208356]
[11]
Warfield SK, Zou KH, Wells WM. Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation. IEEE Trans Med Imaging 2004; 23(7): 903-21.
[http://dx.doi.org/10.1109/TMI.2004.828354] [PMID: 15250643]
[12]
Valindria VV, Lavdas I, Bai W, et al. Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth. IEEE T Med Imaging 2017; 36(8): 1597-606.
[http://dx.doi.org/10.1109/TMI.2017.2665165] [PMID: 28436849]
[13]
Aljabar P, Heckemann RA, Hammers A, Hajnal JV, Rueckert D. Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy. Neuroimage 2009; 46(3): 726-38.
[http://dx.doi.org/10.1016/j.neuroimage.2009.02.018] [PMID: 19245840]
[14]
Xie X, Niu J, Liu X, Chen Z, Tang S, Yu S. A survey on incorporating domain knowledge into deep learning for medical image analysis. Med Image Anal 2021; 69: 101985.
[http://dx.doi.org/10.1016/j.media.2021.101985] [PMID: 33588117]
[15]
Lin L, Dou Q, Jin YM, et al. Deep learning for automated contouring of primary tumor volumes by MRI for nasopharyngeal carcinoma. Radiology 2019; 291(3): 677-86.
[http://dx.doi.org/10.1148/radiol.2019182012] [PMID: 30912722]
[16]
Gao Y, Huang R, Yang Y, et al. FocusNetv2: Imbalanced large and small organ segmentation with adversarial shape constraint for head and neck CT images. Med Image Anal 2021; 67: 101831.
[http://dx.doi.org/10.1016/j.media.2020.101831] [PMID: 33129144]
[17]
Men K, Zhang T, Chen X, et al. Fully automatic and robust segmentation of the clinical target volume for radiotherapy of breast cancer using big data and deep learning. Phys Medica 2018; 50: 13-9.
[http://dx.doi.org/10.1016/j.ejmp.2018.05.006] [PMID: 29891089]
[18]
Liu Z, Liu X, Xiao B, et al. Segmentation of organs-at-risk in cervical cancer CT images with a convolutional neural network. Phys Medica 2020; 69: 184-91.
[http://dx.doi.org/10.1016/j.ejmp.2019.12.008] [PMID: 31918371]
[19]
Van der Veen J, Gulyban A, Willems S, Maes F, Nuyts S. Interobserver variability in organ at risk delineation in head and neck cancer. Radiat Oncol 2021; 16(1): 120.
[http://dx.doi.org/10.1186/s13014-020-01677-2] [PMID: 34183040]
[20]
Van der Veen J, Willems S, Deschuymer S, et al. Benefits of deep learning for delineation of organs at risk in head and neck cancer. Radiother Oncol 2019; 138: 68-74.
[http://dx.doi.org/10.1016/j.radonc.2019.05.010] [PMID: 31146073]
[21]
Chen X, Men K, Chen B, et al. CNN-based quality assurance for automatic segmentation of breast cancer in radiotherapy. Front Oncol 2020; 10: 524.
[http://dx.doi.org/10.3389/fonc.2020.00524] [PMID: 32426272]
[22]
Hall P. Theoretical comparison of bootstrap confidence intervals. Ann Stat 1988; 16(3): 927-53.
[23]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. Int Conference on Med Image Comput & Comput-Assist Intervention. 2015; 9351: 234-41.
[http://dx.doi.org/10.1007/978-3-319-24574-4_28]
[24]
He Y, Zhang S, Luo Y. Quantitative comparisons of deep-learning-based and atlas-based auto-segmentation of the intermediate risk clinical target volume for nasopharyngeal carcinoma. Curr Med Imaging 2022; 18(3): 335-45.
[25]
Rhee DJ, Jhingran A, Rigaud B, et al. Automatic contouring system for cervical cancer using convolutional neural networks. Med Phys 2020; 47(11): 5648-58.
[http://dx.doi.org/10.1002/mp.14467] [PMID: 32964477]
[26]
Crum WR, Camara O, Hill DLG. Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans Med Imaging 2006; 25(11): 1451-61.
[http://dx.doi.org/10.1109/TMI.2006.880587] [PMID: 17117774]
[27]
Huttenlocher D P, Klanderman G A, Rucklidge W J. Comparing images using the hausdorff distance. IEEE T Pattern Anal 1993; 15(9): 850-63.
[http://dx.doi.org/10.1109/34.232073]
[28]
Papaconstadopoulos P, González P, Carbaat C, et al. An anomaly detector as a clinical decision support system for parotid gland delineations. Phys Med Biol 2021; 66(10): 105017.
[http://dx.doi.org/10.1088/1361-6560/abfbf5] [PMID: 33906177]
[29]
Bisong E. TensorFlow 20 and Keras. Building Machine Learning and Deep Learning Models on Google Cloud Platform. Apress, Berkeley, CA 2019; pp. 347-99.
[30]
Weisstein EW. 2002. Available from: https://mathworld.wolfram.com/Skewness.html (Assessed on June 10th, 2022).
[31]
Bishara AJ, Hittner JB. Confidence intervals for correlations when data are not normal. Behav Res Methods 2017; 49(1): 294-309.
[http://dx.doi.org/10.3758/s13428-016-0702-8] [PMID: 26822671]
[32]
Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Inf Process Manage 2009; 45(4): 427-37.
[http://dx.doi.org/10.1016/j.ipm.2009.03.002]
[33]
Chen X, Sun S, Bai N, et al. A deep learning-based auto-segmentation system for organs-at-risk on whole-body computed tomography images for radiation therapy. Radiother Oncol 2021; 160: 175-84.
[http://dx.doi.org/10.1016/j.radonc.2021.04.019] [PMID: 33961914]
[34]
Shah VP, Lakshminarayanan P, Moore J, et al. Data integrity systems for organ contours in radiation therapy planning. J Appl Clin Med Phys 2018; 19(4): 58-67.
[http://dx.doi.org/10.1002/acm2.12353] [PMID: 29893465]
[35]
Min H, Dowling J, Jameson MG, et al. Automatic radiotherapy delineation quality assurance on prostate MRI with deep learning in a multicentre clinical trial. Phys Med Biol 2021; 66(19): 195008.
[http://dx.doi.org/10.1088/1361-6560/ac25d5] [PMID: 34507305]
[36]
Men K, Geng H, Biswas T, Liao Z, Xiao Y. Automated quality assurance of oar contouring for lung cancer based on segmentation with deep active learning. Front Oncol 2020; 10: 986.
[http://dx.doi.org/10.3389/fonc.2020.00986] [PMID: 32719742]
[37]
Rhee DJ, Cardenas CE, Elhalawani H, et al. Automatic detection of contouring errors using convolutional neural networks. Med Phys 2019; 46(11): 5086-97.
[http://dx.doi.org/10.1002/mp.13814] [PMID: 31505046]
[38]
Kim H, Monroe JI, Lo S, et al. Quantitative evaluation of image segmentation incorporating medical consideration functions. Med Phys 2015; 42(6): 3013-23.
[http://dx.doi.org/10.1118/1.4921067] [PMID: 26127054]
[39]
Mir R, Kelly SM, Xiao Y, et al. Organ at risk delineation for radiation therapy clinical trials: Global Harmonization Group Consensus Guidelines. Radiother Oncol 2020; 150: 30-9.
[http://dx.doi.org/10.1016/j.radonc.2020.05.038] [PMID: 32504762]
[40]
Li XA, Tai A, Arthur DW, et al. Variability of target and normal structure delineation for breast cancer radiotherapy: An RTOG Multi-Institutional and Multiobserver Study. Int J Radiat Oncol 2009; 73(3): 944-51.
[http://dx.doi.org/10.1016/j.ijrobp.2008.10.034] [PMID: 19215827]

Rights & Permissions Print Cite
Article Metrics
1
© 2024 Bentham Science Publishers | Privacy Policy