Abstract
Background: Glioma is one of the most common and aggressive primary brain tumors that endanger human health. Tumors segmentation is a key step in assisting the diagnosis and treatment of cancer disease. However, it is a relatively challenging task to precisely segment tumors considering characteristics of brain tumors and the device noise. Recently, with the breakthrough development of deep learning, brain tumor segmentation methods based on fully convolutional neural network (FCN) have illuminated brilliant performance and attracted more and more attention.
Methods: In this work, we propose a novel FCN based network called SDResU-Net for brain tumor segmentation, which simultaneously embeds dilated convolution and separable convolution into residual U-Net architecture. SDResU-Net introduces dilated block into a residual U-Net architecture, which largely expends the receptive field and gains better local and global feature descriptions capacity. Meanwhile, to fully utilize the channel and region information of MRI brain images, we separate the internal and inter-slice structures of the improved residual U-Net by employing separable convolution operator. The proposed SDResU-Net captures more pixel-level details and spatial information, which provides a considerable alternative for the automatic and accurate segmentation of brain tumors.
Results and Conclusion: The proposed SDResU-Net is extensively evaluated on two public MRI brain image datasets, i.e., BraTS 2017 and BraTS 2018. Compared with its counterparts and stateof- the-arts, SDResU-Net gains superior performance on both datasets, showing its effectiveness. In addition, cross-validation results on two datasets illuminate its satisfying generalization ability.
Keywords: Brain tumor, image segmentation, separable convolution, dilated convolution, residual U-Net, fully convolutional network.
Graphical Abstract
[1]
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Proceedings of the 26th Advances in neural information processing systems. Lake Tahoe: NV, USA 2012; pp. 1097-5.
[2]
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3th International Conference on Learning Representations. San Diego: CA, USA 2015. arXiv 1409.1556
[3]
He KM, Zhang X, Ren SQ, Sun J. Deep residual learning for image recognition. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: NV, USA 2016; pp. 770-8.
[4]
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA 2017; pp. 4700-8.
[5]
Kayalibay B, Jensen G, van der Smagt P. CNN-based segmentation
of medical imaging data arXiv preprint arXiv:170103056. 2017.
[6]
Wang D, Khosla A, Gargeya R, Irshad H, Beck H. Deep learning
for identifying metastatic breast cancer arXiv preprint arXiv:
160605718. 2016.
[7]
Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542(7639): 115-8.
[http://dx.doi.org/10.1038/nature21056] [PMID: 28117445]
[http://dx.doi.org/10.1038/nature21056] [PMID: 28117445]
[8]
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition. Boston: USA 2015; pp. 3431-40.
[9]
Verma H, Agrawal RK, Sharan A. An improved intuitionistic fuzzy c-means clustering algorithm incorporating local information for brain image segmentation. Appl Soft Comput 2016; 46: 543-57.
[http://dx.doi.org/10.1016/j.asoc.2015.12.022]
[http://dx.doi.org/10.1016/j.asoc.2015.12.022]
[10]
Zhang W, Li R, Deng H, et al. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 2015; 108: 214-24.
[http://dx.doi.org/10.1016/j.neuroimage.2014.12.061] [PMID: 25562829]
[http://dx.doi.org/10.1016/j.neuroimage.2014.12.061] [PMID: 25562829]
[11]
Meng Y, Hong L, Xu X, et al. An improved label fusion approach with sparse patch-based representation for MRI brain image segmentation. Int J Imaging Syst Technol 2017; 27(1): 23-32.
[http://dx.doi.org/10.1002/ima.22207]
[http://dx.doi.org/10.1002/ima.22207]
[12]
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image seg-mentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention MIC-CAI Munich: Germany. 2015; 234-41.
[13]
Zhao L, Jia K. Multiscale CNNs for brain tumor segmentation and diagnosis. Comput Math Methods Med 2016; 2016(7) 8356294
[http://dx.doi.org/10.1155/2016/8356294] [PMID: 27069501]
[http://dx.doi.org/10.1155/2016/8356294] [PMID: 27069501]
[14]
Qi Dou , Hao Chen , Lequan Yu , et al. Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Trans Med Imaging 2016; 35(5): 1182-95.
[http://dx.doi.org/10.1109/TMI.2016.2528129] [PMID: 26886975]
[http://dx.doi.org/10.1109/TMI.2016.2528129] [PMID: 26886975]
[15]
Dong H, Yang G, Liu F, Mo Y, Guo Y. Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In: Proceedings of the 21st Annual Conference on Medical Image Understanding and Analysis. Edinburgh: UK 2017; pp. 506-17.
[http://dx.doi.org/10.1007/978-3-319-60964-5_44]
[http://dx.doi.org/10.1007/978-3-319-60964-5_44]
[16]
Zeng G, Zheng G. Multi-stream 3D FCN with multi-scale deep
supervision for multi-modality isointense infant brain MR image
segmentation. In: Proceedings of the 2018 IEEE 15th International
Symposium on Biomedical Imaging;. Washington: DC, USA 2018; pp. 136-40.
[http://dx.doi.org/10.1109/ISBI.2018.8363540]
[http://dx.doi.org/10.1109/ISBI.2018.8363540]
[17]
Chen H, Dou Q, Yu LQ, Qin J, Heng PA. VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images. Neuroimage 2017; 170: 446-55.
[http://dx.doi.org/10.1016/j.neuroimage.2017.04.041]
[http://dx.doi.org/10.1016/j.neuroimage.2017.04.041]
[18]
Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Proceedings of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Greece 2016; pp. 424-32.
[19]
Chen S, Roth H, Dorn S, et al. Towards automatic abdominal multi-
organ segmentation in dual energy CT using cascaded 3D fully
convolutional net-work arXiv preprint arXiv:171005379. 2017.
[20]
Zikic D, Ioannou Y, Brown M, Criminisi A. Segmentation of brain tumor tissues with convolutional neural networks. In: Proceedings of MICCAI-BRATS. Boston: Massachusetts 2014; pp. 36-9.
[21]
Havaei M, Dutil F, Pal C, et al. A convolutional neural network
approach to brain tumor segmentation. In: Proceedings of the 1st International
Workshop on Brain-lesion, Brainles 2015 Held in Conjunction
with International Conference on Medical Image Computing
for Computer-Assisted Intervention;. Munich, Germany 2015; pp. 195-208.
[22]
Dvořák P, Menze B. Local structure prediction with convolutional neural networks for multimodal brain tumor segmentation. In: Proceedings
of the 1st International Workshop on Brain-lesion, Brainles
2015 Held in Conjunction with International Conference on
Medical Image Computing for Computer-Assisted Intervention;. Munich: Germany 2015; pp. 59-71.
[23]
Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 2016; 35(5): 1240-51.
[http://dx.doi.org/10.1109/TMI.2016.2538465] [PMID: 26960222]
[http://dx.doi.org/10.1109/TMI.2016.2538465] [PMID: 26960222]
[24]
Urban G, Bendszus M, Hamprecht F. Multi-modal brain tumor segmentation using deep convolutional neural networks. In: International MICCAI Workshop on Medical Computer Vision. Munich: Germany 2015; pp. 59-71.
[25]
Havaei M, Davy A, Warde-Farley D, et al. Brain tumor segmentation with deep neural networks. Med Image Anal 2017; 35: 18-31.
[http://dx.doi.org/10.1016/j.media.2016.05.004] [PMID: 27310171]
[http://dx.doi.org/10.1016/j.media.2016.05.004] [PMID: 27310171]
[26]
Kamnitsas K, Ledig C, Newcombe VFJ, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 2017; 36: 61-78.
[http://dx.doi.org/10.1016/j.media.2016.10.004] [PMID: 27865153]
[http://dx.doi.org/10.1016/j.media.2016.10.004] [PMID: 27865153]
[27]
Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y. A deep learning model integrating FCNNs and CRFs for brain tumor segmentation. Med Image Anal 2018; 43: 98-111.
[http://dx.doi.org/10.1016/j.media.2017.10.002] [PMID: 29040911]
[http://dx.doi.org/10.1016/j.media.2017.10.002] [PMID: 29040911]
[28]
Zheng S, Jayasumana S, Romera-Paredes B, et al. Conditional random fields as recurrent neural networks. In: Proceedings of the 15th IEEE International Conference on Computer Vision. Boston: USA 2015; pp. 1529-37.
[http://dx.doi.org/10.1109/ICCV.2015.179]
[http://dx.doi.org/10.1109/ICCV.2015.179]
[29]
Menze BH, Jakab A, Bauer S, et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans Med Imaging 2015; 34(10): 1993-2024.
[http://dx.doi.org/10.1109/TMI.2014.2377694] [PMID: 25494501]
[http://dx.doi.org/10.1109/TMI.2014.2377694] [PMID: 25494501]
[30]
Wang G, Li W, Ourselin S, Vercauteren T. Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Proceedings of the 3rd International Workshop on Brain-lesion, BrainLes 2017 Held in Conjunction with Medical Image Computing for Computer Assisted Intervention. Quebec City: QC, Canada 2017; pp. 178-90.
[31]
Tseng KL, Lin YL, Hsu W, Huang CY. Joint sequence learning and cross-modality convolution for 3d biomedical segmentation. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: HI, USA 2017; pp. 3739-46.
[http://dx.doi.org/10.1109/CVPR.2017.398]
[http://dx.doi.org/10.1109/CVPR.2017.398]
[32]
Kolnogorov AV. Gaussian two-armed bandit and optimization of batch data processing. Probl Inf Transm 2018; 54(1): 84-100.
[http://dx.doi.org/10.1134/S0032946018010076]
[http://dx.doi.org/10.1134/S0032946018010076]
[33]
Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. In: Proceedings of the 4th International Conference on Learning Representations. San Juan: Puerto Rico 2016; pp. 1-6.
[34]
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-resNET and the impact of residual connections on learning. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco: CA, USA 2017; pp. 4278-84.
[35]
Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: HI, USA 2017; pp. 1800-7.
[http://dx.doi.org/10.1109/CVPR.2017.195]
[http://dx.doi.org/10.1109/CVPR.2017.195]
[36]
Bakas S, Akbari H, Sotiras A, et al. Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci Data 2017; 2017 4170117
[http://dx.doi.org/10.1038/sdata.2017.117] [PMID: 28872634]
[http://dx.doi.org/10.1038/sdata.2017.117] [PMID: 28872634]
[37]
Abadi M. TensorFlow AABP. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation. Savannah: Georgia 2016; pp. 265-83.
[38]
He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the 15th IEEE international conference on computer vision. Boston: USA 2015; pp. 1026-34.
[http://dx.doi.org/10.1109/ICCV.2015.123]
[http://dx.doi.org/10.1109/ICCV.2015.123]
[39]
Kermi A, Mahmoudi I, Khadir MT. Deep convolutional neural networks using U-Net for automatic brain tumor segmentation in multimodal MRI volumes. In: Proceedings of the 4th International Workshop on Brain-lesion, BrainLes 2018 Held in Conjunction with Medical Image Computing for Computer Assisted Intervention, MICCAI 2018. Granada, Spain. Cham 2018. Springer
Verlag; 2018 Sep 16-16 ; pp. 37-48.
Article Metrics
17
4