Abstract
Background: This review aims to identify the development of the algorithms for brain tissue and structure segmentation in MRI images.
Discussion: Starting from the results of the Grand Challenges on brain tissue and structure segmentation held in Medical Image Computing and Computer-Assisted Intervention (MICCAI), this review analyses the development of the algorithms and discusses the tendency from multi-atlas label fusion to deep learning. The intrinsic characteristics of the winners’ algorithms on the Grand Challenges from the year 2012 to 2018 are analyzed and the results are compared carefully.
Conclusion: Although deep learning has got higher rankings in the challenge, it has not yet met the expectations in terms of accuracy. More effective and specialized work should be done in the future.
Keywords: Brain tissue, segmentation, grand challenge, multi-atlas label fusion, deep learning, algorithms.
Graphical Abstract
[1]
Zhu LJ, Zhu YG, Cao HQ, Dong ED. The funding and prospect of global brain research. Sci Found China 2013; 27(6): 359-62.
[2]
Van Erp TG, Hibar DP, Rasmussen JM, et al. Subcortical brain volume abnormalities in 2028 individuals with schizophrenia and 2540 healthy controls via the enigma consortium. Mol Psychiatry 2016; 21(4): 547-53.
[3]
Geevarghese R, Lumsden DE, Hulse N, Samuel M, Ashkan K. Subcortical structure volumes and correlation to clinical variables in Parkinson’s disease. J Neuroimaging 2015; 25(2): 275-80.
[4]
Goldman S, O’Brien LM, Filipek PA, Rapin I, Herbert MR. Motor stereotypies and volumetric brain alterations in children with autistic disorder. Res Autism Spectr Disord 2013; 7(1): 82-92.
[5]
Dolz J, Massoptier L, Vermandel M. Segmentation algorithms of subcortical brain structures on MRI for radiotherapy and radiosurgery: A survey. IRBM 2015; 36(4): 200-12.
[6]
Gui L, Lisowski R, Faundez T, Huppi PS, Lazeyras F, Kocher M. Morphology-driven automatic segmentation of MR images of the neonatal brain. Med Image Anal 2012; 16(8): 1565-79.
[7]
Despotovic I, Goossens B, Philips W. MRI segmentation of the human brain: Challenges, methods, and applications. Comput Math Methods Med 2015; 2015450341
[8]
Mendrik AM, Vincken KL, Kuijf HJ, et al. MRBrainS challenge: an online evaluation framework for brain image segmentation in 3T MRI scans. Comput Intell Neurosci 2015; 2015813696
[9]
Qaiser M, Alipoor M, Chodorowski A, Mehnert A, Persson M. Multimodal MR brain segmentation using Bayesian based adaptive mean-shift (bams). In: Proceedings of the MICCAI Grand Challenge on MR Brain Image Segmentation (MRBrainS13). 2013 Sep
26th; Nagoya, Japan; pp. 1-7.
[10]
Rajchl M, Baxter JSH, McLeod AJ, et al. Map-based brain tissue segmentation using manifold learning and hierarchical max-flow regularization. In: Proceedings of the MICCAI Grand Challenge on MR Brain Image Segmentation (MRBrainS13). 2013 Sep 26th; Nagoya,
Japan; pp 1-5.
[11]
Sarikaya D, Zhao L, Corso JJ. Multi-atlas brain MRI segmentation
with mul- tiway cut. In: Proceedings of the MICCAI Grand Challenge
on MR Brain Image Segmentation (MRBrainS13); 2013 Sep
26th; Nagoya, Japan. Available from: http://hdl.handle.net/10380/3448
[12]
Wang C, Smedby Ö. Fully automatic brain segmentation using model-guided level sets and skeleton-based models. In: Proceedings of the MICCAI Grand Challenge on MR Brain Image Segmentation (MRBrainS13). 2013 Sep 26th; Nagoya, Japan.
[13]
Valverde S, Oliver A, Roura E, et al. Automated tissue segmentation of MR brain images in the presence of white matter lesions. Med Image Anal 2017; 35(1): 446-57.
[14]
Valverde S, Oliver A, Cabezas M, Roura E, Llado X. Comparison of 10 brain tissue segmentation methods using revisited IBSR annotations. J Magn Reson Imaging 2015; 41(1): 93-101.
[15]
Iglesias JE, Sabuncu MR. Multi-atlas segmentation of biomedical images: A survey. Med Image Anal 2015; 24(1): 205-19.
[16]
Sanroma G, Benkarim OM, Piella G, Ballester M. Building an ensemble of complementary segmentation methods by exploiting probabilistic estimates. International Workshop on Machine Learning
in Medical Imaging; Lecture Notes in Computer Science 2016;
Springer: pp. 27-35.
[17]
Huo YK, Xu ZB, Aboud K, et al. Spatially localized atlas network
tiles enable 3D whole brain segmentation from limited data.
arXiv:1806.00546v2 [cs.CV], 2018: 1-8.
[18]
2013 MICCAI Challenge Workshop on Segmentation: Algorithms,
Theory and Applications (“SATA”). Available from: https://masiweb.vuse.vanderbilt.edu/workshop2012/index.php/Main_Page Access date: Jul. 27, 2018.
[20]
Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with
deep convolutional neural networks. In: Proceedings of the 25th International
Confernce on Neural Information Processing Systems,;
2012 Dec 3rd-6th; Lake Tahoe, Nevada; pp. 1097-105
[21]
Large Scale Visual Recognition Challenge 2012 (ILSVRC2012).
Available from: http://image-net.org/challenges/LSVRC/2012/. Access date: Jul.
27, 2018.
[22]
Alom MZ, Taha TM, Yakopcic C, et al. The history began from
AlexNet: A comprehensive survey on deep learning approaches.
arXiv:1803.01164 [cs.CV], 2018: 1-39.
[23]
Chen H, Dou Q, Yu L, Heng PA. VoxResNet: Deep voxelwise
residual networks for volumetric brain segmentation. arXiv:
1608.05895 [cs.CV], 2016.
[24]
Cicek O, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O. 3D
U-Net: Learning dense volumetric segmentation from sparse annotation.
In: International Conference on Medical Image Computing
and Computer-Assisted Intervention-MICCAI 2016; Lecture Notes
in Computer Science 2016: Springer. Athens, Greece; pp. 424-32
[25]
Xu YC, Géraud T, Bloch I. From neonatal to adult brain MR image
segmentation in a few seconds using 3D-like fully convolutional
network and transfer learning. In: IEEE International Conference
on Image Processing (ICIP); Sep 2017, Beijing, China; pp. 17-
4421.
[26]
Greenspan H, Ginneken BV, Summers RM. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Trans Med Imaging 2016; 35(5): 1153-9.
[27]
Dolz J, Desrosiers C, Ayed IB. 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study. Neuroimage 2018; 170(4): 456-70.
[28]
Shen DG, Wu GR, Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng 2017; 19: 221-48.
[29]
Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42(9): 60-88.
[31]
Andermatt S, Pezold S, Amann M, Cattin PC. Multi-dimensional
gated recurrent units for automated anatomical landmark localization.
arXiv: 1708.02766 [cs.CV] 2017: 1-8.
[32]
Dolz J, Gopinath K, Yuan J, Lombaert H, Desrosiers C, Ayed IB. HyperDense-Net: A hyper-densely connected CNN for multimodal
image segmentation. arXiv: 1804.02967v1 [cs.CV] 2018: 1-
13.
[33]
Mahbod A. Chowdhury, Smedby O, Wang CL. Automatic brain segmentation using artificial neural networks with shape context. Pattern Recognit Lett 2018; 101: 74-9.
[34]
Berna l J, Kushibar K, Cabezas M, Valverde S, Oliver A, Llado X.
Quantitative analysis of patch-based fully convolutional neural
networks for tissue segmentation on brain magnetic resonance imaging.
arXiv: 1801.06457v2 [cs.CV] 2018.
[35]
Bernal J, Kushibar K, Asfaw DS, et al. Deep convolutional neural
networks for brain image analysis on magnetic resonance imaging:
A review. arXiv: 1712.03747 [cs.CV] 2017.
[36]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks
for biomedical image segmentation. In: International Conference
on Medical Image Computing and Computer-Assisted Intervention-
MICCAI 2015; Lecture Notes in Computer Science 2015: Springer;
Munich, Germany; pp. 234-41.
[37]
Milletari F, Navab N, Ahmadi SA. V-Net: Fully convolutional
neural networks for volumetric medical image segmentation. In:
Fourth International Conference on 3D Vision; 2016 Oct 25-28;
IEEE. Stanford, CA, USA; pp. 565-71.
[38]
Ganaye PA, Sdika M, Benoit-Cattin H. Towards integrating spatial
localization in convolutional neural networks for brain image segmentation.
arXiv: 1804.04563v1 [cs.CV] 2018: 1-5.
[39]
Rajchl M, Pawlowski N, Rueckert D, Matthews PM, Glocker B. NeuroNet: Fast and robust reproduction of multiple brain image
segmentation pipelines. arXiv: 1806.04224v1 [cs.CV] 2018: 1-9.
[40]
Moeskops P, Veta M, Lafarge MW, Eppenhof KA, Pluim JP. Adversarial training and dilated convolutions for brain MRI segmentation.
Deep Learning in Medical Image Analysis and Multimodal
Learning for Clinical Decision Support 2017; Lecture Notes
in Computer Science 2017; Springer: pp. 56-64
[41]
Kushibar K, Valverde S, Gonzalez-Villa S, et al. Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features. Med Image Anal 2018; 48(6): 177-86.
[42]
Guerrero R, Qin C, Oktay O, et al. White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. Neuroimage Clin 2017; 17: 918-34.
[43]
Ghafoorian M, Karssemeijer N, Heskes T, et al. Location sensitive deep convolutional neural networks for segmentation of white matter hyperintensities. Nat Sci Rep 2017; 7(1): 5110.
[44]
Pham DL, Xu C, Prince JL. Current methods in medical image segmentation. Annu Rev Biomed Eng 2000; 2(1): 315-37.
[45]
Doan NT, de Xivry JO, Macq B. Effect of inter-subject variation on the accuracy of atlas-based segmentation applied to human brain structures. SPIE Med Imaging 2010; 7623(1): 666-73.
[46]
Marcus DS, Fotenos AF, Csemansky JG, Morris JC, Buckner RL. Open Access Series of Imaging Studies (OASIS): Longitudinal MRI data in nondemented and demented older adults. J Cogn Neurosci 2007; 19(9): 1498-507.
[47]
Wang HZ, Suh JW, Sandhitsu RD, Pluta J, Craige C, Yushkevich PA. Multi-atlas segmentation with joint label fusion. IEEE Trans Pattern Anal Mach Intell 2013; 35(3): 611-23.
[48]
Asman AJ, Landman BA. Non-local statistical label fusion for multi-atlas segmentation. Med Image Anal 2013; 17(2): 194-208.
[49]
Simonyan K, Zisserman A. Very deep convolutional networks for
large-scale image recognition. arXiv:1409.1556 [cs.CV] 2014.
[50]
Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions.
In: IEEE Conference on Computer Vision and Pattern Recognition
(CVPR); 2015 Jun 7-12; Boston, USA; pp. 1-9.
[51]
He KM, Zhang XY, Ren SQ, Sun J. Deep residual learning for
image recognition. In: IEEE conference on Computer Vision and
Pattern Recognition (CVPR); 2016 Jun 27-30; Las Vegas, USA;
pp. 770-8,
[53]
Opbroek A, Lijn F, Bruijne M. Automated brain-tissue segmentation
by multi-feature SVM classification. In: Proceedings of the
MICCAI Grand Challenge on MR Brain Image Segmentation
(MRBrainS13); 2013 Sep 26th; Nagoya, Japan.
[54]
Alansary A, Soliman A, Khalifa F, et al. MAP-based framework
for segmentation of MR brain images based on visual appearance
and prior shape. In: Proceedings of the MICCAI Grand Challenge
on MR Brain Image Segmentation (MRBrainS13); 2013 Sep 26th;
Nagoya, Japan.
[55]
Moeskopsb P, Viergever MA, Benders M, Isgum I. Evaluation of an automatic brain segmentation method developed for neonates on adult MR brain images. SPIE Medical Imaging 2015; 9413: 1-6.
[56]
Pereira S, Pinto A, Oliveira J, Mendrik AM, Correia JH, Silva CA. Automatic brain tissue segmentation in MR images using Random Forests and Conditional Random Fields. J Neurosci Methods 2016; 270: 111-23.
[57]
Stollenga MF, Byeon W, Liwicki M, Schmidhuber J. Parallel
Multi-Dimensional LSTM, with application to fast biomedical
volumetric image segmentation. In: Proceedings of the 28th International
Conference on Neural Information Processing Systems
NIPS’15; 2015 Dec 7-12; Cambridge, MA, USA: MIT Press; pp.
2998-3006.
[58]
Wang L, Gao YZ, Shi F, et al. LINKS: Learning-based multi-source Integration framework for segmentation of infant brain images. Neuroimage 2015; 108: 160-72.
[59]
Chen H, Dou Q, Yu LQ, Qin J, Heng PA. VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images. Neuroimage 2018; 170(4): 446-55.
[60]
Andermatt S, Pezold S, Cattin P. Multi-dimensional Gated Recurrent
Units for the segmentation of biomedical 3d-data. In: Carneiro
G Eds. Deep learning and Data Labeling for Medical Applications.
LABELS 2016/DLMIA 2016, Lecture Notes in Computer Science;
Springer; 2016; pp. 142-51.
[61]
Bui TD, Shin J, Moon T. 3D densely convolutional networks for
volumetric segmentation. Comput Vis Patt Recogn 2017: 1-7.
[62]
Fung G, Deng Y, Zhao Q, et al. Distinguishing bipolar and major depressive disorders by brain structural morphometry: A pilot study. BMC Psychiatry 2015; 15(1): 298.
[63]
Chen Y, Storrs J, Tan L, Mazlack LJ, Lee JH, Lu LJ. Detecting brain structural changes as biomarker from magnetic resonance images using a local feature based SVM approach. J Neurosci Methods 2014; 221(2): 22-31.
[64]
Giorgio A, De Stefano N. Clinical use of brain volumetry. J Magn Reson Imaging 2013; 37(1): 1-14.
[65]
Cardoso MJ, Melbourne A, Kendall GS, et al. AdaPT: An adaptive preterm segmentation algorithm for neonatal brain MRI. Neuroimage 2013; 65(1): 97-108.
[66]
Kim YK, Na KS. Application of machine learning classification for
structural brain MRI in mood disorders: Critical review from a
clinical perspective. Prog Neuropsychopharmacol Biol Psychiatry
2018; 80 (Pt B): 71-80.
[67]
Zhang WL, Li RJ, Deng HT, et al. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 2015; 108(3): 214-24.
[68]
Brebisson A, Montana G. Deep neural networks for anatomical
brain segmentation. In: IEEE conference on Computer Vision and
Pattern Recognition (CVPR); 2015 June 7-12; Boston, USA; pp.
20-8.
[69]
Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MA, Isgum I. Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans Med Imaging 2016; 35(5): 1252-62.
[70]
Choi H, Jin KH. Fast and robust segmentation of the striatum using deep convolutional neural networks. J Neurosci Methods 2016; 274(12): 146-53.
[71]
Nie D, Wang L, Gao Y, Shen D. Fully convolutional networks for
multi-modality isointense infant brain image segmentation. In:
IEEE International Symposium on Biomedical Imaging; 2016 April
13-16, Prague, Czech Republic; pp. 1342-5.
[72]
Milletari F, Ahmadi SA, Kroll C, et al. Hough-CNN: Deep learning
for segmentation of deep brain regions in MRI and ultrasound.
Comput Vis Image Underst 2017; 164(SI): 92-102.
[73]
Bao S, Chung ACS. Multi-scale structured CNN with label consistency for brain MR image segmentation. Comput Method Biomec 2016; 6(1): 113-7.
[74]
Shakeri M, Tsogkas S, Ferrante E, et al. Sub-cortical brain structure
segmentation using F-CNNs. In: IEEE International Symposium
on Biomedical Imaging; 2016 April 13-16, Prague, Czech
Republic; pp. 269-72.
Article Metrics
37
2