Generic placeholder image

Current Medical Imaging

Editor-in-Chief

ISSN (Print): 1573-4056
ISSN (Online): 1875-6603

Review Article

Image Integration Procedures in Multisensory Medical Images: A Comprehensive Survey of the State-of-the-art Paradigms

Author(s): Ayush Dogra*, Chirag Kamal Ahuja and Sanjeev Kumar

Volume 18, Issue 5, 2022

Published on: 08 March, 2021

Article ID: e150322192108 Pages: 20

DOI: 10.2174/1573405617666210308112825

Price: $65

Abstract

Background: Obtaining the medical history from a patient is a tedious task for doctors as it depends on a lot of factors which are difficult to keep track from a patient’s perspective. Doctors have to rely upon technological tools to make a swift and accurate judgment about the patient’s health.

Introduction: Out of many such tools, there are two special imaging modalities known as X-ray - Computed Tomography (CT) and Magnetic Resonance imaging (MRI) which are of significant importance in the medical world assisting the diagnosis process.

Methods: The advancement in signal processing theory and analysis has led to the design and implementation of a large number of image processing and fusion algorithms. Each of these methods has evolved in the terms of their computational efficiency and visual results over the years.

Results: Various researches have revealed their properties in terms of their efficiency and outreach and it has been concluded that image fusion can be a very suitable process that can help to compensate for the drawbacks.

Conclusion: In this manuscript, recent state-of-the-art techniques have been used to fuse these image modalities and established its need and importance in a more intuitive way with the help of a wide range of assessment parameters.

Keywords: Multi-modal imaging, image fusion, CT, MRI, transform domain, spatial domain.

Graphical Abstract

[1]
Kaya B, Goceri E, Becker A, et al. Automated fluorescent miscroscopic image analysis of PTBP1 expression in glioma. PLoS One 2017; 12(3): e0170991.
[http://dx.doi.org/10.1371/journal.pone.0170991] [PMID: 28282372]
[2]
Evgin G, Numan G. Deep learning in medical image analysis: Recent advances and future trends. Dig Lib 2017; 305-10.
[3]
Goceri E. Automatic labeling of portal and hepatic veins from MR images prior to liver transplantation. Int J CARS 2016; 11(12): 2153-61.
[http://dx.doi.org/10.1007/s11548-016-1446-8] [PMID: 27338273]
[4]
Dogra A, Goyal B, Agrawal S. From multi-scale decomposition to non-multi-scale decomposition methods: A comprehensive survey of image fusion techniques and its applications. IEEE Access 2017; 5: 16040-67.
[http://dx.doi.org/10.1109/ACCESS.2017.2735865]
[5]
Goceri E, Songul C. Biomedical information technology: Image based computer aided diagnosis systems. International Conference on Advanced Technologies. Antalaya, Turkey. 2018.
[6]
Sharma A, Dogra A, Goyal B, Vig R, Agrawal S. From Pyramids to state-of-the-art: A study and comprehensive comparison of visible-infrared image fusion algorithms. IET Image Process 2019; 14(9): 1671-89.
[http://dx.doi.org/10.1049/iet-ipr.2019.0322]
[7]
Hounsfield GN. Computerized transverse axial scanning (tomography): Part I. description of system. Br J Radiol 1973; 46(552): 1016-22.
[http://dx.doi.org/10.1259/0007-1285-46-552-1016] [PMID: 8542219]
[8]
Hsieh J. Computed tomography: Principles, design, artifacts, and recent advances.WA 2009.
[9]
Dogra A, Bhalla P CT. Image sharpening by gaussian and butterworth high pass filter. Biomed Pharmacol J 2014; 7(2): 20.
[10]
Starkman S, Kidwell CS, Demchuk AM, Leary MC. Validation of computed tomographic middle cerebral artery "dot"sign: An angiographic correlation study. Stroke 2003; 34(11): 2636-40.
[11]
Cantatore A, Müller P. Introduction to computed tomography. Denmark: DTU Mech EngKongens Lyngby 2011.
[12]
Haddadpour M, Daneshvar S, Seyedarabi H. PET and MRI image fusion based on combination of 2-D Hilbert transform and IHS method. Biomed J 2017; 40(4): 219-25.
[http://dx.doi.org/10.1016/j.bj.2017.05.002] [PMID: 28918910]
[13]
Du J, Li W, Lu K, Xiao B. An overview of multi-modal medical image fusion. Neurocomputing 2016; 215: 3-20.
[http://dx.doi.org/10.1016/j.neucom.2015.07.160]
[14]
Diwakar M, Kumar M. Biomedical signal processing and control a review on ct image noise and its denoising. Biomed Signal Process Cont 2018; 42: 73-88.
[http://dx.doi.org/10.1016/j.bspc.2018.01.010]
[15]
Dura E, Domingo J, Ayala G, Marti-Bonmati L, Goceri E. Probabilistic liver atlas construction. Biomed Eng Online 2017; 16(1): 15.
[http://dx.doi.org/10.1186/s12938-016-0305-8] [PMID: 28086965]
[16]
Dura E, Domingo J, Göçeri E, Martí-Bonmatí L. A method for liver segmentation in perfusion MR images using probabilistic atlases and viscous reconstruction. Pattern Anal Appl 2018; 21(4): 1083-95.
[http://dx.doi.org/10.1007/s10044-017-0666-z]
[17]
Xu Z, Tao B, Liu C, et al. Three-dimensional quantitative assessment of myocardial infarction via multimodality fusion imaging: methodology, validation, and preliminary clinical application. Quant Imaging Med Surg 2021; 11(7): 3175-89.
[http://dx.doi.org/10.21037/qims-20-702]
[18]
Lindeberg T. Scale-space theory in computer vision. Springer Science & Business Media 2013; Vol. 256.
[19]
Wang W, Chang F. A multi-focus image fusion method based on laplacian pyramid. JCP 2011; 6(12): 2559-66.
[http://dx.doi.org/10.4304/jcp.6.12.2559-2566]
[20]
Liu Y, Liu S, Wang Z. Multi-focus image fusion with dense SIFT. Inf Fusion 2015; 23(1): 139-55.
[http://dx.doi.org/10.1016/j.inffus.2014.05.004]
[21]
Lowe DG. Object recognition from local scale-invariant features. Proceedings of the seventh IEEE international conference on computer vision; 1999 Sep 20-27; Kerkyra, Greece. IEEE: 2004.
[http://dx.doi.org/10.1109/ICCV.1999.790410]
[22]
Bai X, Zhang Y, Zhou F, Xue B. Quadtree-based multi-focus image fusion using a weighted focus-measure. Inf Fusion 2015; 22: 105-18.
[http://dx.doi.org/10.1016/j.inffus.2014.05.003]
[23]
Finkel R A, Bentley J L. Quad trees a data structure for retrieval on composite keys. Acta Informatica 1974; 4: 1-9.
[http://dx.doi.org/10.1007/BF00288933]
[24]
Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 1990; 12(7): 629-39.
[http://dx.doi.org/10.1109/34.56205]
[25]
Stuttgart BGT. Anisotropic diffusion in image processing.ECMI. 1998; pp. 1-184.
[26]
Bavirisetti DP, Dhuli R. Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform. IEEE Sens J 2016; 16(1): 203-9.
[http://dx.doi.org/10.1109/JSEN.2015.2478655]
[27]
Paul S, Sevcenco IS, Agathoklis P. Multi-exposure and multi-focus image fusion in gradient domain. J Circuits Syst Comput 2016; 25(10): 1650123.
[http://dx.doi.org/10.1142/S0218126616501231]
[28]
Ma J, Chen C, Li C, Huang J. Infrared and visible image fusion via gradient transfer and total variation minimization. Inf Fusion 2016; 31: 100-9.
[http://dx.doi.org/10.1016/j.inffus.2016.02.001]
[29]
Ahmed N, Natarajan T, Rao KR. Discrete cosine transform. IEEE Trans Comput 1974; 100(1): 90-3.
[http://dx.doi.org/10.1109/T-C.1974.223784]
[30]
Rao KR, Yip P. Discrete cosine transform: Algorithms, advantages, applications. Academic press 2014.
[31]
Strang G. The discrete cosine transform. SIAM Rev 1999; 41(1): 135-47.
[http://dx.doi.org/10.1137/S0036144598336745]
[32]
Naidu VPS. Discrete cosine transform-based image fusion. Def Sci J 2010; 60(1): 48-54.
[http://dx.doi.org/10.14429/dsj.60.105]
[33]
Liu Y, Wang Z. Multi-focus image fusion based on wavelet transform and adaptive block. Journal of Image and Graphics 2013; 18(11): 1435-44.
[34]
Misiti M, Misiti Y, Oppenheim G, Poggi JM. Wavelet Toolbox User's Guide 1996.
[35]
Shensa MJ. The discrete wavelet transform: Wedding the trous and Mallat algorithms. IEEE Trans Signal Process 1992; 40(10): 2464-82.
[http://dx.doi.org/10.1109/78.157290]
[36]
Heil CE, Walnut DF. Continuous and discrete wavelet transforms. SIAM Rev 1989; 31(4): 628-66.
[http://dx.doi.org/10.1137/1031129]
[37]
Mallat SG. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans Pattern Anal Mach Intell 1989; 11(7): 674-93.
[http://dx.doi.org/10.1109/34.192463]
[38]
Nason GP, Silverman BW. The stationary wavelet transform and some statistical applications.Wavelets and statistics. New York, NY: Springer 1995; pp. 281-99.
[http://dx.doi.org/10.1007/978-1-4612-2544-7_17]
[39]
Vijayarajan R, Muttan S. Discrete Wavelet Transform based Principal Component Averaging fusion for medical images. AEU 2015; 69(6): 896-902.
[http://dx.doi.org/10.1016/j.aeue.2015.02.007]
[40]
Rodriguez-Sánchez R, García JA, Fdez-Valdivia J. From computational attention to image fusion. Pattern Recognit Lett 2011; 32(14): 1778-95.
[http://dx.doi.org/10.1016/j.patrec.2011.07.003]
[41]
Kumar BS. Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform. Signal Image Video Process 2013; 7(6): 1125-43.
[http://dx.doi.org/10.1007/s11760-012-0361-x]
[42]
Narasimhan SV, Harish M, Haripriya AR, Basumallick N. Discrete cosine harmonic wavelet transform and its application to signal compression and subband spectral estimation using modified group delay. Signal Image Video Process 2009; 3(1): 85-99.
[http://dx.doi.org/10.1007/s11760-008-0062-7]
[43]
Donoho DL, Duncan MR. Digital curvelet transform: Strategy, implementation, and experiments. Wavelet applications VII. Int Soc Optics Photon 2000; 4056: 12-30.
[http://dx.doi.org/10.1117/12.381679]
[44]
da Cunha AL, Zhou J, Do MN. The nonsubsampled contourlet transform: Theory, design, and applications. IEEE Trans Image Process 2006; 15(10): 3089-101.
[http://dx.doi.org/10.1109/TIP.2006.877507] [PMID: 17022272]
[45]
Easley G, Labate D, Lim WQ. Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 2008; 25(1): 25-46.
[http://dx.doi.org/10.1016/j.acha.2007.09.003]
[46]
Sharma AM, Dogra A, Goyal B, Vig R, Agrawal S. Low-light visible and infrared image fusion in NSST domain. Proceedings of International Conference on IoT Inclusive Life (ICIIL 2019), NITTTR Chandigarh, India; pp. 61-8.
[http://dx.doi.org/10.1007/978-981-15-3020-3_7]
[47]
Indira KP, Hemamalini RR, Nandhitha NM. Performance evaluation of DWT, SWT and NSCT for fusion of PET and CT Images using different fusion rules. Biomed Res 2016; 27(1): 123-31.
[48]
Jain AK. Fundamentals of digital image processing. Englewood Cliffs, NJ: Prentice Hall 1989.
[49]
Gong Y, Liu B, Hou X, Qiu G. Sub-window box filter. 2018 IEEE Visual Communications and Image Processing (VCIP); 2018 Dec 9-12; Taichung, Taiwan. IEEE: 2019.
[http://dx.doi.org/10.1109/VCIP.2018.8698682]
[50]
Deng G, Cahill LW. An adaptive Gaussian filter for noise reduction and edge detection. In 1993 IEEE conference record nuclear science symposium and medical imaging conference; 1993 31 Oct.-6 Nov. IEEE : 1993.
[http://dx.doi.org/10.1109/NSSMIC.1993.373563]
[51]
Neycenssac F. Contrast enhancement using the Laplacian-of-a- Gaussian filter. CVGIP Graph Models Image Process 1993; 55(6): 447-63.
[http://dx.doi.org/10.1006/cgip.1993.1034]
[52]
Zhou Z, Wang B, Li S, Dong M. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters. Inf Fusion 2016; 30: 15-26.
[http://dx.doi.org/10.1016/j.inffus.2015.11.003]
[53]
Kumar BKS. Image denoising based on gaussian/bilateral filter and its method noise thresholding. Signal Image Video Process 2013; 6(7): 1159-72.
[http://dx.doi.org/10.1007/s11760-012-0372-7]
[54]
Tomasi C, Manduchi R. Bilateral filtering for gray and color images. Proceedings of the 1998 IEEE International Conference on Computer Vision, Bombay, India.
[http://dx.doi.org/10.1109/ICCV.1998.710815]
[55]
Goceri E, Goksel B, Elder JB, Puduvalli VK, Otero JJ, Gurcan MN. Quantitative validation of anti-PTBP1 antibody for diagnostic neuropathology use: Image analysis approach. Int J Numer Methods Biomed Eng 2017; 33(11): e2862.
[http://dx.doi.org/10.1002/cnm.2862] [PMID: 28024117]
[56]
Durand F, Dorsey J. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans Graph 2002; 21(3): 257-66.
[http://dx.doi.org/10.1145/566570.566574]
[57]
Petschnigg G, Cohen M, Hoppe H. Digital photography with flash and no-flash image Pairs 2004; 1(212): 664-72.
[58]
Shreyamsha Kumar BK. Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 2015; 9(5): 1193-204.
[http://dx.doi.org/10.1007/s11760-013-0556-9]
[59]
He K, Sun J, Tang X. Guided image filtering. IEEE Trans Pattern Anal Mach Intell 2013; 35(6): 1397-409.
[http://dx.doi.org/10.1109/TPAMI.2012.213] [PMID: 23599054]
[60]
Li S, Kang X, Hu J. Image fusion with guided filtering. IEEE Trans Image Process 2013; 22(7): 2864-75.
[http://dx.doi.org/10.1109/TIP.2013.2244222] [PMID: 23372084]
[61]
Bavirisetti DP, Kollu V, Gang X, Dhuli R. Fusion of MRI and CT images using guided image filter and image statistics. Int J Imaging Syst Technol 2017; 27(3): 227-37.
[http://dx.doi.org/10.1002/ima.22228]
[62]
Ule L. Weighted least-squares smoothing filters. IRE Transactions on Circuit Theory 1955; 2(2): 197-203.
[http://dx.doi.org/10.1109/TCT.1955.6373426]
[63]
Eckhorn R, Bauer R, Jordan W, et al. Coherent oscillations: A mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. Biol Cybern 1988; 60(2): 121-30.
[http://dx.doi.org/10.1007/BF00202899] [PMID: 3228555]
[64]
Farbman Z, Fattal R, Lischinski D, Szeliski R. Edge-preserving decompositions for multi-scale tone and detail manipulation. 2008; 1-10.
[http://dx.doi.org/10.1145/1399504.1360666]
[65]
Min D, Choi S, Lu J, Ham B, Sohn K, Do MN. Fast global image smoothing based on weighted least squares. IEEE Trans Image Process 2014; 23(12): 5638-53.
[http://dx.doi.org/10.1109/TIP.2014.2366600] [PMID: 25373085]
[66]
Singh H, Kumar V, Bhooshan S. Weighted least squares based detail enhanced exposure fusion. ISRN Sig Proc 2014; 18
[http://dx.doi.org/10.1155/2014/498762]
[67]
Ma J, Zhou Z, Wang B, Zong H. Infrared and visible image fusion based on visual saliency map and weighted least square optimization. Infrared Phys Technol 2017; 82: 8-17.
[http://dx.doi.org/10.1016/j.infrared.2017.02.005]
[68]
Li W, Xie Y, Zhou H, Han Y, Zhan K. Structure-aware image fusion. Optik (Stuttg) 2018; 172: 1-11.
[http://dx.doi.org/10.1016/j.ijleo.2018.06.123]
[69]
Bavirisetti DP, Dhuli R. Two-scale image fusion of visible and infrared images using saliency detection. Infrared Phys Technol 2016; 76: 52-64.
[http://dx.doi.org/10.1016/j.infrared.2016.01.009]
[70]
Golub GH, Reinsch C. Singular value decomposition and least squares solutions.Linear Algebra. Berlin, Heidelberg: Springer 1971; pp. 134-51.
[http://dx.doi.org/10.1007/978-3-662-39778-7_10]
[71]
Murata Y, Sakatani K, Hoshino T, et al. Effects of cerebral ischemia on evoked cerebral blood oxygenation responses and BOLD contrast functional MRI in stroke patients. Stroke 2006; (10): 2514-20.
[72]
Andrews H, Patterson C. Singular value decompositions and digital image processing. IEEE Trans Acoust Speech Signal Process 1976; 24(1): 26-53.
[http://dx.doi.org/10.1109/TASSP.1976.1162766]
[73]
Naidu VPS. Image fusion technique using multi-resolution singular value decomposition. Def Sci J 2011; 61(5): 479-84.
[http://dx.doi.org/10.14429/dsj.61.705]
[74]
Al-Azzawi NA, Abdullah WAKW. Medical image fusion schemes using Contourlet transform and pca bases. Image Fus App 2011; 93-110.
[75]
Lu H, Wu QX, Maguire LP. Information processing functionality of spiking neurons for image feature extraction. In: Seventh International Workshop on Information Processing in Cells and Tissue. 2007; pp. 1-12.
[76]
Zhan K, Xie Y, Wang H, Min Y. Fast filtering image fusion. J Electron Imaging 2017; 26(06): 1.
[http://dx.doi.org/10.1117/1.JEI.26.6.063004]
[77]
Zhan K, Shi J, Wang H, Xie Y, Li Q. Computational mechanisms of pulse-coupled neural networks: a comprehensive review. Arch Comput Methods Eng 2017; 24(3): 573-88.
[http://dx.doi.org/10.1007/s11831-016-9182-3]
[78]
Wang Z, Ma Y. Medical image fusion using m-PCNN. Inf Fusion 2008; 9(2): 176-85.
[http://dx.doi.org/10.1016/j.inffus.2007.04.003]
[79]
James AP, Dasarathy BV. Medical image fusion: A survey of the state of the art. Inf Fusion 2014; 19: 4-19.
[http://dx.doi.org/10.1016/j.inffus.2013.12.002]
[80]
Ma J, Ma Y, Li C. Infrared and visible image fusion methods and applications: A survey. Inf Fusion 2018; 2019(45): 153-78.
[81]
Li S, Kang X, Fang L, Hu J, Yin H. Pixel-level image fusion: A survey of the state of the art. Inf Fusion 2017; 33: 100-12.
[http://dx.doi.org/10.1016/j.inffus.2016.05.004]
[82]
Deshmukh M, Bhosale U. Image fusion and image quality assessment of fused images. International Journal of Image Processing 2010; 4(5): 484. [IJIP].
[83]
Jagalingam P, Hegde AV. A review of quality metrics for fused image. Aquat Procedia 2015; 4: 133-42.
[http://dx.doi.org/10.1016/j.aqpro.2015.02.019]
[84]
Xydeas CS, Petrović V. Objective image fusion performance measure. Electron Lett 2000; 36(4): 308.
[http://dx.doi.org/10.1049/el:20000267]
[85]
Blasch E, Li X, Chen G, Li W. Image quality assessment for performance evaluation of image fusion. 2008 11th international conference on information fusion; 2008 30 june-3 july; Cologne, Germany. IEEE: 2008.
[86]
Wang Q, Yu D, Shen Y. An overview of image fusion metrics In 2009 IEEE instrumentation and measurement technology conference. IEEE 2009; pp. 918-23.
[http://dx.doi.org/10.1109/IMTC.2009.5168582]
[87]
Liu Z, Blasch E, Xue Z, Zhao J, Laganiere R, Wu W. Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study. IEEE Trans Pattern Anal Mach Intell 2012; 34(1): 94-109.
[http://dx.doi.org/10.1109/TPAMI.2011.109] [PMID: 21576753]
[88]
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Process 2004; 13(4): 600-12.
[http://dx.doi.org/10.1109/TIP.2003.819861] [PMID: 15376593]
[89]
Bavirisetti D, Xiao G. Multi-sensor image fusion based on fourth order partial differential equations. 2017 20th International Conference on Information Fusion (Fusion); 2017 July 10-13; Xi'an, China. IEEE 2017.
[http://dx.doi.org/10.23919/ICIF.2017.8009719]

Rights & Permissions Print Cite
© 2024 Bentham Science Publishers | Privacy Policy