Abstract
Background: Skin disease image analysis has drawn extensive attention from researchers, which can help doctors efficiently diagnose skin disease from medical images. Existing reviews have focused only on the specific task of skin disease diagnosis based on a single medical image type.
Discussion: This paper presents the latest and comprehensive review of image analysis methods in skin diseases, and summarizes over 350 contributions to the field, most of which appeared in the last three years. We first sort out representative publicly available skin datasets and summarize their characteristics. Thereafter, aiming at the typical problems exposed by datasets, we organize the image preprocessing and data enhancement part. Further, we review the single tasks of skin disease image analysis in the literature, such as classification, detection or segmentation, and analyze the improvement direction of their corresponding methods. Additionally, popular multi-task models based on structure and loss function are also investigated.
Conclusions: Challenges involved from the aspects of the dataset and model structure have been discussed.
Keywords: computer-aided diagnosis, skin disease, deep learning, classification, segmentation, multi-task
Graphical Abstract
[http://dx.doi.org/10.1038/jid.2013.446] [PMID: 24166134]
[http://dx.doi.org/10.1016/j.jid.2018.01.028] [PMID: 29428356]
[http://dx.doi.org/10.1001/jama.2016.5415] [PMID: 27458949]
[http://dx.doi.org/10.1023/B:VISI.0000029664.99615.94]
[http://dx.doi.org/10.1109/TIP.2009.2019809] [PMID: 19482579]
[http://dx.doi.org/10.1046/j.1365-2133.2003.05023.x] [PMID: 12786829]
[http://dx.doi.org/10.1001/archderm.134.12.1563] [PMID: 9875194]
[http://dx.doi.org/10.1001/archderm.141.11.1388] [PMID: 16301386]
[http://dx.doi.org/10.1001/jama.292.22.2771] [PMID: 15585738]
[http://dx.doi.org/10.1016/j.jaad.2006.09.003] [PMID: 17190620]
[http://dx.doi.org/10.1007/BF00994018]
[http://dx.doi.org/10.1007/s10100-017-0479-6] [PMID: 29375266]
[http://dx.doi.org/10.1038/nature21056] [PMID: 28117445]
[http://dx.doi.org/10.1111/jdv.15965] [PMID: 31541556]
[http://dx.doi.org/10.1034/j.1600-0846.2000.006001001.x] [PMID: 11428935]
[http://dx.doi.org/10.1093/annonc/mdy166] [PMID: 29846502]
[http://dx.doi.org/10.1016/S0738-081X(02)00231-6] [PMID: 12074856]
[http://dx.doi.org/10.1111/j.1600-0846.1997.tb00152.x]
[http://dx.doi.org/10.1001/archderm.135.12.1538] [PMID: 10606065]
[http://dx.doi.org/10.1001/archderm.143.3.329] [PMID: 17372097]
[http://dx.doi.org/10.1001/archderm.1995.01690150050011] [PMID: 7887657]
[http://dx.doi.org/10.1016/j.compbiomed.2020.104065] [PMID: 33246265]
[http://dx.doi.org/10.1109/RBME.2009.2034865] [PMID: 20671804]
[http://dx.doi.org/10.1109/TCOM.1971.1090769]
[PMID: 25073179]
[http://dx.doi.org/10.1109/TBME.2007.910651] [PMID: 18232357]
[http://dx.doi.org/10.1016/j.compmedimag.2008.11.002] [PMID: 19121917]
[http://dx.doi.org/10.1111/j.1600-0846.2011.00569.x] [PMID: 22092500]
[http://dx.doi.org/10.1016/j.compmedimag.2010.08.003] [PMID: 20933366]
[http://dx.doi.org/10.1016/j.compmedimag.2010.08.004] [PMID: 21035303]
[http://dx.doi.org/10.5220/0005313005770582]
[http://dx.doi.org/10.1016/j.amc.2008.05.130]
[http://dx.doi.org/10.1109/ACCESS.2020.3047258]
[http://dx.doi.org/10.1109/TPAMI.2016.2644615] [PMID: 28060704]
[http://dx.doi.org/10.1109/TPAMI.2017.2699184] [PMID: 28463186]
[http://dx.doi.org/10.1109/TMI.2016.2524985] [PMID: 26863652]
[http://dx.doi.org/10.1109/TMI.2016.2521800] [PMID: 26829785]
[http://dx.doi.org/10.1111/j.1600-0846.2009.00357.x] [PMID: 19624428]
[http://dx.doi.org/10.1111/j.1600-0846.2010.00445.x] [PMID: 20637008]
[http://dx.doi.org/10.1016/j.compmedimag.2010.09.006] [PMID: 20970307]
[http://dx.doi.org/10.1111/j.1600-0846.2007.00251.x] [PMID: 17908199]
[http://dx.doi.org/10.1111/j.1600-0846.2008.00301.x] [PMID: 19159382]
[http://dx.doi.org/10.3390/diagnostics9030072] [PMID: 31295856]
[http://dx.doi.org/10.1109/TBME.2018.2866166] [PMID: 30130171]
[http://dx.doi.org/10.1016/j.compmedimag.2015.02.011] [PMID: 25797605]
[http://dx.doi.org/10.1186/s40537-019-0197-0]
[http://dx.doi.org/10.1613/jair.953]
[http://dx.doi.org/10.1016/j.cmpb.2018.05.027] [PMID: 29903489]
[http://dx.doi.org/10.1016/j.media.2019.101552] [PMID: 31521965]
[http://dx.doi.org/10.1007/s11042-019-7717-y]
[http://dx.doi.org/10.1038/sdata.2018.161] [PMID: 30482902]
[http://dx.doi.org/10.1109/JBHI.2019.2942429] [PMID: 31545748]
[http://dx.doi.org/10.1016/j.eswa.2020.113922]
[http://dx.doi.org/10.1007/s10586-018-1798-7]
[http://dx.doi.org/10.1016/j.eswa.2020.114204]
[http://dx.doi.org/10.1007/s12652-020-02675-8]
[http://dx.doi.org/10.1177/0954411920939829] [PMID: 32643539]
[http://dx.doi.org/10.1109/ICAIIC.2019.8668980]
[http://dx.doi.org/10.1109/TMI.2016.2528162] [PMID: 26886976]
[http://dx.doi.org/10.1016/j.cmpb.2020.105568] [PMID: 32526536]
[http://dx.doi.org/10.1109/CVPRW.2019.00333]
[http://dx.doi.org/10.1016/j.ejca.2019.04.021] [PMID: 31129383]
[http://dx.doi.org/10.1109/ACCESS.2020.2964424]
[http://dx.doi.org/10.1016/j.ejca.2019.02.005] [PMID: 30852421]
[http://dx.doi.org/10.1111/bjd.17189] [PMID: 30207594]
[http://dx.doi.org/10.1016/j.patcog.2020.107413]
[http://dx.doi.org/10.1109/CVPRW.2019.00334]
[http://dx.doi.org/10.1109/TMI.2019.2893944] [PMID: 30668469]
[http://dx.doi.org/10.1109/ICCV.2017.115]
[http://dx.doi.org/10.32604/cmc.2020.05883]
[http://dx.doi.org/10.1016/j.compmedimag.2018.10.007] [PMID: 30458354]
[http://dx.doi.org/10.1016/j.bspc.2020.102041]
[http://dx.doi.org/10.1016/j.eswa.2019.112961]
[http://dx.doi.org/10.1016/j.cmpb.2020.105475] [PMID: 32268255]
[http://dx.doi.org/10.1109/JBHI.2020.2977013] [PMID: 32142460]
[http://dx.doi.org/10.1016/j.eswa.2019.113127]
[http://dx.doi.org/10.1016/j.ebiom.2019.01.028] [PMID: 30674442]
[http://dx.doi.org/10.1109/CVPRW50498.2020.00381]
[http://dx.doi.org/10.3390/biom10081123] [PMID: 32751349]
[http://dx.doi.org/10.1177/1357633x13490890] [PMID: 24163062]
[http://dx.doi.org/10.5070/D3216027826] [PMID: 26158371]
[http://dx.doi.org/10.1109/JBHI.2018.2824327] [PMID: 29994053]
[http://dx.doi.org/10.1111/exd.13777] [PMID: 30187575]
[http://dx.doi.org/10.1016/j.compbiomed.2019.103545] [PMID: 31760271]
[http://dx.doi.org/10.1016/j.patcog.2020.107502]
[http://dx.doi.org/10.2174/1573405616666200129095242]
[http://dx.doi.org/10.1007/978-981-33-4673-4_14]
[http://dx.doi.org/10.1201/b19107-8]
[http://dx.doi.org/10.1016/j.bspc.2017.07.010]
[http://dx.doi.org/10.1111/srt.12744] [PMID: 31228313]
[http://dx.doi.org/10.1186/s12859-019-2625-8] [PMID: 30871471]
[http://dx.doi.org/10.1371/journal.pone.0191493] [PMID: 29352285]
[http://dx.doi.org/10.7717/peerj-cs.268] [PMID: 33816919]
[http://dx.doi.org/10.1007/s11042-017-4788-5]
[http://dx.doi.org/10.1016/j.bspc.2021.102787]
[http://dx.doi.org/10.1016/j.cmpb.2019.105038] [PMID: 31437709]
[http://dx.doi.org/10.1007/s11263-011-0437-z]
[http://dx.doi.org/10.1016/j.media.2020.101915] [PMID: 33260112]
[http://dx.doi.org/10.3390/sym12081224]
[http://dx.doi.org/10.3390/app10093045]
[http://dx.doi.org/10.1007/s00521-012-1149-1]
[http://dx.doi.org/10.1016/j.ijleo.2019.04.022]
[http://dx.doi.org/10.1109/TBME.2013.2283803] [PMID: 24081839]
[http://dx.doi.org/10.1109/JBHI.2015.2390032] [PMID: 25585429]
[http://dx.doi.org/10.1016/j.bspc.2019.101765]
[http://dx.doi.org/10.5565/rev/elcvia.1177]
[http://dx.doi.org/10.1080/21681163.2015.1029080]
[http://dx.doi.org/10.1002/jemt.23009] [PMID: 29464868]
[http://dx.doi.org/10.1016/j.smhl.2019.100088]
[http://dx.doi.org/10.1002/(SICI)1520-6343(1999)5:5<265:AID-BSPY1>3.0.CO;2-F]
[http://dx.doi.org/10.1046/j.1523-1747.2001.00212.x] [PMID: 11168814]
[http://dx.doi.org/10.1080/24699322.2017.1389405]
[http://dx.doi.org/10.1007/s11042-018-6523-2]
[http://dx.doi.org/10.1016/j.eswa.2020.113742]
[http://dx.doi.org/10.1007/s10278-020-00343-z] [PMID: 32378058]
[http://dx.doi.org/10.1016/j.compmedimag.2019.101658] [PMID: 31634739]
[http://dx.doi.org/10.1109/ACCESS.2019.2940794]
[http://dx.doi.org/10.1016/j.media.2020.101716] [PMID: 32492581]
[http://dx.doi.org/10.1016/j.compbiomed.2018.11.010] [PMID: 30471461]
[http://dx.doi.org/10.1016/j.eswa.2018.10.029]
[http://dx.doi.org/10.1109/CVPRW.2019.00332]
[http://dx.doi.org/10.1016/j.bspc.2021.102533]
[http://dx.doi.org/10.1101/2021.02.02.21251038]
[http://dx.doi.org/10.1007/s00607-021-00907-z]
[http://dx.doi.org/10.1177/01617346211014138] [PMID: 33971769]
[http://dx.doi.org/10.3390/s20061601] [PMID: 32183041]
[http://dx.doi.org/10.1016/j.patcog.2021.107994]
[http://dx.doi.org/10.1016/j.bspc.2021.102839]
[http://dx.doi.org/10.1016/j.bspc.2021.102710]
[http://dx.doi.org/10.1515/med-2020-0131] [PMID: 33336044]
[http://dx.doi.org/10.1515/med-2018-0002] [PMID: 29577090]
[http://dx.doi.org/10.1016/j.mcm.2012.09.013]
[http://dx.doi.org/10.3390/app11073025]
[http://dx.doi.org/10.3390/diagnostics11050811] [PMID: 33947117]
[http://dx.doi.org/10.1016/j.compbiomed.2020.103762] [PMID: 32768035]
[http://dx.doi.org/10.1007/s12559-020-09805-6]
[http://dx.doi.org/10.1016/j.cmpb.2019.105241] [PMID: 31837637]
[http://dx.doi.org/10.1109/ACCESS.2020.3007512]
[http://dx.doi.org/10.1007/s00521-018-3439-8]
[http://dx.doi.org/10.3390/diagnostics11030501] [PMID: 33809048]
[http://dx.doi.org/10.1016/j.bspc.2020.102358]
[http://dx.doi.org/10.1007/s12652-021-02933-3]
[http://dx.doi.org/10.1147/JRD.2017.2708299]
[http://dx.doi.org/10.1007/s12652-021-02936-0]
[http://dx.doi.org/10.1109/JBHI.2017.2787487] [PMID: 29990146]
[http://dx.doi.org/10.1016/j.eswa.2019.113129]
[http://dx.doi.org/10.1111/srt.12920] [PMID: 32662570]
[http://dx.doi.org/10.1162/neco.1997.9.8.1735] [PMID: 9377276]
[http://dx.doi.org/10.1109/TBME.2017.2712771] [PMID: 28600236]
[http://dx.doi.org/10.1109/JBHI.2018.2859898] [PMID: 30047917]
[http://dx.doi.org/10.1016/j.compmedimag.2020.101765] [PMID: 32810817]
[http://dx.doi.org/10.1007/s11760-018-01410-3]
[http://dx.doi.org/10.1109/ACCESS.2019.2940418]
[http://dx.doi.org/10.1016/j.compbiomed.2020.104118] [PMID: 33221639]
[http://dx.doi.org/10.1016/j.compbiomed.2020.103738] [PMID: 32421644]
[http://dx.doi.org/10.1016/j.artmed.2019.101756] [PMID: 31980095]
[http://dx.doi.org/10.1016/j.cmpb.2018.01.027] [PMID: 29650319]
[http://dx.doi.org/10.1109/CVPRW.2017.108]
[http://dx.doi.org/10.1016/j.cmpb.2014.12.001] [PMID: 25540998]
[http://dx.doi.org/10.1016/j.ultras.2021.106412] [PMID: 33784575]
[http://dx.doi.org/10.1016/j.eswa.2021.115433]
[http://dx.doi.org/10.1007/s10916-016-0460-2] [PMID: 26872778]
[http://dx.doi.org/10.1016/j.patrec.2019.11.042]
[http://dx.doi.org/10.1016/j.patrec.2020.12.015]
[http://dx.doi.org/10.1002/ima.22377]
[http://dx.doi.org/10.1001/jamadermatol.2019.3807] [PMID: 31799995]
[http://dx.doi.org/10.1016/j.cmpb.2020.105725] [PMID: 32882594]
[http://dx.doi.org/10.2196/21695] [PMID: 33764307]
[http://dx.doi.org/10.1016/j.cmpb.2020.105351] [PMID: 32028084]
[http://dx.doi.org/10.1109/TMI.2020.2972964] [PMID: 32070946]
[http://dx.doi.org/10.3390/s18020556] [PMID: 29439500]
[http://dx.doi.org/10.1109/JBHI.2020.2973614] [PMID: 32071016]
[http://dx.doi.org/10.1016/j.ejca.2020.11.020] [PMID: 33423009]
[http://dx.doi.org/10.1016/j.patcog.2021.108075]
[http://dx.doi.org/10.3390/jimaging7040067] [PMID: 34460517]
[http://dx.doi.org/10.1016/j.patcog.2019.02.024]
[http://dx.doi.org/10.1109/CVPRW50498.2020.00375]
[http://dx.doi.org/10.3389/fmolb.2021.614277] [PMID: 34490342]
[http://dx.doi.org/10.1038/s41467-021-22188-y] [PMID: 33767170]
[http://dx.doi.org/10.1145/3375923.3375948]
[http://dx.doi.org/10.1109/TIP.2019.2913986] [PMID: 31059446]
[http://dx.doi.org/10.1016/j.neunet.2020.10.018] [PMID: 33217685]
[http://dx.doi.org/10.1109/ICCV.2019.00220]
[http://dx.doi.org/10.3390/asi3020020]
[http://dx.doi.org/10.1007/s11042-020-09067-2]
[http://dx.doi.org/10.3389/fmed.2020.00177] [PMID: 32435646]
[http://dx.doi.org/10.1088/1361-6560/ab86d3] [PMID: 32252036]
[http://dx.doi.org/10.1016/j.knosys.2019.01.019]
[http://dx.doi.org/10.1109/ICIP42928.2021.9506741]
[http://dx.doi.org/10.1609/aaai.v33i01.3301371]
[http://dx.doi.org/10.1109/CVPR42600.2020.01339]
[http://dx.doi.org/10.1145/3366423.3380163]
[http://dx.doi.org/10.18653/v1/2020.acl-main.726]
[http://dx.doi.org/10.1145/3394171.3413538]
[http://dx.doi.org/10.26599/TST.2019.9010029]
[http://dx.doi.org/10.1109/TNNLS.2017.2716952] [PMID: 28742048]
[http://dx.doi.org/10.1007/s00521-021-06141-0]
[http://dx.doi.org/10.1109/TRPMS.2021.3066428]
[http://dx.doi.org/10.1109/CVPRW53098.2021.00199]
[http://dx.doi.org/10.1109/JBHI.2021.3052044] [PMID: 33449890]
[http://dx.doi.org/10.1109/TPAMI.2021.3054719] [PMID: 33497328]