Abstract
Background: Metal surface roughness detection is an essential step of quality control in the metal processing industry. Due to the high manual involvement and poor efficiency of traditional roughness testing, rapid automated vision detection has received increasing attention in product quality control. Many methods have focused on extracting features related to roughness from images by means of mathematical statistics. However, these methods often rely on extensive experiments and complex calculations, while being sensitive to external environmental disturbances.
Methods: In this paper, a convolution neural network-based approach for metal surface roughness evaluation has been proposed. The convolutional neural network was initialized using a transfer learning strategy, and the data augmentation technique was applied to the benchmark dataset for sample expansion.
Results: To evaluate this approach, samples of 4 types of roughness classes were prepared. The samples were divided into a training set, validation set, and test set in the ratio of 7:2:1. The accuracy of the neural network on the test set was found to be above 86%.
Conclusion: The effectiveness of the proposed approach and its superiority over manual detection have been demonstrated in the experiments.
Graphical Abstract
[1]
Curry N, Tang Z, Markocsan N, Nylén P. Influence of bond coat surface roughness on the structure of axial suspension plasma spray thermal barrier coatings-Thermal and lifetime performance. Surf Coat Tech 2015; 268: 15-23.
[http://dx.doi.org/10.1016/j.surfcoat.2014.08.067]
[http://dx.doi.org/10.1016/j.surfcoat.2014.08.067]
[2]
Masuo H, Tanaka Y, Morokoshi S, et al. Influence of defects, surface roughness and HIP on the fatigue strength of Ti-6Al-4V manufactured by additive manufacturing. Int J Fatigue 2018; 117: 163-79.
[http://dx.doi.org/10.1016/j.ijfatigue.2018.07.020]
[http://dx.doi.org/10.1016/j.ijfatigue.2018.07.020]
[3]
Maleki E, Bagherifard S, Bandini M, Guagliano M. Surface post-treatments for metal additive manufacturing: Progress, challenges, and opportunities. Addit Manuf 2021; 37: 22.
[http://dx.doi.org/10.1016/j.addma.2020.101619]
[http://dx.doi.org/10.1016/j.addma.2020.101619]
[4]
Jeyapoovan T, Murugan M. Surface roughness classification using image processing. Measurement 2013; 46(7): 2065-72.
[http://dx.doi.org/10.1016/j.measurement.2013.03.014]
[http://dx.doi.org/10.1016/j.measurement.2013.03.014]
[5]
Macek W, Marciniak Z, Branco R, Rozumek D, Królczyk GM. A fractographic study exploring the fracture surface topography of S355J2 steel after pseudo-random bending-torsion fatigue tests. Measurement 2021; 178: 109443.
[http://dx.doi.org/10.1016/j.measurement.2021.109443]
[http://dx.doi.org/10.1016/j.measurement.2021.109443]
[6]
Leach R. Optical measurement of surface topography 2011.
[http://dx.doi.org/10.1007/978-3-642-12012-1]
[http://dx.doi.org/10.1007/978-3-642-12012-1]
[7]
Kobayashi T, Shockey DA. Fracture surface topography analysis (FRASTA)—Development, accomplishments, and future applications. Eng Fract Mech 2010; 77(12): 2370-84.
[http://dx.doi.org/10.1016/j.engfracmech.2010.05.016]
[http://dx.doi.org/10.1016/j.engfracmech.2010.05.016]
[8]
Nikolaev N, Petzing J, Coupland J. Focus variation microscope: Linear theory and surface tilt sensitivity. Appl Opt 2016; 55(13): 3555-65.
[http://dx.doi.org/10.1364/AO.55.003555] [PMID: 27140371]
[http://dx.doi.org/10.1364/AO.55.003555] [PMID: 27140371]
[9]
Wang S, Wan J, Zhang D, Li D, Zhang C. Towards smart factory for industry 4.0: A self-organized multi-agent system with big data based feedback and coordination. Comput Netw 2016; 101: 158-68.
[http://dx.doi.org/10.1016/j.comnet.2015.12.017]
[http://dx.doi.org/10.1016/j.comnet.2015.12.017]
[10]
Oztemel E, Gursev S. Literature review of Industry 4.0 and related technologies. J Intell Manuf 2020; 31(1): 127-82.
[http://dx.doi.org/10.1007/s10845-018-1433-8]
[http://dx.doi.org/10.1007/s10845-018-1433-8]
[11]
Wang X, Han Y, Leung VCM, Niyato D, Yan X, Chen X. Convergence of edge computing and deep learning: A comprehensive survey. IEEE Commun Surv Tutor 2020; 22(2): 869-904.
[http://dx.doi.org/10.1109/COMST.2020.2970550]
[http://dx.doi.org/10.1109/COMST.2020.2970550]
[12]
Köksal G. Batmaz İ Testik MC. A review of data mining applications for quality improvement in manufacturing industry. Expert Syst Appl 2011; 38(10): 13448-67.
[http://dx.doi.org/10.1016/j.eswa.2011.04.063]
[http://dx.doi.org/10.1016/j.eswa.2011.04.063]
[13]
Gupta A, Singh H, Aggarwal A. Taguchi-fuzzy multi output optimization (MOO) in high speed CNC turning of AISI P-20 tool steel. Expert Syst Appl 2011; 38(6): 6822-8.
[http://dx.doi.org/10.1016/j.eswa.2010.12.057]
[http://dx.doi.org/10.1016/j.eswa.2010.12.057]
[14]
Çaydaş U, Ekici S. Support vector machines models for surface roughness prediction in CNC turning of AISI 304 austenitic stainless steel. J Intell Manuf 2012; 23(3): 639-50.
[http://dx.doi.org/10.1007/s10845-010-0415-2]
[http://dx.doi.org/10.1007/s10845-010-0415-2]
[15]
Palani S, Natarajan U, Chellamalai M. On-line prediction of micro-turning multi-response variables by machine vision system using adaptive neuro-fuzzy inference system (ANFIS). Mach Vis Appl 2013; 24(1): 19-32.
[http://dx.doi.org/10.1007/s00138-011-0378-0]
[http://dx.doi.org/10.1007/s00138-011-0378-0]
[16]
Shahabi HH, Ratnam MM. Simulation and measurement of surface roughness via grey scale image of tool in finish turning. Precis Eng 2016; 43: 146-53.
[http://dx.doi.org/10.1016/j.precisioneng.2015.07.004]
[http://dx.doi.org/10.1016/j.precisioneng.2015.07.004]
[17]
Vorburger TV, Teague EC. Optical techniques for on-line measurement of surface topography. Precis Eng 1981; 3(2): 61-83.
[http://dx.doi.org/10.1016/0141-6359(81)90038-6]
[http://dx.doi.org/10.1016/0141-6359(81)90038-6]
[18]
Tian GY, Lu RS, Gledhill D. Surface measurement using active vision and light scattering. Opt Lasers Eng 2007; 45(1): 131-9.
[http://dx.doi.org/10.1016/j.optlaseng.2006.03.005]
[http://dx.doi.org/10.1016/j.optlaseng.2006.03.005]
[19]
Vesselenyi T, Moga I, Mudura P. Surface roughness image analysis using fractal methods. 18th International Symposium of the Danube-Adria-Association-for-Automationand-Manufacturing Zadar, CROATIA 2007. pp. Oct 24-27; 2007; 811-2.
[20]
Ali JM, Jailani HS, Murugan M. Surface roughness evaluation of milled steel surfaces using wavelet transform of laser speckle line images. Lasers Eng 2019; 44(4-6): 371-84.
[21]
Guo R, Bian D. Vision system for surface roughness and surface defect measurement. Opt Eng 2019; 58(6): 1.
[http://dx.doi.org/10.1117/1.OE.58.6.064103]
[http://dx.doi.org/10.1117/1.OE.58.6.064103]
[22]
Yi HA, Zhao XJ, Tang L, Chen YL, Yang J. Measuring grinding surface roughness based on singular value entropy of quaternion. Meas Sci Technol 2020; 31(11): 11.
[http://dx.doi.org/10.1088/1361-6501/ab9aa9]
[http://dx.doi.org/10.1088/1361-6501/ab9aa9]
[23]
Haralick RM, Shanmugam K, Dinstein I. Textural features for image classification. Stud Media Commun 1973; 610-21.
[http://dx.doi.org/ 10.1109/TSMC.1973.4309314]
[http://dx.doi.org/ 10.1109/TSMC.1973.4309314]
[24]
Sebastian VB. Gray level co-occurrence matrices: Generalisation and some new Features. Int J Comp Sci Eng Informa 2012; 2(2): 151-7.
[http://dx.doi.org/10.5121/ijcseit.2012.2213]
[http://dx.doi.org/10.5121/ijcseit.2012.2213]
[25]
Roberti de Siqueira F, Robson Schwartz W, Pedrini H. Multi-scale gray level co-occurrence matrices for texture description. Neurocomputing 2013; 120: 336-45.
[http://dx.doi.org/10.1016/j.neucom.2012.09.042]
[http://dx.doi.org/10.1016/j.neucom.2012.09.042]
[26]
Liang C, Yang W M, Zhou F, Liao Q M. Roughness classification with aggregated discrete fourier transform. IEICE Trans Inf Syst 2014; e97d(10): 2769-79.
[http://dx.doi.org/10.1587/transinf.2014EDP7082]
[http://dx.doi.org/10.1587/transinf.2014EDP7082]
[27]
Chen S, Feng R, Zhang C, Zhang Y. Surface roughness measurement method based on multi-parameter modeling learning. Measurement 2018; 129: 664-76.
[http://dx.doi.org/10.1016/j.measurement.2018.07.071]
[http://dx.doi.org/10.1016/j.measurement.2018.07.071]
[28]
Li W, Lu W, Li S, Zhao Y, Li Q. Research on extraction method of roughness parameters of relay circular contacts. IEEJ Trans Electr Electron Eng 2020; 15(7): 995-1001.
[http://dx.doi.org/10.1002/tee.23143]
[http://dx.doi.org/10.1002/tee.23143]
[29]
Liu H, Li J, Yang Y, Lan J, Xue Y. Automatic process parameters tuning and surface roughness estimation for laser cleaning IEEE Access 2020; 8: 20904-19.
[http://dx.doi.org/10.1109/ACCESS.2020.2970086]
[http://dx.doi.org/10.1109/ACCESS.2020.2970086]
[30]
Pan H, Gao P, Zhou H, Ma R, Yang J, Zhang X. Roughness analysis of sea surface from visible images by texture IEEE Access 2020; 8: 46448-58.
[http://dx.doi.org/10.1109/ACCESS.2020.2978638]
[http://dx.doi.org/10.1109/ACCESS.2020.2978638]
[31]
Ge J, Liu H, Yang S, Lan J. Laser cleaning surface roughness estimation using enhanced glcm feature and IPSO-SVR. Photonics 2022; 9(8): 510.
[http://dx.doi.org/10.3390/photonics9080510]
[http://dx.doi.org/10.3390/photonics9080510]
[32]
Chen ST, Hu HF, Zhang C. Surface roughness modeling based on laser speckle imaging. Wuli Xuebao 2015; 64(23): 9.
[33]
Huaian YI, Jian LIU, Enhui LU, Peng AO. Measuring grinding surface roughness based on the sharpness evaluation of colour images. Meas Sci Technol 2016; 27(2): 025404.
[http://dx.doi.org/10.1088/0957-0233/27/2/025404]
[http://dx.doi.org/10.1088/0957-0233/27/2/025404]
[34]
Gu J, Wang Z, Kuen J, et al. Recent advances in convolutional neural networks. Pattern Recognit 2018; 77: 354-77.
[http://dx.doi.org/10.1016/j.patcog.2017.10.013]
[http://dx.doi.org/10.1016/j.patcog.2017.10.013]
[35]
Wang Z, Yang J, Jiang H, Fan X. CNN training with twenty samples for crack detection via data augmentation. Sensors 2020; 20(17): 4849.
[http://dx.doi.org/10.3390/s20174849] [PMID: 32867223]
[http://dx.doi.org/10.3390/s20174849] [PMID: 32867223]
[36]
Tao X, Zhang D, Ma W, Liu X, Xu D. Automatic metallic surface defect detection and recognition with convolutional neural networks. Appl Sci 2018; 8(9): 1575.
[http://dx.doi.org/10.3390/app8091575]
[http://dx.doi.org/10.3390/app8091575]
[37]
Alqahtani H, Bharadwaj S, Ray A. Classification of fatigue crack damage in polycrystalline alloy structures using convolutional neural networks. Eng Fail Anal 2020; 119(5): 104908.
[http://dx.doi.org/10.1016/j.engfailanal.2020.104908]
[http://dx.doi.org/10.1016/j.engfailanal.2020.104908]
[38]
Tabernik D, Šela S. Skvarč J, Skočaj D. Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 2020; 31(3): 759-76.
[http://dx.doi.org/10.1007/s10845-019-01476-x]
[http://dx.doi.org/10.1007/s10845-019-01476-x]
[39]
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. CoRR 2014; abs./1409.1556.
[http://dx.doi.org/10.48550/arXiv.1409.1556]
[http://dx.doi.org/10.48550/arXiv.1409.1556]
[40]
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. IEEE conference on Computer Vision and Pattern Recognition, 2016, Las Vegas, NV, USA 2016; pp. 770-8.
[http://dx.doi.org/10.48550/arXiv.1512.03385]
[http://dx.doi.org/10.48550/arXiv.1512.03385]
[41]
Huang G, Liu Z, Laurens V, Weinberger KQ. Densely connected convolutional networks. IEEE conference on Computer Vision and Pattern Recognition, Honolulu, HT,USA 2017; 2261-9.
[http://dx.doi.org/10.48550/arXiv.1608.06993]
[http://dx.doi.org/10.48550/arXiv.1608.06993]
[42]
Szegedy C, Liu W, Jia Y, Sermanet P, Rabinovich A. Going deeper with convolutions. 2015 IEEE conference on computer vision and pattern recognition (CVPR), Los Vegas,NV,USA 2014; pp. 1-9.
[43]
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna ZJI. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Vegas, NV, USA 2016; p. 2818-26.
[http://dx.doi.org/10.1109/CVPR.2016.308]
[http://dx.doi.org/10.1109/CVPR.2016.308]
[44]
Gao Y, Mosalam KM. Deep transfer learning for image-based structural damage recognition. Comput Aided Civ Infrastruct Eng 2018; 33(9): 748-68.
[http://dx.doi.org/10.1111/mice.12363]
[http://dx.doi.org/10.1111/mice.12363]
[45]
Apostolopoulos ID, Mpesiana TA. Covid-19: Automatic detection from X-ray images utilizing transfer learning with convolutional neural networks. Physic Eng Sci Medic 2020; 43(2): 635-40.
[http://dx.doi.org/10.1007/s13246-020-00865-4] [PMID: 32524445]
[http://dx.doi.org/10.1007/s13246-020-00865-4] [PMID: 32524445]
[46]
Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 2018; 172(5): 1122-31.
[http://dx.doi.org/10.1016/j.cell.2018.02.010] [PMID: 29474911]
[http://dx.doi.org/10.1016/j.cell.2018.02.010] [PMID: 29474911]
[47]
Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng 2010; 22(10): 1345-59.
[http://dx.doi.org/10.1109/TKDE.2009.191]
[http://dx.doi.org/10.1109/TKDE.2009.191]
[48]
Shin HC, Roth HR, Gao M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 2016; 35(5): 1285-98.
[http://dx.doi.org/10.1109/TMI.2016.2528162] [PMID: 26886976]
[http://dx.doi.org/10.1109/TMI.2016.2528162] [PMID: 26886976]
[49]
Whitehouse DJ. Surface metrology. Meas Sci Technol 1997; 8(9): 955-72.
[http://dx.doi.org/10.1088/0957-0233/8/9/002]
[http://dx.doi.org/10.1088/0957-0233/8/9/002]
[51]
Pimenov DY, Bustillo A, Mikolajczyk T. Artificial intelligence for automatic prediction of required surface roughness by monitoring wear on face mill teeth. J Intell Manuf 2018; 29(5): 1045-61.
[http://dx.doi.org/10.1007/s10845-017-1381-8]
[http://dx.doi.org/10.1007/s10845-017-1381-8]
[52]
CN-GB, Geometrical product specifications (GPS) - Surface texture: Profile method - Surface roughness parameters and their values. 2009. GB/T 12472-2003: 3
[53]
Jurkovic Z, Cukor G, Brezocnik M, Brajkovic T. A comparison of machine learning methods for cutting parameters prediction in high speed turning process. J Intell Manuf 2018; 29(8): 1683-93.
[http://dx.doi.org/10.1007/s10845-016-1206-1]
[http://dx.doi.org/10.1007/s10845-016-1206-1]
[54]
Aich U, Banerjee S. Modeling of EDM responses by support vector machine regression with parameters selected by particle swarm optimization. Appl Math Model 2014; 38(11-12): 2800-18.
[http://dx.doi.org/10.1016/j.apm.2013.10.073]
[http://dx.doi.org/10.1016/j.apm.2013.10.073]
[55]
Gupta MK, Mia M, Pruncu CI, et al. Parametric optimization and process capability analysis for machining of nickel-based superalloy. Int J Adv Manuf Technol 2019; 102(9-12): 3995-4009.
[http://dx.doi.org/10.1007/s00170-019-03453-3]
[http://dx.doi.org/10.1007/s00170-019-03453-3]
[56]
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) Munich, GERMANY. 2015; Oct 05-09: pp. 234-41.
[57]
Li X, Zhang W, Ding Q, Sun JQ. Intelligent rotating machinery fault diagnosis based on deep learning using data augmentation. J Intell Manuf 2020; 31(2): 433-52.
[http://dx.doi.org/10.1007/s10845-018-1456-1]
[http://dx.doi.org/10.1007/s10845-018-1456-1]
[58]
Cirillo MD, Abramian D, Eklund A. In what is the best data augmentation for 3d brain tumor segmentation? IEEE international conference on image processing (ICIP) Sep 19-22 2021; pp. 36-40.
[59]
Kong W, Dong ZY, Hill DJ, Luo F, Xu Y. Short-term residential load forecasting based on resident behaviour levelarning. IEEE Transact Power Sys 2017; 33(1): 1087-8.
[http://dx.doi.org/10.1109/TPWRS.2017.2688178]
[http://dx.doi.org/10.1109/TPWRS.2017.2688178]
[60]
Ketkar N. Deep learning with python. A Hands-on Introduction 2017. xvii, 226.
[http://dx.doi.org/10.1007/978-1-4842-2766-4]
[http://dx.doi.org/10.1007/978-1-4842-2766-4]
[61]
Shore J. Johnson RJITITo. Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. Information Theory IEEE Transact 1980; 26(1): 26-37.
[http://dx.doi.org/10.1109/TIT.1980.1056144]
[http://dx.doi.org/10.1109/TIT.1980.1056144]
[62]
Golik P, Doetsch P, Ney H. In cross-entropy vs. squared error training: A theoretical and experimental comparison. 14th annual conference of the international-speechcommunication association (INTERSPEECH 2013) Lyon,FRANCE. Aug 25 2013; 29: pp.1755-9.
[http://dx.doi.org/10.21437/Interspeech.2013-436]
[http://dx.doi.org/10.21437/Interspeech.2013-436]
[63]
Kingma D, Ba JJCS. Adam: A method for stochastic optimization. arxiv:14142.
[http://dx.doi.org/10.48550/arXiv.1412.6980]
[http://dx.doi.org/10.48550/arXiv.1412.6980]
[64]
Chang Z, Zhang Y, Chen W. Electricity price prediction based on hybrid model of ADAM optimized LSTM neural network and wavelet transform. Energy 2019; 187: 115804.
[http://dx.doi.org/10.1016/j.energy.2019.07.134]
[http://dx.doi.org/10.1016/j.energy.2019.07.134]
[65]
Ou M, Wei H, Zhang Y, Tan J. A dynamic adam based deep neural network for fault diagnosis of oil-immersed power transformers. Energies 2019; 12(6): 995.
[http://dx.doi.org/10.3390/en12060995]
[http://dx.doi.org/10.3390/en12060995]
[66]
Yi D, Ahn J, Ji S. An effective optimization method for machine learning based on ADAM. Appl Sci 2020; 10(3): 1073.
[http://dx.doi.org/10.3390/app10031073]
[http://dx.doi.org/10.3390/app10031073]
[67]
Ide H, Kurita T. Improvement of learning for CNN with ReLU activation by sparse regularization. In international joint conference on neural networks (IJCNN) anchorage, AK 2017 May; 14(19): 2684-91.
[http://dx.doi.org/10.1109/IJCNN.2017.7966185]
[http://dx.doi.org/10.1109/IJCNN.2017.7966185]
[68]
Wang G, Giannakis GB, Chen J. Learning ReLU networks on linearly separable data: algorithm, optimality, and generalization. IEEE Trans Signal Process 2019; 67(9): 2357-70.
[http://dx.doi.org/10.1109/TSP.2019.2904921]
[http://dx.doi.org/10.1109/TSP.2019.2904921]
[69]
Wang SH, Muhammad K, Hong J, Sangaiah AK, Zhang YD. Alcoholism identification viaconvolutional neural network based on parametric ReLU, dropout, and batch normalization. Neural Comput Appl 2020; 32(3): 665-80.
[http://dx.doi.org/10.1007/s00521-018-3924-0]
[http://dx.doi.org/10.1007/s00521-018-3924-0]
[70]
Abadi M, Agarwal A, Barham P, et al. Tensor Flow: Large-scale machine learning on heterogeneous distributed systems. ArXiv 2016; pp. 1603-04467.
[http://dx.doi.org/10.48550/arXiv.1603.04467]
[http://dx.doi.org/10.48550/arXiv.1603.04467]
[72]
Singh P, Singh N, Sing KK, et al. Diagnosing of disease using machine learning. Machine Learning and the Internet of Medical Things in Healthcare 2021; pp. 89-111.
[http://dx.doi.org/10.1016/B978-0-12-821229-5.00003-3]
[http://dx.doi.org/10.1016/B978-0-12-821229-5.00003-3]
[73]
Donna , Katzman . Making MJMD. Analyzing a portion of the roc curve. Mach Learn 1989; 9: 190-5.
[http://dx.doi.org/10.1177/0272989X8900900307] [PMID: 2668680]
[http://dx.doi.org/10.1177/0272989X8900900307] [PMID: 2668680]
[74]
Hand DJ, Till RJ. A simple generalisation of the area under the roc curve for multiple class classification problems. Mach Learn 2001; 45(2): 171-86.
[http://dx.doi.org/10.1023/A:1010920819831]
[http://dx.doi.org/10.1023/A:1010920819831]
[75]
Hui Fang, Tam GK-L, Borgo R, et al. Visualizing natural image statistics. IEEE Trans Vis Comput Graph 2013; 19(7): 1228-41.
[http://dx.doi.org/10.1109/TVCG.2012.312] [PMID: 23661013]
[http://dx.doi.org/10.1109/TVCG.2012.312] [PMID: 23661013]
[76]
Zalama E, Gómez-García BJ, Medina R, Llamas J. Road crack detection using visual features extracted by gabor filters. Comput Aided Civ Infrastruct Eng 2014; 29(5): 342-58.
[http://dx.doi.org/10.1111/mice.12042]
[http://dx.doi.org/10.1111/mice.12042]
[77]
Hua BO, Fu-Long MA, Jiao LC. Research on computation of GLCM of image texture. Tien Tzu Hsueh Pao 2006; 1(1): 155-8.
[78]
Guo Z, Zhang L, Zhang D. Rotation invariant texture classification using LBP variance (LBPV) with global matching. Pattern Recognit 2010; 43(3): 706-19.
[http://dx.doi.org/10.1016/j.patcog.2009.08.017]
[http://dx.doi.org/10.1016/j.patcog.2009.08.017]
