Note! Please note that this article is currently in the "Article in Press" stage and is not the final "Version of record". While it has been accepted, copy-edited, and formatted, however, it is still undergoing proofreading and corrections by the authors. Therefore, the text may still change before the final publication. Although "Articles in Press" may not have all bibliographic details available, the DOI and the year of online publication can still be used to cite them. The article title, DOI, publication year, and author(s) should all be included in the citation format. Once the final "Version of record" becomes available the "Article in Press" will be replaced by that.
Abstract
Background: Deep learning object detection algorithm is widely used in the field of image classification and has become an indispensable part. With the improvement of image classification accuracy, sonar image target detection algorithm based on deep learning has gradually become the focus of more and more people's research.
Objective: This article aims to provide a summary and analysis of deep learning-based sonar image object detection algorithms, with the hope of offering insights for future research in the field of sonar target detection technology.
Method: This paper systematically summarizes sonar image target detection algorithms based on deep learning. According to the method principle, the existing deep learning target detection algorithms are divided into four categories: target detection algorithm based on candidate region, deep target detection method based on regression, Anchor Free deep learning target detection algorithm, and search-based target detection and recognition algorithm. Then, the performance of algorithms based on COCO data sets is compared, and the standard sonar data sets and formats are introduced.
Results: The sonar image object detection algorithm based on deep learning has made significant progress. The combination of deep learning and object detection methods has been applied to sonar images, resulting in the emergence of excellent performing algorithms. However, most algorithms are still in the developmental stage and face challenges in practical applications. Subsequently, several invention patents have been developed based on the aforementioned algorithms, including a feature extraction method for side-scan sonar images based on fully convolutional neural networks, an underwater sonar image target detection method based on improved YOLOv3-tiny, and more.
Conclusion: Sonar image object detection technology based on deep learning has a wide range of application needs but also faces many difficulties and challenges, we still need to continue to learn and explore in future research, and we believe that we can make greater breakthroughs in the future.
[1]
G.J. Dobeck, "Algorithm fusion for the detection and classification of sea mines in the very shallow water region using side-scan sonar imagery", Int. Society for Optics & Photonics, 2000.
[http://dx.doi.org/10.1117/12.396262]
[http://dx.doi.org/10.1117/12.396262]
[2]
P.K. Lehardy, and C. Moore, Deep ocean search for Malaysia airlines flight 370.Proceedings of 2014 Ocean St.John’s., IEEE: St. John’s, NL, Canada, 2014, pp. 1-4.
[http://dx.doi.org/10.1109/OCEANS.2014.7003292]
[http://dx.doi.org/10.1109/OCEANS.2014.7003292]
[3]
G.E. Hinton, and R.R. Salakhutdinov, "Reducing the dimensionality of data with neural networks", Science, vol. 313, no. 5786, pp. 504-507, 2006.
[http://dx.doi.org/10.1126/science.1127647]
[http://dx.doi.org/10.1126/science.1127647]
[4]
J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A.Y. Ng, Multimodal deep learning Proceedings of the 28th International Conference on Machine Learning (CML-11), 2011, pp. 689-696.
[5]
C. Tan, "Mathematical model construction of teaching evaluation in colleges and universities based on convolutional neural network under the background of big data", J. Funct. Spaces, vol. 2022, pp. 1-8, 2022.
[http://dx.doi.org/10.1155/2022/7064287]
[http://dx.doi.org/10.1155/2022/7064287]
[6]
M.Z. Ma, Research on underwater target recognition technology., Harbin Engineering University, 2007.
[7]
G.Y. Liu, Research on object recognition technology based on sonar image., Harbin Engineering University, 2009.
[8]
N. Hurtos, N. Palomeras, S. Nagappa, and J. Salvi, "Automatic detection ofunderwater chain links using a forwardlooking sonar", In: OCEANS-Bergen, 2013 MTS/ IEEE., 2013.
[9]
V. Myers, and J. Fawcett, "A template matching procedure for automatic target recognition in synthetic aperture sonar imagery", IEEE Signal Process. Lett., vol. 17, no. 7, pp. 683-686, 2021.
[10]
Q. Chen, Research on underwater target recognition technology., Harbin Engineering University, 2013.
[11]
E. Dura, Y. Zhang, X. Liao, G.J. Dobeck, and L. Carin, "Active leaming for detection of mine-like objects in side-scan sonar imagery", IEEE J. Oceanic Eng., vol. 30, no. 2, pp. 360-371, 2005.
[http://dx.doi.org/10.1109/JOE.2005.850931]
[http://dx.doi.org/10.1109/JOE.2005.850931]
[12]
J. Tian, Target recognition and ship radiation noise recognition in hydroacoustic imaging., Institute of Acoustics, Chinese Academy of Sciences, 2004.
[13]
D.P. Williams, "Fast target detection in synthetic aperture sonar imagery: A new algorithm and large-scale performance analysis", IEEE J. Oceanic Eng., vol. 40, no. 1, pp. 71-92, 2015.
[http://dx.doi.org/10.1109/JOE.2013.2294532]
[http://dx.doi.org/10.1109/JOE.2013.2294532]
[14]
M. Gao, Research on feature extraction technology of underwater acoustic images., Harbin Engineering University, 2009.
[15]
J. Groena, E. Coirasa, and D. Williamsa, "Detection rate statistics in synthetic aperture sonarimages", 3rd International Conference & Exhibition on "Underwater Acoustic Measurements: Technologies & Results, 2009.
[16]
D. Liu, Object detection and tracking based on multi-resolution processing of sonar images., Harbin Engineering University, 2011.
[17]
M. Valdenegro-Toro, Objectness Scoring and Detection Proposals in Forward-Looking Sonar Images with Convolutional Neural Networks.Artificial Neural Networks in Pattern Recognition. ANNPR 2016. Lecture Notes in Computer Science, vol. 9896. Springer: Cham, 2016.
[http://dx.doi.org/10.1007/978-3-319-46182-3_18]
[http://dx.doi.org/10.1007/978-3-319-46182-3_18]
[18]
M. Valdenegro-Toro, "End-to-end object detection and recognition in forward-looking sonar images with convolutional neural networks", In: Autonomous Underwater Vehicles, 2016.
[http://dx.doi.org/10.1109/AUV.2016.7778662]
[http://dx.doi.org/10.1109/AUV.2016.7778662]
[19]
M. Valdenegro-Toro, "Best practices in convolutional networks for forward-looking sonar image recognition", In: OCEANS 2017 - Aberdeen., 2017.
[http://dx.doi.org/10.1109/OCEANSE.2017.8084987]
[http://dx.doi.org/10.1109/OCEANSE.2017.8084987]
[20]
J. Kim, H. Cho, J. Pyo, and B. Kim, "The convolution neural network based agentvehicle detection using forwardlooking sonar mage", In: OCEANS 2016 MTS/IEEE Monterey. 2016, 2016, pp. 1-5.
[21]
J. Kim, and S.C. Yu, Convolutional neural network-based real-time ROV detection using forward-looking sonar image.In: 2016 IEEE/OES Autonomous Underwater Vehicles., AUV, 2016, pp. 396-400.
[http://dx.doi.org/10.1109/AUV.2016.7778702]
[http://dx.doi.org/10.1109/AUV.2016.7778702]
[22]
W. Hongjian, G. Na, C. Tao, X. Yao, R. Li, and L Benyin, A feature extraction method for sidescan sonar images based on fully convolutional neural networksHeilongjiang Province: CN110781924B February 14, .
[23]
W. Xingmei, J. Jia, S. Boxuan, W. Guoqiang, and L. Anhua, Adaptive Weight Convolutional Neural Network-based Classification Method for Underwater Sonar Images Using Deep LearningHeilongjiang: CN108427958A, August 21, .
[24]
Q. Ye, H. Huang, and C. Zhang, "Image enhancement using stochastic resonance [sonar image processing applications", In 2004 International Conference on Image Processing, 2004, pp. 263-266
[25]
R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587, 2014.
[http://dx.doi.org/10.1109/CVPR.2014.81]
[http://dx.doi.org/10.1109/CVPR.2014.81]
[26]
J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders, "Selective search for object recognition", Int. J. Comput. Vis., vol. 104, no. 2, pp. 154-171, 2013.
[http://dx.doi.org/10.1007/s11263-013-0620-5]
[http://dx.doi.org/10.1007/s11263-013-0620-5]
[27]
N. Bodla, B. Singh, R. Chellappa, and L.S. Davis, "Soft-NMS--improving object detection with one line of code", Proceedings of the IEEE international conference on computer vision, pp. 5561-5569, 2017.
[http://dx.doi.org/10.1109/ICCV.2017.593]
[http://dx.doi.org/10.1109/ICCV.2017.593]
[28]
R. Girshick, "Fast R-CNN", IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 1440-1448, 2015.
[29]
J. Sun, K. He, R. Girshick, and S. Ren, "Faster r-cnn: Towards realtime object detection with region proposal networks", Adv. Neural Inf. Process. Syst., pp. 91-99, 2015.
[30]
K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN", In IEEE International Conference on Computer Vision, 2017, pp. 2980-2988
[31]
T.Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117-2125.
[32]
Z. Cai, and N. Vasconcelos, "Cascade r-cnn: Delving into high quality object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6154-6162.
[http://dx.doi.org/10.1109/CVPR.2018.00644]
[http://dx.doi.org/10.1109/CVPR.2018.00644]
[33]
Y. Li, Y.N. Chen, N. Wang, and Z. Zhang, "Scale-aware trident networks for object detection", Proceedings of the IEEE international conference on computer wision, 2019, pp. 6054-6063.
[34]
J. Redmon, S. Diwala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779-788.
[http://dx.doi.org/10.1109/CVPR.2016.91]
[http://dx.doi.org/10.1109/CVPR.2016.91]
[35]
J. Redmon, and A. Farhadi, "YOL09000: Better, faster, stronger", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263-7271
[37]
Y. Huizhen, Z. Yujia, and L. Yuan, "Underwater Sonar Image Object Detection Method based on Improved YOLOv3-tiny", Shaanxi Province: CN112861919A, May 5.
[38]
A. Bochkovskiy, C.Y. Wang, and H.Y.M. Liao, "YOLOv4: Optimal speed and accuracy of object detection", arXiv, 2020.
[39]
" W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu, and
A.C. Berg, "SSD: Single shot multibox detector", European conference
on computer vision., pp. 21-37, 2016.Springer, Cham",
[http://dx.doi.org/10.1007/978-3-319-46448-0_2]
[http://dx.doi.org/10.1007/978-3-319-46448-0_2]
[40]
C.Y. Fu, W. Liu, A. Ranga, A. Tyagi, and A.C. Berg, "Dssd: Deconvolutional single shot detector", arXiv, 2017.
[41]
Z. Shen, Z. Liu, J. Li, Y.G. Jiang, and X. Xue, "Dsod: Learning deeply supervised object detectors from scratch", Proceedings of the IEEE international conference on computer vision, 2017, pp. 1919-1927.
[http://dx.doi.org/10.1109/ICCV.2017.212]
[http://dx.doi.org/10.1109/ICCV.2017.212]
[43]
J. Jeong, H. Park, and N. Kwak, "Enhancement of SSD by concatenating feature maps for object detection", arXiv, 2017.
[http://dx.doi.org/10.5244/C.31.76]
[http://dx.doi.org/10.5244/C.31.76]
[44]
S. Zhang, L. Wen, X. Bian, Z. Lei, and S.Z. Li, "Single-shot refinement neural network for object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4203-4212.
[http://dx.doi.org/10.1109/CVPR.2018.00442]
[http://dx.doi.org/10.1109/CVPR.2018.00442]
[45]
L. Huang, Y. Yang, Y. Deng, and Y. Yu, "Densebox: Unifying landmark localization with end to end object detection", arXiv, 2015.
[46]
J. Yu, Y. Jiang, Z. Wang, Z. Cao, and T. Huang, "Unitbox: An advanced object detection network", Proceedings of the 24th ACM international conference on Multimedia, 2016, pp. 516-520.
[http://dx.doi.org/10.1145/2964284.2967274]
[http://dx.doi.org/10.1145/2964284.2967274]
[47]
H. Law, and J. Deng, "Cornernet: Detecting objects as paired keypoints", Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 734-750.
[49]
X. Zhou, J. Zhuo, and P. Krahenbuhl, "Bottom-up object detection by grouping extreme and center points", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 850-859.
[http://dx.doi.org/10.1109/CVPR.2019.00094]
[http://dx.doi.org/10.1109/CVPR.2019.00094]
[50]
C. Zhu, Y. He, and M. Savvides, "Feature selective anchor-free module for single-shot object detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 840-849.
[http://dx.doi.org/10.1109/CVPR.2019.00093]
[http://dx.doi.org/10.1109/CVPR.2019.00093]
[51]
Z. Tian, C. Shen, H. Chen, and T. He, "Fcos: Fully convolutional one-stage object detection", Proceedings of the IEEE international conference on computer vision, 2019, pp. 9627-9636.
[52]
T. Kong, F. Sun, H. Liu, Y. Jiang, and J. Shi, "Foveabox: Beyound anchor-based object detection", IEEE Trans. Image Process., pp. 7389-7398, 2022.
[53]
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, "End-to-End Object Detection with Transformers", arXiv .
[http://dx.doi.org/10.1007/978-3-030-58452-8_13]
[http://dx.doi.org/10.1007/978-3-030-58452-8_13]
[54]
D. Yoo, S. Park, J.Y. Lee, A.S. Paek, and I.S. Kweon, AttentionNet: Aggregating Weak Directions for Accurate Object Detection., IEEE, 2016.
[55]
J.W. Li, C.W. Qu, J.Q. Shao, and S.J. Peng, "Deep Learning-based ship detection data set and performance analysis of SAR images", In Proceedings of the Fifth Annual Symposium on High Resolution Earth Observation, 2018.
[56]
P. Xiang, W.W. Guo, Z.H. Zhang, W.X. Yu, P. Xiang, W.W. Guo, Z.H. Zhang, and W.X. Yu, "Opensar data sharing platform for sar interpretation", Inf. Tecnol., no. September, pp. 1-4, 2016.
[57]
X. Sun, Z.R. Wang, Y.R. Sun, W.H. Diao, Y. Zhang, and K. Fu, "Air-sarship-1.0: High-Resolution sarship detection Data set", J. Radar, no. August, pp. 852-862, 2019.
[58]
Y. Zhou, S.C. Chen, K. Wu, M.Q. Ning, H.K. Chen, and P. Zhang, "SCTD1.0: Sonar common target detection data set., vol. 48", Comput. Sci., 2021.
[59]
K. Xie, J. Yang, and K. Qiu, "A dataset with multibeam forward-looking sonar for underwater object detection", Sci. Data, vol. 9, no. 1, p. 739, 2022.
[http://dx.doi.org/10.1038/s41597-022-01854-w] [PMID: 36456623]
[http://dx.doi.org/10.1038/s41597-022-01854-w] [PMID: 36456623]
Related Books
Advanced Characterization Technologies for Secondary Batteries
Design and Construction of Buildings and Foundations with Illustrative Examples
Robotics and Automation in Industry 4.0
Biomedical Applications of Perovskites: The Era of Bio-Piezoelectric Systems
Smart Kinematics For Modern Engineering Students
Computational Modeling and Simulation in Biomedical Research
Introduction to Advanced Soft Robotics
Intelligent Technologies for Research and Engineering
The Production of Biodiesel and Related Fuel Additives
Intelligent Technologies for Research and Engineering
