Abstract
Computer vision is one of the prime domains that enable to derive meaningful and crisp information from digital media, such as images, videos, and other visual inputs.
Background: Detection and correctly tracking the moving objects in a video streaming is still a challenging problem in India. Due to the high density of vehicles, it is difficult to identify the correct objects on the roads.
Methods: In this work, we have used a YOLO.v5 (You Only Look Once) algorithm to identify the different objects on road, such as trucks, cars, trams, and vans. YOLO.v5 is the latest algorithm in the family of YOLO. To train the YOLO.v5, KITTY dataset was used having 11682 images having different objects in a traffic surveillance system. After training and validating the dataset, three different models have been constructed setting various parameters. To further validate the proposed approach, results have also been evaluated on the Indian traffic dataset DATS_2022.
Results: All the models have been evaluated using three performance metrics, such as precision, recall, and mean average precision (MAP). The final model has attained the best performance on KITTY dataset as 93.5% precision, 90.7% recall, and 0.67 MAP for different objects. The results attained on the Indian traffic dataset DATS_2022 included 0.65 precision, 0.78 recall value, and 0.74 MAP for different objects.
Conclusion: The results depict the proposed model to have improved results as compared to stateof- the-art approaches in terms of performance and also reduce the computation time and object loss.
Graphical Abstract
[1]
B.G. Han, J.T. Lee, K.T. Lim, and Y. Chung, "Real-time license plate detection in high-resolution videos using fastest available cascade classifier and core patterns", ETRI J., vol. 37, no. 2, pp. 251-261, 2015.
[http://dx.doi.org/10.4218/etrij.15.2314.0077]
[http://dx.doi.org/10.4218/etrij.15.2314.0077]
[2]
K-J. Kim, P-K. Kim, K-T. Lim, Y-S. Chung, Y-J. Song, S.I. Lee, and D-H. Choi, Vehicle color recognition via representative color region extraction and convolutional neural network. International Conference on Ubiquitous and Future Networks,, ICUFN, Prague, Czech Republic,, 2018, pp. 89-94.
[http://dx.doi.org/10.1109/ICUFN.2018.8436710]
[http://dx.doi.org/10.1109/ICUFN.2018.8436710]
[3]
V.A. Prisacariu, R. Timofte, K. Zimmermann, I. Reid, and L. Van Gool, Integrating object detection with 3D tracking towards a better driver assistance system. 20th International Conference on Pattern Recognition, Istanbul, Turkey, 2010, pp. 3344-3347.
[http://dx.doi.org/10.1109/ICPR.2010.816]
[http://dx.doi.org/10.1109/ICPR.2010.816]
[4]
"Wei, Jian, Jianhua He, Yi Zhou, Kai Chen, Zuoyin Tang, and ZhiliangXiong. “Enhanced object detection with deep convolutional neural networks for advanced driving assistance.”", IEEE Trans. Intell. Transp. Syst., vol. 21, no. 4, pp. 1572-1583, 2019.
[5]
S. Srigrarom, and K.H. Chew, "Hybrid motion-based object detection for detecting and tracking of small and fast moving drones",International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece, 2020, pp. 615-621.
[http://dx.doi.org/10.1109/ICUAS48674.2020.9213912]
[http://dx.doi.org/10.1109/ICUAS48674.2020.9213912]
[6]
L. Hu, and Q. Ni, "IoT-driven automated object detection algorithm for urban surveillance systems in smart cities", IEEE Internet Things J., vol. 5, no. 2, pp. 747-754, 2018.
[http://dx.doi.org/10.1109/JIOT.2017.2705560]
[http://dx.doi.org/10.1109/JIOT.2017.2705560]
[7]
W. Wu, O. Bulan, E.A. Bernal, and R.P. Loce, Detection of Moving Violations.Computer Vision and Imaging in Intelligent Transportation Systems. Hoboken, NJ, USA: Wiley-IEEE Press, 2017, pp. 101-130.
[http://dx.doi.org/10.1002/9781118971666.ch5]
[http://dx.doi.org/10.1002/9781118971666.ch5]
[8]
M.A.A. Al-qaness, A.A. Abbasi, H. Fan, R.A. Ibrahim, S.H. Alsamhi, and A. Hawbani, "An improved YOLO-based road traffic monitoring system", Computing, vol. 103, no. 2, pp. 211-230, 2021.
[http://dx.doi.org/10.1007/s00607-020-00869-8]
[http://dx.doi.org/10.1007/s00607-020-00869-8]
[9]
T. Xu, Z. Zhang, X. Wu, L. Qi, and Y. Han, "Recognition of lane-changing behaviour with machine learning methods at freeway off-ramps", Phys. A: Stat. Mech., vol. 567, p. 125691, 2021.
[http://dx.doi.org/10.1016/j.physa.2020.125691]
[http://dx.doi.org/10.1016/j.physa.2020.125691]
[10]
D. Rosenbaum, F. Kurz, U. Thomas, S. Suri, and P. Reinartz, "Towards automatic near real-time traffic monitoring with an airborne wide angle camera system", Eur. Trans. Res. Rev., vol. 1, no. 1, pp. 11-21, 2009.
[http://dx.doi.org/10.1007/s12544-008-0002-1]
[http://dx.doi.org/10.1007/s12544-008-0002-1]
[11]
M. Fachrie, "A simple vehicle counting system using deep learning with YOLOv3 Model", Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 4, no. 3, pp. 462-468, 2020.
[http://dx.doi.org/10.29207/resti.v4i3.1871]
[http://dx.doi.org/10.29207/resti.v4i3.1871]
[12]
H. Song, H. Liang, H. Li, Z. Dai, and X. Yun, "Vision-based vehicle detection and counting system using deep learning in highway scenes", European Transport Research Review, vol. 11, p. 51, 2019.
[http://dx.doi.org/10.1186/s12544-019-0390-4]
[http://dx.doi.org/10.1186/s12544-019-0390-4]
[13]
S. Alghyaline, N. El-Omari, R.M. Al-Khatib, and Al-Kharbshh H., "Rt-vc: An Efficient Real-Time Vehicle Counting Approach, ", J. Theor. ppl. Inf. Tecnol., vol. 97, pp. 2062-2075, 2019.
[14]
Z. Iftikhar, P. Dissanayake, and P. Vial, Computer Vision Based Traffic Monitoring System for Multi-track Freeways. Intelligent Computing ethodologies. vol 8589 Cham, Switzerland, Springer International Publishing, 2014, pp. 339-349.
[http://dx.doi.org/10.1007/978-3-319-09339-0_35]
[http://dx.doi.org/10.1007/978-3-319-09339-0_35]
[15]
S. Awang, N.M.A.N. Azmi, and M.A. Rahman, "Vehicle type classification using an enhanced sparse-filtered convolutional neural network with layer-skipping strategy", IEEE Access, vol. 8, pp. 14265-14277, 2020.
[http://dx.doi.org/10.1109/ACCESS.2019.2963486]
[http://dx.doi.org/10.1109/ACCESS.2019.2963486]
[16]
W. Sun, X. Zhang, S. Shi, and X. He, "Vehicle classification approach based on the combined texture and shape features with a compressive DL", IET Intell. Transp. Syst., vol. 13, no. 7, pp. 1069-1077, 2019.
[http://dx.doi.org/10.1049/iet-its.2018.5316]
[http://dx.doi.org/10.1049/iet-its.2018.5316]
[17]
Q. Kang, H. Zhao, D. Yang, H.S. Ahmed, and J. Ma, "Lightweight convolutional neural network for vehicle recognition in thermal infrared images", Infrared Phys. Technol., vol. 104, p. 104, 2020.
[18]
W. Sun, X. Zhang, and X. He, "A Two-Stage Vehicle Type Recognition Method Combining the Most Effective Gabor Features", CMC-Computer. Contin., vol. 65, pp. 2489-2510, 2020.
[19]
C. Zhang, and B. Du, "Image-Based Approach for Parking-Spot Detection with Occlusion Handling", J. Transp. Eng. Part Syst., vol. 146, no. 9, 2020.
[20]
Tătulea, Călin, Brad, Brâncovean, and Greavu, "An Image Feature-Based Method for Parking Lot Occupancy", Future Internet, vol. 11, no. 8, p. 169, 2019.
[http://dx.doi.org/10.3390/fi11080169]
[http://dx.doi.org/10.3390/fi11080169]
[21]
B.Y. Cai, R. Alvarez, M. Sit, F. Duarte, and C. Ratti, "Deep Learning-Based Video System for Accurate and Real-Time Parking Measurement", IEEE Internet Things J., vol. 6, no. 5, pp. 7693-7701, 2019.
[http://dx.doi.org/10.1109/JIOT.2019.2902887]
[http://dx.doi.org/10.1109/JIOT.2019.2902887]
[22]
X. Ding, and R. Yang, "Vehicle and Parking Space Detection Based on Improved YOLO Network Model", J. Phys. Conf. Ser., vol. 1325, no. 1, 2019, p. 012084, .
[http://dx.doi.org/10.1088/1742-6596/1325/1/012084]
[http://dx.doi.org/10.1088/1742-6596/1325/1/012084]
[23]
K.J. Kim, P.K. Kim, Y.S. Chung, and D.H. Choi, "Multi-scale detector for accurate vehicle detection in traffic surveillance data", IEEE Access, vol. 7, pp. 78311-78319, 2019.
[http://dx.doi.org/10.1109/ACCESS.2019.2922479]
[http://dx.doi.org/10.1109/ACCESS.2019.2922479]
[24]
Q. Fan, L. Brown, and J. Smith, A closer look at Faster R-CNN for vehicle detection. 2016 IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden, 2016, pp. 124-129.
[http://dx.doi.org/10.1109/IVS.2016.7535375]
[http://dx.doi.org/10.1109/IVS.2016.7535375]
[25]
Q.C. Mao, H.M. Sun, L-Q. Zuo, and R-S. Jia, "Finding every car: A traffic surveillance multi-scale vehicle object detection method", Appl. Intell., vol. 50, no. 10, pp. 3125-3136, 2020.
[http://dx.doi.org/10.1007/s10489-020-01704-5]
[http://dx.doi.org/10.1007/s10489-020-01704-5]
[26]
Soharab Hossain Shaikh, Khalid Saeed, and Nabendu Chaki, Moving Object Detection Using Background Subtraction., Cham Springer, 2014.
[27]
X. Dong, K. Wang, and G. Jia, "Moving Object and Shadow Detection Based on RGB Color Space and Edge Ratio.", 2nd International Congress on Image and Signal Processing, Tianjin, China, 2009, pp. 1-5.
[http://dx.doi.org/10.1109/CISP.2009.5301770]
[http://dx.doi.org/10.1109/CISP.2009.5301770]
[28]
J. Choi, H.J. Chang, Y.J. Yoo, and J.Y. Choi, "Robust moving object detection against fast illumination change", Comput. Vis. Image Underst., vol. 116, no. 2, pp. 179-193, 2012.
[29]
C. Stauffer, and W.E.L. Grimson, Adaptive background mixture models for real-time tracking. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA, 1999, pp. 246-252.
[http://dx.doi.org/10.1109/CVPR.1999.784637]
[http://dx.doi.org/10.1109/CVPR.1999.784637]
[30]
Dar-Shyang Lee, "Effective Gaussian mixture learning for video background subtraction", 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance, Santa Fe, NM, USA, 2008, pp. 41-48.
[http://dx.doi.org/10.1109/TPAMI.2005.102] [PMID: 15875805]
[http://dx.doi.org/10.1109/TPAMI.2005.102] [PMID: 15875805]
[31]
M. Haque, M. Murshed, and M. Paul, "On stable dynamic background generation technique using gaussian mixture models for robust object detection", 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance, Santa Fe, NM, USA, 2008, pp. 41-48.
[http://dx.doi.org/10.1109/AVSS.2008.12]
[http://dx.doi.org/10.1109/AVSS.2008.12]
[32]
D. Chaudhari, and S. Porwal, "Frequency and spatial domains adaptive-based enhancement technique for thermal infrared images", Defence Sci, vol. 64, no. 5, p. 451, 2014.
[33]
Z.P. Dordevic, S.G. Graovac, and S.T. Mitrovic, "Suboptimal threshold estimation for detection of point-like objects in radar images, Eurasip. Image Video Process", Springer Open Access., vol. 5, pp. 1-12, 2015.
[34]
A. Akula, N. Khanna, R. Ghosh, S. Kumar, A. Das, and H.K. Sardana, "Adaptive contour-based statistical background subtraction method for moving target detection in infrared video sequences", Infrared Phys. Technol., vol. 63, pp. 103-109, 2014.
[http://dx.doi.org/10.1016/j.infrared.2013.12.012]
[http://dx.doi.org/10.1016/j.infrared.2013.12.012]
[35]
X. Dong, X. Huang, Y. Zheng, S. Bai, and W. Xu, "A novel infrared small moving target detection method based on tracking interest points under complicated background", Infrared Phys. Technol., vol. 65, pp. 36-42, 2014.
[http://dx.doi.org/10.1016/j.infrared.2014.03.007]
[http://dx.doi.org/10.1016/j.infrared.2014.03.007]
[36]
H. Tao, and X. Lu, "Automatic smoky vehicle detection from traffic surveillance video based on vehicle rear detection and multi‐feature fusion", IET Intell. Transp. Syst., vol. 13, no. 2, pp. 252-259, 2019.
[http://dx.doi.org/10.1049/iet-its.2018.5039]
[http://dx.doi.org/10.1049/iet-its.2018.5039]
[37]
R. Girshick, J. Donahue, T. Darrell, and J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 580-587.
[http://dx.doi.org/10.1109/CVPR.2014.81]
[http://dx.doi.org/10.1109/CVPR.2014.81]
[38]
J. Li, Z. Su, J. Geng, and Y. Yin, "Real-time detection of steel strip surface defects based on improved YOLO detection network", IFAC-PapersOnLine, vol. 51, no. 21, pp. 76-81, 2018.
[http://dx.doi.org/10.1016/j.ifacol.2018.09.412]
[http://dx.doi.org/10.1016/j.ifacol.2018.09.412]
[39]
S. Lu, B. Wang, H. Wang, L. Chen, M. Linjian, and X. Zhang, "A real-time object detection algorithm for video", Comput. Electr. Eng., vol. 77, pp. 398-408, 2019.
[http://dx.doi.org/10.1016/j.compeleceng.2019.05.009]
[http://dx.doi.org/10.1016/j.compeleceng.2019.05.009]
[40]
F. Hong, C.H. Lu, C. Liu, R.R. Liu, and J. Wei, "A traffic surveillance multi-scale vehicle detection object method base on encoder-decoder", IEEE Access, vol. 8, pp. 47664-47674, 2020.
[http://dx.doi.org/10.1109/ACCESS.2020.2979260]
[http://dx.doi.org/10.1109/ACCESS.2020.2979260]
[41]
N. Jain, ""ShreeshaYerragolla, and Tanuja Guha. “Performance Analysis of Object Detection and Tracking Algorithms for Traffic Surveillance Applications using Neural Networks",", 2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Palladam, India 2019, pp. 690-696, .
[http://dx.doi.org/10.1109/I-SMAC47947.2019.9032502]
[http://dx.doi.org/10.1109/I-SMAC47947.2019.9032502]
[42]
F. Zhou, F. Zhou, M. Yang, H. Zhu, and C. Fu, Real-time monocular human depth estimation and segmentation on embedded systems. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021, pp. 55-62.
[43]
D. Wei, S. An, X. Zhang, and J. Tian, Dual Regression for Efficient Hand Pose Estimation. 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 6423-6429.
[44]
I. Kansizoglou, E. Misirlis, K. Tsintotas, and A. Gasteratos, "Continuous emotion recognition for long-term behavior modeling through recurrent neural networks", Technologies, vol. 10, no. 3, p. 59, 2022.
Related Journals
Recent Patents on Engineering
Related Books
Voltammetry for Sensing Applications
