Abstract
Background: Pedestrian detection and tracking are an important area of study in realworld applications, such as mobile robots, human-computer interaction, video surveillance, pedestrian protection systems, etc. As a result, it has attracted the interest of the scientific community.
Objective: Certainly, tracking people is critical for numerous utility areas which cover unusual situations detection, like vicinity evaluation, and sometimes change direction in human gait and partial occlusions.
Researchers' primary focus is to develop a surveillance system that can work in a dynamic environment, but there are major issues and challenges involved in designing such systems. So, it has become a significant issue and challenge to design a tracking system that can be more suitable for such situations. To this end, this paper presents a comparative evaluation of the tracking-by-detection system along with the publicly available pedestrian benchmark databases.
Method: Unlike recent works where person detection and tracking are usually treated separately, our work explores the joint use of the popular Simple Online and Real-time Tracking (SORT) method and the relevant visual detectors. Consequently, the choice of the detector is an important factor in the evaluation of the system's performance.
Results: Experimental results demonstrate that the performance of the tracking-by-detection system is closely related to the optimal selection of the detector and should be required prior to a rigorous evaluation.
Conclusion: The study demonstrates how sensitive the system performance as a whole is to the challenges of the dataset. Furthermore, the efficiency of the detector and the detector-tracker combination is also depending on the dataset.
Keywords: Tracking-by-detection, detection, multi-person tracking, data association.
Graphical Abstract
[1]
Di. Yuan, "Xiaohuan. Lu, Donghao. Li, Yingyi. Liang, Xinming. Zhang, “Particle filter re-detection for visual tracking via correlation fil-ters”", Multimedia Tools Appl., vol. 78, pp. 14277-14301, 2019.
[http://dx.doi.org/10.1007/s11042-018-6800-0]
[http://dx.doi.org/10.1007/s11042-018-6800-0]
[2]
Di. Yuan, "“Wei. Kang, Zhenyu. He ” Robust visual tracking with correlation filters and metric learning", Knowl. Base. Syst., vol. 195, no. 11, 2020.105697
[http://dx.doi.org/10.1016/j.knosys.2020.105697]
[http://dx.doi.org/10.1016/j.knosys.2020.105697]
[3]
A. Dehghan, and M. Shah, "Binary quadratic programming for online tracking of hundreds of people in extremely crowded scenes", IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 3, pp. 568-581, 2018.
[http://dx.doi.org/10.1109/TPAMI.2017.2687462] [PMID: 28358675]
[http://dx.doi.org/10.1109/TPAMI.2017.2687462] [PMID: 28358675]
[4]
M.D. Breitenstein, F. Reichlin, B. Leibe, E. Koller-Meier, and L. Van Gool, "Online multiperson tracking-by-detection from a single, un-calibrated camera", IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 9, pp. 1820-1833, 2011.
[http://dx.doi.org/10.1109/TPAMI.2010.232] [PMID: 21173441]
[http://dx.doi.org/10.1109/TPAMI.2010.232] [PMID: 21173441]
[5]
"Djarah Djalal Benmakhlouf Abdeslam Zidani, Application of Neural Networks in Perception System Management for an Indoor Mobile Robot International Conference on Green Energy & Environmental Engineering (GEEE-2018) Proceedings of Engineering and Technology – PET, vol. Vol.37, pp. 34-38",
[6]
E. Moussy, A.A. Mekonnen, and G. Marion, 12th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2015, 2015
[7]
P. Dollár, R. Appel, S. Belongie, and P. Perona, "Fast feature pyramids for object detection", IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 8, pp. 1532-1545, 2014.
[http://dx.doi.org/10.1109/TPAMI.2014.2300479] [PMID: 26353336]
[http://dx.doi.org/10.1109/TPAMI.2014.2300479] [PMID: 26353336]
[8]
P. Dollár, C. Wojek, B. Schiele, and P. Perona, "Pedestrian detection: an evaluation of the state of the art", IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 743-761, 2012.
[http://dx.doi.org/10.1109/TPAMI.2011.155] [PMID: 21808091]
[http://dx.doi.org/10.1109/TPAMI.2011.155] [PMID: 21808091]
[9]
M.A. Naiel, M.O. Ahmad, M. Swamy, Y. Wu, and M-H. Yang, “Online multi-person tracking via robust collaborative model”, Image Pro-cessing., ICIP, 2014, pp. 431-435.
[http://dx.doi.org/10.1109/ICIP.2014.7025086]
[http://dx.doi.org/10.1109/ICIP.2014.7025086]
[10]
M. Danelljan, G. Häger, F.S. Khan, and M. Felsberg, "Discriminative scale space tracking", IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 8, pp. 1561-1575, 2017.
[http://dx.doi.org/10.1109/TPAMI.2016.2609928] [PMID: 27654137]
[http://dx.doi.org/10.1109/TPAMI.2016.2609928] [PMID: 27654137]
[11]
S-H. Bae, and K-J. Yoon, Robust online multiobject tracking based on tracklet confidence and online discriminative appearance learningProceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1218-1225.
[12]
Y. Dorai, F. Chausse, S. Gazzah, and N. Essoukri Ben Amara, Multi target tracking by linking tracklets with a convolutional neural net-work VISIGRAPP (6: VISAPP), pp. 492-498, 2017.
[http://dx.doi.org/10.5220/0006155204920498]
[http://dx.doi.org/10.5220/0006155204920498]
[13]
S. Tang, M. Andriluka, B. Andres, and B. Schiele, "Multiple people tracking by lifted multicut and person re-identification", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017pp. 3539-3548
[http://dx.doi.org/10.1109/CVPR.2017.394]
[http://dx.doi.org/10.1109/CVPR.2017.394]
[14]
B. Wang, L. Wang, B. Shuai, Z. Zuo, T. Liu, K. Luk Chan, and G. Wang, Joint learning of convolutional neural networks and temporally constrained metrics for tracklet associationProceedings of the IEEE Conference on Computer Vision and Pattern Recognition Work-shops, 2016, pp. 1-8.
[http://dx.doi.org/10.1109/CVPRW.2016.55]
[http://dx.doi.org/10.1109/CVPRW.2016.55]
[15]
Alex Bewley, Zongyuan Ge, Lionel Ott, and Fabio Ramos, Simple online and realtime tracking.In 2016 IEEE International Conference on Image Processing (ICIP) IEEE, 2016, pp. 3464-3468.
[16]
" Simple Online Realtime Tracking Github Repository", https://github.com/abewley/sortavailable under:
[17]
P.F. Felzenszwalb, R.B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models", IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, 2010.
[http://dx.doi.org/10.1109/TPAMI.2009.167] [PMID: 20634557]
[http://dx.doi.org/10.1109/TPAMI.2009.167] [PMID: 20634557]
[18]
N. Dalal, and B. Triggs, "Histograms of oriented gradients for human detection", IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), 2005. CA, USA
[http://dx.doi.org/10.1109/CVPR.2005.177]
[http://dx.doi.org/10.1109/CVPR.2005.177]
[19]
A. Mhalla, H. Maâmatou, T. Chateau, S. Gazzah, and N. Essoukri Ben Amara, Faster R-CNN scene specialization with a sequential Mon-te-Carlo frameworkInternational Conference on Digital Image Computing: Techniques and Applications (DICTA), 2016, pp. 1-7.
[http://dx.doi.org/10.1109/DICTA.2016.7797042]
[http://dx.doi.org/10.1109/DICTA.2016.7797042]
[20]
A. Milan, S.H. Rezatofighi, A.R. Dick, I.D. Reid, and K. Schindler, "Online multi-target tracking using recurrent neural networks", AAAI, pp. 4225-4232, 2017.
[21]
Di. Yuan, Shu Xiu, and He Zhenyu, "TRBACF: Learning temporal regularized correlation filters for high performance online visual ob-ject tracking", J. Vis. Commun. Image Represent., vol. 72, no. October, 2020.102882
[http://dx.doi.org/10.1016/j.jvcir.2020.102882]
[http://dx.doi.org/10.1016/j.jvcir.2020.102882]
[22]
Di. Yuan, Fan Nana., and He Zhenyu., "“Learning target-focusing convolutional regression model for visual object tracking Knowl. Base. Syst., vol. 194, 2020.105526
[http://dx.doi.org/10.1016/j.knosys.2020.105526]
[http://dx.doi.org/10.1016/j.knosys.2020.105526]
[23]
B. Lee, J. Park, Y. Joo, and S. Jin, "Intelligent Kalman filter for tracking a manoeuvring target", IEE Proc., Radar Sonar Navig., vol. 151, no. 6, pp. 344-350, 2004.
[http://dx.doi.org/10.1049/ip-rsn:20040894]
[http://dx.doi.org/10.1049/ip-rsn:20040894]
[24]
J. Vermaak, S.J. Godsill, and P. Perez, "Monte carlofiltering for multi target trackin g and data association", IEEE Trans. Aerosp. Electron. Syst., vol. 41, no. 1, pp. 309-332, 2005.
[http://dx.doi.org/10.1109/TAES.2005.1413764]
[http://dx.doi.org/10.1109/TAES.2005.1413764]
[25]
C. Kim, F. Li, A. Ciptadi, and J.M. Rehg, "Multiple hypothesis tracking revisited", Proceedings of the IEEE International Conference on Computer Vision, 2015pp. 4696-4704
[26]
H. Yoo, K. Kim, M. Byeon, Y. Jeon, and J.Y. Choi, "Online scheme for multiple camera multiple target tracking based on multiple hypoth-esis tracking", IEEE Trans. Circ. Syst. Video Tech., vol. 27, no. 3, pp. 454-469, 2017.
[http://dx.doi.org/10.1109/TCSVT.2016.2593619]
[http://dx.doi.org/10.1109/TCSVT.2016.2593619]
[27]
I.J. Cox, and S.L. Hingorani, "An efficient implementation of reid’s multiple hypothesis tracking algorithm and its evaluation for the pur-pose of visual tracking", IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 2, pp. 138-150, 1996.
[http://dx.doi.org/10.1109/34.481539]
[http://dx.doi.org/10.1109/34.481539]
[28]
M. Han, W. Xu, H. Tao, and Y. Gong, An algorithm for multiple object trajectory tracking The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. vol. 1, , 2004, p. I-I.
[29]
S. Chopra, R. Hadsell, and Y. LeCun, Learning a similarity metric discriminatively, with application to face verification", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. vol. 1,, 2005, pp. 539-546.
[http://dx.doi.org/10.1109/CVPR.2005.202]
[http://dx.doi.org/10.1109/CVPR.2005.202]
[30]
J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, Signature verification using a” siamese” time delay neural network In: Advances in Neural Information Processing Systems, vol. 737., pp. 1994-744.
[31]
J. Wang, Y. Song, T. Leung, C. Rosenberg, J. Wang, J. Philbin, B. Chen, and Y. Wu, "Learning finegrained image similarity with deep rank-ing", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014pp. 1386-1393
[32]
E. Hoffer, and N. Ailon, Deep metric learning using triplet networkInternational Workshop on Similarity-Based Pattern Recognition, 2015, pp. 84-92.
[http://dx.doi.org/10.1007/978-3-319-24261-3_7]
[http://dx.doi.org/10.1007/978-3-319-24261-3_7]
[33]
Y. Taigman, M. Yang, M.A. Ranzato, and L. Wolf, Deepface: Closing the gap to human-level performance in face verificationThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1701-1708.
[http://dx.doi.org/10.1109/CVPR.2014.220]
[http://dx.doi.org/10.1109/CVPR.2014.220]
[34]
Y. Sun, Y. Chen, X. Wang, and X. Tang, "Deep learning face representation by joint identificationverification", Adv. Neural Inf. Process. Syst., vol. •••, pp. 1988-1996, 2014.
[35]
D. Cheng, Y. Gong, S. Zhou, J. Wang, and N. Zheng, Person re-identification by multi-channel partsbased cnn with improved triplet loss functionProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1335-1344.
[36]
M. Andriluka, S. Roth, and B. Schiele, People-tracking-bydetection and people-detection-by-tracking.
[37]
W. Abd-Almageed, M. Hussein, and M. Abdelkader, Realtime human detection and tracking from mobile vehicles IEEE Intelligent Transportation Systems Conference (ITSC’07)., 2007, pp. 149-154.
[38]
J. Zhang, L.L. Presti, and S. Sclaroff, "Online multi-person tracking by tracker hierarchy", IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS’12), 2012. Beijing, China
[39]
L. Leal-Taixé, A. Milan, I. Reid, S. Roth, and K. Schindler, "MOTChallenge", 2015: Towards a benchmark for multitarget tracking, arXiv : 1504.01942 [cs]ArXiv: 1504.01942..
[40]
H.W. Kuhn, "The Hungarian method for the assignment problem", Naval. Research. Logistics Quarterly, vol. 2, pp. 83-97, 1955.
[http://dx.doi.org/10.1002/nav.3800020109]
[http://dx.doi.org/10.1002/nav.3800020109]
[41]
K. Bernardin, and R. Stiefelhagen, Evaluating multiple objects tracking performance: the CLEAR MOT metrics EURASIP Journal on Image and Video Processing, vol. 1, pp. 1-1, January, 2008. 10
[http://dx.doi.org/10.1155/2008/246309]
[http://dx.doi.org/10.1155/2008/246309]
[42]
J. Ferryman, and A. Shahrokni, Pets2009: Dataset and challenge.
[http://dx.doi.org/10.1109/PETS-WINTER.2009.5399556]
[http://dx.doi.org/10.1109/PETS-WINTER.2009.5399556]
[43]
CAVIAR-Project CAVIAR test case scenarios, 2004.http://groups.inf.ed.ac.uk/vision/CAVIAR/CAVIARDATA1/
[44]
W. Choi, "Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor", International Conference on Computer Vision, 2015.
[http://dx.doi.org/10.1109/ICCV.2015.347]
[http://dx.doi.org/10.1109/ICCV.2015.347]
[45]
Y. Min, and J. Yunde, Temporal Dynamic AppearanceM odeling for Online Multi-Person Tracking oct, 2015.
[46]
Y. Xiang, A. Alahi, and S. Savarese, "Learning to Track: Online Multi-Object Tracking by Decision Making", International Conference on Computer Vision, 2015.
[http://dx.doi.org/10.1109/ICCV.2015.534]
[http://dx.doi.org/10.1109/ICCV.2015.534]
[47]
P. Chu, H. Fan, C. Tan, and H. Ling, “Online Multi-Object Tracking with Instance-Aware Tracker and Dynamic Model Refreshment” KCF In: WACV., 2019.
[48]
Jun xiang, Chao Ma, Guohan Xu, Jianhua Hou, "“End-to-End Learning Deep CRF models for Multi-Object Tracking", IEEE Trans. Circ. Syst. Video Tech, 2020.
[49]
N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric”, 2017 IEEE International Con-ference on Image Processing., ICIP, 2017.
[http://dx.doi.org/10.1109/ICIP.2017.8296962]
[http://dx.doi.org/10.1109/ICIP.2017.8296962]
Related Books
Mechanical Engineering Technologies and Applications
Mechanical Engineering Technologies and Applications
Facets of a Smart City: Computational and Experimental Techniques for Sustainable Urban Development
Soft Robotics
Voltammetry for Sensing Applications
Advances in Manufacturing Technologies and Production Engineering
Advances in Additive Manufacturing Processes
Lean Management Solutions for Contemporary Manufacturing Operations
Mechanical Engineering Technologies and Applications
Global Emerging Innovation Summit (GEIS-2021)
Article Metrics
6
1