Login Register Cart 0
Generic placeholder image

Recent Advances in Computer Science and Communications

Editor-in-Chief

ISSN (Print): 2666-2558
ISSN (Online): 2666-2566

Back
Journal Home About Journal Editorial Board Current Issue Volumes/Issues
Subscribe
Research Article

Motion Signal-based Recognition of Human Activity from Video Stream Dataset Using Deep Learning Approach

Author(s): Ram Kumar Yadav*, Daniel Arockiam and Vijay Bhaskar Semwal

Volume 17, Issue 3, 2024

Published on: 26 January, 2024

Article ID: e270124226373 Pages: 15

DOI: 10.2174/0126662558278156231231063935

Price: $65

Abstract

Background: Human physical activity recognition is challenging in various research eras, such as healthcare, surveillance, senior monitoring, athletics, and rehabilitation. The use of various sensors has attracted outstanding research attention due to the implementation of machine learning and deep learning approaches.

Aim: This paper proposes a unique deep learning framework based on motion signals to recognize human activity to handle these constraints and challenges through deep learning (e.g., enhance CNN, LR, RF, DT, KNN, and SVM) approaches.

Method: This research article uses the BML (Biological Motion Library) dataset gathered from thirty volunteers with four various activities to analyze the performance metrics. It compares the evaluated results with existing results, which are found by machine learning and deep learning methods to identify human activity.

Result: This framework was successfully investigated with the help of laboratory metrics with convolutional neural networks (CNN) and achieved 89.0% accuracy compared to machine learning methods.

Conclusion: The novel work of this research is to increase classification accuracy with a lower error rate and faster execution. Moreover, it introduces a novel approach to human activity recognition in the BML dataset using the CNN with Adam optimizer approach.

Graphical Abstract

[1]
K. Chen, D. Zhang, L. Yao, B. Guo, Z. Yu, and Y. Liu, "Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities", ACM Comput. Surv., vol. 54, no. 4, pp. 1-40, 2021.
[CSUR]. [http://dx.doi.org/10.1145/1122445.1122456]
[2]
Z. Hussain, M. Sheng, and W.E. Zhang, "Different approaches for human activity recognition: A survey", 1906.05074.
[3]
L. Minh Dang, K. Min, H. Wang, M. Jalil Piran, C. Hee Lee, and H. Moon, "Sensor-based and vision-based human activity recognition: A comprehensive survey", Pattern Recognit., vol. 108, p. 107561, 2020.
[http://dx.doi.org/10.1016/j.patcog.2020.107561]
[4]
D.R. Beddiar, B. Nini, M. Sabokrou, and A. Hadid, "Vision-based human activity recognition: A survey", Multimedia Tools Appl., vol. 79, no. 41-42, pp. 30509-30555, 2020.
[http://dx.doi.org/10.1007/s11042-020-09004-3]
[5]
D. Singh, I. Psychoula, J. Kropf, S. Hanke, and A. Holzinger, "Users’ perceptions and attitudes towards innovative home technologies. In Smart Homes and Health Telematics, Designing a Better Future: Urban Assisted Living: 16th International Conference, ICOST 2018, Singapore, Singapore, July 10-12, 2018", Proceedings, vol. 16, pp. 203-214, 2018. [Springer International Publishing].
[6]
N. Dua, S.N. Singh, and V.B. Semwal, "Multi-input CNN-GRU based human activity recognition using wearable sensors", Computing, vol. 103, no. 7, pp. 1461-1478, 2021.
[http://dx.doi.org/10.1007/s00607-021-00928-8]
[7]
F. Ordóñez, and D. Roggen, "Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition", Sensors, vol. 16, no. 1, p. 115, 2016.
[http://dx.doi.org/10.3390/s16010115] [PMID: 26797612]
[8]
X. Li, Y. Zhang, I. Marsic, A. Sarcevic, and R.S. Burd, "Deep learning for RFID-based activity recognition", In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, 2016, pp. 164-175
[http://dx.doi.org/10.1145/2994551.2994569]
[9]
L. Gomes, F. Sousa, and Z. Vale, "An intelligent smart plug with shared knowledge capabilities", Sensors, vol. 18, no. 11, p. 3961, 2018.
[http://dx.doi.org/10.3390/s18113961] [PMID: 30445730]
[10]
CD, "Cook DJ Yu Z Sensor-based activity recognition", IEEE Trans. Syst. Man Cybern. Part C, vol. 42, no. 6, p. 790, 2012.
[11]
E. Gomes, L. Bertini, W.R. Campos, A.P. Sobral, I. Mocaiber, and A. Copetti, "Machine learning algorithms for activity-intensity recognition using accelerometer data", Sensors, vol. 21, no. 4, p. 1214, 2021.
[http://dx.doi.org/10.3390/s21041214] [PMID: 33572249]
[12]
M.M. Islam, S. Nooruddin, F. Karray, and G. Muhammad, "Human activity recognition using tools of convolutional neural networks: A state of the art review, data sets, challenges, and future prospects", Comput. Biol. Med., vol. 149, p. 106060, 2022.
[http://dx.doi.org/10.1016/j.compbiomed.2022.106060] [PMID: 36084382]
[13]
L. Babangida, T. Perumal, N. Mustapha, and R. Yaakob, "Internet of things (IoT) based activity recognition strategies in smart homes: A review", IEEE Sens. J., vol. 22, no. 9, pp. 8327-8336, 2022.
[http://dx.doi.org/10.1109/JSEN.2022.3161797]
[14]
A. Liaw, and M. Wiener, "Classification and regression by randomForest", R News, vol. 2, no. 3, pp. 18-22, 2002.
[15]
J. de Lope, and M. Graña, "Deep transfer learning-based gaze tracking for behavioral activity recognition", Neurocomputing, vol. 500, pp. 518-527, 2022.
[http://dx.doi.org/10.1016/j.neucom.2021.06.100]
[16]
P. Khandnor, and N. Kumar, "A survey of activity recognition process using inertial sensors and smartphone sensors", In 2017 International Conference on Computing, Communication and Automation (ICCCA), IEEE. year. , 2017, p. 607-612
[http://dx.doi.org/10.1109/JSEN.2018.2871203]
[17]
N. Manouchehri, and N. Bouguila, "Human activity recognition with an HMM-based generative model", Sensors, vol. 23, no. 3, p. 1390, 2023.
[http://dx.doi.org/10.3390/s23031390] [PMID: 36772428]
[18]
J.R. Kwapisz, G.M. Weiss, and S.A. Moore, "Activity recognition using cell phone accelerometers", SIGKDD Explor., vol. 12, no. 2, pp. 74-82, 2011.
[http://dx.doi.org/10.1145/1964897.1964918]
[19]
M.K. Sain, R.H. Laskar, J. Singha, and S. Saini, "Hybrid deep learning model-based human action recognition in indoor environment", Robotica, vol. 41, no. 12, pp. 3788-3817, 2023.
[http://dx.doi.org/10.1017/S0263574723001327]
[20]
M. Jaberi, and R. Ravanmehr, "Human activity recognition via wearable devices using enhanced ternary weight convolutional neural network", Pervasive Mobile Comput., vol. 83, p. 101620, 2022.
[http://dx.doi.org/10.1016/j.pmcj.2022.101620]
[21]
C. Bu, L. Zhang, H. Cui, G. Yang, and H. Wu, "Dynamic inference via localizing semantic intervals in sensor data for budget-tunable activity recognition", IEEE Trans. Industr. Inform., pp. 1-13, 2023.
[http://dx.doi.org/10.1109/TII.2023.3315773]
[22]
S. Jameer, and H. Syed, "A DCNN-LSTM based human activity recognition by mobile and wearable sensor networks", Alex. Eng. J., vol. 80, pp. 542-552, 2023.
[http://dx.doi.org/10.1016/j.aej.2023.09.013]
[23]
A. Jalal, Y.H. Kim, Y.J. Kim, S. Kamal, and D. Kim, "Robust human activity recognition from depth video using spatiotemporal multi-fused features", Pattern Recognit., vol. 61, pp. 295-308, 2017.
[http://dx.doi.org/10.1016/j.patcog.2016.08.003]
[24]
H. Yu, G. Pan, M. Pan, C. Li, W. Jia, L. Zhang, and M. Sun, "A hierarchical deep fusion framework for egocentric activity recognition using a wearable hybrid sensor system", Sensors, vol. 19, no. 3, p. 546, 2019.
[http://dx.doi.org/10.3390/s19030546] [PMID: 30696100]
[25]
M. Lv, W. Xu, and T. Chen, "A hybrid deep convolutional and recurrent neural network for complex activity recognition using multimodal sensors", Neurocomputing, vol. 362, pp. 33-40, 2019.
[http://dx.doi.org/10.1016/j.neucom.2019.06.051]
[26]
U. Saha, S. Saha, T. Kabir, S.A. Fattah, and M. Saquib, "Decoding human activities: Analyzing wearable accelerometer and gyroscope data for activity recognition", arXiv:2310.02011., 2023.
[27]
S. Mohsen, "Recognition of human activity using GRU deep learning algorithm", Multimedia Tools Appl., vol. 82, no. 30, pp. 47733-47749, 2023.
[http://dx.doi.org/10.1007/s11042-023-15571-y]
[28]
D. Gaur, and S. Kumar Dubey, "Development of activity recognition model using LSTM-RNN deep learning algorithm", J. Inf. Organ. Sci., vol. 46, no. 2, pp. 277-291, 2022.
[29]
O. Nafea, W. Abdul, G. Muhammad, and M. Alsulaiman, "Sensor-based human activity recognition with spatio-temporal deep learning", Sensors, vol. 21, no. 6, p. 2141, 2021.
[http://dx.doi.org/10.3390/s21062141] [PMID: 33803891]
[30]
Z. Zhang, W. Wang, A. An, Y. Qin, and F. Yang, "A human activity recognition method using wearable sensors based on convtransformer model", Evol. Syst., vol. 14, no. 6, pp. 939-955, 2023.
[http://dx.doi.org/10.1007/s12530-022-09480-y]
[31]
S. Xiao, S. Wang, Z. Huang, Y. Wang, and H. Jiang, "Two-stream transformer network for sensor-based human activity recognition", Neurocomputing, vol. 512, pp. 253-268, 2022.
[http://dx.doi.org/10.1016/j.neucom.2022.09.099]
[32]
M. Perkowitz, M. Philipose, K. Fishkin, and D.J. Patterson, "Mining models of human activities from the web", In Proceedings of the 13th international conference on World Wide Web, 2004, pp. 573-582
[http://dx.doi.org/10.1145/988672.988750]
[33]
L. Chen, C.D. Nugent, M. Mulvenna, D. Finlay, X. Hong, and M. Poland, "A logical framework for behavior reasoning and assistance in a smart home", International Journal of Assistive Robotics and Mechatronics, vol. 9, no. 4, pp. 20-34, 2008.
[34]
W. Sousa Lima, E. Souto, K. El-Khatib, R. Jalali, and J. Gama, "Human activity recognition using inertial sensors in a smartphone: An overview", Sensors, vol. 19, no. 14, p. 3213, 2019.
[http://dx.doi.org/10.3390/s19143213] [PMID: 31330919]
[35]
N. Yamada, K. Sakamoto, G. Kunito, Y. Isoda, K. Yamazaki, and S. Tanaka, "Applying ontology and probabilistic model to human activity recognition from surrounding things", IPSJ Digital Courier, vol. 3, pp. 506-517, 2007.
[http://dx.doi.org/10.2197/ipsjdc.3.506]
[36]
L. Chen, C.D. Nugent, L. Chen, and C.D. Nugent, "Semantic smart homes: Towards a knowledge-rich smart environment", In: Human Activity Recognition and Behaviour Analysis: For Cyber-Physical Systems in Smart Environments, 2019, p. 183-199.
[37]
L. Chen, and C. Nugent, "Ontology‐based activity recognition in intelligent pervasive environments. international journal of web information systems", Available from: https://www.emerald.com/insight/content/doi/10.1108/17440080911006199/full/html
[38]
L. Chen, C.D. Nugent, and H. Wang, "A knowledge-driven approach to activity recognition in smart homes", IEEE Trans. Knowl. Data Eng., vol. 24, no. 6, pp. 961-974, 2012.
[http://dx.doi.org/10.1109/TKDE.2011.51]
[39]
B. Logan, J. Healey, M. Philipose, E.M. Tapia, and S. Intille, "A long-term evaluation of sensing modalities for activity recognition. In UbiComp 2007: Ubiquitous Computing: 9th International Conference, UbiComp 2007, Innsbruck, Austria, September 16-19, 2007", Proceedings, vol. 9, pp. 483-500, 2007.
[40]
M.H. Kabir, M.R. Hoque, K. Thapa, and S.H. Yang, "Two-layer hidden Markov model for human activity recognition in home environments", Int. J. Distrib. Sens. Netw., vol. 12, no. 1, p. 4560365, 2016.
[http://dx.doi.org/10.1155/2016/4560365]
[41]
A. Fleury, M. Vacher, and N. Noury, "SVM-based multimodal classification of activities of daily living in Health Smart Homes: Sensors, algorithms, and first experimental results", IEEE Trans. Inf. Technol. Biomed., vol. 14, no. 2, pp. 274-283, 2010.
[http://dx.doi.org/10.1109/TITB.2009.2037317] [PMID: 20007037]
[42]
M. Rahul, R. Shukla, D.K. Yadav, and V. Yadav, "Zernike moment-based facial expression recognition using a two-staged hidden Markov model", In: Advances in Computer Communication and Computational Sciences: Proceedings of IC4S 2018., Springer Singapore, 2019, pp. 661-670.
[http://dx.doi.org/10.1007/978-981-13-6861-5_56]
[43]
M. Rahul, R. Shukla, P.K. Goyal, Z.A. Siddiqui, and V. Yadav, "Gabor filter and ICA-based facial expression recognition using a two-layered hidden Markov model. In Advances in Computational Intelligence and Communication Technology", Proceedings of CICT, vol. 2019, pp. 511-518, 2021. [Springer Singapore].
[44]
R.K. Yadav, S.G. Neogi, and V.B. Semwal, "Special session on recent advances in computational intelligence & technologies (ss_10_racit) development of generic human motion simulation categorization using inception-based cnn", In Proceedings of Third International Conference on Computing, Communications, and Cyber-Security: IC4S 2021, Singapore , 2022, p. 595-608
[45]
R.K. Yadav, S.G. Neogi, and V.B. Semwal, "A Computational Approach to Identify Normal and Abnormal Persons Gait Using Various Machine Learning and Deep Learning Classifiers", In Proceedings, Part I, January 19–20, Cham 2023 p. 14-26
[46]
C.D. Huang, C.Y. Wang, and J.C. Wang, "Human action recognition system for elderly and children care using three stream convent", In 2015 International Conference on Orange Technologies (ICOT), IEEE. , 2015, pp. 5-9
[http://dx.doi.org/10.1109/ICOT.2015.7498476]
[47]
S. Mekruksavanich, and A. Jitpattanakul, Smartwatch-based human activity recognition using hybrid lstm network. In 2020 IEEE SENSORS., IEEE, 2020, pp. 1-4.
[48]
R. Mutegeki, and D.S. Han, “A CNN-LSTM approach to human activity recognition”, In 2020 international conference on artificial intelligence in Information and Communication (ICAIIC) year., IEEE, 2020, pp. 362-366.
[http://dx.doi.org/10.1109/ICAIIC48513.2020.9065078]
[49]
S. Mekruksavanich, and A. Jitpattanakul, Smartwatch-based human activity recognition using hybrid lstm network. In 2020 IEEE SENSORS., IEEE, 2020, pp. 1-4.
[50]
C.T. Yen, J.X. Liao, and Y.K. Huang, "Human daily activity recognition is performed using wearable inertial sensors combined with deep learning algorithms", IEEE Access, vol. 8, pp. 174105-174114, 2020.
[http://dx.doi.org/10.1109/ACCESS.2020.3025938]
[51]
D. Bouchabou, S.M. Nguyen, C. Lohr, B. LeDuc, and I. Kanellos, "Using language model to bootstrap human activity recognition ambient sensors based in smart homes", Electronics, vol. 10, no. 20, p. 2498, 2021.
[http://dx.doi.org/10.3390/electronics10202498]
[52]
D.J. Cook, N.C. Krishnan, and P. Rashidi, "Activity discovery and activity recognition: A new partnership", IEEE Trans. Cybern., vol. 43, no. 3, pp. 820-828, 2013.
[http://dx.doi.org/10.1109/TSMCB.2012.2216873] [PMID: 23033328]
[53]
W.S. Hu, H.C. Li, Y.J. Deng, X. Sun, Q. Du, and A. Plaza, "Lightweight tensor attention-driven ConvLSTM neural network for hyperspectral image classification", IEEE J. Sel. Top. Signal Process., vol. 15, no. 3, pp. 734-745, 2021.
[http://dx.doi.org/10.1109/JSTSP.2021.3063805]
[54]
S. Escalera, V. Athitsos, and I. Guyon, "Challenges in multi-modal gesture recognition", Gesture recognition, p. 1-60. 2017
[55]
M. Asadi-Aghbolaghi, A. Clapes, M. Bellantonio, H.J. Escalante, V. Ponce-López, and X. Baró, "Survey on deep learning based approaches for action and gesture recognition in image sequences", In In 2017 12th IEEE international conference on automatic face & gesture recognition, IEEE, 2017, p. 476-483
[56]
P. Wang, W. Li, P. Ogunbona, J. Wan, and S. Escalera, "RGB-D-based human motion recognition with deep learning: A survey", Comput. Vis. Image Underst., vol. 171, pp. 118-139, 2018.
[http://dx.doi.org/10.1016/j.cviu.2018.04.007]
[57]
G. Sreenu, and M.A. Saleem Durai, "Intelligent video surveillance: A review through deep learning techniques for crowd analysis", J. Big Data, vol. 6, no. 1, p. 48, 2019.
[http://dx.doi.org/10.1186/s40537-019-0212-5]
[58]
R. Chaudhry, F. Ofli, G. Kurillo, R. Bajcsy, and R. Vidal, "Bio-inspired dynamic 3d discriminative skeletal features for human action recognition", In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops year, 2013, pp. 471-478
[http://dx.doi.org/10.1109/CVPRW.2013.153]
[59]
F. Demrozi, G. Pravadelli, A. Bihorac, and P. Rashidi, "Human activity recognition using inertial, physiological and environmental sensors: A comprehensive survey", IEEE Access, vol. 8, pp. 210816-210836, 2020.
[http://dx.doi.org/10.1109/ACCESS.2020.3037715] [PMID: 33344100]
[60]
P. Gupta, and S. Chand, "Human activity recognition using a hybrid dilated CNN and GRU. In Soft Computing: Theories and applications", Proceedings of SoCTA, vol. 2022, pp. 371-380, 2023. [Singapore: Springer Nature Singapore].
[61]
Z. Liao, "Deep CNN-LSTM supervised model and CNN self-supervised model for human activity Recognition Using Wearable Sensor", IEEE J. Transl. Eng. Health Med., vol. 10, p. 2700316, 2023.
[62]
V.B. Semwal, M. Raj, and G.C. Nandi, "Biometric gait identification based on a multilayer perceptron", Robot. Auton. Syst., vol. 65, pp. 65-75, 2015.
[http://dx.doi.org/10.1016/j.robot.2014.11.010]
[63]
H. Zilelioglu, G. Khodabandelou, A. Chibani, and Y. Amirat, "Semisupervised generative adversarial networks with temporal convolutions for human activity recognition", IEEE Sens. J., vol. 23, no. 11, pp. 12355-12369, 2023.
[http://dx.doi.org/10.1109/JSEN.2023.3267243]
[64]
D. Cao, J. Ma, L. Sun, and N. Ma, "A time-series prediction algorithm based on a hybrid model", Recent Advances in Computer Science and Communications, vol. 16, no. 1, pp. 3-17, 2023.
[65]
S. Goyal, "Software measurements using machine learning techniques-a review", Recent Advances in Computer Science and Communications, vol. 16, no. 1, pp. 38-55, 2023.
[66]
K. Dhanasekaren, S.T. Aluri, N. Karthikeyan, S.H. Baskaran, and R. Selvanambi, "A study on the impact of sentiment analysis on stock market prediction", Recent Advances in Computer Science and Communications, vol. 16, no. 1, pp. 73-93, 2023.
[67]
J.Q. Huang, W.L. Guo, and C.Y. Lee, "Dynamic evaluation model of the default risk of online loan borrowers based on K-means and SVM", Recent Advances in Computer Science and Communications, vol. 15, no. 9, pp. 1259-1266, 2022.
[68]
H. Liu, C. Zhang, Z. Wang, Q. Lin, Z. Lan, M. Jiang, and X. Fan, "An effective COVID-19 CT image denoising method based on a deep convolutional neural network", Rec. Adv. Comp. Sci. Comm., vol. 16, no. 4, pp. 75-84, 2023.
[69]
D. Bhattacharya, D. Sharma, W. Kim, M.F. Ijaz, and P.K. Singh, "Ensem-HAR: An ensemble deep learning model for smartphone sensor-based human activity recognition for measurement of elderly health monitoring", Biosensors, vol. 12, no. 6, p. 393, 2022.
[http://dx.doi.org/10.3390/bios12060393] [PMID: 35735541]
[70]
Y. Kaya, and E.K. Topuz, "Human activity recognition from multiple sensors data using deep CNNs", Multimedia Tools Appl., pp. 1-24, 2023.
[http://dx.doi.org/10.1007/s11042-023-15830-y]
[71]
A. Sarkar, S.K.S. Hossain, and R. Sarkar, "Human activity recognition from sensor data using spatial attention-aided CNN with genetic algorithm", Neural Comput. Appl., vol. 35, no. 7, pp. 5165-5191, 2023.
[http://dx.doi.org/10.1007/s00521-022-07911-0] [PMID: 36311167]
[72]
U. Verma, P. Tyagi, and M.K. Aneja, Multi-Branch CNN GRU with attention mechanism for human action recognition., Engineering Research Express, 2023.
[http://dx.doi.org/10.1088/2631-8695/acd98c]
[73]
Y.J. Luwe, C.P. Lee, and K.M. Lim, "Wearable sensor-based human activity recognition with hybrid deep learning model", In Informatics, vol. 9, no. 3, p. 56, 2022.
[http://dx.doi.org/10.3390/informatics9030056]
[74]
A. Yurtman, B. Barshan, and B. Fidan, "Activity recognition invariant to wearable sensor unit orientation using differential rotational transformations represented by quaternions", Sensors, vol. 18, no. 8, p. 2725, 2018.
[http://dx.doi.org/10.3390/s18082725]
[75]
A. Yurtman, and B. Barshan, "Activity recognition invariant to sensor orientation with wearable motion sensors", Sensors, vol. 17, no. 8, p. 1838, 2017.
[http://dx.doi.org/10.3390/s17081838] [PMID: 28792481]
[76]
R.K. Yadav, S.G. Neogi, and V.B. Semwal, "Human activity identification system for video database using deep learning technique", SN Computer Science, vol. 4, no. 5, p. 600, 2023.
[http://dx.doi.org/10.1007/s42979-023-02031-5]

Rights & Permissions Print Cite
© 2025 Bentham Science Publishers | Privacy Policy