Login Register Cart 0
Generic placeholder image

International Journal of Sensors, Wireless Communications and Control

Editor-in-Chief

ISSN (Print): 2210-3279
ISSN (Online): 2210-3287

Back
Journal Home About Journal Editorial Board Journal Insight Current Issue Volumes/Issues
Subscribe
Research Article

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

Author(s): Kyungkoo Jun*

Volume 9, Issue 1, 2019

Page: [16 - 31] Pages: 16

DOI: 10.2174/2210327908666180727123657

Price: $65

Abstract

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data.

Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN).

Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme.

Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

Keywords: CNN, deep learning, human activity recognition, jump, LSTM, sensor.

« Previous Next »
Graphical Abstract

[1]
Kalantarian H, Sideris C, Mortazavi B, Alshurafa N, Sarrafzadeh M. Dynamic computation offloading for low-power wearable health monitoring systems. IEEE Trans Biomed Eng 2017; 64(3): 621-8.
[2]
Pantelopoulos A, Bourbakis NG. A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Trans Syst 2010; 40(1): 1-12.
[3]
Avci A, Bosch S, Marin-Perianu M, Marin-Perianu R, Havinga P. Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey. In: Architecture of computing systems (ARCS). 2010 23rd international conference 2010; 1: pp. 1-10.
[4]
Lin W, Sun MT, Poovandran R, Zhang Z. Human activity recognition for video surveillance. In: Circuits and Systems. 2008. ISCAS 2008. IEEE Int Symp 2008; 12: pp. 2737-40.
[5]
Ahmed SH, Kim D. Named data networking-based smart home. ICT Express 2016; 2(3): 130-4.
[6]
Kumar S. Ubiquitous smart home system using android application. arXiv preprint arXiv :14022114 2014.
[http://dx.doi.org/10.5121/ijcnc.2014.6103.]
[7]
Rautaray SS, Agrawal A. Vision based hand gesture recognition for human computer interaction: a survey. . Artificial Int Rev 2015; 43(1): 1-54.
[8]
Yeo HS, Lee BG, Lim H. Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware. Multimedia Tool Appl 2015; 74(8): 2687-715.
[9]
Ahmadi-Karvigh S, Ghahramani A, Becerik-Gerber B, Soibelman L. Real-time activity recognition for energy efficiency in buildings. Appl Energy 2018; 211: 146-60.
[10]
Mannini A, Sabatini AM. Machine learning methods for classifying human physical activity from on-body accelerometers. Sensor 2010; 10(2): 1154-75.
[11]
Bao L, Intille SS. Activity recognition from user-annotated acceleration data. Int Conf Pervasive Comp 2004; 3: 1-17.
[12]
Aggarwal JK, Ryoo MS. Human activity analysis: a review. ACM Comp Survey 2011; 43(3): 16.
[13]
Wu W, Dasgupta S, Ramirez EE, Peterson C, Norman GJ. Classification accuracies of physical activities using smartphone motion sensors. J Med Internet Res 2012; 14(5): 130.
[14]
Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL. A public domain dataset for human activity recognition using smartphones. In: ESANN . 2013.
[15]
Garcia-Ceja E, Galván-Tejada CE, Brena R. Multi-view stacking for activity recognition with sound and accelerometer data. Inf Fusion 2018; 40: 45-56.
[16]
Ghio A, Oneto L. Byte The bullet: learning on real-world computing architectures. In: ESANN . 2014.
[17]
Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: International workshop on ambient assisted living. 2012; pp. 216-3.
[18]
Plötz T, Hammerla NY, Olivier P. Feature learning for activity recognition in ubiquitous computing. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence. 2011; 1729.
[19]
Khalifa S, Lan G, Hassan M, Seneviratne A, Das SK. Harke: Human activity recognition from kinetic energy harvesting data in wearable devices. IEEE Trans Mobile Comput 2018; 17(6): 1353-68.
[20]
Wang X, Gao L, Song J, Zhen X, Sebe N, Shen HT. Deep appearance and motion learning for egocentric activity recognition. Neurocomputing 2018; 275: 438-47.
[21]
Duong T, Phung D, Bui H, Venkatesh S. Efficient duration and hierarchical modeling for human activity recognition. Artif Intell 2009; 173(7-8): 830-56.
[22]
Barshan B, Yurtman A. Investigating inter-subject and inter-activity variations in activity recognition using wearable motion sensors. Comp J 2016; 59(9): 1345-62.
[23]
Altun K, Barshan B, Tunçel O. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recog 2010; 43(10): 3605-20.
[24]
Bulling A, Blanke U, Schiele B. A tutorial on human activity recognition using body-worn inertial sensors. ACM Comp Survey (CSUR) 2014; 46(3): 33.
[25]
Yao R, Lin G, Shi Q, Ranasinghe DC. Efficient dense labelling of human activity sequences from wearables using fully convolutional networks. Pattern Recognit 2018; 78: 252-66.
[26]
Hammerla NY, Halloran S, Ploetz T. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:160408880 2016.
[27]
Ma X, Hovy E. End-to-end sequence labeling via bidirectional lstm-cnns-crf. arXiv preprint arXiv: 160301354 2016.
[28]
Geras KJ. Mohamed Ar, Caruana R, Urban G, Wang S, Aslan O. Blending lstms into cnns. arXiv preprint arXiv:151106433 2015.
[29]
Wang J, Yu LC, Lai KR, Zhang X. Dimensional sentiment analysis using a regional CNN-LSTM model. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2016; 11: pp. 225-30.
[30]
Donahue J, Anne Hendricks L, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K. Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015; pp. 2625-34.
[31]
Chavarriaga R, Sagha H, Calatroni A, Digumarti ST, Tröster G. Millán JdR. The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recognition Lett 2013; 34(15): 2033-42.
[32]
Foerster F, Smeja M, Fahrenberg J. Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring. Comput Human Behav 1999; 15(5): 571-83.
[33]
Kwapisz JR, Weiss GM, Moore SA. Cell phone-based biometric identification. In: biometrics: theory Applications and Systems (BTAS) 2010. 1-7.
[34]
Sharma A, Lee YD, Chung WY. High accuracy human activity monitoring using neural network. In: Convergence and Hybrid Information Technology ICCIT’08 3rd International Conference. 2008; pp. 430-5.
[35]
Khan AM. Recognizing physical activities using Wii remote. Int J Inf Educ Technol 2013; 3(1): 60.
[36]
Shoaib M, Bosch S, Incel OD, Scholten H. Having a PJM, Fusion of smartphone motion sensors for physical activity recognition. Sensor 2014; 14(6): 10146-76.
[37]
Reyes-Ortiz JL, Oneto L, Samà A, Parra X, Anguita D. Transition-aware human activity recognition using smartphones. Neurocomputing 2016; 171: 754-67.
[38]
Ronao CA, Cho SB. Recognizing human activities from smartphone sensors using hierarchical continuous hidden Markov models. Int J Distr Sensor Netw 2017; 13(1): 1550147716683687.
[39]
Cao L, Wang Y, Zhang B, Jin Q, Vasilakos AV. GCHAR: an efficient Group-based Context-aware human activity recognition on smartphone. J Distr Comp 2018; 118: 67-80.
[40]
Bhattacharya S, Nurmi P, Hammerla N, Plötz T. Using unlabeled data in a sparse-coding framework for human activity recognition. Pervasive Mob Comp 2014; 15: 242-62.
[41]
Li Y, Shi D, Ding B, Liu D. Unsupervised feature learning for human activity recognition using smartphone sensors. Mining Intelligence and Knowledge Exploration: Springer 2014; 13: 99-107.
[42]
Vollmer C, Hellbach S, Eggert J, Gross HM. Sparse coding of human motion trajectories with non-negative matrix factorization. Neurocomputing 2014; 124: 22-32.
[43]
Hassan MM, Uddin MZ, Mohamed A, Almogren A. A robust human activity recognition system using smartphone sensors and deep learning. Future Generat Comput Syst 2018; 81: 307-13.
[44]
Yang J, Nguyen MN, San PP, Li X, Krishnaswamy S. Deep convolutional neural networks on multichannel time series for human activity recognition. In: IJCAI. 2015; 21: pp. 3995-4001.
[45]
Duffner S, Berlemont S, Lefebvre G, Garcia C. 3D gesture classification with convolutional neural networks. In: Acoustics, Speech and Signal Processing (ICASSP),2014. 432-5436.
[46]
Zheng Y, Liu Q, Chen E, Ge Y, Zhao JL. Time series classification using multi-channels deep convolutional neural networks. In: International Conference on Web-Age Information Management. 2014; pp. 298-310.
[47]
Zeng M, Nguyen LT, Yu B, et al. Convolutional neural networks for human activity recognition using mobile sensors. In: Mobile Computing, Applications and Services (MobiCASE). 2014; 11: pp. 197-205.
[48]
Ronao CA, Cho SB. Human activity recognition with smartphone sensors using deep learning neural networks. Exp Syst Appl 2016; 59: 235-44.
[49]
LeCun Y, Bengio Y, Hinton G. Deep learning nature. Int J Sci 2015; 521(7553): 436.
[50]
Ordóñez FJ, Roggen D. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensor 2016; 16(1): 115.
[51]
Yu Z, Rennong Y, Guillaume C, Maoguo G. Deep Residual Bidir-LSTM for Human activity recognition using wearable sensors. arXiv preprint arXiv: 1708 08989 2017.
[52]
Inoue M, Inoue S, Nishida T. Deep recurrent neural network for mobile human activity recognition with high throughput. Artif Life 2018; 23(2): 173-85.
[53]
Nweke HF, Teh YW, Al-garadi MA, Alo UR. Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Exp Syst Appl 2018; 105: 233-61.
[54]
Wang J, Chen Y, Hao S, Peng X, Hu L. Deep learning for sensor-based activity recognition: a survey. Pattern Recog Lett 2018.
[http://dx.doi.org/10.1016/ j.patrec.2018.02.010.]
[55]
Núñez JC, Cabido R, Pantrigo JJ, Montemayor AS, Vélez JF. Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recog 2018; 76: 80-94.
[56]
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comp 1997; 9(8): 1735-80.
[57]
Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 1994; 5(2): 157-66.
[58]
Pascanu R, Mikolov T, Bengio Y. Understanding the exploding gradient problemCoRR, abs/12115063 2012.
[59]
Wollmer M, Eyben F, Keshet J, Graves A, Schuller B, Rigoll G. Robust discriminative keyword spotting for emotionally colored spontaneous speech using bidirectional LSTM networks. In: Acoustics. Speech and Signal Processing. ICASSP 2009; pp. 3949-52.
[60]
Krogh A, Hertz JA. A simple weight decay can improve generalization. In: Advances in neural information processing systems NIPS Proceedings. 1992; 11: pp. 950-7.
[61]
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Res 2014; 15(1): 1929-58.
[62]
Zhang L, Wu X, Luo D. Recognizing human activities from raw accelerometer data using deep neural networks. In: 2015 IEEE 14th International Conference on Machine Learning and Applications ICMLA. 2015; 13: pp. 865-70.
[63]
Cao H, Nguyen MN, Phua C, Krishnaswamy S, Li X. An integrated framework for human activity classification. Ubi Comp 2012; 4: 310-40.

Rights & Permissions Print Cite
Article Metrics
34
2
© 2024 Bentham Science Publishers | Privacy Policy