Using string similarity techniques to detect daily activities in smart homes equipped with a binary sensor network

Document Type : Original Article

Authors

Department of Computer Engineering, Faculty of Engineering, Arak University, Arak, Iran

Abstract

Human activity recognition can be employed in systems that provide support to people who need special care in their daily activities. Various machine learning methods, including Hidden Markov Models and their extensions, are widely used to deal with this problem. In the previous works, HMM-based methods use the conditional independent assumption to compute the probability of a segment of observations. In this research, instead of conditional independent assumption, a new probabilistic model for string space, based on Dynamic Time Warping and Weighted Levenstein Distance is proposed. The model, combined with the Hidden semi-Markov Model, is applied to a publicly available dataset. The results show that the use of the proposed model significantly increases the accuracy of identifying daily activities. All the code and data of this paper are available through the link github.com/ashnik1353.

Keywords


[1] S. Katz, T.D. Downs, H.R. Cash, and R.C. Grotz, “Progress in Development of the Index of ADL,” The Gerontologist, vol. 10, pp. 20-30, 1970, doi: 10.1093/geront/10.1_Part_1.20.
[2] H. Somboon and N. Ramakant, “Large-scale event detection using semi-hidden Markov models,” in 9th IEEE Int. Conf. Comput. Vis. (ICCV 2003), Nice, France, pp. 1455-1462, 2003, doi: 10.1109/ICCV.2003.1238661.
[3] J.K. Aggarwal and M.S. Ryoo, “Human activity analysis: A review,” ACM Comput. Surv., vol. 43, no. 3, pp. 16:1-16:43, 2011, doi: 10.1145/1922649.1922653.
[4] B. Bruno, J. Grosinger, F. Mastrogiovanni, F. Pecora, A. Saffiotti, S. Sathyakeerthy, and A. Sgorbissa, “Multi-modal sensing for human activity recognition,” in 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2015, pp. 594-600, doi: 10.1109/ROMAN.2015.7333653.
[5] L. Chen, J. Hoey, C.D. Nugent, D.J. Cook, and Z. Yu, “Sensor-Based Activity Recognition,” IEEE Trans. Syst. Man Cybern. Part C, vol.  42, no. 6, pp. 790-808, 2012, doi: 10.1109/TSMCC.2012.2198883.
[6] M. Khalily-Dermany, “A multiple criteria algorithm for planning the itinerary of mobile sink in wireless sensor networks,” Soft Comput. J., vol. 7, no. 2, pp. 74-83, 2018, dor: 20.1001.1.23223707.1397.7.2.6.7 [In Persian].
[7] E.M. Tapia, S.S. Intille, and K. Larson, “Activity Recognition in the Home Using Simple and Ubiquitous Sensors,” in Pervasive Computing: 2nd Int. Conf., PERVASIVE 2004, Linz/Vienna, Austria, April 21-23, 2004, pp. 158-175, doi: 10.1007/978-3-540-24646-6_10.
[8] D.C. Ranasinghe, R.L.S. Torres, and A. Wickramasinghe, “Automated activity recognition and monitoring of elderly using wireless sensors: Research challenges,” in 5th IEEE International Workshop on Advances in Sensors and Interfaces IWASI, 2013, pp. 224-227, doi: 10.1109/IWASI.2013.6576067.
[9] U. AVCI, “Recognizing and Discovering Activities of Daily Living in Smart Environments,” Ph.D. dissertation, University of Trento, Italy, 2013.
[10] L.R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-286, 1989, doi: 10.1109/5.18626.
[11] S.-Z. Yu, “Hidden semi-Markov models,” Artif. Intell., vol. 174, no. 2, pp. 215-243, 2010, doi: 10.1016/j.artint.2009.11.011.
[12] T. Kasteren, G. Englebienne, and B.J.A. Krose, “Activity recognition using semi-Markov models on real world smart home datasets,” J. Ambient Intell. Smart Environ., vol. 2, no. 3, pp. 311-325, 2010, doi: 10.3233/AIS-2010-0070.
[13] B. Juang, “On the hidden Markov model and dynamic time warping for speech recognition - A unified view,” AT&T Bell Lab. Tech. J., vol. 63, no. 7, pp. 1213-1243, 1984, doi: 10.1002/j.1538-7305.1984.tb00034.x.
[14] J. Xu and X. Zhang, “Kernels based on weighted Levenshtein distance,” in 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), Budapest, 25-29 July 2004, doi: 10.1109/IJCNN.2004.1381147.
[15] V.G. Sanchez, C.F. Pfeiffer, and N.-O. Skeie, “A Review of Smart House Analysis Methods for Assisting Older People Living Alone,” J. Sens. Actuator Networks, vol. 6, no. 3, p. 11, 2017, doi: 10.3390/jsan6030011.
[16] L. Liu, Y. Peng, M. Liu, and Z.G. Huang, “Sensor-based Human Activity Recognition System with A Multilayered Model Using Time Series Shapelets,” Knowl.-Based Syst., vol. 90, pp. 138-152, 2015, doi: 10.1016/j.knosys.2015.09.024.
[17] H. Ning, F. Shi, T. Zhu, Q. Li, and L. Chen, “A novel ontology consistent with acknowledged standards in smart homes,” Comput. Networks, vol. 148, pp. 101-107, 2019, doi: 10.1016/j.comnet.2018.11.004.
[18] Z. Wang, S. Wang, and Q. Ji, “Capturing Complex Spatio-temporal Relations among Facial Muscles for Facial Expression Recognition,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3422-3429, doi: 10.1109/CVPR.2013.439.
[19] D. Arifoglu and A. Bouchachia, “Activity Recognition and Abnormal Behaviour Detection with Recurrent Neural Networks,” in 14th International Conference on Mobile Systems and Pervasive Computing (MobiSPC 2017), July 24-26, 2017, Leuven, Belgium, pp. 86-93, doi: 10.1016/j.procs.2017.06.121.
[20] L. Song-Mi, Y. Sang Min, and C. Heeryon, “Human activity recognition from accelerometer data using Convolutional Neural Network,” in 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju Island, South Korea, 2017, pp. 131-134, doi: 10.1109/BIGCOMP.2017.7881728.
[21] G. Bhat, R. Deb, V.V. Chaurasia, H. Shill, and U.Y. Ogras, “Online Human Activity Recognition using Low-Power Wearable Devices,” in Proceedings of the International Conference on Computer-Aided Design, San Diego, CA, USA, November 05-08, 2018, pp. 1-8, doi: 10.1145/3240765.3240833.
[22] K.G.M. Chathuramali and R. Rodrigo, “Faster human activity recognition with SVM,” in International Conference on Advances in ICT for Emerging Regions (ICTer2012), Colombo, Sri Lanka, 2012, pp. 197-203, doi: 10.1109/ICTer.2012.6421415.
[23] A. Palaniappan, R. Bhargavi, and V. Vaidehi, “Abnormal human activity recognition using SVM based approach,” in 2012 International Conference on Recent Trends in Information Technology, Chennai, India, 2012, pp. 97-102, doi: 10.1109/ICRTIT.2012.6206829.
[24] P. Palmes, H.K. Pung, T. Gu, W. Xue, and S. Chen, “Object relevance weight pattern mining for activity recognition and segmentation,” Pervasive Mob. Comput., vol. 6, no. 1, pp. 43-57, 2010, doi: 10.1016/j.pmcj.2009.10.004.
[25] P. Rashidi and D.J. Cook, “Mining Sensor Streams for Discovering Human Activity Patterns over Time,” in 2010 IEEE International Conference on Data Mining, Sydney, Australia, 2010, pp. 431-440, doi: 10.1109/ICDM.2010.40.
[26] P. Rashidi, D.J. Cook, L.B. Holder, and M. Schmitter-Edgecombe, “Discovering Activities to Recognize and Track in a Smart Environment,” IEEE Trans. Knowl. Data Eng., vol. 23, no. 4, pp. 527-539, 2011, doi: 10.1109/TKDE.2010.148.
[27] D. Liciotti, M. Bernardini, L. Romeo, and E. Frontoni, “A sequential deep learning application for recognising human activities in smart homes,” Neurocomputing, vol. 396, pp. 501-513, 2020, doi: 10.1016/j.neucom.2018.10.104.
[28] T. Su, H. Sun, C. Ma, L. Jiang, and T. Xu, “HDL: Hierarchical Deep Learning Model based Human Activity Recognition using Smartphone Sensors,” in 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 2019, pp. 1-8, doi: 10.1109/IJCNN.2019.8851889.
[29] T.L.M. Kasteren, A.K. Noulas, and B.J.A. Krose, “Conditional Random Fields versus Hidden Markov Models for activity recognition in temporal sensor data,” in 14th Annual Conference of the Advanced School for Computing and Imaging (ASCI 2008), Heijen, the Netherlands, 2008.
[30] S. Aminikhanghahi, T. Wang, and D.J. Cook, “Real-Time Change Point Detection with Application to Smart Home Time Series Data,” IEEE Trans. Knowl. Data Eng., vol. 31, no. 5, pp. 1010-1023, 2019, doi: 10.1109/TKDE.2018.2850347.
[31] S. Aminikhanghahi and D. Cook, “Enhancing activity recognition using CPD-based activity segmentation,” Pervasive Mob. Comput., vol. 53, 2019, doi: 10.1016/j.pmcj.2019.01.004.
[32] Z. Ramezanpanah, M. Mallem, and F. Davesne, “Human Action Recognition Using Laban Movement Analysis and Dynamic Time Warping,” in 24th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES 2020), Verona (Virtual Event), Italy, 2020, doi: 10.1016/j.procs.2020.08.040.
[33] M. Masnad, G. M. MukitHasan, K.M. Iftekhar, and M.S. Rahman, “Human Activity Recognition using DTW Algorithm,” in 2019 IEEE Region 10 Symposium (TENSYMP), Kolkata, India, 2019, pp. 39-43, doi: 10.1109/TENSYMP46218.2019.8971082.
[34] D.H. Wilson and C. Atkeson, “Simultaneous tracking and activity recognition (STAR) using many anonymous, binary sensors,” in International Conference on Pervasive Computing, Munich, Germany, 2005, pp. 62-79, doi: 10.1007/11428572_5.
[35] D.J. Patterson, D. Fox, H. Kautz, and M. Philipose, “Fine-grained activity recognition by aggregating abstract object usage,” in 9th IEEE International Symposium on Wearable Computers (ISWC'05), Osaka, Japan, 2005, pp. 44-51, doi: 10.1109/ISWC.2005.22.
[36] K.P. Murphy, “Hidden semi-markov models (hsmms),” unpublished notes, 2002.
[37] T. Duong, D. Phung, H. Bui, and S. Venkatesh, “Efficient duration and hierarchical modeling for human activity recognition,” Artif. Intell., vol. 173, no. 7-8, pp. 830-856, 2009, doi: 10.1016/j.artint.2008.12.005.
[38] P. Natarajan and R. Nevatia, “Online, Real-time Tracking and Recognition of Human Actions,” in 2008 IEEE Workshop on Motion and video Computing, Copper Mountain, CO, USA, 2008, pp. 1-8, doi: 10.1109/WMVC.2008.4544064.
[39] C. Vogler and D. Metaxas, “Parallel hidden Markov models for American sign language recognition,” in Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Corfu, Greece, 1999, pp. 116-122, doi: 10.1109/ICCV.1999.791206.
[40] P.-C. Chung and C.-D. Liu, “A daily behavior enabled hidden Markov model for human behavior understanding,” Pattern Recognit., vol. 41, no. 5, pp. 1572-1580, 2008, doi: 10.1016/j.patcog.2007.10.022.
[41] T.T. Truyen, D. Phung, H. Bui, and S. Venkatesh, “Hierarchical semi-Markov conditional random fields for recursive sequential data,” in Advances in Neural Information Processing Systems, 2009, pp. 1657-1664.
[42] E. Marhasev, M. Hadad, and G.A. Kaminka, “Non-stationary Hidden Semi-Markov Models in Activity Recognition,” in: Proceedings of the AAAI Workshop on Modeling Others from Observations (MOO-06), 2006.
[43] P. Natarajan and R. Nevatia, “Coupled Hidden Semi Markov Models for Activity Recognition,” in 2007 IEEE Workshop on Motion and Video Computing (WMVC'07), Austin, TX, USA, 2007, pp. 10-10, doi: 10.1109/WMVC.2007.12.
[44] R.A. Wagner and M.J. Fischer, “The String-to-String Correction Problem,” J. ACM, vol. 21, no. 1, pp. 168-173, 1974, doi: 10.1145/321796.321811.
[45] Z. Geler, V. Kurbalija, M. Ivanovic, and M. Radovanovic, “Weighted kNN and Constrained Elastic Distances for Time-Series Classification,” Expert Syst. Appl., vol. 162, p. 113829, 2020, doi: 10.1016/j.eswa.2020.113829.
[46] P. Arora, Deepali, and S. Varshney, “Analysis of K-Means and K-Medoids Algorithm for Big Data,” Proc. Comput. Sci., vol. 78, pp. 507-512, 2016, doi: 10.1016/j.procs.2016.02.095.
[47] A. Yoosofan and E. Yousofian, “Clustering provinces in Iran Based on Digital Divide Metric Using the K-means Algorithm,” Soft Comput. J., vol. 1, no. 1, pp. 32-45, 2012 [In Persian].
[48] T.L.M. Kasteren, G. Englebienne, and B.J.A. Krose, “Human Activity Recognition from Wireless Sensor Network Data: Benchmark and Software,” in Activity Recognition in Pervasive Intelligent Environments, L. Chen, C.D. Nugent, J. Biswas, and J. Hoey, Eds., ed Paris: Atlantis Press, 2011, pp. 165-186, doi: 10.2991/978-94-91216-05-3_8.
[49] U. Avci and A. Passerini, “Improving Activity Recognition by Segmental Pattern Mining,” in 2012 IEEE International Conference on Pervasive Computing and Communications Workshops, Lugano, Switzerland, 2012, doi: 10.1109/PerComW.2012.6197605.