Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

Sensors
Francisco Javier Ordóñez, Daniel Roggen

Abstract

Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse m...Continue Reading

References

Jul 29, 2006·Science·G E Hinton, R R Salakhutdinov
Apr 4, 2009·Physiological Measurement·Stephen J PreeceRobin Crompton
Apr 24, 2012·Journal of Neuroengineering and Rehabilitation·Shyamal PatelMary Rodgers
Dec 3, 2014·Neural Networks : the Official Journal of the International Neural Network Society·Tara N SainathBhuvana Ramabhadran
Aug 4, 2015·Lab on a Chip·Chee Meng Benjamin HoYong-Jin Yoon

❮ Previous
Next ❯

Citations

Feb 18, 2017·Sensors·Mario Munoz-Organero, Ramona Ruiz-Blazquez
Nov 9, 2017·Sensors·Abdulmajid Murad, Jae-Young Pyun
Dec 15, 2017·Sensors·Sergei VakulenkoAndres Weber
Mar 28, 2018·Scientific Reports·Timothy V PyrkovPeter O Fedichev
Jun 15, 2017·IEEE Journal of Biomedical and Health Informatics·Arnaud MoreauBarry Peterson
Jun 29, 2018·Physiological Measurement·David M BurnsStewart McLachlin
Apr 28, 2018·Movement Disorders : Official Journal of the Movement Disorder Society·Florian LipsmeierMichael Lindemann
Jan 30, 2019·Sensors·Patrícia BotaHugo Gamboa
Apr 3, 2019·Sensors·Carlos Avilés-CruzJuan Villegas-Cortéz
Apr 11, 2019·Journal of Biomechanical Engineering·Tae Hyong KimJoung Hwan Mun
May 12, 2019·Sensors·Sumit Majumder, M Jamal Deen
Sep 25, 2019·Sensors·Philip SchmidtKristof Van Laerhoven
Sep 25, 2019·Sensors·Alexander Diete, Heiner Stuckenschmidt
Aug 8, 2019·Sensors·Nattaya MairitthaSozo Inoue
May 2, 2020·Sensors·Robert D Chambers, Nathanael C Yoder

❮ Previous
Next ❯

Methods Mentioned

BETA
feature extraction

Software Mentioned

Theano
GoogLeNet
RMSProp
Skoda
mHealth
PAMAP
DeepConvLSTM
OPPORTUNITY
Actitracker
OpenCL

Related Concepts

Related Feeds

Cell Checkpoints & Regulators

Cell cycle checkpoints are a series of complex checkpoint mechanisms that detect DNA abnormalities and ensure that DNA replication and repair are complete before cell division. They are primarily regulated by cyclins, cyclin-dependent kinases, and the anaphase-promoting complex/cyclosome. Here is the latest research.

Related Papers

Conference Proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society
John ArevaloMiguel Angel Guevara Lopez
IEEE Transactions on Pattern Analysis and Machine Intelligence
Yannis PanagakisMaja Pantic
IEEE Transactions on Pattern Analysis and Machine Intelligence
Shuiwang JiKai Yu
© 2022 Meta ULC. All rights reserved