A survey of IMU based cross-modal transfer learning in human activity recognition
Kamboj, Abhi
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/124581
Description
Title
A survey of IMU based cross-modal transfer learning in human activity recognition
Author(s)
Kamboj, Abhi
Issue Date
2024-04-30
Director of Research (if dissertation) or Advisor (if thesis)
Do, Minh
Department of Study
Electrical & Computer Eng
Discipline
Electrical & Computer Engr
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Inertial measurement units
human action recognition
transfer learning
cross-modal learning
multimodal learning
sensor fusion
Abstract
Despite living in a multi-sensory world, most AI models are limited to textual and visual understanding of human motion and behavior. Inertial measurement units (IMUs) provide a salient signal to understand human motion; however, in practice, they have been understudied due to numerous difficulties, including the uniterpretability and lack of data. In fact, full situational awareness of human motion could best be understood through a combination of sensors. In this survey, we investigate how knowledge can be transferred and utilized amongst modalities for Human Activity or Action Recognition (HAR), i.e. cross-modality transfer learning. We motivate the importance and potential of IMU data and its applicability in cross-modality learning as well as the importance of studying the HAR problem. We categorize HAR related tasks by time and abstractness and then compare various types of multimodal HAR datasets. We also distinguish and expound on many related but inconsistently used terms in the literature, such as transfer learning, domain adaptation, representation learning, sensor fusion, and multimodal learning, and describe how cross-modal learning fits with all these concepts. Then, we review the literature on IMU-based cross-modal transfer for HAR. The two main approaches for cross-modal transfer are instance-based transfer, where instances of one modality are mapped to another (e.g. knowledge is transferred in the input space), or feature-based transfer, where the model relates the modalities in an intermediate latent space (e.g. knowledge is transferred in the feature space). Finally, we discuss future research directions and applications in cross-modal HAR.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
