A Multimodal Data Model for Four-Dimensional User Attribute Inference in Data Retrieval

Hou, Jingrui; Li, Qiao; Yang, Hanqin; Wang, Ping

Permalink

https://hdl.handle.net/2142/122804 Copy

Description

Title

A Multimodal Data Model for Four-Dimensional User Attribute Inference in Data Retrieval

Author(s)

• Hou, Jingrui
• Li, Qiao
• Yang, Hanqin
• Wang, Ping

Issue Date

2024-03-20

Keyword(s)

• User Portrait
• Data Retrieval
• Multimodal Data Mining
• Deep Learning

Abstract

The construction of a comprehensive portrait of data searchers represents a crucial yet underexplored topic within the field of data retrieval. To address this gap, this study focuses primarily on two key aspects. Firstly, based on the literature review, this study constructs a comprehensive user attribute classification framework that encompasses 52 user attributes derived from the dimensions of user background, expectancy, experience, and system evaluation and usage intention. Secondly, the present study proposes a deep learning-based multimodal data model for inferring these four-dimensional user attributes. The model posits that leveraging deep learning methods to analyze the features of online behaviors, eye movement, facial expressions, and verbal commentary extracted from multimodal data ena-bles accurate inference of the four-dimensional user attributes. A user ex-periment was conducted to collect data. The results from deep learning and ablation experiments provide strong support for the proposed model. The findings suggest that deep learning analysis of multimodal data facilitates the inference of the four-dimensional user attributes. Notably, the proposed model achieved high accuracy in inferring user expectancy and background. Search behavior features and eye movement features are pivotal in accurate-ly inferring the four-dimensional user attributes.

Publisher

iSchools

Series/Report Name or Number

iConference 2024 Proceedings

Type of Resource

Other

Language

eng

Handle URL

https://hdl.handle.net/2142/122804

Copyright and License Information

Copyright 2024 is held by Jingrui Hou, Qiao Li, Hanqin Yang and Ping Wang. Copyright permissions, when appropriate, must be obtained directly from the authors.

Owning Collections