Director of Research (if dissertation) or Advisor (if thesis)
Kumar, Ranjitha
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
design, mobile
Abstract
Given the growing number of mobile apps and their increasing impact on modern life, researchers have developed black-box approaches to mine mobile app design and interaction data. Although the data captured during interaction mining is descriptive, it does not expose the design semantics of UIs: what elements on the screen mean and how they are used. This thesis introduces an automatic approach for semantically annotating the elements comprising a UI given the data captured during interaction mining. Through an iterative open coding of 73k UI elements and 720 screens, we first created a lexical database of 24 types of UI components, 197 text button concepts, and 135 icon classes shared across apps. Using the labeled data created during this process, we learned code-based patterns to detect components, and trained a convolutional neural network which distinguishes between 99 icon classes with 94% accuracy. With this automated approach, we computed semantic annotations for the 72k unique UIs comprising the Rico dataset, assigning labels for 78% of the total visible, non-redundant elements.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
