ILLIXR-A: an open source extended reality system for android platforms

Bhowmik, Madhuparna

Permalink

https://hdl.handle.net/2142/121543 Copy

Description

Title

ILLIXR-A: an open source extended reality system for android platforms

Author(s)

Bhowmik, Madhuparna

Issue Date

2023-07-18

Director of Research (if dissertation) or Advisor (if thesis)

Adve, Sarita

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Extended Reality
• Android
• ILLIXR

Abstract

There has been significant progress in virtual, augmented, and mixed reality (VR, AR, and MR) devices, collectively referred to as extended reality (XR). However, until recently, this do- main was dominated by proprietary software, which slowed down research. This motivated the development of an open-source end-to-end XR system called ILLIXR (Illinois Extended Reality testbed). It was developed to democratize XR systems research, development, and benchmark- ing. ILLIXR has been used to provide quantitative characterizations of performance, power, and quality-of-experience metrics for current XR systems and to develop solutions to address the gap between current and desirable XR systems. However, ILLIXR currently only supports the Linux platform, which limits its use in common consumer-grade devices that do not support Linux. The operating system used in most consumer devices, such as smartphones, smart TVs, tablets, and head-mounted displays (HMD), is Android. Android’s open-source nature allowed it to be highly customizable and adaptable for different purposes, contributing to its widespread usage. Additionally, there is little detailed analysis of XR runtimes on the Android platform. Hence, we present ILLIXR-A, a modular, open-source, and end-to-end XR system for the An- droid platform. ILLIXR-A leverages Monado to provide support for OpenXR-compatible applications, which is an emerging open interface between an XR application and runtime. It will enable researchers to utilize the full potential of ILLIXR on a wide variety of Android devices. ILLIXR-A provides full head-tracking capabilities using Visual-Inertial Odometry (VIO) by utilizing the camera and built-in sensors in an Android device. It also includes an IMU Integrator and a pose predictor, which predicts the user’s pose at a faster rate than VIO to compensate for the high latency of VIO. ILLIXR-A also performs Asynchronous Reprojection (Timewarp). Using ILLIXR-A, we evaluated an end-to-end XR system on an Android smartphone, and we present its performance characterization in this thesis. The experimental results highlight the need for efficient algorithms for low-power devices to meet the tight deadlines for a smooth XR experience. We also show how the quality of experience is affected by the complexity of an algorithm and provide future directions for research.

Graduation Semester

2023-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Madhuparna Bhowmik

Owning Collections