Design and evaluation of extended reality systems

Huzaifa, Muhammad

Permalink

https://hdl.handle.net/2142/120192 Copy

Description

Title

Design and evaluation of extended reality systems

Author(s)

Huzaifa, Muhammad

Issue Date

2022-12-20

Director of Research (if dissertation) or Advisor (if thesis)

Adve, Sarita

Doctoral Committee Chair(s)

Adve, Sarita

Committee Member(s)

• Chowdhry, Girish
• Fletcher, Christopher
• Hauser, Kris
• LaValle, Steve
• Alaghi, Armin
• Brooks, David

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Extended reality
• computer architecture
• system architecture

Abstract

As we enter the era of domain-specific architectures, systems researchers must understand the requirements of emerging application domains. Augmented and virtual reality (AR/VR) or extended reality (XR) is one such important domain that can propel new directions in architecture and systems research, as well as have a profound impact on all aspects of human endeavours. Thus far, conducting end-to-end systems research in XR, and consequently, developing principled design methodologies, has remained a challenge due to a lack of opensource benchmarks and systems. To address this challenge, this thesis develops a novel open-source XR research testbed, proposes several design principles for optimizing the power consumption of XR systems, and demonstrates the efficacy of these design principles in optimizing key XR components. First, we present ILLIXR, Illinois Extended Reality Testbed, the first fully open-source end-to-end XR system with state-of-the-art components, integrated with a modular and extensible multithreaded runtime, providing an OpenXR compliant interface to XR applications (e.g., game engines), and with the ability to report (and trade off) several quality of experience (QoE) metrics. Next, we analyze performance, power, and QoE metrics for the complete ILLIXR system and for its individual components, the first such analysis to be publicly available. Our analysis reveals several properties with implications for architecture and systems research. These include demanding performance, power, and QoE requirements, a large diversity of critical tasks, inter-dependent execution pipelines with challenges in scheduling and resource management, and a large tradeoff space between performance/power and human perception related QoE metrics. We then explore several broadly applicable design principles for optimizing the power consumption of XR systems. We demonstrate the use of distributed on-sensor computing for optimizing head tracking, which reduces VIO subsystem power by 1.3× compared to a centralized baseline. We employ another principle, online frequency control, to optimize scene reconstruction by reducing its frequency, resulting in a 4.9× reduction in power compared to a state-of-the-art baseline. We further reduce the power consumption of scene reconstruction by using scalable hardware mapping, which results in an overall power reduction of 128×–270× compared to an embedded GPU. The end-to-end nature of ILLIXR has enabled research in compute offload, new networking technologies, QoE-driven cross-component approximate computing, QoE-driven scheduling, and automatic accelerator detection. Overall, this thesis presents the research community with a unique testbed that enables architecture and systems research in XR in particular, and domain-specific systems in general. It presents a characterization of the testbed, providing researchers a starting point for conducting XR research. It also presents several generally applicable design principles for optimizing power, and demonstrates their use in optimizing head tracking and scene reconstruction.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Muhammad Huzaifa

Owning Collections