A low latency remote rendering system for interactive mobile graphics

Shi, Shu

Permalink

https://hdl.handle.net/2142/30899 Copy

Description

Title

A low latency remote rendering system for interactive mobile graphics

Author(s)

Shi, Shu

Issue Date

2012-05-22T00:13:59Z

Director of Research (if dissertation) or Advisor (if thesis)

Campbell, Roy H.

Doctoral Committee Chair(s)

Campbell, Roy H.

Committee Member(s)

• Nahrstedt, Klara
• Hart, John C.
• Zhang, Zhengyou

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Remote Rendering
• Interaction Latency
• Mobile Graphics
• Real-time Video Coding
• 3D Image Warping

Abstract

Mobile devices are gradually changing people’s computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to traditional desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for the applications having very strict latency requirements. In this thesis, I present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. My design takes advantage of an image based rendering technique: 3D image warping, to synthesize the final display image on the client from the multiple depth image references generated on the server. This Multi Depth Image approach can successfully reduce the interaction latency for any user interaction that changes the rendering viewpoint and at the same time maintain high rendering quality. The success of the proposed system depends on two key components: how to select the reference viewpoints and how to compress multiple depth image references efficiently. I propose a Reference Prediction Algorithm for real-time reference selection and a high performance warping-assisted video coding method to compress both image and depth frames in real-time. Furthermore, I also discuss about how to evaluate the real interactive performance of the proposed system. A prototype system that runs several visualization and game applications has been implemented to evaluate and demonstrate the proposed design.

Graduation Semester

2012-05

Permalink

http://hdl.handle.net/2142/30899

Copyright and License Information

Copyright 2012 Shu Shi

Owning Collections