Streaming low-bandwidth real-time video using video super-resolution

Vohra, Sanchit

Permalink

https://hdl.handle.net/2142/115801 Copy

Description

Title

Streaming low-bandwidth real-time video using video super-resolution

Author(s)

Vohra, Sanchit

Issue Date

2022-04-28

Director of Research (if dissertation) or Advisor (if thesis)

Patel, Sanjay

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• video
• compression
• H264
• super-resolution
• super resolution
• video super-resolution
• real-time
• streaming
• teleoperations
• autonomy

Abstract

Jalapeño is a real-time video streaming platform designed primarily for tele-operations. The platform uses a traditional video compression approach (H264) and pairs that with unique networking optimizations and artificial super-scaling to increase reliability and decrease bandwidth consumption. Jalapeño assumes the vehicle (client) is running the platform on a device with low computational capabilities. Conversely, the operator is controlling the vehicle on a system with very high resources. Additionally, the client network is assumed to be transmitting over a lossy wireless channel. Real-time streaming also has tighter constraints than traditional live-streaming, hence some networking algorithms that rely on introducing artificial stream delays are infeasible. Jalapeño must alleviate the issues of video streaming under heavy loss conditions with a low-powered client under the challenging real-time constraints. The networking layer closely monitors the quality of the stream and adjusts the parameters of the compression and transmission dynamically. We leverage the temporal scalable video codec capabilities of the open-source OpenH264 codec by Cisco to design a reconstruction algorithm that works under real-time constraints to mitigate packet loss. The prime feature is video super-resolution, a GAN based generator that is able to upscale a low-resolution stream from the client to high-definition frames while maintaining temporal consistency. This thesis also outlines an approach to build a large video dataset for unsupervised video learning and discusses compression-aware super-resolution to advance the framework to the next level.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Sanchit Vohra

Owning Collections