University of Illinois Urbana-Champaign

Structurally consistent, diverse, colorization using generative modeling and GTAV

Jagarlamudi, Shreya

This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.

Permalink

https://hdl.handle.net/2142/105407

Description

Title
Structurally consistent, diverse, colorization using generative modeling and GTAV
Author(s)
Jagarlamudi, Shreya
Contributor(s)
Schwing, Alexander G.
Issue Date
2018-12
Keyword(s)
  • Video Colorization
  • Gaussian Conditional Random Field
  • VAE
  • MDN
Abstract
Colorizing videos is an important task in the media industry and in object tracking due to the nature of temporal coherency that comes with color. Doing this automatically, consistently and diversely has been a challenging problem in computer vision. Existing work in this field only deals with part of the problem to achieve temporally incoherent results by stitching colorized images together. In this thesis, we use a conditional random field based, variational auto encoder to model structural consistency and diversity of videos and train it on our custom dataset generated from GTA V. Instead of predicting colors for each pixel individually, we learn both spatial correlations between pixels in a frame and temporal correlations between pixels across different frames of each video in order to keep colorization consistent across the video. The model also includes a mixture density network to create a distribution of diverse colorizations per gray-scale video input. We also allow optional user controllability for more realistic colorizations. We demonstrate that our model achieves spatially and temporally consistent, diverse colorizations of gray-level videos.
Type of Resource
text
Language
en
Permalink
http://hdl.handle.net/2142/105407

Owning Collections

Senior Theses - Electrical and Computer Engineering PRIMARY
The best of ECE undergraduate research