University of Illinois Urbana-Champaign

Improve OpenMVG and create a novel algorithm for novel view synthesis from point clouds

Tsoi, Ka Wai

Content Files
TSOI-THESIS-2016.pdf

Permalink

https://hdl.handle.net/2142/90821

Description

Title
Improve OpenMVG and create a novel algorithm for novel view synthesis from point clouds
Author(s)
Tsoi, Ka Wai
Issue Date
2016-04-26
Director of Research (if dissertation) or Advisor (if thesis)
Hoiem, Derek W.
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Date of Ingest
2016-07-07T20:27:59Z
Keyword(s)
  • Computer Vision
  • Computer Science
  • Structure-from-Movtion
  • Novel View Synthesis
Abstract
This thesis presents work to improve open source 3D reconstruction software OpenMVG and to create a novel algorithm to render photorealistic images from new views given a photo collection and 3D point cloud. First, the original OpenMVG is parallelized using GPU and its data structure is optimized. Moreover, we integrated the MatchMiner algorithm into OpenMVG to further improve its efficiency. Last but not least, an initial pair selection formulation and a default focal length setting are introduced and implemented to automize OpenMVG. Then 3D sparse point clouds of construction sites are reconstructed by performing Structure-from-Motion (SfM) with the improved version of OpenMVG and source images (images that are used in SfM) are calibrated and registered to point clouds. Furukawa's Patch-based Multi-view Stereo(PMVS) algorithm is used to reconstruct dense point clouds using calibrated cameras as inputs. With known depth values of 3D points in the dense point cloud, we estimate depth maps of source images using optimization similar to Levin's colorization algorithm. For a novel view of the point cloud, we find source images that share some common elements of the construction site that are also visible to the novel view. Then we warp depth maps of these candidate images to the novel view. We estimate a depth map and label pixels for the novel view by solving a multi-label Markov Random Field (MRF) optimization problem using graph-cuts. We introduce a novel energy minimization formulation exploits both 2D and 3D information. Finally, a photorealistic image of the novel view is rendered by copying pixel colors from selected candidate source images using pixel labels computed with graph-cuts. We experimentally validate our approach on several challenging viewing angles of a point cloud model of a complicate construction site. The rendered results show high photo-realistic synthesis quality in planar scenes.
Graduation Semester
2016-05
Type of Resource
text
Permalink
http://hdl.handle.net/2142/90821
Copyright and License Information
2016 Ka Wai Tsoi.

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois