ENS-CVXNet: Convex decomposition of complex scenes

Jain, Seemandhar

Permalink

https://hdl.handle.net/2142/124422 Copy

Description

Title

ENS-CVXNet: Convex decomposition of complex scenes

Author(s)

Jain, Seemandhar

Issue Date

2024-04-30

Director of Research (if dissertation) or Advisor (if thesis)

Forsyth, David

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• 3D Reconstruction
• 3D Vision
• Convex Decomposition
• Ray Tracing

Abstract

Generating accurate convex decompositions of indoor scenes from single RGB images is a challenging task with numerous applications in computer graphics. Current state-of-the-art methods employ encoder-decoder neural networks to convert RGB images into a fixed number of simple primitives (e.g., parallelepipeds). However, these approaches have limitations in capturing long-range dependencies and transferring global information, which can impact their accuracy and generalization capability across diverse indoor environments. To address these limitations, we propose ENS-CVXNet, an ensemble approach that leverages the strengths of various convex decomposition techniques and incorporates additional geometric information as summaries. Our core analysis explores well-established algorithms such as VHACD, COACD, and BSP-Net, as well as the integration of global features extracted by PointNet from the input point cloud. By combining multiple models trained with different summaries and configurations, ENS-CVXNet selects the best-performing model for each input image based on its ability to generate accurate depth predictions, evaluated against ground truth depth maps or predictions from state-of-the-art depth predictors. Through extensive experiments and evaluations on the NYUv2 dataset, we demonstrate that ENS-CVXNet outperforms the baseline method, achieving a 20% decrease in AbsRel from 0.093 to 0.0744, and improving the overall precision and quality of convex decomposition for indoor scenes. Our ensemble approach effectively combines the strengths of various techniques, leveraging geometric summaries and adapting to diverse scene characteristics, results in more accurate and robust 3D geometric reconstruction from single RGB images.

Graduation Semester

2024-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2024 Seemandhar Jain

Owning Collections