From pixels to regions: Toward universal image segmentation

Cheng, Bowen

Permalink

https://hdl.handle.net/2142/116182 Copy

Description

Title

From pixels to regions: Toward universal image segmentation

Author(s)

Cheng, Bowen

Issue Date

2022-07-06

Director of Research (if dissertation) or Advisor (if thesis)

Schwing, Alexander

Doctoral Committee Chair(s)

Schwing, Alexander

Committee Member(s)

• Shi, Humphrey
• Hasegawa-Johnson, Mark
• Darrell, Trevor
• Liang, Zhi-Pei

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• computer vision
• image segmentation
• semantic segmentation
• instance segmentation
• panoptic segmentation

Abstract

Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task: semantic segmentation is usually formulated as per-pixel classification and mask classification dominates instance-level segmentation tasks. In this dissertation, we demonstrate how to build a single unified architecture that can address any image segmentation task. We first introduce an effort in unifying image segmentation with either per-pixel classification (Panoptic-DeepLab) or mask classification (MaskFormer). We observe mask classification is sufficiently general to solve both semantic- and instance-level segmentation tasks. Based on this observation we propose Mask2Former, which outperforms even the best specialized architectures by a significant margin on four popular datasets for three image segmentation tasks (panoptic, instance and semantic). Then we discuss how to evaluate image segmentation models with a new Boundary IoU metric. Finally, we conclude this dissertation with promising future directions to explore.

Graduation Semester

2022-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Bowen Cheng

Owning Collections