University of Illinois Urbana-Champaign

Measuring the robustness and generalization properties of unified vision and language models

Marten, Ryan Lewis

Content Files
MARTEN-THESIS-2023.pdf

Permalink

https://hdl.handle.net/2142/120318

Description

Title
Measuring the robustness and generalization properties of unified vision and language models
Author(s)
Marten, Ryan Lewis
Issue Date
2023-04-24
Director of Research (if dissertation) or Advisor (if thesis)
Hoiem, Derek
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
  • AI
  • computer vision
  • benchmark
Abstract
Computer vision models excel at making predictions when the test distribution closely resembles the training distribution. Such models have yet to match the ability of biological vision to learn from multiple sources and generalize to new data sources and tasks. To facilitate the development and evaluation of more general vision systems, we introduce the General Robust Image Task (GRIT) benchmark. GRIT evaluates the performance, robustness, and calibration of a vision system across a variety of image prediction tasks, concepts, and data sources. The seven tasks in GRIT are selected to cover a range of visual skills: object categorization, object localization, referring expression grounding, visual question answering, segmentation, human keypoint detection, and surface normal estimation. GRIT is carefully designed to enable the evaluation of robustness under image perturbations, image source distribution shift, and concept distribution shift. By providing a unified platform for thorough assessment of skills and concepts learned by a vision model, we hope GRIT catalyzes the development of performant and robust general purpose vision systems.
Graduation Semester
2023-05
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/120318
Copyright and License Information
Copyright 2023 Ryan Marten

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois