Generative models for problems in imaging science

Kelkar, Varun Ajit

Permalink

https://hdl.handle.net/2142/121447 Copy

Description

Title

Generative models for problems in imaging science

Author(s)

Kelkar, Varun Ajit

Issue Date

2023-07-14

Director of Research (if dissertation) or Advisor (if thesis)

Anastasio, Mark A

Doctoral Committee Chair(s)

Anastasio, Mark A

Committee Member(s)

• Do, Minh N
• Zhao, Zhizhen
• Lam, Fan

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• imaging science
• generative models
• inverse problems
• generative adversarial networks
• invertible networks
• tomography
• diffusion models

Abstract

In recent years, generative models have risen to the forefront of machine learning research. Modern generative models such as generative adversarial networks (GANs) and invertible generative models (IGMs) are capable of approximating high-dimensional image distributions and synthesizing images with high perceptual quality. In imaging science, they are being investigated for several potential applications, such as inverse problems and image reconstruction, image-to-image translation, and dataset augmentation. In the first part of this thesis, generative models are investigated for solving inverse problems in imaging. Specifically, we first developed a new image reconstruction method using a multiscale IGM as a prior, which demonstrated high performance on image reconstruction from stylized, simulated magnetic resonance imaging measurements, and was robust to test-time distribution shifts. Next, a style-based GAN was employed as a prior in a framework for estimating an object of interest that is closely related to a known prior image. The approach accurately captured difficult-to-model semantic differences between the sought-after and prior images and estimated the object accurately in terms of conventional metrics. Third, variational Bayesian methods were employed to learn an IGM of objects directly from a dataset of noisy and incomplete images. The second part of this thesis focuses on evaluating generative models and data-driven priors in imaging. Specifically, the concept of "hallucinations" in the context of image reconstruction was formally defined and utilized to illustrate the effects of an incorrect data-driven prior on the image estimate. Lastly, a framework for evaluating GANs in terms of medically relevant statistics was developed. Perceptual measures for evaluating the GAN did not always correlate with the relevant measures developed, highlighting the urgent need to assess generative models in terms of relevant, task-informed statistics. Our findings directly inspired an ongoing large-scale competition on deep generative modeling for learning medical image statistics.

Graduation Semester

2023-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Varun Ajit Kelkar

Owning Collections