Learning distributions with Particle Mirror Descent

Zhang, Ailing

Permalink

https://hdl.handle.net/2142/97411 Copy

Description

Title

Learning distributions with Particle Mirror Descent

Author(s)

Zhang, Ailing

Issue Date

2017-04-24

Director of Research (if dissertation) or Advisor (if thesis)

• Koyejo, Oluwasanmi
• He, Niao

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Constrained Particle Mirror Descent
• Deep generative networks

Abstract

As an effective and provable primal method to estimate posterior distribution, Particle Mirror Descent is appealing for its simplicity and flexibility. In this thesis we explore the applications of Particle Mirror Descent in both supervised and unsupervised learning. In the general classification problem with a parametric discriminative function, we seek a posterior distribution over parameters that maximizes classification accuracy. Existing algorithms usually solve the dual problem and the number of variables to optimize depends on the number of examples in the dataset. Therefore such methods suffer from the poor scalability in large datasets. We propose Constrained Particle Mirror Descent to effectively estimate posterior distribution in primal space even with expectation constraint. By marrying Bayesian probabilistic inference and deep neural networks, deep generative networks have shown remarkable success in various kinds of generative tasks. However, such models usually make an assumption that posterior distribution can be simply characterized as a Gaussian distribution, which is not always true since real data like images and audios yield complex structures in latent space. Motivated by the recent wide application of multi-modal posterior, we introduce a variant of Variational Auto-encoder model that uses a mixture of customized kernels as posterior distribution in latent space. Our deep generative model produces visually plausible images as well as good clustering performance using latent representations.

Graduation Semester

2017-05

Type of Resource

text

Permalink

http://hdl.handle.net/2142/97411

Copyright and License Information

Copyright 2017 Ailing Zhang

Owning Collections