A conditional Bayesian approach with valid inference for high dimensional logistic regression

Ojha, Abhishek

Permalink

https://hdl.handle.net/2142/122255 Copy

Description

Title

A conditional Bayesian approach with valid inference for high dimensional logistic regression

Author(s)

Ojha, Abhishek

Issue Date

2023-12-01

Director of Research (if dissertation) or Advisor (if thesis)

Narisetty, Naveen N.

Doctoral Committee Chair(s)

Narisetty, Naveen N.

Committee Member(s)

• Liang, Feng
• Yang, Yun
• Zhao, Sihai Dave

Department of Study

Statistics

Discipline

Statistics

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Logistic Regression
• High-dimensional setting
• Statistical Inference
• Valid Inference methods
• Bayesian Methods
• Conditional Posterior
• Neyman Orthogonalization

Abstract

We consider the problem of performing inference for a covariate of interest on a binary outcome variable while controlling for high dimensional baseline covariates. While it is relatively easier to address this problem in linear regression, the nonlinearity of the logistic regression poses additional challenges that make it difficult to orthogonalize the effect of the treatment variable from the nuisance variables. When the covariate of interest is continuous, we propose a novel Bayesian framework for performing inference for the desired low-dimensional parameter in a high-dimensional logistic regression model. Our proposed approach provides the first Bayesian alternative to the recent frequentist developments and can incorporate available prior information on the parameters of interest, which plays a crucial role in practical applications. In addition, the proposed approach incorporates uncertainty in orthogonalization in high dimensions instead of relying on a single instance of orthogonalization as done by frequentist methods resulting in intervals with better coverage properties. The main idea behind achieving orthogonality in the context of the continuous covariate of interest is based on a weighted regression between the covariate of interest and the nuisance covariates. However, this approach is not applicable when the covariate of interest is ordinal or categorical. To address this limitation, we design an orthogonal score specifically tailored to these scenarios based on a novel technique of variance-weighted projection. Furthermore, based on the proposed score, we design a novel Bayesian framework to perform inference for the desired low-dimensional parameter. For all the methods proposed in the thesis, we provide rigorous theoretical guarantees in terms of uniform convergence of the proposed conditional posteriors. These results imply the validity of credible intervals resulting from the posterior in a frequentist sense. Based on extensive empirical studies, we demonstrated that our proposed methods have competitive empirical performance when compared with state-of-the-art methods.

Graduation Semester

2023-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Abhishek Ojha

Owning Collections